WO2016052164A1

WO2016052164A1 - Conversation device

Info

Publication number: WO2016052164A1
Application number: PCT/JP2015/076081
Authority: WO
Inventors: 梅原　尚子; 圭司寺島
Original assignee: シャープ株式会社
Priority date: 2014-09-30
Filing date: 2015-09-15
Publication date: 2016-04-07
Also published as: JP6448971B2; JP2016071247A

Abstract

The purpose of the present invention is to provide a conversation device which is capable of responding smoothly, and which is capable of providing a comfortable conversation environment without putting stress on a user. Provided is a conversation device (10), comprising an output control unit (18) which, in response to a voice input, carries out, in series after carrying out a first response process which voice outputs response content which is signified with first response information which is obtained by querying a first response information storage unit (141), a second response process which voice outputs response content which is signified with second response information which is received from a cloud server (30).

Description

Dialogue device

The present invention relates to an interactive apparatus and an interactive system that are connected to a communication network and recognize and respond to a user's voice.

In recent years, robots such as care and healing care robots and robots for housekeeping have gradually permeated the lives of users. For example, as disclosed in Patent Documents 1 to 4, a dialogue engine is provided to recognize a user's voice. Dialogue robots (dialogue devices) that respond in response have also been developed. Such a dialogue robot is difficult to perform complex speech recognition due to the performance and cost of the equipped dialogue engine, and the response is also patterned or simple, which is interesting for the user. It tends to be less likely to get bored.

Therefore, a system in which a dialog robot is connected to a server device via a communication network, and voice recognition is performed by the server device. In the interactive robot, information related to a response based on the recognition result is received from the server device and output (response). Has also been developed. According to such a system, it is possible to respond to a voice whose content cannot be processed by the dialog robot alone, and it is possible to obtain more information for the user. Furthermore, since the server device can be used by a plurality of robots, it is more advantageous in terms of cost than increasing the processing capability of each interactive robot.

International Publication No. WO05 / 076258A1 (released on August 18, 2005) Japanese Patent Publication “Japanese Patent Laid-Open No. 2006-043780 (published February 16, 2006)” Japanese Patent Publication “JP 2010-128281 (published on June 10, 2010)” Japanese Patent Publication “JP 2003-022092 (published on January 24, 2003)”

However, when the interactive robot responds by acquiring the response content from the server device, the response timing is delayed as compared with the case where the interactive robot alone recognizes and responds. For this reason, the user may feel stressed and feel difficult to talk.

Accordingly, the present invention has been made in view of the above problems, and provides an interactive apparatus and an interactive system that can smoothly output a plurality of information and provide a comfortable interactive environment without stressing the user. There is to do.

In order to solve the above-described problem, an interactive apparatus according to an aspect of the present invention shows a speech recognition unit that recognizes input speech that has been input, and a response content corresponding to a result of speech recognition by the speech recognition unit. Response information storage unit for storing first response information, and communication for transmitting the input voice to a server device and receiving second response information indicating response contents according to a result of voice recognition of the input voice by the server device And the second response continuously after performing the first response processing for outputting the response content indicated by the first response information obtained by referring to the response information storage unit to the input speech And an output control unit for performing a second response process for outputting the response content indicated by the information as a voice.

According to the interactive device of one aspect of the present invention, the input speech can be supplemented by the speech recognition of the server device and the response by the second response information in addition to the speech recognition by the device itself and the response by the first response information. Therefore, it is possible to respond with a plurality of pieces of information without improving the processing capability of the voice recognition unit of the interactive apparatus and expanding the capacity of the response information storage unit. Thus, according to the above configuration, a plurality of information can be smoothly output as a voice, and a comfortable interactive environment can be provided without causing stress to the user.

It is a figure which shows schematic structure of the dialogue system which concerns on Embodiment 1 of this invention. It is a figure which shows the flow of the dialogue process in the dialogue system which concerns on Embodiment 1 of this invention. It is a figure which shows schematic structure of the dialogue system which concerns on Embodiment 2 of this invention. It is a figure explaining operation | movement of the dialogue apparatus which concerns on Embodiment 2 of this invention. It is a figure which shows schematic structure of the dialogue system which concerns on Embodiment 3 of this invention. It is a conceptual diagram of the dialogue system which concerns on Embodiment 3 of this invention.

[Embodiment 1]
Hereinafter, an embodiment of the present invention will be described with reference to FIGS.

(Configuration of interactive system)
FIG. 1 is a diagram showing a configuration of a dialogue system 100 according to the present embodiment. As shown in FIG. 1, the dialogue system 100 includes a dialogue device (self device) 10 and a cloud server (server device) 30, which are connected via a communication network. For example, the Internet can be used as this communication network. Further, a telephone line network, a mobile communication network, a CATV (CAble TeleVision) communication network, a satellite communication network, or the like can be used.

In the dialogue system 100, the dialogue device 10 and the cloud server 30 each have a voice recognition function, and the user can interact with the dialogue device 10 by voice using natural language. The dialogue apparatus 10 may be, for example, a dialogue robot, or may be a smartphone, a tablet terminal, a personal computer, a home appliance (home electronic device) or the like having a voice recognition function.

In FIG. 1, only one interactive device 10 connected to the cloud server 30 is shown for simplicity of explanation, but in the interactive system 100, the number of interactive devices 10 connected to the cloud server 30 is as follows. It is not limited. In addition, the type of the interactive device 10 connected to the cloud server 30 is not limited, that is, different types of interactive devices 10 such as an interactive robot and a smartphone may be connected to the cloud server 30.

(Interactive device)
Next, the configuration of the interactive apparatus 10 will be described. The dialogue device 10 is a device that performs voice recognition when a voice (voice signal) is inputted and performs a dialogue according to the recognition result. As shown in FIG. 1, the dialogue apparatus 10 includes a voice input unit 11, a voice output unit 12, a control unit 13, a data storage unit 14, and a communication unit 15.

The voice input unit 11 is a voice input device such as a microphone, and the voice output unit 12 is a voice output device such as a speaker.

The control unit 13 is a block that controls the operation of each unit of the dialogue apparatus 10. For example, the control unit 13 includes a computer device including an arithmetic processing unit such as a CPU (Central Processing Unit) or a dedicated processor. The control unit 13 reads out and executes a program for executing various controls in the interactive device 10 stored in the data storage unit 14, thereby controlling the operation of each unit of the interactive device 10 in an integrated manner.

Further, the control unit 13 has functions as a speech recognition unit 16, a response information acquisition unit 17, an output control unit 18, and a speech synthesis unit 19.

The voice recognition unit 16 is a block that recognizes an input voice from the user. Specifically, the voice recognition unit 16 converts voice data input from the voice input unit 11 into text data, analyzes the text data, and extracts words and phrases. A known technique can be used for voice recognition processing.

The response information acquisition unit 17 is a block that detects response information indicating response content according to the recognition result of the voice recognition unit 16 from a first response information storage unit (response information storage unit) 141 described below. In the present embodiment, the response information that the response information acquisition unit 17 acquires from the first response information storage unit 141 is referred to as first response information. The response information acquisition unit 17 refers to the first response information storage unit 141 and acquires the first response information corresponding to the words and phrases extracted by the voice recognition unit 16. If the information corresponding to the word or phrase extracted by the speech recognition unit 16 is not registered in the first response information storage unit 141 or if the speech recognition unit 16 has failed in speech recognition, the response information acquisition unit 17 Obtains default first response information. Specific examples of the default first response information include information such as “Please wait for a while” or “Let me hear” (for example, “Please wait for a while” or “Listen Information that becomes "I see". Note that the present invention is not limited to these.

The output control unit 18 is a block that performs audio output by causing the audio output unit 12 to output audio data. The output control unit 18 performs the second response process continuously after performing the first response process as a response to the input voice from the voice input unit 11. The first response process is a process of outputting the response content indicated by the first response information acquired by the response information acquisition unit 17 by voice. The second response process is the second response information received from the cloud server 30. This is a process for outputting the displayed response content by voice. The second response information will be described later.

The speech synthesizer 19 is a block that generates speech data (speech synthesis). The voice synthesizer 19 generates voice data having response contents indicated by the first response information. The generated audio data is output via the audio output unit 12. If the first response information is generated as voice data (recorded voice), the voice synthesizer 19 does not generate it.

The data storage unit 14 includes a RAM (Random Access Memory), a ROM (Read Only Memory), an HDD (Hard Disk Drive), and the like, and is a block that stores various information (data) used in the interactive device 10. Further, the data storage unit 14 includes a first response information storage unit 141. The first response information storage unit 141 is a database in which first response information is registered in association with words and phrases. The first response information includes not only information corresponding to one word but also information corresponding to a combination of a plurality of words. In addition, a plurality of pieces of first correspondence information may be registered corresponding to a certain word or a certain phrase, and in this case, what is actually output by voice may be selected. Note that the words, phrases, and first response information may all be stored as text data. Known techniques can be used to construct such a database and to obtain response information from the database.

In this way, the dialogue apparatus 10 can return a response to the user's utterance by referring to the first response information storage unit 141, that is, the dialogue with the user becomes possible.

The communication unit 15 is a block that performs communication with the outside. When the voice data is input from the voice input unit 11 under the control of the control unit 13, the communication unit 15 transmits the voice data to the cloud server 30. And the 2nd response information which shows the response content according to the result of the speech recognition of the input sound by the cloud server 30 is received from the cloud server 30 explained in full detail behind. In the present embodiment, the communication unit 15 transmits the voice data input from the voice input unit to the cloud server 30 as it is. However, the communication unit 15 uses the text data generated by the voice recognition unit 16 or the text data. You may transmit the word and phrase extracted from to the cloud server 30. FIG.

Here, in the present embodiment, the output control unit 18 is configured to perform the first response processing while the communication unit 15 receives the second response information from the cloud server 30.

The dialogue apparatus 10 may further include an imaging unit (camera). For example, the dialogue apparatus 10 is configured to analyze a user's facial expression and position from an image input from the imaging unit and perform a dialogue based on the analysis. Also good. For example, when the dialogue apparatus 10 is a robot, when it is recognized that the user's position is rightward when viewed from the front of the robot, the head of the robot is actually turned to the right, or the face is turned to the right It may be configured to indicate a state of facing the user, that is, being able to respond by displaying a state of moving toward.

(Cloud server configuration)
Next, the cloud server 30 will be described. The cloud server 30 is a server that generates a response to voice data (input voice) received from the dialogue apparatus 10 and transmits the response to the dialogue apparatus 10. In addition, the cloud server 30 is a server that manages the interactive device 10. When a plurality of interactive devices 10 are connected, the cloud server 30 manages each individually. The cloud server 30 may also manage information related to the user of the interactive device 10. In this case, information related to the user can be registered in the cloud server 30 from an external device such as a smartphone or a tablet. Also good.

In this embodiment, the server device connected to the interactive device 10 will be described using the cloud server 30 that provides a cloud service, but is not limited to the cloud server. Moreover, the cloud server 30 may be one unit or a plurality of units connected via a communication network.

The cloud server 30 includes a control unit 31, a data storage unit 32, and a communication unit 33, as shown in FIG.

The control unit 31 includes a computer device configured by an arithmetic processing unit such as a CPU or a dedicated processor, and is a block that controls the operation of each unit of the cloud server 30. In addition, the control unit 31 has functions as a speech recognition unit 34, a response information generation unit 35, and a speech synthesis unit 36.

The voice recognition unit 34 is a block having the same function as the voice recognition unit 16 of the dialogue apparatus 10. However, the speech recognition capability (performance) is higher than that of the speech recognition unit 16. Thereby, even if the dialogue apparatus 10 cannot recognize the voice, the cloud server 30 can recognize the voice.

The response information generation unit 35 is a block that generates response information following the first response information. In the present embodiment, the response information generated by the response information generation unit 35 is referred to as second response information. When the response information generation unit 35 generates the second response information, the response information generation unit 35 detects response information indicating the response content according to the recognition result of the voice recognition unit 34 from the second response information storage unit 321 described below. Then, the second response information is generated.

The voice synthesizer 36 is a block that generates voice data. The voice synthesizer 36 is a block that generates voice data of response contents indicated by the second response information generated by the response information generator 35.

Furthermore, the cloud server 30 is configured to receive information (external provision information) from an external information provision server via a communication network. Therefore, the response information generation unit 35 may generate the second response information based on the externally provided information, the user information registered in the cloud server 30 described above, or a combination thereof. Specific examples of the externally provided information include weather information, traffic information, disaster information, and the like, but are not limited thereto. Further, the number of information providing servers that provide information to the cloud server 30 is not limited.

In this way, by using the externally provided information when generating the second response information, for example, the voice of the response content indicated by the first response information (the first response) with respect to the input voice “Good morning” The voice of the response content indicated by the second response information (the voice output by the second response information) is “Today because today ’s weather is cloudy and rainy” It ’s better to take an umbrella with you. ” In this case, the externally provided information is weather information, and the second response information is generated based on the weather information.

As described above, since the second response information is continuously output after voice output of the first response information, the second response information is a continuous content or a detailed content of the first response information. It is preferable because a sense of unity is born in the response. In this case, the cloud server 30 knows in advance which first response information is output from the interactive device 10 for which input voice, or when the interactive device 10 acquires the first response information. The cloud server 30 may be notified.

The voice data of the response content indicated by the second response information generated by the cloud server 30 is transmitted to the dialogue apparatus 10 by the control unit 31 controlling the communication unit 33.

In the present embodiment, the cloud server 30 is configured to generate the second response information as voice data and then transmit it, so the load on the interactive device 10 can be reduced. Here, since each of the dialogue apparatus 10 and the cloud server 30 has a voice synthesis unit, even if the voice quality of the synthesized voice differs, the first response process as described in the second embodiment. By changing the character that appears in the dialogue apparatus 10 in the second response process, it is possible to eliminate the uncomfortable feeling that the user feels. The cloud server 30 may not have the voice synthesizer 36 and may be configured to transmit the second response information to the dialogue apparatus 10 as text data. In this case, the second response information is generated as voice data by the voice synthesizer 19 of the dialogue apparatus 10.

Also, the cloud server 30 may be able to register recorded voices from an external device such as a smartphone or a tablet, for example. In this case, the response information generation unit 35 may acquire the registered recorded voice as the second response information, or may be included in the generation of the second response information. Since the recorded voice is formed as voice data, if it is transmitted to the dialogue apparatus 10 as it is, the voice synthesis process in the dialogue apparatus 10 is not performed. For example, when the voice “There is a cake in the refrigerator” is registered in the cloud server 30 from the smartphone of the user ’s mother, the dialogue apparatus 10 receives the first response information in response to the input voice of the user “I ’m right”. Use it to output “Okaeri” as a voice, and then use the second response information to say “A message from your mother.“ There is a cake in the refrigerator. ” It is possible to perform an advanced response such as outputting “

The data storage unit 32 is a block that stores various information (data) used in the cloud server 30. Further, the data storage unit 32 includes a second response information storage unit 321. The second response information storage unit 321 is a database in which second response information is registered in association with words and phrases. The second response information storage unit 321 stores a larger amount of information than the first response information storage unit 141. Further, the second response information may be updated periodically.

In the interactive system 100, even if the interactive device 10 cannot perform voice recognition or the first response information has simple response contents, the response information generating unit 35 and the second response information of the cloud server 30 as described above. Since the storage unit 321 is configured, it is possible to correctly recognize the input voice and to return a plurality of information.

The communication unit 33 is a block that performs communication with the outside. The communication unit 33 is connected to an external information providing server (not shown) or an external device such as a smartphone or a tablet through a communication network in addition to the interactive device 10. Note that the number of devices connected to the cloud server 30 is not limited.

(Processing flow in dialogue system)
Next, the flow of dialogue processing in the dialogue system 100 will be described with reference to FIG.

When the dialogue apparatus 10 receives the voice data (input voice) of the utterance from the user 2 (step A1), the dialogue apparatus 10 transmits the received voice data to the cloud server 30 (step A2) and acquires the first response information (step A2). Step A3). In step A3, the input speech is recognized as speech, and first response information indicating response contents according to the result of speech recognition is acquired. Note that either step A2 or step A3 may be started first. Then, the dialogue apparatus 10 outputs the response content indicated by the first response information as a voice (step A4).

On the other hand, when the cloud server 30 receives the voice data from the interactive device 10 (step B1), the cloud server 30 generates second response information (step B2), and transmits the generated second response information to the interactive device 10 (step B3). .

Here, while receiving the second response information from the cloud server 30, the dialogue apparatus 10 outputs the response content indicated by the first response information as a voice.

When the interactive device 10 receives the second response information (step A5), the dialogue device 10 outputs the following response content indicated by the second response information as a voice (step A6). Thus, the dialogue process in the dialogue system 100 ends.

As described above, in the dialogue system 100, the dialogue device 10 continuously outputs the response content indicated by the first response information associated with the result of the voice recognition performed by the voice recognition unit 16, after the voice is output, The response content indicated by the second response information associated with the result of the speech recognition by the server 30 is output as speech.

Here, since the first response information is in accordance with the result of voice recognition in the dialog device 10, the first response information is output from the dialog device 10 earlier than the second response information received through communication with the cloud server 30. Can do. Moreover, since the cloud server 30 can perform advanced processing than the individual interactive devices 10, it can perform advanced speech recognition. Therefore, the interactive device 10 can make a quick response to the input voice by the first response process, and can provide various or advanced information by the second response process that continues after the first response process.

As described above, according to the dialogue system 100, in addition to the voice recognition in the dialogue device 10 and the response in the first response information, the voice recognition in the cloud server 30 and the second response information in response to the input voice from the user. Since it can be supplemented by a response, it is possible to respond with a plurality of pieces of information without improving the processing capability of the voice recognition unit 16 of the interactive apparatus 10 and expanding the capacity of the data storage unit 14. Therefore, the interactive system 100 can smoothly output a plurality of information as voices, and can provide a comfortable interactive environment without stressing the user.

Furthermore, in the interactive system 100, in the interactive device 10, the output control unit 18 is configured to perform the first response process while the communication unit 15 receives the second response information from the cloud server 30. The user can interact without stressing the waiting time for receiving the second response information.

[Embodiment 2]
An interactive system according to another embodiment of the present invention will be described with reference to FIGS. For convenience of explanation, members having the same functions as those explained in the first embodiment are given the same reference numerals and explanations thereof are omitted.

As shown in FIG. 3, the interactive system 100a according to the present embodiment includes an interactive device 10a and a cloud server (server device) 30a. In the present embodiment, the dialogue apparatus 10a will be described as a dialogue robot having a head and a torso that can express facial expressions as shown in FIG.

The interactive device 10 a includes a display unit 21 and an operation unit 22 in addition to the configuration of the interactive device 10 of the first embodiment. The display unit 21 is a block that displays an image of the expression of the interactive robot. In the present embodiment, the display unit 21 performs display using the rear projection method, but is not limited thereto. The operation unit 22 is a block that executes the operation of the interactive apparatus 10a. As will be described below, the operation unit 22 performs an operation of rotating the dialogue apparatus 10a at the time of character switching between the first response process and the second response process. This rotation may be a motion other than the rotation, and as shown in FIG. 4A, the head of the dialog robot as the dialog device 10a rotates in the horizontal direction. Further, the operation unit 22 may be configured to operate the head, the body, or the arm attached to the body of the dialog robot that is the dialog device 10a in various directions. Further, at the time of character switching, the motion unit 22 may perform an exercise other than rotation, and may execute, for example, movement, direction change, vibration, and the like. Here, the exercise means that at least a part of the interactive device performs a physical operation.

Further, the dialogue apparatus 10a includes a character storage unit (character feature amount storage unit) 142 that stores image feature amounts and audio feature amounts of a plurality of characters in the data storage unit 14a. In the interactive device 10a, the control unit 13a has a function as the character switching unit 20. At the time of character switching, the character switching unit 20 controls at least one of the image display, the sound output, and the exercise in the interactive device 10a, and performs an effect process indicating that a different character has appeared in the interactive device 10a. It is a block. The effect process will be described below using a specific example.

In the interactive apparatus 10a, the output control unit 18a selects one of a plurality of characters during the first response process and the second response process, and refers to the character storage unit 142 of the selected character. The character image determined by the feature amount of the obtained image is displayed on the display unit 21, and the character voice determined by the voice feature amount obtained by referring to the character storage unit 142 of the selected character is displayed on the voice output unit 12. Output audio. In this way, the interactive device 10a changes the feature amount of the image displayed on the display unit 21 and the sound quality of the sound output from the audio output unit 12 between the first response process and the second response process, Different characters can appear in the dialogue apparatus 10a. Further, the operation may be changed. The character is, for example, a child, a father, a mother, a teacher, a news caster, or the like.

Furthermore, in the interactive system 100a, the cloud server 30a generates the second response information including the character designation information for designating the character when the dialogue apparatus 10a outputs the second response information as a voice. The character designation information designates a character that can appear on the dialogue apparatus 10a. The character designation information may be, for example, information corresponding to the content to be output as voice by the second response information. For example, if the content to be output in the second response information is content related to study, it is information that specifies the character of father, if it is content related to life, mom, and if it is content related to climate, it is a character that specifies the weather caster. . In addition, these are illustrations and are not limited to these.

Next, the effect process at the time of character switching will be described using a specific example. The output control unit 18a selects a default character during the first response process, and selects a character designated by the character designation information during the second response process. Here, when the output control unit 18a selects a different character during the second response process than during the first response process, the character switching unit 20 performs an effect process indicating that a different character has appeared in the dialogue apparatus 10a during the character switching. The following processing is performed.

As shown in FIG. 4A, the head of the dialogue apparatus 10a is started to rotate while the character image of the first character, which is the character selected during the first response process, is displayed. Then, when the rotation of the head is finished, the display unit 21 and the operation unit 22 are controlled so that the character image of the second character selected during the first response process is displayed instead of the character image of the first character. To do. At this time, in a state where the first character appears, a voice calling the second character (for example, “Mr. XX who is in charge of news!”) May be output as a voice. Further, during the rotation, the footsteps that the first character moves away and the footsteps that the second character approaches may be output as audio.

Alternatively, the character switching unit 20 may perform the following process as the effect process. As shown in (b) or (c) of FIG. 4, the display is gradually switched from the character image of the first character to the character image of the second character that is the character selected during the second response process. The display unit 21 is controlled. In this effect process, the interactive device 10a itself does not rotate, but displays to appear to rotate. Also in this effect processing, an effect by voice output may be performed in the same manner as described above.

Further, when the second response information cannot be correctly received by the dialogue apparatus 10a, the output control unit 18a does not switch the character between the first response process and the second response process. In this case, for example, a default audio output such as “I'm busy, ask me later” or “Newscaster is off today” may be made.

(Processing flow in dialogue system)
The flow of dialogue processing in the dialogue system 100 is basically the same as the flow of processing in the dialogue system 100 of the first embodiment shown in FIG. However, in the interactive system 100a, in the process B2, the second response information is generated including the character designation information for designating the character when the second response information is output as a voice. In step A4, the first character appears in the dialogue device 10a. In step A6, the character switching unit 20 performs the above-described effect processing, and the second character designated in the character designation information is designated as the dialogue device 10a. To appear. Here, for example, if the character switching unit 20 performs the above-described effect processing after the voice output “Please wait a moment” is performed, it is possible to easily tell the user that the character is changed.

In the above description, it has been described that the second response information includes information specifying any one of a plurality of characters and is received by the interactive device 10a. The interactive device 10a is as follows. It may be configured. The output control unit 18 is configured to determine which one of the plurality of characters is designated from the recognition result of the input speech by the speech recognition unit 16, and to select the designated character at the time of the second response process. It may be.

In this case, the output control unit 18 may determine the specified character if the input voice by the user includes information specifying the character itself, or may determine the character from the metadata in the dialog with the user. You may judge by analogy. This will be explained using a specific example. In the former case, if the voice input by the user includes the commands “teacher”, “call the teacher”, and “speak teacher”, select the character “teacher”. The latter means that if the dialogue with the user is related to study, the character of “teacher” is selected.

Note that the above determination may not be performed by the output control unit 18, and a block for performing the above determination based on the result of speech recognition by the speech recognition unit 16 may be provided separately.

As described above, the second response information can be provided using the character reflecting the user's intention by selecting the character from the content of the dialogue with the user. In addition, the conversation can be interesting.

Furthermore, the output control unit 18 selects any one of a plurality of characters when the interactive device 10a is activated or returns from the sleep state, and displays an image of the selected character on the display unit 21 as well as the selection. The voice output unit 12 may output the voice of the character.

When starting up or returning from the sleep state, the user who wants to interact will be kept waiting, which may cause stress on the user. However, by performing the display and voice output as described above, a character can appear in the interactive device, and a user's stress can be reduced with a gap. When the interactive device 10a is activated or the character image or sound shown when the interactive device 10a returns from the sleep state, for example, indicates that the interactive robot has awakened from sleep, the interactive device 10a is in the process of being activated. I can convey this in an easy-to-understand manner.

[Embodiment 3]
Hereinafter, a dialogue system according to still another embodiment of the present invention will be described with reference to FIGS. For convenience of explanation, members having the same functions as those described in the first or second embodiment are denoted by the same reference numerals and description thereof is omitted.

As shown in FIG. 6, the dialogue system 100b of the present embodiment includes a dialogue device 10b and a cloud server (server device) 30b. The dialogue apparatus 10b will be described below assuming that it is a dialogue robot.

As shown in FIG. 5, the interactive device 10 b includes a display unit 21 and an operation unit 22 in the same manner as the interactive device 10 a of the second embodiment, in addition to the configuration of the interactive device 10 of the first embodiment. Moreover, the interactive device 10b has a home appliance operation mode in which home appliances can be operated for each home appliance. As illustrated in FIG. 6, the home appliance in the user home 40 is connected to the infrared communication or wireless LAN communication from the communication unit 15. It is provided so that it can be operated. Home appliances are, for example, air conditioners (air conditioners), washing machines, refrigerators, cooking utensils, lighting devices, hot water supply equipment, photographing equipment, various AV (Audio-Visual) equipment, various household robots (for example, cleaning robots, housework support) Robot, animal type robot, etc.). In the present embodiment, description will be made using an air conditioner 50-1 and a washing machine 50-2 as examples of home appliances.

The interactive device 10b can operate the air conditioner 50-1 and the washing machine 50-2. The control unit 13b of the interactive device 10b has a function as the mode setting unit 23 that sets the interactive device 10b to the home appliance operation mode. The mode setting unit 23 determines a home appliance to be operated from the input voice input from the voice input unit 11, and sets the interactive device 10b to the home appliance operation mode of the determined home appliance. Therefore, when it is inferred that the user wishes to operate the air conditioner 50-1 from the dialog with the user, the dialog device 10b can set and operate the dialog device 10b in the home appliance operation mode for operating the air conditioner 50-1. It becomes possible.

When determining the home appliance to be operated from the input voice, the output control unit 18 may make a determination if the input voice includes information for designating the home appliance to be operated. The determination may be made by analogy with the home appliance to be operated from the data. This will be described using a specific example. The former determines that the home appliance to be operated is the air conditioner 50-1 if the input voice by the user includes the command “turn on air conditioner” or “air conditioner ON”. The latter means that if the input voice includes “hot” metadata, it is determined that the home appliance to be operated is the air conditioner 50-1.

Here, before the operation of the air conditioner 50-1 from the interactive device 10b, a sound for confirming the execution of the operation, for example, a voice such as “Would you like to turn on the air conditioner?” Is output from the interactive device 10b and executed by the user. The operation is executed when a voice to be permitted, for example, a voice input such as “Turn on” or “OK” is made. As described above, it is preferable to secure the user's confirmation before executing the operation of the home appliance in order to ensure safety.

The data storage unit 14b of the interactive device 10b includes a mode information storage unit 143. The mode information storage unit 143 stores information for setting the interactive device 10b for each home appliance so that the home appliance can be operated. Yes. Further, the mode information storage unit 143 stores the feature amount of the character image and the feature amount of the voice associated with the home appliance for each home appliance.

The character associated with the home appliance appears when the interactive device 10b is set to the home appliance operation mode of the home appliance. The appearance of the character can be executed by changing the feature quantity of the image displayed on the display unit 21 and the sound quality of the audio output from the audio output unit 12 as in the second embodiment. Further, the operation may be changed.

As a specific example, when the interactive device 10b is set to the operation mode of the air conditioner 50-1, a character associated with the air conditioner appears in the interactive device 10b. In this case, for example, the mark of the air conditioner is displayed on the forehead part or the stomach part (part of the display unit) of the interactive robot. Similarly, when the interactive device 10b is set to the operation mode of the washing machine 50-2, a character associated with the washing machine appears in the interactive device 10b.

Alternatively, the user may be able to register the association between the home appliance and the character in the dialogue apparatus 10b, such as a rabbit character for an air conditioner and a raccoon character for a washing machine. These are merely examples.

When a character associated with such a home appliance appears in the interaction device 10b, the user is notified of which home appliance operation mode the interaction device 10b can operate, that is, which is the operation target home appliance. be able to.

Further, for example, when a character associated with the air conditioner 50-1 appears in the dialogue apparatus 10b, the dialogue apparatus 10b outputs a voice signal “break” in response to a voice input “break”. In addition, an operation for turning off the power of the air conditioner 50-1 may be performed. When a character associated with the washing machine 50-2 appears, the dialogue apparatus 10b outputs a voice of "Good morning" in response to the voice input of "Good morning", and at the same time, the washing machine 50 -2 may be configured to turn on the power. Thus, you may have a different function for every character matched with the household appliance.

Further, when the interactive device 10b also has a function as a television device, it may be configured to project a broadcast program on the display unit 21 when the home appliance operation mode for operating the television is set. Good.

When operating the air conditioner 50-1 and the washing machine 50-2 from the interactive apparatus 10b, the positions of the air conditioner 50-1 and the washing machine 50-2 may be detected using infrared rays, or the interactive apparatus 10b may be connected to a camera. The positions of the air conditioner 50-1 and the washing machine 50-2 may be detected from information obtained from the camera.

Since the configuration of the cloud server 30b is the same as that of the cloud server 30, description thereof is omitted. When the cloud server 30 is communicatively connected to the air conditioner 50-1 and the washing machine 50-2 and collects state information indicating these states, the cloud server 30 performs the operation based on the state information. Two response information may be generated. For example, the second response information may be generated so as to obtain the state information of the washing machine 50-2 and output a voice saying that the washing machine has finished work.

[Embodiment 4]
The

interactive devices

10, 10a, 10b and the

cloud servers

30, 30a, 30b described in the first to third embodiments are each realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like. Alternatively, it may be realized by software using a CPU (Central Processing Unit).

In the latter case, each of the

interactive devices

10, 10a, 10b and the

cloud servers

30, 30a, 30b includes a CPU that executes instructions of a program that is software that realizes each function, and the program and various data are computers (or CPUs). ROM (Read Only Memory) or storage device (these are referred to as “recording media”), a RAM (Random Access 展開 Memory) that expands the program, and the like. And the objective of this invention is achieved when a computer (or CPU) reads the said program from the said recording medium and runs it. As the recording medium, a “non-temporary tangible medium” such as a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. The program may be supplied to the computer via an arbitrary transmission medium (such as a communication network or a broadcast wave) that can transmit the program. The present invention can also be realized in the form of a data signal embedded in a carrier wave in which the program is embodied by electronic transmission.

The present invention is not limited to the above-described embodiments, and various modifications are possible, and the present invention also relates to embodiments obtained by appropriately combining technical means disclosed in different embodiments. Is included in the technical scope. Furthermore, a new technical feature can be formed by combining the technical means disclosed in each embodiment.

[Summary]
A dialog device (10) according to an aspect 1 of the present invention shows a speech recognition unit (16) that recognizes an input voice that has been input, and a response content corresponding to a result of speech recognition by the speech recognition unit (16). The response information storage unit (first response information storage unit 141) for storing the first response information and the input voice are transmitted to the server device (cloud server 30), and the result of the voice recognition of the input voice by the server device. The communication unit (15) that receives the second response information indicating the corresponding response content, and the response content indicated by the first response information obtained by referring to the response information storage unit to the input voice And an output control unit (18) for performing a second response process for outputting the response content indicated by the second response information continuously after performing the first response process to be output.

According to the above configuration, after the response content indicated by the first response information associated with the result of speech recognition performed by the speech recognition unit of the own device is output as speech, the result of speech recognition by the server device is continuously displayed. The response content indicated by the associated second response information is output as a voice.

Since the first response information is in accordance with the result of speech recognition in the own device, it can generally be output earlier from the dialogue device than the second response information received through communication with the server device. In general, the server device can perform a higher level of processing than an individual interactive device, and thus can perform high-level voice recognition. Therefore, according to the above configuration, a quick response to the input voice can be made by the first response process, and various or advanced information can be provided by the second response process that follows the first response process.

According to the above configuration, since the input speech can be supplemented by the speech recognition of the server device and the response of the second response information in addition to the speech recognition of the own device and the response of the first response information, the speech recognition of the dialogue device It is possible to respond with a plurality of pieces of information without increasing the processing capability of each section and expanding the capacity of the response information storage section. Thus, according to the above configuration, it is possible to smoothly output a plurality of pieces of information, and it is possible to provide a comfortable interactive environment without giving stress to the user.

Furthermore, if the first response process is performed while the second response information is being received, the user can further interact without stressing the waiting time for receiving the second response information.

The interactive apparatus according to aspect 2 of the present invention further includes a character feature amount storage unit (character storage unit 142) that stores image feature amounts and audio feature amounts of a plurality of characters in the above aspect 1, and the output control described above. The unit (18) selects one of the plurality of characters at the time of the first response process and the second response process, and refers to the character feature amount storage unit of the selected character. A character image determined by the feature amount of the obtained image, and voice output of the selected character determined by the feature amount of the voice obtained by referring to the character feature amount storage unit, and the output control When the part (18) selects a character different from the first response process during the second response process, Between the time of 1 response processing and the time of the second response processing, at least one operation of image display, sound output, and exercise is controlled by the own device, and an effect representing the appearance of a different character is performed. A character switching unit (20) is further provided.

According to the above configuration, at the time of the first response process and the second response process, the character is selected, the character image of the selected character is displayed, and the character voice of the selected character is output by voice, thereby the dialogue device You can make characters appear. When a character different from the first response process is selected during the second response process, an effect indicating that a different character has appeared is performed. This effect makes it possible to excite the atmosphere of the appearance of different characters without destroying the image of the character before the appearance due to the appearance of different characters.

In addition, since the user's interest can be attracted by the above-described production by the above control, for example, even when the time from the first response process to the second response process elapses, the user's stress due to the waiting time is reduced. can do.

Here, the character switching unit (a) from the character image of the first character that is the character selected during the first response process, the character of the second character that is the character selected during the second response process. Control the display to be gradually switched to the image, or (b) Start the rotation of the device with the character image of the first character displayed, and when the rotation ends, the first character Different characters appeared by controlling the display of the image and the rotation, movement, direction change, vibration, etc. of the device so that the character image of the second character is displayed instead of the character image of An effect representing this may be performed.

In the interactive device according to aspect 3 of the present invention, in the aspect 2, the second response information includes designation information for designating any one of the plurality of characters, and the output control unit ( 18) selects a character designated by the designation information included in the second response information during the second response process.

According to the above configuration, the character designation information is included in the second response information. Therefore, if a character suitable for the content of the second response information is specified in advance, the specified character can appear at the time of the second response process, so that it is persuasive or interesting. The second response information can be provided to the user.

In the dialog device according to aspect 4 of the present invention, in the aspect 2, the output control unit (18) determines any one of the plurality of characters from the result of speech recognition of the input speech by the speech recognition unit (16). It is determined whether one is designated, and the designated character is selected during the second response process.

According to the above configuration, a character is designated from the voice input by the user, and the designated character can appear in the dialogue device during the second response process. Here, if the information that specifies the character itself is included in the input voice by the user, the specified character may be determined based on the character or by analogizing the character from the metadata in the dialog with the user. You may judge. According to the said structure, 2nd response information can be provided using the character reflecting a user's intention. In addition, the conversation can be interesting.

In the dialog device according to aspect 5 of the present invention, in the above aspect 2, the output control unit (18) selects any one of the plurality of characters when the own device is activated or when returning from the sleep state. The character image determined by the feature amount of the image obtained by referring to the character feature amount storage unit of the selected character is displayed, and the voice of the selected character obtained by referring to the character feature amount storage unit The character voice determined by the feature amount is output as a voice.

According to the above configuration, the character is selected at the time of starting up the device or returning from the sleep state, the character image of the selected character is displayed, and the character voice of the selected character is output as a voice to the dialogue device. Can appear. The appearance of such a character can attract the user's interest and can reduce the stress in the waiting time at the time of activation or return from the sleep state.

The interactive apparatus according to aspect 6 of the present invention has the home appliance operation mode capable of operating home appliances for each home appliance in any one of the above aspects 1 to 5, and the input by the voice recognition unit (16). A mode setting unit (23) is further provided that determines a home appliance to be operated from the result of voice recognition of the voice and sets the own apparatus in the home appliance operation mode of the determined home appliance.

According to the above configuration, the interactive device can be set to the home appliance operation mode in which the home appliance can be operated, and the home appliance to be operated can be determined from the input voice. Therefore, when it is inferred that the user wants to operate the home appliance from the dialog with the user, the interactive device can set the own device to the home appliance operation mode for operating the home appliance and perform the operation.

Here, for example, when the interactive device includes a display unit and is configured to display an operation target home appliance or a character representing the home appliance when the home appliance operation mode is set, the operation target home appliance is displayed to the user. Can be clearly notified.

A dialogue system according to aspect 7 of the present invention is configured by connecting the dialogue apparatus according to any one of aspects 1 to 6 above and a server apparatus having a voice recognition function via a communication network.

According to the above dialogue system, it is possible to respond smoothly to the voice input by the user, and it is possible to provide a comfortable dialogue environment without stressing the user.

The server device according to aspect 8 of the present invention is a server device provided in the interactive system according to aspect 7 described above.

By using the server device, the dialogue system according to aspect 7 can be constructed.

In addition, the interactive device, the server device, or the interactive system according to each aspect of the present invention may be realized by a computer. In this case, each unit (speech recognition unit, computer) included in the interactive device, the server device, or the interactive system. An output control unit, a character switching unit, a voice recognition unit, a mode setting unit), and a program for realizing a dialog device, a server device, or a dialog system on a computer and a computer-readable recording medium on which the program is recorded It falls into the category of the invention.

The present invention can be used for an interactive device connected to a communication network and recognizing and responding to a user's voice.

2

User

10, 10a, 10b Dialogue device 11 Voice input unit 12

Voice output unit

13, 13a,

13b Control unit

14, 14a, 14b Data storage unit 15 Communication unit 16 Voice recognition unit 17 Response

information acquisition unit

18, 18a Output control unit 20 character switching unit 21 display unit 22 operation unit 23

mode setting unit

30, 30a, 30b cloud server (server device)
35 Response information generation unit 40 User's house 50-1 Air conditioner (home appliance)
50-2 Washing machine (home appliance)
100, 100a, 100b Dialog system 141 First response information storage (response information storage)
142 Character storage unit 143 Mode information storage unit

Claims

A speech recognition unit that recognizes the input speech that has been input;
A response information storage unit for storing first response information indicating a response content according to a result of voice recognition by the voice recognition unit;
A communication unit that transmits the input voice to a server device and receives second response information indicating response content according to a result of voice recognition of the input voice by the server device;
After the first response processing for outputting the response content indicated by the first response information obtained by referring to the response information storage unit with respect to the input voice, the second response information is continuously indicated. And an output control unit for performing a second response process for outputting the content of the response to be voiced.
A character feature amount storage unit for storing feature values of images and voice features of a plurality of characters;
The output control unit selects one of the plurality of characters at the time of the first response process and the second response process, and refers to the character feature amount storage unit of the selected character. The character image determined by the feature amount of the obtained image is displayed, and the character voice determined by the feature amount of the voice obtained by referring to the character feature amount storage unit of the selected character is output as voice.
When the output control unit selects a character different from the first response process at the time of the second response process, between the first response process and the second response process, the image display on its own device, The interactive apparatus according to claim 1, further comprising a character switching unit that controls at least one action of voice output and exercise to perform an effect indicating that a different character has appeared.
The second response information includes designation information for designating any one of the plurality of characters.
The dialogue apparatus according to claim 2, wherein the output control unit selects a character designated by the designation information included in the second response information during the second response process.
The output control unit determines which one of the plurality of characters is designated from the result of speech recognition of the input speech by the speech recognition unit, and selects the designated character during the second response process. The interactive apparatus according to claim 2.
Each home appliance has a home appliance operation mode that can operate home appliances.
The apparatus further comprises: a mode setting unit that determines a home appliance to be operated from a result of voice recognition of the input voice by the voice recognition unit, and sets the own device in a home appliance operation mode of the determined home appliance. 5. The interactive apparatus according to any one of 1 to 4.