JP5610283B2 - External device control apparatus, external device control method and program - Google Patents

Description

  The present invention relates to an external device control apparatus, its external device control method, and program, and more particularly to a technique for recognizing a user's voice and operating the external device according to the recognized voice.

  In recent years, many robots that support human life have been developed. Humanoid robots, which are a type of robot, are robots that have the same shape as humans, and have advantages such as less psychological burden on humans and easy adaptation to living spaces designed for humans. Many of these robots are devised so that they can communicate with humans in a natural way by means of speech recognition functions and gesture functions.

  On the other hand, electronic devices having a voice recognition function are also becoming popular. In a car navigation system, since voice recognition has an overwhelming advantage that hands-free input is possible compared to means such as a remote controller and a touch panel, a voice recognition function has been widely used from a relatively early stage. In portable terminals such as mobile phones and smartphones, input devices such as keyboards are small and difficult to use, and voice input is becoming increasingly popular because it is simpler than that.

  Electrical appliances such as air conditioners and televisions have a problem that their functions are diversified while operations are complicated and users cannot use the functions sufficiently. Although solutions with technologies such as speech recognition and speech understanding are desired, there are technical problems such as recognition accuracy and cost, and the current situation is that they are not widely used.

  From another perspective, electronic devices that are currently used tend to include a wide variety of functions for reasons such as pursuing further convenience, stimulating consumers' willingness to purchase, and universal design. For example, technologies have been developed to recommend TV programs and save energy by operating air conditioners by remotely operating devices from outside the network and acquiring and analyzing information about users and their environment using sensors. ing.

  Currently, most of these functions are implemented individually for each electronic device, but in the future all electronic devices will be connected to the network, and will exchange information and realize more advanced functions. It will be possible. As one implementation form, a centralized configuration in which the center server collects and analyzes information on each electronic device and appropriately controls each device is conceivable.

  Based on these backgrounds, the robot plays the role of the center server, collects information on each electronic device, and interacts with the user to clarify the potential demands of the user and comprehensively summarize the results. A system that uses it to control electronic devices is considered useful. For example, Patent Document 1 discloses an invention in which a robot interacts with a user and controls other electronic devices connected to a network.

JP 2005-333495 A

  Various other methods are conceivable for the above-described cooperation method between the robot and the electronic device. However, when the electronic device itself is equipped with a voice recognition function and the robot has spread to the home, it cannot be said that any of the known methods make full use of the voice recognition function. For example, when a robot configures a system that controls other electronic devices as in Patent Document 1, various problems emerge.

  One problem is when it is clear that the user desires to operate an electronic device. When a user tries to operate an electronic device via a robot, if the robot is far away, it is necessary to call it close to you, or if the robot determines that you are bored, you can chat with the electronic device. Inconvenient situations can occur, such as the operation being hindered.

  In such a case, you may want to ignore the robot and operate the electronic device directly. However, speech input to electronic devices is generally poorer in language understanding than robots, and only specific words are often accepted. Therefore, the probability that a user who normally operates an electronic device via a robot will utter an appropriate voice input word and succeed in the operation is low. It is hard to say that such a situation makes effective use of robots.

  Another problem is that the user may feel discomfort when the robot controls the electronic device via the network. The robot's functions and appearance approach humans, making it easier for users to perceive robots as being equivalent to humans. However, when they do so, it is better for users to operate electronic devices in the same way as humans. It moves very naturally.

  One aspect of the present invention is an external device control apparatus that recognizes a user's voice and controls an external device, the receiving unit receiving the voice, and converting the voice received by the receiving device into a character string. Operation information for storing a speech recognition unit, a state information storage area for storing state information including a character string converted by the speech recognition unit, and operation information for associating an operation of the external device control device with a condition in the state information Whether status information including the character string satisfies a condition associated with an operation by voice and / or gesture of an external device with reference to the storage area, the status information storage area, and the motion information storage area And when the operation planning unit determines that the state information including the character string satisfies the condition, the audio waveform and / or gesture corresponding to the external device operation A pattern generator for generating a pattern, characterized in that it comprises an output unit for performing the voice waveform and / or gestures of the output operation in accordance with the pattern.

  According to the present invention, a user can learn an appropriate operation method using voice and / or gestures of an external device.

It is a block diagram showing typically the composition of the service robot concerning a first embodiment of the present invention. It is a flowchart which shows operation | movement of the external apparatus management program of the service robot which concerns on 1st embodiment of this invention. It is a figure which shows the example of the apparatus database of the service robot which concerns on 1st embodiment of this invention. It is a figure which shows the example of the vocabulary database of the service robot which concerns on 1st embodiment of this invention. It is a flowchart which shows operation | movement of the operation planning program of the service robot which concerns on 1st embodiment of this invention. It is a flowchart which shows operation | movement of the operation planning program of the service robot which concerns on 1st embodiment of this invention. It is a flowchart which shows operation | movement of the speech recognition program of the service robot which concerns on 1st embodiment of this invention. It is a figure which shows operation | movement of the user speech synthesis program of the service robot which concerns on 1st embodiment of this invention. It is a flowchart which shows operation | movement of the system speech synthesis program of the service robot which concerns on 1st embodiment of this invention. It is a figure which shows the example of the internal state database of the service robot which concerns on 1st embodiment of this invention. It is a figure which shows the example of the operation | movement database of the service robot which concerns on 1st embodiment of this invention. It is a figure which shows the example of the operation | movement queue of the service robot which concerns on 1st embodiment of this invention. It is a figure which shows the example of the internal state database of the service robot which concerns on 1st embodiment of this invention. It is a figure which shows the example of the internal state database of the service robot which concerns on 1st embodiment of this invention. It is a figure which shows the example of the operation | movement queue of the service robot which concerns on 1st embodiment of this invention. It is a figure which shows the example of the operation | movement queue of the service robot which concerns on 1st embodiment of this invention. It is a block diagram which shows typically the structure of the service robot which concerns on 2nd embodiment of this invention. It is a figure which shows the example of the gesture database of the service robot which concerns on 2nd embodiment of this invention. It is a flowchart which shows operation | movement of the operation planning program of the service robot which concerns on 2nd embodiment of this invention. It is a flowchart which shows operation | movement of the operation planning program of the service robot which concerns on 2nd embodiment of this invention. It is a figure which shows the example of the operation | movement queue of the service robot which concerns on 2nd embodiment of this invention.

  Below, the form for implementing this invention is demonstrated. For clarity of explanation, the following description and drawings are omitted and simplified as appropriate. Moreover, in each drawing, the same code | symbol is attached | subjected to the same element and the duplication description is abbreviate | omitted as needed for clarification of description.

  The service robot according to the present embodiment is characterized by a control function of an external electronic device. Electronic devices include air conditioners, television sets, microwave ovens, stereo sets, lighting devices, drive control devices for opening and closing members such as doors and windows.

  The service robot has a voice recognition function, and recognizes words uttered by the user. When the service robot is instructed to operate the external device by voice, the service robot operates the external device. In a preferred configuration, the service robot interacts with the user and operates an external device while communicating with the user.

  In the operation of the external device, the service robot performs an operation imitating the operation of the external device by human voice and / or gesture. The service robot performs voice, gesture, or a combination of them according to the function of the external device to be operated. Thereby, the user can learn an appropriate operation method by voice and / or gesture of the external device. In addition, it is possible to reduce the possibility that the user will find an inhuman face in the service robot.

<First embodiment>
First, a first embodiment of the present invention will be described. In the present embodiment, the service robot outputs a sound for operating an external device. FIG. 1 is a block diagram schematically showing the configuration of the service robot of this embodiment. The service robot has an external device control function, which is a feature of this embodiment, and is an external device control device. In the present embodiment, an external device control function in the service robot will be described in particular.

  The service robot 10 includes a CPU (central processing unit) 20, a storage device 30, and a bus 40, and further, as an input / output device, an NIF (network interface) 50, a speaker 60, a microphone 70, a temperature sensor 80, a timer 81, and a moving mechanism control. The apparatus 90 and the moving mechanism 91 are provided.

  The CPU 20 is an arithmetic device that executes each program described below included in the storage device 30. The storage device 30 is a storage device that is a data storage device having a volatile medium such as a DRAM or SRAM, a storage device having a nonvolatile medium such as a hard disk drive, or a combination thereof.

  The storage device 30 stores a device database (DB) 311, an operation DB 312, a vocabulary DB 313, an internal state DB 314, an operation queue 315, a system voice quality DB 316, and a user voice quality DB 317 as data for external device control. Each of these DBs is stored in a corresponding storage area in the storage device 30.

  In the present embodiment, the information stored in the storage device 30 does not depend on the data structure and may be expressed in any data structure. For example, a data structure appropriately selected from a table, list, database, or queue can store information. One DB may be constituted by some of the DBs, and one DB may be constituted by a plurality of files.

  The storage device 30 further stores an external device management program 321, an operation plan program 322, a speech recognition program 323, a system speech synthesis program 324, and a user speech synthesis program 325 as programs. The program is executed by the CPU 20 to perform a predetermined process.

  Therefore, in the following description, the description with the program as the subject may be an explanation with the CPU 20 as the subject. The CPU 20 that operates according to the program functions as an operation planning unit, a voice recognition unit, and an output pattern generation unit. The process executed by the program is also a process performed by the service robot 10 on which the program operates. Part or all of the program may be realized by dedicated hardware. The program can be installed in the service robot 10 by a program distribution server or a computer readable medium, and can be stored in the storage device 30.

  The bus 40 is connected to the CPU 20, the storage device 30, the NIF 50, the speaker 60, the microphone 70, the temperature sensor 80, the timer 81, and the moving mechanism control device 90, and each device is used to communicate data with each other. The

  The NIF 50 is a device used for transmitting and receiving data between the service robot 10 and an external device. Specific communication contents will be described later. The speaker 60 converts audio data received via the bus 40 into air vibration and outputs the air vibration. The microphone 70 detects air vibration, converts it into audio data, and outputs it to the bus 40. The temperature sensor 80 detects the temperature and outputs it to the bus 40. The timer 81 outputs the current time to the bus 40.

  The movement mechanism control device 90 executes drive control of the movement mechanism 91 in accordance with a command received from another program via the bus 40. Although the moving mechanism 91 is assumed to be a wheel in this embodiment, the moving mechanism 91 may be any means for moving the service robot 10 such as a leg, a motor, and a steering.

  Next, the operation of the external device management program 321 will be described with reference to FIG. The external device management program 321 manages external devices using the device DB 311 and the vocabulary DB 313. The external device management program 321 adds, changes, and deletes entries in the device DB 311 and the vocabulary DB 313. Examples of the device DB 311 and the vocabulary DB 313 are shown in FIGS. 3 and 4, respectively. These will be described later.

  As shown in the flowchart of FIG. 2, the external device management program 321 is started after the service robot 10 is started (S101). When a participation event packet is received from an external device via the NIF 50 (Y in S102), the external device management program 321 adds an entry to the device DB 311 according to the content of the participation event packet (S103).

  When the disconnection event packet is received from the NIF 50 (Y in S104), the external device management program 321 deletes the entry corresponding to the device ID specified by the disconnection event packet from the device DB 311 (S105). When a vocabulary data packet is received from the NIF 50 (Y in S106), the contents of the vocabulary DB 313 are updated according to the contents of the vocabulary data packet (S107). The external device management program 321 constantly monitors the received packets of the NIF 50 and keeps updating the device DB 311 and the vocabulary DB 313 while the service robot 10 is in operation.

  FIG. 3 shows a preferred example of the device DB 311. This example shows the contents of the device DB 311 when the external device management program 321 receives participation event packets from three external devices. In the participation event packet, the contents corresponding to the column of the device DB 311 in FIG. 3, that is, “device ID”, “device type”, “correspondence / non-correspondence of voice function”, “synthesis profile”, “position”, “Speech recognition recommended position” is described.

  “Device ID” is an identifier unique to the external device. “Device type” is an identifier representing the type of external device. “Support / non-support of voice function” is information indicating whether or not an external device can be operated by voice recognition. “Synthesis profile” is information used by the service robot 10 to output voice to the external device by the system voice synthesis program 324. Details will be described later.

  The “position” is a coordinate in a physical space where the external device exists, and is determined based on information acquired by the position determination function of the external device itself or information set by the user. The “speech recognition recommended position” is a position of a user recommended when operating the external device by voice, and is determined by a relative position from the external device.

  FIG. 4 shows a preferred example of the vocabulary DB 313. The vocabulary DB 313 stores the operation content of the external device and the vocabulary for the operation in association with each other. The operation content is an operation performed by the external device in response to the vocabulary. In the example of FIG. 4, three operation contents (operations of external devices) and vocabularies corresponding to the operation contents are registered in the vocabulary DB 313.

  The service robot 10 does not acquire external device attribute information (information stored in the device DB 311 and the vocabulary DB 313) by the participation event packet, but is stored in advance in the storage device 30 in association with the device ID. Information may be used. In this configuration, external device information may not be described in the participation event packet. The service robot 10 may acquire information on an external device via an external network, or the user may register the information in the service robot 10.

  Next, the operation of the operation planning program 322 will be described with reference to FIGS. 5A and 5B. The operation planning program 322 controls the operation of the service robot 10. The operation planning program 322 determines (plans) future operations of the service robot 10 and instructs other programs and devices on the determined operations.

  The operation planning program 322 determines to operate the external device when the state information registered in the internal state DB 314 satisfies a prescribed condition. The internal state DB 314 may store, as state information, a user voice recognition result (a conversion character string from speech), a temperature detected by the temperature sensor 80, a measurement time of the timer 81, user setting information, and the like. it can. Information stored in the internal state DB 314 depends on the design and operation of the service robot 10.

  The operation DB 312 stores information that associates the specified condition in the state information of the internal state DB 314 with the operation of the external device. As described above, the state information of the present embodiment includes the recognition result of the user's voice. The operation DB 312 operates an external device when the internal state DB 314 includes an instruction to operate the external device by a user's voice.

  5A and 5B are flowcharts showing the operation of the operation planning program 322 in the present embodiment. FIG. 5B shows details of step 207 in FIG. 5A. The operation planning program 322 repeatedly executes the operations shown in these. As shown in FIG. 5A, the motion planning program 322 is started after the service robot 10 is started (S201).

  First, the motion planning program 322 acquires information from each sensor (temperature sensor 80, timer 81) of the service robot 10 and updates the internal state DB 314 (S202). Next, the operation planning program 322 refers to the operation DB 312 and the internal state DB 314, collates the conditions of each item of the operation DB 312 with the contents of the internal state DB 314, and checks whether the conditions are satisfied (S203). If there is an item that satisfies the condition (Y in S203), the action planning program 322 adds the action described in the action column of that item in the action DB 312 to the action queue 315 (S204).

  Next, the action planning program 322 checks whether one or more actions are registered in the action queue 315 (S205). When there is nothing registered (N in S205), the operation planning program 322 returns to Step 202. When there is a registered one (Y in S205), the operation planning program 322 takes out data representing the first operation in the operation queue 315 and specifies the type of the operation (S206). The operation planning program 322 executes processing according to the specified type in subsequent steps.

  When the type specified in step 206 is other than “external device operation” (other types in S206), the operation planning program 322 executes step 207. The action planning program 322 selects and executes a process corresponding to the identified action type from a plurality of processes. FIG. 5B shows a plurality of processes selectively executed in step 207.

  As shown in FIG. 5B, when the type of operation is “move”, the operation plan program 322 issues a command to the movement mechanism control device 90 (S207a). When the type of operation is “system speech output”, the operation planning program 322 issues a command to the system speech synthesis program 324 (S207b). When the operation type is “change state”, the operation plan program 322 updates the internal state DB 314 (S207c).

  If the type is “network transmission”, the operation planning program 322 issues a transmission command to the NIF 50 (S207d). When the type of operation is “user voice output”, the action planning program 322 issues a command to the user voice synthesis program 325 (S207e). When the operation type is “operation completed”, the operation planning program 322 adds a network transmission operation that instructs the target external device to turn on voice recognition (S207f). “Operation complete” means completion of operation of the external device. Steps 207a to 207f will be described later.

  Next, a case where the operation type is “external device operation” in step 206 will be described. As a condition for registering “external device operation” in the operation queue 315, the internal state DB 314 stores a user voice recognition result corresponding to an external device operation instruction. In this configuration example, the voice recognition program 323 stores the character string of the voice recognition result in the internal state DB 314. This point will be described later. The item in the internal state DB 314 associated with the external device operation may be only the user's recognized voice or may include other items.

  The operation plan program 322 first refers to the device DB 311 and selects an external device to be operated (S208). Further, the operation planning program 322 acquires the recommended voice recognition position of the external device from the device DB 311 and adds a “move” operation for moving the service robot 10 to the recommended position to the operation queue 315 (S209). .

  Next, the operation planning program 322 adds a “network transmission” operation that commands “voice recognition OFF” to the target external device to the operation queue 315 (S210). Next, the vocabulary corresponding to the operation content is selected with reference to the vocabulary DB 313 of the target external device. Further, a “user voice output” operation for outputting the vocabulary as a voice is added to the action queue 315 (S211). Finally, a “network transmission” operation for transmitting a packet for instructing the external device to execute the operation content is added to the operation queue 315 (S212).

  The operation planning program 322 executes step 202 to step 206, and then performs processing according to the operation added to the operation queue 315 (S207). As described above, the movement registered in the movement queue 315 includes “movement” to the recommended position (S209), “network transmission” (S210) instructing “voice recognition OFF”, and “user voice output” of the registered vocabulary. (S211) and “network transmission” (S212) of the operation execution command packet.

  The operation planning program 322 executes the above operations in the above order. Specifically, first, the motion planning program 322 instructs the mechanism control device 90 to move to the recommended position (S207a). The mechanism control device 90 controls the moving mechanism 91 according to the command, and moves the service robot 10 to the recommended position.

  Next, the operation planning program 322 instructs the NIF 50 to transmit a “voice recognition OFF” instruction to the target external device (S207d). In response to the instruction, the NIF 50 transmits a “voice recognition OFF” instruction to the target external device. The external device that has received the instruction turns off its voice recognition function.

  Next, the motion planning program 322 instructs the user speech synthesis program 325 to output a vocabulary corresponding to the external device operation (S207e). The user speech synthesis program 325 generates a speech waveform representing the vocabulary in response to a command from the motion planning program 322 and outputs the speech waveform via the speaker 60.

  Next, the operation planning program 322 instructs the NIF 50 to transmit an operation execution instruction packet to the target external device (S207d). In response to the instruction, the NIF 50 transmits the packet to the target external device. The external device that has received the instruction performs an operation in accordance with the instruction indicated by the packet.

  After the voice output, the user voice synthesis program 325 adds a “voice operation complete” operation to the operation queue 315 (S405 in FIG. 7). The operation plan program 322 adds an operation of transmitting an instruction “voice recognition ON” to the target external device to the operation queue 315 (S207f).

  Thereafter, the user speech synthesis program 325 instructs the NIF 50 to transmit a “voice recognition ON” instruction to the target external device in accordance with the instruction registered in the operation queue 315 (S207d). In response to the instruction, the NIF 50 transmits a “voice recognition ON” instruction to the target external device. The external device that has received the instruction turns on its voice recognition function. Thereby, after this series of operations, the user is ready to control the external device by voice recognition again.

  In this way, the service robot 10 recognizes the user's voice and operates the external device according to the external device operation instruction by voice. In the present embodiment, the voice recognition program 323 performs user voice recognition. The operation of the speech recognition program 323 will be described with reference to FIG.

  The voice recognition program 323 is activated after the service robot 10 is activated (S301). The voice recognition program 323 first acquires voice data from the microphone 70 (S302). The voice data is sequentially acquired until the voice of the user is detected from the voice data (S303). Techniques for determining whether a user's voice has been detected are widely known. For example, the speech recognition program 323 uses a general speech detection method as shown in ““ Basics of Speech Recognition ”by Lawrence Rabiner, Biing-Hwang Juang, translated by Sadaaki Furui, published by NTT Advanced Technology Corporation. Use.

  Next, the voice recognition program 323 extracts the characteristics of the user's voice from the acquired voice data and registers it in the user voice quality DB 317 (S304). The characteristics of the voice are the waveform itself for each phoneme, the fundamental frequency, the duration, and the like. By using these parameters in the speech synthesis program 324, a synthesized speech close to the voice of the user can be created. . A technique for extracting voice features is widely known, and can be realized by using speaker adaptation based on a HMM (Hidden Markov Model) speech synthesis method such as HMM-based Speech Synthesis System (HTS).

  Next, the voice recognition program 323 performs voice recognition processing of the user, and when voice is recognized (Y in S305), registers the character string of the voice recognition result in the internal state DB 314. When the voice is not recognized (N in S305), the voice recognition program 323 returns to the first step 302.

  As described with reference to FIG. 5, in a preferred configuration, the service robot 10 outputs an operation instruction sound to an external device in the user's voice. In this configuration example, the user voice synthesis program 325 synthesizes output voice (waveform) by the user's voice. Hereinafter, the operation of the user speech synthesis program 325 will be described with reference to the flowchart of FIG.

  The user speech synthesis program 325 is activated after the service robot 10 is activated (S401). First, the user voice synthesis program 325 waits until a voice output command is received (S402). When the command is received, the waveform of the synthesized speech is calculated (S403). Various techniques for synthesizing speech from character strings are known, and a user speech synthesis program 325 is shown in, for example, “An Introduction to Text-to-Speech Synthesis”, by Thierry Dutoit, KLUWER ACADEMIC PUBLISHERS. A general speech synthesis method can be used.

  At this time, the user speech synthesis program 325 changes parameters used for speech synthesis based on the synthesis profile and user voice quality DB 317 described in the device DB 311. For example, in the device DB shown in FIG. 3, “voice quality” and “recommended utterance speed” are registered as the synthesis profile. “Voice quality” is a parameter for designating voice quality of synthesized speech such as vocal tract characteristics. “Recommended speech rate” indicates the speech rate of the synthesized speech.

  For example, when a synthesized speech for “HTC_AC — 10A” is created, “voice quality” is designated, so the user speech synthesis program 325 uses the designated parameters. Since the “speech rate” is slow, the user speech synthesis program 325 creates a synthesized speech having a speech rate slower than normal (eg, 3 mora per second for normal 4 mora per second). Further, since “volume” is “+3 dB”, the user speech synthesis program 325 creates a synthesized speech whose volume is 3 dB higher than usual.

  Next, the user voice synthesis program 325 outputs the generated synthesized voice waveform to the speaker 60 (S404). When the output to the speaker 60 is completed, an operation of the type “sound operation completed” is added to the operation queue 315 (S405).

  In a preferred configuration, the service robot 10 uses a system voice different from the user's voice in the dialog with the user. In this configuration, the system voice is generated by the system voice synthesis program 324. FIG. 8 is a flowchart showing the operation of the system speech synthesis program 324.

  The system speech synthesis program 324 is activated after the service robot 10 is activated (S501). First, it waits until an audio output command is received (S502). When the command is received (Y in S502), the synthesized speech waveform is calculated based on the system voice quality DB 316 indicating the characteristics of the system voice (S503).

  As a method of synthesizing speech from character strings, the system speech synthesis program 324 is a general speech synthesis method as shown in "An Introduction to Text-to-Speech Synthesis", by Thierry Dutoit, KLUWER ACADEMIC PUBLISHERS. Can be used. The system voice synthesis program 324 outputs the generated synthesized voice waveform to the speaker 60 (S504).

  Hereinafter, a specific example of external device operation by the service robot 10 will be described. In this example, the service robot 10 operates a cooling function of the air conditioner according to an instruction by a user's voice. FIG. 9 shows the internal state DB 314 updated by the operation plan program 322 (S202 in FIG. 5A).

  In this internal state DB 314, the information (detected temperature) of the temperature sensor 80 is reflected in the field of the state name “temperature sensor”, and the information (time) of the timer 81 is reflected in the item of the state name “current time”. In addition, the “user upper limit temperature” is set separately as 28 ° C., which is one of the optional items that can be set by the user.

  The action planning program 322 compares the action DB 312 shown in FIG. 10 with the internal state DB 314 shown in FIG. 9 (S203). In the conditions registered in the operation DB 312 in FIG. 10, the condition “temperature sensor sensed temperature> user upper limit temperature” is satisfied. Therefore, the action planning program 322 adds the corresponding action to the action queue 315 (S204).

  FIG. 11 shows an operation queue 315 in which operations corresponding to the condition “temperature sensor sensed temperature> user upper limit temperature” are registered. The first operation type is “system audio output”. Therefore, the motion planning program 322 instructs the system voice synthesis program 324 to output a voice “Do you want to cooler?” (S207b in FIG. 5B).

  Since the type of the second and third operations in the operation queue 315 in FIG. 11 is “state change”, the operation plan program 322 updates the internal state DB 314 (S207c). FIG. 12 shows the updated internal state DB 314. The status value of “cooler question” is “1”, and the status value of “waiting time” is a value obtained by adding 10 to the status value of “current time”.

  Next, it is assumed that the user utters “Please, please”. The voice recognition program 323 performs voice recognition processing of the user (see FIG. 6) and recognizes the user's word “please please”. The voice recognition program 323 updates the internal state DB 314 according to the recognition result (S306 in FIG. 6). FIG. 13 shows the updated internal state DB 314. The state value of “speech input” is “please please”.

  The action planning program 322 compares the internal state DB 314 of FIG. 13 with the action DB 312 of FIG. The condition of the third item in the operation DB 312 is satisfied. Therefore, the action planning program 322 adds the corresponding action to the action queue 315 (S204). FIG. 14 shows this operation queue 315. The first operation type is “system audio output”, and the operation content is “I understand”. The second operation type is an external device operation, and the operation content is “air conditioner ON, set temperature 26 ° C.”.

  The motion planning program 322 first instructs the system speech synthesis program 324 to output system speech in accordance with the instructions registered in the motion queue 315 shown in FIG. 14 (S207b). Through the processing of the speech synthesis program 324, a sound “I understand” is output from the speaker 60. Specifically, the system speech synthesis program 324 calculates a synthesized speech “I understand” based on the instruction based on the system voice quality DB 316 (S403 in FIG. 8), and outputs the synthesized speech waveform to the speaker ( S404).

  Next, the operation planning program 322 executes “external device operation” processing (S208 to S212). Specifically, the operation plan program 322 adds a series of operations for operating the air conditioner to the operation queue 315 (S209 to S212). FIG. 15 shows an operation queue 315 with new operations added.

  The operation plan program 322 sequentially processes the operations “movement”, “network output”, “voice output”, and “network output” registered in the operation queue 315 of FIG. 15. Specifically, the mechanism control device 90 is instructed to move to the recommended speech recognition position (10, 12, 10), the NIF 50 is instructed to transmit the “speech recognition OFF” instruction to “HTC_AC — 10A”, and Instruct the user voice synthesis program 325 to “set the air conditioner to 26 ° C.” and send the instruction “air conditioner ON, set temperature 26 ° C.” to “HTC_AC — 10A”. Command NIF50.

  As a result, the service robot 10 moves in front of the air conditioner “HTC_AC_10A”, and the voice recognition function of the air conditioner “HTC_AC_10A” is turned off. Thereafter, the service robot 10 emits a voice “Please set the air conditioner to 26 ° C.” from the speaker 60 with a voice close to the user's voice by the user voice synthesis program 325, and the air conditioner “HTC_AC_10A” via the network. Is given a command to “air conditioner ON, set temperature 26 ° C.”.

  The user who sees this series of movements of the service robot 10 says, “Set the air conditioner to 26 ° C.” at that position, as the service robot 10 did when turning on the air conditioner. You can know that you can operate the air conditioner.

  As described above, according to the present embodiment, when operating an electronic device that supports voice input, it can be performed in the same manner as a human being, and the user may find an inhuman side in the service robot. Can be reduced. In addition, since the service robot demonstrates the voice input method of the electronic device on a daily basis, the user can know the standing position at the time of use and the vocabulary that can be used. When the user operates the electronic device in the future, it is only necessary to remember the manners that the robot has performed in the past, which increases convenience.

  The service robot 10 can have a configuration different from the above preferable configuration. Although it is preferable that the service robot 10 can be moved to the recommended voice recognition position of the external device by the moving mechanism 91, the service robot 10 may not include the moving mechanism 91. Or even if it has the moving mechanism 91, you may output an audio | voice, without moving to a voice recognition recommendation position. In the configuration, the service robot 10 emits sound for operating the external device at a fixed position.

  As described above, it is preferable that the service robot 10 sends an instruction to the external device via the network while performing voice output for operating the external device. As a result, the external device can be reliably operated. Depending on the design, the service robot 10 may operate an external device only with output sound.

  In the configuration in which the command is transmitted to the external device via the network, it is preferable that the service robot 10 stops the function of the external device that operates according to the voice recognition as described above. Thereby, the conflict between the voice command and the command via the network can be avoided. If there is no problem in the operation of the external device, the external device may receive the above two commands.

  In the preferred configuration described above, the service robot 10 interacts with the user, and determines whether or not the user's voice includes an instruction to operate the external device in the conversation. The sound corresponding to the external device operation is output according to Whether or not the user's word is an external device operation instruction depends not only on the user's word but also on the values of other items in the status information. The service robot 10 can operate an external device in a natural dialogue with the user. The external device operation instruction may be conditional only on the user's words. An example of a direct instruction from the user to the service robot 10 for operating an external device is an example.

  In the above preferred configuration, the service robot 10 stores information from the temperature sensor 80 and the timer 81 in the internal state DB. Depending on the design, the service robot 10 may operate the external device on the condition of only the user's words without using these data. The service robot 10 may not include devices such as the temperature sensor 80 and the timer 81, and may acquire information on these devices via a network.

  As described above, the service robot 10 uses the user's voice in the voice output for operating the external device. Thereby, the user can clearly identify the voice for the dialogue by the service robot 10 and the voice for the external device. In addition, when the external device has a speaker adaptation function, the voice recognition of the external device can be appropriately adapted to the user by using the voice of the user.

  In order to imitate a user's voice more accurately, the service robot 10 preferably continues to update the user voice quality DB, but may not have an update function. In addition, the service robot 10 may use different system sounds in order to discriminate between the dialogue voice and the voice of the external device.

  As described above, it is preferable that the service robot 10 generates a sound for operating an external device using the composite profile. As a result, it is possible to generate a more appropriate sound for operating the external device. The composite profile is an example, and the composite profile can include other characteristics. Further, depending on the design, the service robot 10 may generate a voice without using the synthesis profile.

  The above description of other configurations in the present embodiment can be applied to the second embodiment described below, except for matters relating to audio output.

<Second Embodiment>
A second embodiment of the present invention will be described. The service robot of this embodiment is shown in FIG. The difference from the configuration shown in FIG. 1 of the first embodiment is that the mechanism control device 90 of the service robot 10 further includes a gesture mechanism 92, and the gesture DB 313b is stored in the storage device 30 instead of the vocabulary DB 313. The robot 10 includes a gesture generation program 325b instead of the user speech synthesis program 325.

  The gesture mechanism 92 is a machine part having an appearance and an operation function similar to those of a human, corresponding to a head, an arm, a chest, and a leg used for a gesture in a human body. Only some of these components may be mounted. A part of the moving mechanism 91 may be shared. In this configuration, the mechanism control device 90 appropriately controls the moving mechanism 91 and the gesture mechanism 92, avoids competition between the moving operation and the gesture operation, and prevents the service robot 10 from entering a dangerous state due to these operations. To do.

  Next, the operation of the external device management program 321 will be described. The operation of the external device management program 321 is almost the same as that of the first embodiment described with reference to FIG. 2, and only the difference will be described here. In Step 106 and Step 107, the external device management program 321 receives a gesture data packet instead of the vocabulary data packet from the NIF 50, and follows the content of the gesture data packet. The content of the gesture DB 313b is updated.

  As illustrated in FIG. 17, the gesture DB 313 b includes information on an operation and a corresponding gesture sequence. In the example of FIG. 17, “[0-10] joint A: (30-50, 5)” indicates that the “joint A” of the gesture mechanism 92 is 30 degrees to 50 degrees and the angular velocity upper limit in time frames 0 to 10. 5 means to move. The external device management program 321 may use the gesture information stored in advance in the storage device 30 instead of updating the gesture DB 313b with the gesture data packet.

  Next, the operation of the operation planning program 322 will be described with reference to FIG. The operation of the operation planning program 322 is substantially the same as that of the first embodiment described with reference to FIGS. 5A and 5B, and only the difference will be described here. In step 206, the operation planning program 322 sends a command to the gesture generation program 325b for the operation of the type “gesture output” (S607e).

  When the type is “operation completed”, the operation plan program 322 adds a network transmission operation that instructs the target external device to turn on gesture recognition (S607f). In step 609, the motion planning program 322 acquires a recommended gesture recognition position from the device DB 311 and adds a movement operation to that position. The device DB 311 holds a recommended gesture recognition position instead of the recommended speech recognition position in the same format as that shown in FIG.

  In S610, the operation planning program 322 adds a network transmission operation that instructs the target external device to turn off gesture recognition. In S611, the operation planning program 322 creates a sequence for gesture output based on the gesture DB 313b of the target external device, and adds a gesture output operation.

  In the description of the operation example of the operation plan program 322, the operation after the “external device operation” operation is processed is different from the configuration of the first embodiment, and the operation queue 315 is as shown in FIG. The operation planning program 322 sequentially processes these operations, “movement”, “network output”, “gesture generation”, and “network output”.

  As a result, the service robot 10 moves in front of the air conditioner “HTC_AC_10A”, and the voice recognition function of the air conditioner “HTC_AC_10A” is turned off. Thereafter, the service robot 10 performs a gesture operation according to the pattern generated by the gesture generation program 325b. Specifically, the gesture mechanism 92 moves in accordance with a gesture sequence corresponding to “air conditioner ON, set temperature 26 ° C.” in FIG. 17, imitating a human gesture. Finally, the service robot 10 gives a command to the air conditioner “HTC_AC — 10A” to “set the air conditioner ON and set temperature 26 ° C.”.

  As described above, the user who sees this series of movements of the service robot 10 performs the same gesture as the service robot 10 at that position as if the service robot 10 did when turning on the air conditioner. You can know that you can operate the air conditioner.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to said embodiment. A person skilled in the art can easily change, add, and convert each element of the above-described embodiment within the scope of the present invention. For example, when the external device has a voice recognition function and an image recognition function, and the operation of the external device is configured by both voice and gesture, the service robot executes both voice output and gesture operation.

  The present invention can be applied mainly to robots that are active in homes and offices.

10 service robot, 30 storage device, 40 bus, 60 speaker 70 microphone, 80 temperature sensor, 81 timer, 90 moving mechanism control device 91 moving mechanism, 92 gesture mechanism, 311 equipment DB, 312 operation DB
313 Vocabulary DB, 313 Vocabulary DB, 313b Gesture DB
314 Internal state DB, 315 Action queue, 316 System voice quality DB
317 User voice quality DB, 321 External device management program 322 Operation planning program, 323 Speech recognition program 324 System speech synthesis program, 325 User speech synthesis program 325b Gesture generation program

Claims (9)

An external device control device that recognizes a user's voice and controls an external device,
A receiver for receiving audio;
A voice recognition unit that converts voice received by the receiving unit into a character string;
A state information storage area for storing state information including a character string converted by the voice recognition unit;
An operation information storage area for storing operation information for associating the operation of the external device control device with the condition in the state information;
An operation for referring to the state information storage area and the operation information storage area to determine whether or not the state information including the character string satisfies a condition associated with an operation by voice and / or gesture of an external device. Planning department,
A pattern generation unit that generates a voice waveform and / or a gesture pattern corresponding to the external device operation when the operation planning unit determines that the state information including the character string satisfies the condition;
An output unit for performing an output operation of the voice waveform and / or gesture according to the pattern ;
A network interface for data communication with the external device,
The operation planning unit
Using the network interface, an operation instruction corresponding to the external device operation is transmitted to the external device,
Before the output operation by the output unit , the external device is configured to instruct the external device to stop the function of recognizing the output operation and executing the corresponding operation using the network interface. Control device.   The external device control device according to claim 1,
  The pattern generation unit converts a natural language character string corresponding to the external device operation into a speech waveform,
  The said output part outputs the said audio | voice waveform as an air vibration, The external apparatus control apparatus characterized by the above-mentioned.   An external device control device according to claim 2,
  A user voice quality storage area for storing user voice quality information including information indicating characteristics of the voice of the user;
  The said pattern production | generation part converts the said character string into the said audio | voice waveform so that it may approximate the characteristic of the user's voice which the said user voice quality information shows, The external apparatus control apparatus characterized by the above-mentioned.   An external device control device according to claim 3,
  The external device control apparatus, wherein the voice recognition unit extracts a voice feature from a voice uttered by a user and updates information in the user voice quality information storage area.   An external device control device according to claim 2,
  The pattern generation unit further comprises a storage area for storing a synthesis profile that specifies parameters used when converting a character string into a speech waveform,
  The said pattern production | generation part converts the input character string into an audio | voice waveform based on the said synthetic | combination profile, The external apparatus control apparatus characterized by the above-mentioned. An external device control device that recognizes a user's voice and controls an external device,
A receiver for receiving audio;
A voice recognition unit that converts voice received by the receiving unit into a character string;
A state information storage area for storing state information including a character string converted by the voice recognition unit;
An operation information storage area for storing operation information for associating the operation of the external device control device with the condition in the state information;
An operation for referring to the state information storage area and the operation information storage area to determine whether or not the state information including the character string satisfies a condition associated with an operation by voice and / or gesture of an external device. Planning department,
A pattern generation unit that generates a gesture sequence corresponding to the operation of the external device when the operation planning unit determines that the state information including the character string satisfies the condition;
An external device control apparatus comprising: an output unit that includes a plurality of movable parts, and that performs a gesture by moving the plurality of movable parts according to the gesture sequence.   An external device control device that recognizes a user's voice and controls an external device,
  A receiver for receiving audio;
  A voice recognition unit that converts voice received by the receiving unit into a character string;
  A state information storage area for storing state information including a character string converted by the voice recognition unit;
  An operation information storage area for storing operation information for associating the operation of the external device control device with the condition in the state information;
  An operation for referring to the state information storage area and the operation information storage area to determine whether or not the state information including the character string satisfies a condition associated with an operation by voice and / or gesture of an external device. Planning department,
  A pattern generation unit that generates a voice waveform and / or a gesture pattern corresponding to the external device operation when the operation planning unit determines that the state information including the character string satisfies the condition;
  An output unit for performing an output operation of the voice waveform and / or gesture according to the pattern;
  Including an area for storing information indicating a recommended position of the output operation for the moving mechanism and the external device;
  The moving mechanism moves the external device control device to the recommended output operation position for the external device,
  The external device control device, wherein the output unit executes the output operation at the recommended output operation position.   A control method of an external device by a control device that recognizes a user's voice and controls the external device,
  Receive user voice,
  The received voice is converted into a character string,
  Storing state information including the converted character string in a data storage device;
  The operation information stored in the data storage device and associating the operation of the control device with the condition in the state information is compared with the state information including the character string, and the state information including the character string is To determine whether the conditions associated with the voice and / or gesture actions of
  If the state information including the character string satisfies the condition, generate a speech waveform and / or gesture pattern corresponding to the external device operation,
  According to the pattern, the voice waveform and / or gesture is output.
  Using the network interface for communicating with the external device, the operation instruction corresponding to the operation of the external device is transmitted to the external device,
  Before the output operation, the network interface is used to instruct the external device to stop the function of recognizing the output operation and executing the corresponding operation.   A program that causes the processor to execute processing for controlling the operation of an external device control device that includes a processor and a data storage device and recognizes a user's voice to control the external device,
  Get status information including the character string converted from the received voice of the user,
  Stored in the storage device, and refers to operation information that associates the operation of the control device with the condition in the state information;
  The state information including the character string is compared with the operation information to determine whether or not the state information including the character string satisfies a condition associated with an operation by a voice and / or gesture of an external device. ,
  If the state information including the character string satisfies the condition, a voice waveform and / or gesture pattern corresponding to the operation of the external device is generated, and the voice waveform and / or gesture output operation is performed according to the pattern. To decide
  Using the network interface for communicating with the external device, the operation instruction corresponding to the operation of the external device is transmitted to the external device,
  Before the output operation, the network interface is used to send a command to the external device to stop the function of recognizing the output operation and executing the corresponding operation.
  A program characterized by causing a step to be executed.

Images