JP2017213612A - Robot and method for controlling robot - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a robot in which utterance and operation are linked to each other.SOLUTION: A robot has: storage means for storing operation definition set which is data, in which operation by a robot having a movable part is defined, and includes a start operation, an intermediate operation and an end operation; voice output means for outputting a voice; driving means for driving the movable part; command generation means for generating a drive command for the driving means by use of the operation definition set; and control means for issuing the drive command for the driving means in synchronization with output of the voice. The command generation means generates the drive command by which the intermediate operation is inserted between the start operation and end operation one or more times so that an operation time of the movable part is substantially same as an output time of the voice.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a robot that interacts with a user by voice.

  In recent years, robots that provide various information by interacting with people have been developed. For example, Patent Document 1 discloses a communication robot that incorporates a camera, a microphone, and an actuator and that has a conversation with a user by voice.

  The robot described in Patent Document 1 has a function of performing a reaction by operating an actuator while it is speaking. For example, when a robot makes an inquiry to a user, an action corresponding to the inquiry is performed by moving a hand or a neck. By configuring in this way, it is possible to link speech and motion in a natural way.

JP 2004-034273 A JP 2001-179667 A

The robot described in Patent Document 1 has a feature that it stores a plurality of motion definitions whose motion times are determined in advance, and dynamically generates a combination of motion definitions that match the duration of the utterance. Thereby, the end of the utterance and the end of the operation can be matched.
However, in such a method, priority is given to the duration time, and as a result, a combination of various operations may result in an unnatural operation as a whole.

  The present invention has been made in consideration of the above problems, and an object thereof is to provide a robot in which speech and motion are linked.

The robot according to the present invention is
Data defining motion by a robot having a movable part, which is a storage means for storing an action definition set including a start action, an intermediate action, and an end action, an audio output means for outputting sound, and driving the movable part Driving means, command generating means for generating a driving command for the driving means using the action definition set, and control means for issuing a driving command to the driving means in synchronization with the output of the sound. And the command generation means includes a drive command in which the intermediate operation is inserted at least once between the start operation and the end operation so that the operation time of the movable part is substantially the same as the output time of the sound. It is characterized by generating.

The motion definition set is data that defines the motion of the robot by driving the movable part, and includes three types of start motion, intermediate motion, and end motion. In addition, the command generation means outputs a drive command in which an intermediate operation is inserted at least once between the start operation and the end operation so that the time during which the sound output unit outputs the sound and the time during which the robot operates Generate.
According to such a configuration, since the overall operation time can be controlled by repeating the intermediate operation, even if the output time of the voice cannot be acquired in advance, the speech and the operation by the robot can be linked. Can do. In addition, since operations that are not supposed to be continuous are not combined, a natural operation can be performed as a whole.

  In addition, the command generation means, when the operation time when the start operation and the end operation are continuously performed is less than the sound output time, the operation time of the movable part reaches the sound output time. Until then, the intermediate operation may be repeatedly inserted between the start operation and the end operation.

  As described above, when the sound output time can be acquired in advance, the intermediate operation may be repeatedly inserted until the operation time exceeds the sound output time.

  The intermediate operation may be an operation that allows looping within one second.

  By making a short operation capable of looping as an intermediate operation, an effect of not appearing unnatural even if the same operation is repeated can be obtained. For example, repeating a long motion such as 3 to 5 seconds many times may give the user a sense of incongruity, but repeating the short loopable motion alleviates the sense of discomfort and makes the action more natural Can be taken.

  The robot according to the present invention further includes a type acquisition unit that acquires a type according to the content of the sound output by the audio output unit, and the storage unit includes a plurality of motion definitions associated with the type. A set is stored, and the command generation unit may select an operation definition set corresponding to the type and generate the drive command using the operation definition set.

  The type corresponding to the content of the voice may represent, for example, a robot's pseudo emotion (feeling emotional), or if the voice provides some information to the user, It may be set based on the content of the information. As described above, a plurality of action definition sets may be set according to the content and type of the response sentence, and a suitable one may be selected. According to such a configuration, it is possible to cause the robot to take different reactions according to the utterance content.

  When the voice output unit outputs two or more sentences, the command generation unit selects a plurality of action definition sets according to the type of each sentence, and uses the plurality of action definition sets. The driving command may be generated.

  When the output target voice is composed of two or more sentences, the response sentence type may be changed in the middle. For example, once the end operation is performed, an operation corresponding to another type may be started, or an intermediate operation corresponding to another type may be shifted in the middle of the intermediate operation. According to such a configuration, richer expressions are possible.

  Further, the command generation means may generate a drive command to which an initialization operation for returning the position of the movable part to an initial position is further added.

  When the robot finishes the operation, it is necessary to stop the movable part at a position that does not hinder the start of the next operation. Therefore, a command for initializing the position of the movable part may be added to the end of the drive command. According to such a configuration, it is not necessary to consider the position of the movable part at the end of driving when designing the operation, and the degree of freedom of expression can be increased.

  In addition, this invention can be specified as a robot containing at least one part of the said means. It can also be specified as a control method of the robot. The above processes and means can be freely combined and implemented as long as no technical contradiction occurs.

  According to the present invention, it is possible to provide a robot in which speech and motion are linked.

1 is a system configuration diagram of a voice interaction system according to an embodiment. 1 is a diagram illustrating a robot 10. FIG. It is a figure explaining the specific example of an action definition set. It is a processing flow figure of the voice dialogue system concerning an embodiment. It is a processing flow figure of the voice dialogue system concerning an embodiment. It is a processing flow figure of the voice dialogue system concerning an embodiment. It is a figure explaining the combination of the speech time and operation | movement of a robot.

Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.
The voice interaction system according to the present embodiment is a system that performs communication by a robot interacting with a user by voice.

(First embodiment)
<System configuration>
FIG. 1 is a system configuration diagram of a spoken dialogue system according to the present embodiment. The voice interaction system according to the present embodiment includes a robot 10 and a control device 20.
The robot 10 has a speaker, a microphone, a camera, and the like, and is a means for performing an interface with a user. The robot 10 has a human type (or character type) as shown in FIG. 2, and various reactions can be performed by moving the joint with an actuator.
The control device 20 is a device that issues a control command to the robot 10. In this embodiment, the robot 10 functions only as a user interface, and the control device 20 performs processing for controlling the entire system, such as generation of a dialog sentence, speech recognition, and other processing.

First, the robot 10 will be described.
The robot 10 includes a voice input unit 11, a communication unit 12, a voice output unit 13, and an operation control unit 14.

  The voice input unit 11 is means for acquiring voice uttered by the user. Specifically, sound is converted into an electrical signal (hereinafter referred to as sound data) using a microphone (not shown). The acquired audio data is transmitted to the control device 20 via the communication unit 12 described later.

  The communication unit 12 is means for performing near field communication with the control device 20. In the present embodiment, the communication unit 12 performs communication using a Bluetooth (registered trademark) connection. The communication unit 12 stores information related to the control device 20 that is a pairing destination, and can be connected by a simple operation.

  The audio output unit 13 is means for outputting audio to be provided to the user. Specifically, the audio data transmitted from the control device 20 is converted into audio using a speaker.

The operation control unit 14 is means for controlling the operation of the robot 10 by driving actuators built in a plurality of movable units of the robot 10. Specifically, based on a command transmitted from the control device 20, for example, an actuator arranged in a joint such as a hand, shoulder, elbow, or foot is driven to cause the robot 10 to perform a predetermined reaction.
Further, the operation control unit 14 stores in advance the operation definition of the actuator (how to move which actuator for what command), and the actuator based on the command transmitted from the control device 20. Drive.
In addition, the movable part may be provided for each joint as shown in FIG. 2, for example, or may be provided at a place other than the joint, such as a wheel. A specific example of the command transmitted to the robot will be described later.

  Next, the control device 20 will be described. The control device 20 is a device that controls the robot 10 and is typically a personal computer, a mobile phone, a smartphone, or the like. The control device 20 can be configured as an information processing device having a CPU, a main storage device, and an auxiliary storage device. Each unit shown in FIG. 1 functions by loading a program stored in the auxiliary storage device into the main storage device and executing it by the CPU. Note that all or part of the illustrated functions may be executed using a circuit designed exclusively.

  The control device 20 includes a communication unit 21, a voice recognition unit 22, a response generation unit 23, a voice synthesis unit 24, and an operation generation unit 25.

  Since the function which the communication part 21 has is the same as that of the communication part 12 mentioned above, detailed description is abbreviate | omitted.

  The voice recognition unit 22 is means for performing voice recognition on the voice acquired by the voice input unit 11 of the robot and converting the voice into text. Speech recognition can be performed by known techniques. For example, the speech recognition unit 22 stores an acoustic model and a recognition dictionary, extracts features by comparing the acquired speech data with the acoustic model, and matches the extracted features with the recognition dictionary to generate speech. Recognize. The recognition result is transmitted to the response generation unit 23.

  The response generation unit 23 is a unit that generates or acquires a response sentence to be provided to the user based on the text output by the voice recognition unit 22. The information to be provided may be, for example, information obtained by searching a database, or information obtained by web search. Further, the provided information may not be an answer to the question. For example, when the robot 10 is a communication robot, it may be a response selected from a dialogue scenario (dialog dictionary). In addition, as long as information can be provided by natural language processing, any text can be input and any response sentence can be output. The response sentence generated or acquired by the response generation unit 23 is converted into synthesized speech, transmitted to the robot 10, and then output to the user.

  The speech synthesizer 24 is a means for converting the response sentence (text) generated by the response generator 23 into speech data using a known speech synthesis technique. The generated voice data is transmitted to the robot 20 and provided to the user.

The action generation unit 25 is means for determining an action to be performed by the robot 10 based on the response sentence generated by the response generation part 23. The motion generation unit 25 stores data (motion definition set) that defines the movement of the movable unit, and based on the data and the response sentence generated by the response generation unit 23, a drive command (hereinafter referred to as motion) is given to the robot. Command).
Here, the action definition set will be described. The action definition set is data that defines how to move a plurality of movable parts of the robot, and includes a combination of three types of actions: “start action”, “intermediate action”, and “end action”. FIG. 3 is an example of an action definition set. Here, an action definition set when the robot greets is shown.

The robot 20 drives the movable part in the order of the start operation, the intermediate operation, and the end operation according to the operation definition set. In the case of the example in FIG. 3, first, the right arm is lifted by moving the three movable parts of the right hand to a position (hereinafter referred to as position X) that has been driven from the initial position. Next, an operation of shaking hands by reciprocating the two movable parts other than the shoulder to the initial position is performed. Finally, the right arm is lowered by returning the three movable parts to the initial position. If this is performed continuously, the operation of waving the right hand to the user is performed.

  Although an action definition set corresponding to “greeting” is shown here, a plurality of kinds of action definition sets are defined according to the content (type) of the response sentence, and an appropriate one is selected. For example, a suitable one may be selected from a plurality of action definition sets according to the emotions and emotions (pseudo emotions) of the robot.

For example, types such as “question of question”, “answer to question”, “praise”, “sympathy”, and “joy” may be defined. There may also be an action definition set corresponding to a specific word.
Further, even if the contents and types of response sentences are the same, different operation definition sets may be selected when the environment is different. For example, different motion definition sets may be selected depending on whether the robot is at home or in a car.
In addition, even if the contents and types of response sentences are the same, different action definition sets may be selected if the emotions of the robot are different. For example, different motion definition sets may be selected depending on whether the robot has an angry emotion or a joyful emotion.
In addition, when the response sentence provides information to the user, an action definition set corresponding to the type and content of the information may be selected.
As described above, a suitable action definition set can be selected according to emotion, response sentence type, environment, and the like.

  Note that the intermediate operation may be omitted or repeated. Also, the end operation may be omitted. This will be explained later.

<Processing flow>
Next, the processing performed by each unit shown in FIG. 1 and the data flow will be described with reference to FIGS. 4 to 6 which are flowcharts illustrating the processing content and the data flow.
First, in step S11, the voice input unit 11 of the robot 10 acquires voice from the user through a microphone. The acquired voice is converted into voice data and transmitted to the voice recognition unit 22 of the control device 20 via the communication unit.
Then, the voice recognition unit 22 performs voice recognition on the acquired voice data and converts it into text (step S12). The text obtained as a result of the speech recognition is transmitted to the response generation unit 23.

  Next, in step S21, the response generation unit 23 generates a response sentence based on the acquired text. As described above, the response sentence may be generated using a dialog dictionary (dialog scenario) that the device itself has, or may be generated using an external information source (database server or web server). The generated response sentence is transmitted to both the speech synthesizer 24 and the action generator 25.

In step S22, the speech synthesizer 24 synthesizes speech based on the response sentence and acquires speech data. The acquired voice data is transmitted to both the action generation unit 25 and the communication unit 21.
In step S23, the action generation unit 25 acquires an action definition set based on the response sentence. Specifically, based on the type of response sentence (what the response is), a suitable one is acquired from a plurality of action definition sets. For example, when the response sentence is a greeting, an action definition set corresponding to “greeting” is acquired.
Note that the execution order of steps S22 and S23 does not matter.

  Next, in step S24, the motion generation unit 25 calculates a playback time based on the acquired audio data, and generates a motion command corresponding to the playback time in step S25.

Here, an operation command generation method will be described.
As shown in FIG. 3, there are three types of action definition sets in the present embodiment: a start action, an intermediate action, and an end action. On the other hand, since the response provided by the robot to the user is dynamically generated, it is not possible to know in advance the time taken for the robot to reproduce the sound. Therefore, the control device 20 according to the present embodiment calculates an audio playback time (hereinafter referred to as “speech time”), and then combines the start operation, the intermediate operation, and the end operation to generate an operation suitable for the speech time.

FIG. 7 is a diagram illustrating a process in which the motion generation unit 25 generates a motion command. Here, the speech time is 3 seconds, the time required for the start operation is 600 milliseconds, the time required for the intermediate operation is 300 milliseconds, and the time required for the end operation is 600 milliseconds.
First, the time when the start operation and the end operation are continuously performed is compared with the speech time. As a result, when the utterance time is longer, an intermediate operation is inserted between the start operation and the end operation. In this example, since the time when the start operation and the end operation are continuously performed is 1.2 seconds and the speech time is 3 seconds, an intermediate operation is inserted (FIG. 7A).

The number of intermediate motions to be inserted is determined so that the difference between the robot motion time and the speech time is minimized. In this example, when six intermediate operations are inserted, the total time is 3.0 seconds (FIG. 7B).
In the case of this example, in step S25, an operation command for performing the start operation once, the intermediate operation six times, and the end operation once is generated and transmitted to the robot 10 via the communication unit.

On the other hand, for example, when the utterance time is 3.2 seconds, when six intermediate motions are inserted, 3.0 seconds (difference is 0.2 seconds), and when seven are inserted, 3.3 seconds (difference is 0). .1 second), the number of intermediate operations to be inserted is seven (FIG. 7C).
However, it is not always necessary to minimize the difference between the operation time and the speech time, and the operation time and the speech time may be substantially the same.

  Here, an example in which an intermediate operation is inserted has been described, but if the utterance time is short, the intermediate operation or the end operation may not be performed. For example, when the utterance time is shorter than the time when the start operation and the end operation are continuously performed, the start operation and the end operation may be performed continuously. Further, when the utterance time is shorter than the time when only the start operation is performed, only the start operation may be performed.

  Further, the start operation is not necessarily performed. For example, when the start operation is performed and the difference between the speech time and the operation time becomes larger than when the start operation is not performed, the start operation may be omitted. Further, an attempt may be made to reduce the difference between the speech time and the operation time by giving priority to the start operation and the end operation and omitting the operation with low priority.

  Moreover, the start operation and the end operation are not necessarily limited to one time. For example, if the difference between the speech time and the operation time can be further reduced by performing the start operation and the end operation a plurality of times, the start operation and the end operation may be repeated a plurality of times.

  When it is desired to make the speech time and the operation time completely coincide with each other, the time may be adjusted by stopping the intermediate operation or the end operation halfway.

When the communication unit on the robot 10 side receives the audio data and the operation command (step S31), the audio data is transmitted to the audio output unit 13 and the operation command is transmitted to the operation control unit 14.
In step S32, the motion control unit 14 acquires the motion definition of the actuator corresponding to the motion command (which actuator is to be moved in what sequence).
On the other hand, when the audio output unit 13 acquires the audio data, the audio output unit 13 starts reproducing the audio (step S33). At the same time, a reproduction start trigger is transmitted to the operation control unit 14, and the operation control unit 14 starts driving the actuator based on the trigger (step S34).
As a result, the robot starts to operate simultaneously with the start of the utterance, and when the utterance ends, the operation of the robot also stops substantially simultaneously.

  As described above, the spoken dialogue system according to the present embodiment allocates an intermediate operation so that the time of speech performed by the robot matches the operation time of the robot, and generates an entire operation command. Thereby, the speech and movement by the robot can be linked. In particular, by making the intermediate operation a short operation that can be looped and repeating the operation until the utterance time is satisfied, a natural motion can be made regardless of the length of the utterance time.

  In the known technology, the movement as a whole is generated by combining relatively large movements. Therefore, there is a possibility that the user may recognize that the movement is connected or that the same movement is performed many times. It was. On the other hand, in the present invention, since a short motion is looped, for example, a natural motion in which the hand or neck shakes swayingly can be performed, and the uncomfortable feeling given to the user can be reduced.

(Second embodiment)
When the robot finishes the operation, it is necessary to stop the movable part at a position that does not hinder the start of the next operation. If this is not performed, there is a possibility that the movable portion may be displaced and the next operation cannot be performed.
In order to cope with this, the second embodiment is an embodiment in which the position of the movable part of the robot is initialized every time the operation by the robot is completed.

  In the second embodiment, in step S25, a command for initializing the position of the movable part is added to the end of the operation command. Thereby, a movable part can be reliably returned to an initial position. That is, when designing the operation, there is no need to consider the position of the movable part at the end of driving, and the degree of freedom of expression can be increased.

(Modification)
The above embodiment is merely an example, and the present invention can be implemented with appropriate modifications within a range not departing from the gist thereof.

For example, in the description of the embodiment, the voice recognition unit 22 performs voice recognition, but the voice recognition may be performed by an external device. In this case, a communication unit for communicating with the network may be added to the control device 20 to transmit / receive voice data and a recognition result.
Similarly, in the description of the embodiment, the response generation unit 23 generates a response, but a response sentence may be generated by an external device. In this case, a communication unit for communicating with the network may be added to the control device 20 to transmit / receive the voice recognition result and the response text.

In the description of the embodiment, one type of action definition set is acquired for one utterance. However, when there are a plurality of utterances, a different action definition set may be used for each utterance. Further, when the robot's pseudo emotions change during the utterance, the type of the response sentence may be changed in the middle (that is, the action definition set is changed). In this case, the operation may be shifted to the next operation after the end operation, or may be shifted from the intermediate operation of the operation definition set A to the intermediate operation of the operation definition set B.

DESCRIPTION OF SYMBOLS 10 ... Robot 11 ... Voice input part 12, 21 ... Communication part 13 ... Voice output part 20 ... Control apparatus 22 ... Voice recognition part 23 ... Response generation part 24 ...・ Speech synthesizer 24 ... Motion generator

Claims (8)

Storage means for storing an operation definition set including data for defining an operation by a robot having a movable part, including a start operation, an intermediate operation, and an end operation;
Audio output means for outputting audio;
Drive means for driving the movable part;
Command generation means for generating a drive command for the drive means using the operation definition set;
Control means for issuing a drive command to the drive means in synchronization with the audio output;
Have
The command generation means generates a drive command in which the intermediate operation is inserted at least once between the start operation and the end operation so that the operation time of the movable part is substantially the same as the output time of the sound.
robot. When the operation time when the start operation and the end operation are continuously performed is less than the sound output time, the operation time of the movable unit reaches the sound output time. Repeatedly inserting the intermediate operation between the start operation and the end operation;
The robot according to claim 1. The intermediate operation is an operation capable of looping within one second.
The robot according to claim 1 or 2. Further comprising a type acquisition unit for acquiring a type according to the content of the audio output by the audio output unit;
The storage means stores a plurality of action definition sets associated with the type,
The command generation means selects an operation definition set corresponding to the type, and generates the drive command using the operation definition set.
The robot according to any one of claims 1 to 3. When the voice output unit outputs two or more sentences, the command generation unit selects a plurality of action definition sets according to the type of each sentence, and uses the plurality of action definition sets. Generate drive instructions,
The robot according to claim 4. The command generation means generates a drive command to which an initialization operation for returning the position of the movable part to an initial position is further added.
The robot according to any one of claims 1 to 5. A control method for a robot having a movable part driven by a driving means, and an audio output means,
An operation definition acquisition step for acquiring an operation definition set including data for defining an operation by the movable part, including a start operation, an intermediate operation, and an end operation;
A command generation step of generating a drive command for the drive means using the motion definition set;
A control step of issuing a drive command to the drive means in synchronization with the output of the sound;
Including
In the command generation step, a drive command in which the intermediate operation is inserted one or more times between the start operation and the end operation is generated so that the operation time of the movable part is substantially the same as the output time of the sound.
Robot control method.   A program for causing a computer to execute the robot control method according to claim 7.

Images