JP3930389B2 - Motion program generation device and robot during robot utterance - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a technique for appropriately operating a robot while the robot is speaking and transmitting the start and end timing of the speech to the user.
[0002]
[Prior art]
In the conventional technology, during the robot's utterance, (1) the determined motion is reproduced, and (2) the robot is operated according to the waveform data (particularly, the strength of the sound) during the utterance.
[0003]
However, since the utterance time is not constant in the playback of the fixed motion of (1), the start and end of the motion often do not exactly match the start and end of the utterance, so when the user interacts with the robot There is a drawback that it is difficult to grasp the timing of remarks. (2) The robot motion according to the waveform data has a drawback that the motion pattern is a combination of momentary motions with respect to the voice data, so that the motion does not match the meaning of the utterance.
[0004]
[Problems to be solved by the invention]
An object of the present invention is to solve the above-described drawbacks of the conventional technique, and to enable a technique to appropriately operate the robot while the robot is speaking and to notify the user of the start and end timing of the speech. Is to provide.
[0005]
[Means for Solving the Problems]
A first invention is an operation program generation device during robot utterance that solves the above-mentioned problems, and includes a database having data of a plurality of operation patterns whose operation times are determined to be different from each other, and before the robot starts utterance An utterance time measuring means for measuring the length of the utterance time from the waveform data of the generated speech before starting the utterance, and an operation start and an end of the utterance with respect to the utterance time measured by the utterance time measuring means, An operation pattern combination generating means for generating a combination of two or more operation patterns, a reproduction order, and a reproduction timing from the contents of the database so as to coincide with the end.
[0006]
In a second aspect based on the first aspect, the motion pattern combination generating means inserts a non-operation time between adjacent motion patterns when the start and end of the motion do not coincide with the start and end of the speech. An operation program generation device during robot utterance characterized by
According to a third aspect of the present invention, in the first aspect of the invention, there is provided an operation program generating apparatus during robot utterance, comprising operation type determining means for determining an operation type from character data to be uttered.
[0007]
According to a fourth aspect of the present invention, there is provided a robot characterized by having the motion program generating apparatus during the speech of the robot according to any one of the first to third aspects.
[0008]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0009]
Prior to the description of the present invention, an example of a robot to which the present invention is applied will be described with reference to FIG. The robot 1 shown in FIG. 3 is similar to a human whose robot body 11 has a head 12, a chest 13, a torso (cart) 14, and left and right arms 15. Can be rotated by a drive mechanism (not shown), and the left and right traveling wheels 16 mounted on the body 14 can be steered and run by a drive mechanism (not shown) and autonomously move in the work space by battery drive. Is configured to do. The robot 1 coexists with a human being as a work space indoors such as a general home, and is used, for example, to assist / support / care for the user's life in a general home. Therefore, the robot 1 has functions of having a conversation with the user, watching the user's behavior, assisting the user's behavior, and acting with the user by using a built-in CPU (computer) and various sensors. ing. Various shapes of the robot 1 may be considered, such as a model imitating an animal for pets.
[0010]
Specifically, the robot 1 shown in FIG. 3 has a control unit using a CPU, two heads 12 are equipped with two cameras 17 and two microphones 18, and a volume / sound source direction sensor and a focus are placed in the center of the chest 13. A human detection sensor 19 comprising an electric infrared sensor and speakers 20a are mounted on the left and right. An image display 20 b is attached to the center of the chest 13, and an ultrasonic obstacle sensor 21 and a laser obstacle sensor 22 are attached to the trunk 4. A keyboard and a touch sensor may be attached.
[0011]
The camera 17 takes a picture of the user or indoors and outputs the image to the control unit, and the microphone 18 captures the user's voice and life sounds such as a telephone bell, doorbell, and TV sound, and outputs them to the control unit. The sensor 19 detects the presence / absence of the user and outputs it to the control unit. The speaker 20a is used for conversation with the user together with the microphone 18, and the image display 20b is used for providing information to the user. The keyboard and touch sensor are data input devices for users, and are used to input user life pattern data and input user intentions. The speaker 20a is also used for providing information to the user. The control unit is provided with a calendar for a predetermined period and a calendar clock for measuring the current date and time.
[0012]
The control unit includes a data input device such as a keyboard, an external sensor that constantly monitors a user's life pattern such as a camera 17, a microphone 18, a human detection sensor 19, and obstacle sensors 21 and 22, and an internal sensor such as a calendar clock. Thus, information on the robot 1's own position, information on the user's position, and information on the user's action are input.
[0013]
The control unit has a self-position recognition function, a user recognition function, a database (data storage unit), a life pattern data processing function, and a drive control function.
[0014]
The self-position recognition function recognizes the position and orientation (direction, orientation) of the robot 1 based on image information captured by the camera 17. The user recognition function recognizes the user's speech content (voice recognition) from the voice captured by the microphone 18, recognizes the individual user from the image information captured by the camera 17 and the voice captured by the microphone 18, and The position, orientation, posture, and amount of activity of the user are recognized from the detection results of the camera 17 and the human detection sensor 19.
[0015]
The database contains voice data for conversation with the user and talking to the user, data on gestures during conversation with the user and talking to the user, life information of the user (room layout and furniture Arrangement, daily life patterns of individual users, user hobbies, health conditions, etc.). A keyword such as a character string, video, or voice is attached to the user's life information as necessary. As life pattern data, events related to actions such as waking up, going to bed, eating, taking medication, taking a walk, free time, bathing, etc. are listed, and these events are stored as a time schedule together with the time data.
[0016]
The life pattern data processing function adds life information directly input from a data input device by a user to a keyword and accumulates it in a database, and various kinds of user information such as voice recognition results obtained by the camera 17, microphone 18, and human detection sensor 19. By processing information, keywords and their dates are attached to each piece of life information and stored in a database. Furthermore, the life pattern data processing function selects corresponding data from the database in response to an instruction or talk from the user, or according to time, etc., and sends a command corresponding to the selected data to the operation control unit. Give and control.
[0017]
The drive control function drives the traveling wheel 16 in accordance with a command from the life pattern data processing function to execute the traveling and steering of the robot 1, and also drives the head 12 and the arm 13 to perform the rotation. Perform the action. The speaker 20a and the image display 20b are driven to perform a conversation with the user.
[0018]
Next, with reference to FIG. 1 and FIG. 2, an operation program generation apparatus (hereinafter simply referred to as an operation program generation apparatus) during robot speech according to an embodiment of the present invention will be described.
[0019]
The robot 1 described above has an operation program generation device illustrated in FIG. 1, and this operation program generation device includes a database 31, an utterance time measurement unit 32, and an operation pattern combination generation unit 33. It is.
[0020]
In FIG. 1, reference numeral 34 denotes a voice synthesizer provided in the robot, which includes a voice waveform generation unit 35 and a waveform reproduction unit 36. The speech waveform generation unit 35 receives character data to be uttered, generates speech waveform data 35a corresponding to the speech data before the start of speech, and the waveform reproduction unit 36 reproduces the waveform data 35a into a real-time speech waveform. . The reproduced voice waveform is amplified by the amplifier 37 and given to the speaker 20a, whereby the robot speaks.
[0021]
Here, since it is difficult for the user to hear when the robot suddenly starts speaking, character data such as “Ah” or “Hmm” is inserted at the beginning of the character data. As a result, the robot utters the original utterance after uttering a meaningless word such as “Ah” or “hmm”, so that the user's attention is drawn and it becomes easy to hear.
[0022]
The database 31 stores data of a plurality of operation patterns whose operation times are different from each other. The movement pattern is a pattern of movements of the gestures of the head 12, the arm part 15, and the trunk part 14, and as an example, a movement pattern for one second as a type of movement that the robot talks to the user, An operation pattern for 2 seconds, an operation pattern for 4 seconds, and an operation pattern for 8 seconds are stored. In addition, an operation pattern for one second and an operation pattern for two seconds are stored as types of movement that the robot asks the user. Furthermore, an operation pattern for 1 second and an operation pattern for 2 seconds are stored as types of movements to be confirmed by the robot.
[0023]
The utterance time measuring unit 32 measures the length of the utterance time before starting the utterance from the waveform data 35a generated before the start of the utterance of the robot by the voice waveform generating unit 35 or the like. Since the waveform data 35a itself is data before being reproduced into a real-time speech waveform, the speech time can be instantaneously measured by examining the data length (number of bytes) of the waveform data 35a.
[0024]
The motion pattern combination generation unit 33 uses the contents of the database 31 so that the start and end of the robot's motion coincide with the start and end of the robot's speech with respect to the speech time measured by the speech time measuring unit 32. A combination of two or more operation patterns is selected, and a reproduction order and reproduction timing thereof are generated.
[0025]
Further, the operation pattern combination generation unit 33 inserts a non-operation time between the operation patterns adjacent to each other when the start and end of the operation do not exactly coincide with the start and end of the utterance.
[0026]
The plurality of operation patterns selected by the operation pattern combination generation unit 33 are preferably suitable for the utterance content, and for this purpose, the operation type selection signal 38a is used.
[0027]
In this example, the action type determining unit 38 analyzes the character data to be uttered, thereby determining the action type such as a question, talking, and confirmation, and obtaining the action type selection signal 38a.
[0028]
The motion control device 39 plays back the plurality of motion patterns selected by the motion pattern combination generation unit 33 in the playback order and playback timing specified by the motion pattern combination generation unit 33, and the head 12, the arm unit 15, and the body unit 14. The actuator (drive mechanism) 40 is actually operated.
[0029]
FIG. 2 shows an example of an operation pattern combination procedure.
[0030]
In FIG. 2, from the length of the waveform data 35a, the speech time is 14.5 seconds, and the action type is a question from the robot to the user.
[0031]
Therefore, the motion pattern combination generation unit 33 first determines a motion pattern for a question (motion to be asked for 2 seconds) that requires 2 seconds as the last motion to be performed from motion type = question. Next, as an operation to be performed first, a motion pattern for talking for 8 seconds and a motion pattern for talking for 4 seconds are combined to fill 14 seconds as a whole. For the remaining 0.5 seconds, a non-operation time of 0.5 seconds is inserted between the movement pattern of talking movement for 4 seconds and the movement pattern of inquiry movement for 2 seconds.
[0032]
As a result, the start and end of the movement exactly coincide with the start and end of the utterance, and while the robot 1 is speaking, the robot 1 operates appropriately and informs the user of the start and end timing of the utterance. be able to.
[0033]
【The invention's effect】
According to the first aspect of the present invention, the length of the utterance time is started from the database having a plurality of movement pattern data whose movement times are different from each other and the waveform data of the voice generated before the robot starts the utterance. Two or more from the contents of the database so that the start and end of the operation coincide with the start and end of the utterance with respect to the utterance time measuring means measured before and the utterance time measured by the utterance time measuring means The operation pattern combination generating means for generating the combination of the operation patterns, the reproduction order and the reproduction timing is provided, so that the robot can be operated appropriately according to the utterance and the start and end timing of the utterance can be transmitted to the user. .
[0034]
In the second invention, when the start and end of the motion do not coincide with the start of speech and the end of speech, the motion pattern combination generation means inserts a non-operation time between the front and back adjacent motion patterns. The start and end of the motion exactly match the end.
[0035]
According to the third aspect of the invention, since the action type determining means for determining the action type from the character data to be uttered is provided, it is possible to cause the robot to perform an action suitable for the utterance content.
[0036]
In the fourth aspect of the invention, since the robot has the operation program generating device during the utterance of the robot according to any of the first to third aspects, the robot operates appropriately according to the utterance, and the timing of the start and end of the utterance to the user Can be communicated.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating a configuration of an operation program generation device during robot speech according to an embodiment of the present invention.
FIG. 2 is a diagram showing an example of an operation pattern combination procedure;
FIG. 3 is a diagram illustrating an example of a robot.
[Explanation of symbols]
1 Robot 12 Head 13 Chest 14 Body (Carriage)
20a Speaker 31 Database 32 Speaking time measurement unit 33 Operation pattern combination generation unit 34 Speech synthesizer 35 Speech waveform generation unit 35a Speech waveform data 36 Waveform reproduction unit 37 Amplifier 38 Operation type determination unit 38a Operation type selection signal 39 Operation control unit 40 Actuator

Claims (4)

An utterance that measures the length of the utterance time before starting the utterance from the database that has the data of multiple motion patterns whose movement times are determined to be different from each other and the waveform data of the speech generated before the robot starts the utterance A combination of two or more action patterns from the contents of the database such that the start and end of the action coincides with the start and end of the utterance with respect to the utterance time measured by the time measurement means and the utterance time measuring means And a motion pattern combination generating means for generating a playback order and a playback timing. 2. The operation pattern combination generation unit according to claim 1, wherein when the start and end of the motion do not coincide with the start and end of the speech, the motion pattern combination generating means inserts a non-operation time between the adjacent motion patterns. A motion program generation device during robot utterance. 2. The motion program generation apparatus during speech of a robot according to claim 1, further comprising motion type determination means for determining a motion type from character data to be uttered. A robot comprising the motion program generating device for speaking a robot according to any one of claims 1 to 3.

Images