JP7399740B2 - Communication robot, control program and control method - Google Patents

Description

Application of Article 30, Paragraph 2 of the Patent Act Posted on October 11, 2019 https://ieeexplore. ieee. org/document/8865356 [Publications, etc.] Humanoids2019 Held from October 15th to 17th, 2019 [Publications, etc.] 37th Academic Conference of the Robotics Society of Japan (RSJ2019) Held from September 3rd to 7th, 2019

The present invention relates to a communication robot, a control program, and a control method, and particularly relates to a communication robot, a control program, and a control method that take communication actions including contact motions that touch people.

As described in Non-Patent Document 1, the present inventors conducted an experiment on how audio stimulation and visual stimulation during a robot's embrace change a person's perceptual impression and stress buffering effect, The results of this experiment revealed that the hugging interaction between humans and robots has a stress-reducing effect on humans.

Masahiro Shiomi, Norihiro Hagita, “Audio-Visual Stimuli Change not Only Robot’s Hug Impressions but Also Its Stress-Buffering Effects” International Journal of Social Robotics, 2019.

In the background technology, it was found that the impression of the robot itself can be changed by tactile stimulation when the robot touches a person, and that tactile stimulation can be expected to have a stress-reducing effect on people. No consideration was given to how to express emotions naturally.

Therefore, the main object of the present invention is to provide a novel communication robot, control program, and control method.

Another object of the present invention is to provide a communication robot, a control program, and a control method that allow the robot to naturally express emotions to a person to be communicated with through contact movements.

A first invention is a humanoid communication robot that includes at least a head, a torso, an arm, and a hand, and that performs a contact action of touching a nearby person with the hand, the communication robot being able to express itself. a first acquisition means for acquiring an occurrence timing at which an event expressing a predetermined emotion included in a plurality of emotions occurs; a second acquisition means for acquiring an emotion for expression expressed in the event; A calculation condition for expression corresponding to the emotion for expression acquired by the second acquisition means from among a plurality of calculation conditions for calculating the start timing of the contact motion, which are set in advance corresponding to each of them. a first setting means for setting a start timing, a calculation means for calculating a start timing according to a calculation condition for expression set by the first setting means, based on the occurrence timing acquired by the first acquisition means; A motion control means is provided for displacing the hand and performing a contact operation when the start timing calculated by the means is reached.

In the first invention, the robot (12) includes at least a head (24), a body part (20), an arm part (30), and a hand part (38), and the robot (12) makes contact with a nearby person by the hand part. perform an action. The first acquisition means (50, 302d, S3) acquires the timing of occurrence of an event in which the communication robot expresses a predetermined emotion included in a plurality of emotions that can be expressed. The second acquisition means (50, 302d, S3) acquires emotions to be expressed at the event. The first setting means (50, 302e, S7) starts a contact motion, which is preset based on experimental results of measuring movements when a person expresses an emotion, corresponding to each of a plurality of emotions. A calculation condition for expression corresponding to the emotion for expression acquired by the second acquisition means is set from among the plurality of calculation conditions for calculating the timing. The calculation means (50, 302f, S9) calculates the start timing using the expression calculation conditions set by the first setting means, using the occurrence timing acquired by the first acquisition means as a reference. The motion control means (50, 302c, S13) displaces the hand and performs the contact motion when the start timing calculated by the calculation means is reached.

In the first invention, a plurality of calculation conditions are prepared in advance based on experimental results of measuring movements when a person expresses an emotion, corresponding to each of a plurality of emotions, and a predetermined When an event that expresses an emotion occurs, the start timing of the contact action is calculated using the expression calculation conditions corresponding to the expression emotion, and when the calculated start timing is reached, Perform a contact action. Therefore, the robot can naturally express emotions through contact movements.

A second invention is dependent on the first invention, and each of the plurality of calculation conditions is determined in advance based on the results of an experiment in which movements when a person expresses an emotion are measured according to each of a plurality of emotions. The calculation means is a communication robot that calculates the start timing according to a random number according to a probability density function including the average and variance included in the expression calculation conditions as parameters.

According to the second invention, the start timing is calculated according to a random number according to a probability density function with parameters of an average and variance that are preset based on experimental results of measuring movements when a person expresses emotions. This allows the robot to express emotions more naturally through touch movements.

A third invention is dependent on the first or second invention, in which a communication robot performs a contact action when viewing visually and/or audibly recognizable content alongside a nearby person. , an event is a communication robot that includes the climax of the content.

According to the third invention, the robot can naturally express emotions through contact motion.

A fourth invention is dependent on any one of the first to third inventions, and the contact movement is an experimental result of measuring movements when a person expresses an emotion in response to each of a plurality of emotions. further comprising a second setting means for setting a motion pattern for expression corresponding to the emotion for expression acquired by the second acquisition means from among the plurality of motion patterns based on the second acquisition means. , the motion control means is a communication robot that performs a contact motion according to a motion pattern for expression when the start timing is reached.

According to the fourth invention, a plurality of motion patterns are prepared based on the results of an experiment to measure movements when a person expresses an emotion, and a motion pattern for expression corresponding to the emotion to be expressed is prepared. Since the robot performs contact actions according to the following, it is possible for the robot to express emotions more naturally through contact actions.

A fifth invention is a control program executed by a computer of a humanoid communication robot that includes at least a head, a torso, an arm, and a hand, and performs a contact action of touching a nearby person with the hand. a first acquisition means for acquiring an occurrence timing at which an event to express a predetermined emotion included in a plurality of emotions that can be expressed by the communication robot occurs; The second acquisition means to acquire the expression information acquired by the second acquisition means from among the plurality of calculation conditions for calculating the start timing of the contact motion, which are preset corresponding to each of the plurality of emotions. a first setting means for setting a calculation condition for expression corresponding to the emotion; and a first setting means for setting a calculation condition for expression corresponding to the emotion; , a calculation means for calculating a start timing, and a control program that displaces a hand when the start timing calculated by the calculation means is reached, and functions as an operation control means for performing a contact operation.

A sixth invention is a method for controlling a humanoid communication robot that includes at least a head, a torso, an arm, and a hand, and performs a contact action of touching a nearby person with the hand, wherein the communication robot a first acquisition step of acquiring an occurrence timing at which an event to express a predetermined emotion included in a plurality of emotions that can be expressed; a second acquisition step of acquiring an emotion for expression to be expressed in the event; Out of a plurality of calculation conditions for calculating the start timing of the contact motion, which are preset corresponding to each of the emotions of a first setting step for setting calculation conditions; a calculation step for calculating a start timing using the expression calculation conditions set in the first setting step, based on the occurrence timing obtained in the first acquisition step; , and a motion control step of displacing the hand and performing a contact motion when the start timing calculated in the calculation step is reached.

According to the fifth and sixth inventions, the same effects as the first invention can be expected.

According to this invention, the robot can naturally express emotions to the person to be communicated with through contact motion.

The above objects, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

FIG. 1 is a diagram showing a system according to an embodiment of the present invention. FIG. 2 is a diagram showing the external configuration of the android robot shown in FIG. 1. FIG. 3(A) is a diagram for explaining actuators that move the eyelids and eyeballs of the android robot shown in FIG. 1. FIG. 3(B) is a diagram for explaining actuators that move the forehead, between the eyebrows, and the corners of the mouth of the android robot shown in FIG. FIG. 3C is a diagram for explaining an actuator that moves the lips of the android robot shown in FIG. FIG. 4 is a diagram for explaining actuators that move the head, shoulders, waist, arms, and hands of the android robot shown in FIG. FIG. 5 is a block diagram showing the electrical configuration of the android robot shown in FIG. 1. FIG. 6 is a diagram showing a state in which a person and an android robot are lined up for a contact operation in one embodiment. FIG. 7(A) is a diagram showing the state of the android robot's hand in a state where no emotion is expressed. FIG. 7(B) is a diagram showing the state of an android robot's hand in a state where it is expressing emotion by touching a target person with its hand. FIG. 7(C) is a diagram showing the state of the android robot's hand in a state where it is expressing emotion by touching the target person with its fingers. FIG. 8 is a table showing an example of the correspondence between climax time and emotion category in a certain content. FIG. 9 is a table showing examples of types of contact actions and calculation conditions corresponding to emotion categories. FIG. 10 is a diagram showing an example of a memory map of the RAM of the android robot. FIG. 11 is a flow diagram showing part of the robot control processing of the CPU of the android robot.

Referring to FIG. 1, an android robot control system (hereinafter sometimes simply referred to as "system") 10 according to an embodiment of the present invention includes an android robot (hereinafter simply referred to as "robot") 12. The robot 12 is a humanoid robot that has a figure (external appearance, etc.) that closely resembles a human (see FIGS. 2 to 4), and performs human-like movements (waving, behavior, speech) that closely resemble a human. Furthermore, although the details will be described later, the robot 12 of this embodiment performs a communication action (or action) including a contact action of touching a person (target person) who is a communication target with whom the robot 12 interacts (communicates).

The robot 12 is connected to a content presentation device 16 and a position detection device 18 via a network 14 such as a LAN (Local Area Network), the Internet, and a telephone communication line. The robot 12 communicates with the content presentation device 16, the position detection device 18, and other computers by wire or wirelessly (for example, infrared, WiFi (registered trademark), or Bluetooth (registered trademark)) without going through the network 14. You can also communicate directly with others.

The content presentation device 16 reproduces content that can be recognized visually and/or audibly by a person. In this embodiment, the content includes video, sound (audio), or both (hereinafter sometimes simply referred to as "video/sound").

Content presentation device 16 includes a playback device, a display, and a speaker. A display and a speaker are each connected to a playback device. A playback device is a device that plays back content and outputs video data, audio data, or both (video/sound data).

As the playback device, a device that plays back content stored in a storage medium or a storage section within the device itself can be used. Devices of this type include optical disc players, video decks, cassette decks, portable music players, smart phones, feature phones, desktop PCs, notebook PCs, tablet PCs, and the like. In addition, the playback device includes a device (for example, a smartphone or a PC) that plays back (streaming playback) content transferred from the outside via the network 14, etc., and a device that plays content by receiving a television signal or radio signal. (for example, a television receiver or a radio receiver), etc. can also be used.

As the display, a general-purpose display such as an LCD (Liquid Crystal Display) or an EL (Electro-Luminescence) display can be used, and as the speaker, a general-purpose speaker can be used. Further, the display and the speaker may be provided separately, or may be configured integrally, such as a display with a built-in speaker. Furthermore, the playback device, display, and speaker may be integrally configured.

When content is played back on a playback device, video data is input from the playback device to the display, the display outputs (displays) video corresponding to the input video data, and sound data is input from the playback device to the speaker. , the speaker outputs sound corresponding to the input sound data. Note that in the case of video-only content, only video data is output from the playback device, and in the case of sound-only content, only sound data is output from the playback device.

As described above, the content presentation device 16 presents video and audio content to the viewer. Further, the content presentation device 16 transmits data related to content (hereinafter referred to as “event data”) to the robot 12. Note that the brightness (brightness) of the display may be changed in conjunction with the contents of the video/sound content, for example, when a scene (climax) in which tension or excitement increases is reached. In this case, variations in display brightness also become part of the content. Additionally, in addition to or in place of the content presentation device 16, content that can be visually and/or audibly recognized by a human (content presenter) is presented to the viewer (picture show, story-telling, reading drama, etc.). You may.

The position detection device 18 detects the position of the subject's body, particularly the position of the subject's hands. In this embodiment, the position detection device 18 includes a computer (position detection computer) and a plurality of distance image sensors. A distance image sensor irradiates light such as infrared light or a laser, and captures the light reflected from an object (reflected light) using an optical sensor such as a CCD (Charge Coupled Device) sensor. A distance image sensor measures the actual distance to an object by measuring the time it takes for light to return for each pixel. Further, the distance image sensor can measure the shape of an object (for example, a predetermined part of a human such as the head and hands). Therefore, based on the output of the distance image sensor, the position of the person and the position of predetermined parts (head and hands) of the person can be detected.

As the distance image sensor, a product called Xtion (registered trademark) manufactured by ASUS (registered trademark) is used. In other examples, the distance image sensor may be a Kinect (registered trademark) sensor manufactured by Microsoft (registered trademark), a three-dimensional distance image sensor D-IMager (registered trademark) manufactured by Panasonic (registered trademark), etc. is also possible. This type of sensor is sometimes referred to as a three-dimensional distance measurement sensor, a 3D scanner, or the like. Note that as the distance image sensor, an omnidirectional laser distance meter such as LiDAR (for example, imaging unit LiDAR (HDL-32e) (trade name) manufactured by Velodine), a stereo camera, etc. can also be used.

A distance image signal from each of the distance image sensors is input to a position detection computer, and the position detection computer stores the distance image signal as digital data in a memory (for example, HDD or RAM). The position detection computer processes the distance image data stored in the memory to determine the position of one or more predetermined parts of a person (in this first embodiment, the three-dimensional position of the subject's hand). data (hereinafter referred to as "person position data") is transmitted to the robot 12.

Note that the configuration of the system 10 shown in FIG. 1 is an example, and another computer such as a server may be interposed between the robot 12, the content presentation device 16, and the position detection device 18.

Although not shown in FIG. 1, an external control terminal capable of communicating with the robot 12 may be provided. The external control terminal is a computer for remotely controlling the robot 12. For example, when the robot 12 communicates with humans and an event that cannot be handled by the robot 12 occurs, an administrator or the like uses an external control terminal to control the robot 12 and respond to the event that the robot 12 cannot handle. . An administrator or the like communicates with humans through the robot 12 or sends commands to the robot 12 to cause it to perform communication actions.

2 to 4 show an example of the appearance and configuration of the robot 12 in FIG. 1. Note that the robot 12 shown in FIGS. 2 to 4 is an example, and any android robot with other appearance and configuration can be used. Furthermore, the robot 12 is not limited to an android robot, and other robots may be used as long as they have a human-like appearance and are capable of contact movements. As an example, a communication robot can be used that can communicate with a human or other robot with whom it communicates through body movements and/or voice. The appearance and configuration of the robot 12 of this embodiment will be briefly described below with reference to FIGS. 2 to 4.

As shown in FIGS. 2 and 4, the robot 12 includes a body part 20 and a head 24 provided on the body part 20 via a neck part 22.

The upper end of the body part 20 (below the head) is the shoulder part 26, and the middle part of the body part 20 is the waist part 28. An upper limb part (arm part) 30 is provided from each of the left and right shoulder parts 26, and a lower limb part 32 is provided from the lower end of the body part 20.

Further, as shown in FIG. 4, the middle part of the upper limb part 30 is the elbow part 34, and the distal end of the upper limb part 30 has a wrist part 36 and a A hand portion 38 is provided, and the hand portion 38 includes five fingers 40 provided at the distal end.

Such a robot 12 is assembled as a whole using a structural material such as metal, and a skin is formed on the structural material using a soft resin such as silicone resin, and the skin is exposed through clothing at necessary parts. I'm letting you do it.

In the system 10 of this embodiment, the robot 12 is set while sitting on a chair 42, and each part is moved by pneumatic actuators to achieve human-like movements. In this first embodiment, the actuators described below for moving each of the above-mentioned parts (target parts) of the robot 12 are stepping motors driven by pulsed electric power, and the rotation amount of the stepping motor is determined by the number of pulses. It is determined by The number of pulses is given as a command value. In addition, in a normal state, each target part is given an initial value to the actuator, and when displacing it, a command value of the number of pulses corresponding to the direction and magnitude is given to the corresponding actuator. In this first embodiment, each actuator is operated with a command value of "0-255", and any value of "0-255" is given as an initial value. The configuration for moving each part of the robot 12 will be described below with reference to the drawings.

Referring to FIG. 3(A), actuator A1 is an actuator for controlling the vertical movement of upper eyelid 44a. Actuators A2, A3, and A4 are actuators for moving the eyeball left, right, up, and down. Actuator A5 is an actuator that controls the vertical movement of the lower eyelid 44b. Therefore, the eyes (eyelids) are opened and closed by actuators A1 and A5.

Referring to FIG. 3(B), actuator A6 is an actuator for moving the forehead, and actuator A7 is an actuator for moving the glabella. Further, the actuator A8 is an actuator for raising the corner of the mouth 46. Actuator A9 is an actuator for moving the tongue up and down.

Referring to FIG. 3(C), actuator A10 is an actuator that spreads the lips left and right, and actuator A11 is an actuator that pushes the lips forward. The actuator A13 is an actuator for protruding or pulling the jaw. This actuator 13 opens and closes the mouth.

Referring to FIG. 4, actuator A14 is an actuator for tilting head 24 left and right, actuator A15 is an actuator for raising and lowering head 24, and actuator A16 is an actuator for rotating head 24 left and right. This is an actuator for moving. The actuator A17 is an actuator for bending the waist portion 28 forward or tilting backward, and the actuator A18 is an actuator for rotating (twisting) the waist portion 28 from side to side.

Further, the actuator A19 is an actuator for vertically moving the shoulder section 26 (controlling the angle of the upper limb section 30 with respect to the torso section 20). The actuator A20 is an actuator for bending and extending the upper limb 30 around the elbow 34 (rotating the wrist 36 and the hand 38 around the elbow 34). The actuator A21 is an actuator for rotating the hand 38 around the wrist 36. The actuator A22 is an actuator for bending, extending or rotating the finger portion 40. Although not shown, the actuator A22 is provided corresponding to each of the five finger sections 40. By driving each of the actuators A19 to A22, the robot 12 uses the upper limb section 30 (elbow section 34, wrist section 36, hand section 38, finger section 40, etc.) to move the human arm, hand, and fingers. It is possible to perform actions that closely resemble movements. For example, it is possible to perform a contact action (touch action) in which the hand 38 or the finger 40 touches a nearby person.

Although not shown in the drawings, the lower limb section 32 is appropriately provided with an actuator for moving each part of the lower limb section 32.

As shown in FIG. 5, the robot 12 includes a processor (in this first embodiment, a CPU) 50 that controls the entire robot 12. The CPU 50 is connected to a RAM 54, a communication module 56, an actuator control circuit 58, a sensor I/F (interface) 60, an input/output I/F 62, etc. through a bus 52.

The CPU 50 is communicably connected via a communication module 56 to the network 14, that is, to external computers such as the content presentation device 16 and the position detection device 18, by wire or wirelessly.

The RAM 54 is used as a buffer area and a work area for the CPU 50. However, instead of the RAM 54, an HDD can also be used. The CPU 50 accesses the RAM 54 through the bus 52, and according to the programs and data stored in the RAM 54 (see FIG. 10), provides command values to the actuator control circuit 58 through the bus 52 to control the operation of each actuator A1-An. .

The actuator control circuit 58 drives each actuator A1-An by generating a number of pulsed electric powers according to the command value given from the CPU 50 and applying them to the corresponding stepping motors. However, in addition to using a stepping motor as the actuator, any actuator such as an actuator using a servo motor, a fluid actuator, etc. can be used.

Sensor I/F 60 receives respective outputs from tactile sensor 64 and eye camera 66.

The tactile sensor 64 or skin sensor is, for example, a touch sensor, and forms part of the tactile sense of the robot 12. That is, the tactile sensor 64 is used to detect whether a human or other object has touched the robot 12. Output (detection data) from the tactile sensor 64 is given to the CPU 50 via the sensor I/F 60. Therefore, the CPU 50 can detect that a human or other object has touched the robot 12 (and the strength of the touch).

The eye camera 66 is an image sensor and forms part of the robot's 12 vision. That is, the eye camera 66 is used to detect images or images seen from the eyes of the robot 12. In this first embodiment, data (image data) corresponding to a captured image (moving image or still image) of the eye camera 66 is provided to the CPU 50 via the sensor I/F 60. CPU 50 stores the image data in RAM 54 or transmits it to an external computer (for example, an external control terminal) via communication module 56 and network 14 (FIG. 1).

Moreover, the speaker 68 and the microphone 70 are connected to the input/output I/F 62. The speaker 68 outputs audio when the robot 12 speaks. The microphone 70 is a sound sensor and constitutes a part of the robot 12's hearing system. This microphone 70 has directivity and is mainly used to detect the voice of a person interacting (communicating) with the robot 12.

The system 10 having such a configuration is a system for presenting physical or physical stimulation (tactile stimulation) to the user through the contact motion of the robot 12. They perform communication behaviors that express various emotions by providing tactile stimulation. However, the emotions that can be expressed by the robot 12 of this embodiment include emotion, enjoyment, sadness, anger, and fear.

As shown in FIG. 6, the robot 12 of this embodiment can be visually and/or audibly recognized by the content presentation device 16 while sitting side by side facing the same direction as the subject. Communicative behavior is performed while watching content (such as a movie) that includes visual images and sounds with the target person. In this embodiment, the robot 12 operates the hand 38 to touch the back of the subject's hand (right hand) with the hand 38 (left hand 38 or left finger 40). Expressing emotions through movement.

As shown in FIG. 7A, when not performing a contact operation, the robot 12 positions the hand 38 a little away from the subject's hand so that the hand 38 does not touch the subject's hand. 7(B) and FIG. 7(C), the robot 12 moves the hand 38 away from the subject. The hand portion 38 is positioned so as to overlap the hand, that is, so that the hand portion 38 touches the subject's hand. Note that, as shown in FIG. 7(B), the entire hand portion 38 may touch the subject's hand, or only the finger portion 40 may touch the subject's hand as shown in FIG. 7(C). You can do it like this.

However, the contact motion of the robot 12 is performed when an event that requires expression of emotion occurs. In this embodiment, the event in which emotions should be expressed is an important scene, a scene where the content changes significantly, a scene where tension or excitement increases, or the like, in the content being viewed, that is, a climax. The content reproduced by the content presentation device 16 includes at least one event, and the contact motion of the robot 12 is performed in response to the event of the content being viewed.

The content presentation device 16 transmits data related to events (hereinafter referred to as "event data") to the robot 12. The event data includes data on the timing at which an event (climax) occurs (hereinafter referred to as "occurrence timing") and data on emotions to be expressed in the event (hereinafter referred to as "emotions for expression"). In this embodiment, since the event is the climax of the content being viewed, the robot 12 acquires data about the climax of the content being viewed from the content presentation device 16 as event data.

Regarding which part (scene) of the content to set as the climax, it is possible to analyze the content and automatically extract the important scene as the climax, or after viewing the content by a human. A scene that humans typically consider may be set as the climax. Note that the technique for extracting important scenes from content is disclosed in detail in Japanese Patent Laid-Open No. 2004-342272, etc., so please refer to it.

FIG. 8 is a diagram showing an example of the contents of event data. As shown in FIG. 8, the event data includes an event ID, which is identification information for identifying (specifying) each event included in the content, and corresponding to each event ID, the occurrence timing and timing of the event are determined. , and the type (category) of emotion for expressing the event. Note that the event ID may be omitted if the robot 12 can recognize the correspondence between the timing of occurrence and the type of emotion to be expressed.

In the example shown in FIG. 8, the occurrence timing is expressed as the elapsed time from the start time of the content (0 minutes 0 seconds). For example, an event that expresses "impression" (event with event ID "1") occurs 6 minutes and 54 seconds after the start of the content, and an event that expresses "sadness" occurs 8 minutes and 48 seconds after the start of the content. An event (event with event ID "2") occurs, and 15 minutes and 18 seconds after the start of the content, an event to express "anger" (event with event ID "3") occurs, and from the start of the content An event that expresses “fear” (event with event ID “4”) occurs 24 minutes and 18 seconds later, and an event that expresses “fun” (event ID “5”) occurs 27 minutes and 12 seconds after the start of the content. event) occurs.

Further, in the system 10, calculation conditions for calculating the timing to start a contact action (hereinafter referred to as "start timing") based on the timing of occurrence of an event are set in advance for each type of emotion. For example, the start timing is calculated according to a random number according to a probability density function with the average and variance as parameters (parameters). As the probability density function, probability density functions such as normal distribution, binomial distribution, and inverse Gaussian distribution can be used.

FIG. 9 is a diagram showing the average and variance as parameters of calculation conditions for each type of emotion, and the movement pattern for each type of emotion. However, the average value (average value) is a value that indicates the time difference (seconds) between the average timing (average timing) set in advance for each type of emotion and the event occurrence timing, and is expressed as a positive number. The average value represented by a negative number indicates a period (time) during which the average timing is delayed (moved down) relative to the occurrence timing, and the average value represented by a negative number represents a period during which the average timing is advanced (moved up) relative to the occurrence timing. In addition, the variance value (dispersion value) is a value that indicates the dispersion of the values taken by random numbers. The possible values of will begin to vary.

Further, the contact motion of the robot 12 includes a plurality of motion patterns (motion types) for each type of emotion. The robot 12 performs a contact motion according to a motion pattern (motion pattern for expression) corresponding to the emotion assigned to the event. A plurality of movement patterns are prepared corresponding to each of the emotions that the robot 12 can express, and each of the plurality of movement patterns includes movement characteristics that are different from each other. For example, the characteristics of the movement include the strength (pressure) of the contact action, the continuity of the contact action, the length of the contact action, and the like.

In addition, the inventors had multiple subjects view content that includes an event that requires the expression of emotions, and when a person expresses an emotion (when an event occurs), they come into contact with another person nearby. We conducted an experiment to measure the start timing of contact movements. In addition, when an android robot expresses a specific emotion, it makes contact with the subject according to multiple types of movement patterns, and an experiment was conducted to measure the movement pattern that the subject recognized as preferable depending on the type of emotion. These experiments were conducted for each of "emotion," "enthusiasm," "sadness," "anger," and "fear," and the average values, variance values, and movement patterns set for each type of emotion are shown in Figure 9. has been determined based on the results of this experiment.

As can be seen from the average values for each type of emotion shown in Figure 9, the average timing of human touch movements when expressing “impression,” “enthusiasm,” “sadness,” and “anger” is faster than the timing of occurrence. The average timing of a person's contact motion when expressing “fear” is set later and is set before the occurrence timing. In other words, people's contact movements to express "impression," "enthusiasm," "sadness," and "anger" are often performed a little later than the timing of the event, and are often used to express "fear." In this case, a person's contact action tends to be performed slightly before the event occurs.

Furthermore, the average timing of people's contact movements when expressing "enthusiasm," "anger," and "fear" tends to have a small time difference with respect to the timing of occurrence, while when expressing "impression" and "sadness." The average timing of a person's contact movements tends to have a large time difference with respect to the timing of occurrence.

Furthermore, as can be seen from the variance values for each type of emotion shown in Figure 9, there tends to be large individual differences in the timing of the start of contact movements when people express “sadness”; There is a tendency for individual differences in the expression of ``to be small.''

As can be seen from Figure 9, from the perspective of the strength (pressure) of the contact motion as an element of the motion pattern, in the motion patterns expressing "impression," "enthusiasm," and "sadness," the contact motion The strength of the motion is set to a predetermined strength (normal strength), and the motion pattern that expresses "anger" and "fear" expresses "impression", "enthusiasm", and "sadness". The strength is set to be stronger than when

In addition, from the perspective of the continuity of the contact motion, the motion pattern when expressing "impression," "enthusiasm," and "anger" is to tap the subject's hand several times with the hand part 38. It is set to intermittent movement like this. On the other hand, the motion pattern for expressing "sadness" and "fear" is set to be a continuous motion in which the hand 38 is placed on the subject's hand for a predetermined period of time.

Furthermore, from the perspective of the length of the contact motion, the length of the contact motion is set to a predetermined length (equivalent to "short") in the motion patterns when expressing "enthusiasm" and "anger." , the length of the contact motion is set longer in the motion patterns for expressing "impression", "sadness", and "fear" than for the motion patterns for expressing "enthusiasm" and "anger". In addition, in the case of a movement pattern that involves intermittent movement, the length of the contact movement is the time from when the subject's hand is first clapped to when the subject's hand is last clapped during the contact movement. Refers to length.

In the system 10, when an emotion for expression to be expressed in an event is set (obtained), calculation conditions for expression are set using the average and variance set for the emotion for expression as calculation parameters. , using the expression calculation conditions, the start timing of the contact motion is calculated based on the event occurrence timing. Further, a movement pattern of the contact movement (a movement pattern for expression) is set according to the emotion to be expressed. Then, when the robot 12 reaches the start timing, it performs a contact motion according to the expression motion pattern.

In this example, the start timing of the contact motion is set (calculated) and the movement pattern of the contact motion is set based on the experimental results of measuring the movements when a person expresses an emotion. It is thought that it is possible to reproduce natural contact movements that are similar to physical movements.

In addition to the contact motion, the robot 12 also performs body motions such as predetermined shaking and behavior, and/or speech motions. In this first embodiment, the control information for the robot 12 to execute various operations such as contact operations is control data for driving and controlling the actuators (A1 to A22) and/or synthesized voice for the content of the utterance. It includes data and is stored in correspondence with the name (command name) given to each communication action. However, body movements also include natural movements (unconscious movements). Typical examples of unconscious movements include blinking and breathing. In addition to such physiological actions, unconscious actions also include actions due to human habit. For example, examples of habitual actions include touching one's hair, touching one's face, and biting one's nails. In addition to body movements, the control information also includes control data for expressing mouth movements and facial expressions.

Although omitted in FIG. 4, control information regarding communication behavior is stored in a nonvolatile memory such as an HDD, and is read from the HDD and written to the RAM 54 as necessary.

FIG. 10 is a diagram showing an example of a memory map 300 of the RAM 54 shown in FIG. 4. As shown in FIG. 10, RAM 54 includes a program storage area 302 and a data storage area 304.

The program storage area 302 stores a communication program 302a, a human position detection program 302b, a motion control program 302c, an event data acquisition program 302d, a calculation condition setting program 302e, a timing calculation program 302f, a motion pattern setting program 302g, and the like.

The communication program 302a is a program for communicating with the content presentation device 16, the position detection device 18, other robots, external computers, and the like. The human position detection program 302b detects the three-dimensional position of a predetermined part (target part) of the subject who is the target of the contact operation based on the human position data transmitted from the position detection device 18 (in this embodiment, the subject's This is a program for detecting or acquiring the three-dimensional position of the right hand.

The motion control program 302c is a program for controlling the motion of the robot 12, and includes an upper body (shoulder) control program 3020, an upper limb (elbow) control program 3022, a wrist control program 3024, and a finger control program 3026.

The upper body (shoulder) control program 3020 uses the actuator shown in FIG. This is a program for controlling A19.

The upper limb (elbow) control program 3022 is a program for controlling the actuator A20 shown in FIG. 4 so that the upper limb part 30 (elbow part 34) of the robot 12 moves in accordance with body movements such as contact movements. .

The wrist control program 3024 is a program for controlling the actuator A21 shown in FIG. 4 so that the wrist portion 36 of the robot 12 moves in accordance with body movements such as contact movements.

The finger control program 3026 is a program for controlling the actuator A22 shown in FIG. 4 so that the finger section 40 of the robot 12 operates in accordance with body motions such as contact motions.

Although a detailed explanation will be omitted, the motion control program 302c includes a facial expression control program that appropriately controls each of the actuators A1 to A18 shown in FIG. 3 to express the facial expressions of the eyes and/or mouth of the robot 12, 12, a head control program that moves the head 24 of the robot 12, and a lower body (waist) control program that moves the lower body (waist) 32 of the robot 12.

The event data acquisition program 302d is a program for acquiring event data transmitted from the content presentation device 16. The calculation condition setting program 302e is a program for setting calculation conditions for expression using the average and variance as calculation parameters, depending on the type of emotion for expression of the event. The timing calculation program 302f is a program for calculating the start timing of a contact motion using the expression calculation condition based on the event occurrence timing. The motion pattern setting program 302g is a program for setting motion patterns for expression according to the type of emotion to be expressed.

Although not shown in the figure, the program storage area 302 includes a voice recognition program for the CPU 50 to recognize the content of what a human has uttered to the robot 12, and a voice for outputting the voice synthesized according to the voice synthesis data to the speaker 68. A synthesis program, other programs for controlling the robot 12, etc. are stored.

Various data are stored in the data storage area 304 of the RAM 54. The human position data 304a is data on the three-dimensional position of a predetermined part of the subject who is the target of the contact motion, which is detected or acquired according to the human position detection program 302b. The event data 304b is data acquired by the event data acquisition program 302d, and includes data on the timing of occurrence of an event and data on emotions to be expressed in the event. The calculation condition data 304c is preset for each emotion that the robot 12 can express, and is calculation condition data for calculating the start timing based on the event occurrence timing. The motion pattern data 304d is motion pattern data that is prepared corresponding to each emotion that the robot 12 can express.

Although not shown, the data storage area 304 also stores voice data input from the microphone 70 and data such as a dictionary necessary for voice recognition. In addition, the data storage area 304 stores other data such as control information for executing communication actions necessary for controlling the robot 12, and also stores timers, registers, flags, etc. necessary for the operation of the CPU 50. provided.

FIG. 11 is a flowchart showing an example of robot control processing by the CPU 50 shown in FIG. This robot control process is started when the content is played back by the content presentation device 16. As shown in FIG. 11, when the CPU 50 starts the robot control process, in step S1, the CPU 50 acquires (receives) event data transmitted from the content presentation device 16 according to the event data acquisition program 302d, and in step S3, The timing of occurrence of the next (immediately) event and the emotion to be expressed included in the event data acquired in step S1 are acquired.

Subsequently, in step S5, according to the motion pattern setting program 302g, the motion pattern data 304d stored in the data storage area 304 of the RAM 54 is read out, and the motion pattern data 304d is prepared corresponding to each emotion that the robot 12 can express. From among the motion patterns, a motion pattern for expression is set according to the emotion to be expressed.

Next, in step S7, according to the calculation condition setting program 302e, the calculation condition data 304c stored in the data storage area 304 of the RAM 54 is read out, and the calculation conditions set in advance for each emotion that the robot 12 can express are read out. From this, calculation conditions for expression are set according to the emotion to be expressed.

Subsequently, in step S9, the start timing is calculated using the expression calculation condition based on the event occurrence timing according to the timing calculation program 302f, and in step S11, the start timing calculated in step S9 is reached. determine whether or not.

If "NO" in step S11, that is, if it is determined that the start timing has not been reached, the process returns to step S11. On the other hand, if "YES" in step S11, that is, if it is determined that the start timing has been reached, then in step S13, a contact motion (a motion of touching the target person) is performed according to the motion pattern for expression. Here, a command value is given to the actuator control circuit 58 to control the operation of each actuator A1-An, and the hand 38 is moved so that the hand 38 touches a predetermined part of the subject who is the target of the contact action. Displace.

Subsequently, in step S15, it is determined whether or not to end the contact operation. If "NO" in step S15, that is, if it is determined that the contact operation is not to be completed, the process returns to step S13. On the other hand, if "YES" in step S15, that is, if it is determined that the contact motion is to be ended, it is determined in step S17 whether or not the next event exists based on the event data acquired in step S1. .

If "YES" in step S17, that is, if it is determined that the next event exists, the process returns to step S3. On the other hand, if "NO" in step S17, that is, if it is determined that the next event does not exist, the robot control process ends.

In this first embodiment, a plurality of calculation conditions are prepared in advance based on the results of experiments that measure movements when people express emotions, corresponding to each of a plurality of emotions. When an event that expresses an emotion occurs, the start timing of the contact action is calculated using the expression calculation conditions corresponding to the expression emotion, and when the calculated start timing is reached, , performs a contact action. Therefore, the robot can naturally express emotions through contact movements.

Note that the specific numerical values shown in each of the above-mentioned examples are merely examples, and there is no need to limit them, and they can be changed as appropriate depending on the actual product and the environment in which the product is applied.

Further, in the robot control processing shown in the above-described embodiment, the order in which each step is processed may be changed as long as the same result is obtained.

Furthermore, the robot 12 and the position detection device 18 may be integrally configured. In this case, the eye camera 66 of the robot 12 functions as a distance image sensor of the position detection device 18.

Furthermore, the robot 12 may perform communication actions while sitting face-to-face with the target person and viewing the content together with the target person.

In the above-described embodiment, the robot 12 expresses emotions by touching the subject's hand (right hand). Emotions may be expressed by performing a touching action.

10...System 12...Android robot 14...Network 16...Content presentation device 18...Position detection device 26...Shoulder 30...Upper limb 36...Wrist 38...Hand 40...Finger 50...CPU
54...RAM
56...Communication module 58...Actuator control circuit

Claims (6)

A humanoid communication robot that includes at least a head, a torso, an arm, and a hand, and performs a contact action of touching a nearby person with the hand,
first acquisition means for acquiring an occurrence timing at which an event for expressing a predetermined emotion included in a plurality of emotions that can be expressed by the communication robot occurs;
a second acquisition means for acquiring emotions for expression expressed in the event;
The emotion for expression acquired by the second acquisition means is selected from among a plurality of calculation conditions for calculating the start timing of the contact motion, which are preset corresponding to each of the plurality of emotions. first setting means for setting calculation conditions for corresponding expression;
a calculation means for calculating the start timing using the expression calculation condition set by the first setting means, based on the occurrence timing acquired by the first acquisition means; A communication robot, comprising a motion control means for displacing the hand and performing the contact motion when the calculated start timing is reached. Each of the plurality of calculation conditions includes an average and a variance that are preset according to each of the plurality of emotions,
2. The communication robot according to claim 1, wherein the calculation means calculates the start timing according to a random number according to a probability density function whose parameters are an average and a variance included in the expression calculation conditions. The communication robot performs the contact action when viewing visually and/or audibly recognizable content alongside a nearby person;
The communication robot according to claim 1 or 2, wherein the event includes a climax of the content. The contact motion includes a plurality of motion patterns corresponding to each of the plurality of emotions,
further comprising a second setting means for setting an expression movement pattern corresponding to the expression emotion acquired by the second acquisition means from among the plurality of movement patterns;
4. The communication robot according to claim 1, wherein the motion control means performs the contact motion according to the expression motion pattern when the start timing is reached. A control program executed by a computer of a humanoid communication robot that includes at least a head, a torso, an arm, and a hand, and that performs a contact action of touching a nearby person with the hand, the computer controlling the computer. ,
first acquisition means for acquiring an occurrence timing at which an event for expressing a predetermined emotion included in a plurality of emotions that can be expressed by the communication robot occurs;
a second acquisition means for acquiring emotions for expression expressed in the event;
The emotion for expression acquired by the second acquisition means is selected from among a plurality of calculation conditions for calculating the start timing of the contact motion, which are preset corresponding to each of the plurality of emotions. first setting means for setting calculation conditions for corresponding expression;
a calculation means for calculating the start timing using the expression calculation condition set by the first setting means, based on the occurrence timing acquired by the first acquisition means; A control program that, when the calculated start timing is reached, causes the hand to function as a motion control means for performing the contact motion by displacing the hand. A method for controlling a humanoid communication robot that includes at least a head, a body, an arm, and a hand, and performs a contact action of touching a nearby person with the hand, the method comprising:
a first acquisition step of acquiring an occurrence timing at which an event for expressing a predetermined emotion included in a plurality of emotions that can be expressed by the communication robot occurs;
a second acquisition step of acquiring emotions for expression to be expressed in the event;
The emotion for expression acquired in the second acquisition step is selected from among a plurality of calculation conditions for calculating the start timing of the contact motion, which are preset corresponding to each of the plurality of emotions. a first setting step of setting calculation conditions for the corresponding expression;
a calculation step of calculating the start timing using the expression calculation condition set in the first setting step based on the occurrence timing obtained in the first obtaining step; A control method including a motion control step of displacing the hand and performing the contact motion when the calculated start timing is reached.

Images