JP2006247780A - Communication robot - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a communication robot, achieving natural interaction as if it is like a human being. <P>SOLUTION: This communication robot 12 includes a CPU, wherein the CPU manages the whole processing of the robot 12. The robot 12 detects the distance (to a personal) between the robot and the human being 14, and the direction of the face of the human being 14 to the robot 12. The robot 12 detects the felicity of interaction parameters (a distance to a person, the watch time, the motion start time, the motion speed), that is, comfort/discomfort of interaction, and the interaction parameters are updated to optimize the above. Thus, comfortable interaction conformable to individuals can be performed. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a communication robot, and more particularly to a communication robot that performs an interaction action including at least one of speech and body movement with a human.

  In order to communicate naturally, adaptation to the partner is important. For example, in order for a person to spend comfortably, an appropriate personal space is required. Non-Patent Document 1 discloses that this personal space differs depending on the content of communication. In addition, as disclosed in Non-Patent Document 2, it is also known that the frequency of matching the line of sight varies from person to person. Comfortable personal spaces and eye-gaze frequency vary from person to person, but people are comfortable with each other according to their communication partners. For example, if the other party feels too close, he / she leaves a little, and as disclosed in Non-Patent Document 3, he / she turns his / her gaze when he / she feels too staring. Furthermore, as disclosed in Non-Patent Document 4 to Non-Patent Document 6, researches are being made on the application of the concept of personal space for robots related to humans.

An example of background technology approximating this type of communication robot is disclosed in Patent Document 1. According to this patent document 1, the behavior pattern generation device is applied to a robot, for example. The behavior pattern generation device detects the user's interpersonal distance with respect to the robot, obtains the user's intimacy with respect to the robot according to the interpersonal distance, and determines whether the user is communicating with the robot according to the intimacy It is. Further, in the behavior pattern generation device, the user's emotion is estimated based on the strength of the user's voice, the level of the tone, the blood pressure, the body temperature, and the like. The behavior pattern generation device causes the robot to perform a behavior according to the interpersonal distance or the user's emotion.
ET Hall. The Hidden Dimension. Double Day Publishing, 1966. S. Duncan jr. And DW Fiske. Face-to-Face Interaction: Research, Methods, and Theory. Lawrence Erlbaum Associates, Inc., Publishers, 1977. E. Sundstrom and I. Altman. Interpersonal relationships and personal space: Research review and theoretical model.Human Ecology, 4 (1), 1976. Nakajima A study on the individual distance of a human to a mobile robot. Doctoral dissertation, Kyushu Institute of Technology, 1998. Y. Nakauchi and R. Simmons.A social robot that stands in line.Autonomous Robots, 1: 313-324, 2002. T. Tasaki, S. Matsumoto, K. Komatani, T. Ogata, HG Okuno.Dynamic communication of humanoid robot with multiple people based on interaction distance.In Proc. Of International Workshop on Robot and Human Interaction (Ro-Man-2004) , pp.81-86, IEEE, 2004. JP 2004-66367 A

  However, in the background art robot, the personal space is fixed, and there is no one adapted to the individual. However, in the behavior pattern generation device disclosed in Patent Literature 1, the user's emotion is estimated based on the strength of the user's voice, the level of the tone, and the blood pressure, the body temperature, and the like. It can be said that the communication (interaction) adapted to the individual and the emotion of the individual is performed, but the interpersonal distance is only determined whether or not the communication with the robot is performed by threshold processing. In other words, an appropriate personal space was not adapted to the individual. For this reason, a user or a person who communicates with the robot may feel uncomfortable in the communication.

  Therefore, the main object of the present invention is to provide a novel communication robot.

  Another object of the present invention is to provide a communication robot capable of realizing natural interactions like people.

  The invention according to claim 1 is a communication robot that interacts with a human, and includes parameter setting means for setting parameters for the interaction, and interaction including at least one of speech and physical movement according to the parameters set by the parameter setting means. It is a communication robot provided with the interaction execution means which performs, the appropriateness detection means which detects the appropriateness of the parameter in interaction, and the optimization means which optimizes the appropriateness detected by the appropriateness detection means.

  According to the first aspect of the present invention, the communication robot performs an interaction action including at least one of body movement and speech with a human. The parameter setting means sets a parameter for interaction (interaction action). The interaction executing means executes the interaction action according to the parameter set by the parameter setting means. The appropriateness detection means detects the appropriateness of the parameter during the interaction. Here, if a person as an interaction (communication) partner feels comfortable, it can be said that the appropriateness of the parameter is high. On the other hand, when a human feels uncomfortable, it can be said that the appropriateness of the parameter is low. For example, whether or not the interaction feels uncomfortable depends on the distance (movement distance) of the human relative to the communication robot, the orientation of the human face relative to the communication robot, whether or not the human is poverty, Laughter (soft), hard (hard) and human footsteps can be known. The optimization means optimizes the appropriateness detected by the appropriateness detection means. That is, the interaction parameter is adapted to the interaction partner.

  According to the first aspect of the present invention, the interaction parameter is adapted to the interaction partner, so that the interaction can be performed comfortably as the interaction is repeated. Therefore, natural communication like people is possible.

  The invention of claim 2 is dependent on claim 1, and further includes a moving distance detecting means for detecting a moving distance of the human during the interaction, and a time detecting means for detecting a time during which the human views the face of the communication robot itself. The appropriateness detection means detects the appropriateness of the parameter based on the detection result of at least one of the action distance detection means and the time detection means when the interaction is executed with the parameter set by the parameter setting means.

  According to a second aspect of the present invention, the communication robot further includes action distance detecting means and time detecting means. The movement distance detection means detects the movement distance of the person during the interaction. The time detection means detects the time during which the human is looking at his / her face during the interaction, that is, the time during which he / she is gazing. For example, when the movement distance of a person during interaction is long (large) or when the gaze time is short, it can be determined that the person feels uncomfortable with the interaction. Conversely, when the moving distance is short (small) or when the gaze time is long, it can be determined that the person feels comfortable in the interaction. The appropriateness detecting means detects the appropriateness of the parameter based on the detection result of at least one of the action distance detecting means and the time detecting means when the interaction is executed with the parameter set by the parameter setting means.

  According to the second aspect of the present invention, it is possible to know the comfort of the interaction based on the human action during the interaction, and it is possible to optimize the parameters so as to increase the comfort.

  The invention of claim 3 is dependent on claim 1 or 2, and the parameters are the interpersonal distance in the interaction with the human, the length of time for directing his / her face to the human face, the delay from the utterance to the start of the movement of the body movement Including at least one of time and speed of physical motion.

  In the invention of claim 3, the parameter includes a component that is considered to determine the comfort of interaction when the robot and the human being communicate with each other. Specifically, the parameters are the interpersonal distance in the interaction with the human, the length of time to turn his face to the human face (gaze time), the delay time from the utterance to the start of the physical motion, and the physical motion Including at least one of the speeds.

  According to the third aspect of the present invention, if the component that is considered to determine the comfort of the interaction is updated, the appropriateness of the parameter can be optimized to realize a comfortable interaction.

  The invention of claim 4 is dependent on claim 3, and the interpersonal distance includes an intimate distance, an individual distance, and a social distance.

  In the invention of claim 4, the interpersonal distance includes an intimate distance, an individual distance, and a social distance. This is to allow an appropriate interpersonal distance to be adapted to the individual according to the type of interaction action. For example, when performing an interaction action such as self-introduction or greeting, social distance is taken.

  According to the invention of claim 4, since the intimate distance, the individual distance, and the social distance are included as the interpersonal distance of the parameter, the interpersonal distance corresponding to the type of interaction action can be taken corresponding to the individual.

  The invention of claim 5 is dependent on any one of claims 1 to 4, and the optimization means includes parameter update means for updating a parameter.

  In the invention of claim 5, the parameter updating means updates the parameter. Thus, for example, each time an interaction is performed, the parameter can be updated so that the appropriateness of the parameter is optimized.

  According to the fifth aspect of the present invention, since the parameter is updated each time the interaction is performed and the appropriateness of the parameter is optimized, more comfortable interaction can be performed as the interaction is repeated.

  The invention of claim 6 is dependent on any one of claims 1 to 5, and further comprises parameter storage means for storing parameters corresponding to humans, and human identification means for identifying humans at the start of interaction, parameter setting means When the parameter corresponding to the person identified by the human identification means is stored in the parameter storage means, the parameter is set, and the parameter corresponding to the person identified by the human identification means is stored in the parameter storage means. If not, the average value of all parameters stored by the parameter storage means is set.

  In the invention of claim 6, the parameter storage means stores the parameter corresponding to the person. That is, an interaction is performed with a human and the optimized parameters are stored corresponding to the human. The human identification means identifies a human at the start of interaction. The parameter setting means sets the parameter when the parameter corresponding to the person identified by the human identification means is stored in the parameter storage means, that is, if the other party has previously interacted. However, if the parameter corresponding to the person identified by the human identification means is not stored by the parameter storage means, that is, if it is not a previously interacting partner, the average value of all parameters stored by the parameter storage means is set. To do. However, in such a case, parameters for a person similar to the person who interacts this time may be set.

  According to the sixth aspect of the present invention, since the parameter optimized last time is used for a person who has experience of interaction, relatively comfortable interaction can be executed from the beginning in this interaction.

  According to the present invention, since the appropriateness of the parameter for the interaction is detected and optimized based on the moving distance and the face direction of the person at the time of interaction, it can be adapted to the interaction partner. In other words, natural communication like people can be realized by interaction adapted to individuals.

  The above object, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

  Referring to FIG. 1, a communication robot system (hereinafter simply referred to as “system”) 10 of this embodiment includes a communication robot (hereinafter simply referred to as “robot”) 12. The robot 12 is interaction-oriented for the purpose of communicating with a communication target (partner) such as a human 14, and uses at least one of body movement (gesture, hand gesture) and speech (voice). It has a function to perform the communication (interaction) action (hereinafter sometimes referred to as “interaction action”).

  The robot 12 has a human-like body and uses the body to generate complex body movements necessary for interaction. Specifically, referring to FIG. 2, the robot 12 includes a carriage 32, and wheels 34 for autonomously moving the robot 12 are provided on the lower surface of the carriage 32. The wheel 34 is driven by a wheel motor (indicated by reference numeral “36” in FIG. 3 showing the internal configuration of the robot 12), and the carriage 32, that is, the robot 12 can be moved in any direction.

  Although not shown in FIG. 2, a collision sensor (indicated by reference numeral “38” in FIG. 3) is attached to the front surface of the carriage 32, and the collision sensor 38 is connected to a person or other person to the carriage 32. Detect obstacle contact. When contact with an obstacle is detected during the movement of the robot 12, the driving of the wheels 34 is immediately stopped and the movement of the robot 12 is suddenly stopped.

  In this embodiment, the height of the robot 12 is about 100 cm so as not to intimidate people, particularly children. However, this height can be arbitrarily changed.

  A polygonal column sensor mounting panel 40 is provided on the carriage 32, and an ultrasonic distance sensor 42 is mounted on each surface of the sensor mounting panel 40. The ultrasonic distance sensor 42 measures the distance between the mounting panel 40, that is, the person around the robot 12 mainly.

  Further, the body of the robot 12 is mounted on the carriage 32 so that the lower portion thereof is surrounded by the mounting panel 40 described above and stands upright. The body is composed of a lower body 44 and an upper body 46, and the lower body 44 and the upper body 46 are connected by a connecting portion 48. Although not shown, the connecting portion 48 has a built-in lifting mechanism, and the height of the upper body 46, that is, the height of the robot 12 can be changed by using the lifting mechanism. As will be described later, the elevating mechanism is driven by a waist motor (indicated by reference numeral “50” in FIG. 3). The height 100 cm of the robot 12 described above is a value when the upper body 46 is at its lowest position. Therefore, the height of the robot 12 can be 100 cm or more.

  One omnidirectional camera 52 and one microphone 16 are provided in the approximate center of the upper body 46. The omnidirectional camera 52 photographs the surroundings of the robot 12 and is distinguished from an eye camera 54 described later. The microphone 16 captures ambient sounds, particularly human voice.

  Upper arms 58R and 58L are attached to both shoulders of the upper body 46 by shoulder joints 56R and 56L, respectively. The shoulder joints 56R and 56L each have three degrees of freedom. That is, the right shoulder joint 56R can control the angle of the upper arm 58R around each of the X, Y, and Z axes. The Y axis is an axis parallel to the longitudinal direction (or axis) of the upper arm 58R, and the X axis and the Z axis are axes orthogonal to the Y axis from different directions. The left shoulder joint 56L can control the angle of the upper arm 58L around each of the A, B, and C axes. The B axis is an axis parallel to the longitudinal direction (or axis) of the upper arm 58L, and the A axis and the C axis are axes orthogonal to the B axis from different directions.

  Forearms 62R and 62L are attached to the respective distal ends of upper arms 58R and 58L via elbow joints 60R and 60L. The elbow joints 60R and 60L can control the angles of the forearms 62R and 62L around the W axis and the D axis, respectively.

  In the X, Y, Z, W axes and the A, B, C, D axes that control the displacement of the upper arms 58R and 58L and the forearms 62R and 62L (FIG. 2), “0 degree” is the home position. In this home position, the upper arms 58R and 58L and the forearms 62R and 62L are directed downward.

  Although not shown in FIG. 2, a touch sensor (FIG. 3) is provided on the shoulder portion including the shoulder joints 56R and 56L of the upper body 46 and the arm portion including the upper arms 58R and 58L and the forearms 62R and 62L. The touch sensor 64 detects whether or not a person has touched these parts of the robot 12.

  Spheres 66R and 66L corresponding to hands are fixedly attached to the tips of the forearms 62R and 62L, respectively. However, when a finger function (gripping, grasping, picking, etc.) is required, a “hand” in the shape of a human hand may be used instead of the spheres 66R and 66L.

  A head 70 is attached to an upper center of the upper body 46 via a neck joint 68. The neck joint 68 has three degrees of freedom and can be controlled in angle around each of the S, T, and U axes. The S-axis is an axis that goes directly from the neck, and the T-axis and the U-axis are axes that are orthogonal to the S-axis in different directions. The head 70 is provided with a speaker 72 at a position corresponding to a human mouth. The speaker 72 is used for the robot 12 to communicate with a person around it by voice or voice. However, the speaker 72 may be provided in another part of the robot 12, for example, the trunk.

  The head 70 is provided with eyeball portions 74R and 74L at positions corresponding to the eyes. Eyeball portions 74R and 74L include eye cameras 54R and 54L, respectively. The right eyeball portion 74R and the left eyeball portion 74L may be collectively referred to as an eyeball portion 74, and the right eye camera 54R and the left eye camera 54L may be collectively referred to as an eye camera 54. The eye camera 54 captures the video signal by photographing the face of the person approaching the robot 12 and other parts or objects.

  Note that each of the omnidirectional camera 52 and the eye camera 54 described above may be a camera using a solid-state imaging device such as a CCD or a CMOS.

  For example, the eye camera 54 is fixed in the eyeball part 74, and the eyeball part 74 is attached to a predetermined position in the head 70 via an eyeball support part (not shown). The eyeball support unit has two degrees of freedom and can be controlled in angle around each of the α axis and the β axis. The α axis and the β axis are axes set with respect to the head 70, the α axis is an axis in a direction toward the top of the head 70, the β axis is orthogonal to the α axis and the front side of the head 70 It is an axis in a direction orthogonal to the direction in which (face) faces. In this embodiment, when the head 70 is at the home position, the α axis is set to be parallel to the S axis and the β axis is set to be parallel to the U axis. In such a head 70, when the eyeball support portion is rotated around each of the α axis and the β axis, the tip (front) side of the eyeball portion 74 or the eye camera 54 is displaced, and the camera axis, that is, the line-of-sight direction is changed. Moved.

  In the α axis and β axis that control the displacement of the eye camera 54, “0 degree” is the home position. At this home position, the camera axis of the eye camera 54 is the head 70 as shown in FIG. The direction of the front side (face) is directed, and the line of sight is in the normal viewing state.

  FIG. 3 is a block diagram showing the internal configuration of the robot 12. As shown in FIG. 3, the robot 12 includes a microcomputer or a CPU 76 for overall control. The CPU 76 is connected to a memory 80, a motor control board 82, a sensor input / output board 84, and a voice through a bus 78. An input / output board 86 is connected.

  Although not shown, the memory 80 includes a ROM, an HDD, a RAM, and the like, and the control program and data for the robot 12 are stored in the ROM or the HDD in advance. The CPU 76 executes processing according to this program. Specifically, a plurality of programs (referred to as action modules) for controlling the body movement of the robot 12 are stored. For example, the body motion indicated by the behavior module includes “handshake”, “holding”, “pointing”, and so on. When the body motion indicated by the behavior module is “handshake”, when the behavior module is executed, the robot 12 presents the right hand forward, for example. Further, when the body motion indicated by the behavior module is “cuckling”, when the behavior module is executed, the robot 12 presents the hands forward with, for example, both hands open, and closes both hands when a human approaches. Further, when the body motion indicated by the behavior module is “pointing”, when the behavior module is executed, the robot 12 indicates a desired direction with, for example, the right hand (right arm) or the left hand (left arm). The RAM can be used as a working memory as well as a temporary storage memory.

  The motor control board 82 is composed of, for example, a DSP (Digital Signal Processor) and controls a motor for driving body parts such as the right arm, the left arm, the head, and the eyes. That is, the motor control board 82 receives control data from the CPU 76 and controls the angles of the X axis, the Y axis and the Z axis of the right shoulder joint 56R and the W axis angle of the right elbow joint 60R. The rotation angle of a total of four motors including one motor (collectively shown as “right arm motor” in FIG. 3) 88 is adjusted. The motor control board 82 includes a total of four motors including three motors that control the angles of the A, B, and C axes of the left shoulder joint 56L and one motor that controls the angle of the D axis of the left elbow joint 60L. The rotation angle of the motor (collectively shown as “left arm motor” in FIG. 3) 90 is adjusted. The motor control board 82 also adjusts the rotation angle of three motors 92 (collectively shown as “head motors” in FIG. 3) that control the angles of the S, T, and U axes of the neck joint 68. To do. The motor control board 82 also controls the waist motor 50 and the two motors 36 that drive the wheels 34 (collectively shown as “wheel motors” in FIG. 3). Further, the motor control board 82 adjusts the rotation angle of two motors 94 (collectively shown as “right eyeball motor” in FIG. 3) that control the angles of the α axis and β axis of the right eyeball portion 74R. In addition, the rotation angles of two motors 96 that collectively control the angles of the α axis and β axis of the left eyeball portion 74L (collectively shown as “left eyeball motor” in FIG. 3) 96 are adjusted.

  The above-described motors of this embodiment are stepping motors or pulse motors for simplifying the control except for the wheel motors 36, but may be direct-current motors as with the wheel motors 36.

  Similarly, the sensor input / output board 84 is also constituted by a DSP, and takes in signals from each sensor and camera and gives them to the CPU 76. That is, data relating to the reflection time from each of the ultrasonic distance sensors 42 is input to the CPU 76 through the sensor input / output board 84. The video signal from the omnidirectional camera 52 is input to the CPU 76 after being subjected to predetermined processing by the sensor input / output board 84 as necessary. Similarly, the video signal from the eye camera 54 is also supplied to the CPU 76. Further, a signal from the touch sensor 64 is given to the CPU 76 via the sensor input / output board 84.

  Synthetic voice data is given to the speaker 72 from the CPU 76 via the voice input / output board 86, and accordingly, voice or voice according to the data is outputted from the speaker 72. Further, the voice input from the microphone 24 is taken into the CPU 76 via the voice input / output board 86.

  Further, a communication LAN board 98 is connected to the CPU 76 through the bus 78. Similarly, the communication LAN board 98 is also configured by a DSP, and sends the transmission data given from the CPU 76 to the wireless communication apparatus 100 and causes the wireless communication apparatus 100 to transmit the transmission data. The communication LAN board 98 receives data via the wireless communication device 100 and provides the received data to the CPU 76.

  Further, the database 102 is connected to the CPU 76 through the bus 78. Although not shown, the database 102 stores an interaction parameter Θ described later together with the name or identification information (tag information, identification number) of the corresponding person (human 14 or the like). Corresponding to the identification information of the person, the height value estimated from the face image and whole body image of the person photographed by the eye camera 54 of the robot 12 is also stored. This is because, as will be described later, the initial value of the interaction parameter Θ is set according to the interaction partner.

  In this embodiment, the database 102 is provided inside the robot 12, but may be provided outside the robot 12 so as to be communicable.

  Returning to FIG. 1, the system 10 includes a motion capture system 20. A known motion capture system is applied as the motion capture system (three-dimensional motion measurement apparatus) 20. For example, an optical motion capture system of VICON (http://www.vicon.com/) can be used. Although illustration is omitted, the motion capture system 20 includes a computer such as a PC or WS, and the computer and the robot 12 are connected to each other by a wired or wireless LAN (not shown).

  Specifically, referring to FIG. 4, in the motion capture system 20, a plurality of (at least three) cameras 20 a having infrared irradiation functions have different directions with respect to the robot 12 and the human 14 existing in the space or environment. Placed in. A plurality of (four in this embodiment) infrared reflection markers 30 are attached to the robot 12 and the human 14. Specifically, as can be seen from FIG. 4, the infrared reflection marker 30 is attached to the top of the eye (the forehead) and the shoulder for both the robot 12 and the human 14. This is because in this embodiment, the position of the robot 12 and the human 14 (three-dimensional position) and the direction of the face (line of sight) are detected. However, in order to accurately detect the position and the direction of the face, the infrared reflection marker 30 may be attached to another part.

  The computer of the motion capture system 20 acquires image data from the camera 20a at, for example, 60 Hz (60 frames per second), and performs image processing on the image data, so that 2 of each marker 30 in all image data at the time of measurement is obtained. Extract dimension position. Then, the computer calculates the three-dimensional position of each marker 30 in the real space based on the two-dimensional position of each marker 30 in the image data, and also calculates the direction of the face of the robot 12 and the human 14. Next, the computer transmits the calculated three-dimensional position coordinate data (position data) and face direction data to the robot 12 in response to a request from the robot 12 (CPU 76).

  The robot 12 acquires coordinate data and direction data transmitted from the motion capture system 20 and acquires a three-dimensional position of itself and the human 14. Then, the robot 12 detects (calculates) the position (distance) of the human 14 when the robot 12 is centered (origin) (robot coordinates). Further, the robot 12 determines whether the human 14 is looking at the face of the robot 12 based on the direction data.

  As described above, the robot 12 having such a configuration performs an interaction action using at least one of a body motion (gesture) and a voice (speech) when communicating with the human 14. For example, the robot 12 detects a gesture or utterance of the human 14 with respect to the robot 12 and determines such interaction behavior.

  Here, considering communication between people, adaptation to the other party is important for natural (comfortable) communication. For example, in order for people to spend comfortably, an appropriate personal space is required, which varies depending on the content of communication. It is also known that the frequency of matching the line of sight varies from person to person. Comfortable personal spaces and eye-gaze frequency vary from person to person, but people are comfortable with each other according to their communication partners. For example, if you feel that the other person is too close, you will move away a little, and if you feel you are staring too much, you will bend your line of sight. People are unconsciously making these adaptations.

  Therefore, when the robot 12 and the human 14 interact (communicate), it is necessary for the robot 12 to secure an appropriate personal space or to adjust the line of sight to the individual according to the opponent.

  In addition, a study analyzing body movements in human-robot interaction (T. Kanda, H. Ishiguro, M. Imai, and T. Ono. Body Movement Analysis of Human-Robot Interaction. In Int. Joint Conference on Artificial Intelligence (IJCAI 2003), pp.177-182, 2003), subjects who have a good impression of robot behavior tend to turn their faces toward the robot, and there is a tendency that the moving distance during the interaction is also short. Yes. It's also natural to think of your face as a partner when your partner's movement is slow and boring, or when it's too fast to understand.

  From the above, it is assumed that in the interaction with the robot 12, the pleasure function of the person unconsciously appears in the distance traveled and the time to face the robot (either one is acceptable) (see FIG. 5). Designed. Here, as the parameter (interaction parameter) Θ for the interaction behavior of the robot 12, three kinds of interpersonal distances (intimacy distance, individual distance, social distance), and the time for which the camera (eye camera 54) is directed toward the human face. Is the length of time, the delay time from speech to motion playback, and the speed of motion. The robot 12 learns the parameter Θ by policy gradient reinforcement learning (PGRL) so as to maximize the reward obtained by calculating the reward function, and the interaction partner (here, human 14) Adapt to individuals. Since it is impossible to directly obtain what parameter Θ is appropriate, unsupervised learning is necessary as a learning method. This is because, for example, the interpersonal distance (personal space or the like) that is appropriate for each person is different. In addition, policy gradient-type reinforcement learning is used because it is important to have a fast convergence in order to learn in human interaction.

  The reward function is processed in software by the CPU 76 of the robot 12. Its functional block diagram is shown in FIG. Referring to FIG. 5, reward function 200 includes input terminals P1 and P2. The position data input from the motion capture system 20 is input as it is to the input terminals P1 and P2. However, as will be described later, every time one interaction action is executed, the calculation by the reward function 200 is executed, so that the input terminals P1 and P2 can be obtained during the execution of one interaction action. In addition, position data according to time changes is input.

  The position data input to the input terminal P1 is noise-removed by the filter unit 202. For example, the filter unit 202 is a 5 Hz LPF, and removes a high frequency component included in the position data. This is because fine movements of the human body 14 are not included in the movement distance. The position data from which the high frequency component has been removed is integrated by the integrating unit 204. That is, the moving distance of the human 14 is calculated. Then, the output of the integration unit 204 is normalized and weighted by the normalization / weighting unit 206 and is inverted and input to the adder 208. This is because, as described above, the movement amount of the human 14 during the interaction is considered to feel the interaction unpleasant and is a negative factor as a reward.

  On the other hand, the position data input to the input terminal P <b> 2 is given to the neck angle calculation unit 210, and the neck angle of the human 14 relative to the robot 12 is calculated by the neck angle calculation unit 210. Strictly speaking, the orientation of the face of the human 14 relative to the face of the robot 12 is calculated. When the neck angle of the human 14 is calculated, the threshold processing unit 212 determines whether the angle is equal to or less than a predetermined angle (10 ° in this embodiment). That is, it is determined whether or not the human 14 is looking at the face of the robot 12. Here, as shown in FIG. 6, when the robot 12 and the human 14 are facing each other, the angle formed by the face direction of the human 14 with respect to a straight line (line segment) connecting the robot 12 and the human 14 is 10. If it is less than 0 °, it is determined that the human 14 is looking at the face of the robot 12. However, when it is strictly determined whether or not the human 14 is looking at the face of the robot 12, it is considered necessary to detect the direction of the line of sight of the human 14 as well. In the threshold processing unit 212, when the neck angle calculated by the neck angle calculating unit 210 is 10 ° or less, the threshold processing unit 212 adds the time. That is, the total time during which the human 14 is looking at the face of the robot 12 during the interaction is calculated. Then, the normalization / weighting unit 214 performs normalization and weighting on the output of the threshold processing unit 212 and inputs the output to the adder 208 as it is. This is because, as described above, the time during which the human 14 looks at the face of the robot 12 during the interaction is thought to feel that the interaction is pleasant, and the reward is a positive factor. Then, the result of the adder 208 is given to the reward acquisition unit 216.

  In this embodiment, the weights in the normalization / weighting units 206 and 214 are set to 1: 1 for simplicity. However, since it is possible to know the reward, that is, the comfort during the interaction, based on either the interpersonal distance or the time to face the robot, it may be weighted, for example, 1 to 0 or 0 to 1. .

  In this embodiment, the comfort of interaction is known based on the interpersonal distance and the time to face the robot. However, the present invention is not limited to this. For example, it is possible to know the comfort of interaction based on the level of human footsteps, whether or not a human is so-called poor, or facial expressions (laughter (soft), hard (hard)). For example, human footsteps are related to annoyance, and if footsteps are low, it is not frustrating and interaction is comfortable, and conversely, if footsteps are high, it is frustrating and interaction is uncomfortable. I can say that. However, human footsteps can be detected by a sound level meter. Further, whether or not a human is poverty and whether or not the human facial expression can be detected by using an image recognition technique.

  Such calculation by the reward function 200 is executed every time the robot 12 performs the interaction action in the interaction. Then, the interaction parameter Θ of the robot 12 is obtained by reinforcement learning so that the human 14 feels pleasant.

  Here, in reinforcement learning represented by Q-learning, in order to learn the optimal behavior (policy or policy), a wide range of space is searched as much as possible, and every policy is tried. Therefore, the learning result can be optimized globally, but the search takes a long time. On the other hand, in policy gradient type reinforcement learning (or reinforcement learning by the policy gradient method), a local optimal solution is obtained by correcting the current policy in a direction in which a reward can be obtained. Since the policy is changed directly from the reward, there is a feature that there is little delay in reward propagation and a short learning time. In this embodiment, policy gradient reinforcement learning of an open loop system in which the interaction is not changed by the sensor after the start of the interaction is adopted.

Specifically, the entire process is executed according to the flowcharts shown in FIGS. 7 and 8, reinforcement learning is executed therein, and the interaction parameter Θ is updated. Here, the algorithm of the policy gradient type reinforcement learning in this embodiment will be briefly described. In the learning, first, the current policy, that is, T policies θ _{ij in} which the interaction parameter Θ is slightly changed is prepared. The policy θ _ij is generated by randomly adding any one of ε _j , 0, and −ε _j to each component θ _j of the interaction parameter Θ. However, the variable step size ε _j may be different for each parameter (component of the interaction parameter Θ) θ _j .

Next, the interaction is performed T times according to each policy R _i to obtain a reward. After performing the interaction for all T policies θ _{ij, the} gradient A with respect to the interaction parameter Θ of the reward function 200 is approximately obtained. For each parameter θ _j , an average reward when ε _j is added, an average reward when 0 is added, and an average reward when −ε _j is added are obtained.

When the average reward when 0 is added is the largest, the gradient A for each parameter θ _j is 0. On the other hand, when the average reward when 0 is added is not the largest, the gradient A for each parameter θ _j is the difference between the average reward when ε is added and the average reward when −ε is added. To do. After obtaining the gradient A, the gradient A is normalized and multiplied by η, each component is weighted with ε _j , and the interaction parameter Θ is updated. The T times of interaction and the update of the interaction parameter Θ are one step. By repeating this, the reward is maximized, that is, the interaction parameter Θ that can execute the interaction behavior that the human 14 feels comfortable is updated.

  As shown in FIG. 7, when starting the entire process, the CPU 76 determines whether or not the interaction partner (for example, the human 14) is a person who has interacted in the past in step S1. Although illustration is omitted, for example, a person corresponding to tag identification information (tag information or number) is referred to the database 102 by attaching a tag to a human 14 and providing a tag receiver in the robot 12. It is determined whether the interaction parameter Θ for is stored. Here, when the interaction parameter Θ for the person is stored, it can be determined that the user has interacted in the past. On the other hand, when the interaction parameter Θ for the person is not stored, it can be determined that no interaction has occurred in the past.

  If “YES” in the step S1, that is, if there has been an interaction in the past, in a step S3, the interaction parameter Θ is read from the database 102 and substituted into the variable Θ, and the process proceeds to a step S11. On the other hand, if “NO” in the step S1, that is, if there is no interaction in the past, it is determined whether or not there is an experience of interacting with a person similar to the partner of the interaction in a step S5.

  In this embodiment, whether or not the person is similar is determined based on the person's face (mainly shape) and height. The face and height of the person are determined (estimated) based on the images (face image and whole body image) taken by the eye camera 54 provided in the robot 12. As described above, since the face image and the estimated height of the interacting person are stored in the database 102 in correspondence with the tag information, the face image of the currently interacting person and the estimated height are compared. Thus, it can be determined whether or not a similar person exists. In other words, you can determine whether you have any experience interacting with similar people.

  If “YES” in the step S5, that is, if there is an experience of interacting with a similar person, in a step S7, the interaction parameter Θ of the similar person is read from the database 102 and substituted into the variable Θ, and the process proceeds to the step S11. . On the other hand, if “NO” in the step S5, that is, if there is no experience of interacting with a similar person, in a step S9, the average interaction parameter Θ is read from the database 102 and substituted into the variable Θ, and the process proceeds to the step S11. move on. Here, the average interaction parameter Θ is, for example, the average value of all the interaction parameters Θ stored in the database 102.

  Although not shown in the figure, when the entire process is executed for the first time, the interaction parameter Θ is not stored in the database 102, and an initial value is set (input) by the user.

In step S11, the number of interactions i is initialized (i = 1). In the subsequent step S13, the process of determining the interaction parameter Θ _{i to be} tested this time based on the variable Θ (see FIG. 9) is executed. Since this determination process will be described later in detail, the detailed description thereof is omitted here. Next, in step S15, an interaction action is executed. However, here, any one of the plurality of interaction actions prepared in advance is selectively executed according to random (predetermined rule) or the behavior of the human 14.

Then, in step S17, it calculates an evaluation of the interaction, into a variable R _i. Here, the evaluation of the interaction is a reward obtained according to the above-described reward function 200 (FIG. 5). As shown in FIG. 8, in the next step S19, 1 is added to the number of interactions i (i = i + 1). In step S21, it is determined whether or not the number of times of interaction i has exceeded a predetermined number of times T (for example, 10). If “NO” in the step S21, that is, if the number of times of interaction i is equal to or less than the predetermined number, the process returns to the step S13 shown in FIG. On the other hand, if “YES” in the step S21, that is, if the number of times of interaction i exceeds the predetermined number T, an update process of the interaction parameter Θ (see FIG. 10) is executed in a step S23, and in step S25, Determine if the interaction is over. Here, for example, it is determined whether an instruction to end the interaction is input or whether a certain time has passed.

  If “NO” in the step S25, that is, if the interaction is not ended, the process returns to the step S11 shown in FIG. On the other hand, if “YES” in the step S25, that is, if the interaction is ended, the updated variable Θ is registered (updated) in the database 102 as an interaction parameter Θ corresponding to the interaction partner, and the entire process is performed. finish.

FIG. 9 is a flowchart showing the determination process of the interaction parameter Θ _{i to be} tried this time in step S13 shown in FIG. Referring to FIG. 9, when starting the determination process of interaction parameter Θ _{i to be} tested this time, CPU 76 sets an initial value to variable j in step S41 (j = 1). In the subsequent step S43, one is randomly selected from 0, ε _j , and −ε _j and is substituted into the variable Δ. In the next step S45, the j-th component θ _ij of the interaction parameter Θ _{i to be} tested this time is calculated (θ _ij = θ _j + Δ). Subsequently, in step S47, the variable j is incremented (j = j + 1). In step S49, it is determined whether or not the variable j exceeds the size (number of all components θ _j ) n of the interaction parameter Θ (interaction parameter vector). If “NO” in the step S49, that is, if the variable j is equal to or less than the magnitude n of the interaction parameter Θ, the process returns to the step S43 as it is. On the other hand, if "YES" in step S49, the words if the variable j exceeds the size n of the interaction parameter theta, it is determined that determine the interaction parameter theta _i try this, the interaction parameter theta _i try this Return the decision process.

FIG. 10 is a flowchart showing the update processing of the interaction parameter Θ in step S23 shown in FIG. Referring to FIG. 10, when starting the update process of the interaction parameter Θ, the CPU 76 sets an initial value to the variable j in step S61 (j = 1). In step S63, the interactions parameter theta _i tried this, the average compensation R0 when the theta _ij was theta _j, the average compensation R1 in the case where the the θ _ij θ _j + ε _j, the θ _ij θ _j -ε _j The average reward R2 is calculated for each. Here, θ _j is the j-th component of the interaction parameter (vector) Θ, θ _ij is the j-th component of the interaction parameter Θ _i , and ε _j is a value that varies the j-th component of the interaction parameter Θ.

Next, in step S65, it is determined whether R0> R1 and R0> R2 using R0, R1, and R2 calculated in step S63. If “YES” in the step S65, that is, if R0> R1 and R0> R2, the j-th component a _j of the gradient A is set to 0 (a _j = 0) in a step S65, Proceed to step S71. On the other hand, if “NO” in the step S65, that is, if at least one of R0 ≦ R1 and R0 ≦ R2 is satisfied, the average reward R1 and the average reward R2 are added to the j-th component a _j of the gradient A in the step S69. The difference (a _j = R1−R2) is set, and the process proceeds to step S71.

In step S71, the variable j is incremented. In step S73, it is determined whether or not the variable j exceeds the size n of the interaction parameter Θ. If “NO” in the step S71, that is, if the variable j exceeds the magnitude n of the interaction parameter Θ, the process returns to the step S63. On the other hand, if “YES” in the step S73, that is, if the variable j is equal to or smaller than the magnitude n of the interaction parameter Θ, the gradient A is normalized (A = A / | A |) in a step S75. In the subsequent step S77, the j-th component a _j of the gradient A is updated (a _j = a _j × ε _j × η). However, η is a scalar and is a parameter that determines the magnitude of the update as a whole. In step S79, the interaction parameter Θ is updated (Θ = Θ + A), and the interaction Θ update processing is returned.

The robot 12 having such a configuration is actually interacted with a human (subject) to update the interaction parameter Θ, and the reinforcement learning is performed from the impression (pleasant / uncomfortable) the subject received from the interaction with the robot 12. The fitness of parameter Θ by is verified by experiment. As described above, the interaction parameter Θ includes three types of interpersonal distances (intimacy distance, individual distance, social distance), the length of time for which the eye camera 54 is directed toward the face of the person (gazing time), and motion playback from speech The delay time until the start of the interaction action and the motion playback speed. All motions (interaction actions) prepared for the robot 12 are classified according to interpersonal distances (see FIG. 12), and the same distances are used for the interaction actions included in the same classification. The interaction parameter Θ related to one motion is four elements (parameter θ _j ) of distance, gaze time, delay time, and reproduction speed. If the adaptive parameter θ _j is increased, it takes time for learning, so it is considered that there is a great influence on the interaction. Also, lesser as possible parameter theta _j is because implementation is easier, it was decided to select the parameter theta _j as described above.

  The distance between the person and the robot 12 (interpersonal distance) was the horizontal distance between the foreheads. The robot 12 looks at a person's face and turns in the other direction with 5 seconds as one cycle. The gaze time was the ratio of the time to see this person's face to 5 seconds. Here, the reason that 5 seconds is set as one cycle is that it is adjusted to the gaze cycle in the interaction between people. The delay time is, for example, the time from when the robot 12 speaks “shake a handshake” until the motion of raising the hand is reproduced. The playback speed is set to 1 when the motion is created.

  The experiment was performed in a laboratory having the motion capture system 20 within a predetermined range (4.5 × 3.5 (m)) in the center where motion capture can be performed with high accuracy. As shown in FIG. 11, a motion capture system 20 including twelve cameras 20a is provided. However, in FIG. 11, for the sake of simplicity, a computer other than the camera 20a is omitted. With such a configuration, there is a measurement accuracy of about 1 (mm) in the experimental area. As described above, the marker 30 is attached to the subject (human 14) and the forehead and shoulders of the robot 12, and the position and direction of each forehead are obtained from the marker 30. The position and direction obtained by the motion capture were sent to the robot 12 via a network such as a LAN, and used for determining the operation of the robot 12 and calculating the reward function 200. During the experiment, the time delay due to communication was within 0.1 seconds, and the delay due to this communication could be ignored.

  FIG. 12 shows a first table regarding the behavior (interaction behavior) of the robot 12 prepared for the experiment. As can be seen with reference to FIG. 12, the interaction behavior is Hug, Shake hands, where did you come from? (Ask where person comes from), is Robotbie (the name of the robot 12) cute? (Ask if robot is cute), Ask person to touch robot, Janken (Play paper-scissors-stone), Play pointing game, Perform arm-swinging exercise, Self There are 10 ways of introduction (Hold “thank you” monologue) and just looking. These interactions were classified by preliminary experiments into three interpersonal distances: an intimate distance, a personal distance, and a social distance. In the preliminary classification experiment, eight subjects were collected, the position of the robot 12 was fixed, and each subject was moved to a distance considered suitable for each interaction, and the distance was measured. Although there were some distance differences between subjects, the variance was small and did not affect the classification.

  Next, an experimental procedure will be described. At the start of the experiment, the robot 12 exists in the center of the measurement area of the motion capture system 20, and the subject was asked to interact with the robot 12 in a relaxed and natural manner from the state of standing in front of the robot 12. Except for requesting to be within the measurement range of the motion capture system 20, the subject is not requested to interact.

In the experiment, an interaction of about 30 minutes was performed between the robot 12 and each subject. During the 30 minutes, the ten interaction actions described above were randomly executed. Although a detailed description is omitted, the robot 12 is accompanied by movements of its arms and heads when performing any interaction action. That is, it involves physical movement. For example, in the hug, when the robot 12 says “Please hold me” and spreads the arm, in response, when a person (subject) stands at an appropriate position (distance) in front of the robot 12, Then hug the person with his arms. After 30 minutes of interaction, reinforcement learning as described above was performed. As described above, the reward function 200 is calculated every time the robot 12 finishes one interaction action (about 10 seconds). Each component (parameter) θ _j of the interaction parameter Θ is changed little by little, and when the interaction of T times (10 times in this embodiment) is completed, the fluctuation direction (gradient A) of the interaction parameter Θ is determined from the reward, Updated the interaction parameter Θ. FIG. 13 shows a second table showing each initial value and step size corresponding to each parameter θ _j . Specifically, in the parameter “intimate distance”, the initial value is 50 (cm) and the step size is 15 (cm). The parameter “individual distance” has an initial value of 80 (cm) and a step size of 15 (cm). In the parameter “social distance”, the initial value is 100 (cm) and the step size is 15 (cm). In the parameter “gazing time”, the initial value is 0.7 and the step size is 0.1. In the parameter “delay time”, the initial value is 0.17 (s) and the step size is 0.3 (s). In the parameter “reproduction speed”, the initial value is 1.0 and the step size is 0.1.

  After the interaction, the subjects were asked about the movement and interaction of the robot 12, the impression of how to adjust the movement, distance, and line of sight of the robot 12, and how they changed during the experiment, and the individual distance was measured. . The subject was allowed to stand at a suitable position on the front of the robot 12 performing the motion for the intimate distance, the individual distance, and the social distance, and the distance was measured by the motion capture system 20. Here, the gaze time is 0.75, the delay time is 0.3 (s), the playback speed is 1.0, the interaction “catch” is used for the intimate distance, the interaction “handshake” is used for the individual distance, and the social distance is determined. Used the “Hold“ thank you ”monologue” interaction. Further, the position at which the subject felt the distance was not appropriate was measured when the robot 12 was brought closer to the appropriate distance and when the robot 12 was moved away.

In addition, the interaction motion of the robot 12 when only one parameter θ _j is changed in three ways, low, medium, and high and the other parameter θ _j is fixed to the average value of all subjects is shown to the subject and feels appropriate. We had you choose thing. For the measurement of the gaze time and the reproduction speed, a “thank you” motion was used, and the distance from the person was set to 1.0 (m). For the measurement of the delay time, a “cuckling” motion was used, and with respect to the distance, the movement of the robot 12 was stopped, and the subject was allowed to stand in an appropriate position. This is because the intimate distance has a large individual difference. Some subjects felt that multiple values were appropriate, and some felt that intermediate values were appropriate.

  Such an experiment was conducted on 15 subjects. Except for one subject, all the subjects were Japanese, and all of them were able to hear the utterance of the robot 12. Subjects were 20 to 35 years old, and many were 20 to 25 years old. Of the subjects, 6 were female and the rest were male. However, some of the subjects knew about the robot 12.

  Three of the subjects did not behave as we expected. Specifically, whether or not the interaction of the robot 12 is appropriate, even if it is not, change the direction of the face or change the standing position to describe the impression to the robot 12 or the experimenter. It was only appearing on the face. Such a subject does not apply to the assumed interaction evaluation model, and the system 10 (reinforcement learning process) does not operate correctly. Therefore, in the experimental results described below, the results of these subjects (three subjects) are excluded. For many other subjects, the interaction parameter Θ converged to an appropriate value in 15 to 20 minutes (about 10 PGRL parameter updates).

  14 (A), (B) and (C) show the values obtained as a result of adaptation and the values determined by the subjects as appropriate for the distances (intimacy distance, individual distance, social distance) of the 12 subjects. Show. Regarding the distance, the average of the last quarter period (about 7 and a half minutes) of all the interactions is shown. This is because PGRL always searches for the optimum local value. 14 (A) to 14 (C), “*” marks are the results of adaptation, the vertical bar indicates the subject, the short bar among the horizontal bars indicates the allowable limit (allowable range), and the long among the horizontal bars. The bar indicates the optimum value (optimum value).

FIG. 15A, FIG. 15B, and FIG. 15C show the results for gaze time (motion meeting ratio), delay time (waiting time), and motion playback speed (motion speed). 15 (A) to 15 (C), “◯” indicates a value determined by the subject as appropriate, and “*” indicates an adaptation result (an average of the last quarter of all interactions). . However, for subjects whose midpoint between the two values is appropriate, a “▽” mark is marked in the middle. From FIG. 14 (A) to FIG. 14 (C) and FIG. 15 (A) to FIG. 15 (C), it can be said that the degree of coincidence with the judgment of the subject greatly varies depending on the parameter θ _j . This is because the importance of each parameter θ _{j to} the interaction is different, and it converges from the parameter θ _{j having} a large contribution to the reward, and the convergence of the parameter θ _{j having} a wide allowable range is delayed.

Also, the impression of the subject who was able to adapt well to the robot 12 tended to not include much change in the parameter θ _j . This suggests that when natural adaptation is performed, the adaptation of the parameter θ _j may not be recognized.

FIG. 16 shows a third table in which the variances from the optimum values (optimum values) of the parameters θ _j are averaged for 12 subjects. The variance was calculated for the last quarter period for all interactions. This is because the influence of the variation of the parameter θ _j during the last quarter period is included in the variance. Each parameter θ _j is normalized so that the update step size is 1. Note that in the third table of FIG. 16, the initial value variance is shown at the right end for reference. As can be seen from the third table, the step size is 1.1 times or less excluding personal distance and social distance. The allowable range was 3 times the step size for personal distance and 5 times for social distance. With regard to social distance, the adaptation results were within acceptable limits, with the exception of one subject. Therefore, it can be said that each parameter θ _j has converged to an appropriate value due to adaptation based on PGRL. In order to further reduce the error, it is considered necessary to reduce the step size or gradually decrease the adaptation.

Moreover, as shown in the third table, the initial value is not so far from the optimum value, but about 10 adaptations are required for convergence to the optimum value. However, as shown in FIGS. 14 (A) to 14 (C) and FIGS. 15 (A) to 15 (C), depending on the subject, the parameter θ that does not converge to the optimum value even after 15 to 20 adaptations. There was _j . Also, in consideration of the reward function 200, when a person consistently behaves in accordance with the current interaction parameter Θ, convergence was achieved with 4 to 5 adaptations. In such a case, the simulation may converge to the optimum value 3 to 4 times. Therefore, one possible reason for the increase in the number of adaptations required before convergence is that the human movement is not constant every time. In the following, the subjects are divided into four groups according to the adaptation results, and further detailed experimental results are described.

First, the case where the adaptation to the optimal value is good and the impression of the subject is good will be described. The robot 12 was smoothly adapted to three subjects (subjects 2, 10, 12). Convergence of each parameter θ _{j to} the optimum value was observed, and these three subjects stated that they felt “interaction is appropriate”. That is, it can be said that the interaction parameter Θ when the robot 12 interacted was appropriate. The common point seen by these three subjects is that they enjoyed interaction with the robot 12, were not conscious of being the robot 12, and were in contact with the robot 12 as in the case of humans.

17A to 17F show changes in the parameters θ _j for the subject 10. The subject 10 states that “the behavior (interaction behavior) of the robot 12 has been improved rapidly”. As can be seen from FIGS. 17A to 17F, each parameter θ _j converges sufficiently close to the optimum value even if the individual distance is slightly away from the optimum value, and agrees with the impression of the subject 10. It can be said that this is the result. In addition, it can be seen that the subject has a wide allowable range with respect to the motion start timing (delay time), and the adaptation result is also within the allowable range.

Next, a case will be described in which some of the parameters θ _j do not converge to the optimum value but the impression of the subject is good. Two subjects (subjects 5 and 8) replied that “the impression of the operation of the robot 12 was good”, but some of the parameters θ _j were greatly deviated from the optimum values. FIG. 18A to FIG. 18F show changes in each parameter θ _j with respect to the subject 5. As can be seen from FIGS. 18 (A) to 18 (F), the three individual distances converge to the optimum value, and the motion start timing (delay time) has a wide allowable range, and is learned appropriately. However, the other two parameters θ _j (gaze time, reproduction speed) are far from the optimum values. However, test subject 5 stated that “the gaze time and playback speed were also appropriate”. This is considered to be due to the fact that the allowable range of the parameter θ _j (especially the gaze time, the reproduction time) of the subject 5 was actually wide. Moreover, the test subject 5 performed the behavior (behavior) which is not seen by other test subjects. Specifically, the subject 5 touched each part of the robot 12 even in the interaction spoken by the robot 12 classified as a social distance. As a result, the social distance is considerably shorter than other subjects. Moreover, although such behavior was not anticipated, it can be said that the robot 12 was able to satisfy the subject 5 with the same reward function 200 (see FIG. 5) as other subjects.

Subsequently, the case where the subject has converged (adapted) to the optimum value but was unhappy with some of the adaptations will be described. FIGS. 19A to 19F show adaptation of each parameter θ _j of the robot 12 to the subject 7. Each parameter θ _j appears to have converged well to the optimal value, but subject 7 stated that it was too close in terms of distance. However, as can be seen from FIGS. 19A to 19C, it converges near the farthest distance allowed by the subject except for the social distance. In addition, the subject 7 stated that the initial impression was “feeling hesitated”, but the impression that gradually became “active” was received. The impression about the distance was not good because the robot 12 increased the motion playback speed according to the preference of the subject 7 due to the adaptation by PGRL, but the optimal distance was measured by setting the motion playback speed to “1”. It is thought that it went. In this case, the optimal distance for the subject 7 may be longer.

Next, a case will be described in which some parameters θ _j do not converge to the optimum values and the subject is also dissatisfied with some adaptations. For five subjects (subjects 1, 3, 4, 6, and 11), some parameters θ _j did not converge to the optimum values, and each subject also complained about the adaptation results of those parameters θ _j . 20A to 20F show how the parameter θ _j changes with respect to the subject 1. In this experiment, the experiment time is 21 minutes shorter than other subjects due to trouble.

As can be seen with reference to FIGS. 20A to 20C, the individual distance and the social distance converge to the optimum values, but the intimate distance is not within the allowable range. This is because the intimate distance taken by the robot 12 with respect to the subject 1 is outside the allowable range, and the subject 1 behaved almost the same for any distance, so the parameter θ _j is changed so as to approach the optimum distance. It is thought that it was not possible. Subject 1 also states that the intimate distance was inappropriate. As shown in FIG. 20D, the frequency (gaze time) for matching the line of sight is about 90% due to adaptation. Since the test subject 1 stated that 100% is the best and about 75% to 50% may be sufficient for the gaze time, it can be said that the subject 1 has converged to an appropriate value. Since the test subject 1 stated that he / she did not care much about the delay time and stated that any value could be used for the playback speed, it can be said that the subject 1 was well adapted except for the intimate distance.

21A to 21F show how the parameter θ _j changes with respect to the subject 3. Subject 3 pointed out that "individual distance was inappropriate in the first half of the experiment". This is consistent with the individual distance graph in FIG. With regard to the line-of-sight frequency (gaze time), as shown in FIG. 21D, the adaptation result was about 75% and the optimum value was 100%, but the subject 3 was “sufficiently satisfied”. Stated. The adaptation result of the intimate distance is a lower limit of 15 (cm) provided so that the robot 12 is not in contact with a person for safety. Since the subject 3 has a wide allowable range with respect to timing (delay time), the adaptation result can be said to be appropriate as shown in FIG. However, as shown in FIG. 21F, in the test subject 3, the adaptation result of the reproduction speed is far from the optimum value. Subject 3 also states that the playback speed was inappropriate. This is probably because the subject 3 has a tendency to stare at the robot 12 when the motion is too fast, and the reward is accidentally increased because the robot 12 increases the reproduction speed too much.

Finally, the case where the adaptation was not successful will be described. 22A to 22F show how the parameter θ _j changes with respect to the subject 9. As can be seen from FIGS. 22A to 22F, the parameter θ _j other than the personal distance and the line-of-sight frequency (gaze time) is far from the optimum value. Subject 9 states that “I was hated by Robbie (robot 12) and Robbie had no impression of trying to act normally. Although it was not observed that there was a big problem from the state during the experiment, there is a possibility that the subject 9 did not react to the robot 12 in a straightforward manner.

  From the above, it was possible to confirm the personal adaptation of the robot 12. Moreover, the test subject has obtained the impression that the behavior of the adapted robot 12 seemed natural. Personal adaptation is one of the important elements for realizing the robot 12 that can interact with people more naturally, and it can be said that the method in this embodiment is one step.

  According to this embodiment, since the behavior of the robot can be matched to the interaction partner by RPGL, natural communication can be executed so that humans can communicate with each other.

  In these embodiments, the motion capture system is used to detect the three-dimensional position of the robot and the human and the direction of the line of sight of the robot and the human, but this may be detected using other sensors. Is possible. For example, if a stereo camera (image sensor) or an ultrasonic sensor is mounted on the robot, the distance from the human can be measured by the output of the ultrasonic sensor or the parallax from the stereo camera. Further, the orientation of the human face can be detected by pattern matching of the orientation of the face from the camera image. However, an ultrasonic distance sensor mounted on the robot can be used as the ultrasonic sensor.

  In this embodiment, the interaction parameter is updated by the policy gradient reinforcement learning. However, the interaction parameter is not limited to this, and can be updated by another algorithm. For example, the interaction parameters can be updated by a genetic algorithm.

Further, in this embodiment, in the determination process of the interaction parameter Θ _{i to be} tried this time (FIG. 9), Δ (0, ε _j , −ε _j ) is used when determining the i-th parameter θ _j to be tried. However, the present invention is not limited to this, and a random number can be used. However, when a random number is used, it is necessary to change the update processing of the interaction parameter Θ (FIG. 10) accordingly.

  Furthermore, in this embodiment, the intimate distance, the individual distance, and the social distance are used as the interpersonal distance, but should not be limited to this. For example, there is a possibility that a better result may be obtained by adjusting other distances such as “shake distance” for handshake and “greeting distance” for greeting.

FIG. 1 is an illustrative view showing one example of a communication robot system of the present invention. FIG. 2 is an illustrative view for explaining the appearance of the robot shown in FIG. 1 embodiment. FIG. 3 is an illustrative view showing an electrical configuration of the robot shown in FIGS. 1 and 2. FIG. 4 is an illustrative view showing a mounting state of a marker robot and a human being detected by a motion capture system and a camera. FIG. 5 is a functional block diagram of a reward function calculated by the CPU of the robot shown in FIGS. FIG. 6 is an illustrative view for explaining an angle when it is determined in the reward function shown in FIG. 5 that a human is facing the robot. FIG. 7 is a flowchart showing a part of the reinforcement learning process of the CPU shown in FIG. FIG. 8 is another part of the CPU reinforcement learning process shown in FIG. 3, and is a flowchart subsequent to FIG. FIG. 9 is a flowchart showing the Θ _i determination process of the CPU shown in FIG. FIG. 10 is a flowchart showing the Θ update process of the CPU shown in FIG. FIG. 11 is an illustrative view for explaining an experimental environment to which the system shown in FIG. 1 is applied. FIG. 12 is an illustrative view showing a first table showing the behavior of the robot with respect to the interpersonal distance. FIG. 13 is an illustrative view showing a second table showing initial values and step sizes of parameters in the experiment. FIG. 14 is an illustrative view showing a value obtained as a result of adaptation and a value determined to be appropriate by the subject with respect to a distance of 12 subjects. FIG. 15 is an illustrative view showing the results of adaptation for the gaze time, delay time, and motion playback speed of 12 subjects. FIG. 16 is an illustrative view showing a third table showing dispersion of values and initial values obtained by averaging dispersion of parameters from the optimum place for 12 subjects. FIG. 17 is a graph showing how the parameters of the subject 10 change. FIG. 18 is a graph showing how the parameters of the subject 5 change. FIG. 19 is a graph showing how the parameters of the subject 7 change. FIG. 20 is a graph showing how the parameters of the subject 1 change. FIG. 21 is a graph showing how the parameters of the subject 3 change. FIG. 22 is a graph showing how the parameters of the subject 9 change.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Communication robot system 12 ... Communication robot 20 ... Motion capture system 38 ... Collision sensor 42 ... Ultrasonic distance sensor 52 ... Omnidirectional camera 54 ... Eye camera 64 ... Touch sensor 76 ... CPU
DESCRIPTION OF SYMBOLS 80 ... Memory 82 ... Motor control board 84 ... Sensor input / output board 86 ... Voice input / output board 88-96 ... Motor 98 ... Communication LAN board 100 ... Wireless communication apparatus 102 ... Database

Claims (6)

A communication robot that interacts with humans,
Parameter setting means to set parameters for interaction,
Interaction executing means for executing an interaction including at least one of speech and physical movement according to the parameter set by the parameter setting means;
A communication robot, comprising: an appropriateness detection unit that detects an appropriateness of the parameter during an interaction; and an optimization unit that optimizes the appropriateness detected by the appropriateness detection unit. A moving distance detecting means for detecting a moving distance of the human during the interaction; and a time detecting means for detecting a time during which the human looks at the face of the communication robot itself during the interaction,
The appropriateness detection means detects the appropriateness of the parameter based on the detection result of at least one of the action distance detection means and the time detection means when the interaction is executed with the parameter set by the parameter setting means. The communication robot according to claim 1.   The parameter includes at least one of an interpersonal distance in the interaction with the human, a length of time for directing his / her face to the human face, a delay time from an utterance to the start of a physical motion, and a motion speed of the physical motion. The communication robot according to claim 1 or 2, further comprising:   The communication robot according to claim 3, wherein the interpersonal distance includes an intimate distance, an individual distance, and a social distance.   The communication robot according to claim 1, wherein the optimization unit includes a parameter update unit that updates the parameter. Parameter storage means for storing the parameter corresponding to the person, and human identification means for identifying the person at the start of an interaction,
The parameter setting means sets the parameter when the parameter corresponding to the person identified by the person identification means is stored in the parameter storage means, and the parameter corresponding to the person identified by the person identification means 6. The communication robot according to claim 1, wherein when the parameter is not stored by the parameter storage unit, an average value of all the parameters stored by the parameter storage unit is set.

Images