JP4517086B2 - Communication robot system and robot controller - Google Patents

Description

  The present invention relates to a communication robot system and a robot control device, and more particularly, to a communication robot system and a robot control device that control a communication lot that performs communication behavior with a human.

The present applicant has proposed a communication robot that interacts with a human, as represented by Patent Document 1.
Japanese Patent Laid-Open No. 2002-355783

  Using the communication robot of the background technology, record the human body motion during communication and the body motion of the communication robot in association with each other, and call and execute the behavior of any communication robot with the human body motion It is possible.

  However, it is not possible to simply associate human body movements with communication robot actions. This is because there are two possibilities for human body movement (behavior). One is an action with respect to the behavior of the communication robot (for example, when the communication robot asks for a handshake, the person also shakes and returns), and the other is an action for searching for the behavior of the communication robot (a human being the communication robot). (In order to shake hands, hand out to the communication robot.) Therefore, if the correspondence between human behavior and communication robot behavior is simply recorded, the two behaviors are mixed.

  Therefore, the main object of the present invention is to provide a novel communication robot system.

  Another object of the present invention is to provide a communication robot system capable of executing communication behavior according to human body movement.

The invention of claim 1 is a communication robot system comprising a communication robot and a control device for controlling the communication robot, wherein the communication robot takes a communication action spontaneously regardless of a human body motion or a human being Communication action execution means for executing communication action according to action information in an interrupt state in which communication action is caused due to the body movement of the person , including action information and state information indicating a normal state or an interrupt state when the communication action is executed transmitting means for transmitting the robot information to the control unit, and includes an interrupt instruction receiving means for receiving an interrupt instruction from the controller, the operation of detecting the operation information about the human body movement Distribution detection means, the robot information detecting means for detecting robot information transmitted from the communication robot, at the same time, the storage means stores an association with operation information and the robot information, the current operation detected by the operation information detecting means The comparison means for comparing the information with the past motion information stored in the storage means, and when there is past motion information indicating that the comparison result by the comparison means is at least similar, it is included in the robot information corresponding to the past motion information State determination means for determining whether the state information to be displayed indicates a normal state or an interrupt state, and when the state determination means determines that the normal state is indicated, past operation information at least similar to the current operation information Read action information included in corresponding robot information, or interrupt by state discrimination means When to show status is determined, the read action information included in the robot information corresponding to the temporally past operation information of the next previous operation information at least similar to the current operation information, the action for communication robot The communication robot system includes an interrupt instruction transmission unit that transmits an interrupt instruction for executing a communication action according to information.

According to a first aspect of the present invention, a communication robot system includes a communication robot and a control device that controls the communication robot. The communication robot includes communication action execution means, transmission means, and interrupt instruction reception means. The communication action execution means executes the communication action according to the action information (behavior module) in the normal state in which the communication action is spontaneously taken regardless of the human body movement or in the interruption state in which the communication action is taken due to the human body movement. To do. For example, a communication robot pushes a hand to a human (shakes a hand) regardless of the human body movement (normal state), and a communication robot pushes a hand to a human (shakes a hand) so that the human holds the hand. A distinction is made from behavior (interrupt state). The transmission means transmits the robot information including the behavior information and the state information of the communication robot indicating the normal state or the interrupt state when the communication behavior is executed to the control device. Further, the interrupt instruction receiving means receives an interrupt instruction from the control device. On the other hand, the control device includes motion information detection means, robot information detection means, storage means, comparison means, state determination means, and interrupt instruction transmission means. The operation information detection means detects the operation information about the human body operation for communication robot and communication. The robot information detecting means detects robot information transmitted from the communication robot. The storage means stores the operation information and the robot information in association with each other at the same time. That is, the correspondence between the human body motion and the robot communication behavior is stored. The comparison unit compares the current operation information with the past operation information stored in the storage unit. Whether the state information included in the robot information corresponding to the past motion information indicates a normal state or an interrupt state when there is past motion information indicating that the comparison result by the comparison unit is at least similar. Is determined. When it is determined that the normal state is indicated by the state determination unit , the interrupt instruction transmission unit reads the behavior information included in the robot information corresponding to the past movement information at least similar to the current movement information, or the state determination When it is determined that the interruption state is indicated by the means, the behavior information included in the robot information corresponding to the past motion information that is temporally next to the past motion information that is at least similar to the current motion information is read, and the communication robot An interruption instruction for execution of the communication action in accordance with the action information is transmitted.

  According to the first aspect of the present invention, since the correspondence relationship between the human body motion and the communication motion of the communication robot is stored, the communication robot is caused to execute the interrupting communication operation based on the current human body motion. The action of the communication robot can be extracted by human body movement. That is, it is possible to cause the communication robot to execute communication behaviors according to human body movements.

The invention of claim 2 is dependent on claim 1, and the interrupt instruction transmitting means does not transmit a new interrupt instruction when the communication robot is performing an interrupting communication action.

According to the second aspect of the present invention, the interrupt instruction transmitting means does not transmit a new interrupt instruction when the communication robot is performing an interrupting communication action. That is, interrupt instructions are not transmitted continuously.

According to the invention of claim 2 , since the interrupt instruction is not continuously transmitted, it is possible to avoid the inconvenience that the communication robot repeatedly executes the same communication action.

The invention of claim 3 is dependent on claim 1 or 2, and the interrupt instruction transmitting means does not transmit a new interrupt instruction when the read action information matches the action information instructed to be executed in the previous interrupt .

In the invention of claim 3 , the interrupt instruction transmitting means does not transmit a new interrupt instruction when the read action information matches the action information instructed to be executed by the previous interrupt. That is, no interruption instruction is performed for the same behavior information as the previous behavior information.

Oite to the invention of claim 3, similarly to the invention of claim 2, it is possible to communicate robot to avoid a disadvantage such as to repeatedly execute the same communication behavior.

The invention according to claim 4 is a communication that executes the communication action according to the action information in a normal state in which the communication action is spontaneously taken regardless of the human body movement or an interruption state in which the communication action is taken due to the human body movement. A robot control device for controlling a robot, comprising: motion information detection means for detecting motion information about a human body motion ; robot information detection means for detecting robot information transmitted from a communication robot; motion information at the same time; Storage means for storing robot information in association with each other, comparison means for comparing current motion information detected by the motion information detection means with past motion information stored in the storage means, and comparison results by the comparison means are at least similar When there is past operation information indicating When it is determined by the state determination means that the state information included in the robot information corresponding to the past motion information indicates a normal state or an interrupt state, and the state determination means indicates the normal state, When the behavior information included in the robot information corresponding to the past motion information at least similar to the motion information is read, or when it is determined by the state determination means to indicate the interrupt state, the past at least similar to the current motion information An action instruction transmitting means for reading action information included in the robot information corresponding to the next past action information of the action information of the action information, and sending an instruction to interrupt the execution of the communication action according to the action information to the communication robot, It is a robot control device.

  According to the invention of the robot control device, the communication robot can be caused to execute a communication action in accordance with a human body motion, similarly to the invention of the communication robot system.

  According to the present invention, since the correspondence between the human body motion and the communication behavior of the communication robot is stored and the communication behavior is executed based on the current body motion, the communication corresponding to the human body motion is performed. Actions can be performed by communication robots.

  The above object, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

  Referring to FIG. 1, a communication robot system (hereinafter simply referred to as “system”) 10 of this embodiment includes a communication robot (hereinafter simply referred to as “robot”) 12. The robot 12 can take a body motion or action (hereinafter sometimes referred to as “communication action”) such as gesture gestures with the human 14. However, the communication behavior may include a conversation between the robot 12 and the human 14.

  The robot 12 has a human-like body and generates complex body movements necessary for communication using the body. Specifically, referring to FIG. 2, the robot 12 includes a carriage 32, and wheels 34 for autonomously moving the robot 12 are provided on the lower surface of the carriage 32. The wheel 34 is driven by a wheel motor (indicated by reference numeral “36” in FIG. 3 showing the internal configuration of the robot 12), and the carriage 32, that is, the robot 12 can be moved in any direction.

  Although not shown in FIG. 2, a collision sensor (indicated by reference numeral “38” in FIG. 3) is attached to the front surface of the carriage 32, and the collision sensor 38 is connected to a person or other person to the carriage 32. Detect obstacle contact. When contact with an obstacle is detected during the movement of the robot 12, the driving of the wheels 34 is immediately stopped and the movement of the robot 12 is suddenly stopped.

  In this embodiment, the height of the robot 12 is about 100 cm so as not to intimidate people, particularly children. However, this height can be arbitrarily changed.

  A polygonal column sensor mounting panel 40 is provided on the carriage 32, and an ultrasonic distance sensor 42 is mounted on each surface of the sensor mounting panel 40. The ultrasonic distance sensor 42 measures the distance between the mounting panel 40, that is, the person around the robot 12 mainly.

  Further, the body of the robot 12 is mounted on the carriage 32 so that the lower portion thereof is surrounded by the mounting panel 40 described above and stands upright. The body is composed of a lower body 44 and an upper body 46, and the lower body 44 and the upper body 46 are connected by a connecting portion 48. Although not shown, the connecting portion 48 has a built-in lifting mechanism, and the height of the upper body 46, that is, the height of the robot 12 can be changed by using the lifting mechanism. As will be described later, the elevating mechanism is driven by a waist motor (indicated by reference numeral “50” in FIG. 3). The height 100 cm of the robot 12 described above is a value when the upper body 46 is at its lowest position. Therefore, the height of the robot 12 can be 100 cm or more.

  One omnidirectional camera 52 and one microphone 16 are provided in the approximate center of the upper body 46. The omnidirectional camera 52 photographs the surroundings of the robot 12 and is distinguished from an eye camera 54 described later. The microphone 16 captures ambient sounds, particularly human voice.

  Upper arms 58R and 58L are attached to both shoulders of the upper body 46 by shoulder joints 56R and 56L, respectively. The shoulder joints 56R and 56L each have three degrees of freedom. That is, the right shoulder joint 56R can control the angle of the upper arm 58R around each of the X, Y, and Z axes. The Y axis is an axis parallel to the longitudinal direction (or axis) of the upper arm 58R, and the X axis and the Z axis are axes orthogonal to the Y axis from different directions. The left shoulder joint 56L can control the angle of the upper arm 58L around each of the A, B, and C axes. The B axis is an axis parallel to the longitudinal direction (or axis) of the upper arm 58L, and the A axis and the C axis are axes orthogonal to the B axis from different directions.

  Forearms 62R and 62L are attached to the respective distal ends of upper arms 58R and 58L via elbow joints 60R and 60L. The elbow joints 60R and 60L can control the angles of the forearms 62R and 62L around the W axis and the D axis, respectively.

  In the X, Y, Z, W axes and the A, B, C, D axes that control the displacement of the upper arms 58R and 58L and the forearms 62R and 62L (FIG. 3), “0 degree” is the home position. In this home position, the upper arms 58R and 58L and the forearms 62R and 62L are directed downward.

  Although not shown in FIG. 3, a touch sensor (FIG. 3) is provided on the shoulder portion including the shoulder joints 56R and 56L of the upper body 46 and the arm portion including the upper arms 58R and 58L and the forearms 62R and 62L. The touch sensor 64 detects whether or not a person has touched these parts of the robot 12.

  Spheres 66R and 66L corresponding to hands are fixedly attached to the tips of the forearms 62R and 62L, respectively. In place of the spheres 66R and 66L, a “hand” in the shape of a human hand can be used when a finger function is required unlike the robot 12 of this embodiment.

  A head 70 is attached to an upper center of the upper body 46 via a neck joint 68. The neck joint 68 has three degrees of freedom and can be controlled in angle around each of the S, T, and U axes. The S-axis is an axis that goes directly from the neck, and the T-axis and the U-axis are axes that are orthogonal to the S-axis in different directions. The head 70 is provided with a speaker 72 at a position corresponding to a human mouth. The speaker 72 is used for the robot 12 to communicate with a person around it by voice or voice. However, the speaker 72 may be provided in another part of the robot 12, for example, the trunk.

  The head 70 is provided with eyeball portions 74R and 74L at positions corresponding to the eyes. Eyeball portions 74R and 74L include eye cameras 54R and 54L, respectively. The right eyeball portion 74R and the left eyeball portion 74L may be collectively referred to as an eyeball portion 74, and the right eye camera 54R and the left eye camera 54L may be collectively referred to as an eye camera 54. The eye camera 54 captures the video signal by photographing the face of the person approaching the robot 12 and other parts or objects.

  Note that each of the omnidirectional camera 52 and the eye camera 54 described above may be a camera using a solid-state imaging device such as a CCD or a CMOS.

  For example, the eye camera 54 is fixed in the eyeball part 74, and the eyeball part 74 is attached to a predetermined position in the head 70 via an eyeball support part (not shown). The eyeball support unit has two degrees of freedom and can be controlled in angle around each of the α axis and the β axis. The α axis and the β axis are axes set with respect to the head 70, the α axis is an axis in a direction toward the top of the head 70, the β axis is orthogonal to the α axis and the front side of the head 70 It is an axis in a direction orthogonal to the direction in which (face) faces. In this embodiment, when the head 70 is at the home position, the α axis is set to be parallel to the S axis and the β axis is set to be parallel to the U axis. In such a head 70, when the eyeball support portion is rotated around each of the α axis and the β axis, the tip (front) side of the eyeball portion 74 or the eye camera 54 is displaced, and the camera axis, that is, the line-of-sight direction is changed. Moved.

  In the α axis and β axis that control the displacement of the eye camera 54, “0 degree” is the home position. At this home position, the camera axis of the eye camera 54 is the head 70 as shown in FIG. The direction of the front side (face) is directed, and the line of sight is in the normal viewing state.

  FIG. 3 is a block diagram showing the internal configuration of the robot 12. As shown in FIG. 3, the robot 12 includes a microcomputer or a CPU 76 for overall control. The CPU 76 is connected to a memory 80, a motor control board 82, a sensor input / output board 84, and a voice through a bus 78. An input / output board 86 is connected.

  Although not shown, the memory 80 includes a ROM, an HDD, a RAM, and the like, and the control program and data for the robot 12 are stored in the ROM or the HDD in advance. The CPU 76 executes processing according to this program. Specifically, a plurality of programs (referred to as action modules) for controlling the body movement of the robot 12 are stored. For example, the body motions indicated by the behavior module include “handshake”, “cuckling”, “many years”, and so on. When the body motion indicated by the behavior module is “handshake”, when the behavior module is executed, the robot 12 presents the right hand forward, for example. Further, when the body motion indicated by the behavior module is “cuddle”, when the behavior module is executed, the robot 12 presents both hands forward, for example. Further, when the body motion indicated by the behavior module is “many years”, when the behavior module is executed, the robot 12 raises and lowers both hands several times (for example, twice), for example. The RAM can be used as a working memory as well as a temporary storage memory.

  The motor control board 82 is composed of, for example, a DSP (Digital Signal Processor) and controls a motor for driving body parts such as the right arm, the left arm, the head, and the eyes. That is, the motor control board 82 receives control data from the CPU 76, and controls the angles of the three motors for controlling the X, Y, and Z axes of the right shoulder joint 56R and the axis W of the right elbow joint 60R. The rotation angles of a total of four motors including one motor (collectively shown as “right arm motor” in FIG. 4) 88 are adjusted. The motor control board 82 includes a total of four motors including three motors that control the angles of the A, B, and C axes of the left shoulder joint 56L and one motor that controls the angle of the D axis of the left elbow joint 60L. The rotation angle of the motor (collectively shown as “left arm motor” in FIG. 4) 90 is adjusted. The motor control board 82 also adjusts the rotation angles of three motors 92 (collectively shown as “head motors” in FIG. 4) that control the angles of the S, T, and U axes of the neck joint 68. To do. The motor control board 82 also controls the waist motor 50 and the two motors 36 that drive the wheels 34 (collectively shown as “wheel motors” in FIG. 4). Further, the motor control board 82 adjusts the rotation angle of two motors 94 (collectively shown as “right eyeball motor” in FIG. 4) that control the angles of the α axis and β axis of the right eyeball portion 74R. In addition, the rotation angle of two motors 96 (collectively shown as “left eyeball motor” in FIG. 4) for controlling the angles of the α axis and the β axis of the left eyeball portion 74L is adjusted.

  The above-described motors of this embodiment are stepping motors or pulse motors for simplifying the control except for the wheel motors 36, but may be direct-current motors as with the wheel motors 36.

  Similarly, the sensor input / output board 84 is also constituted by a DSP, and takes in signals from each sensor and camera and gives them to the CPU 76. That is, data relating to the reflection time from each of the ultrasonic distance sensors 42 is input to the CPU 76 through the sensor input / output board 84. The video signal from the omnidirectional camera 52 is input to the CPU 76 after being subjected to predetermined processing by the sensor input / output board 84 as necessary. Similarly, the video signal from the eye camera 54 is also supplied to the CPU 76. Further, a signal from the touch sensor 64 is given to the CPU 76 via the sensor input / output board 84.

  Synthetic voice data is given to the speaker 72 from the CPU 76 via the voice input / output board 86, and accordingly, voice or voice according to the data is outputted from the speaker 72. Further, the voice input from the microphone 24 is taken into the CPU 76 via the voice input / output board 86.

  Further, a communication LAN board 98 is connected to the CPU 76 through the bus 78. Similarly, the communication LAN board 98 is also configured by a DSP, and sends the transmission data given from the CPU 76 to the wireless communication apparatus 100 and causes the wireless communication apparatus 100 to transmit the transmission data. The communication LAN board 98 receives data via the wireless communication device 100 and provides the received data to the CPU 76.

  Returning to FIG. 1, the system 10 also includes a computer 20 as a control device (robot control device), which can be a personal computer (PC) or a workstation (WS). As is well known, the computer 20 includes a CPU, a ROM, a RAM, an HDD, and the like. A motion capture system 22 and a database 24 are connected to the computer 20. Furthermore, the robot 12 described above is connected to the computer 20 by a wireless LAN via an access point (not shown) connected to a HUB (not shown), for example.

  A known motion capture system is applied as the motion capture system (three-dimensional motion measurement device) 22. For example, an optical motion capture system of VICON (http://www.vicon.com/) can be used. Although illustration is omitted, the motion capture system 22 includes a computer such as a PC or WS, and the computer and the computer 20 are connected to each other by, for example, a wired or wireless LAN via a HUB. The motion capture system 22 detects numerical data (motion information) related to the body motion of the human 14 and transmits it to the computer 20. The computer 20 stores (accumulates) numerical data transmitted from the motion capture system 22 in a database 24 described later.

  Specifically, with reference to FIG. 4, in the motion capture system 22, a plurality (at least three) of cameras 22 a having an infrared irradiation function are arranged in different directions with respect to the human 14. A plurality (10 in this embodiment) of infrared reflection markers 30 are attached to the human 14. Specifically, as can be seen from FIG. 4, the infrared reflective marker 30 is the body motion of the human 14 such as the top of the human 14, the top of the eye (forehead), the chest, the shoulder, the elbow, and the hand (wrist). Is attached to a portion (part) that can specify the body movement of the human 14.

  The computer of the motion capture system 22 acquires image data from the camera 22a at, for example, 60 Hz (60 frames per second), and performs two-dimensional positions of the markers 30 in all image data at the time of measurement by image processing of the image data. To extract. Then, the computer calculates the three-dimensional position of each marker 30 in the real space based on the two-dimensional position of each marker 30 in the image data, and uses the calculated coordinate data (numerical data) of the three-dimensional position as the three-dimensional motion data. As shown in FIG.

  The computer 20 acquires (detects) three-dimensional motion data transmitted from the motion capture system 22 every frame (1/60 second), and inputs (detects) it into the database (three-dimensional motion DB) 24. In addition, the computer 20 detects robot information (information about communication behavior and behavior status, which will be described later) transmitted from the robot 12, and stores it in the three-dimensional motion DB 24 in association with the three-dimensional motion data. However, the robot 12 transmits the robot information to the computer 20 only once when the communication behavior indicated by the information (behavior information: behavior module in this embodiment) about the communication behavior included in the robot information is executed. . Therefore, when receiving the robot information, the computer 20 associates the currently received robot information with the three-dimensional motion data detected for each frame until the next robot information is received.

  For example, the robot 12 performs the action indicated by the action module according to the instruction of its own computer (CPU 76), that is, the body action (the action in the normal state). At this time, the CPU 76 reads out and executes the behavior module stored in the memory 80 at random or according to a predetermined rule. Further, when the robot 12 is instructed by the computer 20 to specify the name or identifier of the behavior module, the robot 12 performs the physical motion indicated by the behavior module (interrupt state operation). However, in an operation in a normal state (normal operation), the robot 12 performs spontaneously regardless of the human body motion or the like. On the other hand, in the interrupted action (interrupt action), the robot 12 performs an action caused by the human 14's physical action.

  As described above, since the robot information (behavior module and action status described later) of the robot 12 and the three-dimensional motion data of the human 14 are stored in association with each other, the past body motion of the human 14 and the human 14 Compared with the current body motion, the body motion of the robot 12 (the motion indicated by the behavior information included in the robot information) associated with the past or similar past body motion can be derived. However, even if the same person 14 performs the same body movement, the three-dimensional movement data can hardly coincide completely, so in reality, the robot 12 associated with a similar past body movement. It is designed to bring out the body movements.

  However, since there are two possibilities for the body motion (behavior) of the human 14, the correspondence relationship between the behavior of the robot 12 and the behavior of the human 14 is recorded and is similar to the current behavior of the human 14. When the robot 12 is simply caused to execute an action associated with a past action, two kinds of actions are mixed. One is an action for the action of the robot 12 (for example, when the robot 14 asks for a handshake, the person 12 also shakes and returns), and the other is an action for searching for the action of the robot 12 (the person 14 In order to shake hands with the robot 12, a hand is put out to the robot 12.)

  Therefore, in this embodiment, the state when the robot 12 behaves (behavior status) is also included in the robot information and stored, and when searching for past physical motion similar to the current physical motion of the human 14, By considering the action status, the two actions as described above are separated and an appropriate communication action can be executed.

  Specifically, the computer 20 provides a flag indicating a state in which the robot 12 takes an action (normal state) or a state in which the robot 12 takes an action (interrupt state) in the memory 80 (RAM). Considering on / off of the flag, a body motion similar to the current body motion is searched, and the robot 12 is caused to execute a communication behavior (interrupt behavior) by interruption.

  FIGS. 5A, 5 </ b> B, and 5 </ b> C are illustrative views showing examples of the communication behavior of the robot 12 and the body motion of the human 14, respectively. FIG. 5A shows a state where the robot 12 and the human 14 are shaking hands. In FIG. 5B, the robot 12 is about to hug the human 14. On the other hand, the human 14 raises both hands up. In FIG. 5 (C), the robot 12 is going to have a great age. On the other hand, the human 14 is waving his left hand, which is difficult to understand in the drawing.

  As described above, the robot information of the communicating robot 12 and the three-dimensional motion data of the human 14 are associated with each other and stored in the three-dimensional motion DB 24, and the human 14 performs some physical motion on the robot 12. At this time, the body motion is compared with the past body motion, and the communication behavior of the robot 12 recorded corresponding to the closest (similar) body motion is executed. Therefore, not only the robot 12 can take any communication action with respect to the human 14, but also the human 14 can take an action (body motion) with respect to the robot 12.

  However, until a certain amount of data is accumulated in the DB 24, the computer 20 does not search for a communication action that causes the robot 12 to perform an interrupt even when communicating with the human 14. Needless to say, at the beginning of the communication, the three-dimensional motion data and the robot information are not sufficiently stored in the DB 24, and the robot 12 cannot execute correct communication behavior for the human body motion. In this embodiment, the computer 20 does not execute a search until communication is performed between the robot 12 and the human 14 for a certain time (for example, 10 minutes). Therefore, in this fixed time, the robot 12 performs communication behavior only in the normal state.

  As illustrated in FIG. 6, the computer 20 stores the three-dimensional motion data, the behavior module, and the behavior status at the same time in the three-dimensional motion DB 24 in association with each other. In the period shown in (1) of FIG. 6, the action module is “handshake”, the action status at that time is “normal”, and the three-dimensional motion data in the period is stored correspondingly. . Further, in the period shown in (2) of FIG. 6, the behavior module is “holding”, the behavior status at that time is “normal”, and the three-dimensional motion data in the period is stored correspondingly. Is done. Further, in the period shown in (3) of FIG. 6, the action module is “many years old”, the action status at that time is “normal”, and the three-dimensional motion data in the period is stored correspondingly. Is done.

  However, as described above, since a plurality of markers 30 are attached to the human 14, the three-dimensional motion data is multidimensional, that is, a plurality of waveforms appear. Only one waveform obtained when one marker 30 is observed is shown. The same applies hereinafter.

  Although illustration is omitted, as described later, when a communication action by search is executed, that is, when an interrupt action is executed, “interrupt (interrupt)” is stored as the action status. become.

  Therefore, as shown in FIG. 7, the three-dimensional motion DB 24 sequentially stores the three-dimensional motion data, the behavior module and the behavior status corresponding to the three-dimensional motion data in time series. However, as described above, the three-dimensional motion data is a set of numerical data according to time series.

  As described above, the data is accumulated in the DB 24. As described above, when a predetermined time has elapsed, the computer 20 executes the search behavior process, and the past physical motion similar to the current physical motion of the human 14 is performed. , And causes the robot 12 to execute an action according to the action module stored corresponding to the detected body motion by interruption.

  Note that, as will be described later, the computer 20 executes a data accumulation process (log storage process) and a search action process in parallel.

  Specifically, in the search process, the currently acquired 3D motion data (hereinafter referred to as “current 3D motion data”) is used as a query, and the 3D motion data stored in the DB 24 (hereinafter referred to as “previous 3D motion data”). Data is referred to sequentially. As shown in FIG. 8, in this embodiment, the query is three-dimensional motion data for the past L frames including the current time. Further, when comparing the previous three-dimensional motion data with the current three-dimensional motion data, that is, the query, as shown in FIG. Compared with dimension motion data. As a method for comparison, a method (LCSS: Longest Common Subsequence) often used for character string comparison is applied. For example, although a method based on the Euclidean distance is also conceivable, since it does not allow temporal expansion and contraction, even if the same person performs the same action, in order to compare the body movements of humans 14 having different temporal lengths and speeds Not suitable. Although a method using DTW (Dynamic Time Warping) is also conceivable, in order to make sure that all points included in the current three-dimensional motion data are associated with any one of the previous three-dimensional motion data, the three-dimensional motion data. Even if a part of the data is missing, since it tries to correspond to any data, the reliability of the comparison result is lacking.

  Therefore, in this embodiment, by applying LCSS, the previous three-dimensional motion data and the current three-dimensional motion data for each marker 30 are compared, that is, the three-dimensional coordinates are compared to obtain the similarity. It is done. The sum of the similarities of each marker 30 is determined as the similarity between the human 14 body movements. However, the threshold value (condition) for determining whether or not they are similar is set to a value empirically obtained in advance by a test or the like.

  Here, as shown in FIG. 9A, for example, it is assumed that, as a result of comparing the query (current three-dimensional motion data) with the previous three-dimensional motion data in the period a1, it is determined that they are similar to each other. . At this time, since the behavior status of the robot 12 in the period indicated by a1 and a2 in FIG. 9A is “normal”, the computer 20 performs the physical action indicated by the previous three-dimensional action data in the period indicated by a1. It is determined that the “handshake” operation of the robot 12 corresponds, and the count value for “handshake” is incremented by one.

  However, as shown in FIG. 9B, when the action status of the robot 12 in the period indicated by b2 is “interrupt”, the computer 20 performs the body movement indicated by the three-dimensional action data in the period indicated by b1. It is determined that this corresponds to the operation of “cuckling” of the robot 12 in the period indicated by b2, and the count value for “cugging” is incremented by one.

  In FIG. 9B, it is assumed that the query (current three-dimensional motion data) and the previous three-dimensional motion data in the period b1 are determined to be similar.

  Thus, when the action status is “interrupt”, it is determined that the physical action of the human 14 corresponds to the communication action next to the communication action of the robot 12 at the same time. This is because the body motion of the human 14 when the communication action is executed by interruption is not used as a query. For example, it is assumed that the human 14 has received a handshake from the robot 12 in the past and wants to shake it again, and has performed the same body movement as the past. At this time, the search result is a communication action of shaking hands, and the human 14 shakes with the robot 12 because the action that the robot 12 wants to take has appeared. Then, if the body motion of shaking the human 14 at this time becomes a query, an interruption to shake the robot 12 again will be entered, and this will prevent the handshake from continuing for a long time. is there.

  In this way, the computer 20 compares the query with all previous three-dimensional motion data, counts the behavior information of the robot 12 that is considered to correspond to the three-dimensional motion data similar to the query, and has the largest count value. The robot 12 is caused to interrupt and execute the action indicated by the action information having a large. That is, the computer 20 transmits to the robot 12 an instruction for interrupting the action indicated by the action information (behavior module) having the largest count value. However, if the robot 12 is currently executing a communication action with an interrupt, the computer 20 does not transmit an interrupt execution instruction. Further, even when the action information (behavior module) having the largest count value is the same as the action information instructed to execute the interrupt last time, the computer 20 does not transmit the instruction to execute the interrupt. This is to avoid repeated execution of the same communication behavior.

  FIG. 10 is a flowchart showing log storage processing of the computer 20. Referring to FIG. 10, when the computer 20 starts the log saving process, in step S1, the capture data from the motion capture system 22, that is, the three-dimensional motion data, is stored in a memory (for example, a virtual memory provided in the HDD; the same applies hereinafter). ) (Temporary save). In step S3, the robot information, that is, the behavior module (behavior information) and the execution status transmitted from the robot 12 are saved (temporarily saved) in the memory. In step S5, the capture data (three-dimensional motion data), the behavior module, and the behavior status at the same time are associated with each other and stored in the DB 24, and the log storage processing is terminated.

  This log storage process is executed at regular intervals, and the communication behavior of the robot 12 and the body movement of the human 14 are recorded in the DB 24. However, when the DB 24 is full, the DB 24 is updated so that the oldest data is overwritten.

  11 to 13 are flowcharts showing the search action process of the computer 20. As described above, this search behavior process is executed in parallel with the log storage process after a predetermined time has elapsed. Referring to FIG. 11, when the computer 20 starts the search action process, in step S11, the computer 20 saves (temporarily saves) capture data from the motion capture system 22, that is, three-dimensional motion data, in the memory 80. Next, in step S13, the current time C is stored (stored), and 0 is substituted into a variable (time) T in step S15. In subsequent step S <b> 17, capture data (three-dimensional motion data) for L frames from time T (frame) is acquired from the DB 24 and copied to the memory 80. In step S19, it is determined whether the input capture data, that is, the current three-dimensional motion data (query), and the capture data copied from the DB 24, that is, the previous three-dimensional motion data are similar.

  If “NO” in the step S19, that is, if the two capture data are not similar, the process proceeds to a step S37 shown in FIG. 13 as it is. However, if “YES” in the step S19, that is, if the two capture data are similar, it is determined whether or not there is a communication action of the robot 12 at the time T + L + 1 in a step S21 shown in FIG. If “NO” in the step S21, that is, if there is no communication action of the robot 12 at the time T + L + 1, the process proceeds to a step S25 as it is. On the other hand, if “YES” in the step S21, that is, if there is a communication action of the robot 12 at the time T + L + 1, it is determined whether or not the action status of the robot 12 at the time T + L + 1 is “interrupt” in a step S23.

  If “NO” in the step S23, that is, if the action status of the robot 12 at the time T + L + 1 is “normal”, the action (behavior module) of the robot at the time T is read into the memory 80 in a step S25, and the process proceeds to the step S29. move on. However, if “YES” in the step S23, that is, if the action status of the robot 12 at the time T + L + 1 is “interrupt”, the action module of the robot 12 at the time T + L + 1 is read into the memory 80 in a step S27, and the process proceeds to the step S29. move on.

  In step S29, it is determined whether or not the action (behavior module) is described in a table (not shown) created in the memory 80. If “YES” in the step S29, that is, if the action read in the step S25 or S27 exists in the table of the memory 80, the action counter of the action is incremented by 1 (incremented) in a step S31, and FIG. The process proceeds to step S37. On the other hand, if “NO” in the step S29, that is, if the action read in the step S25 or S27 does not exist in the table of the memory 80, a column of the action is provided in the table of the memory 80 in a step S33. In S35, an action counter corresponding to the action is set to 0, and the process proceeds to step S31. However, since the table itself is not created at the beginning of the search action process, in step S33, a table is created and a column for the corresponding action is provided.

  As shown in FIG. 13, in step S37, it is determined whether or not the current time CL and the time T + L match (CL = T + L). That is, the similarity between the query and all the three-dimensional motion data stored in the DB 24 is determined, and it is determined whether or not the search for the communication action that causes the robot 12 to perform an interrupt is completed. If “NO” in the step S37, that is, if the current time CL and the time T + L do not coincide with each other, one frame is added to the time T in a step S39. On the other hand, if “YES” in the step S37, that is, if the current time CL and the time T + L coincide with each other, it is determined whether or not the robot 12 is currently executing an interrupt in a step S41. Although not shown, the computer 20 stores a flag (interrupt flag) indicating whether or not the robot 12 is executing an interrupt, and the robot 12 is executing an interrupt based on ON / OFF of the flag. Determine whether or not. However, the interrupting flag is turned on when the computer 20 instructs execution of the communication action by interruption, and is turned off when the action status transmitted from the robot 12 changes to the normal state.

  If “YES” in the step S41, that is, if the robot 12 is currently executing an interruption, the search action process is ended as it is. On the other hand, if “NO” in the step S41, that is, if the robot 12 is currently being executed normally, an action (behavior module) corresponding to the action counter having the maximum count value is specified in a step S43, and in a step S45. , It is determined whether or not the identified action is the same as the action executed last time. If “YES” in the step S45, that is, if the identified action is the same as the action executed previously, the search action process is ended as it is to avoid the same action being repeatedly executed. On the other hand, if “NO” in the step S45, that is, if the specified action is different from the previously executed action, in step S47, the robot 12 is instructed (instructed) to execute the specified action (behavior module) and searched. End the action process.

  According to this embodiment, since the behavior of the person who performs communication is recorded and the behavior of the robot and the status at that time are stored, the robot can execute the normal behavior and the retrieval behavior. In other words, the robot's behavior can be derived from the human behavior.

  Further, in this embodiment, since the movement of a human performing a communication action is stored and the robot is caused to execute the action for the human being, it is possible to communicate according to each person. Therefore, for example, when the communication action with the person B is executed after the robot 12 finishes the communication action with the person A, the computer 20 may delete the three-dimensional motion DB 24, for example.

  In this embodiment, the case where the robot communicates with one specific person has been described, but it is also possible to communicate with two or more persons if the time zone is different. is there. In such a case, each human 14 is made identifiable, a three-dimensional motion DB is provided for each human 14, the human 14 communicating with the robot 12 is specified, and the human 14 according to the specified human body motion is identified. The three-dimensional motion DB for the specified person 14 may be searched to extract the behavior of the robot 12. Each person 14 can be identified by an image or identification information. The identification in the video can be realized by, for example, putting each person 14 wearing clothes with different colors or different marks, and analyzing the image data obtained by photographing this with a camera such as a CCD camera. It is. Identification with identification information can be realized by holding or attaching a tag with different identification information to each person 14 and analyzing the identification information transmitted by the tag.

FIG. 1 is an illustrative view showing one example of a communication robot system of the present invention. FIG. 2 is an illustrative view for explaining the appearance of the robot shown in FIG. 1 embodiment. FIG. 3 is an illustrative view showing an electrical configuration of the robot shown in FIGS. 1 and 2. FIG. 4 is an illustrative view showing a state where a marker detected by the motion capture system is attached to a person. FIG. 5 is an illustrative view showing an example of communication behavior between a robot and a human. FIG. 6 is an illustrative view showing an example of data stored in the database by the communication behavior shown in FIG. FIG. 7 is an illustrative view showing the contents of a database. FIG. 8 is an illustrative view showing a comparison method for comparing a human's current physical motion with a human's past physical motion. FIG. 9 is an illustrative view showing a comparison method for comparing a human's current physical motion with a human's past physical motion. FIG. 10 is a flowchart showing log storage processing of the computer shown in FIG. FIG. 11 is a flowchart showing a part of the search action process of the computer shown in FIG. FIG. 12 is a flowchart showing another part of the search action process subsequent to FIG. FIG. 13 is a flowchart showing another part of the search action process subsequent to FIGS. 11 and 12.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Communication robot system 12 ... Communication robot 16 ... Microphone 20 ... Computer 22 ... Motion capture system 24 ... Database 38 ... Collision sensor 42 ... Ultrasonic distance sensor 52 ... Omnidirectional camera 54 ... Eye camera 64 ... Touch sensor 76 ... CPU
DESCRIPTION OF SYMBOLS 80 ... Memory 82 ... Motor control board 84 ... Sensor input / output board 86 ... Voice input / output board 88-96 ... Motor 98 ... Communication LAN board 100 ... Wireless communication apparatus

Claims (4)

A communication robot system comprising a communication robot and a control device for controlling the communication robot,
Communication robot
A communication action execution means for executing a communication action according to action information in a normal state in which the communication action is spontaneously taken regardless of the human body movement or in an interrupt state in which the communication action is taken due to the human body movement ;
Transmitting means for transmitting robot information including the behavior information and state information indicating the normal state or the interrupt state when the communication behavior is executed, and an interrupt for receiving an interrupt instruction from the control device Comprising an instruction receiving means;
The controller is
Operation information detecting means for detecting operation information about the body movements of the human,
Robot information detecting means for detecting robot information transmitted from the communication robot;
Storage means for storing the motion information and the robot information in association with each other at the same time ;
Comparison means for comparing current motion information detected by the motion information detection means with past motion information stored in the storage means ;
A state for determining whether state information included in the robot information corresponding to the past motion information indicates a normal state or an interrupt state when there is past motion information indicating that the comparison result by the comparison means is at least similar Discrimination means, and
When the state determining means determines that the normal state is indicated, the action information included in the robot information corresponding to the past motion information at least similar to the current motion information is read, or the state When it is determined by the determining means that the interrupt state is indicated, the robot information corresponding to the past motion information that is temporally next to the past motion information that is at least similar to the current motion information is included in the robot information. A communication robot system comprising: interrupt instruction transmission means for reading action information and transmitting an interrupt instruction for executing a communication action according to the action information to the communication robot. It said interrupt instruction transmitting means, when the communication robot is performing communication behavior in the interrupt does not transmit the new interrupt instruction, communication robot system of claim 1, wherein. 3. The communication robot according to claim 1, wherein the interrupt instruction transmission unit does not transmit a new interrupt instruction when the read behavior information matches the behavior information instructed to be executed in the previous interrupt. A robot controller for controlling a communication robot that executes a communication action according to action information in a normal state in which communication action is voluntarily performed regardless of human body movement or in an interruption state in which communication action is caused due to human body movement Because
Operation information detecting means for detecting operation information about the body movements of the human,
Robot information detecting means for detecting robot information transmitted from the communication robot;
Storage means for storing the motion information and the robot information in association with each other at the same time ;
Comparison means for comparing current motion information detected by the motion information detection means with past motion information stored in the storage means ;
A state for determining whether state information included in the robot information corresponding to the past motion information indicates a normal state or an interrupt state when there is past motion information indicating that the comparison result by the comparison means is at least similar Discrimination means, and
When the state determining means determines that the normal state is indicated, the action information included in the robot information corresponding to the past motion information at least similar to the current motion information is read, or the state When it is determined by the determining means that the interrupt state is indicated, the robot information corresponding to the past motion information that is temporally next to the past motion information that is at least similar to the current motion information is included in the robot information. A robot control device comprising: an interrupt instruction transmitting unit that reads out action information and transmits an interrupt instruction for executing a communication action according to the action information to the communication robot.

Images