JP5033994B2 - Communication robot - Google Patents

Description

  The present invention relates to a communication robot, and more particularly to a communication robot that executes a communication action with a human using at least one of voice and body motion.

  An example of a conventional communication robot of this type is disclosed in Patent Document 1. According to the remote control system of Patent Document 1, the robot device is used for security and basically performs autonomous behavior. However, when the robot operation needs to be corrected, the correction operation can be performed by shifting to the autonomous remote cooperation mode. At this time, if the cooperation ratio is 100% remote, the robot apparatus performs a complete remote operation.

  Another example of this type of communication robot is disclosed in Patent Document 2. In the maintenance robot system of Patent Document 2, the maintenance robot patrols the computer group used in the customer's facility in the unattended environment or at night, and when an abnormality is found, reports the failure content to the information management system. Then, according to the work procedure sent from the information management system, the corresponding computer is maintained. However, if the maintenance robot is not completed even after repeating the maintenance work three times, it requests the call center to send maintenance personnel, and if the work is completed within three times, the maintenance robot reports the completion of the work to the call center.

Further, Patent Document 3 (the same applies to Patent Document 4) discloses a remote-to-site dialogue system that promotes communication between humans between remote locations by transmitting body motion together with voice. The intention transmission device disclosed in Patent Document 3 includes a shared robot that behaves as a speaker or a listener, a speaker control unit, a listener control unit, and a voice transmission / reception unit, and is applied to, for example, a telephone. In this intention transmission device, a voice signal transmitted / received through a telephone line is regarded as a time-series electrical signal ON / OFF, and a timing of operation is determined from ON / OFF of the electrical signal to operate each part of the robot. Specifically, a robot as a speaker blinks, opens and closes a mouth, and moves a body part such as an arm and a waist in accordance with a voice signal of a communication partner. On the other hand, the robot as a listener performs a whispering operation according to the voice signal of the user who is the user.
JP 2003-251581 A [B25J 13/00] JP 2005-153104 A [B25J 13/00] JP 2000-349920 A [H04M 11/00] JP 2001-156930 A [H04M 11/00]

  In the invention of Patent Document 1, since the operator monitors various sensor information transmitted from the robot apparatus and sets the operation mode of the robot apparatus, the operator needs to constantly monitor the state of the robot apparatus. was there. That is, it was inefficient.

  Further, in the invention of Patent Document 2, if the maintenance robot is not completed even if the maintenance work is repeated three times, it merely requests the call center to dispatch maintenance personnel. In other words, it was merely a call for maintenance personnel to the site, and it was not intended to solve computer group abnormalities by remote operation by an operator.

  The inventions of Patent Document 3 and Patent Document 4 only control (remote control) a robot's whirling operation in accordance with a human voice or operation that exists in a remote place. Was not intended to remotely control the robot.

  Therefore, a main object of the present invention is to provide a communication robot that can remotely control the robot efficiently.

The invention according to claim 1 is a communication robot that executes a communication action with a human by using at least one of voice and body movement by remote control indicating that the vehicle is operated in accordance with an autonomous control or a control command from an operator terminal. A detecting means for detecting surrounding information including at least a human voice, a storing means for storing an operator calling condition indicating a condition for determining whether or not it is difficult to cope with autonomous control, and a detecting means. A call determination unit that determines whether or not an operator call condition stored in the storage unit is satisfied based on the detected ambient information. When the call determination unit determines that the operator call condition is satisfied, the operator of the communication robot Calls the operator terminal to call the operator Transmitting means for transmitting, receiving means for receiving the remote operation information indicating a control command transmitted from the operator terminal, execution means for executing the communication action based on the remote operation information received by the receiving means, and with the human The communication robot includes call condition adding means for automatically adding an operator call condition to the storage means based on the interaction performed .

According to the first aspect of the present invention, the communication robot (10: reference number corresponding to the embodiment; the same applies hereinafter) executes communication behavior with a human by using at least one of voice and body movement. For example, communication robots are placed in various places and situations such as certain event venues and company receptions, and usually perform communication behaviors such as conversations with human beings by autonomous control. Fulfill (provide service). The detection means (28, 32, 42, 46, 48, 58, 72, 98, S23) detects ambient information including at least a human voice, that is, information about the surroundings of the person and his / her situation. The storage means (100) stores the operator call condition. Here, the operator calling condition is a condition for determining whether or not to call an operator. Specifically, this indicates that the situation or situation is difficult to handle by autonomous control alone, and that it is necessary to respond by remote control. The call determination means (72, S5, S25 to 39) determines whether or not the operator call condition stored in the storage means is satisfied based on the surrounding information detected by the detection means. When it is determined by the call determination means that the operator call condition is satisfied, the transmission means (72, 96, S7) transmits a call signal for calling the operator to the operator terminal (204) via the network (202), for example. To do. In the operator terminal, when the operator inputs remote operation information (control command), it is transmitted to the communication robot. In the communication robot, the receiving means (72, 96, S11) receives the remote operation information input by the operator. The executing means (26, 40, 72, 84 to 92, S13) executes a communication action including at least one of voice and physical action based on the remote operation information received by the receiving means. The calling condition adding means automatically adds an operator calling condition to the storage means based on the interaction performed with a human.

  According to the first aspect of the present invention, the robot can be remotely operated efficiently because the operator can perform the remote operation as necessary.

  The invention of claim 2 is dependent on claim 1, and further comprises keyword determination means for recognizing a human voice detected by the detection means to determine whether or not a keyword is included, and the call determination means includes the keyword When it is determined by the determining means that the keyword is included, it is determined that the operator calling condition is satisfied.

  In the invention of claim 2, the keyword judging means (72, S25) recognizes (voice recognition) the human voice detected by the detecting means, and judges whether or not the keyword is included in the voice-recognized human voice. To do. For example, a keyword (word, phrase) used to express a problem when a human is talking to a communication robot is set as the keyword. Specifically, words (words, phrases) such as “call the responsible person” and “don't know” may be included. When the keyword determining means determines that the keyword is included, the call determining means determines that the operator calling condition is satisfied.

  According to the second aspect of the present invention, since a situation in which a person is in trouble is recognized by voice, the operator can be appropriately called without bothering the person's hand. That is, remote control can be performed more efficiently.

  The invention of claim 3 is dependent on claim 1 or 2, and the detection means includes a facial expression acquisition means for acquiring a facial expression of the human face, and the facial expression of the human face acquired by the facial expression acquisition means is specified. Facial expression determination means for determining whether or not the expression is a facial expression, and the call determination means determines that the operator call condition is satisfied when the facial expression determination means determines that the expression is a specific expression.

  In the invention of claim 3, the facial expression acquisition means (42, 48, 72, S23) acquires a facial expression of a human face. The facial expression determination means (72, S27) determines whether or not the facial expression of the human face acquired by the facial expression acquisition means is a specific expression. As the specific facial expression, for example, a facial expression of a human face to be shown when communication behavior with the communication robot is not good can be set. Specifically, an angry expression or a troubled expression is set. When the facial expression determination unit determines that the facial expression is a specific expression, the call determination unit determines that the operator calling condition is satisfied.

  According to the invention of claim 3, since it is determined by the facial expression of the human face that the communication with the human being is not properly performed, similarly to the invention of claim 2, without bothering the human hand, The operator can be called appropriately.

  The invention of claim 4 is dependent on any one of claims 1 to 3, wherein the detection means includes personal identification means for individually identifying a person, and whether the person identified by the personal identification means is a specific person. It further comprises human judgment means for judging whether or not, and the call judgment means judges that the operator call condition is satisfied when the human judgment means determines that the person is a specific person.

  In the invention of claim 4, the personal identification means (42, 48, 72, 98, S23) identifies a person individually. The human judgment means (72, S29) judges whether or not the person identified by the personal identification means is a specific person. Here, the specific person means a person who is desired to respond appropriately by remote operation by an operator. As the specific person, for example, a so-called VIP (important customer) or a person (for example, an infant) who is difficult to deal with only by autonomous control is set. When it is determined by the human determination means that the person is a specific person, the call determination means determines that the operator call condition is satisfied.

  According to the invention of claim 4, when a specific person is recognized, the operator is quickly called, so that it is possible to cope more appropriately and to prevent troubles in advance. That is, it is possible to provide an appropriate service according to the situation.

  The invention of claim 5 is dependent on any one of claims 1 to 4, wherein the detection means includes distance detection means for detecting a distance from an object including at least a human, and the object detected by the distance detection means Distance determining means for determining whether or not the distance is within a predetermined distance, and the call determining means determines that the operator calling condition is satisfied when the distance determining means determines that the distance is within the predetermined distance. To do.

  In the invention of claim 5, the detecting means includes a distance detecting means (32, 72, S23), and the distance detecting means detects a distance from an object including at least a human being. The distance determining means (72, S31, S33) determines whether or not the distance from the object detected by the distance detecting means is within a predetermined distance. When the distance determining means determines that the distance is within the predetermined distance, the call determining means determines that the operator calling condition is satisfied. That is, based on the distance detected by the distance detection means, a specific situation that requires an action by the operator is estimated. For example, if the distance to a person is within 1 meter and this state continues for 5 minutes or more, the person and the communication robot have been talking for a long time, and the reception and guidance by the communication robot can be performed well. It is speculated that not. Further, for example, when the distance from the human being is 50 cm or less during the movement of the communication robot and this state continues for 10 seconds or longer, it is estimated that the hand going to the human being is blocked.

According to the invention of claim 5, since the operator is called by estimating the situation where the conversation with the human being is not good or the situation where the human being is tampered with, it is possible to appropriately cope with the trouble .

  According to the present invention, the operator is appropriately determined when a situation requiring remote operation by the operator is called, and the operator may perform remote operation as necessary. That is, the robot can be remotely operated efficiently, and an appropriate service according to the situation can be provided.

  The above object, other objects, features, and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

  Referring to FIG. 1, a communication robot 10 (hereinafter, simply referred to as a “robot”) of this embodiment is an interaction-oriented one mainly intended to communicate with a communication target such as a human. It is possible to interact with humans by voice. Moreover, it can also communicate using body movements, such as gesture gesture instead of a voice | voice or with a voice | voice. Hereinafter, in this specification, a communication action or action using at least one of voice and body movement is referred to as communication action.

  The robot 10 is, for example, a reception robot or a road guidance robot, and is arranged in various places and situations such as a certain event venue or company reception, and normally plays a role such as reception and road guidance by autonomous control. Fulfill (provide service). However, when it becomes difficult for the robot 10 to respond by autonomous control alone, for example, when the conversation with the human is not good, or when it becomes a scene that requires more detailed response, etc. Call the operator. The robot 10 receives a control command (remote operation information) input by the calling operator, and executes a communication action (operation or speech) according to the control command. That is, the robot 10 is remotely operated as necessary by the operator.

  FIG. 1 is a front view showing the appearance of the robot 10. As shown in FIG. 1, the robot 10 includes a carriage 22, and wheels 24 for autonomously moving the robot 10 are provided on the lower surface of the carriage 22. The wheel 24 is driven by a wheel motor 26 (see FIG. 2), and the carriage 22, that is, the robot 10 can be moved in any direction, front, rear, left, and right. As described above, the robot 10 is movable in the environment where it is arranged, but may be fixedly provided at a predetermined position in the environment depending on circumstances.

  Although not shown in FIG. 1, a collision sensor 28 (see FIG. 2) is attached to the front surface of the carriage 22, and the collision sensor 28 detects contact of a person and other obstacles with the carriage 22. That is, when contact with an obstacle is detected during the movement of the robot 10, the driving of the wheels 24 is immediately stopped to suddenly stop the movement of the robot 10.

  In this embodiment, the height of the robot 10 is about 100 cm so as not to intimidate people, particularly children. However, this height can be changed.

  A polygonal column sensor mounting panel 30 is provided on the carriage 22, and an ultrasonic distance sensor 32 is mounted on each surface of the sensor mounting panel 30. The ultrasonic distance sensor 32 measures the distance between the sensor mounting panel 30, that is, the person around the robot 10 mainly.

  Moreover, the lower part of the robot 22 is surrounded by the sensor mounting panel 30 so that the body of the robot 10 stands upright. This body is constituted by a lower body 34 and an upper body 36, and the lower body 34 and the upper body 36 are connected to each other by a connecting portion 38. Although illustration is omitted, the connecting portion 38 has a built-in lifting mechanism, and the height of the upper body 36, that is, the height of the robot 10 can be changed by using this lifting mechanism. As will be described later, the lifting mechanism is driven by a waist motor 40 (see FIG. 2).

  The height of the robot 10 described above is that when the upper body 36 is at its lowest position. Therefore, the height of the robot 10 can be 100 cm or more.

  The upper body 36 is provided with one omnidirectional camera 42. The omnidirectional camera 42 is installed, for example, on a support column 44 extending from substantially the center of the upper end on the back side. The omnidirectional camera 42 photographs the surroundings of the robot 10 and is distinguished from an eye camera 48 described later. As this omnidirectional camera 42, for example, a camera using a solid-state imaging device such as a CCD or a CMOS can be adopted. In addition, one microphone 46 is provided substantially at the center of the front side of the upper body 36. The microphone 46 captures ambient sounds, particularly human voices that are communication targets. The installation positions of the omnidirectional camera 42 and the microphone 46 are not limited to the upper body 36 and can be changed as appropriate.

  Upper arms 52R and 52L are provided on both shoulders of the upper body 36 by shoulder joints 50R and 50L, respectively. The shoulder joints 50R and 50L each have three axes of freedom. That is, the shoulder joint 50R can control the angle of the upper arm 52R around each of the X axis, the Y axis, and the Z axis. The Y axis is an axis parallel to the longitudinal direction (or axis) of the upper arm 52R, and the X axis and the Z axis are orthogonal to the Y axis from different directions. On the other hand, the shoulder joint 50L can control the angle of the upper arm 52L around each of the A axis, the B axis, and the C axis. The B axis is an axis parallel to the longitudinal direction (or axis) of the upper arm 52L, and the A axis and the C axis are axes orthogonal to the B axis from different directions.

  Further, forearms 56R and 56L are provided at the respective distal ends of the upper arms 52R and 52L via elbow joints 54R and 54L. The elbow joints 54R and 54L can control the angles of the forearms 56R and 56L around the axes of the W axis and the D axis, respectively.

  In the X axis, Y axis, Z axis, W axis, A axis, B axis, C axis, and D axis that control the displacement of the upper arms 52R and 52L and the forearms 56R and 56L, "0 degree" is the home position, respectively. In this home position, as shown in FIG. 1, the upper arms 52R and 52L and the forearms 56R and 56L are directed downward.

  Although not shown in the drawings, touch sensors (shown comprehensively in FIG. 2) are respectively provided on the shoulder portion of the upper body 36 including the shoulder joints 50R and 50L, the upper arms 52R and 52L, and the forearms 56R and 56L. : 58), and these touch sensors 58 detect whether or not a person touches each part of the robot 10.

  Spheres 60R and 60L corresponding to hands are fixedly provided at the tips of the forearms 56R and 56L, respectively. However, if a finger or palm function is required, a “hand” in the shape of a human hand can be used.

  A head 64 is provided above the center of the upper body 36 via a neck joint 62. The neck joint 62 has a degree of freedom of three axes, and the angle can be controlled around each of the S axis, the T axis, and the U axis. The S-axis is an axis that extends from the neck directly upward (vertically upward), and the T-axis and the U-axis are axes that are orthogonal to the S-axis in different directions. The head 64 is provided with a speaker 66 at a position corresponding to a human mouth. The speaker 66 is used by the robot 10 to execute communication behaviors by voice or sound with respect to people around it. However, the speaker 66 may be provided in another part of the robot 10, for example, the trunk.

  The head 64 is provided with eyeballs 68R and 68L at positions corresponding to the eyes. Eyeball portions 68R and 68L include eye cameras 48R and 48L, respectively. Hereinafter, the right eyeball portion 68R and the left eyeball portion 68L may be collectively referred to as the eyeball portion 68, and the right eye camera 48R and the left eye camera 48L may be collectively referred to as the eye camera 48.

  The eye camera 48 captures a human face approaching the robot 10, other parts or objects, and captures a corresponding video signal. As the eye camera 48, a camera similar to the omnidirectional camera 42 described above can be used.

  For example, the eye camera 48 is fixed in the eyeball part 68, and the eyeball part 68 is attached to a predetermined position in the head 64 via an eyeball support part (not shown). The eyeball support unit has two degrees of freedom and can be controlled in angle around each of the α axis and the β axis. The α axis and the β axis are axes provided with respect to the head 64, the α axis is an axis in a direction toward the top of the head 64, the β axis is orthogonal to the α axis and the front side of the head 64 ( It is an axis in a direction perpendicular to the direction in which the face is facing. In this embodiment, when the head 64 is at the home position, the α axis is set to be parallel to the S axis, and the β axis is set to be parallel to the U axis. In such a head 64, when the eyeball support portion is rotated around each of the α axis and the β axis, the tip (front) side of the eyeball portion 68 or the eye camera 48 is displaced, and the camera axis, that is, the line-of-sight direction is changed. Moved.

  In the α axis and β axis that control the displacement of the eye camera 48, “0 degree” is the home position. At this home position, the camera axis of the eye camera 48 is the head 64 as shown in FIG. The direction of the front side (face) is directed, and the line of sight is in the normal viewing state.

  FIG. 2 is a block diagram showing an electrical configuration of the robot 10. With reference to FIG. 2, the robot 10 includes a CPU 72 for controlling the whole. The CPU 72 is also called a microcomputer or a processor, and is connected to the memory 76, the motor control board 78, the sensor input / output board 80, and the audio input / output board 82 via the bus 74.

  Although not shown, the memory 76 includes a ROM, an HDD, and a RAM, and programs such as a control program for controlling the operation of the robot 10 are stored in advance in the ROM and the HDD. Other programs include a detection program for detecting the output of each sensor 28, 32, 42, 46, 48, 58, a calling program for calling an operator, and an external computer, that is, an operator terminal 204 (see FIG. 3). This corresponds to a communication program for transmitting / receiving necessary data or commands. The memory 76 also stores voice data or voice data (voice synthesis data) to be generated from the speaker 66 when performing a communication action, angle data for presenting a predetermined gesture, and the like. However, the RAM is used as a work memory or a buffer memory.

  As the control program, each of a plurality of programs (referred to as action modules) for controlling the body motion of the robot 10 is stored in correspondence with a command name (control command). For example, the body movements indicated by the behavior module include “what”, “wow”, “standby”, and so on.

  When the body motion indicated by the behavior module is “what”, say “what” and tilt the head. Specifically, the CPU 72 reads out voice data (voice synthesis data) corresponding to “Nani” from the memory 76 and outputs it from the speaker 66 via the voice input / output board 82. At the same time or almost the same time, the CPU 72 reads out the angle data when the neck 62 is tilted from the memory 76 and supplies it to the motor control board 78. Then, the head motor 92 is rotated so as to tilt the neck.

  Also, if the body motion indicated by the behavior module is “Wai”, utter “Wai” and raise both hands. Specifically, the CPU 72 reads out the voice synthesis data corresponding to “Wai” from the memory 76 and outputs it from the speaker 66 via the voice input / output board 82. At the same time or almost the same time, the CPU 72 reads out the angle data for raising both hands from the memory 76, and gives it to the motor control board 78. Then, the right arm motor 88 and the left arm motor 90 are rotated by a predetermined angle so as to raise both hands.

  Further, when the physical motion indicated by the behavior module is “standby”, the user speaks “I am free”, speaks “Play”, or operates to look around. Specifically, the CPU 72 reads out speech synthesis data corresponding to “free time” or “play” from the memory 76 at regular intervals, and outputs it from the speaker 66 via the voice input / output board 82. At the same time or almost the same time, the CPU 72 reads out angle data when the neck 62 is swung left and right from the memory 76 and supplies it to the motor control board 78. Then, the head motor 92 is rotated so as to swing the neck 62 left and right.

  The motor control board 78 is constituted by, for example, a DSP, and controls driving of each axis motor such as each arm, head, and eyeball. That is, the motor control board 78 receives the control data from the CPU 72 and controls two angles of the α axis and β axis of the right eyeball portion 68R (in FIG. 2, collectively referred to as “right eyeball motor”). .) Control the rotation angle of 84. Similarly, the motor control board 78 receives control data from the CPU 72, and controls two angles of the α axis and β axis of the left eyeball portion 68L (in FIG. 2, collectively referred to as “left eyeball motor”). Controls the rotation angle of 86.

  The motor control board 78 receives the control data from the CPU 72, and controls the angles of the X axis, Y axis and Z axis of the right shoulder joint 50R and the W axis angle of the right elbow joint 54R. The rotation angles of a total of four motors (one collectively shown as “right arm motor” in FIG. 2) 88 with one motor to be controlled are adjusted. Similarly, the motor control board 78 receives the control data from the CPU 72 and determines the angles of the three motors for controlling the angles of the A-axis, B-axis and C-axis of the left shoulder joint 50L and the D-axis angle of the left elbow joint 54L. The rotation angle of a total of four motors (one collectively shown as “left arm motor” in FIG. 2) 90 with one motor to be controlled is adjusted.

  Further, the motor control board 78 receives control data from the CPU 72 and controls three motors for controlling the angles of the S-axis, the T-axis, and the U-axis of the head 64 (collectively “head motor” in FIG. 2). The rotation angle of 92 is controlled. Furthermore, the motor control board 78 receives the control data from the CPU 72 and controls the rotation angle of two motors 26 (hereinafter collectively referred to as “wheel motors”) 26 that drive the waist motor 40 and the wheels 24. To do.

  In this embodiment, stepping motors or pulse motors are used for the motors other than the wheel motor 26 in order to simplify the control. However, as with the wheel motor 26, a DC motor may be used.

  Similarly, the sensor input / output board 80 is also constituted by a DSP, and takes in signals from each sensor and gives them to the CPU 72. That is, data relating to the reflection time from each of the ultrasonic distance sensors 32 is input to the CPU 72 through the sensor input / output board 80. Further, a video signal from the omnidirectional camera 42 is input to the CPU 72 after being subjected to predetermined processing by the sensor input / output board 80 as required. Similarly, the video signal from the eye camera 48 is also input to the CPU 72. Further, the plurality of touch sensors described above (collectively indicated as “touch sensor 58” in FIG. 2) and signals from the collision sensor 28 are provided to the CPU 72 via the sensor input / output board 80.

  Similarly, the voice input / output board 82 is also configured by a DSP, and voice or voice in accordance with voice synthesis data provided from the CPU 72 is output from the speaker 66. Also, the voice input from the microphone 46 is taken into the CPU 72 via the voice input / output board 82.

  The CPU 72 is connected to the communication LAN board 94 and the wireless communication device 96 via the bus 74. Although not shown, the communication LAN board 94 and the wireless communication device 96 are connected to a network 202 (see FIG. 3) such as a LAN or the Internet via a wireless LAN access point. The communication LAN board 94 is configured by a DSP, applies transmission data sent from the CPU 72 to the wireless communication device 96, and transmits the transmission data from the wireless communication device 96 to the operator terminal 204 via the network 202. Further, the communication LAN board 94 receives remote operation information (control command) from the operator terminal 204 via the wireless communication device 96 and gives the received remote operation information to the CPU 72. When the robot 10 is installed so as not to move, it may be connected to the network 202 by wire. Further, the robot 10 and the operator terminal 204 may be configured to directly communicate with each other wirelessly or by wire.

  Further, a wireless tag reader 98 is connected to the CPU 72. The wireless tag reader 98 receives a radio wave superimposed with identification information transmitted from a radio tag (RFID tag) via an antenna, amplifies the radio signal, separates the identification information from the radio signal, and The identification information is demodulated (decoded) and given to the CPU 72. The wireless tag is attached to a person in an event venue or company, and the wireless tag reader 98 detects a wireless tag within a communicable range. Note that the wireless tag may be an active type or a passive type that is driven in accordance with radio waves transmitted from the wireless tag reader 98.

  Further, the CPU 72 is connected to a call condition database (call condition DB) 100 via a bus 74. The calling condition DB 100 is a database that stores conditions (operator calling conditions) for determining whether or not to call an operator. Here, as described above, the operator calling condition is a condition for determining whether a trouble (such as speech recognition is impossible) has occurred or a scene in which a more detailed response (such as communication) is required. .

  For example, the operator calling condition may be (1) detecting a certain keyword, (2) obtaining a specific facial expression such as an angry facial expression or a troubled facial expression, or (3) To detect, (4) talk with a person for a long time, (5) block the way to go, (6) have many people around, (7) operate the operator call button, and so on It is judged.

  (1) Specifically, the robot 10 detects and recognizes a human voice (voice) input through the microphone 46. Although illustration is omitted, dictionary data for speech recognition is stored in the memory 76, and input speech is recognized by referring to this dictionary data by DP matching or HMM (Hidden Markov Model) method. To do. Here, for example, when a specific keyword such as “call the person in charge”, “different”, “not so”, or “don't know” is detected, it is determined that the operator calling condition is satisfied. However, the specific keyword is registered in the memory 76 in advance. Further, the specific keyword is merely an example and should not be limited, and may be changed (determined) as appropriate depending on the adaptation environment of the robot 10 and the like. Furthermore, the number of specific keywords may be one or plural. Furthermore, the number of times may be set for each keyword, and it may be determined that the operator's calling condition is satisfied when the number of times the keyword is detected reaches the set number (set number). For example, in the case of a keyword that directly calls an operator, such as “call the responsible person”, the set number of times is set to “1”. In the case of a keyword that indirectly calls the operator, such as “different”, “not so”, “don't know”, etc., the set number of times is set to “3”.

  (2) Further, the robot 10 can acquire facial expressions based on face images taken by the omnidirectional camera 42 or the eye camera 48 and determine whether or not the facial expression is a specific expression. As described above, the facial expression of a specific face is a facial expression of a human being angry or a face that is in trouble. When such a specific facial expression is acquired, it is determined that the operator calling condition is satisfied. A method for recognizing facial expressions has been published in, for example, “The Institute of Electronics, Information and Communication Engineers Journal D-II, Vol.J80-D-II, No.6, pp.1547-1554 (June 1997)”. Yes. Briefly, facial expressions are recognized by converting an image into a spatial frequency domain by two-dimensional discrete cosine transform and capturing power changes in each frequency band corresponding to changes in the facial part. That is, facial expressions are recognized by detecting changes in the shape of the eyes and mouth.

  However, it can be determined that the person is angry based on the sound detected by the microphone 46 instead of acquiring the facial expression. For example, when a sound having a volume higher than a certain value is continuously input for a certain time (for example, 5 seconds) or more, it can be determined that the person is angry by assuming that the person is yelling. .

  (3) Further, the robot 10 uses the wireless tag reader 98 to detect identification information (for example, RFID) of a wireless tag worn or possessed by each person. Although illustration is omitted, the robot 10 stores a human name and its attributes (gender, age, importance, etc.) corresponding to the identification information in the memory 76, and the human name is based on the detected identification information. And at least one of its attributes. Here, for example, when a specific person such as a so-called VIP (important customer) or a person who is difficult to deal with (such as an infant) is detected, it is determined that the operator calling condition is satisfied.

  In this embodiment, a person is individually identified using a wireless tag, but it may be determined whether the person is a specific person based on the video. In such a case, a face image of a specific person is registered (stored) in the memory 76 in advance, and the face image taken by the omnidirectional camera 42 or the eye camera 48 is compared (identified) to obtain a specific image. It is also possible to determine whether or not a person is human. However, when human faces are identified by video processing (image processing), the processing becomes enormous. For example, different specific marks (characters, symbols, figures, etc.) are attached to each person's clothes, It may be determined whether or not the person is a specific person based on the mark.

  (4) Furthermore, the robot 10 specifies a human based on the RFID detected by the wireless tag detection device 98 or the face image taken by the omnidirectional camera 42 or the eye camera 48, and outputs the ultrasonic sensor 32. Based on this, the distance to the person can be detected. Therefore, for example, it can be determined whether or not the user has a long conversation. In this embodiment, the robot 10 assumes that the person and the robot 10 have been talking for a long time if the distance to the person is within 1 m and this state continues for 5 minutes or longer. It is determined that the operator calling condition is satisfied. For long conversations, there is a possibility that communication with humans may not be successful, and there is a possibility that the robot 10 is monopolized by one person. Because. However, the distance and time are examples, and need not be limited to these. Although omitted in FIG. 2, the time is counted by an internal timer.

  (5) Also, when the robot 10 is moving and the distance to the human being is 50 cm or less and this state continues for 10 seconds or more, it is assumed that the hand going to the human being is blocked, It is determined that the operator calling condition is satisfied. Even in such a case, in order to avoid that the human is too close to the robot 10 or to escape from the state where the robot 10 is obstructed by the human, an appropriate action according to the instruction of the operator is executed. Because it should.

  (6) Furthermore, if the number of RFIDs detected by the wireless tag reading device 98 is equal to or greater than a predetermined value (for example, 5), the robot 10 determines that there are a large number of people around and satisfies the operator calling condition. to decide. This is because when there are a large number of humans, speech recognition is difficult and it is impossible to appropriately deal with each human. However, whether or not there are a large number of humans may be determined based on images taken by the omnidirectional camera 42 or the eye camera 48.

  (7) Furthermore, although omitted in FIGS. 1 and 2, it is determined that the operator call condition is satisfied even when an operator call button provided on the robot 10 is operated (pressed). However, the operator call button does not need to be provided on the robot 10, and may be provided around the environment (environment).

  Note that these are merely examples, and any one or more of the above-mentioned conditions may be adopted as the operator calling condition, and other conditions may be set. For example, in the case of the above (4), the robot 10 is within a distance of 10 cm, and if this state continues for 10 seconds or more, the robot 10 is too close to the person. Judge that it satisfies. For example, there is a risk that human clothes may be in a dangerous state such as being caught by the hand of the robot 10. In such a case, appropriate actions should be performed according to instructions from the operator. Because.

  Although not shown, if the robot 10 is provided with a sheet-like pressure sensor using a piezo element instead of the touch sensor 58, the robot 10 may be hit or beaten based on the output (pressure). The situation can be judged. It is determined that such a situation also satisfies the operator calling condition. In such a case, the operator should deal with it to prevent the lot 10 from being damaged.

  Furthermore, although illustration is omitted, if a vibration sensor is provided in the robot 10, it is determined whether the robot 10 is being shaken, and in the same manner as described above, an operator call condition is set to prevent the robot 10 from being damaged. It can be determined that it will be satisfied.

  For example, as described above, the robot 10 having such a configuration functions as a reception robot or a road guide robot, and is applied to a system 200 as shown in FIG. The system 200 includes a robot 10 that is connected to an operator terminal 204 via a network 202 such as the Internet or a LAN.

  The operator terminal 204 is a general-purpose personal computer or a computer such as a workstation or a PDA. Although not shown, the operator terminal 204 includes a display device such as a CRT or LCD, a keyboard, a computer mouse, or the like. An input device, a speaker, a communication device, and the like are connected. The operator terminal 204 is connected to the network 202 via a communication device in a wired or wireless manner. In addition, a control program and necessary data for remotely controlling the robot 10 are stored in a memory such as an HDD or a ROM built in the operator terminal 204.

  In this embodiment, the robot 10 and the operator terminal 204 are connected via the network 202, but may be connected directly. In such a case, short-range radio such as Bluetooth (registered trademark) can be used.

  As described above, the robot 10 performs communication behavior such as dialogue with humans, usually by autonomous control, and performs reception and route guidance. For example, when performing autonomous control, the robot 10 executes a communication action according to a control command selected at random, or executes a communication action according to a control command selected according to an interaction with a human. To do. As a specific example of the latter, when the robot 10 touches the shoulder, the robot 10 speaks “what” and tilts the head. That is, when detecting that the shoulder is touched based on the output of the touch sensor 58, the CPU 72 selects a control command corresponding to “what” and executes a communication action according to the control command. However, this is only an example and should not be limited, and the robot 10 can perform various autonomous actions.

  However, when the operator calling conditions as described above are satisfied, it is considered that communication behavior with a human (user) is not performed smoothly, or that it is difficult to solve by autonomous control alone. It can be said that it is not appropriate for the robot 10 to perform autonomous control.

  In order to avoid this, in this embodiment, the robot 10 transmits a call request to the operator terminal 204 when it is determined that the operator call condition is satisfied. In brief, the CPU 72 outputs an operator call request via the communication LAN board 94 and the wireless communication device 96. Then, it is transmitted to the operator terminal 204 via the network 202.

  When the operator terminal 204 receives a call request from the robot 10, the above-described control program is activated, and a remote operation screen 210 as shown in FIG. 4 is displayed on a display device (not shown). Thus, the operator of the operator terminal 204 can know that there is a call request. However, it is also possible to notify the operator of the necessity for remote operation by outputting a warning sound from a speaker provided (connected) to the operator terminal 204. In such a case, the control program may be activated by an operator using an input device. Then, the operator starts remote operation of the robot 10.

  Here, as shown in FIG. 4, the remote operation screen 210 includes display areas 212, 214 and 216. The display area 212 displays, for example, an image (robot camera image) captured by the eye camera 48 of the robot 10. The display area 214 displays robot information. For example, the robot information includes the name of the place (facility, venue, company, etc.) where the robot 10 is located (here, reception or entrance), the name (or identification information) of the robot 10, and the placement of the robot 10. A map of a place and information (name, age, importance, etc.) of a person with whom the robot 10 is interacting. Further, when the robot 10 and a human are talking, the robot 10 may detect a voice input from the microphone 46 and transmit the voice signal to the operator terminal 204. In this way, the operator terminal 204 can reproduce the voice of a person who is conversing with the robot 10, and the operator can grasp the contents and take appropriate actions. The information presented to these operators is an example, and appropriate information is displayed on the remote operation screen 210 as appropriate.

  Therefore, the operator can remotely operate the robot 10 while looking at the display areas 212 and 214. The display area 216 displays an operation panel for inputting a control command to be transmitted to the robot 10 when the robot 10 is remotely operated. On the operation panel, a plurality of icons or buttons (GUIs) describing control commands are displayed. For example, a button for instructing movement (forward, backward, etc.), a button for instructing communication behavior (bowing, right pointing, etc.), a button for instructing the end of remote operation, and the like are provided. The operator can input a desired control command corresponding to the instructed button by instructing any button using the input device. Then, the input control command is transmitted to the robot 10 via the network 202. When the robot 10 receives the control command transmitted from the operator terminal 204, the robot 10 executes a communication action according to the control command.

Although illustration is omitted, an image (image) included in the robot camera image and the robot information is transmitted from the robot 10 and the environmental camera to the operator terminal 204 via the network 202.


Specifically, the CPU 72 shown in FIG. 2 executes the entire process according to the flowchart shown in FIG. As shown in FIG. 5, when starting the entire process, the CPU 72 determines whether or not there is a stop command in step S1. For example, the stop command is transmitted from the operator terminal 204 to the robot 10 via the network 202 according to the operation of the operator. However, a stop button may be provided in the robot 10 and the button may be operated. If “YES” in the step S1, that is, if there is a stop instruction, the entire processing is ended as it is. On the other hand, if “NO” in the step S1, that is, if there is no stop command, the process proceeds to a step S3.

  In the next step S3, an operator call condition determination process (see FIG. 6), which will be described later, is executed. In step S5, it is determined whether or not the operator's call flag is on. Although not shown in the figure, the operator call flag is composed of a 1-bit register and is provided in the memory 76 (RAM). This call flag is turned on / off by executing an operator call condition determination process. Specifically, if the operator call condition is satisfied, the call flag is turned on. Conversely, if the operator call condition is not satisfied, the call flag is turned off. When the call flag is turned on, the data value “1” is set in the register. Conversely, when the call flag is turned off, the data value “0” is set in the register.

  If “NO” in the step S5, that is, if the calling flag is turned off, an autonomous action process is executed in a step S9, and the process returns to the step S1. Note that, as described above, in step S9, the CPU 72 executes a communication action with a human as needed by autonomous control. For example, when a person is asked for route guidance, the route guidance is performed. On the other hand, if “YES” in the step S5, that is, if the calling flag is on, the operator is called in a step S7. That is, the CPU 72 transmits a call request to the operator terminal 204.

  In a succeeding step S11, it is determined whether or not there is a control command from the operator terminal 204. If “NO” in the step S11, that is, if there is no control command, the process proceeds to a step S15 as it is. On the other hand, if “YES” in the step S11, that is, if there is a control command, a communication action according to the control command is executed in a step S13, and the process proceeds to a step S15.

  In step S15, it is determined whether there is a remote operation end request (end command) from the operator terminal 204. If “NO” in the step S15, that is, if there is no end command, it is determined that the remote operation is continued, and the process returns to the step S11. On the other hand, if “YES” in the step S15, that is, if there is an end command, it is determined that the remote operation is ended, and the process returns to the step S1.

  In this embodiment, when there is a control command from the operator terminal 204, a communication action according to the control command is executed immediately. However, when the communication action is being executed when the control command is received from the operator terminal 204, the communication action according to the control command may be executed after the execution of the communication action is ended. .

  FIG. 6 is a flowchart of the operator call condition determination process in step S3 shown in FIG. As shown in FIG. 6, when starting the operator calling condition determination process, the CPU 72 turns off the calling condition flag in step S21, and detects surrounding information in step S23. In step S23, the CPU 72 detects the outputs of the sensors (ultrasonic distance sensor 32, omnidirectional camera 42, microphone 46, eye camera 48, touch sensor 58, wireless tag reader 98) necessary for determining the operator calling condition. And recognize their surroundings and their own situation (state).

  In a succeeding step S25, it is determined whether or not a specific keyword is detected. If “YES” in the step S25, that is, if a specific keyword is detected, it is determined that the operator calling condition is satisfied. In a step S39, the calling condition flag is turned on, and the operator calling condition determining process is entirely processed. Return to the program. On the other hand, if “NO” in the step S25, that is, if a specific keyword is not detected, it is determined whether or not a specific facial expression is acquired in a step S27. If “YES” in the step S27, that is, if a specific facial expression is acquired, it is determined that the operator calling condition is satisfied, and the process proceeds to the step S39.

  However, if “NO” in the step S27, that is, if a specific facial expression is not acquired, it is determined whether or not a specific human is detected in a step S29. If “YES” in the step S29, that is, if a specific person is detected, it is determined that the operator calling condition is satisfied, and the process proceeds to the step S39. On the other hand, if “NO” in the step S29, that is, if a specific human being is not detected, it is determined whether or not it is a situation in which the user has a long conversation with the human in a step S31.

  If “YES” in the step S31, that is, if it is a situation in which a conversation with a human is performed for a long time, it is determined that the operator calling condition is satisfied, and the process proceeds to a step S39. On the other hand, if “NO” in the step S31, that is, if it is not a situation where the user is talking for a long time, it is determined whether or not the going hand is blocked in a step S33. If “YES” in the step S33, that is, if the going hand is blocked, it is determined that the operator calling condition is satisfied, and the process proceeds to the step S39. On the other hand, if “NO” in the step S33, that is, if the hand is not blocked, it is determined in a step S35 whether or not there are many people around.

  If “YES” in the step S35, that is, if there are many people around, it is determined that the operator calling condition is satisfied, and the process proceeds to the step S39. On the other hand, if “NO” in the step S35, that is, if there are not many people around, it is determined whether or not the operator call button has been pressed in a step S37. If “YES” in the step S37, that is, if the operator call button is pressed, it is determined that the operator call condition is satisfied, and the process proceeds to the step S39. On the other hand, if “NO” in the step S37, that is, if the operator call button is not pressed, the operator call condition determination process is returned to the overall process program.

  According to this embodiment, since an operator call is performed as necessary and a communication action (operation or speech) is executed in accordance with an instruction from the operator, an appropriate service can be provided. Further, since it is not always necessary for the operator to operate, it can be remotely operated efficiently. Furthermore, the efficiency can be improved so that one operator is in charge of a plurality of robots.

  In this embodiment, the operator calling conditions are set in advance and stored in the database. However, the robot itself registers (adds) the operator calling conditions based on the interaction between the robot and the human. It may be. For example, if an action (human action) frequently performed before the “operator call button” is pressed is stored, it can be determined that the operator call condition is satisfied when the action is detected.

  In this embodiment, one operator terminal (one operator) is in charge of remote control of one robot. However, since it is sufficient for the operator to remotely control the robot only when there is a call from the robot, a plurality of robots may be provided for one operator terminal.

  Further, in this embodiment, when the robot speaks, the speech synthesis data stored in the memory in advance is read and output. However, a microphone for inputting the operator's voice (voice) may be provided in the operator terminal, and the operator's voice may be transmitted to the robot and output from the robot speaker. In such a case, the operator can directly talk with the conversation partner of the robot.

FIG. 1 is an illustrative view showing an appearance of a communication robot according to an embodiment of the present invention. FIG. 2 is a block diagram showing an electrical configuration of the communication robot of FIG. FIG. 3 is an illustrative view showing one example of a system to which the communication robot shown in FIG. 1 is applied. FIG. 4 is an illustrative view showing one example of a remote operation screen displayed on the display device of the operator terminal shown in FIG. FIG. 5 is a flowchart showing an example when the CPU shown in FIG. FIG. 6 is a flowchart showing an example when the CPU shown in FIG. 2 performs an operator call condition determination process.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Communication robot 32 ... Ultrasonic distance sensor 42 ... Omnidirectional camera 46 ... Microphone 48 ... Eye camera 58 ... Touch sensor 72 ... CPU
76 ... Memory 96 ... Wireless communication device 98 ... Wireless tag reader 100 ... Database 200 ... Communication robot remote control system 202 ... Network 204 ... Operator terminal

Claims (5)

A communication robot that performs a communication action with a human by using at least one of voice and physical movement by remote control indicating that the vehicle is operated according to a control command from an autonomous control or an operator terminal,
Detection means for detecting ambient information including at least a human voice;
Storage means for storing an operator call condition indicating a condition for determining whether or not the autonomous control is in a difficult state;
Call determination means for determining whether or not an operator call condition stored in the storage means is satisfied based on ambient information detected by the detection means;
A transmission means for transmitting a call signal for calling an operator who is a pilot of the communication robot to the operator terminal when the call determination means determines that the operator call condition is satisfied;
Receiving means for receiving remote operation information indicating a control command transmitted from the operator terminal ;
Execution means for executing a communication action based on remote operation information received by the receiving means ; and
A communication robot comprising calling condition adding means for automatically adding the operator calling condition to the storage means based on an interaction with the human . Recognizing a human voice detected by the detecting means and further determining whether or not a keyword is included;
The communication robot according to claim 1, wherein the call determination unit determines that the operator call condition is satisfied when the keyword determination unit determines that a keyword is included. The detection means includes facial expression acquisition means for acquiring a facial expression of a human face,
A facial expression determination means for determining whether or not the facial expression of the human face acquired by the facial expression acquisition means is a specific expression;
The communication robot according to claim 1, wherein the call determination unit determines that the operator call condition is satisfied when the facial expression determination unit determines that the expression is a specific expression. The detection means includes personal identification means for individually identifying humans,
Human judgment means for judging whether or not the person identified by the personal identification means is a specific person,
The communication robot according to claim 1, wherein the call determination unit determines that the operator call condition is satisfied when the human determination unit determines that the person is a specific person. The detection means includes distance detection means for detecting a distance from an object including at least a human,
Distance determining means for determining whether or not the distance to the object detected by the distance detecting means is within a predetermined distance;
The communication robot according to claim 1, wherein the call determination unit determines that the operator call condition is satisfied when the distance determination unit determines that the distance is within a predetermined distance.

Images