JP2009241166A - Robot remote operation system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable an operator to properly handle, regardless of cultures of a person to talk to, by communication of a robot 12 according to the culture of the person to talk to. <P>SOLUTION: The robot remote operation system 10 includes a central control device 14. If a call is requested from the robot 12, the central control device 14 selects an operator that is in an idle state when determining the language and culture of the person to talk to the robot 12. The selected operator sends a control command for remote controlling the robot 12 to the central control device 14 through an operation terminal 16. When the central control device 14 receives the control command, it converts an action command included in the control command to an action corresponding to the culture of the person to talk to, translates a communication command to the language of the person to talk to, and sends it to the robot 12. Then the robot 12 performs communication activity corresponding to the culture of the person to talk to. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a robot remote control system, and more particularly to a robot remote control system for remotely controlling a robot that communicates with people in a plurality of countries, for example.

An example of this type of conventional robot remote control system is disclosed in Patent Document 1. The communication robot of Patent Document 1 includes a communication robot and an operator terminal connected via a network. The operator terminal is operated by an operator. The communication robot is an autonomous robot capable of communicating with humans, and transmits a call request to an operator terminal when a situation that cannot be handled by autonomous control alone becomes difficult. In response to this, the operator operates the operator terminal to transmit a control command to the communication robot. Then, the communication robot that has called the operator executes a communication action according to the control command from the operator terminal.
JP 2007-190641 A1 [B25J 13/00, B25J 5/00, A63H 3/33]

  However, in the background art disclosed in Patent Document 1, when a communication robot is installed in a venue where people come from various countries, the operator supports multiple languages and cultures in order to cope with people from various countries. It is required to have such a high capability. Requesting such an ability from an operator is harsh and is not considered realistic.

  Therefore, the main object of the present invention is to provide a novel robot remote control system.

  Another object of the present invention is to provide a robot remote control system that can be appropriately handled by an operator regardless of the language or culture of the robot communication partner.

  The present invention employs the following configuration in order to solve the above problems. The reference numerals in parentheses, supplementary explanations, and the like indicate the corresponding relationship with the embodiments described in order to help understanding of the present invention, and do not limit the present invention.

  According to a first aspect of the present invention, there is provided a robot that performs communication with humans through at least one of body movement and voice by remote operation or autonomous control, a plurality of operation terminals that perform remote operation of the robot via a network, and a network This is a robot remote operation system including a central control device that mediates between the robot and one of the plurality of operation terminals via the controller. The robot receives personal information acquisition means for acquiring human personal information, personal information transmission means for transmitting personal information acquired by the personal information acquisition means to the central control device, and first remote operation information from the central control device. First remote operation information receiving means and execution means for executing a communication action based on the first remote operation information received by the first remote operation information receiving means.

  The operation terminal includes second remote operation transmission means for transmitting second remote operation information for remotely operating the robot to the central controller. The central control device includes a storage medium for storing communication information of an operator of the operation terminal, personal information receiving means for receiving personal information transmitted by the personal information transmitting means, and communication information from the personal information received by the personal information receiving means. Is stored in the storage medium and the communication information specified by the communication information specifying means, the second remote operation information receiving means for receiving the second remote operation information transmitted by the second remote operation transmitting means, and the communication information specifying means. The second remote control information received by the second remote operation information receiving means when the judgment results judged by the judgment means judged by the judgment means and the judgment means do not match. Communication information identification means for operating information Therefore comprising a first remote operation information transmitting means for transmitting converting means for converting the first remote control information based on the specified communication information, and the first remote control information converted by the conversion means to the robot.

  In the first invention, the robot remote control system (10) includes a robot (12) that performs a communication action with a human by at least one of body movement and voice by remote control or autonomous control, and a robot via a network. A plurality of operation terminals (16) for performing remote operation, and a central control device (14) for mediating between the robot and one of the plurality of operation terminals via a network.

  The robot personal information acquisition means (66, 80, 116, S305) acquires human personal information. The personal information transmission means (80, 112, 114, S305) transmits the personal information acquired by the personal information acquisition means to the central controller. The first remote operation information receiving means (80, 112, 114, S313) receives the first remote operation information from the central controller. The executing means (80, S317) executes a communication action based on the first remote operation information received by the first remote operation information receiving means.

  The second remote operation transmission means (160, S413) of the operation terminal transmits second remote operation information for remotely operating the robot to the central controller. The storage medium (138) of the central control device stores communication information of the operator of the operation terminal. The personal information receiving means (130, 148, 150, S5) receives the personal information transmitted by the personal information transmitting means. The communication information specifying means (130, S37, S53, S55, S71, S73) specifies communication information from the personal information received by the personal information receiving means. The second remote operation information receiving means (130, S11, S91, S99) receives the second remote operation information transmitted by the second remote operation transmission means. The determining means (130, S61-S65) determines whether or not the communication information specified by the communication information specifying means matches the communication information of the operator of the operating terminal stored by the storage medium. The conversion means (130, S103, S105, S129, S159) uses the second remote operation information received by the second remote operation information reception means when the judgment results judged by the judgment means do not match, the communication information identification means Is converted into first remote control information based on the communication information specified by. The first remote operation information transmission means (130, S107) transmits the first remote operation information converted by the conversion means to the robot.

  For example, a human is a conversation partner who communicates with a robot. If the conversation partner is American, even if the operator (operator) of the operation terminal is not American, the robot speaks in English with the body movement performed by the American. When the determination results determined by the determining means match, the second remote operation information is transmitted to the robot as the first remote operation information.

  According to the first invention, the robot speaks the language of the conversation partner's culture in combination with the body movements performed in the partner's culture, so that the operator appropriately responds regardless of the language or culture of the conversation partner. be able to.

  A second invention is dependent on the first invention, and the communication information includes language information. The communication information specifying means includes language information specifying means for specifying language information from the personal information received by the personal information receiving means. The determining means determines whether or not the language information specified by the language information specifying means matches the language information of the operator of the operation terminal stored by the storage medium. The conversion means includes translation means for translating utterance information included in the second remote control information based on the language information specified by the language specifying means when the determination results determined by the determination means do not match.

  In the second invention, the communication information includes language information. Language information specifying means (130, S37, S53, S55) specifies language information from the personal information received by the personal information receiving means. The determining means determines whether the language information specified by the language information specifying means matches the language information of the operator of the operation terminal stored by the storage medium. The translating means (130, S103, S129) translates the utterance information included in the second remote control information based on the language information specified by the language specifying means when the determination results determined by the determining means do not match. To do.

  For example, the specified language information is Japanese or English. Further, the language information specifying means specifies the language used by the conversation partner. Then, the translation means translates the language spoken by the robot into the language used by the conversation partner if the comparison result by the comparison means does not match. That is, even if the operator who operates the operation terminal does not understand the language used by the conversation partner, the language spoken by the robot is translated into the language used by the conversation partner.

  According to the second invention, even if the operator who remotely operates the robot does not understand the language of the conversation partner, the robot can perform appropriate communication behavior.

  A third invention is dependent on the first invention or the second invention, and the communication information includes information on a type of body movement. The communication information specifying means includes body action type information specifying means for specifying information on the type of body action from the personal information received by the personal information receiving means. The comparison means determines whether or not the information on the type of physical movement specified by the physical action type information specifying means matches the information on the type of physical action of the operator of the operating terminal stored on the storage medium. The converting means converts the motion information included in the second remote operation information based on the body motion type information specified by the body motion type information specifying means when the determination results determined by the determining means do not match. Operation converting means for

  In the third invention, the communication information includes information on the type of body movement. The physical action type information specifying means (130, S71, S73) specifies information on the type of physical action from the personal information received by the personal information receiving means. Further, the comparing means determines whether or not the physical action type information specified by the physical action type information specifying means matches the physical action type information of the operator of the operation terminal stored in the storage medium. To do. The motion conversion means (130, S105, S159) is configured to identify the motion information included in the second remote operation information by the physical action type information specifying means when the judgment results judged by the judgment means do not match. Convert based on the type of information.

  For example, the types of physical movements include movements performed by Americans and movements performed by Japanese. The body movement type determining means determines the type of body movement performed by the conversation partner based on the language used by the conversation partner. The motion conversion means converts the motion performed by the robot into the type of motion performed by the conversation partner. That is, even if the operator of the operation terminal does not know the type of body movement performed by the conversation partner, the robot can cause the robot to perform the same type of movement as the body movement performed by the conversation partner.

  According to the third invention, even if the operator who remotely operates the robot does not know the body movement performed by the conversation partner, the robot can perform the same type of body movement as the conversation partner. Smooth communication with the other party.

  A fourth invention is dependent on either the second invention or the third invention, and the personal information acquisition means includes voice acquisition means for acquiring a human voice. The personal information transmission unit further transmits the voice acquired by the voice acquisition unit. The personal information receiving means further receives the voice transmitted by the personal information transmitting means. The linguistic information specifying means includes a spoken language information specifying means for specifying linguistic information from a human voice received by the personal information receiving means.

  In the fourth invention, the voice acquisition means (66, 80, S305) acquires human voice. The personal information transmission unit further transmits the voice acquired by the voice acquisition unit. The personal information receiving means further receives the voice transmitted by the personal information transmitting means. The speech language information specifying means (130, S53, S55) specifies the language information from the human voice received by the personal information receiving means. That is, the voice of the conversation partner is acquired, and the language used by the conversation partner is specified.

  According to the fourth aspect, the language used by the conversation partner can be specified from the voice of the conversation partner.

  A fifth invention is dependent on the fourth invention, and the central control device includes request signal transmission means for transmitting a request signal for instructing the robot to speak when the human voice is not acquired by the voice acquisition means. Further prepare. The robot further includes a request signal receiving unit that receives the request signal transmitted by the request signal transmitting unit, and a voice output unit that outputs a sound in response to the request signal received by the request signal receiving unit.

  In the fifth invention, the request signal transmitting means (130, S41) transmits a request signal for instructing the robot to speak when the human voice is not acquired by the voice acquiring means. The request signal receiving means (80, S307) receives the request signal transmitted by the request signal transmitting means. The sound output means (80, S309) outputs sound in response to the request signal received by the request signal receiving means. For example, when the language information cannot be specified by the language information specifying means, a communication action requesting the conversation partner's utterance is executed.

  According to the fifth aspect, by requesting human speech, the central control device can specify the language used by the conversation partner.

  A sixth invention is dependent on any one of the second to fifth inventions, and the personal information acquisition means includes identification information acquisition means for acquiring identification information from an identification information tag carried by a human. The personal information transmitting means further transmits the identification information acquired by the identification information acquiring means. The personal information receiving means further receives the identification information transmitted by the personal information transmitting means. The language information specifying means includes identification language specifying means for specifying at least language information from the identification information received by the personal information receiving means.

  In the sixth invention, the identification information acquisition means (80, 116, S305) acquires the identification information from the identification information tag (18) carried by a human. Further, the personal information transmission means further transmits the identification information acquired by the identification information acquisition means. The personal information receiving means further receives the identification information transmitted by the personal information transmitting means. Then, the identification language information specifying means (130, S37) specifies at least language information from the identification information received by the personal information receiving means. For example, the identification information is an ID number assigned to each conversation partner. Based on the ID number, the language used by the conversation partner is specified.

  According to the sixth aspect, the language used by the conversation partner can be specified from the identification information of the conversation partner.

  In a seventh aspect, the personal information acquisition means includes identification information acquisition means for acquiring identification information from an identification information tag carried by a human. The personal information transmitting means further transmits the identification information acquired by the identification information acquiring means. The personal information receiving means further receives the identification information transmitted by the personal information transmitting means. The body motion type determining means includes identification information body motion type determining means for further determining information on the type of body motion from the identification information received by the personal information receiving means.

  In the seventh invention, the identification information acquisition means (80, 116, S305) acquires the identification information from the identification information tag (18) carried by a human. Further, the personal information transmission means further transmits the identification information acquired by the identification information acquisition means. The personal information receiving means further receives the identification information transmitted by the personal information transmitting means. Then, the identification information physical action type determining means (130, S71) further determines information on the type of physical action from the identification information received by the personal information receiving means. In other words, the identification information body movement type determining means determines the type of body movement performed by the conversation partner from the ID number of the conversation partner.

  According to the seventh aspect, if the conversation partner carries the identification information tag, the type of body movement performed by the conversation partner can be determined from the identification information of the conversation partner.

  The eighth invention is dependent on any one of the fourth to seventh inventions, and the central control device receives the voice received by the personal information receiving means when the judgment results judged by the judgment means do not match. Is further provided with speech translation means for translating based on the language information of the operator of the operation terminal stored in the storage medium, and translation result transmission means for transmitting the translation result translated by the speech translation means to the operation terminal. The operation terminal further includes a translation result receiving unit that receives the translation result transmitted by the translated speech result transmitting unit, and an output unit that outputs the translation result received by the translation result receiving unit.

  In the eighth invention, the speech translating means (130, S95) operates the operation terminal in which the speech received by the personal information receiving means is stored in the storage medium when the judgment results judged by the judging means do not match. Translate based on the language information of the person. The translation result transmission means (130, 148, 150, S97) transmits the translation result translated by the speech translation means to the operation terminal. The translation result receiving means (160, 172, 174, S407) receives the translation result transmitted by the translated speech result transmitting means. The output means (160, 176, S409) outputs the translation result received by the translation result receiving means.

  For example, the voice of the conversation partner acquired by the robot is transmitted to the central controller. In the central control unit, the received voice of the conversation partner is translated into the language used by the operator of the operation terminal and transmitted to the operation terminal. The operation terminal outputs the voice of the conversation partner translated by the output means.

  According to the eighth aspect, the operator can easily understand the language of the communication partner of the robot, and can respond appropriately.

  According to a ninth aspect of the present invention, there is provided a robot that performs communication with humans through at least one of body movement and voice by remote operation or autonomous control, a plurality of operation terminals that perform remote operation of the robot via a network, and a network This is a robot remote control system including a central control device that mediates between the robot and one of the plurality of operation terminals via the controller. The robot receives personal information acquisition means for acquiring human personal information, personal information transmission means for transmitting personal information acquired by the personal information acquisition means to the operation terminal, and first remote operation information received from the operation terminal. Remote operation information receiving means, and execution means for executing a communication action based on the first remote operation information received by the first remote operation information receiving means.

  The operation terminal includes a storage medium for storing communication information of the operator of the operation terminal, personal information receiving means for receiving personal information transmitted by the personal information transmitting means, and communication information from the personal information received by the personal information receiving means. Information identifying means for identifying the communication information, judging means for judging whether the communication information identified by the communication information identifying means matches the communication information of the operator of the operation terminal stored in the storage medium, and judged by the judging means Conversion means for converting the second remote operation information for remotely operating the robot into the first remote operation information based on the communication information specified by the communication information specifying means when the determined judgment results do not match, and the conversion means By The first remote control information converted Te comprises a first remote operation information transmitting means for transmitting to the robot.

  In the ninth invention, the robot remote control system (10) includes a robot (12) that performs communication action with at least one of body movement and voice with a human by remote control or autonomous control, and a robot via a network. And a central control device (14) that mediates between the robot and one of the plurality of operation terminals via a network. The robot personal information acquisition means (66, 80, 116, S305) acquires human personal information. The personal information transmitting means (80, 112, 114, S305) transmits the personal information acquired by the personal information acquiring means to the operation terminal. The first remote operation information receiving means (80, 112, 114, S313) receives the first remote operation information from the operation terminal. The executing means (80, S317) executes a communication action based on the first remote operation information received by the first remote operation information receiving means.

  The storage medium (138) of the operation terminal stores communication information of the operator of the operation terminal itself. The personal information receiving means (160, 172, 174, S5) receives the personal information transmitted by the personal information transmitting means. The communication information specifying means (160, S37, S53, S55, S71, S73) specifies communication information from the personal information received by the personal information receiving means. The determining means (160, S61-S65) determines whether or not the communication information specified by the communication information specifying means matches the communication information of the operator of the operating terminal stored by the storage medium. The conversion means (160, S103, S105, S129, S159) specifies the second remote operation information for remotely operating the robot by the communication information specifying means when the determination results determined by the determination means do not match. Based on the communication information, the first remote operation information is converted. The first remote operation information transmission means (160, S107) transmits the first remote operation information converted by the conversion means to the robot.

  According to the ninth invention, similar to the first invention, the robot speaks the language of the conversation partner's culture in combination with the body movements performed in the partner's culture. The operator can respond appropriately.

  According to a tenth aspect of the present invention, there is provided a robot that performs communication with a human by at least one of body movement and voice by remote operation or autonomous control, a plurality of operation terminals that perform remote operation of the robot via a network, and a network This is a robot remote operation system including a central control device that mediates between the robot and one of the plurality of operation terminals via the controller. The operation terminal includes first remote operation transmission means for transmitting first remote operation information for remotely operating the robot to the robot. The robot includes a storage medium for storing communication information of an operator of the operation terminal, personal information acquisition means for acquiring human personal information, communication information specifying means for specifying communication information from the personal information received by the personal information receiving means, The first remote operation information receiving means for receiving the first remote operation information from the operation terminal, whether the communication information specified by the communication information specifying means matches the communication information of the operator of the operation terminal stored by the storage medium When the judgment result judged by the judgment means and the judgment result judged by the judgment means do not match, the first remote operation information received by the first remote operation information receiving means is changed to the communication information specified by the communication information specifying means. 2nd far based Conversion means for converting the operation information, and an execution means for executing the communication action based on the second remote control information converted by the conversion.

  In the tenth invention, the robot remote control system (10) includes a robot (12) that performs communication with humans through at least one of body movement and voice by remote control or autonomous control, and a robot via a network. A plurality of operation terminals (16) for performing remote operation, and a central control device (14) for mediating between the robot and one of the plurality of operation terminals via a network. First remote operation transmission means (160, S413) of the operation terminal transmits first remote operation information for remotely operating the robot to the robot. The storage medium (138) of the robot stores communication information of the operator of the operation terminal. Personal information acquisition means (66, 80, 116, S305) acquires human personal information. The communication information specifying means (80, S37, S53, S55, S71, S73) specifies communication information from the personal information received by the personal information receiving means. The first remote operation information receiving means (80, 112, 114, S313) receives the first remote operation information from the operation terminal. The determining means (80, S61-S65) determines whether or not the communication information specified by the communication information specifying means matches the communication information of the operator of the operating terminal stored by the storage medium. The conversion means (80, S103, S105, S129, S159) converts the first remote operation information received by the first remote operation information reception means when the judgment results judged by the judgment means do not match, the communication information identification means Is converted into second remote control information based on the communication information specified by. The execution means (80, S317) executes a communication action based on the second remote operation information converted by the conversion means.

  In the tenth aspect, similar to the first aspect, the robot speaks the language of the conversation partner's culture with the body movements performed in the partner's culture. Can respond appropriately.

  According to the present invention, the robot speaks the language of the conversation partner's culture with the body movement performed in the partner's culture, so that the operator can appropriately respond regardless of the conversation partner's culture.

  The above object, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

  Referring to FIG. 1, a robot remote control system 10 of this embodiment includes robots 12a and 12b. The robots 12a and 12b are connected to the central control device 14 and the operation terminals 16a, 16b and 16c via the network 100. Hereinafter, when it is not necessary to distinguish the robots 12a and 12b, they will be referred to as robots 12. Similarly, when there is no need to distinguish each of the operation terminals 16a-16c, they will be referred to as operation terminals 16.

  Although FIG. 1 shows two robots 12a and 12b and three operation terminals 16a to 16c for remotely operating the robot 12 for simplicity, it is not necessary to be limited thereto. One robot 12 or three or more robots 12 may be used. Further, any number of operation terminals 16 may be used as long as they are two or more.

  The robot 12 is an interaction-oriented robot (communication robot), and mainly includes a communication action including at least one of body movement such as gesture gesture and voice with a communication target (communication target) such as the human A. It has a function to execute. The robot 12 is, for example, a reception robot, a customer service robot, and the like. The robot 12 is arranged in various places and situations such as a company reception and a certain event venue, and usually plays a role of route guidance by autonomous control.

  The hardware configuration of the robot 12 will be described with reference to FIG. FIG. 2 is a front view showing the appearance of the robot 12 of this embodiment. The robot 12 includes a carriage 30, and two wheels 32 and one slave wheel 34 for autonomously moving the robot 12 are provided on the lower surface of the carriage 30. The two wheels 32 are independently driven by a wheel motor 36 (see FIG. 3), and the carriage 30, that is, the robot 12 can be moved in any direction, front, back, left, and right. The slave wheel 34 is an auxiliary wheel that assists the wheel 32. Therefore, the robot 12 can move in the arranged space by autonomous control. However, the robot 12 may be fixedly arranged at a certain place.

  A cylindrical sensor attachment panel 38 is provided on the carriage 30, and a large number of infrared distance sensors 40 are attached to the sensor attachment panel 38. These infrared distance sensors 40 measure the distance to the sensor mounting panel 38, that is, the object (human being, obstacle, etc.) around the robot 12.

  In this embodiment, an infrared distance sensor is used as the distance sensor, but an ultrasonic distance sensor, a millimeter wave radar, or the like can be used instead of the infrared distance sensor.

  A body 42 is provided on the sensor mounting panel 38 so as to stand upright. Further, the above-described infrared distance sensor 40 is further provided in the upper front upper portion of the body 42 (a position corresponding to a human chest), and measures the distance mainly to a human in front of the robot 12. Further, the body 42 is provided with a support column 44 extending from substantially the center of the upper end of the side surface, and an omnidirectional camera 46 is provided on the support column 44. The omnidirectional camera 46 photographs the surroundings of the robot 12 and is distinguished from an eye camera 70 described later. As this omnidirectional camera 46, for example, a camera using a solid-state imaging device such as a CCD or a CMOS can be adopted. In addition, the installation positions of the infrared distance sensor 40 and the omnidirectional camera 46 are not limited to the portions, and can be changed as appropriate.

  An upper arm 50R and an upper arm 50L are provided at upper end portions on both sides of the torso 42 (position corresponding to a human shoulder) by a shoulder joint 48R and a shoulder joint 48L, respectively. Although illustration is omitted, each of the shoulder joint 48R and the shoulder joint 48L has three orthogonal degrees of freedom. That is, the shoulder joint 48R can control the angle of the upper arm 50R around each of three orthogonal axes. A certain axis (yaw axis) of the shoulder joint 48R is an axis parallel to the longitudinal direction (or axis) of the upper arm 50R, and the other two axes (pitch axis and roll axis) are orthogonal to the axes from different directions. It is an axis to do. Similarly, the shoulder joint 48L can control the angle of the upper arm 50L around each of three orthogonal axes. A certain axis (yaw axis) of the shoulder joint 48L is an axis parallel to the longitudinal direction (or axis) of the upper arm 50L, and the other two axes (pitch axis and roll axis) are orthogonal to the axes from different directions. It is an axis to do.

  In addition, an elbow joint 52R and an elbow joint 52L are provided at the respective distal ends of the upper arm 50R and the upper arm 50L. Although illustration is omitted, each of the elbow joint 52R and the elbow joint 52L has one degree of freedom, and the angle of the forearm 54R and the forearm 54L can be controlled around the axis (pitch axis).

  A sphere 56R and a sphere 56L corresponding to a human hand are fixedly provided at the tips of the forearm 54R and the forearm 54L, respectively. However, when a finger or palm function is required, a “hand” in the shape of a human hand can be used. Although not shown, the front surface of the carriage 30, a portion corresponding to the shoulder including the shoulder joint 48R and the shoulder joint 48L, the upper arm 50R, the upper arm 50L, the forearm 54R, the forearm 54L, the sphere 56R, and the sphere 56L are respectively A contact sensor 58 (shown generically in FIG. 3) is provided. A contact sensor 58 on the front surface of the carriage 30 detects contact of a person or another obstacle with the carriage 30. Therefore, the robot 12 can detect the contact with the obstacle during its movement and immediately stop the driving of the wheel 32 to suddenly stop the movement of the robot 12. Further, the other contact sensors 58 detect whether or not the respective parts are touched. In addition, the installation position of the contact sensor 58 is not limited to the said site | part, and may be provided in an appropriate position (position corresponding to a person's chest, abdomen, side, back, and waist).

  A neck joint 60 is provided at the upper center of the body 42 (a position corresponding to a person's neck), and a head 62 is further provided thereon. Although illustration is omitted, the neck joint 60 has a degree of freedom of three axes, and the angle can be controlled around each of the three axes. A certain axis (yaw axis) is an axis directed directly above (vertically upward) of the robot 12, and the other two axes (pitch axis and roll axis) are axes orthogonal to each other in different directions.

  The head 62 is provided with a speaker 64 at a position corresponding to a human mouth. The speaker 64 is used for the robot 12 to communicate with humans around it by voice or sound. A microphone 66R and a microphone 66L are provided at a position corresponding to a human ear. Hereinafter, the right microphone 66R and the left microphone 66L may be collectively referred to as a microphone 66. The microphone 66 captures ambient sounds, in particular, the voices of humans who are subjects of communication. Furthermore, an eyeball part 68R and an eyeball part 68L are provided at positions corresponding to human eyes. The eyeball portion 68R and the eyeball portion 68L include an eye camera 70R and an eye camera 70L, respectively. Hereinafter, the right eyeball part 68R and the left eyeball part 68L may be collectively referred to as the eyeball part 68. The right eye camera 70R and the left eye camera 70L may be collectively referred to as an eye camera 70.

  The eye camera 70 captures a human face approaching the robot 12, other parts or objects, and captures a corresponding video signal. The eye camera 70 can be the same camera as the omnidirectional camera 46 described above. For example, the eye camera 70 is fixed in the eyeball unit 68, and the eyeball unit 68 is attached to a predetermined position in the head 62 via an eyeball support unit (not shown). Although illustration is omitted, the eyeball support portion has two degrees of freedom, and the angle can be controlled around each of these axes. For example, one of the two axes is an axis (yaw axis) in a direction toward the top of the head 62, and the other is orthogonal to the one axis and goes straight in a direction in which the front side (face) of the head 62 faces. It is an axis (pitch axis) in the direction to be performed. By rotating the eyeball support portion around each of these two axes, the tip (front) side of the eyeball portion 68 or the eye camera 70 is displaced, and the camera axis, that is, the line-of-sight direction is moved. Note that the installation positions of the speaker 64, the microphone 66, and the eye camera 70 described above are not limited to those portions, and may be provided at appropriate positions.

  As described above, the robot 12 of this embodiment includes independent two-axis driving of the wheels 32, three degrees of freedom of the shoulder joint 48 (6 degrees of freedom on the left and right), one degree of freedom of the elbow joint 52 (2 degrees of freedom on the left and right), It has a total of 17 degrees of freedom, 3 degrees of freedom for the neck joint 60 and 2 degrees of freedom for the eyeball support (4 degrees of freedom on the left and right).

  FIG. 3 is a block diagram showing the electrical configuration of the robot 12. With reference to FIG. 3, the robot 12 includes a CPU 80. The CPU 80 is also called a microcomputer or a processor, and is connected to the memory 84, the motor control board 86, the sensor input / output board 88 and the audio input / output board 90 via the bus 82.

  The memory 84 includes a ROM, an HDD, and a RAM (not shown). In the ROM and the HDD, a control program for controlling the operation of the robot 12 is stored in advance. For example, a detection program for detecting the output (sensor information) of each sensor, a communication program for transmitting / receiving necessary data and commands to / from external computers (such as the central control device 14 and the operation terminal 16), etc. To be recorded. The RAM is used as a work memory or a buffer memory.

  The motor control board 86 is constituted by, for example, a DSP, and controls driving of each axis motor such as each arm, neck joint, and eyeball unit. That is, the motor control board 86 receives control data from the CPU 80, and controls two motors (collectively indicated as “right eyeball motor 92” in FIG. 3) that control the angles of the two axes of the right eyeball portion 68R. Control the rotation angle. Similarly, the motor control board 86 receives control data from the CPU 80, and controls two angles of the two axes of the left eyeball portion 68L (in FIG. 3, collectively referred to as “left eyeball motor 94”). ) To control the rotation angle.

  The motor control board 86 receives control data from the CPU 80, and includes a total of four motors including three motors for controlling the angles of the three orthogonal axes of the shoulder joint 48R and one motor for controlling the angle of the elbow joint 52R. The rotation angle of two motors (collectively indicated as “right arm motor 96” in FIG. 3) is controlled. Similarly, the motor control board 86 receives control data from the CPU 80, and includes three motors for controlling the angles of the three orthogonal axes of the shoulder joint 48L and one motor for controlling the angle of the elbow joint 52L. The rotation angles of a total of four motors (collectively indicated as “left arm motor 98” in FIG. 3) are controlled.

  Further, the motor control board 86 receives control data from the CPU 80, and controls three motors for controlling the angles of the three orthogonal axes of the neck joint 60 (in FIG. 3, collectively indicated as “head motor 110”). Control the rotation angle. The motor control board 86 receives control data from the CPU 80 and controls the rotation angles of the two motors (collectively indicated as “wheel motor 36” in FIG. 3) that drive the wheels 32. In this embodiment, a motor other than the wheel motor 36 uses a stepping motor (that is, a pulse motor) in order to simplify the control. However, a DC motor may be used similarly to the wheel motor 36. The actuator that drives the body part of the robot 12 is not limited to a motor that uses a current as a power source, and may be changed as appropriate. For example, in another embodiment, an air actuator or the like may be applied.

  Similar to the motor control board 86, the sensor input / output board 88 is configured by a DSP and takes in signals from each sensor and gives them to the CPU 80. That is, data relating to the reflection time from each of the infrared distance sensors 40 is input to the CPU 80 through the sensor input / output board 88. The video signal from the omnidirectional camera 46 is input to the CPU 80 after being subjected to predetermined processing by the sensor input / output board 88 as necessary. Similarly, the video signal from the eye camera 70 is also input to the CPU 80. Further, signals from the plurality of contact sensors 58 described above (collectively indicated as “contact sensors 58” in FIG. 3) are provided to the CPU 80 via the sensor input / output board 88. Similarly, the voice input / output board 90 is also configured by a DSP, and voice or voice in accordance with voice synthesis data provided from the CPU 80 is output from the speaker 64. In addition, voice input from the microphone 66 is given to the CPU 80 via the voice input / output board 90.

  The CPU 80 is connected to the communication LAN board 112 via the bus 82. The communication LAN board 112 is configured by a DSP, for example, and provides transmission data given from the CPU 80 to the wireless communication device 114. The wireless communication device 114 sends the transmission data to an external computer (the central controller 14 and the operation via the network 100). To the terminal 16). In addition, the communication LAN board 112 receives data via the wireless communication device 104 and provides the received data to the CPU 80. For example, the transmission data may be an operator call request signal (command) from the robot 12 to the central controller 14.

  Further, the wireless tag reader 116 is connected to the CPU 80 via the bus 82. The wireless tag reader 116 receives a radio wave superimposed with identification information transmitted from the wireless tag 18 (RFID tag) via an antenna (not shown). The wireless tag reader 116 amplifies the received radio wave signal, separates the identification signal from the radio wave signal, demodulates (decodes) the identification information, and provides the CPU 80 with the identification information. According to FIG. 1, the wireless tag 18 is attached to a person (person A) who is at the reception of a certain company where the robot 12 is disposed, and the wireless tag reader 116 detects the wireless tag 18 within the communicable range. To do. Note that the wireless tag 18 may be an active type or a passive type that is driven in accordance with a radio wave transmitted from the wireless tag reader 116.

  The identification information acquired from the wireless tag 18 and the voice of the person (dialogue partner) are transmitted to the central controller 14 as personal information of the dialogue partner.

  Returning to FIG. 1, the central control device 14 is a computer that controls the call of the operator in the robot remote operation system 10, and manages information indicating the states of the robot 12 and the operation terminal 16. And when the operation terminal 16 and the robot 12 communicate, the content of data communication is controlled. Referring to FIG. 4, central control device 14 includes a CPU 130. The CPU 130 is also called a microcomputer or a processor, like the CPU 80, and via the bus 132, the memory 134, the robot information database (DB) 136, the operator terminal information DB 138, the conversation partner information DB 140, the language / culture DB 142, and the operation determination DB 144. And connected to the translation dictionary DB 146.

  The memory 134 includes a ROM, an HDD, and a RAM (not shown). In the ROM and the HDD, a control program for controlling the operation of the central controller 14 is stored in advance. The control program includes, for example, a communication program for transmitting and receiving necessary data and commands between the robot 12 and the operation terminal 16, and an appropriate operator (operation terminal 16 when an operator call request is received from the robot 12. ) Including a selection program for selecting. The RAM of the memory 134 is used as a work memory or a buffer memory.

  The robot information DB 136 stores a table (robot information table) indicating the current state of each robot 12. As shown in FIG. 5A, in the robot information table, information such as a state, a position, and a conversation partner is stored in correspondence with the robot name. The name of the robot 12 (for example, R1, R2,...) Is stored in the robot name column. In the status column, the current status of the corresponding robot 12 is stored. Specifically, “BUSY” or “IDLE” is described. BUSY means a state in which a communication action such as dialogue with a person is executed at a reception of a certain company. IDLE means a state in which a communication action such as dialogue with a person is not executed, that is, a free state.

  Information on the current position of the robot 12 corresponding to the robot name is stored in the position column. In this embodiment, information indicating a place such as a building or a venue where the robot 12 is arranged and coordinates of a position where the robot 12 currently exists in the place (position coordinates in the world coordinate system) are registered.

  In the dialogue partner column, information on the person who is currently performing communication behavior by the robot 12 corresponding to the robot name is registered. Specifically, the name (for example, Taro Yamada), the language used (for example, Japanese), and the current position (the position coordinates in the world coordinate system) are registered in association with the identification information (dialogue partner ID) of the conversation partner. The In addition, when the robot 12 corresponding to the robot name is not interacting with a human, “NULL” indicating that there is no target for communication behavior is registered.

  The central control device 14 obtains information indicating the state of the robot 12 or the operation terminal 16 at regular intervals through communication with the robot 12 or the operation terminal 16, and creates and updates the robot information table.

  The operator terminal information DB 138 stores the current state and skills of each operator. As shown in FIG. 5B, the operator terminal information table stores information such as name, state, location, and language in correspondence with the operator ID. In the operator ID column, identification information for identifying the operation terminal 16 or an operator who operates the operation terminal 16 is registered. In this embodiment, the operator ID is represented by a numerical value. In the name column, the name of the operator who operates the operation terminal 16 corresponding to the operator ID is registered. In the status column, the current status of the operation terminal 16 is registered corresponding to the operator ID. Specifically, “IDLE” and “BUSY-R1” are described. “IDLE” means a state of waiting for a call from the central control device 14 (standby state) without currently remotely operating the robot 12. “BUSY” means a state where the robot 12 is remotely operated. For example, when “BUSY-R1” is described, the robot 12 having the robot name “R1” is remotely operated. I understand that there is. In the location column, the location (prefecture or region) where the operation terminal 16 is installed is registered corresponding to the operator ID. In the language column, languages (Japanese, English, etc.) that the operator who operates the operation terminal 16 corresponding to the operator ID can speak are registered.

  The central control device 14 acquires information indicating the state of the robot 12 or the operation terminal 16 at regular intervals through communication with the robot 12 or the operation terminal 16, and creates and updates an operator terminal information table.

  In the robot information table, the address (IP address) of the robot 12 may also be registered. Although illustration is omitted, when the operator does not operate the operation terminal 16 (in the case of being offline), “NULL” (that is, a state in which remote operation is not possible) may be registered.

  The dialogue partner information DB 140 stores a table indicating information of a dialogue partner with whom the robot 12 performs communication behavior. As shown in FIG. 6A, information such as name, language, and culture is stored in the dialogue partner information table based on the dialogue partner ID. In the dialog partner ID column, identification information for identifying a dialog partner (human) with which the robot 12 performs communication behavior is stored. Further, the conversation partner ID is represented by a numerical value, like the operator ID. In the name column, a human name identified by the conversation partner ID is registered. In the language column, a language that can be spoken by a person identified by the conversation partner ID is registered. In the culture column, the country name of the person's country of origin is stored as the person's culture identified by the conversation partner ID. For example, Japan, France, USA, Canada, etc. are stored in the culture column. In other words, Japan indicates that the conversation partner is from Japan. France indicates that the conversation partner is from France. USA indicates that the conversation partner is from the United States. Canada indicates that the conversation partner is from Canada.

  In this embodiment, information composed of language information and culture information is referred to as “communication information”.

  The language / culture DB 142 stores a table indicating the correspondence between languages and cultures. As shown in FIG. 6B, language and culture information is stored in the language / culture table. Here, the culture column stores a culture corresponding to the language stored in the language column. For example, if Japanese is stored in the language column, Japan indicating Japanese culture is stored in the culture column. If English is stored in the language column, USA indicating the United States is stored in the culture column. Further, if Chinese is stored in the language column, China indicating Chinese culture is stored in the culture column. If French is stored in the language column, France indicating French culture is stored in the culture column. In the language column, languages that can be spoken by the robot 12 are stored.

  The action determination DB 144 stores a table for determining an action file corresponding to the action command according to the culture of the conversation partner when the action command is transmitted from the operation terminal 16. In this embodiment, as shown in FIG. 7, the operation and culture are stored in correspondence with the operation file name. In the operation column, an operation command executed by the robot 12 is stored corresponding to the operation file name. For example, in the operation column, an “invitation” operation or an operation command of “bow” is stored. In the culture column, the culture is stored corresponding to the operation file name. For example, the culture column stores “USA” (in the United States) and “Japan” (Japan).

  A specific file name of an operation command transmitted to the robot 12 is stored in the operation file column. For example, “MOVE_temaneki_japan.txt” is a file in which the robot 12 performs the same operation as the beckoning operation performed by the Japanese. “MOVE_temaneki_usa.txt” is a file in which the robot 12 performs the same operation as the beckoning operation performed by an American.

  Further, “MOVE_ojigi_japan.txt” is a file in which the robot 12 performs the same operation as the bowing operation performed by the Japanese. “MOVE_ojigi_usa.txt” is a file in which the robot 12 performs the same operation as the bowing operation performed by an American.

  The reason why the operation commands transmitted from the central control device 14 to the robot 12 are different is that gesture gestures may differ depending on the culture of the conversation partner even in the case of the “invitation” operation. For example, when a Japanese beckones, the palm invites the palm downward, but when an American invites, the palm invites the palm upward. When Japanese greets, they bow, but in other countries they hug, shake hands, and kiss.

  Further, when the robot 12 receives an operation file indicating an operation command from the central control device 14, the robot 12 refers to a library (not shown) stored in the memory 84 and executes an operation corresponding to the operation file. In this library, a command statement (machine language or the like) for controlling the motor control board 86 is recorded corresponding to the operation file name received by the robot 12.

  For example, when the robot 12 receives an operation file of “MOVE_temaneki_usa.txt” as an operation command, the CPU 80 of the robot 12 refers to the library and controls the motor control board 86 according to a command statement corresponding to “MOVE_temaneki_usa.txt”. Then, the right arm motor 96 of the robot 12 is operated.

  Note that the motion file name identifies the culture of the United States and the culture of Japan, but the robot 12 does not need to identify the culture difference from the received motion file name.

  However, the operation terminal 16 transmits not only the operation command but also a control command including the operation command and the speech command to the central control device 14. As described above, the operation command is an operation file name starting with “MOVE_...”. The utterance command is an utterance file name that starts with “SAY_...”. For example, the speech file name of the speech command to the speech to the robot 12 and the "Hello" in Japanese is "SAY_jpn_ Hello .txt". Also, the utterance command name of the utterance command for causing the robot 12 to utter “good weather” is “SAY_jpn_good weather. Txt”. Furthermore, the utterance command name of the utterance command for causing the robot 12 to utter “Come here” is “SAY_jpn_Come here.txt”. Then, the utterance command name of the utterance command for causing the robot 12 to utter “Let's play together” is “SAY_jpn_Let's play together. Txt”.

  Returning to FIG. 4, the translation dictionary DB 146 is a DB that is referred to when the central controller 14 translates speech data, character string data, and the like. Although illustration is omitted, the translation dictionary DB 146 includes a Japanese translation dictionary, an English translation dictionary, a Chinese translation dictionary, a French translation dictionary, and the like.

  The central control device 14 is connected to the network 100 by wire or wirelessly. That is, similar to the robot 12, the CPU 130 is connected to the communication LAN board 148 via the bus 132. The communication LAN board 148 is configured by, for example, a DSP, and provides transmission data given from the CPU 130 to the wireless communication device 150. The wireless communication device 150 sends the transmission data to an external computer (the robot 12 and the operation terminal 16 via the network 100). ).

  The communication LAN board 148 also receives data via the wireless communication device 150 and provides the received data to the CPU 130. For example, the transmission data may be a remote operation request signal from the central control device 14 to the operation terminal 16 or voice data transmitted from the robot 12. The received data may be a control command to the robot 12 transmitted from the operation terminal 16 or the like.

  The memory 134 or the DB (not shown) of the central control device 14 stores map (map) data such as a place where the robot 12 is arranged, and the map data is represented by XY coordinates. Expressed in coordinates. However, since the robot 12 moves, the amount of movement control may be obtained from the robot 12, or the position of the robot 12 may be monitored using a separate environmental sensor (camera, infrared sensor, etc.).

  The operation terminal 16 shown in FIG. 1 is a computer for remotely operating the robot 12 by an operator, and a general-purpose computer can be used. As shown in FIG. 8, the operation terminal 16 includes a CPU 160, to which an LCD driver 162, a memory 164, a speaker 166, a microphone 168, an input device, 170 and a communication LAN board 172 are connected. An LCD 176 is connected to the LCD driver 162.

  The LCD driver 162 is a driving device for the LCD 176 and displays an image on the LCD 176 in accordance with an instruction from the CPU 160. In this embodiment, the LCD 176 is mainly used to display a remote operation screen 210 (see FIG. 9) as will be described later. Although not shown, the memory 164 includes a ROM, an HDD, and a RAM. In the ROM and the HDD, a control program for controlling the operation of the operation terminal 16 is stored. The control program includes, for example, a communication program for transmitting and receiving necessary data and commands between the robot 12 and the central control device 14, a detection program for detecting an operation command input from the input device, and an image on the display device. Includes a display program for display. The RAM of the memory 164 is used as a work memory or a buffer memory.

  The speaker 166 is used for outputting voice data transmitted from the central control device 14 to the operator. The microphone 168 is used to capture the operator's voice. That is, the speaker 166 outputs the human voice detected mainly through the microphone 66 of the robot 12 that is remotely operated. Furthermore, the microphone 168 mainly detects the operator's voice, and the operator's voice is output through the speaker 64 of the robot 12.

  The input device 170 is a computer mouse, a keyboard, a touch panel, or the like. For example, the operator can input a remote operation command for remotely operating the robot 12 by operating the input device while viewing the remote operation screen 210 displayed on the LCD 176.

  The communication LAN board 172 is constituted by, for example, a DSP, and provides transmission data given from the CPU 130 to the wireless communication device 174. The wireless communication device 174 sends the transmission data to an external computer (the robot 12 and the central control device via the network 100). 14).

  In addition, the communication LAN board 172 receives data via the wireless communication device 174 and gives the received data to the CPU 160. For example, the transmission data may be a control command for remotely operating the robot 12. The received data may be a remote operation request signal transmitted from the central controller 14 or voice data.

  Referring to FIG. 9, on this remote operation screen 210, for example, a robot camera image 212, robot information 214, and an operation panel 216 are displayed. In the robot camera image 212, a captured image of the eye camera 70 received from the robot 12 is displayed. Thus, the operator can see an image captured by the eye camera 70 of the robot 12, for example, a human being interacting in real time.

  In the robot information 214, information on the robot 12, the situation, information on the conversation partner, and the like are displayed. Specifically, the location where the robot 12 is placed (such as a reception of a certain company or an event venue), the name (or identification information), a map of the placed location, and information of the person who is interacting are displayed. . On the map, icons indicating the robot 12 and the conversation partner are displayed corresponding to each location. Thus, the operator can easily grasp where the robot 12 is located and where the conversation partner is with respect to the robot 12.

  The operation terminal 16 may store map information in advance or may receive the map information together with information related to the robot 12 from the central control device 14. The positions of the robot 12 and the conversation partner are transmitted from the robot 12 to the operation terminal 16 at regular intervals, for example. Since the robot 12 stores the initial position and orientation, the robot 12 always knows its current position and orientation, and estimates the approximate position of the conversation partner from the output information of the infrared distance sensor 40 and its own position. it can. In addition, an icon indicating a person other than the conversation partner existing at the place may be displayed on the map. In this case, the operator can know the situation of the robot 12 at the location in more detail.

  In addition, the position information of the person at the place where the robot 12 is arranged is obtained by using an environmental sensor such as a ceiling camera or a wireless tag reader 116 installed in the surrounding area by another computer connected to the environmental sensor. It may be detected, given to the robot 12 from the other computer, and transmitted from the robot 12 to the operation terminal 16. Alternatively, the position information may be transmitted from the other computer described above to the operation terminal 16 via the central control device 14. In addition, since the position of the robot 12 and the like can be detected by the environmental sensor, the robot 12 may acquire information such as its own position and orientation from the above-mentioned other computer together with the human position information.

  Further, as the information of the person who is interacting, the name and state (for example, VIP or visitor) are displayed. As a result, the operator can know information on the conversation partner of the robot 12. Information on the conversation partner is transmitted from the central controller 14 to the operation terminal 16. Note that the information presented to these operators is an example, and appropriate information is appropriately displayed in the robot information 214. With such robot information 214, the operator can easily grasp the information of the robot 12 and the conversation partner and the situation where both are placed.

  Further, when the robot 12 and the communication partner communicate with each other, voice information is transmitted from the robot 12 and the voice is output from the speaker 166 of the operation terminal 16. Therefore, the operator can more easily grasp the current situation by listening to the conversation partner's words.

  The question of the conversation partner is acquired by the microphone 66 of the robot 12 and is output from the speaker 166 via the network 100 and the central control device 14. When the operator hears the question asked by the conversation partner, the operator responds by answering the question of the conversation partner with the communication behavior corresponding to the operation panel 216 or the operator's own voice. In other words, the operator can easily determine the content of the question from the conversation partner while confirming the status of the robot 12 on the remote operation screen 210, and answer the question to the conversation partner via the robot 12. Can do.

  On the operation panel 216, a command for controlling the operation of the robot 12 is input by the operator operating the input device. For example, as shown in FIG. 9, the operation panel 216 is provided with a button for instructing movement, a button for instructing an operation, a button for instructing the end of remote operation, and the like. Specifically, the “forward” button is a button for moving the robot 12 forward. When the operator selects the “forward” button, the robot 12 moves forward, for example, by a certain distance. Similarly, the “reverse” button causes the robot 12 to retract, the “right” button causes the robot 12 to turn right, and the “left” button causes the robot 12 to turn left. The “Let's play” button is a button for transmitting a control command “Let's play” to the robot 12 and executing a communication action that invites the conversation partner to play. Specifically, in the communication action to invite to play, the robot 12 speaks “Please come over here”, performs a beckoning operation, and then speaks “Let's play together”.

  The “greeting” button is a button for transmitting a “greeting” control command to the robot and causing a communication action to greet the conversation partner. Specifically, the communication behavior to the greeting, speaks robot 12 as "Hello", after performing the operation of the bow, speaks "It is nice weather". As described above, the beckoning and bowing operations are different depending on the culture of the conversation partner. In addition, the “greeting” button includes a “greeting (morning)” button where the lot 12 speaks “good morning” and a “greeting (evening)” button where the robot 12 speaks “good evening”. May be.

  The “right pointing” button is a button for causing the robot 12 to perform an operation of pointing to the right. Specifically, when the operator selects the “right finger” button, the robot 12 controls the right shoulder joint 48R and the right elbow joint 52R to raise the arm forward. Similarly, when the operator selects another button displayed on the operation panel 216, the robot 12 can perform operations such as nodding and pointing to the left. The “end” button is a button for ending the communication action by remote operation. When the operator selects the “end” button, the robot 12 shifts from the remote operation mode to the autonomous control mode.

  Here, the control command will be described. The control command is composed of at least one of an operation command and an utterance command. For example, the control command of "greeting" is spoken command "Hello", so as to be executed in the order of the operation command "bow" and the spoken command "It is nice weather", consists of three commands. In addition, the control command of “Let's play” is composed of three commands so as to be executed in the order of the utterance command “Come here”, the operation command “Invite” and the utterance command “Let's play together”. . Furthermore, the utterance command included in the control command can be replaced with the voice input by the operator to be a control command. For example, in the control command for “greeting”, if the operator inputs a voice to the microphone 168 “Is there anything you want to do?” And replaces it with the speech command “good weather”, the control command for “greeting” command "Hello" so as to be executed in the order of the operation command "bow" and the spoken command "Do you have any patronage", will be composed of three commands.

  The robot 12 calls an operator in situations where it is difficult to respond by autonomous control alone or when a more detailed response (communication or the like) is required by a human. Specifically, the robot 12 transmits an operator call request (remote operation request) to the central control device 14 via the network 100. In response to the call request from the robot 12, the central control device 14 selects the operation terminal 16 operated by an operator who can respond (remote operation). The central control device 14 notifies the selected operation terminal 16 of the communication information (IP address) of the robot 12. Then, the robot 12 and the operation terminal 16 are communicably connected via the network 100 and the central control device 14 by the operation of the operator of the operation terminal 16. That is, the central controller 14 mediates between the robot 12 and the operation terminal 16. Thereafter, the operator remotely operates the robot 12 using the operation terminal 16. For example, a control command input and transmitted by the operator using the operation terminal 16 is relayed by the central controller 14 and received by the robot 12. Then, the robot 12 performs an operation (communication behavior) according to the received control command.

  Here, although not shown in the figure, the robot 12 stores in advance a condition for calling an operator (operator calling condition) in the memory 84. When it is determined that the operator calling condition is satisfied, the robot 12 calls the operator. That is, the robot 12 transmits an operator call request to the central controller 14.

  When receiving a call request from the operator, the central control unit 14 refers to the dialog partner information table and identifies communication information of the dialog partner in correspondence with the dialog partner ID if the dialog partner is identified. On the other hand, if the conversation partner has not been identified, the language of the conversation partner is identified from the voice of the conversation partner, and the language / culture table (see FIG. 6B) is referred to. Identify the culture corresponding to the language column. Further, the central control device 14 refers to the operator terminal information table, selects an operator in the IDLE state, and specifies the language of the operator corresponding to the operator ID in the IDLE state.

  When the conversation partner and the operator have different languages, when the conversation partner's voice is transmitted from the robot 12 to the operation terminal 16, the central control unit 14 executes the translation process, thereby The voice is translated into the language of the operator and transmitted to the operation terminal 16 used by the selected operator. When a control command is transmitted to the robot 12 from the operation terminal 16 used by the selected operator, the central control device 14 translates the utterance command into the language of the conversation partner by translation processing, and the operation command is a dialogue. It is converted into an action of culture corresponding to the other party and transmitted to the robot 12.

  When the language of the conversation partner and the operator match, when the voice of the conversation partner is transmitted from the robot 12 to the operation terminal 16, the central controller 14 receives the speech without executing the translation process. The transmitted voice is transmitted as it is to the operation terminal 16 used by the selected operator. When a control command is transmitted to the robot 12 from the operation terminal 16 used by the selected operator, the central control device 14 transmits the received control command to the robot 12 as it is.

  Thereby, the robot 12 can perform a communication action in accordance with the language and culture of the conversation partner.

  FIG. 10 is an illustrative view showing one example of a memory map 300 of the memory 134 of the central controller 14 shown in FIG. As shown in FIG. 10, the memory 134 includes a program storage area 302 and a data storage area 304. The program storage area 302 stores a communication program 310, an overall control program 312, a translation processing program 314, and the like. However, the overall control program 312 includes an operator selection program 312a and a communication content control program 312b.

  The communication program 310 is a program for performing communication with the robot 12 and the operation terminal 16. The overall control program 312 updates the information stored in the robot information table and the operator terminal information table shown in FIGS. 5A and 5B at regular intervals, and in response to a call from the robot 12 to the operator, Is a program for controlling the content of communication between the robot 12 and the operation terminal 16 used by the selected operator. However, the operator selection program 312 a is a program that calls an operator in response to an operator call from the robot 12. When the communication between the robot 12 that called the operator and the operation terminal 16 used by the operator is performed, the communication content control program 312b performs communication from the robot 12 to the operation terminal 16 and communication from the operation terminal 16 to the robot 12. It is a program that controls communication contents.

  The translation processing program 314 is a program that is executed when the speech of the conversation partner is translated. For example, when translation processing that translates from Japanese to English is executed, the CPU 130 of the central controller 14 The language information of Japanese and English is read with reference to the DB 146, and the Japanese character string is translated into the English character string.

  The data storage area 304 stores dialogue partner language data 330, operator language data 332, and culture data 334, and is provided with a translation flag 336 and the like. The conversation partner language data 330 is data indicating the language used by the conversation partner. The operator language data 332 is data indicating the language used by the operator. The culture data 334 is information indicating the culture of a conversation partner who performs communication with the robot 12. Specifically, the culture corresponding to the conversation partner is specified, and information on the specified culture is stored in the data storage area 304 as culture data 334.

  The translation flag 336 is a flag for determining whether or not the language used by the conversation partner matches the language used by the operator. For example, the translation flag 336 is composed of a 1-bit register. When the translation flag 336 is established (turned on), a data value “1” is set in the register, and when the translation flag 336 is not established (turned off), a data value “0” is set in the register.

  Specifically, the CPU 130 of the central control device 14 executes a plurality of processes in parallel, including the processes shown in FIGS. The CPU 80 of the robot 12 executes a plurality of processes in parallel including the processes shown in FIGS. 17 and 18.

  As shown in FIG. 11, when starting the overall processing, the CPU 130 of the central control device 14 acquires the state of each operation terminal 16 and updates the table in step S1. That is, in this step S1, information including its position, operator information, current situation, etc. is obtained from each operation terminal 16 connected to the network 100, and the operator terminal information table shown in FIG. Update. In the next step S3, the state of each robot 12 is acquired and the table is updated. That is, in this step S3, information including the position of each robot 12, the current operation state, information on the conversation partner, and the like are acquired, and the robot information table shown in FIG. 5A is updated.

  Subsequently, in step S5, it is determined whether or not there is a call from the robot 12. That is, it is determined whether an operator call request from a certain robot 12 has been received. If “NO” in the step S5, the process proceeds to a step S15. On the other hand, if “YES” in the step S5, that is, if there is a call from the robot 12, an operator selection process (see FIGS. 12 and 13) described later is performed in a step S7. In the next step S9, it is determined whether or not communication between the operation terminal 16 and the robot 12 has been performed. For example, it is determined whether communication is performed with respect to the operation terminal 16 used by the operator selected from the robot 12 or whether communication is performed with respect to the robot 12 from the operation terminal 16.

  If “YES” in the step S9, that is, if communication between the operation terminal 16 and the robot 12 is performed, a communication content control process (see FIG. 14) described later is performed in a step S11, and the process proceeds to a step S13. If “NO” in the step S9, that is, if the communication between the operation terminal 16 and the robot 12 is not performed, the process proceeds to a step S13 as it is.

  In step S13, it is determined whether there is a remote operation termination request. If “NO” in the step S13, that is, if there is no remote operation end request, the process returns to the step S9. On the other hand, if “YES” in the step S13, that is, if there is a remote operation end request, the process proceeds to a step S15.

  In step S15, it is determined whether or not to end the entire process. Here, for example, it is determined whether or not an end instruction is given from the user of the robot remote control system 10 or the administrator of the central controller 14. If “YES” in the step S15, that is, if there is an end instruction, the entire process is ended as it is. On the other hand, if “NO” in the step S15, that is, if there is no end instruction, the process returns to the step S1. As described above, in the central control device 14, the update of the operator operation terminal table, the update of the robot information table, and the determination of the presence / absence of a call request are repeatedly executed at regular intervals.

  12 and 13 are flowcharts of the operator selection process in step S7 shown in FIG. As shown in FIG. 12, when starting the operator selection process, the CPU 130 of the central controller 14 initializes variables in step S31. Specifically, 0 is set to each of the variable jpn and the variable eng. Here, the variable jpn stores the calculation result of the Japanese-likeness calculation from the voice of the person who is interacting with the robot 12, and the variable eng stores the calculation result of the English-likeness calculation from the voice of the person who is talking with the robot 12. It is a variable to do. The Japanese-likeness calculation and the English-likeness calculation will be described in detail in steps S43 and S47.

  Subsequently, in step S33, it is determined whether or not the conversation partner has been identified. That is, it is determined whether or not information such as the name, age, and sex of the conversation partner with whom the robot 12 communicates is known. As described above, the robot 12 receives the tag ID transmitted from the wireless tag 18 possessed by the conversation partner and transmits it to the central controller 14. The CPU 130 of the central control device 14 refers to the conversation partner information DB 140 and acquires information such as the name, age, and sex of the conversation partner corresponding to the tag ID.

  If “YES” in the step S33, that is, if the conversation partner is identified, it is determined whether or not the language used by the conversation partner has been specified in a step S35. That is, referring to the conversation partner information DB 140, it is determined whether or not the language information spoken by the conversation partner is stored in the conversation partner information table. If “NO” in the step S35, that is, if the main language of the conversation partner has not been specified, the process proceeds to a step S39. On the other hand, if “YES” in the step S35, that is, if the main language used by the conversation partner has been specified, the conversation partner language data 330 is read from the conversation partner information DB 140 in a step S37, and the process proceeds to a step S57 shown in FIG. move on. That is, in this step S 37, the language type data corresponding to the name of the conversation partner identified as the conversation partner (dialog partner ID) is stored in the memory 134 as the conversation partner language data 330.

  If “NO” in the step S33, that is, if the conversation partner is not identified, it is determined whether or not the robot 12 has a voice input in a step S39. That is, the CPU 130 of the central control device 14 determines whether or not the voice of the conversation partner is detected by the robot 12 and a voice signal corresponding to the detected voice is transmitted from the robot 12 and received.

  If “NO” in the step S39, that is, if there is no voice input to the robot 12, the robot 12 is instructed to output a voice in a step S41, and the process returns to the step S39. In step S41, for example, a command for causing the robot 12 to output a voice that causes the conversation partner to speak is sent to the robot 12, such as "What is it for?" In this way, the CPU 130 of the central control device 14 listens to the conversation partner's question (request) and identifies the language of the conversation partner.

  On the other hand, if “YES” in the step S39, that is, if there is a voice input to the robot 12, a Japanese-likeness calculation is performed in a step S43. Here, the Japanese-likeness calculation is an operation for obtaining a ratio of how close the language of the conversation partner of the robot 12 is to Japanese. Specifically, the CPU 130 of the central control unit 14 recognizes the received voice signal by the DP matching method or HMM (Hidden Markov Model) method using a Japanese dictionary, and the recognition result and its score (approximation degree). Or a ratio indicating accuracy). In the next step S45, the calculation result is stored in the variable jpn. Here, among the results calculated as described above, the score is stored in the variable jpn as Japanese-likeness.

  In the subsequent step S47, English-likeness calculation is performed. Here, the CPU 130 of the central controller 14 uses the English dictionary to perform voice recognition in the same manner as in step S43, and obtains a recognition result and its score. In step S49, the calculation result (score) is stored in the variable eng as English-likeness.

  In the next step S51, it is determined whether or not the variable jpn is larger than the variable eng. That is, here, the CPU 130 of the central controller 14 determines whether the language of the conversation partner of the robot 12 is close to Japanese or English. If “YES” in the step S51, that is, if the variable jpn is larger than the variable eng, it is determined that the language of the conversation partner of the robot 12 is closer to Japanese than English, and “Japanese” is determined to be the conversation partner in step S53. The language data 330 is stored, and the process proceeds to step S57. That is, it is specified that the language used by the conversation partner is Japanese. On the other hand, if “NO” in the step S51, that is, if the variable jpn is equal to or less than the variable eng, it is determined that the language of the conversation partner of the robot 12 is closer to English than Japanese, and “English” is changed to the language in the step S55. Store as data 330 and proceed to step S57.

  In step S57 shown in FIG. 13, an operator in the IDLE state is selected. That is, an operator in which IDLE is stored is selected in the status column of the operator terminal information table (see FIG. 5B). When there are a plurality of operators in which IDLE is stored, the operator with the lowest operator ID may be selected, the operator with the highest operator ID may be selected, or may be arbitrarily selected. .

  In the next step S59, the operator language data 332 is stored based on the language information used by the operator. In this step S59, referring to the operator terminal information table of the operator terminal information DB 138, the language type data stored in the language column corresponding to the selected operator is stored in the memory 134 as operator language data 332.

  When the operator can handle a plurality of languages, that is, when two or more languages are stored in the language column of the operator terminal table, the data on all the language types is stored in the operator language data 332. Is stored in the memory 134.

  Subsequently, in step S61, it is determined whether or not the conversation partner language data 330 and the operator language data 332 match. That is, it is determined whether or not the language used by the conversation partner matches the language used by the operator. However, when the operator uses a plurality of languages, it is determined whether or not the conversation partner language is included in the plurality of languages.

  If “YES” in the step S61, that is, if the language data 330 and the operator language data 332 match, the translation flag 336 is set to off in a step S63. On the other hand, if “NO” in the step S61, that is, if the language data 330 and the operator language data 332 do not match, the translation flag 336 is set to ON in a step S65, and the conversation partner is identified in a step S67. Determine whether it is done.

  If “YES” in the step S67, that is, if the conversation partner is identified, it is determined whether or not the conversation partner's culture has been specified in a step S69. That is, in step S69, it is determined by referring to the dialogue partner information table of the dialogue partner information DB 140 whether or not culture information is stored corresponding to the dialogue partner. If “NO” in the step S69, that is, if the culture of the conversation partner has not been specified, the process proceeds to a step S73. On the other hand, if “YES” in the step S69, that is, if the culture of the conversation partner has been specified, the culture data 334 is stored based on the conversation partner information DB 140 in a step S71. That is, in this step S71, culture information (data to be identified) corresponding to the name of the conversation partner (dialog partner ID) identified as the conversation partner is stored in the memory 134 as the culture data 334.

If “NO” in the step S67, that is, if the conversation partner is not identified, the culture data 334 is stored on the basis of the conversation partner language data 330 and the language / culture DB 142 in a step S73. For example, in this step S73, if the conversation partner language data 330 indicates English, the language / culture table indicates that the culture is USA, that is, the culture of the United States of America.
That is, in this step S71, USA is stored in the memory 134 as culture data 334.

  In step S75, the operation terminal 16 used by the selected operator is introduced to the robot 12, and the process returns to the entire process shown in FIG. For example, the CPU 130 of the central controller 14 transmits connection information (IP address) of the operation terminal 16 used by the selected operator to the robot 12. At the same time, the remote operation request of the robot 12 and the connection information (IP address) of the robot 12 that transmitted the call request are transmitted to the operation terminal 16 used by the selected operator.

Although not shown, in step S75, the state of the operation terminal 16 used by the selected operator is updated from “IDLE” to “BUSY-Rm” in the operator terminal information table. Here, m is a natural number (0, 1, 2,... M), and “Rm” is a robot name or robot 12 identification information. Thus, it can be seen that the operator is operating the robot Rm remotely. FIG. 14 is a flowchart showing the communication content control process of step S11 shown in FIG. As shown in FIG. 14, when starting the communication content control process, the CPU 130 of the central control device 14 determines whether or not voice data has been received from the robot 12 in step S91. For example, when the robot 12 speaks from the conversation partner, the robot 12 detects the voice of the conversation partner by the microphone 66, acquires the corresponding voice data, and transmits the acquired voice data to the central control device 14. Therefore, in the process of step S91, it is determined whether or not voice data corresponding to the voice of the conversation partner of the robot 12 has been received. If “NO” in the step S91, that is, if voice data is not received from the robot 12, the process proceeds to a step S99 as it is. On the other hand, if “YES” in the step S91, that is, if voice data is received from the robot 12, it is determined whether or not the translation flag 336 is off. If “YES” in the step S93, that is, if the language used by the conversation partner matches the language used by the operator, the process proceeds to a step S99. On the other hand, if “NO” in the step S93, that is, if the translation flag 336 is turned on, the speech data is translated based on the operator language data 332 in a step S95. That is, the question content from the conversation partner is translated into the language used by the selected operator.

  Specifically, the dictionary of the language indicated by the conversation partner language data 330 is read from the translation dictionary DB 146, and speech recognition is performed using the read dictionary by the DP matching method or the HMM (Hidden Markov Model) method. Subsequently, the speech-recognized speech data is subjected to a known morphological analysis process, and each word constituting the sentence is read out. Then, the CPU 130 executes translation processing, reads out the dictionary of each language indicated by the conversation partner language data 330 and the operator language data 332 from the translation dictionary DB 146, and translates the read sentence into the language indicated by the operator language data 332. To do. Further, as a translation function (translation system) including a speech recognition function, a Japanese-English speech translation system “ATR-MATRIX” can be used. The details of “ATR-MATRIX” are detailed in “Reaves et al.,“ ATR-MATRIX: A Speech Translation System between English and Japanese ”, IPSJ Transactions, Vol. 2, pp.87-88, 1999”. Is disclosed.

  In the next step S97, the translation result of the voice data is transmitted to the operation terminal 16 as character string data, and the process proceeds to step S99. That is, in this step S97, the translated voice data result is replaced with the character string data and transmitted to the selected operation terminal 16. On the other hand, when the operation terminal 16 receives the character string data, the operation terminal 16 displays a character string corresponding to the character string data on the LCD 176.

  Note that the operation terminal 16 may output the received character string data from the speaker 166 by using the Text To Speech function (TTS function). Further, the translation result may be transmitted to the operation terminal 16 as voice data not by character string data but by synthesized voice. In this case, if the operation terminal 16 corresponds to the received audio data, the operation terminal 16 may output audio corresponding to the audio data from the speaker 166. In addition, the TTS function needs to specify the language to be read in advance before reading the text.

  Here, the text-to-speech synthesis system “XIMERA” can be used as the TTS function. About this “XIMERA”, “Speech synthesis system XIMERA using large-scale corpus”, IEICE Transactions, vol. J89-D, no. 12, pp. 2688-2698 (2006-12). Are disclosed in detail.

  In step S99, it is determined whether a control command has been received from the operation terminal 16. If “NO” in the step S99, that is, if a control command is not received from the operation terminal 16, the process returns to the entire process. On the other hand, if “YES” in the step S99, that is, if a control command is received from the operation terminal 16, it is determined whether or not the translation flag 336 is turned off in a step S101. If “YES” in the step S101, that is, if the translation flag 336 is turned off, the process proceeds to a step S105 as it is. On the other hand, if “NO” in the step S101, that is, if the translation flag 336 is turned on, an utterance command translation process (see FIG. 15) described later is executed in a step S103, and the process proceeds to a step S105.

  In step S105, an operation command conversion process (see FIG. 16), which will be described later, is executed, a control command is transmitted to the robot 12 in step S107, and the process returns to the overall process shown in FIG. That is, when voice data is received from the robot 12 with the translation flag 336 turned on, the text is read by recognizing the voice data, translated into the language indicated by the operator language data 332, and sent to the operation terminal 16. Send. Further, when a control command is received from the operation terminal 16, the utterance content of the utterance command included in the control command is translated into the language indicated by the conversation partner language data 330 and transmitted to the robot 12. In addition, regardless of the state of the translation flag 336, the operation command is converted into a culture operation indicated by the culture data 334 and transmitted to the robot 12.

  It is not always necessary to convert the operation command, and only the utterance command may be translated. Alternatively, the translation of the operation command may be executed without translating the utterance command.

  FIG. 15 is a flowchart of the utterance command translation process in step S103 shown in FIG. As shown in FIG. 15, when starting the utterance command translation process, the CPU 130 of the central control device 14 arranges the control command com [0, 1,..., N] in the array comTmp [0, 1,. ]. That is, each of the utterance command and the operation command constituting the control command is stored in the array comTmp [0, 1,..., N]. The maximum value N means the total number of commands included in the control command.

  In the next step S123, the variable i is initialized. That is, 0 is stored in the variable i. This variable i is a variable for identifying the command comTmp [i] included in comTmp [0, 1,... N]. In the next step S125, it is determined whether or not the variable i is smaller than the maximum value N. If “YES” in the step S125, that is, if the variable i is smaller than the maximum value N, it is determined whether or not comTmp [i] is an utterance command in a step S127. That is, it is determined whether or not the command specified by the variable i is an utterance command.

  In the process of step S127, a character string up to the first underscore “_” included in each command is read, and if it is a character string “SAY”, it is determined that it is an utterance command, and a character string “MOVE”. If so, it is determined that the command is an operation command.

  If “NO” in the step S127, that is, if the comTmp [i] is not an utterance command, the process proceeds to a step S131 as it is. On the other hand, if “YES” in the step S127, that is, if comTmp [i] is an utterance command, the utterance content of the comTmp [i] is translated into a language indicated by the conversation partner language data 330 in a step S129. The robot 12 has a TTS function. For example, if the speech file name is "SAY_jpn_ Hello .txt", the comTmp [i], "SAY_jpn_ Hello .txt" is stored. Then, the speech command "Hello" is, when it is translated into English from Japanese on the basis of the dialogue partner language data 330, the speech file name is spoken command of "SAY_eng_Hello.txt" is created.

  Further, as described above, the TTS function needs to specify in advance the language to be read before reading the text. Therefore, when the utterance command is executed by the TTS function, the TTS function specifies the language type of the utterance command by “jpn” or “eng” included in the utterance file name. That is, “jpn” is Japanese, and “eng” is English. For example, if you run a speech command indicated by the utterance file name "SAY_jpn_ Hello .txt", running the TTS function identifier included between the first and second underscore contained in spoken filename " “jpn” specifies that the language type to be read is “Japanese”.

  As a result, in the control command of "greeting", and the speech command is translated into English from Japanese, spoken command "Hello" is the spoken command "Hello". Also, the utterance command “Nice weather” becomes the utterance command “Nice weather”.

  In addition, general-purpose text translation software etc. can be used as a function which translates a text.

  In the next step S131, the variable i is incremented (i = i + 1), and the process returns to step S125. That is, the variable i is incremented so that the variable i designates the next command.

  If “NO” in the step S125, that is, if the variable i is equal to or greater than the maximum value N, the array comTmp [0, 1,..., N] is changed to the control command com [0, 1,. N] is overwritten, and the process returns to the communication content control process shown in FIG. That is, in this step S133, the utterance command before translation is replaced with the utterance command after translation. For example, if the speech command is translated into English from Japanese, each of the spoken command "Hello" and the spoken command "It is nice weather" of the control command of "greeting", spoken command "Hello" and the spoken command " Replaced with “Nice weather”.

  FIG. 16 is a flowchart of the operation command conversion process in step S105 shown in FIG. As shown in FIG. 16, when starting the operation command conversion process, the CPU 130 of the central control device 14 arranges the control command com [0, 1,..., N] in an array comTmp [0, 1,. ].

  In the next step S153, the variable j is initialized. That is, 0 is stored in the variable j. The variable j is a variable for identifying the command comTmp [j] included in comTmp [0, 1,..., N] in the operation command conversion process. In the next step S155, it is determined whether or not the variable j is smaller than the maximum value N. If “YES” in the step S155, that is, if the variable j is smaller than the maximum value N, it is determined whether or not comTmp [j] is an operation command in a step S157. That is, it is determined whether or not the command specified by the variable j is an operation command.

  If “NO” in the step S157, that is, if the comTmp [j] is not an operation command, the process proceeds to a step S161 as it is. On the other hand, if “YES” in the step S157, that is, if comTmp [j] is an operation command, the operation content of the comTmp [j] is converted based on the operation determination DB 144 and the culture data 334 in a step S159. That is, the operation command is converted into an operation corresponding to the conversation partner. For example, if the operation command transmitted from the operation terminal 16 is “bow” (MOVE_ojigi.txt) and the culture of the conversation partner indicated by the culture data 334 is “USA”, refer to the operation determination table (see FIG. 7). Then, an operation file name in which “bow” is stored in the operation column and “USA” is stored in the culture column is selected, and the operation file name of the operation command “bow” is “MOVE_ojigi_usa.txt”. To be determined. Therefore, “MOVE_ojigi.txt” stored in comTmp [j] is replaced with “MOVE_ojigi_usa.txt”. In the next step S161, the variable j is incremented (j = j + 1), and the process returns to step S135.

  Here, if “NO” in the step S155, that is, if the variable j is not smaller than the maximum value N, the array comTmp [0, 1,..., N] is set to the control command com [0, 1,. , N] is overwritten on the array comTmp [0, 1,..., N], and the process returns to the communication content control process shown in FIG. That is, in step S163, the operation command before conversion is replaced with the operation command after conversion.

  For example, when the operation command is converted into an operation corresponding to the culture of the United States, the operation command “MOVE_ojigi.txt” (bow) in the “greeting” control command is replaced with the operation command “MOVE_ojigi_usa.txt” (bow). It is done.

  FIG. 17 is a flowchart of the overall processing of the CPU 80 of the robot 12. As shown in FIG. 17, when starting the entire process, the CPU 80 of the robot 12 executes an operator call condition determination process (see FIG. 18) described later in step S301, and the operator call flag is turned on in step S303. Determine whether or not. Although not shown in the figure, the operator call flag is composed of a 1-bit register and is provided in the memory 84 (RAM). This call flag is turned on / off by executing an operator call condition determination process. Specifically, if the operator call condition is satisfied, the call flag is turned on. Conversely, if the operator call condition is not satisfied, the call flag is turned off. When the call flag is turned on, the data value “1” is set in the register. Conversely, when the call flag is turned off, the data value “0” is set in the register.

  If “NO” in the step S303, that is, if the operator calling flag is off, the process proceeds to a step S319. On the other hand, if “YES” in the step S303, that is, if the call flag is on, a call request is transmitted in a step S305, and the process proceeds to the step S307. That is, in this step S305, the operator's call request is transmitted to the central control device 14, and the reception of a response to the request is awaited. In addition, the robot 12 transmits a call request from the operator, and if the conversation partner speaks, acquires the voice of the utterance and transmits voice data corresponding to the voice to the central controller 14. Further, the robot 12 transmits the conversation partner ID acquired from the wireless tag 18 worn by the conversation partner to the central control device 14.

  Subsequently, in step S307, it is determined whether an audio output instruction has been received. That is, in this step S307, it is determined whether or not a voice output instruction (step S41) has been received from the central controller 14. If “NO” in the step S307, the process proceeds to a step S313 as it is. On the other hand, if “YES” in the step S307, an utterance corresponding to the voice output instruction is executed in a step S309, and the process proceeds to a step S311. That is, in this step S309, a sound that causes the conversation partner to speak is output from the speaker 64, such as "What is it for?" In the next step S311, the voice of the conversation partner is acquired and the corresponding voice data is transmitted, and the process proceeds to step S313. That is, voice data corresponding to the voice of the conversation partner acquired by the microphone 66 is transmitted to the central control device 14.

  In step S313, it is determined whether an operation command has been received. If “NO” in the step S313, that is, if an operation command is not received, the process returns to the step S307. On the other hand, if “YES” in the step S313, that is, if an operation command is received from the operation terminal 16, it is determined whether or not a remote operation end request has been made in a step S315. That is, it is determined whether or not the operation command received in step S313 is a remote operation end command.

  If “YES” in the step S315, that is, if a remote operation end request is received, the process proceeds to a step S321. On the other hand, if “NO” in the step S315, that is, if an operation command instructing a communication action is received, an operation corresponding to the received operation command is executed in a step S317, and the process returns to the step S307. By the processing in step S317, the operation of the robot 12 is controlled based on the remote operation. For example, a conversation between an operator who uses the operation terminal 16 using the speaker 64 and the microphone 66 and a conversation partner, body movement such as bowing and pointing, and movement such as forward movement and turning are executed.

  Moreover, in step S319, an independent action process is performed and it progresses to step S321. In the autonomous behavior process, communication behavior is executed with human beings as needed by independent control. For example, when a person is faced, a greeting is given to the person, and when a person is asked about the way, directions are given.

  Subsequently, in step S321, it is determined whether there is a stop instruction. For example, a stop command (stop command) is transmitted from the operation terminal 16 to the robot 12 via the network 100 according to the operation of the operator. However, a stop button may be provided in the environment where the robot 12 or the robot 12 is disposed, and a stop command may be input to the robot 12 by operating the stop button. If “YES” in the step S321, that is, if there is a stop command, the entire processing is ended as it is. On the other hand, if “NO” in the step S321, that is, if there is no stop instruction, the process returns to the step S301.

  FIG. 18 is a flowchart of the operator call condition determination process in step S301 shown in FIG. As shown in FIG. 18, when starting the operator calling condition determination process, the CPU 80 of the robot 12 turns off the calling condition flag in step S341, and detects surrounding information in step S343. That is, the outputs of sensors (infrared distance sensor 40, omnidirectional camera 46, microphone 66, eye camera 70, contact sensor 58, and RFID tag reading device 116) necessary for determining the operator calling condition are detected, and the surrounding and own conditions are detected. Recognize (state).

  In step S345, it is determined whether a specific sentence or word is detected. If “YES” in the step S345, that is, if a specific sentence or word is detected, it is determined that the operator calling condition is satisfied, the calling condition flag is turned on in a step S359, and the robot 12 shown in FIG. Return to the whole process. For example, as a specific sentence or word, a sentence or word that means that the robot 12 cannot understand the intention of the other party (the conversation partner), for example, “No”, “Call the person in charge”, “No. But "and" I don't know "can be considered.

  On the other hand, if “NO” in the step S345, that is, if a specific sentence or word is not detected, it is determined whether or not a specific facial expression is acquired in a step S347. For example, the specific facial expression may be that anger has been detected by recognizing a human facial expression, or that a specific person has been detected by the face specifying function. If “YES” in the step S347, that is, if a specific facial expression is acquired, it is determined that the operator calling condition is satisfied, and the process proceeds to the step S359.

  If “NO” in the step S347, that is, if a specific facial expression is not acquired, it is determined whether or not a specific human is detected in a step S349. For example, as a method of detecting a specific person, it can be considered that a wireless tag 18 having a specific conversation partner ID is detected (for example, a person registered as a VIP appears). If “YES” in the step S349, that is, if a specific person is detected, it is determined that the operator calling condition is satisfied, and the process proceeds to the step S359. On the other hand, if “NO” in the step S349, that is, if a specific person is not detected, it is determined whether or not it is a situation in which the person has a long conversation with the person in a step S351. For example, “YES” may be determined if the distance acquired by the infrared distance sensor 40 is within 1 m for 5 minutes or longer.

  If “YES” in the step S351, that is, if the user is talking for a long time, it is determined that the operator calling condition is satisfied, and the process proceeds to the step S359. On the other hand, if “NO” in the step S351, that is, if it is not a situation where the user is talking for a long time, it is determined whether or not the going hand is blocked in a step S353. If “YES” in the step S353, that is, if the going hand is blocked, it is determined that the operator calling condition is satisfied, and the process proceeds to the step S359. On the other hand, if “NO” in the step S353, that is, if the hand is not blocked, it is determined whether or not there is a large number of people around in a step S355. Here, the state where there are many people in the surroundings may be detected based on the output of the robot 12 or other sensors provided in the surrounding environment.

  If “YES” in the step S355, that is, if there are many people around, it is determined that the operator calling condition is satisfied, and the process proceeds to the step S359. On the other hand, if “NO” in the step S355, that is, if there are not many people around, it is determined whether or not the operator call button has been pressed in a step S357. The operator call button may be provided in the robot 12 or the surrounding environment. If “YES” in the step S357, that is, it is determined that the operator calling condition determination is satisfied, and the process proceeds to a step S359. On the other hand, if “NO” in the step S337, that is, if the operator call button is not pressed, the process returns to the entire processing of the robot 12 shown in FIG.

  FIG. 19 is a flowchart of remote operation processing of the CPU 160 of the operation terminal 16. As shown in FIG. 19, the CPU 160 of the operation terminal 16 determines whether or not there is a remote operation request from the central control device 14 in step S401. Specifically, it is determined whether a remote operation request including the IP address of the robot 12 transmitted from the central control device 14 has been received.

  If “NO” in the step S401, that is, if there is no remote operation request from the central control device 14, the process returns to the same step S401 as it is. However, the process of step S401 is executed every certain time (for example, 30 seconds). On the other hand, if “YES” in the step S401, that is, if there is a remote operation request from the central control device 14, the remote operation screen 210 as shown in FIG. 9 is displayed on the LCD 176 in a step S403.

  Note that at the beginning of displaying the remote operation screen 210, the camera image of the robot 12 that called the operation terminal 16 and the information related to the robot 12 are not acquired, and therefore the content of the robot camera image 212 is not displayed. Further, when the process of step S403 is completed, the subsequent processes of steps S405 to S415 are repeatedly executed at regular intervals.

  Subsequently, in step S405, reception and update of robot information and sensor information (such as voice information, camera information, and distance information) are performed. That is, the robot information is received from the central controller 14 or the robot 12, and the display of the robot information 214 on the remote operation screen 210 is updated. Furthermore, the voice data is received from the robot 12, the voice information is output from the speaker 166 of the operation terminal 16, the camera information is received from the robot 12, and the display of the robot camera image 212 on the remote operation screen is updated. The distance information from the robot 12 is received, and the distance to the obstacle (conversation partner) is displayed.

  When the robot information 214 is displayed for the first time, the position, name, dialogue partner information, etc. of the robot 12 are acquired from the central controller 14. Further, the position information of the robot 12 and the opponent for map display is acquired from the robot 12. In addition, depending on the communication speed between the robot 12 and the operation terminal 16, there is sensor information that is not transmitted from the robot 12, and in such a case, display of some or all sensor information may be limited. Good.

  In step S407, it is determined whether character string data has been received. That is, in this step S407, it is determined whether or not the translated character string (step S97) has been received from the central controller 14. If “NO” in the step S407, that is, if character string data is not received, the process proceeds to a step S411 as it is. On the other hand, if “YES” in the step S407, that is, if character string data is received, the character corresponding to the character string data is displayed on the LCD 176 in a step S409, and the process proceeds to a step S411. That is, in this step S409, the translation result of the content uttered by the conversation partner translated by the central controller 14 is displayed on the LCD 176.

  In step S411, it is determined whether an operation command has been input. For example, based on input data from the input device 170 and button arrangement data on the operation panel 216, it is determined whether or not an operation command button has been selected by the operator. If “NO” in the step S411, that is, if there is no operation command input, the process returns to the step S405 as it is. On the other hand, if “YES” in the step S411, that is, if an operation command is input, the selected operation command is specified in a step S413, and the operation command is transmitted to the robot 12. In step S415, it is determined whether a remote operation end request has been received. That is, it is determined whether or not the input operation command is a command (end command) for instructing the end of the remote operation.

  If “NO” in the step S415, that is, if the end command is not input, the process returns to the step S405 as it is. On the other hand, if “YES” in the step S415, that is, if an end command is input by the operator, the remote operation processing is ended.

  According to this embodiment, when there is a call request from the robot 12, the middle control device 14 specifies the language used by the other party from the voice spoken by the other party, and based on the specified language, Identify cultural behavior. Further, the central control device 14 transmits a request for requesting remote operation of the robot 12 to the operation terminal 16 used by the operator in the IDLE state, and whether or not the language of the conversation partner matches the language used by the operator. Judging. When the language of the conversation partner does not match the language used by the operator, when the control command is transmitted from the operation terminal 16 to the central controller 14, the central controller 14 causes the utterance command included in the control command to be transmitted. Is translated into the language of the conversation partner, and the motion command is converted into a motion of the conversation partner's culture and transmitted to the robot 12.

  As a result, the robot 12 speaks the language of the conversation partner's culture with the body movement performed in the partner's culture, so that the operator can appropriately respond regardless of the conversation partner's culture.

  In this embodiment, the infrared distance sensor 40 is used. However, instead of the infrared distance sensor 40, an ultrasonic distance sensor, a millimeter wave radar, or the like may be used to measure the distance between the robot 12 and the conversation partner. Good. In addition, in the operator selection process, “likeness calculation” of two languages is performed, but the number of languages may be increased as necessary.

  In addition, the call flag of the operator of the robot 12 is detected by a sensor (for example, a skin sensor) that detects contact on the body surface, and detects that the robot 12 itself has been beaten. Good. Further, by providing the terminal 16 with the function of the central control device 14, the operation terminal 16 may communicate directly with the robot 12 without using the central control device 14.

  Moreover, although it was made to translate between different languages, you may make it convert a dialect different in an area. For example, conversion from Kansai dialect to Tohoku dialect or from Tohoku dialect to Kansai dialect may be possible. Similarly, gesture gestures that differ in each region may be converted regardless of cultural differences.

  Further, in this embodiment, the central control device 14 performs the utterance command translation process and the operation command conversion process in the control command, but may be performed by the operation terminal 16 or the robot 12.

  For example, when the operation terminal 16 performs the speech command translation process and the operation command conversion process in the control command, the CPU 160 of the operation terminal 16 performs steps S31-S55 (see FIG. 12) and steps S59-S73 ( The processing shown in steps S101 to S107 (see FIG. 14), the utterance command translation processing (see FIG. 15), and the operation command conversion processing (see FIG. 16) may be performed. In step S 413, the command input to the operator is not transmitted to the robot 12, but is transmitted to the robot 12 after performing the utterance command translation process and the operation command conversion process.

  When the robot 12 performs the speech command translation process and the motion command conversion process in the control command, the CPU 80 of the robot 12 performs steps S33-55, S59-S73, and steps S101-S107. The process shown and the utterance command translation process and the operation command conversion process may be performed. In step S41, the process of step S309 is executed without transmitting a request for an audio output instruction. When the process of step S317 is executed, the received control command is executed after performing the utterance command translation process and the operation command conversion process.

FIG. 1 is an illustrative view showing an outline of a robot remote control system according to an embodiment of the present invention. FIG. 2 is an illustrative view showing the appearance of the robot shown in FIG. 1 from the front. FIG. 3 is a block diagram showing an electrical configuration of the robot shown in FIG. FIG. 4 is a block diagram showing an electrical configuration of the central controller shown in FIG. FIG. 5 is an illustrative view showing one example of a table stored in the database of the central control unit shown in FIG. FIG. 6 is an illustrative view showing one example of another table stored in the database of the central controller shown in FIG. FIG. 7 is an illustrative view showing one example of other tables stored in the database of the central controller shown in FIG. FIG. 8 is a block diagram showing an electrical configuration of the operation terminal shown in FIG. FIG. 9 is an illustrative view showing one example of a remote operation screen displayed on the display device of the operation terminal shown in FIG. FIG. 10 is an illustrative view showing one example of a memory map of a memory of the central control unit shown in FIG. FIG. 11 is a flowchart showing the entire processing of the CPU of the central control measure shown in FIG. FIG. 12 is a flowchart showing a part of the CPU operator selection process of the central control measure shown in FIG. FIG. 13 is another part of the CPU operator selection process of the central control measure shown in FIG. 1, and is a flowchart subsequent to FIG. FIG. 14 is a flowchart showing a communication content control process of the CPU of the central control measure shown in FIG. FIG. 15 is a flowchart showing the utterance command translation processing of the CPU of the central control measure shown in FIG. FIG. 16 is a flowchart showing operation command conversion processing of the CPU of the central control measure shown in FIG. FIG. 17 is a flowchart showing the overall processing of the CPU of the robot shown in FIG. FIG. 18 is a flowchart showing operator call condition determination processing of the CPU of the robot shown in FIG. FIG. 19 is a flowchart showing remote operation processing of the CPU of the operation terminal shown in FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Robot remote control system 12 ... Communication robot 14 ... Central controller 16 ... Operation terminal 18 ... Wireless tag 40 ... Infrared distance sensor 46 ... Omnidirectional camera 58 ... Contact sensor 64 ... Speaker 66 ... Microphone 70 ... Eye camera 80 ... CPU
82 ... Bus 84 ... Memory 86 ... Motor control board 88 ... Sensor input / output board 90 ... Voice input / output board 100 ... Network 112 ... Communication LAN board 114 ... Wireless communication device 116 ... Wireless tag reader 130 ... CPU
132 ... Bus 134 ... Memory 148 ... Communication LAN board 150 ... Wireless communication device 160 ... CPU
162 ... LCD driver 164 ... Memory 166 ... Speaker 168 ... Microphone 170 ... Input device 172 ... Communication LAN board 174 ... Wireless communication device 176 ... LCD

Claims (10)

A robot that performs communication with humans through at least one of body movement and voice by remote control or autonomous control, a plurality of operation terminals that perform remote control of the robot via a network, and the network via the network In a robot remote operation system comprising a central control device that mediates between a robot and one of the plurality of operation terminals,
The robot is
Personal information acquisition means for acquiring the human personal information;
Personal information transmitting means for transmitting the personal information acquired by the personal information acquiring means to the central control device;
First remote operation information receiving means for receiving first remote operation information from the central control device; and execution means for executing communication behavior based on the first remote operation information received by the first remote operation information receiving means. Prepared,
The operation terminal is
Second remote operation transmitting means for transmitting second remote operation information for remotely operating the robot to the central control device;
The central controller is
A storage medium for storing communication information of an operator of the operation terminal;
Personal information receiving means for receiving personal information transmitted by the personal information transmitting means;
Communication information specifying means for specifying communication information from personal information received by the personal information receiving means;
Second remote operation information receiving means for receiving the second remote operation information transmitted by the second remote operation transmission means;
Determining means for determining whether the communication information specified by the communication information specifying means and the communication information of the operator of the operating terminal stored by the storage medium match;
When the determination result determined by the determining means does not match, the second remote operation information received by the second remote operation information receiving means is changed to the first remote control information based on the communication information specified by the communication information specifying means. A robot remote operation system comprising: conversion means for converting to operation information; and first remote operation information transmission means for transmitting the first remote operation information converted by the conversion means to the robot. The communication information includes language information,
The communication information specifying means includes language information specifying means for specifying language information from the personal information received by the personal information receiving means,
The determining means determines whether or not the language information specified by the language information specifying means matches the language information of the operator of the operation terminal stored by the storage medium,
The converting means translates the utterance information included in the second remote control information based on the language information specified by the language specifying means when the determination results determined by the determining means do not match The robot remote control system according to claim 1, comprising: The communication information includes information on the type of body movement,
The communication information specifying means includes physical action type information specifying means for specifying information on the type of physical action from the personal information received by the personal information receiving means,
The comparison means determines whether or not the physical action type information specified by the physical action type information specifying means matches the physical action type information of the operator of the operating terminal stored by the storage medium. Judging
When the determination result determined by the determination means does not match, the conversion means converts the motion information included in the second remote operation information into the physical action type information specified by the physical action type information specifying means. The robot remote control system according to claim 1, further comprising motion conversion means for converting based on the motion conversion means. The personal information acquisition means includes voice acquisition means for acquiring human voice,
The personal information transmission means further transmits the voice acquired by the voice acquisition means;
The personal information receiving means further receives the voice transmitted by the personal information transmitting means;
4. The robot remote control system according to claim 2, wherein the language information specifying unit includes a spoken language information specifying unit that specifies language information from a human voice received by the personal information receiving unit. The central controller is
A request signal transmitting means for transmitting a request signal for instructing the robot to speak when a human voice is not acquired by the voice acquiring means;
The robot is
5. The apparatus according to claim 4, further comprising: a request signal receiving unit that receives the request signal transmitted by the request signal transmitting unit; and an audio output unit that outputs a sound in response to the request signal received by the request signal receiving unit. Robot remote control system. The personal information acquisition means includes identification information acquisition means for acquiring identification information from an identification information tag carried by a human,
The personal information transmitting means further transmits the identification information acquired by the identification information acquiring means;
The personal information receiving means further receives the identification information transmitted by the personal information transmitting means;
6. The robot remote control system according to claim 2, wherein the language information specifying means includes identification language information specifying means for specifying at least language information from the identification information received by the personal information receiving means. The personal information acquisition means includes identification information acquisition means for acquiring identification information from an identification information tag carried by a human,
The personal information transmitting means further transmits the identification information acquired by the identification information acquiring means;
The personal information receiving means further receives the identification information transmitted by the personal information transmitting means;
4. The robot remote operation system according to claim 3, wherein the body motion type determining means includes identification information body motion type determining means for further determining information on the type of body motion from the identification information received by the personal information receiving means. The central controller is
Speech translation means for translating the speech received by the personal information receiving means based on the language information of the operator of the operation terminal stored in the storage medium when the judgment results judged by the judgment means do not match And a translation result transmitting means for transmitting the translation result translated by the speech translation means to the operation terminal,
The operation terminal is
The translation result receiving means for receiving the translation result transmitted by the translated speech result transmitting means, and the output means for outputting the translation result received by the translation result receiving means, further comprising: The robot remote control system described. A robot that performs communication with humans through at least one of body movement and voice by remote control or autonomous control, a plurality of operation terminals that perform remote control of the robot via a network, and the network via the network In a robot remote control system comprising a central controller that mediates between a robot and one of the plurality of operation terminals,
The robot is
Personal information acquisition means for acquiring the human personal information;
Personal information transmitting means for transmitting the personal information acquired by the personal information acquiring means to the operation terminal;
First remote operation information receiving means for receiving first remote operation information from the operation terminal, and execution means for executing a communication action based on the first remote operation information received by the first remote operation information receiving means. ,
The operation terminal is
A storage medium for storing communication information of an operator of the operation terminal itself;
Personal information receiving means for receiving personal information transmitted by the personal information transmitting means;
Communication information specifying means for specifying communication information from personal information received by the personal information receiving means;
Determining means for determining whether the communication information specified by the communication information specifying means and the communication information of the operator of the operating terminal stored by the storage medium match;
When the determination result determined by the determination means does not match, the second remote operation information for remotely operating the robot is converted into the first remote operation information based on the communication information specified by the communication information specifying means A robot remote operation system, comprising: a conversion unit that transmits the first remote operation information converted by the conversion unit to the robot. A robot that performs communication with humans through at least one of body movement and voice by remote control or autonomous control, a plurality of operation terminals that perform remote control of the robot via a network, and the network via the network In a robot remote operation system comprising a central control device that mediates between a robot and one of the plurality of operation terminals,
The operation terminal is
First remote operation transmission means for transmitting first remote operation information for remotely operating the robot to the robot;
The robot is
A storage medium for storing communication information of an operator of the operation terminal;
Personal information acquisition means for acquiring the human personal information;
Communication information specifying means for specifying communication information from personal information received by the personal information receiving means;
First remote operation information receiving means for receiving first remote operation information from the operation terminal;
Determining means for determining whether the communication information specified by the communication information specifying means and the communication information of the operator of the operating terminal stored by the storage medium match;
When the determination results determined by the determining means do not match, the first remote operation information received by the first remote operation information receiving means is changed to the second remote operation information based on the communication information specified by the communication information specifying means. A robot remote operation system comprising: conversion means for converting to operation information; and execution means for executing a communication action based on the second remote operation information converted by the conversion means.

Images