JP2007216363A - Communication robot - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a communication robot capable of realizing smooth communication. <P>SOLUTION: The communication robot 10 performs the communication while facing a first object person 301, and performs the communication with a second object person 40 via a communication means. The communication robot performs a first mode for performing the autonomous operation, a second mode for performing the operation instructed from the second object person 40, and a third mode for performing the operation to reflect the operation of the second object person 40 and for mediating the communication with the first object person 301 and the second object person 40. The robot 10 can perform the smooth communication with the facing person 301, perform the smooth communication with the person 40 in contact via a communication means, and performs the smooth communication between the facing person 301 and the person 40 in contact via the communication means. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a robot that communicates with humans, and in particular, to achieve smooth communication.

In Japan, research and development related to robots is thriving, and many types of robots are manufactured.
Patent Document 1 below discloses a robot that starts an action in response to an instruction given by an instructor by hand gesture or gesture. This robot holds a setting file in which the relationship between the posture of the instructor and the instruction content is defined, recognizes the posture of the person from the image captured by the camera, and executes the instruction content corresponding to the posture.
For example, when signaling to follow the robot, the robot follows the person who issued the instruction. At this time, “when the robot moves while maintaining a constant distance from the human, the distance to ensure safety is the first predetermined distance (eg, 150 cm)”, “distance from the human Is stopped when the distance is less than the second predetermined distance (for example, 90 cm) "," the distance from the human being is greater than or equal to the second predetermined distance (for example, 90 cm) to less than the first predetermined distance (for example, 150 cm) If a condition such as “retreat or adjust the pace when it is reached” is set, the operation of the robot is controlled to satisfy this condition.

Patent Document 2 listed below describes a robot that supports communication between two people who are located apart from each other. This robot includes a travel means, a drive means, various sensors, a controller, etc. for autonomous operation, as well as a speaker, a microphone, a camera, a display screen, and a communication means with a control device for remotely operating the robot. ing.
For example, a control device is placed in the hands of a customer service representative, and an image of a robot camera showing a store visitor is displayed on the control device. When the customer service staff finds a visitor from this video, the robot approaches the customer remotely. The face of the customer service is photographed by the camera of the control device and displayed on the robot display screen, and the voice of the customer service is sent to the robot and output from the speaker. On the other hand, the customer's face is photographed by the robot camera and displayed on the control device, and the customer's voice is collected by the robot microphone and sent to the customer service.
Therefore, video and audio can be transmitted and received between the communication robot and the control device in real time, and communication between the operator of the control device and the person facing the robot can be performed.
JP 2004-78316 A JP 2004-261194 A

In a situation where people communicate with each other face-to-face, for example, even if they suddenly say “take it”, they often know what it is. This is because the receiver side obtains information about the surrounding situation and the behavior of the other party with the five senses.
However, when communication is performed via communication means, only information transmitted through the communication means can be used for communication. Therefore, when using communication means, how to achieve smooth communication is a major issue.
When people communicate with each other via a robot, at least one person faces the robot, so that the advantage of face-to-face communication can be utilized.
However, for that purpose, it is necessary for the robot to show the performance that appeals to the five senses of the person who faces it, and the other person who exchanges information with the robot through the communication means communicates smoothly with the robot. It needs to be realized.

  The present invention has been made in consideration of such circumstances, and an object thereof is to provide a communication robot capable of realizing smooth communication.

The present invention is a communication robot that communicates with a first target person and communicates with the second target person via a communication means, the first mode for autonomously operating, A second mode for performing an operation instructed by the second target person, a third mode for performing an operation reflecting the action of the second target person, and mediating communication between the first target person and the second target person. Execute the mode.
The first target person facing the robot can know the operation of the second target person who is the communication partner from the operation of the robot.

In addition, the communication robot according to the present invention executes the first mode, the second mode, and the third mode by switching them according to the situation.
Therefore, it is possible to communicate in a manner desired by the target person.

In addition, the communication robot of the present invention communicates the behavior of the communication robot and the surrounding situation to the second target person and communicates the operation of the second target person to the communication robot. It has a transmission means.
By knowing the behavior of the robot and the surrounding situation, the second target person can perform an action when the robot is at the position of the robot, and the robot can obtain information on the action of the second target person through the transmission means. By obtaining, the operation of the second subject can be imitated.

In addition, the communication robot of the present invention autonomously communicates with the first target person in the first mode.
This robot autonomously communicates with the first target person facing.

In addition, the communication robot of the present invention explains the exhibit to the first target person in the first mode, and the question from the first target person and the answer by the second target person in the third mode. Mediates exchanges with
This robot operates as a guide robot for the exhibition.

In addition, the communication robot of the present invention mediates a conversation between the first target person in the hall and the second target person outside the hall in the third mode.
This robot looks around the venue on behalf of the second subject who cannot come to the venue due to physical circumstances, etc., and performs the operation of conveying the situation in the venue to the second subject.

  The communication robot according to the present invention can perform smooth communication with a person who meets the person, can smoothly communicate with a person who contacts through the communication means, and can communicate with the person who meets the person. You can have smooth communication with people who come in contact with you.

(First embodiment)
In the first embodiment of the present invention, a communication robot in charge of a museum guide role will be described.
FIG. 1 is a schematic diagram showing the communication robot and the operation side device. FIG. 2 is a block diagram showing the configuration of the communication robot, and FIG. 3 is a block diagram showing the configuration of the operation side device.

As shown in FIG. 1, the communication robot 10 includes a head 11, a neck 16, a body 12, a carriage 13, wheels 14, and arms 151 and 152. A head camera 17 is installed at the apex of the head 11, and the body 12 has three body cameras 201, 202, 203, a microphone 24, and a speaker 25 on the front side and three on the back side. A plurality of ultrasonic sensors 23 are installed on the carriage 13.
The arms 151 and 152 can be raised and lowered. The head 11 can be moved with two or more degrees of freedom by rotating or tilting the neck 16. Further, the carriage 13 can move in an arbitrary direction by the rotation of the wheels 14.
Among the front body cameras 201 to 203, the center camera 202 has a shooting range in the front, the right camera 203 has a right direction, and the left camera 201 has a left range. Therefore, the entire area in front of the robot is photographed by the three cameras 201 to 203 on the front side. Similarly, the entire area behind the robot 10 is photographed by three cameras (not shown) on the back side.

The operation side device is installed in the room of the expert 40 in the museum, and includes six monitors 501-503, 601-603, and a camera 51 for photographing the expert 40.
Among the six monitors, the monitors 501, 502, and 503 in the lower row display images of the body cameras 201, 202, and 203 installed on the front side of the robot 10, respectively, and the monitors 601, 602, and 603 in the upper row display , Respectively, display images of three torso cameras installed on the back side.
Further, the image 53 of the head camera 17 is displayed in an overlapping manner on the images of the lower row monitors 501, 502, and 503 that coincide with the shooting direction of the head camera 17. For example, when the head 11 of the robot 10 turns to the left, the image 53 of the head camera 17 is displayed on the image of the left monitor 501, and when the head 11 turns to the right, the head camera 17 Is displayed on the image of the monitor 503 on the right side.
Further, a CG image 52 of the robot 10 is displayed on the monitor 602 in the center of the upper row so as to be superimposed on the image of the torso camera.

As shown in FIG. 2, the communication robot 10 includes an image processing unit 63 that processes image data of the head camera 17 and the torso cameras 201 to 203, an image analysis unit 62 that analyzes these images, a microphone 25, and a speaker. A voice processing unit 66 that processes 24 voices, a voice synthesis unit 65, a voice recognition unit 64, an obstacle detection unit 70 that detects an obstacle using the ultrasonic sensor 23, a travel drive unit 71, and an arm drive unit 72. And a drive control unit 69 that controls the drive of the neck drive unit 73, a data storage unit 67, a transmission / reception unit 61 that transmits and receives data to and from the operation side device by wireless, and a control unit 68 that controls the operation of each unit. And.
Among these, the image processing unit 63, the image analysis unit 62, the voice processing unit 66, the voice synthesis unit 65, the voice recognition unit 64, the obstacle detection unit 70, the drive control unit 69, and the control unit 68 are built in the robot 10. This is realized by a computer or CPU executing a process defined by a program.

The travel drive unit 71 moves the carriage 13, the arm drive unit 72 moves the arms 151 and 152, and the neck drive unit 73 rotates or tilts the neck 16 to change the direction and tilt of the head 11.
As shown in FIG. 3, the operation side device 80 includes a display control unit 90 that controls the display of the monitors 501 to 503 and 601 to 603, a CG creation unit 89 that creates a CG image 52, and an image of the camera 51. An image processing unit 82 that processes data, an image analysis unit 83 that analyzes an image of the camera 51 to detect a face direction 831 and a pointing direction 832 of the expert 40, a microphone 84, a speaker 85, An audio processing unit 86, a data storage unit 87, a transmission / reception unit 81 for transmitting / receiving data to / from the robot 10 by radio or the like, and a control unit 88 for controlling the operation of each unit are provided.
Among these, the display control unit 90, the CG creation unit 89, the image processing unit 82, the image analysis unit 83, and the control unit 88 execute processing specified by a program by a computer or CPU built in the operation side device 80. It is realized by.

As shown in FIG. 4, the communication robot 10 communicates with an “autonomous mode” that operates autonomously, an expert 40 (that is, a person who exchanges information through a communication means), and gives instructions from the expert 40. Either “interactive mode” to be executed or “mediation mode” to mediate communication between the expert 40 and the visitor is executed.
The communication robot 10 autonomously communicates with surrounding people 301 (visitors) in the “autonomous mode”. Normally, the robot 10 operates in an autonomous mode, and switches to a dialogue mode or a mediation mode depending on the situation. Then, when the operation in those modes ends, the mode returns to the autonomous mode.

The data storage unit 67 of the robot 10 stores data for explaining the exhibits and map data of the venue, and further, the operation to be executed in each mode of the autonomous mode, the dialogue mode, and the mediation mode, and other operations from one mode to another. The conditions for moving to the mode are set.
For example, in autonomous mode,
(1) A person who is near the exhibit and looking at the robot 10 is detected from the images of the front and back body cameras 201 to 203.
(2) When a corresponding person is detected in (1), the head 11 is directed toward the person's face and the face is photographed by the head camera 17.
(3) If the face photographed by the head camera 17 is looking at the robot 10 for S1 seconds or longer, the face moves toward the person and stops at a position d1 m away from the person.
(4) Subsequent to (3), the explanatory text of the exhibit is uttered by voice synthesis.
(5) At the time of utterance of the explanatory sentence, the face of the robot 10 is directed to a visitor who hears the explanation after an important word specified in advance or at a sentence break point.
Etc. are set.

In interactive mode,
(1) The head 11 of the robot 10 is directed in the direction instructed from the operation side device 80.
(2) The pamphlet instructed from the operation side device 80 is given to the instructed person.
Etc. are set.
In the medium mode,
(1) The head 11 is directed in the direction in which the face of the person on the operation side device 80 faces.
(2) Raise the arms 151 and 152 in the direction in which the person on the operation side device 80 points.
(3) The voice sent from the operation side device 80 is output.
Etc. are set.

Moreover, the conditions for transition between modes are set as follows, for example.
(1) When a voice instruction to the robot 10 is input from the operation side device 80, the mode shifts from the autonomous mode to the dialog mode, and when the commanded operation ends, the mode shifts from the dialog mode to the autonomous mode.
(2) When there is a question (speech) from a person around the robot 10 toward the robot 10, the mode shifts from the autonomous mode to the mediation mode, and when the exchange of the questions ends, the mode shifts from the mediation mode to the autonomous mode. To do.
On the other hand, the data storage unit 87 of the operation side device 80 stores a table indicating the relationship between various parameters of the robot 10 and actual data.

Next, the operation of this system will be described.
The robot 10 performs processing such as sampling and quantization on the images projected by the front and back torso cameras 201 to 203 and the image projected by the head camera 17 by the image processing unit 63, and then transmitted and received. The data is transmitted from the unit 61 to the operation side device 80. The audio signal input from the microphone 25 is converted into digital data by the audio processing unit 66 and transmitted from the transmission / reception unit 61 to the operation side device 80. Further, the drive control unit 69 transmits control parameters used for drive control of the travel drive unit 71, the arm drive unit 72, and the neck drive unit 73 from the transmission / reception unit 61 to the operation side device 80.
In the operation-side device 80, the display controller 90 divides and displays the received images of the front and back body cameras 201 to 203 on the six monitors 501 to 503 and 601 to 603 (the front left body of the robot 10). The image of the camera 201 is directed to the left monitor 501 toward the lower row, the image of the front-side torso camera 202 is directed to the monitor 502 at the center of the lower row, and the image of the front right torso camera 203 is directed to the right monitor 503 toward the lower row. The image of the back fuselage camera on the back side of the torso camera 201 is displayed on the right monitor 603 toward the upper row, the image of the back fuselage camera on the back side of the torso camera 202 is displayed on the monitor 602 in the center of the upper row, and the back side of the torso camera 203 Are displayed on the left monitor 601 toward the upper row.)

In addition, the display control unit 70 refers to the table stored in the data storage unit 87 based on the control parameters of the neck driving unit 73 among the control parameters received from the robot 10, so that the head 11 of the robot 10 The direction and inclination are obtained, and the image 53 of the head camera 17 is displayed in an overlapping manner on the images of the lower monitors 501, 502, and 503 that coincide with the shooting direction of the head camera 17.
Further, the CG creation unit 89 refers to the table of the data storage unit 87 based on the control parameters of the travel drive unit 71, the arm drive unit 72, and the neck drive unit 73 received from the robot 10, and the travel direction and arm of the robot 10. The angles of 151 and 152 and the direction and inclination of the head 11 are obtained, and a CG image of the robot 10 is generated. The display control unit 70 displays the created CG image 52 so as to be superimposed on the image of the monitor 602 at the center of the upper row.
The received audio data is converted into an analog signal by the audio processor 86 and output from the speaker 85 as audio.
The expert 40 can know the situation around the robot 10 and the behavior of the robot 10 by looking at various images displayed on the monitors 501 to 503 and 601 to 603.

In the robot 10 in the autonomous mode, the control unit 68 controls each unit to execute the operation in the autonomous mode set in the data storage unit 67.
The image analysis unit 62 analyzes the images of the front and back torso cameras 201 to 203 based on instructions from the control unit 68, and is a visitor who is near the exhibition and faces in the direction of the robot 10. Is detected.
When a corresponding person is detected by image analysis, the control unit 68 instructs the drive control unit 69 to turn the head 11 in the detected direction, and the drive control unit 69 follows the instruction, and the neck drive unit 73. To change the direction of the head 11.
The image analysis unit 62 analyzes the image of the head camera 17 and extracts the visitor's face. The control unit 68 measures the time when the extracted face is facing the robot 10, and when the time exceeds S1 seconds, the robot 10 meets the visitor (eye contact is established). The drive control unit 69 is instructed to approach until the distance from the visitor becomes d1 m. The drive control unit 69 refers to the map data in the data storage unit 67 and controls the travel drive unit 71 to advance the robot 10 in the direction of the visitor. And it stops when the obstacle detection part 70 detects the distance with the visitor as d1m.

When the robot 10 moves to the instructed position, the voice synthesizer 65 synthesizes the description data stored in the data storage unit 67 with the voice based on the instruction from the control unit 68, and the synthesized voice is output from the speaker. 24 is uttered (this utterance is also transmitted to the operation side device 80).
Accordingly, a visitor who has eyes in contact with the robot 10 receives an explanation of the exhibit from the robot 10.

During this explanation, the control unit 68 confirms the current position of the visitor's face from the analysis result of the image analysis unit 62, and when important words designated in advance during the explanation are uttered, or When the break point is reached, the drive control unit 69 is instructed to turn the head 11 toward the visitor. The drive control unit 69 changes the rotation and inclination of the neck drive unit 73 in accordance with the instruction, and turns the head 11 toward the visitor.
The movement of the robot 10 is such that a general guide moves his / her line of sight to the visitor in the middle of the explanation, and the visitor also looks at the guide accordingly (that is, the guide is “Do you understand? ”And the same effect as a visitor who answers“ I know ”with the eyes), and enables smooth communication with the face-to-face visitors.
The expert 40 of the operation side device 80 uses the position and display contents of the image 53 of the head camera 17 displayed on the monitors 501 to 503, the CG image 52 displayed on the monitors 601 to 603, the sound flowing from the speaker 85, and the like. It is easy to see what direction the robot 10 is looking at.

When the expert 40 finds a person who wants to hear the explanation of the robot 10 from the images of the torso cameras 201 to 203 displayed on the monitors 501-503 and 601-603, the robot 10 notices the presence of the person. If not, face the person on the monitors 501 to 503 and 601 to 603, or point to the person and say "Tell me about that person".
The voice of the expert 40 is converted into an electric signal by the microphone 84 of the operation side device 80, and converted into digital data by the voice processing unit 86. The control unit 88 instructs the image analysis unit 83 to analyze the image of the camera 51 that reflects the appearance of the expert 40 at this time, and the image analysis unit 83 detects the face direction detection processing 831 or the finger from this image. An insertion direction detection process 832 is performed. The control unit 88 transmits data indicating the detected direction and audio data to the robot 10 through the transmission / reception unit 81.

In the robot 10 that has received these data, the control unit 68 instructs the voice recognition unit 64 to perform voice recognition of the voice data. The voice recognition unit 64 converts the voice data into a character string. When the control unit 68 compares the character string with a dictionary word or character string stored in advance and identifies the instruction to the robot 10, the control unit 68 shifts from the autonomous mode to the interactive mode and executes the operation in the interactive mode.
The control unit 68 transmits data indicating the direction received from the operation side device 80 to the drive control unit 69 and instructs the head 11 to face the direction. The drive control unit 69 drives the neck drive unit 73 in accordance with the instruction, and directs the head 11 in the designated direction. As a result, a visitor who the expert 40 says “that person” appears in the head camera 17.
When the operation set in the dialogue mode is completed, the robot 10 returns to the autonomous mode, analyzes the image of the head camera 17 and extracts the face of the visitor. When the extracted face faces the robot 10 for S1 seconds or longer, it approaches until the distance from the visitor becomes d1 m, and the explanation of the exhibit is started.

Thus, in this system, the expert 40 who does not directly face the robot 10 can communicate with the robot 10 with the expression “this person” or “that person”. This indicates that smooth communication is possible between the robot 10 and the expert 40.
Similarly, when instructing the robot 10 to hand over the pamphlet, the expert 40 similarly looks at the pamphlets displayed on the monitors 501 to 503 and 601 to 603 (or points to them). You can give instructions by saying "Give that person to that person".

When a question comes from a person who has heard the explanation of the robot 10, the voice is converted into an electric signal by the microphone 25 and converted into digital data by the voice processing unit 66. The control unit 68 of the robot 10 instructs the voice recognition unit 64 to perform voice recognition of the voice data. When the voice recognition unit 64 converts the voice data into a character string, the control unit 68 compares the character string with a dictionary word or character string stored in advance, and when it is identified as a question, the autonomous mode is changed to the mediation mode. Transition to execute the operation of the mediation mode.
First, the control unit 68 of the robot 11 notifies the operation side device 80 that the mode has been changed to the mediation mode. For example, a specific sound is generated from the speaker 24 of the operation side device 80, or the displays of the monitors 501 to 503 and 601 to 603 of the operation side device 80 are surrounded by a special color display frame, so that the transition to the mediation mode is performed. To inform.
Further, the robot 11 utters the answer voice sent from the operation side device 80 from the speaker 24.

The figure of the expert 40 answering the question is photographed by the camera 51 of the operation side device 80, and the face direction and the pointing direction are detected from this image by the image analysis unit 83 which receives an instruction from the control unit 88. . The control unit 88 transmits the detected face direction data and pointing direction data to the robot 10 through the transmission / reception unit 81.
The control unit 68 of the robot 11 transmits the face direction data and the pointing direction data of the expert 40 received from the operation side device 80 to the drive control unit 69 so that the direction of the head 11 matches the face direction. And instructing the directions of the arms 151 and 152 to coincide with the pointing direction. The drive control unit 69 drives the neck drive unit 73 and the arm drive unit 72 according to the instructions, and directs the head 11 and the arms 151 and 152 in the designated direction. As a result, the robot 10 takes the same posture as the expert 40.

For example, when a visitor asks "What is this?", The expert 40 points to "this" on the monitors 501-503 and 601-603 and answers "This is xxxxxxx" The voice of the expert 40 flows from the robot 10, the head 11 of the robot 10 faces “this”, and the arms 151 and 152 of the robot 10 point in the “this” direction.
Therefore, the visitor can perform smooth communication as when facing the expert 40 by looking at the robot 10 even though he / she does not face the expert 40 directly. Moreover, the expert 40 can aim at communication with a visitor by the expression "this" or "that". This indicates that smooth communication is possible between the visitor and the expert 40.
When the exchange of questions ends, the control unit 68 of the robot 10 returns from the mediation mode to the autonomous mode.

  Here, when there is a question from the visitor, the mode immediately shifts to the mediation mode, but in the case of a question that the robot 10 can answer, the robot 10 in the autonomous mode answers and a question that cannot be answered to the robot 10 appears. In addition, the mode may be shifted to the intermediate mode.

In this way, as shown in FIG. 4, the communication robot 10 corresponds to each mode of the autonomous mode, the dialogue mode, and the mediation mode, and switches the mode by itself according to the situation. With such behavior, the robot 10 can communicate smoothly with the visitors.
The mode switching by the robot 10 is naturally performed and is not perceived by the visitor. As a result, visitors will be impressed with the high functionality of the robot.
Moreover, the expert 40 who contacts the robot 10 through the communication means can know the behavior of the robot 10 in detail from the image and can tell the robot 10 the direction of his / her face and hand. Therefore, smooth communication equivalent to that when facing each other is possible between the expert 40 and the robot 10.
In addition, since visitors around the robot 10 can see the operation of the expert 40 through the robot 10, smooth communication equivalent to the case of facing the expert 40 is possible.

As a method for displaying the CG image 52 on the monitor of the operation side device 80, a method other than that illustrated in FIG. To display the CG image 52,
(1) An image when the robot 10 is viewed from the front (front image) is displayed on the upper row of the central monitor 602. (2) A left-right inverted image obtained by reversing the left and right of the front image is displayed on the upper row of the central monitor 602. 3) Display the left and right reverse image on the lower row of the central monitor 502 (4) Display the image of the robot 10 viewed from the back (rear image) on the lower row of the central monitor 502 (5) Display the front image on the lower row of the central monitor 502 Display (6) A method of displaying the rear image on the central monitor 602 in the upper row can be considered.

Among these, the display method (1) (display method in FIG. 1) is advantageous in that the state of the robot 10 is displayed in the same background as when the expert 40 is actually there and the robot 10 is viewed. There is. However, the left and right movements of the face and hand of the robot 10 do not match the front monitor display. For example, in the situation of FIG. 1, the robot 10 is looking at the left side and the CG image 52 is also facing the left side (when looking from the expert 40), the image 53 of the head camera 17 is an expert. When viewed from 40, it is displayed on the monitor 501 on the left side.
In the display method (2), the left and right movements of the face and hand of the robot 10 coincide with the front monitor display, but the actual movement and the reverse image are displayed. Since the background image is the right and left images as viewed from the expert 40, there is a problem that an image different from the actual situation is displayed.
In the display method (3), since the CG image 52 of the reverse image is displayed on the lower center monitor 502 that displays an image from the viewpoint of the robot 10, the uncomfortable feeling is reduced compared to the display method (2). That is, the expert 40 is conscious of the viewpoint of the robot 10 when viewing the monitor 502. However, in this case, there is a problem that a part of the front image including important information is generally hidden by the CG image.
In the display method (4), the operation of the robot 10 and the positional relationship of the monitor 502 coincide with each other, which is good. However, the operation is difficult to understand because a backward-facing image is displayed. In order to prevent this, you may change the image from slightly behind. Further, as in (3), there is a problem that a part of the front image is hidden by the CG image. In addition, since the robot 10 is facing backward, the user may not have a favorable feeling particularly in the interactive mode.
The display method (5) is a display method employed when the robot 10 does not have a back body camera, but is not particularly advantageous in other cases.
The display method (6) is unnatural because it looks as if the robot 10 is looking backward, and has no particular advantage.
Thus, each display method has its own characteristics. Therefore, a plurality of CG display methods may be prepared so that the user can select according to his / her preference. Further, the CG display method may be switched according to the mode of the robot 10 (for example, in the interactive mode, the CG image is displayed by any one of methods (1), (2), (3), and (5)). In the intermediate mode, it is displayed by the method (4) or (6), and in the autonomous mode, it is displayed by the same method as the interactive mode or the intermediate mode.)

  Here, the operation side device 80 is provided with a six-sided display, but the display area of one display is divided into six parts so that the contents displayed on the six-sided display are displayed in a small size. May be. By doing so, as shown in FIG. 5, the expert 40 can handle a plurality of robots alone. However, in this case, since the movement of the face of the expert 40 who sees the display becomes small, it is necessary on the robot side to enlarge it and reflect it in the movement of the head 11 of the robot.

  In this case, the camera for detecting the orientation of the face of the expert 40 can be replaced by a single wide-angle camera. However, as shown in FIG. 5, a single display is provided for each robot. It is desirable to install them one by one. In this way, when determining the robot to which the expert 40 pays attention, it is sufficient to search for an image in which the face of the expert 40 is almost in front of each camera image. .

  Here, the video 53 of the head camera 17 of the robot 10 is displayed on the monitors 501 to 503 of the operation side device 80, but the installation of the head camera 17 is stopped and the direction of the head 11 of the robot 10 is indicated. You may make it display a symbol (for example, a cursor, a head-shaped mark, a rectangular frame, etc.) in the corresponding position of the monitors 501 to 503.

  Also, when finding the orientation of the face or hand of the expert 40 from the camera image, if the head and arms of the expert 40 are marked and detected, reliable information can be obtained through simple processing. Obtainable. Further, instead of obtaining the orientation of the face or hand from the camera image, it may be detected using a magnetic sensor or the like. In this case, the expert 40 needs to wear a detection device.

(Second Embodiment)
In the second embodiment of the present invention, a communication robot that visits a museum on behalf of a person who cannot go to the museum will be described.
FIG. 6 schematically shows a situation where a friend 303 of the home-staying 302 and the robot 10 take a tour of the museum, and the home-staying 302 sees an image reflected on the camera of the robot 10 on the home operating side device 80. It is a figure.

Visiting museums and art galleries is not just about appreciating the works, but also about creating fun while talking with friends. This communication robot 10 is intended to visit a museum with friends 303 on behalf of a resident 302 who cannot enjoy such enjoyment due to illness or physical reasons, and to make the resident 302 feel the pleasure of visiting.
The configuration of the robot 10 is the same as that of the first embodiment (FIG. 2). The configuration of the operation side device 80 is functionally the same as that of the first embodiment (FIG. 3), although the outer shape is small.

The communication robot 10 normally operates in a mediation mode, and switches to an autonomous mode or a dialogue mode depending on the situation. When the operation of these modes is completed, the mode returns to the mediation mode.
In the data storage unit 67 of the robot 10, operations to be executed in each mode of the autonomous mode, the dialogue mode, and the mediation mode, and conditions for shifting from one mode to another mode are set as follows.

For example, in autonomous mode,
(1) When an obstacle is found during movement, the head 11 of the robot 10 is directed toward the obstacle.
(2) Avoid the obstacles found.
Etc. are set.
In interactive mode,
(1) A place designated by the operation side device 80 is set as a target point.
(2) A person designated by the operation side device 80 is set as a companion target.
Etc. are set.
In the medium mode,
(1) The head 11 is directed in the direction in which the face of the person on the operation side device 80 faces.
(2) Raise the arms 151 and 152 in the direction in which the person on the operation side device 80 points.
(3) The voice sent from the operation side device 80 is output.
(4) Move to the set target point.
(5) Move according to the person set as the companion target.
Etc. are set.

Moreover, the conditions for transition between modes are set as follows, for example.
(1) When a voice instruction to the robot 10 is input from the operation side device 80, the mode is switched from the mediation mode to the dialog mode, and when the commanded operation is completed, the mode is switched from the dialog mode to the mediation mode.
(2) When an obstacle is found during movement, the mode is switched from the mediation mode to the autonomous mode, and when the operation of the robot 10 and the motion of the person on the operation side device 80 coincide, the mode is switched from the autonomous mode to the mediation mode. To do.

Next, the operation of this system will be described.
When the at-home person 302 points to the exhibition displayed on the screen of the operation side device 80 and utters “I want to see this”, the robot 10 identifies that it is an instruction for itself, and determines the designated location as the target point. Set to. Further, when the home person 302 points to the friend 303 reflected on the screen of the operation side device 80 and utters “Go with this person”, the robot 10 identifies that it is an instruction for itself, and the person Set as a companion target.
The robot 10 shifts to the mediation mode and starts moving to the target point or following the friend 303.

In the mediation mode, a voice uttered by the person at home 302 is uttered from the speaker 24 of the robot 10, and a voice of the friend 303 is input from the microphone 25 of the robot 10 and uttered from the speaker 85 of the operation side device 80. In addition, the face orientation and hand shake of the person at home 302 are reproduced on the head 11 and arms 151 and 152 of the robot 10, and the gesture of the friend 303 is displayed on the screen of the operation side device 80. Therefore, the person at home 302 can observe the exhibition while enjoying a conversation with gestures and hand gestures with the friend 303.
When the home person 302 turns his / her face to a place he / she wants to see on the screen of the operation side device 80, the head 11 of the robot 10 faces the same direction, and the head camera 17 projects the direction the home person 302 wants to see.

If an obstacle is present during the movement, the robot 10 that has detected the obstacle by the ultrasonic sensor 23 shifts to the autonomous mode and directs the head 11 in the direction of the obstacle. The resident 302 notices the presence of an obstacle from the movement of the CG image displayed on the screen of the operation side device 80 and the image of the head camera 17 that reflects the obstacle, and sees the obstacle displayed on the screen. At this time, since the movement of the head 11 of the robot 10 and the movement of the head of the person 302 are the same, the robot 10 returns to the mediation mode.
When the robot 10 returns to the mediation mode, the at-home person 302 will negotiate with the friend 303 about a change in the tour route. If the home person 302 does not notice the obstacle, the robot 10 in the autonomous mode avoids a collision with the obstacle.

  In this way, this communication robot can make people who cannot go to museums feel the pleasure of visiting.

  The communication robot of the present invention can be widely used as a guide robot in museums and various exhibitions, or as a welfare service robot that supports people who are sick or physically disabled.

The figure which shows typically the usage type of the robot in the 1st Embodiment of this invention. The block diagram which shows the structure of the robot in the 1st Embodiment of this invention. The block diagram which shows the structure of the operation side apparatus in the 1st Embodiment of this invention. The figure explaining the mode of the robot in the 1st Embodiment of this invention The figure which shows the modification of the usage pattern of the robot in the 1st Embodiment of this invention. The figure which shows typically the usage type of the robot in the 2nd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Communication robot 11 Head 12 Body 13 Cart 14 Wheel 16 Neck 17 Head camera 23 Ultrasonic sensor 24 Microphone 25 Speaker 40 Expert 51 Camera 52 CG image 61 Transmission / reception part 62 Image analysis part 63 Image processing part 64 Voice recognition part 65 Speech synthesis unit 66 Speech processing unit 67 Data storage unit 68 Control unit 69 Drive control unit 70 Obstacle detection unit 71 Travel drive unit 72 Arm drive unit 73 Neck drive unit 80 Operation side device 81 Transmission / reception unit 82 Image processing unit 83 Image analysis unit 84 Microphone 85 Speaker 86 Audio processing unit 87 Data storage unit 88 Control unit 89 CG creation unit 90 Display control unit 151 Arm 152 Arm 201 Torso camera 202 Torso camera 203 Torso camera 301 Visitor 302 Home visitor 303 Friends 501 to 503 Monitor 601 603 Data 831 face direction detection 832 pointing direction detection

Claims (6)

A communication robot that communicates with a first target person and communicates with a second target person through communication means,
A first mode that operates autonomously;
A second mode for performing an operation instructed by the second subject;
A communication robot that performs an operation reflecting the operation of the second target person and executes a third mode that mediates communication between the first target person and the second target person.   The communication robot according to claim 1, wherein the communication robot executes the first mode, the second mode, and the third mode by switching according to a situation.   The communication robot according to claim 1 or 2, wherein the second target person is informed of the behavior of the communication robot and the surrounding situation with the second target person. A communication robot having a transmission means for transmitting the movement of the robot to the communication robot.   The communication robot according to claim 1, wherein the communication robot autonomously communicates with the first target person in the first mode.   The communication robot according to claim 4, wherein an exhibit is explained to the first subject in the first mode, and a question from the first subject is explained in the third mode. A communication robot that mediates exchange with an answer by the second target person. The communication robot according to claim 3, wherein in the third mode, the communication robot mediates a conversation between the first target person in the hall and the second target person outside the hall.