JP5028568B2 - Robot control system - Google Patents

Description

  The present invention relates to a robot control system, and more particularly to a robot control system in which an operator remotely operates a communication robot capable of autonomous control as required.

For example, Patent Document 1 discloses an example of a conventional robot control system. In the technique of Patent Document 1, a control panel is displayed on a monitor of a personal computer operated by a user. On this control panel, an image taken by the robot camera, a distance to the object on the image detected by the robot distance sensor, a 3D CG image having the same posture as the current posture of the robot, and the like are displayed. The Further, when the touch sensor of the robot is touched, the corresponding part on the three-dimensional CG image is displayed in blue.
JP 2003-136455 A

  In the prior art, since only the information detected by the sensor provided on the robot was displayed on the control panel, the operator could not immediately grasp the environment where the robot was placed. . For example, in the case of remotely controlling a communication robot that often operates near a person, such as interacting with or touching a person, it is important to grasp the surrounding environment of the robot to ensure safety. However, in the prior art, even if the operator moves the head provided with the camera by remote control, only the images viewed from the viewpoint of the robot are sequentially displayed. Therefore, in the prior art, the surrounding environment of the robot cannot be grasped instantaneously, it is inefficient, and it has not been possible to immediately cope with changes in the environment. Thus, it is difficult to grasp the surrounding environment of the robot only by presenting information on the sensors installed in the robot, and the efficiency and safety of remote operation of the robot are not good.

  Therefore, a main object of the present invention is to provide a robot control system capable of efficiently operating a robot remotely.

  Another object of the present invention is to provide a robot control system that can improve the safety of remote operation of a robot.

The invention of claim 1 includes a communication robot existing in an environment provided with a plurality of types of environmental sensors, and an operation computer provided with an input device. The operation computer is operated by an operator in response to an input from the input device. A robot control system for identifying an operation and transmitting operation information for the operation to the robot, a plurality of environment display means for individually displaying the environment information detected using a plurality of types of environment sensors, and the environment includes a map display means for displaying a map showing a part or the whole of, a map display means, the current position robot position display means to display on a map the communication robot, which is detected based on detection information of the environmental sensor Note information display means for displaying warning information in the environment on the map, and a plurality of types of environments Sen placed in the environment Including environmental sensor position display means for displaying the respective position of a robot control system.

According to the first aspect of the present invention, the robot control system remotely operates the communication robot existing in the environment provided with the environment sensor in accordance with an instruction from the operator . In an embodiment described later, the environmental sensor includes a camera, a distance sensor, a sound level meter, a wireless tag reader, and the like. The environment display means displays the environment information detected using the environment sensor. The environmental information is, for example, sensor information detected by an environmental sensor, for example, and displays a camera image of a predetermined area, a noise level of the predetermined area, distance information from a specific object such as an exhibit to a person, and the like. . The map display means displays a map showing part or all of such an environment, the robot position display means displays the current position of the communication robot on the map, and the caution information display means displays detection information of the environment sensor. the warning information in the detected environment is displayed on the map based on, thereby to prompting attention to the operator when the remote operation of the communication robot. As this, since the information of the robot environment such as environmental information and warning information is displayed together with the current position of the communication robot, it is possible to easily grasp the surrounding environment of the communication robot operator. Therefore, the operator can remotely control the robot efficiently and safely.

The invention of claim 2 is dependent on the invention of claim 1, and the environment display means displays the environment information detected by the environment sensor selected by the operation of the operator.

In the invention of claim 2 , the operator can select the environmental sensor and display necessary environmental information while viewing the position of the robot and the position of the environmental sensor displayed on the map.
The invention according to claim 3 is the robot control system according to claim 1 or 2, wherein the map display means further includes environment sensor display means for displaying the positions of a plurality of types of environment sensors arranged in the environment. .
In the invention of claim 3, since the installation positions of the plurality of types of environmental sensors are displayed on the map , the operator's attention can be alerted when the robot is remotely operated.
According to a fourth aspect of the present invention, there is provided a program executed by a computer of a robot control system that remotely operates a communication robot existing in an environment provided with a plurality of types of environmental sensors in accordance with an instruction from an operator. Are operated as a plurality of environment display means for individually displaying environment information detected by using a plurality of types of environment sensors, and a map display means for displaying a map showing a part or the whole of the environment. is arranged the current position of the communication robot robot position display means to display on the map, note information display means a warning information in the detected environment displayed on the map on the basis of the detection information of the environmental sensors, and in the environment Environmental sensor position display hand that displays the position of each of multiple types of environmental sensors To function as a program.

  According to the present invention, information on the surrounding environment of the robot is displayed, so that the operator can easily grasp the surrounding environment of the robot. Therefore, the robot can be remotely operated efficiently and safely by the operator.

  The above object, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

  Referring to FIG. 1, a robot control system (hereinafter also simply referred to as “system”) 10 of this embodiment includes a robot 12 and an operation computer 14. This system 10 is for remotely operating a robot 12 from an operation computer 14. The robot 12 and the operation computer 14 are connected via a network 16 such as a wireless LAN or the Internet.

  The robot 12 is a communication robot in this embodiment. The communication robot 12 is a robot capable of interacting mainly with a communication target such as a human being, and has a function of executing a communication action including at least one of body movement such as gesture and gesture. As an example, the robot 12 may be a guide robot, a reception robot, or the like disposed in a predetermined place such as a museum, a science museum, an event venue, a company, a shopping street, or an underground mall. The robot 12 can basically interact with humans or play a predetermined role by autonomous control, but is remotely operated by an operator from the operation computer 14 as necessary.

  As will be described later, a remote operation screen is displayed on the operation computer 14 for an operator who remotely operates the robot 12. The screen of this embodiment includes an operation screen (see FIG. 6) for instructing the operation of the robot 12 and a sensor information screen (see FIG. 5) for displaying information on the environment in which the robot 12 is arranged. This sensor information screen provides the operator with information on the surrounding environment of the robot 12. The operator remotely operates the robot 12 by inputting an operation command on the operation screen while confirming environmental information on the sensor information screen.

  Therefore, the system 10 includes a sensor (environment sensor) that is provided in a predetermined place as described above where the robot 12 is arranged and detects environmental information of the place. In this embodiment, for example, a plurality of cameras 18, a plurality of wireless tag readers 20, a plurality of distance sensors 22, and a plurality of sound level meters 24 are provided as sensors embedded in such an environment. Computers 18 a, 20 a, 22 a, 24 a that acquire information detected by the sensors are connected to the environmental sensors 18, 20, 22, 24. Each computer 18a, 20a, 22a, 24a is connected to the network 16 by wire or wireless via a communication device. The computers 18 a, 20 a, 22 a, and 24 a acquire information detected by the sensors 18, 20, 22, and 24 at regular intervals, for example, and transmit the information to the operation computer 14 via the network 16.

  In another embodiment, each computer 18a, 20a, 22a, 24a to which each environmental sensor 18, 20, 22, 24 is connected transmits environmental sensor information to another computer via the network 16, and Another computer may provide the operation computer 14 with the environmental sensor information. Similarly, the sensor information of the robot 12 may be given to the operation computer 14 via the other computer. The operation information of the operation computer 14 may also be given to the robot 12 via another computer.

  FIG. 2 shows an example of a scene in which the robot 12 is active at the placement location. In FIG. 2, the robot 12 explains a specific object 26 such as an exhibit or a guide board to a human. In FIG. 2, the computers 18a, 20a, 22a, 24a of the environmental sensors 18, 20, 22, 24 are not shown.

  Referring also to FIG. 2, the plurality of cameras 18 are arranged in a distributed manner in the space where the robot 12 is arranged. That is, each of the plurality of cameras 18 is installed in a predetermined direction at a predetermined position such as a ceiling, a wall, or a stand so as to photograph a predetermined region or range in the environment where the robot 12 is arranged. A video or image of a predetermined area captured by each environmental camera 18 is given from each computer 18 a to the operation computer 14. The operation computer 14 stores the positional information (coordinates, installed location or area, etc.) of each environmental camera 18 in association with the identification information. In addition, when the place where the robot 12 is arranged is small or the activity range of the robot 12 is small, the environment camera 18 may be one.

  The plurality of RFID tag readers 20 are arranged in a distributed manner in the space where the robot 12 is arranged. That is, each of the plurality of wireless tag readers 20 is installed at a predetermined position such as a ceiling, a wall, a floor, or a stand in order to estimate the position of a wireless tag (not shown) in the environment where the robot 12 is arranged. In this embodiment, the arrangement of the wireless tag readers 20 is determined so that the wireless tags can be detected by at least three wireless tag readers 20. For example, the wireless tag reader 20 can control the reception sensitivity in a stepwise manner. Therefore, it is possible to detect a rough distance from the wireless tag reader 20 to the wireless tag. The identification information and distance information of the wireless tag detected by each environmental wireless tag reader 20 are given to the operation computer 14 from each computer 20a. The operation computer 14 stores the position information (coordinates, installed location or area, etc.) of each wireless tag reader 20 in association with the identification information. If the wireless tag is detected by at least three wireless tag readers 20 at the same time, the operation computer 14 can estimate the position of the wireless tag based on the position information and the detection distance information of the wireless tag reader 20. In this embodiment, the wireless tag is attached to the robot 12, so that the current position of the robot 12 is estimated and grasped. If the wireless tag is attached to a person, the position of the person can also be estimated.

  Note that the method for detecting the position of the robot 12 is not limited to the above-described method, and may be changed as appropriate. In another embodiment, for example, a plurality of infrared cameras are installed in the environment and an infrared LED is attached to the robot 12 so that the operation computer 14 detects the position of the robot 12 by image processing of the infrared camera image. Also good. Further, a floor sensor including a large number of detection elements (pressure sensors) is provided on the floor of the space where the robot 12 is arranged, and the operation computer 14 detects the position of the robot 12 based on the detection pressure signal of the floor sensor. You may make it do.

  Each of the plurality of distance sensors 22 is arranged in front of the object 26 in order to detect whether a human is present in front of a specific object 26 such as an exhibit or a guide board in the environment where the robot 12 is arranged. It is provided at a predetermined position in the vicinity thereof. The distance sensor 22 is an ultrasonic distance sensor or an infrared distance sensor. The distance information measured by each environmental distance sensor 22 is given to the operation computer 14 from each computer 22a. The operation computer 14 stores the position information (coordinates, installed location, area or object 26) of each distance sensor 22 in association with the identification information. In addition, when only one specific object 26 is provided in the environment, the environment distance sensor 22 may be one. Alternatively, a plurality of distance sensors 22 may be arranged in a specific area such as a passage to detect how many people exist in that area.

  The plurality of sound level meters 24 are distributed in the space where the robot 12 is disposed. That is, each of the plurality of sound level meters 24 is provided at a predetermined position such as a ceiling, a wall, or a stand in order to detect a noise level in a predetermined region in the environment where the robot 12 is arranged. The noise level information detected by each environmental sound level meter 24 is given to the operation computer 14 from each computer 24a. The operation computer 14 stores the position information (coordinates, installed location or area, etc.) of each sound level meter 24 in association with the identification information. Note that there may be only one environmental sound level meter 24 for the same reason as in the case of the environmental camera 18 described above.

  FIG. 3 shows an example of the appearance of the robot 12 as viewed from the front, and FIG. 4 shows an example of the electrical configuration of the robot 12. Referring to FIG. 3, the robot 12 includes a carriage 28, and wheels 30 for autonomously moving the robot 12 are provided on the lower surface of the carriage 28. The wheel 30 is driven by a wheel motor (indicated by reference numeral “32” in FIG. 4), and the carriage 28, that is, the robot 12 can be moved in any direction. Although not shown in FIG. 3, a collision sensor (indicated by reference numeral “34” in FIG. 4) is attached to the front surface of the carriage 28. Detects contact with obstacles. When contact is detected while the robot 12 is moving, the driving of the wheels 30 can be stopped immediately.

  A polygonal column sensor mounting panel 36 is provided on the carriage 28, and an ultrasonic distance sensor 38 is mounted on each surface of the sensor mounting panel 36. This ultrasonic distance sensor 38 is for measuring the distance between the mounting panel 36, that is, the person around the robot 12 mainly.

  Further, the body of the robot 12 is mounted on the carriage 28 so that the lower part of the body is surrounded by the mounting panel 36 described above. The body includes a lower body 40 and an upper body 42, and the lower body 40 and the upper body 42 are connected by a connecting portion 44. Although not shown, the connecting portion 44 has a built-in lifting mechanism, and by using this lifting mechanism, the height of the upper body 42, that is, the height of the robot 12 can be changed. The lifting mechanism is driven by a waist motor (indicated by reference numeral “46” in FIG. 4).

  One omnidirectional camera 48 is provided on the upper body 42. The omnidirectional camera 48 is installed, for example, on a column 50 extending from substantially the center of the upper end on the back side. The omnidirectional camera 48 captures the surroundings of the robot 12 and is distinguished from an eye camera 52 described later. As this omnidirectional camera 48, for example, a camera using a solid-state imaging device such as a CCD or a CMOS can be adopted. In addition, one microphone 54 is provided at approximately the center of the front side of the upper body 42. The microphone 54 captures ambient sounds, particularly human voices that are communication targets. The installation positions of the omnidirectional camera 48 and the microphone 54 are not limited to the upper body 40 and can be changed as appropriate.

  Upper arms 58R and 58L are attached to both shoulders of the upper body 42 by shoulder joints 56R and 56L, respectively. The shoulder joints 56R and 56L each have three degrees of freedom. That is, the right shoulder joint 56R can control the angle of the upper arm 58R around each of the X, Y, and Z axes. The Y axis is an axis parallel to the longitudinal direction (or axis) of the upper arm 58R, and the X axis and the Z axis are axes orthogonal to the Y axis from different directions. The left shoulder joint 56L can control the angle of the upper arm 58L around each of the A, B, and C axes. The B axis is an axis parallel to the longitudinal direction (or axis) of the upper arm 58L, and the A axis and the C axis are axes orthogonal to the B axis from different directions.

  Forearms 62R and 62L are attached to the respective distal ends of upper arms 58R and 58L via elbow joints 60R and 60L. The elbow joints 60R and 60L can control the angles of the forearms 62R and 62L around the W axis and the D axis, respectively.

  In the X, Y, Z, W axes and A, B, C, and D axes that control the displacement of the upper arms 58R and 58L and the forearms 62R and 62L, “0 degree” is the home position. The upper arms 58R and 58L and the forearms 62R and 62L are directed downward.

  Although not shown in FIG. 3, touch sensors (FIG. 4) are provided on the shoulder portion including the shoulder joints 56R and 56L of the upper body 42 and the arm portion including the upper arms 58R and 58L and the forearms 62R and 62L. The touch sensor 64 detects whether or not a person has touched these parts of the robot 12.

  Spheres 66R and 66L corresponding to hands are fixedly attached to the tips of the forearms 62R and 62L, respectively. In place of the spheres 66R and 66L, a “hand” in the shape of a human hand can be used when a finger function is required unlike the robot 12 of this embodiment.

  A head 70 is attached to an upper center of the upper body 42 via a neck joint 68. The neck joint 68 has three degrees of freedom and can be controlled in angle around each of the S, T, and U axes. The S-axis is an axis that goes directly from the neck, and the T-axis and the U-axis are axes that are orthogonal to the S-axis in different directions. The head 70 is provided with a speaker 72 at a position corresponding to a human mouth. The speaker 72 may be used for the robot 12 to communicate with the surrounding people by voice or voice. Further, the speaker 72 may be provided in another part of the robot 12, for example, the body.

  The head 70 is provided with eyeball portions 74R and 74L at positions corresponding to the eyes. Eyeball portions 74R and 74L include eye cameras 52R and 52L, respectively. Note that the left and right eyeball portions 74R and 74L may be collectively denoted by reference numeral “74”, and the left and right eye cameras 52R and 52L may be collectively denoted by reference numeral “52”. The eye camera 52 captures the video signal by photographing the face of the person approaching the robot 12 and other parts or objects.

  Note that each of the omnidirectional camera 48 and the eye camera 52 described above may be a camera using a solid-state imaging device such as a CCD or a CMOS.

  For example, the eye camera 52 is fixed in the eyeball part 74, and the eyeball part 74 is attached to a predetermined position in the head 70 via an eyeball support part (not shown). The eyeball support unit has two degrees of freedom and can be controlled in angle around each of the α axis and the β axis. The α axis and the β axis are axes set with respect to the head 70, the α axis is an axis in a direction toward the top of the head 70, the β axis is orthogonal to the α axis and the front side of the head 70 It is an axis in a direction orthogonal to the direction in which (face) faces. In this embodiment, when the head 70 is at the home position, the α axis is set to be parallel to the S axis and the β axis is set to be parallel to the U axis. In such a head 70, when the eyeball support portion is rotated around each of the α axis and the β axis, the tip (front) side of the eyeball portion 74 or the eye camera 52 is displaced, and the camera axis, that is, the line-of-sight direction is changed. Moved.

  In the α axis and β axis that control the displacement of the eye camera 52, “0 degree” is the home position, and at this home position, the camera axis of the eye camera 52 is the head 70 as shown in FIG. The direction of the front side (face) is directed, and the line of sight is in the normal viewing state.

  Referring to FIG. 4, this robot 12 includes a microcomputer or CPU 76 for overall control, and this CPU 76 is connected to a memory 80, a motor control board 82, a sensor input / output board 84 and a voice through a bus 78. An input / output board 86 is connected.

  Although not shown, the memory 80 includes a ROM, an HDD, a RAM, and the like, and a program and data for controlling the operation of the robot 12 are stored in the ROM or the HDD in advance. The CPU 76 executes processing according to this program. Examples of the program include a detection program for detecting detection information by each sensor, a communication program for transmitting / receiving necessary data to / from an external computer such as the operation computer 14, an autonomous control program for autonomously controlling operation, and received operation information There is a remote control program that controls the operation based on this. Individual communication behaviors such as greeting, whispering, pointing, etc. are realized by each modularized behavior program (behavior module), and a plurality of behavior programs are stored in the memory 80 in association with each identification information (operation command). Has been. The CPU 76 can autonomously control the operation of the robot 12 itself by determining the next action based on a plurality of rules and the current situation. The memory 80 also stores voice data or voice data (speech synthesis data) to be generated when executing each action, angle data for presenting a predetermined gesture, and the like. The RAM is used as a buffer memory or a working memory.

  The motor control board 82 is composed of, for example, a DSP (Digital Signal Processor) and controls a motor for driving body parts such as the right arm, the left arm, the head, and the eyes. That is, the motor control board 82 receives control data from the CPU 76, and controls the angles of the three motors for controlling the X, Y, and Z axes of the right shoulder joint 56R and the axis W of the right elbow joint 60R. The rotation angle of a total of four motors including one motor (collectively shown as “right arm motor” in FIG. 4) 88 is adjusted. The motor control board 82 includes a total of four motors including three motors that control the angles of the A, B, and C axes of the left shoulder joint 56L and one motor that controls the angle of the D axis of the left elbow joint 60L. The rotation angle of the motor (collectively shown as “left arm motor” in FIG. 4) 90 is adjusted. The motor control board 82 also adjusts the rotation angle of three motors (collectively shown as “head motors” in FIG. 4) 92 that control the angles of the S, T, and U axes of the neck joint 68. . The motor control board 82 also controls the waist motor 46 and the two motors 32 that drive the wheels 30 (collectively shown as “wheel motors” in FIG. 4). Furthermore, the motor control board 82 adjusts the rotation angle of two motors 94 (collectively shown as “right eyeball motors” in FIG. 4) that control the angles of the α axis and β axis of the right eyeball portion 74R. In addition, the rotation angle of two motors 96 that collectively control the angles of the α axis and β axis of the left eyeball portion 74L (collectively shown as “left eyeball motor” in FIG. 4) 96 is adjusted.

  The above-described motors of this embodiment are stepping motors or pulse motors, respectively, for simplifying the control except for the wheel motors 32. However, like the wheel motors 32, they may be DC motors. In this embodiment, a motor using electric power as a driving source is used as an actuator for driving a body part such as an arm, a head, and an eye of the robot 12. However, the robot 12 may be a robot that drives a body part by other actuators such as air pressure (or negative pressure), oil pressure, a piezoelectric element, or a shape memory alloy.

  Similarly, the sensor input / output board 84 is configured by a DSP, and takes in signals from each sensor and gives them to the CPU 76. That is, data relating to the reflection time from each of the ultrasonic distance sensors 38 is input to the CPU 76 through the sensor input / output board 84. The video signal from the omnidirectional camera 48 is input to the CPU 76 after being subjected to predetermined processing by the sensor input / output board 84 as necessary. Similarly, the video signal from the eye camera 52 is also supplied to the CPU 76. Further, signals from the touch sensor 64 and the collision sensor 34 are given to the CPU 76 via the sensor input / output board 84. In addition, angle information from the joint angle sensor 98 is given to the CPU 76 via the sensor input / output board. The joint angle sensor 98 is a rotary encoder, a potentiometer, or the like that detects the rotation angle of each axis of each joint 56, 60, 68.

  Audio data is given to the speaker 72 from the CPU 76 via the audio input / output board 86, and accordingly, audio or voice according to the data is output from the speaker 72. Also, the voice input from the microphone 54 is taken into the CPU 76 as voice data via the voice input / output board 86.

  A communication LAN board 100 is further connected to the CPU 76. Similarly, the communication LAN board 100 is configured by a DSP, and sends transmission data given from the CPU 76 to the wireless communication device 102, and the data is sent from the wireless communication device 102 to an external computer such as the operation computer 14 via the network 16. Send it. Further, the communication LAN board 100 receives data from an external computer such as the operation computer 14 via the network 16 and the wireless communication device 102 and supplies the received data to the CPU 76.

  Although not shown, the operation computer 14 includes a CPU, and a memory, a display device, an input device, a speaker, a communication device, and the like are connected to the CPU. The operation computer 14 is connected to the network 16 via a communication device in a wired or wireless manner.

  The memory of the operation computer 14 stores a program for controlling the operation and necessary data. The program includes, for example, a communication program for transmitting / receiving necessary data to / from an external computer such as the robot 12 and each computer 18a, 20a, 22a, 24a, environment information, information on the robot 12, and the like. An information display program for displaying on the apparatus, an operation program for specifying an operation by an operator according to an input from the input device of the operation computer 14 and transmitting operation information to the robot 12 are included. In addition, position information of each environmental sensor, environmental map information, and the like are also stored.

  A remote operation screen including a sensor information screen as shown in FIG. 5 and an operation screen as shown in FIG. 6 is displayed on the display device of the operation computer 14. The operation computer 14 obtains necessary information such as sensor information and the action state of the robot 12 from the computers 18a, 20a, 22a, 24a and the robot 12 through the communication device at regular intervals, for example. Generate and update screens. An input device of the operation computer 14 is a mouse, a keyboard, a touch panel, or the like, and an operator can input an operation command for remotely operating the robot 12 on the operation screen by operating the input device. Further, the auditory information acquired by the microphone 54 of the robot 12 may be output from the speaker of the operation computer 14 as necessary.

  The environmental information detected using the environmental sensors 18, 20, 22, 24 is displayed on the sensor information screen of FIG. In this embodiment, sensor information of the robot 12 is also displayed. Specifically, an environmental camera window 110, a robot distance sensor window 112, a robot motion window 114, a sound level meter window 116, an environmental distance sensor window 118, a map window 120, and the like are displayed. The environmental camera window 110, the sound level meter window 116, and the environmental distance sensor window 118 display environmental sensor information detected by each environmental sensor, and the map window 120 shows the environment detected based on the environmental sensor information. Information to be noted in remote control of is displayed.

  Note that the type of information to be displayed on the sensor information screen is determined in advance by initial setting, and can be selected by an operator from a display setting menu (not shown). A display flag relating to information selected by the initial setting or the operator is turned on, and information on which the display flag is on is displayed on the sensor information screen. Therefore, only sensor information and environment information necessary for remote operation of the robot 12 can be displayed on the sensor information screen, so that the operator can efficiently control the behavior of the robot 12.

  The environment camera window 110 displays an image or image of the environment taken by the environment camera 18. In the upper title field of the window 110, the name or identification information (EC1, EC2,...) Of the environmental camera 18 is displayed. For example, based on the current position of the robot 12, the environmental camera 18 that includes the position in the imaging range is specified, the display flag corresponding to the environmental camera 18 is turned on, and the window 110 is displayed. Further, the display flag of the environmental camera 18 within a predetermined distance from the current position of the robot 12 may be turned on. If the total number of environment cameras 18 is small, the display flags of all environment cameras 18 may be turned on. In addition, the operator can select a desired display of the environmental camera 18 from a plurality of environmental cameras 18 using a display setting menu (not shown). The operator can view images around and near the place where the robot 12 is present through this window 110, and can view an image of a place he / she wants to see by menu selection.

  In the robot distance sensor window 112, distance information detected by the ultrasonic distance sensor 38 of the robot 12 is displayed. In the robot 12, for example, a plurality of distance sensors 38 are provided at predetermined angular intervals all around the robot 12. Therefore, the operator can confirm the distance to the object (such as a human being or an obstacle) existing around the robot 12 and its direction through the window 112.

In the robot motion window 114, the posture or movement of the robot 12 based on the angle information detected by the joint angle sensor 98 of the robot 12 is displayed as, for example, a three-dimensional CG image. The operator can confirm the current posture or movement of the robot 12, specifically, the orientation of the head 70 and the orientations of both arms (upper arm 58 and forearm 62) by this window 114.

  In the sound level meter window 116, the noise level detected by the environmental sound level meter 24 is displayed. In the upper title column of the window 116, the name or identification information (EN1,...) Of the environmental sound level meter 24 is displayed. For example, the sound level meter 24 that detects the noise in the area including the current position of the robot 12 is specified, the display flag corresponding to the sound level meter 24 is turned on, and the window 116 is displayed. Further, the display flag of the sound level meter 24 within a predetermined distance from the current position of the robot 12 may be turned on. In addition, the operator can select a desired sound level meter 24 from among a plurality of environmental sound level meters 24 using the display setting menu. The operator can confirm the noise level around or near the place where the robot 12 is present by using this window 116, and can confirm the noise level of the place he / she wants to check by selecting a menu.

  In the environmental distance sensor window 118, distance information detected by the environmental distance sensor 22 is displayed. In the upper title column of the window 118, the name or identification information (ED1,...) Of the environmental distance sensor 22 is displayed. For example, the environmental distance sensor 22 closest to the current position of the robot 12 or within a predetermined distance is specified, the display flag corresponding to the environmental distance sensor 22 is turned on, and the window 118 is displayed. In addition, the operator can select a desired environmental distance sensor 22 from the plurality of environmental distance sensors 22 using the display setting menu. When the environmental distance sensor 22 is installed in association with an object 26 such as an exhibit, the name of the object 26 may be displayed in the selection list. When the environmental distance sensor 22 is installed in association with a specific place or region such as a passage, the name of the place may be included in the selection list. The environmental distance sensor 22 can detect whether or not a person is present in front of the distance sensor 22, and when installing a plurality of environmental distance sensors 22 in a specific place or region, It is possible to detect how many people are in the area. In the environmental distance sensor window 118, for example, an icon indicating a person or the like existing in the detection area of the environmental distance sensor 22 is displayed at a position away from the origin by a detection distance in the detection direction. The window 118 allows the operator to check whether or not there is a person or the like in the object 26 or the place near the current position of the robot 12 and how many persons are present. It is possible to confirm the presence of a human or the like regarding the object 26 or place to be known.

  In the map window 120, for example, a map showing the space or place where the robot 12 such as a museum is arranged is displayed. This map may show the entire environment in which the robot 12 is arranged, or a part thereof. And the information regarding the environment where the robot 12 detected based on environmental sensor information is arrange | positioned is displayed on this map. In this embodiment, attention information in the environment is displayed as the environment information, and specifically, a congested area and a troublesome area are shown.

  The crowded area is detected based on the captured image of each environmental camera 18. Specifically, since each environmental camera 18 is fixedly provided at a predetermined position and in a predetermined direction, a certain area is always photographed. By paying attention to a specific area in this captured image, and estimating the amount of difference between the background image acquired in advance in the specific area and the captured image, it is estimated how many people or objects exist in that area. Can do. For example, a congested area is calculated using a discriminant such as a congestion if the difference occurrence area is a predetermined ratio (for example, 50%) or more of the specific area. On the map, the congested area is displayed using a mark of a predetermined color (for example, red) that is different from the map background color and different from the color indicating other environmental information. In FIG. 5, this congested area is indicated by a circle with a hatched pattern for convenience. For example, when the difference image is presented to the operator as it is, it is difficult for the operator to instantly determine which area or place is congested. It is very easy to know where you are doing.

  The annoying area is detected based on the noise level measured by each sound level meter 24. Specifically, when the noise level is equal to or higher than a predetermined threshold, it is determined that the area or place where the sound level meter 24 is installed is a troublesome area. On the map, the troublesome area is displayed using a mark of a predetermined color (for example, yellow) that is different from the map background color and different from the color indicating other environmental information. In FIG. 5, this troublesome area is indicated by a circle with a checkered pattern for convenience. For example, rather than looking at the value (noise level) measured by the sound level meter 24 as it is, the operator can know the troublesome place very easily by using such a display form.

In this way, the map indicating the environment in which the robot 12 exists is displayed, and the environment information is displayed on the map. Therefore, the environment information can be provided to the operator in an easily understandable manner. Further, in this embodiment, as the environment information, a place that is likely to interfere with the movement of the robot 12 or the body movement in the environment such as a congested area or a troubled area is displayed. That is, information for alerting or prompting the operator to perform remote operation of the robot 12 is displayed. As described above, the caution information in the environment where the robot 12 exists can be provided very easily.

  Further, on this map, a mark or the like (black triangle in FIG. 5) indicating the robot 12 is displayed at a position corresponding to the current position of the robot 12. In this embodiment, as described above, the position of the robot 12 can be estimated by attaching the wireless tag to the robot 12 and detecting the distance information to the wireless tag by the wireless tag reader 20. The operator can grasp at a glance the positional relationship between the current robot 12 and the congested area or the troubled area by viewing the above-described environmental information together with the current position of the robot 12 on the map. For example, when the operator determines that it is necessary to avoid inputting a movement operation command when the robot 12 is present in a congested area or a troubled area or referring to the current motion of the robot 12 in the window 114. In this case, it is easy to make remote operation determinations such as refraining from input of operation commands including body movements, or avoiding movement toward the area when the robot 12 is outside a congested area or annoying area. be able to.

  Further, on the map, the position of the sensors 18, 20, 22, 24 installed in the environment, the position of the object 26 such as an exhibit, etc. may be displayed. In FIG. 5, the installation position of the wireless tag reader 20 is indicated using a mark (white square) indicating the wireless tag reader 20. Although not shown in FIG. 5, names or identification information of the sensors 18, 20, 22, 24 and the object 26 may be attached to the marks. Thus, when the position of each sensor 18, 20, 22, 24, object 26, etc. is shown on the map, the window 110, 116, 118 of which sensor 18, 20, 22, 24 is displayed on the sensor information screen. It becomes possible to make it easy for the operator to decide whether to open the door. For example, since it is possible to easily identify the environmental camera 18 that captures a congested area or an annoying area, the operator selects the display of the window 110 of the environmental camera 18 from the display setting menu, so that the congested area or the annoying area is selected. You can check the actual situation on the video.

  The operator can remotely operate the robot 12 by inputting an operation command on the operation screen as shown in FIG. 6 while easily grasping the environmental information on the sensor information screen as shown in FIG.

  The operation screen includes a first area 130 indicating the current state of the robot 12 and a second area 132 indicating an operation command. Specifically, in the first area 130, the current state (state) of the robot 12 and the communication behavior being executed are displayed in characters. In the example illustrated in FIG. 6, the robot 12 is in the “IDLE” state, and it is indicated that the communication behavior “ASOBO” is being executed. When the robot 12 is acting autonomously without being instructed by the operator, the state is “IDLE”. Conversely, when the robot 12 is performing communication behavior according to the instruction to the operator, the state is “BUSY”. is there. The communication behavior is executed according to a program called a behavior module for realizing the behavior. The communication behavior displayed in the first area 130 may be the name or identification information (that is, the operation command) of the behavior module. As described above, the robot 12 transmits the behavior state together with the sensor information to the operation computer 14 at regular time intervals, for example, and the display of the first region 130 is performed based on the received behavior state.

  Further, an end button (icon) 134 is displayed in the first area 130. The end button 134 is for closing the operation screen, that is, for ending the remote operation of the robot 12.

  In the second area 132, an operation command instructed by the operator is displayed. For example, display areas 136, 138, 140, 142 are provided. In the display area 136-140, operation commands for instructing each communication action are displayed. The operation command names are classified into three types: behavior when the robot 12 is busy (when there are few people around the robot 12), behavior when crowded (when there are many people around the robot 12), and normal operation. Is displayed. This improves the efficiency of operator selection. However, operation commands for all communication actions may be displayed in one display area without classification. Although omitted in the drawings, since many command names are displayed, a scroll bar for scrolling the display contents is actually provided. The same applies to other display areas.

  In the display area 142, a command history is displayed. That is, the operation command name about the communication action transmitted to the robot 12 and executed is displayed in time series, for example. Thereby, the operator can confirm the action history of the robot 12 executed by remote operation.

  In the second area 132, a command transmission button 144, an emergency stop button 146, a left button 148, a front button 150, and a right button 152 are further provided. Further, as buttons relating to the movement command, a forward button 154, a left rotation button 156, a right rotation button 158, a backward button 160, and a stop button 162 are provided. An end button 164 for ending the operation of the robot 12 is also provided.

  The command transmission button 144 is for transmitting the operation command selected by the operator to the robot 12. The emergency stop button 146 is for urgently stopping the movement of the robot 12.

  The left button 148 is for rotating the neck joint 68 of the robot 12 to the left by a predetermined angle. For example, each time this button 148 is clicked, the head 70 of the robot 12 is rotated by a predetermined angle (for example, 5 degrees) leftward (S axis). The front button 150 is for directing the neck joint 68 of the robot 12 in the front direction. When this button 150 is clicked, the neck joint 68 is controlled so that the face of the robot 12 (front of the head 70) faces the front. The right button 152 is for rotating the neck joint 68 of the robot 12 to the right by a predetermined angle. For example, each time this button 152 is clicked, the head 70 of the robot 12 is rotated by a predetermined angle (for example, 5 degrees) in the right direction (S axis).

  The advance button 154 is for advancing the robot 12. For example, the robot 12 moves forward when the button 154 is clicked, and stops when the button 162 is clicked and a stop command is given. The left rotation button 156 is a button for turning the robot 12 to the left. For example, the robot 12 turns left when the button 156 is clicked, and stops when a stop command is given. The right rotation button 158 is for turning the robot 12 to the right. For example, the robot 12 turns right when the button 158 is clicked, and stops when a stop command is given. The retreat button 160 is for retreating the robot 12. For example, the robot 12 moves backward when the button 160 is clicked, and stops when a stop command is given. The stop button 162 is for giving a stop command to the moving robot 12. For example, when this button 162 is clicked, the robot 12 stops moving forward, rotating left, rotating right, or moving backward.

  The end button 164 is for ending (stopping) the operation of the robot 12. For example, when the button 164 is clicked, an end command is given to the robot 12, and the robot 12 stops supplying power to the circuit components as shown in FIG. 4 after returning each part to the home position.

  FIG. 7 and FIG. 8 show an example of the operation in the information display process of the CPU of the operation computer 14. This information display process is repeatedly executed at regular intervals. The information display process is executed in parallel with other processes such as the operation process of FIG.

  When the information display process is started, first, in step S1, information of each sensor 18, 20, 22, 24 installed in the environment is acquired from each computer 18a, 20a, 22a, 24a via the network 16. Next, in step S <b> 3, information about each sensor installed in the robot 12 is acquired via the network 16. For example, the information of the ultrasonic distance sensor 38 provided in the robot 12 and the information of the joint angle sensor 98 are acquired. However, information on other sensors 34, 48, 52, 54, and 64 may also be acquired.

  In step S5, the position of the robot 12 is estimated. That is, as described above, the current position of the wireless tag attached to the robot 12 is calculated based on the detection information of the wireless tag reader 20 provided in the environment and the position information stored in advance.

  Subsequently, in step S7, attention information (congested area and troublesome area in this embodiment) as environment information is detected. That is, as described above, the congested area is detected based on the difference image of the captured image of the environmental camera 18, and the troublesome area is detected based on the noise level detected by the environmental sound level meter 24. When there is a predetermined area such as a passage where a plurality of environmental distance sensors 22 are arranged, it is determined based on information detected by the plurality of environmental distance sensors 22 whether or not the predetermined area is congested. You may do it.

  In step S9, a display flag is set. For example, at the start of the information display process, the display flag of each window 110-120 is set based on the initial setting data stored in advance in the memory. When a display setting menu selection operation is performed, a setting screen (not shown) is displayed on the display device, and each sensor information and map information displayed on the window 110-120 in the sensor information screen is selected by the operator. Make it. In response to the operator's selection, a display flag corresponding to the selected sensor information or map information is turned on. The display flag may be automatically set based on the current position of the robot 12 estimated in step S5 and the position information of each environmental sensor stored in advance in the memory. For example, the environmental camera 18 in which the current position of the robot 12 is included in the imaging region is specified, and the display flag of the window 110 of the environmental camera 18 is turned on. Alternatively, each sensor 18, 22, and 24 existing within a predetermined distance from the current position of the robot 12 may be specified, and the display flag of the sensor window 110-118 may be turned on.

  In subsequent steps S11 to S43, display processing of the window 110-120 is performed according to the display flag. Specifically, in step S11, it is determined whether information of each environmental camera 18 at the current time has been received. If “YES” in the step S11, an environmental camera window 110 showing a captured image of the environmental camera 18 whose display flag is on is displayed in a step S13.

  If “NO” in the step S11, or when the step S13 is ended, it is determined whether or not the information of each environmental distance sensor 22 at the current time is received in a step S15. If “YES” in the step S15, an environmental distance sensor window 118 showing the distance information of the environmental distance sensor 22 whose display flag is turned on is displayed in a step S17.

  If “NO” in the step S15, or when the step S17 is ended, it is determined whether or not the information of the environmental noise meters 24 at the current time is received in a step S19. If “YES” in the step S19, a sound level meter window 116 indicating the noise level of the environmental sound level meter 24 whose display flag is turned on is displayed in a step S21. If “NO” in the step S19, or when the step S21 is ended, the process proceeds to a step S23 in FIG.

  In step S23 of FIG. 8, it is determined whether or not the information of the joint angle sensor 98 of the robot 12 at the current time has been received. If “YES” in the step S23, it is determined whether or not a display flag related to the movement of the robot 12 is turned on in a step S25. If “YES” in the step S25, a robot motion window 114 showing the movement of the robot 12 is displayed in a step S27.

  If “NO” in step S23 or S25, or when step S27 ends, it is determined in step S29 whether information of the ultrasonic distance sensor 38 of the robot 12 at the current time has been received. If “YES” in the step S29, it is determined whether or not a display flag regarding the distance sensor information of the robot 12 is turned on in a step S31. If “YES” in the step S31, the robot distance sensor window 112 showing the distance sensor information of the robot 12 is displayed in a step S33.

  If “NO” in step S29 or S31, or when step S33 ends, it is determined in step S35 whether the display flag relating to the map is on. If “YES” in the step S35, the map window 120 is displayed in a step S37, and a map of the environment where the robot 12 exists is displayed.

  Subsequently, in step S39, the mark of the robot 12 (black triangle in FIG. 5) is displayed on the map at a position corresponding to the current position calculated in step S5.

  In step S41, it is determined whether or not a display flag relating to environmental information is on. If “YES” in the step S41, environmental information, in this embodiment, environmental caution information is displayed on the map in a step S43. Specifically, on the map, a mark indicating a congested area (a circle with a diagonal line pattern in FIG. 5) or the like is displayed at a position corresponding to the congested area detected in step S7. Further, a mark indicating a troublesome area (circle with a lattice pattern in FIG. 5) or the like is displayed at a position corresponding to the troublesome area detected in step S7.

  If “NO” in the step S35, if “NO” in the step S41, or when the step S43 is ended, it is determined whether or not an end instruction is issued in a step S45. That is, it is determined whether or not a menu for instructing to end the display of the sensor information screen is selected by the operator's operation of the input device. If “NO” in the step S45, the process returns to the step S1 in FIG. 7 to repeat the process, and the sensor information screen is updated. On the other hand, if “YES” in the step S45, the sensor information screen is closed and the information display processing is ended. Note that when an end instruction is given on the sensor information screen, the operation process of FIG. 9 is also ended.

  FIG. 9 shows an example of the operation in the operation process of the CPU of the operation computer 14. When the operation process is started, first, in step S61, an operation screen as shown in FIG. 6 is displayed. The operation screen and the sensor information screen are displayed side by side on the display device, for example.

  The subsequent steps S63 to S71 are repeatedly executed at regular intervals. That is, in step S63, the action state of the robot 12 is acquired via the network 16, and the operation state, the communication action being executed, and the like are displayed in the first area 130 of the operation screen.

  Subsequently, in step S65, it is determined whether or not an operation command has been selected based on the input data from the input device and the operation command and button arrangement data stored in the memory. Specifically, in the case of a command instructing a communication action, it is determined whether or not the command transmission button 144 is selected in a state where the command is selected in the display areas 136, 138 and 140. In the case of a command assigned to each button 146-164, it is determined whether or not the button 146-164 has been selected.

  If “YES” in the step S65, the selected operation command is specified in a step S67, and the operation command is transmitted to the robot 12 via the network 16. Subsequently, in step S69, the transmitted operation command is added to the display area 142, and the command history is displayed in time series.

  If “NO” in the step S65, or when the step S69 is ended, it is determined whether or not an end instruction is issued in a step S71. Specifically, it is determined whether or not the end button 134 has been selected. If “NO” in the step S71, the process returns to the step S63 to repeat the process, update the display of the first area 130 of the operation screen as described above, and transmit the operation command according to the selection of the operator. On the other hand, if “YES” in the step S71, the operation screen is closed and the operation process is ended. Note that when an end instruction is given on the operation screen, the information display processing in FIGS. 7 and 8 is also ended.

  According to this embodiment, since not only the sensor information of the robot 12 but also the sensor information of the environment where the robot 12 exists is displayed, the information on the surrounding environment of the robot 12 can be provided to the operator. In addition, since the environmental information is displayed in an easy-to-understand manner, for example, the caution information in the environment such as a congested area or a troubled area is displayed on a map, the surrounding environment of the robot 12 can be very easily and instantly displayed to the operator. It can be grasped. For this reason, for example, it can be expected that the time until the operator decides the behavior control of the robot 12 is reduced. Accordingly, the operator can efficiently perform remote operation of the robot 12.

  In addition, by displaying caution information in an environment such as a congested area or an annoying area, it is possible to easily grasp whether or not the robot 12 is in a situation that hinders movement or physical movement, or for physical movement or movement. It is possible to easily grasp where the trouble is likely to be. As described above, since the operator can accurately grasp the surrounding environment and the situation of the robot 12 and control the action thereof, the efficiency of the remote operation of the robot 12 and the safety can be improved.

  Furthermore, since the efficiency and safety of remote control of the autonomously controllable robot 12 can be improved as described above, for example, one operator can take charge of remote operation of a plurality of robots 12 at the same time. That is, a small number of operators can control many robots 12 simultaneously.

  In the above-described embodiment, the operation computer 14 receives each environment sensor information from each computer 18a, 20a, 22a, 24a to which each environment sensor 18, 20, 22, 24 is connected, and the robot 12 The sensor information was received from. However, in another embodiment, another computer such as a central control device connected to the network 16 acquires the environmental sensor information or the sensor information of the robot 12, and the computer uses the sensor information as the operation computer 14. You may make it give to.

  Further, in each of the above-described embodiments, the operation computer 14 calculates the current position of the robot 12, a congested area, a troublesome area, and the like. However, in another embodiment, another computer such as the central control device may calculate the information necessary for displaying the sensor information screen and give it to the operation computer 14.

It is an illustration figure which shows the structure of the robot control system of one Example of this invention. It is an illustration figure which shows an example of the activity scene of the robot of FIG. 1 Example. It is an illustration figure which shows an example of the external appearance which looked at the robot of FIG. 1 Example from the front. It is a block diagram which shows an example of the electrical structure of the robot of FIG. 1 Example. It is an illustration figure which shows an example of the sensor information screen displayed on the computer for operation. It is an illustration figure which shows an example of the operation screen displayed on the computer for operation. It is a flowchart which shows a part of example of operation | movement of the information display process in the computer for operation. FIG. 8 is a flowchart showing a continuation of FIG. 7. It is a flowchart which shows an example of the operation | movement process in an operation computer.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Robot control system 12 ... Communication robot 14 ... Operation computer 16 ... Network 18 ... Camera 20 ... Wireless tag reader 22 ... Distance sensor 24 ... Sound level meter

Claims (4)

A communication robot that exists in an environment provided with a plurality of types of environmental sensors, and an operation computer including an input device, the operation computer specifying an operation by an operator in response to an input from the input device A robot control system for transmitting operation information for the operation to the robot,
A plurality of environment display means for individually displaying environment information detected using the plurality of types of environment sensors, and a map display means for displaying a map showing a part or the whole of the environment;
Said map display means displays the warning information at the current position the robot position display means to display on said map, said detected environment based on the detected information before Symbol environment sensor of the communication robot on the map A robot control system comprising: attention information display means ; and environmental sensor position display means for displaying the positions of the plurality of types of environmental sensors arranged in the environment . The robot control system according to claim 1 , wherein the environment display unit displays environment information detected by an environment sensor selected by the operation of the operator . It said map display means further comprises, according to claim 1 or 2 the robot control system according to environmental sensor display means for displaying the position of each of the plurality of types of environmental sensors disposed in the environment. A program executed by a computer of a robot control system for remotely operating a communication robot existing in an environment provided with a plurality of types of environmental sensors in accordance with an instruction from an operator, the program comprising:
A plurality of environment display means for individually displaying environment information detected using the plurality of types of environment sensors; and a map display means for displaying a map showing a part or the whole of the environment, and further the map display means the current position robot position display means to display on said map, note information display means a warning information in the environment detected based on the detection information before Symbol environment sensor to display on the map of the communication robot And a program for functioning as environmental sensor position display means for displaying the respective positions of the plurality of types of environmental sensors arranged in the environment .

Images