JP2017061026A - Robot control device, robot control method, robot system and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a robot control device and a robot control method capable of improving productivity more by suppression a decline of work efficiency of work by a robot while securing the safety of a person.SOLUTION: The robot control device for controlling the operation of a robot includes acquisition means for acquiring the position of the robot, first detection means for detecting the position of a person, notification means for giving notification information that can be perceived by the person, second detection means for detecting a reaction of the person who responds to the notification information outputted by the notification means, and operation determination means for determining the operation of the robot on the basis of the reaction of the person detected by the second detection means.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a robot control apparatus, a robot control method, a robot system, and a computer program for avoiding human danger in the field of industrial robots.

When using an industrial robot (hereinafter referred to as a “robot”), it is necessary to control for avoiding the danger of humans (operators using the robot, maintenance personnel, etc.) located around the robot. Become. As this risk avoidance control, there is known control for stopping or decelerating a robot when a human enters or approaches an area set in advance by a safety manager or an operator.
Patent Document 1 discloses a robot control device that stops a robot or shuts off a power source of a robot when an operator has entered a prohibited area where entry is prohibited for safety. Patent Document 2 discloses a robot control device that decelerates a robot to a predetermined speed or less when the position of the robot is within the operator's work area. Patent Document 3 predicts the future positions of the workers and robots from the current positions and moving speeds, determines the possibility of contact between the workers and the robot, and stops or slows down according to the determination results. A safety control device that performs processing is disclosed.

JP 2009-050958 A JP 2007-283448 A JP 2010-120139 A

However, the conventional control has the following problems. In Patent Document 1, each time an operator enters the operation area of the robot, the robot stops and the work efficiency of the robot decreases. In Patent Document 2, the robot is decelerated when the robot is in the work area. For this reason, even if the robot and the worker just approach each other at a certain point and the possibility of future contact is low, the robot is decelerated each time, and the work efficiency is lowered.
Further, in Patent Document 3, whether or not danger avoidance is necessary is determined only by contact determination based on the presence / absence of human intrusion into the work area of the robot, the position and movement speed of the human and the robot. For this reason, since humans recognize the robot, it is possible to perform danger avoidance operations from the human side, and even when there is no need to stop or decelerate the robot, stop and decelerate the robot each time. End up. Therefore, the robot is frequently stopped and decelerated, the work of the robot is delayed, and the work efficiency and productivity of the robot are lowered.

In particular, in robot systems that tune the motion path on the robot side or pick parts randomly, it is difficult to predict the trajectory of a robot that has been changed (updated) by a human because the robot motion path can be changed from the previous time. It is. In this case, if the robot is working on the assumption that it always performs the same operation, it becomes difficult for a human to work in accordance with the operation of the robot when the robot performs an unpredictable operation. For this reason, the danger that the robot will come into contact with a human or the like increases, and the necessity of risk avoidance control increases.
The present invention has been made in view of the above-described problems, and the object thereof is to improve productivity by suppressing a decrease in work efficiency of a robot operation while ensuring human safety. A robot control apparatus and a robot control method are provided.

In order to solve the above-described problem, a robot control device according to an aspect of the present invention is a robot control device for controlling the operation of a robot, and includes an acquisition unit that acquires the position of the robot, and a human position. First detecting means for detecting, notifying means for notifying the notification information perceivable by the human, and second detecting means for detecting the human reaction according to the notification information output by the notifying means; And an action determining means for determining an action of the robot based on the reaction of the human detected by the second detecting means.
A robot control method according to another aspect of the present invention is a robot control method for controlling the operation of a robot, the step of obtaining the position of the robot to be controlled, and the detection of a human position. Based on the detected reaction of the human, the step of detecting the notification information perceivable by the human, the step of detecting the human reaction according to the notified notification information, Determining the operation of the robot.

  ADVANTAGE OF THE INVENTION According to this invention, the fall of the working efficiency of a robot can be suppressed and productivity can be improved more, ensuring a human safety.

It is a figure which shows an example of a structure of the robot system of Embodiment 1 of this invention. 4 is a flowchart illustrating a flow of processing of the robot control apparatus according to the first embodiment. It is a figure which shows an example of a structure of the robot system of Embodiment 2. FIG. It is a flowchart which shows the flow of a process of the robot control apparatus of Embodiment 2. It is a figure which shows an example of a structure of the robot system of Embodiment 3. 10 is a flowchart illustrating a processing flow of the robot control apparatus according to the third embodiment. It is a figure which shows an example of the hardware constitutions of the robot control apparatus of each embodiment.

DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the accompanying drawings.
The embodiment described below is an example as means for realizing the present invention, and should be appropriately modified or changed depending on the configuration and various conditions of the apparatus to which the present invention is applied. It is not limited to the embodiment.

Embodiment 1
The robot control apparatus according to the first embodiment notifies predetermined notification information to a person who should avoid danger, detects a human reaction to the notification information, and whether the person is aware of the notification information visually (perception) Estimated). Here, the notification information is information that is notified as a stimulus that can be perceived by a human to obtain a detected human reaction. A risk avoidance operation to be executed by the robot is determined according to risk prediction information such as contact and an estimation result of whether or not the notification information is conscious (perceived). In determining the risk avoidance operation, for example, when a person is conscious of the notified information, the robot is operated as it is. On the other hand, when the person is not conscious of the notified information, the robot is decelerated. To stop or stop.

When a human works in close proximity to the robot, if the robot's motion changes from the previous time by changing the motion already generated by the robot, the human may not be able to predict the changed motion of the robot.
In such a case, according to the first embodiment, when a danger is predicted, a future risk avoidance operation is determined depending on whether a person is conscious (perceived) of the notified information (notification information). . Here, in the first embodiment, whether or not a human being is conscious (perceived) by viewing the notification information by notifying a human being of a situation including a change in the operation of the robot and observing the human reaction. Is estimated.
For this reason, the robot can be continuously operated even in the situation where the robot is stopped conventionally, and the productivity is improved. In the first embodiment, “being conscious” means a state in which a human perceives notification information. Further, “perceive” means that a human senses an external event through a sensory organ, and can be estimated, for example, when notification information enters the human visual field as described later.

  FIG. 1 shows a configuration of a robot system 8 according to the first embodiment. The robot system 8 includes a robot control device 10, a robot 11, and a sensor 12. The robot control apparatus 10 includes a control unit 101, a robot position acquisition unit 102 (acquisition unit), an image information acquisition unit 103, a human position detection unit 104 (first detection unit), and a risk prediction unit 105 (prediction unit). ). In the first embodiment, the robot control apparatus 10 further includes an information notification unit 106 (notification unit), a reaction information detection unit 107 (second detection unit), and a perception (consciousness) state estimation unit 108 (consciousness state estimation unit). ) And a danger avoidance handling operation determination unit 109 (operation determination means).

The robot 11 includes a robot arm (not shown) and a robot hand provided at the tip of the robot arm. The robot arm includes a plurality of links and one or a plurality of joints (joints) that connect the links.
The robot 11 is connected to the control unit 101 of the robot control apparatus 10, operates in accordance with an operation instruction from the control unit 101, and outputs the position of each part of the operating robot to the control unit 101. The robot 11 includes a base portion serving as an arm base, a servo motor that drives the arm, an encoder that detects the angular position of the servo motor, and the like as necessary.

In the first embodiment, the robot 11 is provided with one or more information notification units 106 that are controlled by the danger prediction unit of the robot control device 10 and output information for notification.
The control unit 101 of the robot control apparatus 10 controls the operation of the robot based on an instruction (teaching) of the operation of the robot 11 from the operator. Specifically, the control unit 101 moves the part from the position A to the position B while avoiding the obstacle based on the result of the operator instructing the part (work) to be moved from the position A to the position B, for example. To generate (plan) motion plan information that defines the motion of the robot. Here, position A is the robot operation start point, and position B is the robot operation end point. The control unit 101 holds, as operation plan information, an operation plan for moving a part from position A to position B by the robot 11 in storage means (such as the external memory 714 in FIG. 7).

In the first embodiment, the control unit 101 has a function of tuning the operation for the purpose of shortening the tact time, shortening the trajectory, smoothing the operation, etc., for the operation plan information generated at the beginning, and productivity is improved. Change the robot motion plan information to improve it. That is, the robot 11 does not always repeat the same operation every time. When the control unit 101 changes the operation plan information, the robot trajectory is changed from the previous time.
Other cases in which the operation plan information is changed include the following cases. First, the control unit 101 changes the operation plan information of the robot 11 in accordance with a risk avoidance response operation determined by a risk avoidance response operation determination unit 109 (operation determination unit) described later.
Further, there may be a case where the position A of the component is not at a position designated in advance. In this case, the position A where the part is placed is newly acquired by the sensor, and the operation plan information for moving the part to the position B where the part is moved based on the acquired new position A is obtained by planning. Become. Thus, also when the position A at which the robot 11 grips a part is different from the initial position, the operation plan information of the robot 11 is changed from the previous time. Further, even in the work of assembling several types of parts, the position A and the position B are different each time, and the operation plan information generated by the robot 11 is changed each time.

When the operation plan information of the robot 11 is changed, the control unit 101 retains and stores the change status as change information regarding the robot status.
The robot position acquisition unit 102 (acquisition means) acquires the robot arm (each link configuring the robot) and the trajectory of the robot hand from the operation plan information of the robot 11 held by the control unit 101. . Then, the robot position acquisition unit 102 calculates a spatial area that passes when moving the part from the current position of the robot 11 to the position B as information on the position of the robot. At this time, the robot position acquisition unit 102 acquires the current position of the robot 11 from the control unit 101 in addition to the trajectory of the robot 11.

In the first to third embodiments, for example, the sensor 12 can use an imaging camera that captures an image of a robot work area or a worker. Specifically, the sensor 12 includes a plurality of imaging cameras that capture images, and includes a circuit and an interface for outputting the captured images to the outside. The sensor 12 captures an image so that a human (such as an operator) is included in the image captured by the imaging camera.
As long as the sensor 12 is a sensor that detects a robot work area, a robot position, a worker position, and the like, other means may be used. For example, a three-dimensional laser measuring device other than the imaging camera can be used.
The image information acquisition unit 103 includes a capture board, a memory, and the like, and acquires a two-dimensional image and a distance image from the captured image input from the sensor 12.

In order to acquire a distance image, the sensor 12 includes a plurality of imaging cameras having different viewpoints, and the image information acquisition unit 103 may acquire image information captured by the plurality of imaging cameras. Here, the distance image refers to distance data in the depth direction in two-dimensional image data, that is, image data having a distance value from the viewpoint in the image to the object. Data in any format may be used as long as data regarding distance is provided. Further, for example, image data having a distance value may be used in addition to or in place of the color and shading of normal two-dimensional image data.
The acquired image information may be a three-dimensional image instead of a two-dimensional image, and the apparatus configuration of the image information acquisition unit 103, the sensor 12, and the like is not limited to the above configuration as long as these can be acquired. For example, the sensor 12 may generate a distance image and supply it to the image information acquisition unit 103.

In addition, a distance image generation unit (not shown) may generate a distance image of the photographed object by a stereo method from images captured by a plurality of imaging cameras.
The acquisition of the distance image is not limited to the configuration described so far, and other configurations may be used. For example, an active three-dimensional measurement method such as a pattern projection method of projecting a light pattern onto a photographing object from a position different from that of the imaging camera may be used. Alternatively, the distance image of the photographed object may be calculated using a three-dimensional measurement method other than the pattern projection method. In addition, the distance image may be acquired using various methods such as a TOF (Time Of Flight) method. The image information acquisition unit 103 supplies the acquired two-dimensional image and distance image to the human position detection unit 104 (first detection unit) and the reaction information detection unit 107 (second detection unit).

The human position detection unit 104 divides the distance image acquired by the image information acquisition unit 103 into regions (segmentation). The human position detection unit 104 also detects the human head from the two-dimensional image acquired by the image information acquisition unit 103. The human position detection unit 104 sets a region including a portion corresponding to the head of the two-dimensional image among a plurality of regions of the divided distance image as a human region, and a point cloud of the distance image corresponding to the human region Is the current human position.
Here, the point cloud means point cloud data (Point Cloud Data), and the point cloud is an image of the surface shape of an object, environment, or person existing in 3D space using, for example, a 3D sensor. It means data of “3D point coordinates”. As a result, a 3D point cloud is obtained by perspectively projecting the captured distance image onto a point in the 3D space.

  The risk prediction unit 105 (prediction means) performs contact prediction between the robot and the human as an aspect of human risk prediction. Specifically, the danger prediction unit 105 can contact the human and the robot at a position where the robot position acquired by the robot position acquisition unit 102 and the human position detected by the human position detection unit 104 overlap. Predict that the position has sex. Information indicating a position where the human and the robot may come into contact is referred to as “contact information”. If it is predicted that there is a possibility of contact, that is, if contact information exists, if the human does not move at the current position and the robot's movement remains in the current trajectory, the time elapses Then, humans and robots may come into contact. In this case, the danger prediction unit 105 predicts that there is a possibility of danger to humans. That is, the danger prediction unit 105 (prediction means) predicts a contact between the robot and a human as a danger.

The information notification unit 106 is disposed on a movable part (movable part) of the robot 11, for example, each link of the robot arm or the robot hand, and outputs the notification information so that it can be perceived by humans. The information notification unit 106 may be configured by a device such as a liquid crystal display device or an LED that can change color and brightness.
In the first embodiment, the liquid layer display device or LED that can optically notify is used as the notification means. However, a speaker or a buzzer that performs voice notification can also be used.
Note that the information notification unit 106 may be disposed on a surface or a side surface that is outside each link of the robot 11. When the links constituting the robot arm and the robot hand constituting the robot 11 are bent at the joints (joints) connecting these links, the information notification unit 106 is located on the inner surface, and thus from the human field of view. This is because it is difficult to see. Further, the number of information notification units 106 is not limited to one for each link, and a plurality of information notification units 106 may be provided on the outer side and the side surface of each link.

  The danger prediction unit 105 obtains the contact information when predicting that there is a possibility of danger to humans. Based on the contact information, the danger prediction unit 105 turns on the information notification unit 106 other than the contact information in light blue. For example, the danger prediction unit 105 lights the information notification unit 106 arranged on the link or the robot hand that is determined to have the same trajectory of the robot 11 as the previous one and that there is no possibility of contact with the light blue ( Luminescence). By turning on the light blue display color, the link (or robot hand) on which the information notification unit 106 is disposed is notified to the human that the trajectory is not changed and there is no possibility of contact with the human. .

On the other hand, the risk predicting unit 105 changes only the information notification unit 106 disposed on the link or the robot hand in which the trajectory of the robot 11 is changed and it is determined that there is a possibility of contact based on the contact information. Lights up (emits light) in pink. By turning on the pink display color, the link (or robot hand) on which the information notification unit 106 is arranged is notified to the human that the trajectory is changed and there is a risk of contact with the human.
Regardless of whether or not the trajectory of the robot 11 has changed, the information notification unit 106 that has continued to display the pink light emission from the previous operation of the robot 11 can make contact with the corresponding link of the robot 11 or the robot hand. The pink light emission display is continued while there is sex. This ensures safety to humans when the robot 11 is stopped due to an abnormality, emergency stop, or other cause, and then the robot 11 is restarted from the stop position or restarted from the position A. It is to do.

In addition, when there is a time until the time when the human and the robot 11 are predicted to come into contact, the display color of the information notification unit 106 changed from light blue to pink is maintained pink, and the human reaction is continuously detected. May be.
In the first embodiment, examples of light blue and pink have been described as display colors. However, any color may be used as long as information notification of the status of the robot 11 can be perceived by humans.
Further, as another method of presenting the notification information, when the robot 11 has the same trajectory instead of changing the display color, the display is turned on. On the other hand, when the robot trajectory changes and there is a possibility of contact, Alternatively, it may be notified to a human by performing blinking display with a long cycle. Or you may alert | report to a person using light extinction and blinking.

Furthermore, the notification content may be changed not only for the information notification unit 106 that should output the notification information but also for a plurality of information notification units 106 that are located in the vicinity of the information notification unit 106 that should output the notification information. Further, the notification contents of all the information notification units 106 arranged in the robot 11 may be changed. This makes it easier for humans to perceive the notification information and react, thereby reducing the chance of the robot 11 to perform the danger avoidance reaction operation.
In the first embodiment, a light emitting unit such as an LED is used as the information notification unit 106, but another light emitting unit may be used. Further, the information notification unit 106 may be configured by an LED matrix or a liquid crystal display device, and information notification may be performed by character display. Further, a speaker or the like may be used as the information notification unit 106 to perform information notification by voice.

In the first embodiment, the information notification unit 106 has been described as being provided at each link of the robot 11. However, the installation location is not particularly limited and may be installed anywhere. For example, it may be arranged (concentrated) at any part of the robot. In addition, for example, a liquid crystal display that is the information notification unit 106 can be arranged in the vicinity of the robot 11 to notify information to a human. Further, for example, the information notification unit 106 may be provided on a ceiling or a wall to perform information notification.
Furthermore, the information notification unit 106 may be configured to be worn on a human body (for example, an operator). For example, the information notification unit 106 may be configured by providing an LED on a hat, a clock, or the like. In addition, an information notification unit 106 using a vibrator can be carried by a person and information can be notified to the person by vibration. In addition, when the information notification unit 106 uses a speaker to perform information notification by voice, the information notification unit 106 may be installed anywhere as long as a human can hear the voice. For example, headphones or earphones worn by humans may be used as the information notification unit 106, or a speaker attached to a wall may be used as the information notification unit 106.

The reaction information detection unit 107 detects a human head from the two-dimensional image acquired by the image information acquisition unit 103, and further extracts a face direction or a line-of-sight direction as a human reaction. The reaction information refers to information related to a human reaction to the notified information, and includes, for example, a face direction, a line-of-sight direction, and the like.
The perception (consciousness) state estimation unit 108 includes contact information (contact position between the robot and human) predicted by the risk prediction unit 205 and a face direction or line-of-sight direction detected by the reaction information detection unit 107 (hereinafter referred to as “reaction”). "Information")). Based on this contact information and reaction information, the perception (consciousness) state estimation unit 108 determines whether or not the position of the information notification unit 106 that has changed the display color from light blue to pink has entered the human visual field. .

  In the first embodiment, the perception (consciousness) state estimation unit 108 determines whether the display color has been changed by the human depending on whether the information notification unit 106 that has changed the display color from light blue to pink has entered the human visual field. It is estimated whether or not the information notification unit 106 is seen. Here, the “viewed state” means that the information notification unit 106 whose display color has been changed enters the human visual field, so that the information output from the information notification unit 106 by the human (the display color is changed). It can be assumed that it has been perceived or conscious. That is, the perception (consciousness) state estimation unit 108 (estimating means) estimates whether or not a person is aware of the notification information.

When the perception (consciousness) state estimation unit 108 determines that the person has seen the information notification unit 106 whose display color has been changed, the human responds to the notification information output from the information notification unit 106, It is estimated that the movement of the robot 11 is conscious (perceived). On the other hand, when it is determined that the information notification unit 106 whose display color has been changed is not in a state where a human has seen, the human does not react to the notification information output from the information notification unit 106 and the 11 operations of the robot have changed. This is presumed to be unconscious (perceived).
The danger avoidance corresponding motion determination unit 109 (motion determination means) continues the current state of the robot 11 without changing the motion of the robot 11 according to the estimation result estimated by the perception (consciousness) state estimation unit 108, or avoids danger Decide whether to take action. Here, the danger avoidance response operation includes deceleration of the operation of the robot, stop of the operation, and change of the operation trajectory (change of the operation path).

(Robot controller processing flow)
Next, the processing flow of the robot control apparatus 10 will be described based on the flowchart of FIG. Each processing operation in FIG. 2 may be achieved by the hardware of the robot controller 10 shown in FIG. 1 executing a program.
First, in S <b> 200, the robot position acquisition unit 102 calculates an operation range from the trajectory data of the robot 11 held by the control unit 101 and the current position of the robot 11 to the completion of the operation of the robot 11. The trajectory data of the robot 11 includes the time-series positions and joint angles of the members of the robot arm and the robot hand constituting the robot 11 and the members when the robot 11 moves the part from the current position to the position B. Including the dimensions of From this trajectory data, the robot position acquisition unit 102 calculates a spatial region through which the robot 11 passes when moving from the current position to the position B, and uses this as the position of the robot 11.

In S201, the image information acquisition unit 103 acquires a two-dimensional image and a distance image.
In S202, the human position detection unit 104 detects the current human position. Specifically, the human position detection unit 104 divides the point group of the distance image acquired in S201 into regions, and detects the human head from the two-dimensional image. Then, the human position detection unit 104 sets a divided area including a point group corresponding to the head of the two-dimensional image among a plurality of areas of the distance image obtained by dividing the area as a human area. A point group included in the human region is detected as the current human position.
In S <b> 203, the risk prediction unit 105 performs human risk prediction by predicting the possibility of contact between the human and the robot 11. Specifically, the risk prediction unit 105 determines the position of the point group existing in the spatial region that is the position of the robot obtained in S200 among the point groups detected as the human position obtained in S202. It is obtained as a position (contact information) where a human and a robot may contact each other.

Here, the position of the robot 11 is obtained in S200 as a spatial region through which the robot 11 passes when moving from the current position to the position B. In this way, by determining the position of the robot 11 as a spatial region, the danger prediction unit 105 can predict whether there is a possibility of contact with the robot 11 in the future from the current human position. .
In S203, the danger prediction unit 105 obtains a position (contact information) where the human and the robot 11 may come into contact. The position (contact information) is determined by the robot 11 when the human and the robot 11 come into contact with each other. This is the position of the contact part. Note that the position (contact information) may be estimated by predicting the future positions of the robot 11 and the human from the current position and moving speed of the robot 11 and the current position and moving speed of the human.

In S <b> 204, the danger prediction unit 105 determines whether or not there is a possibility of contact between the human and the robot 11. Specifically, in S203, if there is no position (contact information) where the human and the robot 11 contact each other, the risk prediction unit 105 determines that there is no possibility of contact, and the process proceeds to S212.
In S <b> 212, the danger prediction unit 105 lights all the information notification units 106 of the robot 11 in light blue.
In S213, the control unit 101 returns to S200 again without changing the operation of the robot 11, and continues the process.

On the other hand, if the risk prediction unit 105 determines in S203 that there is a position (contact information) where the human and the robot 11 contact each other, the risk prediction unit 105 may contact the human and the robot 11 in S204. And the process proceeds to S205.
In S <b> 205, the danger prediction unit 105 changes the display color of the information notification unit 106 arranged on the link or the robot hand in which the robot trajectory is changed and which may be contacted from light blue to pink. As a result, the information detection unit 106 notifies the human being of the change in the status of the robot due to the change in the operation of the robot 11.
By changing the display color of some of the information notification units 106 of the plurality of information notification units 106 arranged in the robot 11, it is possible to know which link is likely to be contacted and predicted to be dangerous. There is an advantage of immediately knowing which link (or hand) a human may contact. Here, “immediately grasp” means notifying a position where there is a possibility of contact so that a human can immediately perceive (in a short time). For example, an example is shown in which notification is performed by changing the color instead of a method of displaying characters and allowing a human to read (it takes some time).

In S206, the reaction information detection unit 107 detects the human head from the two-dimensional image acquired in S200, and detects the face direction and the line-of-sight direction as human reaction information. The head may be detected by detecting the Ω (omega) shape that is the shape of the head from the human shoulder. Moreover, the orientation of the face can be detected and the eye region can be extracted by the Haar-like feature using local contrast of the eyes, nose, and mouth which are facial organs.
Furthermore, the direction of the line of sight can be estimated from the pupil position and the face direction in the extracted eye region.
In S207, the perception (consciousness) state estimation unit 108 calculates a human visual field range. Specifically, the direction of the line of sight estimated in S206 is the center of the human visual field, and the human visual angle is, for example, 90 degrees in the horizontal direction with respect to the center of the visual field, and 60 in the vertical direction is up. The human visual field range can be calculated with 70 degrees and 70 degrees below.

In S208, the perception (consciousness) state estimation unit 108 calculates whether or not the position of the information notification unit 106 whose display color has been changed from light blue to pink in S205 is within the viewing angle that is the viewing range calculated in S207. To do.
In S209, the perception (consciousness) state estimation unit 108 is a state in which the person has seen the position of the information notification unit 106 that has changed the display color from light blue to pink, which may contact the robot with time. Whether or not. That is, the perception (consciousness) state estimation unit 108 (estimating means) estimates whether or not a human is aware of the notification information.
Specifically, in S208, when the information notification unit 106 whose display color is changed to pink is in the human visual field, the state in which the human has viewed the information notification unit 106 (perception or conscious state) Judge (estimate), and proceed to S210. On the other hand, if the information notification unit 106 is not within the human visual field, it is determined (estimated) that the human is not looking at the information notification unit 106, and the process proceeds to S214.
As described above, in the first embodiment, in S207 to S209, the perception (consciousness) state estimation unit 108 estimates the human consciousness state about the danger posed from the robot 11.

In other words, in the first embodiment, the display color of the information notification unit 106 arranged in the link or the robot hand in which the trajectory of the robot 11 is changed and which may be in contact with humans is changed to other information notification units. 106 so as to be identifiable. Thereby, the state of the robot such as a change in the operation state of the robot 11 is notified to the human. If it is determined that the notification information notified by the information notification unit 106 whose display color has been changed in this manner is viewed by a person, it is estimated that the person is aware of the change in the situation of the robot 11.
Alternatively, the notification information may be notified via the information notification unit 106 either when the danger prediction unit 105 predicts a danger or when the operation is changed.
In S210, the danger avoidance corresponding operation determination unit 109 determines not to change the operation of the robot 11 as the danger avoidance corresponding operation. In this case, the control unit 101 proceeds to S211 without changing the control of the robot 11. In step S <b> 211, the control unit 101 determines whether the operation of the robot 11 (for example, movement of a part (work)) is continued or completed. If the movement of the part by the robot 11 has not been completed, the process returns to S200 to continue the process, and if the movement of the part has been completed, the process ends.

On the other hand, in S214, the danger avoidance corresponding action determining unit 109 determines the danger avoidance corresponding action to be executed by the robot 11, and instructs the control part 101 of the determined danger avoidance corresponding action. The control unit 101 causes the robot 11 to execute the risk avoidance response operation determined by the risk avoidance response operation determination unit 109. This danger avoidance response operation may be to reduce the operation speed of the robot 11 in order to avoid contact between the human and the robot. Further, the robot 11 may be stopped when the current distance between the human position and the position of the robot 11 is equal to or less than a predetermined distance. Alternatively, the trajectory of the robot 11 may be changed so that the robot 11 does not pass through the position of the contact portion of the robot when the human and the robot 11 are in contact with each other in order to avoid contact with the human rather than deceleration or stop. .
In addition, when there is a time until the time when the human and the robot 11 may come into contact and the preset time until contact is not reached, the following processing may be performed. That is, the display color of the information notification unit 106 that has been changed from light blue to pink may be kept pink while the detection of human reaction information may be continued.

In this case, in S209, it may be determined whether or not the current time has reached a preset time until a time when the human and the robot 11 may come into contact with each other. If not reached, the process proceeds to S210 and the robot does not have to perform the normal operation and perform the danger avoidance response operation. On the other hand, when the preset time is reached, the process proceeds to S214, and the robot 11 may perform the danger avoidance response operation.
Here, an example has been described in which the time until the time at which the human and the robot 11 may come into contact has not reached the predetermined grace time until the contact is set in advance. In addition, distance requirements may be employed.
That is,
・ The time until the time when the human and the robot 11 may come into contact has not reached the predetermined grace period. ・ When the distance between the human and the robot 11 satisfies the two requirements of less than the predetermined distance, avoiding danger. The corresponding action determination unit 109 can also perform a process of maintaining the display color of the information notification unit 106 in pink without changing the operation of the robot 11.

Also in this case, for example, in the above-described S209, it is preferable to check whether or not both of the two types of requirements are satisfied. Specifically, it is determined whether or not the time until the time when the human and the robot 11 may come into contact has reached a preset time (grace time), and the distance between the human and the robot 11 Is determined to be less than a predetermined distance.
Such processing takes into account, for example, the case where the worker is an expert. When the worker is a skilled worker, the worker grasps the movement pattern of the robot, so it is bothersome to see the display on the information notification unit 106 until a certain timing, and the work efficiency is also lowered. Therefore, in order to allow the work to be continued without looking at the information notification unit 106, when there is a certain amount of time, the risk avoidance response operation is not performed until the preset time is reached. .

In this way, for example, when there is a time longer than a preset time until the human and the robot 11 contact each other, the human (for example, an operator) changes the display color of the information notification unit 106 each time. The work can be continued without confirmation. Accordingly, it is not necessary for the human to work while confirming the notification information output from the information notification unit 106 each time until a predetermined time until the time when the human touches the robot 11, and the work efficiency is improved. Furthermore, it is possible to further reduce the frequency at which the danger avoidance response operation is performed.
If a human notices and confirms notification information (for example, a change in display color) output from the information notification unit 106 before reaching the set time, human safety is ensured. Even if the notification information is not noticed before the set time is reached, or even if it is not noticed, even if human beings have not been able to avoid danger, the time after reaching the set time is reached In addition, the control unit 101 may stop the robot 11. Since the robot 11 finally stops before contact, human safety is ensured.

(Modification of Embodiment 1)
In the first embodiment, it is estimated from the orientation of the face and the direction of the line of sight whether or not the human has seen the robot 11 with the state where the information notification unit 106 is in the human visual field. Instead, it is estimated whether the human is looking at the information notification unit 106 based on whether the human face is facing the information notification unit 106 or the back from the direction of the face. May be.
In any case, in the first embodiment, the human consciousness state (whether or not it is perceived) is estimated from the image including the person captured in the image acquired by the image information acquisition unit 103. Therefore, not only the worker who performs work in cooperation with the robot 11 but also the safety of humans other than the worker, such as an intruder into the work area, a parts replenisher, or a maintenance person, can be similarly secured.

In the first embodiment, the situation in which the motion trajectory of the robot 11 is changed has been described as the motion situation of the robot 11. However, the operation state (robot state) of the robot 11 serving as a trigger for notifying the notification information is internal information of the operation of the robot 11 and is related to the temperature, current value, voltage value, Information such as pressure, gripping force, operation speed, and external force may be included.
The temperature may include electronic component soldering or reflow soldering temperature, and the internal information related to temperature may include a current value for controlling the temperature of soldering or the like. The danger prediction regarding the temperature is based on a preset distance indicating how dangerous the danger prediction unit 105 should be from the human position and the position of the robot 11 if the distance between the human being and the human being is not separated from the temperature. Just do it. When a human danger due to temperature is predicted, the danger prediction unit 105 may change the display color of the information notification unit 106 to notify the person of the danger. Further, when the current value exceeds a preset threshold value, the display color of the information notification unit 106 may be changed to notify the danger to the human.

  In addition, the situation around the robot 11 serving as a trigger for notifying the notification information includes the temperature, shape, size, mass, reaction of the chemical substance, and the radioactive substance (radioactivity of the substance (part) held by the robot 11. It may include situations such as the impact on humans). Furthermore, the surrounding situation is the situation such as dedicated jigs, transfer devices, other robots, objects to be assembled, placement of assembled parts and boxes containing them, which exist in the work environment around humans and robots Can also be included. In addition, the surrounding conditions may include surrounding conditions such as brightness, temperature, humidity, pressure, vibration, sound, dust, wind, and light.

  Moreover, in the said Embodiment 1, the danger prediction part 105 alert | reports by the period of a change of the display color of the information alerting | reporting part 106, lighting, or blinking as a method of alerting | reporting information to a person via the information alerting | reporting part 106. Explained the method. However, the information may be notified by gradually changing the intensity of light emitted by the information notification unit 106 in accordance with the various parameters and various numerical values of the operation state of the robot 11 described above. The change in light intensity may be changed linearly according to a numerical value or the like, or may be changed stepwise such as small, medium, and large.

  In addition to light display color and light emission intensity, notification information may be notified to humans using sound, pressure, wind pressure, temperature, and vibration. In this case, the information notification unit 106 may include a speaker that outputs sound, an actuator that generates vibration, and the like. Further, instead of notifying the notification information by the information notification unit 106, the notification information may be notified using the movement of the robot 11. In this case, instead of controlling the display of the information notification unit 106, the danger prediction unit 105 may send a motion of the robot 11 for notifying the control unit 101 of a person. The control unit 101 may cause the robot 11 to perform a motion for notification of the robot 11 that is sent out.

Furthermore, the risk prediction unit 105 displays a parameter that causes danger to humans or a risk level based on the value, on the information notification unit 106 as a numerical value or a variable-length bar corresponding to the level of the numerical value, etc. Then, the notification information may be notified to a human. Or the information alerting | reporting part 106 may change the brightness | luminance of a display according to the value of a danger level. When performing notification based on the degree of risk, the information notification unit 106 may include a device that can display numerical values, a device that can perform bar display, and the like in order to perform such display. Also, numerical information such as the degree of danger may be notified by analog display such as an indicator instead of digital display.
Further, the notification information to the human may be continuously notified regardless of whether the operation state of the robot 11 is changed or when the operation state of the robot 11 is changed. May be notified.

As described above, in the first embodiment, in addition to the risk prediction, the operation status of the robot 11 is notified to the human as notification information, and a human reaction to the notification of the notification information is detected. Then, by estimating whether or not a person has seen the notification position (for example, the position of the information notification unit 106 that outputs the notification information), it is estimated whether or not the change in the situation of the robot 11 is conscious (perceived). To do. Based on this estimation result, the danger avoidance action is determined.
Therefore, when a human is conscious (perceived) of a change in the situation of the robot 11, it is not necessary to avoid danger, so that a decrease in the work efficiency of the robot 11 due to the stop or deceleration of the robot 11 is suppressed. The It is possible to execute a danger avoidance operation corresponding to a case where a human motion cannot be predicted. Thereby, the working efficiency and productivity of the robot 11 are further improved.

Embodiment 2
Hereinafter, the second embodiment of the present invention will be described with respect to differences from the first embodiment. Similarly to the first embodiment, the second embodiment notifies the human being of the change state of the operation of the robot as notification information, and observes the human reaction. As the human reaction, the second embodiment detects whether the central visual field of the human has moved in accordance with the notification of the notification information to the human, so that the human has recognized the change in the status of the robot based on the notification information. Estimate whether or not. Here, “recognition” is a kind of “awareness”.
FIG. 3 shows a configuration of the robot system 38 according to the second embodiment. 3, components having the same functions as those in the robot control apparatus 10 according to the first embodiment shown in FIG. 1 are denoted by the same reference numerals as those in the first embodiment, and description thereof is omitted. The robot system 38 according to the second embodiment includes the sensor 12, the robot control device 30, and the robot 11. The robot control apparatus 30 according to the second embodiment includes a recognition state estimation unit 308 (recognition state estimation unit) instead of the perception (consciousness) state estimation unit 108 in the robot control apparatus 10 according to the first embodiment.

When the notification information to be notified via the information notification unit 106 that notifies the human of the change state of the operation of the robot 11 changes, the recognition state estimation unit 308 causes the human to gaze at the information notification unit 106 in which the notification information has changed. Is detected. Then, the recognition state estimation unit 308 estimates whether or not the human has recognized the change in the situation of the robot 11 depending on whether or not the human has moved his / her line of sight to the information notification unit 106 whose notification information has changed.
Even if the position of the information notification unit 106 that has changed the notification information is within the human field of view, the consciousness is concentrated on places other than the position and other matters. In some cases, attention is not directed to the position.

  On the other hand, the recognition information estimation unit 308 according to the second exemplary embodiment is configured such that the human being changes the notification information output from the information notification unit 106 after the danger prediction unit 105 changes the notification information. It is detected whether the central visual field has been moved to the position. After the notification information is changed, if the central visual field of the person has moved toward the information notification unit 106, it is estimated that the notification of the notification information has been recognized. On the other hand, if the human has not moved the central visual field, it is estimated that even if the notification of the notification information is conscious (perceived), it is not recognized. Here, “recognize” means that an external event has been identified and determined, and can be estimated, for example, by moving a central visual field to notification information as described later.

FIG. 4 is a flowchart illustrating a process flow of the robot control device 30 according to the second embodiment.
Each processing operation in FIG. 4 may be achieved by the hardware of the robot controller 30 shown in FIG. 3 executing a program. The flowchart of FIG. 4 according to the second embodiment is different from the flowchart (FIG. 2) showing the processing flow of the robot control apparatus 10 described in the first embodiment in the processing operations of S407, S407b, S408, and S409. That is, the process for estimating the human recognition state for the robot is different. The other processes are the same as those in the flowchart shown in FIG. 2 and are denoted by the same reference numerals as those in FIG.

First, in S407, the recognition state estimation unit 308 calculates a human visual field range. Specifically, the direction of the line of sight detected in S206 is the center of the human visual field, and the human visual angle is, for example, 90 degrees in the horizontal direction and 60 in the vertical direction with respect to the center of the visual field. The visual field range of human beings is calculated with 70 degrees and below.
In S407b, the recognition state estimation unit 308 acquires the movement of the human eye line before and after the risk prediction unit 105 changes the display color of the notification information of the information notification unit 106.
In the first embodiment, using the current line-of-sight direction extracted from the current image acquired by the image information acquisition unit 103, a human being is aware of the notification information output by the information notification unit 106 (whether it has been perceived. The state of whether or not) was estimated. On the other hand, in the second embodiment, not only the present time but also the previous gaze direction detected in the past is used. From the past line-of-sight direction, it is possible to obtain the line-of-sight direction before the display color of the information notification unit 106 is changed. For example, if the image information acquisition unit 103 acquires image information every fixed time Δt, the movement of the line of sight on the time series at a constant interval Δt can be acquired as time series data. In S408, the recognition state estimation unit 308 calculates the movement of the position of the information notification unit 106 that has changed the display color from light blue to pink in S205 within the viewing angle that is the viewing range calculated in S207.

Next, in S409, the recognition state estimation unit 308 determines whether or not the position of the information notification unit 106 whose notification information has been changed has moved within the range of the central visual field after the display color of the information notification unit 106 has been changed. To detect. As the viewing angle of the central visual field, a range of 2 degrees from the center of the visual field, for example, up, down, left, and right may be used. The viewing angle of the central visual field is not limited to 2 degrees, and may be a numerical value that is different between the upper and lower sides and the left and right sides.
If the position of the information notification unit 106 in which the notification information has been changed has moved within the range of the central visual field, the recognition state estimation unit 308 causes the human to react to the change of the notification information and move the line of sight at least for a moment. Judge that In this case, the recognition state estimation unit 308 assumes that the human has recognized (consciously) that the robot status has changed, and proceeds to S210 without changing the operation of the robot 11, and the subsequent operation These are as described in FIG. 2 of the first embodiment. As described above, the recognition state estimation unit 308 (estimating means) estimates whether or not a person is aware of the notification information. In this embodiment, in order to facilitate understanding, the term “recognized” is used as a type of “awareness”.

Returning to S409, when the position of the information notification unit 106 whose notification information has been changed has not moved within the range of the central visual field, the recognition state estimation unit 308 responds to a change in the display color of the display unit 106 by a human. Judge that it was not. In this case, the recognition state estimation unit 308 estimates that the change in the state of the robot 11 has not been recognized by the human, and proceeds to S214, in which a risk avoidance response operation is performed. 2 as described above.
As described above, in the second embodiment, in addition to danger prediction, the state of operation of the robot 11 is notified to the human as notification information, and the human gazes at the position of the information notification unit 106 where the notification information is changed. It is detected whether or not it has been moved. Then, it is estimated whether or not the person recognizes the change in the situation of the robot 11 based on whether or not the person has moved his / her line of sight to the information notification unit 106, and based on the estimation result, determination of the risk avoidance response operation ( Judgment).

In the second embodiment, whether or not a person has moved his / her line of sight to the information notification unit 106 is determined based on whether or not the position of the information notification unit 106 whose notification information has been changed has moved within the range of the central visual field of the person. to decide. If the human is moving the central visual field toward the information notification unit 106 that notifies the notification information, the possibility that the human is recognizing the situation of the robot is considered higher.
As a result, when it can be determined that the human is recognizing a change in the situation of the robot 11, it is not necessary for the robot 11 to perform the danger avoidance response operation, the reduction in work efficiency is suppressed, and productivity is improved.
In particular, even if a human is conscious (perceived) to change the notification information, the state of the robot 11 is changed according to the second embodiment in a state where attention is not paid to the change of the notification information and the recognition is not performed. It can be suppressed that the change is recognized erroneously. In other words, unless it is estimated that a human can reliably perform danger avoidance, the safety of human beings can be more reliably ensured by causing the robot 11 side to perform the danger avoidance response operation.

Embodiment 3
Hereinafter, the third embodiment of the present invention will be described with respect to differences from the second embodiment. In the third embodiment, as in the second embodiment, information including a change in the state of operation of the robot is notified to a person as notification information, and a human reaction is observed. As this human reaction, the third embodiment detects whether or not a human has made a movement (motion) for avoiding danger, and based on the detection result, whether or not the human has recognized a change in the situation of the robot. presume.
FIG. 5 shows a configuration of a robot system 58 according to the third embodiment. In FIG. 5, the same reference numerals as those in the first embodiment are assigned to the components having the same functions as those in the robot control device 10 (the robot control device 30 in FIG. 3) according to the first embodiment shown in FIG. Description is omitted. The robot system 58 according to the third embodiment includes the sensor 12, the robot control device 50, and the robot 11. The robot control apparatus 50 according to the third embodiment includes a reaction information detection unit 507 in place of the reaction information detection unit 107 of the first and second embodiments. The robot control apparatus 50 according to the third embodiment includes a recognition state estimation unit 508 instead of the recognition state estimation unit 308 of the second embodiment.

The reaction information detection unit 507 detects the position of the human after the danger prediction unit 105 notifies the change of the situation of the robot 11 to the human through the display unit 106 and supplies the detected position to the recognition state estimation unit 508. The reaction information detection unit 507 can acquire the image information used in the detection of the human position by the image information acquisition unit 103 of the first embodiment, and the human position can be acquired by the human position detection unit 104 of the first embodiment. It can be detected.
The recognition state estimation unit 508 may contact the robot 11 with the person whose human position is detected by the risk prediction unit 105 after the display color of the notification information output from the information notification unit 106 is changed. It is determined whether or not it has moved from a certain position (contact information). The recognition state estimation unit 508 estimates whether or not the person has recognized the change in the situation of the robot 11 based on whether or not the person has moved from a position where contact is likely (contact information has been displaced). . In the third embodiment, it is estimated whether or not a human has performed a reaction operation such as a danger avoidance operation by determining whether or not the human has moved from a position where there is a possibility of contact according to the notification of the notification information. .

  FIG. 6 is a flowchart showing the process flow of the robot control apparatus 50 according to the third embodiment. Each processing operation of FIG. 6 may be achieved by the hardware of the robot control device 50 shown in FIG. 5 executing a program. The flowchart shown in the example of FIG. 6 is different from the flowchart (FIG. 4) showing the processing flow of the robot control apparatus 30 described in the second embodiment in the processing operations of S606 and S607. That is, the process for detecting human reaction information is different from the process for estimating the human recognition state of the robot 11. The other processes are the same as those in the flowchart shown in FIG. 4, and the same reference numerals as those in FIG.

In step S <b> 606, the reaction information detection unit 507 detects the position of the person after changing the notification information output from the information notification unit 106 and informing the person, and supplies the detected position to the recognition state estimation unit 508. The human position detection process can be executed by the same process as the process of S201 and S202.
In step S <b> 607, the recognition state estimation unit 508 (estimation unit) estimates whether a human has recognized (consciously) a change in the situation of the robot 11.
In the third embodiment, after the notification information of the information notification unit 106 is changed, if the position of the person has moved from a position in contact with the robot 11, the person will recognize the change in the situation of the robot 11 and avoid contact. (Dangerous avoidance) Presumed that the operation was performed. That is, the recognition state estimation unit 508 (estimating means) estimates whether or not a human is aware of information notification. In this embodiment, in order to facilitate understanding, the term “recognized” is used as a kind of “awareness”.

  Specifically, after the risk prediction unit 105 changes the notification information of the information notification unit 106 and notifies the human being, the point cloud that is the position of the human obtained in S606 and the robot 11 obtained in S203. A point cloud that becomes the position of the contact portion (contact information) is compared. If there is no point group area that is the position (contact information) of the contact portion of the robot 11 in the point cloud of the human position after the notification of the notification information, the human and the robot 11 may contact each other. From the position, the human has moved to avoid danger. In this case, in the third embodiment, it is estimated that a human has recognized a change in the situation of the robot 11 (and has taken a risk avoiding operation), and the process proceeds to S210. On the other hand, if there is a point cloud area that is the position (contact information) of the contact portion of the robot 11 in the point cloud of the human position after the notification of the notification information, the human and the robot 11 may contact each other. From a certain position, the human is not performing the danger avoiding operation. In the third embodiment, when the human is not performing the danger avoiding operation, it is assumed that the human is not aware of the change in the situation of the robot 11 (not conscious), and the process proceeds to S214, and the danger on the robot 11 side. Determine (determine) the avoidance action.

  As described above, in the third embodiment, whether or not a human sees the notification information output from the danger prediction unit 105 to the information notification unit 106 is determined based on whether the robot 11 performs a danger avoidance response operation. Not used for decision. Instead, in the third embodiment, the detection result of the human reaction movement (human movement) is acquired as the reaction information, and the subsequent change in the situation such as the robot movement based on the change of the reaction information is always information. What is necessary is just to continue alerting | reporting (display) as alerting | reporting information via the alerting | reporting part 106. FIG.

On the other hand, the human may notice the change in the situation of the robot 11 by looking at the notification information output by the information notification unit 106 and take action that the human avoids contact with the robot 11. Alternatively, if the person is skilled in the work, there is a case in which there are a plurality of patterns of future robot movements without switching to the notification information output by the information notification unit 106 and it is sufficiently understood that the switching is performed. is there. In this case, since a change in the situation such as the operation of the robot 11 can be predicted, it is possible for the human to take action to avoid contact with the robot 11.
In the third embodiment, in any case, the information notification unit 106 displays the display color of the information notification unit 106 when it is determined that the human is not in contact with the robot 11 and the human danger is avoided. May be changed from pink to light blue (S204Y and S212 in FIG. 6).

On the other hand, if the situation in which the robot 11 and the human are in contact is not avoided and the situation in which the human danger is not avoided, the robot may be decelerated or stopped as a risk avoidance action to avoid the danger (see FIG. 6 S214). The same applies when the risk value calculated by the risk prediction unit 105 is higher than a predetermined reference value.
Here, if the person is a skilled worker, the notification information output by the information notification unit 106 may be viewed only when necessary to view the information, and if the user is familiar with the movement of the robot 11, the information is not necessary. Also good. In the unlikely event that the risk of contact increases due to a lack of viewing, it may be determined that danger avoidance will be performed on the robot controller 50 side, and the operation of the robot 11 may be decelerated or stopped.

As described above, in the third embodiment, in addition to danger prediction, after notifying a person of the operation status of the robot 11 as notification information, it is detected whether or not the person has made a movement to avoid danger. Then, it is estimated whether or not the human has recognized a change in the situation of the robot 11 depending on whether or not the human has made a motion that can be estimated as risk avoidance, and based on the estimation result, a risk avoidance response operation is determined.
Accordingly, when it can be estimated that a human is aware of a change in the situation of the robot 11, it is not necessary for the robot 11 to perform a danger avoidance response operation, a decrease in the work efficiency of the robot is suppressed, and productivity is improved. .

(Other configuration examples)
The present invention supplies a program that realizes one or more functions of the configurations described in the first to third embodiments to a predetermined system or apparatus via a network or a storage medium, and the system or apparatus executes the program. This can also be realized. The program may be executed by a process in which one or more processors in the computer of the system or apparatus read and execute the program.
Further, it can be realized by a circuit that realizes one or more functions (for example, ASIC (Application Specific Integrated Circuit)).
FIG. 7 shows a hardware configuration of a computer 710 that can constitute all or part of each component of the robot control apparatus according to each of the above embodiments. The computer 710 includes a CPU 711, a ROM 712, a RAM 713, an external memory 714, an input unit 716, a display unit 717, a communication I / F 718, and a system bus 719. The computer 710 may further include an imaging unit 715.

The CPU 711 comprehensively controls the operation of each unit of the robot control device, and controls each component unit via the system bus 719.
The ROM 712 is a non-volatile memory that stores a control program and the like necessary for the CPU 711 to execute processing. These programs may be stored in the external memory 714 or a removable storage medium (not shown).
The RAM 713 functions as a main memory, work area, and the like for the CPU 711. That is, the CPU 711 implements various functional operations by loading a program or data required from the ROM 22 into the RAM 713 when executing the processing and executing the program.
The external memory 714 stores, for example, various data and various information necessary for the CPU 711 to execute processing for executing a program. The external memory 714 also stores various data and various information obtained by the CPU 711 executing a program.

The input unit 716 includes, for example, a keyboard and a mouse, and is configured so that the user can give instructions such as teaching to the robot control device via the input unit 716.
The display unit 717 may be configured with a monitor such as a liquid crystal display (LCD).
The communication I / F 718 is an interface for communicating with an external device. A system bus 719 connects the CPU 711, the ROM 712, the RAM 713, the external memory 714, the imaging unit 715, the input unit 716, the display unit 717, and the communication I / F unit 718 so that they can communicate with each other.

In the first to third embodiments, the case where the number of the robots 11 is one has been described. However, the present invention may be applied to the case where a plurality of robots 11 are controlled. Further, the robot control device 10 and the like and the robot 11 may be arranged apart from each other as well as being arranged close to each other. Further, the robot control device 10 or the like and the robot 11 may be configured to be connected via a predetermined network.
As mentioned above, although embodiment of this invention was described in detail, these embodiment only showed the specific example in implementing this invention, and the technical scope of this invention is limited to the said embodiment. It is not a thing. The present invention can be variously modified without departing from the gist thereof, and these are also included in the technical scope of the present invention.

  DESCRIPTION OF SYMBOLS 8 ... Robot control system, 10 ... Robot control apparatus, 11 ... Robot, 12 ... Sensor, 101 ... Control part, 102 ... Robot position acquisition part, 103 ... Image information acquisition part, 104 ... Human position detection part, 105 ... Risk prediction , 106 ... Information notification unit, 107 ... Reaction information detection unit, 108 ... Perception (consciousness) state estimation unit, 109 ... Danger avoidance corresponding action determination unit, 308 ... Recognition state estimation unit

Claims (17)

A robot control device for controlling the operation of a robot,
Obtaining means for obtaining the position of the robot;
First detecting means for detecting a human position;
Informing means for informing the human perceptible notification information;
Second detection means for detecting the human reaction according to the notification information output by the notification means;
An action determining means for determining an action of the robot based on the reaction of the human detected by the second detecting means;
A robot control device comprising: Predicting means for predicting the danger of the robot to the human based on the position of the robot acquired by the acquiring means and the position of the human detected by the first detecting means;
The movement determining means determines the movement of the robot based on the danger predicted by the prediction means and the human reaction detected by the second detection means.
The robot control apparatus according to claim 1. The predicting means predicts contact between the robot and the human as dangerous;
The robot control apparatus according to claim 2. Even if the predicting means predicts the danger to the human being of the robot, the operation determining means, when the predicted time until the occurrence of the danger has not reached a predetermined grace time, Does not change the behavior of the robot,
The robot control apparatus according to claim 2 or 3, wherein The operation determining means, even if the predicting means predicts the danger to the human being of the robot, the time to occurrence of the predicted danger has not reached a predetermined grace period, and When the distance between the robot and the human is a predetermined distance or more, do not change the operation of the robot.
The robot control apparatus according to claim 2 or 3, wherein The informing means informs the situation of the robot or the situation around the robot,
The reaction detected by the second detection means is the human reaction to the informed robot status.
The robot control apparatus according to any one of claims 1 to 5, wherein The state of the robot includes at least one of a temperature, a current value, a voltage value, a pressure, a gripping force, an operation speed, and an external force related to the operation of the robot, and a trajectory change state of the robot. ,
The robot control apparatus according to claim 6. The surrounding environment of the robot includes at least one of temperature, shape, size, mass, chemical reaction, and radioactive substance influence of an object held by the robot.
The robot control apparatus according to claim 6. The notification means includes a light emission means, and performs information notification by a display color of emitted light or an emission intensity of emitted light.
The robot control apparatus according to any one of claims 1 to 8, wherein An estimation means for estimating whether or not the person is aware of the notification information based on the reaction of the person detected by the second detection means;
The motion determining means determines the motion based on an estimation result of the estimating means;
The robot control device according to claim 1, wherein the robot control device is a robot. The reaction detected by the second detection means includes the human gaze or face orientation,
Whether the estimation means is aware of the notification information by obtaining the human visual field from the line of sight of the human or the face direction and determining whether the notification means is located in the visual field. To estimate,
The robot control apparatus according to claim 10. The reaction detected by the second detection means is the movement of the human eye or the movement of the face,
The estimating means obtains the human central visual field from the movement of the line of sight of the human or the movement of the face, and determines whether the obtained central visual field of the human has moved, thereby determining the notification information. Estimating whether or not
The robot control apparatus according to claim 10. Whether the estimation means is aware of the notification information by determining whether the person has moved from a position where danger is predicted by the prediction means based on the detected movement of the person. To estimate,
The robot control apparatus according to claim 10. The operation determining means determines at least one of operation stop, operation deceleration, and operation path change as the operation.
The robot control device according to claim 1, wherein the robot control device is a robot. The robot control device according to any one of claims 1 to 14, comprising at least one of a sensor and the robot,
The second detection means detects the human reaction based on information from the sensor,
The first detection means detects the position of the person based on information from the sensor.
A robot system characterized by this. A robot control method for controlling the operation of a robot,
Obtaining the position of the robot to be controlled;
Detecting a human position;
Informing the human-perceivable notification information;
Detecting the human reaction according to the notified information,
Determining the movement of the robot based on the detected human response;
A robot control method comprising:   The computer program for functioning the said computer as each means of the robot control apparatus of any one of Claim 1 to 14 by a computer reading and executing.

Images