JP6572092B2 - A moving body system using a visual sensor - Google Patents

Description

  The present invention relates to a moving body system and a robot system including a moving body whose safety for humans is enhanced by using a visual sensor.

  In Patent Document 1, the robot operates in an operation region where human entry is restricted. Then, when it is detected that a human has visited the operation area while the robot is performing a certain process, stopping the robot operation at that time will stop at a predetermined inconvenient stage. In some cases, the robot is stopped after going to another process after a certain process.

  Further, in Patent Document 2, when a maintenance request signal is input while the robot is operating, the block for the maintenance request signal input is operated up to the block end of the block to stop the robot. Then, it is disclosed that the robot is in a predetermined posture suitable for maintenance.

  Further, in Patent Document 3, the images recorded by the first image recording unit and the second image recording unit are analyzed by at least two different methods, and the robot is stopped when a foreign object is detected by any one of the analysis. It is disclosed.

  Further, in Patent Document 4, an operation designation area for outputting a command corresponding to the set processing level to the robot controller is stored, and the robot hand position calculated by the robot controller from the joint angle of the robot arm is stored. Is determined to be included in the motion designation area, and when the hand position of the robot is included in the motion designation area, the processing level is stopped and the robot is stopped. ing.

  Furthermore, as disclosed in Patent Document 5, in a machine with a numerical control (NC) device, a limit switch is attached to the machine in order to prevent the movable part of the machine from exceeding the movable region and damaging the machine. ing. When the limit switch is pressed when the movable part of the machine exceeds the movable region, a signal is output from the limit switch to the NC apparatus, and the NC apparatus receives the signal and stops the movable part. In Patent Document 5, the position of a limit (stored stroke limit) is stored in a non-volatile memory built in the NC device, and the machine coordinate value updated every time the movable part moves is the coordinate value of the stored stroke limit. It is disclosed that limit switchless in which the NC device stops the movable part when the value exceeds the limit is becoming common. Similarly, in the case of a robot, when the movable area of the robot is set as a stored stroke limit based on the coordinate position of the hand of the robot arm, for example, when the robot arm moves and the coordinate position exceeds the stored stroke limit, the robot controller A stored stroke limit function for stopping the robot arm is common.

Japanese Patent No. 5582422 JP-A-5-77179 Japanese Patent No. 4405468 Japanese Patent Laid-Open No. 3-37701 JP 60-230205 A

  However, in Patent Document 1, it is necessary to define and program inconvenient steps in advance. Therefore, when the technique of Patent Document 1 is applied to an existing robot system, it is necessary to change the program of the robot system or the circuit of the control unit. The cost and labor required for such a change are relatively large. Further, since a high degree of skill and carefulness are required for changing the program and circuit, the number of workers is limited, and it is difficult to freely change the program and the like at the site of the robot system. In addition, if there is a change mistake, there is a possibility that a human being will be dangerous by the robot system after the change. For this reason, in general, the program of an existing robot system is not actively changed.

  In Patent Document 2, it is necessary to set in advance information on a predetermined posture suitable for maintenance and information for moving the robot to that posture. In Patent Document 2, after a maintenance request signal is input, the robot temporarily stops at the block end. For this reason, when moving from the stop position of the block end to a predetermined posture suitable for maintenance, the robot may come into contact with an obstacle. There is a similar problem when returning to the stop position at the block end after maintenance.

  In Patent Document 3, since the robot stop position is a position when a foreign object is detected, the robot stop position is not set. However, depending on the stop position of the robot, it may be difficult to resume the operation of the robot.

  In Patent Document 4, the robot control device outputs a stop command to the robot. Further, the robot control device detects the hand position of the robot and determines whether or not the hand position is included in the motion designation area. Therefore, as in the case of Patent Document 1, when the technique of Patent Document 4 is applied to an existing robot, it is necessary to change the program of the robot or the circuit of the control unit. There are the same problems as Patent Document 1 regarding the cost and labor required for such a change, and the high level of skill and carefulness required for changing the program and circuit.

  Further, the stop command to the robot in Patent Document 5 depends on the detection of the robot hand position by the robot control device and the determination by the robot control device whether it is included in the movable restriction range. Therefore, for example, if the setting of the movement limit range of the robot is wrong, there is a risk that the tip position of the robot moves to an unexpected place. In addition, if there is a device independent of the robot control device that detects the robot's hand position and determines whether it is included in the operation restriction area, it is possible to monitor the movement of the robot twice, so that further safety is achieved. Can be planned.

  The present invention has been made in view of such circumstances, and can be applied to an existing system without changing a program or the like, and also presents a danger to human beings caused by mistakes that can occur by changing the program. It is an object to provide a moving body system and a robot system that can be eliminated as much as possible.

In order to achieve the above object, according to a first invention, in a moving body system including at least a moving body that is moved by a driving device and a moving body control unit that controls a drive command of the moving body to the driving device, An imaging unit that captures a predetermined section including at least a moving section; and an image processing unit that processes an image captured by the imaging unit. The moving object reaches a stop position taught in advance by the image processing unit. If it is detected, the image processing unit outputs a command to stop the moving object, and a moving object system is provided.
According to a second invention, in the first invention, the stop position is taught as a space of a predetermined size, and the image processing unit detects that the moving object has reached the space. In some cases, the moving body is regarded as having reached the stop position.
According to a third aspect, in the first or second aspect, the image processing unit includes a stop request unit, and when the moving object is requested to stop by an operation of the stop request unit. As long as the moving body has reached the stop position.
According to a fourth aspect of the present invention, in a moving body system including at least a moving body that is moved by a driving device and a moving body control unit that controls a driving command of the moving body to the driving device, the predetermined section including at least a section in which the moving body moves is provided. An image capturing unit that captures an image, and an image processing unit that processes an image captured by the image capturing unit, and when the position of the moving object detected by the image processing unit deviates from a predetermined space, the image A processing unit outputs a command to stop the moving body, and a moving body system is provided.
According to a fifth aspect of the present invention, in a moving body system including at least a moving body that is moved by a driving device and a moving body control unit that controls a driving command of the moving body to the driving device, the predetermined section including at least a section in which the moving body moves is provided. An imaging unit that captures an image and an image processing unit that processes an image captured by the imaging unit, and when the position of a moving object detected by the image processing unit exceeds a predetermined straight line or curve, An image processing unit outputs a command to stop the moving object, and a moving object system is provided.
According to a sixth invention, there is provided a robot system according to any one of the first to fifth inventions, wherein the moving object is a robot.

In the first invention, when the image processing unit detects that the moving object has reached the stop position, the moving object is stopped at the stop position. The stop position of the moving object is previously taught to the image processing unit as teaching data, and the image processing unit detects that the moving object has reached the stop position based on the teaching data, so that the moving object is stopped at the stop position. There is no need to adjust the program that controls the movement.
In the second invention, since the stop position of the robot is defined by a space of a predetermined size, a margin can be given to the stop position of the robot.
In the third invention, the robot can be stopped only when the stop is requested by the operation of the stop request unit.
In the fourth aspect of the invention, the image processing unit tracks the position of the moving object, and can stop the moving object when it deviates from a predetermined space. Therefore, even if the moving object deviates from the predetermined space due to a mistake in the program for controlling the moving object, the moving object can be stopped by detecting it by the image processing unit. Thus, it is not necessary to adjust the program for controlling the moving object in order to stop the moving object.
In the fifth aspect of the invention, the image processing unit tracks the position of the moving object and can stop the moving object when it exceeds a predetermined straight line or curve. When a straight line or curve is exceeded, the moving object can be stopped by detecting it with the image processing unit. Thus, it is not necessary to adjust the program for controlling the moving object in order to stop the moving object.
In the sixth invention, since the moving body is a robot, the effects of the first to fifth inventions can be achieved also in the robot.

  These and other objects, features and advantages of the present invention will become more apparent from the detailed description of exemplary embodiments of the present invention illustrated in the accompanying drawings.

1 is a schematic diagram of a robot system according to the present invention. FIG. 3 is a diagram of a reference image and an image model according to the present invention. It is a 1st flowchart which shows the process of an image process part. It is a 2nd flowchart which shows the process of an image process part. It is a 3rd flowchart which shows the process of an image process part.

Embodiments of the present invention will be described below with reference to the accompanying drawings. In the following drawings, the same members are denoted by the same reference numerals. In order to facilitate understanding, the scales of these drawings are appropriately changed.
A specific example of the moving body system according to claim 1 will be described using a robot system. FIG. 1 is a schematic diagram of a robot system according to the present invention. As shown in FIG. 1, the robot system 1 of the present invention includes at least a robot 10 and a robot controller 20 that controls the robot 10. Further, FIG. 1 shows a visual sensor 11 that includes the robot 10 and its surroundings in the field of view, and an image processing unit 25 that processes an image captured by the visual sensor 11 and outputs various signals.

  The robot 10 is an industrial robot 10, for example, an articulated robot 10. Therefore, the robot 10 is driven by at least one driving device, for example, a motor. Accordingly, the robot control unit 20 controls a drive command for the drive device. The robot 10 is an example of a moving object, and may be a system including other moving objects. The visual sensor 11 is disposed at the tip of the post 14 disposed in the vicinity of the robot 10 and images the robot 10 and the like from above. The visual sensor 11 includes a camera incorporating a TOF (Time of Flight) area sensor device and an LED light projecting device. The visual sensor 11 measures the time when the reflected light of the LED light returns and takes a two-dimensional depth image.

  Further, referring to FIG. 1, a safety fence 19 is arranged around the robot 10. The safety fence 19 restricts the entry of people to the robot operation section where the robot 10 operates. In the visual field of the visual sensor 11, a predetermined section including at least a part of the robot operation section is imaged. The predetermined section may be defined by means other than the safety fence 19, such as a white line arranged on the floor. In FIG. 1, the visual sensor 11 images a predetermined section from above, but the visual sensor 11 may capture images from other directions.

  Images picked up by the visual sensor 11 every predetermined time are processed by the image processing unit 25 of the robot control unit 20. Thereby, a robot operating in a predetermined section is detected as described later. By performing such processing every predetermined time, it is possible to track the robot within a predetermined section.

  For the image processing unit 25, an image model related to the robot 10 is taught in advance. The image processing unit 25 identifies and detects the robot 10 by matching the image captured by the visual sensor 11 with these image models. The image model is created by cutting out only the characteristic portion of the robot 10 from a preprocessed reference image (described later).

  As shown in FIG. 1, the robot control unit 20 is connected to a stop unit 21. The stop unit 21 outputs a command to stop the robot 10 according to the processing result of the image processing unit 25. Note that the image processing unit 25 may output a stop command for stopping the robot 10.

  The command to stop the robot 10 is a command to stop the robot 10 after decelerating, for example. In this case, the load on the robot 10 can be avoided. Alternatively, the command for stopping the robot 10 may be a command for shutting off the power source of the robot 10. In this case, the robot 10 can be stopped reliably. The image processing unit 25 includes a stop request unit 29 operated by an operator. It may be detected that the robot 10 has reached the stop position only when the stop request unit 29 is operated to request the robot 10 to stop. That is, in this case, the robot is stopped only when it is desired to stop the robot at the stop position, and in other cases, the robot is not stopped even when the robot reaches the stop position.

  As described above, the visual sensor 11 is attached to the tip of the post 14. For this reason, the image imaged by the visual sensor 11 corresponds to a plan view of a predetermined section inside the safety fence 19 as shown in FIG.

  A reference image is registered in the image processing unit 25 in advance. The reference image is an image captured by the visual sensor 11 when the robot 10 is moved to the stop position taught by the image processing unit 25. For example, in the image shown in FIG. 2, the robot 10 is stopped on the peripheral device 40. In this case, a reference image is taught in advance to the image processing unit 25 so that the robot 10 can be stopped at a position on the peripheral device 40. In the present invention, an image captured in advance in such a state is used as a reference image.

  A plurality of stop positions may be taught, and in that case, a plurality of reference images corresponding to the respective stop positions are registered. Even in such a case, it is possible to easily teach the stop position of the robot in the robot system 1 simply by moving the robot 10 to each stop position and taking an image with the visual sensor 11. Further, since the stop position of the robot 10 is displayed on the display unit of an operation panel (not shown) connected to the image processing unit 25 based on the reference image, the operator intuitively obtains information on the stop position of the robot 10 and the like. Can be obtained.

  Moreover, it is preferable that the stop position is set on the track of the work operation when the robot 10 operates according to a predetermined operation program. For example, considering the case where the robot 10 is maintained, a place suitable for the maintenance is often on the trajectory of the robot's work operation. Therefore, when there is a stop position on the trajectory of the work operation, the maintenance can be easily performed, and the operation of the robot after the maintenance can be resumed very smoothly.

  FIG. 3 is a first flowchart showing the processing of the image processing unit. With reference to FIG. 3, the teaching of the stop position in advance and the detection of the robot reaching the stop position will be described. The robot is moved to a position where the robot is to be stopped, and the image captured by the visual sensor is used as a reference image at the stop position. In step S5, an image model is created by cutting out the part that is characteristic of the robot from the reference image. Then, the image processing unit collates the image captured when the robot reaches the stop position (step S1) with the image model when the image model is matched (steps S2 and S3), so that the image processing unit stops the stop position taught in advance by the robot. Detects that it has reached.

  Steps S1, S2, and S3 in FIG. 3 are repeatedly performed at predetermined control cycles. The captured image captured by the visual sensor 11 of the tap S1 is model-matched in step S2 with the image model for detecting the robot created in step S5 from the reference image captured in the stop position in FIG. . And if it collates by model matching in step S3, it will detect that the robot arrived at the stop position taught beforehand. If not collated, the process of model matching in step S2 and collation in step S3 are repeated from the imaging in step S1.

  If it is detected in step S3 that the robot 10 has reached the previously taught stop position, the stop unit 21 outputs a command to stop the robot 10 in step S4. Thereby, the robot 10 stops at the stop position.

  Note that the stop position of the robot 10 may be taught as a space having a predetermined size. When the image processing unit 25 detects that the robot 10 has reached these spaces, it is considered that the robot 10 has reached the stop position and the passage position. In this case, a margin can be given to the stop position of the robot.

  FIG. 4 is a second flowchart showing processing of the image processing unit. A robot system that outputs a command for stopping the robot 10 when the position of the robot 10 detected by the image processing unit 25 deviates from a predetermined space will be described with reference to FIG. As shown in FIG. 4, in order to track the robot 10 at each control cycle, a reference image for creating an image model of the feature mark of the robot 10 is displayed, and the robot 10 is placed so that the feature mark enters the visual field of the visual sensor 11. The image is taken by the visual sensor 11 after being moved. In step S1a, the feature mark portion is cut out from the reference image to create an image model of the feature mark.

  As shown in FIG. 4, in step S2a, the visual sensor 11 captures the robot 10 and the like for each control period, and in step S3a, the captured image and the feature mark image model are model-matched. When the robot 10 is in the visual field of the visual sensor 11, matching is performed by model matching, and the collated position becomes the position of the robot 10 in the visual field of the visual sensor 11 of the feature mark. As shown in FIG. 4, in the visual field of the visual sensor 11, a range in which the robot 11 is allowed to move is previously taught as a movable space A of the robot feature mark. In step S4a, it is determined whether or not the feature mark is within the movable space A of the feature mark due to the movement of the robot 10. If the characteristic mark of the robot 10 deviates from the movable space A, a stop command is output to the robot 10 in step S5a. The visual sensor 11 includes a camera incorporating a TOF (Time of Flight) area sensor device and an LED floodlight device, and measures the time when the reflected light of the LED light returns to obtain a two-dimensional depth image. Therefore, not only a two-dimensional space in the captured image but also a three-dimensional space using depth data can be set. In other words, in the embodiment shown in FIG. 4, the stop command is output when the position of the robot 10 deviates from the space A that is a predetermined three-dimensional space.

  FIG. 5 is a third flowchart showing the processing of the image processing unit. FIG. 5 illustrates a robot system that outputs a command to stop the robot 10 when the position of the robot 10 detected by the image processing unit 25 exceeds a predetermined straight line or curve. Steps S1a, S2a, S3a, and S5a are the same as in FIG. As shown in FIG. 5, in the visual field of the visual sensor 11, a range allowing the robot to move is previously taught as a movable range of the robot characteristic mark. While the movable range of FIG. 4 is the closed space A, in FIG. 5, the visual field of the visual sensor is divided by a straight line B or a curve, and one of the divided spaces is a movable space. In step S4a, it is determined whether or not the feature mark is within the movable space of the feature mark due to the movement of the robot. Next, when the feature mark of the robot 10 is out of the movable space, a stop command is output to the robot 10 in step S5a. Even when the straight line B or the curve does not completely divide the field of view and is partially arranged in the field of view, a stop command is output to the robot 10 when the robot goes over the partial line or curve. May be. Also, the movable space can be set three-dimensionally using depth data, as in the case of FIG. In other words, in the embodiment shown in FIG. 5, a stop command is output when the position of the robot 10 exceeds a predetermined straight line B or curve.

  Thus, in the present invention, when the image processing unit 25 detects that the robot 10 has reached the stop position, a stop command is output so as to stop the moving object at the stop position. For this reason, it is not necessary to change the program in the present invention. For this reason, it is possible to eliminate as much as possible the danger to humans caused by mistakes that can occur by changing the program. Therefore, it will be understood that human safety can be ensured.

  Although the present invention has been described using exemplary embodiments, those skilled in the art can make the above-described changes and various other changes, omissions, and additions without departing from the scope of the invention. You will understand.

1 Robot system (moving body system)
10 Robot (moving object)
11 Visual sensor (imaging part)
14 Post 19 Safety fence 20 Robot controller (moving object controller)
21 Stop unit 25 Image processing unit 29 Stop request unit

Claims (6)

In a moving body system including at least a moving body that is moved by a driving device and a moving body control unit that controls a drive command of the moving body to the driving device,
An imaging unit for imaging a predetermined section including at least a section in which the moving body moves;
An image processing unit that processes an image captured by the imaging unit,
The image processing unit extracts an image model created by cutting out a characteristic part of the moving object from a reference image as an image captured by the imaging unit when the moving object is moved to a stop position taught in advance. Including
When the image processing unit detects that the moving object has reached the previously taught stop position by matching the image captured by the image capturing unit with the image model , the image processing unit A moving body system that outputs a command to stop the moving body. The stop position is taught as a space of a predetermined size;
2. The moving object according to claim 1, wherein, when the image processing unit detects that the moving object has reached the space, the moving object is considered to have reached the stop position. 3. system.   The image processing unit includes a stop request unit, and the operation of the stop request unit considers that the moving object has reached the stop position only when there is a request to stop the moving object. The moving body system according to claim 1 or claim 2. In a moving body system including at least a moving body that is moved by a driving device and a moving body control unit that controls a drive command of the moving body to the driving device,
An imaging unit for imaging a predetermined section including at least a section in which the moving body moves;
An image processing unit that processes an image captured by the imaging unit,
The image processing unit extracts an image model created by cutting out a characteristic part of the moving object from a reference image as an image captured by the imaging unit when the moving object is moved to a stop position taught in advance. Including
By the image processing unit for matching with the image model and an image captured by the imaging unit, when the position of the moving object is detected to be deviating from the predetermined space, the image processing section the A moving system that outputs a command to stop the moving body. In a moving body system including at least a moving body that is moved by a driving device and a moving body control unit that controls a drive command of the moving body to the driving device,
An imaging unit for imaging a predetermined section including at least a section in which the moving body moves;
An image processing unit that processes an image captured by the imaging unit,
The image processing unit extracts an image model created by cutting out a characteristic part of the moving object from a reference image as an image captured by the imaging unit when the moving object is moved to a stop position taught in advance. Including
By the image processing unit for matching with the image model and an image captured by the imaging unit, when the position of the moving object is detected that exceeds a predetermined straight line or curve, the image processing unit Outputs a command to stop the moving body.   The robot system according to claim 1, wherein the moving body is a robot.

Images