JP2020116717A - Robot control system - Google Patents

Abstract

To accurately perform abnormality diagnosis of a robot control system by using a camera.SOLUTION: Mark parts 61 and 62 are provided in a position having the prescribed positional relationship with respect to work areas 38 and 57, the mark parts 61 and 62 are imaged by both of a hand camera 40 and a fixed camera 51 in calibration time of a robot 11 thereby obtaining captured images, which are then processed thereby detecting positions of the mark parts 61 and 62, and coordinate values of the positions of the mark parts 61 and 62 are stored in a storage section 46. The mark parts 61 and 62 are then imaged by both of the hand camera 40 and the fixed camera 51 in the inspection time thereby obtaining captured images, which are then processed thereby detecting the positions of the mark parts 61 and 62, then coordinate values thereof are compared with coordinate values of the positions of the mark parts 61 and 62 in the calibration time stored in a storage section 46, and an abnormality diagnosis of a robot control system is performed based on these comparison results, and when determined that there is abnormality, an abnormality occurrence place is identified and a proper coping method for the abnormality is notified to a user by display or voice.SELECTED DRAWING: Figure 2

Description

This specification discloses a technique relating to a robot control system having an abnormality diagnosis function.

In a robot system for the purpose of gripping, conveying, and working a workpiece, a camera (hand camera) attached to the tip of the robot arm is used for applications requiring high-precision positioning and automated teaching of the workpiece position. , The position coordinate of the work and the position coordinate of the work target are detected using the camera (fixed camera) fixed above the work supply device and the work target part, and the work grip position and the work target position are automatically corrected. System is used.

When instructing the robot the work position by using the image processing result of the image captured by such a camera (two-dimensional), each coordinate system, that is, the vision coordinate system (two-dimensional) which is the coordinate system of the camera and the coordinate of the robot. Since the systems (three-dimensional) are different, it is necessary to measure the relative positions of the two coordinate systems to convert the coordinate systems. Obtaining this conversion formula and correction parameters is called calibration.

If you continue to operate the robot after you have calibrated the robot coordinate system when installing the robot, and if you continue to operate the robot, the secular change of the robot and the installation base after the start of operation, the vibration of the robot operation and the movement of the robot arm, The robot coordinate system may change due to a shock, a change in peripheral equipment, or the like, which becomes a factor that deteriorates the work accuracy of the robot.

In particular, the fixed camera fixedly placed on the robot above the work supply device and the work target part has a wide field of view, and the deterioration of the position detection accuracy causes the work accuracy of the robot to deteriorate, and the work accuracy required for the system can be secured. Therefore, it may be difficult to continue production.

In recent years, as described in Patent Document 1 (Japanese Patent Laid-Open No. 2018-034271), when a work transfer operation by a robot fails, a cause of failure is obtained by using a camera attached to the tip of the robot arm. Have been proposed. In the failure factor identification method of Patent Document 1, a reference mark indicating a reference position and a reference direction in a robot coordinate system is set on a calibration stage, and a master work is held by a hand at the tip of an arm of a robot to calibrate the calibration stage. Then, the camera mounted on the tip of the robot arm is moved above the calibration stage, the reference mark and the master work are captured within the field of view of the camera, and the image is taken. By measuring the deviation amount of the position and orientation of the reference mark and the deviation amount of the position and orientation of the master work, and determining whether the cause of failure is the camera or the robot based on these deviation amounts. I am trying to judge.

JP, 2008-034271, A

However, in Patent Document 1, when the position or orientation of the calibration stage provided with the reference mark is misaligned, it is erroneously determined that the cause of failure is in the camera (the position or orientation of the camera is misaligned). Or, there is a possibility of erroneously determining that the cause of the failure lies in the robot (the robot coordinate system is misaligned). Moreover, when the master work is gripped by the robot hand, if the master work is misaligned, the gripping position at which the robot hand grips the master work is misaligned, so the master work placed on the calibration stage The position of is also displaced, and there is a possibility that the cause of failure may be erroneously determined.

In order to solve the above problems, a supply device that supplies a work to a work area, a robot that performs a predetermined work on the work supplied to the work area, and a hand camera attached to an arm of the robot, A fixed camera, which is fixed downward at a position higher than the arm movable area of the robot, and which picks up an image of the work supplied to the work area from above, and a position of an image pickup target by processing an image picked up by the hand camera. A first image processing unit that detects a coordinate value of a coordinate system whose origin is the reference point of the image (hereinafter referred to as "hand camera vision coordinate system"), and a position of an image capturing target by processing an image captured by the fixed camera. A second image processing unit that detects a coordinate value of a coordinate system having the reference point of the image as an origin (hereinafter referred to as “fixed camera vision coordinate system”), and the coordinate value of the hand camera vision coordinate system and the fixed value. A coordinate conversion unit that converts the coordinate value of the vision coordinate system of the camera into the coordinate value of the world coordinate system that is the coordinate system that controls the operation of the robot arm, and the position of the robot arm is the coordinate value of the world coordinate system. In a robot control system including a control unit for controlling by the above, a mark portion provided in a position having a predetermined positional relationship with the work area within a range that fits in the visual field of the fixed camera, and a visual field of the hand camera The hand camera is moved to a position where the mark portion is accommodated, the mark portion is imaged by the hand camera, the image is processed by the first image processing portion, and the position of the mark portion is set by the hand camera. The operation of detecting the coordinate value of the vision coordinate system and converting it to the coordinate value of the world coordinate system by the coordinate conversion unit (hereinafter referred to as "mark position detection operation from the hand camera image"), and the fixed camera An operation of capturing an image of the mark portion, processing the image by the second image processing portion, and detecting the position of the mark portion by the coordinate value of the vision coordinate system of the fixed camera (hereinafter, “mark portion from fixed camera image”). And a coordinate value of a position of the mark portion in the past detected by a mark portion position detecting operation from the hand camera image performed in the past by the inspection executing unit (hereinafter referred to as "position detecting operation"). The “coordinate value of the past position of the mark portion detected from the past hand camera image” is stored and detected by the mark portion position detection operation from the fixed camera image performed in the past by the inspection execution unit. The coordinate value of the position of the mark portion in the past (hereinafter, "the past coordinate value detected from the past fixed camera image (Hereinafter referred to as the “coordinate value of the position of the mark portion”), the inspection execution unit stores the position of the mark portion detected this time by the mark position detection operation from the hand camera image performed this time. The coordinate value is compared with the coordinate value of the past position of the mark portion detected from the past hand camera image stored in the storage unit, and the mark portion position detection operation from the fixed camera image performed this time is performed. The coordinate value of the position of the mark portion detected this time is compared with the coordinate value of the past position of the mark portion detected from the past fixed camera image stored in the storage unit, and the comparison result thereof is obtained. The abnormality is diagnosed based on the above.

In this configuration, both the hand camera attached to the robot arm and the fixed camera fixed downward at a position higher than the robot arm movable area are provided in a predetermined positional relationship with respect to the work area. Since the image of the mark portion is picked up and the position of the same mark portion is detected in both the image of the hand camera and the image of the fixed camera to perform abnormality diagnosis, the calibration stage which is a factor of erroneous determination in Patent Document 1 Abnormality can be diagnosed without using masterwork. Moreover, since the same mark portion is imaged by both the hand camera and the fixed camera, it is possible to determine whether the camera position is displaced or the mark portion is displaced, and the accuracy of abnormality diagnosis can be improved.

FIG. 1 is a front view showing the appearance of the robot control system. FIG. 2 is a plan view for explaining the arrangement of mark parts in the robot control system. FIG. 3 is a plan view illustrating the arrangement of mark portions provided on the upper surface of the tray transport device. FIG. 4 is a block diagram showing the electrical configuration of the robot control system. FIG. 5 is a flowchart showing the flow of processing of the mark position detection/storage program during calibration. FIG. 6 is a flowchart (No. 1) showing the flow of a part of the processing of the robot control system abnormality diagnosis program. FIG. 7 is a flowchart (part 2) showing the flow of part of the processing of the robot control system abnormality diagnosis program. FIG. 8 is a flowchart (part 3) showing the flow of part of the processing of the robot control system abnormality diagnosis program.

An embodiment disclosed in the present specification will be described below with reference to the drawings.
First, the configuration of the robot 11 will be described with reference to FIG.

The robot 11 is, for example, a 5-axis vertical articulated robot, and includes a fixed base 13 installed on the factory floor 12, and a first base provided on the fixed base 13 so as to be rotatable about a first joint shaft 14 (J1). One arm 15, a second arm 17 provided at the tip of the first arm 15 so as to be pivotable by a second joint shaft 16 (J2), and a third joint shaft 18 (J3) at the tip of the second arm 17. The third arm 19 rotatably provided by the third arm 19, the wrist portion 21 rotatably provided by the fourth joint shaft 20 (J4) at the tip of the third arm 19, and the fifth joint shaft 21 by the wrist portion 21. 22 (J5) and an end effector 23 which is attached so as to be rotatable and replaceable. As a result, the end effector 23 attached to the wrist portion 21 is swung by the fourth joint shaft 20 which is the joint shaft of the wrist portion 21.

In this case, the end effector 23 may be any one of a suction nozzle, a hand, a gripper, a welding machine, and the like. The first to fifth joint shafts 14, 16, 18, 20, 22 of the robot 11 are driven by servomotors 25 to 29 (see FIG. 4), respectively. As shown in FIG. 4, each of the servo motors 25 to 29 is provided with an encoder 31 to 35 for detecting a rotation angle, and information about the rotation angle detected by each of the encoders 31 to 35 is transmitted via the servo amplifier 36. It is fed back to the control unit 37. As a result, the controller 37 feeds back each servo motor 25-29 via the servo amplifier 36 so that the rotation angle of each servo motor 25-29 detected by each encoder 31-35 matches each target rotation angle. By controlling, the positions of the arms 15, 17, 19 of the robot 11, the wrist part 21, and the end effector 23 are feedback-controlled to the respective target positions.

In the configuration example of FIG. 4, the servo amplifier 36 is a multi-axis amplifier that feedback-controls the plurality of servo motors 25 to 29, but the servo motors 25 to 29 are feedback-controlled by separate servo amplifiers one by one. Is also good.

As shown in FIGS. 1 and 2, at a predetermined position of an arm movable region of the robot 11 (a region where the end effector 23 on the distal end side of the wrist 21 can move), the work 30 to be worked is positioned at a certain height position. A supply device 39 for supplying the work area 38 is installed. The supply device 39 may be configured by a conveyor, or a parts feeder having any structure such as a vibration type parts feeder may be used. In short, the height position of the work area 38 is known. It may be a constant height position. In this embodiment, a plurality of supply devices 39 are arranged side by side.

As shown in FIG. 1, a tray transfer device 55 is installed between the robot 11 and a plurality of supply devices 39, and an empty tray 56 is provided near the work area 38 of the supply device 39 by the tray transfer device 55. It is carried in to another work area 57 arranged at. The robot 11 picks up the work 30 supplied to the work area 38 of each supply device 39 by the end effector 23, transfers the work 30 to the tray 56 stopped in the work area 57 of the tray transfer device 55, and aligns it. Do the work. Each time the robot 11 transfers a predetermined number of works 30 to the tray 56, the tray transfer device 55 restarts the transfer operation to carry out the tray 56, and replaces it with the next empty tray 56.

As shown in FIGS. 1 and 2, a hand camera 40 for picking up an image of the work 30 supplied to the work area 38 from above is attached to the wrist 21, which is the arm tip of the robot 11. The hand camera 40 is a two-dimensional camera that captures a two-dimensional image, the optical axis of which is parallel to the fifth joint axis 22, and the lens 41 of the hand camera 40 has a tip end side of the fifth joint axis 22 ( It is fixed so that it is located on the arm tip side).

Further, a fixed camera 51 is fixed downward at a predetermined position of a fixed structure 50 (for example, the ceiling of a robot protection fence) installed at a position higher than the arm movable area of the robot 11, and its lens 52 side is directed downward. Has been. The fixed camera 51 is used as a two-dimensional camera that captures the work 30 supplied to the work area 38 of the supply device 39 and the work 30 transferred to the tray 56 in the field of view from the upper side.

When imaging the work 30 supplied to the work area 38 of the supply device 39, the hand camera 40 and the fixed camera 51 are selectively used depending on the required resolution, the position of the work area 38 in the arm movable area of the robot 11, and the like. I am trying. Here, the visual field of the fixed camera 51 and the visual field of the hand camera 40 are narrower than the arm movable region of the robot 11. Comparing the hand camera 40 and the fixed camera 51, the fixed camera 51 has a wider field of view, but the hand camera 40 has a finer resolution (actual length of the imaging target per pixel), and a high-precision image. It can be processed. Therefore, for example, when the size of the work 30 on the work area 38 is small and a fine resolution is required, the hand camera 40 is used, and the work area installed outside the visual field of the fixed camera 51 is used. For 38, the hand camera 40 may be used. Further, when a plurality of work areas 38 fit within the field of view of the fixed camera 51, if the plurality of work areas 38 are housed within the field of view of the fixed camera 51 and imaged, an image of the work 30 on the plurality of work areas 38 can be obtained. The processing can be performed efficiently. Alternatively, as shown in FIG. 2, when there are a plurality of work areas 38, the hand camera 40 may be moved for each work area 38 to image the work 30.

As shown in FIG. 2, for each work area 38 of each supply device 39 within the range of the field of view of the fixed camera 51, one or a plurality of mark portions 61 are provided at a position having a predetermined positional relationship with the work area 38. They are provided individually. Each mark portion 61 is integrated with each supply device 39, and when each supply device 39 is displaced, each mark portion 61 is also displaced in the same direction by the same amount. The shape of each mark portion 61 is formed in a shape, for example, a circle, which makes it easy to accurately recognize the position of the center of each mark portion 61 by image processing.

Also in the work area 57 of the tray transfer device 55, one or a plurality of mark portions 62 are provided at a position having a predetermined positional relationship with the work area 57 within the range of the fixed camera 51. The mark portions 62 are integrated with the tray transport device 55 and the tray 56, respectively, and when the tray transport device 55 and the tray 56 are displaced, the mark portions 62 are displaced in the same direction by the same amount. It is set. The shape of each mark portion 62 is formed in a shape such that the position can be easily recognized accurately by image processing, for example, a circle.

The mark parts 61, 62 are used for abnormality diagnosis described later, and when the mark parts 61, 62 are housed in the field of view of the fixed camera 51 during the image pickup during production, the position of the work 30 is the closest. It is expressed by the coordinate value of the relative position from the position of the mark portions 61, 62. For this reason, the coordinate values (X1w, Y1w) on the world coordinates are measured in advance when the mark portions 61 and 62 are calibrated and stored in the storage portion 46.

In general, the mark portions 61 and 62 may be arranged such that one mark portion 61 is arranged for one supply device 39 when the work detection target area is narrow as in the supply device 39 shown in FIG. On the other hand, when the work detection target area is relatively wide like the tray 56 shown in FIGS. 2 and 3, a plurality of mark portions 62 are arranged around the area, and the work detection target area is marked by the mark portion 62. The position of the work 30 on the tray 56 is calculated by the coordinate value of the relative position with the position of the nearest mark portion 62 as the reference point.

In the example of FIG. 3, four mark portions 62 are arranged around the tray 56, the work detection target area is divided into four, and the position of each work 30 on the tray 56 is set to the nearest mark portion 62. It is calculated by the coordinate value of the relative position with the position as the reference point.

Of the workpieces 30 on the tray 56, the mark portion 62 closest to the workpiece 30 (A) to be detected is the mark portion 62 (a) at the upper left of FIG. ) The world coordinate values (X1w, Y1w), which are the coordinate values of the world coordinate system, and the relative position offset (ΔX1, ΔY1) to the workpiece 30 (A) to be detected are used to detect the workpiece 30 on the tray 56. The coordinate value (Xw, Yw) of the world coordinate system of (A) is calculated by the following formula.
[Xw, Yw]=[X1w, Y1w]+[ΔX1, ΔY1]
The other works 30 on the tray 56 are also calculated by the relative position from the position of the nearest mark portion 62, as in the above.

As shown in the above equation, by expressing the position of the work 30 as a relative position from the position of the nearest mark portion 62, it is possible to have a plurality of reference points in the world coordinate system. In the calculation of the world coordinate value from the image of the normal fixed camera 51, the world coordinate value of one point at the center of the image is detected and expressed as a relative value, so that the image peripheral portion is affected by factors such as image distortion. However, the plurality of mark portions 61 and 62 are arranged in the peripheral portion of the image, and the position coordinate of the work 30 is calculated by the relative value from the absolute coordinate of each mark portion 61 and 62. As a result, it is possible to improve the detection accuracy in the peripheral portion of the image.

As shown in FIG. 4, the robot control unit 42 for controlling the operation of the robot 11 configured as above includes an image processing unit 43, a coordinate conversion unit 44, a control unit 37, an inspection execution unit 45, a servo amplifier 36, and the like. It is equipped with it. The image processing unit 43 processes the image captured by the hand camera 40, and uses the coordinate value of the coordinate system (hereinafter referred to as “vision coordinate system of the hand camera 40”) whose origin is the reference point of the image to be captured. A first image processing unit for detection and a two-dimensional Cartesian coordinate system (hereinafter referred to as “fixed”) that processes an image captured by the fixed camera 51 and sets the position of the image capturing target as the origin of a reference point (for example, the center of the image) of the image. It functions as a second image processing unit that detects the coordinate value of the "vision coordinate system of the camera 51". The first image processing unit and the second image processing unit may be configured by the same image processing device or may be configured by different image processing devices. The coordinate axes of each vision coordinate system are the Xv axis and the Yv axis that are orthogonal to each other on the image. When the hand camera 40 images the work 30 on the work area 38 of the supply device 39, since the optical axis of the hand camera 40 is vertically downward, the image taken by the hand camera 40 is the fixed camera 51. Similar to the image captured in (1), the image is obtained by capturing a horizontal plane, and the Xv axis and the Yv axis of the vision coordinate system are orthogonal coordinate axes on the horizontal plane. The image processing unit 43 detects the position of the work 30 by image processing by the coordinate value of each pixel in the vision coordinate system.

On the other hand, while the robot 11 is in operation, the coordinate conversion unit 44 converts the coordinate values of the two-dimensional vision coordinate system detected as the position of the work 30 by the image processing of the image processing unit 43 into the arms 15 and 17 of the robot 11. , 19, the wrist 21, and the end effector 23 are converted into coordinate values in the world coordinate system for controlling the positions. This world coordinate system is a three-dimensional orthogonal coordinate system whose origin is the reference point, and the coordinate axes of the world coordinate system are the orthogonal coordinate axes (X axis and Y axis) on the horizontal plane and the vertically upward coordinate axis (Z axis). is there. The unit of the coordinate value of this world coordinate system is the unit of length (for example, μm unit). The origin (reference point) of this world coordinate system is, for example, the center of the arm movable region of the robot 11 (the region where the end effector 23 on the tip side of the wrist 21 can move).

In this case, when the coordinate conversion unit 44 converts the coordinate value of the vision coordinate system of the fixed camera 51 into the coordinate value of the world coordinate system while the robot 11 is in operation, the vision of the fixed camera 51 is converted using the coordinate conversion parameter. Converts coordinate values in the coordinate system to coordinate values in the world coordinate system. Here, the coordinate conversion parameter includes an origin correction parameter that corrects the origin of the vision coordinate system of the fixed camera 51 and a coordinate axis tilt correction parameter that corrects the tilt of the coordinate axis of the fixed camera 51 in the vision coordinate system.

On the other hand, when converting the coordinate value of the vision coordinate system of the hand camera 40 into the coordinate value of the world coordinate system, the inclination angle Rz and the origin of the coordinate axis of the vision coordinate system of the hand camera 40 with respect to the coordinate axis of the world coordinate system are corrected. Thus, the coordinate value of the hand camera 40 in the vision coordinate system is converted into the coordinate value of the world coordinate system. This correction is performed by obtaining the rotation matrix indicating the inclination angle Rz of each coordinate axis and the translation vector of the vision coordinate origin to the position (X1w, Y1w) on the world coordinate system, and thus the coordinates of the vision coordinate system of the hand camera 40. The value (Xav, Yav) can be converted into the coordinate value (Xw', Yw') in the world coordinate system by correcting it with the following equation.

In order to perform coordinate conversion using the above equation, it is necessary to measure the tilt angle Rz of the coordinate axis of the vision coordinate system of the hand camera 40 and the coordinate value of the origin.

Since the hand camera 40 is fixed to the wrist portion 21 of the robot 11, the relative positional relationship between the arm position of the robot 11 and the hand camera 40 is always kept constant. Therefore, the tilt angle Rz of the coordinate axes of the vision coordinate system of the hand camera 40 with respect to the XY coordinate axes of the world coordinate system is calculated by the following equation.

Rz=[installation angle of hand camera 40]+[rotation angle of first joint shaft 14]
Here, the installation angle of the hand camera 40 is the installation angle of the hand camera 40 with respect to the wrist 21 of the robot 11, and when the hand camera 40 is installed on the wrist 21 of the robot 11, the installation of the hand camera 40 with respect to the wrist 21. The installation angle is measured and the value stored in the nonvolatile memory (not shown) of the robot control unit 42 is referred to.

Regarding the method of detecting the rotation angle of the first joint shaft 14, the output of the encoder 31 of the servo motor 25 of the first joint shaft 14 when the arm position of the robot 11 (position of the hand camera 40) is stopped at the imaging position The control unit 37 of the robot control unit 42 acquires the value in real time and calculates the rotation angle of the first joint shaft 14.

On the other hand, the coordinate value of the origin of the vision coordinate system of the hand camera 40 is handed to the robot 11 so that the control unit 37 of the robot control unit 42 matches the coordinate value of the world coordinate system designated as the imaging position of the hand camera 40. The camera 40 is left in a calibrated state when assembled. Since the relative positional relationship between the arm position of the robot 11 and the hand camera 40 is always constant, the distance between the reference point of the robot 11 and the optical axis center of the hand camera 40 is measured, and this is controlled as a correction value. When the robot 11 is instructed to the position (X1w, Y1w) on the world coordinate system by reflecting it on the coordinate command value from the unit 37, the origin of the vision coordinate system of the hand camera 40 is on the world coordinate system (X1w, Y1w) is controlled to move.

The control unit 37 of the robot control unit 42 is based on the rotation angle of each servo motor 25 to 29 detected by each encoder 31 to 35 and the relative positional relationship between the hand camera 40 and the wrist 21 (end effector 23) in real time. Then, the coordinate value (X1w, Y1w) of the origin of the vision coordinate system of the hand camera 40 is calculated.

The control unit 37 of the robot control unit 42 controls each arm 15, 17, 19 of the robot 11 based on the position of the work 30 converted into the coordinate values of the three-dimensional world coordinate system by the coordinate conversion unit 44 during the operation of the robot 11. The target positions of the wrist 21 and the end effector 23 are set by the coordinate values of the world coordinate system, and the positions of the arms 15, 17, 19 and the wrist 21 and the end effector 23 are controlled by the coordinate values of the world coordinate system. ..

Next, the function of the inspection execution unit 45 will be described.
The inspection execution unit 45 executes the abnormality diagnosis of the robot control system using the fixed camera 51 and the hand camera 40, and when the predetermined inspection execution condition described later is satisfied, the inspection execution unit 45 is within the visual field of the hand camera 40. The hand camera 40 is moved to a position where the mark portions 61 and 62 are accommodated, the hand camera 40 takes an image of the mark portions 61 and 62, and the image processing unit 43 processes the image to determine the position of the mark portions 61 and 62. The operation of detecting with the coordinate value of the vision coordinate system of the hand camera 40 and converting into the coordinate value of the world coordinate system with the coordinate conversion unit 44 (hereinafter referred to as "mark portion position detection operation from the hand camera image"), and the fixed camera 51. Takes all the mark parts 61 and 62 with, and processes the image by the image processing part 43 to detect the positions of all the mark parts 61 and 62 by the coordinate values of the vision coordinate system of the fixed camera 51 to perform coordinate conversion. The unit 44 performs an operation of converting the coordinate value into a coordinate value in the world coordinate system (hereinafter referred to as "mark portion position detection operation from fixed camera image").

The coordinate values of the positions of the past mark portions 61 and 62 detected by the mark position detecting operation from the hand camera image performed by the inspection execution unit 45 in the past (hereinafter, “the past mark portion 61 detected from the past hand camera image”). , 62) is stored in the storage unit 46, and the past positions of the mark portions 61 and 62 detected by the mark portion position detection operation from the fixed camera image performed by the inspection execution unit 45 in the past. (Hereinafter, referred to as “coordinate values of past positions of the mark portions 61 and 62 detected from past fixed camera images”) in the storage unit 46. Here, the coordinate values of the past positions of the mark portions 61 and 62 stored in the storage unit 46 are the mark portion position detection operation from the hand camera image and the mark from the fixed camera image when the robot control system is calibrated. These are the coordinate values of the positions of the mark parts 61 and 62 detected by executing the part position detection operation.

The storage unit 46 that stores the coordinate values of the past positions of the mark units 61 and 62 is a rewritable nonvolatile storage medium that retains stored data even when the power is off, and is configured by, for example, an HDD, SSD, flash memory, or the like. Has been done.

The inspection execution unit 45 stores the past coordinate values of the positions of the mark portions 61 and 62 detected by the mark position detection operation from the hand camera image performed this time in the storage unit 46. The coordinate values of the positions of the mark portions 61 and 62 detected from the above are compared, and the coordinate values of the position of the mark portions 61 and 62 detected this time by the mark portion position detection operation from the fixed camera image performed this time are stored. Comparing with the coordinate values of the past positions of the mark portions 61 and 62 detected from the past fixed camera images stored in the portion 46, abnormality diagnosis of the robot control system is performed based on the comparison result, and As a result, when it is determined that there is an abnormality, the abnormality occurrence location is based on the combination of the comparison results of the coordinate values of the positions of the current mark portions 61 and 62 and the coordinate values of the positions of the past mark portions 61 and 62. Specify.

The inspection execution unit 45 displays the result of the abnormality diagnosis on a display device 47 such as a liquid crystal display or a portable terminal (not shown), or performs a process of notifying the operator who manages the robot 11 by voice. Further, when the inspection execution unit 45 identifies the abnormal place, the inspection executing unit 45 displays the abnormal place and the coping method on the display device 47 or the mobile terminal, or performs a process of notifying the operator by voice.

In this embodiment, the inspection execution unit 45 executes the abnormality diagnosis of the robot control system before the start of production, and performs the mark portion position detection operation from the hand camera image and the mark portion position from the fixed camera image performed before the production start. The coordinate values of the positions of the mark portions 61 and 62 before the production start detected by the detection operation are compared with the coordinate values of the past positions of the mark portions 61 and 62 stored in the storage unit 46, and the coordinate value of each position. Is calculated, and if any one of them exceeds the predetermined threshold value corresponding to the allowable error, it is determined that there is an abnormality.

Further, in the present embodiment, the inspection executing unit 45 executes the mark portion position detecting operation from the fixed camera image when a predetermined inspection executing condition, which will be described later, is satisfied during the production, and the inspection executing unit 45 executes the mark portion during the production. The coordinate values of the positions 61 and 62 are detected, and the coordinate values of the positions of the mark portions 61 and 62 in production are stored in the storage unit 46. The past mark portion 61 detected from the past fixed camera image, When the abnormality is diagnosed based on the comparison result during the production by comparing with the coordinate value of the position of 62, and as a result, it is determined that there is abnormality, the mark position detection operation from the hand camera image is executed. The coordinate values of the positions of the mark portions 61, 62 are detected, and the coordinate values of the position of the mark portions 61, 62 are stored in the storage unit 46. The past mark portion 61 detected from the past hand camera image. , 62, and the position of abnormality is specified based on the combination of the comparison result and the comparison result during production.

Alternatively, the inspection execution unit 45 executes the mark position detection operation from the hand camera image while the robot 11 stops the arm operation for a predetermined time or longer during production, and the fixed camera 51 causes the mark unit 61 to perform the mark position detection operation. , 62 may be performed while the arm of the robot 11 is moving to a position that does not interfere with the imaging of the robot 62, and the abnormality detection of the robot control system is performed by executing the mark position detecting operation from the fixed camera image. good.

Alternatively, when the robot 11 performs a predetermined work by capturing an image of the work 30 with the hand camera 40 during production and detecting the position of the work 30 during production, the inspection execution unit 45 uses the hand camera 40 during production. When the work 30 is imaged, at least one of the work 30 and the mark portions 61, 62 is captured in the visual field of the hand camera 40 to perform the mark portion position detection operation from the hand camera image, and the work is fixed. During the period in which the arm of the robot 11 moves to a position that does not hinder the image pickup of the mark portions 61 and 62 by the camera 51, the mark portion position detection operation is executed from the fixed camera image to perform abnormality diagnosis of the robot control system. May be performed.

Further, the abnormality management of the robot control system may be performed when the operator who manages the robot 11 operates the input device 48 (keyboard, mouse, touch panel, etc.) to perform the inspection execution start operation.

By the way, the robot 11 picks up the work 30 supplied to the work area 38 of each supply device 39 by the end effector 23, and transfers the work 30 to the tray 56 stopped in the work area 57 of the tray transfer device 55. The following three causes (1) to (3) are conceivable as causes for the work of the robot 11 to become abnormal.

[Cause (1)]
The cause (1) may be that the installation position or installation angle of the fixed camera 51 or the hand camera 40 has changed from the past (during calibration). It is considered that this change occurs due to, for example, the secular change of the camera installation member or the deviation of the installation position or installation angle when the fixed camera 51 is reinstalled due to line movement or the like.

[Cause (2)]
The cause (2) is considered to be that the position or angle of the tray 56 in the work area 38 of each supply device 39 or the work area 57 of the tray transport device 55 has changed from the past (during calibration). It is considered that this change is caused by, for example, the vibration of the robot 11, the vibration of each supply device 39 itself, the impact and the change of the installation position and the installation angle due to the movement and replacement of the supply device 39.

[Cause (3)]
The cause (3) is considered to be deterioration of the arm movement accuracy of the robot 11. It is considered that the deterioration of the arm movement accuracy is caused by, for example, deterioration of the robot 11 over time, a change in the assembly state due to vibration or shock, and a change in the coordinate system of the robot 11 due to replacement or reassembly of the end effector 23.

Therefore, in the present embodiment, the robot control unit 42 executes the abnormality diagnosis of the robot control system in the following procedure.
First, when the robot control system is calibrated, the robot control unit 42 executes the calibration mark position detection/storage program of FIG. 5, and the world coordinate values of the positions of the mark parts 61 and 62 at the time of calibration. Is detected and stored in the storage unit 46. If the mark parts 61 and 62 are not installed, the operator installs the mark parts 61 and 62, and then the robot control unit 42 executes the calibration mark part position detection/storage program of FIG.

When the calibration mark position detection/storage program of FIG. 5 is started, the robot control unit 42 first determines in step 101 that one or more mark parts 61 to be detected are within the visual field of the hand camera 40. The arms 15, 17, and 19 of the robot 11 are operated to move the hand camera 40 to a position where the hand camera 40 fits. After that, the process proceeds to step 102, the mark portion 61 is imaged by the hand camera 40, and in the next step 103, the image of the hand camera 40 is processed by the image processing unit 43 to determine the positions of the mark portions 61, 62. The coordinate conversion unit 44 detects the coordinate value of the vision coordinate system 40 and converts it into the world coordinate value [A0] which is the coordinate value of the world coordinate system. Then, in the next step 104, the world coordinate value [A0] of the positions of the mark portions 61 and 62 is stored in the storage portion 46.

After that, the process proceeds to step 105, and it is determined whether or not the detection and storage of the positions of all the mark portions 61 and 62 have been completed. If not completed, the processes of steps 101 to 104 described above are repeated. The positions of the remaining mark portions 61 and 62 are detected and stored. By repeating such processing, when the detection and storage of the positions of all the mark portions 61 and 62 are completed, the determination result of the above step 105 becomes “Yes”, the process proceeds to step 106, and all the fixed cameras 51 are used. The image of the fixed camera 51 is processed by the image processing unit 43, and the positions of all the mark portions 61 and 62 are detected by the coordinate values of the fixed camera 51's vision coordinate system in the next step 107. Then, the coordinate conversion unit 44 converts it into a world coordinate value [B0]. Then, in the next step 108, the world coordinate values [B0] of the positions of all the mark portions 61 and 62 are stored in the storage portion 46, and this program ends.

After that, the robot control unit 42 executes the abnormality diagnosis program of the robot control system by executing the robot control system abnormality diagnosis program of FIGS. 6 to 8 before and during the production.

When the robot control system abnormality diagnosis program of FIGS. 6 to 8 is started, the robot control unit 42 first waits in step 201 until the inspection execution condition before the start of production is satisfied. Here, the inspection execution condition before the start of production is, for example, that the robot 11 is operable and other work is not performed. Alternatively, the inspection execution condition before the start of production may be satisfied when the operator operates the input device 48 to perform the inspection execution start operation. Further, the abnormality diagnosis may be performed every time before the production is started, but the abnormality diagnosis may be performed each time the number of production executions or the cumulative production time reaches a predetermined value.

At this step 201, when it is determined that the inspection execution condition before the start of production is satisfied, the process proceeds to step 202, until the position where one or a plurality of mark portions 61 to be detected fall within the visual field of the hand camera 40. The arms 15, 17, and 19 of the robot 11 are operated to move the hand camera 40. After that, the process proceeds to step 203, the mark portion 61 is imaged by the hand camera 40, and in the next step 204, the image of the hand camera 40 is processed by the image processing unit 43 so that the positions of the mark portions 61 and 62 are determined by the hand camera. The coordinate conversion unit 44 detects the coordinate value of the vision coordinate system 40 and converts it into the world coordinate value [A1]. Then, in the next step 205, the world coordinate value [A1] of the positions of the mark portions 61 and 62 this time is detected from the image of the hand camera 40 at the time of calibration stored in the storage unit 46. Then, the change amount |[A0]-[A1]| of the world coordinate value of the position of each mark portion 61, 62 is calculated by comparing with the world coordinate value [A0] of the position of each mark portion 61, 62.

After that, the process proceeds to step 206, and it is determined whether or not the detection/comparison of the positions of all the mark portions 61 and 62 is completed. If not completed, the processes of steps 202 to 205 described above are repeated, The positions of the remaining mark portions 61 and 62 are detected and compared.

By repeating such processing, when the detection/comparison of the positions of all the mark portions 61 and 62 is completed, the determination result of the above step 206 becomes “Yes”, the process proceeds to step 207, and all the fixed cameras 51 are used. The image of the fixed camera 51 is processed by the image processing unit 43, and the positions of all the mark portions 61 and 62 are detected by the coordinate values of the fixed camera 51's vision coordinate system in the next step 208. Then, the coordinate conversion unit 44 converts it into the world coordinate value [B1]. After that, the process proceeds to step 209, in which the world coordinate values [B1] of the positions of all the mark parts 61 and 62 this time are detected from the image of the fixed camera 51 at the time of calibration stored in the storage part 46. The world coordinate value [B0] of the positions of the mark portions 61 and 62 at the time of the operation is calculated, and the change amount |[B0]-[B1]| of the world coordinate value of the positions of the mark portions 61 and 62 is calculated.

After that, the process proceeds to step 210 of FIG. 7 and the amount of change in the world coordinate values of the positions of the mark portions 61 and 62 calculated in step 205 and step 209 |[A0]-[A1]| and |[B0]- [B1]| is compared with a predetermined threshold value corresponding to the allowable error, respectively, and the amount of change in the world coordinate values of the positions of all the mark portions 61, 62 |[A0]-[A1]| and |[B0]- If both [B1]| are within the threshold, the routine proceeds to step 211, where it is determined that the robot control system is normal (no abnormality), and the routine proceeds to step 212 to start production.

On the other hand, in step 210, one of the change amount |[A0]-[A1]| and |[B0]-[B1]| of the world coordinate value of the position of each mark portion 61, 62 However, if it is determined that it is equal to or more than the threshold value, the process proceeds to step 217, it is determined that the robot control system is abnormal, and the process proceeds to step 218, and the comparison result between |[A0]-[A1]| , |[B0]-[B1]| and the threshold value is identified based on the combination of the result of comparison with the threshold value, and in the next step 219, an alarm warning is displayed on the display device 47 or the portable terminal. Alternatively, the process of notifying the operator of the abnormality by voice is performed, the location of the abnormality and the coping method are displayed on the display device 47 or the mobile terminal, and the process of notifying the operator by voice is performed.

Before the start of production, for example, the location of the abnormality is specified according to the following criteria (1) to (3), and the location of the abnormality and the countermeasure are notified to the operator.

(1) In the case of |[A0]-[A1]|≧threshold value and |[B0]-[B1]|≧threshold value, the position or angle of the supply device 39 or the tray 56 (tray transfer device 55) is abnormal. It is determined that the deviation amount exceeds the allowable error. As a coping method in this case, the operator is instructed to correct the positions and angles of the supply device 39 and the tray 56 (tray transfer device 55) and recalibrate the vision coordinate system of the hand camera 40 and the fixed camera 51. To do.

(2) If |[A0]-[A1]|≧threshold value and |[B0]-[B1]|<threshold value, the arm movement accuracy of the robot 11 (coordinate system in the robot 11) is abnormal (shifted). Quantity exceeds the allowable error). As a coping method in this case, the operator is notified to confirm the mounting state of the end effector 23 and recalibrate the robot 11.

(3) In the case of |[A0]-[A1]|<threshold value and |[B0]-[B1]|≧threshold value, the installation position or installation angle of the fixed camera 51 is abnormal (the deviation amount causes an allowable error). Exceeded). As a coping method in this case, the operator is notified to correct the installation position and installation angle of the fixed camera 51 and recalibrate the vision coordinate system of the fixed camera 51.

After the start of production, the process proceeds to step 213, and the abnormality diagnosis during production is executed by the processing from step 214 onward every time when the fixed camera 51 images the work 30 during production. The abnormality diagnosis may be performed every time when the fixed camera 51 picks up an image of the work 30 during production, but the abnormality diagnosis may be performed each time the number of times of image pickup reaches a predetermined number.

When it comes to the timing of imaging the work 30 (abnormality diagnosis execution timing), the process proceeds to step 214, and the work 30 and the mark portions 61 and 62 are imaged by the fixed camera 51. Then, in the next step 215, the image of the fixed camera 51 is processed by the image processing unit 43, the positions of all the mark portions 61 and 62 are detected by the coordinate values of the vision coordinate system of the fixed camera 51, and the coordinate conversion unit 44 is detected. Convert to world coordinate value [B2] with. After that, the process proceeds to step 216, and the world coordinate values [B2] of the positions of all the mark parts 61 and 62 this time are detected from the image of the fixed camera 51 at the time of calibration stored in the storage part 46. The world coordinate values [B0] of the positions of the mark portions 61 and 62 at the time of the operation are compared with each other to calculate the change amount |[B0]-[B2]| of the world coordinate values of the positions of the mark portions 61 and 62.

After that, the process proceeds to step 220 in FIG. 8 and the change amount |[B0]-[B2]| of the world coordinate value of the position of each mark portion 61, 62 is compared with a predetermined threshold value corresponding to the allowable error, and all are compared. If the amount of change |[B0]-[B2]| in the world coordinate values at the positions of the mark portions 61 and 62 is within the threshold, the process proceeds to step 221, and it is determined that the robot control system is normal (no abnormality). In this case, the process proceeds to step 222, it is determined whether the production is finished, and if it is determined that the production is not finished (in production), the process returns to step 213 in FIG. 7. As a result, during production, the abnormality diagnosis of the robot control system is repeatedly executed every time when the fixed camera 51 captures an image of the work 30 (abnormality diagnosis execution timing), and this program also ends when the production ends. ..

On the other hand, if it is determined in step 220 that any one of the amounts of change in the world coordinate values |[B0]-[B2]| at the positions of the mark portions 61 and 62 is equal to or greater than the threshold value, In step 223, it is determined that the robot control system is abnormal, and in step 224, each robot 11 is moved to a position where one or more mark portions 61 to be detected are within the visual field of the hand camera 40. The arms 15, 17, and 19 are operated to move the hand camera 40. After that, the process proceeds to step 225, the mark portion 61 is imaged by the hand camera 40, and in the next step 226, the image of the hand camera 40 is processed by the image processing unit 43 so that the positions of the mark portions 61 and 62 are determined by the hand camera. The coordinate conversion unit 44 detects the coordinate value of the vision coordinate system 40 and converts it into the world coordinate value [A2]. After that, the routine proceeds to step 227, where the world coordinate value [A2] of the positions of the mark portions 61 and 62 this time is detected from the image of the hand camera 40 at the time of calibration stored in the storage portion 46. Then, the change amount |[A0]-[A2]| of the world coordinate value at the position of each mark portion 61, 62 is calculated by comparing with the world coordinate value [A0] at the position of each mark portion 61, 62.

In the next step 228, the location of the abnormality is identified based on the combination of the comparison result of |[A0]-[A2]| and the threshold value and the comparison result of |[B0]-[B2]| and the threshold value. Then, in the next step 229, a warning of the abnormality is displayed on the display device 47 or the mobile terminal, or a process of notifying the operator of the abnormality by voice is performed, and the location of the abnormality and the coping method are displayed on the display device 47. Then, the program is displayed on a mobile terminal or is notified to the operator by voice, and the program is terminated.

During production, for example, the location of the abnormality is specified according to the following criteria (1) to (3), and the location of the abnormality and the countermeasure are notified to the operator.

(1) In the case of |[A0]-[A2]|≧threshold value and |[B0]-[B2]|≧threshold value, the position or angle of the supply device 39 or the tray 56 (tray transfer device 55) is abnormal. It is determined that the deviation amount exceeds the allowable error. As a coping method in this case, the operator is instructed to correct the positions and angles of the supply device 39 and the tray 56 (tray transfer device 55) and recalibrate the vision coordinate system of the hand camera 40 and the fixed camera 51. To do.

(2) If |[A0]-[A2]|≧threshold and |[B0]-[B2]|<threshold, the robot 11 arm movement accuracy (coordinate system in the robot 11) is abnormal (shifted). Quantity exceeds the allowable error). As a coping method in this case, the operator is notified to confirm the mounting state of the end effector 23 and recalibrate the robot 11.

(3) In the case of |[A0]-[A2]|<threshold value and |[B0]-[B2]|≧threshold value, the installation position or installation angle of the fixed camera 51 is abnormal (the deviation amount causes an allowable error). Exceeded). As a coping method in this case, the operator is notified to correct the installation position and installation angle of the fixed camera 51 and recalibrate the vision coordinate system of the fixed camera 51.

According to the present embodiment described above, work is performed with both the hand camera 40 attached to the wrist portion 21 of the robot 11 and the fixed camera 51 fixed downward at a position higher than the arm movable area of the robot 11. The mark portions 61 and 62 provided in a predetermined positional relationship with the areas 38 and 57 are imaged, and the positions of the same mark portions 61 and 62 are detected in both the image of the hand camera 40 and the image of the fixed camera 51. Therefore, the abnormality diagnosis can be performed without using the calibration stage or the master work, which has been a factor of erroneous determination in Patent Document 1. Moreover, since the same mark portions 61 and 62 are imaged by both the hand camera 40 and the fixed camera 51, it is possible to determine whether the positional deviation of the cameras 40 and 51 or the positional deviation of the mark portions 61 and 62, and the abnormality diagnosis. The accuracy of can be improved.

Moreover, in this embodiment, the abnormality control of the robot control system is performed before the production is started, and the abnormality control of the robot control system is performed every time the fixed camera 51 captures the image of the work 30 during the production. Before the robot 11 reaches a state where it cannot operate normally, it is possible to give an abnormality warning to the operator and notify the location of the abnormality and an appropriate coping method. As a result, it is possible to prevent unplanned stoppages and failures due to the coordinate system abnormality of the robot 11 in advance, reduce the time and cost of maintenance at the location of the abnormality, and realize predictive maintenance of the robot control system. can do.

Further, even during production, since the abnormality diagnosis of the robot control system is performed every time the fixed camera 51 captures an image of the work 30, even if an abnormality occurs during production, the abnormality can be immediately detected. The influence on the production tact can be minimized.

Moreover, since there is no additional cost and man-hours due to additional equipment or sensors dedicated to abnormality detection, introduction is easy. Furthermore, since the robot 11 can be operated by confirming the normal state of the coordinate system of the robot 11 by the abnormality diagnosis, it is possible to improve the position detection accuracy of the work 30.

In the present embodiment, the mark portions 61, 62 have a circular shape, but may have other shapes such as a quadrangle. In short, the positions of the centers of the mark portions 61, 62 may be changed by image processing. Any shape can be used as long as it can be accurately recognized.

In addition, when the abnormality of the robot control system is diagnosed only before the production is started, or when a work error of the robot 11 occurs during the production (or when the number or the rate of occurrence of the work error exceeds a predetermined value). You may make it perform the abnormality diagnosis of a robot control system.

In addition, it goes without saying that the present invention can be variously modified and implemented without departing from the scope of the invention, for example, the configuration of the robot 11 may be modified appropriately.

11... Robot, 14... 1st joint axis, 15... 1st arm, 16... 2nd joint axis, 17... 2nd arm, 18... 3rd joint axis, 19... 3rd arm, 20... 4th joint axis, 21... Wrist part (arm tip part), 22... 5th joint axis, 23... End effector, 25-29... Servo motor, 31-35... Encoder, 36... Servo amplifier, 37... Control part, 38... Work area, 39... Supply device, 40... Hand camera, 41... Lens, 42... Robot control unit, 43... Image processing unit (first image processing unit, second image processing unit), 44... Coordinate conversion unit, 45... Inspection execution unit , 46... Storage section, 50... Fixed structure, 51... Fixed camera, 52... Lens, 55... Tray transport device, 56... Tray, 57... Work area, 61, 62... Mark section

Claims (14)

A supply device for supplying the work to the work area,
A robot that performs a predetermined work on the work supplied to the work area,
A hand camera attached to the robot arm,
A fixed camera that is fixed downward at a position higher than the arm movable region of the robot and that images the work supplied to the work area from above,
A first image processing unit that processes an image captured by the hand camera and detects a position of an image capturing target with a coordinate value of a coordinate system having a reference point of the image as an origin (hereinafter referred to as a “vision coordinate system of the hand camera”). When,
A second image processing unit that processes an image captured by the fixed camera and detects a position of an image capturing target by a coordinate value of a coordinate system whose origin is a reference point of the image (hereinafter, referred to as “fixed camera vision coordinate system”). When,
A coordinate conversion unit that converts the coordinate values of the vision coordinate system of the hand camera and the coordinate values of the vision coordinate system of the fixed camera into coordinate values of the world coordinate system that is a coordinate system that controls the operation of the arm of the robot;
A robot control system including a control unit that controls the position of the arm of the robot by coordinate values of the world coordinate system,
A mark portion provided at a position that has a predetermined positional relationship with the work area within a range that falls within the field of view of the fixed camera;
The hand camera is moved to a position where the mark portion fits within the field of view of the hand camera, the mark portion is imaged by the hand camera, and the image is processed by the first image processing portion so that the mark portion An operation of detecting a position with a coordinate value of the vision coordinate system of the hand camera and converting the coordinate value into a coordinate value of the world coordinate system with the coordinate conversion unit (hereinafter referred to as "mark portion position detection operation from a hand camera image"); The fixed camera takes an image of the mark portion, the second image processing unit processes the image, the position of the mark portion is detected by the coordinate value of the vision coordinate system of the fixed camera, and the coordinate conversion unit detects the position. An inspection execution unit that executes an operation of converting to coordinate values in the world coordinate system (hereinafter referred to as "mark portion position detection operation from fixed camera image");
The coordinate value of the past position of the mark portion detected by the mark portion position detection operation from the hand camera image performed by the inspection execution unit in the past (hereinafter, “the past mark portion detected from the past hand camera image "Coordinate value of position"), and the coordinate value of the past position of the mark portion detected by the mark position detecting operation from the fixed camera image performed by the inspection execution unit in the past (hereinafter referred to as "past coordinate value"). Storage unit for storing "the coordinate values of the past position of the mark portion detected from the fixed camera image").
The inspection execution unit stores, from the past hand camera image, the coordinate value of the current position of the mark portion detected by the mark portion position detection operation from the hand camera image performed this time, which is stored in the storage unit. The coordinate value of the detected position of the mark portion is compared, and the coordinate value of the current position of the mark portion detected by the mark position detection operation from the fixed camera image performed this time is stored in the storage unit. The robot control system compares the past coordinate value of the position of the mark portion detected from the past fixed camera image, and performs abnormality diagnosis based on the comparison result. When it is determined that there is an abnormality, the inspection execution unit determines the location of the abnormality based on the combination of the comparison results of the coordinate value of the position of the mark portion this time and the coordinate value of the position of the mark portion in the past. The robot control system according to claim 1, which is specified. The robot control system according to claim 1, wherein the inspection execution unit performs a process of displaying a result of the abnormality diagnosis and/or notifying a worker by voice. The robot control system according to claim 2, wherein, when the inspection execution unit specifies the abnormality occurrence place, the inspection execution unit performs a process of displaying the abnormality occurrence place and a coping method to the operator by displaying and/or voice. The inspection execution unit is a coordinate of the position of the mark portion before the production start detected by the mark portion position detection operation from the hand camera image and the mark portion position detection operation from the fixed camera image performed before the production start. The amount of change in the coordinate value at each position is calculated by comparing the value with the coordinate value of the past position of the mark portion stored in the storage unit, and any one of them allows the amount of change in the coordinate value. The robot control system according to claim 1, wherein an abnormality is determined if a predetermined threshold value corresponding to an error is exceeded. The inspection execution unit executes a mark portion position detection operation from the fixed camera image during production to detect the coordinate value of the position of the mark portion during production, and detects the position of the mark portion during production. The coordinate value is stored in the storage unit, compared with the coordinate value of the past position of the mark portion detected from the past fixed camera image, and the abnormality is diagnosed based on the comparison result during the production. As a result, when it is determined that there is an abnormality, the mark position detection operation from the hand camera image is executed to detect the coordinate value of the position of the mark portion, and the coordinate value of the position of the mark portion is stored in the storage unit. The stored position is compared with the stored coordinate value of the position of the mark portion in the past detected from the hand camera image in the past, and the abnormal place is specified based on the combination of the comparison result and the comparison result during the production. The robot control system according to any one of claims 1 to 5. The inspection execution unit executes a mark position detection operation from the hand camera image during a period in which the robot has stopped the arm operation for a predetermined time or longer during production, and images the mark portion with the fixed camera. 6. The robot control according to claim 1, wherein an abnormality diagnosis is performed by executing a mark position detection operation from the fixed camera image while the arm of the robot is moving to a position that does not disturb the robot. system. When the inspection execution unit captures an image of a work with the hand camera during production, detects the position of the work, and the robot performs a predetermined work, the inspection execution unit captures an image of the work with the hand camera during production. In addition, the work and the mark portion are captured in the field of view to perform the mark portion position detection operation from the hand camera image, and the fixed camera is moved to a position that does not interfere with the image pickup of the mark portion. The robot control system according to any one of claims 1 to 5, wherein an abnormality diagnosis is performed by performing a mark position detection operation from the fixed camera image while the robot arm is moving. The mark portion is provided at a plurality of locations,
The inspection execution unit detects the coordinate value of the position of the mark portion for each mark portion by the mark portion position detection operation from the hand camera image and the mark portion position detection operation from the fixed camera image, The coordinate value of the position of the mark portion is compared for each time with the coordinate value of the position of the mark portion in the past stored in the storage unit for each mark portion. Robot control system. The robot picks up the work supplied to the work area and transfers it to another work area,
Also for the other work area, a mark portion is provided at a position that has a predetermined positional relationship with the other work area within a range that falls within the field of view of the fixed camera,
The robot control system according to any one of claims 1 to 9, wherein the inspection execution unit captures an image by accommodating the mark portions of both the work area and the another work area in a field of view of the fixed camera. A plurality of the supply devices are arranged side by side and a plurality of the work areas are arranged side by side,
The robot control system according to claim 1, wherein the mark portion is provided for each work area. 9. The inspection execution unit, when it is not possible to fit all the mark portions in the visual field of the hand camera at the same time, performs the image pickup of the mark portion by the hand camera in two or more times. 11. The robot control system according to any one of 11. The coordinate value of the position of the mark portion in the past is detected by executing a mark portion position detecting operation from the hand camera image and a mark portion position detecting operation from the fixed camera image when the robot control system is calibrated. The robot control system according to claim 1, wherein the robot control system is a coordinate value of the position of the marked portion. When the second image processing unit processes the image captured by the fixed camera during production to detect the position of the work included in the image, the second image processing unit determines the position of the mark portion closest to the detection target work. The robot control system according to claim 1, wherein the position of the workpiece to be detected is detected as a reference point by a relative position from the reference point.

Images