JP2021012203A - Parallelism determination method of camera - Google Patents

Abstract

To more easily determine parallelism for a reference surface of a multi-joint robot.SOLUTION: A parallelism determination method of a multi-joint robot determines parallelism for a reference surface of a multi-joint robot 20 and comprises: arranging a plate P that has a design of a predetermined pattern on a working table 11 of the multi-joint robot; imaging a plate by means of a camera 24 mounted on an arm of the multi-joint robot; measuring a gap of the design which is recognized in an imaged image of the plate at a plurality of places; and determining the parallelism on the basis of contrast of each measured gap.SELECTED DRAWING: Figure 5

Description

The present specification discloses a method for determining parallelism of an articulated robot and an inclination adjusting device for the articulated robot.

Conventionally, those for determining the parallelism of a work with respect to a reference plane have been known. For example, in Patent Document 1, a wafer positioned on an alignment plate (magnetic plate) is photographed by a camera mounted on a robot arm, and the parallelism of the wafer with respect to the magnetic plate is calculated based on the photographed image of the wafer. Those provided with a control device are disclosed. Specifically, the parallelism is calculated by the control device as follows. That is, the control device images the two alignment marks attached to the wafer by the camera. Subsequently, the control device calculates the defocus at each alignment mark based on the captured image of each alignment mark, and calculates the parallelism of the wafer with respect to the magnetic plate from the defocus. Then, the control device controls the actuators of each joint of the robot arm so that the wafer is parallel to the magnetic plate based on the calculated parallelism of the wafer, and tilts the wafer.

Japanese Unexamined Patent Publication No. 2014-53343

The method described in Patent Document 1 detects the defocus from the captured image of the mark, but it requires a dedicated detection device or the processing becomes complicated in order to secure sufficient detection accuracy. Problems arise. These problems can also occur when determining the parallelism of an articulated robot with respect to a reference plane.

An object of the present disclosure is to more easily determine the parallelism of an articulated robot with respect to a reference plane.

The present disclosure has taken the following steps to achieve the above-mentioned main objectives.

The present disclosure is a method for determining parallelism of an articulated robot that determines parallelism with respect to a reference plane of the articulated robot. A plate having a predetermined pattern is placed on a work table of the articulated robot, and the articulated robot is articulated. The plate is imaged by a camera mounted on the arm of the robot, the intervals of patterns recognized in the captured image of the plates are measured at a plurality of points, and the parallelism is determined based on the comparison of the measured intervals. The gist is to do.

In the captured image of the pattern held by the plate, when the camera (arm) and the plate are relatively tilted, there are areas where the pattern appears larger than the actual pattern and areas where the pattern appears smaller than the actual pattern. The greater the tilt between the camera and the plate, the greater the difference in the size of the patterns captured in each area. The parallelism determination method of the articulated robot of the present disclosure measures the intervals of patterns in a captured image at a plurality of points and determines the parallelism by comparing the measured intervals, so that the articulated robot can be more easily determined. The parallelism with respect to the reference plane can be determined.

It is a block diagram which shows the outline of the structure of the robot 20. It is explanatory drawing which shows the range of motion of a robot 20. It is explanatory drawing which shows the range of motion of a robot 20. It is a block diagram which shows the electrical connection relationship of a robot 20, a robot control device 70, and an image processing device 80. It is explanatory drawing which shows an example of the inclination adjustment process. It is explanatory drawing which shows the state of the imaging of the jig plate P. It is explanatory drawing which shows the upper surface pattern of a jig plate P. It is explanatory drawing which shows the upper surface pattern of a jig plate P. It is explanatory drawing which shows the state of the imaging of the jig plate P when the camera 24 is parallel to the jig plate P. It is explanatory drawing which shows the captured image of the jig plate P image | photographed from the position of FIG. 7A. It is explanatory drawing which shows the state of the imaging of the jig plate P when the camera 24 is not parallel to the jig plate P. It is explanatory drawing which shows the captured image of the jig plate P image | photographed from the position of FIG. 8A. It is explanatory drawing which shows the size H, the focal length F, the field of view FOV, and the working distance WD of an image sensor. It is explanatory drawing which shows the relationship between working distance WD and resolution RES. It is explanatory drawing which shows the working distance WD1 at the left end (A end) of the field of view, and the working distance WD2 at the right end (B end) of a field of view when the camera 24 is tilted by a predetermined angle θ with respect to an object. It is explanatory drawing which shows the dot center distance La1, La2 of A end and the dot center distance Lb1, Lb2 of B end.

Next, a mode for carrying out the invention of the present disclosure will be described with reference to the drawings.

FIG. 1 is a configuration diagram showing an outline of the configuration of the robot 20. 2A and 2B are explanatory views showing the range of motion of the robot 20. FIG. 3 is a block diagram showing an electrical connection relationship between the robot 20, the robot control device 70, and the image processing device 80.

The robot 20 is controlled by the robot control device 70 (see FIG. 3) to perform a predetermined work on a work (work object) transported by the work transfer device 12 (see FIG. 3). In addition, as a predetermined work, a pick-up work for picking up a work, a place work for placing a work at a predetermined position, an assembling work for assembling the work at a predetermined position, and the like can be mentioned.

As shown in FIG. 1, the robot 20 includes a 5-axis vertical articulated arm (hereinafter referred to as an arm) 22. The arm 22 has six links (first to sixth links 31 to 36) and five joints (first to fifth joints 41 to 45) that rotatably or swivelly connect each link. Each joint (1st to 5th joints 41 to 45) includes a motor (servo motor) 51 to 55 for driving the corresponding joint and an encoder (rotary encoder) 61 to 65 for detecting the rotation position of the corresponding motor. Is provided.

A work tool as an end effector can be attached to and detached from the tip link (sixth link 36) of the arm 22. Examples of the work tool include an electromagnetic chuck, a mechanical chuck, and a suction nozzle. The tool to be attached to the tip link is appropriately selected according to the shape and material of the work to be worked.

A camera 24 is attached to the tip of the arm 22 (fifth link 35). The camera 24 is for taking an image of the work in order to recognize the position and orientation of the work.

As shown in FIG. 2A, the arm 22 of the present embodiment configured in this way has the X-axis in the front (front) and rear (back) directions when the robot 20 is viewed from the front, and the vertical direction (first joint) of the robot 20. It is possible to move in a three-dimensional space with the Z axis as the extension direction of the rotation axis of 41) and the Y axis as the direction orthogonal to the X axis and the Z axis, and to move in the rotation direction (RC) around the Z axis. Is. Further, as shown in FIG. 2B, the arm 22 can move in the rotation direction (RB) around the Y axis.

As shown in FIG. 3, the robot control device 70 is configured as a microprocessor centered on a CPU 71, and includes a ROM 72, an HDD 73, a RAM 74, an input / output interface (not shown), a communication interface (not shown), and the like in addition to the CPU 71. The HDD 73 stores an operation program of the robot 20 and the like. Detection signals from encoders 61 to 65 and the like are input to the robot control device 70. Further, the robot control device 70 outputs control signals to the motors 51 to 55, the work transfer device 12, and the like.

The robot control device 70 drives and controls the motors 51 to 55 of the robot 20 to move the work roots attached to the tip link (sixth link 36) of the arm 22 toward the work, and uses the work tool. Perform a predetermined work on the work. Specifically, the robot control device 70 acquires the target position (X, Y, Z) and the target posture (RB, RC) of the work tool for performing the work on the work from the image processing device 80. Subsequently, the robot control device 70 sets the acquired target position (X, Y, Z) and target posture (RB, RC) to the target position (target angle) of each joint of the arm 22 by using well-known HD parameters and the like. Coordinate conversion. Then, the robot control device 70 drives and controls the corresponding motors 51 to 55 so that the position (angle) of each joint matches the coordinate-converted target position (target angle), and the work is performed on the work. Drive and control work tools.

Further, the image processing device 80 is communicably connected to the robot control device 70. The image processing device 80 is configured as a microprocessor centered on a CPU 81, and includes a ROM 82, an HDD 83, a RAM 84, an input / output interface (not shown), a communication interface (not shown), and the like, in addition to the CPU 81. An image signal from the camera 24, an input signal from the input device 85, and the like are input to the image processing device 80. Further, the image processing device 80 outputs a drive signal to the camera 24, an output signal to the output device 86, and the like. Here, the input device 85 is an input device such as a keyboard or a mouse in which an operator performs an input operation. The output device 86 is a display device for displaying various information such as a liquid crystal display. The image processing device 80 is communicably connected to the robot control device 70, and exchanges control signals and data with each other.

Next, an inclination adjusting step for adjusting the inclination of the robot 20 configured in this way with respect to the reference plane will be described. FIG. 4 is an explanatory diagram showing an example of the inclination adjusting process. In the tilt adjusting step, after the operator installs the jig plate P on the workbench 11 of the robot 20 as shown in FIG. 5 (S100), the CPU 81 of the image processing device 80 executes the following processes S110 to S160. It is done by. 6A and 6B are explanatory views showing an upper surface pattern of the jig plate P. As shown in FIG. 6A, a pattern in which circular dots (marks) are arranged in a matrix is formed on the upper surface of the jig plate P. As shown in FIG. 6B, a checkered pattern in which squares of two colors (white and black) are alternately arranged may be formed on the upper surface of the jig plate P.

The CPU 81 of the image processing device 80 images the jig plate P with the camera 24 mounted on the arm 22 of the robot 20 (S110). This process is performed by transmitting a control signal including the imaging position of the camera 24 corresponding to the installation position of the jig plate P to the robot control device 70. When the robot control device 70 receives the control signal from the image processing device 80, the robot control device 70 drives and controls the motors 51 to 55 so that the camera 24 comes above the jig plate P as shown in FIG.

Subsequently, the CPU 81 corrects the lens aberration (distortion) included in the captured image of the jig plate P by using a well-known distortion correction algorithm (S120). In the lens distortion correction, for example, the position of each pixel of the image is associated with the amount of distortion, obtained in advance, stored in the ROM 82 as distortion amount mapping data, and the position of each pixel of the captured image is corrected by the corresponding amount of distortion. It can be done by doing. Then, the CPU 81 separates the background color (white) and the mark color (black) from the captured image after the lens distortion correction, and extracts the contours of the marks at the four corners of the jig plate P (S130). Next, the CPU 81 has a distance between the centers of the two dots at the A end (dot center distance La) on one side of the robot 20 in the X direction (front-back direction of the robot 20) and two at the B end on the other side. The distance between the center of the dots (distance between the center of the dots Lb) is measured by counting the number of pixels included between the centers of the dots (S140). Then, the CPU 81 has a parallelism θ (Y) with respect to the reference surface (upper surface of the workbench 11) of the robot 20 based on the ratio (La / Lb) of the dot center distance La at the B end to the dot center distance Lb at the A end. The inclination in the rotation direction around the axis) is derived (S150).

FIG. 7A is an explanatory diagram showing a state of imaging of the jig plate P when the camera 24 is parallel to the jig plate P. FIG. 7B is an explanatory view showing a captured image of the jig plate P captured from the position of FIG. 7A. FIG. 8A is an explanatory diagram showing a state of imaging of the jig plate P when the camera 24 is not parallel to the jig plate P. FIG. 8B is an explanatory view showing a captured image of the jig plate P captured from the position of FIG. 8A. As shown in FIG. 7A, when the camera 24 is parallel to the jig plate P, the captured image of the jig plate P shows all the marks having the same size and the same interval as shown in FIG. 7B. On the other hand, as shown in FIG. 8A, when the camera 24 is tilted about the Y axis relative to the jig plate P, the captured image of the jig plate P has a large and wide mark as shown in FIG. 8B. There are areas that appear at intervals and areas where the marks are small and appear at narrow intervals. The larger the relative inclination of the camera 24 with respect to the jig plate P, the larger the size difference and the width difference of the marks tend to be. Utilizing such a phenomenon, the CPU 81 measures the intervals between the marks reflected in the captured image of the jig plate P at a plurality of points and compares them, so that the relative inclination of the camera 24 and the jig plate P, that is, the robot The parallelism θ (slope) with respect to the reference plane of 20 can be obtained.

Specifically, in the processing of S150, the relationship between the ratio (La / Lb) of the dot center distance La at the B end to the dot center distance Lb at the A end and the parallelism θ is experimentally obtained in advance and used as a map. It is stored in the ROM 82, and when the ratio (La / Lb) is given, it is performed by deriving the corresponding parallelism θ from the map. Here, the steps S100 to S150 described above correspond to the parallelism determination step of determining the parallelism (inclination) of the robot 20 with respect to the reference plane.

When the CPU 81 derives the parallelism θ with respect to the reference plane of the robot 20 in this way, the inclination correction value of the robot 20 is set based on the derived parallelism θ (S160), and the inclination adjustment step is completed. The processing of S160 is performed by setting the offset angle ΔRB with respect to the target posture (RB) of the work tool so as to be offset in the opposite direction by the inclination in the rotation direction around the Y axis. As a result, the robot control device 70 controls the robot 20 based on the target posture (RB) of the work tool offset by the offset angle ΔRB.

Here, the correspondence between the main elements of the embodiment and the main elements of the invention described in the column of disclosure of the invention will be described. That is, the camera 24 corresponds to the camera, and the CPU 81 of the image processing device 80 corresponds to the control device.

In the tilt correction device of the articulated robot described above, the CPU 81 of the image processing device 80 uses the camera 24 to image the jig plate P having a pattern (matrix-like mark) formed on the upper surface (imaging surface). Subsequently, the CPU 81 measures the dot center distance La of the two non-diagonal marks among the marks at the four corners in the captured image of the jig plate P and the dot center distance Lb of the remaining two marks. Then, the CPU 81 derives the parallelism θ of the robot 20 based on the measured ratio (La / Lb) of the distances between the dots. This makes it possible to more easily determine the parallelism of the articulated robot with respect to the reference plane without using a dial gauge or the like.

Further, the CPU 81 applies lens distortion correction to the image captured by the jig plate P, and measures the dot center distances La and Lb based on the image captured after the lens distortion correction. As a result, the CPU 81 can measure the dot center distances La and Lb more accurately, and can more appropriately determine the parallelism.

Further, the CPU 81 sets the inclination correction value of the robot 20 based on the derived parallelism θ. As a result, the inclination of the robot 20 with respect to the reference plane can be adjusted by a simpler method.

In the present embodiment, the CPU 81 derives the parallelism θ with respect to the reference plane of the robot 20 by a map based on the ratio (La / Lb) of the dot center distance La at the B end to the dot center distance Lb at the A end. However, the CPU 81 may calculate the parallelism θ by calculation using a predetermined calculation formula. FIG. 9 is an explanatory diagram showing the size H, the focal length F, the field of view FOV, and the working distance WD of the image sensor. In a camera 24 having a lens 24a and an image sensor 24b (CCD, etc.), the size of the image sensor 24b is H [mm], the focal length which is the distance from the image sensor 24b to the lens 24a is F [mm], and working. The distance is WD [mm], and the field of view, which is the imaging range in the working distance range, is FOV [mm]. In this case, as can be seen from FIG. 9, since the following equation (1) holds by H: F = FOV: WD, the visual field FOV is calculated by the following equation (2). On the other hand, the resolution RES indicates the size per pixel in the captured image, and is calculated by the following equation (3) using the visual field FOV in the X direction and the number of pixels PN in the X direction of the image sensor 24b. Will be done. Substituting the visual field FOV equation (2) into the equation (3), the resolution RES is calculated by the following equation (4). It can be seen from Eq. (4) that the resolution RES is proportional to the working distance WD. Α in the equation (4) is the resolution per unit length of the working distance (also called unit resolution), and depends on the specifications of the image sensor 24b (number of pixels, sensor size) and the individual lens 24a (focal length, magnification). It becomes a fixed constant.

F * FOV = H * WD… (1)
FOV = (H / F) * WD… (2)
RES = FOV / PN… (3)
RES = (H / (F * PN)) * WD
= α * WD However, α = H / (F * PN)… (4)

Here, as shown in FIG. 9, when WD1 shorter than the reference working distance WD and WD2 longer than the reference working distance WD are set, the relationship between the working distance and the resolution is such that the unit resolution α is the slope as shown in FIG. It can be represented by a straight line. The unit resolution α is calculated by the following equation (5).

α = RES / WD
= ΔRES1 / ΔWD1
= ΔRES1 / ΔWD1
= (RES2-RES1) / (WD2-WD1)… (5)

FIG. 11 is an explanatory diagram showing a working distance WD1 at the left end (A end) of the field of view and a working distance WD2 at the right end (B end) of the field of view when the camera 24 is tilted by a predetermined angle θ with respect to the object. .. FIG. 12 is an explanatory diagram showing the dot center distances La1 and La2 at the A end and the dot center distances Lb1 and Lb2 at the B end. When the jig plate P of FIG. 6A in which dots (marks) are arranged in a matrix is imaged by the camera 24, the CPU 81 arms the CPU 81 so that the center of the field of view of the camera 24 coincides with the central dot DC (see FIG. 12). 22 is moved to perform imaging. In this case, as shown in FIG. 11, when the camera 24 is tilted by a predetermined angle θ with respect to the upper surface of the jig plate P, the working distance WD1 at the left end (A end) of the field of view is closer than the original working distance WD. Therefore, the working distance WD2 at the right end (B end) of the field of view is farther than the original working distance WD. Therefore, as shown in FIG. 12, the captured image of the camera 24 appears large at the A end on the short distance side and small at the B end on the long distance side. At this time, the inclination θ of the camera 24 is calculated by the following equation (6) from the difference between WD1 and WD2 (ΔWD = WD2-WD1) and the distance L between the A end and the B end of the jig plate P. To. Since the distance L is known, the inclination θ of the camera 24 can be calculated by detecting ΔWD.

sin θ = ΔWD / L
θ = sin ^-1 (ΔWD / L)… (6)

Equation (5) can be modified as in equation (7) below. On the other hand, the resolution RES1 at the A end and the resolution RES2 at the B end can be calculated by processing the captured image of FIG. That is, the resolution RES1 at the A end is the average value of the dot center distances La1 and La2 [pix] of two adjacent dots including the dot at the center of the end side at the A end in FIG. 12 shown in the captured image, and the jig plate P. It is calculated by the following equation (8) using the inter-center distance DP [mm] of the specifications of the two adjacent dots. The distances between dot centers La1 and La2 [pix] can be measured by counting the number of pixels included between the centers of adjacent dots in the captured image, respectively. On the other hand, the resolution RES2 at the B end is similarly the average value of the dot center distances Lb1 and Lb2 [pix] of two adjacent dots including the dot at the center of the end side at the B end in FIG. 12 and the above specifications. It is calculated by the following equation (9) using the center-to-center distance DP [mm]. As described above, the resolutions RES1 and RES2 are included in the specified distance between any two dots of the dot group at each end edge of the A end and the B end, and between the two dots in the captured image. It can be calculated using the number of pixels.

WD2-WD1 = ΔWD
= (RES2-RES1) / α… (7)
RES1 = DP / Average (La1, La2)
= 2 * DP / (La1 + La2)… (8)
RES2 = DP / Average (Lb1, Lb2)
= 2 * DP / (Lb1 + Lb2)… (9)

By substituting the equations (7) to (9) into the equation (6), the inclination θ of the camera 24, that is, the parallelism θ with respect to the reference plane of the robot 20 can be calculated.

In the present embodiment, it is assumed that a matrix-like dot pattern or a checkered pattern (pattern) is formed on the upper surface (imaging surface) of the jig plate P. However, the pattern formed on the jig plate P may be any pattern as long as it has at least four marks arranged in a square or rectangular shape. Further, the pattern formed on the jig plate P is the parallelism of the camera 24 with respect to the jig plate P, that is, the reference of the robot 20 by measuring the intervals of the patterns reflected in the image captured by the jig plate P by the camera 24 at a plurality of points. Anything that can determine the parallelism with respect to the surface is sufficient.

In the present embodiment, the robot 20 is provided with a 5-axis vertical articulated arm 22. However, the articulated arm is not limited to five axes, and may have, for example, six or more axes. For example, a 6-axis articulated arm can move in three-dimensional space of X (front-back direction of robot), Y (horizontal direction), Z (vertical direction) and rotation direction (RA) around the X-axis. , The movement in the rotation direction (RB) around the Y axis and the movement in the rotation direction (RC) around the Z axis are possible. In this case, the CPU of the image processing device determines the distance between the dot centers of the two marks on one side in the Y direction and the two marks on the other side among the marks at the four corners included in the image captured by the jig plate by the camera. Measure the distance between the dots. As a result, the CPU of the image processing device can determine the inclination (parallelism) with respect to the reference plane of the robot in the rotation direction around the X axis based on the comparison of the measured distances between the centers of the dots.

In the present embodiment, the invention of the present disclosure has been described as a form of the tilt adjusting device, but it may be a form of a parallelism determination method.

It is needless to say that the present invention is not limited to the above-described embodiments, and can be implemented in various embodiments as long as it belongs to the technical scope of the invention of the present disclosure.

The present disclosure can be used in the robot manufacturing industry and the like.

11 workbench, 20 robot, 22 arm, 24 camera, 31 1st link, 32 2nd link, 33 3rd link, 34 4th link, 35 5th link, 36 6th link, 41 1st joint, 42nd 2 joints, 43 3rd joint, 44 4th joint, 45 5th joint, 51-55 motor, 61-65 encoder, 70 robot control device, 71 CPU, 72 ROM, 73 HDD, 74 RAM, 80 image processing device, 81 CPU, 82 ROM, 83 HDD, 84 RAM, 85 input device, 86 output device, P jig plate.

This specification discloses with parallel determination how the camera.

Claims (5)

This is a method for determining the parallelism of an articulated robot, which determines the parallelism of the articulated robot with respect to the reference plane.
A plate having a predetermined pattern is placed on the workbench of the articulated robot.
The plate was imaged by a camera mounted on the arm of the articulated robot.
The intervals of patterns recognized in the captured image of the plate are measured at a plurality of points, and the parallelism is determined based on the comparison of the measured intervals.
A method for determining parallelism of an articulated robot. The method for determining parallelism of an articulated robot according to claim 1.
Lens distortion correction is performed on the captured image, and the intervals of patterns recognized in the captured image after lens distortion correction are measured at a plurality of points.
A method for determining parallelism of an articulated robot. The method for determining parallelism of an articulated robot according to claim 1 or 2.
The pattern of the predetermined pattern is a pattern having at least four identification information arranged in a square shape or a rectangular shape.
The parallelism is determined based on the comparison between the distance between two non-diagonal identification information among the four identification information and the distance between the other two identification information.
A method for determining parallelism of an articulated robot. The method for determining parallelism of an articulated robot according to claim 3.
The pattern of the predetermined pattern is a pattern in which the identification information is arranged in a matrix.
A method for determining parallelism of an articulated robot. An articulated robot tilt adjustment device that adjusts the tilt of an articulated robot with respect to a reference plane.
A camera mounted on the arm of the articulated robot and
A plate having a pattern of a predetermined pattern installed on the work table of the articulated robot is imaged by the camera, the interval of the pattern recognized in the captured image of the plate is measured at a plurality of places, and each of the measured intervals is measured. A control device that determines the parallelism of the articulated robot with respect to the reference plane based on the comparison of the above, and sets the tilt correction value of the articulated robot based on the determined parallelism.
An articulated robot tilt adjustment device equipped with.

Images