TWI832770B - Calibration method and system for mechanical arm based on image processing - Google Patents

Abstract

A method for calibrating a mechanical arm, used for calibrating a mechanical arm, comprising: setting a visit sequence of at least one calibration point; moving to at least three shooting positions, and shooting at least one image respectively at least three shooting positions by a photographing device, at least one image has a center point respectively, wherein the step of generating the at least one image respectively at the at least three shooting positions by the photographing device comprises: generating at least one image after identifying that a shooting range of the photography device covers at least one calibration point; and calculating vector information of the at least one calibration point from the at least one image to adjust a mechanical arm.

Description

Robotic arm correction method and system based on image processing

The present disclosure relates to a processing technology, and in particular to a robotic arm correction method and system based on image processing.

With the advancement of industrial technology, various automation devices have been widely developed and used in life and industry. Generally speaking, robotic arms are important components of automation devices. Although the stability of processing by a robotic arm is much greater than that of manual work, because the robotic arm has too many joints, it leads to the accumulation and transmission of multiple errors (for example, errors caused by the adjustment of the transmission mechanism or belt). As a result, the accuracy of robotic arms is generally low. How to correct these errors to improve the accuracy of robotic arm processing is a problem that those skilled in the art are eager to solve.

The present disclosure proposes an image processing-based robotic arm calibration method for calibrating the robotic arm, including: setting an access sequence for at least one calibration point on a calibration board; and driving the robotic arm according to the access sequence to cause a mechanical arm of the robotic arm to A photographing device on the end of the arm moves to at least three photographing positions, and the photographing device captures at least one image respectively at at least three photographing positions, and at least one image respectively has a center point, wherein the photographing device captures at least three photographing positions with The step of generating at least one image respectively includes: generating at least one image after identifying that the shooting range of the photography device covers at least one correction point; and calculating vector information of at least one correction point from the at least one image to adjust the robot arm.

This disclosure also proposes a robotic arm correction system based on image processing, including a correction plate, a robotic arm, a photography device and a controller. The calibration plate includes at least one calibration point. The photography device is installed at the end of the robot arm to take pictures of the correction plate. The controller is connected to the robot arm and is used to control the robot arm to perform the following operations: set the access sequence of at least one calibration point on the calibration plate; and drive the robot arm according to the access sequence to make the robot arm on one end of the robot arm A photographing device moves to at least three photographing positions, and the photographing device captures at least one image respectively at at least three photographing positions. At least one image has a center point, and the photographing device produces at least one image at at least three photographing positions respectively. The image step includes: generating at least one image after identifying that the shooting range of the photography device covers at least one correction point; and calculating vector information of at least one correction point from the at least one image to adjust the robot arm.

During the operation of the robot arm, the origin of the end of the robot arm may be offset due to various unexpected conditions (for example, offset caused by assembly or manufacturing, offset caused by power outage or external impact, transmission mechanism or belt in the offset caused during adjustment, etc.). In view of this, this disclosure proposes a non-contact method and system for robotic arm correction based on image processing, which uses the correction points on the correction plate to be photographed to calculate the distance between the correction points, and uses these distances to calculate the robot arm. The respective length offsets of the multiple connecting rods and the respective axis angle offsets of the multiple reference points of the robotic arm are used to correct the offset of the origin of the end of the robotic arm.

FIG. 1 is a schematic diagram of a robot arm correction system 1 based on image processing according to some embodiments of the disclosure. As shown in FIG. 1 , the robot arm correction system 1 based on image processing includes a robot arm 11 , a photography device 12 , a correction plate 13 and a controller 14 .

Furthermore, the imaging device 12 is provided at the position of the robot arm end 111 (that is, the flange) of the robot arm 11 . The robotic arm 11 drives the photographing device 12 to move to at least three photographing positions, and the photographing device 12 photographs the correction plate 13 at at least three photographing positions to generate at least one image respectively. In other words, at each shooting position, the photography device 12 will capture an image of the correction plate 13 . Furthermore, the photography device 12 mainly obtains images in the two-dimensional space of the actual space, and performs judgment and analysis based on the images. In other words, the entire field of view of the photography device 12 may be a two-dimensional space.

In some embodiments, the correspondence between the pixel distance (ie, the distance between pixels) in the image captured by the photography device 12 and the distance in the actual space can be pre-stored in the controller 14 or other remote storage device (For example, remote database or cloud drive, etc.). In other words, the absolute distance in the real space (ie, the real distance in the real field) can be calculated directly from the distance in the image captured by the photography device 12 according to the above-mentioned corresponding relationship. In some embodiments, the pixel position of the center point of the image (ie, the coordinates in the X and Y directions in the image) will be adjacent to the pixel position of one of the correction points in the image. In some embodiments, the photography device 12 can be implemented by various image capture devices, such as cameras or photosensitive devices, etc., for capturing images of the correction plate 13 within a preset shooting range (field of view, FoV). .

Furthermore, the robot arm 11 includes a plurality of links and a plurality of reference points between the links. In some embodiments, the robotic arm 11 may be, for example, but not limited to, a six-axis robotic arm or a four-axis SCARA robotic arm. In some embodiments, the multiple reference points may include multiple joints on the robotic arm 11 and the robotic arm end 111 . In some embodiments, the lengths of the multiple links between the multiple joints are also preset and stored in the controller 14 or other remote storage device. In some embodiments, each of the plurality of joints may have a sensor (not shown) at an axis angle through which each of the plurality of joints rotates. In some embodiments, when the robotic arm 11 is a six-axis robotic arm, each of the plurality of joints can rotate in three axes (X, Y, Z directions in the real field) as the center. In some embodiments, when the robot arm 11 is a four-axis SCARA robot arm, each of the plurality of joints can rotate centered in a vertical single-axis direction (Z direction in the real field).

Furthermore, the calibration plate 13 has at least one calibration point p~r. In some embodiments, the correction points p~r are respectively in the shooting range of the photographing device 12 at at least three shooting positions. In some embodiments, the photographing device 12 can be preset to photograph the photographing positions of the correction points p~r, and obtain the absolute coordinates of these photographing positions in the actual space (ie, the cartesian coordinates in the real field). coordinate)) are also stored in the controller 14 or other remote storage devices. In some embodiments, the normal vector N of the entire plane of the correction plate 13 is parallel to the Z direction in the real field. It is worth noting that the shooting range of the photography device 12 at at least three shooting positions only needs to cover the correction points p~r, and there is no need to align the center point of the shooting range with the correction points p~r. This approach will reduce errors generated by the robot arm 11 during fine-tuning (for example, errors generated during the adjustment of the transmission mechanism or belt) and save time spent during fine-tuning. In addition, although three correction points p~r are taken as an example here, in actual applications, the number of correction points can also be any other positive integer greater than 3.

It is worth noting that the transformation matrix between the robot arm coordinate system used by the base of the robot arm 11 and the absolute coordinate system of the actual space (i.e., homogeneous transformation matrix) is also preset and stored in controller 14 or other remote storage device. In some embodiments, the correction plate 13 may be any checkerboard or grid-type correction plate used for calibrating the robot arm 11 . In some embodiments, the correction points p~r may be marks with various different patterns on the correction plate 13 (for example, marks with patterns of triangles, stars, or circular centers). In some embodiments, the absolute coordinates of the calibration points p~r in the actual space can also be stored in the controller 14 or other remote storage devices in advance.

Furthermore, the controller 14 is connected to the robotic arm 11 and controls to drive the robotic arm 11 to move the photography device 12 . In some embodiments, the controller 14 can drive the robotic arm 11 to move the photography device 12 to the shooting position according to the respective lengths of the multiple links of the robotic arm 11 and the respective axis angles of multiple reference points between these links. . In some embodiments, the controller 14 may be implemented by processing circuitry or dedicated logic circuitry, such as a processor, a field programmable gate array (FPGA), or an application specific integrated circuit (ASIC). ).

The operation of the robot arm correction system 1 based on image processing will be further described below. Referring also to FIG. 2A , FIG. 2A illustrates a flow chart of a robotic arm correction method based on image processing in some embodiments. The components in the image processing-based robot arm correction system 1 in Figure 1 are used to perform steps S210 to S240 in the image processing-based robot arm correction method.

As shown in FIG. 2A , first, in step S210 , the controller 14 sets the access sequence of at least one calibration point p~r on the calibration plate 13 . In some embodiments, the access sequence may be pre-stored in the controller 14 . In some embodiments, the visit sequence may indicate that at least one correction point p~r is visited sequentially from the initial position of the robot arm 11 from near to far. For example, assuming that the calibration point p is the closest position to the initial position of the robot arm 11, and the calibration point r is the farthest position from the initial position of the robot arm 11, the robot arm 11 can move to the adjacent calibration point p~ in sequence. At the position of r, the correction points p~r can be respectively within the shooting range of the photography device 12 at these three shooting positions.

In some embodiments, the visit sequence may indicate: arbitrarily selecting a first correction point from at least one correction point p~r as the initial position, and sequentially visiting the second correction point among at least one correction point and according to the shortest path. The third correction point. In some embodiments, the shortest path may be generated using a shortest path algorithm (eg, Dijkstra algorithm (Dijkstra algorithm) or shortest path faster algorithm (SPFA), etc.). In some embodiments, the visit sequence may indicate: arbitrarily selecting a first correction point from at least one correction point p~r as the initial position, and returning after visiting a second correction point among at least one correction point p~r. to the initial position, and then visit the third calibration point among at least one calibration point p~r.

Furthermore, as shown in Figure 2A, in step S220, the robot arm 11 is driven according to the access sequence to move the photography device 12 on the robot arm end 111 of the robot arm 11 to at least three shooting positions, and the photography device 12 At least one image is captured in at least three shooting positions. And at least one image respectively has a center point. In some embodiments, the visit sequence in which the photography device 12 moves to at least three shooting positions can be preset by the user and stored in the controller 14 or other remote storage devices.

The following uses practical examples to describe photographing the correction plate 13 at at least three photographing positions. Referring also to FIG. 3 , FIG. 3 illustrates a schematic diagram of photographing the correction plate 13 at at least three photographing positions in some embodiments of the disclosure. As shown in FIG. 3 , assuming that the photography device 12 can be moved to an initial position in advance, the correction point closest to the initial position is the correction point q, and the correction point farthest from the initial position is the correction point r. At this time, the correction points q~r are not yet in the shooting range VSo of the photography device. Then, the photographing device 12 can be moved to the first photographing position. At this time, the correction point p is in the imaging range VSp of the imaging device 12, and the imaging device 12 will photograph the correction plate 13 at the first imaging position. In addition, after the shooting at the first shooting position is completed, the shooting device 12 can be moved back to the initial position. In other embodiments, the initial position can also be directly set to the position of the calibration point p, and the calibration point r and the calibration point q can be visited sequentially.

Then, the photographing device 12 can be moved to the second photographing position. At this time, the correction point q is in the imaging range VSq of the imaging device 12, and the imaging device 12 will photograph the correction plate 13 at the second imaging position. In addition, after the shooting at the second shooting position is completed, the shooting device 12 can be moved back to the initial position. Finally, the photographing device 12 can be moved to the third photographing position. At this time, the correction point r is in the imaging range VSr of the imaging device 12, and the imaging device 12 will photograph the correction plate 13 at the third imaging position. In addition, after the shooting at the third shooting position is completed, the shooting device 12 can be moved back to the initial position.

The operation of step S220 in Figure 2A will be further described below. Referring also to FIG. 2B , FIG. 2B illustrates a flowchart illustrating further steps of the image processing-based robotic arm correction method of FIG. 2A in some embodiments. The components in the robot arm correction system 1 based on image processing in Figure 1 are used to perform steps S221~S222 in the further steps of step S220 in Figure 2A. As shown in Figure 2B, first, as shown in Figure 2B, in step S221, the controller 14 generates at least one image after identifying that the shooting range of the photography device 12 covers at least one correction point p~r. In other words, as long as the shooting range of the photography device 12 can cover the correction point, an image will be generated.

Furthermore, in step S222, vector information of at least one correction point p~r is calculated from at least one image to adjust the robot arm 11. In other words, the robot arm can be corrected and adjusted based on the vector information of the correction points p~r calculated from the image. In some embodiments, the vector information may include a plurality of first vectors and a plurality of second vectors.

Furthermore, the following embodiments will further describe the generation of vector information and the operation of step S222 in Figure 2B. Referring also to FIG. 2C , FIG. 2C illustrates a flowchart illustrating further steps of the image processing-based robotic arm correction method of FIG. 2B in some embodiments. The components in the robot arm correction system 1 based on image processing in Figure 1 are used to perform steps S222A~S222C in the further steps of step S222 in Figure 2C. As shown in Figure 2C, first, as shown in Figure 2C, in step S222A, the controller 14 calculates the multiplicity between the positions of the correction points p~r in the at least one image and the respective center points of the at least one image. first vectors, and calculate multiple vector differences between multiple first vectors. In some embodiments, controller 14 may identify a two-dimensional pixel difference between the pixel location of the correction point in each of the at least one image and the pixel location of the center point of each of the at least one image (e.g., The two-dimensional pixel difference between the pixel position of the correction point and the pixel position of the center point in the image is 3 pixels in the X direction and 2 pixels in the Y direction). Then, the controller 14 may convert the plurality of two-dimensional pixel differences recognized from at least one image into a plurality of first vectors in the real space (for example, 3 pixels in the X direction and 3 pixels in the Y direction in the image). 2 pixels are converted into a first vector, which consists of 3 cm in the X direction, 2 cm in the Y direction, and 0 cm in the Z direction in the actual space).

The following uses a practical example to illustrate the first vector. Referring to FIGS. 4A to 4C together, FIGS. 4A to 4C illustrate schematic diagrams of at least one image IMGp, IMGq, and IMGr in some embodiments. As shown in Figures 4A to 4C, the images IMGp, IMGq, and IMGr have multiple center points Op, Oq, and Or. Through this, multiple two-dimensional pixel differences between the correction points p~r and the center points Op, Oq, Or in the images IMGp, IMGq, and IMGr can be calculated. , , , where the two-dimensional pixel difference , , Both include the X-direction component and the Y-direction component of the number of pixels. In this way, the two-dimensional pixel difference can be , , Convert to multiple first vectors in real space and calculate multiple vector differences between the multiple first vectors.

Furthermore, as shown in Figure 2C, in step S222B, the controller 14 calculates a plurality of second vectors between the correction points p~r on the correction plate 13, and captures the images when the photography device 12 is at at least three shooting positions. multiple axis angles of multiple reference points on the robot arm 11 at the time. In some embodiments, the controller 14 may calculate a plurality of second vectors between the absolute coordinates of the correction points p~r in the real space.

The following uses a practical example to illustrate the second vector. As shown in Figure 1, the second vector of the distance difference in the three-dimensional direction between the correction points p and r can be calculated based on the stored absolute coordinates of the correction points p~r in the actual space. and the second vector that corrects the distance difference in the three-dimensional direction between points p and q. , where the second vector , They are all 3×1 vectors, and the elements in the vector are distance differences in the X, Y, and Z directions respectively.

Furthermore, as shown in FIG. 2C , in step S222C, the controller 14 calculates the length offsets of the multiple links on the robotic arm 11 based on a plurality of vector differences, a plurality of second vectors and a plurality of axis angles. The length vector and the rotation offset vector of multiple reference points on the robot arm 11 are used to correct the robot arm 11 using the length offset vector and the rotation offset vector. In some embodiments, the axis angle may be an angle at which multiple reference points on the robot arm 11 rotate when the photography device 12 is in at least three shooting positions. In some embodiments, the length offset vector may include respective length offsets of multiple links, and the rotational offset vector may include respective axis angle offsets of multiple reference points. Thereby, the respective length offsets of the multiple connecting rods can be used to adjust their respective lengths, and the respective axis angle offsets of the multiple reference points can be used to adjust the axis angles of their respective rotations.

In some embodiments, the controller 14 may utilize inverse kinematics to generate a prediction function of the robotic arm 11, and calculate multiple angles when the photography device 12 moves to at least three shooting positions based on multiple axis angles and the prediction function. predicted positions, where the prediction function takes as variables multiple lengths of multiple links and multiple axis angles of multiple reference points. Then, the controller 14 may use the prediction function to calculate the length offset vector and the rotation offset vector according to the plurality of vector differences, the plurality of second vectors and the plurality of predicted positions.

The prediction function is explained below with a practical example. Referring also to FIG. 5 , FIG. 5 illustrates a schematic diagram of a plurality of joints J1 - J3 and a robotic arm end 111 in some embodiments. As shown in Figure 5, the robotic arm 11 is a four-axis SCARA robotic arm, which includes multiple joints J1~J3 and a robotic arm end 111. The joint J1 is connected to the base B of the robotic arm 11, and the joints J1~J3 rotate respectively. The axis angles are q1, q2, and q4 respectively, the length of the connection LK1 between joints J1~J2 is a1, the length of the connection LK2 between joints J2~J3 is a2, and the vertical offset between joints J2~J3 The quantity is q3 (that is, the length of the connecting part LK3), the joint J3 and the robot arm end 111 are connected by the connecting part LK4, the length of the connecting part LK4 in the horizontal direction is W, the vertical offset between the joint J3 and the robot arm end 111 The amount is h (that is, the length of the connecting portion LK4 in the vertical direction).

In detail, since Figure 5 is a side view of the robot arm viewed in the Y direction, for the convenience of explanation, the Y axis can be ignored below (because the Y direction of all coordinate systems is the same) to illustrate the axis angle of the above rotation. The origin and coordinate axes of the robot arm coordinate system of base B overlap with the origin and coordinate axes of the coordinate system of joint J1 (that is, the origin overlaps at joint J1 and the coordinate axes , overlap, axes , overlap), at this time, the coordinate axis can , as the center, move the coordinate axis Rotate the axis angle q1 counterclockwise and joint the coordinate axis of the coordinate system of J2 as the center, move the coordinate axis Rotate the axis angle q2 counterclockwise. Next, the origin and coordinate axes of the coordinate system at the end of the connecting portion LK3 overlap with the origin and coordinate axes of the coordinate system of the joint J3 (that is, the origin overlaps at the joint J3 and the coordinate axes , overlap, axes , overlap), at this time, the coordinate axis can , as the center, move the coordinate axis Rotate the axis angle q4 counterclockwise.

Then, based on the structure and parameters of the robot arm 11 mentioned above, the robot arm coordinate system of the base B, the coordinate system of the joints J1 to J2, the coordinate system of the end of the connecting part LK3, the coordinate system of the joint J3, and the coordinate system of the robot arm end 111 can be calculated. Multiple transformation matrices between the arm end coordinate systems, these transformation matrices will be expressed by the following formulas (1)~(5). …Formula 1) …Formula (2) …Formula (3) …Formula (4) …Formula (5)

in is the transformation matrix between the robot arm coordinate system of base B and the coordinate system of joint J1, is the transformation matrix between the coordinate system of joint J1 and the coordinate system of joint J2, is the transformation matrix between the coordinate system of joint J2 and the coordinate system of the end of connecting part LK3, is the transformation matrix between the coordinate system of the end of the connecting part LK3 and the coordinate system of the joint J3, is the transformation matrix between the coordinate system of joint J3 and the arm end coordinate system of robot arm end 111, , , , , is the rotation matrix among these transformation matrices, , , , , is the displacement matrix among these transformation matrices. Thereby, the function of the following formula (6) can be generated from the transformation matrix. …Formula (6)

in is the four-axis expression, is the displacement matrix between the robot arm coordinate system and the arm end coordinate system, and is also the above prediction function, which is the above displacement matrix , , , , The product of , , are the distance differences in the X, Y, and Z directions between the origin of the robot arm coordinate system and the origin of the arm end coordinate system, respectively. It is the angle difference between the Z axis of the robot arm coordinate system and the Z axis of the arm end coordinate system.

It can be seen from this that the prediction function uses multiple lengths of multiple connecting rods (ie, the above-mentioned connecting portions LK1 ~ LK4 ) and multiple axis angles of multiple reference points (ie, the above-mentioned joints J1 ~ J3 ) as variables. Once the detected lengths and these axis angles are brought in, the absolute coordinates of the robot arm end 111 in the actual space (ie, the above-mentioned predicted position) can be predicted. It is worth noting that this absolute coordinate is predicted by the prediction function, so it may not necessarily be equivalent to the actual absolute coordinate of the end of the robot arm 111 in the actual space (maybe because after the robot arm 11 has been used for a long time, Resulting in the deflection of the links and joints at the end of the robotic arm 111).

In an ideal state, when the controller 14 controls to drive the photographing device 12 on the robotic arm 11 to move to 2 or more shooting positions according to the set shooting position, the shooting device 12 will move at every 2 shooting positions. The difference between the actual positions minus the difference between the predicted positions calculated by the camera 12 at every two shooting positions will be equal to 0.

The following is explained with Figure 1 and Figures 4A to 4B. As shown in Figure 1 and Figures 4A to 4B, there are three shooting positions at this time. The second vector between the correction points p and q And the second vector between the correction points p and r are respectively , . It is assumed that the two-dimensional pixel difference in the images IMGp and IMGq can be , Generates multiple vector differences in real space , can be determined by the two-dimensional pixel difference in the images IMGp and IMGr , Generates multiple vector differences in real space , at this time , The sum of is the difference between the actual positions of the photographing device 12 at the photographing positions corresponding to the correction points p and q, and , The sum of is the difference between the actual positions of the photographing device 12 at the photographing positions corresponding to the correction points p and r. Thus, the above difference can be expressed by the following formula (7) ~ formula (8). …Formula (7) …Formula (8)

in is the prediction function obtained above (i.e., in the above formula (6) ), is the predicted position of the robot arm end 111 when the imaging device 12 is at the imaging position corresponding to the correction point q, is the sum of multiple axis angles when the imaging device 12 is at the imaging position corresponding to the correction point q, is the predicted position of the robot arm end 111 when the imaging device 12 is at the imaging position corresponding to the correction point p, is the sum of multiple axial angles when the photographing device 12 is at the photographing position corresponding to the correction point p, is the predicted position of the robot arm end 111 when the imaging device 12 is at the imaging position corresponding to the correction point r, is the sum of multiple axis angles when the imaging device 12 is at the imaging position corresponding to the correction point r.

Next, two 3×1 vectors can be generated from the above formula (7) and formula (8). Under ideal conditions, these two vectors should be zero vectors. However, due to length or axis angle errors that may occur due to manufacturing or assembly, these two vectors may not be 0. The following explains how to calculate length or axis angle errors caused by manufacturing or assembly.

In some embodiments, the controller 14 may calculate partial derivatives of multiple lengths and partial derivatives of multiple axis angles under the prediction function to generate a Jacobian matrix. Then, the controller 14 may calculate the length offset vector and the rotation offset vector according to the plurality of vector differences, the plurality of second vectors, the plurality of predicted positions, the Jacobian matrices, and the plurality of axis angles. Then, the controller 14 may use the length offset vector and the rotation offset vector to correct multiple lengths of multiple links and multiple axis angles of multiple rotation axes on the robotic arm 11 .

The following uses practical examples to illustrate the calculation of the length offset vector and the rotation offset vector. As shown in the following formula (9). …Formula (9)

in for the above reasons The calculated Jacobian matrix, for Bring in the matrix of multiple lengths and multiple axis angles when the photography device 12 is at the shooting position corresponding to the correction point q, for Bring in the matrix of multiple lengths and multiple axis angles when the photography device 12 is at the shooting position corresponding to the correction point p, for Bring in the matrix of multiple lengths and multiple axis angles when the photography device 12 is at the shooting position corresponding to the correction point r, is a length offset vector composed of the respective length offsets of multiple links on the robotic arm 11, is a rotation offset vector composed of the respective axis angle offsets of multiple reference points on the robot arm 11 .

Thereby, the length offset vector and the rotation offset vector can be calculated from the simultaneous equations of formula (9), and the length offset vector and the rotation offset vector can be used to adjust multiple connections on the robot arm 11 The respective lengths of the rods and the respective axis angles of the multiple reference points.

In some embodiments, the controller 14 may determine the difference between the actual positions of the photography device 12 at every two shooting positions minus the difference between the predicted positions calculated by the camera 12 at every two shooting positions. Whether the value is less than a convergence value, where this convergence value can be preset by the user. Then, when it is not less than the convergence value, the length offset vector and the rotation offset vector will continue to be calculated using the above simultaneous equations to correct the robot arm 11 . On the contrary, the length offset vector and the rotation offset vector will not be calculated to end the robot arm correction method based on image processing of the present disclosure.

Through the above steps, the length offset vector and the rotation offset vector can be calculated using the non-contact image capturing method. In addition, when it is not less than the convergence value, new length offset vectors and new rotation offset vectors can be continuously calculated to correct the robot arm 11 until it is less than the convergence value. Through this continuous correction method, the offset of the origin of the robot arm end 111 of the robot arm 11 can be corrected.

In summary, the present disclosure provides a robotic arm correction method based on image processing, which uses photographing the correction points on the correction plate to calculate the distance between the correction points, and uses these distances to calculate multiple connections of the robotic arm. The respective length offsets of the rods and the respective axis angle offsets of the multiple reference points of the robotic arm are used to achieve alignment using the respective length offsets of the multiple connecting rods and the respective axis angle offsets of the multiple reference points. Origin correction of robotic arm. Therefore, during the operation of the robot arm, if it is not sure whether the origin of the robot arm has shifted, the robot arm can be calibrated immediately in the working environment of the robot arm, and there is no need to reteach the points after the correction is completed. In this way, the time and cost required for calibration can be saved, and the work efficiency of the robotic arm can be greatly improved. In addition, since only the camera device at the end of the robotic arm is used to capture the calibration plate, when the robotic arm performs calibration, there is no need to install any additional tools before calibration, thus saving the calibration process and time, and indirectly improving the work efficiency of the robotic arm. .

1: Robotic arm correction system based on image processing

11: Robotic arm

111: End of robotic arm

12: Photography installation

13:Calibration plate

14:Controller

p~r: correction point

, :Second vector

S210~S240: steps

VSp, VSq, VSr: shooting range

IMGp, IMGq, IMGr: images

Op, Oq, Or: center point

, , : Two-dimensional pixel difference

B:Base

J1~J3: joints

LK1~LK4: Connector

a1~a2: length

q1, q2, q4: axis angle

w: horizontal length

q3, h: vertical offset

, , , , , , , , , , , :Coordinate axis

FIG. 1 is a schematic diagram of a robotic arm correction system based on image processing according to some embodiments of the disclosure. FIG. 2A illustrates a flowchart of a robotic arm correction method based on image processing in some embodiments. FIG. 2B is a flowchart illustrating further steps of the image processing-based robotic arm correction method of FIG. 2A in some embodiments. Figure 2C is a flowchart illustrating further steps of the image processing-based robotic arm correction method of Figure 2B in some embodiments. Figure 3 is a schematic diagram of photographing the correction plate at at least three photographing positions in some embodiments of the present disclosure. Figures 4A to 4C illustrate schematic diagrams of at least one image in some embodiments. Figure 5 is a schematic diagram of multiple joints and the end of a robotic arm in some embodiments of the disclosure.

S210~S220: steps

Claims (18)

A method for calibrating a robotic arm based on image processing for calibrating a robotic arm, including: setting an access sequence for at least one calibration point on a calibration plate; and driving the robotic arm according to the access sequence to make a mechanical arm of the robotic arm A photography device on the end of the arm moves to at least three shooting positions, and the photography device captures at least one image respectively at the at least three shooting positions, and the at least one image respectively has a center point, wherein the photography device captures at least one image at the at least three shooting positions. The steps of generating the at least one image from at least three shooting positions include: generating the at least one image after identifying that the shooting range of the photography device covers the at least one correction point; and calculating the at least one correction point from the at least one image. A vector information to adjust the robot arm. The robotic arm correction method based on image processing as described in claim 1, wherein the vector information includes a plurality of first vectors and a plurality of second vectors, wherein the vector information of the at least one correction point is calculated from the at least one image, The step of adjusting the robotic arm includes: calculating a plurality of first vectors between the positions of the at least one correction point in the images and respective center points of the images, and calculating the distance between the first vectors. A plurality of vector differences; calculating a plurality of second vectors between the at least one calibration point on the correction plate; and calculating based on the vector differences, the second vectors and a plurality of axis angles A length offset vector of a plurality of links on the robot arm, and a rotation offset vector of a plurality of reference points on the robot arm, so as to utilize the length offset vector and the rotation offset vector Calibrate the robot arm. The robot arm correction method based on image processing as described in claim 2, wherein the step of calculating the first vectors between the position of the at least one correction point in the images and the respective center points of the images includes : identifying a one-dimensional pixel difference between the pixel position of the correction point in each of the images and the pixel position of the center point of each of the images; and identifying from the images The two-dimensional pixel differences are respectively converted into the first vectors in a real space. The robotic arm correction method based on image processing as described in claim 2, wherein the length offset vector may include the respective length offsets of the links, and the rotation offset vector may include the respective length offsets of the reference points. Axial angle offset, wherein the step of calculating the second vectors between the at least one correction point includes: calculating the second vectors between the absolute coordinates of the at least one correction point of the correction plate in a real space vector. Robotic arm based on image processing as described in request 2 Correction method, wherein the length offset vectors of the links on the robotic arm and the reference points on the robotic arm are calculated based on the vector differences, the second vectors and the axis angles The step of using the length offset vector and the rotation offset vector to correct the robotic arm includes: using forward kinematics to generate a prediction function of the robotic arm, and based on the These axis angles and the prediction function calculate a plurality of predicted positions when the photography device moves to the shooting positions, wherein the prediction function uses the lengths of the connecting rods and the axis angles of the reference points as variables. ; And use the prediction function to calculate the length offset vector and the rotation offset vector according to the vector differences, the second vectors and the predicted positions. The robotic arm correction method based on image processing as described in claim 5, wherein the length offset of the links on the robotic arm is calculated based on the vector differences, the second vectors and the axis angles The length vector and the rotation offset vector of the reference points on the robot arm, so that the step of using the length offset vector and the rotation offset vector to correct the robot arm includes: calculating the predicted The partial derivatives of the lengths and the partial derivatives of the axis angles under the function are used to generate a Jacobian matrix; according to the vector differences, the second vectors, the predicted positions, the Jacobian matrix and the axis angles , calculate the length offset vector and the rotation offset vector; and The length offset vector and the rotation offset vector are used to correct the lengths of the links on the robot arm and the axis angles of the rotation axes. The robotic arm correction method based on image processing as described in claim 1, wherein the visit sequence indicates: visiting the at least one calibration point sequentially from an initial position of the robotic arm from near to far. The robot arm correction method based on image processing as described in claim 1, wherein the access sequence indicates: arbitrarily selecting the position of a first correction point from the at least one correction point as an initial position, and based on a shortest path A second calibration point and a third calibration point of the at least one calibration point are visited sequentially. The robot arm correction method based on image processing as described in claim 1, wherein the visit sequence indicates: arbitrarily select a first correction point position from the at least one correction point as the initial position, and visit the at least one correction point Point a second correction point among the points and then return to the initial position, and then visit a third correction point among the at least one correction points. A robotic arm correction system based on image processing, including: a correction plate, including at least one correction point; a robotic arm; A photographic device is provided at the end of a robotic arm of the robotic arm for photographing the correction plate; a controller is connected to the robotic arm and used to control the robotic arm to perform the following operations: setting a calibration point of the at least one calibration point Visit sequence; drive the robot arm according to the visit sequence to move the photography device on the end of the robot arm to at least three shooting positions, and use the photography device to capture at least one image respectively at at least three shooting positions, and the at least one image is respectively There is a center point, wherein the step of generating the at least one image respectively by the photography device at the at least three shooting positions includes: generating the at least one image after identifying that the shooting range of the photography device covers the at least one correction point; and calculating a vector information of the at least one correction point from the at least one image to adjust the robotic arm. The robotic arm correction system based on image processing as described in claim 10, wherein the vector information includes a plurality of first vectors and a plurality of second vectors, wherein the vector information of the at least one correction point is calculated from the at least one image, Adjusting the operation of the robotic arm includes: calculating a plurality of first vectors between the positions of the at least one correction point in the images and respective center points of the images, and calculating the distance between the first vectors. A plurality of vector differences; calculating a plurality of second vectors between the at least one calibration point on the calibration plate; and calculating a length offset vector of multiple links on the robotic arm and a rotation of multiple reference points on the robotic arm based on the vector differences, the second vectors and the multiple axis angles The offset vector is used to correct the robotic arm using the length offset vector and the rotation offset vector. The robot arm correction system based on image processing as described in claim 11, wherein the controller further performs the following steps: identifying the pixel position of the correction point in each of the images and the each of the images. a two-dimensional pixel difference between the pixel positions of the center points; and converting the two-dimensional pixel differences recognized from the images into the first vectors in a real space respectively. The robotic arm correction system based on image processing as described in claim 11, wherein the length offset vector may include the respective length offsets of the links, and the rotation offset vector may include the respective length offsets of the reference points. Axial angle offset, wherein the controller further performs the following steps: calculating the second vectors between the absolute coordinates of the at least one calibration point of the calibration plate in a real space. The robot arm correction system based on image processing as described in claim 11, wherein the controller further performs the following steps: Use forward kinematics to generate a prediction function of the robotic arm, and calculate a plurality of predicted positions when the photography device moves to the shooting positions based on the axis angles and the prediction function, wherein the prediction function uses the connections The plurality of lengths of the rod and the axis angles of the reference points are variables; and the length offset vector and the rotation offset are calculated using the prediction function according to the vector differences, the second vectors and the predicted positions. Shift vector. The robotic arm correction system based on image processing as described in claim 14, wherein the controller further performs the following steps: calculating partial derivatives of the lengths and partial derivatives of the axis angles under the prediction function to generate an Comparable matrix; calculate the length offset vector and the rotation offset vector according to the vector differences, the second vectors, the predicted positions, the Jacobian matrix and the axis angles; and use the length offset vector The displacement vector and the rotation offset vector correct the lengths of the links on the robot arm and the axis angles of the rotation axes. The robotic arm calibration system based on image processing as described in claim 10, wherein the visit sequence indicates: visiting the at least one calibration point sequentially from an initial position of the robotic arm from near to far. Image processing-based manipulator as claimed in claim 10 Arm calibration system, wherein the visit sequence indicates: arbitrarily select the position of a first calibration point from the at least one calibration point as an initial position, and sequentially visit a first calibration point in the at least one calibration point according to a shortest path two calibration points and a third calibration point. The robotic arm calibration system based on image processing as described in claim 10, wherein the access sequence indicates: arbitrarily selecting a position of a first calibration point from the at least one calibration point as an initial position, and visiting the at least one calibration point. A second correction point among the correction points then returns to the initial position, and then visits a third correction point among the at least one correction points.

Images