TWI758626B - Positioning method for a vision system of a robot arm - Google Patents

Abstract

The invention is to disclose a positioning method for a vision system of a robot arm. The positioning method disposes a standard calibration plate at several different z-axis heights of the vision system to form and store imagine plane data. A label put on a work plane is detected to acquire the needed datum of the work plane and vision system. An imagine plane datum, closest to the needed datum among store imagine plane data, is adopted, and the vision system is moved to the z-axis heights of the adopted imagine plane datum for taking photograph to promptly position.

Description

Vision system positioning method of robotic arm

The present invention relates to a robot arm, in particular to a method for quickly and accurately positioning a vision system of the robot arm on a work plane.

With the rapid development of industrial production technology, factory automation uses the vision system of the robotic arm to locate the position and posture of the workpiece, and guide the robotic arm to automatically and accurately grasp the workpiece for assembly and manufacturing, so as to improve the efficiency of precise and rapid production. The main key that affects the automatic and precise movement of the robot arm lies in the vision system for guiding and positioning. Improving the precise positioning of the vision system and reducing the error of guiding the robot arm is an important topic for the robot arm.

Please refer to FIG. 10 , which is a schematic diagram of the positioning of the robotic arm 10 in the prior art. One end of the robot arm 10 is fixed on the main body 11 , and the other end is a movable part 12 , and a vision system 13 such as an Eye In Hand is fixed on the movable part 12 . The robot arm 10 controls and moves the vision system 13 carried by the movable part 12 at the other end by driving the multiple toggle joints 14 to capture the image in front of the movable part 12 , position the workpiece 15 , and guide the robot arm 10 to approach and pick up and place the workpiece 15 . When the robot arm 10 is installed on the production line, a positioning correction operation needs to be performed to ensure the operation precision. When positioning and calibrating the robot arm 10 , in order to avoid hindering the positioning and calibration, the working table 16 needs to be cleared first, so as to fix the standard rectangular checkerboard-shaped calibration plate 17 on the working plane 18 of the working machine 16 .

When performing positioning correction, the vision system 13 on the movable part 12 of the robot arm 10 Taking an image, guiding the mobile robot arm 10 to drive the vision system 13 to approach and align the calibration plate 17 at a preset position, and focus the captured image of the calibration plate 17 . According to the captured image of the calibration plate 17 , the Z-axis height and direction of the vision system 13 relative to the calibration plate 17 and the plane X and Y-axis coordinates of each grid point of the calibration plate 17 are detected. In order to reduce the positioning error, the robot arm 10 needs to perform the aforesaid correction actions one by one at a plurality of preset different positions to perform comparison and adjustment. Since the distance between the checkerboard points of the standard calibration plate 17 is constant, the positioning error caused by the peripheral distortion of the captured image of the calibration plate 17 can be avoided, so that the plane coordinates of the working plane 18 can be accurately positioned. Finally, the calibration plate 17 is removed, and the workpiece 15 is placed on the working plane 18 of the working table 16 again. The robot arm 10 uses the vision system 13 to capture the image of the working plane 18 at the preset point, and the coordinates of the workpiece 15 on the working plane 18 can be accurately positioned. To pick and place the workpiece 15 correctly.

However, when positioning and calibrating the robot arm in the prior art, it takes a lot of effort to clear the working table so that a larger calibration plate can be placed, and the robot arm needs to move to multiple preset positions to perform calibration one by one. The operation is too time-consuming and cumbersome, and it is more difficult to perform positioning correction for the work table where large workpieces have been placed, which seriously affects the production efficiency. Although there are other cases such as US Published Patent No. US6885759, a small calibration plate is used, which is placed at a corner of the work plane to perform image detection and positioning calibration, so as to improve the aforementioned shortcomings. However, the area of the small calibration plate is too small relative to the working plane, and the positioning error of the working plane cannot be greatly reduced by using the small calibration plate to deduce to a larger working plane. Therefore, there are still problems to be solved in the positioning method of the vision system of the robot arm.

The object of the present invention is to provide a method for positioning a vision system of a robot arm, by placing a standard calibration plate at a preset number of Z-axis heights of the vision system by the robot arm, creating and storing different The image plane information of the Z-axis height is directly applied and stored according to the image plane information required by the vision system and the operation plane, so that the robot arm vision system can accurately locate the operation plane.

Another object of the present invention is to provide a method for locating a vision system of a robotic arm. An icon is placed on the working plane, the robotic arm moves the vision system to a desired working height to shoot the icon, aligns the image plane of the vision system, and detects the difference between the vision system and the vision system. The Z-axis height of the working plane, apply the image plane information of the closest Z-axis height in the storage to quickly locate the working plane.

Another object of the present invention is to provide a method for locating a vision system of a robotic arm, which utilizes the inputted length and width of the working plane plus a reserved peripheral distance to form a preset field of view of the vision system of the robotic arm, and applies the image plane of the closest field of view in storage. information to correctly position the work plane.

In order to achieve the purpose of the foregoing invention, the positioning method of the vision system of the robotic arm of the present invention, the positioning method of the vision system of the robotic arm of the present invention, the standard calibration board is arranged at several preset Z-axis heights of the vision system, and the vision system is used to take pictures separately. The standard calibration plate establishes the information of the image plane, and stores the image plane information of each Z-axis height, and then detects the information of the vision system and the operation plane. The operation plane can be the plane on the working machine or the workpiece, according to the detected information Apply the closest image plane information in the storage, and then move the vision system away from the Z-axis height of the operation plane to capture the image according to the Z-axis height of the applied image plane information, and overlap the X and Y-axis coordinates of the applied image plane information to the camera. Image to locate the working plane.

The visual system positioning method of the mechanical arm of the present invention, when detecting the information of the visual system and the work plane, sticks the icon on the work plane, and moves the visual system to make the icon enter the field of vision of the visual system, and calculates the position of the icon according to the photographed image of the icon. The coordinates of the arm coordinate system, Position and attitude, information that determines the relative position of the work plane and the vision system. The standard calibration plate is a rectangular checkerboard, and the icon is a small and directional calibration plate. According to the orientation provided by the icon, the vision system can be aligned with the working plane to ensure that the image plane captured by the vision system is parallel to the working plane.

In the positioning method of the vision system of the robot arm of the present invention, the detected information is the height information of the vision system and the icon, which is used as the height information between the operation plane and the vision system. According to the detected height, the stored image plane information is compared to the closest one The Z-axis height of , and the image plane information of the Z-axis height is applied. In another embodiment, the detected information is to calculate the required visual system field of view by using the length and width of the input working plane and the preset increased peripheral distance in advance, and then compare and store the image according to the visual system field of view required by the calculation. The field of view in the plane information, the image plane information of the closest field of view is applied.

The positioning method of the vision system of the robotic arm of the present invention is that, for the standard calibration plate placed at the height of the Z-axis of the vision system, the vision system of the robotic arm shoots an image, and detects the direction and each direction of the standard calibration plate according to the captured image of the standard calibration plate. The coordinates of the grid points are deduced to the X and Y axis coordinates of the same plane of the standard calibration plate to establish the image plane and field of view of the visual coordinate system of the vision system at the height of the Z axis, and form the image plane information at the height of the Z axis.

20: Robot Arm

21: Pedestal

22: Axle section

23: Activities Department

24: Vision System

25: Actuating Motor

26: Standard calibration plate

27: Working machine

28,32: Work plane

29,31: Artifacts

30: Icons

FIG. 1 is a schematic diagram of establishing an image plane of a vision system for a robot arm of the present invention.

FIG. 2 is a schematic diagram of establishing a plurality of image planes with Z-axis heights by the vision system of the present invention.

FIG. 3 is a schematic diagram of the positioning operation plane of the vision system of the present invention.

Fig. 4 is a schematic diagram of positioning by the vision system of the present invention

FIG. 5 is a flow chart of the positioning method of the robotic arm vision system of the present invention.

FIG. 6 is a schematic diagram of setting a visual field of a vision system according to another embodiment of the present invention.

FIG. 7 is a schematic diagram of applying a stored image plane to a comparison field of view according to another embodiment of the present invention.

FIG. 8 is a flowchart of a positioning method of a robotic arm vision system according to another embodiment of the present invention.

FIG. 9 is a schematic diagram of positioning of a work plane according to another embodiment of the present invention.

FIG. 10 is a schematic diagram of the positioning correction of the vision system of the robotic arm in the prior art.

Regarding the technical means adopted by the present invention to achieve the above-mentioned objects and their effects, preferred embodiments are given and described below with the accompanying drawings.

Please refer to FIG. 1 to FIG. 4 at the same time. FIG. 1 is a schematic diagram of a robot arm of the present invention establishing an image plane of a vision system. FIG. 2 is a schematic diagram of a vision system of the present invention establishing several image planes with a height of the Z-axis. FIG. 3 is a schematic diagram of the vision system of the present invention. A schematic diagram of the positioning operation plane, FIG. 4 is a schematic diagram of positioning by the vision system of the present invention. In FIG. 1, one end of the robot arm 20 of the present invention is a fixed base 21, and the multi-axis joints 22 are connected in series to form a movable part 23 at the other end. An actuating motor 25 is provided. The robot arm 20 rotates via the actuating motors 25 controlling the joints 22 to move the vision system 24 carried by the movable part 23 of the robot arm 20 to capture images. The depth of field of the captured images forms the visual coordinate system V of the vision system 24 . The robot arm 20 of the present invention uses the fixed base 21 as a reference point to form an arm coordinate system R of the robot arm 20 . The moving coordinates of the movable portion 23 in the coordinate system R are calculated based on the known lengths of the joints 22 and the movable portion 23 of the robot arm 20 , and the rotation angle of the actuator motor 25 controlled by the toggles 22 . Since the relative positions of the movable part 23 and the visual system 24 are fixed, the arm coordinate system R and the visual coordinate system V have a certain conversion relationship, so the robot arm 20 can move the movable part according to the coordinates of the visual coordinate system V twenty three.

When the vision system 24 of the robot arm 20 of the present invention establishes the positioning operation plane 28 in advance, the standard calibration plate 26 in the shape of a rectangular checkerboard is fixed on the operation plane 28 of the working table 27 . The vision system 24 captures images, guides the mobile robot arm 20 to drive the vision system 24 , aligns the standard calibration plate 26 at the preset Z-axis height H relative to the standard calibration plate 26 of the visual coordinate system V, and captures an image of the standard calibration plate 26 . According to the captured image of the standard calibration plate 26, the direction of the standard calibration plate 26 at the preset Z-axis height H and the coordinates of each grid point are detected, and the X and Y-axis coordinates of the same plane of the standard calibration plate 26 are deduced to establish a visual The coordinate system V is at the image plane P and the field of view F at the preset Z-axis height H, and is stored as the image plane information at the preset Z-axis height H. Similarly, in FIG. 2 , a plurality of Z-axis heights H1 , H2 , H3 of different heights are preset, and the vision system 24 of the robot arm 20 performs the same operation as the aforementioned robot arm 20 to capture the image of the standard calibration plate 26 to perform the operation of positioning the operation plane 28 . , establish the image planes P1, P2, P3 and fields of view F1, F2, F3 of the visual coordinate system V at the preset Z-axis heights H1, H2, H3, and store the image planes at the preset Z-axis heights H1, H2, H3 News.

In FIG. 3 , when positioning the work plane 28 , the vision system 24 of the robot arm 20 of the present invention does not need to clear the workpiece 29 on the work plane 28 , and directly sticks the small icon 30 with the directional correction plate on the work table 27 In this embodiment, the icon 30 is aligned with the corner of the work plane 28 whose relative position is relatively easy to distinguish, but it includes but is not limited to the position where the icon 30 is placed in this embodiment. The present invention moves the vision system 24 of the robot arm 20 to the required working height and allows the icon 30 to enter the field of view of the moving vision system 24. The robot arm 20 uses the vision system 24 to photograph the icon 30 and guides the vision system 24 to move closer to the icon 30. An image of the icon 30 is captured. Then, according to the captured image of the icon 30, calculate the coordinates of the icon 30 in the arm coordinate system R, including the position and posture, determine the relative position of the work plane 28 and the vision system 24, and then obtain the height between the vision system 24 and the icon 30. The degree D information is used as the height D information of the operation plane 28 and the vision system 24 .

In FIG. 4, the present invention compares the closest Z-axis height from the pre-stored information of a plurality of image planes P according to the detection height D between the vision system 24 and the icon 30, and applies the image plane P of the Z-axis height H, Then, according to the operation plane 28 positioned by the aforementioned detection icon 30, the robot arm 20 moves the vision system 24 at the Z-axis height H from the operation plane 28 according to the applied image plane P to capture an image, and the applied image plane information X, X, X, X, X, and X of the applied image plane information are captured. The Y-axis coordinates are superimposed on the captured image, and the X and Y-axis coordinates of the image plane P are applied from where the workpiece 29 is located to quickly and accurately locate the workpiece 29 .

As shown in FIG. 5 , it is a flow chart of the positioning method of the robotic arm vision system of the present invention. The detailed flow of the positioning method of the vision system of the robotic arm of the present invention is described as follows: in step S1, the robotic arm arranges a standard calibration plate at several preset Z-axis heights of the vision system; in step S2, the vision system is used to separately photograph the standard calibration plate to establish image plane information; step S3, storing image plane information of each Z-axis height; then step S4, detecting the information of the vision system and the operation plane, and applying the stored closest image plane information according to the detected information; finally, step S4 S5, the robot arm moves the vision system from the Z-axis height of the working plane to capture images according to the Z-axis height of the applied image plane information, and superimposes the X and Y-axis coordinates of the applied image plane information to the captured image for positioning.

Please refer to FIG. 3 , FIG. 6 and FIG. 7 at the same time. FIG. 6 is a schematic diagram of setting a visual system field of view according to another embodiment of the present invention, and FIG. 7 is a schematic diagram of applying a stored image plane to a comparison field of view according to another embodiment of the present invention. To simplify the description, the same components in this embodiment and the previous embodiment use the same part numbers. Since the plurality of image planes P pre-stored in the foregoing embodiment have preset Z-axis heights H different from those of the vision system 24, the smaller the Z-axis height H, the closer to the vision system 24. Although the positioning accuracy is higher, its image The plane P is photographed by the vision system 24 and the field of view F of the working plane 28 is smaller, and vice versa, the Z-axis height H is The larger the distance, the further away from the vision system 24. Although the positioning accuracy is poor, the larger the field of view F of the image plane P captured by the vision system 24 on the operation plane 28, will be limited by the positioning accuracy. Therefore, the size of the field of view F is related to the height of the Z axis. H has a corresponding relationship (see FIG. 2 ), and the relative Z-axis height H can also be obtained by using the detection field of view F in this embodiment.

In FIG. 3 , in this embodiment, the icon 30 is also pasted on the working plane 28 of the working machine 27 , and the icon 30 is moved into the field of view of the mobile vision system 24 by moving the robot arm 20 , and the vision system 24 is guided to move close to the icon 30 to shoot. Image of icon 30. Then, according to the captured image of the icon 30 , the coordinates of the icon 30 in the arm coordinate system R, including the position and posture, are calculated, and then the relative position of the work plane 28 and the arm coordinate system R is obtained. In addition, in FIG. 3 , the robot arm 20 can also further strengthen the alignment of the vision system 24 with the working plane 28 according to the orientation provided by the icon 30 to ensure that the image plane captured by the vision system 24 is parallel to the working plane 28 , so that the applied image plane P is more accurate .

In FIG. 6 , when the robot arm 20 is used for positioning, the icon on the work plane 28 can be removed. In the present invention, the user pre-inputs the required length (L) and width (W) of the work plane 28 and presets the work plane 28 The peripheral visual field increases the peripheral distance K, and the size of the required visual field F is calculated by the robot arm 20 . In FIG. 7 , according to the required field of view F, the field of view F with the closest size is compared with the information of a plurality of image planes P stored in advance, and the image plane P is applied, and then the robot arm 20 is positioned according to the operation plane 28 of the aforementioned detection icon 30 . Then, according to the Z-axis height H of the applied image plane P, the vision system 24 is moved from the Z-axis height H of the work plane 28 to capture an image, and the X and Y-axis coordinates of the applied image plane information are superimposed on the captured image. The X and Y axis coordinates of the image plane P are applied to locate the workpiece 29 quickly and accurately.

As shown in FIG. 8 , it is a flowchart of a positioning method of a robotic arm vision system according to another embodiment of the present invention. The detailed flow of the positioning method of the robotic arm vision system according to another embodiment of the present invention The process is described as follows: in step T1, the robot arm arranges standard calibration plates at preset Z-axis heights of several different vision systems; in step T2, the information of the image planes is established by using the vision system to photograph the standard calibration plates; step T3, saves the information of each Z axis The image plane information of the axis height; Step T4, place the icon on the operation plane, and move the vision system to make the icon enter the field of view of the vision system; Then step T5, shoot the icon image, to calculate the relative positional relationship between the operation plane and the vision system; In step T6, the length and width of the input operation plane and the preset peripheral distance are used to calculate the required visual system field of view; in step T7, the field of view in the stored image plane information is compared, and the image plane information of the field of view with the closest size is applied; Step T8, the robot arm moves the vision system from the Z-axis height of the working plane to capture images according to the Z-axis height of the applied image plane information, and superimposes the X and Y-axis coordinates of the applied image plane information to the captured image for positioning.

As shown in FIG. 9 , it is a schematic diagram of working plane positioning according to another embodiment of the present invention. In the previous embodiment, although the positioning method of the vision system of the robot arm of the present invention is described by taking the positioning of the work plane on the working machine as an example, the present invention includes and is not limited to the working plane of the working machine. In FIG. 9 , taking the working plane 32 of the workpiece 31 as an example, the icon 30 is directly attached to the working plane 32 of the workpiece 31 , and the image of the icon 30 is captured by the vision system 24 on the robot arm 20 , the orientation of the icon 30 is calculated, and then the positioning is performed. The relative position of the work plane 32 to the vision system 24 . Measure the detection height D of the working plane 32 and the vision system 24, or input the length (L) and width (W) of the working plane 32, increase the peripheral distance K from the preset field of view, calculate the size of the required field of view F, compare and apply The closest stored image plane P information, the robot arm can move the vision system to the Z-axis height of the operation plane 32 to shoot and position the operation plane 32 according to the applied image plane P information. Therefore, the present invention can be applied to any plane in the working environment of the robot arm 20 for positioning.

Therefore, the positioning method of the vision system of the robot arm of the present invention can be The arm arranges a standard calibration board at several preset Z-axis heights of the vision system, and pre-establishes and stores image plane information of different Z-axis heights. When positioning, detect the icons on the working plane, strengthen the visual system to parallel the working plane, and determine the relative position of the working plane and the vision system, and then according to the image plane information required by the vision system and the working plane, such as detecting the Z-axis height, Or use the input operation plane length and width plus the reserved peripheral distance to form the required field of view and other image plane information, directly apply the image plane information of the closest Z-axis height or field of view in the storage, and then move the vision system to the applied image plane Z The axis height is used for shooting and positioning, so as to achieve the invention purpose of accurately and quickly positioning the working plane of the present invention.

The above descriptions are only for the convenience of describing the preferred embodiments of the present invention, and the scope of the present invention is not limited to these preferred embodiments. Any changes made according to the present invention shall not depart from the spirit of the present invention. , all belong to the scope of the patent application of the present invention.

20‧‧‧Robot arm

24‧‧‧Vision System

26‧‧‧Standard Calibration Plate

28‧‧‧Working plane

Claims (10)

A vision system positioning method of a robotic arm, the steps of which include: the robotic arm arranges a standard calibration plate at several preset Z-axis heights of the vision system, and uses the vision system to separately photograph the standard calibration plate to establish the information of the image plane; The height of the image plane information; detect the information of the vision system and the operation plane, and apply the closest image plane information in the storage according to the detected information; according to the Z-axis height of the applied image plane information, move the vision system to the Z axis of the operation plane Shoot the image at a height, and superimpose the X and Y-axis coordinates of the applied image plane information to the shooting image to locate the working plane. The method for positioning the vision system of a robotic arm as described in item 1 of the scope of application, wherein the step of detecting the information of the vision system and the work plane comprises: sticking an icon on the work plane, and moving the vision system to make the icon enter the vision system According to the captured image of the icon, the coordinates, position and posture of the icon in the arm coordinate system are calculated to determine the relative position information of the operating plane and the vision system. The positioning method for a vision system of a robotic arm as described in item 2 of the scope of the patent application, wherein the standard calibration plate is in the shape of a rectangular checkerboard, and the icon is a small and directional calibration plate. The method for positioning the vision system of a robotic arm as described in item 3 of the scope of the patent application, wherein when the vision system captures the icon, according to the orientation provided by the icon, the vision system is aligned with the working plane to ensure that the image plane captured by the vision system is parallel to the working plane . The method for positioning a vision system of a robotic arm as described in item 3 of the scope of the patent application, wherein the detected information is the height information of the vision system and the icon, which is used as the information between the operation plane and the vision system. Altitude information. The vision system positioning method of the robotic arm as described in item 5 of the scope of the patent application, wherein according to the detection height, the stored image plane information is compared with the closest Z-axis height, and the image plane information of the Z-axis height is applied. The vision system positioning method of a robotic arm as described in item 2 of the scope of the application, wherein the detected information is to calculate the required vision system by using the length and width of the input working plane and the preset enlarged peripheral distance in advance. vision. The vision system positioning method of the robotic arm described in item 7 of the scope of the patent application, wherein according to the vision system field of view required for the calculation, the field of view in the stored image plane information is compared, and the image plane information of the closest field of view is applied. The method for positioning a vision system of a robotic arm as described in item 1 of the scope of the patent application, wherein the standard calibration plate arranged at the height of the Z-axis of the vision system is captured by the vision system of the robotic arm, and the image of the calibration plate is detected according to the captured image of the standard. Measure the direction of the standard calibration plate and the coordinates of each grid point, deduce to the X and Y-axis coordinates of the same plane of the standard calibration plate, and establish the image plane and field of view of the visual coordinate system of the vision system at the Z-axis height, forming the Z-axis height. Image plane information. The vision system positioning method of a robotic arm as described in item 1 of the scope of the application, wherein the working plane is a plane on a work table or a workpiece.

Images