TWI617995B - Method for checking the vision position of a robot - Google Patents

Abstract

A method for verifying the visual positioning of a robot. After the robot corrects the visual positioning, it compensates the set robot verification posture to capture the shape feature image in the working environment by compensating the posture, compares the image difference value with the reference feature image, and the image difference value is greater than a preset When the threshold is different, the visual positioning is performed again to ensure the correct visual positioning.

Description

Verification method of robot vision positioning

The invention relates to a robot, in particular to a method for industrial robots to capture images using a vision system, locate a work position, and then verify the positioning.

The robot has the characteristics of flexible movement, precise positioning and continuous operation, and has become the best weapon for manufacturing and assembly on product production lines. Flexible operation of the robot, allowing the robot to quickly switch production lines and efficiently produce various products, has become an important issue to improve the production efficiency of robots.

The prior art robot sets the peripheral fixtures in a relatively fixed relationship, so that the robot maintains a certain relative relationship with the workpiece and the work surface, so that the robot can pick up and process the workpiece. However, this method of fixed relationship, which fixes the entire operating unit including the robot, requires a large number of precise mechanical fixtures, and the mechanical combination can form a relatively fixed relationship. Once the produced workpieces and processing procedures change, all mechanical fixtures, mechanical combinations, relatively fixed relationships, and operating points in the operating unit must be accurately re-taught and set up, and the robot is fixed in the operating unit and it is difficult to use it for other purposes. The robot cannot be scheduled flexibly, and it lacks flexibility in use.

Therefore, there are other previous technologies that increase the flexibility of using robots. For example, the national patent TW385269 patent case puts the robot on a movable trolley and moves and switches between working units on the production line. In order to solve the positioning of the trolley to each operating unit every time, due to the workbench The three levels of surface, floor, and trolley are inconsistent, resulting in an uncertain relationship between the robot and the work surface in three dimensions. In the prior art, a positioning mark was set on the work surface and the robot's vision system was used to allow the robot to automatically complete the positioning of the three-dimensional space of the robot coordinate system and the work surface before starting work. Therefore, all of the robot's teaching operation points on the work surface will re-establish the original relative fixed relationship with the positioning results to perform the preset automatic operation.

However, the aforementioned prior art uses a vision system for spatial positioning, often due to the error of the vision system itself, the working environment, light and shadow, etc., causing its positioning accuracy to deviate, and it is difficult to completely avoid the misidentification of the vision system. However, when a misidentification occurs in the vision system, an incorrect positioning result will be generated, which will affect the automatic operation of all teaching points, which will often cause collisions, inadequate machining accuracy or damage to the workpiece. Therefore, how to ensure the correct visual positioning of the robot is an urgent problem.

The object of the present invention is to provide a method for verifying the visual positioning of a robot. After the visual positioning, the image of the shape feature in the working environment is captured by compensating and setting the robot verification posture, and the difference is compared with the taught reference feature image to verify the correct visual positioning. Sex.

Another object of the present invention is to provide a method for verifying the visual positioning of a robot. By using the difference between a captured feature image and a reference feature image, a preset image difference threshold is preset, and when the image difference is greater than the difference threshold, visual positioning is performed again to ensure Visual positioning is correct.

In order to achieve the purpose of the foregoing invention, the method for verifying the visual positioning of a robot of the present invention teaches the robot to establish and record a visual positioning reference, and the robot moves into a set position, automatically controls the movement to the reference posture recorded by the visual positioning reference, and captures positioning marks. Anchor mark image, image processing and comparison of the captured anchor mark image with the reference setting of the visual positioning reference record Mark the difference of the image, and use the image difference to compensate and correct the positioning of the robot, and make the same compensation for the verification posture of the visual positioning reference record to form the compensation posture. The robot automatically controls the movement to the compensation posture, and captures the feature image for the shape feature. Image processing and comparison of the difference value of the shape feature between the extracted feature image and the reference feature image recorded by the visual positioning reference. Check that the image difference value is not less than the preset difference threshold, and then restart the visual positioning verification. Check that the image difference value is less than the preset difference threshold, and then complete the visual positioning verification operation.

When teaching the robot to establish and record the visual positioning reference, the mobile robot enters the set position, pulls the robot to the first point, and captures the positioning mark image with the reference attitude as the reference positioning mark image, records the reference positioning mark image and the robot's reference attitude , Towing the robot to the second point, the image of any shape feature in the working environment is taken to verify the posture, as the reference feature image, the reference feature image and the verification posture of the robot are recorded, and the visual positioning reference is established.

When the invention uses the image difference to correct the positioning of the robot, the positioning mark image captured by the robot's reference attitude and the recorded reference positioning mark image are used to compare the features of the positioning mark in the two images to calculate the difference between the displacement and the rotation angle. Serve the mobile robot according to the difference amount, search and compare the captured positioning mark image, make the captured positioning mark image be the same as the reference positioning mark image, or the difference amount is less than the preset threshold, record the robot's correction posture, and calculate the correction posture The coordinate offset from the reference attitude compensates the robot coordinate offset to complete the robot's calibration and positioning.

The setting position of the invention is set at the working position of the robot in the work unit, and the positioning mark is set at the work table in the work unit. The positioning mark and the working environment in the work unit maintain a relatively fixed spatial position relationship, and the shape characteristic is in the working environment Any identifiable special Image. When the visual positioning verification is restarted, the number of re-calibration positioning is counted, and once the preset number of times is exceeded, an alarm is activated.

1‧‧‧Operating unit

2‧‧‧ Robot

3‧‧‧ Controller

4‧‧‧Workbench

5‧‧‧Working environment

6‧‧‧ fixed end

7‧‧‧ base

8‧‧‧ positioning mark

9‧‧‧ event side

10‧‧‧Vision System

11‧‧‧ Position marker image

12‧‧‧ fiducial mark image

13‧‧‧ Feature image

14‧‧‧ benchmark feature image

FIG. 1 is a schematic diagram of teaching a robot to establish a visual positioning reference.

FIG. 2 is a schematic diagram of verification of the visual positioning of a robot according to the present invention.

FIG. 3 is a schematic diagram of comparing image differences of a reference positioning mark according to the present invention.

FIG. 4 is a schematic diagram of an image of a robot according to the present invention to adjust a posture and search for a reference positioning mark.

FIG. 5 is a schematic diagram of a robot capturing a characteristic image with a corrected posture according to the present invention.

FIG. 6 is a schematic diagram of comparing differences in reference feature images according to the present invention.

FIG. 7 is a flowchart of a method for establishing a visual positioning reference for a robot according to the present invention.

FIG. 8 is a flowchart of a method for verifying visual positioning of a robot according to the present invention.

Regarding the technical means adopted by the present invention to achieve the above-mentioned objectives, and their effects, preferred embodiments are described below with reference to the drawings.

As shown in FIG. 1, it is a schematic diagram of teaching a robot to establish a visual positioning reference. The working unit 1 of the present invention mainly includes a robot 2, a controller 3, a workbench 4, and a working environment 5. The fixed end 6 of the robot 2 is fixed on the base 7 to form a robot coordinate system R, and the base 7 can be a mobile body such as a trolley. The robot 2 can be moved to each work unit 1 by the robot 2 and approached or left the work unit 1. Positioning marks 8 are provided on the workbench 4, and the positioning marks 8 maintain a relatively fixed spatial position relationship with the workbench 4 and the surrounding working environment 5. The working environment 5 has multiple shape features F, shape features F. It can be a recognizable special image such as a sharp corner of an object or a mark.

The mobile terminal 9 of the robot 2 is provided with a vision system 10, and the robot 2 is connected to the controller 3. The controller 3 controls the movement of the robot 2 according to programming, so that the mobile terminal 9 carries the vision system 10 to capture characteristic images of the workbench 4 or the working environment 5, The attitude and characteristic image of the robot 2 when capturing the image are recorded in the controller 3. The controller 3 captures the posture of the image by the robot 2 and can recognize and record the coordinates of the movable end 9 in the robot coordinate system R according to the rotation relationship of the servo motor of each axis section. Since the vision system 10 is fixed on the movable end 9 of the robot 2, the coordinates of the vision system 10 can also be obtained from the relatively fixed relationship between the vision system 10 and the movable end 9. The controller 3 then performs image processing on the stored feature images, and determines the spatial relationship between the features in the images and the visual system 10 from the focus status of the visual system 10 to locate the coordinates of the features.

The present invention teaches the robot 2 to establish a visual positioning reference, first to make the robot 2 enter a set position in the work unit 1, the set position is the work position of the robot 2, and the user pulls the movable end 9 of the robot 2 to the first point P1, the controller 3 is used to operate the vision system 10 to capture an image of the positioning mark 8 on the table 4 as a reference positioning mark image 12 (see FIG. 3). The controller 3 then records the reference positioning mark image 12 and the robot 2 reference posture A when the positioning mark 8 image is captured. From the reference attitude A, the robot 2 can calculate the coordinates of the movable end 9 at the first point P1 according to the rotation relationship of the servo motor of each axis section. The reference positioning mark image 12 provides the relative spatial relationship between the positioning mark 8 and the moving end 9 , Further obtain the relative positional relationship between the table 4 and the working environment 5 which maintain a fixed positional relationship with the positioning mark 8, so that the robot 2 can be clearly positioned in the work unit 1.

The user then moves the movable end 9 of the robot 2 to the second point P2, and uses the controller 3 to operate the vision system 10 to capture an image of any shape feature F of the working environment 5 as a reference feature image 14 (see Figure 6). The controller 3 records the reference feature image and captures the shape. The robot 2 verifies the posture B at the time of the image of the feature F. Similarly, the robot 2 can verify the posture B and calculate the coordinates of the movable end 9 at the second point P2 according to the rotation relationship of the servo motor of each axis section. The reference feature image 14 provides the relative space between the shape feature F and the movable end 9 relationship. Since the shape feature F is fixed in the position of the working environment 5, a relatively fixed spatial relationship is maintained relatively with the positioning mark 8. Therefore, under the condition of correct visual positioning, as long as the positioning mark 8 is used to position the robot 2 in the work unit 1, the robot 2 can capture the same reference feature image 14 with the same verification posture B.

Please refer to FIG. 2 to FIG. 6 at the same time. FIG. 2 is a schematic diagram of verification of the visual positioning of the robot according to the present invention, FIG. 3 is a schematic diagram of comparing the difference between the reference positioning mark images of the present invention, and FIG. Schematic diagram, FIG. 5 is a schematic diagram of a robot capturing feature images with a corrected posture according to the present invention, and FIG. 6 is a schematic diagram of comparing a reference feature image difference according to the present invention. In FIG. 2, after the visual positioning reference is established in the present invention, when the robot 2 moves and then enters the set position of the work unit 1, the positioning of the robot 2 needs to be corrected first. The robot 2 automatically controls the movement formation based on the information recorded when the reference is established. The reference attitude A, and control the vision system 10 to capture a positioning mark image 11 (see FIG. 3), compare the difference between the positioning mark image 11 and the reference positioning mark image 12 (see FIG. 3) after image processing, and use the image difference to correct the robot 2 Positioning.

In FIG. 3, the present invention uses image differences to perform positioning of the correction robot 2. The positioning mark image 11 captured by the robot 2 with the reference attitude A is compared with the recorded reference positioning mark image 12, and the feature comparison of the positioning mark 8 in the two images is performed to calculate the difference between the displacement amount and the rotation angle. Then in FIG. 4, the mobile robot 2 is servoed according to the difference amount, and the positioning mark 8 images are continuously captured for search and comparison, so that the captured positioning mark image 11 is the same as the reference positioning mark image 12 or the difference is smaller than the preset valve. When the value is set, the corrected posture A ′ of the robot 2 is recorded. The captured positioning mark image 11 is the same as the reference positioning mark image 12 at this time, indicating that the mobile terminal 9 and the fixed position of the robot 2 The bit mark 8 maintains the relative positional relationship when the visual positioning reference is established. Based on the corrected attitude A ′ and the rotation relationship of the servo motor of each axis of the robot 2, the coordinates of the movable end 9 after correction in the robot 2 coordinate system R can be obtained, and the movable end 9 coordinates of the corrected attitude A ′ and the reference attitude A are calculated. The offset of the robot 2 is compensated, and the robot 2 can be repositioned in the work unit 1 to complete the correction and positioning of the robot 2. The foregoing is merely an example. There are various methods for correcting and positioning the robot 2. The present invention includes and is not limited to the foregoing examples.

According to the present invention, after the calibration and positioning of the robot 2 is completed, the coordinates of the positioning mark 8 that has been offset due to the offset have been compensated, and the coordinates of the shape feature F that has a fixed spatial relationship with the positioning mark 8 will also be shifted. The verification attitude B at this time performs the same offset compensation to form a compensated attitude B ′. In FIG. 5, when the present invention performs verification, the robot 2 first performs the same offset compensation on the verification posture B as the compensation posture B ′ according to the information recorded when establishing the reference, and the robot 2 then automatically controls the movement to the compensation posture B ′, The vision system 10 is controlled to capture the feature image 13 of the shape feature F. In FIG. 6, the image difference value e between the shape feature F ′ in the feature image 13 and the shape feature F in the reference feature image 14 is compared through image processing. For example, the X axis of the image difference value e: 3 pixels (pixels), Y: 6 pixels (pixels), angle: 0.5 degrees, the pixels are converted into actual distances such as 6mm, 12mm, and so on. Compared with the preset difference threshold E, if the image difference value e is smaller than the difference threshold E, it means that the positioning error has been verified to be in an acceptable range, and the robot 2 can continue the operation after completing the positioning. Once the image difference value e is larger than the difference threshold value E, it means that the positioning error is too large to continue the operation, and the robot 2 needs to re-correct the positioning. For the robot 2 that repeatedly recalibrates the positioning, the limit of the number of times of recalibration should be further set, and the number of times of recalibration should be counted. Once the number of times exceeds the limit, an alarm will be initiated to notify maintenance to avoid collision damage.

As shown in Figure 7, the flow of establishing a visual positioning reference method for the robot of the present invention Cheng. According to the description of the previous embodiment, the detailed steps of the method for establishing a visual positioning reference of the present invention are described as follows: First, in step S1, the mobile robot enters the set position of the work unit and starts to establish a visual positioning reference for the robot; step S2, the robot is pulled to In the first position, the positioning mark image of the workbench is acquired with the reference posture as the reference positioning mark image; step S3, the reference positioning mark image and the reference posture of the robot are recorded; in step S4, the robot is pulled to the second position to verify The posture captures an image of any shape feature in the working environment as a reference feature image, and then proceeds to step S5 to record the reference feature image and the verification posture of the robot; step S6, the end of establishing a visual positioning reference.

As shown in FIG. 8, it is a flowchart of a method for verifying the visual positioning of a robot according to the present invention. According to the description of the foregoing embodiment, after teaching the robot to establish and record a visual positioning reference, the present invention requires a robot to perform visual positioning and positioning verification for a robot that moves into a position set by a work unit. The process of the method for verifying the visual positioning of a robot according to the present invention The detailed steps are described as follows: First, in step T1, the robot moves into the position set by the work unit and starts visual positioning verification; in step T2, the robot automatically controls the movement to the reference posture recorded by the visual positioning reference, and captures the positioning marks of the workbench. Positioning mark image; step T3, image processing and comparing the difference between the captured positioning mark image and the reference positioning mark image recorded by the visual positioning reference, and using the image difference to compensate and correct the positioning of the robot; to step T4, verifying the establishment of the reference record Perform the same compensation on the attitude to form a compensated attitude; then, go to step T5, the robot automatically controls the movement to the compensated attitude, and captures the feature image of the shape features in the working environment; step T6, the image processing compares the captured feature image with the reference feature As shape feature values in the difference image; step T7, checked image difference value is smaller than a predetermined difference threshold value? If the image difference value is not less than the preset difference threshold, go to step T8, return to step T1, and restart the visual positioning verification. If the image difference value is less than the preset difference threshold, go to step T9 to complete the visual positioning verification and end the operation.

Therefore, the method for verifying the visual positioning of the robot of the present invention can verify the visual positioning by compensating the set robot to verify the posture, capture the shape feature image in the working environment, and compare the difference with the taught reference feature image. The correctness of the image, and use the image difference value between the captured feature image and the reference feature image, and the preset image difference threshold. When the image difference value is greater than the difference threshold, the visual positioning is performed again to ensure the correct visual positioning. the goal of.

The above are only for the convenience of explaining the preferred embodiments of the present invention, and the scope of the present invention is not limited to these preferred embodiments. Any changes made according to the present invention can be made without departing from the spirit of the present invention. , All belong to the scope of patent application of the present invention.

1‧‧‧Operating unit

2‧‧‧ Robot

3‧‧‧ Controller

4‧‧‧Workbench

5‧‧‧Working environment

6‧‧‧ fixed end

7‧‧‧ base

8‧‧‧ positioning mark

9‧‧‧ event side

10‧‧‧Vision System

Claims (8)

A method for verifying a robot's visual positioning includes the steps of: moving a robot into a set position; towing the robot to a first position as a reference posture; capturing a positioning mark image as a reference positioning mark image; recording the reference positioning mark image and the robot's Reference posture; Towing the robot to the second point for the verification posture, capture an image of any identifiable shape feature in the working environment as the reference characteristic image; record the reference characteristic image and the verification posture of the robot; end the establishment Visual positioning reference; the robot enters the set position; the robot automatically controls to move to the reference attitude recorded by the visual positioning reference, and captures the positioning mark image for the positioning mark; image processing and comparing the captured positioning mark image with the reference of the visual positioning reference record Position the mark image difference, and use the image difference to compensate and correct the positioning of the robot; perform the same compensation on the verification posture recorded by the visual positioning reference to form the compensation posture; the robot automatically controls the movement to the compensation posture, and extracts the shape features Image sign; difference image characteristic of the reference image shape characteristics and comparing the captured image processing and visual image features recorded positioning reference value; checking the image difference value is not less than a preset difference threshold, positioning is restarted visual verification. The method for verifying the visual positioning of a robot according to item 1 of the scope of patent application, wherein the set position is set at the working position of the robot in the work unit. The method for verifying the visual positioning of a robot as described in item 2 of the scope of the patent application, wherein the positioning mark is provided on a workbench in a work unit. The method for verifying the visual positioning of a robot according to item 3 of the scope of the patent application, wherein the positioning mark maintains a relatively fixed spatial position relationship with the working environment in the work unit. The method for verifying the visual positioning of a robot according to item 1 of the scope of patent application, wherein when correcting the positioning of the robot using image differences, the positioning mark image captured by the robot's reference attitude and the recorded reference positioning mark image are used for positioning in two images. Compare the features of the marks, and calculate the difference between the displacement and the rotation angle. The method for verifying the visual positioning of a robot according to item 5 of the scope of patent application, wherein the mobile robot is controlled according to the direction of reducing the difference amount, and the captured positioning mark image is searched for comparison, so that the captured positioning mark image is the same as the reference positioning mark image When the difference is less than the preset threshold, the robot's correction posture is recorded, the offset between the coordinates of the correction posture and the reference posture is calculated, and the robot coordinate offset is compensated to complete the robot's correction and positioning. According to the robot visual positioning verification method described in item 1 of the scope of the patent application, wherein the inspection image difference value is less than a preset difference threshold, the visual positioning verification operation is completed. The method for verifying the visual positioning of a robot according to item 1 of the scope of the patent application, wherein when the visual positioning verification is restarted, the number of times of re-calibrating the positioning is counted, and once the preset number of times is exceeded, an alarm is activated.