JP2022516852A - Robot visual guidance method and device by integrating overview vision and local vision - Google Patents

Abstract

The robot visual guidance method by integrating overview vision and local vision is S10: machining target data collection, S20: overall machining guide path, S30: sub-target area division, S40: optimum machining path planning, S50: machining guide point conversion. , S60: High-precision local collection of point cloud, S70: High-precision machining guidance information is included. The robot visual guidance device by integrating the overview visual and the local visual is a machining target data acquisition module 10, an overall machining guide path module 20, a sub-target area division module 30, an optimum machining path planning module 40, and a machining guide point conversion module 50. , Point cloud high precision local collection module 60, high precision machining guidance information module 70 is included. While satisfying the accuracy requirements for high-efficiency machining of large volume machining targets, the requirements for real-time machining can be met by reducing the amount of calculation and complexity, accelerating the processing speed, and reducing the calculation time. In addition, software hardware performance requirements are reduced, reducing cost and development difficulty. It can be adapted to the demands of high-speed and large-scale production modes.

Description

The present invention relates to the visual area of the robot, and more particularly to a robot visual guidance method and apparatus by integrating overview vision and local vision.

Automation equipment (robot systems), which form the basis of a powerful country, must move in the direction of speeding up and becoming intelligent. An important means of intelligentizing automation equipment is to equip the machine with "eyes" and "brain" that fits these eyes. This "eye" can be a monocular camera, a binocular camera, a multi-eye camera, a three-dimensional scanner, or an RGB-D (RGB + Depth) sensor. The core work of smartening automation equipment includes analyzing image data acquired by this "eye" (eg image recognition) and guiding the robot system to a specific machining or assembly operation based on the analysis results. As machining technology has advanced, the surfaces of parts that need to be machined have become more complex and require more precision. Therefore, surface treatment (sharpening work) of parts is an indispensable and important process. In order to realize automation and intelligent surface treatment for machined parts, the above "eyes" are used to acquire images of the parts to be machined, and the above "brain".
The analysis process is performed using, and the spatially accurate positioning for the machined part is realized, and the target is detected accurately. In addition, we will guide the tool to sharpen the edge of the robot and work on the machining target of the machined part. Conventionally, a robot vision guide with a large field of view has been adopted, but
Detection accuracy is usually not high and machining accuracy cannot be required. In order to obtain highly accurate spatial positioning information, the robot vision guide needs to set a smaller detection field of view, so block division detection is required for large machining targets, so the calculation complexity is high and the amount of calculation is high. I also need a lot. The calculation time is long and the work efficiency of the entire system is not high. again,
The demands on the performance of the above-mentioned "brain" software and hardware are high, it is difficult to achieve real-time processing, and it does not meet the needs of the current high-speed industrial production process. Therefore, the present invention discloses a highly accurate visual guidance method and apparatus for a polishing toss robot for a large volume machining target that combines global vision and local vision, and is highly efficient for the large volume machining target. Satisfies the precision requirements that can be processed.

The present invention provides a robot visual guidance method and device by integrating overview vision and local vision, and adopts an existing robot visual guidance with a large detection field of view. Therefore, the detection accuracy is not usually high, and processing accuracy is required. Although it cannot be achieved, it is necessary to set the robot vision in order to obtain highly accurate spatial positioning information, so block division detection is required for large machining targets, and the calculation is highly complicated. In addition, a large amount of calculation is required, the calculation time is long, the work efficiency of the entire system is not high, and the performance requirements of software hardware are high.
It is difficult to achieve real-time processing, and it solves these technical problems required in the current accelerated industrial production process.

In order to solve the above problem, the robot visual guidance method by integrating the overview vision and the local vision
Step 1:
The whole RGB- _D composite sensor D70 installed in the terminal of the detection robot acquires the whole registration RGB two-dimensional screen I _RGB and the whole registration depth data ID of the machining target, and the whole registration RGB two-dimensional screen I _RGB . Obtain the entire region S _RGB of the processing target O10 from, and use the calibration matrix of the overall RGB- _D composite sensor to obtain the overall registration depth data ID according to the overall region S _RGB .
Steps to extract the overall 3D point cloud data S _3D of the processing target O10 from
Step 2:
Overall 3D point cloud data S By analyzing _3D , the set point {AX _j } _{j = 1-> n} of the overall machining guide path of the machining target is to be obtained, but AX respectively.
_j is the overall 3D point cloud data of the machining target, the point of the overall machining guide path in _S3D , j is the point of the overall machining guide path AX _j number, the range of the value of j is [1, n], and n is the whole. Steps to make the total number of points AX _j of the machining guide path, and
Step 3:
Overall 3D point cloud data S of the machining target by the high-precision detection parameter of the predetermined machining sub-target region based on the set point {AX _j } _{j = 1-> n} of the entire machining guide path.
_3D is divided into a point cloud collection {S _3D-j } _{j = 1-> n} , which is a sub-target area requiring machining, except that S _3D-j requires machining of the point AX _j of the entire machining guide path. Steps corresponding to the point cloud of the sub-target area,
Step 4:
Point cloud collection of sub-target areas requiring machining { _S3D by optimal route planning algorithm
_-J } By ordering _{j = 1-> n} , a point cloud array {S _3D-i } _{i = 1-> n} of the optimum machining required sub-target region is generated, but S _3D respectively.
_-I is the point cloud array of the optimum machining sub-target area {S _3D-i } _{i = 1-> n} point cloud of the machining required sub-target area, i is the point cloud array of the optimum machining sub-target area {S _3D-i } _{i = 1-> n} numbers, i and j correspond one by one, and the set point {AX _j } _{j = 1-> n} of the whole machining guide path is totally machined according to the one-to-one correspondence relationship between i and j. Point cloud array of guide path {AX _i } _{i = 1-> n} . And the steps to convert to
Step 5:
Assuming that i is 1 to n, the point AX _i of the entire machining guide path corresponding to the point cloud _S3D-i of the machining required sub-target region is converted into the point BX _i of the overall machining guide path of the detection robot-based coordinate system. Then, the point BX _i of the whole machining guide path of the detection robot-based coordinate system is polished. By converting the point CX _i of the whole machining guide path of the robot-based coordinate system, the point array of the whole machining guide path {AX _i } _{i = 1 -> Detect n} Point cloud array of machining guides in robot-based coordinate system {BX _i
} _{I = 1-> n} , and the point array of the machining guide of the detection robot-based coordinate system {BX _i } _{i =}
The step of converting _1- > n to the point array {CX _i } _{i = 1-> n} of the entire machining guide of the polishing robot base coordinate system,
Step 6:
Assuming that i is 1 to n, the machining target is scanned while guiding the local RGB-D composite sensor installed at the terminal of the polishing robot according to the point CX _i of the entire machining guide path of the polishing robot-based coordinate system. , The step to acquire the high-precision local point cloud SS _3D-i in the corresponding area of the point cloud S _3D-i corresponding to the machining required sub-target area of the machining required target.
Step 7:
Assuming that i is 1 to n, the high-precision local point cloud SS _{3D-i is used as a module to register with the high-precision local point cloud SS 3D-i from} the sample point cloud RS _3D scheduled by the registration algorithm. Search point cloud RS _3D-i and high precision local point cloud SS _3D
-I and the corresponding difference between the high-precision local point cloud RS _3D-i Point cloud D
Steps to calculate S _3D-i , analyze and search the difference point cloud DS _3D-i , and acquire high-precision 3D machining guidance information for polishing robots,
including.

Preferably, the optimal route planning algorithm in step 4 is an annealing simulated intelligence optimization algorithm.

Preferably, the registration algorithm in step 7 is an iterative recent contact algorithm based on a normal distribution transformation.

Preferably, the method of the corresponding difference point cloud DS _3D-i between the calculated high-precision local point cloud SS _3D-i and the high-precision local point cloud RS _3D-i in step 7 is a quick nearest neighbor search algorithm.

Preferably, the high-precision detection parameter of the predetermined machining required sub-target region in step 3 is
Includes field size, detection accuracy, and detection distance for high-precision detection.

The robot visual guidance device 1 by integrating the overview vision and the local vision is
The entire RG installed in the terminal of the detection robot, which is the processing target data collection module 10.
The overall registration RGB two-dimensional screen I _RGB and the overall registration depth data ID of the machining target are acquired by the _BD composite sensor D70, and the entire region S of the machining target O10 is acquired from the overall registration RGB two-dimensional screen I _RGB . It is configured to acquire _RGB and extract the overall 3D point cloud data S _3D of the machining target O10 from the overall registration depth data ID according to the overall area S _RGB by the calibration matrix of the overall RGB- _D composite sensor. Machining target data collection module 10
When,
The overall machining guide path module 20 is configured to acquire the set point {AX _j } _{j = 1-> n} of the overall machining guide path of the machining target by analyzing the overall 3D point cloud data S _3D . However, AX _j is the point of the overall machining guide path in the overall 3D point cloud data S _3D of the machining target, and j is the point of the overall machining guide path AX.
The number of _j , the range of the value of j is [1, n], and n is the total number of points AX _j of the entire machining guide path.
The sub-target area division module 30 is a set point of the entire machining guide path {AX _j } _{j = 1} .
Based _{on-> n} , the overall 3D point cloud data S _3D of the machining target area is converted into the point cloud collection of the machining sub-target area {S _3D-j } _j by the high-precision detection parameter of the predetermined machining-required sub-target region. _{= 1-> n} , but it is configured to be divided into S _3D-
j is a sub-target area division module 30 corresponding to a point cloud of a sub-target area requiring machining of AX _j , which is a point of the entire machining guide path, and
In the optimum machining route planning module 40, the point cloud collection {S _3D-j } _{j = 1-> n} of the optimum machining sub-target region is ordered by the optimum route planning algorithm to obtain the optimum machining sub-target region. It is configured to generate a point cloud array {S _3D-i } _{i = 1-> n} , where each S _3D-i is a point cloud array {S _3D-i } _{i =} of the optimum machining required sub-target area. _{1-> n} point cloud of the required machining sub-target area, i is the point cloud array of the optimum machining sub-target area {S _3D-i } _{i = 1-
> N} numbers, i and j correspond one by one, and the set point of the whole machining guide path {AX _j } _{j = 1-> n} is the point array of the whole machining guide path {AX _i . } _{I = 1-> n} .
Optimal machining path planning module 40 to convert to
In the machining guide point conversion module 50, assuming that i is 1 to n, the point AX _i of the entire machining guide path corresponding to the point cloud _S3D-i in the machining required sub-target region is detected. By converting the point BX i of the guide path to the point BX _i of the whole machining guide path of the detection robot-based coordinate system and converting the point BX _i of the whole machining guide path of the detection robot-based coordinate system to the point CX _i of the whole machining guide path of the polishing robot-based coordinate system, the whole machining guide path. Point cloud {AX _i } _{i =} 1-> n is converted to the point cloud of the processing guide of the detection robot-based coordinate system {BX _i } _{i = 1-> n} , and the point cloud of the processing guide of the detection robot-based coordinate system A machining guide point conversion module 50 configured to convert {BX _i } _{i = 1-> n} into a point cloud of the entire machining guide of the polishing robot-based coordinate system {CX _i } _{i = 1-> n} .
In the high-precision local collection module 60 of the point cloud, assuming that i is 1 to n,
By scanning the machining target while guiding the local RGB-D composite sensor installed in the terminal of the polishing robot according to the point CX _i of the entire machining guide path of the polishing robot base coordinate system, the machining target sub of the machining target Point cloud S _3D-i corresponding to the target area
High-precision local point cloud of the point cloud configured to acquire the high-precision local point cloud SS _3D-i of the corresponding area of the point cloud high-precision local collection module 60,
including.

Preferably, the optimum route planning algorithm of the optimum route planning module 40 is an annealing simulated intelligence optimization algorithm.

Preferably, the registration algorithm of the high-precision machining guidance information module 70 is an iterative recent contact algorithm based on a normal distribution transformation.

Preferably, the calculation of the high-precision machining guidance module 70 and the corresponding difference between the high-precision local point cloud SS _3D-i and the high-precision local point cloud RS _{3D- i The method of the point cloud DS 3D-i} is the quick nearest neighbor search algorithm. do.

Preferably, the high-precision detection parameters of the predetermined machining-required sub-target region of the sub-target region division module 30 include the size of the field of view for high-precision detection, the detection accuracy, and the detection distance.

As described above, the present invention satisfies the accuracy requirement in the high-efficiency machining of the large-volume machining target and the calculation amount by the high-precision visual guide technology plan by the robot of the large-volume machining target by integrating the overview vision and the local vision. And the demands of real-time processing can be met by reducing computational complexity, accelerating processing speeds, and reducing computational time. In addition, software hardware performance requirements are reduced, reducing cost and development difficulty. It can be adapted to the demands of high-speed and large-scale production modes.

It is a flowchart of 1st Embodiment of the robot visual guidance method by integrating the overview vision and the local vision by this invention. FIG. 3 is a functional block diagram of a first embodiment of a robot visual guidance device that integrates overview vision and local vision according to the present invention. It is a schematic diagram of the RGB-D composite sensor which carries out this invention. It is a schematic diagram of the robot which realizes this invention.

The realization, functional features and advantages of the present invention will be further described with reference to the drawings in connection with embodiments.

It should be understood that the specific embodiments described herein are used only to illustrate the invention and not to limit the invention.

Hereinafter, mobile terminals that implement various embodiments of the present invention will be described with reference to the drawings. In later discussion, suffixes such as "module", "component", or "unit" to represent an element are used solely to favor the description of the invention and have a particular meaning in their own right. do not have. Therefore, "modules" and "parts" can be mixed and used.

FIG. 1 is a schematic flow chat diagram of a first embodiment of a robot visual guidance method by integrating overview vision and local vision according to the present invention. As shown in FIG. 1, the robot visual guidance method by integrating the overview vision and the local vision includes the following steps.

S10: Processing target data collection, that is, the entire registration RGB two-dimensional screen I _RGB and the entire registration depth data ID of the processing target are acquired by the entire RGB- _D composite sensor D70 installed in the terminal of the detection robot, and the entire registration is performed. Registration RGB Two-dimensional screen I Obtain the entire area S _RGB of the processing target O10 from _RGB , and use the calibration matrix of the overall RGB- _D composite sensor to obtain the processing target from the entire registration depth data ID according to the entire area S _RGB . To extract the overall 3D point cloud data S _3D of O10,
As shown in FIG. 3, the RGB-D composite sensor D70 is installed at the top end of the arm D40 of the detection robot, and the RGB camera D20 is installed at an intermediate position of the RGB-D composite visual sensor. The color screen data is to be compressed before being transmitted to the computer to guarantee the speed of analysis on the RGB data, and RGB.
-Sensors D10 and D30 on both the left and right sides of the D compound visual sensor are supposed to transmit and receive infrared rays, respectively. First, the infrared transmitter D10 on the left side emits infrared rays to the processing target O10, and here the infrared rays are transmitted. Since infrared rays have high randomness, they form a three-dimensional "code" with respect to the environment because the light spots formed by being reflected at any two different positions in the space are also different, and then the infrared rays on the right side. Depth data in the field of view is collected by the receiver D30 and finally by performing a series of complex calculations on the infrared screen by the calibration matrix of the RGB-D composite visual sensor D70. To get and include.

S20: Overall machining guide path, that is, by analyzing the overall 3D point cloud data S _3D , the set point {AX _j } _{j = 1-> n} of the overall machining guide path of the machining required target is obtained. However, AX _j is the point of the overall machining guide path in the overall _3D point cloud data S3D of the machining target, j is the number of the point AX _j of the overall machining guide path, and the range of the value of j is [1, n]. , N is the total number of points AX _j of the entire machining guide path.
AX _i corresponds to the coordinate vector of the overall 3D point cloud data S _3D , and the set point of the overall machining guide path {AX _j } _{j = 1-> n} is the set of all AX _i of the overall 3D point cloud data S _3D . To correspond to.

S30: Sub-target region division, that is, based on the set point {AX _j } _{j = 1-> n} of the entire machining guide path, the overall 3D of the machining target is determined by the high-precision detection parameter of the predetermined machining-required sub-target region. The point cloud data S _3D is divided into the point cloud collection {S _3D-j } _{j = 1-> n} in the sub-target area requiring machining, except that S _3D-j is the point AX of the entire machining guide path. Correspond to the point cloud of the sub-target area requiring machining of _j .

S40: Optimal machining path planning, that is, a point cloud collection of sub-target areas requiring machining by the optimal path planning algorithm {
By ordering S _3D-j } _{j = 1-> n} , a point cloud array {S _3D-i } _{i = 1-> n} of the optimum machining required sub-target region is generated, but each S _3D-i is a point cloud array of the optimum sub-target area requiring machining {S _3D-i } _{i = 1->}
Of _n , the point cloud of the required machining sub-target area, i is the point cloud array of the optimum machining sub-target area {S _3D-i } _{i = 1-> n} numbers, i and j correspond one by one, and i and j
The set point {AX _j } _{j = 1-> n} of the entire machining guide path is set to the point array {AX _i } _{i = 1-> n} of the entire machining guide path according to the one-to-one correspondence relationship. To convert to
The order of the point cloud sub-machining area S _3D-j of the machining-required sub-target region by the point cloud array {S _3D-i } _{i = 1-> n} of the optimum machining-required sub-target region generated based on the optimum route planning algorithm. In other words, the total working time of the polishing robot D50 is minimized by allowing the polishing robot D50 to pass through all the regions of the machining target O20 without repeating.

S50: Machining guide point conversion That is, assuming that i is 1 to n, the point AX _i of the entire machining guide path corresponding to the point cloud _S3D-i of the machining required sub-target region is detected. Point BX
Convert to _i , and polish the point BX _i of the whole machining guide path of the detection robot-based coordinate system. By converting the point CX _i of the whole machining guide path of the robot-based coordinate system, the point array of the whole machining guide path {AX _i } _{i = 1-> n} detected Point array of machining guides in the robot-based coordinate system {
BX _i } _{i = 1-> n} is converted, and the point array of the processing guide of the detection robot-based coordinate system {BX _i
} _{I = 1-> n} is polished Point array of the entire machining guide of the robot-based coordinate system {CX _i } _{i = 1-}
To convert to _{> n}
As shown in FIG. 4, the detection robot D detects the point AX _i of the entire machining guide path corresponding to the point cloud S _3D-i in the machining required sub-target O30 region acquired from the arm D40 of the detection robot.
40 Converts to the point BX _i of the whole machining guide path of the base coordinate system, and converts the point BX _i of the whole machining guide path of the detection robot D40 base coordinate system to the point CX _i of the whole machining guide path of the polishing robot D50 base coordinate system. The polishing tool D is converted and further guided to the terminal of the polishing robot D50.
Have 60 do the rest of the work.

S60: High-precision local collection of point cloud, that is, if i is 1 to n, the point CX _i of the entire machining guide path of the polishing robot-based coordinate system.
Corresponding to the point cloud _S3D-i corresponding to the machining required sub-target area of the machining target by scanning the machining target while guiding the local RGB-D composite sensor installed in the terminal of the polishing robot according to the above. Area high precision local point cloud SS _{3D
-Obtain i} .

S70: High-precision machining guidance information, that is, if i is 1 to n, the high-precision local point cloud SS _3D-i will be used as a module by the registration algorithm. Sample point cloud RS ₃
Search for high-precision local point cloud RS _3D-i that registers with high-precision local point cloud SS _3D-i from _D , and high-precision local point cloud S
Calculate the corresponding difference point cloud DS _3D-i between S _3D-i and high-precision local point cloud RS _3D- i, analyze and search the difference point cloud DS _3D-i , and obtain the high-precision 3D machining guidance information of the polishing robot. To acquire.

Therefore, the processing step is applied to the processing target O20 having a large volume so as to integrate the overview visual system and the local visual system. First, the visual system D70 of the detection robot D40 performs rough positioning of the machining target O20, and realizes the area division and route planning of the machining target O20. Next, the target is accurately detected by the high-precision visual detection system on the polishing robot D50. And the polishing robot D
While guiding the tool D60 at the terminal of 50, high-precision and high-efficiency automated polishing work is performed on the machining target area O30. This makes it possible to meet the accuracy requirements for high-efficiency machining of large volume machining targets, as well as meet the requirements for real-time machining by reducing computational complexity and complexity, accelerating processing speeds, and reducing computational time. .. In addition, software hardware performance requirements are reduced, reducing cost and development difficulty. It can be adapted to the demands of high-speed and large-scale production modes.

The optimum route planning algorithm in step S40 is an annealing simulated intelligence optimization algorithm (s).
Simulated annealing algorithm, SAA). The annealing simulated intelligence optimization algorithm has the advantages of being reliable and easy to realize the process.

The registration algorithm in step S70 is an iterative recent contact (I) based on normal distribution transformation.
The algorithm is a tertiary Closet Point (ICP) algorithm. The iterative recent contact algorithm based on the normal distribution conversion has the advantages of being reliable and easy to realize the process.

Calculation of step S70 Corresponding difference between high-precision local point cloud SS _3D-i and high-precision local point cloud RS _{3D- i The method of point cloud DS 3D-i} is a quick nearest neighbor search algorithm. The quick nearest neighbor search algorithm has the advantages of being reliable and easy to realize the process.

The high-precision detection parameters of the predetermined machining-required sub-target region in step S30 include the size of the field of view for high-precision detection, the detection accuracy, and the detection distance. The parameters have the advantages of easy measurement and acquisition and high reliability.

In the present invention, the robot visual guidance method by integrating the overview visual and the local visual in the first embodiment of the robot visual guidance method by integrating the overview visual and the local visual is the overview visual and the local according to the present invention. It can be realized by the robot visual guidance device by integrating the overview visual and the local visuals proposed in the first embodiment of the robot visual guidance device by integrating the visuals.

As shown in FIG. 2, the robot visual guidance device 1 by integrating the overview vision and the local vision proposed in the first embodiment of the robot visual guidance device by integrating the overview vision and the local vision has the following modules. including.

The entire RG installed in the terminal of the detection robot, which is the processing target data collection module 10.
The overall registration RGB two-dimensional screen I _RGB and the overall registration depth data ID of the machining target are acquired by the _BD composite sensor D70, and the entire region S of the machining target O10 is acquired from the overall registration RGB two-dimensional screen I _RGB . It is configured to acquire _RGB and extract the overall 3D point cloud data S _3D of the machining target O10 from the overall registration depth data ID according to the overall area S _RGB by the calibration matrix of the overall RGB- _D composite sensor. And
As shown in FIG. 3, the RGB-D composite sensor D70 is installed at the top end of the arm D40 of the detection robot, the RGB camera D20 is installed at the intermediate position of the RGB-D composite visual sensor, and the color screen data is , Configured to be compressed before being transmitted to the computer to ensure the speed of analysis on RGB data, the sensors D10 and D30 on both the left and right sides of the RGB-D composite visual sensor transmit and receive infrared rays, respectively. First, the infrared transmitter D10 on the left side emits infrared rays to the processing target O10, where the infrared rays have high randomness and are reflected at any two different positions in the space. Since the light spots formed are also different, they form a three-dimensional "code" for the environment, then the infrared receiver D30 on the right collects the infrared screen in the visual field, and finally,
A module configured to acquire depth data in the field of view by performing a series of complex calculations on the infrared screen with the calibration matrix of the RGB-D composite visual sensor D70.

The overall machining guide path module 20 is configured to acquire the set point {AX _j } _{j = 1-> n} of the overall machining guide path of the machining target by analyzing the overall 3D point cloud data S _3D . However, AX _j is the point of the overall machining guide path in the overall 3D point cloud data S _3D of the machining target, and j is the point of the overall machining guide path AX.
The range of the number of _j and the value of j is [1, n], and n is the total number of points AX _j of the entire machining guide path.
AX _i corresponds to the coordinate vector of the overall 3D point cloud data S _3D , and the set point of the overall machining guide path {AX _j } _{j = 1-> n} is the set of all AX _i of the overall 3D point cloud data S _3D . The corresponding module.

The sub-target area division module 30 is a set point of the entire machining guide path {AX _j } _{j = 1} .
Based _{on-> n} , the overall 3D point cloud data S _3D of the machining target area is converted into the point cloud collection of the machining sub-target area {S _3D-j } _j by the high-precision detection parameter of the predetermined machining-required sub-target region. _{= 1-> n} , but it is configured to be divided into S _3D-
j is a module corresponding to the point cloud of the point AX _j of the entire machining guide path, which is the sub-target area requiring machining.

In the optimum machining route planning module 40, the point cloud collection {S _3D-j } _{j = 1-> n} of the optimum machining sub-target region is ordered by the optimum route planning algorithm to obtain the optimum machining sub-target region. It is configured to generate a point cloud array {S _3D-i } _{i = 1-> n} , where each S _3D-i is a point cloud array {S _3D-i } _{i =} of the optimum machining required sub-target area. _{1-> n} point cloud of the required machining sub-target area, i is the point cloud array of the optimum machining sub-target area {S _3D-i } _{i = 1-
> N} numbers, i and j correspond one by one, and the set point of the whole machining guide path {AX _j } _{j = 1-> n} is the point array of the whole machining guide path {AX _i . } _{I = 1-> n} .
To convert to
The order of the point cloud sub-machining area S _3D-j of the machining-required sub-target region by the point cloud array {S _3D-i } _{i = 1-> n} of the optimum machining-required sub-target region generated based on the optimum route planning algorithm. A module that minimizes the total working time of the polishing robot D50 by matching the above to the desired robot processing process, that is, by passing the polishing robot D50 through all the regions of the machining target O20 without repeating.

In the machining guide point conversion module 50, assuming that i is 1 to n, the point AX _i of the entire machining guide path corresponding to the point cloud _S3D-i in the machining required sub-target region is detected. By converting the point BX i of the guide path to the point BX _i of the whole machining guide path of the detection robot-based coordinate system and converting the point BX _i of the whole machining guide path of the detection robot-based coordinate system to the point CX _i of the whole machining guide path of the polishing robot-based coordinate system, the whole machining guide path. Point cloud {AX _i } _{i =} 1-> n is converted to the point cloud of the processing guide of the detection robot-based coordinate system {BX _i } _{i = 1-> n} , and the point cloud of the processing guide of the detection robot-based coordinate system It is configured to convert {BX _i } _{i = 1-> n} into a point cloud {CX _i } _{i = 1-> n} of the entire machining guide of the polishing robot-based coordinate system.
As shown in FIG. 4, the detection robot D detects the point AX _i of the entire machining guide path corresponding to the point cloud S _3D-i in the machining required sub-target O30 region acquired from the arm D40 of the detection robot.
40 Converts to the point BX _i of the whole machining guide path of the base coordinate system, and converts the point BX _i of the whole machining guide path of the detection robot D40 base coordinate system to the point CX _i of the whole machining guide path of the polishing robot D50 base coordinate system. The polishing tool D is converted and further guided to the terminal of the polishing robot D50.
A module configured to allow 60 to perform subsequent tasks.

In the high-precision local collection module 60 of the point cloud, assuming that i is 1 to n,
By scanning the machining target while guiding the local RGB-D composite sensor installed in the terminal of the polishing robot according to the point CX _i of the entire machining guide path of the polishing robot base coordinate system, the machining target sub of the machining target Point cloud S _3D-i corresponding to the target area
A module configured to acquire the high precision local point cloud SS _3D-i in the corresponding area of.

If i is 1 to n in the high-precision machining guidance information module 70, the high-precision local point cloud SS _3D-i will be used as a module by the registration algorithm. High-precision local point cloud from the sample point cloud RS _3D . SS
Search for high-precision local point cloud RS _{3D- i} that registers with 3D-i, and high-precision local point cloud SS _3D-i and high-precision local point cloud R.
A module configured to calculate the corresponding difference point cloud DS _3D- i of S _3D-i , analyze and search the difference point cloud DS _3D-i , and acquire the high-precision 3D machining guidance information of the polishing robot.
Therefore, the processing module is adapted for the machining target O20 with a large volume to integrate the overview visual system and the local visual system. First, the visual system D70 of the detection robot D40 performs rough positioning of the machining target O20, and realizes the area division and route planning of the machining target O20. Next, the polishing robot D
The target is accurately detected by the high-precision visual detection system on the 50. Then, while guiding the tool D60 at the terminal of the polishing robot D50, highly accurate and highly efficient automated polishing work is performed on the machining target area O30. This makes it possible to meet the accuracy requirements for high-efficiency machining of large volume machining targets, as well as meet the requirements for real-time machining by reducing computational complexity and complexity, accelerating processing speeds, and reducing computational time. .. In addition, software hardware performance requirements are reduced, reducing cost and development difficulty. It can be adapted to the demands of high-speed and large-scale production modes.

The optimum route planning algorithm of the optimum route planning module 40 is simulated annealing algorithm (SAA).
). The annealing simulated intelligence optimization algorithm has the advantages of being reliable and easy to realize the process.

The registration algorithm of the high-precision machining guidance information module 70 is an iterative recent contact algorithm (Iterative Closest Point, based on normal distribution transformation).
ICP). The iterative recent contact algorithm based on the normal distribution conversion has the advantages of being reliable and easy to realize the process.

High-precision machining guidance Calculation of information module 70 High-precision local point cloud SS _3D-i
And high precision local point cloud RS _3D-i corresponding difference point cloud DS ₃
The method of _Di is a quick nearest neighbor search algorithm. The quick nearest neighbor search algorithm has the advantages of being reliable and easy to realize the process.

The high-precision detection parameters of the predetermined machining-required sub-target region of the sub-target region division module 30 include the size of the field of view for high-precision detection, the detection accuracy, and the detection distance. Parameters are easy to measure and obtain,
Moreover, it has the advantage of high reliability.

It is necessary to explain that, in the present specification, the terms "include", "include" or any other variant are used.
A process, method, article or appliance that comprises a set of elements is intended to include not only those elements, but also other elements that are not explicitly listed, or such a process,. Includes elements specific to methods, devices, objects and devices. If there are no more restrictions,
An element defined by the phrase "contains one ..." does not preclude the existence of the same other element in the process, method, article, or device that contains that element.

The above-mentioned number of the embodiment of the present invention does not represent the superiority or inferiority of the embodiment only for the sake of explanation.

Those skilled in the art will appreciate that each module unit or each step of the invention described above can optionally be implemented by an executable program code. The steps illustrated or described may be performed in a different order than here, or may be created for each integrated circuit module, or multiple modules or steps of them may be combined into a single integrated circuit module. It may be created and realized. As such, the invention is not limited to any particular hardware and software combination.

According to the above description of the embodiment, the method of the above-described embodiment may be realized by the general-purpose hardware platform required for Software Plus, and of course it is possible by the hardware, but in many cases, the former is better. It became clear that it was an embodiment. Based on this understanding, the technical aspects of the present invention essentially or contribute to existing technology in a single terminal device (mobile phone, computer, server, air conditioner, etc.).
It can be embodied as a software product stored in one storage medium, including several instructions to enable. Alternatively, the network device, etc.) implements the methods according to various embodiments of the present invention.

The above is merely a preferred embodiment of the present invention, and does not limit the scope of the patent of the present invention. It is indirectly applied to other related technical fields, and all of them are included in the claims of the present invention.

Claims (10)

It is a robot visual guidance method that integrates overview vision and local vision.
Step 1:
The whole RGB- _D composite sensor D70 installed in the terminal of the detection robot acquires the whole registration RGB two-dimensional screen I _RGB and the whole registration depth data ID of the machining target, and the whole registration RGB two-dimensional screen I _RGB . Obtain the entire region S _RGB of the processing target O10 from, and use the calibration matrix of the overall RGB- _D composite sensor to obtain the overall registration depth data ID according to the overall region S _RGB .
Steps to extract the overall 3D point cloud data S _3D of the processing target O10 from
Step 2:
Overall 3D point cloud data S By analyzing _3D , the set point {AX _j } _{j = 1-> n} of the overall machining guide path of the machining target is to be obtained, but AX respectively.
_j is the overall 3D point cloud data of the machining target, the point of the overall machining guide path in _S3D , j is the point of the overall machining guide path AX _j number, the range of the value of j is [1, n], and n is the whole. Steps to make the total number of points AX _j of the machining guide path, and
Step 3:
Overall 3D point cloud data S of the machining target by the high-precision detection parameter of the predetermined machining sub-target region based on the set point {AX _j } _{j = 1-> n} of the entire machining guide path.
_3D is divided into a point cloud collection {S _3D-j } _{j = 1-> n} , which is a sub-target area requiring machining, except that S _3D-j requires machining of the point AX _j of the entire machining guide path. Steps corresponding to the point cloud of the sub-target area,
Step 4:
Point cloud collection of sub-target areas requiring machining { _S3D by optimal route planning algorithm
_-J } By ordering _{j = 1-> n} , a point cloud array {S _3D-i } _{i = 1-> n} of the optimum machining required sub-target region is generated, but S _3D respectively.
_-I is the point cloud array of the optimum machining sub-target area {S _3D-i } _{i = 1-> n} point cloud of the machining required sub-target area, i is the point cloud array of the optimum machining sub-target area {S _3D-i } _{i = 1-> n} numbers, i and j correspond one by one, and the set point {AX _j } _{j = 1-> n} of the whole machining guide path is totally machined according to the one-to-one correspondence relationship between i and j. Point cloud array of guide path {AX _i } _{i = 1-> n} . And the steps to convert to
Step 5:
Assuming that i is 1 to n, the point AX _i of the entire machining guide path corresponding to the point cloud _S3D-i of the machining required sub-target region is converted into the point BX _i of the overall machining guide path of the detection robot-based coordinate system. Then, the point BX _i of the whole machining guide path of the detection robot-based coordinate system is polished. By converting the point CX _i of the whole machining guide path of the robot-based coordinate system, the point array of the whole machining guide path {AX _i } _{i = 1 -> Detect n} Point cloud array of machining guides in robot-based coordinate system {BX _i
} _{I = 1-> n} , and the point array of the machining guide of the detection robot-based coordinate system {BX _i } _{i =}
The step of converting _1- > n to the point array {CX _i } _{i = 1-> n} of the entire machining guide of the polishing robot base coordinate system,
Step 6:
Assuming that i is 1 to n, the machining target is scanned while guiding the local RGB-D composite sensor installed at the terminal of the polishing robot according to the point CX _i of the entire machining guide path of the polishing robot-based coordinate system. , The step to acquire the high-precision local point cloud SS _3D-i in the corresponding area of the point cloud S _3D-i corresponding to the machining required sub-target area of the machining required target.
Step 7:
Assuming that i is 1 to n, the high-precision local point cloud SS _{3D-i is used as a module to register with the high-precision local point cloud SS 3D-i from} the sample point cloud RS _3D scheduled by the registration algorithm. Search point cloud RS _3D-i and high precision local point cloud SS _3D
-I and the corresponding difference between the high-precision local point cloud RS _3D-i Point cloud D
Steps to calculate S _3D-i , analyze and search the difference point cloud DS _3D-i , and acquire high-precision 3D machining guidance information for polishing robots,
A robot visual guidance method that integrates overview vision and local vision, characterized by including. The robot visual guidance method by integrating overview vision and local vision according to claim 1, wherein the optimum route planning algorithm in step 4 is an annealing simulated intelligence optimization algorithm. The robot visual guidance method by integrating overview vision and local vision according to claim 1, wherein the registration algorithm of step 7 is an iterative recent contact algorithm based on a normal distribution transformation. The calculation of step 7 Corresponding difference between the high-precision local point cloud SS _3D-i and the high-precision local point cloud RS _{3D- i The method of the point cloud DS 3D-i} is a quick nearest neighbor search algorithm. Item 1. The robot visual guidance method by integrating the overview vision and the local vision according to Item 1. The overview vision and the overview visual according to claim 1, wherein the high-precision detection parameter of the predetermined machining-required sub-target region in step 3 includes the size of the field of view for high-precision detection, the detection accuracy, and the detection distance. Robot visual guidance method by integrating local vision. A robot visual guidance device 1 that integrates overview vision and local vision.
The entire RG installed in the terminal of the detection robot, which is the processing target data collection module 10.
The overall registration RGB two-dimensional screen I _RGB and the overall registration depth data ID of the machining target are acquired by the _BD composite sensor D70, and the entire region S of the machining target O10 is acquired from the overall registration RGB two-dimensional screen I _RGB . It is configured to acquire _RGB and extract the overall 3D point cloud data S _3D of the machining target O10 from the overall registration depth data ID according to the overall area S _RGB by the calibration matrix of the overall RGB- _D composite sensor. Machining target data collection module 10
When,
The overall machining guide path module 20 is configured to acquire the set point {AX _j } _{j = 1-> n} of the overall machining guide path of the machining target by analyzing the overall 3D point cloud data S _3D . However, AX _j is the point of the overall machining guide path in the overall 3D point cloud data S _3D of the machining target, and j is the point of the overall machining guide path AX.
The number of _j , the range of the value of j is [1, n], and n is the total number of points AX _j of the entire machining guide path.
The sub-target area division module 30 is a set point of the entire machining guide path {AX _j } _{j = 1} .
Based _{on-> n} , the overall 3D point cloud data S _3D of the machining target area is converted into the point cloud collection of the machining sub-target area {S _3D-j } _j by the high-precision detection parameter of the predetermined machining-required sub-target region. _{= 1-> n} , but it is configured to be divided into S _3D-
j is a sub-target area division module 30 corresponding to a point cloud of a sub-target area requiring machining of AX _j , which is a point of the entire machining guide path, and
In the optimum machining route planning module 40, the point cloud collection {S _3D-j } _{j = 1-> n} of the optimum machining sub-target region is ordered by the optimum route planning algorithm to obtain the optimum machining sub-target region. It is configured to generate a point cloud array {S _3D-i } _{i = 1-> n} , where each S _3D-i is a point cloud array {S _3D-i } _{i =} of the optimum machining required sub-target area. _{1-> n} point cloud of the required machining sub-target area, i is the point cloud array of the optimum machining sub-target area {S _3D-i } _{i = 1-
> N} numbers, i and j correspond one by one, and the set point of the whole machining guide path {AX _j } _{j = 1-> n} is the point array of the whole machining guide path {AX _i . } _{I = 1-> n} .
Optimal machining path planning module 40 to convert to
In the machining guide point conversion module 50, assuming that i is 1 to n, the point AX _i of the entire machining guide path corresponding to the point cloud _S3D-i in the machining required sub-target region is detected. By converting the point BX i of the guide path to the point BX _i of the whole machining guide path of the detection robot-based coordinate system and converting the point BX _i of the whole machining guide path of the detection robot-based coordinate system to the point CX _i of the whole machining guide path of the polishing robot-based coordinate system, the whole machining guide path. Point cloud {AX _i } _{i =} 1-> n is converted to the point cloud of the processing guide of the detection robot-based coordinate system {BX _i } _{i = 1-> n} , and the point cloud of the processing guide of the detection robot-based coordinate system A machining guide point conversion module 50 configured to convert {BX _i } _{i = 1-> n} into a point cloud of the entire machining guide of the polishing robot-based coordinate system {CX _i } _{i = 1-> n} .
In the high-precision local collection module 60 of the point cloud, assuming that i is 1 to n,
By scanning the machining target while guiding the local RGB-D composite sensor installed in the terminal of the polishing robot according to the point CX _i of the entire machining guide path of the polishing robot base coordinate system, the machining target sub of the machining target Point cloud S _3D-i corresponding to the target area
High-precision local point cloud of the point cloud configured to acquire the high-precision local point cloud SS _3D-i of the corresponding area of the point cloud high-precision local collection module 60,
A robotic visual guidance device with integrated overview vision and local vision, characterized by including. The robot visual guidance device by integrating the overview vision and the local vision according to claim 6, wherein the optimum route planning algorithm of the optimum route planning module 40 is an annealing simulated intelligence optimization algorithm. The robot visual guidance device by integrating the overview vision and the local vision according to claim 6, wherein the registration algorithm of the high-precision machining guidance information module 70 is an iterative recent contact algorithm based on a normal distribution transformation. High-precision machining guidance Calculation of information module 70 High-precision local point cloud SS _3D-i
And high precision local point cloud RS _3D-i corresponding difference point cloud DS ₃
The robot visual guidance device according to claim 6, wherein the method of _Di is a quick nearest neighbor search algorithm, which integrates the overview vision and the local vision. 6. The high-precision detection parameter of a predetermined sub-target region requiring machining of the sub-target region division module 30 includes the size of the field of view for high-precision detection, the detection accuracy, and the detection distance, claim 6.
A robot visual guidance device that integrates overview vision and local vision as described in.

Images