WO2021103824A1

WO2021103824A1 - Key point position determining method and device in robot hand-eye calibration based on calibration block

Info

Publication number: WO2021103824A1
Application number: PCT/CN2020/120103
Authority: WO
Inventors: 郑振兴; 刁世普
Original assignee: 广东技术师范大学
Priority date: 2019-11-26
Filing date: 2020-10-10
Publication date: 2021-06-03
Also published as: CN110930442A; CN110930442B

Abstract

A key point position determining method and device in robot hand-eye calibration based on a calibration block. The method comprises: enabling the projection of a connection line of any two points in key points on a three-dimensional calibration block on an XY plane to be not parallel to any coordinate axis of a robot base coordinate system by adjusting the placement posture of the three-dimensional calibration block; adjusting the posture of the robot, such that a three-dimensional vision system located at the tail end of the robot can obtain a three-dimensional calibration block point cloud containing the peripheral surface of the key points; then converting a CAD model of the three-dimensional calibration block into a three-dimensional calibration block model point cloud; then registering the three-dimensional calibration block model point cloud and the three-dimensional calibration block point cloud; finally, setting a threshold value to obtain point clouds near the key points from the three-dimensional calibration block point cloud so as to determine coordinate values of the key points in a three-dimensional vision system coordinate system. According to the method and device, the key points can be extracted conveniently with low cost and high precision, so that hand-eye calibration can be carried out in a robot vision system conveniently with low cost and high precision.

Description

Method and device for determining position of key points in robot hand-eye calibration based on calibration block

Technical field

The invention relates to the calibration of a vision guidance system in a robot automatic processing system, the calibration of a vision system in the detection of the position and related parameters of parts to be assembled in a robot automatic assembly system, and the conversion of target position information after a defect is obtained by analyzing sensor data in a processing center The technical field of hand-eye calibration for visual inspection system calibration and other automated processing (operation) processes such as visual guidance tasks in the automated field, specifically relates to a method and device for determining key point positions in robot hand-eye calibration based on a calibration block.

Background technique

Automated equipment is a weapon to make a powerful country. Therefore, it must move towards high speed and intelligence. An important method is to equip the machine with "eyes" and a "brain" that can cooperate with the eyes. This eye can be a monocular camera, a binocular camera, a multi-eye camera, a 3D scanner, or an RGB-D sensor. Through the visual sensor to obtain the relevant data, you can analyze the obtained processing information, where the processing information is defined in the coordinate system of the visual sensor, and the processing information must be transformed into the robot base coordinate system before being executed by the robot. Therefore, the calibration of the hand-eye relationship of the robot vision guidance system is very important.

At present, there are many hand-eye calibration methods for the eye-on-hand vision system, but for the robot dynamic 3D vision system, these existing calibration methods have either low calibration accuracy or high calibration cost (requiring expensive laser trackers, etc.) Equipment), and is not conducive to rapid calibration. Therefore, there is an urgent need for a low-cost, convenient, and high-precision hand-eye calibration method. Before hand-eye calibration, it is necessary to provide a method for extracting key points in the hand-eye calibration of the robot vision system to quickly perform hand-eye calibration. Method and device for extracting key points of three-dimensional calibration block in hand-eye calibration of robot vision system

Summary of the invention

In view of the deficiencies of the prior art, the present invention proposes a method and device for determining the position of key points in the hand-eye calibration of a robot based on a calibration block, which can extract key points with low cost, convenience, and high precision, thereby achieving low cost, convenience, and high precision. Accurately perform hand-eye calibration in the robot vision system.

The technical scheme of the present invention is realized as follows:

A method for determining the position of key points in the hand-eye calibration of a robot based on a calibration block, the calibration block is a three-dimensional calibration block, the three-dimensional calibration block has a polyhedral structure and an irregular shape, and the key points are on the three-dimensional calibration block and there are many At the three preset points, the preset points do not overlap in the height direction; the key point extraction method includes the following steps:

Step 1. Adjust the posture of the 3D calibration block so that the projection of any two points of the key points on the 3D calibration block on the XY plane is not parallel to any coordinate axis of the robot base coordinate system;

Step 2: Adjust the posture of the robot so that the three-dimensional vision system at the end of the robot can obtain the three-dimensional calibration block point cloud containing the peripheral surface of the key point on the three-dimensional calibration block;

Step 3: Convert the CAD model of the 3D calibration block into a point cloud to obtain the point cloud of the 3D calibration block model;

Step 4: Register the 3D calibration block model point cloud with the obtained 3D calibration block point cloud;

Step 5: Taking the position of the key points on the 3D calibration block model point cloud as the reference, set the corresponding threshold to obtain the point cloud near the key points from the 3D calibration block point cloud to determine that the key points on the 3D calibration block are in the 3D vision system coordinate system The coordinate value.

Further, step 3 includes the following sub-steps:

Step 301: Obtain the CAD model of the 3D calibration block and convert it into a PLY format file;

Step 302: According to the PLY format file, use the data format conversion function in the PCL library to convert it into a point cloud data format to obtain a three-dimensional calibration block model point cloud.

Further, step 4 includes the following sub-steps:

Step 401: Sampling the 3D calibration block point cloud and the 3D calibration block model point cloud respectively;

Step 402: Calculate the feature point descriptors of the three-dimensional calibration block point cloud and the three-dimensional calibration block model point cloud respectively to obtain respective fast point feature histograms;

Step 403, according to the fast point feature histogram of the 3D calibration block point cloud and the 3D calibration block model point cloud, perform rough registration on the point cloud by using a sampling consistent initial registration algorithm;

In step 404, the point cloud is accurately registered through the iterative closest point algorithm.

Further, in step 5, set the corresponding threshold to search for the closest point to the key point cloud on the 3D calibration block model point cloud from the 3D calibration block point cloud through the nearest neighbor search method, and determine that the coordinate value of the point is 3D The coordinate value of the key point on the calibration block in the coordinate system of the 3D vision system.

A device for determining the position of key points in the hand-eye calibration of a robot based on a calibration block, the calibration block is a three-dimensional calibration block, the three-dimensional calibration block has a polyhedral structure and an irregular shape, and the key points are on the three-dimensional calibration block and there are not many At three preset points, the preset points do not overlap in the height direction; the key point extraction device includes

The 3D calibration block posture adjustment module is used to adjust the placement posture of the 3D calibration block, so that the connection between any two points of the key points on the 3D calibration block is projected on the XY plane, and any coordinates in the robot base coordinate system The axis is not parallel;

The robot posture adjustment module is used to adjust the posture of the robot so that the three-dimensional vision system at the end of the robot can obtain the three-dimensional calibration block point cloud containing the peripheral surface of the key point on the three-dimensional calibration block;

The model point cloud conversion module is used to convert the CAD model of the 3D calibration block into a point cloud to obtain the 3D calibration block model point cloud;

The registration module is used to register the 3D calibration block model point cloud with the obtained 3D calibration block point cloud;

The key point coordinate determination module is used to set the corresponding threshold value based on the key point position on the point cloud of the 3D calibration block model to obtain the point cloud near the key point from the 3D calibration block point cloud to determine the key point on the 3D calibration block. The coordinate value of the coordinate system of the 3D vision system.

Further, the model point cloud conversion module includes

The PLY format file conversion unit is used to obtain the CAD model of the 3D calibration block and convert it into a PLY format file;

The model point cloud acquisition unit is used to convert the PLY format file into a point cloud data format by using the data format conversion function in the PCL library to acquire a three-dimensional calibration block model point cloud.

Further, the registration module includes a sampling unit, a fast point feature histogram unit, a rough configuration unit, and a precise registration unit, wherein

The sampling unit is used to sample the 3D calibration block point cloud and the 3D calibration block model point cloud respectively;

The fast point feature histogram unit is used to calculate the feature point descriptors of the 3D calibration block point cloud and the 3D calibration block model point cloud respectively to obtain their respective fast point feature histograms;

The coarse configuration unit is used to coarsely register the point cloud based on the 3D calibration block point cloud and the fast point feature histogram of the 3D calibration block model point cloud by using the sampling consistent initial registration algorithm;

The precise registration unit is used to accurately register the point cloud through the iterative closest point algorithm.

Further, the key point coordinate determination module is used to set a corresponding threshold to search for the closest point to the key point cloud on the 3D calibration block model point cloud from the 3D calibration block point cloud through the nearest neighbor search method, and determine the The coordinate value of the point is the coordinate value of the key point on the 3D calibration block in the coordinate system of the 3D vision system.

Compared with the prior art, the present invention has the following advantages: the present invention uses a three-dimensional calibration block with a polyhedral structure and irregular shape, and multiple key points on the three-dimensional calibration block do not overlap in the height direction, thereby being low-cost, convenient, and Determine the coordinate value of the key point in the robot vision system with high precision; specifically, by adjusting the placement posture of the three-dimensional calibration block, the connection between any two points of the multiple key points is projected on the XY plane, and the Any coordinate axis of the robot base coordinate system is not parallel; then by adjusting the robot's posture, the 3D vision system can obtain the point cloud of the peripheral surface of the key point; finally, the 3D calibration block model point cloud and the collected 3D The calibration block point cloud is registered, and the corresponding threshold is set to determine the point cloud near the key point, so as to obtain the coordinate value of the key point in the coordinate system of the three-dimensional vision system. According to the coordinate values of the key points in the robot base coordinate system and the coordinates of the key points in the coordinate system of the three-dimensional vision system, the transformation matrix of the hand-eye relationship of the robot dynamic three-dimensional vision system can be quickly solved, so as to achieve low-cost, convenient and high-precision realization Hand-eye calibration in robot 3D dynamic vision system.

Description of the drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor.

Figure 1 is a schematic diagram of the structure of a calibration block used in the present invention;

Figure 2 is a schematic diagram of the robot detection attitude adjustment of the present invention;

3 is a flow chart of an embodiment of the method for extracting key points in the calibration of the robot dynamic three-dimensional vision system according to the present invention;

4 is a structural block diagram of an embodiment of the device for extracting key points in the calibration of the robot dynamic three-dimensional vision system according to the present invention;

5 is a flowchart of step 3 in an embodiment of the method for extracting key points in the calibration of the robot dynamic three-dimensional vision system according to the present invention;

6 is a structural block diagram of a model point cloud conversion module in an embodiment of the method for extracting key points in the calibration of the robot dynamic three-dimensional vision system according to the present invention;

7 is a flowchart of step 4 in an embodiment of the method for extracting key points in the calibration of the robot dynamic 3D vision system according to the present invention;

FIG. 8 is a structural block diagram of a model point cloud registration module in an embodiment of the method for extracting key points in the calibration of the robot dynamic three-dimensional vision system of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

Figure 1 is a calibration block used in the present invention, wherein the calibration block is a three-dimensional calibration block, and the key points are points P1, P2, P3 in Figure 1; Figure 2 is a schematic diagram of the robot detection attitude adjustment of the present invention , After adjusting the placement posture of the three-dimensional calibration block, the detection posture of the robot is adjusted to determine the coordinate values of points P1, P2, and P3 in the coordinate system of the three-dimensional vision system. When the coordinate values of points P1, P2, and P3 in the robot base coordinate system are also determined, according to the coordinate values of points P1, P2, and P3 in the three-dimensional vision system coordinate system and the coordinates of points P1, P2, P3 in the robot base coordinate system The coordinate value can solve the transformation matrix of the hand-eye relationship of the robot dynamic 3D vision system, and realize the hand-eye calibration in the robot 3D dynamic vision system. A detailed description will be given below with reference to FIGS. 1 to 4.

Among them, the calibration block used in the embodiment of the present invention is a three-dimensional calibration block with a special shape, which is specifically expressed as: as shown in FIG. 1, the three-dimensional calibration block has a polyhedral structure and an irregular shape, and the key points are three-dimensional Points P1, P2, P3 on the calibration block, among which the key points P1, P2, P3 do not overlap in the height direction, and are basically evenly distributed in the height direction. In the embodiment of the present invention, the coordinate value of the key point in the coordinate system of the three-dimensional vision system is determined by the three-dimensional calibration block of this special structure.

Referring to Fig. 3, the embodiment of the present invention discloses a method for determining the position of key points in the hand-eye calibration of a robot based on a calibration block, which includes the following steps:

Step 1, as shown in Figure 2, adjust the placement posture of the 3D calibration block so that the connection of any two points of P1, P2, P3 on the 3D calibration block is projected on the XY plane, and the robot base coordinates Any coordinate axis of the system is not parallel;

Step 2, as shown in Figure 2, adjust the posture of the robot so that the 3D vision system at the end of the robot can obtain the 3D calibration block point cloud containing the peripheral surface of the P1, P2, and P3 points on the 3D calibration block;

Step 5. Take the key point positions on the point cloud of the three-dimensional calibration block model (ie P1', P2', P3' points, where P1' corresponds to P1, P2' corresponds to P2, and P3' corresponds to P3) as Benchmark, set the corresponding threshold to obtain the point cloud near the key point from the point cloud of the three-dimensional calibration block to determine the coordinate value of the key point on the three-dimensional calibration block in the coordinate system of the three-dimensional vision system.

Referring to Fig. 4, the embodiment of the present invention also discloses a device for determining the position of key points in the hand-eye calibration of a robot based on a calibration block, including:

The three-dimensional calibration block posture adjustment module 10 is used to adjust the placement posture of the three-dimensional calibration block, so that the line of any two points of P1, P2, P3 on the three-dimensional calibration block is projected on the XY plane, and the robot base Any coordinate axis of the coordinate system is not parallel;

The robot posture adjustment module 20 is used to adjust the posture of the robot so that the three-dimensional vision system at the end of the robot can obtain the three-dimensional calibration block point cloud including the peripheral surface of the P1, P2, and P3 points on the three-dimensional calibration block;

The model point cloud conversion module 30 is used to convert the CAD model of the three-dimensional calibration block into a point cloud to obtain the point cloud of the three-dimensional calibration block model;

The registration module 40 is used to register the three-dimensional calibration block model point cloud with the obtained three-dimensional calibration block point cloud;

The key point coordinate determination module 50 is used for three-dimensional calibration of the key point positions on the block model point cloud (ie P1', P2', P3' points, where P1' corresponds to P1, P2' corresponds to P2, and P3' Corresponding to P3) as the reference, set the corresponding threshold to obtain the point cloud near the key point from the point cloud of the three-dimensional calibration block to determine the coordinate value of the key point on the three-dimensional calibration block in the coordinate system of the three-dimensional vision system.

In the embodiment of the present invention, the method for determining the position of the key point in the hand-eye calibration of the robot based on the calibration block uses the device for determining the position of the key point in the hand-eye calibration of the robot based on the calibration block as the execution target of the steps. Among them, step 1 uses the 3D calibration block posture adjustment module 10 as the execution object of the steps, step 2 uses the robot posture adjustment module 20 as the execution object of the steps, and step 3 uses the model point cloud conversion module 30 as the execution object of the steps. Step 4 uses the registration module 40 as the execution object of the step, and step 5 uses the key point coordinate determination module 50 as the execution object of the step.

In the present invention, the determination of the coordinate values of the key points P1, P2, P3 in the coordinate system of the three-dimensional vision system and the coordinate values of the key points P1, P2, P3 in the robot base coordinate system are the key to the solution of the transformation matrix, and the key The coordinate values of points P1, P2, and P3 in the robot base coordinate system are quickly determined by the probe set at the end of the robot. Specifically, when P1, P2, and P3 are touched by the probe, the robot controller The coordinate value after length compensation can be the coordinate value of the key points P1, P2, P3 in the robot base coordinate system. Therefore, determining the coordinate values of the key points P1, P2, and P3 in the coordinate system of the three-dimensional vision system is the key to solving the transformation matrix. The embodiment of the present invention utilizes the key points on the three-dimensional calibration block with the aid of the three-dimensional calibration block with the polyhedral structure and irregular shape, so that the coordinates of the key points in the coordinate system of the three-dimensional vision system can be determined with low cost, convenience and high precision. In addition, the hand-eye calibration in the robot's three-dimensional dynamic vision system can be realized at low cost, conveniently and with high precision.

In step 1, the placement posture of the three-dimensional calibration block is related to whether the acquired data is available. Therefore, in the embodiment of the present invention, as shown in Figures 1 and 2, when adjusting the posture of the three-dimensional calibration block, the key The projection of the line connecting any two of the points P1, P2, and P3 on the XY plane is not parallel to any coordinate axis of the robot base coordinate system, so that the robot end can obtain the key points at the same time under the same detection posture. Point cloud data of each surface.

In step 2, the detection posture of the robot also needs to be adjusted so that the three-dimensional vision system, such as monocular camera, binocular camera, multi-eye camera, three-dimensional scanner, etc., can obtain usable spatial position data. As shown in Figure 2, during adjustment, the three-dimensional vision system installed at the end of the robot can simultaneously obtain the target position points P1, P2, P3 on the three-dimensional calibration block shown in Figure 1 under the same end detection posture. Point cloud on the peripheral surface.

Specifically, as shown in Figure 5, step 3 includes the following sub-steps:

Correspondingly, as shown in FIG. 6, the model point cloud conversion module 30 in the device for determining the position of the key point in the hand-eye calibration of the robot based on the calibration block includes

The PLY format file conversion unit 31 is used to obtain the CAD model of the three-dimensional calibration block and convert it into a PLY format file;

The model point cloud acquisition unit 32 is configured to convert the PLY format file into a point cloud data format by using the data format conversion function in the PCL library to acquire a three-dimensional calibration block model point cloud.

In the embodiment of the present invention, step 3 uses each unit in the model point cloud conversion module 30 as the execution target of the step. Specifically, step 301 uses the PLY format file conversion unit 31 as the execution object of the step, and step 302 uses the model point cloud acquisition unit 32 as the execution object of the step.

Specifically, as shown in Fig. 7, step 4 includes the following sub-steps:

Step 403, according to the 3D calibration block point cloud and the fast point feature histogram of the 3D calibration block model point cloud, perform rough registration on the point cloud by using a sampling consistent initial registration algorithm;

In step 404, the point cloud is accurately registered by using the iterative closest point algorithm.

Correspondingly, as shown in FIG. 8, the registration module 40 in the device for determining the position of the key point in the hand-eye calibration of the robot based on the calibration block includes

The sampling unit 41 is used to sample the three-dimensional calibration block point cloud and the three-dimensional calibration block model point cloud respectively;

The fast point feature histogram unit 42 is used to calculate the feature point descriptors of the three-dimensional calibration block point cloud and the three-dimensional calibration block model point cloud respectively to obtain respective fast point feature histograms;

The coarse configuration unit 43 is used to perform coarse registration on the point cloud by using the sampling consistent initial registration algorithm according to the fast point feature histogram of the 3D calibration block point cloud and the 3D calibration block model point cloud;

The precise registration unit 44 is used for precise registration of the point cloud through the iterative closest point algorithm.

In the embodiment of the present invention, step 4 uses each unit in the registration module 40 as the execution target of the step. Specifically, step 401 uses the sampling unit 41 as the execution object of the step, step 402 uses the fast point feature histogram unit 42 as the execution object of the step, step 403 uses the rough configuration unit 43 as the execution object of the step, and step 404 is The precise registration unit 44 is used as the execution target of the steps.

In step 401, the three-dimensional calibration block point cloud and the three-dimensional calibration block model point cloud can be sampled by using a Volgograd filter to improve the registration speed of the point cloud pair.

In step 402, the registration of the point cloud pair depends on the feature description. Therefore, in the present invention, it is necessary to calculate the feature point descriptors of the 3D calibration block point cloud and the 3D calibration block model point cloud separately to obtain respective fast point feature histograms (FPFH , Fast Point Feature Histograms);

In step 403, it is usually necessary to coarsely register the point cloud pair before accurately registering the point cloud pair. Therefore, in the present invention, the sample consensus initial registration algorithm (SAC-IA, Sample Consensus Initial Aligment) is used to realize the rough matching of the point cloud pair. quasi.

In step 404, after the rough registration of the point cloud pair, the iterative closest point algorithm (ICP, Iterative Closest Point) is used to realize the precise registration of the point cloud pair.

Correspondingly, the key point coordinate determination module 50 in the key point position determination device in the robot hand-eye calibration based on the calibration block is used to set the corresponding threshold to search for the distance 3D calibration from the 3D calibration block point cloud through the nearest neighbor search method. The closest point of the key point cloud on the block model point cloud, and the coordinate value of this point is determined to be the coordinate value of the key point on the 3D calibration block in the coordinate system of the 3D vision system.

In the embodiment of the present invention, the key point positions (ie P1', P2', P3' points on the point cloud of the block model are calibrated in three dimensions, where P1' corresponds to P1, P2' corresponds to P2, and P3' corresponds to P3. Correspondence) as the benchmark, through the nearest neighbor search method, from the 3D calibration block point cloud, search for the closest point to the key point on the 3D calibration block model point cloud (ie P1', P2', P3' point), The coordinate value of this point is the required key point coordinate value, that is, the coordinate value of the key points P1, P2, P3 in the coordinate system of the three-dimensional vision system.

In summary, the present invention uses a three-dimensional calibration block with a polyhedral structure and an irregular shape, and multiple key points on the three-dimensional calibration block do not overlap in the height direction, so as to determine the key points in the robot with low cost, convenience and high precision. The coordinate value in the vision system; specifically, by adjusting the placement posture of the three-dimensional calibration block, the connection of any two points of multiple key points is projected on the XY plane, and it is with any coordinate axis of the robot base coordinate system Non-parallel; then by adjusting the robot's posture, the 3D vision system can obtain the point cloud of the peripheral surface of the key point; finally, the 3D calibration block model point cloud is registered with the collected 3D calibration block point cloud, Set the corresponding threshold to determine the point cloud near the key point, so as to obtain the coordinate value of the key point in the coordinate system of the 3D vision system. According to the coordinate values of the key points in the robot base coordinate system and the coordinates of the key points in the coordinate system of the three-dimensional vision system, the transformation matrix of the hand-eye relationship of the robot dynamic three-dimensional vision system can be quickly solved, so as to achieve low-cost, convenient and high-precision realization Hand-eye calibration in robot 3D dynamic vision system.

In the description of the embodiments of the present invention, "plurality" means two or more, unless otherwise specifically defined.

The following disclosure provides many different embodiments or examples for realizing different structures of the embodiments of the present invention. In order to simplify the disclosure of the embodiments of the present invention, the components and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the embodiments of the present invention may repeat reference numbers and/or reference letters in different examples. This repetition is for the purpose of simplification and clarity, and does not indicate the relationship between the various embodiments and/or settings discussed. . In addition, the embodiments of the present invention provide examples of various specific processes and materials, but those of ordinary skill in the art may be aware of the application of other processes and/or the use of other materials.

Any process or method description in the flowchart or described in other ways herein can be understood as a module, segment or part of code that includes one or more executable instructions for implementing specific logical functions or steps of the process , And the scope of the preferred embodiment of the present invention includes additional implementations, which may not be in the order shown or discussed, including the functions involved in a substantially simultaneous manner or in the reverse order, which should be It is understood by those skilled in the art to which the embodiments of the present invention belong.

The logic and/or steps represented in the flowchart or described in other ways herein, for example, can be considered as a sequenced list of executable instructions for realizing logic functions, and can be embodied in any computer-readable medium, For use by instruction execution systems, devices, or equipment (such as computer-based systems, systems including processing modules, or other systems that can fetch and execute instructions from instruction execution systems, devices, or equipment), or combine these instruction execution systems, devices Or equipment. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transmit a program for use by an instruction execution system, device, or device or in combination with these instruction execution systems, devices, or devices. More specific examples (non-exhaustive list) of computer readable media include the following: electrical connections (electronic devices) with one or more wiring, portable computer disk cases (magnetic devices), random access memory (RAM), Read only memory (ROM), erasable and editable read only memory (EPROM or flash memory), fiber optic devices, and portable compact disk read only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program can be printed, because it can be used for example by optically scanning the paper or other medium, and then editing, interpreting, or other suitable media if necessary. The program is processed in a manner to obtain the program electronically, and then stored in the computer memory.

It should be understood that each part of the embodiments of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the foregoing embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if it is implemented by hardware, as in another embodiment, it can be implemented by any one or a combination of the following technologies known in the art: Discrete logic circuits, application-specific integrated circuits with suitable combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), etc.

A person of ordinary skill in the art can understand that all or part of the steps carried in the method of the foregoing embodiments can be implemented by a program instructing relevant hardware to complete. The program can be stored in a computer-readable storage medium, and the program can be stored in a computer-readable storage medium. When executed, it includes one of the steps of the method embodiment or a combination thereof.

In addition, the functional units in the various embodiments of the present invention may be integrated into one processing module, or each unit may exist alone physically, or two or more units may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or software functional modules. If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer readable storage medium.

The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

Although the embodiments of the present invention have been shown and described above, it can be understood that the above-mentioned embodiments are exemplary and should not be construed as limiting the present invention. A person of ordinary skill in the art can comment on the above-mentioned embodiments within the scope of the present invention. The embodiment undergoes changes, modifications, substitutions, and modifications.

Claims

A method for determining the position of key points in a robot hand-eye calibration based on a calibration block, wherein the calibration block is a three-dimensional calibration block, the three-dimensional calibration block has a polyhedral structure and an irregular shape, and the key points are a three-dimensional calibration block The above and not less than three preset points, the preset points do not overlap in the height direction; the key point extraction method includes the following steps:

Step 1. Adjust the posture of the 3D calibration block so that the projection of any two points of the key points on the 3D calibration block on the XY plane is not parallel to any coordinate axis of the robot base coordinate system;

Step 2: Adjust the posture of the robot so that the three-dimensional vision system at the end of the robot can obtain the three-dimensional calibration block point cloud containing the peripheral surface of the key point on the three-dimensional calibration block;

Step 3: Convert the CAD model of the 3D calibration block into a point cloud to obtain the point cloud of the 3D calibration block model;

Step 4: Register the 3D calibration block model point cloud with the obtained 3D calibration block point cloud;

Step 5: Taking the position of the key points on the 3D calibration block model point cloud as the reference, set the corresponding threshold to obtain the point cloud near the key points from the 3D calibration block point cloud to determine that the key points on the 3D calibration block are in the 3D vision system coordinate system The coordinate value.
The method for determining the position of key points in the hand-eye calibration of the robot based on the calibration block according to claim 1, wherein step 3 includes the following sub-steps:

Step 301: Obtain the CAD model of the 3D calibration block and convert it into a PLY format file;

Step 302: According to the PLY format file, use the data format conversion function in the PCL library to convert it into a point cloud data format to obtain a three-dimensional calibration block model point cloud.
The method for determining the position of key points in the hand-eye calibration of a robot based on a calibration block according to claim 1, wherein step 4 includes the following sub-steps:

Step 401: Sampling the 3D calibration block point cloud and the 3D calibration block model point cloud respectively;

Step 402: Calculate the feature point descriptors of the three-dimensional calibration block point cloud and the three-dimensional calibration block model point cloud respectively to obtain respective fast point feature histograms;

Step 403, according to the fast point feature histogram of the 3D calibration block point cloud and the 3D calibration block model point cloud, perform rough registration on the point cloud by using a sampling consistent initial registration algorithm;

In step 404, the point cloud is accurately registered through the iterative closest point algorithm.
The method for determining the position of key points in the hand-eye calibration of the robot based on the calibration block according to claim 1, wherein in step 5, corresponding thresholds are set to search for the distance 3D from the 3D calibration block point cloud through the nearest neighbor search method. The closest point of the key point cloud on the point cloud of the calibration block model is determined, and the coordinate value of this point is determined to be the coordinate value of the key point on the 3D calibration block in the coordinate system of the 3D vision system.
A device for determining the position of key points in the hand-eye calibration of a robot based on a calibration block, wherein the calibration block is a three-dimensional calibration block, the three-dimensional calibration block has a polyhedral structure and an irregular shape, and the key point is a three-dimensional calibration block Above and not less than three preset points, the preset points do not overlap in the height direction; the key point extraction device includes

The 3D calibration block posture adjustment module is used to adjust the placement posture of the 3D calibration block, so that the connection between any two points of the key points on the 3D calibration block is projected on the XY plane, and any coordinates in the robot base coordinate system The axis is not parallel;

The robot posture adjustment module is used to adjust the posture of the robot so that the three-dimensional vision system at the end of the robot can obtain the three-dimensional calibration block point cloud containing the peripheral surface of the key point on the three-dimensional calibration block;

The model point cloud conversion module is used to convert the CAD model of the 3D calibration block into a point cloud to obtain the 3D calibration block model point cloud;

The registration module is used to register the 3D calibration block model point cloud with the obtained 3D calibration block point cloud;

The key point coordinate determination module is used to set the corresponding threshold value based on the key point position on the point cloud of the 3D calibration block model to obtain the point cloud near the key point from the 3D calibration block point cloud to determine the key point on the 3D calibration block. The coordinate value of the coordinate system of the 3D vision system.
The device for determining the position of key points in the hand-eye calibration of the robot based on the calibration block according to claim 5, wherein the model point cloud conversion module comprises

The PLY format file conversion unit is used to obtain the CAD model of the 3D calibration block and convert it into a PLY format file;

The model point cloud acquisition unit is used to convert the PLY format file into a point cloud data format by using the data format conversion function in the PCL library to acquire a three-dimensional calibration block model point cloud.
The device for determining the position of key points in the hand-eye calibration of a robot based on a calibration block according to claim 5, wherein the registration module includes a sampling unit, a fast point feature histogram unit, a rough configuration unit, and a precise registration unit, wherein

The sampling unit is used to sample the 3D calibration block point cloud and the 3D calibration block model point cloud respectively;

The fast point feature histogram unit is used to calculate the feature point descriptors of the 3D calibration block point cloud and the 3D calibration block model point cloud respectively to obtain their respective fast point feature histograms;

The coarse configuration unit is used to coarsely register the point cloud based on the 3D calibration block point cloud and the fast point feature histogram of the 3D calibration block model point cloud by using the sampling consistent initial registration algorithm;

The precise registration unit is used to accurately register the point cloud through the iterative closest point algorithm.
The device for determining the position of key points in the hand-eye calibration of the robot based on the calibration block according to claim 5, wherein the key point coordinate determination module is used to set corresponding thresholds to separate the point cloud from the three-dimensional calibration block through the method of nearest neighbor search. Search for the closest point to the key point cloud on the 3D calibration block model point cloud, and determine that the coordinate value of this point is the coordinate value of the key point on the 3D calibration block in the coordinate system of the 3D vision system.