CN110706285A - Object pose prediction method based on CAD model - Google Patents

Abstract

The invention discloses an object pose prediction method based on a CAD model, and relates to the technical field of image processing methods. The method comprises the following steps: obtaining relevant parameters of the monocular camera through calibration, and generating data required by rough matching by using a CAD model; detecting and identifying an object in the image and outputting a mask of the image, and obtaining related outline information of the object through the mask of the object; and obtaining a rough matching pose of the object by combining the relevant contour information of the object with rough matching data, and then obtaining an accurate pose of the object by an iterative algorithm. The method can be used as an algorithm for detecting the pose of an object when the real-time requirement is not high, and has high detection precision and strong anti-interference performance.

Description

Object pose prediction method based on CAD model

technical field

The invention relates to the technical field of image processing methods, in particular to a CAD model-based object pose prediction method.

Background technique

Augmented Reality (AR) is based on computer graphics technology and visualization technology, adding and positioning virtual objects in three-dimensional space, which can integrate the information of real scenes and virtual scenes, and has real-time interactivity. Since the concept of induced maintenance based on augmented reality was put forward, the research of AR in the field of maintenance has gradually deepened. For example, robots with augmented reality technology need to obtain accurate three-dimensional pose information of objects in advance through visual information collected by cameras when performing tasks such as grasping and welding. In other aspects, it is necessary to use visual sensor information to pre-determine the three-dimensional pose of the object. At present, augmented reality sensors mainly rely on cameras, lidars, ultrasonic radars, etc. Among them, cameras are divided into monocular cameras and binocular cameras. Among them, binocular cameras have the problems of large size, heavy weight, high price, and easy damage. Ultrasonic radar has the problems of low accuracy, poor real-time performance, no occlusion, and easy to be affected by noise.

SUMMARY OF THE INVENTION

The technical problem to be solved by the present invention is how to provide a low-cost and accurate identification method for obtaining the pose of an object.

In order to solve the above-mentioned technical problems, the technical scheme adopted by the present invention is: a method for predicting the pose of an object based on a CAD model, which is characterized in that it comprises the following steps:

Obtain the relevant parameters of the monocular camera through calibration, and use the CAD model to generate the data required for rough matching;

Detect and identify objects in the image and output the mask of the image, and obtain the relevant contour information of the object through the mask of the object;

The rough matching pose of the object is obtained by combining the relevant contour information of the object with the rough matching data, and then the precise pose of the object is obtained through an iterative algorithm.

A further technical solution is that the method for obtaining the relevant parameters of the monocular camera by calibration includes the following steps:

Build the camera imaging model:

M is a three-dimensional space point, and m is the image point projected by M on the image plane. According to the relationship between the coordinate systems involved in the camera, the projection of the world coordinate system to the pixel coordinate can be obtained:

(1) can be written in the form of (2)

where a _x and a _y are the scale factors of the horizontal and vertical axes of the image respectively; K is the camera's internal parameter matrix; M ₁ contains the rotation matrix and translation vector, and the parameters in M ₁ are determined by the camera coordinate system relative to the world coordinate system The position is determined, so M ₁ is called the camera external parameter matrix; the product M of the internal parameter and the external parameter matrix is the projection matrix; X _W is the x-axis coordinate of the object center W in the world coordinate system, and Y _W is the world coordinate system. The y-axis coordinate where the object center W is located, and Z _W is the z-axis coordinate where the object center W is located in the world coordinate system;

The focal length of the camera is that the axis where f is located is the positive direction of z, the x and y axes are in the plane where the optical center O is located, and the optical center O is the origin of the camera coordinate system. In this camera coordinate system, the position of the object center is represented by W, where :

W=(W _x ,W _y ,W _z ) (3)

It is stipulated that the center of the object is the position of the center of the CAD model of the object. If P=(u, v) is the coordinate of the corresponding pixel of the object on the image, and K is the camera internal parameter matrix, this equation can be obtained:

This equation expresses the two-dimensional coordinate position P of the actual object center position W in the camera coordinate system after passing through the camera internal reference K and projected to the image.

A further technical solution is that the method for generating rough matching data by using the CAD model is as follows:

First, the mask of the object is rendered under the specified pose through the CAD model of the object, and the bounding box of the object is obtained through the mask of the object, and then the outline of the object is sampled at certain distances on the bounding box according to different needs;

Taking the length L of the left bounding box as the benchmark, divide L into n equal parts, and every L/n is a sampling abscissa point, traverse the points on each contour and calculate the left side when the abscissa is equal to the sampling abscissa point. The distance of the border, since each sampling abscissa point may correspond to multiple contour sampling distances, the maximum and minimum values of the multiple distances are taken as the sampling values of the contour sampling on the sampling abscissa, and the contour information is changed into a group. sample value;

Normalize the sampled values, that is, unify the length of the left bounding box to one unit;

At a specified distance, take the center of the CAD model of the object as the center, sample the contour of the object at different rotation angles, save the contour sampling information and the corresponding pose information, and obtain the rough matching template data of the object.

A further technical solution is that the method for detecting and recognizing objects in the image and outputting the mask of the image is as follows:

The Mask-RCNN neural network is used for image recognition, and the category of the object and the mask of the object are output.

A further technical solution is to use blender and Opencv software to automatically generate a dataset for training when training the Mask-RCNN neural network.

A further technical solution is that the coarse matching pose method is as follows:

The pose of the rigid body includes two parts: rotation R and displacement T. The matching process of the rotation part is as follows:

First, the output contour information is normalized and unified to the same scale for comparison;

If the actual mask sampling data for the object is S ⁱⁿ , the i-th group of data in the template data is S ⁱ , and each group has n sampling values, then calculate the L ₁ distance between the actual mask sampling data and each group of data in the template, and the first The L ₁ distance Li of the _i group of data is:

Ideally, the sampling values should be the same when the poses are the same, that is, the rotation angle at which this distance is 0 in the template data is the rotation angle corresponding to this contour, so take the minimum value of all results satisfying the threshold. The corresponding rotation angle is the rotation angle obtained by the current matching, and the matching fails if the threshold is not met;

During rough matching, the error is controlled at the Euler angle and the error of each degree of freedom is not greater than 12°; then the Euler angle information is converted into the rotation matrix R, that is, the rotation information of the object is obtained;

The translation algorithm is as follows:

When generating template data, since the object is sampled at a specified distance and the size of the CAD model is known, the size of the bounding box corresponding to the object is inversely proportional to its distance. If the knowledge is consistent, the distance D between the model center point and the camera optical center can be calculated by (5):

D=(w _in /w _i )·D _i (6)

where w _in is the width of the object recognition output bounding box, w _i is the bounding box width of the template data matching its rotation, D _i is the distance specified when the template data is collected, and D is the distance between the model center point and the camera optical center;

The prior information of the size of the CAD model is known, and the actual physical distance represented by each pixel in the template can be calculated, and then the displacement vector t _z of the object can be calculated:

Among them, t _x is the displacement of the object on the x-axis, and ty is the displacement of the object on the _y -axis. After the rotation R and displacement T of the object are obtained, the world coordinates of the object are obtained by combining the internal and external parameters of the camera.

A further technical solution is that, through an iterative algorithm, the method for obtaining the precise pose of the object is as follows:

If the rotation of the coarse matching object is A=(ψ, θ, φ), then on this basis, each coordinate axis is added or subtracted by an angle Δε, Δε is set to half of the coarse matching interval, and it is calculated in the coarse matching rotation space. Several angles of the adjacent space, using the several angles of the adjacent space combined with the CAD model to obtain the contour of the object, using the contour sampling method and combining the formula (5) to find the rotation A ₁ that _minimizes the value of Li in the formula (5) =(ψ ₁ , θ ₁ , φ ₁ ), that is, the rotation angle of the object after one iteration is obtained;

Then, by continuously halving the angle Δε to obtain a smaller range of angle values for iteration, the rotation angle that makes the formula (3) equal to 0 can be finally obtained;

Combined with the object translation information obtained during rough matching, the precise pose of the object is obtained.

The beneficial effect of adopting the above technical solution is that: the method described in the present application first obtains the relevant parameters of the camera through calibration, and uses the CAD model to generate the data required for rough matching, and then uses the deep neural network or other algorithms to detect and recognize objects in the image. And output the mask of the image. The relevant contour information can be obtained through the mask of the object. This contour information can be combined with the rough matching data to obtain the rough matching pose of the object, and then through the iterative algorithm, the precise pose of the object can be obtained.

Description of drawings

The present invention will be described in further detail below with reference to the accompanying drawings and specific embodiments.

Fig. 1 is the flow chart of the method described in the embodiment of the present invention;

Fig. 2 is the relational diagram of the coordinate system in the embodiment of the present invention;

3 is a schematic diagram of a camera pinhole model in an embodiment of the present invention;

4 is a result diagram of object contour sampling in an embodiment of the present invention;

Fig. 5 is the Mask-RCNN image segmentation effect diagram in the embodiment of the present invention;

Fig. 6 is the Mask-RCNN image recognition result diagram in the embodiment of the present invention;

Fig. 7 is the position and attitude comparison diagram after rough matching and iteration in the embodiment of the present invention;

FIG. 8 is an accuracy diagram of the pose of an object under occlusion in an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Many specific details are set forth in the following description to facilitate a full understanding of the present invention, but the present invention can also be implemented in other ways different from those described herein, and those skilled in the art can do so without departing from the connotation of the present invention. Similar promotion, therefore, the present invention is not limited by the specific embodiments disclosed below.

As shown in FIG. 1 , an embodiment of the present invention discloses a method for predicting the pose of an object based on a CAD model, including the following steps:

First, the relevant parameters of the camera are obtained through calibration, and the CAD model is used to generate the data required for rough matching, and then the deep neural network or other algorithms are used to detect and identify the objects in the image and output the mask of the image, and the relevant contour can be obtained through the mask of the object. The contour information can be combined with the rough matching data to obtain the rough matching pose of the object, and then through the iterative algorithm, the precise pose of the object can be obtained.

The above methods are described in detail below:

Camera imaging model:

M is a three-dimensional space point, and m is an image point projected by M on the image plane. According to the relationship between the coordinate systems involved in the camera, the projection of the world coordinate system to the pixel coordinate can be obtained (the coordinate system relationship is shown in Figure 2):

(1) can be written in the form of (2)

where a _x and a _y are the scale factors of the horizontal and vertical axes of the image respectively; K contains the camera's internal parameters such as focal length and principal point coordinates, so K is called the internal parameter matrix; M ₁ contains the rotation matrix and translation vector, M ₁ The middle parameter is determined by the position of the camera coordinate system relative to the world coordinate system, so it is called the external parameter matrix of the camera; the product M of the internal parameter and the external parameter matrix is called the projection matrix. By comparing Equation (1) and Equation (2), it is easy to determine the specific expressions of the camera's internal and external parameters represented by these matrices. Camera calibration is to determine the camera's internal parameters and external parameters.

Camera calibration:

Assuming that the camera is a pinhole model, as shown in Figure 3, the pose of an object is a general term for the position and pose of the object.

The focal length of the camera is that the axis where f is located is the positive direction of z, the x and y axes are in the plane where the optical center O is, and the optical center O is the origin of the camera coordinate system. In this camera coordinate system, the position of the object center can be represented by W, in:

W=(W _x ,W _y ,W _z ) (3)

It is stipulated that the center of the object is the position of the center of the CAD model of the object, which is generally the center of the volume. If P=(u, v) is the coordinate of the corresponding pixel of the object on the image, and K is the camera's internal parameter matrix, this equation can be obtained:

This equation expresses the two-dimensional coordinate position P of the actual object center position W in the camera coordinate system after passing through the camera internal reference K and projected to the image.

Therefore, in order to obtain the three-dimensional coordinate position of the object, the camera internal parameter K must be calibrated. There are many calibration methods. In this application, the camera calibration method provided by the Opencv software is used to obtain K.

Use the CAD model to generate coarse matching data:

The core algorithm in using object CAD model to generate template data is the sampling algorithm based on object outline. First, the mask of the object can be rendered under the specified pose through the CAD model of the object, and the bounding box of the object can be obtained through the mask of the object, and then the outline of the object can be sampled at certain distances on the bounding box according to different needs, as shown in Figure 4 It is shown that the object contour is sampled every certain distance in the left border.

Based on the frame length L of the left border (other borders are similar), divide L into n equal parts, and every L/n is a sampling abscissa point, traverse the points on each contour whose abscissa is equal to the sampling abscissa point When calculating the distance to the left border, since each sampling abscissa point may correspond to multiple contour sampling distances, the maximum and minimum values of the multiple distances are taken as the sampling value for contour sampling on this sampling abscissa. This turns the contour information into a set of sampled values.

Since the outline may change in size, the sampling value needs to be normalized, that is, the length of the left bounding box is unified to one unit. In the experiment, the unified length of the left bounding box is 128px, which can not only ensure the accuracy of sampling, Sampling speed can also be guaranteed.

The advantage of the above sampling method is to obtain a set of features of the contour, that is, the sampling value. This value has scaling invariance to the contour, but is sensitive to the rotation of the object, and the data dimensions are consistent, which is convenient for comparison.

At a specified distance, take the center of the CAD model of the object as the center, sample the contour of the object at different rotation angles, and save the contour sampling information and the corresponding pose information to obtain the rough matching template data of the object.

Image recognition output object mask:

At present, the most prominent method of image recognition is to use deep neural network, and Mask-RCNN is a model with better effect in image recognition using deep neural network. The effect is shown in Figure 5. After the deep neural network model is trained, The category of the object and the mask of the object can be output with high precision in real time, so this method uses this model as the processing module of the image detection end.

With the continuous development of neural networks, the performance of different algorithms and deep neural network models will definitely surpass Mask-RCNN. This algorithm can be applied to any algorithm or deep neural network model that outputs masks or contours, that is, it can be used as a general-purpose algorithm. solution. When training the Mask-RCNN neural network, blender and Opencv software are used to automatically generate data sets for training, and the recognition accuracy is high.

Rough matching algorithm:

The pose of the rigid body includes two parts: rotation R and displacement T. The matching process of the rotation part is as follows:

Since the resolution of the object mask output by different frameworks is different, if the object resolution is too low, it will affect the data quality collected by the sampling algorithm. If the object resolution is too high, the sampling speed will decrease during sampling, so the same as the sampling algorithm, first of all The output contour information is normalized and unified to the same scale for comparison.

If the actual mask sampling data for the object is S ⁱⁿ , the i-th group of data in the template data is S ⁱ , and each group has n sampling values, then calculate the L ₁ distance between the actual mask sampling data and each group of data in the template, and the first The L ₁ distance Li of the _i group of data is:

Ideally, the sampling values should be the same when the pose is the same, that is, the rotation angle at which the distance is 0 in the template data is the rotation angle corresponding to the contour. In practice, if the angle is divided too finely, a large number of Data, the matching is too slow, so take the rotation angle corresponding to the minimum value of all results satisfying the threshold as the rotation angle obtained by the current matching, and the matching fails if the threshold is not met. In order to ensure the matching speed, during rough matching, the error is controlled to be no more than 12° in each degree of freedom of the Euler angle (that is, 360° is divided into 30 equal parts for sampling to generate a rough matching template). Then, the Euler angle information can be converted into the rotation matrix R, that is, the rotation information of the object can be obtained.

The translation algorithm is as follows:

When generating template data, since the object is sampled at a specified distance and the size of the CAD model is known, the size of the bounding box corresponding to the object is inversely proportional to its distance. If the knowledge is consistent, the distance between the model center point and the camera optical center can be calculated by (5):

D=(w _in /w _i )·D _i (6)

where w _in is the width of the object recognition output bounding box (can also be calculated by using the frame length), w _i is the bounding box width of the template data matching its rotation, D _i is the distance specified when the template data is collected, and D is the model center The distance of the point from the camera's optical center.

Similarly, because the prior information of the size of the CAD model is known, the actual physical distance represented by each pixel in the template can be calculated, and then the displacement vector of the object can be calculated:

In the actual experiment, since the sub-pixel level information cannot be obtained, when only pixels are used for calculation, the displacement vector error is large. This problem can be solved by increasing the resolution of the camera image, that is, the higher the camera resolution, the better the object. The more accurate the position. After the rotation R and displacement T of the object are obtained, the world coordinates of the object can be obtained by combining the internal and external parameters of the camera.

Iterative Algorithms:

After the rough matching pose of the object is obtained, there is still an error of less than 12° in the rotation of the object in theory. In order to eliminate this error, an iterative algorithm is introduced. This algorithm calculates the object rotation information obtained by the rough matching, and finally obtains the error as Rotation information for zero (when float is set to 8 decimal places)

If the rotation of the coarse matching object is A=(ψ, θ, φ), then on this basis, each axis adds or subtracts a small angle Δε. Since the coarse matching interval was previously set to 12°, Δε is set as the coarse matching interval The half of φ is 6°, so the 26 angles of the adjacent space are obtained in the coarse matching rotation space, and the contour of the object is obtained by combining these 26 angles with the CAD model, and the contour sampling method is combined with the formula (5) to obtain ( 5) The rotation A ₁ =( _{ψ 1} , θ ₁ , φ ₁ ) with the smallest value of formula Li, that is, the rotation angle of the object after one iteration is obtained.

Then by continuously halving Δε to obtain a smaller range of angle values for iteration, we can finally get the rotation angle that makes equation (3) 0 (when the floating-point number is set to 8 decimal places), which can be set by setting the floating-point number of the computer. Get more accurate rotation information.

Combined with the object translation information obtained during rough matching, the precise 6Dof position of the object is obtained.

Experimental data:

The experimental environment is configured as: the notebook is Lenovo Y7000, the system is ubuntul6.04, and the programming language is python3.6.

Object recognition accuracy: The mask data input by this method adopts the mask output by the Mask-RCNN neural network. Due to the strong performance of the Mask-RCNN neural network itself, it has been trained with a self-developed dataset to achieve a relatively ideal recognition accuracy. The requirements of this method can be met, as shown in Figure 6.

Rotation accuracy: The rotation accuracy of rough matching is determined by the generated rough matching data template. In this experiment, each degree of freedom of Euler angle is divided into 30 equal parts, so the accuracy is not greater than 12° (360°/30). The highest precision of 8-bit floating-point numbers is reached after 6 iterations. Figure 7 is a comparison of the effect of the mask image after rough matching and iteration, where from left to right are the random pose mask image (ie input) of the object, the object rendering image obtained by using the pose after rough matching, and the two difference graph.

It has been proved by experiments that the rotation error after iteration is 0 when it is an 8-bit floating point number, and the green error in Fig. 5 is caused by the position error.

Position accuracy: Since the position of this method is calculated by the bounding box of the object, the accuracy is limited by the pixel accuracy. In extreme cases, such as the far object is smaller, the bounding box is proportionally reduced, so that the bounding box is one pixel difference, and the position error is It increases a lot, so the position accuracy of the object depends on the camera pixel. The higher the camera pixel, the smaller the bounding box error, and the higher the object position accuracy.

After experiments, when the camera resolution is 512x512 pixels, the position accuracy of the 4x4x3(cm) object in this method changes with the position of the object from the camera, as shown in Table 1:

Table 1 The relationship between position error and distance

Object and camera distance (mm) Error(mm) 500 0-5 1000 2-12 2000 10-100 5000 >100

Comparison with other related pose methods:

Compared with the representative methods such as SSD-6D and BB8 in the current neural network, when the evaluation standard is the current general standard 2Dprojection, 5cm5° or 6Dpose, the rotation accuracy of this method is close to 100%, far exceeding For other types of algorithms, the main source of error is the position error. After considering that the cause of the position error depends on the resolution and accuracy of the camera image, it is considered that there is no comparison with other methods.

There is a big gap in real-time performance. This method takes about 0.6s to detect a rough matching of a picture in the above-mentioned personal notebook environment, and the average time after iteration is about 40-60s. The general neural network-based 6Dof pose method can basically achieve real-time (>20fps), and the algorithm-based 6Dof pose method is more representative that Linemod can basically reach 15-18fps.

This method is more prominent in anti-interference ability. As long as the object contour sampling is basically correct, the absence of the object mask has little effect on the pose estimation of this method, as shown in Figure 8.

In summary, the above method can be used as a general algorithm for object pose detection when real-time requirements are not high. It has high detection accuracy and strong anti-interference performance. In practical applications, it can be considered by using C++ code and parallelism. The calculation improves its real-time performance to meet the usage requirements.

Claims (7)

1. An object pose prediction method based on a CAD model is characterized by comprising the following steps:

obtaining relevant parameters of the monocular camera through calibration, and generating data required by rough matching by using a CAD model;

detecting and identifying an object in the image and outputting a mask of the image, and obtaining related outline information of the object through the mask of the object;

and obtaining a rough matching pose of the object by combining the relevant contour information of the object with rough matching data, and then obtaining an accurate pose of the object by an iterative algorithm.

2. The CAD model-based object pose prediction method of claim 1, wherein the method for obtaining monocular camera related parameters by calibration comprises the steps of:

constructing a camera imaging model:

m is a three-dimensional space point, M is an image point projected by M on an image plane, and the projection of a world coordinate system to a pixel coordinate can be obtained according to the relation between coordinate systems related to a camera:

can write (1) into (2)

Wherein a is_x，a_yScale factors for the horizontal and vertical axes of the image, respectively; k is a camera internal parameter matrix;M₁containing a rotation matrix and a translation vector, M₁The medium parameter is determined by the position of the camera coordinate system relative to the world coordinate system, hence the name M₁Is a camera extrinsic parameter matrix; the product M of the internal parameter matrix and the external parameter matrix is a projection matrix; x_WIs the x-axis coordinate, Y, of the center W of the object in the world coordinate system_WIs the y-axis coordinate, Z, of the center W of the object in the world coordinate system_WIs the z-axis coordinate of the object center W in the world coordinate system;

the focal length of the camera is f, the axis is the positive z direction, the x and y axes are on the plane of the optical center O, the optical center O is used as the origin of the camera coordinate system, and the position of the center of the object is represented by W in the camera coordinate system, wherein:

W＝(W_x,W_y,W_z) (3)

specifying the object center as the position of the object CAD model center, and if P ═ u (u)_,v) is the coordinates of the corresponding pixels of the object on the image, and K is the camera internal reference matrix, then the equation can be obtained:

this equation represents the two-dimensional coordinate position P of the actual object center position W after passing through the camera intrinsic parameters K and being projected onto the image in the camera coordinate system.

3. The CAD model-based object pose prediction method of claim 1, wherein the method of generating coarse match data using a CAD model is as follows:

firstly, rendering a mask of an object under a specified pose through an object CAD model, obtaining a boundary frame of the object through the mask of the object, and then sampling the outline of the object at intervals on the boundary frame according to different requirements;

dividing L into n equal parts by taking the length L of a left frame as a reference, taking every L/n as a sampling abscissa point, traversing points on each contour, and calculating the distance from the points to the left frame when the abscissa of the points is equal to the sampling abscissa point;

normalizing the sampling value, namely unifying the lengths of the left boundary frames to a unit;

and sampling the contour of the object at different rotation angles by taking the center of the object CAD model as the center at the specified distance, and storing the contour sampling information and the corresponding pose information to obtain rough matching template data of the object.

4. The CAD model-based object pose prediction method of claim 1, wherein the method of detecting masks that identify objects in an image and output an image is as follows:

and performing image recognition by using a Mask-RCNN neural network, and outputting the category of the object and the Mask of the object.

5. The CAD model based object pose prediction method of claim 4, wherein during training of the Mask-RCNN neural network, a dataset is automatically generated by a blend and Opencv software for training.

6. The CAD model-based object pose prediction method of claim 1, wherein the coarse matching pose method is as follows:

the pose of the rigid body comprises a rotation part R and a displacement part T, and the matching process of the rotation part is as follows:

firstly, normalizing the output contour information, unifying the output contour information to the same scale for comparison;

if the sampling data of the actual shade of the object is SⁱⁿThe ith group of data in the template data is SⁱAnd each group has n sampling values, calculating L of each group of actual mask sampling data and template₁Distance, L of the ith group of data₁Distance L_iComprises the following steps:

ideally, the sampling values should be consistent when the positions are the same, that is, the rotation angle with the distance of 0 in the template data is the rotation angle corresponding to the contour, so that the rotation angle corresponding to the minimum value in all the results when the threshold is met is taken as the rotation angle obtained by current matching, and the threshold is not met to consider that the matching fails;

in coarse matching, the error is controlled to be not more than 12 degrees in each degree of freedom of the Euler angle; then, the Euler angle information is converted into a rotation matrix R, and the rotation information of the object is obtained;

the algorithm of the translation part is as follows:

when generating the template data, since the object is sampled at a predetermined distance and the CAD model size is known, the size of the bounding box corresponding to the object is inversely proportional to the distance thereof, that is, the distance between the center point of the model and the optical center of the camera can be found by (5) if the distance becomes longer as the bounding box becomes smaller, which is consistent with the recognition by the human eye:

D＝(w_in/w_i)·D_i(6)

wherein w_inOutput bounding box width, w, for object identification_iBounding box width of template data matched to its rotation, D_iThe distance is specified when the template data is collected, and D is the distance between the center point of the model and the optical center of the camera;

the prior information of the size of the CAD model is known, so that the actual physical distance represented by each pixel in the template can be calculated, and the displacement vector t of the object can be further calculated_z：

Wherein, t_xIs an object in xAmount of displacement of the shaft, t_yThe world coordinate of the object is obtained by combining internal and external parameters of the camera after the rotation R and the displacement T of the object are obtained.

7. The CAD model-based object pose prediction method of claim 6, wherein the exact pose of the object is obtained by an iterative algorithm as follows:

if the coarse matching object rotation is a ═ phi (psi, theta, phi), then, on the basis of the coarse matching object rotation, each coordinate axis is added and subtracted by an angle delta epsilon, the delta epsilon is set to be half of the coarse matching interval, a plurality of angles of adjacent spaces of the coarse matching object rotation are obtained in the coarse matching rotation space, the outlines of the objects are obtained by combining a CAD model by the plurality of angles of the adjacent spaces, and the outlines are obtained by combining the formula (5) by a contour sampling method, so that the formula (5) L is obtained_iRotation A of minimum value₁＝(ψ₁，θ₁,φ₁) Obtaining the rotation angle of the object after one iteration;

then, continuously halving the angle delta epsilon to obtain an angle value in a smaller range for iteration, and finally obtaining a rotation angle with the formula (3) being 0;

and combining the translation information of the object obtained in the rough matching process to obtain the accurate pose of the object.

Images