CN113077518B

CN113077518B - Camera parameter calibration method, device and storage medium

Info

Publication number: CN113077518B
Application number: CN202110278012.0A
Authority: CN
Inventors: 魏晓林
Original assignee: China Mobile Communications Group Co Ltd; China Mobile Shanghai ICT Co Ltd; CM Intelligent Mobility Network Co Ltd
Current assignee: China Mobile Communications Group Co Ltd; China Mobile Shanghai ICT Co Ltd; CM Intelligent Mobility Network Co Ltd
Priority date: 2021-03-15
Filing date: 2021-03-15
Publication date: 2022-02-11
Anticipated expiration: 2041-03-15
Also published as: WO2022193636A1; CN113077518A

Abstract

The application discloses a camera parameter calibration method, which comprises the following steps: determining the pose of the elliptical calibration plate based on the integral inclination quantization vector of the elliptical calibration plate; adjusting external parameters of a camera based on the data on the elliptical calibration plate after the pose is adjusted; wherein the data is acquired by the camera. The application also discloses a camera parameter calibration device and a storage medium, and based on the camera parameter calibration method, the camera parameter calibration device and the storage medium, the accuracy of the external parameter of the camera can be improved, and the complexity of camera parameter calculation can be reduced.

Description

Camera parameter calibration method, device and storage medium

Technical Field

The application relates to the technical field of multi-source heterogeneous fusion sensing, in particular to a camera parameter calibration method, a device and a storage medium.

Background

In wisdom traffic autopilot field, in the multisource heterogeneous perception application research that fuses based on the roadside, spatial synchronization is one of main technical prerequisite, however in the spatial synchronization scheme implementation process of correlation technique, need frequently carry out the calculation of camera parameter, the process of camera parameter calculation not only can increase the complexity of calculation many times, still can lead to the error accumulation of camera parameter, it is extremely low to lead to the accuracy of camera parameter, can't satisfy spatial synchronization's demand, therefore, the precision of the external parameter of promotion camera and the complexity that reduces camera parameter calculation are the problem that needs to solve urgently.

Disclosure of Invention

The embodiment of the application provides a camera parameter calibration method, a camera parameter calibration device and a storage medium, which can improve the accuracy of external parameters of a camera and reduce the complexity of camera parameter calculation.

The technical scheme of the embodiment of the application is realized as follows:

in a first aspect, an embodiment of the present application provides a camera parameter calibration method, including: determining the pose of the elliptical calibration plate based on the integral inclination quantization vector of the elliptical calibration plate; adjusting external parameters of a camera based on the data on the elliptical calibration plate after the pose is adjusted; wherein the data is acquired by the camera.

In the foregoing solution, the determining the pose of the elliptical calibration plate based on the whole tilt quantization vector of the elliptical calibration plate includes:

determining at least one first connected region comprised by the elliptical calibration plate;

determining an average value of the azimuthal arc curvature of the inner ellipse of each first communication region on the elliptical calibration plate;

and determining the integral inclination quantization vector of the elliptical calibration plate based on the average value of the orientation arc curvature of the inner ellipse of each first communication area on the elliptical calibration plate.

determining whether a maximum value of a parameter in the global tilt quantized vector is within a first tilt threshold range and a minimum value of the parameter in the global tilt quantized vector is within a second tilt threshold range;

if the maximum value of the parameters in the overall inclination quantization vector is not in a first inclination threshold range and/or the minimum value of the parameters in the overall inclination quantization vector is not in a second inclination threshold range, determining the pose of the elliptical calibration plate based on the relation between each parameter in the overall inclination quantization vector and the first threshold range, the second threshold range and a third threshold range;

wherein the first threshold range, the second threshold range, and the third threshold range are determined based on a curvature of an ellipse in the ellipse scaling plate.

In the foregoing solution, the determining an average value of the curvature of the azimuth arc of the inner ellipse of each first communication region on the elliptical calibration plate includes:

determining a left side azimuth arc curvature, a right side azimuth arc curvature, an upper side azimuth arc curvature, a lower side azimuth arc curvature, a left upper side azimuth arc curvature, a right upper side azimuth arc curvature, a left lower side azimuth arc curvature and a right lower side azimuth arc curvature of each first communication region inner ellipse;

determining an average of the azimuth arc curvatures of each of the first communication region inner ellipses on the elliptical calibration plate based on the left side azimuth arc curvature, the right side azimuth arc curvature, the upper side azimuth arc curvature, the lower side azimuth arc curvature, the upper left side azimuth arc curvature, the upper right side azimuth arc curvature, the lower left side azimuth arc curvature, and the lower right side azimuth arc curvature of each of the first communication region inner ellipses.

In the above scheme, the external parameter of the camera is acquired.

In the above solution, the adjusting the external parameter of the camera based on the adjusted data on the elliptical calibration plate includes:

converting the three-dimensional point cloud data on the elliptical calibration plate after the pose is adjusted into a two-dimensional gray scale map;

at least a first number of boundary points of each ellipse on the two-dimensional gray scale map is determined.

In the above scheme, the method further comprises:

determining a measured ellipse parameter vector based on a four-point ellipse-determining algorithm and at least a first number of boundary points for each ellipse;

determining a standard ellipse parameter vector of the two-dimensional gray scale image based on a Hough ellipse detection algorithm;

adjusting an external parameter of the camera based on the measured elliptical parameter vector and the standard elliptical parameter vector;

wherein the measured ellipse parameter vector comprises the center coordinate, the major axis length and the minor axis length of each ellipse on the two-dimensional gray scale map.

In the foregoing aspect, the adjusting the external parameter of the camera based on the measured elliptical parameter vector and the standard elliptical parameter vector includes:

determining at least one of a gain value corresponding to the first rotation angle, a gain value corresponding to the second rotation angle, a gain value corresponding to the third rotation angle, a gain value corresponding to the first translation amount, a gain value corresponding to the second translation amount, and a gain value corresponding to the third translation amount based on the measured elliptical parameter vector;

if the sum of the gain value corresponding to the first rotation angle, the gain value corresponding to the second rotation angle, the gain value corresponding to the third rotation angle, the gain value corresponding to the first translation amount, the gain value corresponding to the second translation amount, and the gain value corresponding to the third translation amount is less than the benefit gain threshold, adjusting the external parameters of the camera based on the gain value corresponding to the first rotation angle, the gain value corresponding to the second rotation angle, the gain value corresponding to the third rotation angle, the gain value corresponding to the first translation amount, the gain value corresponding to the second translation amount, and the gain value corresponding to the third translation amount.

In a second aspect, an embodiment of the present application further provides a camera parameter calibration apparatus, where the apparatus includes: the determining unit is used for determining the pose of the oval calibration plate based on the integral inclination quantization vector of the oval calibration plate; the adjusting unit is used for adjusting external parameters of the camera based on the data on the elliptical calibration plate after the pose is adjusted; wherein the data is acquired by the camera.

In a third aspect, an embodiment of the present application provides a storage medium, which stores an executable program, and when the executable program is executed by a processor, the method for calibrating camera parameters performed by the apparatus is implemented.

In a fourth aspect, an embodiment of the present application provides a camera parameter calibration apparatus, where the camera parameter calibration apparatus enables a processor to execute the camera parameter calibration method.

The camera parameter calibration method, the device and the storage medium provided by the embodiment of the application determine the pose of the elliptical calibration plate based on the integral inclination quantization vector of the elliptical calibration plate; adjusting external parameters of a camera based on the data on the elliptical calibration plate after the pose is adjusted; the data are acquired by the camera, so that the accuracy of external parameters of the camera can be improved, and the complexity of camera parameter calculation can be reduced.

Drawings

Fig. 1 is a schematic view of an alternative flow chart of a camera parameter calibration method according to an embodiment of the present disclosure;

fig. 2 is a schematic view of another alternative flow chart of a camera parameter calibration method according to an embodiment of the present disclosure;

fig. 3 is a schematic view of another alternative flow chart of a camera parameter calibration method according to an embodiment of the present application;

fig. 4 is a schematic view of an alternative structure of a camera parameter calibration apparatus provided in an embodiment of the present application;

fig. 5 is a schematic diagram of a hardware component structure of the camera external reference calibration apparatus according to the embodiment of the present application.

Detailed Description

The present application will be described in further detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

In the field of intelligent traffic automatic driving and roadside-based multi-source heterogeneous fusion perception application research, time synchronization and space synchronization are two main technical premises. In the road side multisource heterogeneous fusion sensing scheme, three different sensing devices, namely a camera, a laser radar and a millimeter wave radar of a camera are in a group and are erected at the same height position on a rod. Converting coordinate systems of other sensing equipment by taking a laser radar coordinate system as a center in the group; and converting the laser radar coordinate system into a world coordinate system between the groups to perform space synchronization. In the actual implementation process of the space synchronization scheme, the parameters need to be calculated frequently, the prior art cannot meet the requirements of simplicity and accuracy of the space synchronization parameter measurement, and most parameter measurement processes need manual participation; because no corresponding operation instruction and accurate real-time evaluation data quantitative display exist, the parameter calibration implementation process is complex, the operation difficulty is high, and in addition, the requirements on testing personnel and testing equipment are also high.

In the related art, the following schemes are mainly adopted to adjust camera parameters:

scheme 1

The sensor is aligned to the cubic target to carry out one-time shooting, the laser radar shoots three-dimensional point cloud of the cubic target, and a rotation matrix R from a laser radar coordinate system to a world coordinate system where the cubic target is located is obtained_LWAnd translation matrix T_LW(ii) a Shooting by a camera to obtain an image of the cubic target, and obtaining a rotation matrix R from a camera coordinate system to a world coordinate system where the cubic target is located_CWAnd translation matrix T_CW(ii) a Finally, according to the transformation of the two previous pairs of rotation translation matrixes and the rotation matrix, obtaining a rotation matrix R between the laser radar and the camera_LCAnd translation matrix T_LC。

However, in the scheme 1, the three-dimensional target is aligned, the image coordinates and the three-dimensional point cloud data under the camera and the laser radar are obtained, and the conversion relation between the laser radar and the world coordinate system is obtained by combining the normal vectors of three adjacent surfaces of the box; and (4) utilizing an external parameter matrix from three planes to a camera coordinate system by a Zhang calibration method. And calculating external parameters of the laser radar under a camera coordinate system by taking the world coordinate as a central medium. As can be seen from the process, the method has the following problems: 1) the image coordinate is changed to a world coordinate, the laser radar coordinate is changed to the world coordinate, then the laser radar is changed to the image coordinate, three processes are carried out, and errors of the three processes are accumulated; 2) no corresponding error evaluation quantization index exists; 3) the working and placing positions of the checkerboard target have large influence on parameter measurement, and the stability of precision is difficult to control due to manual control.

Scheme 2

By means of the deep neural network and the multi-model target state prediction method of reinforcement learning, the problems of complex strong nonlinear environment expression and long-time target state prediction of multi-model fusion are effectively solved, the aim is to improve the target state prediction longitude and prediction duration, and online parameter self-correction of multi-model fusion is achieved. The method is suitable for combined calibration of an automatic driving vehicle end, adopts a multi-sensor calibration reward mechanism, adapts to a benefit function of calibration matrix transformation under a dynamic environment, and realizes an off-line initial learning and on-line real-time learning updated calibration model.

However, the problems of the above scheme 2 include: 1) the three-layer calibration benefit function only provides a combined calibration scheme, so that the technical problems are still not solved, and the implementation is poor; 2) target detection and characteristic acquisition under various sensors have large errors in a real scene, and the whole scheme has more influence factors on the final calibration precision of combined calibration and has some defects in precision; 3) because of adopting deep reinforcement learning, the calculation amount is large.

Scheme 3

And directly solving a pose transformation matrix from a world coordinate system to a pixel coordinate system after fusing internal and external parameters of the camera. The method comprises the steps of forming discrete point cloud coordinates obtained by a known and fixed plane plate through laser radar irradiation, further fitting a large amount of point cloud data to obtain a space plane where a calibration plane plate is located and a sideline equation, obtaining corner point three-dimensional coordinate cloud information of the calibration plane plate through intersection points of sidelines, and fitting Gaussian distribution to continuous multi-frame data to greatly reduce uncertainty of the laser radar in observation and can be used for any relative pose.

However, the problems of the above scheme 3 include: 1) the requirement on the manufacturing precision of the calibration plate is high; 2) the human intervention is more, and the professional literacy of the calibration operator is higher; 3) for the placing mode of the calibration objects, the placing mode is more and lacks necessary instructions, and the degree quantization guidance is provided.

Based on the problems existing in the current space synchronization, the camera parameter calibration method provided by the application can solve the technical problems and defects which cannot be solved in the prior art.

Fig. 1 shows an optional flowchart of a camera parameter calibration method provided in an embodiment of the present application, which will be described according to various steps.

And S101, determining the pose of the oval calibration plate based on the integral inclination quantization vector of the oval calibration plate.

In some embodiments, the camera parameter calibration device (hereinafter referred to as a device) determines the pose of the elliptical calibration plate based on the overall tilt quantization vector of the elliptical calibration plate, so that the optical axis of the camera can be perpendicular to the elliptical calibration plate, thereby performing subsequent adjustment on the external parameters of the camera.

In some alternative embodiments, the device may be configured by determining at least one first connected region comprised by the elliptical calibration plate; determining an average value of the azimuthal arc curvature of the inner ellipse of each first communication region on the elliptical calibration plate; and determining the integral inclination quantization vector of the elliptical calibration plate based on the average value of the orientation arc curvature of the inner ellipse of each first communication area on the elliptical calibration plate.

In specific implementation, the device determines a left side azimuth arc curvature, a right side azimuth arc curvature, an upper side azimuth arc curvature, a lower side azimuth arc curvature, a left upper side azimuth arc curvature, a right upper side azimuth arc curvature, a left lower side azimuth arc curvature and a right lower side azimuth arc curvature of each first communication region inner ellipse; determining an average of the azimuth arc curvatures of each of the first communication region inner ellipses on the elliptical calibration plate based on the left side azimuth arc curvature, the right side azimuth arc curvature, the upper side azimuth arc curvature, the lower side azimuth arc curvature, the upper left side azimuth arc curvature, the upper right side azimuth arc curvature, the lower left side azimuth arc curvature, and the lower right side azimuth arc curvature of each of the first communication region inner ellipses.

In some optional embodiments, the apparatus may determine the slope of the parameter by determining whether a maximum value of the parameter in the global slope quantized vector is within a first slope threshold range and whether a minimum value of the parameter in the global slope quantized vector is within a second slope threshold range; if the maximum value of the parameters in the overall inclination quantization vector is not in a first inclination threshold range and/or the minimum value of the parameters in the overall inclination quantization vector is not in a second inclination threshold range, determining the pose of the elliptical calibration plate based on the relation between each parameter in the overall inclination quantization vector and the first threshold range, the second threshold range and a third threshold range; wherein the first threshold range, the second threshold range, and the third threshold range are determined based on a curvature of an ellipse in the ellipse scaling plate.

And S102, adjusting external parameters of the camera based on the data on the elliptical calibration plate after the pose is adjusted.

In some embodiments, the device adjusts the external parameters of the camera based on the data on the elliptical calibration plate after the pose is adjusted. The data on the elliptical calibration plate can comprise three-dimensional point cloud data acquired by a laser radar from the elliptical calibration plate after the position and orientation are adjusted and two-dimensional image coordinates acquired by a camera from the elliptical calibration plate after the position and orientation are adjusted.

In specific implementation, the device converts the three-dimensional point cloud data on the elliptical calibration plate after the pose is adjusted into a two-dimensional gray scale map; at least a first number of boundary points of each ellipse on the two-dimensional gray scale map is determined. Determining a measured ellipse parameter vector based on a four-point ellipse-determining algorithm and at least a first number of boundary points for each ellipse; determining a standard ellipse parameter vector of the two-dimensional gray scale image based on a Hough ellipse detection algorithm; adjusting an external parameter of the camera based on the measured elliptical parameter vector and the standard elliptical parameter vector;

Accordingly, the apparatus determines at least one of a gain value corresponding to the first rotation angle, a gain value corresponding to the second rotation angle, a gain value corresponding to the third rotation angle, a gain value corresponding to the first translation amount, a gain value corresponding to the second translation amount, and a gain value corresponding to the third translation amount based on the measured elliptical parameter vector;

if the sum of the gain value corresponding to the first rotation angle, the gain value corresponding to the second rotation angle, the gain value corresponding to the third rotation angle, the gain value corresponding to the first translation amount, the gain value corresponding to the second translation amount, and the gain value corresponding to the third translation amount is less than the benefit gain threshold, adjusting the external parameter of the camera based on the positive and negative of the gain value corresponding to the first rotation angle, the gain value corresponding to the second rotation angle, the gain value corresponding to the third rotation angle, the gain value corresponding to the first translation amount, the gain value corresponding to the second translation amount, and the gain value corresponding to the third translation amount.

Therefore, by the camera parameter calibration method provided by the embodiment of the application, the pose of the elliptical calibration plate is determined based on the integral inclination quantization vector of the elliptical calibration plate; adjusting external parameters of a camera based on the data on the elliptical calibration plate after the pose is adjusted; wherein the data is acquired by the camera. Manual operation can be reduced, so that the adjustment of the external parameter of the complex camera becomes general and easy to operate; the external parameter of the camera is adjusted based on the data on the elliptical calibration plate, so that the accuracy of the external parameter of the camera can be improved, and the complexity of the calculation of the camera parameter can be reduced.

Fig. 2 shows another alternative flow chart of the camera parameter calibration method provided in the embodiment of the present application, which will be described according to various steps.

Step S201, determining the integral inclined quantization vector of the oval calibration plate.

In some embodiments, the camera parameter calibration device determines at least one first connected region comprised by the elliptical calibration plate; determining an average value of the azimuthal arc curvature of the inner ellipse of each first communication region on the elliptical calibration plate; and determining the integral inclination quantization vector of the elliptical calibration plate based on the average value of the orientation arc curvature of the inner ellipse of each first communication area on the elliptical calibration plate.

Wherein the first communication area may be a white communication area of the elliptical calibration plate; accordingly, the apparatus may also determine at least one second connected region comprised by the elliptical calibration plate; the second communication region may be a black communication region of the elliptical calibration plate.

In specific implementation, the device can acquire the whole white connected region and the whole black region on the oval calibration plate based on the HSV color space with Hue (Hue), Saturation (Saturation) and Value (Value) as parameters; then acquiring at least one first connected region based on the whole white connected region; alternatively, the at least one first connected region may be obtained based on a method of corrosive expansion.

In specific implementation, the device may determine eight azimuthal arc curvatures of an inner ellipse of each first communication region through a hough ellipse curvature detection algorithm, where the eight azimuthal arc curvatures include: left side azimuth arc curvature, right side azimuth arc curvature, upper side azimuth arc curvature, lower side azimuth arc curvature, left upper side azimuth arc curvature, right upper side azimuth arc curvature, left lower side azimuth arc curvature, and right lower side azimuth arc curvature; determining an average of the azimuth arc curvatures of each of the first communication region inner ellipses on the elliptical calibration plate based on the left side azimuth arc curvature, the right side azimuth arc curvature, the upper side azimuth arc curvature, the lower side azimuth arc curvature, the upper left side azimuth arc curvature, the upper right side azimuth arc curvature, the lower left side azimuth arc curvature, and the lower right side azimuth arc curvature of each of the first communication region inner ellipses.

For example, the equation for the inner ellipse of any of the first communication regions is

The device determines a second number of sets of points on each azimuthal arc of the ellipse, each set of points including at least two points; based on the determined second number of set points on each orientation arc and the equation for the ellipse, a second number of arc curvatures on each orientation arc may be determined, the second number of arc curvatures averaged, and an arc curvature for each orientation arc of the ellipse determinedAnd (4) rate. Optionally, the overall tilt quantization vector of the elliptical calibration plate is determined based on an average value of each azimuthal arc curvature of each ellipse inside each first communication region on the elliptical calibration plate. The second number is a positive integer and can be set according to actual needs.

The dimension of the integral inclined quantization vector is equal to the number of ellipses on the ellipse calibration plate; in the overall skewing vector, each parameter corresponds to the arc curvature of the ellipse on the ellipse calibration plate. The arc curvature of the ellipse is determined based on an average of each azimuthal arc curvature of the ellipse.

And S202, determining the pose of the oval calibration plate based on the integral inclination quantization vector of the oval calibration plate.

In some embodiments, the device determines the pose of the elliptical calibration plate based on an overall tilt quantization vector of the elliptical calibration plate.

In a specific implementation, the determining, by the apparatus, whether a maximum value and a minimum value of the overall tilting vector are within a tilt threshold range may specifically include: whether a maximum value of the parameter in the global tilt quantization vector is within a first tilt threshold range, and whether a minimum value of the parameter in the global tilt quantization vector is within a second tilt threshold range.

If the maximum value of the parameters in the overall inclination quantization vector is within a first inclination threshold range and the minimum value of the parameters in the overall inclination quantization vector is within a second inclination threshold range, determining that the pose of the graph circular calibration plate does not need to be adjusted, and executing step S203;

or if the maximum value of the parameters in the whole inclined quantized vector is not within the first inclination threshold range and/or the minimum value of the parameters in the whole inclined quantized vector is not within the second inclination threshold range, adjusting the pose of the elliptical calibration plate based on the relationship between each parameter in the whole inclined quantized vector and the first threshold range, the second threshold range and the third threshold range, and repeatedly executing the steps S201 to S202 until the maximum value of the parameters in the whole inclined quantized vector is within the first inclination threshold range and the minimum value of the parameters in the whole inclined quantized vector is within the second inclination threshold range, determining that the pose of the circular calibration plate does not need to be adjusted, and executing the step S203.

Wherein the first threshold range of inclination, the second threshold range of inclination, the first threshold range, the second threshold range and the third threshold range are determined based on a proportional relationship between a major axis and a minor axis of the ellipse (i.e. a curvature of the ellipse or a curvature of the ellipse) on the ellipse calibration plate.

In specific implementation, adjusting the pose of the elliptical calibration plate based on the relationship between each parameter in the whole tilt quantization vector and the first threshold range, the second threshold range, and the third threshold range may include: if any parameter is within the first threshold range, no adjustment is needed; and if any parameter is within the second threshold range or the third threshold range, adjusting the position of the parameter within the second threshold range or the third threshold range.

Optionally, a corresponding color may be indicated on the elliptical scaling plate based on a relationship of each parameter in the global tilt quantization vector to a first threshold range, a second threshold range, and a third threshold range.

Specifically, if any one of the parameters is within a first threshold range, indicating green at the corresponding position of the oval calibration plate; if any one parameter is within a second threshold range, indicating orange at the corresponding position of the oval calibration plate; and if any one parameter is within the third threshold value range, indicating red at the corresponding position of the oval calibration plate. Alternatively, different shades may be displayed depending on the magnitude of the difference between each parameter and the end of the first threshold range, the end of the second threshold range, and the end of the third threshold range. For example, the third threshold range is [ c, + ∞), the smaller the difference with c, the lighter the red color is indicated; the greater the difference with c, the darker the red color is indicated.

The device selects the orientation with the darker red or yellow to adjust until the orientation indicates green.

For example, the red color of the upper left side of the oval calibration plate is deepest, the device keeps the lower right side of the oval calibration plate still, the upper left corner of the oval calibration plate is slowly moved backwards, and in the moving process, the device repeatedly executes the steps S201 to S202 until the upper left side of the oval calibration plate indicates green, and/or all other positions of the oval calibration plate indicate green, and the pose adjustment of the oval calibration plate is determined to be completed.

Step S203, obtaining external parameter of the camera.

In some embodiments, the device acquires external parameters of the camera; optionally, the apparatus may also acquire internal parameters of the camera.

In specific implementation, the apparatus may obtain the internal reference parameter according to the prior art, which is not limited herein. For example, the apparatus may acquire the internal parameters of the camera according to the zhang-shi scaling method.

In specific implementation, the device determines N groups of corresponding three-dimensional point cloud data and two-dimensional image coordinates (namely, the three-dimensional point cloud data and the two-dimensional image coordinates are in a corresponding relationship) in a laser radar coordinate system and the camera picture pixel coordinates; and acquiring the external parameters of the camera based on the N groups of corresponding three-dimensional point cloud data and the two-dimensional image coordinates. The three-dimensional point cloud data is acquired by the laser radar from the elliptical calibration plate after the pose is adjusted; and the two-dimensional image coordinates are acquired by the camera from the elliptical calibration plate after the pose is adjusted.

Optionally, the apparatus may acquire the external parameters of the camera based on an angle-N-Point (PnP) algorithm and the N sets of corresponding three-dimensional Point cloud data and two-dimensional image coordinates.

Thus, the initialization of the internal parameter and the external parameter of the camera is completed.

Step S204, determining a measured ellipse parameter vector and a standard ellipse parameter vector.

In some alternative embodiments, the device determines a measured ellipse parameter vector and a standard ellipse parameter vector.

Optionally, the device converts the three-dimensional point cloud data on the pose-adjusted elliptical calibration plate into a two-dimensional gray scale map; determining at least a first number of boundary points for each ellipse on the two-dimensional gray scale map; determining a measured ellipse parameter vector based on a four-point ellipse-determining algorithm and at least a first number of boundary points for each ellipse; and determining a standard ellipse parameter vector of the two-dimensional gray scale image based on a Hough ellipse detection algorithm.

In specific implementation, the device converts the three-dimensional point cloud data into a two-dimensional image coordinate system to form a two-dimensional gray scale image, and performs gray scale value conversion according to the reflection intensity of the three-dimensional point cloud data. The three-dimensional point cloud data can be obtained based on a laser radar, and because the detection performance of the laser radar to dark objects is larger than the difference between white, the oval area on the oval calibration plate is bright white, and other areas are dark black, when the three-dimensional point cloud data is converted into a two-dimensional gray scale image, points in the oval area on the oval calibration plate are bright white, and points in other areas are gray. Optionally, the device may further adjust the size of the corresponding pixel value on the two-dimensional gray scale image according to the intensity of the three-dimensional point cloud data. The greater the intensity of the three-dimensional point cloud data is, the greater the corresponding pixel value on the two-dimensional gray scale image is. Because the reflection intensity of the laser radar to the dark object is weak, on the two-dimensional gray scale map, the point of the bright white oval area of the oval calibration plate will be off white.

In some alternative embodiments, the apparatus determining at least a first number of boundary points for each ellipse on the two-dimensional gray scale map may include: at least a first number of boundary points is determined based on the bright-colored cross-hatching of the lidar on the elliptical calibration plate. The first number is determined according to the line number of the laser radar, and the higher the line number of the laser radar is, the larger the value of the first number is, and the higher the accuracy of the adjusted external parameter of the camera is.

In specific implementation, the device determines a measured ellipse parameter vector based on a four-point ellipse-determining algorithm and at least a first number of boundary points of each ellipse; wherein the dimension of the measured ellipse parameter vector is equal to the number of ellipses on the ellipse calibration plate; the measured ellipse parameter vector comprises at least one of the measurement center coordinate of any ellipse on the ellipse calibration plate, the measurement major axis length, the measurement minor axis length, the measurement left length of the ellipse major axis, the measurement right length of the ellipse major axis, the measurement upper length of the ellipse minor axis and the measurement lower length of the ellipse minor axis.

In specific implementation, the device determines a standard ellipse parameter vector of the two-dimensional gray scale image based on a Hough ellipse detection algorithm; the dimension of the standard ellipse parameter vector is equal to the number of ellipses on the ellipse calibration plate; the standard ellipse parameter vector comprises at least one of standard center coordinates of any ellipse on the ellipse calibration plate, standard major axis length, standard minor axis length, left side length of standard ellipse major axis, right side length of standard ellipse major axis, upper side length of standard ellipse minor axis and lower side length of standard ellipse minor axis.

Step S205, adjusting the external parameter of the camera based on the measured elliptical parameter vector and the standard elliptical parameter vector.

In some embodiments, the device adjusts an external parameter of the camera based on the measured ellipse parameter vector and the standard ellipse parameter vector.

In some optional embodiments, the device adjusts the external parameter of the camera based on a degree of coincidence of the measured elliptical parameter vector and the standard elliptical parameter vector.

When the external parameter of the camera does not need to be adjusted, the difference value of corresponding elements in the measured elliptical parameter vector and the standard elliptical parameter vector is small, for example, smaller than a first threshold value; and if the difference value of any one corresponding element in the measured elliptical parameter vector and the standard elliptical parameter vector is greater than or equal to a first threshold value, indicating that the external parameter of the camera needs to be adjusted.

Optionally, the external parameters of the camera may at least include: at least one of the first rotation angle, the second rotation angle, the third rotation angle, the first translation amount, the second translation amount, and the third translation amount.

Specifically, the first rotation angle may be a rotation angle of an x-axis; the gain value corresponding to the first rotation angle may be determined according to a difference between the lengths of the left half axes of all the major axes of the measurement ellipses and the lengths of the right half axes of all the major axes of the measurement ellipses in the measurement ellipse parameter vector, and may be determined by a gain benefit function as follows:

wherein, w_{x_recs}The gain value is corresponding to the first rotation angle; lambda [ alpha ]_{x_recs}Weighting the gain benefit function corresponding to the first rotation angle; l_rTo determine the length of the right side of the major axis of the ellipse; l_lTo determine the length of the left side of the major axis of the ellipse.

Specifically, the second rotation angle may be a rotation angle of the y-axis; the gain value corresponding to the second rotation angle may be determined according to a difference between the upper half-axis length of all the measured ellipse minor axes and the lower half-axis length of all the measured ellipse minor axes in the measured ellipse parameter vector, and may be determined by the following gain benefit function:

wherein, w_{y_recs}The gain value is corresponding to the second rotation angle; lambda [ alpha ]_{y_recs}The weight of the gain benefit function corresponding to the second rotation angle; l_uThe length of the upper side of the minor axis of the ellipse is measured; l_dTo determine the lower length of the minor axis of the ellipse.

Specifically, the third rotation angle may be a rotation angle of the z-axis; the angle between the major axis included in the measured ellipse parameter vector and the major axis included in the standard ellipse parameter vector and the angle between the minor axis included in the measured ellipse parameter vector and the minor axis included in the standard ellipse parameter vector may be determined, and the gain value corresponding to the third rotation angle may be determined by a gain benefit function as follows:

wherein, w_{z_recs}The gain value is corresponding to the third rotation angle; lambda [ alpha ]_{z_recs}The weight of the gain benefit function corresponding to the third rotation angle; theta is an included angle between a long axis included in the measured elliptical parameter vector and a long axis included in the corresponding standard elliptical parameter vector;

the included angle between the minor axis included in the measured elliptical parameter vector and the minor axis included in the corresponding standard elliptical parameter vector is determined.

Specifically, the first translation amount may be an offset amount of the x-axis; the first translation amount may be determined according to an x-direction component value of a measurement center coordinate in the measurement elliptical parameter vector and an x-direction component value of a standard center coordinate in the standard elliptical parameter vector, and the gain value corresponding to the first translation amount may be determined by a gain benefit function as follows:

wherein, w_{x_tvecs}The gain value corresponding to the first translation quantity; lambda [ alpha ]_{x_tvecs}The weight of the gain benefit function corresponding to the first translation quantity; o_xThe component value in the x direction of the standard center coordinate in the standard elliptic parameter vector is obtained;

and determining the x-direction component value of the central coordinate in the determined elliptic parameter vector.

Specifically, the second shift amount may be an offset amount of the y-axis; the second translation amount may be determined according to a y-direction component value of a measurement center coordinate in the measurement elliptical parameter vector and a y-direction component value of a standard center coordinate in the standard elliptical parameter vector, and the gain value corresponding to the second translation amount may be determined by a gain benefit function as follows:

wherein, w_{y_tvecs}The gain value corresponding to the second shift quantity; lambda [ alpha ]_{y_tvecs}The weight of the gain benefit function corresponding to the second shift quantity; o_yThe component value in the y direction of the standard center coordinate in the standard elliptic parameter vector is obtained;

and determining the y-direction component value of the central coordinate in the determined elliptic parameter vector.

Specifically, the third translation amount may be an offset amount of the z-axis; the length of the long axis in the measured elliptical parameter vector, the length of the short axis in the measured elliptical parameter vector, the length of the long axis in the standard elliptical parameter vector, and the length of the short axis in the labeled elliptical parameter vector may be determined, and the gain value corresponding to the third translation may be determined by the following gain benefit function:

wherein, w_{z_tvecs}The gain value corresponding to the third translation quantity; lambda [ alpha ]_{z_tvecs}The weight of the gain benefit function corresponding to the third translation quantity; a is the length of a standard long axis in the standard elliptic parameter vector; b is the length of a standard short axis in the standard elliptical parameter vector;

determining major axis length for the determined elliptic parameter vector;

and determining the length of the short axis in the determined elliptic parameter vector.

In some alternative embodiments, λ_{x_recs}+λ_{y_recs}+λ_{z_recs}+λ_{x_tvecs}+λ_{y_tvecs}+λ_{z_tvecs}1, optionally λ_{x_recs}＝λ_{y_recs}＝λ_{z_recs}＝λ_{x_tvecs}＝λ_{y_tvecs}＝λ_{z_tvecs}＝1/6。

In some optional embodiments, if the gain value corresponding to the first rotation angle is not within the range of the fourth threshold, adjusting the external parameter of the camera based on the positive, negative and magnitude of the gain value corresponding to the first rotation angle; or if the gain value corresponding to the second rotation angle is not within a fifth threshold range, adjusting the external parameter of the camera based on the positive and negative values and the size of the gain value corresponding to the second rotation angle; or if the gain value corresponding to the third rotation angle is not within a sixth threshold range, adjusting the external parameter of the camera based on the positive and negative values and the size of the gain value corresponding to the third rotation angle; or if the gain value corresponding to the first translation amount is not within a seventh threshold range, adjusting the external parameter of the camera based on the positive and negative values and the size of the gain value corresponding to the first translation amount; or if the gain value corresponding to the second translation amount is not within the range of an eighth threshold, adjusting the external parameter of the camera based on the positive and negative values and the size of the gain value corresponding to the second translation amount; or if the gain value corresponding to the third translation amount is not within a ninth threshold range, adjusting the external parameter of the camera based on the positive and negative sum of the gain value corresponding to the third translation amount. Any one of the fourth threshold range to the ninth threshold range may be set according to actual needs.

For example, if the gain value corresponding to the first rotation angle is a positive value and is not at the right end point of the fourth threshold range, the external parameter of the camera is adjusted so that the gain value corresponding to the first rotation angle is within the fourth threshold range.

In some embodiments, the sum of the gain value corresponding to the first rotation angle, the gain value corresponding to the second rotation angle, the gain value corresponding to the third rotation angle, the gain value corresponding to the first shift amount, the gain value corresponding to the second shift amount, and the gain value corresponding to the third shift amount is the overall benefit gain w, that is, the total benefit gain w is

w＝|w_{x_recs}|+|w_{y_recs}|+|w_{z_recs}|+|w_{x_tvecs}|+|w_{y_tvecs}|+|w_{z_tvecs}| (7)

And if the overall benefit gain is smaller than a first benefit gain threshold value, adjusting the external parameters of the camera based on the positive and negative sum of the gain value corresponding to the first rotation angle, the positive and negative sum of the gain value corresponding to the second rotation angle, the positive and negative sum of the gain value corresponding to the third rotation angle, the positive and negative sum of the gain value corresponding to the first translation amount, the positive and negative sum of the gain value corresponding to the second translation amount, and the positive and negative sum of the gain value corresponding to the third translation amount. Until the overall benefit gain is greater than or equal to the first benefit gain threshold.

Therefore, according to the camera parameter calibration method provided by the embodiment of the application, the position and posture modulation mode of the elliptical calibration plate is quantized by using the characteristics of the long axis and the short axis of the elliptical calibration plate, and the position and posture of the elliptical calibration plate are adjusted according to each quantized parameter index. The three-dimensional point cloud data are converted into the two-dimensional gray scale map, the gain benefit value is calculated by acquiring information such as the center coordinate, the major axis and the minor axis of the ellipse, so that the deviation of the external parameters of the camera is quantified, multiple automatic optimization of the modulation translation amount and the rotation amount of the six parameters included by the external parameters is realized, the accuracy of the external parameters of the camera is improved, and meanwhile, the complexity of calculation of the parameters of the camera can be reduced.

Fig. 3 shows a schematic flow chart of yet another alternative method for calibrating camera parameters, which will be described according to various steps.

In this embodiment, the camera parameter calibration method may be divided into three stages, which are respectively: the method comprises the steps of pose adjustment of an oval calibration plate, initialization of external parameters and adjustment of the external parameters of a camera. The pose adjustment of the elliptical calibration plate may specifically include steps S301 to S305; the initializing of the external parameter may specifically include steps S306 to S307; adjusting the external parameters of the camera may specifically include steps S308 to S312.

Step S301, determining parameters based on the oval calibration plate.

In some embodiments, the camera parameter calibration device determines a first threshold range of tilt, a second threshold range of tilt, a first threshold range, a second threshold range, and a third threshold range based on the curvature of the ellipse on the elliptical calibration plate.

In this embodiment, a case will be described in which 54 ellipses are provided on the ellipse calibration plate, and the ratio of the length of the major axis to the length of the minor axis of each ellipse is 2. The ellipses of the ellipse calibration plate can be divided into 6 rows according to 9 ellipses in each row; or 6 ellipses per row, for a total of 9 rows.

For example, in the present embodiment, the curvature of the ellipse is 2, and accordingly, the first inclination range may be (1.12, + ∞) and the second inclination range may be (0, 0.86). Wherein 1.12 and 0.86 may be replaced by other values. The first threshold range may be (0.86,1.12), the second threshold range may be (0.5,0.86 [. sup.1.2, 2 ], and the third threshold range may be (- ∞,0.5 ]. sup.2 [ + ∞).

Step S302, determining eight orientation arc curvatures of an ellipse in each white connected region on the oval calibration plate.

In some embodiments, the camera parameter calibration device obtains an entire white connected region and at least one black region (i.e., a second connected region) on the elliptical calibration plate through an HSV color space, obtains boundary information of each white connected region (i.e., a first connected region) through erosion and expansion, obtains eight azimuthal arc curvatures, i.e., left, right, upper, lower, upper left, upper right, lower left, and lower right, of an ellipse in each white connected region by using a hough ellipse curvature detection mechanism, and obtains an entire gradient measurement quantization vector E, i.e., an 8-dimensional vector, of the ellipse. Where a is the length of the major axis of the ellipse and b is the length of the minor axis of the ellipse.

Wherein the equation of the ellipse is

Eight groups of points are uniformly selected on each orientation arc, each group of two points can calculate the arc curvature by using an elliptic equation, and then the average value is calculated to be used as the curvature (namely the curvature) of the orientation arc.

Step S303, averaging the curvatures of all the eight-direction arcs of the ellipse on the ellipse calibration plate.

In this embodiment, the specification of the elliptic calibration plate is 80cm × 80cm, 9 × 6 ellipses are formed on the elliptic calibration plate, the curvature is 2(a/b is 2), and therefore, 54 white connected regions are formed, and the curvature of eight azimuth arcs of the ellipses in the 54 white connected regions is averaged to obtain the final overall inclination quantization vector of the elliptic calibration plate

Wherein the vector is quantized by global tilt

Is 54 dimensions, and has 54 parameters, each of which is determined by averaging the curvatures of eight azimuthal arcs of an ellipse.

In step S304, it is determined whether the maximum value and the minimum value of the entire inclination vector are within the inclination range.

In some embodiments, the device determines the global tilted quantization vector

Median maximum value

E and minimum value

And determining said maximum value

And minimum value

Whether or not within a set threshold value of inclination, i.e.

In the interval (0,0.86) (i.e. the second inclination range) and

in the interval (1.12, + ∞) (i.e. the first range of inclination), it should be noted that 0.86 and 1.12 can be replaced by other values depending on the ratio of the major axis a and the minor axis b of the ellipse on the calibration plate used, i.e. (b/a) × 2 being defined for curvature. Here, an elliptical calibration plate with a/b of 2 is used, and thus the standard curvature is 1. If either of the two conditions is not satisfied, the apparatus prompts an instruction to adjust the azimuth, and then proceeds to step S305. Otherwise, it is prompted that the orientation of the elliptical calibration plate has satisfied the condition, and step S306 is executed.

In some alternative embodiments, the eight orientations of the elliptical calibration plate will be displayed on the image with a color indication of tilt measure, green-yellow-red, and three colors are set according to the degree of curvature approaching 1, namely (0.86,1.12) (i.e. the first threshold range) with green between, the color of green being darker as the curvature approaches 1; yellow between (0.5, 0.86) and [1.12, 2) (i.e., the second threshold range), the closer to 1 the lighter the yellow color; (∞, 0.5) U [2, + ∞) (i.e., the third threshold range) is red, with the depth of red being lighter the closer to 1.

And S305, adjusting the pose of the elliptical calibration plate based on the integral inclination quantization vector of the elliptical calibration plate.

In some embodiments, the device adjusts the orientation of the prompt or optionally selects one of the orientations with a darker red or yellow color to indicate a green color based on the color change of the inclination indication icon. For example, in the upper left side with the deepest red color, the lower right corner of the elliptical calibration plate is kept still, the upper left corner of the elliptical calibration plate is slowly moved backwards, and the color is indicated as green by modulating the inclination of the upper left corner. And repeatedly executing the steps S302 to S304 until whether the maximum value and the minimum value of the whole inclination vector are within the inclination range, and executing the step S306.

Step S306, acquiring the internal parameters of the camera.

In some embodiments, the apparatus may obtain the internal reference parameter according to the prior art, which is not limited herein. For example, the apparatus may acquire the internal parameters of the camera according to the zhang-shi scaling method.

Step S307, obtaining external parameter of the camera.

The device determines N groups of corresponding three-dimensional point cloud data and two-dimensional image coordinates (namely the three-dimensional point cloud data and the two-dimensional image coordinates are in a corresponding relation) in a laser radar coordinate system and the camera picture pixel coordinates; and acquiring the external parameters of the camera based on the N groups of corresponding three-dimensional point cloud data and the two-dimensional image coordinates. The three-dimensional point cloud data is acquired by the laser radar from the elliptical calibration plate after the pose is adjusted; and the two-dimensional image coordinates are acquired by the camera from the elliptical calibration plate after the pose is adjusted.

Optionally, the apparatus may acquire the external parameters of the camera based on the PnP algorithm and the N sets of corresponding three-dimensional point cloud data and two-dimensional image coordinates.

And step S308, converting the three-dimensional point cloud data on the elliptical calibration plate after the pose is adjusted into a two-dimensional gray scale map.

In some embodiments, the device converts the three-dimensional point cloud data into a two-dimensional gray scale map through initialized internal and external parameters, and assigns a pixel value [0, 255 ] of the gray scale map according to the intensity, wherein the greater the intensity, the greater the pixel value, the weaker the reflection intensity of the lidar to a dark object, and therefore, on the two-dimensional gray scale map, the point of a bright white elliptical area of the elliptical calibration plate will be off white.

In specific implementation, the device converts the three-dimensional point cloud data into a two-dimensional image coordinate system to form a two-dimensional gray scale image, and performs gray scale value conversion according to the reflection intensity of the three-dimensional point cloud data. The three-dimensional point cloud data can be obtained based on a laser radar, and because the detection performance of the laser radar to dark objects is larger than the difference between white, the oval area on the oval calibration plate is bright white, and other areas are dark black, when the three-dimensional point cloud data is converted into a two-dimensional gray scale image, points in the oval area on the oval calibration plate are bright white, and points in other areas are gray.

Step S309, determining at least a first number of boundary points of each ellipse on the two-dimensional grayscale map.

In some embodiments, the apparatus determines at least a first number of boundary points based on a bright-colored cross-line of the lidar on the elliptical calibration plate. The first number is determined according to the line number of the laser radar, and the higher the line number of the laser radar is, the larger the first number value is, and the higher the accuracy of subsequent results is. Optionally, the first number is 4.

For example, in this embodiment, a laser radar with 16 lines is used, so that on the two-dimensional gray scale map, there are at least 5 bright color horizontal lines per ellipse of the ellipse calibration plate, and thus 10 ellipse boundary points can be obtained. The boundary points are used for calculating the parameters of the ellipse, and the larger the number of the boundary points is (at least 4), the more the calculated a, b and x of the ellipse₀、y₀The higher the accuracy of (c).

In step S310, a measurement ellipse parameter vector is determined.

In some embodiments, the apparatus may determine at least one of a measured center coordinate of the ellipse, a measured length of the major axis, a measured length of the minor axis, a measured length of the left side of the major axis of the ellipse, a measured length of the right side of the major axis of the ellipse, a measured length of the upper side of the minor axis of the ellipse, and a measured length of the lower side of the minor axis of the ellipse based on a four-point ellipsometry method based on any set of boundary points of a single elliptical region on a two-dimensional grayscale map. Then proceed withAnd (5) carrying out equalization processing. By analogy, the measurement center coordinates of 54 ellipses on the gray-scale map, the length of the long axis, the length of the short axis, the length of the left side of the long axis, the length of the right side of the long axis, the length of the upper side of the short axis and the length of the lower side of the short axis can be obtained, and the measurement ellipse parameter vector can be formed

In step S311, a standard elliptical parameter vector is determined.

By utilizing a Hough ellipse detection algorithm, the standard center coordinates, the standard major axis length, the standard minor axis length, the left length of the standard ellipse major axis, the right length of the standard ellipse major axis, the upper length of the standard ellipse minor axis and the lower length of the standard ellipse minor axis of an accurate image (a two-dimensional gray scale image after three-dimensional point cloud data conversion) can be obtained and used as the standard ellipse parameter vector

Step S312, adjusting the external parameter of the camera based on the measured elliptical parameter vector and the standard elliptical parameter vector.

Optionally, the external parameters of the camera may at least include: a first rotation angle (alpha), a second rotation angle (beta), a third rotation angle (gamma), a first translation amount (T)_x) Second amount of translation (T)_y) And a third translation amount (T)_z) At least one of them.

wherein, w_{z_tvecs}The gain value corresponding to the third translation quantity; lambda [ alpha ]_{z_tvecs}The weight of the gain benefit function corresponding to the third translation quantity; a is the length of a standard long axis in the standard elliptic parameter vector; b is the standard ellipseThe length of a standard short axis in the parameter vector;

determining major axis length for the determined elliptic parameter vector;

In some optional embodiments, if the gain value corresponding to the first rotation angle is not within the range of the fourth threshold, adjusting the external parameter of the camera based on the positive, negative and magnitude of the gain value corresponding to the first rotation angle; or if the gain value corresponding to the second rotation angle is not within a fifth threshold range, adjusting the external parameter of the camera based on the positive and negative values and the size of the gain value corresponding to the second rotation angle; or if the gain value corresponding to the third rotation angle is not within a sixth threshold range, adjusting the external parameter of the camera based on the positive and negative values and the size of the gain value corresponding to the third rotation angle; or if the gain value corresponding to the first translation amount is not within a seventh threshold range, adjusting the external parameter of the camera based on the positive and negative values and the size of the gain value corresponding to the first translation amount; or if the gain value corresponding to the second translation amount is not within the range of an eighth threshold, adjusting the external parameter of the camera based on the positive and negative values and the size of the gain value corresponding to the second translation amount; or if the gain value corresponding to the third translation amount is not within a ninth threshold range, adjusting the external parameter of the camera based on the positive and negative sum of the gain value corresponding to the third translation amount.

In some optional embodiments, after the external parameter of the camera is adjusted, step S308 to step S312 are repeatedly executed until the gain value corresponding to the first rotation angle, the gain value corresponding to the second rotation angle, the gain value corresponding to the third rotation angle, the gain value corresponding to the first translation amount, the gain value corresponding to the second translation amount, and the gain value corresponding to the third translation amount meet the condition.

w＝|w_{x_recs}|+|w_{y_recs}|+|w_{z_recs}|+|w_{x_tvecs}|+|w_{y_tvecs}|+|w_{z_tvecs}| (14)

In some optional embodiments, the method further comprises:

in step S313, a store instruction is received.

In some embodiments, the apparatus continues to perform steps S308-S312 until the store instruction is received. Before the device confirms that the storage instruction is received, in the execution results of the step S308 to the step S312, the first rotation angle, the second rotation angle, the third rotation angle, the first translation amount, the second translation amount and the third translation amount corresponding to the maximum value of the overall benefit gain are determined as the external parameters of the camera.

In this way, in the embodiment of the application, the hough ellipse detection is adopted, the curvature of the elliptic calibration plate in the camera picture of the camera is calculated, and the ellipticity threshold (at least including the first inclination threshold range, the second inclination threshold range, the first threshold range, the second threshold range and the third threshold range) is set, so that the pose of the elliptic calibration plate is adjusted, and the elliptic calibration plate can be perpendicular to the optical axis of the camera as much as possible. Meanwhile, the ellipticity quantization indexes of the ellipticity range in eight directions can be measured according to the least square method of the curvature, and the adjustment indication of the elliptical calibration plate is given. After the pose of the elliptical calibration plate is set, selecting three-dimensional point cloud and two-dimensional image coordinates corresponding to N (preferably 10 to 20) groups in a laser radar coordinate system and a camera picture pixel coordinate. Since there are many calibration methods for calibrating the internal reference of the camera, it is not discussed here, and the internal reference is considered to be well measured. Then, M groups of the N groups are randomly extracted, and a PnP algorithm is adopted to solve the external parameter matrix, so that the initial external parameter is set.

And then, converting the point cloud data under the laser radar coordinate system into a two-dimensional gray scale image under a two-dimensional image coordinate system through the three-dimensional point cloud data, and performing gray scale value conversion according to the reflection intensity of the point cloud data. Next, two-dimensional boundary points of the three-dimensional point cloud on the two-dimensional gray scale are obtained by using the 6 × 9 standard of the ellipse calibration plate, and the center points and the radii of the ellipses are determined by taking the average value of the coordinates and the radii of the plurality of sets of center points under the determination condition of the four-point ellipse calibration method (the three-point ellipse calibration method may be used), and are determined as 54(6 × 9 — 54). At pixel coordinates, the center points and radii of 54(6 × 9 ═ 54) ellipses of the ellipse calibration plate were detected by hough circle transformation (circle detection). Finally, by measuring the elliptical parameter vector

And standard elliptical parameter vector

A contact ratio determination mechanism. And finally, according to the deviation measurement and the IOU of the central points of the 54 groups of ellipses on the x axis and the y axis, the automatic and intelligent adjustment of the external parameters of the camera is realized, and the accuracy of the external parameters of the camera is improved by carrying out multiple times of adjustment and optimization.

Fig. 4 shows an alternative structural schematic diagram of the camera parameter calibration apparatus provided in the embodiment of the present application, which will be described according to various parts.

In some embodiments, the camera parameter calibration apparatus 400 includes: a determination unit 401 and an adjustment unit 402.

The determining unit 401 is configured to determine the pose of the elliptical calibration plate based on the overall tilt quantization vector of the elliptical calibration plate;

the adjusting unit 402 is configured to adjust external parameters of the camera based on the data on the elliptical calibration plate after the pose is adjusted;

wherein the data is acquired by the camera.

In some embodiments, the determining unit 401 is specifically configured to:

determining an overall tilt quantization vector of the elliptical calibration plate based on an average of the azimuthal arc curvature of the inner ellipse of each of the first communication regions on the elliptical calibration plate;

In some embodiments, the determining unit 401 is specifically configured to:

and determining the average value of the azimuth arc curvature of each first communication region inner ellipse on the oval calibration plate based on the left side azimuth arc curvature, the right side azimuth arc curvature, the upper side azimuth arc curvature, the lower side azimuth arc curvature, the left upper side azimuth arc curvature, the right upper side azimuth arc curvature, the left lower side azimuth arc curvature and the right lower side azimuth arc curvature of each first communication region inner ellipse.

In some embodiments, the apparatus 400 may further include: an acquisition unit 403.

The acquiring unit 403 is configured to acquire external parameters of the camera.

The adjusting unit 402 is further configured to convert the three-dimensional point cloud data on the pose-adjusted elliptical calibration plate into a two-dimensional grayscale map;

the determining unit 401 is further configured to determine at least a first number of boundary points of each ellipse on the two-dimensional gray scale map.

The determining unit 401 is further configured to determine a measured ellipse parameter vector based on a four-point ellipse-determining algorithm and at least a first number of boundary points of each ellipse; determining a standard ellipse parameter vector of the two-dimensional gray scale image based on a Hough ellipse detection algorithm;

the adjusting unit 402 is further configured to adjust the external parameter of the camera based on the measured ellipse parameter vector and the standard ellipse parameter vector;

In some embodiments, the adjusting unit 402 is specifically configured to:

Fig. 5 is a schematic diagram of a hardware component structure of a camera external reference calibration apparatus according to an embodiment of the present application, where the camera external reference calibration apparatus 700 includes: at least one processor 701, a memory 702, and at least one network interface 704. The various components in the camera external reference calibration apparatus 700 are coupled together by a bus system 705. It is understood that the bus system 705 is used to enable communications among the components. The bus system 705 includes a power bus, a control bus, and a status signal bus in addition to a data bus. But for clarity of illustration the various busses are labeled in figure 5 as the bus system 705.

It will be appreciated that the memory 702 can be either volatile memory or nonvolatile memory, and can include both volatile and nonvolatile memory. The non-volatile Memory may be ROM, Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), magnetic random access Memory (FRAM), Flash Memory (Flash Memory), magnetic surface Memory, optical Disc, or Compact Disc Read-Only Memory (CD-ROM); the magnetic surface storage may be disk storage or tape storage. Volatile Memory can be Random Access Memory (RAM), which acts as external cache Memory. By way of illustration and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), Enhanced Synchronous Dynamic Random Access Memory (Enhanced DRAM), Synchronous Dynamic Random Access Memory (SLDRAM), Direct Memory (DRmb Access), and Random Access Memory (DRAM). The memory 702 described in embodiments herein is intended to comprise, without being limited to, these and any other suitable types of memory.

The memory 702 in the embodiments of the present application is used to store various types of data to support the operation of the camera external reference calibration apparatus 700. Examples of such data include: any computer program for operating on camera external reference calibration apparatus 700, such as application 722. A program implementing the method of an embodiment of the present application may be included in the application 722.

The method disclosed in the embodiment of the present application may be applied to the processor 701, or implemented by the processor 701. The processor 701 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the method may be performed by integrated logic circuits of hardware or instructions in the form of software in the processor 701. The Processor 701 may be a general purpose Processor, a Digital Signal Processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like. The processor 701 may implement or perform the methods, steps, and logic blocks disclosed in the embodiments of the present application. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of the method disclosed in the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium located in the memory 702, and the processor 701 may read the information in the memory 702 and perform the steps of the aforementioned methods in conjunction with its hardware.

In an exemplary embodiment, the camera external parameter calibration apparatus 700 may be implemented by one or more Application Specific Integrated Circuits (ASICs), DSPs, Programmable Logic Devices (PLDs), Complex Programmable Logic Devices (CPLDs), FPGAs, general purpose processors, controllers, MCUs, MPUs, or other electronic components for performing the aforementioned methods.

The embodiment of the application also provides a storage medium for storing the computer program.

Optionally, the storage medium may be applied to the first client in the embodiment of the present application, and the computer program enables the computer to execute corresponding processes in each method in the embodiment of the present application, which is not described herein again for brevity.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

The above description is only exemplary of the present application and should not be taken as limiting the scope of the present application, as any modifications, equivalents, improvements, etc. made within the spirit and principle of the present application should be included in the scope of the present application.

Claims

1. A camera parameter calibration method is characterized by comprising the following steps:

adjusting the pose of the elliptical calibration plate based on the integral inclination quantization vector of the elliptical calibration plate; the method comprises the following steps: determining at least one first connected region comprised by the elliptical calibration plate; determining an average value of the azimuthal arc curvature of the inner ellipse of each first communication region on the elliptical calibration plate; determining an overall tilt quantization vector of the elliptical calibration plate based on an average of the azimuthal arc curvature of the inner ellipse of each of the first communication regions on the elliptical calibration plate; determining whether a maximum value of a parameter in the global tilt quantized vector is within a first tilt threshold range and a minimum value of the parameter in the global tilt quantized vector is within a second tilt threshold range; if the maximum value of the parameters in the overall inclination quantization vector is not in a first inclination threshold range and/or the minimum value of the parameters in the overall inclination quantization vector is not in a second inclination threshold range, adjusting the pose of the elliptical calibration plate based on the relation between each parameter in the overall inclination quantization vector and the first threshold range, the second threshold range and a third threshold range; wherein the first threshold range, the second threshold range, and the third threshold range are determined based on a curvature of an ellipse in the ellipse scaling plate;

adjusting external parameters of a camera based on the data on the elliptical calibration plate after the pose is adjusted;

wherein the data is acquired by the camera.

2. The method of claim 1, wherein said determining an average of azimuthal arc curvature of an inner ellipse of each of said first communication regions on said elliptical calibration plate comprises:

3. The method of claim 1, wherein adjusting the external parameters of the camera based on the data on the adjusted elliptical calibration plate comprises:

4. The method of claim 3, further comprising:

5. The method of claim 4, wherein the adjusting the external parameters of the camera based on the measured elliptical parameter vector and the standard elliptical parameter vector comprises:

6. A camera parameter calibration device is characterized in that the device comprises:

the determining unit is used for adjusting the pose of the oval calibration plate based on the integral inclination quantization vector of the oval calibration plate; the method comprises the following steps: determining at least one first connected region comprised by the elliptical calibration plate; determining an average value of the azimuthal arc curvature of the inner ellipse of each first communication region on the elliptical calibration plate; determining an overall tilt quantization vector of the elliptical calibration plate based on an average of the azimuthal arc curvature of the inner ellipse of each of the first communication regions on the elliptical calibration plate; determining whether a maximum value of a parameter in the global tilt quantized vector is within a first tilt threshold range and a minimum value of the parameter in the global tilt quantized vector is within a second tilt threshold range; if the maximum value of the parameters in the overall inclination quantization vector is not in a first inclination threshold range and/or the minimum value of the parameters in the overall inclination quantization vector is not in a second inclination threshold range, adjusting the pose of the elliptical calibration plate based on the relation between each parameter in the overall inclination quantization vector and the first threshold range, the second threshold range and a third threshold range; wherein the first threshold range, the second threshold range, and the third threshold range are determined based on a curvature of an ellipse in the ellipse scaling plate;

the adjusting unit is used for adjusting external parameters of the camera based on the data on the elliptical calibration plate after the pose is adjusted;

wherein the data is acquired by the camera.

7. A storage medium storing an executable program, wherein the executable program, when executed by a processor, implements the camera parameter calibration method of any one of claims 1 to 5.

8. A camera parameter calibration apparatus, comprising a memory, a processor and an executable program stored on the memory and capable of being executed by the processor, wherein the processor executes the executable program to perform the steps of the camera parameter calibration method according to any one of claims 1 to 5.