CN116801100B

CN116801100B - Calibration focal length verification method and device for binocular camera module in automatic driving system

Info

Publication number: CN116801100B
Application number: CN202310814566.7A
Authority: CN
Inventors: 陈轩黎; 李江涛
Original assignee: Shanghai Jianzhi Qiji Technology Co ltd
Current assignee: Shanghai Jianzhi Qiji Technology Co ltd
Priority date: 2023-07-05
Filing date: 2023-07-05
Publication date: 2023-11-10
Anticipated expiration: 2043-07-05
Also published as: CN116801100A

Abstract

The disclosure relates to a calibration focal length verification method of a binocular camera module in an automatic driving system, a calibration focal length verification device of the binocular camera module in the automatic driving system, a calibration target, a storage medium and electronic equipment. The calibration focal length verification method comprises the steps of obtaining an original image with random stripe characteristics and dot characteristics and a target image; carrying out feature extraction and matching on random stripe features in an original image and a target image based on an image processing model, and determining a rotation angle of an imaging plane of the binocular camera module relative to a target plane of a calibration target; rotating the binocular camera module based on the rotation angle to enable the binocular camera module to reach a preset state, and obtaining a measurement focal length of the binocular camera module in the preset state; and carrying out parameter verification on the calibration focal length of the binocular camera module based on the measurement focal length of the binocular camera module to obtain a verification result, and improving the reliability of the calibration focal length verification result.

Description

Calibration focal length verification method and device for binocular camera module in automatic driving system

Technical Field

The disclosure relates to the technical field of image processing, in particular to a calibration focal length verification method of a binocular camera module in an automatic driving system, a calibration focal length verification device of the binocular camera module in the automatic driving system, a calibration target, a storage medium and electronic equipment.

Background

Camera calibration is a very important technique in the fields of computer vision and machine vision, which can be used to estimate internal and external parameters of a camera. Camera calibration is the basis for many computer vision tasks such as stereoscopic vision, motion estimation, three-dimensional reconstruction, image matching, etc. The checkerboard or dot grid calibration plate is a common tool in camera calibration, and uses a corner detection algorithm and a calibration algorithm to calculate internal parameters and external parameters of a camera by shooting a plurality of checkerboard images with different angles and positions. When the accuracy of camera calibration is not high enough, the performance of the computer vision task can also be affected. Therefore, for each specific computer vision application, a corresponding camera calibration needs to be performed, and the calibration accuracy is verified. Verifying the correctness of calibration is critical, which can affect camera performance and user usage.

Disclosure of Invention

In view of the foregoing, it is desirable to provide a calibration focal length verification method for a binocular camera module in an automatic driving system, a calibration focal length verification device for a binocular camera module in an automatic driving system, a calibration target, a storage medium and an electronic device.

The technical scheme of the present disclosure is realized as follows:

in a first aspect, the present disclosure provides a calibration focus verification method for a binocular camera module in an autopilot system.

The calibration focal length verification method of the binocular camera module in the automatic driving system provided by the embodiment of the disclosure comprises the following steps:

acquiring an original image with random stripe characteristics and dot characteristics and a target image of a calibration target manufactured based on the original image; the target image is obtained by acquiring the original image on the calibration target through the binocular camera module;

based on an image processing model, extracting and matching the characteristics of the random stripes in the original image and the target image, and determining the rotation angle of the imaging plane of the binocular camera module relative to the target plane of the calibration target;

rotating the binocular camera module based on the rotation angle of the imaging plane of the binocular camera module relative to the target plane of the calibration target, so that the binocular camera module reaches a preset state; the predetermined state is determined that the imaging plane of the binocular camera module is parallel to the target plane of the calibration target;

Obtaining a measurement focal length of the binocular camera module based on a vertical distance between the binocular camera module and the calibration target in the preset state and a mapping relation between the vertical distance and a focal length of the binocular camera module; the mapping relation between the vertical distance and the focal length of the binocular camera module is related to a first distance between two dot features of an original image on the calibration target and a second distance between two dot features of the target image; the two dot features of the target image are two feature points corresponding to the two dot features of the original image when imaging;

and carrying out parameter verification on the calibrated focal length of the binocular camera module based on the measured focal length of the binocular camera module to obtain a verification result.

In some embodiments, a first distance between two dot features of an original image on the calibration target and a second distance between two dot features of the target image are vectors; the first distance between two dot features of the original image on the calibration target comprises a first X-axis distance in the X-axis direction and a first Y-axis distance in the Y-axis direction; the second distance between the two dot features of the target image comprises a second X-axis distance in the X-axis direction and a second Y-axis distance in the Y-axis direction;

The mapping relation between the vertical distance and the focal length of the binocular camera module comprises the following steps: a first mapping relationship associated with a first X-axis distance between two dot features of the original image and a second X-axis distance between two dot features of the target image, a second mapping relationship associated with a first Y-axis distance between two dot features of the original image and a second Y-axis distance between two dot features of the target image;

the obtaining the measurement focal length of the binocular camera module based on the vertical distance between the binocular camera module and the calibration target and the mapping relation between the vertical distance and the focal length of the binocular camera module comprises the following steps:

based on the vertical distance between the binocular camera module and the calibration target and the first mapping relation, a first measurement focal length of the binocular camera module in the X-axis direction is obtained;

and obtaining a second measurement focal length of the binocular camera module in the Y-axis direction based on the vertical distance between the binocular camera module and the calibration target and the second mapping relation.

In some embodiments, the performing parameter verification on the calibration focal length of the binocular camera module based on the measurement focal length of the binocular camera module to obtain a verification result includes:

Based on the first measurement focal length, performing parameter verification on a focal length component of a calibration focal length of the binocular camera module in the X-axis direction to obtain a first verification result;

based on the second measurement focal length, performing parameter verification on a focal length component of the calibrated focal length of the binocular camera module in the Y-axis direction to obtain a second verification result;

and if the first verification result and the second verification result show that the verification is passed, determining that the calibration focal length of the binocular camera module passes the verification.

In some embodiments, the performing parameter verification on the focal length component of the calibration focal length of the binocular camera module in the X-axis direction based on the first measured focal length, to obtain a first verification result, includes:

if the difference value of the focal length components of the first measurement focal length and the calibration focal length in the X-axis direction is within a first error threshold value range, determining that the first verification result is verification passing;

and performing parameter verification on a focal length component of a calibration focal length of the binocular camera module in a Y-axis direction based on the second measurement focal length to obtain a second verification result, wherein the parameter verification comprises the following steps:

and if the difference value of the focal length components of the second measurement focal length and the calibration focal length in the Y-axis direction is within a second error threshold range, determining that the second verification result is verification passing.

In some embodiments, the random streak features of the original image have gradient information for identification by the image processing model;

the feature extraction and matching are performed on the random stripe features in the original image and the target image based on an image processing model, and the rotation angle of the imaging plane of the binocular camera module relative to the target plane of the calibration target is determined, including:

analyzing gradient information of random stripe features in the original image and the target image through the image processing model, and extracting a plurality of random stripe features from the original image and the target image;

and performing feature matching on a plurality of extracted random stripe features between the original image and the target image, and determining the rotation angle of the imaging plane of the binocular camera module relative to the target plane of the calibration target.

In some embodiments, the feature matching the extracted plurality of random streak features between the original image and the target image, determining a rotation angle of an imaging plane of the binocular camera module relative to a target plane of the calibration target, comprises:

Performing coordinate transformation between the target image and the original image based on a plurality of random stripe features matched between the original image and the target image to obtain a homography matrix representing a coordinate transformation relationship between the target image and the original image;

performing matrix solving based on the homography matrix to obtain a rotation matrix between the target image and the original image;

and carrying out matrix decomposition on the rotation matrix to obtain the rotation angle of the imaging plane of the binocular camera module relative to the target plane of the calibration target.

In some embodiments, the mapping between the vertical distance and the focal length of the binocular camera module comprises:

the method comprises the steps of carrying out a first treatment on the surface of the Wherein f is the focal length of the binocular camera module, ">For a first distance between two dot features of an original image on said calibration target, +.>For a second distance between two dot features of the target image,/>And the vertical distance between the binocular camera module and the calibration target is set.

In some embodiments, the dot feature is a combination of an inner circle feature and an outer ring feature; wherein the inner circle feature is a black inner circle feature; the outer ring features are white outer ring features;

A plurality of dot features are distributed in an array in the original image; the random stripe feature surrounds each of the dot features;

the first distance between two dot features of the original image on the calibration target and the second distance between two dot features of the target image are determined as the distance between the circle centers of the two inner circle features.

In a second aspect, the present disclosure provides a calibration target for a calibration focal length verification method of a binocular camera module in an autopilot system according to the first aspect, including:

a body;

the body is provided with an original image;

the original image has random stripe characteristics and dot characteristics; wherein,

the random streak feature has gradient information for identification by an image processing model.

In some embodiments, the dot feature is a combination of an inner circle feature and an outer ring feature; wherein the inner circle feature is a black inner circle feature; the outer ring feature is a white outer ring feature.

In a third aspect, the present disclosure provides a calibration focus verification apparatus for a binocular camera module in an autopilot system, comprising:

The image acquisition module is used for acquiring an original image with random stripe characteristics and dot characteristics and a target image of a calibration target manufactured based on the original image; the target image is obtained by acquiring the original image on the calibration target through the binocular camera module;

the rotation angle determining module is used for extracting and matching the characteristics of the random stripes in the original image and the target image based on an image processing model, and determining the rotation angle of the imaging plane of the binocular camera module relative to the target plane of the calibration target;

the distance correction module is used for rotating the binocular camera module based on the rotation angle of the imaging plane of the binocular camera module relative to the target plane of the calibration target so that the binocular camera module reaches a preset state; the predetermined state is determined that the imaging plane of the binocular camera module is parallel to the target plane of the calibration target;

the measuring focal length determining module is used for obtaining the measuring focal length of the binocular camera module based on the vertical distance between the binocular camera module and the calibration target in the preset state and the mapping relation between the vertical distance and the focal length of the binocular camera module; the mapping relation between the vertical distance and the focal length of the binocular camera module is related to a first distance between two dot features of an original image on the calibration target and a second distance between two dot features of the target image; the two dot features of the target image are two feature points corresponding to the two dot features of the original image when imaging;

And the parameter verification module is used for carrying out parameter verification on the calibrated focal length of the binocular camera module based on the measured focal length of the binocular camera module to obtain a verification result.

the measuring focal length determining module is used for

In some embodiments, the parameter verification module is configured to

the rotation angle determining module is used for

In some embodiments, the rotation angle determination module is configured to

In a fourth aspect, the present disclosure provides a computer readable storage medium, on which a calibration focus verification program of a binocular camera module in an autopilot system is stored, where the calibration focus verification program of the binocular camera module in the autopilot system is executed by a processor, to implement the calibration focus verification method of the binocular camera module in the autopilot system described in the first aspect.

In a fifth aspect, the disclosure provides an electronic device, including a memory, a processor, and a calibration focal length verification program of a binocular camera module in an autopilot system stored in the memory and capable of running on the processor, where the processor implements the calibration focal length verification method of the binocular camera module in the autopilot system described in the first aspect when executing the calibration focal length verification program of the binocular camera module in the autopilot system.

The calibration focal length verification method of the binocular camera module in the automatic driving system comprises the steps of obtaining an original image with random stripe characteristics and dot characteristics and a target image of a calibration target manufactured based on the original image; the target image is obtained by acquiring an original image on a calibration target through a binocular camera module; based on an image processing model, carrying out feature extraction and matching on random stripe features in an original image and a target image, and determining a rotation angle of an imaging plane of the binocular camera module relative to a target plane of a calibration target; rotating the binocular camera module based on the rotation angle of the imaging plane of the binocular camera module relative to the target plane of the calibration target, so that the binocular camera module reaches a preset state; the preset state is determined that the binocular camera module is parallel to the target plane of the calibration target; obtaining a measurement focal length of the binocular camera module based on a vertical distance between the binocular camera module and the calibration target in a preset state and a mapping relation between the vertical distance and the focal length of the binocular camera module; wherein, the mapping relation between the vertical distance and the focal length of the binocular camera module is related to the first distance between two dot features of the original image on the calibration target and the second distance between two dot features of the target image; the two dot features of the target image are two corresponding feature points when the two dot features of the original image are imaged; and carrying out parameter verification on the calibrated focal length of the binocular camera module based on the measured focal length of the binocular camera module to obtain a verification result. In the application, the original image with random stripe characteristics and dot characteristics is adopted to manufacture the calibration target. And carrying out feature extraction and matching on random stripe features in the original image and the target image through an image processing model, and determining the rotation angle of the imaging plane of the binocular camera module relative to the target plane of the calibration target. Because the random stripe features have gradient information for the image processing model to recognize, the image processing model can also extract a large number of random stripe features for matching under an illumination scene, so that the more accurate rotation angle of the imaging plane of the binocular camera module relative to the target plane of the calibration target is obtained, the angle rotation of the binocular camera module is carried out, the preset state is reached, the vertical distance between the binocular camera module and the calibration target is the calibration distance, and the data support is provided for obtaining the accurate measurement focal length of the binocular camera module. Parameter verification is carried out by accurately measuring the focal length, so that the reliability of a calibration focal length verification result is improved.

Additional aspects and advantages of the disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the disclosure.

Drawings

FIG. 1 is a flow chart illustrating a method of calibrating focal length verification of a binocular camera module in an autopilot system, according to one exemplary embodiment;

FIG. 2 is a schematic diagram of an original image of a calibration focus verification method for a binocular camera module in an autopilot system, according to one exemplary embodiment;

FIG. 3 is a schematic view of an original image under illumination and after rotation angle in a calibration focus verification method of a binocular camera module in an automatic driving system according to an exemplary embodiment;

FIG. 4 is a schematic diagram of a target image of an original image acquired by a binocular camera module under illumination or after a rotational angle, according to an exemplary embodiment;

fig. 5 is a schematic diagram illustrating a calibration focus verification device of a binocular camera module in an automatic driving system according to an exemplary embodiment.

Detailed Description

Embodiments of the present disclosure are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and intended for the purpose of explaining the present disclosure and are not to be construed as limiting the present disclosure.

In view of the above, the present disclosure provides a calibration focal length verification method for a binocular camera module in an autopilot system. FIG. 1 is a flow chart illustrating a method for calibrating focus verification of a binocular camera module in an autopilot system, according to one exemplary embodiment. As shown in fig. 1, the calibration focal length verification method of the binocular camera module in the automatic driving system includes:

Step 10, acquiring an original image with random stripe characteristics and dot characteristics and a target image of a calibration target manufactured based on the original image; the target image is obtained by acquiring the original image on the calibration target through the binocular camera module;

step 11, based on an image processing model, extracting and matching the characteristics of the random stripes in the original image and the target image, and determining the rotation angle of the imaging plane of the binocular camera module relative to the target plane of the calibration target;

step 12, rotating the binocular camera module based on the rotation angle of the imaging plane of the binocular camera module relative to the target plane of the calibration target, so that the binocular camera module reaches a preset state; the predetermined state is determined that the imaging plane of the binocular camera module is parallel to the target plane of the calibration target;

step 13, obtaining a measurement focal length of the binocular camera module based on a vertical distance between the binocular camera module and the calibration target in the preset state and a mapping relation between the vertical distance and a focal length of the binocular camera module; the mapping relation between the vertical distance and the focal length of the binocular camera module is related to a first distance between two dot features of an original image on the calibration target and a second distance between two dot features of the target image; the two dot features of the target image are two feature points corresponding to the two dot features of the original image when imaging;

And 14, performing parameter verification on the calibrated focal length of the binocular camera module based on the measured focal length of the binocular camera module to obtain a verification result.

In the present exemplary embodiment, when verifying calibration parameters of a camera, a common method is to calculate a re-projection error by using a checkerboard or dot grid calibration plate to evaluate the calibration accuracy of the camera. The basic flow comprises the steps of shooting checkerboard images, detecting angular points, calibrating cameras, calculating re-projection errors and the like. Specifically, it is first necessary to take a plurality of checkerboard images of different angles and positions using a camera, and to ensure that the checkerboard is in a planar state, and that all corner points of the checkerboard can be completely seen by the camera view angle. Then, for each checkerboard image, corner points of the checkerboard are detected using a corner detection algorithm.

Although the reprojection error is an evaluation index commonly used in camera calibration, it can be used to evaluate the accuracy and consistency of parameters inside and outside the camera. However, the re-projection error does not evaluate the depth calculation accuracy of the camera well, because it only considers pixel coordinate errors on the image plane, and does not consider the depth calculation error of the camera. Therefore, in some applications requiring high-precision depth information, other evaluation indicators need to be used. In addition, in practical application, the universal checkerboard is often used as a target for calibration and then used as a target for verification, so that two problems are coupled together, and the verification and positioning problems are not facilitated. Thus, a different validation target needs to be selected to separate the two problems.

In this regard, the present application employs an original image having random stripe features and dot features, and a target image of a target that is calibrated based on the original image. Fig. 2 is a schematic diagram of an original image in a calibration focus verification method of a binocular camera module in an automatic driving system according to an exemplary embodiment. As shown in fig. 2, the original image has random stripe features and dot features. Fig. 3 is a schematic diagram of an original image under illumination and after a rotation angle in a calibration focus verification method of a binocular camera module in an automatic driving system according to an exemplary embodiment. Fig. 4 is a schematic diagram of a target image of an original image acquired by a binocular camera module under illumination or after a rotation angle, according to an exemplary embodiment. And the target image is obtained by acquiring the original image on the calibration target by the binocular camera module.

In the present exemplary embodiment, random streak features in the original image and the target image are feature extracted and matched to determine the rotation angle of the imaging plane of the binocular camera module relative to the target plane of the target. Based on the rotation angle of the imaging plane of the binocular camera module relative to the target plane of the calibration target, the angle rotation of the binocular camera module is carried out to reach a preset state, so that the vertical distance between the imaging plane of the binocular camera module and the calibration target is the calibration distance. Based on the vertical distance between the binocular camera module and the calibration target and the mapping relation between the vertical distance and the focal length of the binocular camera module, the measurement focal length of the binocular camera module is obtained, and parameter verification is carried out on the calibration focal length of the binocular camera module. The whole process only needs to carry out distance correction on the calibration distance between the binocular camera module and the calibration target, does not need to calculate the re-projection error, and does not have the problem of coupling of the depth calculation error and the re-projection error.

Meanwhile, the original image with random stripe characteristics and dot characteristics is adopted to manufacture the calibration target. And carrying out feature extraction and matching on random stripe features in the original image and the target image through an image processing model, and determining the rotation angle of the imaging plane of the binocular camera module relative to the target plane of the calibration target. As shown in fig. 4, since the random stripe features have gradient information for the image processing model to recognize, the image processing model can also extract a large number of random stripe features for matching in the illumination scene, so as to obtain a more accurate rotation angle of the imaging plane of the binocular camera module relative to the target plane of the calibration target, and perform the angular rotation of the binocular camera module to reach a predetermined state, so that the vertical distance between the binocular camera module and the calibration target is the calibration distance, thereby providing data support for obtaining the accurate measurement focal length of the binocular camera module. Parameter verification is carried out by accurately measuring the focal length, so that the reliability of a calibration focal length verification result is improved.

In the present exemplary embodiment, an original image having gradient information may be generated. And then printing the original image according to a certain proportion to manufacture a calibration target. The binocular camera module is spaced from the calibration target by a predetermined distance, and the camera imaging plane is parallel to the target plane of the calibration target to acquire the original image on the calibration target. Wherein, the calibration target can be arranged on the wall surface.

In the present exemplary embodiment, the image processing model may be a SIFT (Scale-invariant feature transform ) algorithm. And carrying out feature extraction and matching on the random stripe features in the original image and the target image through SIFT. The present example is only, and other feature extraction matching algorithms that can identify gradient information may be employed instead of SIFT.

In this exemplary embodiment, the performing parameter verification on the calibration focal length of the binocular camera module based on the measurement focal length of the binocular camera module to obtain a verification result includes:

In the present exemplary embodiment, a coordinate system may be established first when feature point extraction is performed, and coordinates correspond to each pixel point in the original image and the target image. The original image and the target image may be established under the same coordinate system. When the parameter verification is performed on the calibration focal length of the binocular camera module, the calibration focal length needs to be divided into a focal length component in the X-axis direction and a focal length component in the Y-axis direction for parameter verification.

Meanwhile, the measurement focal length for performing parameter verification on the calibration focal length is also divided into a first measurement focal length in the X-axis direction and a second measurement focal length in the Y-axis direction. And carrying out parameter verification on the focal length component of the calibrated focal length of the binocular camera module in the X-axis direction through a first measurement focal length, and carrying out parameter verification on the focal length component of the calibrated focal length of the binocular camera module in the Y-axis direction through a second measurement focal length.

When the first measurement focal length and the second measurement focal length are determined, the first measurement focal length of the binocular camera module in the X-axis direction can be obtained according to the vertical distance between the binocular camera module and the calibration target and the first mapping relation, and the second measurement focal length of the binocular camera module in the Y-axis direction can be obtained according to the vertical distance between the binocular camera module and the calibration target and the second mapping relation. Wherein,

The mapping relation between the vertical distance and the focal length of the binocular camera module comprises the following steps:

The first mapping relation is:wherein->For the first measurement focal length of the binocular camera module in the X-axis direction, < >>For a second X-axis distance between two dot features of the target image,/>A first X-axis distance between two dot features of the original image; />A vertical distance between the binocular camera module and the calibration target;

the second mapping relation is:wherein->Second measurement focal length of binocular camera module in Y-axis direction, ">For a second Y-axis distance between two dot features of the target image,/>Is the first Y-axis distance between two dot features of the original image.

The method is used for verifying the precision of the calibration focal length of the binocular camera module, if the verification is not passed, the calibration focal length is not qualified, and the binocular camera module can be recalibrated according to the requirement.

In the present exemplary embodiment, a first error threshold value and a second error threshold value are set for judging the calibration accuracy. If the difference value of the focal length component of the first measurement focal length and the focal length component of the calibration focal length in the X-axis direction is within a first error threshold value range, the calibration accuracy verification of the focal length component of the calibration focal length in the X-axis direction is verified, and the first verification result is determined to be verification passing;

And if the difference value of the focal length component of the second measurement focal length and the focal length component of the calibration focal length in the Y-axis direction is within a second error threshold value range, the calibration accuracy verification of the focal length component of the calibration focal length in the Y-axis direction is verified, and the second verification result is determined to be verification passing.

In this exemplary embodiment, the feature matching the extracted plurality of random stripe features between the original image and the target image, determining a rotation angle of an imaging plane of the binocular camera module with respect to a target plane of the calibration target, includes:

In the present exemplary embodiment, a large number of accurate matching points can be extracted from the target image and the original image by a feature expression based on gradient information such as SIFT. The transformation relationship between the target image and the original image can be described by a homographic homography. Solving for homography using matching points can be accomplished by a findhomography conversion matrix lookup function of OpenCV (Open Source Computer VisionLibrary, open-source computer vision library). Because of the large number of matching points provided, the final result is solved by the overdetermined equation with all points, which is more stable and reliable than the result of using only tens of points for dots and checkerboards. The homographic theory is calculated by using the matching points as follows:

And performing feature matching on the target image and the original image, wherein two point sets are respectively marked as X and X'. Fitting the relationship of the two with a homography transformation can be expressed as:

；

wherein,is the coordinates of the feature point in X' -, a +.>The coordinate of the feature point in X is the homography matrix, and H represents the transformation relation between the homography matrix and the homography matrix. H is a matrix of 3*3, and there are 8 degrees of freedom, so there are 8 unknown parameters to be solved, where (u, v) is the feature point coordinates of the original image, and (x, y) is the feature point coordinates of the target image.

；

The DLT algorithm derivation process is as follows:

expanding the formula (1), dividing the first 2 rows by the third row to obtain

；

Formulas (2), (3) may be arranged as:

the method comprises the steps of carrying out a first treatment on the surface of the Wherein,

；

from the number of unknown variables, solveHAt least 4 pairs of matching points are required. Typically, more than 4 pairs of feature matches will be used in order to obtain more stable results. Therefore, this equation becomes overdetermined, and the least squares solution can be regarded as the final solution.

The least squares solution of the equation has a defined conclusion, i.e. forAAnd (3) carrying out SVD singular value decomposition, wherein the right singular vector corresponding to the minimum singular value of A is the solution of h. Performing reshape matrix transformation on h (reshape transforming the specified matrix into a matrix with a specific dimension) to obtain H。

Obtaining homography matrixHThen, extracting the rotation matrix from the HomographyRAnd obtaining the rotation angle and the angle of the camera relative to the wall surface.

Because the default calibration target is attached to a flat wall surface, the rotation matrix can be calculated by using the following formula, and the rotation angle is extracted:

（5）；

（6）；/>

(7) The method comprises the steps of carrying out a first treatment on the surface of the Wherein t is a translation vector,Kis a camera intrinsic matrix (3*3),Ris a rotation matrix (3*3); from equation (6), get (7). For a pair ofRDecomposing to obtain Euler angles, namely the rotation angles of the imaging planes of the binocular camera modules relative to the target plane of the calibration target; wherein (1)>、/>Are both rotation vectors.

The Eulerian angles are used to determine 3 independent sets of angular parameters for the fixed point rotating rigid body position, consisting of nutation angle α, precession angle β, and rotation angle γ. Wherein,

；/>；

the method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>、/>、/>、/>、/>Is thatRAnd 5 values corresponding to the rows and columns are taken in the matrix. And after the Euler angle is obtained, the binocular camera module is rotated, so that the binocular camera module is in a preset state, and the vertical distance between the binocular camera module and the calibration target is the calibration distance.

In this exemplary embodiment, when the first distance between the two dot features of the original image on the calibration target and the second distance between the two dot features of the target image are obtained, the distance between the circle centers of the inner circle features of the two dot features of the original image on the calibration target may be taken as the first distance between the two dot features of the original image on the calibration target, and the distance between the circle centers of the inner circle features of the two dot features of the target image may be taken as the second distance between the two dot features of the target image. When the dot features are extracted from the original image and the target image, the dot feature extraction can be performed through a dot detection equation of OpenCV.

The present disclosure provides a calibration target for a calibration focal length verification method of a binocular camera module in an automatic driving system according to the above embodiments, including:

A body;

the body is provided with an original image;

The disclosure provides a calibration focal length verification device of a binocular camera module in an automatic driving system. Fig. 5 is a schematic diagram illustrating a calibration focus verification device of a binocular camera module in an automatic driving system according to an exemplary embodiment. As shown in fig. 5, the calibration focal length verification device of the binocular camera module in the automatic driving system includes:

an image acquisition module 50 for acquiring an original image having random stripe features and dot features, and a target image of a calibration target made based on the original image; the target image is obtained by acquiring the original image on the calibration target through the binocular camera module;

The rotation angle determining module 51 is configured to perform feature extraction and matching on the random stripe features in the original image and the target image based on an image processing model, and determine a rotation angle of an imaging plane of the binocular camera module relative to a target plane of the calibration target;

a distance correction module 52, configured to rotate the binocular camera module based on a rotation angle of an imaging plane of the binocular camera module relative to a target plane of the calibration target, so that the binocular camera module reaches a predetermined state; the predetermined state is determined that the imaging plane of the binocular camera module is parallel to the target plane of the calibration target;

a measurement focal length determining module 53, configured to obtain a measurement focal length of the binocular camera module based on a vertical distance between the binocular camera module and the calibration target in the predetermined state and a mapping relationship between the vertical distance and a focal length of the binocular camera module; the mapping relation between the vertical distance and the focal length of the binocular camera module is related to a first distance between two dot features of an original image on the calibration target and a second distance between two dot features of the target image; the two dot features of the target image are two feature points corresponding to the two dot features of the original image when imaging;

And the parameter verification module 54 is configured to perform parameter verification on the calibration focal length of the binocular camera module based on the measurement focal length of the binocular camera module, so as to obtain a verification result.

In this regard, the present application employs an original image having random stripe features and dot features, and a target image of a target that is calibrated based on the original image. And the target image is obtained by acquiring the original image on the calibration target by the binocular camera module.

In the present exemplary embodiment, random streak features in the original image and the target image are feature extracted and matched to determine the rotation angle of the imaging plane of the binocular camera module relative to the target plane of the target. Based on the rotation angle of the imaging plane of the binocular camera module relative to the target plane of the calibration target, the angle rotation of the binocular camera module is carried out to reach a preset state, so that the vertical distance between the binocular camera module and the calibration target is the calibration distance. Based on the vertical distance between the binocular camera module and the calibration target and the mapping relation between the vertical distance and the focal length of the binocular camera module, the measurement focal length of the binocular camera module is obtained, and parameter verification is carried out on the calibration focal length of the binocular camera module. The whole process only needs to carry out distance correction on the calibration distance between the binocular camera module and the calibration target, does not need to calculate the re-projection error, and does not have the problem of coupling of the depth calculation error and the re-projection error.

In the present exemplary embodiment, an original image having gradient information may be generated. And then printing the original image according to a certain proportion to manufacture a calibration target. The binocular camera module is spaced from the calibration target by a predetermined distance, and the camera imaging plane is parallel to the target plane of the calibration target to acquire the original image on the calibration target.

The measuring focal length determining module is used for

In the present exemplary embodiment, the parameter verification module is configured to

the method comprises the steps of carrying out a first treatment on the surface of the Wherein f is the focal length of the binocular camera module, ">For a first distance between two dot features of an original image on said calibration target, +.>For a second distance between two dot features of the target image,/ >And the vertical distance between the binocular camera module and the calibration target is set.

In some embodiments, the parameter verification module is configured to

The random streak feature of the original image in the present exemplary embodiment has gradient information for the image processing model to recognize;

the rotation angle determining module is used for

In the present exemplary embodiment, the rotation angle determination module is configured to

In the present exemplary embodiment, a large number of accurate matching points can be extracted in the target image and the original image by SIFT. The transformation relationship between the target image and the original image may be described by a homographic. Solving the homograph using the matching points may be implemented by a findhomograph function of OpenCV. Because of the large number of matching points provided, the final result is solved by the overdetermined equation with all points, which is more stable and reliable than the result of using only tens of points for dots and checkerboards. The homographic theory is calculated by using the matching points as follows:

and performing feature matching on the target image and the original image, wherein two point sets are respectively marked as X and X'. Fitting the relationship between the two with homography transformation can be expressed as

；

Wherein,is the coordinates of the feature point in X' -, a +.>The coordinate of the feature point in X is the homography matrix, and H represents the transformation relation between the homography matrix and the homography matrix. HIs a 3*3 matrix, with 8 degrees of freedom, so there are 8 unknown parameters to be solved,

；

the DLT algorithm derivation process is as follows:

expanding the formula (1), dividing the first 2 rows by the third row to obtain

；

Formulas (2), (3) may be arranged as:

；/>

from the number of unknown variables, at least 4 pairs of matching points are needed to solve for HH. Typically, more than 4 pairs of feature matches will be used in order to obtain more stable results. Therefore, this equation becomes overdetermined, and the least squares solution can be regarded as the final solution.

The least squares solution of the equation has a defined conclusion, i.e. forAThe SVD decomposition is carried out and,Athe right singular vector corresponding to the smallest singular value of (a) is the solution of h. The reshape is carried out on h to obtainH。

（5）；

（6）；

(7) The method comprises the steps of carrying out a first treatment on the surface of the Wherein t is a translation vector, K is a camera internal reference matrix (3*3),Ris a rotation matrix (3*3); from equation (6), get (7). For a pair ofRAnd decomposing to obtain an Euler angle, namely the rotation angle of the imaging plane of the binocular camera module relative to the target plane of the calibration target.

；/>；

the method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>、/>、/>、/>、/>Is thatRAnd 5 values corresponding to the rows and columns are taken in the matrix.

The present disclosure provides a computer readable storage medium, on which a calibration focus verification program of a binocular camera module in an automatic driving system is stored, where the calibration focus verification program of the binocular camera module in the automatic driving system is executed by a processor, to implement the calibration focus verification method of the binocular camera module in the automatic driving system described in the above embodiments.

The present disclosure provides an electronic device, including a memory, a processor, and a calibration focal length verification program of a binocular camera module in an autopilot system stored in the memory and capable of running on the processor, where the processor implements the calibration focal length verification method of the binocular camera module in the autopilot system described in the above embodiments when executing the calibration focal length verification program of the binocular camera module in the autopilot system.

It should be noted that the logic and/or steps represented in the flowcharts or otherwise described herein, for example, may be considered as a ordered listing of executable instructions for implementing logical functions, and may be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium may even be paper or other suitable medium upon which the program is printed, as the program may be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.

It should be understood that portions of the present disclosure may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.

In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present disclosure. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

In the description of the present disclosure, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings are merely for convenience in describing the present disclosure and simplifying the description, and do not indicate or imply that the device or element being referred to must have a specific orientation, be configured and operated in a specific orientation, and therefore should not be construed as limiting the present disclosure.

Furthermore, the terms "first," "second," and the like, as used in embodiments of the present disclosure, are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated in the present embodiment. Thus, a feature of an embodiment of the present disclosure that is defined by terms such as "first," "second," and the like may explicitly or implicitly indicate that at least one such feature is included in the embodiment. In the description of the present disclosure, the word "plurality" means at least two or more, for example, two, three, four, etc., unless explicitly specified otherwise in the examples.

In this disclosure, unless expressly specified or limited otherwise in the examples, the terms "mounted," "connected," and "secured" and the like as used in the examples are intended to be broadly construed, as for example, the connection may be a fixed connection, may be a removable connection, or may be integral, and as may be a mechanical connection, an electrical connection, or the like; of course, it may be directly connected, or indirectly connected through an intermediate medium, or may be in communication with each other, or in interaction with each other. The specific meaning of the terms in this disclosure will be understood by those of ordinary skill in the art depending on the specific implementation.

In this disclosure, unless expressly stated or limited otherwise, a first feature "up" or "down" a second feature may be the first and second features in direct contact, or the first and second features in indirect contact through an intervening medium. Moreover, a first feature being "above," "over" and "on" a second feature may be a first feature being directly above or obliquely above the second feature, or simply indicating that the first feature is level higher than the second feature. The first feature being "under", "below" and "beneath" the second feature may be the first feature being directly under or obliquely below the second feature, or simply indicating that the first feature is less level than the second feature.

Although embodiments of the present disclosure have been shown and described above, it will be understood that the above embodiments are illustrative and not to be construed as limiting the present disclosure, and that variations, modifications, alternatives, and variations may be made to the above embodiments by one of ordinary skill in the art within the scope of the present disclosure.

Claims

1. A calibration focal length verification method of a binocular camera module in an automatic driving system is characterized by comprising the following steps:

2. The method for verifying the calibration focal length of a binocular camera module in an autopilot system of claim 1 wherein a first distance between two dot features of an original image on the calibration target and a second distance between two dot features of the target image are both vectors; the first distance between two dot features of the original image on the calibration target comprises a first X-axis distance in the X-axis direction and a first Y-axis distance in the Y-axis direction; the second distance between the two dot features of the target image comprises a second X-axis distance in the X-axis direction and a second Y-axis distance in the Y-axis direction;

3. The method for verifying the calibration focal length of the binocular camera module in the automatic driving system according to claim 2, wherein the performing parameter verification on the calibration focal length of the binocular camera module based on the measurement focal length of the binocular camera module to obtain a verification result includes:

4. The method for verifying the calibration focal length of the binocular camera module in the automatic driving system according to claim 3, wherein the performing parameter verification on the focal length component of the calibration focal length of the binocular camera module in the X-axis direction based on the first measurement focal length to obtain a first verification result includes:

5. The method for verifying the calibration focus of a binocular camera module in an autopilot system of claim 1 wherein the random streak features of the raw image have gradient information for recognition by the image processing model;

6. The method of claim 5, wherein the feature matching the extracted plurality of random streak features between the original image and the target image, determining the rotation angle of the imaging plane of the binocular camera module relative to the target plane of the calibration target comprises:

7. The method for verifying the calibrated focal length of a binocular camera module in an autopilot system of claim 1, wherein the mapping between the vertical distance and the focal length of the binocular camera module comprises:

8. The method for calibrating focal length verification of binocular camera module in automatic driving system according to any one of claims 1-7, wherein the dot feature is a combination of an inner circle feature and an outer circle feature; wherein the inner circle feature is a black inner circle feature; the outer ring features are white outer ring features;

9. A calibration target for use in a method of calibrating focal length verification of a binocular camera module in an autopilot system according to any one of claims 1-8, wherein the calibration target comprises:

A body;

the body is provided with an original image;

10. The calibration target of claim 9, wherein the dot features are a combination of inner circle features and outer circle features; wherein the inner circle feature is a black inner circle feature; the outer ring feature is a white outer ring feature.

11. A calibration focus verification device for a binocular camera module in an autopilot system, comprising:

12. The device for verifying the calibrated focal length of a binocular camera module of claim 11, wherein a first distance between two dot features of an original image on the calibrated target and a second distance between two dot features of the target image are vectors; the first distance between two dot features of the original image on the calibration target comprises a first X-axis distance in the X-axis direction and a first Y-axis distance in the Y-axis direction; the second distance between the two dot features of the target image comprises a second X-axis distance in the X-axis direction and a second Y-axis distance in the Y-axis direction;

the measuring focal length determining module is used for

13. The apparatus of claim 12, wherein the parameter verification module is configured to

14. The apparatus of claim 13, wherein the parameter verification module is configured to

15. The calibration focus verification device of a binocular camera module in an autopilot system of claim 11 wherein the random streak features of the raw image have gradient information for recognition by the image processing model;

the rotation angle determining module is used for

16. The apparatus of claim 15, wherein the rotation angle determining module is configured to

17. The apparatus for verifying a calibrated focal length of a binocular camera module of an autopilot system of claim 11 wherein the mapping between the vertical distance and the focal length of the binocular camera module comprises:

18. The calibration focus verification device of a binocular camera module in an autopilot system of any one of claims 11-17, wherein the dot feature is a combination of an inner circle feature and an outer circle feature; wherein the inner circle feature is a black inner circle feature; the outer ring features are white outer ring features;

19. A computer readable storage medium, wherein a calibration focus verification program for a binocular camera module in an autopilot system is stored thereon, which when executed by a processor, implements the calibration focus verification method for a binocular camera module in an autopilot system of any one of claims 1-8.

20. An electronic device comprising a memory, a processor and a calibration focus verification program for a binocular camera module in an autopilot system stored on the memory and operable on the processor, the processor implementing the calibration focus verification method for a binocular camera module in an autopilot system of any one of claims 1-8 when executing the calibration focus verification program for a binocular camera module in the autopilot system.