CN117726691A - Binocular camera calibration method and device, aircraft and storage medium - Google Patents
Binocular camera calibration method and device, aircraft and storage medium Download PDFInfo
- Publication number
- CN117726691A CN117726691A CN202311711413.6A CN202311711413A CN117726691A CN 117726691 A CN117726691 A CN 117726691A CN 202311711413 A CN202311711413 A CN 202311711413A CN 117726691 A CN117726691 A CN 117726691A
- Authority
- CN
- China
- Prior art keywords
- image
- binocular
- matching
- points
- feature point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 238000001514 detection method Methods 0.000 claims abstract description 40
- 238000010586 diagram Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 6
- 230000006870 function Effects 0.000 description 4
- 230000008447 perception Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- 230000032683 aging Effects 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Landscapes
- Image Processing (AREA)
Abstract
The invention discloses a binocular camera calibration method, a binocular camera calibration device, an aircraft and a storage medium, wherein the binocular camera calibration method comprises the following steps: acquiring a binocular image of a binocular camera; detecting the feature points of the binocular image to obtain a left image feature point and a right image feature point; and matching the left image characteristic points and the right image characteristic points through strong constraint based on the binocular structure characteristics to obtain a matching result. The invention can improve the efficiency of feature point detection and optimize the efficiency and accuracy of feature point matching.
Description
Technical Field
The invention relates to the technical field of aircrafts, in particular to a binocular camera calibration method and device, an aircraft and a storage medium.
Background
The binocular camera module is one of important equipment for sensing the surrounding environment of the aerocar, and the binocular camera module can cause structural change due to long-term flight vibration, high temperature, ageing of structural devices and the like, so that the surrounding environment cannot be sensed normally, and the calibration parameters are generally required to be calculated and updated by a self-calibration algorithm.
Feature detection and matching in the self-calibration algorithm are very critical links. The current self-calibration algorithm generally adopts an ORB (Oriented FAST and Rotated BRIEF, rapid feature point extraction description), SIFT (Scale-invariant feature transform, scale invariant feature transform) or SURF (Speeded Up Robust Features, accelerated robust feature) algorithm to extract feature points on feature detection, and the feature point algorithms extract feature points in an image and rank the features, so that the top N good feature points are reserved. The disadvantage of this approach is that: if a local strong texture scene exists in the image, the N characteristic points are piled up, so that the characteristic points are unevenly distributed, and the accuracy of a calibration result is affected; meanwhile, the feature point algorithms can take multi-layer pyramid downsampling for scale invariance, which results in time consumption. However, the present feature matching algorithm generally uses fast nearest neighbor search matching (FLANN) and Brute Force matching (Brute Force), and the disadvantages of these matching algorithms are that: one feature point of the left image needs to be compared with all feature points of the right image, even if KNN is adopted, the complexity is between O (Log 2 (N)) and O (N), when the number of feature points is large, the matching efficiency is extremely low, and if more repeated textures exist in a scene, the matching error rate is also high.
Disclosure of Invention
The invention mainly aims to provide a binocular camera calibration method, a binocular camera calibration device, an aircraft and a storage medium, and aims to improve the efficiency of binocular camera calibration and the accuracy of calibration results.
In order to achieve the above object, the present invention provides a binocular camera calibration method, which includes:
acquiring a binocular image of a binocular camera;
detecting the feature points of the binocular image to obtain a left image feature point and a right image feature point;
and matching the left image characteristic points and the right image characteristic points through strong constraint based on the binocular structure characteristics to obtain a matching result.
Optionally, the step of matching the left graph feature point and the right graph feature point through strong constraint based on the binocular structure feature to obtain a matching result includes:
based on binocular structure characteristics, the left-right uniqueness check and the matching cost uniqueness check are used for carrying out strong constraint, and the left-image characteristic points and the right-image characteristic points are matched to obtain a matching result.
Optionally, the step of matching the left graph feature point and the right graph feature point includes:
matching the left image characteristic points and the right image characteristic points;
if the ith feature point of the left graph is matched with the jth feature point of the right graph and the jth feature point of the right graph is matched with the ith feature point of the left graph to be closest, reserving the matched feature point pair (i, j).
Optionally, the step of matching the left graph feature point and the right graph feature point includes:
matching the left image characteristic points and the right image characteristic points;
and selecting feature points with optimal distance and suboptimal distance from the right graph feature points matched with the left graph feature points, wherein the matching cost with optimal distance is smaller than the matching cost with suboptimal distance multiplied by a preset coefficient.
Optionally, the step of acquiring the binocular image of the binocular camera includes:
acquiring an original left image and an original right image of a binocular camera;
unifying the original left image and the original right image to a preset fixed focal length;
and generating a undistorted left image and a undistorted right image under the unified focal length based on the fixed focal length.
Optionally, the step of detecting feature points of the binocular image to obtain the left image feature points and the right image feature points includes:
and detecting the feature points of the binocular image based on a preset feature point detection algorithm to obtain the left image feature points and the right image feature points.
Optionally, the step of detecting feature points of the binocular image to obtain the left image feature points and the right image feature points includes:
and respectively dividing the undistorted left graph and the undistorted right graph into M x N grids, and reserving local strongest characteristic points in each grid to obtain left graph characteristic points and right graph characteristic points, wherein M, N is a positive integer.
The embodiment of the invention also provides a binocular camera calibration device, which comprises:
the acquisition module is used for acquiring binocular images of the binocular camera;
the detection module is used for detecting the characteristic points of the binocular image to obtain a left image characteristic point and a right image characteristic point;
and the matching module is used for matching the left image characteristic points and the right image characteristic points through strong constraint based on the binocular structure characteristics to obtain a matching result.
The embodiment of the invention also provides an aircraft, which comprises: the system comprises a memory, a processor and a binocular camera calibration program stored on the memory and executable on the processor, wherein the binocular camera calibration program is configured to implement the steps of the binocular camera calibration method.
The embodiment of the invention also provides a storage medium, wherein a binocular camera calibration program is stored on the storage medium, and the binocular camera calibration program realizes the steps of the binocular camera calibration method when being executed by a processor.
The binocular camera calibration method, the binocular camera calibration device, the terminal equipment and the storage medium provided by the embodiment of the invention are used for acquiring binocular images of the binocular camera; detecting the feature points of the binocular image to obtain a left image feature point and a right image feature point; and matching the left image characteristic points and the right image characteristic points through strong constraint based on the binocular structure characteristics to obtain a matching result. According to the scheme, the binocular structure characteristics of the binocular camera are considered, and the strong constraint is added to match the characteristic points of the binocular image, so that the characteristic point matching efficiency and accuracy can be optimized.
Drawings
FIG. 1 is a schematic diagram of functional modules of a terminal device to which a binocular camera calibration apparatus of the present invention belongs;
FIG. 2 is a flow chart of a first exemplary embodiment of a binocular camera calibration method of the present invention;
FIG. 3 is a flow chart of a second exemplary embodiment of a binocular camera calibration method of the present invention;
FIG. 4 is a flow chart of a third exemplary embodiment of a binocular camera calibration method of the present invention;
FIG. 5 is a schematic diagram of a conventional feature point detection and matching process;
FIG. 6 is a detailed flowchart of the method for calibrating a binocular camera for detecting and matching feature points according to an embodiment of the present invention;
fig. 7 is a schematic diagram of functional modules of the binocular camera calibration apparatus of the present invention.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
The main solutions of the embodiments of the present invention are: obtaining a binocular image of a binocular camera; detecting the feature points of the binocular image to obtain a left image feature point and a right image feature point; and matching the left image characteristic points and the right image characteristic points through strong constraint based on the binocular structure characteristics to obtain a matching result. Because the binocular structure characteristics of the binocular camera are considered, strong constraint is added to match the characteristic points of the binocular image, so that the characteristic point matching efficiency and accuracy can be optimized.
Technical terms related to the embodiment of the invention:
binocular camera: refers to a camera system consisting of two lenses placed in parallel. The method acquires richer information by capturing and analyzing images from multiple angles simultaneously, and can acquire depth information of objects in a scene by calculating parallax between two cameras, thereby realizing stereoscopic vision perception and depth calculation, realizing three-dimensional reconstruction and depth perception and achieving stereoscopic vision effect. The binocular camera is widely applied to the fields of aircrafts, robot navigation, augmented reality, virtual reality and the like, and can provide more accurate functions of target positioning, obstacle detection, distance measurement and the like. Because the binocular camera has two lenses, the binocular camera is more complex than the monocular camera, and camera calibration and image registration are required, namely, the calibration of the mutual positions of the two cameras is realized.
ORB, SIFT or SURF algorithm
The ORB (Oriented FAST and Rotated BRIEF, FAST feature point extraction description) algorithm is a feature detection algorithm, which is a combination of FAST and BRIEF feature detection algorithms.
The basic principle of the ORB algorithm is: firstly, using a FAST feature point detection algorithm to detect corner points from an image, then adopting a Harris corner point detection algorithm to carry out non-maximum suppression on the corner points so as to obtain key points, and finally using a BRIEF algorithm to calculate descriptors of each key point so as to extract the key point features of the image.
The SIFT (Scale-invariant feature transform) algorithm is a descriptor for the field of image processing. The description has scale invariance, can detect key points in an image, and is a local characteristic descriptor.
SIFT feature detection mainly includes the following 4 basic steps:
(1) Extremum detection of scale space: images on all scale spaces are searched, and potential scale-versus-selection-invariant interest points are identified through Gaussian differential functions.
(2) Characteristic point positioning: at each candidate location, the location scale is determined by a fitting fine model, and the key points are selected according to their degree of stability.
(3) Feature direction assignment: based on the direction of the gradient of the image portion, assigned to each keypoint location one or more directions, all subsequent operations are transformations of the direction, scale and location of the keypoint, providing invariance to these features.
(4) Description of characteristic points: within a neighborhood around each feature point, local gradients of the image are measured on a selected scale, and these gradients are transformed into a representation that allows for deformation and illumination transformation of relatively large local shapes.
SIFT feature matching mainly includes two phases: the generation of SIFT features and the matching of SIFT feature vectors.
The SIFT feature is generated by extracting feature vectors irrelevant to scale, rotation and brightness change from a plurality of images. Wherein, generation of SIFT features generally includes: 1. constructing a scale space, detecting extreme points, and obtaining scale invariance; 2. filtering the characteristic points and accurately positioning; 3. assigning a direction value to the feature point; 4. and generating a feature descriptor.
Taking a neighborhood of 16 multiplied by 16 as a sampling window by taking the characteristic point as the center, classifying the relative directions of the sampling point and the characteristic point into a direction histogram containing 8 bins after Gaussian weighting, and finally obtaining the 128-dimensional characteristic descriptor of 4 multiplied by 8.
The SURF (Speeded Up Robust Features, accelerated robust features) algorithm is a robust image recognition and description algorithm that can be used for computer vision tasks such as object recognition and 3D reconstruction. The SURF algorithm is an accelerated version of the SIFT algorithm.
In the embodiment of the invention, considering the current self-calibration algorithm, feature points are generally extracted by ORB, SIFT or SURF algorithms in feature detection, and the feature points in the image are extracted and ranked by the feature point algorithms, so that the top N good feature points are reserved. The disadvantage of this approach is that: if a local strong texture scene exists in the image, the N feature points are piled up, so that the feature points are unevenly distributed; meanwhile, the feature point algorithms can take multi-layer pyramid downsampling for scale invariance, which results in time consumption. However, the present feature matching algorithm generally uses fast nearest neighbor search matching (FLANN) and Brute Force matching (Brute Force), and the disadvantages of these matching algorithms are that: one feature point of the left image needs to be compared with all feature points of the right image, even if KNN is adopted, the complexity is between O (Log 2 (N)) and O (N), when the number of feature points is large, the matching efficiency is extremely low, and if more repeated textures exist in a scene, the matching error rate is also high.
Based on the above, the embodiment of the invention provides a solution, which can provide characteristic detection and matching for self-calibration more accurately, stably and efficiently, so that the self-calibration result is more accurate.
Specifically, referring to fig. 1, fig. 1 is a schematic diagram of functional modules of a terminal device to which the binocular camera calibration apparatus of the present invention belongs. The binocular camera calibration apparatus may be a terminal-independent apparatus capable of data processing, which may be carried on the terminal apparatus in the form of hardware or software. The terminal device may be an aircraft with flight functions, such as a car, which in this embodiment is exemplified by a car.
In this embodiment, the terminal device to which the binocular camera calibration apparatus belongs at least includes an output module 110, a processor 120, a memory 130 and a communication module 140.
The memory 130 stores an operating system and a binocular camera calibration program, and the binocular camera calibration apparatus may store information such as binocular images, left image feature points, right image feature points, and matching results of the binocular camera in the memory 130; the output module 110 may be a display screen or the like. The communication module 140 may include a WIFI module, a bluetooth module, and the like, and communicate with an external device or a server through the communication module 140.
Wherein the binocular camera calibration program in the memory 130, when executed by the processor, performs the steps of:
acquiring a binocular image of a binocular camera;
detecting the feature points of the binocular image to obtain a left image feature point and a right image feature point;
and matching the left image characteristic points and the right image characteristic points through strong constraint based on the binocular structure characteristics to obtain a matching result.
Further, the binocular camera calibration program in the memory 130, when executed by the processor, also performs the steps of:
based on binocular structure characteristics, the left-right uniqueness check and the matching cost uniqueness check are used for carrying out strong constraint, and the left-image characteristic points and the right-image characteristic points are matched to obtain a matching result.
Matching the left image characteristic points and the right image characteristic points;
if the ith feature point of the left graph is matched with the jth feature point of the right graph and the jth feature point of the right graph is matched with the ith feature point of the left graph to be closest, reserving the matched feature point pair (i, j).
Further, the binocular camera calibration program in the memory 130, when executed by the processor, also performs the steps of:
matching the left image characteristic points and the right image characteristic points;
and selecting feature points with optimal distance and suboptimal distance from the right graph feature points matched with the left graph feature points, wherein the matching cost with optimal distance is smaller than the matching cost with suboptimal distance multiplied by a preset coefficient.
Further, the binocular camera calibration program in the memory 130, when executed by the processor, also performs the steps of:
acquiring an original left image and an original right image of a binocular camera;
unifying the original left image and the original right image to a preset fixed focal length;
and generating a undistorted left image and a undistorted right image under the unified focal length based on the fixed focal length.
Further, the binocular camera calibration program in the memory 130, when executed by the processor, also performs the steps of:
and detecting the feature points of the binocular image based on a preset feature point detection algorithm to obtain the left image feature points and the right image feature points.
Further, the binocular camera calibration program in the memory 130, when executed by the processor, also performs the steps of:
and respectively dividing the undistorted left graph and the undistorted right graph into M x N grids, and reserving local strongest characteristic points in each grid to obtain left graph characteristic points and right graph characteristic points, wherein M, N is a positive integer.
According to the embodiment, through the scheme, the binocular images of the binocular camera are obtained; detecting the feature points of the binocular image to obtain a left image feature point and a right image feature point; and matching the left image characteristic points and the right image characteristic points through strong constraint based on the binocular structure characteristics to obtain a matching result. According to the scheme, the binocular structure characteristics of the binocular camera are considered, and the strong constraint is added to match the characteristic points of the binocular image, so that the characteristic point matching efficiency and accuracy can be optimized.
The method embodiment of the invention is proposed based on the above-mentioned terminal equipment architecture but not limited to the above-mentioned architecture.
Referring to fig. 2, fig. 2 is a flowchart of a first exemplary embodiment of the binocular camera calibration method of the present invention. The embodiment provides a binocular camera calibration method, which can be applied to an aircraft, and comprises the following steps:
step S101, obtaining a binocular image of a binocular camera;
the main implementation body of the method of the embodiment may be an aircraft, such as a flying car, and the embodiment is exemplified by the flying car.
The binocular camera or the binocular camera module is arranged on the aerocar, and the surrounding environment of the aerocar is perceived through the binocular camera or the binocular camera module to obtain binocular images.
Wherein the binocular image includes a left image and a right image.
The embodiment proposal mainly relates to the self-calibration of the binocular camera, so as to provide characteristic detection and matching for the self-calibration more accurately, stably and efficiently, and ensure that the self-calibration result of the binocular camera is more accurate.
First, a binocular image of a binocular camera is acquired so that feature point detection is performed on the binocular image of the binocular camera.
When the binocular image of the binocular camera is acquired, the distortion removal processing can be performed on the binocular image, so that the accuracy of the subsequent feature point detection on the binocular image of the binocular camera is improved, and the accuracy of the self-calibration result of the binocular camera is further improved.
Step S102, detecting characteristic points of the binocular image to obtain a left image characteristic point and a right image characteristic point;
the feature points of the image can be simply understood as more significant points in the image, such as contour points, bright points in darker areas, dark points in lighter areas, and the like.
For the detection of the image feature points, the detection may be based on a preset feature point detection algorithm.
Specifically, the feature points of the left image and the right image in the binocular image may be detected based on a preset feature point detection algorithm, so as to obtain the feature points of the left image and the feature points of the right image.
The preset feature point detection algorithm includes, but is not limited to ORB, SIFT, SURF, FAST, shi-tomasi and the like.
In addition, the characteristic points of the left image and the right image in the binocular image can be detected by dividing the image into grids, so that the characteristic points of the left image and the characteristic points of the right image are obtained, the problem of uneven distribution of the characteristic points can be relieved, and the accuracy of the calibration result of the binocular camera is improved.
And step S103, matching the left image characteristic points and the right image characteristic points through strong constraint based on the binocular structure characteristics to obtain a matching result.
The binocular structure characteristic of the binocular camera mainly means that the binocular camera is a camera system formed by two lenses which are placed in parallel, richer information is obtained by capturing and analyzing images from multiple angles at the same time, depth information of objects in a scene can be obtained by calculating parallax between the two cameras, and stereoscopic perception and depth calculation are realized, so that three-dimensional reconstruction and depth perception are realized, and stereoscopic visual effect is achieved.
In the embodiment, based on the binocular structure characteristics of the binocular camera, strong constraint is added in feature point matching to match the left image feature point and the right image feature point, so that the efficiency and the accuracy of feature point matching are improved.
Wherein the strong constraint may include: left-right uniqueness checking and matching cost uniqueness checking.
In one embodiment, the step of matching the left graph feature point and the right graph feature point by strong constraint based on the binocular structure feature to obtain a matching result may include:
based on binocular structure characteristics, the left-right uniqueness check and the matching cost uniqueness check are used for carrying out strong constraint, and the left-image characteristic points and the right-image characteristic points are matched to obtain a matching result.
In one embodiment, the step of matching the left graph feature point and the right graph feature point by performing a strong constraint through a left-right uniqueness check may include:
matching the left image characteristic points and the right image characteristic points;
if the ith feature point of the left graph is matched with the jth feature point of the right graph and the jth feature point of the right graph is matched with the ith feature point of the left graph to be closest, reserving the matched feature point pair (i, j).
In one embodiment, the step of matching the left graph feature point and the right graph feature point by performing strong constraint through matching cost uniqueness check may include:
matching the left image characteristic points and the right image characteristic points;
and selecting feature points with optimal distance and suboptimal distance from the right graph feature points matched with the left graph feature points, wherein the matching cost with optimal distance is smaller than the matching cost with suboptimal distance multiplied by a preset coefficient.
More specifically, the binocular structure characteristic based on the binocular camera matches the left image feature point and the right image feature point through strong constraint to obtain a matching result, and the following implementation scheme can be adopted:
from the binocular structural feature analysis of the binocular camera, a certain feature point (lx, ly) of the left image should not be excessively different in the y direction of the matching position of the right image, so that comparison with feature points of all positions of the right image is not required, only feature points corresponding to the right image ly±t can be searched, wherein a proper value can be defined according to the image resolution.
In order to improve the efficiency and accuracy of feature point matching, optimization is performed by adding two constraints, wherein:
one is the uniqueness check of the left and right images, when the ith feature point of the left image is required to be matched with the jth feature point of the right image to be nearest, the jth feature point of the right image is required to be matched with the ith feature point of the left image to be nearest, so that the matched feature point pairs are reserved;
and the other is the matching cost uniqueness check, wherein the characteristic points with optimal distance and suboptimal distance are selected from the right graph characteristic points matched with the left graph characteristic points, the matching cost required to be optimal in distance is smaller than the matching cost with suboptimal distance multiplied by the coefficient theta, and optionally, the theta can be set to be about 0.4.
According to the embodiment, through the scheme, the binocular images of the binocular camera are obtained; detecting the feature points of the binocular image to obtain a left image feature point and a right image feature point; and matching the left image characteristic points and the right image characteristic points through strong constraint based on the binocular structure characteristics to obtain a matching result. According to the scheme, the binocular structure characteristics of the binocular camera are considered, and the strong constraint is added to match the characteristic points of the binocular image, so that the characteristic point matching efficiency and accuracy can be optimized.
Referring to fig. 3, fig. 3 is a flowchart illustrating a second exemplary embodiment of a binocular camera calibration method according to the present invention. Based on the first embodiment shown in fig. 2, the present embodiment refines the binocular image acquired by the binocular camera in step S101.
In one embodiment, the step S101, acquiring the binocular image of the binocular camera may include:
s1011, acquiring an original left image and an original right image of the binocular camera;
s1012, unifying the original left image and the original right image to a preset fixed focal length;
and S1013, generating a undistorted left image and a undistorted right image under a unified focal length based on the fixed focal length.
Specifically, in this embodiment, by acquiring an original left image and an original right image of the binocular camera, unifying the original left image and the original right image to a preset fixed focal length, then generating a de-distorted left image and a de-distorted right image under the unified focal length based on the fixed focal length, and then detecting feature points of the de-distorted left image and the de-distorted right image under the unified focal length.
Compared with the prior art, the method has the advantages that the distortion removal processing is independently carried out on the left image and the right image, and the focal lengths of the left image and the right image which are subjected to distortion removal are not completely consistent, so that when the characteristic points are detected later, the image is required to be subjected to multi-layer pyramid downsampling operation for the purpose of scale invariance, and time is consumed.
In the scheme of the embodiment, the left and right images are unified under a proper fixed focal length, and then the left and right undistorted images are generated based on the fixed focal length, so that the scale problem is not needed to be considered in the detection of the characteristic points, and the detection efficiency of the characteristic points can be improved.
More specifically, as an embodiment, the appropriate focal length value f 'preset in the left-right diagram can be obtained by the following formula' x And f' y ：
f′ x ＝max(fl x ，fr x )；
f′ y ＝max(fl y ，fr y )；
Wherein the focal length in the x, y directions of the left graph is (fl x ，fl y ) The focal length in the x, y directions of the right graph is (fr x ，fr y )。
Therefore, the left image and the right image can be unified to a proper focal length, the problem of scale scaling is not needed to be considered under the unified focal length, the pyramid downsampling step is omitted, the optimization is time-consuming, and the calibration efficiency of the binocular camera is improved.
According to the technical scheme, the original left image and the original right image of the binocular camera are obtained, the original left image and the original right image are unified to a preset fixed focal length, then a de-distorted left image and a de-distorted right image under the unified focal length are generated based on the fixed focal length, and feature point detection is carried out on the de-distorted left image and the de-distorted right image under the unified focal length, so that left image feature points and right image feature points are obtained; and matching the left image characteristic points and the right image characteristic points through strong constraint based on the binocular structure characteristics to obtain a matching result. According to the scheme, the left image and the right image are unified to a proper fixed focal length, and then the left and right undistorted images are generated based on the fixed focal length, so that the scale problem is not required to be considered in the detection of the feature points, the efficiency of the feature point detection can be improved, the feature points of the binocular images are matched by adding strong constraint in consideration of the binocular structure characteristics of the binocular cameras, and the feature point matching efficiency and accuracy can be optimized.
Referring to fig. 4, fig. 4 is a flowchart of a third exemplary embodiment of the binocular camera calibration method of the present invention. Based on the above embodiments, in the present embodiment, in step S102, feature points of the binocular image are detected, and the left image feature points and the right image feature points are obtained for refinement.
Specifically, the step S102, detecting the feature points of the binocular image, and obtaining the left image feature points and the right image feature points includes:
step S1021, dividing the undistorted left image and the undistorted right image into M grids;
in step S1022, local strongest feature points are reserved in each grid, and a left graph feature point and a right graph feature point are obtained, where M, N is a positive integer.
Specifically, in order to alleviate the problem of uneven distribution of feature points, in the embodiment, the de-distorted left graph and the de-distorted right graph are segmented into m×n grids, and local strongest feature points are reserved in each grid, so that a left graph feature point and a right graph feature point are obtained, and then the left graph feature point and the right graph feature point can be matched. The method improves the problem of uneven distribution of the characteristic points through the method of dividing the grid by the image, and enables the self-calibration result to be more accurate.
According to the technical scheme, the original left image and the original right image of the binocular camera are obtained, unified to a preset fixed focal length, then the undistorted left image and the undistorted right image under the unified focal length are generated based on the fixed focal length, feature point detection is carried out on the undistorted left image and the undistorted right image under the unified focal length, the undistorted left image and the undistorted right image are respectively segmented into M x N grids, and local strongest feature points are reserved in each grid to obtain left image feature points and right image feature points; based on binocular structure characteristics, the left image characteristic points and the right image characteristic points are matched through strong constraint to obtain a matching result, so that the problem of uneven distribution of the characteristic points is solved through the method of dividing the grid by the image, the self-calibration result is more accurate, the binocular structure characteristics of the binocular camera are considered, the characteristic points of the binocular image are matched through the strong constraint, and therefore the characteristic point matching efficiency and accuracy can be optimized.
The flow of feature point detection and matching in the binocular camera calibration method of the present embodiment can be compared with the flow of existing feature point detection and matching, and reference may be made to fig. 5 and 6, where fig. 5 is a schematic flow diagram of existing feature point detection and matching, and fig. 6 is a detailed flow diagram of feature point detection and matching in the binocular camera calibration method of the present embodiment.
In addition, as shown in fig. 7, an embodiment of the present invention further provides a binocular camera calibration apparatus, where the apparatus includes:
the acquisition module is used for acquiring binocular images of the binocular camera;
the detection module is used for detecting the characteristic points of the binocular image to obtain a left image characteristic point and a right image characteristic point;
and the matching module is used for matching the left image characteristic points and the right image characteristic points through strong constraint based on the binocular structure characteristics to obtain a matching result.
The principle and implementation process of the binocular camera calibration are implemented in this embodiment, please refer to the above embodiments, and will not be described in detail here.
In addition, an embodiment of the present invention also proposes an aircraft, including: the system comprises a memory, a processor and a binocular camera calibration program stored on the memory and executable on the processor, wherein the binocular camera calibration program is configured to implement the steps of the binocular camera calibration method as described in the above embodiments.
Because all the technical solutions of all the embodiments are adopted when the binocular camera calibration program is executed by the processor, at least all the beneficial effects brought by all the technical solutions of all the embodiments are provided, and the description is omitted herein.
In addition, the embodiment of the invention also provides a storage medium, and a binocular camera calibration program is stored on the storage medium, and the binocular camera calibration program is configured to implement the steps of the binocular camera calibration method described in the embodiment.
Because all the technical solutions of all the embodiments are adopted when the binocular camera calibration program is executed by the processor, at least all the beneficial effects brought by all the technical solutions of all the embodiments are provided, and the description is omitted herein.
It should be noted that, the functions of the data storage operation in the embodiment may be integrated into one functional unit alone, or may be integrated into a plurality of functional unit modules, where the storage of the data is not limited to the eeprom, and other storage media such as the ram, the rom, the Flash memory, the usb, the removable hard disk, the magnetic disk and the optical disk may be replaced as the storage media.
Compared with the prior art, the binocular camera calibration method, the device, the aircraft and the storage medium provided by the embodiment of the invention are characterized in that the original left image and the original right image of the binocular camera are obtained, the original left image and the original right image are unified to a preset fixed focal length, then a de-distorted left image and a de-distorted right image under the unified focal length are generated based on the fixed focal length, feature point detection is carried out on the de-distorted left image and the de-distorted right image under the unified focal length, the de-distorted left image and the de-distorted right image are respectively segmented into M and N grids, and local strongest feature points are reserved in each grid to obtain left image feature points and right image feature points; based on binocular structural characteristics, the left image characteristic points and the right image characteristic points are matched through strong constraint to obtain a matching result, therefore, the left image and the right image are unified under a proper fixed focal length, and then the left image and the right image which are subjected to distortion removal are generated based on the fixed focal length, so that the scale problem is not required to be considered in the detection of the characteristic points, the efficiency of the detection of the characteristic points can be improved, the problem of uneven distribution of the characteristic points is solved through the method of dividing the grids by the images, the self-calibration result is more accurate, the binocular structural characteristics of the binocular camera are considered, the strong constraint is added to match the characteristic points of the binocular image, and the characteristic point matching efficiency and accuracy can be optimized.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) as described above, comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.
Claims (10)
1. A binocular camera calibration method, the method comprising:
acquiring a binocular image of a binocular camera;
detecting the feature points of the binocular image to obtain a left image feature point and a right image feature point;
and matching the left image characteristic points and the right image characteristic points through strong constraint based on the binocular structure characteristics to obtain a matching result.
2. The method according to claim 1, wherein the step of matching the left graph feature point and the right graph feature point by strong constraint based on the binocular structure characteristics, to obtain a matching result comprises:
based on binocular structure characteristics, the left-right uniqueness check and the matching cost uniqueness check are used for carrying out strong constraint, and the left-image characteristic points and the right-image characteristic points are matched to obtain a matching result.
3. The method of claim 2, wherein the step of matching the left graph feature point and the right graph feature point by strong constraint through left-right uniqueness check comprises:
matching the left image characteristic points and the right image characteristic points;
if the ith feature point of the left graph is matched with the jth feature point of the right graph and the jth feature point of the right graph is matched with the ith feature point of the left graph to be closest, reserving the matched feature point pair (i, j).
4. The method of claim 2, wherein the step of matching the left graph feature point and the right graph feature point by strongly constraining a matching cost uniqueness check comprises:
matching the left image characteristic points and the right image characteristic points;
and selecting feature points with optimal distance and suboptimal distance from the right graph feature points matched with the left graph feature points, wherein the matching cost with optimal distance is smaller than the matching cost with suboptimal distance multiplied by a preset coefficient.
5. The method of claim 1, wherein the step of acquiring a binocular image of a binocular camera comprises:
acquiring an original left image and an original right image of a binocular camera;
unifying the original left image and the original right image to a preset fixed focal length;
and generating a undistorted left image and a undistorted right image under the unified focal length based on the fixed focal length.
6. The method of claim 5, wherein the step of detecting feature points of the binocular image to obtain left and right map feature points comprises:
and detecting the feature points of the binocular image based on a preset feature point detection algorithm to obtain the left image feature points and the right image feature points.
7. The method of claim 5, wherein the step of detecting feature points of the binocular image to obtain left and right map feature points comprises:
and respectively dividing the undistorted left graph and the undistorted right graph into M x N grids, and reserving local strongest characteristic points in each grid to obtain left graph characteristic points and right graph characteristic points, wherein M, N is a positive integer.
8. A binocular camera calibration apparatus, the apparatus comprising:
the acquisition module is used for acquiring binocular images of the binocular camera;
the detection module is used for detecting the characteristic points of the binocular image to obtain a left image characteristic point and a right image characteristic point;
and the matching module is used for matching the left image characteristic points and the right image characteristic points through strong constraint based on the binocular structure characteristics to obtain a matching result.
9. An aircraft, the aircraft comprising: a memory, a processor and a binocular camera calibration program stored on the memory and executable on the processor, the binocular camera calibration program configured to implement the steps of the binocular camera calibration method of any one of claims 1 to 7.
10. A storage medium having stored thereon a binocular camera calibration program which, when executed by a processor, implements the steps of the binocular camera calibration method of any of claims 1 through 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311711413.6A CN117726691A (en) | 2023-12-12 | 2023-12-12 | Binocular camera calibration method and device, aircraft and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311711413.6A CN117726691A (en) | 2023-12-12 | 2023-12-12 | Binocular camera calibration method and device, aircraft and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117726691A true CN117726691A (en) | 2024-03-19 |
Family
ID=90199251
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311711413.6A Pending CN117726691A (en) | 2023-12-12 | 2023-12-12 | Binocular camera calibration method and device, aircraft and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117726691A (en) |
-
2023
- 2023-12-12 CN CN202311711413.6A patent/CN117726691A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110135455B (en) | Image matching method, device and computer readable storage medium | |
| US9177381B2 (en) | Depth estimate determination, systems and methods | |
| EP2711670B1 (en) | Visual localisation | |
| JP6091560B2 (en) | Image analysis method | |
| US10311595B2 (en) | Image processing device and its control method, imaging apparatus, and storage medium | |
| CN110070096B (en) | Local frequency domain descriptor generation method and device for non-rigid shape matching | |
| CN107240130B (en) | Remote sensing image registration method, device and system | |
| Son et al. | A multi-vision sensor-based fast localization system with image matching for challenging outdoor environments | |
| Byrne et al. | Maximizing feature detection in aerial unmanned aerial vehicle datasets | |
| CN111798453A (en) | Point cloud registration method and system for unmanned auxiliary positioning | |
| Paradowski et al. | Local keypoints and global affine geometry: Triangles and ellipses for image fragment matching | |
| Avidar et al. | Local-to-global point cloud registration using a dictionary of viewpoint descriptors | |
| US20230100238A1 (en) | Methods and systems for determining the 3d-locations, the local reference frames and the grasping patterns of grasping points of an object | |
| Atik et al. | An automatic image matching algorithm based on thin plate splines | |
| Wan et al. | A performance comparison of feature detectors for planetary rover mapping and localization | |
| CN117726691A (en) | Binocular camera calibration method and device, aircraft and storage medium | |
| JP2016206909A (en) | Information processor, and information processing method | |
| Lv et al. | Three-dimensional laser scanning under the pinhole camera with lens distortion | |
| KR102511821B1 (en) | Methods and apparatus for processing image data for machine vision | |
| Ye et al. | Rotation invariant feature lines transform for image matching | |
| Ito et al. | A 3D reconstruction method with color reproduction from multi-band and multi-view images | |
| Yaman et al. | Performance evaluation of similarity measures for dense multimodal stereovision | |
| Ruiz-Rodriguez et al. | Reconstruction of 3D deformable objects using a single Kinect sensor | |
| CN113570667B (en) | Visual inertial navigation compensation method and device and storage medium | |
| CN113177903B (en) | Fusion method, system and equipment of foreground point cloud and background point cloud |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |