CN110874854A - Large-distortion wide-angle camera binocular photogrammetry method based on small baseline condition - Google Patents
Large-distortion wide-angle camera binocular photogrammetry method based on small baseline condition Download PDFInfo
- Publication number
- CN110874854A CN110874854A CN202010060072.0A CN202010060072A CN110874854A CN 110874854 A CN110874854 A CN 110874854A CN 202010060072 A CN202010060072 A CN 202010060072A CN 110874854 A CN110874854 A CN 110874854A
- Authority
- CN
- China
- Prior art keywords
- image
- binocular
- camera
- point
- space
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 38
- 238000005259 measurement Methods 0.000 claims abstract description 36
- 238000012937 correction Methods 0.000 claims abstract description 19
- 238000009434 installation Methods 0.000 claims abstract description 10
- 238000001914 filtration Methods 0.000 claims abstract description 9
- 238000006243 chemical reaction Methods 0.000 claims description 15
- 239000011159 matrix material Substances 0.000 claims description 13
- 230000009466 transformation Effects 0.000 claims description 13
- 238000004364 calculation method Methods 0.000 claims description 10
- 230000001360 synchronised effect Effects 0.000 claims description 10
- 238000003384 imaging method Methods 0.000 claims description 9
- 238000001514 detection method Methods 0.000 claims description 4
- 238000007689 inspection Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 abstract description 7
- 238000005516 engineering process Methods 0.000 description 8
- 230000036544 posture Effects 0.000 description 7
- 238000012986 modification Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000004927 fusion Effects 0.000 description 4
- UDHXJZHVNHGCEC-UHFFFAOYSA-N Chlorophacinone Chemical compound C1=CC(Cl)=CC=C1C(C=1C=CC=CC=1)C(=O)C1C(=O)C2=CC=CC=C2C1=O UDHXJZHVNHGCEC-UHFFFAOYSA-N 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 2
- 241000282414 Homo sapiens Species 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 210000003128 head Anatomy 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T7/85—Stereo camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
- G06T2207/10012—Stereo images
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Length Measuring Devices By Optical Means (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a binocular photogrammetry method of a large-distortion wide-angle camera based on a small baseline condition, which comprises the following steps of firstly calibrating a relative position relation and an internal parameter coefficient between binocular cameras based on a preset indoor high-precision calibration field; then, correcting the binocular camera in real time by adopting a distortion coefficient of the wide-angle camera obtained by calibration to obtain a binocular image after distortion correction; then, obtaining the positions of the feature points with the same name based on epipolar geometry, SIFT feature matching and Randac filtering; and calculating to obtain the light direction of the binocular image matched with the homonymous point according to the installation position relationship between the POS system and the binocular camera, the attitude information measured by the POS system, the relative position relationship between the binocular cameras and the position of the homonymous feature point, and finally obtaining the position information of the ground feature. The method of the invention has the advantages of small measurement system equipment, simple measurement process, large measurement range, high precision and good robustness.
Description
Technical Field
The invention relates to the technical field of mobile photogrammetry and computer vision, in particular to a binocular photogrammetry method for a large-distortion wide-angle camera based on a small baseline condition.
Background
Binocular stereo vision is a method for acquiring three-dimensional geometric information of an object from a plurality of images based on the parallax principle. In a machine vision system, in binocular vision, two digital images of surrounding scenery are generally acquired simultaneously from different angles by two cameras, or two digital images of the surrounding scenery are acquired from different angles at different times by a single camera, and three-dimensional geometric information of an object can be recovered based on a parallax principle, so that the three-dimensional shape and position of the surrounding scenery are reconstructed. Binocular vision, sometimes also called stereoscopic vision, is a main way for human beings to acquire environmental three-dimensional information by using both eyes. From the present, with the development of the machine vision theory, binocular stereo vision plays an increasingly important role in the machine vision research.
The mathematical principle of binocular vision is as follows: the binocular stereo vision is based on parallax, and three-dimensional information is acquired by a trigonometry principle, namely a triangle is formed between the image planes of two cameras and a north object. The three-dimensional size of the object in the common field of view of the two cameras and the three-dimensional coordinates of the characteristic points of the space object can be obtained by keeping the position relationship between the two cameras. Therefore, binocular vision systems are generally composed of two cameras. The binocular vision calibration method generally adopted in the prior art is mainly used for acquiring three-dimensional information based on the trigonometry principle.
The inventor of the present application finds that the method of the prior art has at least the following technical problems in the process of implementing the present invention:
the existing method can not be realized due to the problems of equipment volume, measurement flow and the like when the hidden requirement of a special measurement task is met.
Therefore, the method in the prior art has the technical problems that the method is not suitable for the hidden condition and the measurement precision is not high.
Disclosure of Invention
In view of the above, the present invention provides a binocular photogrammetry method for a large-distortion wide-angle camera based on a small baseline condition, so as to solve or at least partially solve the technical problems that the method in the prior art is not suitable for a hidden condition and has low measurement accuracy.
The invention provides a binocular photogrammetry method for a large-distortion wide-angle camera based on a small baseline condition, which comprises the following steps:
step S1: calibrating a relative position relation and an internal reference coefficient between the binocular cameras based on a preset indoor high-precision calibration field, wherein the internal reference coefficient comprises a distortion coefficient;
step S2: correcting the binocular camera in real time by adopting a distortion coefficient of the wide-angle camera obtained by calibration to obtain a binocular image after distortion correction;
step S3: reducing a matching region based on a epipolar geometry theory, and then performing SIFT feature matching and Randac filtering in the reduced matching region to obtain a binocular image matching homonymy point pair after distortion correction; establishing a Gaussian pyramid in a Gaussian kernel in a scale space by an SIFT feature matching algorithm in an epipolar geometry constrained region; and constructing a Gaussian difference pyramid based on the Gaussian pyramid: a DOG pyramid; carrying out extreme value detection in the DOG pyramid to detect and obtain the positions of the characteristic points with the same name;
step S4: according to the installation position relation between the POS system and the binocular cameras, the attitude information obtained by measurement of the POS system, the relative position relation between the binocular cameras and the positions of the homonymous feature points, the light direction of the binocular images matched with the homonymous points is obtained through calculation, then forward intersection operation is carried out according to the light direction of the binocular images matched with the homonymous points in a synchronous and asynchronous measurement mode, and the position information of the ground objects is obtained.
In one embodiment, step S1 specifically includes:
step S1.1: the photoelectric pod respectively shoots a section of video towards the left, the middle and the right of an inspection school field, so that six videos are obtained, and the left camera and the right camera respectively correspond to 3 videos;
step S1.2: checking and correcting the shot video screenshot, manually selecting a preset number of control points with known geodetic coordinates as calculation conditions, and calculating an initial value of the attitude by using DLT direct linear transformation in photogrammetry;
step S1.3: based on the initial value of the attitude, performing least square adjustment by using back intersection, iteratively eliminating gross error points, and obtaining the instantaneous attitude of six photos
And the internal reference coefficient of the camera
；
Step S1.4: according to the instantaneous posture of six photos
The relative positional relationship between the binocular cameras is obtained.
In one embodiment, the gesture in step S1.3
And the internal reference coefficient of the camera
The formula of (1) is:
wherein,
in (1),
as a camera
The focal length of,
Is like a main point,
Is a distortion coefficient;
in (1),
as a camera
The relative position of the two or more of the three or more of the,
as a camera
The posture of (2).
In one embodiment, step S1.3 specifically includes:
step S1.3.1: calculating a conversion relation between the object space coordinate and the image space coordinate by utilizing the principle of a collinear equation, and obtaining a proportional relation between the object space coordinate and the image coordinate of the point according to the conversion relation:
wherein,
representing the object space coordinates of the ground object point,
representing the image space coordinates of the ground feature point, when the incident light is 90 degrees, the distance between the corresponding image point and the image principal point is R, and when the incident light is R
When the distance between the corresponding image point and the image principal point is R, the ratio between R and R is:
Step S1.3.2: obtaining a unified model calibrated by the wide-angle camera according to the conversion relation between the object space coordinates and the image space coordinates and the relation between the distances between the image points corresponding to the incident angles of different light rays and the image principal point:
wherein,
s1, s2 denotes a conversion coefficient between an image coordinate system and a camera coordinate system, r denotes an imaging radius,
representing the distance between the image point and the center of the image principal point;
step S1.3.3: according to the unified model calibrated by the wide-angle camera, iterative solution is carried out on the parameters to be solved by utilizing a least square method to obtain the attitude
And inside of cameraParameter coefficient
。
In one embodiment, step S2 specifically includes: distortion coefficient in unified model calibrated by wide-angle camera
、
、
、
、
、
、
The binocular camera is corrected in real time to obtain a binocular image after distortion correction, wherein a distortion correction formula is as follows:
wherein,
、
respectively representing the amounts of distortion correction in the x-direction and y-direction,
is the distance between a pixel point and the image principal point and the distance between the pixel point and the image principal pointAngle of incidence
The relationship of (1) is:
in one embodiment, in step S3, performing SIFT feature matching and ranac filtering in the reduced matching area to obtain distortion-corrected binocular image matching homonymy point pairs, including:
correcting the original image into a epipolar image to obtain a binocular image matched with the homonymy point pairs, wherein the original image is the projection from an object space point to an original image space, the epipolar image is the projection from the object space point to a baseline coordinate system, and the relationship of converting the original image space into the object space is as follows:
wherein,
the coordinates of the space of the object are represented,
the spatial coordinates of the original image are represented,
a transformation matrix representing image space to object space;
the relationship of object space to epipolar image space is:
wherein,
the coordinates of the epipolar line image space are represented,
a transformation matrix representing the object space to a baseline coordinate system;
the conversion relationship from the original image to the epipolar line image is:
conversely, the relationship from the epipolar image to the original image is:
wherein,
the representation represents the epipolar line image spatial coordinates.
In one embodiment, step S4 specifically includes:
step S4.1: determining the position of the camera by using the position information obtained by the POS system and the installation position relationship between the POS system and the binocular camera;
step S4.2: calculating the light direction of the homonymous point by using the position information obtained by the POS system, the relative position relation between the binocular cameras obtained by calibration and the position of the homonymous feature point;
step S4.3: and performing forward intersection operation by using a synchronous measurement mode and an asynchronous measurement mode according to the light direction of the same-name point to obtain the position information of the ground object.
One or more technical solutions in the embodiments of the present application have at least one or more of the following technical effects:
the invention provides a binocular photogrammetry method of a large-distortion wide-angle camera based on a small baseline condition, which comprises the steps of firstly calibrating a relative position relation and an internal parameter coefficient between binocular cameras based on a preset indoor high-precision calibration field; then, correcting the binocular camera in real time by adopting a distortion coefficient of the wide-angle camera obtained by calibration to obtain a binocular image after distortion correction; then, obtaining the positions of the feature points with the same name based on a epipolar geometry theory, SIFT feature matching and Randac filtering; and calculating to obtain the light direction of the binocular image matched with the homonymous point according to the installation position relationship between the POS system and the binocular camera, the attitude information measured by the POS system, the relative position relationship between the binocular cameras and the position of the homonymous feature point, and finally obtaining the position information of the ground feature.
According to the method provided by the invention, because the attitude between the binocular cameras of the small baseline measurement system has the characteristic of small angle, and meanwhile, the wide-angle camera image has large distortion, the robustness and the precision of measurement can be seriously influenced, the scheme adopts an indoor high-precision stereo control point to realize the integrated calculation of the mutual attitude between the cameras, the camera internal parameters and the distortion coefficient, and the optimal system parameters of the system are obtained; then, real-time distortion correction of the wide-angle camera is realized by using the distortion coefficient obtained by calculation; then, the binocular image is rapidly calculated by using a visual geometry theory and an image processing technology; and finally, based on the POS information and high-precision homonymy points in the images, realizing high-precision measurement of the ground objects by using a synchronous and asynchronous fusion measurement technology. The volume of the measuring equipment can be maximally reduced; the wide-angle camera is utilized, so that the measurement range can be maximized, and the measurement efficiency is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a schematic flowchart of a binocular photogrammetry method for a large-distortion wide-angle camera based on a small baseline condition according to an embodiment of the present invention;
FIG. 2 is a diagram illustrating the relationship between lens coordinate systems of a wide-angle camera used in an embodiment of the present invention;
FIG. 3 is a schematic view of a wide-angle camera according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of epipolar geometry constraints in an embodiment of the present invention.
Detailed Description
Aiming at the defects of low measurement accuracy and low robustness of a large-distortion wide-angle camera binocular photogrammetry system under the condition of a small baseline, the invention provides a large-distortion wide-angle camera binocular photogrammetry method based on the condition of a small baseline, which adopts the accurate calibration technology of small-angle postures between binocular cameras under the condition of a small baseline, the integrated calibration technology of a large-distortion wide-angle camera, the rapid matching technology of high-overlapping images based on visual geometry and the synchronous asynchronous fusion high-accuracy measurement technology based on high-accuracy homonymy points, thereby greatly improving the accuracy of the photogrammetry system under the covert measurement.
In order to achieve the above object, the main concept of the present invention is as follows:
the method comprises the steps that integrated calculation of mutual postures of cameras, camera internal parameters and distortion coefficients is achieved through indoor high-precision three-dimensional control points, and system optimal system parameters (namely, relative position relations and internal parameter coefficients between binocular cameras) are obtained; then, real-time distortion correction of the wide-angle camera is realized by using the distortion coefficient obtained by calculation; then, the binocular image is rapidly calculated by using a visual geometry theory and an image processing technology; and finally, based on the POS information and high-precision homonymy points in the images, realizing high-precision measurement of the ground objects by using a synchronous and asynchronous fusion measurement technology.
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The present embodiment provides a binocular photogrammetry method for a large-distortion wide-angle camera based on a small baseline condition, please refer to fig. 1, and the method includes:
step S1: based on a preset indoor high-precision calibration field, calibrating a relative position relation and an internal reference coefficient between the binocular cameras, wherein the internal reference coefficient comprises a distortion coefficient.
Step S2: and correcting the binocular camera in real time by adopting the distortion coefficient of the wide-angle camera obtained by calibration to obtain a binocular image after distortion correction.
Step S3: reducing a matching region based on a epipolar geometry theory, and then performing SIFT feature matching and Randac filtering in the reduced matching region to obtain a binocular image matching homonymy point pair after distortion correction; establishing a Gaussian pyramid in a Gaussian kernel in a scale space by an SIFT feature matching algorithm in an epipolar geometry constrained region; and constructing a Gaussian difference pyramid based on the Gaussian pyramid: a DOG pyramid; and carrying out extreme value detection in the DOG pyramid, and detecting to obtain the positions of the characteristic points with the same name.
Specifically, SIFT is called Scale Invariant Feature Transform, and is a local Feature that is very stable and Invariant to rotation, scaling, luminance change, and the like. Randac is an abbreviation for "RANdom SAmple Consensus". It can iteratively estimate the parameters of the mathematical model from a set of observed data sets comprising "outliers". It is an uncertain algorithm-it has a certain probability to get a reasonable result. The DOG pyramid is the Difference of Gaussian Difference.
And the positions of the homonymous feature points are the positions of the homonymous feature points in the images shot by the left eye camera and the right eye camera in the binocular camera, so that matching is performed.
Step S4: according to the installation position relation between the POS system and the binocular cameras, the attitude information obtained by measurement of the POS system, the relative position relation between the binocular cameras and the positions of the homonymous feature points, the light direction of the binocular images matched with the homonymous points is obtained through calculation, then forward intersection operation is carried out according to the light direction of the binocular images matched with the homonymous points in a synchronous and asynchronous measurement mode, and the position information of the ground objects is obtained.
Specifically, the installation Position relationship between the POS System and the binocular camera may be obtained by tool measurement in advance, and the POS System (Position and Orientation System) is a Position and Orientation measurement System.
In one embodiment, step S1 specifically includes:
step S1.1: the photoelectric pod respectively shoots a section of video towards the left, the middle and the right of an inspection school field, so that six videos are obtained, and the left camera and the right camera respectively correspond to 3 videos;
step S1.2: checking and correcting the shot video screenshot, manually selecting a preset number of control points with known geodetic coordinates as calculation conditions, and calculating an initial value of the attitude by using DLT direct linear transformation in photogrammetry;
step S1.3: based on the initial value of the attitude, performing least square adjustment by using back intersection, iteratively eliminating gross error points, and obtaining the instantaneous attitude of six photos
And the internal reference coefficient of the camera
；
Step S1.4: according to the instantaneous posture of six photos
The relative positional relationship between the binocular cameras is obtained.
In particular, DLT direct linear transformation is an algorithm that establishes a direct linear relationship between the coordinates of an image point and the coordinates of the object space of the corresponding object point. The camera can be calibrated without internal and external orientation elements.
In a specific implementation process, the left camera and the right camera respectively correspond to 3 videos, a group of calibration projects can be established for the 3 videos of the left camera, calibration is carried out by utilizing the principle in 1.2, and an initial value of the posture of the left camera is obtained.
In one embodiment, the gesture in step S1.3
And the internal reference coefficient of the camera
The formula of (1) is:
wherein,
in (1),
as a camera
The focal length of,
Is like a main point,
Is a distortion coefficient;
in (1),
as a camera
The relative position of the two or more of the three or more of the,
is a phase ofMachine for working
The posture of (2).
In one embodiment, step S1.3 specifically includes:
step S1.3.1: and (3) calculating a conversion relation between the object space coordinates and the image space coordinates by using the principle of a collinearity equation:
wherein,
representing the object space coordinates of the ground object point,
representing the image space coordinates of the ground feature point, when the incident light is 90 degrees, the distance between the corresponding image point and the image principal point is R, and when the incident light is R
When the distance between the corresponding image point and the image principal point is R, the ratio between R and R is:
Step S1.3.2: obtaining a unified model calibrated by the wide-angle camera according to the conversion relation between the object space coordinates and the image space coordinates and the relation between the distances between the image points corresponding to the incident angles of different light rays and the image principal point:
s1, s2 denotes a conversion coefficient between an image coordinate system and a camera coordinate system, r denotes an imaging radius,
representing the distance between the image point and the center of the image principal point;
step S1.3.3: according to the unified model calibrated by the wide-angle camera, iterative solution is carried out on the parameters to be solved by utilizing a least square method to obtain the attitude
And the internal reference coefficient of the camera
。
Specifically, first, the image-object space conversion is performed, as shown in FIG. 2, for the relationship between the lens coordinate systems,
wherein:
the rotation matrix is obtained according to the rotation sequence of Y _ X _ Z, and the rotation angles are respectively
The definition of the angle is the same as that in photogrammetry, and the rotation matrix is specifically as follows:
according to fig. 3, the proportional relationship between the object coordinates and the image coordinates of the points can be obtained:
then, the relation of formula (4), i.e. the unified model of the wide-angle camera calibration, can be obtained according to formulas (2) and (3). In step S1.3.3, the total number of the parameters to be solved is 14, including the outside orientation line element and the angle element of the camera, the principal point coordinate of the camera, the imaging radius, and the radial and tangential distortion parameters of the camera, and the parameters to be solved are iteratively solved by using a least square method, so that the calibration of the fisheye camera (wide-angle camera) can be completed. The specific implementation process is as follows:
equation 5 is first linearized:
then, the model is solved, and the formula is written into a matrix form, namely
It should be noted that the requirement for solving the nonlinear equation for the initial value of the parameter is high, and if the initial value is far from the true value, the equation is easy to be not converged. Thus, the internal orientation element
The initial value of (is) taken as the center of the image
A (b) a
、
Representing the number of columns and rows of the image respectively),
the initial value of (1) is the radius of the imaging part on the image and the unit pixel; distortion parameter
、
、
、
、
、
、
The initial values of all the parameters are zero; exterior orientation element
The initial value of the exterior orientation element is settled by using space rear intersection and a small number of control points, wherein the exterior orientation element comprises a posture
And the internal reference coefficient of the camera
。
In one embodiment, step S2 specifically includes: distortion coefficient in unified model calibrated by wide-angle camera
、
、
、
、
、
、
The binocular camera is corrected in real time to obtain a binocular image after distortion correction, wherein a distortion correction formula is as follows:
wherein,
、
respectively representing the amounts of distortion correction in the x-direction and y-direction,
the distance between the pixel point and the image principal point and the incident angle
The relationship of (1) is:
see, in particular, the wide-angle camera ray diagram in fig. 3.
In one embodiment, the SIFT feature matching and the ranac filtering are performed in the reduced matching area to obtain the distortion-corrected binocular image matching homonymous point pairs, which includes:
correcting the original image into a epipolar image to obtain a binocular image matched with the homonymy point pairs, wherein the original image is the projection from an object space point to an original image space, the epipolar image is the projection from the object space point to a baseline coordinate system, and the relationship of converting the original image space into the object space is as follows:
wherein,
the coordinates of the space of the object are represented,
the spatial coordinates of the original image are represented,
a transformation matrix representing image space to object space;
the relationship of object space to epipolar image space is:
wherein,
the coordinates of the epipolar line image space are represented,
a transformation matrix representing the object space to a baseline coordinate system;
the conversion relationship from the original image to the epipolar line image is:
conversely, the relationship from the epipolar image to the original image is:
wherein,
the representation represents the epipolar line image spatial coordinates.
Specifically, a epipolar geometric theory is used to effectively reduce a matching region, then SIFT feature matching and ranac filtering are performed in the region to quickly obtain distortion-corrected binocular image matching homonymous point pairs, and a epipolar geometric constraint schematic diagram is shown in fig. 4:
p represents object point coordinates; i1, I2 respectively represent left and right eye camera images; o1, O2 denote camera imaging centers, respectively; p1 and P2 respectively represent imaging points of P in the image; e1, e2 represent poles, respectively, and l1, l2 represent epipolar lines, respectively.
After the relative position relationship of the two photos is determined according to the calibration parameters, the original image can be corrected into the epipolar line image, namely, the optical axes of the two photos are parallel and vertical to the base line (the central connecting line of the camera head images), meanwhile, the row (or column) of the epipolar line image is parallel to the base line, and at the moment, the same name points on the two photos are aligned in the row or column, so the process of searching for the same name is limited to one-dimensional search. The original image is actually a projection of the object space points to the original image space, and the epipolar image is actually a projection of the object space points to the baseline coordinate system.
Establishing a Gaussian pyramid and a DOG pyramid in a Gaussian kernel in a scale space by an SIFT feature matching algorithm in an epipolar geometry constrained region; carrying out extreme value detection in the DOG pyramid; calculating a key scale; calculating the main direction of the key points; calculating a descriptor; and finally, matching by using the distance information to determine the positions of the characteristic points with the same name, wherein the specific process is not repeated.
In one embodiment, step S4 specifically includes:
step S4.1: determining the position of the camera by using the position information obtained by the POS system and the installation position relationship between the POS system and the binocular camera;
step S4.2: calculating the light direction of the homonymous point by using the position information obtained by the POS system, the relative position relation between the binocular cameras obtained by calibration and the position of the homonymous feature point;
step S4.3: and performing forward intersection operation by using a synchronous measurement mode and an asynchronous measurement mode according to the light direction of the same-name point to obtain the position information of the ground object.
Specifically, the position of the camera can be determined by using the position obtained by the POS system and the installation position parameters measured in advance, the light direction of the same-name point can be calculated by using the attitude information of the POS system and the relative attitude information between the cameras obtained by calibration, the forward intersection calculation in a synchronous measurement mode and an asynchronous measurement mode is used, and finally high-precision positioning is obtained by fusion.
The distance from a target object to a camera in the small-baseline binocular vehicle-mounted mobile measurement is generally far longer than the length of a baseline, and on the premise, the principle that 3 points of a photographing center, an image point and an object point are collinear is still met in an imaging model, namely a collinear condition equation is still established. Under the binocular imaging model, the two homonymous rays intersect in the front of the space, and the space position of the target point can be determined.
Generally, the measuring system is small in equipment, simple in measuring process, large in measuring range, high in precision and robust, is suitable for light, small and low-cost concealed measuring application scenes, and finally can achieve measuring precision within a 15m range due to 0.2, and has high precision and practical production significance.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various modifications and variations can be made in the embodiments of the present invention without departing from the spirit or scope of the embodiments of the invention. Thus, if such modifications and variations of the embodiments of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to encompass such modifications and variations.
Claims (6)
1. A large-distortion wide-angle camera binocular photogrammetry method based on a small baseline condition is characterized by comprising the following steps:
step S1: calibrating a relative position relation and an internal reference coefficient between the binocular cameras based on a preset indoor high-precision calibration field, wherein the internal reference coefficient comprises a distortion coefficient;
step S2: correcting the binocular camera in real time by adopting a distortion coefficient of the wide-angle camera obtained by calibration to obtain a binocular image after distortion correction;
step S3: reducing a matching region based on a epipolar geometry theory, and then performing SIFT feature matching and Randac filtering in the reduced matching region to obtain a binocular image matching homonymy point pair after distortion correction; establishing a Gaussian pyramid in a Gaussian kernel in a scale space by an SIFT feature matching algorithm in an epipolar geometry constrained region; and constructing a Gaussian difference pyramid based on the Gaussian pyramid: a DOG pyramid; carrying out extreme value detection in the DOG pyramid to detect and obtain the positions of the characteristic points with the same name;
step S4: calculating to obtain the light direction of the binocular image matched with the same-name point according to the installation position relationship between the POS system and the binocular camera, the attitude information obtained by measurement of the POS system, the relative position relationship between the binocular cameras and the position of the same-name feature point, and then performing front intersection operation by using a synchronous measurement mode and an asynchronous measurement mode according to the light direction of the binocular image matched with the same-name point to obtain the position information of the ground object;
wherein, step S4 specifically includes:
step S4.1: determining the position of the camera by using the position information obtained by the POS system and the installation position relationship between the POS system and the binocular camera;
step S4.2: calculating the light direction of the homonymous point by using the position information obtained by the POS system, the relative position relation between the binocular cameras obtained by calibration and the position of the homonymous feature point;
step S4.3: and performing forward intersection operation by using a synchronous measurement mode and an asynchronous measurement mode according to the light direction of the same-name point to obtain the position information of the ground object.
2. The method according to claim 1, wherein step S1 specifically comprises:
step S1.1: the photoelectric pod respectively shoots a section of video towards the left, the middle and the right of an inspection school field, so that six videos are obtained, and the left camera and the right camera respectively correspond to 3 videos;
step S1.2: checking and correcting the shot video screenshot, manually selecting a preset number of control points with known geodetic coordinates as calculation conditions, and calculating an initial value of the attitude by using DLT direct linear transformation in photogrammetry;
step S1.3: based on the initial value of the attitude, performing least square adjustment by using back intersection, iteratively eliminating gross error points, and obtaining the instantaneous attitude of six photos
And the internal reference coefficient of the camera
；
Step S1.4: according to the instantaneous posture of six photos
The relative positional relationship between the binocular cameras is obtained.
3. Method according to claim 2, characterized in that the gesture in step S1.3
And the internal reference coefficient of the camera
The formula of (1) is:
wherein,
in (1),
as a camera
The focal length of,
Is like a main point,
Is a distortion coefficient;
in (1),
as a camera
The relative position of the two or more of the three or more of the,
as a camera
The posture of (2).
4. The method according to claim 3, characterized in that step S1.3 comprises in particular:
step S1.3.1: calculating a conversion relation between the object space coordinate and the image space coordinate by utilizing the principle of a collinear equation, and obtaining a proportional relation between the object space coordinate and the image coordinate of the point according to the conversion relation:
wherein,
representing the object space coordinates of the ground object point,
image space coordinates representing feature points, whenWhen the incident light is 90 degrees, the distance between the corresponding image point and the image principal point is R, and when the incident light is R
When the distance between the corresponding image point and the image principal point is R, the ratio between R and R is:
Step S1.3.2: obtaining a unified model calibrated by the wide-angle camera according to the conversion relation between the object space coordinates and the image space coordinates and the relation between the distances between the image points corresponding to the incident angles of different light rays and the image principal point:
wherein,
s1, s2 denotes a conversion coefficient between an image coordinate system and a camera coordinate system, r denotes an imaging radius,
representing the distance between the image point and the center of the image principal point;
step S1.3.3: according to the unified model calibrated by the wide-angle camera, iterative solution is carried out on the parameters to be solved by utilizing a least square method to obtain the attitude
And the internal reference coefficient of the camera
。
5. The method according to claim 3, wherein step S2 specifically comprises: distortion coefficient in unified model calibrated by wide-angle camera
、
、
、
、
、
、
The binocular camera is corrected in real time to obtain a binocular image after distortion correction, wherein a distortion correction formula is as follows:
wherein,
、
respectively representing the amounts of distortion correction in the x-direction and y-direction,
the distance between the pixel point and the image principal point and the incident angle
The relationship of (1) is:
6. the method as claimed in claim 3, wherein the step S3 of performing SIFT feature matching and ranaca filtering in the reduced matching area to obtain distortion-corrected binocular image matching homonymous point pairs comprises:
correcting the original image into a epipolar image to obtain a binocular image matched with the homonymy point pairs, wherein the original image is the projection from an object space point to an original image space, the epipolar image is the projection from the object space point to a baseline coordinate system, and the relationship of converting the original image space into the object space is as follows:
wherein,
the coordinates of the space of the object are represented,
the spatial coordinates of the original image are represented,
a transformation matrix representing image space to object space;
the relationship of object space to epipolar image space is:
wherein,
the coordinates of the epipolar line image space are represented,
a transformation matrix representing the object space to a baseline coordinate system;
the conversion relationship from the original image to the epipolar line image is:
conversely, the relationship from the epipolar image to the original image is:
wherein,
the representation represents the epipolar line image spatial coordinates.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010060072.0A CN110874854B (en) | 2020-01-19 | 2020-01-19 | Camera binocular photogrammetry method based on small baseline condition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010060072.0A CN110874854B (en) | 2020-01-19 | 2020-01-19 | Camera binocular photogrammetry method based on small baseline condition |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110874854A true CN110874854A (en) | 2020-03-10 |
| CN110874854B CN110874854B (en) | 2020-06-23 |
Family
ID=69717690
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010060072.0A Active CN110874854B (en) | 2020-01-19 | 2020-01-19 | Camera binocular photogrammetry method based on small baseline condition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110874854B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111815703A (en) * | 2020-07-16 | 2020-10-23 | 中国农业机械化科学研究院 | Binocular camera-based handheld heap survey device and method |
| CN111882621A (en) * | 2020-07-22 | 2020-11-03 | 武汉大学 | Rice thickness parameter automatic measurement method based on binocular image |
| CN112819900A (en) * | 2021-02-23 | 2021-05-18 | 自然资源部第一海洋研究所 | Method for calibrating internal azimuth, relative orientation and distortion coefficient of intelligent stereography |
| CN114185071A (en) * | 2021-12-10 | 2022-03-15 | 武汉市虎联智能科技有限公司 | Positioning system and method based on object recognition and spatial position perception |
| CN114383575A (en) * | 2021-12-22 | 2022-04-22 | 中国测绘科学研究院 | Geological disaster monitoring method for gyroscope-assisted close-range survey |
| CN115962758A (en) * | 2022-12-15 | 2023-04-14 | 中铁第四勘察设计院集团有限公司 | Measuring method and system based on unmanned aerial vehicle and binocular camera measuring platform |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102155955A (en) * | 2011-03-11 | 2011-08-17 | 天津理工大学 | Stereoscopic vision mile meter and measuring method |
| CN105678787A (en) * | 2016-02-03 | 2016-06-15 | 西南交通大学 | Heavy-duty lorry driving barrier detection and tracking method based on binocular fisheye camera |
| CN109035200A (en) * | 2018-06-21 | 2018-12-18 | 北京工业大学 | A kind of bolt positioning and position and posture detection method based on the collaboration of single binocular vision |
| US10349040B2 (en) * | 2015-09-21 | 2019-07-09 | Inuitive Ltd. | Storing data retrieved from different sensors for generating a 3-D image |
| CN209231985U (en) * | 2018-05-31 | 2019-08-09 | 福州大学 | A kind of camera parameters automatic calibration device based on virtual Binocular Vision Principle |
| WO2019228523A1 (en) * | 2018-05-31 | 2019-12-05 | 上海微电子装备（集团）股份有限公司 | Method and device for determining spatial position shape of object, storage medium and robot |
-
2020
- 2020-01-19 CN CN202010060072.0A patent/CN110874854B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102155955A (en) * | 2011-03-11 | 2011-08-17 | 天津理工大学 | Stereoscopic vision mile meter and measuring method |
| US10349040B2 (en) * | 2015-09-21 | 2019-07-09 | Inuitive Ltd. | Storing data retrieved from different sensors for generating a 3-D image |
| CN105678787A (en) * | 2016-02-03 | 2016-06-15 | 西南交通大学 | Heavy-duty lorry driving barrier detection and tracking method based on binocular fisheye camera |
| CN209231985U (en) * | 2018-05-31 | 2019-08-09 | 福州大学 | A kind of camera parameters automatic calibration device based on virtual Binocular Vision Principle |
| WO2019228523A1 (en) * | 2018-05-31 | 2019-12-05 | 上海微电子装备（集团）股份有限公司 | Method and device for determining spatial position shape of object, storage medium and robot |
| CN109035200A (en) * | 2018-06-21 | 2018-12-18 | 北京工业大学 | A kind of bolt positioning and position and posture detection method based on the collaboration of single binocular vision |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111815703A (en) * | 2020-07-16 | 2020-10-23 | 中国农业机械化科学研究院 | Binocular camera-based handheld heap survey device and method |
| CN111815703B (en) * | 2020-07-16 | 2024-05-03 | 中国农业机械化科学研究院 | Handheld grain pile measuring device and grain pile measuring method based on binocular camera |
| CN111882621A (en) * | 2020-07-22 | 2020-11-03 | 武汉大学 | Rice thickness parameter automatic measurement method based on binocular image |
| CN112819900A (en) * | 2021-02-23 | 2021-05-18 | 自然资源部第一海洋研究所 | Method for calibrating internal azimuth, relative orientation and distortion coefficient of intelligent stereography |
| CN112819900B (en) * | 2021-02-23 | 2022-10-11 | 自然资源部第一海洋研究所 | Method for calibrating internal azimuth, relative orientation and distortion coefficient of intelligent stereography |
| CN114185071A (en) * | 2021-12-10 | 2022-03-15 | 武汉市虎联智能科技有限公司 | Positioning system and method based on object recognition and spatial position perception |
| CN114383575A (en) * | 2021-12-22 | 2022-04-22 | 中国测绘科学研究院 | Geological disaster monitoring method for gyroscope-assisted close-range survey |
| CN114383575B (en) * | 2021-12-22 | 2024-03-22 | 中国测绘科学研究院 | Geological disaster monitoring method for gyroscope-assisted close-range measurement |
| CN115962758A (en) * | 2022-12-15 | 2023-04-14 | 中铁第四勘察设计院集团有限公司 | Measuring method and system based on unmanned aerial vehicle and binocular camera measuring platform |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110874854B (en) | 2020-06-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110874854B (en) | Camera binocular photogrammetry method based on small baseline condition | |
| CN105678742B (en) | A kind of underwater camera scaling method | |
| TWI555379B (en) | An image calibrating, composing and depth rebuilding method of a panoramic fish-eye camera and a system thereof | |
| CN112767542A (en) | Three-dimensional reconstruction method of multi-view camera, VR camera and panoramic camera | |
| CN109859272B (en) | Automatic focusing binocular camera calibration method and device | |
| CN109272574B (en) | Construction method and calibration method of linear array rotary scanning camera imaging model based on projection transformation | |
| CN109325981B (en) | Geometric parameter calibration method for micro-lens array type optical field camera based on focusing image points | |
| WO2020001120A1 (en) | Light field image correction method, computer-readable storage medium, and electronic terminal | |
| Chatterjee et al. | Algorithms for coplanar camera calibration | |
| CN101354796B (en) | Omnidirectional stereo vision three-dimensional rebuilding method based on Taylor series model | |
| CN110807815B (en) | Quick underwater calibration method based on corresponding vanishing points of two groups of mutually orthogonal parallel lines | |
| CN113947638B (en) | Method for correcting orthographic image of fish-eye camera | |
| CN110345921A (en) | Stereoscopic fields of view vision measurement and vertical axial aberration and axial aberration bearing calibration and system | |
| CN113920206B (en) | Calibration method of perspective tilt-shift camera | |
| Kannala et al. | Geometric camera calibration. | |
| JP2024537798A (en) | Photographing and measuring method, device, equipment and storage medium | |
| Perdigoto et al. | Calibration of mirror position and extrinsic parameters in axial non-central catadioptric systems | |
| CN111429571A (en) | Rapid stereo matching method based on spatio-temporal image information joint correlation | |
| CN108269234A (en) | A kind of lens of panoramic camera Attitude estimation method and panorama camera | |
| JP2012198031A (en) | Image correction method and image correction device | |
| CN111383264A (en) | Positioning method, positioning device, terminal and computer storage medium | |
| Liu et al. | Calibration method based on the image of the absolute quadratic curve | |
| CN112950727B (en) | Large-view-field multi-target simultaneous ranging method based on bionic curved compound eye | |
| CN114092647A (en) | Three-dimensional reconstruction system and method based on panoramic binocular stereo vision | |
| CN112258581B (en) | On-site calibration method for panoramic camera with multiple fish glasses heads |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |