CN115100284A - Binocular feature matching displacement measurement method and device - Google Patents
Binocular feature matching displacement measurement method and device Download PDFInfo
- Publication number
- CN115100284A CN115100284A CN202210730878.5A CN202210730878A CN115100284A CN 115100284 A CN115100284 A CN 115100284A CN 202210730878 A CN202210730878 A CN 202210730878A CN 115100284 A CN115100284 A CN 115100284A
- Authority
- CN
- China
- Prior art keywords
- displacement
- current frame
- frame image
- camera
- feature point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/97—Determining parameters from multiple pictures
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention relates to a binocular feature matching displacement measurement method and device, which comprises the steps of obtaining a history frame image and a current frame image shot by a binocular camera, wherein the history frame image is a l0 image and a r0 image which are obtained by shooting a target by a left camera and a right camera respectively at a first moment, and the current frame image is an ln image and an rn image which are obtained by shooting the target by the left camera and the right camera respectively at a second moment; extracting feature points on the historical frame image and the current frame image, wherein the feature points comprise corner sub-pixel coordinates and corresponding ORB descriptions, matching the feature points of the historical frame image and the current frame image according to the ORB descriptions, and determining the sub-pixel coordinates of the same feature point on l0, r0, ln and rn images; and executing displacement measurement operation on each characteristic point, and further obtaining the displacement of the characteristic point in the z, x and y directions relative to the first moment at the second moment.
Description
Technical Field
The invention relates to the technical field of displacement measurement, in particular to a binocular characteristic matching displacement measurement method and device.
Background
With the continuous development of measurement technology, many current measurement modes are limited in some scenes, displacement measurement based on Binocular Stereo Vision starts to play a role, Binocular Stereo Vision (Binocular Stereo Vision) is an important form of machine Vision, and the method is a method for acquiring three-dimensional geometric information of an object by acquiring two images of the object to be measured from different positions by using imaging equipment based on a parallax principle and calculating position deviation between corresponding points of the images.
Three-dimensional displacement measurement based on binocular stereo vision is higher in measurement accuracy than monocular measurement, and therefore the method is widely applied to target displacement detection in the scenes of geological disaster monitoring, foundation pit monitoring, bridge monitoring, tunnel monitoring and the like.
However, in the prior art, in the aspect of binocular vision, the relative accuracy of dense matching algorithms such as BM, SGBM, GC and the like is coarse, so that the accuracy of the displacement measurement result is low.
Disclosure of Invention
Aiming at the problems, the invention provides a binocular feature matching displacement measurement method and device.
The binocular feature matching displacement measurement method provided by the invention comprises the following steps:
acquiring historical frame images and current frame images shot by a binocular camera, wherein the historical frame images are l0 images and r0 images which are obtained by shooting a target by a left camera and a right camera at a first moment respectively, and the current frame images are ln images and rn images which are obtained by shooting the target by the left camera and the right camera at a second moment respectively;
extracting feature points on the historical frame image and the current frame image, wherein the feature points comprise corner sub-pixel coordinates and corresponding ORB descriptions, matching the feature points of the historical frame image and the current frame image according to the ORB descriptions, and determining the sub-pixel coordinates of the same feature point on l0, r0, ln and rn images;
for each feature point, the following displacement measurement operations are performed:
respectively calculating a historical frame distance and a current frame distance corresponding to the feature point according to the parallax of the feature point in the historical frame image and the parallax of the current frame image, and calculating the Z-direction displacement of the feature point at the second moment relative to the first moment according to the historical frame distance and the current frame distance;
obtaining the pixel variation in the x direction and the pixel variation in the y direction according to the difference between the sub-pixel coordinates of the feature point on the l0 and the ln image or the sub-pixel coordinates of the feature point on the r0 and the rn image; calculating to obtain an x-direction scale and a y-direction scale corresponding to the feature point according to the current frame distance corresponding to the feature point and parameters of the binocular camera; and calculating the x-direction displacement and the y-direction displacement of the characteristic point relative to the first time at the second time according to the x-direction pixel variation, the y-direction pixel variation, the x-direction scale and the y-direction scale corresponding to the characteristic point.
Further, the method further comprises: and calibrating the binocular camera in advance to obtain the internal parameter, the external parameter and the distortion correction parameter of the binocular camera.
Further, before extracting feature points on the historical frame image and the current frame image, the method further includes: and carrying out distortion correction on the historical frame image and the current frame image according to the distortion correction parameters.
Further, according to the parallax of the feature point corresponding to the historical frame image and the parallax of the current frame image, respectively calculating to obtain a historical frame distance and a current frame distance corresponding to the feature point as follows:
respectively calculating the historical frame distance and the current frame distance corresponding to the feature point by using the formula distance d (baseline) fx/disparity;
wherein, baseline refers to the distance between the optical centers of the left camera and the right camera, fx is the focal length in the internal reference calibrated in advance, and disparity is parallax.
Further, the Z-direction displacement of the feature point at the second time relative to the first time is calculated according to the historical frame distance and the current frame distance as follows: the Z-direction displacement is the current frame distance-the historical frame distance.
Further, the binocular camera parameters include camera horizontal visual angle, camera vertical visual angle, camera horizontal resolution and camera vertical resolution, and according to the current frame distance and the binocular camera parameters that this feature point corresponds, the x direction scale that the calculation obtains this feature point corresponds is with y direction scale:
according to the formula: the x-direction scale is tan (horizontal visual angle of the camera) dn 2/horizontal resolution of the camera, and the x-direction scale corresponding to the characteristic point is obtained through calculation;
according to the formula: the y-direction scale is tan (camera vertical visual angle) dn 2/camera vertical resolution, and the y-direction scale corresponding to the feature point is obtained through calculation; where dn is the current frame distance.
Further, according to the x-direction pixel variation, the y-direction pixel variation, the x-direction scale and the y-direction scale corresponding to the feature point, the x-direction displacement and the y-direction displacement of the feature point at the second time relative to the first time are calculated as follows:
calculating the x-direction displacement of the characteristic point according to an x-direction pixel variation quantity x-direction scale;
and calculating the y-direction displacement of the characteristic point according to a y-direction pixel variation quantity y-direction scale.
Further, the displacement measurement operation further includes correcting for x-direction displacement and y-direction displacement:
according to the x-direction displacement-x-direction adjustment angle dn and the y-direction displacement-y-direction adjustment angle dn, re-assigning the x-direction displacement and the y-direction displacement to finish the correction of the x-direction displacement and the y-direction displacement;
the x-direction adjustment angle is the median of all the characteristic point x-direction deflection angles, the y-direction adjustment angle is the median of all the characteristic point y-direction deflection angles, and the calculation formula of each characteristic point x-direction deflection angle is as follows:
the x-direction deflection angle is atan (x-direction displacement/dn), and the calculation formula of the y-direction deflection angle of each feature point is as follows: the y-direction deflection angle is atan (y-direction displacement/dn), and dn is the current frame distance.
The invention also provides a binocular feature matching displacement measurement device, which comprises an acquisition module, a feature point extraction and matching module and a displacement measurement module, wherein:
the acquisition module is connected with the characteristic point extraction and matching module and is used for acquiring a history frame image and a current frame image shot by the binocular cameras, wherein the history frame image is a l0 image and a r0 image which are respectively shot by the left camera and the right camera at a first moment, and the current frame image is an ln image and an rn image which are respectively shot by the left camera and the right camera at a second moment;
the characteristic point extracting and matching module is connected with the displacement measuring module and is used for extracting characteristic points on the historical frame image and the current frame image, the characteristic points comprise corner sub-pixel coordinates and corresponding ORB descriptions, the characteristic point matching is carried out on the historical frame image and the current frame image according to the ORB descriptions, and the sub-pixel coordinates of the same characteristic point on l0, r0, ln and rn images are determined;
a displacement measurement module for performing the following displacement measurement operations for each feature point:
respectively calculating a historical frame distance and a current frame distance corresponding to the feature point according to the parallax of the feature point in the historical frame image and the parallax of the current frame image, and calculating the Z-direction displacement of the feature point at the second moment relative to the first moment according to the historical frame distance and the current frame distance;
obtaining the pixel variation in the x direction and the pixel variation in the y direction according to the difference between the sub-pixel coordinates of the feature point on the l0 and the ln image or the sub-pixel coordinates of the feature point on the r0 and the rn image; calculating to obtain an x-direction scale and a y-direction scale corresponding to the feature point according to the current frame distance and the binocular camera parameters corresponding to the feature point; and calculating the x-direction displacement and the y-direction displacement of the characteristic point relative to the first time at the second time according to the x-direction pixel variation, the y-direction pixel variation, the x-direction scale and the y-direction scale corresponding to the characteristic point.
Furthermore, the device further comprises an image distortion correction module, wherein the image distortion correction module is respectively connected with the acquisition module and the characteristic point extraction and matching module and is used for carrying out distortion correction on the historical frame image and the current frame image according to distortion correction parameters, and the distortion correction parameters are obtained by calibrating the binocular camera.
The binocular characteristic matching displacement measurement method and the device provided by the invention at least have the following beneficial effects:
(1) the characteristic points extracted from the historical frame image and the current frame image comprise corner sub-pixel coordinates and corresponding ORB description, the sub-pixel coordinates are used, the coordinate precision is improved, more accurate coordinates are obtained, meanwhile, the characteristic points can be accurately matched by using the ORB description, and subsequently, for each characteristic point, the sub-pixel coordinates of the characteristic point on l0, r0, ln and rn images are obtained by matching, namely, displacement change can be accurately identified, and accurate measurement of spatial displacement is realized.
(2) After the displacement is measured, the displacement in the x direction and the displacement in the y direction are corrected according to the deflection angle in the x direction and the deflection angle in the y direction, so that the accuracy of displacement measurement is further improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the prior art descriptions will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a schematic structural diagram of a displacement measurement system based on a binocular camera;
fig. 2 is a first flowchart of a binocular feature matching displacement measurement method in an embodiment of the present invention;
FIG. 3 is a schematic diagram of a current frame and a historical frame;
FIG. 4 is a flow chart of a method of operation of displacement measurement in an embodiment of the present invention;
fig. 5 is a flowchart of a binocular feature matching displacement measurement method according to yet another embodiment of the present invention;
fig. 6 is a flow chart of a binocular feature matching displacement measurement method in yet another embodiment of the present invention;
FIG. 7 is a flow chart of a displacement measurement operation method in accordance with yet another embodiment of the present invention;
FIG. 8 is a schematic view of a binocular feature matching displacement measurement device in one embodiment of the present invention;
fig. 9 is a schematic view of a binocular feature matching displacement measuring apparatus according to still another embodiment of the present invention;
801-acquisition module, 802-feature point extraction and matching module, 803-displacement measurement module and 804-image distortion correction module.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is to be understood that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In an embodiment of the present invention, a binocular feature matching displacement measurement method is provided, and the method is applied to a displacement measurement system based on a binocular camera, as shown in fig. 1, which is a schematic diagram of the system, the system includes the binocular camera and an industrial control computing host connected to the binocular camera, and an execution main body of the method is the industrial control computing host. The further binocular camera includes left camera and right camera.
When the displacement measurement is carried out on the target, the binocular camera shoots the target to obtain a shot image, specifically, the target is shot by the left camera and the right camera in the binocular camera simultaneously every time, and therefore images shot by the left camera and the right camera simultaneously at a plurality of moments are obtained.
As shown in fig. 2, the method comprises the steps of:
step S10: and acquiring a historical frame image and a current frame image shot by the binocular camera.
The historical frame images are l0 images (left camera shooting) and r0 images (right camera shooting) obtained by shooting the target by the left camera and the right camera at the first moment respectively. The current frame image is an ln image (shot by the left camera) and an rn image (shot by the right camera) which are respectively shot by the left camera and the right camera at the second moment. The second time is after the first time.
According to the invention, a historical frame image obtained by shooting the target at the first moment is used as a reference, and the displacement of the target at the second moment relative to the first moment is determined according to a current frame image obtained by shooting the target at the second moment. It should be understood that the selection of the first time and the second time can be determined by a technician according to the actual detection requirement, and the invention is not limited thereto.
Fig. 3 is a schematic diagram of a history frame image and a current frame image.
Step S20: extracting feature points on the historical frame image and the current frame image, wherein the feature points comprise corner sub-pixel coordinates and corresponding ORB descriptions, matching the feature points of the historical frame image and the current frame image according to the ORB descriptions, and determining the sub-pixel coordinates of the same feature point on the l0, r0, ln and rn images.
In this step, the feature points are feature points generated at the positions of the corner points, and the feature points include sub-pixel coordinates of the corner points and orb (organized Fast and Rotated brief) descriptions.
The feature points extracted from the historical frame image and the current frame image in this step may be:
detecting and identifying corner points (a plurality of corner points) on l0, r0, ln and rn images by using a Shi-Tomasi corner point detection algorithm, generating sub-pixel coordinates for the corner points by using a sub-pixel regression algorithm, and generating an ORB description of each sub-pixel coordinate corner point by using an ORB algorithm, so as to generate feature points at the positions of the corner points, wherein the generated feature points comprise the corner point sub-pixel coordinates and the ORB description. Specifically, the subpixel regression algorithm may adopt a subpixel-level corner detection algorithm in the prior art, which is not limited in the present invention.
Further, in this step, feature point matching is performed on the historical frame image and the current frame image according to the ORB description, and sub-pixel coordinates of the same feature point on the l0, r0, ln, rn images are determined, which may be:
and matching the ORB description by using BF matching, and further finding the sub-pixel coordinates of the same characteristic point on l0, r0, ln and rn images.
Step S30: for each feature point, a displacement measurement operation is performed.
Specifically, in an implementation manner of this embodiment, as shown in fig. 4, the displacement measurement operation specifically includes the following steps:
step S301: and respectively calculating the historical frame distance and the current frame distance corresponding to the feature point according to the parallax of the feature point corresponding to the historical frame image and the parallax of the current frame image.
The distance in this step refers to the z-direction distance from the measured point (the feature point) to the optical center of the camera.
Specifically, a historical frame distance and a current frame distance corresponding to the feature point are respectively calculated by using a formula distance d ═ baseline ×/disparity;
wherein, baseline refers to the distance between the optical centers of the left camera and the right camera, fx is the focal length in the internal reference calibrated in advance, and disparity is parallax. When the historical frame distance is calculated, the parallax of the feature point corresponding to the historical frame image is substituted into the disparity, and when the current frame distance is calculated, the parallax of the feature point corresponding to the current frame image is substituted into the disparity.
Furthermore, in this step, distance data may be filtered using spatial constraints, where left-right disparity and spatial distance have constraints, and neighborhood space has continuity constraints. It should be understood that, if the historical frame distance or the current frame distance corresponding to the feature point is filtered out by the spatial constraint in this step, the feature point is discarded during the displacement measurement, and the z-direction displacement, the x-direction displacement, and the y-direction displacement corresponding to the feature point are not measured.
Step S302: and calculating the Z-direction displacement of the characteristic point at the second moment relative to the first moment according to the historical frame distance and the current frame distance.
Specifically, the Z-direction displacement is the current frame distance — the historical frame distance.
Step S303: the x-direction pixel variation and the y-direction pixel variation are obtained from the difference between the sub-pixel coordinates of the feature point on the l0 and the ln image or the r0 and the rn image.
Specifically, assuming that the sub-pixel coordinates of the feature point on the l0 image are (x1, y1) and the sub-pixel coordinates on the ln image are (x2, y2), the x-direction pixel variation amount is x2-x1, and the y-direction pixel variation amount is y2-y 1.
Or assuming that the sub-pixel coordinates of the feature point on the r0 image are (x3, y3) and the sub-pixel coordinates on the rn image are (x4, y4), the x-direction pixel variation amount is x4-x3, and the y-direction pixel variation amount is y4-y 3.
Step S304: and calculating to obtain an x-direction scale and a y-direction scale corresponding to the feature point according to the current frame distance corresponding to the feature point and the parameters of the binocular camera.
Further, the binocular camera parameters include a camera horizontal viewing angle, a camera vertical viewing angle, a camera horizontal resolution and a camera vertical resolution, and are obtained by calibrating the binocular camera in advance.
According to the current frame distance corresponding to the characteristic point and the binocular camera parameters, calculating to obtain an x-direction scale and a y-direction scale corresponding to the characteristic point as follows:
calculating to obtain an x-direction scale corresponding to the feature point according to the x-direction scale tan (horizontal view angle of the camera) dn 2/horizontal resolution of the camera;
calculating to obtain a y-direction scale corresponding to the characteristic point according to the y-direction scale tan (camera vertical visual angle) dn 2/camera vertical resolution; where dn is the current frame distance.
Specifically, when calibrating the binocular camera, the view angle parameters of the left and right cameras are not completely consistent, and therefore in this step, it is necessary to consider whether the left camera image or the right camera image is used when calculating the x-direction pixel variation and the y-direction pixel variation in step S303, if the left camera image is used, the camera horizontal view angle and the camera vertical view angle in this step are the camera horizontal view angle and the camera vertical view angle corresponding to the left camera, and if the right camera image is used, the camera horizontal view angle and the camera vertical view angle in this step are the camera horizontal view angle and the camera vertical view angle corresponding to the right camera.
Step S305: and calculating the x-direction displacement and the y-direction image displacement of the feature point relative to the first time at the second time according to the x-direction pixel variation, the y-direction pixel variation, the x-direction scale and the y-direction scale corresponding to the feature point.
Specifically, in this step, the x-direction displacement of the feature point may be calculated according to the x-direction displacement, i.e., the x-direction pixel variation amount, i.e., the x-direction scale;
and calculating the y-direction displacement of the characteristic point according to a y-direction pixel variation quantity y-direction scale.
In the binocular feature matching displacement measurement method provided by this embodiment, when displacement measurement of a target is performed, a binocular camera is used to shoot a history frame image and a current frame image, feature points extracted from the history frame image and the current frame image include corner sub-pixel coordinates and corresponding ORB descriptions, the sub-pixel coordinates are used, coordinate precision is improved, and more accurate coordinates of each corner point are obtained, meanwhile, the ORB descriptions can be used to perform accurate matching on the feature points, and subsequently, for each feature point, the sub-pixel coordinates of the feature point on l0, r0, ln, and rn images obtained through matching are used, that is, displacement change can be accurately identified, and accurate measurement of spatial displacement is achieved
In yet another embodiment of the present invention, as shown in fig. 5, the method further comprises:
step S40: and calibrating the binocular camera in advance to obtain the internal parameter, the external parameter and the distortion correction parameter of the binocular camera.
Specifically, the method for calibrating the binocular camera in this step may be a calibration method in the prior art, which is not limited in the present invention.
In another embodiment of the present invention, as shown in fig. 6, before feature points are extracted from the historical frame image and the current frame image, the method further includes:
step S50: and carrying out distortion correction on the historical frame image and the current frame image according to the distortion correction parameters.
In this embodiment, before the displacement measurement is performed, distortion correction is performed on the historical frame image and the current frame image, so that the accuracy of subsequent displacement measurement can be improved.
Further, as shown in fig. 7, the displacement measurement operation further includes step S306: the x-direction displacement and the y-direction displacement are corrected.
And re-assigning the displacement in the x direction and the displacement in the y direction according to the displacement in the x direction, namely the displacement in the x direction-the x direction adjustment angle dn and the displacement in the y direction, namely the displacement in the y direction-the y direction adjustment angle dn so as to finish the correction of the displacement in the x direction and the displacement in the y direction.
It should be understood that, in the above formula, the x-direction displacement before the equal sign is the x-direction displacement after the characteristic point (the characteristic point to be corrected) is corrected, the x-direction displacement after the equal sign is the x-direction displacement before the characteristic point (the characteristic point to be corrected) is corrected, and dn is the current frame distance corresponding to the characteristic point (the characteristic point to be corrected).
Furthermore, the x-direction adjustment angle is the median of the x-direction deflection angles of all the characteristic points, the y-direction adjustment angle is the median of the y-direction deflection angles of all the characteristic points,
that is, the x-direction deflection angle and the y-direction deflection angle of each feature point need to be obtained first, and then median is obtained through calculation and is used as the x-direction adjustment angle and the y-direction adjustment angle.
The calculation formula of the x-direction deflection angle of each characteristic point is as follows:
the x-direction deflection angles of all the feature points can be calculated by substituting the x-direction displacement corresponding to each feature point and the current frame distance dn into the above formula.
The calculation formula of the y-direction deflection angle of each feature point is as follows:
the y-direction deflection angles of all the feature points can be calculated by substituting the y-direction displacement corresponding to each feature point and the current frame distance dn into the above formula. In this embodiment, the adjustment correction is performed on the calculated x-direction displacement and y-direction displacement according to the characteristics of local displacement before and after deformation of the feature points, the synchronization errors have the same deflection angle in the vision measurement, the deflection angles in the x-direction and the y-direction are calculated first, then the deflection angle adjustment is calculated, the adjustment correction value is calculated, the x-direction displacement and the y-direction displacement are corrected by the correction value, and the synchronization error in the same direction caused by illumination or jitter is eliminated.
The present invention further provides a binocular feature matching displacement measurement apparatus, as shown in fig. 8, the apparatus includes an acquisition module 801, a feature point extraction and matching module 802, and a displacement measurement module 803, wherein:
an obtaining module 801 connected to the feature point extracting and matching module 802, configured to obtain a history frame image and a current frame image captured by a binocular camera, where the history frame image is an l0 image and an r0 image obtained by left and right cameras capturing a target at a first time, respectively, and the current frame image is an ln image and an rn image obtained by left and right cameras capturing the target at a second time, respectively;
a feature point extracting and matching module 802, connected to the displacement measuring module 803, configured to extract feature points on the historical frame image and the current frame image, where the feature points include corner sub-pixel coordinates and corresponding ORB descriptions, and perform feature point matching on the historical frame image and the current frame image according to the ORB descriptions, to determine sub-pixel coordinates of the same feature point on the l0, r0, ln, and rn images;
a displacement measurement module 803, configured to perform the following displacement measurement operations for each feature point:
respectively calculating a historical frame distance and a current frame distance corresponding to the feature point according to the parallax of the feature point in the historical frame image and the parallax of the current frame image, and calculating the Z-direction displacement of the feature point at the second moment relative to the first moment according to the historical frame distance and the current frame distance;
obtaining the pixel variation in the x direction and the pixel variation in the y direction according to the difference between the sub-pixel coordinates of the feature point on the l0 and the ln image or the sub-pixel coordinates of the feature point on the r0 and the rn image; calculating to obtain an x-direction scale and a y-direction scale corresponding to the feature point according to the current frame distance corresponding to the feature point and parameters of the binocular camera; and calculating the x-direction displacement and the y-direction image displacement of the feature point relative to the first time at the second time according to the x-direction pixel variation, the y-direction pixel variation, the x-direction scale and the y-direction scale corresponding to the feature point.
The binocular feature matching displacement measurement device provided by this embodiment, when performing displacement measurement of a target, a history frame image and a current frame image are shot by using a binocular camera, feature points extracted from the history frame image and the current frame image include corner sub-pixel coordinates and corresponding ORB descriptions, and the sub-pixel coordinates are used, so that coordinate accuracy is improved, and more accurate coordinates of each corner are obtained, meanwhile, the ORB descriptions can be used to perform accurate matching on the feature points, and subsequently, for each feature point, the sub-pixel coordinates of the feature point on l0, r0, ln, and rn images obtained by matching are used, so that displacement change can be accurately identified, and accurate measurement of spatial displacement is realized.
As shown in fig. 9, in another embodiment of the present invention, the binocular feature matching displacement measurement apparatus further includes an image distortion correction module 804, wherein the image distortion correction module 804 is respectively connected to the acquisition module 801 and the feature point extraction and matching module 802, and is configured to perform distortion correction on the historical frame image and the current frame image according to distortion correction parameters, where the distortion correction parameters are obtained by calibrating the binocular cameras.
According to the binocular feature matching displacement measurement method and device, when displacement measurement is carried out, the feature points extracted from the historical frame image and the current frame image comprise corner sub-pixel coordinates and corresponding ORB descriptions, the sub-pixel coordinates are used, coordinate precision is improved, more accurate coordinates are obtained, meanwhile, the ORB descriptions can be used for accurately matching the feature points, and subsequently, for each feature point, the sub-pixel coordinates of the feature point on l0, r0, ln and rn images obtained through matching are used, so that displacement change can be accurately identified, and accurate measurement of space displacement is achieved. In addition, after the displacement is measured, the displacement in the x direction and the displacement in the y direction are corrected according to the deflection angle in the x direction and the deflection angle in the y direction, so that the accuracy of displacement measurement is further improved.
The terms and expressions used in the specification of the present invention have been set forth for illustrative purposes only and are not meant to be limiting. The terms "first" and "second" used herein in the claims and the description of the present invention are for the purpose of convenience of distinction, have no special meaning, and are not intended to limit the present invention. It will be appreciated by those skilled in the art that changes could be made to the details of the above-described embodiments without departing from the underlying principles thereof. The scope of the invention is, therefore, to be determined only by the following claims, in which all terms are to be interpreted in their broadest reasonable sense unless otherwise indicated.
Claims (10)
1. A binocular feature matching displacement measurement method is characterized by comprising the following steps:
acquiring historical frame images and current frame images shot by a binocular camera, wherein the historical frame images are l0 images and r0 images obtained by shooting a target by a left camera and a right camera at a first moment respectively, and the current frame images are ln images and rn images obtained by shooting the target by the left camera and the right camera at a second moment respectively;
extracting feature points from the historical frame image and the current frame image, wherein the feature points comprise corner sub-pixel coordinates and corresponding ORB descriptions, matching the feature points of the historical frame image and the current frame image according to the ORB descriptions, and determining the sub-pixel coordinates of the same feature point on l0, r0, ln and rn images;
for each feature point, the following displacement measurement operations are performed:
respectively calculating the historical frame distance and the current frame distance corresponding to the feature point according to the parallax of the feature point corresponding to the historical frame image and the parallax of the current frame image, and calculating the Z-direction displacement of the feature point at the second moment relative to the first moment according to the historical frame distance and the current frame distance;
obtaining the pixel variation in the x direction and the pixel variation in the y direction according to the difference between the sub-pixel coordinates of the feature point on the l0 and the ln image or the sub-pixel coordinates of the feature point on the r0 and the rn image; calculating to obtain an x-direction scale and a y-direction scale corresponding to the feature point according to the current frame distance corresponding to the feature point and parameters of the binocular camera; and calculating the x-direction displacement and the y-direction displacement of the characteristic point relative to the first time at the second time according to the x-direction pixel variation, the y-direction pixel variation, the x-direction scale and the y-direction scale corresponding to the characteristic point.
2. The binocular feature matching displacement measurement method of claim 1, further comprising: and calibrating the binocular camera in advance to obtain the internal parameter, the external parameter and the distortion correction parameter of the binocular camera.
3. The binocular feature matching displacement measurement method of claim 2, wherein before the feature points are extracted on the history frame image and the current frame image, the method further comprises:
and carrying out distortion correction on the historical frame image and the current frame image according to the distortion correction parameters.
4. The binocular feature matching displacement measurement method of claim 2, wherein the historical frame distance and the current frame distance corresponding to the feature point are respectively calculated according to the parallax of the feature point corresponding to the historical frame image and the parallax of the current frame image as follows:
respectively calculating the historical frame distance and the current frame distance corresponding to the feature point by using a formula distance d (baseline) fx/disparity;
wherein, baseline refers to the distance between the optical centers of the left camera and the right camera, fx is the focal length in the internal reference calibrated in advance, and disparity is parallax.
5. The binocular feature matching displacement measurement method of claim 1, wherein the Z-direction displacement of the feature point at the second time relative to the first time calculated according to the historical frame distance and the current frame distance is: the Z-direction displacement is the current frame distance-the historical frame distance.
6. The binocular feature matching displacement measurement method of claim 1, wherein the binocular camera parameters include a camera horizontal viewing angle, a camera vertical viewing angle, a camera horizontal resolution and a camera vertical resolution, and the x-direction scale and the y-direction scale corresponding to the feature points are calculated according to the current frame distance and the binocular camera parameters corresponding to the feature points as follows:
according to the formula: the x-direction scale is tan (horizontal view angle of the camera) dn 2/horizontal resolution of the camera, and the x-direction scale corresponding to the feature point is obtained through calculation;
according to the formula: calculating a y-direction scale corresponding to the characteristic point, wherein the y-direction scale is tan (vertical visual angle of the camera) dn 2/vertical resolution of the camera;
where dn is the current frame distance.
7. The binocular feature matching displacement measurement method of claim 1, wherein according to the x-direction pixel variation, the y-direction pixel variation, the x-direction scale and the y-direction scale corresponding to the feature point, the x-direction displacement and the y-direction displacement of the feature point at the second time relative to the first time are calculated as:
calculating the displacement of the characteristic point in the x direction according to the displacement in the x direction, namely the pixel variation in the x direction, namely the x direction scale;
and calculating the y-direction displacement of the characteristic point according to a y-direction pixel variation quantity y-direction scale.
8. The binocular feature matching displacement measurement method of claim 1, wherein the displacement measurement operation further includes correcting for x-direction displacement and y-direction displacement:
according to the x-direction displacement-x-direction flat angle dn and the y-direction displacement-y-direction flat angle dn, re-assigning the x-direction displacement and the y-direction displacement to finish the correction of the x-direction displacement and the y-direction displacement;
the x-direction adjustment angle is the median of all the characteristic point x-direction deflection angles, the y-direction adjustment angle is the median of all the characteristic point y-direction deflection angles, and the calculation formula of each characteristic point x-direction deflection angle is as follows:
the x-direction deflection angle is atan (x-direction displacement/dn), and the calculation formula of the y-direction deflection angle of each feature point is as follows: the y-direction deflection angle is atan (y-direction displacement/dn), and dn is the current frame distance.
9. The utility model provides a binocular feature matching displacement measurement device, its characterized in that, the device is including obtaining module, characteristic point and extract and match module and displacement measurement module, wherein:
the acquisition module is connected with the feature point extraction and matching module and is used for acquiring a history frame image and a current frame image shot by a binocular camera, wherein the history frame image is a l0 image and a r0 image which are respectively shot by a left camera and a right camera at a first moment, and the current frame image is an ln image and an rn image which are respectively shot by the left camera and the right camera at a second moment;
the characteristic point extracting and matching module is connected with the displacement measuring module and is used for extracting characteristic points on the historical frame image and the current frame image, wherein the characteristic points comprise corner sub-pixel coordinates and corresponding ORB descriptions, and the characteristic points are matched with the historical frame image and the current frame image according to the ORB descriptions to determine the sub-pixel coordinates of the same characteristic point on l0, r0, ln and rn images;
the displacement measurement module is used for executing the following displacement measurement operations for each characteristic point:
respectively calculating the historical frame distance and the current frame distance corresponding to the feature point according to the parallax of the feature point corresponding to the historical frame image and the parallax of the current frame image, and calculating the Z-direction displacement of the feature point at the second moment relative to the first moment according to the historical frame distance and the current frame distance;
obtaining the pixel variation in the x direction and the pixel variation in the y direction according to the difference of the sub-pixel coordinates of the characteristic point on the l0 and the ln images or the r0 and the rn images; calculating to obtain an x-direction scale and a y-direction scale corresponding to the feature point according to the current frame distance and the binocular camera parameters corresponding to the feature point; and calculating the x-direction displacement and the y-direction displacement of the characteristic point relative to the first time at the second time according to the x-direction pixel variation, the y-direction pixel variation, the x-direction scale and the y-direction scale corresponding to the characteristic point.
10. The binocular feature matching displacement measuring device of claim 9, further comprising an image distortion correction module, wherein the image distortion correction module is respectively connected to the acquisition module and the feature point extraction and matching module, and is configured to perform distortion correction on the historical frame image and the current frame image according to distortion correction parameters, wherein the distortion correction parameters are obtained by calibrating the binocular camera.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210730878.5A CN115100284A (en) | 2022-06-24 | 2022-06-24 | Binocular feature matching displacement measurement method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210730878.5A CN115100284A (en) | 2022-06-24 | 2022-06-24 | Binocular feature matching displacement measurement method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115100284A true CN115100284A (en) | 2022-09-23 |
Family
ID=83293059
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210730878.5A Pending CN115100284A (en) | 2022-06-24 | 2022-06-24 | Binocular feature matching displacement measurement method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115100284A (en) |
-
2022
- 2022-06-24 CN CN202210730878.5A patent/CN115100284A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP5715735B2 (en) | Three-dimensional measurement method, apparatus and system, and image processing apparatus | |
| WO2011125937A1 (en) | Calibration data selection device, method of selection, selection program, and three dimensional position measuring device | |
| EP3332387B1 (en) | Method for calibration of a stereo camera | |
| JP4055998B2 (en) | Distance detection device, distance detection method, and distance detection program | |
| WO2017199285A1 (en) | Image processing device and image processing method | |
| US11295478B2 (en) | Stereo camera calibration method and image processing device for stereo camera | |
| JP3842988B2 (en) | Image processing apparatus for measuring three-dimensional information of an object by binocular stereoscopic vision, and a method for recording the same, or a recording medium recording the measurement program | |
| CN110044266B (en) | Photogrammetry system based on speckle projection | |
| JP2016217944A (en) | Measurement device and measurement method | |
| WO2021070415A1 (en) | Correction parameter calculation method, displacement amount calculation method, correction parameter calculation device, and displacement amount calculation device | |
| JP2014202661A (en) | Range finder | |
| US20220076399A1 (en) | Photographing guide device | |
| JP2008224323A (en) | Stereoscopic photograph measuring instrument, stereoscopic photograph measuring method, and stereoscopic photograph measuring program | |
| CN114754695B (en) | Multi-view-field bridge deflection measuring device and method and storage medium | |
| CN114608463B (en) | Laser optical axis soft calibration method and device, electronic equipment and medium | |
| CN115100284A (en) | Binocular feature matching displacement measurement method and device | |
| CN112805607A (en) | Measurement device, measurement method, and microscope system | |
| JP2017040542A (en) | Information processing device, information processing method, and program | |
| KR101703715B1 (en) | The apparatus for measuring camera principal point and the method thereof | |
| CN112822481A (en) | Detection method and detection system for correction quality of stereo camera | |
| US20230260159A1 (en) | Information processing apparatus, information processing method, and non-transitory computer readable medium | |
| JP6091092B2 (en) | Image processing apparatus and image processing method | |
| Liu et al. | Vision measurement method for impact point in large planar region | |
| CN115511933A (en) | Binocular image combination detection device and method and storage medium | |
| JP2011227073A (en) | Three-dimensional position measuring device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |