CN110319776B - SLAM-based three-dimensional space distance measuring method and device - Google Patents

SLAM-based three-dimensional space distance measuring method and device Download PDF

Info

Publication number
CN110319776B
CN110319776B CN201910596753.6A CN201910596753A CN110319776B CN 110319776 B CN110319776 B CN 110319776B CN 201910596753 A CN201910596753 A CN 201910596753A CN 110319776 B CN110319776 B CN 110319776B
Authority
CN
China
Prior art keywords
camera
real
point
calculating
matching
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910596753.6A
Other languages
Chinese (zh)
Other versions
CN110319776A (en
Inventor
颜冰
陈宇民
代维菊
马仪
彭兆裕
马御棠
邹德旭
王山
洪志湖
文刚
龚泽威一
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Electric Power Research Institute of Yunnan Power Grid Co Ltd
Original Assignee
Electric Power Research Institute of Yunnan Power Grid Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Electric Power Research Institute of Yunnan Power Grid Co Ltd filed Critical Electric Power Research Institute of Yunnan Power Grid Co Ltd
Priority to CN201910596753.6A priority Critical patent/CN110319776B/en
Publication of CN110319776A publication Critical patent/CN110319776A/en
Application granted granted Critical
Publication of CN110319776B publication Critical patent/CN110319776B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/02—Measuring arrangements characterised by the use of optical techniques for measuring length, width or thickness
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C3/00—Measuring distances in line of sight; Optical rangefinders
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S11/00—Systems for determining distance or velocity not using reflection or reradiation
    • G01S11/12—Systems for determining distance or velocity not using reflection or reradiation using electromagnetic waves other than radio waves
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20228—Disparity calculation for image-based rendering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Electromagnetism (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Image Analysis (AREA)

Abstract

The application provides a three-dimensional space distance measuring method and device based on SLAM, wherein the method comprises the following steps: acquiring internal parameters of a camera; the method comprises the steps of carrying out shaking removal processing on a video to be processed to obtain a processed video; aiming at the processed video, calculating an initial depth value of the camera image corresponding to the real three-dimensional point; obtaining an external parameter of the camera according to the internal parameter and the initial depth value; and calculating the space distance of the real three-dimensional point according to the external parameters. The method and the device for measuring the three-dimensional space distance based on the SLAM can effectively solve the problems of inaccurate depth estimation and unstable characteristics caused by video jitter in the conventional distance measuring method.

Description

SLAM-based three-dimensional space distance measuring method and device
Technical Field
The application relates to the technical field of computer vision, in particular to a three-dimensional space distance measuring method and device based on SLAM.
Background
The existing distance measurement method generally adopts a calibration object in a scene to solve a monocular scale, which requires a marker in the scene, but the method is very inconvenient to implement in an actual scene. In order to eliminate the scale uncertainty of monocular SLAM (Simultaneous Localization and Mapping, instantaneous Localization and Mapping), when the ORB-SLAM (0 object Request Broker-Simultaneous Localization and Mapping, ORB feature-based three-dimensional Localization and Mapping) method is adopted, the ORB feature needs to be calculated once for each key frame at the front end, which is time-consuming. And the three-thread structure of the ORB-SLAM also imposes a heavy burden on the CPU. Therefore, LSD-SLAM (Large Scale Direct monomer-Simultaneous Localization and Mapping, ORB feature-based three-dimensional positioning and map construction) is mostly used for realizing reconstruction of semi-dense scenes, and time consumption of attitude estimation is reduced.
The LSD-SLAM method needs to initialize the image depth by a random number when estimating the depth map, and then continuously updates the reference frame and the depth map by an incremental stereo matching method, which may cause inaccuracy of depth estimation. Meanwhile, in practical application, video jitter causes instability of characteristics between video frames, which affects optimization effect of the SLAM back end based on characteristics.
Disclosure of Invention
The application provides a three-dimensional space distance measuring method and device based on SLAM, and aims to solve the problems of inaccurate depth estimation and unstable characteristics caused by video jitter in the conventional distance measuring method.
In a first aspect, the present application provides a method for measuring a three-dimensional spatial distance based on SLAM, where the method includes:
acquiring an internal parameter of a camera, wherein the internal parameter is a mapping relation between a pixel coordinate and a camera coordinate corresponding to a real three-dimensional point on a camera picture;
the method comprises the steps of carrying out shake removal processing on a video to be processed to obtain a processed video, wherein the video to be processed is a video obtained based on feature matching;
aiming at the processed video, calculating an initial depth value of the camera image corresponding to the real three-dimensional point;
obtaining an external parameter of the camera according to the internal parameter and the initial depth value, wherein the external parameter is a posture parameter corresponding to the camera;
and calculating the space distance of the real three-dimensional point according to the external parameters, wherein the space distance is the distance from the real three-dimensional point to the optical center of the camera.
Optionally, the acquiring intrinsic parameters of the camera includes:
acquiring 15-20 calibration pictures of different angles shot by a camera on a calibration plate;
and carrying out corner feature detection and feature matching on each calibration picture to obtain the internal parameters of the camera.
Optionally, the dithering removing processing is performed on the video to be processed, and obtaining the processed video includes:
matching feature points between adjacent frames of the camera picture by utilizing an SIFT feature matching method to obtain matching points;
adopting a random sampling consistency method to eliminate error points in the matching points to obtain effective matching points;
calculating the number of points of the average effective matching points of the frame to be processed and the two adjacent frames;
determining a maximum number of points and a minimum number of points in the number of points;
if the ratio of the minimum point number to the maximum point number is smaller than or equal to a preset jitter threshold value, determining the frame to be processed as a jitter frame;
and eliminating all jittering frames from the video to be processed to obtain the processed video.
Optionally, for the processed video, calculating an initial depth value of the real three-dimensional point corresponding to the camera image includes:
determining an initial reference frame and a sequence of key frames from the processed video;
matching the initial reference frame with each feature point in the key frame sequence to obtain a matching result;
calculating the parallax of each feature point in the adjacent key frames according to the matching result;
calculating a depth value corresponding to the feature point according to the parallax, the focal length of the camera and the baseline distance between two adjacent frames, wherein the depth value is the distance from the real three-dimensional point to the optical center of the camera;
and obtaining an initial depth value corresponding to the depth value of each characteristic point by adopting a least square method.
Optionally, the calculating the spatial distance of the real three-dimensional point according to the external parameters includes:
calculating the coordinate of the real three-dimensional point in a world coordinate system according to the following formula,
Figure BDA0002117046630000021
where u and v represent pixel coordinates of a projection of a true three-dimensional point in the camera image, fxRepresenting the transverse proportional focal length of the camera, fyRepresenting the longitudinal proportional focal length, u, of the camera0And v0Representing the coordinates of the principal point of the camera, R and t representing the extrinsic parameters of the camera, XW、YW、ZWRepresenting the coordinates of the real three-dimensional point in a world coordinate system;
the spatial distance of the true three-dimensional point is calculated according to the following formula,
Figure BDA0002117046630000031
where D represents the distance of the real three-dimensional point to the camera optical center.
In a second aspect, the present application provides a SLAM-based three-dimensional spatial distance measuring device, the device comprising:
the camera comprises an internal parameter acquisition unit, a parameter analysis unit and a parameter comparison unit, wherein the internal parameter acquisition unit is used for acquiring internal parameters of a camera, and the internal parameters are the mapping relation between corresponding pixel coordinates of a real three-dimensional point on a camera picture and camera coordinates;
the de-jittering processing unit is used for performing de-jittering processing on a video to be processed to obtain a processed video, wherein the video to be processed is a video obtained based on feature matching;
an initial depth value calculation unit, configured to calculate an initial depth value of the real three-dimensional point corresponding to the camera image for the processed video;
the external parameter calculating unit is used for obtaining an external parameter of the camera according to the internal parameter and the initial depth value, wherein the external parameter is a posture parameter corresponding to the camera;
and the spatial distance calculation unit is used for calculating the spatial distance of the real three-dimensional point according to the external parameters, wherein the spatial distance is the distance from the real three-dimensional point to the optical center of the camera.
Optionally, the internal parameter obtaining unit includes:
the calibration picture acquisition unit is used for acquiring 15-20 calibration pictures of different angles shot by the camera on the calibration plate;
and the internal parameter determining unit is used for carrying out corner feature detection and feature matching on each calibration picture to obtain the internal parameters of the camera.
Optionally, the debounce processing unit comprises:
a matching point obtaining unit, configured to match feature points between adjacent frames of the camera picture by using an SIFT feature matching method to obtain matching points;
the effective matching point determining unit is used for eliminating error points in the matching points by adopting a random sampling consistency method to obtain effective matching points;
the point number calculating unit is used for calculating the point number of the average effective matching points of the frame to be processed and the two adjacent frames;
a special dot number determination unit for determining a maximum dot number and a minimum dot number among the dot numbers;
a jitter frame determining unit, configured to determine that the frame to be processed is a jitter frame if a ratio of the minimum point number to the maximum point number is less than or equal to a preset jitter threshold;
and the jitter frame eliminating unit is used for eliminating all jitter frames from the video to be processed to obtain the processed video.
Optionally, the initial depth value calculation unit includes:
a special frame determining unit, configured to determine an initial reference frame and a key frame sequence from the processed video;
the matching result calculation unit is used for matching the initial reference frame with each feature point in the key frame sequence to obtain a matching result;
the parallax calculation unit is used for calculating the parallax of each feature point in the adjacent key frames according to the matching result;
the depth value calculating unit is used for calculating the depth value corresponding to the characteristic point according to the parallax, the focal length of the camera and the baseline distance between two adjacent frames, wherein the depth value is the distance from the real three-dimensional point to the optical center of the camera;
and the initial depth value obtaining unit is used for obtaining the initial depth value corresponding to the depth value of each characteristic point by adopting a least square method.
Optionally, the spatial distance calculation unit includes:
a world coordinate calculation unit for calculating the coordinates of the real three-dimensional point in a world coordinate system according to the following formula,
Figure BDA0002117046630000041
where u and v represent pixel coordinates of a projection of a true three-dimensional point in the camera image, fxRepresenting the transverse proportional focal length of the camera, fyRepresenting the longitudinal proportional focal length, u, of the camera0And v0Representing the coordinates of the principal point of the camera, R and t representing the extrinsic parameters of the camera, XW、YW、ZWRepresenting the coordinates of the real three-dimensional point in a world coordinate system;
a spatial distance obtaining unit for calculating a spatial distance of the real three-dimensional point according to the following formula,
Figure BDA0002117046630000042
where D represents the distance of the real three-dimensional point to the camera optical center. The method comprises the following steps:
as can be seen from the above technologies, the present application provides a method and an apparatus for measuring a three-dimensional spatial distance based on SLAM, where the method includes: acquiring an internal parameter of a camera, wherein the internal parameter is a mapping relation between a pixel coordinate and a camera coordinate corresponding to a real three-dimensional point on a camera picture; the method comprises the steps of carrying out shaking removal processing on a video to be processed to obtain a processed video; aiming at the processed video, calculating an initial depth value of the camera image corresponding to the real three-dimensional point; obtaining an external parameter of the camera according to the internal parameter and the initial depth value; and calculating the space distance of the real three-dimensional point according to the external parameters. When the method is used, the camera is used for shooting an image of a real three-dimensional point, and internal parameters of the camera are determined according to the image of the real three-dimensional point; and then, carrying out shake removal processing on the video formed by the images of the cameras to obtain a processed video, and calculating the initial depth value of the real three-dimensional point according to the internal parameters and the processed video. And finally, calculating to obtain an external parameter of the camera, namely an attitude parameter of the camera by using the internal parameter and the initial depth value, and accurately calculating the space distance of the real three-dimensional point according to the external parameter. The method and the device for measuring the three-dimensional space distance based on the SLAM can effectively solve the problems of inaccurate depth estimation and unstable characteristics caused by video jitter in the conventional distance measuring method.
Drawings
In order to more clearly explain the technical solution of the present application, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious to those skilled in the art that other drawings can be obtained according to the drawings without any creative effort.
Fig. 1 is a flowchart of a three-dimensional spatial distance measurement method based on SLAM according to an embodiment of the present disclosure;
fig. 2 is a flowchart of a method for acquiring intrinsic parameters of a camera according to an embodiment of the present disclosure;
fig. 3 is a flowchart of a method for video de-jittering according to an embodiment of the present disclosure;
FIG. 4 is a flowchart illustrating a method for calculating initial depth values according to an embodiment of the present disclosure;
fig. 5 is a schematic diagram of a SLAM-based three-dimensional spatial distance measuring device according to an embodiment of the present disclosure.
Detailed Description
The technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, a flowchart of a method for measuring a three-dimensional spatial distance based on SLAM according to an embodiment of the present application is shown, where the method includes:
s1, obtaining intrinsic parameters of the camera, wherein the intrinsic parameters are the mapping relation between the corresponding pixel coordinates of the real three-dimensional points on the camera picture and the camera coordinates.
Aiming at the problem of distance measurement from a camera to a three-dimensional space point, the invention aims to provide a three-dimensional space distance measurement method based on an SLAM, which can eliminate the scale uncertainty of the monocular SLAM without markers in a scene, eliminate the influence of the distribution of characteristic points on the distance measurement precision, weaken the influence of video jitter on the optimization effect of the SLAM rear end, and estimate a depth map more accurately.
And acquiring the internal parameters of the camera by adopting a Zhangyingyou calibration method.
Specifically, as shown in fig. 2, a flowchart of a method for acquiring intrinsic parameters of a camera provided in an embodiment of the present application is provided, where the method includes:
s101, acquiring 15-20 calibration pictures of different angles shot by a camera on a calibration plate;
s102, performing corner feature detection and feature matching on each calibration picture to obtain internal parameters of the camera.
After the angular point feature detection and the feature matching are carried out on each calibration picture, the mapping relation between the pixel coordinates of the camera picture and the camera coordinates can be obtained by calculation
Figure BDA0002117046630000061
Where u and v represent pixel coordinates of a projection of a true three-dimensional point in the camera image, fxRepresenting the transverse proportional focal length of the camera, fyRepresenting the longitudinal proportional focal length, u, of the camera0And v0Representing the principal point coordinates, X, of the cameraC、YC、ZCRepresenting the coordinates of the real three-dimensional point in the camera coordinate system.
It should be noted that this step can be skipped if there are camera intrinsic parameters that have been calibrated in advance.
And S2, performing shake removal processing on the video to be processed to obtain the processed video, wherein the video to be processed is the video obtained based on feature matching.
Specifically, as shown in fig. 3, a flowchart of a method for video de-jittering provided by an embodiment of the present application is shown, where the method includes:
s201, matching feature points between adjacent frames of a camera picture by utilizing an SIFT feature matching method to obtain matching points;
s202, eliminating error points in the matching points by adopting a random sampling consistency method to obtain effective matching points;
s203, calculating the number of points of the average effective matching points of the frame to be processed and the two adjacent frames;
s204, determining the maximum point number and the minimum point number in the point numbers;
s205, if the ratio of the minimum point number to the maximum point number is less than or equal to a preset jitter threshold, determining that the frame to be processed is a jitter frame;
s206, all the jittering frames are removed from the video to be processed, and the processed video is obtained.
For input n-frame video sequence, matching adjacent frame time by SIFT (Scale-invariant Feature Transform) Feature matching methodAnd (4) eliminating error points in all the matching points by adopting a random sampling consistency method to obtain effective matching points. Assuming that the frame to be processed is the ith frame, the adjacent frames are the (i-1) th frame and the (i + 1) th frame, and it should be noted that if the frame is the first and last frames, there is only one adjacent frame. The calculated average effective matching point number is siS can be determinediThe minimum number of points in is sminThe maximum number of dots is smaxThen the ratio of the minimum number of points to the maximum number of points is
Figure BDA0002117046630000062
Assume a preset jitter threshold of saThen if
Figure BDA0002117046630000063
The ith frame is a jitter frame, and the jitter frame will not continuously participate in the selection and loop-back detection process of the key frame of the LSD-SLAM algorithm; if it is not
Figure BDA0002117046630000064
Then the ith frame is not a jittery frame and should be retained.
And S3, aiming at the processed video, calculating an initial depth value of the camera image corresponding to the real three-dimensional point.
Specifically, as shown in fig. 4, a flowchart of a method for calculating an initial depth value provided in an embodiment of the present application is shown, where the method includes:
s301, determining an initial reference frame and a key frame sequence from the processed video;
s302, matching the initial reference frame with each feature point in the key frame sequence to obtain a matching result;
s303, calculating the parallax of each feature point in the adjacent key frames according to the matching result;
s304, calculating a depth value corresponding to the feature point according to the parallax, the focal length of the camera and the baseline distance between two adjacent frames, wherein the depth value is the distance from the real three-dimensional point to the optical center of the camera;
s305, obtaining an initial depth value corresponding to the depth value of each feature point by adopting a least square method.
Determining an initial reference frame f from a processed videorAnd a sequence of key frames FimIf the set of feature points in the keyframes is F, a pair of initial keyframes can be corrected based on the matching result, and each feature point P is calculated according to the following equationjIn the case of the disparity of the neighboring key frames,
Z＝f·T/xd
wherein Z represents the depth value, f represents the focal length of the camera, T represents the baseline distance between two adjacent frames, and xdRepresenting parallax.
Finally, the initial depth value of each feature point can be calculated by using the least square method.
And S4, obtaining an external parameter of the camera according to the internal parameter and the initial depth value, wherein the external parameter is a posture parameter corresponding to the camera.
The video sequence is input into the LSD-SLAM, so that the processes of tracking, map construction and loop detection can be completed, the attitude estimation of the camera, namely the external parameters of the camera, and a semi-dense scene reconstruction can be obtained.
And S5, calculating the space distance of the real three-dimensional point according to the external parameters, wherein the space distance is the distance from the real three-dimensional point to the optical center of the camera.
Specifically, the coordinates of the real three-dimensional point in the world coordinate system are calculated according to the following formula,
Figure BDA0002117046630000071
where u and v represent pixel coordinates of a projection of a true three-dimensional point in the camera image, fxRepresenting the transverse proportional focal length of the camera, fyRepresenting the longitudinal proportional focal length, u, of the camera0And v0Representing the coordinates of the principal point of the camera, R and t representing the extrinsic parameters of the camera, XW、YW、ZWRepresenting the coordinates of the real three-dimensional point in a world coordinate system;
the spatial distance of the true three-dimensional point is calculated according to the following formula,
Figure BDA0002117046630000072
where D represents the distance of the real three-dimensional point to the camera optical center.
Referring to fig. 5, a schematic diagram of a SLAM-based three-dimensional spatial distance measuring device according to an embodiment of the present application is shown, where the device includes:
the camera system comprises an internal parameter acquisition unit 1, a parameter analysis unit and a parameter comparison unit, wherein the internal parameter acquisition unit is used for acquiring internal parameters of a camera, and the internal parameters are the mapping relation between corresponding pixel coordinates of a real three-dimensional point on a camera picture and camera coordinates;
the de-jittering processing unit 2 is used for performing de-jittering processing on a video to be processed to obtain a processed video, wherein the video to be processed is a video obtained based on feature matching;
an initial depth value calculating unit 3, configured to calculate, for the processed video, an initial depth value of the camera image corresponding to the real three-dimensional point;
the external parameter calculating unit 4 is used for obtaining an external parameter of the camera according to the internal parameter and the initial depth value, wherein the external parameter is a posture parameter corresponding to the camera;
and the spatial distance calculation unit 5 is configured to calculate a spatial distance of the real three-dimensional point according to the external parameter, where the spatial distance is a distance from the real three-dimensional point to an optical center of a camera.
Optionally, the intrinsic parameter acquiring unit 1 includes: the calibration picture acquisition unit is used for acquiring 15-20 calibration pictures of different angles shot by the camera on the calibration plate; and the internal parameter determining unit is used for carrying out corner feature detection and feature matching on each calibration picture to obtain the internal parameters of the camera.
Optionally, the debounce processing unit 2 comprises: a matching point obtaining unit, configured to match feature points between adjacent frames of the camera picture by using an SIFT feature matching method to obtain matching points; the effective matching point determining unit is used for eliminating error points in the matching points by adopting a random sampling consistency method to obtain effective matching points; the point number calculating unit is used for calculating the point number of the average effective matching points of the frame to be processed and the two adjacent frames; a special dot number determination unit for determining a maximum dot number and a minimum dot number among the dot numbers; a jitter frame determining unit, configured to determine that the frame to be processed is a jitter frame if a ratio of the minimum point number to the maximum point number is less than or equal to a preset jitter threshold; and the jitter frame eliminating unit is used for eliminating all jitter frames from the video to be processed to obtain the processed video.
Optionally, the initial depth value calculation unit 3 includes: a special frame determining unit, configured to determine an initial reference frame and a key frame sequence from the processed video; the matching result calculation unit is used for matching the initial reference frame with each feature point in the key frame sequence to obtain a matching result; the parallax calculation unit is used for calculating the parallax of each feature point in the adjacent key frames according to the matching result; the depth value calculating unit is used for calculating the depth value corresponding to the characteristic point according to the parallax, the focal length of the camera and the baseline distance between two adjacent frames, wherein the depth value is the distance from the real three-dimensional point to the optical center of the camera; and the initial depth value obtaining unit is used for obtaining the initial depth value corresponding to the depth value of each characteristic point by adopting a least square method.
Optionally, the spatial distance calculating unit 5 includes: a world coordinate calculation unit for calculating the coordinates of the real three-dimensional point in a world coordinate system according to the following formula,
Figure BDA0002117046630000081
where u and v represent pixel coordinates of a projection of a true three-dimensional point in the camera image, fxRepresenting the transverse proportional focal length of the camera, fyRepresenting the longitudinal proportional focal length, u, of the camera0And v0Representing the coordinates of the principal point of the camera, R and t representing the extrinsic parameters of the camera, XW、YW、ZWRepresenting the coordinates of the real three-dimensional point in a world coordinate system;
a spatial distance obtaining unit for calculating a spatial distance of the real three-dimensional point according to the following formula,
Figure BDA0002117046630000091
where D represents the distance of the real three-dimensional point to the camera optical center.
It should be noted that, in specific implementations, the present invention also provides a computer storage medium, where the computer storage medium may store a program, and when the program is executed, the program may include some or all of the steps in each embodiment of the user identity service providing method or the user registration method provided by the present invention. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a Random Access Memory (RAM).
Those skilled in the art will readily appreciate that the techniques of the embodiments of the present invention may be implemented as software plus a required general purpose hardware platform. Based on such understanding, the technical solutions in the embodiments of the present invention may be essentially or partially implemented in the form of a software product, which may be stored in a storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments or some parts of the embodiments.
Other embodiments of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.
It will be understood that the present application is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims (10)

1. A three-dimensional space distance measuring method based on SLAM is characterized by comprising the following steps:
acquiring an internal parameter of a camera, wherein the internal parameter is a mapping relation between a pixel coordinate and a camera coordinate corresponding to a real three-dimensional point on a camera picture;
the method comprises the steps of carrying out shake removal processing on a video to be processed to obtain a processed video, wherein the video to be processed is a video obtained based on feature matching;
aiming at the processed video, calculating an initial depth value of the camera image corresponding to the real three-dimensional point;
obtaining an external parameter of the camera according to the internal parameter and the initial depth value, wherein the external parameter is a posture parameter corresponding to the camera;
and calculating the space distance of the real three-dimensional point according to the external parameters, wherein the space distance is the distance from the real three-dimensional point to the optical center of the camera.
2. The method of claim 1, wherein the acquiring intrinsic parameters of the camera comprises:
acquiring 15-20 calibration pictures of different angles shot by a camera on a calibration plate;
and carrying out corner feature detection and feature matching on each calibration picture to obtain the internal parameters of the camera.
3. The method of claim 1, wherein the de-jittering the to-be-processed video to obtain the processed video comprises:
matching feature points between adjacent frames of the camera picture by utilizing an SIFT feature matching method to obtain matching points;
adopting a random sampling consistency method to eliminate error points in the matching points to obtain effective matching points;
calculating the number of points of the average effective matching points of the frame to be processed and the two adjacent frames;
determining a maximum number of points and a minimum number of points in the number of points;
if the ratio of the minimum point number to the maximum point number is smaller than or equal to a preset jitter threshold value, determining the frame to be processed as a jitter frame;
and eliminating all jittering frames from the video to be processed to obtain the processed video.
4. The method of claim 1, wherein calculating, for the processed video, initial depth values for real three-dimensional points corresponding to a camera image comprises:
determining an initial reference frame and a sequence of key frames from the processed video;
matching the initial reference frame with each feature point in the key frame sequence to obtain a matching result;
calculating the parallax of each feature point in the adjacent key frames according to the matching result;
calculating a depth value corresponding to the feature point according to the parallax, the focal length of the camera and the baseline distance between two adjacent frames, wherein the depth value is the distance from the real three-dimensional point to the optical center of the camera;
and obtaining an initial depth value corresponding to the depth value of each characteristic point by adopting a least square method.
5. The method of claim 1, wherein said calculating the spatial distance of the true three-dimensional point from the extrinsic parameters comprises:
calculating the coordinate of the real three-dimensional point in a world coordinate system according to the following formula,
Figure FDA0002947062300000021
where u and v represent the pixel coordinates of the projection of a real three-dimensional point in the camera image, fxRepresenting the transverse proportional focal length of the camera, fyRepresenting the longitudinal proportional focal length, u, of the camera0And v0Representing the coordinates of the principal point of the camera, R and t representing the extrinsic parameters of the camera, XW、YW、ZWRepresenting the coordinates of the real three-dimensional point in a world coordinate system;
the spatial distance of the true three-dimensional point is calculated according to the following formula,
Figure FDA0002947062300000022
where D represents the distance of the real three-dimensional point to the camera optical center.
6. A SLAM-based three-dimensional spatial distance measuring apparatus, comprising:
the camera comprises an internal parameter acquisition unit, a parameter analysis unit and a parameter comparison unit, wherein the internal parameter acquisition unit is used for acquiring internal parameters of a camera, and the internal parameters are the mapping relation between corresponding pixel coordinates of a real three-dimensional point on a camera picture and camera coordinates;
the de-jittering processing unit is used for performing de-jittering processing on a video to be processed to obtain a processed video, wherein the video to be processed is a video obtained based on feature matching;
an initial depth value calculation unit, configured to calculate an initial depth value of the real three-dimensional point corresponding to the camera image for the processed video;
the external parameter calculating unit is used for obtaining an external parameter of the camera according to the internal parameter and the initial depth value, wherein the external parameter is a posture parameter corresponding to the camera;
and the spatial distance calculation unit is used for calculating the spatial distance of the real three-dimensional point according to the external parameters, wherein the spatial distance is the distance from the real three-dimensional point to the optical center of the camera.
7. The apparatus of claim 6, wherein the intrinsic parameter obtaining unit comprises:
the calibration picture acquisition unit is used for acquiring 15-20 calibration pictures of different angles shot by the camera on the calibration plate;
and the internal parameter determining unit is used for carrying out corner feature detection and feature matching on each calibration picture to obtain the internal parameters of the camera.
8. The apparatus of claim 6, wherein the debounce processing unit comprises:
a matching point obtaining unit, configured to match feature points between adjacent frames of the camera picture by using an SIFT feature matching method to obtain matching points;
the effective matching point determining unit is used for eliminating error points in the matching points by adopting a random sampling consistency method to obtain effective matching points;
the point number calculating unit is used for calculating the point number of the average effective matching points of the frame to be processed and the two adjacent frames;
a special dot number determination unit for determining a maximum dot number and a minimum dot number among the dot numbers;
a jitter frame determining unit, configured to determine that the frame to be processed is a jitter frame if a ratio of the minimum point number to the maximum point number is less than or equal to a preset jitter threshold;
and the jitter frame eliminating unit is used for eliminating all jitter frames from the video to be processed to obtain the processed video.
9. The apparatus of claim 6, wherein the initial depth value calculation unit comprises:
a special frame determining unit, configured to determine an initial reference frame and a key frame sequence from the processed video;
the matching result calculation unit is used for matching the initial reference frame with each feature point in the key frame sequence to obtain a matching result;
the parallax calculation unit is used for calculating the parallax of each feature point in the adjacent key frames according to the matching result;
the depth value calculating unit is used for calculating the depth value corresponding to the characteristic point according to the parallax, the focal length of the camera and the baseline distance between two adjacent frames, wherein the depth value is the distance from the real three-dimensional point to the optical center of the camera;
and the initial depth value obtaining unit is used for obtaining the initial depth value corresponding to the depth value of each characteristic point by adopting a least square method.
10. The apparatus according to claim 6, wherein the spatial distance calculating unit comprises:
a world coordinate calculation unit for calculating the coordinates of the real three-dimensional point in a world coordinate system according to the following formula,
Figure FDA0002947062300000031
where u and v represent the pixel coordinates of the projection of a real three-dimensional point in the camera image, fxRepresenting the transverse proportional focal length of the camera, fyRepresenting the longitudinal proportional focal length, u, of the camera0And v0Representing the coordinates of the principal point of the camera, R and t representing the extrinsic parameters of the camera, XW、YW、ZWRepresenting the coordinates of the real three-dimensional point in a world coordinate system;
a spatial distance obtaining unit for calculating a spatial distance of the real three-dimensional point according to the following formula,
Figure FDA0002947062300000032
where D represents the distance of the real three-dimensional point to the camera optical center.
CN201910596753.6A 2019-07-03 2019-07-03 SLAM-based three-dimensional space distance measuring method and device Active CN110319776B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910596753.6A CN110319776B (en) 2019-07-03 2019-07-03 SLAM-based three-dimensional space distance measuring method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910596753.6A CN110319776B (en) 2019-07-03 2019-07-03 SLAM-based three-dimensional space distance measuring method and device

Publications (2)

Publication Number Publication Date
CN110319776A CN110319776A (en) 2019-10-11
CN110319776B true CN110319776B (en) 2021-05-07

Family

ID=68122500

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910596753.6A Active CN110319776B (en) 2019-07-03 2019-07-03 SLAM-based three-dimensional space distance measuring method and device

Country Status (1)

Country Link
CN (1) CN110319776B (en)

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106289071B (en) * 2016-08-18 2018-10-19 温州大学 A kind of structure three-dimensional displacement monocular photographing measurement method
DE102017107336A1 (en) * 2017-04-05 2018-10-11 Testo SE & Co. KGaA Measuring device and corresponding measuring method
CN108648240B (en) * 2018-05-11 2022-09-23 东南大学 Non-overlapping view field camera attitude calibration method based on point cloud feature map registration
CN109855822B (en) * 2019-01-14 2019-12-06 中山大学 unmanned aerial vehicle-based high-speed rail bridge vertical dynamic disturbance degree measuring method
CN109737874B (en) * 2019-01-17 2021-12-03 广东省智能制造研究所 Object size measuring method and device based on three-dimensional vision technology

Also Published As

Publication number Publication date
CN110319776A (en) 2019-10-11

Similar Documents

Publication Publication Date Title
US10559090B2 (en) Method and apparatus for calculating dual-camera relative position, and device
US8447099B2 (en) Forming 3D models using two images
US8452081B2 (en) Forming 3D models using multiple images
CN111652086B (en) Face living body detection method and device, electronic equipment and storage medium
CN110996082B (en) Projection adjusting method and device, projector and readable storage medium
JP2003317117A (en) Method and system for 3d smoothing within bound of error region of matching curve
US8531505B2 (en) Imaging parameter acquisition apparatus, imaging parameter acquisition method and storage medium
US20220414908A1 (en) Image processing method
JP6061770B2 (en) Camera posture estimation apparatus and program thereof
US11741615B2 (en) Map segmentation method and device, motion estimation method, and device terminal
JP2022509329A (en) Point cloud fusion methods and devices, electronic devices, computer storage media and programs
US20170289516A1 (en) Depth map based perspective correction in digital photos
JP2019091122A (en) Depth map filter processing device, depth map filter processing method and program
US11475629B2 (en) Method for 3D reconstruction of an object
EP3588437B1 (en) Apparatus that generates three-dimensional shape data, method and program
CN110319776B (en) SLAM-based three-dimensional space distance measuring method and device
JP6579816B2 (en) Image processing apparatus, image processing method, and program
KR20180019329A (en) Depth map acquisition device and depth map acquisition method
CN108510578B (en) Three-dimensional model building method and device and electronic equipment
JP2002077941A (en) Apparatus and method for generating depth image as well as computer readable recording medium recording program to execute the method in computer
CN112396117A (en) Image detection method and device and electronic equipment
JP2006300656A (en) Image measuring technique, device, program, and recording medium
RU2791081C2 (en) Method of three-dimensional object reconstruction
CN111369612A (en) Three-dimensional point cloud image generation method and equipment
CN112529848B (en) Panorama updating method and device, readable medium and equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant