CN114663486A - Building height measurement method and system based on binocular vision - Google Patents

Building height measurement method and system based on binocular vision Download PDF

Info

Publication number
CN114663486A
CN114663486A CN202111664762.8A CN202111664762A CN114663486A CN 114663486 A CN114663486 A CN 114663486A CN 202111664762 A CN202111664762 A CN 202111664762A CN 114663486 A CN114663486 A CN 114663486A
Authority
CN
China
Prior art keywords
camera
building
disparity map
height
binocular
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111664762.8A
Other languages
Chinese (zh)
Inventor
陆涛
曹颂
钟星
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing Tuling Video Technology Co ltd
Original Assignee
Nanjing Tuling Video Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing Tuling Video Technology Co ltd filed Critical Nanjing Tuling Video Technology Co ltd
Priority to CN202111664762.8A priority Critical patent/CN114663486A/en
Publication of CN114663486A publication Critical patent/CN114663486A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G06T7/85—Stereo camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20228—Disparity calculation for image-based rendering

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Geometry (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

The invention discloses a building height measurement method and a building height measurement system based on binocular vision, wherein the method comprises the following steps: calibrating the binocular camera, including calculating a basic matrix F, an essential matrix E and a distortion parameter D of the camera, and performing epipolar correction on the binocular camera; a binocular camera field building step; a step of calculating a disparity map, which comprises calculating the disparity map based on an SGM algorithm; calculating the height of the floor according to the v-disparity map calculated by the disparity map, comprising the following steps: by finding the geometric correspondence on the v-disparity map, the position of the building in the image is located, and the vertical height of the building is calculated. According to the invention, a binocular vision-based building height measurement algorithm and system are constructed, the pixel coordinates of the object height are more flexible to obtain from the building of the camera to the calculation of the building height, and a measurer can measure the height of any floor only by placing a binocular camera.

Description

Building height measurement method and system based on binocular vision
Technical Field
The invention relates to a building height measurement method and system based on binocular vision, and belongs to the technical field of building height measurement.
Background
In recent years, urbanization is rapidly developed, high buildings are pulled out, and the height and construction condition of floors are measured, so that the method plays an important role in monitoring construction progress and preventing engineering hidden dangers. The common building height measurement technology at present is triangulation height measurement. Triangulation is a method of determining the height difference between two control points by observing the horizontal distance and zenith distance (or elevation angle) between the two control points. The observation method is simple and is slightly limited by terrain conditions, and is a basic method for measuring the elevation of the geodetic control point. However, errors are brought to the triangular elevation measurement result by measurement errors of the side length, the vertical angle, the height of the instrument, the target opening and measurement errors of the atmospheric vertical refractive index K. Moreover, the measurement method cannot achieve the effect of real-time monitoring, and needs strong professional knowledge and professionals to measure and calculate. This time consuming and expertise-demanding task can be completely replaced by a computer and a camera.
By using Binocular Stereo Vision (Binocular Stereo Vision) technology, the height of the floor and the construction situation can be accurately calculated in real time. Binocular stereo vision is an important form of machine vision, and is a method for acquiring three-dimensional geometric information of an object by acquiring two images of the object to be measured from different positions by using imaging equipment based on a parallax principle and calculating position deviation between corresponding points of the images. The binocular vision can acquire the geometric information of the target to be detected through a stereo matching algorithm, and the position and the size of the target can be inferred through some simple calibration data. The stereo vision matching algorithm is most commonly the matching of characteristic points, and the physical information of a target object in a three-dimensional world is obtained by calculating and matching the characteristic points of the left image and the right image.
Patent 1 discloses a building height measuring method based on a single digital photo, with application number CN200910028010.5, specifically disclosing: based on a single digital photo, the height of the building is measured by adopting a reverse direct linear transformation calculation method on the basis of a direct linear change formula.
Patent 2 discloses a method and a device for measuring height of a human body based on a binocular vision technology, wherein the publication number is CN 104173054A, and the method comprises the steps of obtaining image coordinates of a human head tip point of a human body target in a scene image shot by a binocular camera and depth information corresponding to the image coordinates, calculating world coordinates of the human head tip point under a camera coordinate system, and further measuring the height of the human body target. The fit of the tested personnel is required to be low, the height measurement can be completed only by standing in the measurable range shot by the camera, the heights of a plurality of human targets in a scene can be measured simultaneously, and the measurement precision is high.
Patent 3 discloses a method and a device for measuring height of a height-limited vehicle road based on binocular vision, the publication number is CN 110207650A, the method and the device for measuring height of the height-limited vehicle road based on binocular vision are provided, two wide-angle cameras are used, a binocular vision system is used for shooting, tracking and positioning a height-limited beam target in real time, the binocular vision system is used for measuring the distance between a vehicle and the height of the relative roof of the height-limited frame in real time in the driving process of the vehicle, and early warning information is provided for a driver.
The authors of the patent are Zhao hong, Zhang Shi Hua, Hu Xinyu, and so on, and relate to application research on building height measurement by building shadow of satellite image No. Beijing [ J ] Beijing surveying and mapping, 2010,000(002):38-40. the building height is calculated by satellite photos, building shadow, sun height angle, and other information.
Disclosure of Invention
The invention aims to overcome the technical defects in the prior art, solve the technical problems and provide a building height measuring method and system based on binocular vision.
The invention specifically adopts the following technical scheme: a building height measurement method based on binocular vision comprises the following steps:
step SS 1: the binocular camera calibration step comprises: a StereoCalibration tool carried by the Matlab Toolbox observes a chessboard of the same chess with a known size from different angles to calculate a basic matrix F and an essential matrix E of the camera and a distortion parameter D of the camera, and performs epipolar correction on the binocular camera;
step SS 2: performing field construction of a binocular camera;
step SS 3: a step of calculating a disparity map, which comprises a disparity map calculated based on an SGM algorithm;
step SS 4: calculating the height of the floor according to the v-disparity map calculated by the disparity map, comprising the following steps: by finding the geometric correspondence on the v-disparity map, the position of the building in the image is located, and the vertical height of the building is calculated.
As a preferred embodiment, the epipolar line correction in step SS1 specifically includes: and (3) rotating the imaging plane of the binocular camera by half respectively by adopting a Bouguet epipolar line correction method, so that the error caused by the re-projection of the left and right images is minimum, and the common area of the left and right views is maximum.
As a preferred embodiment, step SS2 specifically includes: the binocular camera is built on site, the camera is horizontally placed and faces the floor to be detected, the floor can be ensured to be seen from the left lens and the right lens, and the bottom of the image can be used for clearly seeing a part of ground.
As a preferred embodiment, step SS3 specifically includes: calculating a disparity map by an SGM dense matching algorithm, wherein in a calibrated binocular system, P (X, Y, Z) represents any point of a world coordinate system, (u)l,vl) And (u)r,vr) Respectively representing the projection of the point P on the image plane coordinate systems of the two cameras; (u)0,v0) Is the projection of the optical center of the camera onto the image plane; f represents the focal length of the camera; b represents the base length in binocular vision; the formula for calculating the parallax value delta is shown in formula (1),
Figure BDA0003448061720000041
any point on the left camera (u)l,vl) Finding the corresponding point on the right camera (u) by the SGM dense algorithmr,vr) Then, the point (u) is calculated by the formula (1)l,vl) The disparity value delta is calculated for each point, and then a disparity map is formed.
As a preferred embodiment, step SS4 specifically includes:
step SS 41: projecting the disparity map according to the horizontal direction to obtain a v-disparity map; the horizontal coordinate of the v-disparity value represents a disparity value delta, and the vertical coordinate represents an image height;
step SS 42: calculating and obtaining the lowest point p of the building on the v-disparity map1(d△,y1) And highest point p of building2(d△,y2) Because the building to be measured is vertical to the ground, the building forms a vertical line segment l on the v-disparity mapb(ii) a While the road is horizontal, a slope is reflected on the v-disparity map<Inclined line l of 0r，lbIs a group of points which are perpendicular to an x axis on a v-disparity map and have the same or close values of delta, and l is obtained by a least square fitting modebFitting out; l obtained by fittingbIs point p1(d△,y1) (ii) a Choosing a Ransac method fitting that is robust to noise points lr(ii) a Finally, find lbAnd lrThe intersection of (A) is p2(d△,y2) Then calculate | y2-y1|，|y2-y1I is the height difference of the coordinate system of the floor on the image plane;
step SS 43: combining the formula (1) and the formula (2) to obtain a formula (3), calculating a conversion equation of point coordinates P (X, Y, Z) of a world coordinate system and points (u, v) in the image,
Figure BDA0003448061720000042
where dx, dy, f are camera focal length related parameters, (u)0,v0) Representing the origin of coordinates of the phase plane; (u, v) are building coordinates captured on the image, and (X, Y, Z,1) are their corresponding physical world coordinates; the leftmost (u, v,1) represents the coordinates of a point on the image plane, 1 is a homogeneous representation, R represents the rotation matrix of the camera, and t represents the translation matrix;
Figure BDA0003448061720000051
substituting Z into X and Y to obtain new expression modes of X and Y, see formula (4),
Figure BDA0003448061720000052
f is the focal length of the camera, dx is the actual size of each pixel on the horizontal axis, dy is the actual size of each pixel on the vertical axis, and the three parameters are obtained by calibrating the camera; will point p1And p2Substituting the formula (4) to obtain a calculation formula (5) of the final floor height,
Figure BDA0003448061720000053
where b represents the base length of the left and right cameras, which is a fixed value in m, the actual height H of the target is calculated by equation (5).
The invention also provides a building height measuring system based on binocular vision, which comprises:
the binocular camera calibration module specifically executes: a StereoCalibration tool carried by the Matlab Toolbox observes a chessboard of the same chess with a known size from different angles to calculate a basic matrix F and an essential matrix E of the camera and a distortion parameter D of the camera, and performs epipolar correction on the binocular camera;
the camera building module specifically executes: building a binocular camera on site;
the disparity map generation module specifically executes: calculating a disparity map based on an SGM algorithm;
the height calculation module specifically executes: calculating the height of the floor according to the v-disparity map calculated by the disparity map, comprising the following steps: by finding the geometric correspondence on the v-disparity map, the position of the building in the image is located, and the vertical height of the building is calculated.
As a preferred embodiment, the epipolar line correction in the binocular camera calibration module specifically includes: and (3) rotating the imaging plane of the binocular camera by half respectively by adopting a Bouguet epipolar line correction method, so that the error caused by the re-projection of the left and right images is minimum, and the common area of the left and right views is maximum.
As a preferred embodiment, the camera building module specifically performs: the binocular camera is built on site, the camera is horizontally placed and faces the floor to be detected, the floor can be ensured to be seen from the left lens and the right lens, and part of the ground can be seen from the bottom of the image.
As a preferred embodiment, the disparity map generating module specifically executes: the disparity map is generated by a dense matching algorithm such as SGM.
As a preferred embodiment, the height calculating module specifically executes:
projecting the disparity map in the horizontal direction to obtain a v-disparity map; the horizontal coordinate of the v-disparity value represents a disparity value delta, and the vertical coordinate represents an image height; calculating and obtaining the lowest point p of the building on the v-disparity map1(d△,y1) And highest point p of building2(d△,y2) (ii) a The building forms a vertical line segment l on the v-disparity mapb(ii) a While the road is horizontal, a slope is reflected on the v-disparity map<Inclined line l of 0r；lbIs a group of points which are perpendicular to an x axis on a v-disparity map and have the same or close values of delta, and l is obtained by a least square fitting modebFitting out; l is fitted outbThe upper vertex of (1) is point p1(d△,y1) (ii) a Selecting a Randac method fitting l more robust to noise pointsr(ii) a Finally, find lbAnd lrIs p2(d△,y2) Then calculate | y2-y1|；
The actual height H of the floor is calculated using equation (6),
Figure BDA0003448061720000061
where, fy is f/dy, f is the focal length of the camera, dy is the actual size of each pixel on the vertical axis, b represents the base length of the left and right cameras, and the unit is m.
The invention achieves the following beneficial effects: 1, the building height measurement method and system based on binocular vision, disclosed by the invention, have the advantages that the pixel coordinates of the height of an object are more flexible, and the height of any floor can be measured only by placing a binocular camera without professional height measurement knowledge. 2, the invention can monitor the height of the floor in real time through the binocular vision system, such as the building construction progress of a construction site, and does not need people to survey and measure on site, and after the system is accessed into the network, the system can automatically report the measurement result every day. 3, the height measurement method provided by the invention is more accurate than the height measurement method of the monocular camera.
Drawings
Fig. 1 is a schematic view of the binocular vision system coordinate system of the present invention.
Fig. 2 is a schematic view of the arrangement position of the binocular camera according to the present invention.
Fig. 3 is a flow chart of a building height measuring method based on binocular vision according to the invention.
Fig. 4 is a v-disparity diagram illustration of the present invention, wherein a pixel map (left) and a v-disparity map (right).
Detailed Description
The invention is further described below with reference to the accompanying drawings. The following examples are only for illustrating the technical solutions of the present invention more clearly, and the protection scope of the present invention is not limited thereby.
In monocular vision, a picture obtained by shooting by a camera is obtained by arranging pixel points one by one. Each pixel point is a projection of a point of the physical world onto the phase plane. Assuming that the coordinates of a certain point in the physical world are (X, Y, Z,1), it is generally expressed in a homogeneous coordinate system, and when the fourth coordinate is equal to 0, it represents a point at infinity. This coordinate is an absolute value, and if a certain origin is set, any physical world coordinate is fixed. The projection of the camera coordinate system on the image constitutes an image, and the image coordinate system constitutes a phase plane coordinate system. The leftmost (u, v,1) in formula (1) represents the coordinates of a point on the image plane, 1 is a homogeneous representation, and this formula represents the conversion of a point of the world coordinate system into a point of the phase plane coordinate system, where dy, dy and f are camera focal length related parameters, (u, v,1) is a homogeneous representation0,v0) Representing the phase plane origin of coordinates, R the rotation matrix of the camera and t the translation matrix.
Figure BDA0003448061720000081
Finally, the binocular matching method for detecting the building height is to find its corresponding physical world coordinates (X, Y, Z) through the building coordinates (u, v) photographed on the image, so that its upper and lower height differences can be calculated.
The method provided by the invention comprises the following steps: 1) calibrating a binocular camera, 2) placing the binocular phase on site, 3) calculating a disparity map by using an SGM algorithm, 4) calculating a v-disparity map from the disparity map, locating the position of the building in the image by finding a geometric corresponding relation on the v-disparity map, and calculating the vertical height of the building. The flow diagram is shown in FIG. 3.
2.1 binocular Camera calibration
The purpose of the binocular camera calibration is to calculate the Fundamental Matrix (F Matrix) of the camera, the Essential Matrix (E Matrix) and the distortion parameters D of the camera. The F matrix contains dx, du, F in equation (1). The E matrix contains R and T in equation (1). The calculation of the F matrix is consistent with the monocular camera calibration and will not be described here too much. The calculation accuracy of the E matrix directly influences the calculation of the disparity map, and has important influence on building height measurement. The present invention uses the Matlab Toolbox self-contained Stereo Calibration tool to determine F, E and D by observing the same chess checkerboard with known size at different angles.
In addition, the binocular camera calibration process is different from the monocular camera calibration process in that the rectification of the left camera and the right camera, namely epipolar line correction, is needed finally after the R, T matrix is calculated. The correction has the advantages that when the stereo matching search is carried out later, the search is only carried out near the same height (epipolar line), and the efficiency is greatly improved. The most common correction method is a Bouguet epipolar line correction method, and the simple expression is that the imaging planes of the left camera and the right camera are respectively rotated by half, so that the error caused by the re-projection of the left image and the right image is minimum, and the common area of the left view and the right view is maximum. After correction, the image can be cropped as required, and the center and the edge of one image need to be reselected, so that the left and right overlapped parts are maximized. Before correction, the optical centers of the cameras are not mutually parallel, after correction, poles are at infinity, the optical axes of the two cameras are parallel, and the projection heights of the same physical target point on the left phase plane and the right phase plane are consistent.
2.2 construction of the Camera
Camera set-up referring to the schematic diagram set forth in fig. 2, the binocular camera looks up the floor level with the bottom to see the ground, and in order to be able to see the higher floors, the present invention allows the horizontal distance of the camera from the floor to be between 50m and 100 m.
2.3 computing disparity maps
In a calibrated stereoscopic vision coordinate system, the image planes of the left camera and the right camera belong to the same plane in a physical space. FIG. 1 shows the structure of a binocular vision system (u)l,vl) And (u)r,vr) Respectively belonging to the image plane coordinate systems of the left camera and the right camera. (u)0,v0) Is the projection of the optical center of the camera onto the image plane. f denotes the focal length of the camera, b denotes the base length in binocular vision (two camera optical centers O)lAnd OrThe distance therebetween). (X, Y, Z) represents a world coordinate system, and in the present invention, the camera looks up the floor so that the X-axis is parallel to the horizontal, the Y-axis is parallel to the vertical, and the Z-axis is perpendicular to the camera plane. Point O is the origin of the world coordinate system, which is distributed at the midpoint of the connecting line of the centers of the two cameras. In the system, a certain point P (X, Y, Z) under the world coordinate system is respectively projected on two image planes to obtain respective image plane coordinates, the calculation formula is as follows,
Figure BDA0003448061720000101
wherein u islAnd urThe connecting line of (c) is called epipolar line, point (u)l,vl) The parallax value Δ of (a) can be calculated by the formula (2). Each pixel in the image corresponds to its own viewAnd the difference value is used for forming a disparity map. Each disparity value is inversely proportional to the distance, the closer the distance, the greater the disparity. The height measurement algorithm provided by the invention has no too much requirement on the calculation time, and because the floor height does not change within a certain time, the parallax map calculation method SGM based on dense matching with higher precision is adopted, so that more accurate distance information can be obtained.
2.4 building height Using Dual Vision
After the disparity map is calculated in step 2.3, the present invention needs to start calculating the height of the target building. In the disparity map, the shade of the color represents the disparity of the pixel point, and the darker the color is, the stronger the intensity of the corresponding pixel value is, and the closer the actual physical distance of the target to the camera is. The disparity map is projected in the horizontal direction to obtain a v-disparity map, as shown on the right of fig. 4. The horizontal coordinate of the v-disparity value represents the disparity value delta (from small to large), the vertical coordinate represents the image height, wherein the depth of the color represents the number of pixel points, the deeper the color is, the more the pixel points sharing the disparity value are, namely, the actual distances from the physical targets corresponding to the pixel points to the camera are the same. Since the detected target is substantially perpendicular to the ground, what is reflected on the v-disparity map is a vertical line segment (e.g. p in the right of fig. 4)1To p2Line segment of) is given asb(ii) a While the road is horizontal, it is a line (slope) that reacts on the v-disparity map<0) The inclined line (see the right oblique line in FIG. 4) is represented by lr。
In the following, the present invention shows how to calculate the height H of the actual floor through a process of formula derivation.
Step 1: first, p needs to be obtained by calculation1(d△,y1) And p2(d△,y2) As shown in fig. 4, right. lbIs a group of points which are vertical to the x axis on the v-disparity map and have the same or close delta values, and l is easy to be fitted by a least square methodbAnd (6) fitting. lbThe upper vertex of (1) is point p1(d△,y1). Because the left and right images in the ground area are very small in the system designed by the invention, the invention selects pairsRanac method fitting l with more robust noise pointsr. Finally, find lbAnd lrThe intersection of (A) is p2(d△,y2) Then calculate | y2-y1|，|y2-y1And | is the height difference of the coordinate system of the floor on the image plane. Finally, the invention calculates the height difference Y in the world coordinate system after conversion2-Y1|。
And 2, step: combining equation (1) and equation (2), the point coordinates (X, Y, Z) of the world coordinate system can be calculated:
Figure BDA0003448061720000111
substituting Z into X and Y to obtain new expression of X and Y, see formula (4),
Figure BDA0003448061720000121
where fx is f/dxfy is f/dy, f is the focal length of the camera, dx is the actual size of each pixel on the horizontal axis, and dy is the actual size of each pixel on the vertical axis, which can be obtained by camera calibration.
Will point p1And p2Substituting the formula (4) to obtain a calculation formula (5) of the final floor,
Figure BDA0003448061720000122
where b represents the baseline length of the left and right cameras, and is a fixed value in m. The actual height H of the target can be calculated by equation (5).
As can be seen from equation (5), the higher the performance of the binocular camera, the higher the parallax value d△The more accurate the calculation, the more accurate the height measurement. In the specific implementation process, if only one calibrated binocular camera is used for height measurement and no multi-camera or camera motion is involved, the camera coordinate system can be directly set as the world coordinate system sourcePoints, thereby avoiding the computation of the external parameter matrix.
Figure BDA0003448061720000123
Consider an identity matrix, E is a 3x3 identity matrix.
The internal parameters of the binocular camera mainly comprise a lens focal length, a lens distortion coefficient, a pixel size and a principal point coordinate, and the external parameters comprise a rotation matrix and a translation vector of the camera and the installation height of the binocular camera. The binocular matching is to establish the alignment of target points of two projected images under different visual angles under the same scene. The binocular vision measuring method has the advantages of high efficiency, proper precision, simple system structure, low cost and the like, and is very suitable for online and non-contact product detection and quality control in a manufacturing field. In the measurement of a moving object, the binocular vision method is a more effective measurement method because the image acquisition is completed instantaneously.
The specific implementation structure chart is as shown in fig. 2, wherein the binocular camera is horizontally shot, the camera can be arranged at a high position near a building to be measured, and the unmanned aerial vehicle carrying the binocular camera can be used for shooting.
The flow chart of the invention is shown in fig. 3, and the specific flow is as follows.
Step (1), binocular camera calibration: selecting a binocular camera, preparing a chessboard grid of chess with 10 grids (width) × 8 grids (height) and 60mm width of each grid, printing the chessboard grid by a piece of big paper, closely adhering the chessboard grid to a flat wood board, horizontally placing the binocular camera on a table with a certain height, fixing the binocular camera, and starting shooting. The positions (upper, lower, left and right) of the wood board are continuously changed in the shooting process, and the checkerboard on the wood board is ensured to be always present in the two lenses, so that the pictures with different angles and different distances are shot by about 100, and the sum of the two lenses is 200. Calibrating the 200 images by using a Stereo Calibration tool box built in Matlab to obtain basic Calibration parameters of the camera: f, fx, fy, b, D, and performing Rectify on the left and right lenses.
Step (2), building a binocular camera: according to the schematic diagram shown in fig. 2, a binocular camera is built on site, the camera is horizontally arranged and faces a floor to be detected, the floor can be ensured to be seen from the left lens and the right lens, a part of ground can be clearly seen from the bottom of an image, the horizontal distance from the camera to a building is at most within 100m, and if the resolution of the binocular camera is high enough, the distance can be properly lengthened. If it has difficulty to set up the camera actually, also can control an unmanned aerial vehicle who has carried on binocular camera and hover and survey the height, in the measurement process, unmanned aerial vehicle should hover still, and the visual angle of shooing is on a parallel with ground as far as possible.
Step (3), calculating a disparity map: in the binocular vision system, a basic schematic thereof is shown in fig. 1. The invention calculates the disparity map by a full map matching algorithm SGM to obtain delta in a formula (2).
Step (4), calculating the height of the floor: after the disparity map is obtained in step (3), a v-disparity map can be obtained by projecting the disparity map in the horizontal direction, as shown in fig. 4 on the right. The horizontal coordinate of the v-disparity value represents the disparity value delta (from small to large), the vertical coordinate represents the image height, wherein the depth of the color represents the number of pixel points, and the deeper the color is, the more pixel points sharing the disparity value are represented.
First, p needs to be calculated and obtained on a v-disparity map1(d△,y1) And p2(d△,y2) As shown in fig. 4, right. lbIs a group of points which are vertical to the x axis on the v-disparity map and have the same or close values, and l is easy to be fitted by least squaresbIs fitted out, < i >bThe upper vertex of (1) is point p1(d△,y1). Since the ground area has smaller left and right image occupation ratio in the system designed by the invention, the invention selects a Randac method more robust to noise points to fitr. Find lbAnd lrIs p2(d△,y2). Finally, p is added1And p2Substituting equation (5) calculates the actual height H of the floor.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting the same, and although the present invention is described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: modifications and equivalents may be made to the embodiments of the invention without departing from the spirit and scope of the invention, which is to be covered by the claims.

Claims (10)

1. A building height measurement method based on binocular vision is characterized by comprising the following steps:
step SS 1: the binocular camera calibration step comprises the following steps: calculating a basic matrix F, an essential matrix E and a distortion parameter D of the camera, and performing epipolar rectification on the binocular camera;
step SS 2: performing field construction of a binocular camera;
step SS 3: a step of calculating a disparity map, which comprises calculating the disparity map based on an SGM algorithm;
step SS 4: calculating a v-disparity map according to the disparity map, and calculating the height of the floor by combining the v-disparity map and calibration information, wherein the method comprises the following steps: by finding the geometric correspondence on the v-disparity map, the position of the building in the image is located, and the vertical height of the building is calculated.
2. The building height measuring method based on binocular vision according to claim 1, wherein the epipolar line correction in the step SS1 specifically comprises: and (3) rotating the imaging plane of the binocular camera by half respectively by adopting a Bouguet epipolar line correction method, so that the error caused by the re-projection of the left and right images is minimum, and the common area of the left and right views is maximum.
3. The binocular vision based building height measuring method according to claim 1, wherein the step SS2 specifically comprises: the binocular camera is built on site, the camera is horizontally placed and faces the floor to be detected, the floor can be ensured to be seen from the left lens and the right lens, and part of the ground can be seen from the bottom of the image.
4. The building height measuring method based on binocular vision according to claim 1, wherein the building height measuring method is characterized in thatStep SS3 specifically includes: calculating a disparity map by an SGM dense matching algorithm, wherein in a calibrated binocular system, P (X, Y, Z) represents any point of a world coordinate system, (u)l,vl) And (u)r,vr) Respectively representing the projection of the point P on the image plane coordinate systems of the two cameras; (u)0,v0) Is the projection of the optical center of the camera onto the image plane; f represents the focal length of the camera; b represents the base length in binocular vision; the formula for calculating the parallax value delta is shown in formula (1),
Figure FDA0003448061710000011
Figure FDA0003448061710000021
any point on the left camera (u)l,vl) Finding the corresponding point on the right camera (u) by the SGM dense algorithmr,vr) Then, the point (u) is calculated by the formula (1)l,vl) The disparity value delta is calculated for each point, and then a disparity map is formed.
5. The binocular vision based building height measuring method according to claim 4, wherein the step SS4 specifically comprises:
step SS 41: projecting the disparity map according to the horizontal direction to obtain a v-disparity map; the horizontal coordinate of the v-disparity value represents a disparity value delta, and the vertical coordinate represents an image height;
step SS 42: calculating and obtaining the lowest point p of the building on the v-disparity map1(d△,y1) And highest point p of building2(d△,y2) Because the building to be height-measured is vertical to the ground, the building forms a vertical line segment l on the v-disparity mapb(ii) a While the road is horizontal, a slope is reflected on the v-disparity map<Inclined line l of 0r，lbIs a segment of a v-disparity mapA group of points which are perpendicular to the x axis and have the same or close values of delta, and l is obtained by a least square fitting modebFitting out; l obtained by fittingbIs the point p1(d△,y1) (ii) a Choosing a Rannac method fitting that is robust to noise points lr(ii) a Finally, find lbAnd lrThe intersection of (A) is p2(d△,y2) Then calculate | y2-y1|，|y2-y1I is the height difference of the coordinate system of the floor on the image plane;
step SS 43: combining the formula (1) and the formula (2) to obtain a formula (3), calculating a conversion equation of point coordinates P (X, Y, Z) of a world coordinate system and points (u, v) in the image,
Figure FDA0003448061710000022
where dx, dy, and f are parameters related to the focal length of the camera, and (u)0,v0) Representing the origin of coordinates of the phase plane; (u, v) are building coordinates captured on the image, and (X, Y, Z,1) are their corresponding physical world coordinates; the leftmost (u, v,1) represents the coordinates of a point on the image plane, 1 is a homogeneous representation, R represents the rotation matrix of the camera, and t represents the translation matrix;
Figure FDA0003448061710000031
substituting Z into X and Y to obtain new expression modes of X and Y, see formula (4),
Figure FDA0003448061710000032
f is the focal length of the camera, dx is the actual size of each pixel on the horizontal axis, dy is the actual size of each pixel on the vertical axis, and the three parameters are obtained through the calibration of the SS1 camera; will point p1And p2Substituting into formula (4) to obtain the final floorThe calculation formula (5) of the height,
Figure FDA0003448061710000033
where b represents the base length of the left and right cameras, which is a fixed value in m, the actual height H of the target is calculated by equation (5).
6. A building height measurement system based on binocular vision is characterized by comprising:
the binocular camera calibration module specifically executes: calculating a basic matrix F, an essential matrix E and a distortion parameter D of the camera, and performing epipolar correction on the binocular camera;
the camera building module specifically executes: building a binocular camera on site;
the disparity map generation module specifically executes: calculating a disparity map based on an SGM algorithm;
the height calculation module specifically executes: calculating a v-disparity map according to the disparity map, and calculating the height of the floor by combining calibration parameters, wherein the method comprises the following steps: by finding the geometric correspondence on the v-disparity map, the position of the building in the image is located, and the vertical height of the building is calculated.
7. The binocular vision based building height finding system of claim 6, wherein the epipolar rectification in the binocular camera calibration module specifically comprises: and (3) rotating the imaging plane of the binocular camera by half respectively by adopting a Bouguet epipolar line correction method, so that the error caused by the re-projection of the left and right images is minimum, and the common area of the left and right views is maximum.
8. The binocular vision based building height finding system of claim 6, wherein the camera building module specifically performs: the binocular camera is built on site, the camera is horizontally placed and faces the floor to be detected, the floor can be ensured to be seen from the left lens and the right lens, and part of the ground can be seen from the bottom of the image.
9. The binocular vision based building height measurement system of claim 6, wherein the disparity map generation module specifically performs: the disparity map is generated by a dense matching algorithm such as SGM.
10. The binocular vision based building height finding system of claim 6, wherein the height calculation module specifically performs:
projecting the disparity map in the horizontal direction to obtain a v-disparity map; the horizontal coordinate of the v-disparity value represents a disparity value delta, and the vertical coordinate represents the image height; calculating and obtaining the lowest point p of the building on the v-disparity map1(d△,y1) And highest point p of building2(d△,y2) (ii) a The building forms a vertical line segment l on the v-disparity mapb(ii) a While the road is horizontal, a slope is reflected on the v-disparity map<Inclined line l of 0r；lbIs a group of points which are perpendicular to an x axis on a v-disparity map and have the same or close values of delta, and l is obtained by a least square fitting modebFitting out; l obtained by fittingbThe upper vertex of (1) is point p1(d△,y1) (ii) a Selecting a Ransac method fitting l that is more robust to noise pointsr(ii) a Finally, find lbAnd lrHas a point of intersection of p2(d△,y2) Then calculate | y2-y1|；
The actual height H of the floor is calculated using equation (6),
Figure FDA0003448061710000041
where, fy is f/dy, f is the focal length of the camera, dy is the actual size of each pixel on the vertical axis, b represents the base length of the left and right cameras, and the unit is m.
CN202111664762.8A 2021-12-30 2021-12-30 Building height measurement method and system based on binocular vision Pending CN114663486A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111664762.8A CN114663486A (en) 2021-12-30 2021-12-30 Building height measurement method and system based on binocular vision

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111664762.8A CN114663486A (en) 2021-12-30 2021-12-30 Building height measurement method and system based on binocular vision

Publications (1)

Publication Number Publication Date
CN114663486A true CN114663486A (en) 2022-06-24

Family

ID=82026133

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111664762.8A Pending CN114663486A (en) 2021-12-30 2021-12-30 Building height measurement method and system based on binocular vision

Country Status (1)

Country Link
CN (1) CN114663486A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116977600A (en) * 2023-07-03 2023-10-31 上海鱼微阿科技有限公司 XR equipment and XR equipment height acquisition method

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116977600A (en) * 2023-07-03 2023-10-31 上海鱼微阿科技有限公司 XR equipment and XR equipment height acquisition method
CN116977600B (en) * 2023-07-03 2024-04-09 玩出梦想(上海)科技有限公司 XR equipment and XR equipment height acquisition method

Similar Documents

Publication Publication Date Title
CN105424006B (en) Unmanned plane hovering accuracy measurement method based on binocular vision
CN104034263B (en) A kind of non-contact measurement method of forging&#39;s block dimension
CN105115560B (en) A kind of non-contact measurement method of cabin volume of compartment
CN109855603B (en) Focus measurement method and terminal
CN110208771B (en) Point cloud intensity correction method of mobile two-dimensional laser radar
JP2004127239A (en) Method and system for calibrating multiple cameras using calibration object
JP2004163292A (en) Survey system and electronic storage medium
CN112254670B (en) 3D information acquisition equipment based on optical scanning and intelligent vision integration
WO2022126339A1 (en) Method for monitoring deformation of civil structure, and related device
CN106840011A (en) Steel tower deformation measuring device and its method
CN111307046B (en) Tree height measuring method based on hemispherical image
CN111879354A (en) Unmanned aerial vehicle measurement system that becomes more meticulous
CN112254675B (en) Space occupancy rate acquisition and judgment equipment and method containing moving object
CN113340272B (en) Ground target real-time positioning method based on micro-group of unmanned aerial vehicle
Bergström et al. Automatic in-line inspection of shape based on photogrammetry
CN114663486A (en) Building height measurement method and system based on binocular vision
CN112082486B (en) Handheld intelligent 3D information acquisition equipment
CN112254638B (en) Intelligent visual 3D information acquisition equipment that every single move was adjusted
Wu Photogrammetry: 3-D from imagery
CN112254676A (en) Portable intelligent 3D information acquisition equipment
CN109712200B (en) Binocular positioning method and system based on least square principle and side length reckoning
CN112254678B (en) Indoor 3D information acquisition equipment and method
CN112254669B (en) Intelligent visual 3D information acquisition equipment of many bias angles
CN106482665B (en) One kind combination point group high-precision three-dimensional information vision measuring method
Xue et al. An improved method of depth measurement based on line laser

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination