CN111340942A - Three-dimensional reconstruction system based on unmanned aerial vehicle and method thereof - Google Patents

Abstract

The invention relates to a three-dimensional reconstruction system based on an unmanned aerial vehicle and a method thereof, comprising a workbench, a transmission device is arranged in the middle of the upper surface of the workbench, a plurality of positioning vehicles are arranged on the upper part of the transmission device, and two positioning vehicles are arranged in each positioning vehicle. Each loading position, the worktable is located on the side of the conveying device and is provided with mirror ring feeding and bending mechanism, head bending and welding mechanism, nose bridge bending and welding mechanism, reinforcement rod grinding and welding mechanism, deburring mechanism from left to right. Mechanism, nose pad welding mechanism, temple marking and welding mechanism, rotary riveting and pressing mechanism, leg cover assembling mechanism, temple hot pressing and bending mechanism and blanking manipulator; the present invention transmits the mirror ring to each mechanism through the transmission device. Corresponding processing operations, and then assemble and weld the parts together according to the set process; fully mechanized operations replace manual loading and unloading, improve the quality and efficiency of processing and assembly, improve the yield of sunglasses, and reduce labor costs; other types of glasses can be assembled The frame body has good market application value.

Description

A three-dimensional reconstruction system and method based on UAV

technical field

The invention belongs to the technical field of three-dimensional reconstruction, and in particular relates to a three-dimensional reconstruction system and a three-dimensional reconstruction method based on an unmanned aerial vehicle.

Background technique

The method for acquiring an image for three-dimensional reconstruction includes: controlling the periodic change of the brightness of each of the at least two light sources that are spatially separated, and using three cameras in at least three positions to separately acquire images for three-dimensional reconstruction. image. Image-based three-dimensional reconstruction is to automatically calculate and match two or more two-dimensional images captured by an object or scene, calculate the two-dimensional geometric information and depth information of the object or scene, and establish a three-dimensional The process of the diorama. However, there are also some shortcomings. First of all, when the real scene cannot be obtained when the real perception image is to be reconstructed, for example, the object or scene does not exist at all, it is fictitious, or the scene is in the setting

When the planning stage is changing at all times, image-based modeling techniques cannot be used. Secondly, because the objects in the scene have become two-dimensional objects in the image, it is difficult for users to interact with these two-dimensional graphic objects to obtain the required information; in addition, there are certain requirements for cameras and photographic equipment, which are The need to obtain a true perceptual image. At the same time, these large number of image files also need enough storage space to save. In the photovoltaic industry, pipelines are often damaged, but large-scale renovations are not carried out immediately. The scene at that time is often recorded first, and the positioning can be carried out when subsequent rectifications are made. 3D reconstruction is widely used in such practical scenarios. The 3D model of the overall environment is reconstructed every once in a while, and changes will occur each time, and differences can be found in the later comparison and maintenance. Traditional 3D reconstruction relies on human hand-held lasers to scan the environment to be reconstructed, which is not very reliable, and only point cloud information can be obtained by simple lidar. With the vision sensor, information including shape, texture, color, etc. is obtained.

With the development of UAV technology, the price of consumer UAVs continues to drop, and the miniaturization and portability of lidars make it possible for small and medium-sized UAVs to carry lidars for geographic mapping. At present, UAV Lidar surveying and mapping generally uses the method of ground target positioning and ground base station cooperation, but it is not adaptable to different landforms. It needs to be artificially explored in advance to set up targets around the surveying and mapping site, which is cumbersome and inefficient. The point cloud data obtained at the same time needs to be processed offline at the base station, and the real-time performance is not good enough.

In recent years, with the development of computer vision and the enhancement of graphics computing capabilities, there has also been a technique for 3D modeling of urban terrain by relying on sequential images to recover the structure from motion. However, the accuracy of modeling by taking pictures is not high enough. , the model is prone to voids and distortions, and the adjustment after modeling relies on human participation, manual editing and optimization. As a result, the model is not accurate enough, and it is easy to mix human subjective factors. The 3D map function of maps such as Baidu Maps also relies on manual modeling, and the proportion of automated modeling is low.

Therefore, the prior art has shortcomings and needs to be improved.

SUMMARY OF THE INVENTION

The present invention provides a 3D reconstruction method based on a 3D laser and a camera, in order to solve the above problems, which integrates laser data and visual data to avoid the occurrence of fictitious scenes when a single camera exists. solve the above problem.

In order to solve the above-mentioned shortcomings of the prior art, the present invention provides a three-dimensional reconstruction method of urban terrain, in order to scan a target area quickly and at low cost, so as to realize automatic real-time three-dimensional reconstruction of urban terrain, and to ensure high In order to solve the above problems, the technical solutions provided by the present invention are as follows:

A 3D reconstruction system based on

Step S1, calibrate the three-dimensional reconstruction coordinate system; obtain the point cloud data of the first frame of three-dimensional measurement data of the measured object, which is called global point cloud data, and the coordinate system based on the first frame of three-dimensional measurement data is called global coordinates Step S2: 3D measurement is performed on the surface of a certain local area of the object to be measured, and 3D reconstruction is performed using the principle of binocular stereo vision to obtain the point cloud data of the local area, which is called local point cloud data, wherein the There is an overlapping area between the local area and the area corresponding to the global point cloud data; step S3, transform the local point cloud data into the global coordinate system, and according to the overlapping area, compare the local point cloud data with the The global point cloud data is registered, and the global point cloud data is updated; step S4, keeping the measured object still, changing the measurement perspective to measure the measured object, and repeating the steps S2 to S3 , until the measurement of the measured object is completed; step S5, performing global optimization processing on the global point cloud data updated after the measurement is completed, to obtain a point cloud model; the step S2 specifically includes the following steps: step S21, on the Three-dimensional measurement is performed on the surface of the local area to obtain a first speckle image and a second speckle image of the local area; step S22, determining each pixel in the first speckle image in the second speckle image In step S23, according to the integer pixel level corresponding points and the coordinates of each pixel point in the first speckle image, perform sub-pixel corresponding point search on the second speckle image, and obtain sub-pixel corresponding points in the second speckle image; step S24, using the principle of binocular stereo vision, using Kirsch algorithm to combine the sub-pixel correspondence of the second speckle image to perform three-dimensional reconstruction, to obtain the measured Local point cloud data of the object surface.

The Kirsch algorithm includes:

Step 1: Smooth the original image with a Gaussian filter with a specified standard deviation σ, then compute the local gradient g(x,y) and edge direction α(x,y) at each point;

Step 2: Perform Kirsch calculation on the calculated gradient image. Assuming that the image has H×W pixels, the edge pixels generally do not exceed 5×H. This is a looser limit for images with certain goals. Take the initial threshold T ₀ , and calculate its Kirsch operator for each pixel point i. If K(i)>T ₀ is satisfied, then i is an edge point, and the number of edge points N is increased by 1. Once the number of edge points exceeds 5×H and i is still If the number of pixels is less than the whole image, it means that the threshold is too low, so that many pixels that are not edge points are also taken out. Therefore, it is necessary to increase the threshold. The calculation process will record the minimum K(i) that satisfies the condition of K(i)>T ₀ , record it as K _min , and use this minimum value as the new threshold. The whole process is adjusted as follows:

(1) If K(i)>T, then i is an edge point, record the coordinates of edge point i and the minimum value of K(i) K _min =min[K(i)], and add 1 to N at the same time;

(2) Once N≥5×H, adjust the threshold to the minimum value that satisfies the minimum edge requirement, that is, take T=K _min ;

(3) Compare the previously obtained edge point with the new threshold, take the point greater than the new threshold as the new edge point, recalculate K _min under the new threshold, and record the newN of the new edge point;

(4) assigning the number of new edge points to the count behind N starts from N=newN;

(5) Continue to process the remaining edge points in the first step, and return to (2) if N≥5×H;

(6) If N<5×H, let T ₂ =T, T ₁ =βT ₂ (where 0<β<1, and β is a constant obtained in the experiment);

Step 3: Edge extraction. Use two thresholds T ₁ and T ₂ to threshold the gradient image generated in step 1, where the ridge pixels with a value greater than T ₂ are called strong edge pixels, and the ridge pixels between T ₁ and T ₂ are called weak edge pixels. , the weak edge is included in the output only if the strong and weak edges are connected.

The three-dimensional reconstruction method is:

Step 1. Data acquisition:

Step 1.1. Use the airborne GPS on the quadrotor UAV to obtain the positioning information set in RMC format in real time and send it to the ground base station frame by frame in order for storage, wherein any αth positioning information includes: the αth GPS timestamp RMCα.timestamp, the αth latitude and longitude RMCα.position, the αth heading information RMCα.track;

Step 1.2. Use the airborne lidar on the quadrotor UAV to obtain the urban terrain data set D and send it to the ground base station frame by frame in sequence for storage, where any jth urban terrain data dj includes: the jth point number dj .PointID, the jth spatial coordinate point (xj, yj, zj), the jth adjustment time dj.adjustedtime, the jth azimuth angle dj.Azimuth, the jth distance dj.Distance, the jth reflection intensity dj. Intensity, jth radar channel dj.Laser_id, jth point timestamp dj.timestamp;

Step 2. Data integration:

From the urban terrain data set D and the positioning information set, select each piece of urban terrain data and positioning information that satisfies the formula (1), thereby obtaining n data and forming a data set Pfit:

To the rotated n space coordinate points:

Step 4, point cloud denoising:

Step 4.1, use the threshold method to obtain invalid point cloud data in the n point cloud data PN and clear the invalid point cloud data, thereby obtaining the removed point cloud data set;

Step 4.2, using the KNN algorithm with double constraints of distance and quantity to perform denoising and smoothing processing on the removed point cloud data set to obtain a denoised point cloud data set;

Step 5, point cloud thinning:

Using a point cloud reduction algorithm based on K-means++ clustering to perform thinning processing on the denoised point cloud data set to obtain thinned point cloud data;

Step 6: Visually process the diluted point cloud data to obtain a three-dimensional point cloud model of the urban terrain.

Compared with the prior art, the beneficial effect is that, by adopting the above solution, the multi-view Lanwei reconstruction method and system based on the UAV of the present invention, through the drone carried by the UAV.

The image acquisition device collects a plurality of two-dimensional images of the target building from a plurality of preset orientations of the target building with a plurality of preset viewing angles, and uses an image processor according to the plurality of two-dimensional images collected by the image acquisition device. image, using the preset image processing software to generate the blue-dimensional model of the target building, and realizes the generation of the target building according to the multi-view image of the target building.

The Lanwei model of the target building can comprehensively obtain the surface data of the target building cultural relics. The method is less difficult to implement, the equipment is cheap, the work cycle is short, and the investment is reduced. The established Lanwei reconstruction has high accuracy and good effect.

The invention adopts the time stamp data integration method, which can effectively match GPS data and laser radar data, accelerate the speed of data integration, improve the efficiency of extracting valid data from massive point cloud data, and can perform three-dimensional reconstruction of different types of terrain.

The invention adopts the KNN algorithm with double constraints of distance and quantity, which can automatically remove outlier points and noise points, and can smooth the point cloud to a certain degree, thereby speeding up the drying speed.

Has good market application value.

Description of drawings

In order to more clearly illustrate the embodiments or technical solutions in the prior art, the following briefly introduces the accompanying drawings used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only for invention. In some embodiments, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without any creative effort.

Fig. 1 is a schematic diagram of the present invention;

Detailed ways

In order to facilitate understanding of the present invention, the present invention will be described in more detail below with reference to the accompanying drawings and specific embodiments. Preferred embodiments of the invention are shown in the accompanying drawings. However, the present invention may be embodied in many different forms and is not limited to the embodiments described in this specification. Rather, these embodiments are provided so that a thorough and complete understanding of the present disclosure is provided.

It should be noted that when an element is referred to as being "fixed to" another element, it can be directly on the other element or intervening elements may also be present. When an element is referred to as being "connected" to another element, it can be directly connected to the other element or intervening elements may also be present. The terms "fixed", "integrated", "left", "right" and similar expressions used in this specification are only for the purpose of illustration, and in the drawings, elements with similar structures are marked with the same reference numerals.

Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the technical field of the present invention. The terms used in the description of the present invention in this specification are only for the purpose of describing specific embodiments, and are not used to limit the present invention.

As shown in Figure 1, a 3D reconstruction system based on

Step S1, calibrate the three-dimensional reconstruction coordinate system; obtain the point cloud data of the first frame of three-dimensional measurement data of the measured object, which is called global point cloud data, and the coordinate system based on the first frame of three-dimensional measurement data is called global coordinates Step S2: 3D measurement is performed on the surface of a certain local area of the object to be measured, and 3D reconstruction is performed using the principle of binocular stereo vision to obtain the point cloud data of the local area, which is called local point cloud data, wherein the There is an overlapping area between the local area and the area corresponding to the global point cloud data; step S3, transform the local point cloud data into the global coordinate system, and according to the overlapping area, compare the local point cloud data with the The global point cloud data is registered, and the global point cloud data is updated; step S4, keeping the measured object still, changing the measurement perspective to measure the measured object, and repeating the steps S2 to S3 , until the measurement of the measured object is completed; step S5, performing global optimization processing on the global point cloud data updated after the measurement is completed, to obtain a point cloud model; the step S2 specifically includes the following steps: step S21, on the Three-dimensional measurement is performed on the surface of the local area to obtain a first speckle image and a second speckle image of the local area; step S22, determining each pixel in the first speckle image in the second speckle image In step S23, according to the integer pixel level corresponding points and the coordinates of each pixel point in the first speckle image, perform sub-pixel corresponding point search on the second speckle image, and obtain sub-pixel corresponding points in the second speckle image; step S24, using the principle of binocular stereo vision, using Kirsch algorithm to combine the sub-pixel correspondence of the second speckle image to perform three-dimensional reconstruction, to obtain the measured Local point cloud data of the object surface.

The Kirsch algorithm includes:

Step 1: Smooth the original image with a Gaussian filter with a specified standard deviation σ, then compute the local gradient g(x,y) and edge direction α(x,y) at each point;

Step 2: Perform Kirsch calculation on the calculated gradient image. Assuming that the image has H×W pixels, the edge pixels generally do not exceed 5×H. This is a looser limit for images with certain goals. Take the initial threshold T ₀ , and calculate its Kirsch operator for each pixel point i. If K(i)>T ₀ is satisfied, then i is an edge point, and the number of edge points N is increased by 1. Once the number of edge points exceeds 5×H and i is still If the number of pixels is less than the whole image, it means that the threshold is too low, so that many pixels that are not edge points are also taken out. Therefore, it is necessary to increase the threshold. The calculation process will record the minimum K(i) that satisfies the condition of K(i)>T ₀ , record it as K _min , and use this minimum value as the new threshold. The whole process is adjusted as follows:

(1) If K(i)>T, then i is an edge point, record the coordinates of edge point i and the minimum value of K(i) K _min =min[K(i)], and add 1 to N at the same time;

(2) Once N≥5×H, adjust the threshold to the minimum value that satisfies the minimum edge requirement, that is, take T=K _min ;

(3) Compare the previously obtained edge point with the new threshold, take the point greater than the new threshold as the new edge point, recalculate K _min under the new threshold, and record the newN of the new edge point;

(4) assigning the number of new edge points to the count behind N starts from N=newN;

(5) Continue to process the remaining edge points in the first step, and return to (2) if N≥5×H;

(6) If N<5×H, let T ₂ =T, T ₁ =βT ₂ (where 0<β<1, and β is a constant obtained in the experiment);

Step 3: Edge extraction. Use two thresholds T ₁ and T ₂ to threshold the gradient image generated in step 1, where the ridge pixels with a value greater than T ₂ are called strong edge pixels, and the ridge pixels between T ₁ and T ₂ are called weak edge pixels. , the weak edge is included in the output only if the strong and weak edges are connected.

The three-dimensional reconstruction method is:

Step 1. Data acquisition:

Step 1.1. Use the airborne GPS on the quadrotor UAV to obtain the positioning information set in RMC format in real time and send it to the ground base station frame by frame in order for storage, wherein any αth positioning information includes: the αth GPS timestamp RMCα.timestamp, the αth latitude and longitude RMCα.position, the αth heading information RMCα.track;

Step 1.2. Use the airborne lidar on the quadrotor UAV to obtain the urban terrain data set D and send it to the ground base station frame by frame in sequence for storage, where any jth urban terrain data dj includes: the jth point number dj .PointID, the jth spatial coordinate point (xj, yj, zj), the jth adjustment time dj.adjustedtime, the jth azimuth angle dj.Azimuth, the jth distance dj.Distance, the jth reflection intensity dj. Intensity, jth radar channel dj.Laser_id, jth point timestamp dj.timestamp;

Step 2. Data integration:

From the urban terrain data set D and the positioning information set, select each piece of urban terrain data and positioning information that satisfies the formula (1), thereby obtaining n data and forming a data set Pfit:

To the rotated n space coordinate points:

Step 4, point cloud denoising:

Step 4.1, use the threshold method to obtain invalid point cloud data in the n point cloud data PN and clear the invalid point cloud data, thereby obtaining the removed point cloud data set;

Step 4.2, using the KNN algorithm with double constraints of distance and quantity to perform denoising and smoothing processing on the removed point cloud data set to obtain a denoised point cloud data set;

Step 5, point cloud thinning:

Using a point cloud reduction algorithm based on K-means++ clustering to perform thinning processing on the denoised point cloud data set to obtain thinned point cloud data;

Step 6: Visually process the diluted point cloud data to obtain a three-dimensional point cloud model of the urban terrain.

Compared with the prior art, the beneficial effect is that, by adopting the above solution, the multi-view Lanwei reconstruction method and system based on the UAV of the present invention, through the drone carried by the UAV.

The image acquisition device collects a plurality of two-dimensional images of the target building from a plurality of preset orientations of the target building with a plurality of preset viewing angles, and uses an image processor according to the plurality of two-dimensional images collected by the image acquisition device. image, using the preset image processing software to generate the blue-dimensional model of the target building, and realizes the generation of the target building according to the multi-view image of the target building.

The Lanwei model of the target building can comprehensively obtain the surface data of the target building cultural relics. The method is less difficult to implement, the equipment is cheap, the work cycle is short, and the investment is reduced. The established Lanwei reconstruction has high accuracy and good effect.

The invention adopts the time stamp data integration method, which can effectively match GPS data and laser radar data, accelerate the speed of data integration, improve the efficiency of extracting valid data from massive point cloud data, and can perform three-dimensional reconstruction of different types of terrain.

The invention adopts the KNN algorithm with double constraints of distance and quantity, which can automatically remove outlier points and noise points, and can smooth the point cloud to a certain degree, thereby speeding up the drying speed.

Has good market application value.

It should be noted that the above-mentioned technical features continue to be combined with each other to form various embodiments not listed above, which are all regarded as the scope of the description of the present invention; and, for those of ordinary skill in the art, improvements can be made according to the above descriptions or transformation, and all these improvements and transformations should belong to the protection scope of the appended claims of the present invention.

Claims (3)

1. A three-dimensional reconstruction system based on a three-dimensional reconstruction method,

step S1, calibrating the three-dimensional reconstruction coordinate system; acquiring point cloud data of a first frame of three-dimensional measurement data of a measured object, wherein the point cloud data is called global point cloud data, and a coordinate system based on the first frame of three-dimensional measurement data is called a global coordinate system; step S2, performing three-dimensional measurement on the surface of a certain local area of the object to be measured, and performing three-dimensional reconstruction by using a binocular stereo vision principle to obtain point cloud data of the local area, wherein an overlapping area exists between the local area and an area corresponding to the global point cloud data; step S3, transforming the local point cloud data to the global coordinate system, registering the local point cloud data and the global point cloud data according to the overlapping area, and updating the global point cloud data; step S4, keeping the object to be measured still, changing the angle of view of measurement to measure the object to be measured, and repeating the steps S2 to S3 until the object to be measured is measured; step S5, performing global optimization processing on the global point cloud data updated after the measurement is completed to obtain a point cloud model; the step S2 specifically includes the following steps: step S21, performing three-dimensional measurement on the surface of the local area to obtain a first speckle image and a second speckle image of the local area; step S22, determining a whole pixel level corresponding point of each pixel in the first speckle image in the second speckle image; step S23, according to the whole pixel level corresponding point and the coordinates of each pixel point in the first speckle image, performing sub-pixel corresponding point search on the second speckle image to obtain a sub-pixel corresponding point in the second speckle image; and step S24, performing three-dimensional reconstruction by using a binocular stereoscopic vision principle and combining a Kirsch algorithm with the corresponding relation of the sub-pixels of the second speckle image to obtain local point cloud data of the surface of the measured object.

2. The three-dimensional reconstruction system of claim 1, wherein the Kirsch algorithm comprises:

step 1, smoothing the original image by a Gaussian filter with a specified standard deviation sigma, and then calculating local gradient g (x, y) and edge direction α (x, y) at each point;

step 2, performing Kirsch calculation on the calculated gradient image, assuming that the image has H × W pixel points, the edge pixels of the image generally do not exceed 5 × H, and the image with a certain target has a more relaxed limit value, and taking an initial threshold value T₀Calculating Kirsch operator of each pixel point i, and if K (i) > T is satisfied₀I is the edge point, N is the number of edge points plus 1, and once the number of edge points exceeds 5 × H and i is less than the number of pixels in the whole image, it means that the threshold value is taken too low, so that many pixels which are not edge points are also taken₀Minimum K (i) of the conditions, denoted K_minTaking the minimum value as a new threshold value, the whole process is adjusted according to the following method:

(1) if K (i) > T, i is the edge point, recording the coordinate of the edge point i and the minimum value K of K (i)_min＝min[K(i)]While N is added by 1;

(2) once N ≧ 5 × H, the threshold is adjusted to the minimum that satisfies the lowest edge requirement, i.e., T ═ K_min；

(3) Comparing the edge points with the new threshold, taking the points greater than the new threshold as new edge points, and recalculating K under the new threshold_minRecording newN of the new edge point;

(4) assigning new edge points to the count after N starts with N ═ newN;

(5) continuing to process the rest edge points in the step 1, and returning to the step (2) if N is more than or equal to 5 × H;

(6) if N < 5 × H, let T₂＝T，T₁＝βT₂(wherein 0 < β < 1, β is a constant obtained in the test);

and 3, step 3: and (5) extracting edges. Using two thresholds T₁And T₂Performing threshold processing on the gradient image generated in the step 1, wherein the value is greater than T₂Is called a strong edge pixel, T₁And T₂The ridge pixels in between are called weak edge pixels, and the weak edge is only included in the output when the strong edge and the weak edge are connected.

3. The three-dimensional reconstruction system of claim 1, wherein the three-dimensional reconstruction method is:

step 1, data acquisition:

step 1.1, acquiring a positioning information set in an RMC format by using an onboard GPS on a four-rotor unmanned aerial vehicle in real time, and sending the positioning information set to a ground base station for storage frame by frame in sequence, wherein any α th piece of positioning information comprises a α th GPS time stamp RMC α, a timetag, a α th longitude and latitude RMC α, a position and a α th piece of heading information RMC α, a track;

step 1.2, acquiring an urban terrain data set D by using an airborne laser radar on a quad-rotor unmanned aerial vehicle, and sending the urban terrain data set D to a ground base station for storage frame by frame according to a sequence, wherein any jth urban terrain data dj comprises: a jth point number dj.pointid, a jth spatial coordinate point (xj, yj, zj), a jth adjusting time dj.adjust time, a jth azimuth angle dj.azimuth, a jth distance dj.distance, a jth reflection intensity dj.intensity, a jth radar channel dj.laser _ id, and a jth point timestamp dj.time;

step 2, data integration:

selecting each piece of urban terrain data and positioning information which satisfy the formula (1) from the urban terrain data set D and the positioning information set, thereby obtaining n data and forming a data set Pfail:

to the rotated n spatial coordinate points:

step 4, point cloud denoising:

step 4.1, obtaining invalid point cloud data in the n point cloud data PN by using a threshold method and clearing the invalid point cloud data to zero, thereby obtaining a removed point cloud data set;

step 4.2, denoising and smoothing the removed point cloud data set by using a distance and quantity double-constrained KNN algorithm to obtain a denoised point cloud data set;

step 5, point cloud rarefying:

performing rarefaction treatment on the denoised point cloud data set by using a point cloud reduction algorithm based on K-means + + clustering to obtain rarefied point cloud data;

and 6, performing visualization processing on the diluted point cloud data to obtain a three-dimensional point cloud model of the urban terrain.

Images