CN117456092A - Three-dimensional live-action modeling system and method based on unmanned aerial vehicle aerial survey - Google Patents

Abstract

The invention discloses a three-dimensional real-scene modeling method based on UAV aerial survey, which includes: setting processing stations and base points on the ground and rationally arranging multiple groups of image control points, generating a strip aerial survey area, generating coordinate maps based on the image control points, and obtaining Navigate the route; control the drone to go to the image control point for oblique photography, obtain image data with image control points, preprocess the acquired image data through the image control point recognition and classification model, and screen out waste films; each acquisition The image data is input into the image quality model for accuracy evaluation. If the accuracy does not meet the preset value, the image is reacquired until the real scene of all image control points is updated, and a real scene model diagram is formed after modeling. The invention belongs to the technical field of UAV aerial survey. Specifically, it provides a three-dimensional real scene modeling system and method based on UAV aerial survey, which is used to solve the problem of insufficient aerial survey quality accuracy and poor measurement accuracy in the existing technology. , the problem of heavy workload.

Description

Three-dimensional reality modeling system and method based on UAV aerial survey

Technical field

The invention belongs to the technical field of UAV aerial survey, and specifically refers to a three-dimensional real scene modeling system and method based on UAV aerial survey.

Background technique

In the field of construction surveying, with the development of drone technology, the advantages of drone aerial survey are gradually growing and occupying a dominant position in the market. In the existing surveying and mapping modeling technology, surveyors first need to remotely control the drone to shoot the real scene, and then return with the data after the drone shooting is completed, and then obtain the survey model map, and then use modeling and other methods. When conducting aerial survey by drone, a rectangular or irregular polygon area is usually generated first, and then the route is planned by adjusting the control points on the area. However, in conventional operations, it is necessary to adjust the control points multiple times to generate a strip-shaped aerial photography area. The operation is extremely inconvenient, and the drawn aerial photography area may cover non-measurement areas, which increases redundant data and data processing workload.

The existing Chinese invention patent with application number 2019110557828 discloses a three-dimensional reality modeling method based on drone aerial survey. The three-dimensional real-scene modeling method based on UAV aerial survey includes the following steps: using UAV aerial survey to obtain aerial survey data, performing image analysis through aerial triangulation analysis, and converting a series of two-dimensional aerial images into three-dimensional dense images of the construction project to be measured. point cloud, and then perform data post-processing to obtain a digital line map and a digital surface model of the construction project to be measured, and obtain a real-scene three-dimensional model; based on the real-scene three-dimensional model and the real surface point cloud, the real-life scene of the construction project to be measured is obtained Inspect and obtain the construction execution data of the construction project to be tested; based on the comparison of the three-dimensional planning and design of the construction project to be tested and the construction execution data, study and issue construction scheduling instructions, and check and correct the execution effect of the scheduling instructions. This three-dimensional real-scene modeling method based on UAV aerial survey is highly efficient and low-cost.

In the above scheme, the accuracy problem caused by manual measurement can be solved by aerial triangulation. However, there are problems with the flight quality and scanning quality obtained by aerial triangulation. If the film base itself has certain systematic deformation, or During aerial photography, photographic processing, and scanning processes, certain strain forces may occur, causing dynamic geometric deformation, resulting in poor measurement accuracy and increased workload.

Contents of the invention

In view of the above situation, in order to overcome the shortcomings of the existing technology, the present invention provides a three-dimensional real-scene modeling system and method based on UAV aerial survey, which is used to solve the problem of insufficient aerial survey quality accuracy and measurement accuracy in the prior art. Not good, a problem with heavy workload.

The technical solutions adopted by the present invention are as follows:

This solution discloses a three-dimensional reality modeling method based on UAV aerial survey, including the following steps:

S1: Set up processing stations and base points on the ground and reasonably arrange multiple groups of image control points. The processing station pre-sets the aerial photography parameters of the UAV, generates a strip aerial survey area, and inputs the position information of the image control points to the processing station to form a coordinate map. , obtain the navigation route between the current position and the image control point;

S2: The processing station controls the drone to move from the reference coordinate control point to the corresponding image control point, uses the drone to perform oblique photography, and obtains image data with image control points. It preprocesses the acquired image data and screens out waste. piece;

S3: During the photography process, input the image data obtained each time into the image quality model for accuracy evaluation; if the accuracy meets the preset value, continue to obtain the image data of the next image control point; if the accuracy does not meet the preset value, then Re-acquire the image;

S4: Use the reacquired image that meets the accuracy to cover the original blurred scene to update the real scene, and form the final real scene model diagram based on the collected image modeling.

In a further solution, in the step S1, the image control points are laid out using the measurement method of the graph root point and the axially symmetric layout method, so as to increase the layout density of the image control points at the complex boundary of the aerial survey area.

Further, in step S1, the aerial photography parameter settings include: setting of heading overlap, side overlap, aerial survey height and reference point height, and a strip aerial survey area is generated based on the aerial photography parameters and UAV lens parameters.

In a further solution, the image quality model process includes the following steps:

Step 1: Select a target point of the captured image for example analysis;

Step 2: Select different layouts of course overlap rate, side overlap rate, aerial survey height and reference point height, and count the elevation error and horizontal error of each instance;

Step 3: Calculate the accuracy of various situations through the fuzzy comprehensive evaluation model and conduct comparative analysis of the elevation error and horizontal error of each instance.

Further, in the step S2, an image control point recognition and classification model is used to identify the image control point identifiers in the image data and perform image preprocessing.

Further, the construction and training process of the image control point recognition and classification model includes the following steps:

Step 1: Build a number of image control point markers, take the original image of the image control point marker, and preprocess the image, including image cropping, size adjustment, grayscale conversion, and normalization to ensure the consistency and usability of the image data;

Step 2: Use the convolutional neural network in the deep learning model to automatically learn features and convert the image into a feature expression form that can be processed by the machine learning algorithm; use the encoder-decoder neural network based on the VGG11 network and train it.

Step 3: Annotate the preprocessed image data set to form an annotation file, assign the correct classification label to each sample, select an appropriate classification model according to the characteristics of the task, and form a sample set corresponding to the annotation file and the original image;

Step 4: When training the model, first perform image classification on the original image to determine whether the image resembles a control point. If so, perform image segmentation on the image to accurately classify each pixel;

Step 5: Divide the sample set into a training set and a test set, where the training set accounts for 70% of the total samples and the test set accounts for 30% of the total samples; use the test set to evaluate the trained model and calculate the accuracy of the model. Accuracy, the model is tuned based on the evaluation results;

Step 6: Finally, deploy the trained model to the actual application environment to realize the image control point recognition and classification function of the image.

This solution also discloses a three-dimensional real-scene modeling system based on UAV aerial survey, including UAV, processing station and communication base station. The UAV establishes a connection with the communication base station, thereby realizing communication with the ground processing station. communication, the processing station thereby controls the drone;

The drone includes a drone control unit, a data collection unit and a processing station control unit;

The drone control unit includes: a drone processor, an inertial measurement system, a positioning system, a power supply system, a storage system, and a wireless communication system. The above systems are all connected to the drone processor, and the drone processor is used to Receive and process signals for use;

The inertial measurement system is composed of an accelerometer and a gyroscope, and is used to sense the acceleration of the drone and obtain related data such as the speed and attitude of the drone through integral operations. Among them, the accelerometer is used to measure the acceleration of the UAV relative to the inertial space during its movement, indicating the local vertical direction; the gyroscope is used to measure the angular displacement of the UAV relative to the direction of rotation of the mounting platform, indicating the direction of the Earth's rotation axis. Direction, through the setting of the inertial measurement system, the attitude of the drone is obtained, which facilitates the control of angle shooting.

The positioning system uses GPS positioning or Beidou navigation positioning to locate the position of the drone. The power supply system uses a lithium-ion power battery, and the storage system uses a memory card combined with an external hard disk to store data. The wireless communication system uses a long-distance WiFi module to achieve communication.

The data collection unit includes: a camera, an infrared range finder and a battery information sensor. The camera is used for taking photos and videos. The infrared range finder is used to measure the relationship between the drone body and base points and image control points. The battery information sensor is used to collect battery power information. During the aerial survey, when the real-time power of the drone battery is less than the power setting threshold, the drone processor sends a signal to the processing station through the wireless communication system. , reminding staff to control the drone to return to avoid causing losses. During use, the power supply system supplies power to the drone. The battery information sensor is used to monitor the battery power information and return to home for charging according to the power. During the flight of the drone, the speed, attitude, and speed of the drone are obtained in real time. and filming footage.

The processing station control unit is communicatively connected to the processing station. The processing station includes a data receiving unit and a monitoring screen. The monitoring screen and data receiving unit are connected to the UAV processor and data collection unit through a wireless communication system. It plays the role of monitoring, control and information processing.

This solution discloses a three-dimensional real-scene modeling method for UAV aerial survey. The beneficial effects achieved by using the above solution are as follows:

1. Before the aerial survey, make pre-test preparations based on the characteristics of the area to be measured. The image control points are laid out using the measurement method of the root point and the axially symmetric layout method. The density of the image control points is increased at the complex boundaries of the aerial survey area. It can reduce the redundant workload contained in the strip aerial survey area and the overlap problem of non-survey areas.

2. Go to the corresponding image control points and perform oblique photography to obtain image data with image control points. During the construction and training process of the image control point recognition and classification model, the images are preprocessed through neural network learning, and Carry out image classification and segment the image by judging whether there are image control points in the image. It is used to realize the classification and recognition of the image control points. While screening out the waste films, it reduces the workload in the later stage and facilitates the classification and post-processing of the image. Real-life modeling.

3. Input the image data obtained according to each image control point into the image quality model for accuracy evaluation. The image quality model uses different heading overlap rates, side overlap degrees, aerial survey heights and reference point heights to perform fuzzy comprehensive evaluation. Through the calculation of the model, the accuracy of various situations is obtained and the elevation error and horizontal error of the embodiment are compared and analyzed to determine the accuracy of the image quality, determine whether the composite image requires standards, and delete redundant images to solve the problem in the existing technology. The problem of heavy workload is solved at the same time, the problem of insufficient quality accuracy of aerial survey in the existing technology, and the problem of poor measurement accuracy.

4. In addition, the system provided by this solution includes real-time monitoring functions. The processing station is equipped with a monitoring screen for real-time monitoring, which plays the role of monitoring, control and information processing. It also includes monitoring of drone battery power. Ensure the normal use of the drone.

Description of the drawings

The drawings are used to provide a further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

Figure 1 is the method flow chart of this modeling method;

Figure 2 shows the composition diagram of this modeling system;

Figure 3 is a flow chart of the construction and training process of the image control point recognition and classification model in the embodiment;

Figure 4 is a flow chart for image quality model evaluation to obtain image accuracy in the embodiment.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, not all of them; based on The embodiments of the present invention and all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

Embodiment, as shown in Figure 2, the present invention provides a three-dimensional real-scene modeling system based on UAV aerial survey, including a UAV, a processing station and a communication base station. The UAV establishes a connection with the communication base station. The connection enables communication with the ground processing station, which controls the drone.

Referring to Figure 2, the drone includes a drone control unit, a data collection unit and a processing station control unit.

The UAV control unit is used to monitor and obtain various states of the UAV to ensure the normal operation of the UAV in aerial survey work; the UAV control unit includes: a UAV processor, an inertial measurement system, and a positioning system , power supply system, storage system, wireless communication system, the above systems are all connected to the drone processor, and the drone processor is used to receive and process signals. In a preferred embodiment, the inertial measurement system is composed of an accelerometer and a gyroscope, and is used to sense the acceleration of the drone and obtain relevant data such as the speed and attitude of the drone through integral operations. Among them, the accelerometer is used to measure the acceleration of the UAV relative to the inertial space during its movement, indicating the local vertical direction; the gyroscope is used to measure the angular displacement of the UAV relative to the direction of rotation of the mounting platform, indicating the direction of the Earth's rotation axis. Direction, through the setting of the inertial measurement system, the attitude of the drone is obtained, which facilitates the control of angle shooting. Among them, the positioning system adopts GPS positioning or Beidou navigation positioning to locate the position of the drone. The power supply system uses a lithium-ion power battery, and the storage system uses a memory card combined with an external hard disk to store data. The wireless communication system uses a long-distance WiFi module to achieve communication.

The data collection unit includes: a camera, an infrared range finder and a battery information sensor. The camera is used for taking photos and videos. The infrared range finder is used to measure the relationship between the drone body and base points and image control points. The battery information sensor is used to collect battery power information. During the aerial survey, when the real-time power of the drone battery is less than the power setting threshold, the drone processor sends a signal to the processing station through the wireless communication system. , reminding staff to control the drone to return to avoid causing losses. During use, the power supply system supplies power to the drone. The battery information sensor is used to monitor the battery power information and return to home for charging according to the power. During the flight of the drone, the speed, attitude, and speed of the drone are obtained in real time. and filming footage.

The processing station control unit: communicates with the processing station. The processing station includes a data receiving unit and a monitoring screen. The monitoring screen and data receiving unit are connected to the UAV processor and data collection unit through a wireless communication system. , plays the role of monitoring, control and information processing.

Referring to Figures 1, 3 and 4, on the basis of the above system, this solution also discloses a three-dimensional reality modeling method based on UAV aerial survey, which includes the following steps:

S1: Set up processing stations and base points on the ground and arrange multiple groups of image control points reasonably. The image control points are laid out using the measurement method of the root point and the axially symmetric layout method. The density of the image control points is increased at the complex boundary of the aerial survey area. The processing station pre-sets the aerial photography parameters of the UAV, generates a strip aerial survey area, inputs the position information of the image control point to the processing station to form a coordinate map, and obtains the navigation route between the current position and the image control point;

The aerial photography parameter settings include: heading overlap, side overlap, aerial survey height and reference point height settings, and a strip aerial survey area is generated based on the aerial photography parameters and UAV lens parameters.

S2: The processing station controls the drone to move from the reference coordinate control point to the corresponding image control point, uses the drone to perform oblique photography, and obtains image data with image control points, and uses the image control point recognition and classification model to classify the image data The image control point identification in the image is identified and image pre-processed, and waste films are screened out;

Wherein, the construction and training process of the image control point recognition and classification model includes the following steps:

S2.1: Build a number of image control point markers, capture the original images of the image control point markers, and preprocess the images, including image cropping, size adjustment, grayscale conversion, and normalization to ensure the consistency and availability of image data. ;

S2.2: Use the convolutional neural network in the deep learning model to automatically learn features and convert the image into a feature expression form that can be processed by the machine learning algorithm; use the encoder-decoder neural network based on the VGG11 network and train it;

S2.3: Annotate the preprocessed image data set to form an annotation file, assign the correct classification label to each sample, select an appropriate classification model according to the characteristics of the task, and form a sample set corresponding to the annotation file and the original image;

S2.4: When training the model, first perform image classification on the original image to determine whether the image resembles a control point. If so, perform image segmentation on the image to accurately classify each pixel;

S2.5: Divide the sample set into a training set and a test set, where the training set accounts for 70% of the total samples and the test set accounts for 30% of the total samples; use the test set to evaluate the trained model and calculate the accuracy of the model , accuracy, and tune the model based on the evaluation results;

S2.6: Finally, deploy the trained model to the actual application environment to realize the image control point recognition and classification function of the image.

Go to the corresponding image control point and perform tilt photography to obtain image data with image control points. During the construction and training process of the image control point recognition and classification model, the image is preprocessed through neural network learning, and the image is Classification, by judging whether there are image control points in the image, thereby segmenting the image, which is used to realize the classification and recognition of the object control points, while filtering out waste films, reducing the workload in the later stage, and facilitating the classification of the image and the real scene in the later stage. Modeling.

S3: During the photography process, input the image data acquired each time into the image quality model for accuracy evaluation; if the accuracy meets the preset value, continue to acquire the image data of the next image control point until all image control point images are collected. data; if the accuracy does not meet the preset value, reacquire the image;

Among them, the evaluation process of the image quality model includes the following steps:

S3.1: Select a target point of the captured image for instance analysis;

S3.2: Select different layouts of course overlap rate, side overlap rate, aerial survey height and reference point height, and count the elevation error and horizontal error of each instance;

S3.3: Calculate the accuracy of various situations through the fuzzy comprehensive evaluation model and conduct comparative analysis of the elevation error and horizontal error of each instance.

S4: Use the reacquired image that meets the accuracy to cover the original blurred scene to update the real scene, and form the final real scene model diagram based on the collected image modeling.

According to the image data obtained by each image control point, the image quality model is input into the image quality model for accuracy evaluation. The image quality model uses different heading overlap rates, side overlap degrees, aerial survey heights and reference point heights to calculate the fuzzy comprehensive evaluation model. The accuracy of various situations and the elevation error and horizontal error of the embodiment are compared and analyzed to determine the accuracy of the image quality and determine whether the composite image requires standards, thereby deleting redundant images and solving the heavy workload in the existing technology. It also solves the problems of insufficient aerial survey quality accuracy and poor measurement accuracy in the existing technology.

It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that these entities or operations are mutually exclusive. any such actual relationship or sequence exists between them. Furthermore, the terms "comprises," "comprises," or any other variations thereof are intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus that includes a list of elements includes not only those elements, but also those not expressly listed other elements, or elements inherent to the process, method, article or equipment.

Claims (7)

1. A three-dimensional reality modeling method based on UAV aerial survey, which is characterized by including the following steps: S1: Set up processing stations and base points on the ground and reasonably arrange multiple groups of image control points. The processing station pre-sets the aerial photography parameters of the UAV, generates a strip aerial survey area, and inputs the position information of the image control points to the processing station to form a coordinate map. , obtain the navigation route between the current position and the image control point; S2: The processing station controls the drone to move from the reference coordinate control point to the corresponding image control point, uses the drone to perform oblique photography, and obtains image data with image control points. It preprocesses the acquired image data and screens out waste. piece; S3: During the photography process, input the image data acquired each time into the image quality model for accuracy evaluation; if the accuracy meets the preset value, continue to acquire the image data of the next image control point until all image control point images are collected. data; if the accuracy does not meet the preset value, reacquire the image; S4: Use the reacquired image that meets the accuracy to cover the original blurred scene to update the real scene, and form the final real scene model diagram based on the collected image modeling. 2. A three-dimensional real scene modeling method based on UAV aerial survey according to claim 1, characterized in that: the aerial photography parameter setting in step S1 includes: heading overlap degree, side overlap degree, aerial survey height and benchmark. Set the point height to generate a strip aerial survey area based on the aerial photography parameters and UAV lens parameters. 3. A three-dimensional real scene modeling method based on UAV aerial survey according to claim 2, characterized in that: in step 2, an image control point recognition and classification model is used to identify the image control points in the image data. Perform recognition and image preprocessing work. 4. A three-dimensional reality modeling method based on UAV aerial survey according to claim 3, characterized in that: the construction and training process of the image control point recognition and classification model includes the following steps: Step 1: Build a number of image control point markers, take the original image of the image control point marker, and preprocess the image, including image cropping, size adjustment, grayscale conversion, and normalization to ensure the consistency and usability of the image data; Step 2: Use the convolutional neural network in the deep learning model to automatically learn features and convert the image into a feature expression form that can be processed by the machine learning algorithm; use the encoder-decoder neural network based on the VGG11 network and train it; Step 3: Annotate the preprocessed image data set to form an annotation file, assign the correct classification label to each sample, select an appropriate classification model according to the characteristics of the task, and form a sample set corresponding to the annotation file and the original image; Step 4: When training the model, first perform image classification on the original image to determine whether the image resembles a control point. If so, perform image segmentation on the image to accurately classify each pixel; Step 5: Divide the sample set into a training set and a test set, where the training set accounts for 70% of the total samples and the test set accounts for 30% of the total samples; use the test set to evaluate the trained model and calculate the accuracy of the model. Accuracy, the model is tuned based on the evaluation results; Step 6: Finally, deploy the trained model to the actual application environment to realize the image control point recognition and classification function of the image. 5. A three-dimensional reality modeling method based on UAV aerial survey according to claim 4, characterized in that: the evaluation process of the image quality model in step S3 includes the following steps: Step 1: Select a target point of the captured image for example analysis; Step 2: Select different layouts of course overlap rate, side overlap rate, aerial survey height and reference point height, and count the elevation error and horizontal error of each instance; Step 3: Calculate the accuracy of various situations through the fuzzy comprehensive evaluation model and conduct comparative analysis of the elevation error and horizontal error of each instance. 6. A three-dimensional real scene modeling method based on UAV aerial survey according to claim 1, characterized in that: the layout of the image control points in step S1 adopts the measurement method of the root point and the axially symmetric layout method. , increasing the layout density of image control points at the complex boundaries of the aerial survey area. 7. A modeling system based on a three-dimensional real-scene modeling method based on UAV aerial survey according to any one of claims 1 to 6, characterized in that it includes a UAV, a processing station and a communication base station, and the UAV The drone establishes a connection with the communication base station to achieve communication with the ground processing station, and the processing station controls the drone; The drone includes a drone control unit, a data collection unit and a processing station control unit; The drone control unit includes: a drone processor, an inertial measurement system, a positioning system, a power supply system, a storage system, and a wireless communication system. The above systems are all connected to the drone processor; The UAV processor is used to receive and process signals; The inertial measurement system is composed of an accelerometer and a gyroscope, and is used to sense the acceleration of the drone and obtain data related to the speed and attitude of the drone through integral operations. The accelerometer is used to measure the movement of the drone. The acceleration relative to the inertial space indicates the local vertical direction; the gyroscope is used to measure the angular displacement of the drone relative to the direction of rotation of the mounting platform, indicating the direction of the earth's rotation axis. The drone is obtained through the settings of the inertial measurement system. Attitude, easy to control angle shooting; The positioning system uses GPS positioning or Beidou navigation positioning to locate the position of the drone; The power supply system uses lithium-ion power batteries; The storage system uses a combination of a memory card and an external hard disk to store data; The wireless communication system uses a long-distance WiFi module to achieve communication; The data collection unit includes: a camera, an infrared range finder and a battery information sensor. The camera is used for taking photos and videos. The infrared range finder is used to measure the relationship between the drone body and base points and image control points. The battery information sensor is used to collect battery power information. During the aerial survey, when the real-time power of the drone battery is less than the power setting threshold, the drone processor sends a signal to the processing station through the wireless communication system. , remind staff; The processing station control unit is communicatively connected to the processing station. The processing station includes a data receiving unit and a monitoring screen. The monitoring screen and data receiving unit are connected to the UAV processor and data collection unit through a wireless communication system. It plays the role of monitoring, control and information processing.