CN117115063A - A multi-source data fusion application method - Google Patents

Abstract

The application discloses a multi-source data fusion application method, which comprises the following steps: step one, processing point cloud data to obtain target point cloud data; step two, processing the image to obtain a target image; step three, matching the target point cloud data with the target image to obtain modeling data; step four, model reconstruction: constructing a three-dimensional model and a two-dimensional model of the detected area based on the modeling data; fifthly, model modification: modifying the two-dimensional model and the three-dimensional model; step six, model display: and integrally displaying the two-dimensional space data of the two-dimensional model and the three-dimensional space data of the three-dimensional model. The application has simple structure and reasonable design, adopts a method of rough segmentation and fine adjustment to process point cloud data, adopts a learnable weight to fuse historical images and newly added images, then fuses multi-source point cloud data and multi-source images to obtain modeling data, respectively establishes a two-dimensional model and a three-dimensional model, and integrates two-dimensional space data and three-dimensional space display.

Description

A multi-source data fusion application method

Technical field

The invention belongs to the technical field of geographical information, and specifically relates to a multi-source data fusion application method.

Background technique

In modern society, the type and amount of data collected by people is constantly increasing, which often comes from different sources, including sensors, social media, mobile devices, etc. How to integrate these data for analysis and application has become an important technical challenge.

2D and 3D integration is an emerging technology that can integrate data from different sources into a unified model for better visualization and analysis.

In terms of geographical information collection and display, it is of great significance to achieve multi-source data fusion, comprehensively utilize geographical and geological data, form a comprehensive application platform, intuitively analyze the geographical environment, and manage data scenarios.

Contents of the invention

The technical problem to be solved by this invention is to provide a multi-source data fusion application method in view of the above-mentioned deficiencies in the prior art. It has a simple structure and reasonable design. It can fuse multi-source point cloud data and multi-source images to obtain modeling data. Establish two-dimensional models and three-dimensional models respectively, integrate two-dimensional space data and three-dimensional space display, and two-dimensional space data and three-dimensional space data are located in the same coordinate system.

In order to solve the above technical problems, the technical solution adopted by the present invention is: a multi-source data fusion application method, which is characterized by:

Step 1. Obtain target point cloud data: Input the first point cloud data, second point cloud data and historical point cloud data into the coarse segmentation network, and learn the relationship between pixels and object area features based on the coarse segmentation network to enhance the pixel features. Description, obtain the rough segmentation result; fuse the rough segmentation result with the historical point cloud data to obtain the fused point cloud, input the fused point cloud into the fine-tuning network, and the fine-tuning network outputs accurate segmentation results to obtain the target point cloud data;

Step 2. Obtain the target image: perform feature extraction on the first new image, the second new image and the historical image respectively to obtain the first feature map F ₁ , the second feature map F ₂ and the third feature map F ₃ . The first feature map F ₁ and the second feature map F ₂ are fused to obtain the fused feature map F _R ; based on the multi-layer perception mlp model, the learnable weight is output, and the third feature map F ₃ and the fused feature map F are fused based on the learnable weight _R , get the target image;

Step 3: Match the target point cloud data and the target image to obtain modeling data;

Step 4. Model reconstruction: Based on the modeling data, construct a three-dimensional model and a two-dimensional model of the measured area;

Step 5. Model modification: modify the two-dimensional model and the three-dimensional model;

Step 6. Model display: display the two-dimensional spatial data of the two-dimensional model and the three-dimensional spatial data of the three-dimensional model in an integrated manner.

The above-mentioned multi-source data fusion application method is characterized by: the specific method of step one is:

Step 101. The first aviation equipment collects the first new point cloud data of the measured area, preprocesses the first new point cloud data, corrects the first new point cloud data, and obtains the first point cloud data;

Step 102: The first vehicle-mounted device collects the second newly added point cloud data of the measured area, preprocesses the second newly added point cloud data, corrects the second newly added point cloud data, and obtains the second point cloud data;

Step 103: Obtain historical point cloud data of the measured area;

Step 104. The first point cloud data, the second point cloud data and the historical point cloud data constitute a point cloud data set. The point cloud data set is divided into a training set, a verification set and a test set, and corresponding points are added to the point cloud data of the training set. Tag of;

Step 105: Construct a segmentation network, input the point cloud data of the training set into the segmentation network to obtain the predicted segmentation results, and adjust the network parameters of the segmentation network according to the predicted segmentation results until the training stop conditions are met, and a trained segmentation network is obtained;

Step 106: Input the first point cloud data and the second point cloud data into the segmentation network respectively to obtain a rough segmentation result, fuse the rough segmentation result with the historical point cloud data to obtain a fused point cloud, and slice the fused point cloud;

Step 107. Construct a fine-tuning network: Input the slice data into the fine-tuning network, and the fine-tuning network outputs accurate segmentation results to obtain target point cloud data.

The above-mentioned multi-source data fusion application method is characterized by: the specific method of step two is:

Step 201: The second aviation equipment collects the first new image of the measured area, and performs aerial three-dimensional encryption on the preprocessed first new image to obtain the first new image;

Step 202: The second vehicle-mounted device collects the second new image of the measured area, and performs air triple encryption on the preprocessed second new image to obtain the second new image;

Step 203: Obtain historical images of the measured area;

Step 204: Perform feature extraction on the first newly added image and the second newly added image respectively to obtain the first feature map F ₁ and the second feature map F ₂ , perform feature extraction on the historical image to obtain the third feature map F ₃ ;

Step 205: Use the BCA network to fuse the first feature map F ₁ and the second feature map F ₂ to obtain the fused feature map _FR ;

Step 206: Build a multi-layer perception mlp model;

Step 207: Perform global maximum pooling processing on the fused feature map F _R to obtain the first pooling value, and use the first pooling value to construct the first channel dimension vector C _FR ;

Step 208: Perform global average pooling processing on the third feature map F ₃ to obtain the second pooling value, and use the second pooling value to construct the second channel dimension vector C _F3 ;

Step 209: Input the first channel dimension vector C _FR and the second channel dimension vector C _F3 into the multi-layer perception mlp model, and the mlp model outputs weight ω ₁ ;

Step 2010: Perform weighted fusion of the fused feature map F _R and the third feature map F ₃ based on the formula F=ω ₁ ·FR + (1-ω ₁ )·F ₃ to obtain _the target image.

The above-mentioned multi-source data fusion application method is characterized by: the specific method for matching target point cloud data and target images is:

Step 301. Determine the matching domain: Use the CornerNet model to read the corner points of the m-th target image, form the target frame from the corner points, convert the ground coordinates X _j of each corner point, and connect the ground coordinates of each corner point to form the matching domain. ;

Step 302. Determine the matching line: Divide one side of the target frame into n equal parts to obtain n+1 parallel lines. Randomly select a point on each parallel line and convert to obtain the ground coordinates X _d of each point. The ground coordinates of each point are obtained. The coordinates X _d are connected to form the first matching line;

Step 303. In the target point cloud, find a target point cloud subset corresponding to the matching domain, and determine at least 3 coordinate points corresponding to the ground coordinates X _d in the target point cloud subset to form a second matching line. Calculate The rotation angle θ between the first matching line and the second matching line;

Step 304: Correspond to the rotation angle θ of the target point cloud subset, and add the rotated target point cloud subset to the feature point set of the m-th target image to obtain the complete feature point set of the m-th target image;

Step 305: Repeat steps 301-304 to complete the matching of all target point cloud data and target images.

The above-mentioned multi-source data fusion application method is characterized in that: modifying the two-dimensional model in step five includes: detecting the invalid value area of the two-dimensional model and filling it; repairing the noise area of the two-dimensional model; and clearing the two-dimensional model. Suspended objects; set the number of layers, map multiple colors to multiple layered data, and render the two-dimensional model with layered coloring.

The above-mentioned multi-source data fusion application method is characterized in that: modifying the three-dimensional model in step five includes: detecting invalid value areas of the three-dimensional model and filling them; repairing the noise areas of the three-dimensional model; clearing suspended objects of the three-dimensional model; setting The number of layers corresponds to multiple colors to multiple layer data, and the three-dimensional model is rendered with layered coloring; the three-dimensional model is automatically processed with light and color uniformity.

The above-mentioned multi-source data fusion application method is characterized in that: the segmentation network in step one adopts the V-net network, and at the end of the encoder of the V-net network, a pooling layer, a 1×1 convolution layer and three 3×3 convolutional layers with different dilation rates.

The above-mentioned multi-source data fusion application method is characterized in that: the fine-tuning network in step 106 adopts the U-net network, and the U-net network adopts the weighted sum of the boundary loss function and the binary cross-entropy loss as the overall loss function.

The above-mentioned multi-source data fusion application method is characterized in that: two-dimensional spatial data includes vector data, elevation data, image data, and tilt data; three-dimensional spatial data includes model data and tilt data.

The above-mentioned multi-source data fusion application method is characterized in that: the preprocessing of point cloud data in step one includes fully automatic filtering of point clouds, and the fully automatic filtering of point clouds includes adaptive filtering, flattening filtering, smoothing filtering, and fusion. Filtering, general filtering, lower elevation filtering, profile filtering.

Compared with the prior art, the present invention has the following advantages:

1. The present invention has a simple structure, reasonable design, and is easy to implement, use and operate.

2. When processing point cloud data, the present invention divides it into two stages. In the first stage, a 3D convolutional segmentation network is used to segment the point cloud data. The segmentation result is integrated with the historical point cloud data in the channel dimension and input. to the fine-tuning network, and then use the 2D convolution fine-tuning network to fine-tune the segmentation results, so that the network can make full use of the characteristics of three-dimensional convolution and two-dimensional convolution, thereby improving segmentation accuracy.

3. The present invention uses the BCA network to perform feature fusion on the first newly added image and the second newly added image to obtain a fused feature map, and updates the network weight based on the fused feature map and the pooling value of the historical image, thereby updating the entire multi-layer Perceptual MLP model finally obtains the target image of the same size as the input, and the effect is good.

4. In the present invention, two-dimensional spatial data and three-dimensional spatial data are displayed in an integrated manner.

To sum up, the present invention has a simple structure and reasonable design. It uses a rough segmentation and then fine-tuning method to process point cloud data, uses learnable weights to fuse historical images and new images, and integrates multi-source point cloud data and multi-source images. Obtain modeling data, establish two-dimensional models and three-dimensional models respectively, and integrate two-dimensional space data and three-dimensional space display.

The technical solution of the present invention will be further described in detail below through the accompanying drawings and examples.

Description of drawings

Figure 1 is a flow chart of the method for obtaining target point cloud data in the present invention.

Figure 2 is a flow chart of the method for obtaining target images according to the present invention.

Figure 3 is a flow chart of the method of the present invention.

Detailed ways

The present invention will be described in further detail below in conjunction with the accompanying drawings and embodiments of the present invention.

It should be noted that, as long as there is no conflict, the embodiments and features in the embodiments of this application can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, the singular forms are also intended to include the plural forms unless the context clearly indicates otherwise. Furthermore, it will be understood that when the terms "comprises" and/or "includes" are used in this specification, they indicate There are features, steps, operations, means, components and/or combinations thereof.

It should be noted that the terms "first", "second", etc. in the description and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances so that the embodiments of the application described herein, for example, can be practiced in sequences other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, e.g., a process, method, system, product, or apparatus that encompasses a series of steps or units and need not be limited to those explicitly listed. Those steps or elements may instead include other steps or elements not expressly listed or inherent to the process, method, product or apparatus.

For the convenience of description, spatially relative terms can be used here, such as "on...", "on...", "on the upper surface of...", "above", etc., to describe what is shown in the figure. The spatial relationship between one device or feature and other devices or features. It will be understood that the spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. For example, if a feature in the figure is turned upside down, then one feature described as "above" or "on top of" other features or features would then be oriented "below" or "below" the other features or features. under other devices or structures". Thus, the exemplary term "over" may include both orientations "above" and "below." The device may be otherwise oriented (rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein interpreted accordingly.

As shown in Figure 1-3, this embodiment includes a multi-source data fusion application method, which is characterized by:

Step 1. Obtain target point cloud data: Input the first point cloud data, second point cloud data and historical point cloud data into the coarse segmentation network, and learn the relationship between pixels and object area features based on the coarse segmentation network to enhance the pixel features. Description, get the rough segmentation result; fuse the rough segmentation result with the historical point cloud data to get the fused point cloud, input the fused point cloud into the fine-tuning network, and the fine-tuning network outputs accurate segmentation results to get the target point cloud data.

In a possible embodiment, the specific method of obtaining target point cloud data is:

Step 101. The first aviation equipment collects the first new point cloud data of the measured area, preprocesses the first new point cloud data, corrects the first new point cloud data, and obtains the first point cloud data;

Step 102: The first vehicle-mounted device collects the second newly added point cloud data of the measured area, preprocesses the second newly added point cloud data, corrects the second newly added point cloud data, and obtains the second point cloud data;

Preprocessing of point cloud data includes fully automatic point cloud filtering, which includes adaptive filtering, flattening filtering, smoothing filtering, fusion filtering, general filtering, elevation reduction filtering, and profile filtering.

Step 103: Obtain historical point cloud data of the measured area;

Step 104. The first point cloud data, the second point cloud data and the historical point cloud data constitute a point cloud data set. The point cloud data set is divided into a training set, a verification set and a test set, and corresponding points are added to the point cloud data of the training set. Tag of.

In actual use, methods for preprocessing the first newly added point cloud data and preprocessing the second newly added point cloud data include normalization and denoising using Gaussian filtering.

Normalization is scaling the point cloud data to the same scale range, subtracting the mean and dividing by the standard deviation. Using Gaussian filter to denoise is to use Gaussian filter to denoise point cloud data.

The data set is divided into training set, verification set and test set according to the ratio of 8:1:1.

It should be noted that the labels corresponding to the point cloud data are used to characterize the categories of each coordinate point in it. Example: The corresponding label can be set to three categories: target A, target B, target C, and background that need to be recognized, or two categories: target and background.

Taking building recognition as an example, each coordinate point in the point cloud data can be labeled as a building or background, the building can be identified, and then each coordinate point can be labeled as a building or background.

In a possible embodiment, taking a data set composed of point cloud data obtained once as an example, the data set is divided into a total of 800 images. Among them, 640 pictures are used for model training, 80 pictures are used for test verification, and 80 pictures are used for model testing.

In a possible embodiment, point cloud data can be obtained through lidar. The point cloud data obtained by lidar is transmitted to a remote controller on the ground in real time. The remote controller on the ground then transmits it to a mobile phone, tablet computer, notebook computer, super mobile phone, etc. in real time. On terminal devices such as ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs), point cloud data can be collected and transmitted in real time, and the terminal devices can perform point cloud data segmentation. There is no restriction on the acquisition method of point cloud data in the embodiment of this application.

Step 105: Construct a segmentation network, input the point cloud data of the training set into the segmentation network to obtain the predicted segmentation results, and adjust the network parameters of the segmentation network according to the predicted segmentation results until the training stop conditions are met, and a trained segmentation network is obtained;

Step 106: Input the first point cloud data and the second point cloud data into the segmentation network respectively to obtain a rough segmentation result, fuse the rough segmentation result with the historical point cloud data to obtain a fused point cloud, and slice the fused point cloud;

Step 107. Construct a fine-tuning network: Input the slice data into the fine-tuning network, and the fine-tuning network outputs accurate segmentation results to obtain target point cloud data.

In the first stage, a 3D convolutional segmentation network is used to segment the point cloud data. The segmentation results are merged with the historical point cloud data in the channel dimension and input to the fine-tuning network. The segmentation results are then processed using a 2D convolutional fine-tuning network. Fine-tuning allows the network to make full use of the features of three-dimensional convolution and two-dimensional convolution, thereby improving segmentation accuracy.

Adding historical point cloud data to the training set can fully express local geometric feature distribution information, so that when the segmentation network performs semantic segmentation on the first point cloud data and the second point cloud data, it can have good adaptability and improve the segmentation accuracy. . Using the segmentation network to extract key points of point cloud data not only reduces the number of point clouds but also ensures the representativeness of the point cloud data, which can improve the efficiency of matching and fusion in step three.

Fusion of the coarse segmentation results and historical point cloud data makes full use of the rich features of buildings that are widely present in historical point cloud data to achieve effective positioning of buildings when the point clouds of the coarse segmentation results are sparse.

Use the fine-tuning network to subdivide pixels to further optimize the segmentation results of the segmentation network.

In this embodiment, the segmentation network in step 1 adopts the V-net network. At the end of the encoder of the V-net network, a pooling layer, a 1×1 convolution layer and three 3×3 convolution layers with different expansion rates are set in sequence. .

In this embodiment, the fine-tuning network in step 1 uses the U-net network, and the U-net network uses the weighted sum of the boundary loss function and the binary cross-entropy loss as the overall loss function.

U-net network refers to a convolutional network used for biomedical image segmentation. V-net network refers to a fully convolutional neural network used for three-dimensional medical image segmentation.

Step 2: Process the image to obtain the target image;

Step 201: The second aviation equipment collects the first new image of the measured area, and performs aerial three-dimensional encryption on the preprocessed first new image to obtain the first new image;

Step 202: The second vehicle-mounted device collects the second new image of the measured area, and performs air triple encryption on the preprocessed second new image to obtain the second new image;

Step 203: Obtain historical images of the measured area;

Step 204: Perform feature extraction on the first new image and the second new image respectively to obtain the first feature map F ₁ and the second feature map F ₂ , and perform feature extraction on the historical image to obtain the third feature map F ₃ .

The feature extraction module has a built-in deep neural network, and its feature extraction method is well known to those skilled in the art. Therefore, its specific implementation method will not be described in detail.

Step 205: Use the BCA network to fuse the first feature map F ₁ and the second feature map F ₂ . The BCA network uses a convolutional layer to fuse the first feature map F ₁ and the second feature map F ₂ . The BCA network uses a cross-entropy loss function for supervision and an attention loss function to enhance the synergy between edge detection and semantic segmentation. The weights of the cross-entropy loss function and attention loss function are 1 and 0.5 respectively.

The full English name of BCA is: Boundary guided Context Aggregation module. BCA is translated as boundary guidance network.

Step 206: Build a multi-layer perception mlp model;

The full English name of MLP is Multilayer Perceptron. MLP is translated as multi-layer perceptron, which is a feedforward artificial neural network model.

Step 207: Perform global maximum pooling processing on the fused feature map F _R to obtain the first pooling value, and use the first pooling value to construct the first channel dimension vector C _FR ;

Step 208: Perform global average pooling processing on the third feature map F ₃ to obtain the second pooling value, and use the second pooling value to construct the second channel dimension vector C _F3 ;

Step 209: Input the first channel dimension vector C _FR and the second channel dimension vector C _F3 into the multi-layer perception mlp model, and the mlp model outputs weight ω ₁ ;

Step 2010: Perform weighted fusion of the fused feature map F _R and the third feature map F ₃ based on the formula F=ω ₁ ·FR + (1-ω ₁ )·F ₃ to obtain _the target image.

First, perform a pooling operation on the fused feature map F _R and the third feature map F ₃ with the same number of channels, use the obtained pooling value to construct a channel dimension vector, and combine the first channel dimension vector C _FR and the second channel dimension vector C _F3 inputs the multi-layer perception mlp model, and the mlp model outputs weight ω ₁ .

The learnable weight ω ₁ is obtained based on the pooling value. Therefore, if the image features are different, the pooling value is different, and the corresponding learnable weight ω ₁ is different. Therefore, the learnable weight ω ₁ is updated based on the multi-layer perception mlp model, and finally we get The target image is the same size as the input, and the effect is good.

Affected by the shooting angle, the top area of the first newly added image collected by the second aerial equipment is rich in information, but its sides are lacking in information or even empty due to the shooting angle or occlusion. The second newly added image collected by the second vehicle-mounted device is the opposite. The information in the top area of the second newly added image is lacking or even empty, but the side information is complete and rich. Therefore, fusing the first newly added image and the second newly added image can provide more detailed geometric constraint information. At the same time, fusing historical images can make full use of the sparse feature information of the first newly added image and the second newly added image. The rich features of buildings that are widely present in historical images avoid image information gaps and achieve effective positioning of buildings.

Step 3: Match the target point cloud data and the target image to obtain the modeling data. The specific method is:

Step 301. Determine the matching domain: Use the CornerNet model to read the corner points of the m-th target image, form the target frame from the corner points, convert the ground coordinates X _j of each corner point, and connect the ground coordinates of each corner point to form the matching domain. .

It should be noted that the CornerNet model predicts two sets of heat maps through a convolutional network, representing the upper left corner position and the lower right corner position of the target image respectively. The upper left corner position and the lower right corner position are the corner points, and the upper left corner position and the lower right corner position constitute the target frame.

Step 302. Determine the matching line: Divide one side of the target frame into n equal parts to obtain n+1 parallel lines. Randomly select a point on each parallel line and convert to obtain the ground coordinates X _d of each point. The ground coordinates of each point are obtained. The coordinates X _d are connected to form the first matching line;

Step 303. In the target point cloud, find a target point cloud subset corresponding to the matching domain, and determine at least 3 coordinate points corresponding to the ground coordinates X _d in the target point cloud subset to form a second matching line. Calculate The rotation angle θ between the first matching line and the second matching line;

Step 304: Correspond to the rotation angle θ of the target point cloud subset, and add the rotated target point cloud subset to the feature point set of the m-th target image to obtain the complete feature point set of the m-th target image;

Step 305: Repeat steps 301-304 to complete the matching of all target point cloud data and target images.

Rotate the target point cloud data corresponding to the building so that it matches and fuses with the target image corresponding to the building, thereby completing the fusion of open space images. Under the building framework provided by the target image, fill in the target point cloud data to bring Information with centimeter-level accuracy can improve the accuracy of building positioning.

Step 4. Model reconstruction: Based on the modeling data, construct the three-dimensional model and the two-dimensional model of the measured area; it should be noted that the model reconstruction is based on the two-dimensional and three-dimensional integration technology to establish the three-dimensional model and the two-dimensional model of the buildings in the measured area. Model.

Step 5. Model modification: modify the two-dimensional model and the three-dimensional model;

Modifying the 2D model includes: detecting the invalid value areas of the 2D model and filling them; repairing the noise areas of the 2D model; clearing the suspended matter of the 2D model; setting the number of layers and mapping multiple colors to multiple layers. In terms of data, the two-dimensional model is rendered with layered coloring.

Modifying the 3D model includes: detecting the invalid value areas of the 3D model and filling them; repairing the noise areas of the 3D model; clearing the suspended objects of the 3D model; setting the number of layers and corresponding multiple colors to multiple layered data. The three-dimensional model is rendered with layered coloring; the three-dimensional model is automatically processed with light and color uniformity.

Step 6. Model display: The two-dimensional spatial data of the two-dimensional model and the three-dimensional spatial data of the three-dimensional model are integrated and displayed based on the two-dimensional and three-dimensional integration technology.

Two-dimensional spatial data includes vector data, elevation data, image data, and tilt data; three-dimensional spatial data includes model data and tilt data.

Two-dimensional and three-dimensional integration technology is a new generation of GIS technology. The full English name of GIS is: Geographic Information System, which is translated as geographical information system. To put it simply, 2D and 3D integration technology can integrate 2D spatial data and 3D spatial data in GIS on the same platform. Among them, the established two-dimensional data can be used directly on the three-dimensional platform, and the two-dimensional spatial data can be directly visualized in the three-dimensional scene without any conversion processing. In the process of using this two-dimensional and three-dimensional integration technology, users can directly operate the building data on the two-dimensional GIS map, and at the same time, the three-dimensional GIS map of the building will be generated simultaneously in the three-dimensional scene.

For example: when a user creates location data of a building on a two-dimensional GIS map, the topological relationship of the building in the two-dimensional scene can be established through two-dimensional and three-dimensional integration technology, and by obtaining the elevation data of the building in the two-dimensional scene, A three-dimensional GIS map of the building will be generated simultaneously.

Among them, the contents not described in detail in the specification belong to the prior art known to those skilled in the art.

The above are only embodiments of the present invention and do not limit the present invention in any way. Any simple modifications, changes and equivalent structural changes made to the above embodiments based on the technical essence of the present invention still belong to the technical solutions of the present invention. within the scope of protection.

Claims (10)

1. A multi-source data fusion application method, characterized by: Step 1. Obtain target point cloud data: Input the first point cloud data, second point cloud data and historical point cloud data into the coarse segmentation network, and learn the relationship between pixels and object area features based on the coarse segmentation network to enhance the pixel features. Description, obtain the rough segmentation result; fuse the rough segmentation result with the historical point cloud data to obtain the fused point cloud, input the fused point cloud into the fine-tuning network, and the fine-tuning network outputs accurate segmentation results to obtain the target point cloud data; Step 2. Obtain the target image: perform feature extraction on the first new image, the second new image and the historical image respectively to obtain the first feature map F ₁ , the second feature map F ₂ and the third feature map F ₃ . The first feature map F ₁ and the second feature map F ₂ are fused to obtain the fused feature map F _R ; based on the multi-layer perception mlp model, the learnable weight is output, and the third feature map F ₃ and the fused feature map F are fused based on the learnable weight _R , get the target image; Step 3: Match the target point cloud data and the target image to obtain modeling data; Step 4. Model reconstruction: Based on the modeling data, construct a three-dimensional model and a two-dimensional model of the measured area; Step 5. Model modification: modify the two-dimensional model and the three-dimensional model; Step 6. Model display: display the two-dimensional spatial data of the two-dimensional model and the three-dimensional spatial data of the three-dimensional model in an integrated manner. 2. A multi-source data fusion application method according to claim 1, characterized in that: the specific method of step one is: Step 101. The first aviation equipment collects the first new point cloud data of the measured area, preprocesses the first new point cloud data, corrects the first new point cloud data, and obtains the first point cloud data; Step 102: The first vehicle-mounted device collects the second newly added point cloud data of the measured area, preprocesses the second newly added point cloud data, corrects the second newly added point cloud data, and obtains the second point cloud data; Step 103: Obtain historical point cloud data of the measured area; Step 104. The first point cloud data, the second point cloud data and the historical point cloud data constitute a point cloud data set. The point cloud data set is divided into a training set, a verification set and a test set, and corresponding points are added to the point cloud data of the training set. Tag of; Step 105: Construct a segmentation network, input the point cloud data of the training set into the segmentation network to obtain the predicted segmentation results, and adjust the network parameters of the segmentation network according to the predicted segmentation results until the training stop conditions are met, and a trained segmentation network is obtained; Step 106: Input the first point cloud data and the second point cloud data into the segmentation network respectively to obtain a rough segmentation result, fuse the rough segmentation result with the historical point cloud data to obtain a fused point cloud, and slice the fused point cloud; Step 107: Construct a fine-tuning network, input the slice data into the fine-tuning network, and the fine-tuning network outputs accurate segmentation results to obtain target point cloud data. 3. A multi-source data fusion application method according to claim 1, characterized in that: the specific method of step two is: Step 201: The second aviation equipment collects the first new image of the measured area, and performs aerial three-dimensional encryption on the preprocessed first new image to obtain the first new image; Step 202: The second vehicle-mounted device collects the second new image of the measured area, and performs air triple encryption on the preprocessed second new image to obtain the second new image; Step 203: Obtain historical images of the measured area; Step 204: Perform feature extraction on the first newly added image and the second newly added image respectively to obtain the first feature map F ₁ and the second feature map F ₂ , perform feature extraction on the historical image to obtain the third feature map F ₃ ; Step 205: Use the BCA network to fuse the first feature map F ₁ and the second feature map F ₂ to obtain the fused feature map _FR ; Step 206: Build a multi-layer perception mlp model; Step 207: Perform global maximum pooling processing on the fused feature map F _R to obtain the first pooling value, and use the first pooling value to construct the first channel dimension vector C _FR ; Step 208: Perform global average pooling processing on the third feature map F ₃ to obtain the second pooling value, and use the second pooling value to construct the second channel dimension vector C _F3 ; Step 209: Input the first channel dimension vector C _FR and the second channel dimension vector C _F3 into the multi-layer perception mlp model, and the mlp model outputs weight ω ₁ ; Step 2010: Perform weighted fusion of the fusion feature map F _R and the third feature map F ₃ based on the formula F=ω ₁ · _FR + (1-ω ₁ )·F ₃ to obtain the target image F. 4. A multi-source data fusion application method according to claim 1, characterized in that: the specific method for matching target point cloud data and target images is: Step 301. Determine the matching domain: Use the CornerNet model to read the corner points of the m-th target image, form the target frame from the corner points, convert the ground coordinates X _j of each corner point, and connect the ground coordinates of each corner point to form the matching domain. ; Step 302. Determine the matching line: Divide one side of the target frame into n equal parts to obtain n+1 parallel lines. Randomly select a point on each parallel line and convert to obtain the ground coordinates X _d of each point. The ground coordinates of each point are obtained. The coordinates X _d are connected to form the first matching line; Step 303. In the target point cloud, find a target point cloud subset corresponding to the matching domain, and determine at least 3 coordinate points corresponding to the ground coordinates X _d in the target point cloud subset to form a second matching line. Calculate The rotation angle θ between the first matching line and the second matching line; Step 304: Correspond to the rotation angle θ of the target point cloud subset, and add the rotated target point cloud subset to the feature point set of the m-th target image to obtain the complete feature point set of the m-th target image; Step 305: Repeat steps 301-304 to complete the matching of all target point cloud data and target images. 5. A multi-source data fusion application method according to claim 1, characterized in that: modifying the two-dimensional model in step five includes: detecting the invalid value area of the two-dimensional model and filling it; repairing the noise of the two-dimensional model area; clear the suspended objects of the 2D model; set the number of layers, map multiple colors to multiple layered data, and render the 2D model with layered coloring. 6. A multi-source data fusion application method according to claim 1, characterized in that: modifying the three-dimensional model in step five includes: detecting invalid value areas of the three-dimensional model and filling them; repairing the noise areas of the three-dimensional model; and clearing Suspended objects of the 3D model; set the number of layers, map multiple colors to multiple layered data respectively, perform layered color rendering of the 3D model; perform automatic light and color uniformity processing on the 3D model. 7. A multi-source data fusion application method according to claim 2, characterized in that: the segmentation network in step one adopts the V-net network, and the end of the encoder of the V-net network is provided with a pooling layer, a 1× 1 convolutional layer and 3 3×3 convolutional layers with different dilation rates. 8. A multi-source data fusion application method according to claim 2, characterized in that: the fine-tuning network in step one adopts the U-net network, and the U-net network adopts the weighting of the boundary loss function and the binary cross-entropy loss. and as the overall loss function. 9. A multi-source data fusion application method according to claim 1 or 6, characterized in that: two-dimensional spatial data includes vector data, elevation data, image data, and tilt data; three-dimensional spatial data includes model data and tilt data. . 10. A multi-source data fusion application method according to claim 1 or 6, characterized in that: preprocessing the point cloud data in step one includes fully automatic filtering of the point cloud, and the fully automatic filtering of the point cloud includes adaptive filtering. , leveling filter, smoothing filter, fusion filter, general filter, elevation reduction filter, profile filter.