CN117725141A - Open-pit mining area vector road network map construction method and related device based on deep learning - Google Patents
Open-pit mining area vector road network map construction method and related device based on deep learning Download PDFInfo
- Publication number
- CN117725141A CN117725141A CN202311723219.XA CN202311723219A CN117725141A CN 117725141 A CN117725141 A CN 117725141A CN 202311723219 A CN202311723219 A CN 202311723219A CN 117725141 A CN117725141 A CN 117725141A
- Authority
- CN
- China
- Prior art keywords
- image
- road
- model
- open
- area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000005065 mining Methods 0.000 title claims abstract description 84
- 238000013135 deep learning Methods 0.000 title claims abstract description 56
- 238000010276 construction Methods 0.000 title claims abstract description 33
- 238000000034 method Methods 0.000 claims abstract description 52
- 238000000605 extraction Methods 0.000 claims abstract description 23
- 238000005457 optimization Methods 0.000 claims abstract description 21
- 238000012545 processing Methods 0.000 claims abstract description 11
- 238000004519 manufacturing process Methods 0.000 claims abstract description 10
- 230000009467 reduction Effects 0.000 claims abstract description 7
- 238000012549 training Methods 0.000 claims description 26
- 230000008569 process Effects 0.000 claims description 21
- 238000004364 calculation method Methods 0.000 claims description 18
- 238000007781 pre-processing Methods 0.000 claims description 14
- 238000012360 testing method Methods 0.000 claims description 9
- 238000012795 verification Methods 0.000 claims description 9
- 238000003860 storage Methods 0.000 claims description 8
- 230000006870 function Effects 0.000 claims description 7
- 235000015220 hamburgers Nutrition 0.000 claims description 6
- 230000000877 morphologic effect Effects 0.000 claims description 6
- 238000004422 calculation algorithm Methods 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 238000004590 computer program Methods 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 6
- 239000011159 matrix material Substances 0.000 description 6
- 230000000694 effects Effects 0.000 description 5
- 230000011218 segmentation Effects 0.000 description 5
- 238000005286 illumination Methods 0.000 description 4
- 235000008733 Citrus aurantifolia Nutrition 0.000 description 2
- 235000011941 Tilia x europaea Nutrition 0.000 description 2
- 238000000354 decomposition reaction Methods 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 239000004571 lime Substances 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000009412 basement excavation Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000004883 computer application Methods 0.000 description 1
- 238000013499 data model Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000007599 discharging Methods 0.000 description 1
- 239000000428 dust Substances 0.000 description 1
- 238000005530 etching Methods 0.000 description 1
- 230000006698 induction Effects 0.000 description 1
- 229910052500 inorganic mineral Inorganic materials 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 239000011707 mineral Substances 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000007670 refining Methods 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 239000004576 sand Substances 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses a deep learning-based open-pit mining area vector road network map construction method and a related device, wherein the method comprises the steps of collecting open-pit mining area oblique photographic images containing geographic position information, and carrying out low-light image enhancement and noise reduction pretreatment on the open-pit mining area oblique photographic images; carrying out image extraction on the preprocessed image through the trained deep learning network model, and extracting road area images in the open sky mining area; carrying out optimization processing on the road area image to obtain an optimized road area image; constructing an opencut road network orthophoto model and a digital surface model based on the optimized road area image; and carrying out vectorization on a road area in the open-pit mining area by using the open-pit mining road network orthographic image model and the digital surface model, and manufacturing to obtain an open-pit mining vector road network map. The map obtained by the invention can carry out road network matching on mining vehicles with errors in position, thereby meeting the requirement of high-precision positioning of vehicles in open-air mining areas.
Description
Technical Field
The invention belongs to the technical field of vector road network map drawing, and simultaneously belongs to the field of computer application, relates to image semantic segmentation in deep learning, and particularly relates to an open pit mining area vector road network map construction method and a related device based on the deep learning.
Background
Under the environment of the strong development of intelligent mine construction, an intelligent dispatching system and an unmanned technology of a mining truck are not separated from an accurate strip mine high-precision road network map, and the intelligent dispatching system and the unmanned technology take an important leading role in stope transportation. And with the implementation and application of unmanned technology in strip mines, high-precision road network map construction of mining area roads becomes more and more important. The accurate open-pit road network map information provides great application significance for unmanned intelligent traffic navigation, intelligent scheduling, digital map updating, path planning, emergency guarantee and the like of the open-pit.
However, along with continuous movement of mining and discharging of the strip mine, the transportation route of the vehicle is changed along with the continuous movement, and minerals are required to be transported to corresponding unloading points, so that the road network of the strip mine is changed very frequently in the mining process, and if the map of the road network of the strip mine is updated in an insufficiently timely manner, an intelligent dispatching system of the vehicle and unmanned operation can be affected, and further the production condition of a mining area is affected. Therefore, the construction of the strip mine road network map must be efficient and accurate.
The current deep learning-based method achieves great results in the aspect of strip mine road extraction, can not only completely reserve the surrounding environment information of the road, but also can extract the road area efficiently and accurately. However, road network data constructed by the road network data are not vectorized, and longitude and latitude information of each area of the strip mine cannot be reflected, so that the road network data cannot be used for real-time monitoring of vehicle running tracks, and effective assistance cannot be provided for intelligent dispatching and unmanned driving of mining trucks directly. How to update and manufacture the vector road network map of the strip mine at regular intervals under the background of complex environment is a serious difficulty of intelligent mine construction.
Disclosure of Invention
In order to efficiently and accurately update and manufacture the strip mine road network map and take the defect of extracting a road area based on a deep learning technology in the past into consideration, the invention aims to provide a strip mine area vector road network map construction method based on deep learning and a related device.
The technical scheme adopted by the invention is as follows:
the open-pit mining area vector road network map construction method based on deep learning comprises the following steps:
collecting an open-air mining area inclined photographic image containing geographical position information, and preprocessing the collected open-air mining area inclined photographic image, wherein the preprocessing process comprises low-light image enhancement and noise reduction;
carrying out image extraction on the preprocessed strip mine inclined photographic image through a trained deep learning network model, and extracting road area images in the strip mine;
carrying out optimization processing on the road area image to obtain an optimized road area image;
constructing an opencut road network orthophoto model and a digital surface model based on the optimized road area image;
and vectorizing a road area in the open pit area by using the open pit road network orthophoto model and the digital surface model to manufacture an open pit vector road network map.
Preferably, when the inclined photographic image of the open-pit mining area containing the geographical position information is acquired, the image is photographed at an orthographic angle and at an oblique angle of 45 degrees under the conditions of a preset height from the ground and a ground overlapping rate of 75%.
Preferably, the establishing process of the deep learning network model includes:
establishing the deep learning network model by combining cross window self-attention, local enhancement position coding and a lightweight Hamburg decoder;
the training process of the deep learning network model comprises the following steps:
collecting an open-air mining area inclined photographic image containing geographical position information, and preprocessing the collected open-air mining area inclined photographic image;
manually marking a road area in the preprocessed strip mine inclined photographic image based on a labelme tool and establishing a road data set;
dividing the road data set into a training set, a verification set and a test set, wherein more than 50% of images containing temporary roads are divided into the training set, and other images are randomly divided into the verification set and the test set;
combining a binary cross entropy Loss function BCE and a Focal Loss for solving the problem of class unbalance, and establishing a Loss function in the training process of a CSHfomer network model;
based on the CSHformer network model, training the deep learning network model by utilizing the training set, the verification set and the test set.
Preferably, the deep learning network model framework is as follows:
an input image of 512×512×3 size is input through Convolutional Token EmbeedingLine convolution operations, obtaining patch samples of 128×128 size, then generating different hierarchical representations by 4 stages, each stage containing N t A number of consecutive Cswin tranformer block, a convolution operation with a convolution kernel size of 3×3 and a step size of 2 is used between each adjacent stage to reduce the number of token and increase the number of channels, wherein the i-th layer feature map has a size ofAfter the feature layers of stage2, stage3 and stage4 are connected, the global environment is further modeled through a Hamburger model, and then a result is output through an MLP layer and an auxiliary layer FCN.
Preferably, the optimizing the road area image includes:
and carrying out open circuit connection on the road area image by adopting morphological operation, carrying out refinement treatment on the road network area by adopting a Zhang-Suen refinement algorithm, and carrying out image super-resolution treatment by adopting a SwinIR model.
Preferably, the process of constructing the opencut road network orthographic image model and the digital surface model based on the optimized road area image comprises the following steps:
according to the method of homonymy matching, geographic position information in the collected strip mine inclined photographic image is endowed to the optimized road area image, and the optimized road area image with the geographic position information is obtained;
importing the optimized road area image with the geographic position information into ContextCapture software, performing air triangulation calculation on the imported optimized road area image with the geographic position information by ContextCapture software, performing air three optimization on the image with the puncture point according to the puncture point and image control point information, generating a high-resolution triangular mesh model with real texture, restoring the geometric appearance and texture characteristics of an open pit modeling object, and then deriving a road network orthographic image model and a digital surface model in a TIFF format.
Preferably, the strip mine road network orthophoto model and the digital surface model are loaded through a GIS platform, the road area in the strip mine area is vectorized, and the strip mine vector road network map is manufactured, comprising the following steps:
importing the strip mine road network orthographic image model and the digital surface model into a GIS platform, and removing the black edges of the strip mine road network orthographic image model;
converting a coordinate system of an opencast road network orthophoto model and a digital surface model into a WGS84 geodetic coordinate system by using a coordinate conversion tool, vectorizing road network areas, road network central lines and road network central line inflection points in a mining area, and attaching elevation information to vectorized road data to finally form a vector road network map of the whole opencast mining area.
The invention also provides an open-air mining area vector road network map construction system based on deep learning, which is used for realizing the open-air mining area vector road network map construction method based on deep learning, and comprises the following steps:
and an image acquisition module: the method comprises the steps of acquiring an open-air mining area oblique photographic image containing geographical position information, and preprocessing the acquired open-air mining area oblique photographic image, wherein the preprocessing process comprises low-light image enhancement and noise reduction;
an image extraction module: the method comprises the steps of performing image extraction on preprocessed strip mine inclined photographic images through a trained deep learning network model, and extracting road area images in a strip mine;
and an image optimization module: the method comprises the steps of carrying out optimization processing on the road area image to obtain an optimized road area image;
modeling module: the method is used for constructing an opencut road network orthophoto model and a digital surface model based on the optimized road area image;
and a map construction module: the method is used for vectorizing the road area in the open-pit mining area by utilizing the open-pit mining road network orthophoto model and the digital surface model, and a open-pit mining vector road network map is manufactured.
The invention also provides an electronic device, comprising:
one or more processors;
a storage device having one or more programs stored thereon;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the deep learning based strip mine vector road network map construction method of the present invention as described above.
The invention also provides a storage medium on which a computer program is stored, wherein the computer program, when being executed by a processor, implements the deep learning-based open pit mining area vector road network map construction method according to the invention.
The invention has the following beneficial effects:
the vector road network map of the open-air mining area is constructed by the open-air mining area vector road network map construction method based on deep learning, so that the open-air mining area vector road network map can be updated in time by collecting the open-air mining area oblique photographic image containing geographical position information, and the vector road network map can embody longitude and latitude information of each area of the open-air mining, so that the vector road network map can be used for real-time monitoring of vehicle running tracks, and can provide effective help for intelligent dispatching and unmanned driving of mining trucks. The map construction method can greatly save manpower and material resources and more efficiently and accurately extract the road information of the open-pit mining area.
Drawings
Fig. 1 is a flowchart of an open air mining area vector road network map construction method based on deep learning in an embodiment of the invention.
Fig. 2 is an exemplary diagram of an image preprocessing result in an embodiment of the present invention, where (a) is an original image, (b) is a noise-reduced image, and (c) is an enhanced image of a low-light image.
Fig. 3 is a framework diagram of a CSHformer network model constructed in an embodiment of the present invention.
FIG. 4 is a schematic diagram of cross window self-attention calculation in an embodiment of the present invention, where (a) is a cross window graph, (b) is a cross stripe self-attention calculation process graph, and (c) is a final calculation result graph of cross window self-attention output by connecting the cross stripe and vertical stripe self-attention calculations together.
Fig. 5 is a schematic diagram of a morphological open circuit connection procedure in an embodiment of the invention.
Fig. 6 is an example of the result of optimizing an image after extracting a road area in the embodiment of the present invention, where (a) is an image after extracting a road area, (b) is an image after road connection, (c) is an image after road network refinement, and (d) is an image after super resolution.
Fig. 7 is a flowchart of a road network data vectoring operation in an embodiment of the present invention.
Fig. 8 is a block diagram of a surface mining area vector road network map construction system based on deep learning in an embodiment of the invention.
Detailed Description
The invention will now be described in detail with reference to the drawings and examples.
The invention discloses a deep learning-based open-air mining area vector road network map construction method, which is used for constructing a deep learning network model-CSHformer which integrates cross-shaped window self-attention, local enhancement position coding (LePE) and a lightweight Hamburg decoder, has less calculation amount and can ensure a higher-precision extraction effect. The road image extracted from the strip mine is subjected to optimization processing such as disconnection connection, refinement and super-resolution. In the specific implementation, the original longitude and latitude information of the unmanned aerial vehicle oblique photographic image is endowed to the extracted road image, an opencut road network orthophoto model and a digital surface model are constructed, and the manufacture of an opencut vector road network map is realized through vectorized road network areas, road network central lines and road network central line inflection points.
Specifically, the open-pit mining area vector road network map construction method based on deep learning comprises the following steps:
step one: collecting an open-air mining area oblique photographic image by using an unmanned aerial vehicle, performing low-light image enhancement and noise reduction treatment on the image, and manually marking a road area in the image based on a labelme tool to establish a road data set; specifically, special light and shadow effects are captured in the early morning by the unmanned aerial vehicle, pictures under normal illumination conditions are recorded in noon, and the road image of the strip mine is shot under the conditions of soft light, lighter shadow and prominent detail in the cloudy day. Meanwhile, the unmanned aerial vehicle is used for making accurate map measurement by shooting according to a positive angle under the conditions of a height of 200 meters from the ground and a ground overlapping rate of 75%, and shooting at an oblique angle of 45 degrees so as to obtain more stereoscopic impression and detail.
The resolution of the road image collected by the unmanned aerial vehicle is 5472 multiplied by 3648, each image records the corresponding geographic position information, and the adopted coordinate system is a CGCS2000 geodetic coordinate system. Considering the performance of a computer, firstly dividing an image into 6 parts in a 3-column 2-row segmentation mode, then dividing the resolution of the divided image into 512X 512 from 1824X 1824Resize, and naming the divided image as follows: original image names_01 to 06.
Due to poor natural environment of open-pit mining areas and complex mining area operations, such as mechanical operations of truck carrying, excavation and the like, the mining area sites are often accompanied by phenomena of dust, sand lifting and the like, sometimes even under the influence of weather, and the objective conditions can influence the image quality. Through analysis of the original data set, the fact that part of pictures are blurred and have dark colors can influence the deep learning network model to learn and extract road features in the images. The method utilizes a Retinex modeling-based Low-Light Image Enhancement (LIME) method to restore the strip mine road information in a darker illumination environment as clearly as possible, and utilizes a restorer model to effectively remove noise points of a strip mine road image.
Road extraction is a classification task, and a road area in an image is manually marked with a "road" label based on a labelme tool, and the background is a "background" label. And analyzing the duty ratio of the fixed road, the semi-fixed road, the mobile road and the temporary road contained in the strip mine road image and the pixel distinction condition from the surrounding environment. Considering that the boundary between the temporary road and the surrounding environment is not obvious, in order to enable the deep learning network model to learn more detail features of the temporary road, the images containing 50% of the temporary road and more than the temporary road are divided into training sets, and other images are randomly divided. And 4558 training sets, 1140 verification sets and 1424 test sets are finally obtained and used for pre-training the deep learning network model.
Step two: establishing a deep learning network model, pre-training based on a strip mine road data set, and extracting a road area in a strip mine area by adopting the pre-trained model; specifically, a deep learning network model-CSHformer is built by combining cross-window self-attention, localized enhanced position coding (LePE), and lightweight Hamburger decoders. The model utilizes CSWin Transformer Block to extract multi-level characteristics, forms an encoder-decoder structure with a lightweight Hamburg model, can effectively reduce the loss of characteristic information by fusing low-level characteristic information and high-level characteristic information, has less calculation amount, can ensure higher-precision extraction effect, and can meet the condition that a road network map needs to be updated frequently, efficiently and accurately by a strip mine.
Step three: carrying out open circuit connection on the extracted road image by adopting morphological operation, carrying out refinement treatment on the road network area by adopting a Zhang-Suen refinement algorithm, and carrying out image super-resolution treatment by adopting a SwinIR model; in particular, for open pit temporary roads with extremely insignificant characteristic information, the CSHformer network model still cannot segment them well. The road network extraction result can have the problems of road disconnection, holes, unobvious road network skeleton structure and the like. In order to construct an opencut road network orthophoto model, the image extracted by the road network is restored to the original resolution size of 5472 multiplied by 3648, and the image information is damaged by the direct resize operation, so that further optimization processing is needed. The specific optimization process is as follows:
(1) And (3) breaking connection: based on morphological image processing technology, the structure of an input image is measured by using 3×3 square structural elements, and the image to be short-circuited is first connected by expansion operation to the separated road area, but the edge characteristics of the road are strengthened. In order to restore the shape of the original region to the maximum extent, an etching operation is again employed to repair the edge portion expanded during the expansion process.
(2) And (3) refining: and the Zhang-Suen refinement algorithm is adopted to refine the road area extracted from the strip mine, and the road skeleton information is extracted, so that the road structure of the final map is more visual and clear.
(3) Super resolution: the resolution of the road image extracted by the strip mine is 512 multiplied by 512, and as geographical position information in the original image acquired by the unmanned aerial vehicle is required to be endowed to the corresponding image when the strip mine road network orthophoto model is subsequently constructed, the 6 equally divided images are combined according to the method of matching the images with the names, and then the SwinIR model is adopted to carry out super resolution on the combined images, so that the original image information is restored as far as possible.
Step four: constructing a strip mine road network orthophoto model (DOM) and a Digital Surface Model (DSM) based on the strip mine road image after the optimization treatment; specifically, the strip mine road image extracted through the CSHfomer network model loses geographic position information, geographic information in an original image shot by the unmanned aerial vehicle is endowed to the strip mine road image after the extraction and optimization treatment according to a homonymous matching method, and the strip mine road image is used for constructing an orthographic image of a strip mine road network; performing aerial triangulation calculation on the imported open pit road image with geographical position information by using ContextCapture software, performing aerial three-dimensional optimization on the image with the pricked point according to the pricked point and image control point information, generating a high-resolution triangular mesh model with real texture, accurately restoring the geometric appearance and texture characteristics of the open pit modeling object, and deriving a road network orthographic image model (DOM) and a Digital Surface Model (DSM) in TIFF format.
Step five: and loading DOM and DSM through the GIS platform, vectorizing the road network area, and manufacturing the vector road network map of the strip mine. Specifically, an opencut road network orthophoto image model (DOM) and a Digital Surface Model (DSM) are imported into a GIS platform, and the image black edge of the DOM model is removed. To match the base map (world map) in the GIS platform, the coordinate system of DOM and DSM is converted to WGS84 geodetic coordinate system using a coordinate conversion tool. And vectorizing the road network area, the road network center line and the inflection point of the road network center line in the mining area, and attaching elevation information to the vectorized road network data to finally form a vector road network map of the whole open-air mining area.
According to the scheme, the road network information is rapidly and accurately extracted by using a deep learning technology under the environment that the unstructured road edges of the strip mine are not obvious and the background interference is more. Road network matching can be carried out on mining area vehicles with errors based on vectorized road network data, and the requirement of high-precision positioning of the vehicles in the open-air mining areas is met.
Examples
In the embodiment, henan Luoyang certain open-pit mining area roads are taken as a study object. As shown in fig. 1, a flow chart of the method for constructing the vector road network map of the strip mine is mainly divided into five steps.
And step 1, capturing special light and shadow effects in the early morning by the unmanned aerial vehicle, recording pictures under normal illumination conditions in noon, and shooting a strip mine road image under the conditions of soft light, lighter shadow and prominent detail in the cloudy day. Meanwhile, the unmanned aerial vehicle is used for making accurate map measurement by shooting according to a positive angle under the conditions of a height of 200 meters from the ground and a ground overlapping rate of 75%, and shooting at an oblique angle of 45 degrees so as to obtain more stereoscopic impression and detail.
The resolution of the road image collected by the unmanned aerial vehicle is 5472 multiplied by 3648, each image records the geographic position information corresponding to the image, and the adopted coordinate system is a CGCS2000 geodetic coordinate system. Considering the performance of a computer, firstly dividing an image into 6 parts in a 3-column 2-row segmentation mode, then dividing the resolution of the divided image into 512X 512 from 1824X 1824Resize, and naming the divided image as follows: original image names_01 to 06.
Because the Low light area and noise in the image may affect the process of extracting road features of the subsequent deep learning network model, the strip mine road information in the darker illumination environment is restored as clearly as possible by using a Low-Light Image Enhancement (LIME) method based on Retinex modeling, and noise points of the strip mine road image are effectively removed by using a Restormer model, as shown in fig. 2, which is a result example after preprocessing.
Road extraction is a classification task, and a road area in an image is manually marked with a "road" label based on a labelme tool, and the background is a "background" label. And analyzing the duty ratio of the fixed road, the semi-fixed road, the mobile road and the temporary road contained in the strip mine road image and the pixel distinction condition from the surrounding environment. Considering that the boundary between the temporary road and the surrounding environment is not obvious, in order to enable the deep learning network model to learn more detail characteristics of the temporary road, the images containing 50% of the temporary road and more than the temporary road are divided into training sets, and other images are randomly divided into a verification set and a test set. And 4558 training sets, 1140 verification sets and 1424 test sets are finally obtained and used for pre-training the deep learning network model.
And 2, establishing a deep learning network model CSHfomer training strip mine road data set, and extracting a road area in the strip mine area by adopting a pre-trained model.
The overall architecture of the cshfomer network model constructed therein is shown in fig. 3, with an input image size of 512 x 3, and patch keys of 128 x 128 size, each with a latitude of C, were obtained by Convolutional Token Embeeding (using a convolution operation with a convolution kernel size of 7 x 7 and a step size of 4). Then a different hierarchical representation is generated by four stages, each stage containing N t A number of consecutive Cswin tranformer block, a convolution operation with a convolution kernel size of 3×3 and a step size of 2 is used between each adjacent stage to reduce the number of token and increase the number of channels, wherein the i-th layer feature map has a size ofAfter the feature layers of stage2, stage3 and stage4 are connected, the global environment is further modeled through a Hamburger model, and then a result is output through an MLP layer and an auxiliary layer FCN.
Calculation of self-attention of cross-shaped Window As shown in FIG. 4, the input feature X ε R (H×W)×C The linear projection is K token, self-attention by executing horizontal and vertical stripes in parallel on each token. The width of the horizontal window and the width of the vertical window are sw, and the width of the window can be increased along with the deepening of the hierarchy, so that the learning capacity and the computational complexity of the network model are adjusted. In order to make sw divisible by the input features H and W, the experimental embodiment sets sw for four stages to 1,2,8, respectively. Self-attention as in FIG. 4 (b) horizontal stripesThe calculation of the kth token of (c) is as follows:
wherein X is E [ X ] 1 ,X 2 ,…,X M ],X∈R (sw×W)×C ,M＝H/sw，i＝(1,2,…M)，d k ＝C/K，Q, K, V are the queries, keys and values, respectively, for each token.
Local enhancement position coding (LePE) can generate position codes for input images of arbitrary resolution sizes, impose position information in each transducer module, and operate directly on values in the attention mechanism. In order to reduce the calculation amount, capture the local Attention, use Depth-wise Conv to convolve value, then add the result to the result of Self-Attention, in this way, can not only keep the relative position information, can also strengthen the local induction bias, in order to improve the generalization ability of the model, the calculation of LePE is shown as follows:
the calculation of vertical stripe self-Attention can be analogized, and the calculation of kth token can be expressed as V-Attention k (X). Finally H-Attention k (X) and V-Attention k The outputs of (X) are connected together in parallel to be the final calculation of the self-attention of the cross window.
A decoder based on a lightweight Hamburger model learns the global environment of image features by modeling global context information. The Hamburg model consists of a matrix decomposition model M between two linear transformation layers, where Hams isModeling an objective function of "global information". Let x= [ X 1 ,…,x n ]∈R d×n I.e. there is a dictionary matrix d= [ D ] 1 ,…d r ]∈R d×r Sum coefficient matrix c= [ C ] 1 ,…,c n ]∈R r×n X may be defined asWherein low rank global information ∈ ->Residual terms (noise, redundancy or missing) E.epsilon.R d×n The Hamburger model may be defined as:
Y＝Z+BN(H(Z))
H(Z)＝W u M(W l Z)
wherein the features are inputLower break linear transformation +.>Upper break linear transformationL in the matrix decomposition model M represents reconstruction errors, which can be derived from the element distribution of the residual term E, R 1 And R is 2 Representing regularization of the dictionary matrix D and coefficient matrix C, respectively, can be derived from their prior distributions, H (Z) is fused with the input features via Batch Normalization (BN), and finally output Y.
The extraction of the strip mine road network can be regarded as a classification task, the road part to be extracted is regarded as foreground information, the problem of unbalance of the road and the background category can be obviously found, in order to reduce the influence of data unbalance on the training effect, a classification cross entropy Loss function BCE is combined with a Focal Loss for solving the problem of unbalance of the category, and a Loss function Loss of network model training is established:
Loss＝W 1 L BCE +W 2 L Focal
wherein W is 1 And W is 2 Is the Loss weighting coefficient of BCE Loss and Focal Loss,for model prediction of the probability that the sample is a road, y is the real label of the sample, alpha is the weight of the road class, gamma is the adjustment factor, and in this embodiment, alpha=0.8, gamma=2, and w are set 1 ＝W 2 ＝1。
According to the semantic segmentation network model, training is performed based on the strip mine road data set. The specific parameters of the training process are shown in table 2.
Table 2 model training parameters
By analyzing the training result of the network model, mAcc, mIoU, mPA values of the network model, which respectively reach 95.75%, 86.54% and 92.47%, can be obtained, and the model has smaller calculation amount and higher extraction precision. And semantic segmentation is carried out on all road areas of the open-pit mining area based on the trained network model, the road edge contour after the road network extraction is clear, and the open-pit mining road area can be accurately and completely extracted basically.
Step 3, further optimizing the extracted road network image; in order to make the result of the open-pit mine road network map more accurate, the extracted open-pit mine road network map is subjected to optimization processing such as open-circuit connection, refinement and super-resolution. As shown in fig. 5, a schematic diagram of the disconnection process based on morphological operations is shown in fig. 6, which is an example of the result after the optimization process.
And 4, giving geographic information in an original image shot by the unmanned aerial vehicle to the strip mine road image after the optimization processing after extraction according to a homonymous matching method, performing air triangulation calculation on the imported strip mine road image with geographic position information by using ContextCapture software for constructing an orthographic image of the strip mine road network, and performing air three optimization on the image with the pricked point according to the pricked point and the image control point information. Generating a high-resolution triangular mesh model with real textures, accurately restoring the geometric appearance and texture characteristics of the strip mine modeling object, and deriving a road network orthographic image model (DOM) and a Digital Surface Model (DSM) in a TIFF format.
And 5, importing a strip mine road network orthophoto model (DOM) and a Digital Surface Model (DSM) into the GIS platform, and removing the image black edge of the DOM model. And simultaneously, in order to match the base map (world map) in the GIS platform, the coordinate system of the DOM and the DSM is converted into a WGS84 geodetic coordinate system by using a coordinate conversion tool. The DOM model cannot directly represent geographic elements and geometric features of a specific road, and needs to vector road network areas, road network center lines and road network center line inflection points based on a GIS platform, and describe the spatial position relationship of the road network areas, the road network center lines and the road network center line inflection points through a coordinate system. As shown in fig. 7, a specific operation flow for making the strip mine vector road network map through the GIS platform is as follows:
(1) Loading a day map in the GIS platform. In order to align and check the subsequently imported raster data model with the geographic position of the world-level electronic map;
(2) A digital surface model DSM is imported. After vectorization operation, vector data such as a point line and a plane only record longitude and latitude coordinates, elevation information is not directly extracted, and the elevation information is imported into the vector data for subsequent extraction of the elevation information;
(3) An orthophoto model DOM is imported. In order to align and check the vectorized data with the actual image;
(4) And importing a road network model DOM. Carrying out vectorization on road network raster data for the follow-up;
(5) And importing a road network center line model DOM. Carrying out vectorization on the grid data of the central line of the road network for the subsequent step;
(6) And removing the black edge of the image. Opening the layer attribute panel, and setting the additional data value in the transparency to 0;
(7) Grid vectorization. Clicking the grid, converting, vectorizing, selecting a grid layer to be vectorized, and selecting a file storage position after vectorizing;
(8) And extracting the vertex from the road network center line. Clicking a vector-geometric figure tool-vertex extraction, selecting a line diagram layer of a road network center to be vector, and selecting a file storage position after vertex extraction;
(9) And extracting elevation information. Clicking the paste (Z value is set from a grid) in a vector geometry tool box, selecting a layer to be extracted with elevation information, selecting a DSM layer, and selecting a file storage position after the elevation information is extracted;
after the line-network point-line-surface data vectorization, setting indexes such as compass, scale, geographical position range and the like, and manufacturing a vector line-network map of the open-pit mining area.
In conclusion, the method can efficiently and accurately update and manufacture the vector road network map of the strip mine, and finally the extracted and optimized result can well cover the road area in the strip mine.
As shown in fig. 8, the present invention further provides a system for constructing a vector road network map of an open pit mining area based on deep learning, which comprises:
and an image acquisition module: the method comprises the steps of acquiring an open-air mining area oblique photographic image containing geographical position information, and preprocessing the acquired open-air mining area oblique photographic image, wherein the preprocessing process comprises low-light image enhancement and noise reduction;
an image extraction module: the method comprises the steps of performing image extraction on preprocessed strip mine inclined photographic images through a trained deep learning network model, and extracting road area images in a strip mine;
and an image optimization module: the method comprises the steps of carrying out optimization processing on the road area image to obtain an optimized road area image;
modeling module: the method is used for constructing an opencut road network orthophoto model and a digital surface model based on the optimized road area image;
and a map construction module: the method is used for vectorizing the road area in the open-pit mining area by utilizing the open-pit mining road network orthophoto model and the digital surface model, and a open-pit mining vector road network map is manufactured.
The embodiment of the invention also provides corresponding equipment and a computer readable storage medium, which are used for realizing the scheme provided by the embodiment of the invention.
The device comprises a memory and a processor, wherein the memory is used for storing instructions or codes, and the processor is used for executing the instructions or codes so that the device can execute the trusted DCS upper computer trusted state synchronization method according to any embodiment of the application.
Claims (10)
1. The open-pit mining area vector road network map construction method based on deep learning is characterized by comprising the following steps of:
collecting an open-air mining area inclined photographic image containing geographical position information, and preprocessing the collected open-air mining area inclined photographic image, wherein the preprocessing process comprises low-light image enhancement and noise reduction;
carrying out image extraction on the preprocessed strip mine inclined photographic image through a trained deep learning network model, and extracting road area images in the strip mine;
carrying out optimization processing on the road area image to obtain an optimized road area image;
constructing an opencut road network orthophoto model and a digital surface model based on the optimized road area image;
and vectorizing a road area in the open pit area by using the open pit road network orthophoto model and the digital surface model to manufacture an open pit vector road network map.
2. The deep learning-based surface mining area vector road network map construction method according to claim 1, wherein when the surface mining area oblique photographic image containing geographical position information is acquired, the surface mining area oblique photographic image is photographed at an orthonormal angle and at an oblique angle of 45 degrees under the condition that the ground is at a preset height and the ground overlapping rate is 75%.
3. The deep learning-based open pit area vector road network map construction method according to claim 1, wherein the deep learning network model construction process comprises the following steps:
establishing the deep learning network model by combining cross window self-attention, local enhancement position coding and a lightweight Hamburg decoder;
the training process of the deep learning network model comprises the following steps:
collecting an open-air mining area inclined photographic image containing geographical position information, and preprocessing the collected open-air mining area inclined photographic image;
manually marking a road area in the preprocessed strip mine inclined photographic image based on a labelme tool and establishing a road data set;
dividing the road data set into a training set, a verification set and a test set, wherein more than 50% of images containing temporary roads are divided into the training set, and other images are randomly divided into the verification set and the test set;
combining a binary cross entropy Loss function BCE and a Focal Loss for solving the problem of class unbalance, and establishing a Loss function in the training process of a CSHfomer network model;
based on the CSHformer network model, training the deep learning network model by utilizing the training set, the verification set and the test set.
4. A deep learning-based surface mining area vector road network map construction method according to claim 1 or 3, characterized in that the deep learning network model framework is as follows:
convolution of an input image of size 512×512×3 by Convolutional Token Embeeding to obtain patch samples of size 128×128, followed by 4 stages to produce different hierarchical representations, each stage containing N t A number of consecutive Cswin tranformer block, a convolution operation with a convolution kernel size of 3×3 and a step size of 2 is used between each adjacent stage to reduce the number of token and increase the number of channels, wherein the i-th layer feature map has a size ofAfter the feature layers of stage2, stage3 and stage4 are connected, the global environment is further modeled through a Hamburger model, and then a result is output through an MLP layer and an auxiliary layer FCN.
5. The deep learning-based strip mine vector road network map construction method according to claim 1, wherein the optimizing the road area image comprises:
and carrying out open circuit connection on the road area image by adopting morphological operation, carrying out refinement treatment on the road network area by adopting a Zhang-Suen refinement algorithm, and carrying out image super-resolution treatment by adopting a SwinIR model.
6. The deep learning-based strip mine area vector road network map construction method according to claim 1, wherein the process of constructing strip mine road network orthographic image model and digital surface model based on the optimized road area image comprises:
according to the method of homonymy matching, geographic position information in the collected strip mine inclined photographic image is endowed to the optimized road area image, and the optimized road area image with the geographic position information is obtained;
importing the optimized road area image with the geographic position information into ContextCapture software, performing air triangulation calculation on the imported optimized road area image with the geographic position information by ContextCapture software, performing air three optimization on the image with the puncture point according to the puncture point and image control point information, generating a high-resolution triangular mesh model with real texture, restoring the geometric appearance and texture characteristics of an open pit modeling object, and then deriving a road network orthographic image model and a digital surface model in a TIFF format.
7. The method for constructing the vector road network map of the open-pit mining area based on deep learning according to claim 1, wherein the open-pit mining area vector road network map is manufactured by loading the open-pit mining road network orthographic image model and the digital surface model through a GIS platform and vectorizing a road area in the open-pit mining area, and the method comprises the following steps:
importing the strip mine road network orthographic image model and the digital surface model into a GIS platform, and removing the black edges of the strip mine road network orthographic image model;
converting a coordinate system of an opencast road network orthophoto model and a digital surface model into a WGS84 geodetic coordinate system by using a coordinate conversion tool, vectorizing road network areas, road network central lines and road network central line inflection points in a mining area, and attaching elevation information to vectorized road data to finally form a vector road network map of the whole opencast mining area.
8. Open-air mining area vector road network map construction system based on deep learning, characterized by comprising:
and an image acquisition module: the method comprises the steps of acquiring an open-air mining area oblique photographic image containing geographical position information, and preprocessing the acquired open-air mining area oblique photographic image, wherein the preprocessing process comprises low-light image enhancement and noise reduction;
an image extraction module: the method comprises the steps of performing image extraction on preprocessed strip mine inclined photographic images through a trained deep learning network model, and extracting road area images in a strip mine;
and an image optimization module: the method comprises the steps of carrying out optimization processing on the road area image to obtain an optimized road area image;
modeling module: the method is used for constructing an opencut road network orthophoto model and a digital surface model based on the optimized road area image;
and a map construction module: the method is used for vectorizing the road area in the open-pit mining area by utilizing the open-pit mining road network orthophoto model and the digital surface model, and a open-pit mining vector road network map is manufactured.
9. An electronic device, comprising:
one or more processors;
a storage device having one or more programs stored thereon;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the deep learning based strip mine vector road network map construction method of any one of claims 1 to 7.
10. A storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the deep learning based strip mine vector road network map construction method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311723219.XA CN117725141A (en) | 2023-12-14 | 2023-12-14 | Open-pit mining area vector road network map construction method and related device based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311723219.XA CN117725141A (en) | 2023-12-14 | 2023-12-14 | Open-pit mining area vector road network map construction method and related device based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117725141A true CN117725141A (en) | 2024-03-19 |
Family
ID=90210049
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311723219.XA Pending CN117725141A (en) | 2023-12-14 | 2023-12-14 | Open-pit mining area vector road network map construction method and related device based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117725141A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118172363A (en) * | 2024-05-14 | 2024-06-11 | 大连海事大学 | Prostate sporadic cancer prediction method and system based on multitasking learning |
-
2023
- 2023-12-14 CN CN202311723219.XA patent/CN117725141A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118172363A (en) * | 2024-05-14 | 2024-06-11 | 大连海事大学 | Prostate sporadic cancer prediction method and system based on multitasking learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110619283B (en) | Automatic extraction method for unmanned aerial vehicle ortho-image road | |
| US20230135512A1 (en) | Method for updating road signs and markings on basis of monocular images | |
| CN113780296B (en) | Remote sensing image semantic segmentation method and system based on multi-scale information fusion | |
| Abdollahi et al. | Improving road semantic segmentation using generative adversarial network | |
| CN114120102A (en) | Boundary-optimized remote sensing image semantic segmentation method, device, equipment and medium | |
| CN114596500B (en) | Remote sensing image semantic segmentation method based on channel-space attention and DeeplabV plus | |
| CN112287807A (en) | Remote sensing image road extraction method based on multi-branch pyramid neural network | |
| CN106295613A (en) | A kind of unmanned plane target localization method and system | |
| Zhang et al. | Learning from GPS trajectories of floating car for CNN-based urban road extraction with high-resolution satellite imagery | |
| CN117725141A (en) | Open-pit mining area vector road network map construction method and related device based on deep learning | |
| CN113628180B (en) | Remote sensing building detection method and system based on semantic segmentation network | |
| CN113610070A (en) | Landslide disaster identification method based on multi-source data fusion | |
| CN112633140A (en) | Multi-spectral remote sensing image urban village multi-category building semantic segmentation method and system | |
| CN110443862A (en) | Lithologic map filling method and system based on unmanned aerial vehicle and electronic equipment | |
| CN116168246A (en) | Method, device, equipment and medium for identifying waste slag field for railway engineering | |
| CN111461006A (en) | Optical remote sensing image tower position detection method based on deep migration learning | |
| CN112001293A (en) | Remote sensing image ground object classification method combining multi-scale information and coding and decoding network | |
| CN111090717A (en) | Automatic building model based on CGA rule | |
| CN116958827A (en) | Deep learning-based abandoned land area extraction method | |
| CN116630818A (en) | Plateau lake boundary online extraction method and system based on GEE and deep learning | |
| Ruiz-Lendínez et al. | Deep learning methods applied to digital elevation models: state of the art | |
| CN116977750B (en) | Construction method and classification method of land covering scene classification model | |
| Schuegraf et al. | Deep learning for the automatic division of building constructions into sections on remote sensing images | |
| CN113705538A (en) | High-resolution remote sensing image road change detection device and method based on deep learning | |
| CN115797184B (en) | Super-resolution extraction method for surface water body |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |