CN117725141A - Method and related devices for constructing vector road network map of open-pit mining area based on deep learning - Google Patents

Abstract

The invention discloses a deep learning-based open-pit mining area vector road network map construction method and a related device, wherein the method comprises the steps of collecting open-pit mining area oblique photographic images containing geographic position information, and carrying out low-light image enhancement and noise reduction pretreatment on the open-pit mining area oblique photographic images; carrying out image extraction on the preprocessed image through the trained deep learning network model, and extracting road area images in the open sky mining area; carrying out optimization processing on the road area image to obtain an optimized road area image; constructing an opencut road network orthophoto model and a digital surface model based on the optimized road area image; and carrying out vectorization on a road area in the open-pit mining area by using the open-pit mining road network orthographic image model and the digital surface model, and manufacturing to obtain an open-pit mining vector road network map. The map obtained by the invention can carry out road network matching on mining vehicles with errors in position, thereby meeting the requirement of high-precision positioning of vehicles in open-air mining areas.

Description

Method and related devices for constructing vector road network map of open-pit mining area based on deep learning

Technical field

The invention belongs to the technical field of vector road network map drawing, and also belongs to the field of computer application. It relates to image semantic segmentation in deep learning, and in particular to a deep learning-based vector road network map construction method for open-pit mining areas and related devices.

Background technique

Nowadays, in the environment of vigorous development of smart mine construction, the intelligent dispatching system and driverless technology of mining trucks are inseparable from the accurate high-precision road network map of open-pit mines, which plays an important and leading role in stope transportation. And with the implementation and application of driverless technology in open-pit mines, the construction of high-precision road network maps for mining areas has become increasingly important. Accurate open-pit mine road network map information provides significant application significance for open-pit mine unmanned intelligent traffic navigation, intelligent dispatching, digital map updates, path planning, emergency support and other aspects.

However, with the continuous movement of open-pit mine mining and discharge, vehicle transportation routes also change, and minerals need to be transported to corresponding unloading points, resulting in the open-pit mine road network changing very frequently during the mining process. If the open-pit mine road network If the map update is not timely enough, it may affect the vehicle's intelligent dispatching system and driverless driving, thereby affecting the production status of the mining area. Therefore, the construction of open pit mine road network maps must be efficient and accurate.

At present, methods based on deep learning have achieved great results in road extraction in open-pit mines. It can not only completely retain the road surrounding environment information, but also extract road areas efficiently and accurately. However, the road network data they constructed is not vectorized and cannot reflect the longitude and latitude information of each area of the open-pit mine. Therefore, it cannot be used for real-time monitoring of vehicle movement trajectories, and it cannot directly provide intelligent dispatching and driverless driving for mining trucks. Provide effective help. How to regularly and accurately update and produce open-pit mine vector road network maps in complex environmental backgrounds is a major difficulty in the construction of smart mines.

Contents of the invention

In order to update and produce open-pit mine road network maps efficiently and accurately, and taking into account the shortcomings of extracting road areas based on deep learning technology in the past, the purpose of the present invention is to provide a deep learning-based vector road network map construction method for open-pit mine areas and related devices.

The technical solutions adopted by the present invention are as follows:

The deep learning-based vector road network map construction method for open-pit mining areas includes the following processes:

Collect oblique photography images of the open-pit mining area that contain geographical location information, and preprocess the collected oblique photography images of the open-pit mining area. The pre-processing process includes low-light image enhancement and noise reduction processing;

The pre-processed oblique photography images of the open-pit mining area are used to extract images through the trained deep learning network model, and the road area images in the open-pit mining area are extracted;

Perform optimization processing on the road area image to obtain an optimized road area image;

Construct an open-pit mine road network orthophoto model and a digital surface model based on the optimized road area image;

The open-pit mine road network orthophoto model and digital surface model are used to vectorize the road areas in the open-pit mine area, and a vector road network map of the open-pit mine is produced.

Preferably, when collecting oblique photography images of open-pit mining areas that contain geographical location information, they are taken at a preset height above the ground and at a ground overlap rate of 75% at an orthographic angle and at an oblique angle of 45 degrees.

Preferably, the establishment process of the deep learning network model includes:

The deep learning network model is established by combining cross-shaped window self-attention, local enhanced position encoding and lightweight Hamburger decoder;

The training process of the deep learning network model includes:

Collect oblique photography images of open-pit mining areas that contain geographical location information, and preprocess the collected oblique photography images of open-pit mining areas;

Based on the labelme tool, the road areas in the pre-processed oblique photography images of the open-pit mining area are manually labeled and a road data set is established;

Divide the road data set into a training set, a verification set and a test set, in which more than 50% of the images containing temporary roads are divided into the training set, and other images are randomly divided into the verification set and the test set;

The binary cross-entropy loss function BCE is combined with the Focal Loss used to solve the class imbalance problem to establish the loss function in the CSHformer network model training process;

Based on the CSHformer network model, the deep learning network model is trained using the training set, verification set and test set.

Preferably, the deep learning network model framework is as follows:

The input image of size 512×512×3 is convolved through Convolutional Token Embeeding to obtain patch tokens of size 128×128, and then different hierarchical expressions are generated through 4 stages. Each stage contains N _t consecutive Cswin transformer block, a convolution operation with a convolution kernel size of 3×3 and a step size of 2 is used between each adjacent stage to reduce the number of tokens and increase the number of channels. The size of the i-th layer feature map is After connecting the feature layers of stage2, stage3, and stage4, the global environment is further modeled through the Hamburger model, and then the results are output through the MLP layer and the auxiliary layer FCN.

Preferably, optimizing the road area image includes:

Morphological operations are used to perform disconnection and connection on the road area images, the Zhang-Suen thinning algorithm is used to refine the road network area, and the SwinIR model is used for image super-resolution processing.

Preferably, the process of constructing the open-pit mine road network orthophoto model and digital surface model based on the optimized road area image includes:

According to the matching method of the same name, the geographical location information in the collected oblique photography images of the open-pit mining area is assigned to the optimized road area image, and the optimized road area image with the geographical location information is obtained;

Import the optimized road area image with geographical location information into the ContextCapture software. The ContextCapture software performs aerial triangulation calculation on the imported optimized road area image with geographical location information. Based on the punctum and image control points information, perform aerial triangulation optimization on the punctuated image, generate a high-resolution triangular mesh model with real texture, restore the geometric appearance and texture characteristics of the open-pit mine modeling object, and then export the road network orthophoto model in TIFF format and digital surface models.

Preferably, the open-pit mine road network orthoimage model and digital surface model are loaded through the GIS platform, the road areas in the open-pit mining area are vectorized, and the open-pit mine vector road network map is produced, including the following processes:

Import the open-pit mine road network orthophoto model and digital surface model into the GIS platform to remove the image black edges of the open-pit mine road network orthophoto model;

Use coordinate conversion tools to convert the coordinate systems of the open-pit mine road network orthophoto model and digital surface model into the WGS84 geodetic coordinate system, and then vectorize the road network area, road network centerline, and road network centerline inflection points within the mining area. And the elevation information is added to the vectorized road data, finally forming a vector road network map of the entire open-pit mining area.

The present invention also provides a deep learning-based open-pit mining area vector road network map construction system, which is used to implement the above-mentioned deep learning-based open-pit mining area vector road network map construction method, including:

Image acquisition module: used to collect oblique photography images of open-pit mining areas that contain geographical location information, and pre-process the collected oblique photography images of open-pit mining areas. The pre-processing process includes low-light image enhancement and noise reduction processing;

Image extraction module: used to extract the pre-processed oblique photography images of the open-pit mining area through the trained deep learning network model, and extract the road area image in the open-pit mining area;

Image optimization module: used to optimize the road area image to obtain an optimized road area image;

Modeling module: used to construct an open-pit mine road network orthophoto model and a digital surface model based on the optimized road area image;

Map construction module: used to vectorize the road areas in the open-pit mining area using the orthophoto model and digital surface model of the open-pit mine road network, and produce a vector road network map of the open-pit mine.

The invention also provides an electronic device, including:

one or more processors;

A storage device on which one or more programs are stored;

When the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the deep learning-based open-pit mining area vector road network map construction method of the present invention as described above.

The present invention also provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the deep learning-based vector road network map construction method for open-pit mining areas of the present invention is implemented as described above.

The invention has the following beneficial effects:

The present invention constructs a vector road network map of the open-pit mining area based on the deep learning vector road network map construction method of the open-pit mining area. In this way, by collecting oblique photographic images of the open-pit mining area containing geographical location information, the open-pit mining area can be updated in a timely manner through the method of the present invention. Vector road network map. This vector road network map can reflect the longitude and latitude information of each area of the open-pit mine. Therefore, it can be used for real-time monitoring of vehicle movement trajectories, and can directly provide effective help for the intelligent dispatching and driverless driving of mining trucks. . The map construction method of the present invention can greatly save manpower and material resources, and extract road information in open-pit mining areas more efficiently and accurately.

Description of the drawings

Figure 1 is a flow chart of a method for constructing a vector road network map in an open-pit mining area based on deep learning in an embodiment of the present invention.

Figure 2 is an example diagram of image preprocessing results in an embodiment of the present invention, in which (a) is the original image, (b) is the image after noise reduction, and (c) is the image after low-light image enhancement.

Figure 3 is a framework diagram of the CSHformer network model constructed in the embodiment of the present invention.

Figure 4 is a schematic diagram of the calculation of cross-shaped window self-attention in the embodiment of the present invention, in which (a) is a cross-shaped window diagram, (b) is a calculation process diagram of horizontal stripe self-attention, and (c) is a calculation process diagram of horizontal stripes and The calculations of the vertical stripe self-attention are connected together to output the final calculation result map of the cross-shaped window self-attention.

Figure 5 is a schematic diagram of the morphological disconnection process in the embodiment of the present invention.

Figure 6 is an example of the result of optimization processing of the image after extracting the road area in the embodiment of the present invention, where (a) is the image after extracting the road area, (b) is the image after the roads are connected, and (c) is the road network details. The image after transformation, (d) is the image after super-resolution.

Figure 7 is a flow chart of road network data vectorization operations in the embodiment of the present invention.

Figure 8 is a structural block diagram of a vector road network map construction system for an open-pit mining area based on deep learning in an embodiment of the present invention.

Detailed ways

The present invention will be described in detail below with reference to the drawings and examples.

This invention is based on a deep learning method for constructing vector road network maps in open-pit mining areas, and constructs a deep learning network model-CSHformer that integrates cross-shaped window self-attention, local enhanced position encoding (LePE) and lightweight Hamburger decoder. It has less calculation amount and can also ensure higher precision extraction effect. The road images extracted from the open-pit mine are optimized through disconnection, thinning and super-resolution. During the specific implementation, the original longitude and latitude information of the UAV oblique photography image was assigned to the extracted road image, and an orthophoto model and a digital surface model of the open-pit mine road network were constructed. By vectorizing the road network area, road network The center line and the inflection point of the road network center line realize the production of open-pit mine vector road network map.

Specifically, the deep learning-based vector road network map construction method for open-pit mining areas of the present invention includes the following steps:

Step 1: Use drones to collect oblique photography images of open-pit mining areas, and perform low-light image enhancement and noise reduction processing on them. Based on the labelme tool, the road areas in the images are manually annotated to establish a road data set; specifically, through unmanned The camera captures special light and shadow effects in the early morning, records images under normal lighting conditions at noon, and captures open-pit mine road images under cloudy conditions with soft light, lighter shadows and outstanding details. At the same time, the drone must shoot at an orthogonal angle to make accurate maps at a height of 200 meters from the ground and with a ground overlap rate of 75%, and at an oblique angle of 45 degrees to obtain more three-dimensional images. sense and detail.

The resolution of road images collected by drones is 5472×3648, and each image records its corresponding geographical location information. The coordinate system used is the CGCS2000 geodetic coordinate system. Considering the performance of the computer, first divide the image into 6 equal parts by dividing it into 3 columns and 2 rows, and then resize the image resolution from 1824×1824 to 512×512. The naming format of the divided image is: Original image name_01~06.

Due to the poor natural environment of open-pit mining areas and complex mining operations, such as truck transportation, excavation and other mechanical operations, mining sites are often accompanied by dust, blowing sand and other phenomena, and sometimes even affected by weather. These objective conditions will affect the image quality. . After analysis of the original data set, it was found that some images were blurry and dim, which would affect the deep learning network model's learning and extraction of road features in the image. This article uses the Low-Light Image Enhancement (LIME) method based on Retinex modeling to restore the open-pit mine road information in a darker lighting environment as clearly as possible, and uses the Restormer model to effectively remove the noise points of the open-pit mine road image.

Road extraction is a two-classification task. The road area in the image is manually labeled with the "road" label based on the labelme tool, and the background is labeled with the "background" label. Analyze the proportion of fixed roads, semi-fixed roads, mobile roads and temporary roads contained in open-pit mine road images and the pixel differences with the surrounding environment. Considering that the boundary between temporary roads and the surrounding environment is not obvious, in order to allow the deep learning network model to learn more detailed features of temporary roads, images containing 50% or more of temporary roads are divided into training sets, and other images are randomly divided. Finally, 4558 training sets, 1140 verification sets, and 1424 test sets were obtained for pre-training of the deep learning network model.

Step 2: Establish a deep learning network model, conduct pre-training based on the open-pit mine road data set, and use the pre-trained model to extract the road area in the open-pit mining area; specifically, by combining cross-shaped window self-attention and local enhanced position coding ( LePE) and lightweight Hamburger decoder to build a deep learning network model-CSHformer. This model uses CSWin Transformer Block to extract multi-level features, and forms an encoder-decoder structure with the lightweight Hamburger model. By integrating low-level feature information with high-level feature information, the loss of feature information can be effectively reduced, and it has fewer features. While reducing the amount of calculations, it can also ensure a higher-precision extraction effect, which can meet the conditions of open-pit mines that require frequent and accurate updates of road network maps.

Step 3: For the extracted road image, use morphological operations for disconnection, use Zhang-Suen thinning algorithm to refine the road network area, and use the SwinIR model for image super-resolution processing; specifically, for feature information The CSHformer network model still cannot segment the extremely obscure temporary road in the open-pit mine. The road network extraction results will have problems such as road disconnections, holes, and unclear road network skeleton structure. In order to construct an orthophoto model of the open-pit mine road network, it is necessary to restore the extracted image of the road network to the original resolution size of 5472×3648. Direct resize operation will damage the image information and requires further optimization. The specific optimization process is as follows:

(1) Open-circuit connection: Based on morphological image processing technology, 3×3 square structural elements are used to measure the structure of the input image. For images that require short-circuit connection, an expansion operation is first performed to connect the previously separated road areas, but it will also be strengthened at the same time. Road edge features. In order to restore the shape of the original area to the greatest extent, an etching operation is used to repair the edge portion that expanded during the expansion process.

(2) Refinement processing: The Zhang-Suen refinement algorithm is used to refine the road area extracted from the open pit mine, and extract the road skeleton information, making the road structure of the final map more intuitive and clear.

(3) Super-resolution: The resolution of the road images extracted from the open-pit mine is 512×512. Since the subsequent construction of the orthophoto model of the open-pit mine road network requires the geographical location information in the original images collected by the drone to be assigned to In the corresponding images, first merge the six equally divided images according to the image name matching method, and then use the SwinIR model to super-resolve the merged images to restore the original image information as much as possible.

Step 4: Construct the open-pit mine road network orthophoto model (DOM) and digital surface model (DSM) based on the optimized open-pit mine road images; specifically, the open-pit mine road images extracted through the CSHformer network model will lose geographical location information , according to the matching method of the same name, the geographical information in the original image taken by the drone is assigned to the extracted and optimized open-pit mine road image, which is used to construct the orthophoto of the open-pit mine road network; ContextCapture software is used to import the imported Aerial triangulation calculations are performed on open-pit mine road images with geographical location information. Based on the punctum and image control point information, aerial triangulation optimization is performed on the punctuated image to generate a high-resolution triangular mesh model with real textures, which is accurate Restore the geometric appearance and texture characteristics of open-pit mine modeling objects, and export the road network orthophoto model (DOM) and digital surface model (DSM) in TIFF format.

Step 5: Load DOM and DSM through the GIS platform, vectorize the road network area, and create a vector road network map of the open-pit mine. Specifically, the open-pit mine road network orthophoto model (DOM) and digital surface model (DSM) were imported into the GIS platform to remove the black edges of the DOM model. In order to match the base map (world map) in the GIS platform, use the coordinate conversion tool to convert the coordinate systems of the DOM and DSM into the WGS84 geodetic coordinate system. The road network area, road network centerline, and road network centerline inflection points in the mining area are vectorized, and the elevation information is added to the vectorized road network data, ultimately forming a vector road network map of the entire open-pit mining area.

It can be seen from the above solution that the present invention uses deep learning technology to quickly and accurately extract road network information in an environment where the edges of unstructured roads in open-pit mines are not obvious and there is a lot of background interference. Based on the vectorized road network data, the road network matching of mining area vehicles with position errors can be performed to meet the needs of high-precision positioning of vehicles in open-pit mining areas.

Example

This embodiment takes the road in an open-pit mining area in Luoyang, Henan as the research object. As shown in Figure 1, the method flow chart for constructing the vector road network map of this open-pit mine is mainly divided into five steps.

Step 1: Use a drone to capture special light and shadow effects in the early morning, record images under normal lighting conditions at noon, and capture open-pit mine road images under cloudy conditions with soft light, lighter shadows, and outstanding details. At the same time, the drone must shoot at an orthogonal angle to make accurate maps at a height of 200 meters from the ground and with a ground overlap rate of 75%, and at an oblique angle of 45 degrees to obtain more three-dimensional images. sense and detail.

The resolution of road images collected by drones is 5472×3648, and each image records the geographical location information corresponding to the image. The coordinate system used is the CGCS2000 geodetic coordinate system. Considering the performance of the computer, first divide the image into 6 equal parts by dividing it into 3 columns and 2 rows, and then resize the image resolution from 1824×1824 to 512×512. The naming format of the divided image is: Original image name_01~06.

Since the low-light areas and noise in the image may affect the subsequent road feature extraction process of the deep learning network model, the Low-Light Image Enhancement (LIME) method based on Retinex modeling is used to restore the image in the darker lighting environment as clearly as possible. Open-pit mine road information, and use the Restormer model to effectively remove noise points from the open-pit mine road image, as shown in Figure 2, which is an example of the preprocessed results.

Road extraction is a two-classification task. The road area in the image is manually labeled with the "road" label based on the labelme tool, and the background is labeled with the "background" label. Analyze the proportion of fixed roads, semi-fixed roads, mobile roads and temporary roads contained in open-pit mine road images and the pixel differences with the surrounding environment. Considering that the boundary between temporary roads and the surrounding environment is not obvious, in order to allow the deep learning network model to learn more detailed features of temporary roads, images containing 50% or more of temporary roads are divided into training sets, and other images are randomly divided into verification sets and test set. Finally, 4558 training sets, 1140 verification sets, and 1424 test sets were obtained for pre-training of the deep learning network model.

Step 2: Establish a deep learning network model CSHformer to train the open-pit mine road data set, and use the pre-trained model to extract the road area in the open-pit mine area.

The overall architecture of the CSHformer network model constructed is shown in Figure 3. The input image size is 512×512×3, which is obtained through Convolutional Token Embeeding (using a convolution operation with a convolution kernel size of 7×7 and a step size of 4) The size of patch tokens is 128×128, and the latitude of each token is C. Then different hierarchical expressions are generated through four stages. Each stage contains N _t consecutive Cswin transformer blocks. A convolution operation with a convolution kernel size of 3×3 and a step size of 2 is used between each adjacent stage. To reduce the number of tokens and increase the number of channels, the size of the i-th layer feature map is After connecting the feature layers of stage2, stage3, and stage4, the global environment is further modeled through the Hamburger model, and then the results are output through the MLP layer and the auxiliary layer FCN.

The calculation of cross-shaped window self-attention is shown in Figure 4. The input feature X∈R ^(H×W)×C is linearly projected into K tokens, and horizontal and vertical stripes of self-attention are performed in parallel on each token. . The width of the horizontal and vertical windows is sw. As the level deepens, the width of the window will also increase, thereby adjusting the learning ability and computational complexity of the network model. In order to make sw divisible by the input features H and W, after experiments, this embodiment sets the sw of the four stages to 1, 2, 8, and 8 respectively. As shown in Figure 4(b), the calculation of the k-th token of horizontal stripe self-attention is as follows:

Among them, X∈[X ¹ , ^{X 2} , _… ,X ^M ], K, Q, K, and V are the queries, keys, and values of each token respectively.

Locally enhanced position encoding (LePE) can generate position encoding for input images of any resolution size, impose position information in each Transformer module, and directly operate on the values in the attention mechanism. In order to reduce the amount of calculation and capture local attention, use Depth-wise Conv to convolve the value, and then add the result to the Self-Attention result. This not only retains the relative position information, but also enhances the local sensing bias. To improve the generalization ability of the model, the calculation of LePE is as follows:

The calculation of vertical stripe self-attention can be derived by analogy, and the calculation of the k-th token can be expressed as V-Attention _k (X). Finally, the outputs of H-Attention _k (X) and V-Attention _k (X) are connected in parallel to obtain the final calculation result of the cross-shaped window self-attention.

A decoder based on the lightweight Hamburger model learns the global context of image features by modeling global contextual information. The Hamburger model consists of two linear transformation layers containing a matrix decomposition model M, where Hams is the objective function for modeling "global information". _{Assume that} ^given _{​ ​} ^​ _​ ,c _n ]∈R ^r×n , X can be defined as Among them, low-rank "global information"/> The residual term (noise, redundancy or missing) E∈R ^d×n , then the Hamburger model can be defined as:

Y＝Z+BN(H(Z))

H(Z)＝W _u M(W _l Z)

Among them, the input features Lower Bread linear transformation/> Upper BreadLinear Transformation L in the matrix decomposition model M represents the reconstruction error, which can be derived from the element distribution of the residual term E. R ₁ and R ₂ represent the regularization of the dictionary matrix D and coefficient matrix C respectively, which can be derived from their prior distribution. H(Z) is fused with input features through BatchNormalization (BN), and finally outputs Y.

Open-pit mine road network extraction can be regarded as a two-class classification task. The road part to be extracted is regarded as foreground information, and the problem of imbalance between road and background categories can be clearly found. In order to reduce the impact of data imbalance on the training effect, the two-class classification task is The cross-entropy loss function BCE is combined with the Focal Loss used to solve the category imbalance problem to establish the loss function Loss for network model training:

Loss＝W ₁ L _BCE + W ₂ L _Focal

Among them, W ₁ and W ₂ are the loss weight coefficients of BCE Loss and Focal Loss, The model predicts the probability that the sample is a road, y is the true label of the sample, α is the weight of the road category, and γ is the adjustment factor. In this embodiment, α=0.8, γ=2, and W ₁ =W ₂ =1.

According to the above semantic segmentation network model, training is carried out based on the open-pit mine road data set. The specific parameters of the training process are shown in Table 2.

Table 2 Model training parameters

By analyzing the training results of the network model, the mAcc, mIoU, and mPA values of the network model can be obtained, which reach 95.75%, 86.54%, and 92.47% respectively. This model has a smaller amount of calculation and higher extraction accuracy. . Based on the trained network model, all road areas in the open-pit mining area are semantically segmented. After extracting the road network, the road edge contours are clear, and the road areas in the open-pit mining area can be extracted accurately and completely.

Step 3: Further optimize the extracted road network image; in order to make the result of the open-pit mine road network map more accurate, perform optimization processes such as disconnection, thinning, and super-resolution on the extracted open-pit mine road network map. . As shown in Figure 5, it is a schematic diagram of the disconnection and connection process based on morphological operations, and as shown in Figure 6, it is an example of the results after optimization processing.

Step 4: According to the matching method of the same name, the geographical information in the original image captured by the drone is assigned to the extracted and optimized open-pit mine road image, which is used to construct the orthophoto of the open-pit mine road network and import it using ContextCapture software. Aerial triangulation calculation is performed on the open-pit mine road image with geographical location information, and aerial triangulation optimization is performed on the punctuated image based on the punctum and image control point information. Generate high-resolution triangular mesh models with real textures, accurately restore the geometric appearance and texture characteristics of open-pit mine modeling objects, and export road network orthophoto models (DOM) and digital surface models (DSM) in TIFF format.

Step 5: Import the open-pit mine road network orthophoto model (DOM) and digital surface model (DSM) into the GIS platform to remove the black edges of the DOM model. At the same time, in order to match the base map (world map) in the GIS platform, the coordinate conversion tool is used to convert the DOM and DSM coordinate systems into the WGS84 geodetic coordinate system. The DOM model cannot directly represent the geographical elements and geometric features of specific roads. It is necessary to vectorize the road network area, road network centerline, and road network centerline inflection points based on the GIS platform, and describe their spatial position relationships through a coordinate system. As shown in Figure 7, the specific operation process of making an open-pit mine vector road network map through the GIS platform is as follows:

(1) Load the sky map in the GIS platform. In order to check the alignment of the subsequently imported raster data model with the geographical location of the world-class electronic map;

(2) Import the digital surface model DSM. After the vectorization operation, vector data such as points, lines, and planes will only record the latitude and longitude coordinates, and the elevation information will not be directly extracted. The purpose of importing ground elevation data is to subsequently extract the elevation information into the vector data;

(3) Import the orthophoto model DOM. In order to check the alignment of the vectorized data with the actual image;

(4) Import the road network model DOM. In order to vectorize the road network raster data in the future;

(5) Import the road network centerline model DOM. In order to subsequently vectorize the road network centerline raster data;

(6) Remove black edges from the image. Open the layer properties panel and set "Additional No Data Value" in "Transparency" to 0;

(7) Raster vectorization. Click "Raster" - "Convert" - "Raster Vectorization", select the raster layer to be vectorized, and select the location where the vectorized file will be saved;

(8) Extract vertices from the center line of the road network. Click "Vector" - "Geometry Tools" - "Extract Vertices", select the centerline layer of the vector road network, and select the file save location after extracting the vertices;

(9) Extract elevation information. In the "Vector Geometry" toolbox, click "Add (set Z value from raster)", select the layer from which you want to extract elevation information, select the DSM layer, and select the location where the file will be saved after the elevation information is extracted;

After the road network point, line, and surface data are vectorized, indicators such as the north compass, scale, and geographical location range are set to create a vector road network map of the open-pit mining area.

In summary, the present invention can efficiently and accurately update and produce an open-pit mine vector road network map. The final extracted and optimized results can well cover the road areas in the open-pit mine area. The road network map produced based on the present invention can be practically used in open-pit mines. In the intelligent construction of mining areas, the operating trajectories of vehicles in open-pit mining areas are monitored in real time, and vehicles with position errors can be map-matched to obtain high-precision positioning information of vehicles, which provides accurate implementation of intelligent scheduling and path planning in open-pit mining areas. Lay the foundation.

As shown in Figure 8, the present invention also provides a system for realizing the above-mentioned deep learning-based vector road network map construction for open-pit mining areas. The system includes:

Image acquisition module: used to collect oblique photography images of open-pit mining areas that contain geographical location information, and pre-process the collected oblique photography images of open-pit mining areas. The pre-processing process includes low-light image enhancement and noise reduction processing;

Image extraction module: used to extract the pre-processed oblique photography images of the open-pit mining area through the trained deep learning network model, and extract the road area image in the open-pit mining area;

Image optimization module: used to optimize the road area image to obtain an optimized road area image;

Modeling module: used to construct an open-pit mine road network orthophoto model and a digital surface model based on the optimized road area image;

Map construction module: used to vectorize the road areas in the open-pit mining area using the orthophoto model and digital surface model of the open-pit mine road network, and produce a vector road network map of the open-pit mine.

Embodiments of the present invention also provide corresponding equipment and computer-readable storage media for implementing the solution provided by the embodiments of the present invention.

Wherein, the device includes a memory and a processor, the memory is used to store instructions or codes, and the processor is used to execute the instructions or codes, so that the device executes the trusted method described in any embodiment of this application. DCS host computer trusted status synchronization method.

Claims (10)

1. A deep learning-based vector road network map construction method for open-pit mining areas, which is characterized by including the following processes: Collect oblique photography images of the open-pit mining area that contain geographical location information, and preprocess the collected oblique photography images of the open-pit mining area. The pre-processing process includes low-light image enhancement and noise reduction processing; The pre-processed oblique photography images of the open-pit mining area are used to extract images through the trained deep learning network model, and the road area images in the open-pit mining area are extracted; Perform optimization processing on the road area image to obtain an optimized road area image; Construct an open-pit mine road network orthophoto model and a digital surface model based on the optimized road area image; The open-pit mine road network orthophoto model and digital surface model are used to vectorize the road areas in the open-pit mine area, and a vector road network map of the open-pit mine is produced. 2. The deep learning-based open-pit mining area vector road network map construction method according to claim 1, characterized in that when collecting oblique photography images of the open-pit mining area containing geographical location information, the preset height from the ground and the ground overlap rate are Photographed at 75% orthogonal angle and 45-degree oblique angle. 3. The method for constructing a vector road network map in an open-pit mining area based on deep learning according to claim 1, characterized in that the establishment process of the deep learning network model includes: The deep learning network model is established by combining cross-shaped window self-attention, local enhanced position encoding and lightweight Hamburger decoder; The training process of the deep learning network model includes: Collect oblique photography images of open-pit mining areas that contain geographical location information, and preprocess the collected oblique photography images of open-pit mining areas; Based on the labelme tool, the road areas in the pre-processed oblique photography images of the open-pit mining area are manually labeled and a road data set is established; Divide the road data set into a training set, a verification set and a test set, in which more than 50% of the images containing temporary roads are divided into the training set, and other images are randomly divided into the verification set and the test set; The binary cross-entropy loss function BCE is combined with the Focal Loss used to solve the class imbalance problem to establish the loss function in the CSHformer network model training process; Based on the CSHformer network model, the deep learning network model is trained using the training set, verification set and test set. 4. The deep learning-based vector road network map construction method for open-pit mining areas according to claim 1 or 3, characterized in that the deep learning network model framework is as follows: The input image of size 512×512×3 is convolved through Convolutional Token Embeeding to obtain patch tokens of size 128×128, and then different hierarchical expressions are generated through 4 stages. Each stage contains N _t consecutive Cswin transformer block, a convolution operation with a convolution kernel size of 3×3 and a step size of 2 is used between each adjacent stage to reduce the number of tokens and increase the number of channels. The size of the i-th layer feature map is After connecting the feature layers of stage2, stage3, and stage4, the global environment is further modeled through the Hamburger model, and then the results are output through the MLP layer and the auxiliary layer FCN. 5. The deep learning-based vector road network map construction method for open-pit mining areas according to claim 1, characterized in that optimizing the road area image includes: Morphological operations are used to perform disconnection and connection on the road area images, the Zhang-Suen thinning algorithm is used to refine the road network area, and the SwinIR model is used for image super-resolution processing. 6. The deep learning-based vector road network map construction method for open-pit mine areas according to claim 1, characterized in that the process of constructing an open-pit mine road network orthophoto model and a digital surface model based on the optimized road area image include: According to the matching method of the same name, the geographical location information in the collected oblique photography images of the open-pit mining area is assigned to the optimized road area image, and the optimized road area image with the geographical location information is obtained; Import the optimized road area image with geographical location information into the ContextCapture software. The ContextCapture software performs aerial triangulation calculation on the imported optimized road area image with geographical location information. Based on the punctum and image control points information, perform aerial triangulation optimization on the punctuated image, generate a high-resolution triangular mesh model with real texture, restore the geometric appearance and texture characteristics of the open-pit mine modeling object, and then export the road network orthophoto model in TIFF format and digital surface models. 7. The method for constructing a vector road network map in an open-pit mine area based on deep learning according to claim 1, characterized in that the orthophoto model and digital surface model of the open-pit mine road network are loaded through a GIS platform to map the open-pit mine road network in the open-pit mine area. The road area is vectorized to produce a vector road network map of the open-pit mine, which includes the following processes: Import the open-pit mine road network orthophoto model and digital surface model into the GIS platform to remove the image black edges of the open-pit mine road network orthophoto model; Use coordinate conversion tools to convert the coordinate systems of the open-pit mine road network orthophoto model and digital surface model into the WGS84 geodetic coordinate system, and then vectorize the road network area, road network centerline, and road network centerline inflection points within the mining area. And the elevation information is added to the vectorized road data, finally forming a vector road network map of the entire open-pit mining area. 8. A vector road network map construction system for open-pit mining areas based on deep learning, which is characterized by: Image acquisition module: used to collect oblique photography images of open-pit mining areas that contain geographical location information, and pre-process the collected oblique photography images of open-pit mining areas. The pre-processing process includes low-light image enhancement and noise reduction processing; Image extraction module: used to extract the pre-processed oblique photography images of the open-pit mining area through the trained deep learning network model, and extract the road area image in the open-pit mining area; Image optimization module: used to optimize the road area image to obtain an optimized road area image; Modeling module: used to construct an open-pit mine road network orthophoto model and a digital surface model based on the optimized road area image; Map construction module: used to vectorize the road areas in the open-pit mining area using the orthophoto model and digital surface model of the open-pit mine road network, and produce a vector road network map of the open-pit mine. 9. An electronic device, characterized in that it includes: one or more processors; A storage device on which one or more programs are stored; When the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the deep learning-based open-pit mine vector path according to any one of claims 1 to 7. Web map construction method. 10. A storage medium, characterized in that a computer program is stored thereon, wherein when the computer program is executed by a processor, the vector path of an open-pit mine area based on deep learning according to any one of claims 1 to 7 is implemented. Web map construction method.