CN117128977A - High-quality green road path planning method, device and equipment based on double-image fusion - Google Patents

High-quality green road path planning method, device and equipment based on double-image fusion Download PDF

Info

Publication number
CN117128977A
CN117128977A CN202311396981.1A CN202311396981A CN117128977A CN 117128977 A CN117128977 A CN 117128977A CN 202311396981 A CN202311396981 A CN 202311396981A CN 117128977 A CN117128977 A CN 117128977A
Authority
CN
China
Prior art keywords
path
road
green
data
street view
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202311396981.1A
Other languages
Chinese (zh)
Other versions
CN117128977B (en
Inventor
王浩
张天阁
宁晓刚
马晓康
刘若文
曹银璇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chinese Academy of Surveying and Mapping
Original Assignee
Chinese Academy of Surveying and Mapping
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chinese Academy of Surveying and Mapping filed Critical Chinese Academy of Surveying and Mapping
Priority to CN202311396981.1A priority Critical patent/CN117128977B/en
Publication of CN117128977A publication Critical patent/CN117128977A/en
Application granted granted Critical
Publication of CN117128977B publication Critical patent/CN117128977B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01CMEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/20Instruments for performing navigational calculations
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01CMEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/005Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 with correlation of navigation data from several sources, e.g. map or contour matching
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00Computing arrangements using knowledge-based models
    • G06N5/01Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/25Determination of region of interest [ROI] or a volume of interest [VOI]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/10Terrestrial scenes
    • G06V20/182Network patterns, e.g. roads or rivers
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02ATECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A30/00Adapting or protecting infrastructure or their operation
    • Y02A30/60Planning or developing urban green infrastructure

Landscapes

  • Engineering & Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Automation & Control Theory (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses a high-quality green road path planning method, device and equipment based on double-image fusion, which belong to the technical field of geographic information, and comprise the following steps: acquiring OSM road network data, performing road screening and simplifying processing, extracting a road center line and generating street view sampling points; collecting street view image data based on street view sampling points, performing image segmentation processing, and calculating the green vision rate and sky width; DEM data are collected, and road network data are combined to calculate the average gradient of a road; the green vision rate, the road average gradient and the sky opening are weighted and calculated to obtain the path cost; acquiring POI data, giving a starting point and an end point of a planning path, and determining a high-quality green path with the lowest total path cost based on the path cost; and visually displaying the high-quality green road diameter. The invention effectively fuses the street view image and the remote sensing image to plan the high-quality green road path, and provides decision support for the fields of city planning, city greening, traffic planning and the like.

Description

High-quality green road path planning method, device and equipment based on double-image fusion
Technical Field
The invention relates to a high-quality green road path planning method, device and equipment based on double-image fusion, in particular to a high-quality green road path planning method, device and equipment based on double-image fusion of street view images and remote sensing images, and belongs to the technical field of geographic information.
Background
Urban green land is an important component for improving urban ecology and living environment, and plays a positive role in improving urban quality and improving resident happiness. The areas with rich greening generally have lower pedestrian and driving accident rates and are positively correlated with the sense of well-being of the residents, helping to avoid sub-health conditions. However, the existing pedestrian path planning systems are based on the overlooking view angles of satellite remote sensing images, lack cognition on vegetation below vegetation canopy, and therefore difficulty in quantifying the real greening quality of the street at the pedestrian view angles. The existing pedestrian path planning system is mainly based on the overlooking view angle of satellite remote sensing images, and can not accurately evaluate greening conditions below vegetation canopies, so that real greening quality of streets under the pedestrian view angle is difficult to quantify.
The street view data provides a brand new mode for observing the urban physical space, has the advantages of high efficiency and low cost, and is convenient to express the urban building environment (such as road green vision rate) by using street view images. It has the following advantages: (1) large data volume and large coverage; (2) Compared with geographic information data (VGI) of volunteers, street view data has a unified data format, high data quality and small data bias; (3) the acquisition is more convenient and the cost is lower; (4) Most importantly, the street view data reflects the actual landscape of the city from the "human perspective".
Therefore, in order to improve the quality and effect of high-quality green road path planning, the information of street view images and remote sensing images needs to be comprehensively utilized so as to quantify the real greening quality of streets. How to improve the high-quality green road path planning method in the existing method, the advantages of street view images are utilized, the viewing angle of a person is added to improve the quality and effect of the high-quality green road path planning while ensuring the precision, and the method has important significance for improving the quality of the generated lines of residents. However, no high-quality green road path planning method for fusing street view images and remote sensing images exists at present.
Disclosure of Invention
In order to solve the problems, the invention provides a high-quality green road path planning method, device and equipment based on double-image fusion, which can improve the quality and effect of high-quality green road path planning and improve the quality of residential production.
The technical scheme adopted by the invention for solving the technical problems is as follows:
in a first aspect, the method for planning a high-quality green road path based on dual-image fusion provided by the embodiment of the invention comprises the following steps:
acquiring OSM road network data, performing road screening and simplifying processing, extracting a road center line and generating street view sampling points;
Collecting street view image data based on street view sampling points, performing image segmentation processing, and calculating the green vision rate and sky width;
DEM data are collected, and road average gradient is calculated by combining with OSM road network data;
the green vision rate, the road average gradient and the sky opening are weighted and calculated to obtain the path cost;
acquiring POI data, giving a starting point and an end point of a planning path, and determining a high-quality green path with the lowest total path cost based on the path cost;
and visually displaying the high-quality green road diameter.
As a possible implementation manner of this embodiment, the acquiring OSM road network data, performing road screening and simplifying processing, extracting a road center line, and generating a street view sampling point includes:
downloading OSM road network data from an Open Street Map official network, and screening a data source according to the attribute;
performing buffer area and rasterization operation processing on the screened data sources, and converting the data sources into a vectorized simplified road network;
converting line elements into point elements by using an element-to-point tool, and extracting a road center line;
generating street view sampling points every 100 meters according to the simplified road network;
calculating longitude and latitude of each street view sampling point WGS84 coordinate system;
The hundred degree coordinate conversion API is called to convert the WGS84 coordinate system into the bd09II coordinate system.
As a possible implementation manner of this embodiment, the capturing street view image data based on the street view sampling point, performing image segmentation processing, and calculating a green view rate and a sky width, includes:
accessing hundred-degree static image API by using a crawler technology to crawl street view image data of all sampling points in batches;
semantic segmentation is carried out on the street view image by using a deep Labv3 model and a Cityscapes data set, and different objects and areas in the street view image are identified;
the number of vegetation pixels whose area is green land is counted, and a green vision rate GVI is calculated:
(1)
wherein i is the number of street view images crawled by one sampling point, area green To identify the number of pixels as vegetation, area all Total number of pixels per image;
and calculating the sky width by using the sky pixel number and the total pixel number in the street view image.
As a possible implementation manner of this embodiment, the acquiring DEM data and calculating the average gradient of the road with reference to OSM road network data includes:
acquiring DEM data and acquiring elevation values from a DEM layer;
segmenting an OSM road network according to a distance of 100 meters, and adding endpoints at two ends of the segmented road;
Assigning the obtained elevation numerical value to each endpoint;
the road average gradient is calculated using the elevation values of the two end points of each segment road.
As a possible implementation manner of this embodiment, the calculation formula of the path cost is:
route cost=road average gradient×a 1 +green vision rate x a 2 +sky width of opening of sky of width of opening of sky 3 (2)
Wherein a is 1 、a 2 And a 3 Weights of road average gradient, green vision rate and sky opening width, respectively, a 1 +a 2 +a 3 =1, and a 1 +a 3 ≤a 2 ≤1。
As a possible implementation manner of this embodiment, the obtaining the POI data, given the start point and the end point of the planned path, determines a high-quality green path with the lowest overall path cost based on the path cost, includes:
creating a network dataset using arccatalyst or ArcGIS Pro;
selecting POI points of a starting point and an ending point of a planned path or giving coordinates of the starting point and the ending point of the planned path as input of path planning;
defining COST attributes, inputting path COST COST as attribute information by using a network analysis function in Arcgis, and executing a path analysis tool in network analysis to find a high-quality green path with the lowest overall COST.
As a possible implementation manner of this embodiment, the obtaining the POI data, given the start point and the end point of the planned path, determines a high-quality green path with the lowest overall path cost based on the path cost, includes:
Acquiring POI data;
giving the starting point and the end point of a planned path or giving the coordinates of the starting point and the end point of the planned path;
considering the path passing cost, using Dijkstra algorithm to find the path with the lowest path passing cost as the high-quality green path.
As a possible implementation manner of this embodiment, the visually displaying the high-quality green road path includes:
the visualized high-quality green path is presented under the image taking the remote sensing image as the substrate.
In a second aspect, an embodiment of the present invention provides a high-quality green road path planning device based on dual-image fusion, including:
the sampling point generation module is used for collecting OSM road network data, carrying out road screening and simplifying processing, extracting a road center line and generating street view sampling points;
the image segmentation module is used for acquiring street view image data based on street view sampling points, carrying out image segmentation processing and calculating the green vision rate and sky width;
the gradient calculation module is used for acquiring DEM data and calculating the average gradient of the road by combining with OSM road network data;
the path cost calculation module is used for carrying out weighted calculation on the green vision rate, the road average gradient and the sky opening degree to obtain path cost;
The high-quality green road path determining module is used for acquiring POI data of interest points, giving a starting point and an ending point of a planning path, and determining a high-quality green road path with the lowest total path cost based on the path cost;
and the path planning output module is used for visually displaying the high-quality green road path.
In a third aspect, an embodiment of the present invention provides a computer device, including: the system comprises a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, when the computer device runs, the processor and the memory are communicated through the bus, and the processor executes the machine-readable instructions to realize the high-quality green road path planning method based on double-image fusion.
In a fourth aspect, an embodiment of the present invention provides a computer readable storage medium, where the storage medium stores a computer program, where the computer program is configured to perform the method for planning a high-quality green road path based on dual image fusion as described in any one of the above.
The technical scheme of the embodiment of the invention has the following beneficial effects:
the technical scheme of the embodiment of the invention relates to a high-quality green road path planning method based on double-image fusion, which comprises the following steps: acquiring OSM road network data, performing road screening and simplifying processing, extracting a road center line and generating street view sampling points; collecting street view image data based on street view sampling points, performing image segmentation processing, and calculating the green vision rate and sky width; DEM data are collected, and road average gradient is calculated by combining with OSM road network data; the green vision rate, the road average gradient and the sky opening are weighted and calculated to obtain the path cost; acquiring POI data, giving a starting point and an end point of a planning path, and determining a high-quality green path with the lowest total path cost based on the path cost; and visually displaying the high-quality green road diameter. The invention effectively fuses the street view image and the remote sensing image to plan a high-quality green road path, provides reference for green travel of citizens, and provides decision support for the fields of urban planning, urban greening, traffic planning and the like. According to the invention, the green vision rate index, the road average gradient index, the sky opening degree index and other factors are taken into the path planning, so that the importance of multi-index comprehensive consideration on high-quality green road path planning is emphasized, the relationship among different factors is balanced, the path with good comprehensive performance is provided, and the path planning effect and the user satisfaction are improved.
According to the invention, by collecting and analyzing multi-source data, factors such as road conditions, beautiful landscapes, comfort level, traffic convenience and the like are comprehensively considered, and an optimized path planning scheme is provided, so that the quality and effect of high-quality green road path planning can be improved, people can select more proper paths, enjoy better travel experience, and meanwhile, urban sustainable development and improvement of resident life quality are promoted.
Drawings
FIG. 1 is a flow chart of a method for planning a high-quality green road path based on dual image fusion, according to an exemplary embodiment;
FIG. 2 is a schematic diagram of a high-quality green road path planning device based on dual image fusion according to an exemplary embodiment;
FIG. 3 is a detailed implementation flow chart of a quality green road path plan for merging street view images with remote sensing images, according to an example embodiment;
fig. 4 (a) and 4 (b) are diagrams showing the result of a high-quality green road path planning process defined by the present invention according to an exemplary embodiment (wherein fig. 4 (a) is a schematic diagram showing the result of calculation of the average gradient of the road, and fig. 4 (b) is a schematic diagram showing the result of calculation of the green vision rate);
FIG. 5 is a schematic diagram illustrating an interface of a high-quality green road path planning system delineated by the present invention in accordance with an exemplary embodiment;
Fig. 6 is a schematic diagram showing a result of planning a green road path of good quality defined by the present invention according to an exemplary embodiment.
Detailed Description
In order to more clearly illustrate the technical features of the solution of the present invention, the present invention will be described in detail below with reference to the following detailed description and the accompanying drawings.
Example 1
As shown in fig. 1, the method for planning a high-quality green road path based on dual-image fusion provided by the embodiment of the invention comprises the following steps:
acquiring OSM road network data, performing road screening and simplifying processing, extracting a road center line and generating street view sampling points;
collecting street view image data based on street view sampling points, performing image segmentation processing, and calculating the green vision rate and sky width;
DEM data are collected, and road average gradient is calculated by combining with OSM road network data;
the green vision rate, the road average gradient and the sky opening are weighted and calculated to obtain the path cost;
acquiring POI data, giving a starting point and an end point of a planning path, and determining a high-quality green path with the lowest total path cost based on the path cost;
and visually displaying the high-quality green road diameter.
As a possible implementation manner of this embodiment, the acquiring OSM road network data, performing road screening and simplifying processing, extracting a road center line, and generating a street view sampling point includes:
Downloading OSM road network data from an Open Street Map official network, and screening a data source according to the attribute; primary roads (primary), secondary roads (secondary), tertiary roads (tertiary roads), residential roads, living-street roads (trunk roads) are selected as main data sources, and points on roads such as overpasses, viaducts and tunnels are removed to further reduce the generation of low-quality images, so that a large amount of shielding is often generated on the roads;
performing buffer area and rasterization operation processing on the screened data sources, and converting the data sources into a vectorized simplified road network; the purpose of road network simplification is to merge multiple lanes of the same road. Taking 50m as a radius to make a buffer area for the road network, performing binarization processing to obtain a grid image, and then converting grid data into vector data through a vectorization tool to obtain a simplified road network;
converting line elements into point elements by using an element-to-point tool, and extracting a road center line; in ArcGIS, a road element is converted into a line element using an "element-to-line" tool, and a line element is converted into a point element using an "element-to-point" tool, thereby extracting a road center line.
Generating street view sampling points every 100 meters according to the simplified road network;
calculating longitude and latitude of each street view sampling point WGS84 coordinate system;
the hundred degree coordinate conversion API is called to convert the WGS84 coordinate system into the bd09II coordinate system.
As a possible implementation manner of this embodiment, the capturing street view image data based on the street view sampling point, performing image segmentation processing, and calculating a green view rate and a sky width, includes:
accessing hundred-degree static image API by using a crawler technology to crawl street view image data of all sampling points in batches; accessing corresponding street view image provider hundred-degree static image APIs by using a crawler technology, crawling street view images of all sampling points in batches, analyzing and storing returned street view image data, and storing the returned street view image data as an image file;
semantic segmentation is carried out on the street view image by using a deep Labv3 model and a Cityscapes data set, and different objects and areas in the street view image are identified;
the number of vegetation pixels whose area is green land is counted, and a green vision rate GVI is calculated:
(1)
wherein i is the number of street view images crawled by one sampling point, area green To identify the number of pixels as vegetation, area all Total number of pixels per image;
and calculating the sky width by using the sky pixel number and the total pixel number in the street view image. The street view image is derived from a hundred-degree street view image.
As a possible implementation manner of this embodiment, the acquiring DEM data and calculating the average gradient of the road with reference to OSM road network data includes:
acquiring DEM data and acquiring elevation values from a DEM layer;
segmenting an OSM road network according to a distance of 100 meters, and adding endpoints at two ends of the segmented road;
assigning the obtained elevation numerical value to each endpoint;
the road average gradient is calculated using the elevation values of the two end points of each segment road.
As a possible implementation manner of this embodiment, the calculation formula of the path cost is:
route cost=road average gradient×a 1 +green vision rate x a 2 +sky width of opening of sky of width of opening of sky 3 (2)
Wherein a is 1 、a 2 And a 3 Weights of road average gradient, green vision rate and sky opening width, respectively, a 1 +a 2 +a 3 =1, and a 1 +a 3 ≤a 2 ≤1。
As one possible implementation of this embodiment, in a specific implementation of the invention,the weight values are as follows: a, a 1 =0.2,a 2 =0.6,a 3 =0.2。
As a possible implementation manner of this embodiment, the obtaining the POI data, given the start point and the end point of the planned path, determines a high-quality green path with the lowest overall path cost based on the path cost, includes:
creating a network dataset using arccatalyst or ArcGIS Pro;
Selecting POI points of a starting point and an ending point of a planned path or giving coordinates of the starting point and the ending point of the planned path as input of path planning;
defining COST attributes, inputting path COST COST as attribute information by using a network analysis function in Arcgis, and executing a path analysis tool in network analysis to find a high-quality green path with the lowest overall COST.
As a possible implementation manner of this embodiment, the visually displaying the high-quality green road path includes:
the visualized high-quality green path is presented under the image taking the remote sensing image as the substrate.
The invention effectively fuses the street view image and the remote sensing image to plan a high-quality green road path, provides reference for green travel of citizens, and provides decision support for the fields of urban planning, urban greening, traffic planning and the like. According to the invention, the green vision rate index, the road average gradient index, the sky opening degree index and other factors are taken into the path planning, so that the importance of multi-index comprehensive consideration on high-quality green road path planning is emphasized, the relationship among different factors is balanced, the path with good comprehensive performance is provided, and the path planning effect and the user satisfaction are improved.
According to the invention, by collecting and analyzing multi-source data, factors such as road conditions, beautiful landscapes, comfort level, traffic convenience and the like are comprehensively considered, and an optimized path planning scheme is provided, so that the quality and effect of high-quality green road path planning can be improved, people can select more proper paths, enjoy better travel experience, and meanwhile, urban sustainable development and improvement of resident life quality are promoted.
Example 2
As shown in fig. 2, a high-quality green road path planning device based on dual-image fusion provided by an embodiment of the present invention includes:
the sampling point generation module is used for collecting OSM road network data, carrying out road screening and simplifying processing, extracting a road center line and generating street view sampling points;
the image segmentation module is used for acquiring street view image data based on street view sampling points, carrying out image segmentation processing and calculating the green vision rate and sky width;
the gradient calculation module is used for acquiring DEM data and calculating the average gradient of the road by combining with OSM road network data;
the path cost calculation module is used for carrying out weighted calculation on the green vision rate, the road average gradient and the sky opening degree to obtain path cost;
the high-quality green road path determining module is used for acquiring POI data of interest points, giving a starting point and an ending point of a planning path, and determining a high-quality green road path with the lowest total path cost based on the path cost;
And the path planning output module is used for visually displaying the high-quality green road path.
As a possible implementation manner of this embodiment, the high-quality green road path determining module is specifically configured to:
creating a network dataset using arccatalyst or ArcGIS Pro;
selecting POI points of a starting point and an ending point of a planned path or giving coordinates of the starting point and the ending point of the planned path as input of path planning;
defining COST attributes, inputting path COST COST as attribute information by using a network analysis function in Arcgis, and executing a path analysis tool in network analysis to find a high-quality green path with the lowest overall COST.
As shown in fig. 3, the process of planning a high-quality green road path by fusing street view images and remote sensing images by using the device of the invention is as follows.
Step 1: and acquiring OSM road network data and street view image data.
The specific implementation process of the step 1 comprises the following substeps:
step 1.1: road network data acquisition: downloading road network data in an Open Street Map official network, selecting proper Map data in a downloading area, and screening and acquiring road network data for acquiring Street view images according to attribute requirements of table 1;
table 1: OSM road network attribute selection and Chinese-English comparison table
Step 1.2: road network simplification: performing buffer area and rasterization operation on the screened road network by using ArcGIS software, then converting the road network into a vectorized simplified road network by using ArcScan tool and converting line elements into point elements by using element-to-point tool, thereby extracting a road center line and facilitating subsequent path planning and analysis;
step 1.3: generating street view sampling points: generating street view sampling points every 100 meters according to the simplified road network by using a line generating point tool in the ArcGIS;
step 1.4: longitude and latitude calculation: for each generated street view sampling point, calculating the longitude and latitude of the street view sampling point by using a proper geographic coordinate system (such as WGS 84);
step 1.5: coordinate conversion: the hundred degree coordinate conversion API is called, and the WGS84 coordinate system is used in the sample point coordinates before conversion and converted into the bd09II coordinate system.
Step 1.6: obtaining street view image data: and accessing a corresponding hundred-degree static diagram API of the street view image data provider by writing a web crawler program. And analyzing the returned street view image data, and storing the street view image data as an image file or adopting a proper data storage and processing method.
Step 2: and calculating the green vision rate, the sky opening degree and the average gradient of the road.
The specific implementation process of the step 2 comprises the following substeps:
step 2.1: semantic segmentation: semantic segmentation of street view images was performed using the deep labv3 model and the Cityscapes dataset. The pretrained deep labv3 model is used and fine-tuned on the Cityscapes dataset to identify different objects and regions in the street view image, identifying greenbelts and non-greenbelts.
Step 2.2: counting the number of pixels identified as vegetation and further calculating the green vision rate:
(1)
i is the number of street view images crawled by one sampling point, area green To identify the number of pixels as vegetation, area all Total number of pixels per image; as shown in fig. 4 (b), the calculated green vision rate is shown in fig. 4 (b), and the green vision rate of different vegetation is shown by different color bands.
Step 2.3: calculating sky width: the number of sky pixels in the image and the total number of pixels are used for calculation. The sky proportion is obtained by counting the number of pixels identified as sky and dividing the number of pixels of the whole image, and the sky proportion is calculated.
Step 2.4: calculating the average gradient of the road: segmenting a road network according to a distance of 100 meters, adding endpoints at two ends of the segmented road network, assigning elevation values of a Digital Elevation Model (DEM) layer to the endpoints, calculating the average gradient of a section of road by using the elevation values of the two endpoints, wherein the calculation result of the average gradient of the road is shown in fig. 4 (a), and lines with different thickness represent roads with different average gradients in fig. 4 (a).
Step 3: and constructing a weighted path cost.
The specific implementation process of the step 3 comprises the following substeps:
step 3.1: the calculation of the path weighted cost involves three metrics: gradient index, green vision rate index and sky opening index, and weighting calculation is carried out on different factors of the path according to the indexes. The specific calculation method can be described by the following formula:
route cost=gradient index×0.2+green vision rate index×0.6+sky opening width index×0.2
The green vision rate index weight is 0.6, which indicates that the green vision rate has a larger influence in path cost calculation; the slope index and sky-opening index weight are both 0.2, indicating that they have less impact on path cost.
Step 4: merging path planning of street view images and remote sensing images and visualizing path planning results;
the specific implementation process of the step 4 comprises the following substeps:
step 4.1: as shown in fig. 5, the "create network dataset" tool of arccatalyst or ArcGIS Pro is used to create the network dataset. The POI points of the starting point and the ending point or the coordinates of the starting point and the ending point are selected at the operation interface, the COST attribute is defined as the input of the path planning, the path COST is input as attribute information, and the path analysis tool in the network analysis is executed to find the high-quality green path with the lowest overall COST, as shown in fig. 6.
Step 4.2: visualization of results: the visualized path planning results are presented to the user or decision maker in a remote sensing image based image, as shown in fig. 6, for analysis and evaluation. By observing the performance of the path on street view and remote sensing images, the feasibility and environmental characteristics of the path can be known, which is helpful for better understanding the path planning result and making adjustments or decisions when needed.
The method is tested by adopting real data, and the result of high-quality green road strength planning by combining street view images and remote sensing images is reasonable, so that the method is proved to be correct and effective in theory and is also practical in practical application.
The method is characterized by comprising the following steps of performing experiments on Beijing urban areas, wherein earth surface coverage data are derived from geographical national condition census data achievements, POI points are derived from Goldmap, street view images are derived from hundred degree static image API, road network data are derived from OSM official networks, and the specific implementation steps of the method are as follows:
step 1: preprocessing road network data and generating sampling points;
(1) And selecting specific attributes in OSM (OpenStreetMap) -path network data through attribute screening by utilizing an ArcGIS tool, wherein the specific attributes include primary roads, secondary roads, tertiary roads, residential roads, living-area streets and trunk roads. These road classes represent roads of different categories and importance.
(2) And screening road sections according to whether road names contain keywords such as overpass, tunnel, overpass and the like by using a wild screening function in ArcGIS, and deleting the relevant road sections. This makes it possible to exclude a particular road section unsuitable as a path.
(3) The road network is subjected to buffer processing, an appropriate radius (for example, 50 m) is selected, and the road line is buffered and regionalized, and converted into a raster image. In this way, the road can be expressed as a binarized raster image, and subsequent processing is convenient.
(4) The grid version of the road network is converted into vector data using an Arcscan et al Guan Shi quantization tool. Therefore, the simplified road network can be obtained, and the line elements are converted into the point elements by using an element-to-point tool, so that the road center line is extracted, and the subsequent path planning and analysis are convenient.
(5) Sampling points are generated on the simplified road network at certain intervals (every 100 m) by using a line generation point tool in ArcGIS. These sampling points will be used to represent specific locations and features of street view images;
(6) For each generated point, its longitude and latitude in WGS84 coordinate system is calculated. This ensures that the sampling points have accurate geographical coordinate information.
(7) The coordinates of each sampling point are converted into a bd09II coordinate system by using a hundred degree API coordinate conversion interface. The method is used for keeping consistency with the coordinate system of the hundred-degree map, and is convenient for acquiring and displaying the subsequent street view images.
Step 2: street view image acquisition and image segmentation;
(1) The corresponding street view image provider hundred degree static map API is accessed by using a crawler technology (such as BeautifluSoup library or Scrapy framework in Python) to crawl the street view images of each sampling point in batches, and the acquisition parameters are shown in Table 2. The returned street view image data is parsed and saved, which may be stored as an image file or using other suitable data storage and processing methods.
Table 2: street view image acquisition parameters
(2) Semantic segmentation is performed on the street view image using the deep labv3 model and the Cityscapes dataset to identify different objects and regions in the street view image. Semantic segmentation is carried out on street view images by adopting a deep Labv3 model and a Cityscapes data set, the number of pixels classified into vegetation is counted, and the green vision rate GVI is calculated by using the following formula;
(1)
i is the number of street view images crawled by one sampling point, area green To identify the number of pixels as vegetation, area all Total number of pixels per image;
step 3: calculating sky width: the number of sky pixels in the image and the total number of pixels are used for calculation. The sky proportion is obtained by counting the number of pixels identified as sky and dividing the number of pixels of the whole image, and the sky proportion is calculated.
Step 4: calculating the average gradient of the road: segmenting the road network according to the distance of 100 meters, adding endpoints at two ends of the segmented road network, assigning the elevation numerical value of a Digital Elevation Model (DEM) layer to the endpoints, and calculating the average gradient of a section of road by utilizing the elevation numerical values of the two endpoints.
Step 5: path weighted cost: the calculation of the path weighted cost involves three metrics: gradient index, green vision rate index and sky opening index, and weighting calculation is carried out on different factors of the path according to the indexes. The specific calculation method can be described by the following formula:
route cost=gradient index×0.2+green vision rate index×0.6+sky opening width index×0.2
The green vision rate index weight is 0.6, which indicates that the green vision rate has a larger influence in path cost calculation; the slope index and sky-opening index weight are both 0.2, indicating that they have less impact on path cost.
Step 6: the "create network dataset" tool of arccatalyst or arcgipro is used to create the network dataset. And selecting POI points of the starting point and the ending point or giving coordinates of the starting point and the ending point at an operation interface, defining COST attributes as input of path planning, inputting path COST COST as attribute information, and executing a path analysis tool in network analysis to find a high-quality green path with the lowest overall COST. The visualized path planning result is presented to a user or a decision maker under an image taking the remote sensing image as a base, and the best green path between the starting point and the end point is obtained.
The invention effectively fuses the street view image and the remote sensing image to plan a high-quality green road path, provides reference for green travel of citizens, and provides decision support for the fields of urban planning, urban greening, traffic planning and the like. According to the invention, the green vision rate index, the road average gradient index, the sky opening degree index and other factors are taken into the path planning, so that the importance of multi-index comprehensive consideration on high-quality green road path planning is emphasized, the relationship among different factors is balanced, the path with good comprehensive performance is provided, and the path planning effect and the user satisfaction are improved.
Example 3
Unlike embodiment 1, the acquiring the POI data, given the start point and the end point of the planned route, determines a high-quality green path having the lowest overall path cost based on the path cost, including:
acquiring POI data;
giving the starting point and the end point of a planned path or giving the coordinates of the starting point and the end point of the planned path;
considering the path passing cost, using Dijkstra algorithm to find the path with the lowest path passing cost as the high-quality green path.
Example 4
The difference from embodiment 2 is that the high-quality green road diameter determination module is specifically configured to:
acquiring POI data;
giving the starting point and the end point of a planned path or giving the coordinates of the starting point and the end point of the planned path;
considering the path passing cost, using Dijkstra algorithm to find the path with the lowest path passing cost as the high-quality green path.
Example 5
The embodiment of the invention provides a computer device, which comprises: the system comprises a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, when the computer device runs, the processor and the memory are communicated through the bus, and the processor executes the machine-readable instructions to realize the high-quality green road path planning method based on double-image fusion.
The above memory and the processor are all general-purpose memories and processors, and are not particularly limited herein, and when the processor runs the computer program stored in the memory, the above-mentioned high-quality green road path planning method based on double-image fusion can be executed.
Those skilled in the art will appreciate that the structure of the computer device is not limiting of the computer device and may include more or fewer components than shown, or may combine certain components, or may split certain components, or may be arranged in different components.
In some embodiments, the computer device may also include a touch screen for displaying a graphical user interface (e.g., a launch interface for an application, etc.) and receiving user actions with respect to the graphical user interface (e.g., launch operations with respect to an application, etc.). The touch screen may include a display panel, which may be configured in the form of an LCD (Liquid Crystal Display ), an OLED (Organic Light-Emitting Diode), or the like, and a touch panel, which may collect a touch or non-touch operation of a user thereon or thereabout and generate a preset operation instruction, such as an operation of the user on or thereabout using any suitable object or accessory such as a finger, a stylus, or the like. In addition, the touch panel may include two parts, a touch detection device and a touch controller. The touch detection device detects the touch azimuth and the touch gesture of a user, detects signals brought by touch operation and transmits the signals to the touch controller; the touch controller receives touch information from the touch detection device, converts the touch information into information which can be processed by the processor, sends the information to the processor, and can receive and execute commands sent by the processor. In addition, the touch panel can be realized by adopting various types such as resistance type, capacitance type, infrared ray, surface acoustic wave and the like, and can also be realized by adopting any technology developed in the future. Further, the touch panel may overlay the display panel, and a user may operate on or near the touch panel overlaid on the display panel according to a graphical user interface displayed by the display panel, and upon detection of an operation thereon or thereabout, the touch panel is transferred to the processor to determine a user input, and the processor then provides a corresponding visual output on the display panel in response to the user input. The touch panel and the display panel may be implemented as two separate components or may be integrated.
Corresponding to the starting method of the application program, the embodiment of the application also provides a computer readable storage medium, wherein the storage medium stores a computer program, and the computer program is used for executing the high-quality green road path planning method based on double-image fusion.
The starting device of the application program provided by the embodiment of the application can be specific hardware on the equipment or software or firmware installed on the equipment. The device provided by the embodiment of the present application has the same implementation principle and technical effects as those of the foregoing method embodiment, and for the sake of brevity, reference may be made to the corresponding content in the foregoing method embodiment where the device embodiment is not mentioned. It will be clear to those skilled in the art that, for convenience and brevity, the specific operation of the system, apparatus and unit described above may refer to the corresponding process in the above method embodiment, which is not described in detail herein.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. The above-described apparatus embodiments are merely illustrative, for example, the division of modules is merely a logical function division, and there may be additional divisions in actual implementation, and for example, multiple modules or components may be combined or integrated into another system, or some features may be omitted, or not performed.
The modules illustrated as separate components may or may not be physically separate, and components shown as modules may or may not be physical modules, i.e., may be located in one place, or may be distributed over a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional module in the embodiment provided by the application may be integrated in one processing module, or each module may exist alone physically, or two or more modules may be integrated in one module.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the above embodiments, it should be understood by those skilled in the art that modifications and equivalents may be made to the specific embodiments of the present invention without departing from the spirit and scope of the present invention, and it should be covered by the claims of the present invention.

Claims (10)

1. A high-quality green road path planning method based on double image fusion is characterized by comprising the following steps:
acquiring OSM road network data, performing road screening and simplifying processing, extracting a road center line and generating street view sampling points;
collecting street view image data based on street view sampling points, performing image segmentation processing, and calculating the green vision rate and sky width;
DEM data are collected, and road average gradient is calculated by combining with OSM road network data;
the green vision rate, the road average gradient and the sky opening are weighted and calculated to obtain the path cost;
acquiring POI data, giving a starting point and an end point of a planning path, and determining a high-quality green path with the lowest total path cost based on the path cost;
and visually displaying the high-quality green road diameter.
2. The method for planning a high-quality green road path based on double image fusion according to claim 1, wherein the acquiring OSM road network data, performing road screening and simplifying processing, extracting a road center line and generating street view sampling points comprises:
downloading OSM road network data from an Open Street Map official network, and screening a data source according to the attribute;
performing buffer area and rasterization operation processing on the screened data sources, and converting the data sources into a vectorized simplified road network;
Converting line elements into point elements by using an element-to-point tool, and extracting a road center line;
generating street view sampling points every 100 meters according to the simplified road network;
calculating longitude and latitude of each street view sampling point WGS84 coordinate system;
the hundred degree coordinate conversion API is called to convert the WGS84 coordinate system into the bd09II coordinate system.
3. The method for planning a high-quality green road path based on double-image fusion according to claim 1, wherein the step of acquiring street view image data based on street view sampling points, performing image segmentation processing and calculating a green vision rate and sky opening degree comprises the steps of:
accessing hundred-degree static image API by using a crawler technology to crawl street view image data of all sampling points in batches;
semantic segmentation is carried out on the street view image by using a deep Labv3 model and a Cityscapes data set, and different objects and areas in the street view image are identified;
the number of vegetation pixels whose area is green land is counted, and a green vision rate GVI is calculated:
(1)
wherein i is the number of street view images crawled by one sampling point, area green To identify the number of pixels as vegetation, area all Total number of pixels per image;
and calculating the sky width by using the sky pixel number and the total pixel number in the street view image.
4. The method for planning a high-quality green road path based on double image fusion according to claim 1, wherein the steps of collecting DEM data and calculating the average gradient of the road by combining OSM road network data include:
acquiring DEM data and acquiring elevation values from a DEM layer;
segmenting an OSM road network according to a distance of 100 meters, and adding endpoints at two ends of the segmented road;
assigning the obtained elevation numerical value to each endpoint;
the road average gradient is calculated using the elevation values of the two end points of each segment road.
5. The method for planning a path of a high-quality green road based on double image fusion according to claim 1, wherein the calculation formula of the path cost is:
route cost=road average gradient×a 1 +green vision rate x a 2 +sky width of opening of sky of width of opening of sky 3 (2)
Wherein a is 1 、a 2 And a 3 Weights of road average gradient, green vision rate and sky opening width, respectively, a 1 +a 2 +a 3 =1, and a 1 +a 3 ≤a 2 ≤1。
6. The method for planning a path of a green road based on dual image fusion according to any one of claims 1 to 5, wherein the obtaining the POI data, giving a starting point and an ending point of a planned path, determining a path of a green road with a highest quality and a lowest overall path cost based on path costs comprises:
Creating a network dataset using arccatalyst or ArcGIS Pro;
selecting POI points of a starting point and an ending point of a planned path or giving coordinates of the starting point and the ending point of the planned path as input of path planning;
defining COST attributes, inputting path COST COST as attribute information by using a network analysis function in Arcgis, and executing a path analysis tool in network analysis to find a high-quality green path with the lowest overall COST.
7. The method for planning a path of a green road based on dual image fusion according to any one of claims 1 to 5, wherein the obtaining the POI data, giving a starting point and an ending point of a planned path, determining a path of a green road with a highest quality and a lowest overall path cost based on path costs comprises:
acquiring POI data;
giving the starting point and the end point of a planned path or giving the coordinates of the starting point and the end point of the planned path;
considering the path passing cost, using Dijkstra algorithm to find the path with the lowest path passing cost as the high-quality green path.
8. High-quality green road path planning device based on double image fuses, its characterized in that includes:
the sampling point generation module is used for collecting OSM road network data, carrying out road screening and simplifying processing, extracting a road center line and generating street view sampling points;
The image segmentation module is used for acquiring street view image data based on street view sampling points, carrying out image segmentation processing and calculating the green vision rate and sky width;
the gradient calculation module is used for acquiring DEM data and calculating the average gradient of the road by combining with OSM road network data;
the path cost calculation module is used for carrying out weighted calculation on the green vision rate, the road average gradient and the sky opening degree to obtain path cost;
the high-quality green road path determining module is used for acquiring POI data of interest points, giving a starting point and an ending point of a planning path, and determining a high-quality green road path with the lowest total path cost based on the path cost;
and the path planning output module is used for visually displaying the high-quality green road path.
9. A computer device, comprising: the system comprises a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, when the computer device runs, the processor and the memory are communicated through the bus, and the processor executes the machine-readable instructions to realize the high-quality green road path planning method based on double-image fusion as claimed in any one of claims 1 to 7.
10. A computer-readable storage medium, wherein the storage medium stores a computer program for executing the high-quality green road path planning method based on the double image fusion as set forth in any one of claims 1 to 7.
CN202311396981.1A 2023-10-26 2023-10-26 High-quality green road path planning method, device and equipment based on double-image fusion Active CN117128977B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311396981.1A CN117128977B (en) 2023-10-26 2023-10-26 High-quality green road path planning method, device and equipment based on double-image fusion

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311396981.1A CN117128977B (en) 2023-10-26 2023-10-26 High-quality green road path planning method, device and equipment based on double-image fusion

Publications (2)

Publication Number Publication Date
CN117128977A true CN117128977A (en) 2023-11-28
CN117128977B CN117128977B (en) 2024-01-19

Family

ID=88858603

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311396981.1A Active CN117128977B (en) 2023-10-26 2023-10-26 High-quality green road path planning method, device and equipment based on double-image fusion

Country Status (1)

Country Link
CN (1) CN117128977B (en)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9217649B1 (en) * 2012-10-11 2015-12-22 The Arizona Board Of Regents On Behalf Of The University Of Arizona Systems and methods for providing topographical graphs and travel routes based on perceived exertion
CN108304470A (en) * 2017-12-27 2018-07-20 浙江工业大学 A kind of city underground paths planning method based on ArcGIS
CN110956089A (en) * 2019-11-04 2020-04-03 李苗裔 Historical block walking performance measuring method based on ICT technology
CN113792650A (en) * 2021-09-13 2021-12-14 上海市建筑科学研究院有限公司 Method and system for measuring green vision rate
CN114459488A (en) * 2022-03-11 2022-05-10 中国民航大学 Indoor and outdoor integrated emergency disposal path planning method based on three-dimensional fusion road network
CN114863706A (en) * 2022-04-26 2022-08-05 山东高速集团有限公司 Expressway-oriented vehicle-road cooperative automatic driving simulation test system and method
CN116187638A (en) * 2022-11-24 2023-05-30 洛阳市规划建筑设计研究院有限公司 Green channel line selection method based on GIS software
CN116204576A (en) * 2023-05-04 2023-06-02 北京城建交通设计研究院有限公司 Method and system for generating GTFS format data by public transportation data

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9217649B1 (en) * 2012-10-11 2015-12-22 The Arizona Board Of Regents On Behalf Of The University Of Arizona Systems and methods for providing topographical graphs and travel routes based on perceived exertion
CN108304470A (en) * 2017-12-27 2018-07-20 浙江工业大学 A kind of city underground paths planning method based on ArcGIS
CN110956089A (en) * 2019-11-04 2020-04-03 李苗裔 Historical block walking performance measuring method based on ICT technology
CN113792650A (en) * 2021-09-13 2021-12-14 上海市建筑科学研究院有限公司 Method and system for measuring green vision rate
CN114459488A (en) * 2022-03-11 2022-05-10 中国民航大学 Indoor and outdoor integrated emergency disposal path planning method based on three-dimensional fusion road network
CN114863706A (en) * 2022-04-26 2022-08-05 山东高速集团有限公司 Expressway-oriented vehicle-road cooperative automatic driving simulation test system and method
CN116187638A (en) * 2022-11-24 2023-05-30 洛阳市规划建筑设计研究院有限公司 Green channel line selection method based on GIS software
CN116204576A (en) * 2023-05-04 2023-06-02 北京城建交通设计研究院有限公司 Method and system for generating GTFS format data by public transportation data

Also Published As

Publication number Publication date
CN117128977B (en) 2024-01-19

Similar Documents

Publication Publication Date Title
JP6937355B2 (en) Crack analysis device, crack analysis method, and crack analysis program
Dhanani et al. From the axial line to the walked line: Evaluating the utility of commercial and user-generated street network datasets in space syntax analysis
CN115409069A (en) Village and town building identification method, classification method, device, electronic equipment and medium
Khayyal et al. Creation and spatial analysis of 3D city modeling based on GIS data
Ye et al. Urban commerce distribution analysis based on street view and deep learning
CN113139529B (en) Linear cultural heritage exploration method and system, storage medium and electronic equipment
Zhang et al. Using street view images to identify road noise barriers with ensemble classification model and geospatial analysis
Hatta Antah et al. Perceived usefulness of airborne LiDAR technology in road design and management: A review
CN109447436A (en) A kind of ring lake Parkway Landscape Vision Impact Assessment method
Frajer et al. A historical curiosity or a source of accurate spatial information on historical land use? The issue of accuracy of old cadastres in the example of Josephian Cadastre from the Habsburg Empire
CN107944089A (en) A kind of plot limit for height analysis system and its analysis method for regarding corridor because of the present circumstance
Lawrence Early urbanism in the northern Fertile Crescent: a comparison of regional settlement trajectories and millennial landscape change
CN117128977B (en) High-quality green road path planning method, device and equipment based on double-image fusion
Chen et al. Evaluating implied urban nature vitality in San Francisco: An interdisciplinary approach combining census data, street view images, and social media analysis
Huang et al. Simultaneous update of high-resolution land-cover mapping attempt: Wuhan and the surrounding satellite cities cartography using L2HNet
Pérez et al. Sustainable acoustics: A smart cities approach
Mao et al. Population spatialization at building scale based on residential population index—A case study of Qingdao city
Khine et al. Change analysis of indices (NDWI, NDVI, NDBI) for Mawlamyine City area using google earth engine
CN106339423A (en) Method and device for dynamically updating sugarcane planting information
Kazemi Knowledge-based generalisation of road networks
Lai Three-dimensional Visualisation of Cultural Landscape under the Perspective of Culture and Tourism Integration
US20230358561A1 (en) Method, apparatus, and computer program product for lane geometry generation based on graph estimation
Hu et al. Method for selecting viewpoints of architectural heritage landscapes
Eschner Real-time avalanche risk visualization on a large-scale geospatial dataset
Naidu CONCEPT OF GEOGRAPHIC INFORMATION SYSTEM FOR A GEOINFOMATICS ENGINEER

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant