WO2020073936A1 - 地图要素提取方法、装置及服务器 - Google Patents

地图要素提取方法、装置及服务器 Download PDF

Info

Publication number
WO2020073936A1
WO2020073936A1 PCT/CN2019/110259 CN2019110259W WO2020073936A1 WO 2020073936 A1 WO2020073936 A1 WO 2020073936A1 CN 2019110259 W CN2019110259 W CN 2019110259W WO 2020073936 A1 WO2020073936 A1 WO 2020073936A1
Authority
WO
WIPO (PCT)
Prior art keywords
map
image
feature
depth
target scene
Prior art date
Application number
PCT/CN2019/110259
Other languages
English (en)
French (fr)
Inventor
舒茂
陈偲
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Priority to EP19871308.3A priority Critical patent/EP3779358B1/en
Publication of WO2020073936A1 publication Critical patent/WO2020073936A1/zh
Priority to US17/075,134 priority patent/US11380002B2/en

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
    • G01C11/02—Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
    • G01C21/34—Route searching; Route guidance
    • G01C21/36—Input/output arrangements for on-board computers
    • G01C21/3602—Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
    • G01C11/04—Interpretation of pictures
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/38—Electronic maps specially adapted for navigation; Updating thereof
    • G01C21/3804—Creation or updating of map data
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/38—Electronic maps specially adapted for navigation; Updating thereof
    • G01C21/3804—Creation or updating of map data
    • G01C21/3807—Creation or updating of map data characterised by the type of data
    • G01C21/3811—Point data, e.g. Point of Interest [POI]
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/38—Electronic maps specially adapted for navigation; Updating thereof
    • G01C21/3804—Creation or updating of map data
    • G01C21/3807—Creation or updating of map data characterised by the type of data
    • G01C21/3815—Road data
    • G01C21/3822—Road feature data, e.g. slope data
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/86—Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/93—Lidar systems specially adapted for specific applications for anti-collision purposes
    • G01S17/931—Lidar systems specially adapted for specific applications for anti-collision purposes of land vehicles
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/05—Geographic models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G06T7/337—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving reference images or patches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/521—Depth or shape recovery from laser ranging, e.g. using interferometry; from the projection of structured light
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G06T7/593—Depth or shape recovery from multiple images from stereo images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/7715—Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/64—Three-dimensional objects
    • G06V20/647—Three-dimensional objects by matching two-dimensional images to three-dimensional objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle

Definitions

  • Embodiments of the present application relate to the field of computer technology, and in particular, to a method, device, and server for extracting map elements.
  • a high-precision map is a map used to assist driving, semi-automatic driving, or unmanned driving, and consists of a series of map elements.
  • Map elements include: lane lines, ground signs, road teeth, fences, traffic signs and other elements.
  • Various embodiments of the present application provide a method, device, and server for extracting map elements.
  • a method for extracting map elements executed by an electronic device, includes: acquiring a laser point cloud and an image of a target scene, the target scene including at least one element entity corresponding to a map element; performing the laser point cloud and all Registration between the images to obtain a depth map of the image; performing image segmentation on the depth map of the image to obtain a segmented image of the map element in the depth map; according to the laser point cloud and The registration relationship between the images converts the two-dimensional position of the segmented image in the depth map to the three-dimensional of the map element in the target scene.
  • a map element extraction device includes: an image acquisition module for acquiring a laser point cloud and an image of a target scene, the target scene including at least one element entity corresponding to a map element; a depth map construction module for Perform registration between the laser point cloud and the image to obtain a depth map of the image; an image segmentation module is used to perform image segmentation on the depth map of the image to obtain the map element at the depth A segmented image in the figure; a position conversion module for converting the two-dimensional position of the segmented image in the depth map to the map element according to the registration relationship between the laser point cloud and the image The three-dimensional position in the target scene.
  • a server includes a processor and a memory.
  • Computer-readable instructions are stored on the memory.
  • the map element extraction method described above is implemented.
  • a computer-readable storage medium has stored thereon a computer program, and when the computer program is executed by a processor, the map element extraction method described above is implemented.
  • FIG. 1 is a schematic diagram of an implementation environment involved in a method for extracting map elements according to an embodiment of the present application
  • Fig. 2 is a block diagram of a hardware structure of a server according to an exemplary embodiment
  • Fig. 3 is a flowchart illustrating a method for extracting map elements according to an exemplary embodiment
  • FIG. 4 is a schematic diagram of a laser point cloud and an image of a target scene before registration according to the embodiment corresponding to FIG. 3;
  • FIG. 5 is a schematic diagram of the laser point cloud and image of the target scene after registration according to the embodiment corresponding to FIG. 3;
  • Fig. 6 is a flowchart illustrating steps of registering the laser point cloud and the image to obtain depth information corresponding to pixels in the image according to an exemplary embodiment
  • step 350 is a flowchart of step 350 in an embodiment corresponding to the embodiment of FIG. 3;
  • Fig. 8 is a flowchart of a process of constructing a semantic segmentation network according to an exemplary embodiment
  • FIG. 9 is a schematic diagram of a segmented image of a map element in a depth map related to the embodiment corresponding to FIG. 7;
  • step 351 is a flowchart of step 351 in an embodiment corresponding to the embodiment of FIG. 7;
  • FIG. 11 is a schematic structural diagram of a residual neural network involved in the embodiment corresponding to FIG. 10;
  • Fig. 12 is a flowchart of another method for extracting map elements according to an exemplary embodiment
  • FIG. 13 is a schematic diagram of displaying lane line elements in a target scene matching map related to the embodiment corresponding to FIG. 12;
  • FIG. 14 is a schematic diagram of displaying ground mark elements in a target scene matching map related to the embodiment corresponding to FIG. 12;
  • Fig. 15 is a structural diagram of a device for extracting map elements according to an exemplary embodiment
  • Fig. 16 is a structural diagram of a server according to an exemplary embodiment.
  • FIG. 1 is a schematic diagram of an implementation environment involved in a method for extracting map elements according to an embodiment of the present application.
  • the implementation environment includes a user side 110 and a server side 130.
  • the user terminal 110 is deployed in vehicles, airplanes, and robots, and may be a desktop computer, a laptop computer, a tablet computer, a smart phone, a palmtop computer, a personal digital assistant, a navigator, a smart computer, etc., which is not limited herein.
  • the user terminal 110 and the server terminal 130 establish a network connection in advance through a wireless or wired network, etc., and realize data transmission between the user terminal 110 and the server terminal 130 through this network connection.
  • the transmitted data includes: a high-precision map of the target scene.
  • the server 130 may be a single server or a server cluster composed of multiple servers. As shown in FIG. 1, it may also be a cloud computing center composed of multiple servers.
  • the server is an electronic device that provides users with background services.
  • the background services include: map feature extraction service, high-precision map generation service, and so on.
  • the server 130 can extract map elements from the laser point cloud and image of the target scene to obtain the three-dimensional position of the map element in the target scene.
  • the map element After obtaining the three-dimensional position of the map element in the target scene, the map element can be displayed on the target scene map according to the three-dimensional position through the display screen configured by the server 130 to generate a high-precision map of the target scene.
  • map feature extraction and map feature editing can be deployed on the same server or separately on different servers.
  • map feature extraction is deployed on servers 131 and 132
  • map feature editing is deployed on server 133. 134.
  • the high-precision map of the target scene is further stored, for example, stored in the server 130, or may be stored in other cache spaces, which is not limited herein.
  • the user terminal 110 For the user terminal 110 using a high-precision map, for example, when an unmanned vehicle wants to pass through a target scene, the user terminal 110 carried by it will obtain a high-precision map of the target scene accordingly, so as to assist the unmanned vehicle Pass the target scene safely.
  • the laser point cloud and image of the target scene can be pre-collected by another collection device and stored in the server side 130, or when the vehicle, aircraft, or robot carrying the user side 110 passes the target scene , Collected by the user terminal 110 in real time and uploaded to the server terminal 130, which is not limited herein.
  • Fig. 2 is a block diagram of a hardware structure of a server according to an exemplary embodiment. This kind of server is applicable to the server in the implementation environment shown in FIG. 1.
  • this kind of server is only an example adapted to the embodiments of the present application, and cannot be considered as providing any limitation on the scope of use of the embodiments of the present application.
  • This kind of server cannot also be interpreted as needing to depend on or must have one or more components in the exemplary server 200 shown in FIG. 2.
  • the hardware structure of the server 200 may vary greatly depending on the configuration or performance.
  • the server 200 includes: a power supply 210, an interface 230, at least one memory 250, and at least one central processing unit (CPU, Central Processing Units) 270.
  • CPU Central Processing Unit
  • the power supply 210 is used to provide an operating voltage for each component on the server 200.
  • the interface 230 includes at least one wired or wireless network interface 231, at least one serial-parallel conversion interface 233, at least one input-output interface 235, at least one USB interface 237, etc., for communicating with external devices. For example, interacting with other servers in the client 110 or server 130 in the implementation environment shown in FIG. 1.
  • the memory 250 may be a read-only memory, a random access memory, a magnetic disk, or an optical disk.
  • the resources stored on the memory 250 include an operating system 251, application programs 253, and data 255. .
  • the operating system 251 is used to manage and control the components and application programs 253 on the server 200 to realize the calculation and processing of the massive data 255 by the central processor 270, which may be Windows Server TM , Mac OS X TM , Unix TM , Linux TM , FreeBSD TM, etc.
  • the application program 253 is a computer program that completes at least one specific job based on the operating system 251, and may include at least one module (not shown in FIG. 2), and each module may separately include a series of computers for the server 200. Readable instructions.
  • the device for extracting map elements may be regarded as an application program 253 deployed on the server 200 to implement the method for extracting map elements described in any embodiment of the present application.
  • the data 255 may be photos, pictures, or laser point clouds and images of the target scene, which are stored in the memory 250.
  • the central processor 270 may include one or more processors, and is configured to communicate with the memory 250 through a communication bus to read computer-readable instructions stored in the memory 250, thereby implementing operations on the massive data 255 in the memory 250 With processing. For example, the method for extracting map elements described in any embodiment of the present application is completed by the central processor 270 reading a series of computer-readable instructions stored in the memory 250.
  • the display screen 280 may be a liquid crystal display screen or an electronic ink display screen, etc.
  • This display screen 280 provides an output interface between the electronic device 200 and the user, through which any form or combination of text, pictures, or video
  • the formed output content displays the output to the user. For example, display map elements that are available for editing in a map that matches the target scene.
  • the input component 290 may be a touch layer covered on the display screen 280, a button, a trackball or a touchpad provided on the casing of the electronic device 200, or an external keyboard, mouse, touchpad, etc.
  • embodiments of the present application can also be implemented through a hardware circuit or a combination of hardware circuits and software. Therefore, the implementation of the embodiments of the present application is not limited to any specific hardware circuit, software, or combination of the two.
  • a method for extracting map elements is applicable to a server in the implementation environment shown in FIG. 1, and the structure of the server may be as shown in FIG.
  • the method for extracting map elements may be executed by an electronic device such as a server, or may be understood to be executed by a device for extracting map elements deployed in the server.
  • a device for extracting map elements deployed in the server may be executed by an electronic device such as a server, or may be understood to be executed by a device for extracting map elements deployed in the server.
  • the execution body of each step is used as a map element extraction device for description, but this is not a limitation.
  • the method for extracting map elements may include the following steps:
  • Step 310 Acquire the laser point cloud and image of the target scene.
  • the target scene may be a road that can be driven by the vehicle and its surrounding environment, or it can be inside a building where the robot can travel, or a channel and its surrounding environment that can be used by the drone to fly at low altitude. This is not restricted.
  • the method for extracting map elements provided in this embodiment can be adapted to different application scenarios according to different target scenarios, for example, the road and its surrounding environment are suitable for auxiliary vehicle driving scenarios, and the interior of the building is suitable for auxiliary robot traveling scenarios. , The channel and its surrounding environment are suitable for assisting UAV low-altitude flight scenarios.
  • the target scene includes at least one element entity, which corresponds to the map element.
  • the feature entity is an entity that actually exists in the target scene
  • the map feature corresponding to the feature entity is the feature that is presented in the map matched by the target scene.
  • the map elements and their corresponding element entities are different according to different application scenarios.
  • the map elements include: lane lines, ground signs, road teeth, fences, traffic signs, etc.
  • the element entities refer to lane lines, ground signs, road teeth, fences, fences, traffic Signs and other entities that actually exist in the target scene.
  • the map elements include: street lights, vegetation, buildings, traffic signs and other elements, and correspondingly, the element entities refer to street lights, vegetation, buildings, traffic signs and other real The entity that exists in the target scene.
  • the laser point cloud is generated by laser scanning the entities in the target scene, and its essence is a dot matrix image, that is, it is composed of several sampling points corresponding to the entities in the target scene. Therefore, the laser point cloud only reflects the spatial structure of the entity in the target scene, but cannot reflect the color texture contour of the entity in the target scene. This may cause the map elements in the laser point cloud due to the lack of sampling points of the corresponding feature entities The outline of is missing, which in turn affects the accuracy of map feature extraction.
  • the image of the target scene is also acquired, so as to reflect the color texture outline of the entity in the target scene.
  • the laser point cloud and image can be derived from the pre-stored laser point cloud and image, and can also be derived from the laser point cloud and image collected in real time, and then obtained through local reading or network download.
  • the laser point cloud and image of the target scene collected in real time can be obtained to facilitate real-time extraction of map elements, and the laser point cloud of the target scene collected in a historical time period can also be obtained And images to facilitate extraction of map elements when there are fewer processing tasks, or extraction of map elements at an appropriate time, which is not specifically limited in this embodiment.
  • the laser point cloud is generated by the laser scan emitted by the laser, and the image is collected by a camera device (such as a camera).
  • the laser and camera equipment can be pre-deployed in the acquisition device to facilitate the acquisition of laser point clouds and images for the target scene.
  • the acquisition device is a vehicle, and the laser and camera equipment are pre-deployed as a vehicle-mounted component on the vehicle. When the vehicle travels through the target scene, the laser point cloud and image of the target scene are collected accordingly.
  • Step 330 Perform registration between the laser point cloud and the image to obtain a depth map of the image.
  • a depth map can be constructed for the image according to the spatial structure described by the laser point cloud.
  • the construction process of the depth map not only utilizes the spatial structure of the element entity in the target scene reflected by the laser point cloud, but also combines the color texture outline of the element entity in the target scene reflected in the image.
  • the depth map not only describes the color texture outline of the map elements, but also describes the spatial structure of the map elements, which greatly enriches the data basis of image segmentation, thereby fully ensuring the accuracy of subsequent image segmentation in the depth map. .
  • the depth map of the image is constructed based on the depth information corresponding to the pixels in the image, that is, the depth map of the image is essentially a two-dimensional image carrying the depth information corresponding to the pixels in the image.
  • the depth information is used to represent the geometric transformation form between the laser point cloud (3D) and the image (2D), that is, the registration relationship.
  • the purpose of registration is to ensure that the matching geographic location between the laser point cloud and the image for the same target scene but from different sources is essentially the process of determining the geometric transformation form between the laser point cloud and the image.
  • the laser point cloud comes from the laser
  • the image comes from the camera equipment.
  • registration can be achieved based on grayscale features. In another embodiment, registration can also be implemented based on image features, where the image features include color features, texture features, shape features, spatial relationship features, and so on.
  • the registration includes processing methods such as geometric correction, projection transformation, and uniform scale, which are not limited in this embodiment.
  • the depth information corresponding to the pixels in the image can be obtained, and then the depth of the image can be obtained based on the geometric transformation relationship between the laser point cloud and the image represented by the depth information Figure.
  • Step 350 Perform image segmentation on the depth map of the image to obtain a segmented image of the map element in the depth map.
  • the target scene includes not only the element entities corresponding to the map elements, but also other entities not related to the map elements, such as vehicles. Then, for the depth map, in addition to the map elements corresponding to the element entities, there are also non-map elements corresponding to other entities.
  • image segmentation refers to distinguishing map elements from non-map elements in a depth map. Then, the distinguished map elements form corresponding segmented images in the depth map.
  • the segmented image can be used to describe the location, category, color, etc. of map elements in the depth map.
  • the category refers to the type of map elements, for example, lane line elements are regarded as a type of map elements.
  • image segmentation includes: ordinary segmentation, semantic segmentation, instance segmentation, etc., where ordinary segmentation further includes: threshold segmentation, region segmentation, edge segmentation, histogram segmentation, etc., this embodiment does not address this Make specific restrictions.
  • the position of the described map element in the depth map is essentially a two-dimensional position.
  • Step 370 according to the registration relationship between the laser point cloud and the image, convert the two-dimensional position of the segmented image in the depth map to the three-dimensional of the map element in the target scene position.
  • the high-precision map matching the target scene is a true reflection of the actual style of the target scene according to the specified scale.
  • the entity is a road
  • a high-precision map not only the road needs to be drawn according to the geographic location of the road in the target scene, but also the shape of the road, including width, slope, curvature, etc.
  • This three-dimensional location refers to the geographic location of the feature entity corresponding to the map feature in the target scene. Further, the three-dimensional position can be uniquely identified by coordinates.
  • the map element data further includes the color and category of the map element in the target scene.
  • the map elements are lane line elements.
  • the map element data includes: the three-dimensional position of the lane line in the target scene, the color of the lane line, and the form of the lane line.
  • the form of lane line includes solid line, dashed line, double yellow line and so on.
  • the map elements are extracted quickly and accurately, which provides a highly accurate data basis for the generation of high-precision maps, avoiding manual editing of map elements. It not only improves the production efficiency of high-precision maps, but also reduces the production cost of high-precision maps.
  • the embodiments of the present application make full use of the image of the target scene, which not only effectively expands the data source, but also makes the map element data more abundant and complete, which is beneficial to ensuring the accuracy of the high-precision map.
  • the step of registering the laser point cloud with the image to obtain depth information corresponding to pixels in the image may further include the following steps:
  • Step 3311 Construct a projection transformation function between the laser point cloud and the image.
  • Step 3313 Extract feature points corresponding to the laser point cloud and the image, and estimate parameters of the projection transformation function according to the extracted feature points.
  • Step 3315 Calculate the depth information corresponding to the pixels in the image according to the projection transformation function that completes the parameter estimation.
  • the registration is implemented based on the projection transformation method of image features.
  • f x represents the ratio of the camera focal length to the physical size of pixels in the image in the x-axis direction
  • f y represents the ratio of the camera focal length to the physical size of pixels in the image in the y-axis direction
  • (u 0 , v 0 ) indicates The origin of the two-dimensional coordinate system
  • R represents the rotation relationship between the camera coordinate system and the three-dimensional coordinate system
  • t represents the translation relationship between the camera coordinate system and the three-dimensional coordinate system.
  • the two-dimensional coordinate system refers to the image coordinate system
  • the three-dimensional coordinate system refers to the coordinate system where the target scene is located, that is, the real-world coordinate system.
  • (u, v) represents the two-dimensional coordinates of pixels in the image
  • (X w , Y w , Z w ) represents the three-dimensional coordinates of a point on the entity corresponding to the pixel in the target scene, that is, in the laser point cloud
  • Z c represents the depth information corresponding to the pixel point, that is, the coordinate of the pixel point along the z-axis direction in the camera coordinate system.
  • determining the registration relationship between the laser point cloud and the image is essentially estimating the parameters of the projection transformation function, namely f x , f y , (u 0 , v 0 ), R, and t.
  • Feature points refer to pixels that can describe the characteristics of the image.
  • the sampling points such as corners, vertices, endpoints, center of gravity points, and inflection points
  • 6 pixels that are evenly distributed in the image Points are used as feature points to reflect the significant features of the entities in the target scene, which is beneficial to improve the accuracy of registration between the laser point cloud and the image.
  • the registration relationship between the laser point cloud and the image is determined. Then, through the determination of the laser point cloud (X w , Y w , Z w ) and the determination of the image (u, v), the depth information corresponding to the pixels in the image, that is, Z c , can be calculated.
  • registration based on image features is realized, which not only greatly reduces the calculation amount in the registration process, but also helps to improve the efficiency of extracting map elements, thereby promoting the production efficiency of high-precision maps, and the feature points It embodies the salient features of the entities in the target scene, and can be more sensitive to changes in the spatial structure of the entities in the target scene, which is beneficial to improve the accuracy of the registration process.
  • step 370 may include the following steps:
  • the two-dimensional position of the segmented image in the depth map and the depth information corresponding to the pixels in the image are input into the projection transformation function that completes the parameter estimation, and the map element in the target scene is calculated Three-dimensional location.
  • step 350 may include the following steps:
  • Step 351 Perform feature extraction on the depth map of the image to obtain a feature map corresponding to the image.
  • the feature map is used to represent the image features of the depth map.
  • the image features include color features, texture features, shape features, spatial relationship features, and so on.
  • the feature map not only reflects the global features of the depth map, such as color features, but also reflects the local features of the depth map, such as spatial relationship features.
  • feature extraction may be performed using a convolutional neural network.
  • feature extraction may also be performed based on a residual neural network, which is not used in this embodiment. Make specific restrictions.
  • Step 353 Perform category prediction on the pixels in the feature map to obtain the categories of the pixels in the feature map.
  • the pixel-level category prediction on the feature map is implemented through a pre-built semantic segmentation network.
  • Semantic segmentation networks are not limited to: convolutional neural networks, residual neural networks, etc.
  • the construction process of the semantic segmentation network may include the following steps:
  • step 510 an image sample is obtained, and the image sample is marked with pixel points.
  • Step 530 Guide a specified mathematical model for model training according to the acquired image samples.
  • Step 550 Construct the semantic segmentation network from a specified mathematical model that completes model training.
  • the semantic segmentation network is generated by mass model training on the specified mathematical model.
  • the image sample refers to an image that has been marked with pixel categories.
  • Model training is essentially to iteratively optimize the parameters of the specified mathematical model so that the specified algorithm function constructed from the parameters satisfies the convergence conditions.
  • specified mathematical models including but not limited to: machine learning models such as logistic regression, support vector machines, random forests, neural networks, etc.
  • Specify algorithm functions including but not limited to: maximum expectation function, loss function, etc.
  • the parameters of the specified mathematical model are updated, and the loss value of the loss function constructed by the updated parameters is calculated according to the latter image sample.
  • the map element extraction device After the construction of the semantic segmentation network is completed, the map element extraction device has the ability to predict the pixel type of the feature map.
  • Step 355 Fit pixels of the same category in the feature map to form a segmented image of the corresponding map element in the depth map, and each category corresponds to a map element.
  • the fitting method is used to classify the same category in the feature map Pixels are structured.
  • the fitting method includes a least square fitting method, a Ransac-based curve fitting method, and the like.
  • the category of the map element is a lane line
  • the pixel points belonging to the lane line in the feature map are fitted as a straight line, as shown by 601 in FIG. 9.
  • the pixels in the feature map that belong to road teeth and fences are also fitted as a straight line, as shown in 602 and 603 in FIG. 9 respectively.
  • the pixel points belonging to the traffic sign in the feature map are fitted into a rectangular frame, as shown by 604 in FIG. 9.
  • the pixel points belonging to the ground mark in the feature map are also fitted into a rectangular frame, as shown by 605 in FIG. 9.
  • the segmented images formed based on the semantic segmentation network can directly know the location and category of the map elements, and avoid manually editing the map elements of different categories one by one, which greatly saves the time spent by manual editing , Fully reduce the production cost of high-precision maps, effectively improve the production efficiency of high-precision maps.
  • step 351 may include the following steps:
  • Step 3511 Use a high-level network in the residual neural network to extract global features of the depth map, and use a low-level network in the residual neural network to extract local features of the depth map.
  • the semantic segmentation network is a residual neural network.
  • the residual neural network adopts the Encoder-Decoder structure, including several high-level networks and several low-level networks.
  • Image represents the input of the residual neural network, that is, the depth map.
  • 701 represents the Encoder part of the residual neural network for feature extraction of the depth map; 701 'represents the Decoder part of the residual neural network for fusion of the extracted features.
  • 7011 and 7012 represent low-level networks in the residual neural network to extract local features of the depth map
  • 7013 and 7014 represent high-level networks in the residual neural network to extract global features of the depth map.
  • Step 3513 Fusion of the extracted global features and local features to obtain an intermediate feature map.
  • deconvolution 7023 is performed on the global features corresponding to the next highest layer network 7013, and through fusion with the fusion feature map, an updated fusion feature map 7031 is formed, and then the updated fusion feature map 7031 is upsampled 7024 To form the fusion feature map of the second update.
  • the global features corresponding to the remaining high-level networks (not shown in FIG. 11) and the local features corresponding to the low-level networks 7011 and 7012 are traversed according to the traversed global features Or the local feature updates the fused feature map that is updated twice.
  • the deconvolution process 7025 is performed on the local features corresponding to the low-level network 7012, and by fusion with the second-updated fusion feature map, a re-updated fusion feature map 7032 is formed, and then the re-updated fusion feature map 7032 Upsampling process 7026 is performed to form a fusion feature map updated four times.
  • the last updated fusion feature map 7033 is used as the intermediate feature map.
  • Step 3515 Perform linear interpolation on the intermediate feature map to obtain a feature map corresponding to the image.
  • the resolution of the intermediate feature map is substantially 1/2 of the resolution of the depth map Image. Therefore, before performing pixel-level category prediction, the intermediate The feature map is linearly interpolated so that the resolution of the feature map thus formed is consistent with the resolution of the depth map Image.
  • the method described above may further include the following steps:
  • Step 810 Display the map element in the target scene map according to the three-dimensional position of the map element in the target scene.
  • Step 830 Acquire and respond to control instructions for map elements in the target scene map, and generate a high-precision map of the target scene.
  • the target scene map refers to a map that matches the target scene.
  • the corresponding lane line feature data will be loaded in the target scene map to display the lane line feature according to the three-dimensional position of the lane line feature in the target scene indicated by the lane line feature data, as shown in the figure 13 is shown.
  • the ground mark element will be displayed in the target scene map accordingly, as shown in Figure 14.
  • map feature data such as lane line feature data
  • map feature data will be stored in advance according to the specified storage format after extraction is completed, so that it can be read when editing map features.
  • map elements are displayed on the target scene map, you can refer to the laser point cloud and image of the target scene to view the map elements.
  • map elements do not meet the requirements, for example, the accuracy requirements are not met, or the location, shape, category are deviated, or the map elements are missing due to vehicle obstruction, then the map elements can be further edited Operation, at this time, the editing instructions for the map elements in the map will be obtained accordingly, and then the corresponding editing processing will be performed on the map elements in the map in response to the editing instructions, and finally a high-precision map containing the edited map elements .
  • control instructions at least include editing instructions and one-key generation instructions.
  • high-precision maps are an indispensable part of realizing driverless driving. It can truly restore the target scene to improve the positioning accuracy of unmanned devices (such as unmanned vehicles, drones, and robots); it can also solve the failure of environmental awareness devices (such as sensors) in unmanned devices under special circumstances Problem, effectively compensate for the lack of environment-aware devices; at the same time, it can realize global path planning for unmanned devices, and formulate reasonable travel strategies for unmanned devices based on pre-judgments. Therefore, high-precision maps play an irreplaceable role in unmanned driving.
  • the map element extraction methods provided by the embodiments of this application not only fully ensure the accuracy of high-precision maps, but also effectively reduce the high-precision maps.
  • the production cost has improved the production efficiency of high-precision maps, which is conducive to mass production of high-precision maps.
  • the following is an embodiment of the device of the present application, which can be used to execute the method of extracting map elements involved in any embodiment of the present application.
  • the method embodiment of the map element extraction method involved in the present application please refer to the method embodiment of the map element extraction method involved in the present application.
  • a map element extraction device 900 includes but is not limited to: an image acquisition module 910, a depth map construction module 930, an image segmentation module 950, and a position conversion module 970.
  • the image acquisition module 910 is used to acquire a laser point cloud and an image of a target scene, where the target scene includes at least one element entity corresponding to a map element.
  • the depth map construction module 930 is used to perform registration between the laser point cloud and the image to obtain a depth map of the image.
  • the image segmentation module 950 is used to perform image segmentation on the depth map of the image to obtain a segmented image of the map element in the depth map.
  • the position conversion module 970 is configured to convert the two-dimensional position of the segmented image in the depth map to the map element in the target scene according to the registration relationship between the laser point cloud and the image Three-dimensional location.
  • the depth map construction module includes, but is not limited to, a registration unit and a construction unit.
  • the registration unit is used to register the laser point cloud and the image to obtain depth information corresponding to the pixels in the image.
  • the construction unit is configured to construct the depth map for the image according to the depth information corresponding to the pixels in the image.
  • the registration unit includes, but is not limited to, a function construction subunit, a feature point extraction subunit, and an information calculation subunit.
  • the function construction subunit is used to construct a projection transformation function between the laser point cloud and the image.
  • a feature point extraction subunit is used to extract feature points corresponding to the laser point cloud and the image, and estimate parameters of the projection transformation function according to the extracted feature points.
  • the information calculation subunit is used for calculating the depth information corresponding to the pixels in the image according to the projection transformation function for completing the parameter estimation.
  • the position conversion module includes, but is not limited to, a position conversion unit.
  • the position conversion unit is used to input the projection transformation function that completes the parameter estimation to the two-dimensional position of the segmented image in the depth map and the depth information corresponding to the pixels in the image to calculate the map element The three-dimensional position in the target scene.
  • the image segmentation module includes but is not limited to: a feature extraction unit, a category prediction unit, and a fitting unit.
  • the feature extraction unit is used to perform feature extraction on the depth map of the image to obtain a feature map corresponding to the image.
  • a category prediction unit is used to perform category prediction on pixels in the feature map to obtain the category of pixels in the feature map.
  • the fitting unit is used to fit pixels of the same category in the feature map to form a segmented image of the corresponding map element in the depth map, and each category corresponds to a map element.
  • the feature extraction unit includes, but is not limited to: a feature extraction subunit, a feature fusion subunit, and an interpolation subunit.
  • the feature extraction sub-unit is used to extract the global features of the depth map by using a high-level network in the residual neural network, and extract the local features of the depth map by using a low-level network in the residual neural network.
  • the feature fusion subunit is used to fuse the extracted global features and local features to obtain an intermediate feature map.
  • the interpolation subunit is used to linearly interpolate the intermediate feature map to obtain a feature map corresponding to the image.
  • the feature fusion subunits include but are not limited to: processing subunits, deconvolution processing subunits, upsampling processing subunits, traversal subunits, and definition subunits.
  • the processing subunit is used to perform deconvolution and upsampling processing on the global features corresponding to the highest layer network in the residual neural network to obtain a fusion feature map.
  • the deconvolution processing subunit is used to perform deconvolution processing on the global features corresponding to the next highest layer network in the residual neural network, and update the fusion feature map through fusion with the fusion feature map.
  • the up-sampling processing sub-unit is used for up-sampling the updated fusion feature map and updating the fusion feature map twice.
  • the traversal subunit is used to traverse the global features corresponding to the remaining high-level networks and the local features corresponding to the low-level networks according to the order of the networks in the residual neural network from high to low.
  • the fusion feature map after the second update is updated.
  • a subunit is defined to use the last updated feature map as the intermediate feature map after completing the traversal.
  • the apparatus further includes a network construction module, and the network construction module includes but is not limited to: a sample acquisition unit, a model training unit, and a network construction unit.
  • the sample acquiring unit acquires the image sample, and the image sample is marked with the pixel type.
  • the model training unit is used to guide a specified mathematical model to perform model training according to the acquired image samples.
  • the network construction unit is configured to construct the residual neural network from a specified mathematical model that completes model training.
  • the device further includes, but is not limited to: a display module and an editing module.
  • the display module is configured to display the map element in the target scene map according to the three-dimensional position of the map element in the target scene.
  • the editing module is configured to obtain a control instruction for map elements in the target scene map and respond to the control instruction to generate a high-precision map of the target scene.
  • map element extraction device only uses the division of the above functional modules as an example to illustrate the map element extraction.
  • the above functions can be assigned to different functions as needed Module completion, that is, the internal structure of the map element extraction device will be divided into different functional modules to complete all or part of the functions described above.
  • map element extraction device and the map element extraction method embodiment provided in the above embodiments belong to the same concept, and the specific manner in which each module performs operations has been described in detail in the method embodiments, and will not be repeated here.
  • a server 1000 includes at least one processor 1001, at least one memory 1002, and at least one communication bus 1003.
  • the memory 1002 stores computer readable instructions, and the processor 1001 reads the computer readable instructions stored in the memory 1002 through the communication bus 1003.
  • a computer-readable storage medium has stored thereon a computer program which, when executed by a processor, implements the method of extracting map elements in the above embodiments.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Remote Sensing (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Automation & Control Theory (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Electromagnetism (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Geometry (AREA)
  • Optics & Photonics (AREA)
  • Computer Graphics (AREA)
  • Image Analysis (AREA)
  • Traffic Control Systems (AREA)
  • Image Processing (AREA)
  • Processing Or Creating Images (AREA)

Abstract

一种地图要素提取方法、装置（900）及服务器（131，132，133，134，200，1000），地图要素提取方法包括：获取目标场景的激光点云（401）和图像（402）（步骤310），目标场景包括至少一要素实体对应于地图要素；进行激光点云（401）和图像（402）之间的配准，得到图像（402）的深度图（步骤330）；对图像（402）的深度图进行图像（402）分割，得到地图要素在深度图中的分割图像（步骤350）；根据激光点云（401）与图像（402）之间的配准关系，将分割图像在深度图中的二维位置转换为地图要素在目标场景中的三维位置（步骤370）。

Description

地图要素提取方法、装置及服务器
本申请要求于2018年10月12日提交中国专利局、申请号为201811186664.6，申请名称为“地图要素提取方法、装置及服务器”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及计算机技术领域，尤其涉及一种地图要素提取方法、装置及服务器。
背景技术
高精度地图，是用于辅助驾驶、半自动驾驶或者无人驾驶的地图，由一系列地图要素构成。地图要素包括：车道线、地面标志、路牙、栅栏、交通标志牌等要素。
发明内容
本申请各实施例提供一种地图要素提取方法、装置及服务器。
其中，本申请实施例所采用的技术方案为：
第一方面，一种地图要素提取方法，由电子设备执行，包括：获取目标场景的激光点云和图像，所述目标场景包括至少一要素实体对应于地图要素；进行所述激光点云与所述图像之间的配准，得到所述图像的深度图；对所述图像的深度图进行图像分割，得到所述地图要素在所述深度图中的分割图像；根据所述激光点云与所述图像之间的配准关系，将所述分割图像在所述深度图中的二维位置转换为所述地图要素在所述目标场景中的三维。
第二方面，一种地图要素提取装置，包括：图像获取模块，用于获取目标场景的激光点云和图像，所述目标场景包括至少一要素实体对应于地图要素；深度图构建模块，用于进行所述激光点云与所述图像之间的配准，得到所述图像的深度图；图像分割模块，用于对所述图像的深度图进行图像分割，得到所述地图要素在所述深度图中的分割图像；位 置转换模块，用于根据所述激光点云与所述图像之间的配准关系，将所述分割图像在所述深度图中的二维位置转换为所述地图要素在所述目标场景中的三维位置。
第三方面，一种服务器，包括处理器及存储器，所述存储器上存储有计算机可读指令，所述计算机可读指令被所述处理器执行时实现如上所述的地图要素提取方法。
第四方面，一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现如上所述的地图要素提取方法。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本申请的实施例，并于说明书一起用于解释本申请实施例的原理。
图1是根据本申请实施例的一种地图要素提取方法所涉及的实施环境的示意图；
图2是根据一示例性实施例示出的一种服务器的硬件结构框图；
图3是根据一示例性实施例示出的一种地图要素提取方法的流程图；
图4是图3对应实施例所涉及的配准前目标场景的激光点云与图像的示意图；
图5是图3对应实施例所涉及的配准后目标场景的激光点云与图像的示意图；
图6是根据一示例性实施例示出的对所述激光点云与所述图像进行配准，得到所述图像中像素点对应的深度信息步骤的流程图；
图7是图3对应实施例中步骤350在一个实施例的流程图；
图8是根据一示例性实施例示出的语义分割网络的构建过程的流程图；
图9是图7对应实施例所涉及的地图要素在深度图中的分割图像的示意图；
图10是图7对应实施例中步骤351在一个实施例的流程图；
图11是图10对应实施例所涉及的残差神经网络的结构示意图；
图12是根据一示例性实施例示出的另一种地图要素提取方法的流程图；
图13是图12对应实施例所涉及的目标场景匹配地图中显示车道线要素的示意图；
图14是图12对应实施例所涉及的目标场景匹配地图中显示地面标志要素的示意图；
图15是根据一示例性实施例示出的一种地图要素提取装置的结构图；
图16是根据一示例性实施例示出的一种服务器的结构图。
具体实施方式
这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本申请实施例相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本申请实施例的一些方面相一致的装置和方法的例子。
通常，在高精度地图的生成过程中，首先从激光点云中提取地图要素，再通过人工方式对提取的地图要素进行手动编辑，以生成高精度地图。可以理解，如果地图要素提取的准确性不高，将导致人工手动编辑地图要素的耗时长、工序复杂、效率低，并最终影响高精度地图的生产效率。因此，如何提高高精度地图的生产效率仍亟待解决。
图1为根据本申请实施例的一种地图要素提取方法所涉及的实施环境的示意图。该实施环境包括用户端110和服务器端130。
其中，用户端110部署于车辆、飞机、机器人中，可以是台式电脑、笔记本电脑、平板电脑、智能手机、掌上电脑、个人数字助理、导航仪、智能计算机等，在此不进行限定。
用户端110与服务器端130通过无线或者有线网络等方式预先建立网络连接，并通过此网络连接实现用户端110与服务器端130之间的数 据传输。例如，传输的数据包括：目标场景的高精度地图等。
在此说明的是，服务器端130可以是一台服务器，也可以是由多台服务器构成的服务器集群，如图1所示，还可以是由多台服务器构成的云计算中心。其中，服务器是为用户提供后台服务的电子设备，例如，后台服务包括：地图要素提取服务、高精度地图生成服务等。
服务器端130在获取到目标场景的激光点云和图像之后，便可通过目标场景的激光点云和图像进行地图要素提取，以获取地图要素在目标场景中的三维位置。
在获得地图要素在目标场景中的三维位置之后，便可通过服务器端130所配置的显示屏幕，按照此三维位置在目标场景地图中显示地图要素，以生成目标场景的高精度地图。
当然，根据实际需要，地图要素提取和地图要素编辑既可以部署于同一服务器中，也可以分别部署于不同服务器中，例如，地图要素提取部署于服务器131、132，地图要素编辑部署于服务器133、134。
然后，进一步存储目标场景的高精度地图，例如，存储至服务器端130，也可以存储至其它缓存空间，在此并未加以限定。
对于使用高精度地图的用户端110而言，例如，当无人驾驶车辆欲经过目标场景时，其所承载的用户端110将相应地获得目标场景的高精度地图，以便于辅助无人驾驶车辆安全经过目标场景。
值得一提的是，关于目标场景的激光点云和图像，可以是由另外的采集设备预先采集并存储至服务器端130，也可以是在承载用户端110的车辆、飞机、机器人经过目标场景时，由用户端110实时采集并上传至服务器端130，在此并未加以限定。
图2是根据一示例性实施例示出的一种服务器的硬件结构框图。该种服务器适用于图1所示实施环境中的服务器。
需要说明的是，该种服务器只是一个适配于本申请实施例的示例，不能认为是提供了对本申请实施例的使用范围的任何限制。该种服务器也不能解释为需要依赖于或者必须具有图2中示出的示例性的服务器 200中的一个或者多个组件。
服务器200的硬件结构可因配置或者性能的不同而产生较大的差异，如图2所示，服务器200包括：电源210、接口230、至少一个存储器250、以及至少一个中央处理器(CPU,Central Processing Units)270。
具体地，电源210用于为服务器200上的各组件提供工作电压。
接口230包括至少一有线或无线网络接口231、至少一串并转换接口233、至少一输入输出接口235以及至少一USB接口237等，用于与外部设备通信。例如，与图1所示出实施环境中的用户端110或服务端130中的其它服务器交互。
存储器250作为资源存储的载体，可以是只读存储器、随机存储器、磁盘或者光盘等，其上所存储的资源包括操作系统251、应用程序253及数据255等，存储方式可以是短暂存储或者永久存储。
其中，操作系统251用于管理与控制服务器200上的各组件以及应用程序253，以实现中央处理器270对海量数据255的计算与处理，其可以是Windows Server TM、Mac OS X TM、Unix TM、Linux TM、FreeBSD TM等。
应用程序253是基于操作系统251之上完成至少一项特定工作的计算机程序，其可以包括至少一模块(图2中未示出)，每个模块都可以分别包含有对服务器200的一系列计算机可读指令。例如，地图要素提取装置可视为部署于服务器200的应用程序253，以实现本申请任一实施例所述的地图要素提取方法。
数据255可以是照片、图片，还可以是目标场景的激光点云和图像，存储于存储器250中。
中央处理器270可以包括一个或多个以上的处理器，并设置为通过通信总线与存储器250通信，以读取存储器250中存储的计算机可读指令，进而实现对存储器250中海量数据255的运算与处理。例如，通过中央处理器270读取存储器250中存储的一系列计算机可读指令的形式来完成本申请任一实施例所述的地图要素提取方法。
显示屏幕280可以是液晶显示屏或者电子墨水显示屏等，此显示屏 幕280在电子设备200与用户之间提供一个输出界面，以通过该输出界面将文字、图片或者视频任意一种形式或者组合所形成的输出内容向用户显示输出。例如，将可供编辑的地图要素显示在目标场景匹配的地图中。
输入组件290，可以是显示屏幕280上覆盖的触摸层，也可以是电子设备200外壳上设置的按键、轨迹球或者触控板，还可以是外接的键盘、鼠标、触控板等，用于接收用户输入的各种控制指令，以便于生成目标场景的高精度地图。例如，针对地图中地图要素的编辑指令。
此外，通过硬件电路或者硬件电路结合软件也能同样实现本申请实施例，因此，实现本申请实施例并不限于任何特定硬件电路、软件以及两者的组合。
请参阅图3，在一示例性实施例中，一种地图要素提取方法适用于图1所示实施环境的服务器，该服务器的结构可以如图2所示。
该地图要素提取方法可以由服务器等电子设备执行，也可以理解为由服务器中部署的地图要素提取装置执行。在下述方法实施例中，为了便于描述，以各步骤的执行主体为地图要素提取装置加以说明，但是并不对此构成限定。
该地图要素提取方法可以包括以下步骤：
步骤310，获取目标场景的激光点云和图像。
其中，目标场景可以是可供车辆行驶的道路及其周边环境，还可以是可供机器人行进的建筑物内部，又或者是可供无人机低空飞行的航道及其周边环境，本实施例并未对此加以限定。
相应地，本实施例所提供的地图要素提取方法可根据目标场景的不同而适用于不同的应用场景，例如，道路及其周边环境适用于辅助车辆行驶场景，建筑物内部适用于辅助机器人行进场景，航道及其周边环境适用于辅助无人机低空飞行场景。
在本申请一实施例中，目标场景包括至少一要素实体，其对应于地图要素。其中，要素实体是真实存在于目标场景中的实体，而要素实体 所对应的地图要素则是呈现于目标场景所匹配地图中的要素。
具体而言，地图要素及其对应的要素实体根据应用场景的不同有所区别。例如，在辅助车辆行驶场景中，地图要素包括：车道线、地面标志、路牙、栅栏、交通标志牌等要素，相应地，要素实体指的是车道线、地面标志、路牙、栅栏、交通标志牌等真实存在于目标场景的实体。又例如，在辅助无人机低空飞行场景中，地图要素包括：路灯、植被、建筑物、交通标志牌等要素，相应地，要素实体则是指路灯、植被、建筑物、交通标志牌等真实存在于目标场景的实体。
如前所述，为了生成高精度地图，需要从激光点云中提取地图要素。可以理解，激光点云是通过激光扫描目标场景中实体所生成的，其实质是点阵图像，即是由对应目标场景中实体的若干采样点构成。故而，激光点云仅反映了实体在目标场景中的空间结构，而无法体现实体在目标场景中的色彩纹理轮廓，这就可能因对应要素实体的采样点缺失而使得地图要素在激光点云中的轮廓缺失，进而影响地图要素提取的准确性。
基于此，在本实施例中，在获取目标场景的激光点云时，还获取目标场景的图像，以此来反映实体在目标场景中的色彩纹理轮廓。
针对目标场景，激光点云和图像可以来源于预先存储的激光点云和图像，还可以来源于实时采集的激光点云和图像，进而通过本地读取或者网络下载的方式获取。
换句话说，对于地图要素提取装置而言，可以获取实时采集的目标场景的激光点云和图像，以便于实时进行地图要素提取，还可以获取一历史时间段内采集的目标场景的激光点云和图像，以便于在处理任务较少的时候进行地图要素提取，或者，在适当的时机进行地图要素提取，本实施例并未对此作出具体限定。
应当说明的是，激光点云是由激光器发射的激光扫描生成，图像则是通过摄像设备(例如摄像机)采集。在采集过程中，激光器和摄像设备可预先部署于采集设备中，以便于采集设备针对目标场景进行激光点云和图像的采集。例如，采集设备为车辆，激光器和摄像设备作为车载组件预先部署于该车辆，当该车辆行驶经过目标场景时，便相应地采集 得到该目标场景的激光点云和图像。
步骤330，进行所述激光点云与所述图像之间的配准，得到所述图像的深度图。
在获取到目标场景的激光点云和图像之后，便可根据激光点云所描述的空间结构，为图像构建深度图。
换句话说，本实施例中，深度图的构建过程，既利用了激光点云所反映的目标场景中要素实体的空间结构，还结合了图像所反映的目标场景中要素实体的色彩纹理轮廓，使得深度图不仅描述了地图要素的色彩纹理轮廓，而且描述了地图要素的空间结构，极大地丰富了图像分割的数据依据，从而充分地保证了后续地图要素在深度图中进行图像分割的准确性。
具体而言，根据图像中像素点对应的深度信息构建图像的深度图，也即是，图像的深度图实质上是携带了图像中像素点所对应深度信息的二维图像。其中，深度信息用于表示激光点云(三维)与图像(二维)之间的几何变换形式，亦即配准关系。
配准的目的在于保证针对同一目标场景却来源不同的激光点云与图像之间保持相匹配的地理位置，实质是确定激光点云与图像之间几何变换形式的过程。其中，激光点云来源于激光器，而图像来源于摄像设备。
针对同一目标场景，例如，配准前，如图4所示，激光点云401区域与图像402区域不匹配，仅存在部分区域重合。配准后，如图5所示，激光点云401区域与图像402区域基本重合，即视为达到最佳匹配效果，以此保证了配准后的激光点云与图像之间保持相匹配的地理位置，也即是，激光点云与图像可视为同源。
在一实施例中，配准可以根据灰度特征实现。在另一实施例中，配准还可以根据图像特征实现，其中，图像特征包括颜色特征、纹理特征、形状特征、空间关系特征等。
进一步地，配准包括：几何纠正、投影变换、统一比例尺等处理方式，本实施例并未对此加以限定。
通过激光点云与图像之间进行的配准，便可得到图像中像素点对应的深度信息，进而可基于该深度信息所表示的激光点云与图像之间的几何变换关系，得到图像的深度图。
步骤350，对所述图像的深度图进行图像分割，得到所述地图要素在所述深度图中的分割图像。
可以理解，目标场景中不仅包括对应地图要素的要素实体，还包括与地图要素无关的其它实体，例如车辆。那么，对于深度图而言，除了存在对应于要素实体的地图要素，也还存在对应于其它实体的非地图要素。
由此，本实施例中，图像分割是指将地图要素与非地图要素在深度图中区分开。那么，完成区分的地图要素在深度图中便形成了相应的分割图像。换而言之，分割图像可用于描述地图要素在深度图中的位置、类别、颜色等。其中，类别是指地图要素的种类，例如，车道线要素视为地图要素的一种。
在本申请一实施例中，图像分割包括：普通分割、语义分割、实例分割等，其中，普通分割进一步包括：阈值分割、区域分割、边缘分割、直方图分割等，本实施例并未对此作出具体限定。
值得一提的是，由于深度图仅是携带有深度信息的二维图像，故而，对于分割图像而言，所描述的地图要素在深度图中的位置，本质上是二维位置。
步骤370，根据所述激光点云与所述图像之间的配准关系，将所述分割图像在所述深度图中的二维位置，转换为所述地图要素在所述目标场景中的三维位置。
目标场景匹配的高精度地图，是按照指定比例真实地反映目标场景的实际样式。例如，实体为道路，则在高精度地图中，不仅需要按照道路在目标场景中的地理位置绘制道路，而且需要绘制该道路的形状，包括宽度、坡度、曲率等，甚至需要绘制该道路所包含的车道数，以此真实地反映该道路在目标场景中的实际样式。
那么，对于高精度地图而言，至少需要获知地图要素在目标场景中 的三维位置。此三维位置即是指地图要素所对应的要素实体在目标场景中的地理位置。进一步地，三维位置可通过坐标进行唯一标识。
由此，在得到地图要素在深度图中的分割图像之后，便需要按照激光点云与图像之间的配准关系，针对该分割图像在深度图中的二维位置，进行坐标转换，进而得到地图要素在目标场景中的三维位置，以作为地图要素数据。
在本申请一实施例中，地图要素数据还包括地图要素在目标场景中的颜色、类别等。
例如，地图要素为车道线要素，相应地，地图要素数据包括：车道线在目标场景中的三维位置、车道线的颜色、车道线的形式等。其中，车道线的形式包括实线、虚线、双黄线等。
通过如上所述的过程，通过结合目标场景的激光点云与图像，实现了快速准确地自动化提取地图要素，为高精度地图的生成提供了准确性高的数据依据，避免人工手动编辑地图要素，不仅提高了高精度地图的生产效率，而且降低了高精度地图的生产成本。
此外，本申请实施例还充分利用了目标场景的图像，不仅有效地扩展了数据来源，还使得地图要素数据更加地丰富完整，进而有利于保障高精度地图的精度。
请参阅图6，在一示例性实施例中，所述对所述激光点云与所述图像进行配准，得到所述图像中像素点对应的深度信息步骤，可以进一步包括以下步骤：
步骤3311，构建所述激光点云与所述图像之间的投影变换函数。
步骤3313，提取所述激光点云与所述图像中相对应的特征点，并根据提取到的特征点估计所述投影变换函数的参数。
步骤3315，根据完成参数估计的投影变换函数，计算得到所述图像中像素点对应的深度信息。
本实施例中，配准是基于图像特征的投影变换方式实现的。
具体地，激光点云与图像之间构建的投影变换函数如计算公式(1)所示：
Figure PCTCN2019110259-appb-000001
其中，f x表示摄像机焦距和图像中像素点在x轴方向上的物理尺寸比值，f y表示摄像机焦距和图像中像素点在y轴方向上的物理尺寸比值，(u 0,v 0)表示二维坐标系的原点，R表示摄像机坐标系与三维坐标系之间的旋转关系，t表示摄像机坐标系与三维坐标系之间的平移关系。应当说明的是，二维坐标系是指图像坐标系，三维坐标系是指目标场景所在坐标系，即真实世界坐标系。(u,v)表示图像中像素点的二维坐标，(X w,Y w,Z w)表示该像素点所对应实体上某个点在目标场景中的三维坐标，亦即激光点云中对应实体的采样点的三维坐标，Z c则表示该像素点对应的深度信息，亦即摄像机坐标系中该像素点沿z轴方向的坐标。
由上可知，确定激光点云与图像之间的配准关系，实质是估计投影变换函数的参数，即f x、f y、(u 0,v 0)、R、t。
为此，需要获取激光点云与图像中相对应的6组特征点。特征点是指能够描述图像特征的像素点。
在本申请一实施例中，针对激光点云中边界清晰显示、棱角分明的采样点(例如角点、顶点、端点、重心点、拐点等)，对应提取尽量均匀分布在图像中的6个像素点作为特征点，以此体现目标场景中实体的显著特征，进而有利于提高激光点云与图像之间配准的准确性。
在完成投影变换函数中参数的估计后，便确定了激光点云与图像之间的配准关系。那么，通过激光点云确定(X w,Y w,Z w)，以及通过图像确定(u,v)，便可计算得到图像中像素点对应的深度信息，即Z c。
在上述实施例的配合下，实现了基于图像特征的配准，不仅大幅度降低配准过程的计算量，还有利于提高地图要素提取的效率，进而促进高精度地图的生产效率，而且特征点体现了目标场景中实体的显著特征，能够对目标场景中实体空间结构的变化较敏感，有利于提高配准过程的精度。
进一步地，在一示例性实施例中，步骤370可以包括以下步骤：
将所述分割图像在所述深度图中的二维位置、以及所述图像中像素点对应的深度信息，输入完成参数估计的投影变换函数，计算得到所述地图要素在所述目标场景中的三维位置。
结合计算公式(1)对坐标变换过程加以说明如下：
当投影变换函数中的参数完成估计，即f x、f y、(u 0,v 0)、R、t已知。
那么，将图像中像素点对应的深度信息，即Z c、以及分割图像在深度图中的二维位置，即(u,v)，输入计算公式(1)，便可计算得到地图要素在目标场景中的三维位置，即(X w,Y w,Z w)，进而以此作为地图要素数据，以便于后续生成高精度地图。
请参阅图7，在一示例性实施例中，步骤350可以包括以下步骤：
步骤351，对所述图像的深度图进行特征提取，得到所述图像对应的特征图。
其中，特征图用于表示深度图的图像特征，此图像特征包括颜色特征、纹理特征、形状特征、空间关系特征等。那么，也可以理解为，特征图不仅体现了深度图的全局特征，例如颜色特征，还体现了深度图的局部特征，例如空间关系特征。
基于图像分割中的语义分割，在一实施例中，特征提取可采用卷积神经网络进行，在另一实施例中，特征提取还可以根据残差神经网络进行，本实施例中并未对此作出具体限定。
步骤353，对所述特征图中的像素点进行类别预测，得到所述特征图中像素点的类别。
本实施例中，在特征图上进行像素点级别的类别预测，是通过预先构建的语义分割网络实现的。
语义分割网络不限于：卷积神经网络、残差神经网络等。
下面对语义分割网络的构建过程加以说明。
如图8所示，语义分割网络的构建过程可以包括以下步骤：
步骤510，获取图像样本，所述图像样本进行了像素点类别标注。
步骤530，根据获取到的图像样本引导指定数学模型进行模型训练。
步骤550，由完成模型训练的指定数学模型构建得到所述语义分割 网络。
语义分割网络是通过海量的图像样本对指定数学模型进行模型训练生成的。其中，图像样本，是指进行了像素点类别标注的图像。
模型训练，实质上是对指定数学模型的参数加以迭代优化，使得由此参数构建的指定算法函数满足收敛条件。
其中，指定数学模型，包括但不限于：逻辑回归、支持向量机、随机森林、神经网络等机器学习模型。
指定算法函数，包括但不限于：最大期望函数、损失函数等等。
举例来说，随机初始化指定数学模型的参数，根据当前一个图像样本计算随机初始化的参数所构建的损失函数的损失值。
如果损失函数的损失值未达到最小，则更新指定数学模型的参数，并根据后一个图像样本计算更新的参数所构建的损失函数的损失值。
如此迭代循环，直至损失函数的损失值达到最小，即视为损失函数收敛，此时，指定数学模型也收敛，并符合预设精度要求，则停止迭代。
否则，迭代更新指定数学模型的参数，并根据其余图像样本迭代计算所更新参数构建的损失函数的损失值，直至损失函数收敛。
值得一提的是，如果在损失函数收敛之前，迭代次数已经达到迭代阈值，也将停止迭代，以此保证模型训练的效率。
当指定数学模型收敛并符合预设精度要求时，表示指定数学模型完成模型训练，由此便可构建得到语义分割网络。
在完成语义分割网络的构建之后，对于地图要素提取装置而言，便具有了对特征图进行像素点类别预测的能力。
那么，将特征图输入语义分割网络，便能够对特征图中的像素点进行类别预测，由此得到特征图中像素点的类别。
步骤355，将所述特征图中同一类别的像素点，拟合形成对应地图要素在所述深度图中的分割图像，每一类别对应一种地图要素。
可以理解，对于像素点的类别而言，并不能够形成地图要素在深度图中的分割图像，即为非结构化表示，故而，本实施例中，采用拟合方法，对特征图中同一类别的像素点进行结构化表示。
在本申请一实施例中，拟合方法包括：最小二乘拟合方法、基于Ransac的曲线拟合方法等。
结合图9对地图要素在深度图中的分割图像进行如下说明。
当地图要素的类别为车道线，则针对特征图中属于车道线的像素点，即被拟合为一条直线，如图9中601所示。
当地图要素的类别为路牙、栅栏，则针对特征图中属于路牙、栅栏的像素点，亦被拟合为一条直线，分别如图9中602、603所示。
当地图要素的类别为交通标志牌，则针对特征图中属于交通标志牌的像素点，即被拟合为一个矩形框，如图9中604所示。
当地图要素的类别为地面标志，则针对特征图中属于地面标志的像素点，亦被拟合为一个矩形框，如图9中605所示。
通过上述过程，基于语义分割网络所形成的分割图像，便可直接获知地图要素所在的位置、类别，而避免以人工方式对不同类别的地图要素逐个编辑，大大节省了人工手动编辑所耗费的时间，充分地降低了高精度地图的生产成本，有效地提高了高精度地图的生产效率。
此外，利用丰富的语义信息，在图像分割过程中相互验证，避免出现误检，能够有效地提高地图要素提取的准确性。
请参阅图10，在一示例性实施例中，步骤351可以包括以下步骤：
步骤3511，采用残差神经网络中的高层网络提取得到所述深度图的全局特征，并采用所述残差神经网络中的低层网络提取得到所述深度图的局部特征。
本实施例中，语义分割网络为残差神经网络。
具体地，残差神经网络采用了Encoder-Decoder结构，包括若干高层网络和若干低层网络。
如图11所示，Image表示残差神经网络的输入，即深度图。
701表示残差神经网络的Encoder部分，用于进行深度图的特征提取；701’表示残差神经网络的Decoder部分，用于对提取到的特征进行融合。
7011、7012表示残差神经网络中的低层网络，用于提取深度图的局 部特征；7013、7014表示残差神经网络中的高层网络，用于提取深度图的全局特征。
步骤3513，对提取得到的全局特征与局部特征进行融合，得到中间特征图。
结合图11，对残差神经网络提取图像所对应特征图的过程加以说明。
首先，对7014最高一层网络对应的全局特征进行反卷积处理7021和上采样处理7022，得到融合特征图。
然后，对次高一层网络7013对应的全局特征进行反卷积处理7023，并通过与融合特征图的融合，形成更新的融合特征图7031，再对更新的融合特征图7031进行上采样处理7024，形成二次更新的融合特征图。
接着，按照残差神经网络中网络由高至低的顺序，对其余高层网络对应的全局特征(图11中未体现)和低层网络7011、7012对应的局部特征进行遍历，根据遍历到的全局特征或者局部特征对二次更新的融合特征图进行更新。
具体而言，对低层网络7012对应的局部特征进行反卷积处理7025，并通过与二次更新的融合特征图的融合，形成再次更新的融合特征图7032，再对再次更新的融合特征图7032进行上采样处理7026，形成四次更新的融合特征图。
继续对低层网络7011对应的局部特征进行反卷积处理7027，并通过与四次更新的融合特征图的融合，形成最后更新的融合特征图7033，由此即完成残差神经网络中所有网络对应特征的遍历。
完成遍历后，则将最后更新的融合特征图7033作为中间特征图。
步骤3515，对所述中间特征图进行线性插值，得到所述图像对应的特征图。
如图11所示，可以看出，经过3次上采样处理，中间特征图的分辨率实质为深度图Image的分辨率的1/2，故而，在进行像素级的类别预测之前，需要针对中间特征图进行线性插值，以使得由此形成的特征图的分辨率与深度图Image的分辨率保持一致。
在上述过程中，实现了基于残差神经网络的特征提取，有利于提高 特征提取的准确性，进而充分地保证了地图要素提取的鲁棒性和稳定性。
请参阅图12，在一示例性实施例中，如上所述的方法还可以包括以下步骤：
步骤810，根据所述地图要素在所述目标场景中的三维位置，在目标场景地图中显示所述地图要素。
步骤830，获取针对所述目标场景地图中地图要素的控制指令并响应，生成目标场景的高精度地图。
目标场景地图是指与目标场景相匹配的地图。
在本申请实施例中，可以选择同时对全部类别的地图要素进行编辑，也可以选择一个类别的地图要素进行编辑，本实施例并未对此加以限定。
如果选择编辑车道线要素，则目标场景地图中，将加载对应的车道线要素数据，以根据车道线要素数据所指示的该车道线要素在目标场景中的三维位置显示该车道线要素，如图13所示。
同理，如果选择编辑地面标志要素，则目标场景地图中相应地显示该地面标志要素，如图14所示。
值得一提的是，地图要素数据，例如车道线要素数据，在完成提取之后，将按照指定的存储格式预先存储，以便于进行地图要素编辑时读取。
在目标场景地图中显示出地图要素之后，便可以参照目标场景的激光点云和图像，对该地图要素进行查看。
如果地图要素不符合要求，例如，不符合精度要求，或者，位置、形状、类别有所偏差，又或者，因车辆阻挡而导致地图要素有所缺失，那么，便可进一步地对地图要素进行编辑操作，此时，将相应获取到针对地图中地图要素的编辑指令，进而通过对编辑指令的响应，对地图中的地图要素进行相应的编辑处理，并最终生成包含编辑后地图要素的高精度地图。
反之，如果地图要素符合要求，则无需任何修改，即可一键生成高精度地图，这大大减少了手动编辑的工作量，有效地提升了编辑效率，进而有利于降低高精度地图的生产成本，提高高精度地图的生产效率。
由上可知，控制指令至少包括编辑指令、一键生成指令。
在具体应用场景中，高精度地图是实现无人驾驶不可或缺的重要环节。它能够真实还原目标场景，以此提高无人驾驶设备(例如无人驾驶车辆、无人机、机器人)的定位精度；还能够解决特殊情况下无人驾驶设备中环境感知设备(例如传感器)失效的问题，有效地弥补了环境感知设备的不足；同时能够实现为无人驾驶设备进行路径全局规划，并且基于预判为无人驾驶设备制定合理的行进策略。因此，高精度地图在无人驾驶中发挥着不可替代的作用，通过本申请各实施例所提供的地图要素提取方法，不仅充分地保证了高精度地图的精度，还有效地降低了高精度地图的生产成本，提高了高精度地图的生产效率，有利于实现高精度地图的大规模批量生产。
下述为本申请装置实施例，可以用于执行本申请任一实施例所涉及的地图要素提取方法。对于本申请装置实施例中未披露的细节，请参照本申请所涉及的地图要素提取方法的方法实施例。
请参阅图15，在一示例性实施例中，一种地图要素提取装置900包括但不限于：图像获取模块910、深度图构建模块930、图像分割模块950和位置转换模块970。
其中，图像获取模块910用于获取目标场景的激光点云和图像，所述目标场景包括至少一要素实体对应于地图要素。
深度图构建模块930用于进行所述激光点云与所述图像之间的配准，得到所述图像的深度图。
图像分割模块950用于对所述图像的深度图进行图像分割，得到所述地图要素在所述深度图中的分割图像。
位置转换模块970用于根据所述激光点云与所述图像之间的配准关系，将所述分割图像在所述深度图中的二维位置转换为所述地图要素在所述目标场景中的三维位置。
在一示例性实施例中，所述深度图构建模块包括但不限于：配准单元和构建单元。
其中，配准单元，用于对所述激光点云与所述图像进行配准，得到所述图像中像素点对应的深度信息。
构建单元，用于根据所述图像中像素点对应的深度信息，为所述图像构建所述深度图。
在一示例性实施例中，所述配准单元包括但不限于：函数构建子单元、特征点提取子单元和信息计算子单元。
其中，函数构建子单元，用于构建所述激光点云与所述图像之间的投影变换函数。
特征点提取子单元，用于提取所述激光点云与所述图像中相对应的特征点，并根据提取到的特征点估计所述投影变换函数的参数。
信息计算子单元，用于根据完成参数估计的投影变换函数，计算得到所述图像中像素点对应的深度信息。
在一示例性实施例中，所述位置转换模块包括但不限于：位置转换单元。
其中，位置转换单元，用于将所述分割图像在所述深度图中的二维位置以及所述图像中像素点对应的深度信息，输入完成参数估计的投影变换函数，计算得到所述地图要素在所述目标场景中的三维位置。
在一示例性实施例中，所述图像分割模块包括但不限于：特征提取单元、类别预测单元和拟合单元。
其中，特征提取单元，用于对所述图像的深度图进行特征提取，得到所述图像对应的特征图。
类别预测单元，用于对所述特征图中的像素点进行类别预测，得到所述特征图中像素点的类别。
拟合单元，用于将所述特征图中同一类别的像素点，拟合形成对应地图要素在所述深度图中的分割图像，每一类别对应一种地图要素。
在一示例性实施例中，所述特征提取单元包括但不限于：特征提取子单元、特征融合子单元和插值子单元。
其中，特征提取子单元，用于采用残差神经网络中的高层网络提取得到所述深度图的全局特征，并采用所述残差神经网络中的低层网络提 取得到所述深度图的局部特征。
特征融合子单元，用于进行提取得到的全局特征与局部特征的融合，得到中间特征图。
插值子单元，用于对所述中间特征图进行线性插值，得到所述图像对应的特征图。
在一示例性实施例中，所述特征融合子单元包括但不限于：处理子单元、反卷积处理子单元、上采样处理子单元、遍历子单元和定义子单元。
其中，处理子单元，用于对所述残差神经网络中最高一层网络对应的全局特征进行反卷积和上采样处理，得到融合特征图。
反卷积处理子单元，用于对所述残差神经网络中次高一层网络对应的全局特征进行反卷积处理，并通过与所述融合特征图的融合，更新所述融合特征图。
上采样处理子单元，用于对所述更新的融合特征图进行上采样处理，二次更新所述融合特征图。
遍历子单元，用于按照所述残差神经网络中网络由高至低的顺序，对其余高层网络对应的全局特征和低层网络对应的局部特征进行遍历，根据遍历到的全局特征或者局部特征对所述二次更新后的融合特征图进行更新。
定义子单元，用于完成所述遍历后，以最后更新的所述融合特征图作为所述中间特征图。
在一示例性实施例中，所述装置还包括网络构建模块，所述网络构建模块包括但不限于：样本获取单元、模型训练单元和网络构建单元。
其中，样本获取单元，获取图像样本，所述图像样本进行了像素点类别标注。
模型训练单元，用于根据获取到的图像样本引导指定数学模型进行模型训练。
网络构建单元，用于由完成模型训练的指定数学模型构建得到所述残差神经网络。
在一示例性实施例中，所述装置还包括但不限于：显示模块和编辑模块。
其中，显示模块，用于根据所述地图要素在所述目标场景中的三维位置，在目标场景地图中显示所述地图要素。
编辑模块，用于获取针对所述目标场景地图中地图要素的控制指令并响应所述控制指令，生成所述目标场景的高精度地图。
需要说明的是，上述实施例所提供的地图要素提取装置在进行地图要素提取时，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即地图要素提取装置的内部结构将划分为不同的功能模块，以完成以上描述的全部或者部分功能。
另外，上述实施例所提供的地图要素提取装置与地图要素提取方法的实施例属于同一构思，其中各个模块执行操作的具体方式已经在方法实施例中进行了详细描述，此处不再赘述。
请参阅图16，在一示例性实施例中，一种服务器1000，包括至少一处理器1001、至少一存储器1002、以及至少一通信总线1003。
其中，存储器1002上存储有计算机可读指令，处理器1001通过通信总线1003读取存储器1002中存储的计算机可读指令。
该计算机可读指令被处理器1001执行时实现上述各实施例中的地图要素提取方法。
在一示例性实施例中，一种计算机可读存储介质，其上存储有计算机程序，该计算机程序被处理器执行时实现上述各实施例中的地图要素提取方法。
上述内容为本申请的示例性实施例，并非用于限制本申请实施例的实施方案，本领域普通技术人员根据本申请实施例的主要构思和精神，可以十分方便地进行相应的变通或修改，故本申请实施例的保护范围应以权利要求书所要求的保护范围为准。

Claims (15)

  1. 一种地图要素提取方法，由电子设备执行，包括：
    获取目标场景的激光点云和图像，所述目标场景包括至少一要素实体对应于地图要素；
    进行所述激光点云与所述图像之间的配准，得到所述图像的深度图；
    对所述图像的深度图进行图像分割，得到所述地图要素在所述深度图中的分割图像；
    根据所述激光点云与所述图像之间的配准关系，将所述分割图像在所述深度图中的二维位置转换为所述地图要素在所述目标场景中的三维位置。
  2. 如权利要求1所述的方法，所述进行所述激光点云与所述图像之间的配准，得到所述图像的深度图，包括：
    对所述激光点云与所述图像进行配准，得到所述图像中像素点对应的深度信息；
    根据所述图像中像素点对应的深度信息，为所述图像构建所述深度图。
  3. 如权利要求2所述的方法，所述对所述激光点云与所述图像进行配准，得到所述图像中像素点对应的深度信息，包括：
    构建所述激光点云与所述图像之间的投影变换函数；
    提取所述激光点云与所述图像中相对应的特征点，并根据提取到的特征点估计所述投影变换函数的参数；
    根据完成参数估计的投影变换函数，计算得到所述图像中像素点对应的深度信息。
  4. 如权利要求3所述的方法，所述根据所述激光点云与所述图像之间的配准关系，将所述分割图像在所述深度图中的二维位置转换为所述地图要素在所述目标场景中的三维位置，包括：
    将所述分割图像在所述深度图中的二维位置以及所述图像中像素点对应的深度信息，输入完成参数估计的投影变换函数，计算得到所述地图要素在所述目标场景中的三维位置。
  5. 如权利要求1所述的方法，所述对所述图像的深度图进行图像分割，得到所述地图要素在所述深度图中的分割图像，包括：
    对所述图像的深度图进行特征提取，得到所述图像对应的特征图；
    对所述特征图中的像素点进行类别预测，得到所述特征图中像素点的类别；
    将所述特征图中同一类别的像素点，拟合形成对应地图要素在所述深度图中的分割图像，其中，每一类别对应一种地图要素。
  6. 如权利要求5所述的方法，所述对所述图像的深度图进行特征提取，得到所述图像对应的特征图，包括：
    采用残差神经网络中的高层网络提取得到所述深度图的全局特征，并采用所述残差神经网络中的低层网络提取得到所述深度图的局部特征；
    对提取得到的全局特征与局部特征进行融合，得到中间特征图；
    对所述中间特征图进行线性插值，得到所述图像对应的特征图。
  7. 如权利要求6所述的方法，所述对提取得到的全局特征与局部特征进行融合，得到中间特征图，包括：
    对所述残差神经网络中最高一层网络对应的全局特征进行反卷积和上采样处理，得到融合特征图；
    对所述残差神经网络中次高一层网络对应的全局特征进行反卷积处理，并通过与所述融合特征图的融合，更新所述融合特征图；
    对所述更新的融合特征图进行上采样处理，二次更新所述融合特征图；
    按照所述残差神经网络中网络由高至低的顺序，对其余高层网络对应的全局特征和低层网络对应的局部特征进行遍历，根据遍历到的全局特征或者局部特征对所述二次更新后的融合特征图进行更新；
    完成所述遍历后，以最后更新的所述融合特征图作为所述中间特征图。
  8. 如权利要求6所述的方法，还包括：
    获取图像样本，所述图像样本进行了像素点类别标注；
    根据获取到的图像样本引导指定数学模型进行模型训练；
    由完成模型训练的指定数学模型构建得到所述残差神经网络。
  9. 如权利要求1至8任一项所述的方法，还包括：
    根据所述地图要素在所述目标场景中的三维位置，在目标场景地图中显示所述地图要素；
    获取针对所述目标场景地图中地图要素的控制指令并响应所述控制指令，生成所述目标场景的高精度地图。
  10. 一种地图要素提取装置，包括：
    图像获取模块，用于获取目标场景的激光点云和图像，所述目标场景包括至少一要素实体对应于地图要素；
    深度图构建模块，用于进行所述激光点云与所述图像之间的配准，得到所述图像的深度图；
    图像分割模块，用于对所述图像的深度图进行图像分割，得到所述地图要素在所述深度图中的分割图像；
    位置转换模块，用于根据所述激光点云与所述图像之间的配准关系，将所述分割图像在所述深度图中的二维位置转换为所述地图要素在所述目标场景中的三维位置。
  11. 如权利要求10所述的装置，所述深度图构建模块包括：
    配准单元，用于对所述激光点云与所述图像进行配准，得到所述图像中像素点对应的深度信息；
    构建单元，用于根据所述图像中像素点对应的深度信息，为所述图像构建所述深度图。
  12. 如权利要求11所述的装置，所述配准单元包括：
    函数构建子单元，用于构建所述激光点云与所述图像之间的投影变换函数；
    特征点提取子单元，用于提取所述激光点云与所述图像中相对应的特征点，并根据提取到的特征点估计所述投影变换函数的参数；
    信息计算子单元，用于根据完成参数估计的投影变换函数，计算得到所述图像中像素点对应的深度信息。
  13. 如权利要求10所述的装置，所述图像分割模块包括：
    特征提取单元，用于对所述图像的深度图进行特征提取，得到所述图像对应的特征图；
    类别预测单元，用于对所述特征图中的像素点进行类别预测，得到所述特征图中像素点的类别；
    拟合单元，用于将所述特征图中同一类别的像素点，拟合形成对应地图要素在所述深度图中的分割图像，其中，每一类别对应一种地图要素。
  14. 一种服务器，包括：
    处理器；及
    存储器，所述存储器上存储有计算机可读指令，所述计算机可读指令被所述处理器执行时实现如权利要求1至9中任一项所述的地图要素提取方法。
  15. 一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现如权利要求1至9中任一项所述的地图要素提取方法。
PCT/CN2019/110259 2018-10-12 2019-10-10 地图要素提取方法、装置及服务器 WO2020073936A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP19871308.3A EP3779358B1 (en) 2018-10-12 2019-10-10 Map element extraction method and apparatus
US17/075,134 US11380002B2 (en) 2018-10-12 2020-10-20 Map element extraction method and apparatus, and server

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811186664.6 2018-10-12
CN201811186664.6A CN110160502B (zh) 2018-10-12 2018-10-12 地图要素提取方法、装置及服务器

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/075,134 Continuation US11380002B2 (en) 2018-10-12 2020-10-20 Map element extraction method and apparatus, and server

Publications (1)

Publication Number Publication Date
WO2020073936A1 true WO2020073936A1 (zh) 2020-04-16

Family

ID=67645031

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/110259 WO2020073936A1 (zh) 2018-10-12 2019-10-10 地图要素提取方法、装置及服务器

Country Status (4)

Country Link
US (1) US11380002B2 (zh)
EP (1) EP3779358B1 (zh)
CN (1) CN110160502B (zh)
WO (1) WO2020073936A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112232271A (zh) * 2020-10-29 2021-01-15 上海有个机器人有限公司 一种基于激光的人流检测方法以及设备
CN112686197A (zh) * 2021-01-07 2021-04-20 腾讯科技（深圳）有限公司 一种数据处理方法和相关装置
CN113724329A (zh) * 2021-09-01 2021-11-30 中国人民大学 融合平面与立体信息的目标姿态估计方法、系统和介质
CN113902047A (zh) * 2021-12-10 2022-01-07 腾讯科技（深圳）有限公司 图像元素匹配方法、装置、设备以及存储介质

Families Citing this family (36)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110160502B (zh) * 2018-10-12 2022-04-01 腾讯科技（深圳）有限公司 地图要素提取方法、装置及服务器
CN110488830B (zh) * 2019-08-26 2021-11-19 吉林大学 面向智能车辆速度节能规划的高精度地图信息预解析系统及预解析方法
CN110598597A (zh) * 2019-08-29 2019-12-20 上海眼控科技股份有限公司 一种多场景路口信息分类与提取方法及设备
CN110956100A (zh) * 2019-11-15 2020-04-03 北京三快在线科技有限公司 一种高精度地图生成方法、装置、电子设备和存储介质
KR20210061722A (ko) * 2019-11-20 2021-05-28 팅크웨어(주) 고정밀 지도 제작 방법, 고정밀 지도 제작 장치, 컴퓨터 프로그램 및 컴퓨터 판독 가능한 기록 매체
CN111192311B (zh) * 2019-12-31 2022-05-17 武汉中海庭数据技术有限公司 一种高精度地图制作中纵向减速标线自动提取方法和装置
CN111311709B (zh) * 2020-02-05 2023-06-20 北京三快在线科技有限公司 一种生成高精地图的方法及装置
CN111414968B (zh) * 2020-03-26 2022-05-03 西南交通大学 一种基于卷积神经网络特征图的多模态遥感影像匹配方法
KR20210121628A (ko) * 2020-03-31 2021-10-08 주식회사 만도 강화학습 기반 포인트 클라우드 자동화 처리 방법 및 그 시스템
CN111523409B (zh) * 2020-04-09 2023-08-29 北京百度网讯科技有限公司 用于生成位置信息的方法和装置
CN111667545B (zh) * 2020-05-07 2024-02-27 东软睿驰汽车技术（沈阳）有限公司 高精度地图生成方法、装置、电子设备及存储介质
CN113724311B (zh) * 2020-05-25 2024-04-02 北京四维图新科技股份有限公司 深度图获取方法、设备及存储介质
CN111652179B (zh) * 2020-06-15 2024-01-09 东风汽车股份有限公司 基于点线特征融合激光的语义高精地图构建与定位方法
CN113936018A (zh) * 2020-06-29 2022-01-14 千寻位置网络有限公司 地面印刷物的边界提取方法
CN111797189B (zh) * 2020-07-03 2024-03-19 武汉四维图新科技有限公司 数据源质量评价方法与装置、设备、计算机可读存储介质
CN111881790B (zh) * 2020-07-14 2023-04-28 武汉中海庭数据技术有限公司 一种高精度地图制作中道路人行横道自动化提取方法和装置
CN114120795B (zh) * 2020-09-01 2023-03-10 华为技术有限公司 一种地图绘制方法及装置
CN112184901B (zh) * 2020-09-08 2024-04-19 北京三快在线科技有限公司 一种深度图确定方法及装置
CN112740225B (zh) * 2020-09-30 2022-05-13 华为技术有限公司 一种路面要素确定方法及装置
US11681780B2 (en) * 2020-09-30 2023-06-20 Nissan North America, Inc. Annotation and mapping for vehicle operation in low-confidence object detection conditions
CN112258568B (zh) * 2020-10-12 2022-07-01 武汉中海庭数据技术有限公司 一种高精度地图要素的提取方法及装置
CN112433193B (zh) * 2020-11-06 2023-04-07 山东产研信息与人工智能融合研究院有限公司 一种基于多传感器的模位置定位方法及系统
CN113034566B (zh) * 2021-05-28 2021-09-24 湖北亿咖通科技有限公司 高精度地图构建方法、装置、电子设备及存储介质
CN113409459B (zh) * 2021-06-08 2022-06-24 北京百度网讯科技有限公司 高精地图的生产方法、装置、设备和计算机存储介质
CN113514053B (zh) * 2021-07-13 2024-03-26 阿波罗智能技术(北京)有限公司 生成样本图像对的方法、装置和更新高精地图的方法
CN113674287A (zh) * 2021-09-03 2021-11-19 阿波罗智能技术(北京)有限公司 高精地图的绘制方法、装置、设备以及存储介质
CN113657691B (zh) * 2021-10-19 2022-03-01 北京每日优鲜电子商务有限公司 信息显示方法、装置、电子设备和计算机可读介质
CN114155415B (zh) * 2021-12-07 2024-05-03 华东交通大学 一种多数据融合的车辆检测方法、系统、设备及存储介质
CN114413881B (zh) * 2022-01-07 2023-09-01 中国第一汽车股份有限公司 高精矢量地图的构建方法、装置及存储介质
WO2023164845A1 (zh) * 2022-03-02 2023-09-07 深圳市大疆创新科技有限公司 三维重建方法、装置、系统及存储介质
CN114663612A (zh) * 2022-03-24 2022-06-24 北京百度网讯科技有限公司 一种高精度地图构建方法、装置及电子设备
CN115019048B (zh) * 2022-07-08 2023-04-07 北京百度网讯科技有限公司 三维场景分割方法、模型训练方法、装置和电子设备
CN115410173B (zh) * 2022-11-01 2023-03-24 北京百度网讯科技有限公司 多模态融合的高精地图要素识别方法、装置、设备及介质
CN116189145A (zh) * 2023-02-15 2023-05-30 清华大学 一种线形地图要素的提取方法、系统和可读介质
CN116499477B (zh) * 2023-06-21 2023-09-26 小米汽车科技有限公司 地图的融合方法、装置、介质及车辆
CN117036576B (zh) * 2023-10-09 2024-01-30 腾讯科技（深圳）有限公司 地图渲染方法、装置、电子设备及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102308320A (zh) * 2009-02-06 2012-01-04 香港科技大学 从图像生成三维模型
CN103017739A (zh) * 2012-11-20 2013-04-03 武汉大学 基于激光雷达点云与航空影像的真正射影像的制作方法
CN104599314A (zh) * 2014-06-12 2015-05-06 深圳奥比中光科技有限公司 三维模型重建方法与系统
US9177481B2 (en) * 2013-12-13 2015-11-03 Sikorsky Aircraft Corporation Semantics based safe landing area detection for an unmanned vehicle
CN110160502A (zh) * 2018-10-12 2019-08-23 腾讯科技（深圳）有限公司 地图要素提取方法、装置及服务器

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN100559398C (zh) * 2007-06-19 2009-11-11 北京航空航天大学 自动的深度图像配准方法
CN102473303B (zh) * 2009-08-12 2015-07-29 皇家飞利浦电子股份有限公司 生成对象数据
AU2012244276A1 (en) * 2012-10-30 2014-05-15 Canon Kabushiki Kaisha Method, apparatus and system for detecting a supporting surface region in an image
US9111444B2 (en) * 2012-10-31 2015-08-18 Raytheon Company Video and lidar target detection and tracking system and method for segmenting moving targets
CN103150747A (zh) * 2012-11-16 2013-06-12 佳都新太科技股份有限公司 一种基于拓扑特征的点云配准算法
CN103955939B (zh) * 2014-05-16 2018-06-19 重庆理工大学 三维扫描系统中点云拼接用边界特征点配准方法
US10282591B2 (en) * 2015-08-24 2019-05-07 Qualcomm Incorporated Systems and methods for depth map sampling
CN106485690A (zh) * 2015-08-25 2017-03-08 南京理工大学 基于点特征的点云数据与光学影像的自动配准融合方法
CN108088444A (zh) * 2016-11-22 2018-05-29 广州映博智能科技有限公司 基于三维激光的室内点云地图生成系统及方法
CN107403430B (zh) * 2017-06-15 2020-08-07 中山大学 一种rgbd图像语义分割方法
CN107622244B (zh) * 2017-09-25 2020-08-28 华中科技大学 一种基于深度图的室内场景精细化解析方法
CN107993282B (zh) * 2017-11-06 2021-02-19 江苏省测绘研究所 一种动态的可量测实景地图制作方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102308320A (zh) * 2009-02-06 2012-01-04 香港科技大学 从图像生成三维模型
CN103017739A (zh) * 2012-11-20 2013-04-03 武汉大学 基于激光雷达点云与航空影像的真正射影像的制作方法
US9177481B2 (en) * 2013-12-13 2015-11-03 Sikorsky Aircraft Corporation Semantics based safe landing area detection for an unmanned vehicle
CN104599314A (zh) * 2014-06-12 2015-05-06 深圳奥比中光科技有限公司 三维模型重建方法与系统
CN110160502A (zh) * 2018-10-12 2019-08-23 腾讯科技（深圳）有限公司 地图要素提取方法、装置及服务器

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
See also references of EP3779358A4 *
WANG KE: "Mobile Robot 3D Environment Modeling Based on Variational Model", CHINESE DOCTORAL DISSERTATIONS FULL-TEXT DATABASE, 1 June 2016 (2016-06-01), pages 1 - 141, XP055742723, ISSN: 1674-022X *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112232271A (zh) * 2020-10-29 2021-01-15 上海有个机器人有限公司 一种基于激光的人流检测方法以及设备
CN112232271B (zh) * 2020-10-29 2023-09-12 上海有个机器人有限公司 一种基于激光的人流检测方法以及设备
CN112686197A (zh) * 2021-01-07 2021-04-20 腾讯科技（深圳）有限公司 一种数据处理方法和相关装置
CN112686197B (zh) * 2021-01-07 2022-08-19 腾讯科技（深圳）有限公司 一种数据处理方法和相关装置
CN113724329A (zh) * 2021-09-01 2021-11-30 中国人民大学 融合平面与立体信息的目标姿态估计方法、系统和介质
CN113902047A (zh) * 2021-12-10 2022-01-07 腾讯科技（深圳）有限公司 图像元素匹配方法、装置、设备以及存储介质

Also Published As

Publication number Publication date
US20210035314A1 (en) 2021-02-04
CN110160502A (zh) 2019-08-23
EP3779358A4 (en) 2021-11-10
EP3779358B1 (en) 2023-12-06
EP3779358A1 (en) 2021-02-17
CN110160502B (zh) 2022-04-01
US11380002B2 (en) 2022-07-05

Similar Documents

Publication Publication Date Title
WO2020073936A1 (zh) 地图要素提取方法、装置及服务器
CN109493407B (zh) 实现激光点云稠密化的方法、装置及计算机设备
JP6862409B2 (ja) 地図生成及び移動主体の位置決めの方法及び装置
US11698449B2 (en) User interface for displaying point clouds generated by a LiDAR device on a UAV
JP6745328B2 (ja) 点群データを復旧するための方法及び装置
US8467810B2 (en) Method and system for reporting errors in a geographic database
US10297074B2 (en) Three-dimensional modeling from optical capture
JP7440005B2 (ja) 高精細地図の作成方法、装置、デバイス及びコンピュータプログラム
WO2021223465A1 (zh) 一种高精度地图构建方法及系统
CN113139453B (zh) 一种基于深度学习的正射影像高层建筑基底矢量提取方法
Li et al. Camera localization for augmented reality and indoor positioning: a vision-based 3D feature database approach
CN111739005B (zh) 图像检测方法、装置、电子设备及存储介质
CN113989450A (zh) 图像处理方法、装置、电子设备和介质
WO2023155581A1 (zh) 一种图像检测方法和装置
US10089417B2 (en) Structure determination in a geographic area
WO2023164845A1 (zh) 三维重建方法、装置、系统及存储介质
US20230177707A1 (en) Post-processing of mapping data for improved accuracy and noise-reduction
WO2022077790A1 (en) Representation data generation of three-dimensional mapping data
CN113932796A (zh) 高精地图车道线生成方法、装置和电子设备
CN114627438A (zh) 目标检测模型生成方法、目标检测方法、设备及介质
CN115937449A (zh) 高精地图生成方法、装置、电子设备和存储介质
CN115578432B (zh) 图像处理方法、装置、电子设备及存储介质
CN116883504B (zh) 车辆朝向的校准方法、装置、计算机设备及存储介质
CN113487741B (zh) 稠密三维地图更新方法及装置
CN115993121A (zh) 一种室内移动机器人三维地图构建与维护方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19871308

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2019871308

Country of ref document: EP

Effective date: 20201014

NENP Non-entry into the national phase

Ref country code: DE