WO2022257358A1 - 高精地图的生产方法、装置、设备和计算机存储介质 - Google Patents

高精地图的生产方法、装置、设备和计算机存储介质 Download PDF

Info

Publication number
WO2022257358A1
WO2022257358A1 PCT/CN2021/131180 CN2021131180W WO2022257358A1 WO 2022257358 A1 WO2022257358 A1 WO 2022257358A1 CN 2021131180 W CN2021131180 W CN 2021131180W WO 2022257358 A1 WO2022257358 A1 WO 2022257358A1
Authority
WO
WIPO (PCT)
Prior art keywords
point cloud
point
registration
sequence
data
Prior art date
Application number
PCT/CN2021/131180
Other languages
English (en)
French (fr)
Inventor
夏德国
黄际洲
王海峰
Original Assignee
北京百度网讯科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京百度网讯科技有限公司 filed Critical 北京百度网讯科技有限公司
Priority to EP21916644.4A priority Critical patent/EP4174786A4/en
Priority to KR1020227023591A priority patent/KR20220166779A/ko
Priority to JP2022541610A priority patent/JP7440005B2/ja
Priority to US17/758,692 priority patent/US20240185379A1/en
Publication of WO2022257358A1 publication Critical patent/WO2022257358A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/38—Electronic maps specially adapted for navigation; Updating thereof
    • G01C21/3804—Creation or updating of map data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/14—Transformations for image registration, e.g. adjusting or mapping for alignment of images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/05—Geographic models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/80—Geometric correction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/38—Registration of image sequences
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle

Definitions

  • the present disclosure relates to the field of computer application technology, in particular to automatic driving and deep learning technology in the field of artificial intelligence technology.
  • high-precision map is one of the key factors to promote the development of automatic driving.
  • Traditional maps have low accuracy and can only provide road-level route planning.
  • high-precision maps can help you know location information in advance, accurately plan driving routes, predict complex road surface information, and better avoid potential risks, etc. Therefore, how to realize the production of high-precision maps has become an urgent problem to be solved.
  • the present disclosure provides a high-definition map production method, device, equipment and computer storage medium.
  • a method for producing a high-precision map including:
  • Map elements are identified on the top view to obtain high-precision map data.
  • a high-precision map production device including:
  • the acquiring unit is used to acquire the point cloud data and the front view image data respectively collected by the acquisition device at each position point, so as to obtain the point cloud sequence and the front view image sequence;
  • a registration unit configured to register the point cloud sequence and the front view image sequence with the front view image and point cloud data
  • a conversion unit configured to convert the front view image sequence into a top view according to the registration result and determine the coordinate information of each pixel in the top view
  • the identification unit is configured to identify map elements on the top view to obtain high-precision map data.
  • an electronic device including:
  • the memory stores instructions executable by the at least one processor, the instructions are executed by the at least one processor to enable the at least one processor to perform the method as described above.
  • a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method as described above.
  • a computer program product comprises a computer program which, when executed by a processor, implements the method as described above.
  • FIG. 1 is a flowchart of a production method of a high-precision map provided by an embodiment of the present disclosure
  • FIG. 2 is a flow chart of a preferred registration process provided by an embodiment of the present disclosure
  • FIG. 3 is a flow chart of a method for frame-by-frame registration of point cloud data provided by an embodiment of the present disclosure
  • Fig. 4a and Fig. 4b are respectively the example figure of front view image and top view
  • FIG. 5 is a structural diagram of a high-precision map production device provided by an embodiment of the present disclosure.
  • FIG. 6 is a block diagram of an electronic device used to implement an embodiment of the present disclosure.
  • high-precision maps Although there are already some productions of high-precision maps, they are mainly based on point cloud technology. That is, a large amount of dense point cloud data is collected by the lidar equipment, and after processing and identifying the point cloud data, information such as roads and ground signs are obtained, and then these data are manually corrected to finally generate high-precision map data.
  • this traditional approach highly relies on point cloud data.
  • due to the complex spatial structure of urban roads in order to ensure the accuracy of high-precision maps, a large amount of manpower is required for registration, resulting in low production efficiency of high-precision maps, high labor costs, and high requirements for the professional skills of operators. Ultimately, it will affect the large-scale production of high-precision maps.
  • the present disclosure provides a production method of a high-definition map that is different from the above-mentioned traditional methods.
  • the method provided by the present disclosure will be described in detail below with reference to the embodiments.
  • Fig. 1 is a flowchart of a production method of a high-precision map provided by an embodiment of the present disclosure.
  • the execution body of the method may be a recommendation device, which may be an application located on a local terminal, or may also be a plug-in in an application located on a local terminal or a functional unit such as a software development kit (Software Development Kit, SDK), or may also be located at the server side, which is not particularly limited in the embodiment of the present invention.
  • the method may include:
  • the point cloud data and front view image data respectively collected by the acquisition device at each position point are obtained to obtain a point cloud sequence and a front view image sequence.
  • the point cloud sequence and the front-view image sequence are registered with the front-view image and point cloud data.
  • step 103 the orthographic image sequence is converted into a top view according to the registration result, and the coordinate information of each pixel in the top view is determined.
  • map elements are identified on the top view to obtain high-precision map data.
  • the idea of the present disclosure is to fuse the image data collected by the image acquisition device with the point cloud data collected by the laser radar device, so as to realize the automatic registration of mutual fusion, and generate the final result based on the registration result.
  • HD map This method does not require a lot of extra manpower for manual registration, improves production efficiency, reduces labor costs and requires professional skills for operators, and provides a basis for large-scale production of high-precision maps.
  • step 101 that is, "obtain point cloud data and front-view image data respectively collected by the acquisition device at each position point, to obtain a point cloud sequence and front-view image sequence" will be described in detail in conjunction with an embodiment.
  • the acquisition equipment involved in this step mainly includes the following two types:
  • Image acquisition devices such as cameras and video cameras, can perform image acquisition at regular intervals or after being triggered.
  • the laser radar device can acquire the data of the collection of reflection points on the surface of the surrounding environment by emitting laser scanning at regular intervals or after being triggered, that is, point cloud data.
  • point cloud data include point coordinate information, usually the coordinate information is the coordinates in the lidar device coordinate system.
  • GNSS Global Navigation Satellite System, global satellite navigation system
  • a movable device such as a collection vehicle
  • each collection equipment performs data collection according to a certain frequency, or is triggered to perform data collection at the same point. collection.
  • the orthographic images collected by the image acquisition device according to a certain acquisition frequency constitute an orthographic image sequence Among them, I i is a frame of orthographic image collected at time t i .
  • the point cloud data collected by the laser radar device according to a certain acquisition frequency constitutes a point cloud sequence
  • P i is a frame of point cloud data collected at time t i .
  • the position data collected by the position acquisition device according to a certain acquisition frequency constitutes a position sequence where L i is the location data collected at time t i .
  • the above N is the number of data collections performed by the collection device, that is, the amount of data obtained by each collection device.
  • clock synchronization and/or joint calibration can be performed on the acquisition device in advance.
  • the specific synchronization method can choose GPS-based "PPS (Pulse Per Second, pulses per second) + NMEA (National Marine Electronics Association, National Marine Electronics Association)", or Ethernet-based IEEE 1588 (or IEEE 802.1AS) Clock synchronization protocol.
  • PPS Pulse Per Second, pulses per second
  • NMEA National Marine Electronics Association, National Marine Electronics Association
  • Ethernet-based IEEE 1588 or IEEE 802.1AS
  • the joint calibration of the acquisition equipment is mainly to obtain the internal and external parameter information of the image acquisition equipment in the acquisition equipment, the external parameter information of the laser radar equipment, the transformation from the laser radar coordinate system to the image acquisition equipment coordinate system and the translation matrix M 1 , and an internal reference information matrix M 2 of the image acquisition device.
  • the way of joint calibration is mainly to preset a calibration board, adjust the lidar equipment and image acquisition equipment to take pictures and capture point clouds of the calibration board. Then find at least three corresponding two-dimensional points on the image and three-dimensional points of the point cloud, that is, three point pairs are formed. Using these three point pairs to perform PNP (perspective-n-point, multi-point perspective imaging) solution, the transformation relationship between the coordinate system of the lidar device and the coordinate system of the image acquisition device can be obtained.
  • PNP perspective-n-point, multi-point perspective imaging
  • step 102 that is, "registering the point cloud sequence and the orthographic image sequence with the orthographic image and the point cloud data" will be described in detail below in conjunction with an embodiment.
  • FIG. 2 is a flowchart of a preferred registration process provided by an embodiment of the present disclosure. As shown in FIG. 2, the process may include the following steps:
  • the adjacent images in the front-view image sequence are registered to obtain a set of corresponding pixels in the adjacent images.
  • the K pixels in are the sum image
  • the K pixels in are corresponding, respectively expressed as a set: where the image of pixels with image pixels correspond, and correspond, and so on.
  • the feature-based method mainly includes: determining the features of each pixel in the two frames of images, where the features can be such as SIFT (Scale-invariant feature transform, scale-invariant feature transform); and then performing feature matching based on similarity, so as to obtain corresponding pixels. For example, the matching between two pixel points whose similarity between features exceeds the preset similarity threshold is successful.
  • SIFT Scale-invariant feature transform, scale-invariant feature transform
  • the deep learning method mainly includes: using such as convolutional neural network, VGG (Visual Geometry Group Network, visual geometry group network) layer to generate the feature vector representation of each pixel, and then perform feature vector representation based on the feature vector representation of each pixel in the two frames of images. Match to get the corresponding pixel. For example, the feature vector indicates that the matching between two pixel points whose similarity exceeds a preset similarity threshold is successful.
  • VGG Visual Geometry Group Network, visual geometry group network
  • distortion correction is performed on the point cloud data according to the amount of movement of the lidar device that collects the point cloud data for one revolution.
  • This step is preferably performed, and helps to improve the accuracy of the point cloud data in the subsequent registration process.
  • the image acquisition device uses a global shutter, which can be considered to be acquired in an instant.
  • the laser radar equipment is not obtained instantaneously, but is usually collected after the transmitter and receiver rotate a circle, that is, 360 degrees. Assuming that one revolution is 100ms, then in a frame of point cloud data formed in one acquisition period, the difference between the first point and the last point is 100ms, and the lidar device is collected during motion, so the point Cloud data is distorted and cannot truly reflect the real environment at a certain moment. In order to better register the image data and the point cloud data, distortion correction is performed on the point cloud data in this step.
  • the lidar calculates the laser point coordinates based on the lidar's own coordinate system when the laser beam is received, the reference coordinate system of each column of laser points is different during the movement of the lidar. But they are in the same frame point cloud, so they need to be unified in the same coordinate system during the process of distortion correction.
  • the idea of distortion correction is to calculate the movement of the lidar during the acquisition process, and then compensate the movement amount on each frame point cloud, including the compensation of rotation and translation.
  • First determine the first laser point in a frame point cloud the subsequent laser points can determine the rotation angle and translation relative to the first laser point, and then perform the compensation conversion of first rotation and then translation to get the corrected Coordinate information of the laser point.
  • a set of corresponding point clouds in adjacent images may also be determined.
  • the projection from the point cloud to the image can be obtained first according to the internal reference information matrix of the image acquisition device, the rotation matrix from the coordinate system of the image acquisition device to the image plane, and the conversion and translation matrix from the lidar coordinate system to the coordinate system of the image acquisition device matrix; the point cloud data is then projected onto the image using the projection matrix.
  • the corresponding set of point clouds in adjacent images can be determined. Assuming two consecutive frames of images and image After the above projection, the projection to the image is obtained The set of K 1 points in and projected to the image In the set of K 2 points, the two sets are intersected, which is the image and image The collection of corresponding point clouds in The use of this collection will be covered in subsequent examples.
  • a reference point cloud in the point cloud sequence is determined.
  • the first frame in the point cloud sequence can be used as the reference point cloud.
  • the point cloud of the first frame may not be the most accurate in the point cloud sequence. Therefore, a preferred way of determining a reference point cloud is provided in the present disclosure.
  • the first frame in the point cloud sequence can be used as a reference, and other point cloud data can be registered frame by frame; in the point cloud image sequence, the frame with the highest proportion of registration points with the two frames of point cloud data before and after The point cloud is used as the reference point cloud.
  • a transformation matrix between two frames of point clouds is learned from the reference point cloud and its unregistered adjacent point clouds.
  • the point cloud of the first frame can theoretically be obtained after rotating and translating the point cloud of the second frame.
  • a method such as ICP (Iterative Closest Point, Iterative Closest Point) can be used to learn the transformation matrix.
  • the rotation matrix is expressed as R
  • the translation matrix is expressed as t
  • the loss function can be: each point in the point cloud as a reference is converted according to the transformation matrix and each conversion point in the adjacent point cloud The average or weighted average of the distances between the nearest points of each conversion point.
  • loss function For example, the following loss function can be used:
  • E(R, t) represents the loss function
  • xi represents the point cloud as the reference point cloud such as the point in the first frame point cloud
  • R( xi )+t represents the conversion of xi according to the transformation matrix
  • n is the number of points that can be matched.
  • loss function can also be used:
  • the weighting coefficient w i is added, and its value can be determined according to whether the point in the point cloud as a reference belongs to the set of the corresponding point cloud to make sure.
  • the following formula can be used:
  • DGR Deep global registration, deep global registration
  • the transformation matrix is used to transform the reference point cloud to obtain the registered adjacent point cloud.
  • the registered points in the point cloud of the second frame are obtained.
  • step 304 it is judged whether there are adjacent point clouds that have not yet been registered in the point cloud sequence, and if yes, step 304 is performed; otherwise, the current registration process is ended.
  • step 304 use the adjacent point cloud as a new reference, and go to step 301.
  • the registration point ratio A j of the point cloud P j of the jth frame can be determined by the following formula:
  • match() indicates the intersection of the points that can be registered in the two frames of point clouds before and after, which can be reflected as the intersection of each point obtained after converting the point cloud of one frame according to the transformation matrix and each point of the point cloud of another frame.
  • indicates the number of points in the set, for example
  • the reference point cloud After the reference point cloud is determined, use the method shown in Figure 3 to register other point cloud data frame by frame based on the reference point cloud. If the reference point cloud is the first frame, the point clouds of subsequent frames are registered sequentially. If the reference point cloud is not the first frame, the point cloud of each frame is registered forward and backward based on the reference point cloud. Finally, the registered point cloud sequence is obtained.
  • the registered point cloud data is projected into the set obtained in step 201, and the coordinate information of each pixel in the set is obtained.
  • it may include projecting the coordinates of the point cloud data to the set to obtain the coordinate information of the point cloud corresponding to the pixel in the front-view image;
  • the coordinate information of the point cloud is converted into the coordinate information of the pixel.
  • the above-mentioned set is actually corresponding pixels obtained after adjacent image registration.
  • the coordinate information of the point cloud corresponding to each pixel in these sets in the front-view image can be obtained.
  • step 103 that is, "converting the front-view image sequence into a top view according to the registration result and determining the coordinate information of each pixel in the top view" will be described in detail below in conjunction with an embodiment.
  • each frame of the front-view image in the front-view sequence can be converted into a top view based on inverse perspective transformation first; then, the coordinate information of each pixel in the top view is determined by matching on the top view according to the coordinate information of the pixels in the front-view image.
  • R x the horizontal resolution of the image acquisition device
  • R y vertical resolution of the image acquisition device.
  • the inverse perspective transformation model can be expressed as follows:
  • h is the height of the image acquisition device from the ground
  • cot() is the cotangent function.
  • the front view image such as that shown in FIG. 4a can be transformed into a top view image as shown in FIG. 4b.
  • each frame of front-view images in the front-view sequence can be converted to obtain a top view. If there are N frames of front-view images, N top views can be obtained. These top views actually overlap each other, especially two adjacent top views, most of the areas overlap. Since the coordinate information of the pixels in the top view can be obtained in the above process, stitching can be performed one by one based on the position information of the pixels in each top view, and finally a high-definition map can be obtained.
  • step 104 that is, "recognizing map elements on the top view to obtain high-precision map data" will be described in detail below in conjunction with an embodiment.
  • road information identification can be performed on the bird's-eye view obtained in step 103; then, the recognized road information can be superimposed on the bird's-eye view to obtain high-precision map data.
  • the road information can include lane lines, lane line types (such as white solid lines, single yellow solid lines, double yellow solid lines, yellow dashed solid lines, diversion lines, yellow no-stop lines, etc.), colors, and lane guide arrow information , lane type (such as main lane, bus lane, tidal lane, etc.), etc.
  • lane line types such as white solid lines, single yellow solid lines, double yellow solid lines, yellow dashed solid lines, diversion lines, yellow no-stop lines, etc.
  • colors and lane guide arrow information
  • lane type such as main lane, bus lane, tidal lane, etc.
  • a semantic segmentation model based on a deep neural network can be used to segment road information, such as DeepLabV3. It is also possible to use image recognition technology based on deep neural network to identify the above road information, such as Faster-RCNN (Regions with CNN features, convolutional neural network feature area).
  • the recognition based on the top view above is mainly the recognition of ground elements, that is, the recognition of road information.
  • map elements such as traffic signs, buildings, etc.
  • This part may adopt methods in the prior art, which are not limited in this disclosure.
  • the operator can directly compare the top view data with the superimposed road information, correct the problematic data, and produce the final high-precision map data.
  • FIG. 5 is a structural diagram of a high-precision map production device provided by an embodiment of the present disclosure.
  • the device 500 may include: an acquisition unit 510 , a registration unit 520 , a conversion unit 530 and an identification unit 540 .
  • the main functions of each component unit are as follows:
  • the acquiring unit 510 is configured to acquire point cloud data and front-view image data respectively collected by the collection device at each position point, to obtain a point cloud sequence and a front-view image sequence.
  • the above-mentioned collection device at least includes an image collection device for collecting front-view images and a laser radar device for collecting point cloud data.
  • clock synchronization and/or joint calibration can be performed on the acquisition device in advance.
  • the registration unit 520 is configured to perform registration between the front view image and the point cloud data of the point cloud sequence and the front view image sequence.
  • the conversion unit 530 is configured to convert the front view image sequence into a top view according to the registration result and determine the coordinate information of each pixel in the top view.
  • the identification unit 540 is configured to identify map elements in the top view to obtain high-precision map data.
  • the registration unit 520 may include: a first registration subunit 521 and a projection subunit 522, and may further include a correction subunit 523, a reference subunit 524, a second registration subunit 525, and a third registration subunit. Unit 526.
  • the first registration subunit 521 is configured to register adjacent images in the front-view image sequence to obtain a set of corresponding pixels in the adjacent images.
  • the projection subunit 522 is used to project the point cloud data to the set to obtain the coordinate information of each pixel in the set.
  • the correction subunit 523 is configured to provide the projection subunit 522 with distortion correction on the point cloud data according to the amount of movement of the lidar device that collects the point cloud data for one rotation.
  • the reference subunit 524 is configured to determine a reference point cloud in the point cloud sequence.
  • the second registration subunit 525 is configured to register other point cloud data frame by frame based on the reference point cloud, and provide the registered point cloud data to the projection subunit 522 .
  • the correction subunit 523 is used to correct the distortion of the point cloud data, and then the reference subunit 524 determines the reference point cloud, and the second registration subunit 525 performs registration.
  • the reference subunit 524 may use the point cloud of the first frame in the point cloud sequence as the basic point cloud. However, as a preferred embodiment, the reference subunit 524 is specifically used to provide the first frame in the point cloud sequence to the second registration subunit 525 as a reference, and to register other point cloud data frame by frame. The second registration subunit 525 obtains the registration result; in the point cloud sequence, the point cloud of the frame with the highest proportion of the registration points of the two frames of point cloud data before and after is used as the reference point cloud.
  • the second registration subunit 525 is used for:
  • the second registration subunit 525 learns the transformation matrix between two frames of point clouds from the point cloud as a reference and its adjacent point clouds that have not yet been registered, it is specifically used for:
  • the iterative closest point ICP algorithm is used to learn the transformation matrix between two frames of point clouds from the point cloud as the reference point cloud and the adjacent point cloud;
  • the loss function of the ICP algorithm is: each point in the point cloud as the reference point cloud is based on the transformation matrix
  • the third registration subunit 526 is configured to determine a set of corresponding point clouds in adjacent images.
  • the weights used by each distance are determined according to whether the point in the point cloud as a reference belongs to the set formed by the corresponding point cloud.
  • the projection subunit 522 is specifically used to project the coordinates of the point cloud data to the set, and obtain the coordinate information of the point cloud corresponding to the pixel in the front-view image;
  • the coordinate information of the point cloud corresponding to the pixel in the image is converted into the coordinate information of the pixel.
  • the conversion unit 530 is specifically configured to convert each frame of the front view image in the front view sequence into a top view based on inverse perspective transformation; perform matching on the top view according to the coordinate information of the pixels in the front view image, and determine the coordinate information of each pixel in the top view.
  • the identification unit 540 is specifically used for identifying road information on the top view; superimposing the recognized road information on the top view to obtain high-precision map data.
  • the road information can include lane lines, lane line types (such as white solid lines, single yellow solid lines, double yellow solid lines, yellow dashed solid lines, diversion lines, yellow no-stop lines, etc.), colors, and lane guide arrow information , lane type (such as main lane, bus lane, tidal lane, etc.), etc.
  • lane line types such as white solid lines, single yellow solid lines, double yellow solid lines, yellow dashed solid lines, diversion lines, yellow no-stop lines, etc.
  • colors and lane guide arrow information
  • lane type such as main lane, bus lane, tidal lane, etc.
  • a semantic segmentation model based on a deep neural network can be used to segment road information, such as DeepLabV3. It is also possible to use image recognition technology based on deep neural network to identify the above road information, such as Faster-RCNN (Regions with CNN features, convolutional neural network feature area).
  • each embodiment in this specification is described in a progressive manner, the same and similar parts of each embodiment can be referred to each other, and each embodiment focuses on the differences from other embodiments.
  • the description is relatively simple, and for relevant parts, please refer to part of the description of the method embodiment.
  • the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.
  • FIG. 6 it is a block diagram of an electronic device according to a method for producing a high-definition map according to an embodiment of the present disclosure.
  • Electronic device is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers.
  • Electronic devices may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smart phones, wearable devices, and other similar computing devices.
  • the components shown herein, their connections and relationships, and their functions, are by way of example only, and are not intended to limit implementations of the disclosure described and/or claimed herein.
  • the device 600 includes a computing unit 601 that can execute according to a computer program stored in a read-only memory (ROM) 602 or loaded from a storage unit 608 into a random-access memory (RAM) 603. Various appropriate actions and treatments. In the RAM 603, various programs and data necessary for the operation of the device 600 can also be stored.
  • the computing unit 601, ROM 602, and RAM 603 are connected to each other through a bus 604.
  • An input/output (I/O) interface 605 is also connected to the bus 604 .
  • the I/O interface 605 includes: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a magnetic disk, an optical disk, etc. ; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, and the like.
  • the communication unit 609 allows the device 600 to exchange information/data with other devices over a computer network such as the Internet and/or various telecommunication networks.
  • the computing unit 601 may be various general-purpose and/or special-purpose processing components having processing and computing capabilities. Some examples of computing units 601 include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processing processor (DSP), and any suitable processor, controller, microcontroller, etc.
  • the calculation unit 601 executes various methods and processes described above, for example, a production method of a high-definition map.
  • the method for producing a high-definition map may be implemented as a computer software program tangibly contained in a machine-readable medium, such as the storage unit 608 .
  • part or all of the computer program may be loaded and/or installed on the device 600 via the ROM 802 and/or the communication unit 609.
  • the computer program When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the production method of the high-definition map described above can be performed.
  • the computing unit 601 may be configured in any other appropriate way (for example, by means of firmware) to execute the high-precision map production method.
  • Various implementations of the systems and techniques described herein can be implemented in digital electronic circuitry, systems integrated circuits, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips system (SOC), complex programmable logic device (CPLD), computer hardware, firmware, software, and/or a combination thereof.
  • FPGAs field programmable gate arrays
  • ASICs application specific integrated circuits
  • ASSPs application specific standard products
  • SOC systems on chips system
  • CPLD complex programmable logic device
  • computer hardware firmware, software, and/or a combination thereof.
  • programmable processor can be special-purpose or general-purpose programmable processor, can receive data and instruction from storage system, at least one input device, and at least one output device, and transmit data and instruction to this storage system, this at least one input device, and this at least one output device an output device.
  • Program codes for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, so that the program codes, when executed by the processor or controller, make the flow diagrams and/or block diagrams specified The function/operation is implemented.
  • the program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
  • a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device.
  • a machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.
  • a machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing.
  • machine-readable storage media would include one or more wire-based electrical connections, portable computer discs, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.
  • RAM random access memory
  • ROM read only memory
  • EPROM or flash memory erasable programmable read only memory
  • CD-ROM compact disk read only memory
  • magnetic storage or any suitable combination of the foregoing.
  • the systems and techniques described herein can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user. ); and a keyboard and pointing device (eg, a mouse or a trackball) through which a user can provide input to the computer.
  • a display device e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor
  • a keyboard and pointing device eg, a mouse or a trackball
  • Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and can be in any form (including Acoustic input, speech input or, tactile input) to receive input from the user.
  • the systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., as a a user computer having a graphical user interface or web browser through which a user can interact with embodiments of the systems and techniques described herein), or including such backend components, middleware components, Or any combination of front-end components in a computing system.
  • the components of the system can be interconnected by any form or medium of digital data communication, eg, a communication network. Examples of communication networks include: Local Area Network (LAN), Wide Area Network (WAN) and the Internet.
  • a computer system may include clients and servers.
  • Clients and servers are generally remote from each other and typically interact through a communication network.
  • the relationship of client and server arises by computer programs running on the respective computers and having a client-server relationship to each other.
  • the server can be a cloud server, also known as cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the management problems existing in traditional physical host and virtual private server (VPs, VI irtual Private Server) services. Difficulty and weak business expansion.
  • the server can also be a server of a distributed system, or a server combined with a blockchain.
  • steps may be reordered, added or deleted using the various forms of flow shown above.
  • each step described in the present disclosure may be executed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution disclosed in the present disclosure can be achieved, no limitation is imposed herein.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Remote Sensing (AREA)
  • Geometry (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computer Graphics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Automation & Control Theory (AREA)
  • Image Processing (AREA)
  • Instructional Devices (AREA)
  • Image Analysis (AREA)

Abstract

本公开公开了高精地图的生产方法、装置、设备和计算机存储介质，涉及人工智能技术领域中的自动驾驶和深度学习技术。具体实现方案为：获取采集设备在各位置点分别采集的点云数据和正视图像数据，得到点云序列和正视图像序列；将所述点云序列和正视图像序列进行正视图像与点云数据的配准；依据所述配准结果将所述正视图像序列转换为俯视图并确定所述俯视图中各像素的坐标信息；对所述俯视图进行地图元素的识别，得到高精地图数据。

Description

高精地图的生产方法、装置、设备和计算机存储介质
本申请要求了申请日为2021年06月08日，申请号为202110637791.9发明名称为“高精地图的生产方法、装置、设备和计算机存储介质”的中国专利申请的优先权。
技术领域
本公开涉及计算机应用技术领域，尤其涉及人工智能技术领域中的自动驾驶和深度学习技术。
背景技术
高精地图作为自动驾驶系统中重要的一部分，是推动自动驾驶发展的关键因素之一。传统地图精度低，只能够提供道路级别的路线规划。高精地图通过提供高精度定位、车道级路径规划能力以及丰富的道路元素信息，可以帮助提前知晓位置信息，精确规划行驶路线，预知路面复杂信息更好地规避潜在风险，等等。因此，如何实现高精地图的生产成为亟待解决的问题。
发明内容
有鉴于此，本公开提供了一种高精地图的生产方法、装置、设备和计算机存储介质。
根据本公开的第一方面，提供了一种高精地图的生产方法，包括：
获取采集设备在各位置点分别采集的点云数据和正视图像数据，得到点云序列和正视图像序列；
将所述点云序列和正视图像序列进行正视图像与点云数据的配准；
依据所述配准结果将所述正视图像序列转换为俯视图并确定所述俯视图中各像素的坐标信息；
对所述俯视图进行地图元素的识别，得到高精地图数据。
根据本公开的第二方面，提供了一种高精地图的生产装置，包括：
获取单元，用于获取采集设备在各位置点分别采集的点云数据和正视图像数据，得到点云序列和正视图像序列；
配准单元，用于将所述点云序列和正视图像序列进行正视图像与点云数据的配准；
转换单元，用于依据所述配准结果将所述正视图像序列转换为俯视图并确定所述俯视图中各像素的坐标信息；
识别单元，用于对所述俯视图进行地图元素的识别，得到高精地图数据。
根据本公开的第三方面，提供了一种电子设备，包括：
至少一个处理器；以及
与所述至少一个处理器通信连接的存储器；其中，
所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行如上所述的方法。
根据本公开的第四方面，提供了一种存储有计算机指令的非瞬时计算机可读存储介质，其中，所述计算机指令用于使所述计算机执行如上所述的方法。
根据本公开的第五方面，一种计算机程序产品，包括计算机程序，所述计算机程序在被处理器执行时实现如上所述的方法。
应当理解，本部分所描述的内容并非旨在标识本公开的实施例的关键或重要特征，也不用于限制本公开的范围。本公开的其它特征将通过以下的说明书而变得容易理解。
附图说明
附图用于更好地理解本方案，不构成对本公开的限定。其中：
图1为本公开实施例提供的高精地图的生产方法流程图；
图2为本公开实施例提供的一种优选的配准处理流程图；
图3为本公开实施例提供的逐帧配准点云数据的方法流程图；
图4a和图4b分别为正视图像和俯视图的实例图；
图5为本公开实施例提供的高精地图的生产装置结构图；
图6是用来实现本公开实施例的电子设备的框图。
具体实施方式
以下结合附图对本公开的示范性实施例做出说明，其中包括本公开实施例的各种细节以助于理解，应当将它们认为仅仅是示范性的。因此，本领域普通技术人员应当认识到，可以对这里描述的实施例做出各种改变和修改，而不会背离本公开的范围和精神。同样，为了清楚和简明，以下的描述中省略了对公知功能和结构的描述。
虽然目前已经存在有一些高精地图的生产，但其主要基于点云技术。即由激光雷达设备采集大量稠密的点云数据，通过对点云数据进行处理和识别后，得到诸如道路、地面标志等信息，再通过人工方式对这些数据进行修正，最终生成高精地图数据。然而，这种传统的方式高度依赖点云数据。而由于城市道路空间结构复杂，为了保证高精地图的精度，需要花费大量的人力进行配准工作，从而导致高精地图的生产效率低下，人工成本高昂，且对作业人员的专业技能要求高，最终影响高精地图的规模化生产。
有鉴于此，本公开提供了一种不同于上述传统方式的高精地图的生产方法。下面结合实施例对本公开提供的方法进行详细描述。
图1为本公开实施例提供的高精地图的生产方法流程图，该方法的执行主体可以为推荐装置，该装置可以是位于本地终端的应用，或者还可以为位于本地终端的应用中的插件或软件开发工具包(Software Development Kit，SDK)等功能单元，或者，还可以位于服务器端，本发明实施例对此不进行特别限定。如图1中所示，该方法可以包括：
在101中，获取采集设备在各位置点分别采集的点云数据和正视图像数据，得到点云序列和正视图像序列。
在102中，将点云序列和正视图像序列进行正视图像与点云数据的配准。
在103中，依据配准结果将正视图像序列转换为俯视图并确定俯视图中各像素的坐标信息。
在104中，对俯视图进行地图元素的识别，得到高精地图数据。
由以上技术方案可以看出，本公开的思路是将图像采集设备采集的图像数据与激光雷达设备采集的点云数据进行融合，从而实现相互融合的自动化配准，并基于配准结果生成最终的高精地图。这种方式无需额外花费大量人力进行人工配准，提高了生产效率，降低了人工成本以及 对作业人员的专业技能要求，为高精地图的规模化生产提供基础。
下面对上述实施例中的各步骤进行详细描述。
首先结合实施例对上述步骤101即“获取采集设备在各位置点分别采集的点云数据和正视图像数据，得到点云序列和正视图像序列”进行详细描述。
在本步骤中涉及的采集设备主要包括以下两种：
图像采集设备，例如相机、摄影机等设备，能够定时或者被触发后进行图像采集。
激光雷达设备，能够定时或者被触发后通过发射激光扫描的方式获取周围环境表面反射点集合的数据，即点云数据。这些点云数据包括点的坐标信息，通常该坐标信息是在激光雷达设备坐标系下的坐标。
还可以包括具有定位功能的设备，即负责进行位置信息的采集，例如GNSS(Global Navigation Satellite System，全球卫星导航系统)设备。
在公开中，可以采用一个可移动的设备(例如采集车)承载上述采集设备，然后在可移动设备行进的过程，各采集设备按照一定的频率进行数据采集，或者在相同位置点被触发进行数据采集。
例如，图像采集设备按照一定的采集频率采集得到的正视图像构成正视图像序列
Figure PCTCN2021131180-appb-000001
其中，I i为在时间t i采集的一帧正视图像。
激光雷达设备按照一定的采集频率采集得到的点云数据构成点云序列
Figure PCTCN2021131180-appb-000002
其中P i为在时间t i采集的一帧点云数据。每一帧点云数据包括M个点的坐标信息，即一帧点云数据包括的坐标信息为P i＝{p 1,p 2,…,p M}，其中p j为第j个点的坐标信息。
位置采集设备按照一定的采集频率采集得到的位置数据构成位置序列
Figure PCTCN2021131180-appb-000003
其中L i为在时间t i采集的位置数据。
上述N为采集设备进行数据采集的次数，即每一种采集设备分别得到的数据数量。
作为一种优选的实施方式，为了保证数据的同步和后续配准过程，可以预先对采集设备进行时钟同步和/或联合标定。
其中，在对采集设备之间进行时钟同步时，优选需要达到毫米级别。具体的同步方式可以选择基于GPS的“PPS(Pulse Per Second，每秒脉冲数)+NMEA(National Marine Electronics Association，美国国家海洋电 子协会)”，或基于以太网的IEEE 1588(或IEEE 802.1AS)时钟同步协议。
对采集设备进行联合标定，主要是为了得到采集设备中图像采集设备的内、外参信息，激光雷达设备的外参信息，激光雷达坐标系到图像采集设备坐标系的转换及平移矩阵M 1，以及图像采集设备的内参信息矩阵M 2。
联合标定的方式主要是预设一个标定板，调整激光雷达设备和图像采集设备对标定板进行拍照和点云捕捉。然后找到至少三个对应的图像上的二维点和点云的三维点，即构成三个点对。利用这三个点对进行PNP(pespective-n-point，多点透视成像)求解，就可以得到激光雷达设备的坐标系和图像采集设备的坐标系之间的变换关系。
由于设备间的时钟同步和联合标定可以采用目前比较成熟的技术，在此不做详述。
下面结合实施例对上述步骤102即“将点云序列和正视图像序列进行正视图像与点云数据的配准”进行详细描述。
本步骤中将点云序列和正视图像序列进行正视图像和点云数据的配准，其思路是将正视图像序列中相邻图像首先进行配准，得到相邻图像中对应像素构成的集合，然后将点云数据投影到该集合中，从而得到集合中各像素的坐标信息。配准的过程实际上一方面是确定出较为准确的像素，一方面是确定出这部分像素的坐标信息。下面描述一种优选的实施方式，图2为本公开实施例提供的一种优选的配准处理流程图，如图2中所示，该流程可以包括以下步骤：
在201中，将正视图像序列中相邻图像进行配准，得到相邻图像中对应像素构成的集合。
由于在图像采集设备中是按照一定的频率进行的图像采集，因此相邻两帧图像是不相同的。本步骤的目的是为了得到相邻图像中哪些像素点是对应的，假设连续两帧图像
Figure PCTCN2021131180-appb-000004
和图像
Figure PCTCN2021131180-appb-000005
经过图像配准后，得到图像
Figure PCTCN2021131180-appb-000006
中的K个像素是和图像
Figure PCTCN2021131180-appb-000007
中的K个像素是对应的，分别表示为集合：
Figure PCTCN2021131180-appb-000008
其中，图像
Figure PCTCN2021131180-appb-000009
的像素
Figure PCTCN2021131180-appb-000010
与图像
Figure PCTCN2021131180-appb-000011
的像素点
Figure PCTCN2021131180-appb-000012
对应，
Figure PCTCN2021131180-appb-000013
与
Figure PCTCN2021131180-appb-000014
对应，以此类推。
在进行配准时，可以采用基于特征的方法或深度学习方法等。其中 基于特征的方法主要包括：确定两帧图像中各像素的特征，其中特征可以采用诸如SIFT(Scale-invariant feature transform，尺度不变特征转换)；然后基于相似度的方式进行特征匹配，从而得到对应的像素点。例如特征之间的相似度超过预设相似度阈值的两个像素点之间匹配成功。
深度学习方法主要包括：使用诸如卷积神经网络、VGG(Visual Geometry Group Network，视觉几何组网络)层等来生成各像素的特征向量表示，然后基于两帧图像中各像素的特征向量表示进行特征匹配，从而得到对应的像素点。例如，特征向量表示之间的相似度超过预设相似度阈值的两个像素点之间匹配成功。
在202中，依据采集点云数据的激光雷达设备旋转一周的运动量，对点云数据进行畸变校正。
本步骤是优选执行的，有助于提高点云数据在后续配准过程中的准确性。
图像采集设备采用的是全局快门，可以认为是在瞬间获得的。而激光雷达设备不是瞬时获得的，通常是发射接收机旋转一周即360度后采集到。假设旋转一周为100ms，那么在一个采集周期内形成的一帧点云数据中，最开始的点和最后的点之间相差100ms，再加上激光雷达设备是在运动过程中采集的，因此点云数据存在畸变，不能真实反映某一时刻真实的环境。为了更好地对图像数据以及点云数据进行配准，在本步骤中对点云数据进行畸变校正。
由于激光雷达在计算激光点坐标时，是以接收到激光束时刻激光雷达自身坐标系为基础的，所以在激光雷达运动过程中，每一列激光点的基准坐标系都是不一样的。但是它们在同一帧点云里，因此进行畸变校正的过程中需要将其统一在同一个坐标系下。
畸变校正的思路是计算采集过程中激光雷达的运动，然后在每帧点云上补偿这个运动量，包括旋转和平移的补偿。首先确定一帧点云中的第一个激光点，后续激光点可以确定出相对于第一个激光点的旋转角度和平移量，然后进行先旋转再平移的补偿转换，就可以得到矫正后的激光点的坐标信息。
更进一步地，在步骤202之后，还可以确定相邻图像中对应点云构成的集合。
具体地，可以首先根据图像采集设备的内参信息矩阵、图像采集设备的坐标系到图像平面的旋转矩阵以及激光雷达坐标系到图像采集设备坐标系的转换及平移矩阵，得到点云到图像的投影矩阵；然后利用投影矩阵将点云数据投影到图像上。进行投影后，可以确定相邻图像中对应的点云集合。假设连续两帧图像
Figure PCTCN2021131180-appb-000015
和图像
Figure PCTCN2021131180-appb-000016
经过上述投影后，得到投影到图像
Figure PCTCN2021131180-appb-000017
中的K 1个点构成的集合以及投影到图像
Figure PCTCN2021131180-appb-000018
中的K 2个点构成的集合，将两个集合求交集，即为图像
Figure PCTCN2021131180-appb-000019
和图像
Figure PCTCN2021131180-appb-000020
中对应点云构成的集合
Figure PCTCN2021131180-appb-000021
该集合的用途将在后续实施例中涉及。
在203中，确定点云序列中的基准点云。
具体地，可以将点云序列中的首帧作为基准点云。
但在实际情况中，点云序列中可能并非首帧点云是最为准确的。因此，本公开中提供了一种优选的确定基准点云的方式。具体地，可以首先将点云序列中的首帧作为基准，逐帧对其他点云数据进行配准；将点云图像序列中，与前后两帧点云数据的配准点占比最高的一帧点云作为基准点云。
其中，在逐帧对其他点云数据进行配准时，可以采用如图3中的方式，包括以下步骤：
在301中，从作为基准的点云及其尚未配准过的相邻点云中学习两帧点云之间的转换矩阵。
若从首帧开始作为基准，那么确定首帧点云和第二帧点云，从这两帧点云中学习两者的转换矩阵。
由于相邻两帧点云实际上是旋转和平移的关系，即将首帧点云进行旋转和平移后理论上能够得到第二帧点云。而实际场景下，由于采集设备在行进过程中可能存在颠簸等，可能会有一些偏差，因此，可以采用诸如ICP(Iterative Closest Point，迭代最近点)方法进行转换矩阵的学习。例如，旋转矩阵表示为R，平移矩阵表示为t，那么在学习R和t时，损失函数可以为：作为基准的点云中各点依据转换矩阵转换得到的各转换点与相邻点云中各转换点最近的点之间的距离均值或者加权均值。
例如，可以采用如下损失函数：
Figure PCTCN2021131180-appb-000022
其中，E(R,t)表示损失函数，x i表示作为基准的点云例如首帧点云中 的点，R(x i)+t表示依据转换矩阵对x i进行的转换，
Figure PCTCN2021131180-appb-000023
为对x i进行变换后在相邻点云例如第二帧点云中匹配上的最近点。n为能够匹配上的点的数量。逐个点进行学习后，以最小化上述损失函数为目标，最终可以学习到转换矩阵R和t。
再例如，还可以采用如下损失函数：
Figure PCTCN2021131180-appb-000024
与上述公式(1)不同的是，增加了加权系数w i，其取值可以依据作为基准的点云中的点是否属于对应点云构成的集合
Figure PCTCN2021131180-appb-000025
来确定。例如可以采用如下公式：
Figure PCTCN2021131180-appb-000026
其中，α≥1，例如可以取α＝1.5，或α＝2.0。
除了上述ICP方法之外，还可以采用诸如DGR(Deep global registration，深度全局配准)等基于深度特征学习的方法。
在302中，利用该转换矩阵将作为基准的点云进行转换，得到配准后的相邻点云。
例如，将首帧点云中的各点利用上述转换矩阵R和t进行转换后，得到第二帧点云中配准后的各点。
在303中，判断点云序列中是否还存在尚未配准过的相邻点云，如果是，执行步骤304；否则，结束当前配准流程。
在304中，将相邻点云作为新的基准，转至执行步骤301。
例如，继续将配准后的第二帧点云作为新的基准，对第三帧点云执行上述过程进行配准，得到配准后的各点。然后再以配准后的第三帧作为新的基准，对第四帧点云执行上述过程进行配准，以此类推。
完成点云序列中所有帧点云的配准后，将点云图像序列中，与前后两帧点云数据的配准点占比最高的一帧点云作为基准点云。其中，第j帧点云P j的配准点占比A j可以采用如下公式确定：
Figure PCTCN2021131180-appb-000027
其中，match()表示前后两帧点云中能够配准上的点的交集，可以体现为一帧点云按照转换矩阵进行转换后得到的各点与另一帧点云中各点 的交集。||表示集合中点的数量，例如|P j-1|表示第j-1帧点云中的点数量。
继续参见图2。在204中，以基准点云为基准，逐帧对其他点云数据进行配准。
确定出基准点云后，采用如图3中所示的方法，以基准点云为基准逐帧对其他点云数据进行配准。如果基准点云为首帧，则依次对后续各帧点云进行配准。如果基准点云为非首帧，则以基准点云为基准，向前和向后对各帧点云进行配准。最终得到配准后的点云序列。
在205中，将配准后的点云数据投影到步骤201得到的集合中，得到集合中各像素的坐标信息。
具体可以包括将点云数据的坐标投影到集合，得到正视图像中像素对应的点云的坐标信息；依据激光雷达坐标系到图像采集设备坐标系的转换及平移矩阵，将正视图像中像素对应的点云的坐标信息转换为像素的坐标信息。
其中，上述集合实际上是经过相邻图像配准后得到的对应像素。经过将点云数据分别投影到图像上，取落入上述集合的点(即激光点)，就可以得到正视图像中这些集合中各像素对应的点云的坐标信息。关于点云数据到图像的投影方式可以参见之前实施例中的相关记载，在此不做赘述。
在得到正视图像中像素对应的点云的坐标信息后，由于激光雷达设备的坐标系与图像采集设备的坐标系不同，因此需要将点云的坐标信息转换为像素的坐标信息，即转换到图像采集设备的坐标系下。
下面结合实施例对上述步骤103即“依据配准结果将正视图像序列转换为俯视图并确定俯视图中各像素的坐标信息”进行详细描述。
本步骤中，可以首先基于逆透视变换，将正视图序列中的各帧正视图像转换为俯视图；然后根据正视图像中像素的坐标信息在俯视图上进行匹配，确定俯视图中各像素的坐标信息。
其中，逆透视变换是目前进行图像投影变换的一种常用方式。其实质就是将图像采集设备采集的正视图像变换到世界坐标系下的z＝0平面中。
假设正视图中像素的坐标信息表示为(u,v)，需要转换为世界坐标系 下的坐标(x,y,z)。在联合标定过程中能够获取到如下参数：
γ：图像采集设备光轴o在z＝0平面的投影与y轴的夹角；
θ：图像采集设备光轴o偏离z＝0平面的角度；
2α：图像采集设备的视角；
R x：图像采集设备水平方向的分辨率；
R y：图像采集设备垂直方向的分辨率。
逆透视变换模型可以表示如下：
Figure PCTCN2021131180-appb-000028
Figure PCTCN2021131180-appb-000029
其中，h为图像采集设备距离地面的高度，cot()为余切函数。
经过上述逆透视变换，能够将诸如图4a中所示的正视图像转换为如图4b中所示的俯视图。
按照上述的逆透视变换理论实际上可以将正视图序列中的每帧正视图像都分别转换得到一张俯视图，若存在N帧正视图像，则得到N张俯视图。这些俯视图实际是相互重叠的，特别是相邻的两张俯视图，大部分区域都是重叠的。由于在上述过程中俯视图中的像素的坐标信息可以得到，因此可以基于各张俯视图中像素的位置信息进行逐张的拼接，最终得到高清地图。
下面结合实施例对上述步骤104即“对俯视图进行地图元素的识别，得到高精地图数据”进行详细描述。
在本步骤中，可以对步骤103中得到的俯视图进行道路信息的识别；然后将识别得到的道路信息叠加至俯视图中展现，得到高精地图数据。
其中道路信息可以包括车道线、车道线类型(例如白色实线、单黄实线、双黄实线、黄色虚实线、导流线、黄色禁止停车线等等)、颜色、车道的导向箭头信息、车道类型(例如主车道、公交车道、潮汐车道等等)等。
进行上述识别时，可以采用基于深度神经网络的语义分割模型来分割道路信息，例如DeepLabV3。也可以采用基于深度神经网络的图像识别技术来识别上述道路信息，例如Faster-RCNN(Regions with CNN  features，卷积神经网络特征区域)。
在此需要说明的是，基于上述俯视图的识别主要是对于地面元素的识别，即主要是道路信息的识别。而对于诸如交通标志、建筑物等其他地图元素的识别则从正视图像中识别。这部分可以采用现有技术中的方式，在本公开中不做限制。
将识别得到的道路信息叠加至俯视图中展现后，作业人员能够直接依据俯视图的数据与叠加的道路信息进行比对，对存在问题的数据进行修正，产出最终的高精地图数据。
以上是对本公开所提供方法进行的详细描述，下面结合实施例对本公开提供的装置进行详细描述。
图5为本公开实施例提供的高精地图的生产装置结构图，如图5所示，该装置500可以包括：获取单元510、配准单元520、转换单元530和识别单元540。其中各组成单元的主要功能如下：
获取单元510，用于获取采集设备在各位置点分别采集的点云数据和正视图像数据，得到点云序列和正视图像序列。
其中，上述采集设备至少包括用于采集正视图像的图像采集设备和用于采集点云数据的激光雷达设备。
作为一种优选的实施方式，为了保证数据的同步和后续配准过程，可以预先对采集设备进行时钟同步和/或联合标定。
配准单元520，用于将点云序列和正视图像序列进行正视图像与点云数据的配准。
转换单元530，用于依据配准结果将正视图像序列转换为俯视图并确定俯视图中各像素的坐标信息。
识别单元540，用于对俯视图进行地图元素的识别，得到高精地图数据。
具体地，配准单元520可以包括：第一配准子单元521和投影子单元522，还可以进一步包括校正子单元523、基准子单元524、第二配准子单元525和第三配准子单元526。
其中，第一配准子单元521，用于将正视图像序列中相邻图像进行配准，得到相邻图像中对应像素构成的集合。
投影子单元522，用于将点云数据投影到集合，得到集合中各像素 的坐标信息。
作为一种优选的实施方式，校正子单元523，用于依据采集点云数据的激光雷达设备旋转一周的运动量，对点云数据进行畸变校正后提供给投影子单元522。
作为另一种优选的实施方式，基准子单元524，用于确定点云序列中的基准点云。
第二配准子单元525，用于以基准点云为基准，逐帧对其他点云数据进行配准，将配准后的点云数据提供给投影子单元522。
上述两种方式可以结合，例如先采用校正子单元523对点云数据进行畸变校正，然后由基准子单元524确定基准点云，并由第二配准子单元525进行配准。
基准子单元524可以将点云序列中的首帧点云作为基础点云。但作为一种优选的实施方式，基准子单元524，具体用于将点云序列中的首帧提供给第二配准子单元525作为基准，逐帧对其他点云数据进行配准，从第二配准子单元525获取配准结果；将点云序列中，与前后两帧点云数据的配准点占比最高的一帧点云作为基准点云。
第二配准子单元525，用于：
从作为基准的点云及其尚未配准过的相邻点云中学习两帧点云之间的转换矩阵；
利用转换矩阵将作为基准的点云进行转换，得到配准后的相邻点云；
将相邻点云作为新的基准，转至执行从作为基准的点云及其尚未配准过的相邻点云中学习两帧点云之间的转换矩阵的操作，直至完成对点云图像序列中的所有点云数据的配准。
作为一种可实现的方式，第二配准子单元525在从作为基准的点云及其尚未配准过的相邻点云中学习两帧点云之间的转换矩阵时，具体用于：采用迭代最近点ICP算法，从作为基准的点云以及相邻点云中学习两帧点云之间的转换矩阵；其中，ICP算法的损失函数为：作为基准的点云中各点依据转换矩阵转换得到的各转换点与相邻点云中各转换点最近的点之间的距离均值或者加权均值。
作为一种优选的实施方式，第三配准子单元526，用于确定相邻图像中对应点云构成的集合。
第二配准子单元525在确定加权均值时，各距离采用的权值依据作为基准的点云中的点是否属于对应点云构成的集合确定。
投影子单元522，具体用于将点云数据的坐标投影到集合，得到正视图像中像素对应的点云的坐标信息；依据激光雷达坐标系到图像采集设备坐标系的转换及平移矩阵，将正视图像中像素对应的点云的坐标信息转换为像素的坐标信息。
转换单元530，具体用于基于逆透视变换，将正视图序列中的各帧正视图像转换为俯视图；根据正视图像中像素的坐标信息在俯视图上进行匹配，确定俯视图中各像素的坐标信息。
识别单元540，具体用于对俯视图进行道路信息的识别；将识别得到的道路信息叠加至俯视图中展现，得到高精地图数据。
其中道路信息可以包括车道线、车道线类型(例如白色实线、单黄实线、双黄实线、黄色虚实线、导流线、黄色禁止停车线等等)、颜色、车道的导向箭头信息、车道类型(例如主车道、公交车道、潮汐车道等等)等。
进行上述识别时，可以采用基于深度神经网络的语义分割模型来分割道路信息，例如DeepLabV3。也可以采用基于深度神经网络的图像识别技术来识别上述道路信息，例如Faster-RCNN(Regions with CNN features，卷积神经网络特征区域)。
本说明书中的各个实施例均采用递进的方式描述，各个实施例之间相同相似的部分互相参见即可，每个实施例重点说明的都是与其他实施例的不同之处。尤其，对于装置实施例而言，由于其基本相似于方法实施例，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。
根据本公开的实施例，本公开还提供了一种电子设备、一种可读存储介质和一种计算机程序产品。
如图6所示，是根据本公开实施例的高精地图的生产方法的电子设备的框图。电子设备旨在表示各种形式的数字计算机，诸如，膝上型计算机、台式计算机、工作台、个人数字助理、服务器、刀片式服务器、大型计算机、和其它适合的计算机。电子设备还可以表示各种形式的移动装置，诸如，个人数字处理、蜂窝电话、智能电话、可穿戴设备和其它类似的计算装置。本文所示的部件、它们的连接和关系、以及它们的 功能仅仅作为示例，并且不意在限制本文中描述的和/或者要求的本公开的实现。
如图6所示，设备600包括计算单元601，其可以根据存储在只读存储器(ROM)602中的计算机程序或者从存储单元608加载到随机访问存储器(RAM)603中的计算机程序，来执行各种适当的动作和处理。在RAM 603中，还可存储设备600操作所需的各种程序和数据。计算单元601、ROM 602以及RAM 603通过总线604彼此相连。输入/输出(I/O)接口605也连接至总线604。
设备600中的多个部件连接至I/O接口605，包括：输入单元606，例如键盘、鼠标等；输出单元607，例如各种类型的显示器、扬声器等；存储单元608，例如磁盘、光盘等；以及通信单元609，例如网卡、调制解调器、无线通信收发机等。通信单元609允许设备600通过诸如因特网的计算机网络和/或各种电信网络与其他设备交换信息/数据。
计算单元601可以是各种具有处理和计算能力的通用和/或专用处理组件。计算单元601的一些示例包括但不限于中央处理单元(CPU)、图形处理单元(GPU)、各种专用的人工智能(AI)计算芯片、各种运行机器学习模型算法的计算单元、数字信号处理器(DSP)、以及任何适当的处理器、控制器、微控制器等。计算单元601执行上文所描述的各个方法和处理，例如高精地图的生产方法。例如，在一些实施例中，高精地图的生产方法可被实现为计算机软件程序，其被有形地包含于机器可读介质，例如存储单元608。
在一些实施例中，计算机程序的部分或者全部可以经由ROM 802和/或通信单元609而被载入和/或安装到设备600上。当计算机程序加载到RAM 603并由计算单元601执行时，可以执行上文描述的高精地图的生产方法的一个或多个步骤。备选地，在其他实施例中，计算单元601可以通过其他任何适当的方式(例如，借助于固件)而被配置为执行高精地图的生产方法。
此处描述的系统和技术的各种实施方式可以在数字电子电路系统、集成电路系统、场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、芯片上系统的系统(SOC)、复杂可编程逻辑设备(CPLD)、计算机硬件、固件、软件、和/或它们的组合中实现。 这些各种实施方式可以包括：实施在一个或者多个计算机程序中，该一个或者多个计算机程序可在包括至少一个可编程处理器的可编程系统上执行和/或解释，该可编程处理器可以是专用或者通用可编程处理器，可以从存储系统、至少一个输入装置、和至少一个输出装置接收数据和指令，并且将数据和指令传输至该存储系统、该至少一个输入装置、和该至少一个输出装置。
用于实施本公开的方法的程序代码可以采用一个或多个编程语言的任何组合来编写。这些程序代码可以提供给通用计算机、专用计算机或其他可编程数据处理装置的处理器或控制器，使得程序代码当由处理器或控30制器执行时使流程图和/或框图中所规定的功能/操作被实施。程序代码可完全在机器上执行、部分地在机器上执行，作为独立软件包部分地在机器上执行且部分地在远程机器上执行或完全在远程机器或服务器上执行。
在本公开的上下文中，机器可读介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。机器可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。
为了提供与用户的交互，可以在计算机上实施此处描述的系统和技术，该计算机具有：用于向用户显示信息的显示装置(例如，CRT(阴极射线管)或者LCD(液晶显示器)监视器)；以及键盘和指向装置(例如，鼠标或者轨迹球)，用户可以通过该键盘和该指向装置来将输入提供给计算机。其它种类的装置还可以用于提供与用户的交互；例如，提供给用户的反馈可以是任何形式的传感反馈(例如，视觉反馈、听觉反馈、或者触觉反馈)；并且可以用任何形式(包括声输入、语音输入或者、触觉输入)来接收来自用户的输入。
可以将此处描述的系统和技术实施在包括后台部件的计算系统(例如，作为数据服务器)、或者包括中间件部件的计算系统(例如，应用服务器)、或者包括前端部件的计算系统(例如，具有图形用户界面或者网络浏览器的用户计算机，用户可以通过该图形用户界面或者该网络浏览器来与此处描述的系统和技术的实施方式交互)、或者包括这种后台部件、中间件部件、或者前端部件的任何组合的计算系统中。可以通过任何形式或者介质的数字数据通信(例如，通信网络)来将系统的部件相互连接。通信网络的示例包括：局域网(LAN)、广域网(WAN)和互联网。
计算机系统可以包括客户端和服务器。客户端和服务器一般远离彼此并且通常通过通信网络进行交互。通过在相应的计算机上运行并且彼此具有客户端-服务器关系的计算机程序来产生客户端和服务器的关系。服务器可以是云服务器，又称为云计算服务器或云主机，是云计算服务体系中的一项主机产品，以解决传统物理主机与虚拟专用服务器(VPs，Ⅵ irtual Private Server)服务中存在的管理难度大，业务扩展性弱的缺陷。服务器也可以为分布式系统的服务器，或者是结合了区块链的服务器。
应该理解，可以使用上面所示的各种形式的流程，重新排序、增加或删除步骤。例如，本公开中记载的各步骤可以并行地执行也可以顺序地执行也可以不同的次序执行，只要能够实现本公开公开的技术方案所期望的结果，本文在此不进行限制。
上述具体实施方式，并不构成对本公开保护范围的限制。本领域技术人员应该明白的是，根据设计要求和其他因素，可以进行各种修改、组合、子组合和替代。任何在本公开的精神和原则之内所作的修改、等同替换和改进等，均应包含在本公开保护范围之内。

Claims (25)

  1. 一种高精地图的生产方法，包括：
    获取采集设备在各位置点分别采集的点云数据和正视图像数据，得到点云序列和正视图像序列；
    将所述点云序列和正视图像序列进行正视图像与点云数据的配准；
    依据所述配准结果将所述正视图像序列转换为俯视图并确定所述俯视图中各像素的坐标信息；
    对所述俯视图进行地图元素的识别，得到高精地图数据。
  2. 根据权利要求1所述的方法，其中，将所述点云序列和正视图像序列进行正视图像与点云数据的配准包括：
    将所述正视图像序列中相邻图像进行配准，得到相邻图像中对应像素构成的集合；
    将点云数据投影到所述集合，得到所述集合中各像素的坐标信息。
  3. 根据权利要求2所述的方法，其中，在所述将点云数据投影到所述集合之前，还包括：
    依据采集所述点云数据的激光雷达设备旋转一周的运动量，对所述点云数据进行畸变校正。
  4. 根据权利要求2所述的方法，其中，所述将点云数据投影到所述集合之前，还包括：
    确定所述点云序列中的基准点云；
    以所述基准点云为基准，逐帧对其他点云数据进行配准。
  5. 根据权利要求4所述的方法，其中，确定所述点云序列中的基准点云包括：
    将所述点云序列中的首帧作为基准，逐帧对其他点云数据进行配准；
    将所述点云序列中，与前后两帧点云数据的配准点占比最高的一帧点云作为基准点云。
  6. 根据权利要求4或5所述的方法，其中，所述逐帧对其他点云数据进行配准包括：
    从作为基准的点云及其尚未配准过的相邻点云中学习两帧点云之间的转换矩阵；
    利用所述转换矩阵将作为基准的点云进行转换，得到配准后的所述相邻点云；
    将所述相邻点云作为新的基准，转至执行所述从作为基准的点云及其尚未配准过的相邻点云中学习两帧点云之间的转换矩阵的步骤，直至完成对所述点云图像序列中的所有点云数据的配准。
  7. 根据权利要求6所述的方法，其中，所述从作为基准的点云及其尚未配准过的相邻点云中学习两帧点云之间的转换矩阵包括：
    采用迭代最近点ICP算法，从作为基准的点云以及所述相邻点云中学习两帧点云之间的转换矩阵；
    其中，所述ICP算法的损失函数为：所述作为基准的点云中各点依据转换矩阵转换得到的各转换点与所述相邻点云中各转换点最近的点之间的距离均值或者加权均值。
  8. 根据权利要求7所述的方法，其中，在确定所述点云图像序列中的基准点云之前，还包括：
    确定相邻图像中对应点云构成的集合；
    在确定所述加权均值时，各距离采用的权值依据作为基准的点云中的点是否属于所述对应点云构成的集合确定。
  9. 根据权利要求2所述的方法，其中，所述将点云数据投影到所述集合，得到所述集合中各像素的坐标信息包括：
    将点云数据的坐标投影到所述集合，得到正视图像中像素对应的点云的坐标信息；
    依据激光雷达坐标系到图像采集设备坐标系的转换及平移矩阵，将所述正视图像中像素对应的点云的坐标信息转换为像素的坐标信息。
  10. 根据权利要求1所述的方法，其中，依据所述配准结果将所述正视图像序列转换为俯视图并确定所述俯视图中各像素的坐标信息包括：
    基于逆透视变换，将所述正视图序列中的各帧正视图像转换为各张俯视图；
    根据正视图像中像素的坐标信息在对应俯视图上进行匹配，确定俯视图中像素的坐标信息；
    依据俯视图中像素的坐标信息，对所述各张俯视图进行拼接处理得到最终的俯视图。
  11. 根据权利要求1所述的方法，其中，对所述俯视图进行地图元素的识别，得到高精地图数据包括：
    对所述俯视图进行道路信息的识别；
    将识别得到的道路信息叠加至所述俯视图中展现，得到高精地图数据。
  12. 一种高精地图的生产装置，包括：
    获取单元，用于获取采集设备在各位置点分别采集的点云数据和正视图像数据，得到点云序列和正视图像序列；
    配准单元，用于将所述点云序列和正视图像序列进行正视图像与点云数据的配准；
    转换单元，用于依据所述配准结果将所述正视图像序列转换为俯视图并确定所述俯视图中各像素的坐标信息；
    识别单元，用于对所述俯视图进行地图元素的识别，得到高精地图数据。
  13. 根据权利要求12所述的装置，其中，所述配准单元包括：
    第一配准子单元，用于将所述正视图像序列中相邻图像进行配准，得到相邻图像中对应像素构成的集合；
    投影子单元，用于将点云数据投影到所述集合，得到所述集合中各像素的坐标信息。
  14. 根据权利要求13所述的装置，其中，所述配准单元还包括：
    校正子单元，用于依据采集所述点云数据的激光雷达设备旋转一周的运动量，对所述点云数据进行畸变校正后提供给所述投影子单元。
  15. 根据权利要求13所述的装置，其中，所述配准单元还包括：
    基准子单元，用于确定所述点云序列中的基准点云；
    第二配准子单元，用于以所述基准点云为基准，逐帧对其他点云数据进行配准，将配准后的点云数据提供给所述投影子单元。
  16. 根据权利要求15所述的装置，其中，所述基准子单元，具体用于将所述点云序列中的首帧提供给所述第二配准子单元作为基准，逐帧对其他点云数据进行配准，从所述第二配准子单元获取配准结果；将所述点云序列中，与前后两帧点云数据的配准点占比最高的一帧点云作为基准点云。
  17. 根据权利要求15或16所述的装置，其中，所述第二配准子单元，具体用于：
    从作为基准的点云及其尚未配准过的相邻点云中学习两帧点云之间的转换矩阵；
    利用所述转换矩阵将作为基准的点云进行转换，得到配准后的所述相邻点云；
    将所述相邻点云作为新的基准，转至执行所述从作为基准的点云及其尚未配准过的相邻点云中学习两帧点云之间的转换矩阵的操作，直至完成对所述点云图像序列中的所有点云数据的配准。
  18. 根据权利要求17所述的装置，其中，所述第二配准子单元在从作为基准的点云及其尚未配准过的相邻点云中学习两帧点云之间的转换矩阵时，具体用于：
    采用迭代最近点ICP算法，从作为基准的点云以及所述相邻点云中学习两帧点云之间的转换矩阵；
    其中，所述ICP算法的损失函数为：所述作为基准的点云中各点依据转换矩阵转换得到的各转换点与所述相邻点云中各转换点最近的点之间的距离均值或者加权均值。
  19. 根据权利要求18所述的装置，其中，所述配准单元还包括：
    第三配准子单元，用于确定相邻图像中对应点云构成的集合；
    所述第二配准子单元在确定所述加权均值时，各距离采用的权值依据作为基准的点云中的点是否属于所述对应点云构成的集合确定。
  20. 根据权利要求13所述的装置，其中，所述投影子单元，具体用于将点云数据的坐标投影到所述集合，得到正视图像中像素对应的点云的坐标信息；依据激光雷达坐标系到图像采集设备坐标系的转换及平移矩阵，将所述正视图像中像素对应的点云的坐标信息转换为像素的坐标信息。
  21. 根据权利要求12所述的装置，其中，所述转换单元，具体用于基于逆透视变换，将所述正视图序列中的各帧正视图像转换为各张俯视图；根据正视图像中像素的坐标信息在对应俯视图上进行匹配，确定俯视图中像素的坐标信息；依据俯视图中像素的坐标信息，对所述各张俯视图进行拼接处理得到最终的俯视图。
  22. 根据权利要求12所述的装置，其中，所述识别单元，具体用于对所述俯视图进行道路信息的识别；将识别得到的道路信息叠加至所述俯视图中展现，得到高精地图数据。
  23. 一种电子设备，包括：
    至少一个处理器；以及
    与所述至少一个处理器通信连接的存储器；其中，
    所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行权利要求1-11中任一项所述的方法。
  24. 一种存储有计算机指令的非瞬时计算机可读存储介质，其中，所述计算机指令用于使所述计算机执行权利要求1-11中任一项所述的方法。
  25. 一种计算机程序产品，包括计算机程序，所述计算机程序在被处理器执行时实现根据权利要求1-11中任一项所述的方法。
PCT/CN2021/131180 2021-06-08 2021-11-17 高精地图的生产方法、装置、设备和计算机存储介质 WO2022257358A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
EP21916644.4A EP4174786A4 (en) 2021-06-08 2021-11-17 METHOD AND APPARATUS FOR GENERATING HIGH-PRECISION MAP, DEVICE AND COMPUTER STORAGE MEDIUM
KR1020227023591A KR20220166779A (ko) 2021-06-08 2021-11-17 고정밀 지도의 생산 방법, 장치, 설비 및 컴퓨터 저장 매체
JP2022541610A JP7440005B2 (ja) 2021-06-08 2021-11-17 高精細地図の作成方法、装置、デバイス及びコンピュータプログラム
US17/758,692 US20240185379A1 (en) 2021-06-08 2021-11-17 Method for generating high definition map, device and computer storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110637791.9 2021-06-08
CN202110637791.9A CN113409459B (zh) 2021-06-08 2021-06-08 高精地图的生产方法、装置、设备和计算机存储介质

Publications (1)

Publication Number Publication Date
WO2022257358A1 true WO2022257358A1 (zh) 2022-12-15

Family

ID=77676974

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/131180 WO2022257358A1 (zh) 2021-06-08 2021-11-17 高精地图的生产方法、装置、设备和计算机存储介质

Country Status (6)

Country Link
US (1) US20240185379A1 (zh)
EP (1) EP4174786A4 (zh)
JP (1) JP7440005B2 (zh)
KR (1) KR20220166779A (zh)
CN (1) CN113409459B (zh)
WO (1) WO2022257358A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115965756A (zh) * 2023-03-13 2023-04-14 安徽蔚来智驾科技有限公司 地图构建方法、设备、驾驶设备和介质
CN116467323A (zh) * 2023-04-11 2023-07-21 北京中科东信科技有限公司 一种基于路侧设施的高精地图的更新方法及系统
CN116863432A (zh) * 2023-09-04 2023-10-10 之江实验室 基于深度学习的弱监督激光可行驶区域预测方法和系统
CN117934573A (zh) * 2024-03-25 2024-04-26 北京华航唯实机器人科技股份有限公司 点云数据的配准方法、装置及电子设备

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111311743B (zh) * 2020-03-27 2023-04-07 北京百度网讯科技有限公司 三维重建精度测试方法、测试装置和电子设备
CN113409459B (zh) * 2021-06-08 2022-06-24 北京百度网讯科技有限公司 高精地图的生产方法、装置、设备和计算机存储介质
CN114419165B (zh) * 2022-01-17 2024-01-12 北京百度网讯科技有限公司 相机外参校正方法、装置、电子设备和存储介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105678689A (zh) * 2015-12-31 2016-06-15 百度在线网络技术（北京）有限公司 高精地图数据配准关系确定方法及装置
CN110160502A (zh) * 2018-10-12 2019-08-23 腾讯科技（深圳）有限公司 地图要素提取方法、装置及服务器
CN112105890A (zh) * 2019-01-30 2020-12-18 百度时代网络技术（北京）有限公司 用于自动驾驶车辆的基于rgb点云的地图生成系统
CN112434706A (zh) * 2020-11-13 2021-03-02 武汉中海庭数据技术有限公司 一种基于图像点云融合的高精度交通要素目标提取方法
CN112434119A (zh) * 2020-11-13 2021-03-02 武汉中海庭数据技术有限公司 一种基于异构数据融合的高精度地图生产装置
US20210148722A1 (en) * 2019-11-20 2021-05-20 Thinkware Corporation Method, apparatus, computer program, and computer-readable recording medium for producing high-definition map
CN113409459A (zh) * 2021-06-08 2021-09-17 北京百度网讯科技有限公司 高精地图的生产方法、装置、设备和计算机存储介质

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101976429B (zh) * 2010-10-27 2012-11-14 南京大学 基于游弋图像的水面鸟瞰图成像方法
CN106910217A (zh) * 2017-03-17 2017-06-30 驭势科技（北京）有限公司 视觉地图建立方法、计算装置、计算机存储介质和智能车辆
US10452927B2 (en) * 2017-08-09 2019-10-22 Ydrive, Inc. Object localization within a semantic domain
CN108230247B (zh) * 2017-12-29 2019-03-15 达闼科技（北京）有限公司 基于云端的三维地图的生成方法、装置、设备及计算机可读的存储介质
CN109059942B (zh) * 2018-08-22 2021-12-14 中国矿业大学 一种井下高精度导航地图构建系统及构建方法
CN108801171B (zh) * 2018-08-23 2020-03-31 南京航空航天大学 一种隧道断面形变分析方法及装置
CN109543520B (zh) * 2018-10-17 2021-05-28 天津大学 一种面向语义分割结果的车道线参数化方法
CN111160360B (zh) * 2018-11-07 2023-08-01 北京四维图新科技股份有限公司 图像识别方法、装置及系统
CN110568451B (zh) * 2019-08-02 2021-06-18 北京三快在线科技有限公司 一种高精度地图中道路交通标线的生成方法和装置
CN111311709B (zh) * 2020-02-05 2023-06-20 北京三快在线科技有限公司 一种生成高精地图的方法及装置
CN111508021B (zh) * 2020-03-24 2023-08-18 广州视源电子科技股份有限公司 一种位姿确定方法、装置、存储介质及电子设备
CN111652179B (zh) * 2020-06-15 2024-01-09 东风汽车股份有限公司 基于点线特征融合激光的语义高精地图构建与定位方法
CN111784836B (zh) * 2020-06-28 2024-06-04 北京百度网讯科技有限公司 高精地图生成方法、装置、设备及可读存储介质

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105678689A (zh) * 2015-12-31 2016-06-15 百度在线网络技术（北京）有限公司 高精地图数据配准关系确定方法及装置
CN110160502A (zh) * 2018-10-12 2019-08-23 腾讯科技（深圳）有限公司 地图要素提取方法、装置及服务器
CN112105890A (zh) * 2019-01-30 2020-12-18 百度时代网络技术（北京）有限公司 用于自动驾驶车辆的基于rgb点云的地图生成系统
US20210148722A1 (en) * 2019-11-20 2021-05-20 Thinkware Corporation Method, apparatus, computer program, and computer-readable recording medium for producing high-definition map
CN112434706A (zh) * 2020-11-13 2021-03-02 武汉中海庭数据技术有限公司 一种基于图像点云融合的高精度交通要素目标提取方法
CN112434119A (zh) * 2020-11-13 2021-03-02 武汉中海庭数据技术有限公司 一种基于异构数据融合的高精度地图生产装置
CN113409459A (zh) * 2021-06-08 2021-09-17 北京百度网讯科技有限公司 高精地图的生产方法、装置、设备和计算机存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4174786A4

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115965756A (zh) * 2023-03-13 2023-04-14 安徽蔚来智驾科技有限公司 地图构建方法、设备、驾驶设备和介质
CN115965756B (zh) * 2023-03-13 2023-06-06 安徽蔚来智驾科技有限公司 地图构建方法、设备、驾驶设备和介质
CN116467323A (zh) * 2023-04-11 2023-07-21 北京中科东信科技有限公司 一种基于路侧设施的高精地图的更新方法及系统
CN116467323B (zh) * 2023-04-11 2023-12-19 北京中科东信科技有限公司 一种基于路侧设施的高精地图的更新方法及系统
CN116863432A (zh) * 2023-09-04 2023-10-10 之江实验室 基于深度学习的弱监督激光可行驶区域预测方法和系统
CN116863432B (zh) * 2023-09-04 2023-12-22 之江实验室 基于深度学习的弱监督激光可行驶区域预测方法和系统
CN117934573A (zh) * 2024-03-25 2024-04-26 北京华航唯实机器人科技股份有限公司 点云数据的配准方法、装置及电子设备

Also Published As

Publication number Publication date
KR20220166779A (ko) 2022-12-19
JP2023533625A (ja) 2023-08-04
EP4174786A1 (en) 2023-05-03
CN113409459B (zh) 2022-06-24
US20240185379A1 (en) 2024-06-06
JP7440005B2 (ja) 2024-02-28
CN113409459A (zh) 2021-09-17
EP4174786A4 (en) 2024-04-03

Similar Documents

Publication Publication Date Title
WO2022257358A1 (zh) 高精地图的生产方法、装置、设备和计算机存储介质
CN112894832B (zh) 三维建模方法、装置、电子设备和存储介质
WO2021196294A1 (zh) 一种跨视频人员定位追踪方法、系统及设备
WO2020073936A1 (zh) 地图要素提取方法、装置及服务器
CN111174799B (zh) 地图构建方法及装置、计算机可读介质、终端设备
US20210366155A1 (en) Method and Apparatus for Detecting Obstacle
EP4116462A2 (en) Method and apparatus of processing image, electronic device, storage medium and program product
CN110675450B (zh) 基于slam技术的正射影像实时生成方法及系统
WO2021218123A1 (zh) 用于检测车辆位姿的方法及装置
WO2022262160A1 (zh) 传感器标定方法及装置、电子设备和存储介质
CN104361628A (zh) 一种基于航空倾斜摄影测量的三维实景建模系统
US20210248390A1 (en) Road marking recognition method, map generation method, and related products
CN103426165A (zh) 一种地面激光点云与无人机影像重建点云的精配准方法
CN112799096B (zh) 基于低成本车载二维激光雷达的地图构建方法
CN108629829A (zh) 一种球幕相机与深度相机结合的三维建模方法和系统
WO2023065657A1 (zh) 地图构建方法、装置、设备、存储介质及程序
CN112947526A (zh) 一种无人机自主降落方法和系统
CN116188893A (zh) 基于bev的图像检测模型训练及目标检测方法和装置
Gao et al. Multi-source data-based 3D digital preservation of largescale ancient chinese architecture: A case report
WO2022246812A1 (zh) 定位方法、装置、电子设备及存储介质
TW202132804A (zh) 地圖建構系統以及地圖建構方法
US9240055B1 (en) Symmetry-based interpolation in images
CN113129422A (zh) 一种三维模型构建方法、装置、存储介质和计算机设备
CN107784666B (zh) 基于立体影像的地形地物三维变化检测和更新方法
CN115937449A (zh) 高精地图生成方法、装置、电子设备和存储介质

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2022541610

Country of ref document: JP

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 17758692

Country of ref document: US

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21916644

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021916644

Country of ref document: EP

Effective date: 20230127

NENP Non-entry into the national phase

Ref country code: DE