WO2022041257A1 - 三维地图处理方法、装置、可移动平台及存储介质 - Google Patents

三维地图处理方法、装置、可移动平台及存储介质 Download PDF

Info

Publication number
WO2022041257A1
WO2022041257A1 PCT/CN2020/112700 CN2020112700W WO2022041257A1 WO 2022041257 A1 WO2022041257 A1 WO 2022041257A1 CN 2020112700 W CN2020112700 W CN 2020112700W WO 2022041257 A1 WO2022041257 A1 WO 2022041257A1
Authority
WO
WIPO (PCT)
Prior art keywords
map
dimensional
memory
coordinate information
dimensional coordinate
Prior art date
Application number
PCT/CN2020/112700
Other languages
English (en)
French (fr)
Inventor
颜钊
匡鑫
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to PCT/CN2020/112700 priority Critical patent/WO2022041257A1/zh
Publication of WO2022041257A1 publication Critical patent/WO2022041257A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
    • G01C21/28—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network with correlation of data from several navigational instruments
    • G01C21/30—Map- or contour-matching
    • G01C21/32—Structuring or formatting of map data

Definitions

  • the present application relates to the technical field of three-dimensional reconstruction, and in particular, to a three-dimensional map processing method, device, movable platform and storage medium.
  • 3D reconstruction refers to the process of reconstructing 3D information from single-view or multi-view images. For example, 3D reconstruction is performed based on a depth map. Due to the limited information obtained by a single depth map, multiple depth maps are usually used for fusion when generating a 3D map.
  • the method used is to back-project all the depth information in a single depth map to a three-dimensional space and mark the three-dimensional space with obstacle information, and then repeat this action, using other depth maps in turn. Update the same three-dimensional space, and finally get a three-dimensional map.
  • the 3D map needs to be read from the memory every time the depth map is used to update the same 3D space.
  • one of the objectives of the present application is to provide a three-dimensional map processing method, device, movable platform and storage medium.
  • an embodiment of the present application provides a three-dimensional map processing method, including:
  • each pixel in each depth image obtains the three-dimensional coordinate information corresponding to the pixel
  • the historical three-dimensional map is updated by using three-dimensional coordinate information corresponding to all pixels in the plurality of depth images.
  • an embodiment of the present application provides a three-dimensional map processing method, including:
  • any pixel in the depth image corresponds to a depth value
  • the three-dimensional coordinate information is sequentially read from a storage location pointed to by one of the map blocks in the first memory, and a three-dimensional map corresponding to the map block is generated according to the three-dimensional coordinate information belonging to the same map block.
  • an embodiment of the present application provides a three-dimensional map processing device, including:
  • the coordinate information generation module is used to read a plurality of depth images from the first memory, and any pixel in the depth image corresponds to a depth value; according to the depth value of each pixel in each depth image, obtain all the depth images.
  • a map update module for reading historical three-dimensional maps from the first memory after obtaining the three-dimensional coordinate information corresponding to all pixels in the multiple depth images; using the three-dimensional corresponding to all pixels in the multiple depth images The coordinate information updates the historical three-dimensional map.
  • an embodiment of the present application provides a three-dimensional map processing device, including:
  • a coordinate information generation module configured to read a depth image from the first memory; any pixel in the depth image corresponds to a depth value; according to the depth value of each pixel in the depth image, obtain the corresponding pixel 3D coordinate information of ;
  • a block module configured to determine the map block to which the three-dimensional coordinate information corresponding to each pixel in the depth image belongs; store the three-dimensional coordinate information belonging to different map blocks in different storage locations in the first memory;
  • a map generation module configured to sequentially read three-dimensional coordinate information from a storage location pointed to by one of the map blocks in the first memory, and generate a three-dimensional map corresponding to the map block according to the three-dimensional coordinate information belonging to the same map block.
  • an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a three-dimensional map processing apparatus, is used to implement any one of the first aspect or the second aspect the method described.
  • an embodiment of the present application provides a movable platform, including the device according to any one of the third aspect or the fourth aspect.
  • the three-dimensional map processing method, device, movable platform and storage medium provided by the embodiments of the present application only need to read the historical three-dimensional map once, which is beneficial to reduce the bandwidth consumption of the memory, and utilizes the process of updating multiple depth maps at one time It is also beneficial to improve the processing speed to meet real-time requirements.
  • Fig. 1, Fig. 5 and Fig. 8 are different schematic flowcharts of the three-dimensional map processing method provided by the embodiment of the present application;
  • FIG. 2 , FIG. 3 , and FIG. 9 to FIG. 15 are schematic diagrams of different structures of the three-dimensional map processing apparatus provided by the embodiments of the present application;
  • FIG. 4 is a schematic diagram of generating a three-dimensional map based on a depth map provided by an embodiment of the present application
  • FIG. 6 is a schematic diagram of dividing a map block provided by an embodiment of the present application.
  • FIG. 7 is a schematic diagram of generating a three-dimensional map corresponding to one of the blocks provided by an embodiment of the present application.
  • the method used is to back-project all the depth information in a single depth map to the three-dimensional space and mark the three-dimensional space with obstacle information, and then repeat this action, and use other
  • the depth map updates the same three-dimensional space, and finally a three-dimensional map is obtained.
  • the three-dimensional map needs to be read from the memory, and the update is performed on the basis of the read three-dimensional map.
  • the process of reading the 3D map requires a lot of memory bandwidth.
  • the updated 3D map needs to be written into the memory.
  • reading and writing the 3D map multiple times will bring a lot of processing delay, which is difficult to satisfy. real-time requirements.
  • the depth map is continuously used to repeat the above method to refresh the 3D map, so that the 3D map is updated at a certain frame rate to ensure The real-time nature of the three-dimensional map, but the above method needs to read and write the three-dimensional map multiple times, and there is a large processing delay, which is difficult to meet the real-time requirements.
  • the SLAM system is used to solve the SLAM (simultaneous localization and mapping, simultaneous localization and mapping) problem.
  • the SLAM problem can be described as: put a robot into an unknown location in an unknown environment, is there a way to make the robot move while gradually depicting To get a complete map of this environment, the so-called complete map (a consistent map) refers to unobstructed travel to every corner of the room that can be accessed.
  • the embodiments of the present application provide a three-dimensional map processing method, device, and movable platform, which simultaneously read multiple depth maps and obtain three-dimensional coordinate information corresponding to all pixels in the multiple depth images.
  • the historical three-dimensional map is updated at one time by using the three-dimensional coordinate information corresponding to all pixels in the multiple depth images, so as to realize that the historical three-dimensional map only needs to be read from the first memory once , which is beneficial to reduce the bandwidth consumption of the first memory, and at the same time, the one-time update process is also beneficial to improve the processing speed, so as to meet the real-time requirement.
  • the three-dimensional map processing method of the embodiments of the present application may be applied to a three-dimensional map processing apparatus, and the three-dimensional map processing apparatus may be an electronic device with data processing capability, such as a computer, a server, a cloud server or a terminal, a movable platform ( Such as unmanned aerial vehicles, unmanned vehicles or mobile robots, etc.), etc.; it can also be a computer chip or integrated circuit with data processing capabilities, such as a central processing unit (Central Processing Unit, CPU), a digital signal processor (Digital Signal Processor) , DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC) or off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) and so on.
  • a computer Central Processing Unit, CPU
  • DSP digital signal processor
  • ASIC Application Specific Integrated Circuit
  • FPGA Field-Programmable Gate Array
  • the three-dimensional map processing device when the three-dimensional map processing device is a computer chip or integrated circuit with data processing capability, the three-dimensional map processing device can be installed on a movable platform, and the movable platform includes but is not limited to a Human flying vehicles, unmanned vehicles, mobile robots, unmanned boats or gimbal.
  • the unmanned aerial vehicle can perform aerial survey, navigation, obstacle avoidance or positioning, etc. according to the three-dimensional map updated in real time by the three-dimensional map processing device.
  • the unmanned vehicle can perform automatic driving, obstacle avoidance, etc. according to the three-dimensional map updated in real time by the three-dimensional map processing device.
  • one or more photographing devices may also be installed on the movable platform, and the photographing devices are used to acquire depth images required by the three-dimensional map processing device.
  • the movable platform may send the 3D map to a terminal associated with the movable platform, so as to display the 3D map on the terminal. 3D map.
  • the three-dimensional map processing method in the embodiments of the present application can be applied to but not limited to map navigation scenarios, vehicle driving scenarios, location query scenarios, survey scenarios, monitoring scenarios, or rescue scenarios, and the like.
  • the three-dimensional map processing method provided in the embodiment of the present application can be used to generate and update a three-dimensional map in real time, so that the car can perform automatic driving and obstacle avoidance based on the three-dimensional map.
  • FIG. 1 is a schematic flowchart of a three-dimensional map processing method provided by an embodiment of the present application. The method can be applied to a three-dimensional map processing device, wherein the three-dimensional map processing device needs to store data from a first memory. Read multiple depth images and historical three-dimensional maps; in an example, please refer to FIG.
  • the first memory 200 may be provided outside the three-dimensional map processing apparatus 100, for example, the three-dimensional map processing apparatus 100 is a dedicated integrated Circuit (Application Specific Integrated Circuit, ASIC), the first memory 200 is a DDR memory; in another example, the first memory can also be set inside the three-dimensional map processing device, such as the three-dimensional map processing device For a computer, the first memory is an internal memory provided on the computer.
  • the method includes:
  • step S101 a plurality of depth images are read from the first memory; any pixel in the depth images corresponds to a depth value.
  • step S102 according to the depth value of each pixel in each depth image, three-dimensional coordinate information corresponding to the pixel is acquired.
  • step S103 after obtaining the three-dimensional coordinate information corresponding to all the pixels in the plurality of depth images, read the historical three-dimensional map from the first memory.
  • step S104 the historical three-dimensional map is updated using the three-dimensional coordinate information corresponding to all pixels in the plurality of depth images.
  • the embodiments of the present application do not impose any restrictions on the acquisition of the multiple depth images, and specific settings can be made according to actual application scenarios; in an example, the multiple depth images can be processed by the three-dimensional map.
  • the photographing device (such as a depth camera) mounted on the device is obtained by real-time photographing; in another example, the multiple depth images may be obtained by the three-dimensional map processing device from other terminals.
  • the multiple depth images can be obtained by shooting the same target by multiple photographing devices (such as depth cameras) in different positions, so as to obtain the information of the same target and its surroundings from different directions, so that based on the multiple images
  • the information around the target (such as obstacle information) contained in the three-dimensional map obtained after the fusion of the depth images is more comprehensive; and/or (and/or representing both or one of the two), the multiple depth images can be composed of A shooting device (such as a depth camera) shoots the same target at different times. Since the information of the same target obtained at different times is also different, the 3D map obtained after fusion based on the multiple depth images can be better filtered. noise, with better robustness.
  • the plurality of depth maps may be stored in the first memory so as to be read by the three-dimensional image processing apparatus when performing three-dimensional image processing.
  • the first memory may be provided external to the three-dimensional map processing apparatus.
  • the first memory may also be provided inside the three-dimensional map processing apparatus, which is not limited in this embodiment of the present application.
  • the first memory may be an internal memory such as a DDR (Double Data Rate, double rate) memory, etc., which may be specifically set according to actual application scenarios.
  • DDR Double Data Rate, double rate
  • the three-dimensional map processing apparatus 100 further includes a second memory 300 disposed inside, and the second memory 300 is used for buffering (or temporarily saving) the three-dimensional map. Some data in the process of data processing by the processing device. During three-dimensional map processing, a plurality of depth maps read from the first memory may be buffered (or temporarily stored) in the second memory.
  • the three-dimensional map processing device When performing three-dimensional map processing, the three-dimensional map processing device first reads the plurality of depth maps from the first memory, and any pixel in the depth images corresponds to a depth value, and then according to each depth map The depth value of each pixel in the image is obtained, and the three-dimensional coordinate information corresponding to the pixel is obtained.
  • the three-dimensional map processing apparatus may read at least one depth image from the first memory each time, wherein the number of read depth images is Adapted to the storage capacity of the second memory, then the three-dimensional map processing device obtains the three-dimensional coordinate information corresponding to the pixel according to the depth value of each pixel in the read at least one depth image, and then The three-dimensional coordinate information of all the pixels corresponding to the at least one depth image is stored in the first memory, and then the three-dimensional map processing apparatus continues to read the unprocessed depth image from the first memory and perform the same steps as the above steps. step until the three-dimensional coordinate information corresponding to all pixels in the multiple depth images is obtained.
  • the three-dimensional map processing apparatus may read the plurality of depth images from the first memory at one time, and then, according to each depth image The depth value of each pixel in the depth image is obtained, and the three-dimensional coordinate information corresponding to the pixel is obtained and buffered in the second memory until the three-dimensional coordinate information corresponding to all the pixels in the multiple depth images is obtained.
  • the three-dimensional coordinate information includes the three-dimensional coordinate information corresponding to the pixel, including the three-dimensional coordinate corresponding to the pixel and the display radius of the target point indicated by the three-dimensional coordinate; the target point represents the three-dimensional coordinate in the point cloud. Points indicated for display in the 3D map.
  • Map_info The storage format of the three-dimensional coordinate information (represented by map_info) is shown in Table 1.
  • idx_x, idx_y, and idx_z respectively represent the coordinates of the target point in the three-dimensional map. For example, it is represented by 8 bits, and tab_info represents The display radius information of the target point, for example, is represented by 2 bits, and one target point occupies a 26-bit bit width.
  • the display radius of the target point indicated by the three-dimensional coordinates may be determined according to the depth value of the pixel.
  • different ranges of the depth value may be preset to correspond to different display radii, and the display radius is determined based on the range corresponding to the depth value of the pixel, and the three-dimensional map processing device determines the range of the depth value.
  • the display radius first determine the range to which the depth value of the pixel belongs, and obtain the corresponding display radius according to the range to which the depth value belongs as the display radius of the target point indicated by the three-dimensional coordinates.
  • a preset correspondence relationship may be preset, and the preset correspondence relationship indicates that different depth values correspond to different display radii.
  • the three-dimensional map processing apparatus The depth value of the pixel obtains the corresponding display radius from the preset correspondence relationship as the display radius of the target point indicated by the three-dimensional coordinates.
  • the depth value of the pixel can be set to have a positive correlation with the display radius, that is, the smaller the depth value of the pixel is. , the display radius of the target point indicated by the three-dimensional coordinates corresponding to the pixel is also smaller.
  • the larger the depth value of the pixel the larger the display radius of the target point indicated by the three-dimensional coordinates corresponding to the pixel. It can be said that the further the distance is, the rougher the determined spatial position, the larger the display radius needs to be.
  • a larger spatial position range is indicated to avoid errors in spatial position determination.
  • the method used is to back-project all the depth information in a single depth map to a three-dimensional space and mark the three-dimensional space as obstacles. Specifically, it is to obtain all the pixels in the depth map.
  • the 3D coordinate information of each pixel in the memory stores the 3D coordinate information of each pixel in the memory, and then obtain the 3D coordinate information of each pixel from the memory to update the 3D map, that is, use the 3D coordinate information of a single pixel in turn to generate a 3D map, and then use the generated 3D map to update Historical three-dimensional map; since the target points indicated by the three-dimensional coordinate information corresponding to each pixel in the depth image are randomly distributed in the three-dimensional space, for example, as shown in Figure 4, the target points indicated by the three-dimensional coordinate information corresponding to the pixel P1 and the pixel P2 The target points indicated by the corresponding three-dimensional coordinate information are distributed in different positions in the three-dimensional map, and the distance between the two is far away. Therefore, if the three-dimensional coordinate information corresponding to the pixels in the depth map is processed sequentially, the three-dimensional map can be updated. , there may be a large number of random accesses to the memory, resulting in low memory access efficiency.
  • the three-dimensional map processing apparatus may All three-dimensional coordinate information corresponding to each depth image is stored in the first memory.
  • the first memory is provided outside the three-dimensional map processing device, and the second memory is provided inside the three-dimensional map processing device. After obtaining the three-dimensional coordinate information corresponding to all the pixels in the multiple depth images and storing the three-dimensional coordinate information corresponding to the multiple depth images in the first memory, the three-dimensional map processing apparatus retrieves the information from the first memory.
  • the three-dimensional map processing device uses the three-dimensional coordinate information corresponding to all pixels in the multiple depth images to update the historical three-dimensional map. map.
  • the three-dimensional map processing apparatus may sequentially read the three-dimensional coordinate information corresponding to all pixels in at least one depth image from the first memory (the number of read depth images is adapted to the number of the second memory).
  • the memory to generate a three-dimensional map corresponding to each depth image, and then fuse the three-dimensional map corresponding to all depth images with the historical three-dimensional map, for example, the three-dimensional map corresponding to each depth image can be sequentially combined with the historical three-dimensional map Fusion is performed to obtain a fused three-dimensional map; in this embodiment, the three-dimensional coordinate information corresponding to all pixels in the depth map will be obtained in units of each depth map and cached in the second memory, and implemented in the second memory.
  • the fusion update of the three-dimensional map instead of obtaining the three-dimensional coordinate information of each pixel in turn, realizes the update of the three-dimensional map, thereby reducing a large number of random accesses to the first memory; and in this embodiment, considering the second memory storage
  • the problem of limited capacity, and the bandwidth required to read the three-dimensional coordinate information from outside the first storage is far less than the bandwidth required to read the three-dimensional map, so the three-dimensional coordinate information corresponding to the depth image is stored in the In the first memory, when updating the historical three-dimensional map, it is realized that the historical three-dimensional map only needs to be read from the first memory once, which is beneficial to reduce the bandwidth consumption of the first memory, and at the same time, the historical three-dimensional map is updated at one time.
  • the process of historical 3D map is also beneficial to improve the processing speed to meet real-time requirements.
  • the second memory when the second memory has sufficient storage capacity, after acquiring the three-dimensional coordinate information corresponding to each pixel in each depth image, the three-dimensional coordinate information can be cached in the second memory.
  • the first memory 200 is disposed outside the three-dimensional map processing apparatus 100
  • the second memory 300 is disposed inside the three-dimensional map processing apparatus 100 .
  • the three-dimensional map processing device After obtaining the three-dimensional coordinate information corresponding to all the pixels in the plurality of depth images, the three-dimensional map processing device reads a historical three-dimensional map from the first memory, and the historical three-dimensional map is generated based on the historical depth image, The historical three-dimensional map is also cached in the second memory 300, wherein the read and write speed for the second memory is faster than the read and write speed for the first memory; then the three-dimensional map is processed In the second memory, the device updates the historical three-dimensional map by using three-dimensional coordinate information corresponding to all pixels in the plurality of depth images.
  • the 3D map processing device generates a 3D map by using 3D coordinate information corresponding to all pixels in the plurality of depth images, and then fuses the 3D map with the historical 3D map to obtain a fused 3D map .
  • the three-dimensional coordinate information corresponding to the plurality of depth images is cached in the second memory.
  • the data for the second memory is The read and write speed is better than the read and write speed of the first memory, and the fusion update of the three-dimensional map is realized in the second memory, which also reduces a large number of random accesses to the first memory;
  • the three-dimensional coordinate information corresponding to all pixels in the depth image is updated at one time to the historical three-dimensional map, so that the historical three-dimensional map only needs to be read from the first memory once, which is beneficial to reduce the bandwidth consumption of the first memory.
  • the one-time update process is also conducive to improving the processing speed, so as to meet the real-time requirements.
  • the embodiment of the present application uses N (N is an integer greater than 1) depth maps of h (height)*w (width) to generate a three-dimensional map, and any one of the depth maps in each depth map is used to generate a three-dimensional map
  • a pixel corresponds to a depth value
  • the three-dimensional coordinate information corresponding to the pixel can be obtained according to the depth value
  • a three-dimensional map is generated according to the three-dimensional coordinate information.
  • the three-dimensional map can be used to update the historical three-dimensional map.
  • the depth map has Pixel p1 and pixel p2 obtain their three-dimensional coordinate information according to the depth value of pixel p1 and obtain their three-dimensional coordinate information according to the depth value of pixel p2, and generate corresponding three-dimensional coordinate information according to all pixels of the N depth maps.
  • the figure shows the position of the target point P1' corresponding to the pixel p1 and the target point P2' corresponding to the pixel p2 in the three-dimensional map
  • the target point represents the point indicated by the three-dimensional coordinate information in the point cloud
  • the target point Points are used for display in a three-dimensional map that can be used to update historical three-dimensional maps.
  • the three-dimensional map, the historical three-dimensional map and the fused three-dimensional map are all represented in the form of three-dimensional grid maps. Then, when generating a three-dimensional map by using the three-dimensional coordinate information corresponding to all pixels in the plurality of depth images, the three-dimensional map processing device builds the three-dimensional map, the three-dimensional map includes several grids, and the three-dimensional map processing device creates the three-dimensional map. The device determines the corresponding grid in the three-dimensional map according to the three-dimensional coordinate information corresponding to the pixel, and projects the target point indicated by the three-dimensional coordinate information corresponding to the pixel on the corresponding grid in the three-dimensional map.
  • the target point represents the point indicated by the three-dimensional coordinate information in the point cloud, and then the attribute value of the corresponding grid is updated according to the three-dimensional coordinate information; the attribute value indicates that there is an obstacle at the position indicated by the grid. probability.
  • the three-dimensional map can generate a three-dimensional map corresponding to the depth image according to the three-dimensional coordinate information corresponding to each depth image, and each depth image corresponds to a three-dimensional map;
  • the three-dimensional map processing apparatus may generate a three-dimensional map according to the three-dimensional coordinate information corresponding to the multiple depth images, and the multiple depth images correspond to one three-dimensional map.
  • each grid in the three-dimensional map has a corresponding storage location in the second memory, and the storage location stores the attribute value of the grid;
  • an attribute value may be initialized for each grid in the three-dimensional map, and the initialized attribute value may be stored in the corresponding storage position of the grid in the second memory. It is understandable that this application does not impose any restrictions on the specific values of the initialized attribute values, and can be specifically set according to actual application scenarios.
  • the three-dimensional map processing device can determine the corresponding grid in the three-dimensional map according to the three-dimensional coordinate information, and store the grid corresponding to the grid in the second memory.
  • the attribute value at the position overlays the preset value.
  • the three-dimensional map processing device determines the grids corresponding to the same three-dimensional coordinate information in the three-dimensional map and the historical three-dimensional map, and superimposes the attribute values of the grids corresponding to the same three-dimensional coordinate information to obtain all the grids corresponding to the same three-dimensional coordinate information.
  • the fused 3D map if the attribute value of the grid in the fused three-dimensional map is greater than the upper threshold, the attribute value corresponding to the grid is modified to the upper threshold; The attribute value in the grid in is less than the lower threshold, and the attribute value corresponding to the grid is modified to the lower threshold.
  • an upper threshold and a lower threshold are set, and the attribute values of the three-dimensional map are controlled within a range, so as to prevent the fused three-dimensional map from being inaccurate due to the influence of abnormal attribute values.
  • the embodiment of the present application also provides another three-dimensional map processing method
  • the method can be executed by the three-dimensional map processing device, the three-dimensional map processing device divides the historical three-dimensional map according to the preset direction in advance, and determines a plurality of map blocks, and the map blocks indicate the three-dimensional map.
  • Classified storage the method includes:
  • step S201 a plurality of depth images are read from the first memory; any pixel in the depth images corresponds to a depth value.
  • step S202 according to the depth value of each pixel in each depth image, the three-dimensional coordinate information corresponding to the pixel is acquired.
  • step S203 the map block to which the three-dimensional coordinate information corresponding to each pixel in the depth image belongs is determined, and the three-dimensional coordinate information belonging to different map blocks is stored in different storage locations in the first memory.
  • step S204 after obtaining the three-dimensional coordinate information corresponding to all the pixels in the plurality of depth images, read the historical three-dimensional map from the first memory.
  • step S205 the three-dimensional coordinate information corresponding to the multiple depth images is read from different storage locations pointed to by different map blocks in the first memory, and the historical three-dimensional map is updated using the three-dimensional coordinate information.
  • the three-dimensional coordinate information corresponding to the pixels in the depth image is stored according to the different map blocks to which they belong.
  • the historical three-dimensional map is updated by acquiring corresponding three-dimensional coordinate information on a unit basis, so as to further improve the three-dimensional map processing efficiency on the basis of reducing a large number of random accesses to the first memory.
  • the three-dimensional map processing device divides the historical three-dimensional map in advance according to a preset direction, and determines a plurality of map blocks, and the map blocks indicate a preset range where the target point indicated by the three-dimensional coordinate information is located.
  • the spatial area, or in other words, the map block indicates three-dimensional coordinates within a certain range.
  • the historical three-dimensional map is along the direction of the three-dimensional coordinate axis, for example, five map blocks are determined along the direction of the three-dimensional coordinate axis X-axis, which are map block 01, map block 02, map block 03, map block 04 and map block 05, respectively.
  • the map tiles indicate three-dimensional coordinates in different ranges, in other words, the target points indicated by the three-dimensional coordinates in different ranges are located in different map tiles.
  • the first memory is installed outside the three-dimensional map processing device, and the three-dimensional map processing device includes a second memory installed inside, and the read and write speed of the second memory is better than For the read and write speed of the first memory and the price per unit capacity of the second memory is higher than that of the first memory, and in order to reduce costs, the storage capacity of the second memory is smaller than that of the first memory .
  • the first memory may be DDR memory and the second memory may be SRAM.
  • the three-dimensional map processing apparatus reads a plurality of depth images from the first memory, and buffers the plurality of depth images in the second memory, wherein the second memory has limited storage resources.
  • the three-dimensional map processing apparatus may read one depth image buffer from the first memory to the second memory at a time; if the storage resources of the second memory are sufficient, the three-dimensional map processing apparatus may A plurality of depth image buffers are read from the first memory to the second memory at one time.
  • any pixel in the depth image corresponds to a depth value
  • the three-dimensional map processing device acquires the three-dimensional coordinate information corresponding to the pixel according to the depth value of each pixel in each depth image, and then determines the depth
  • the three-dimensional coordinate information corresponding to each pixel in the image belongs to a map block, and the three-dimensional coordinate information belonging to different map blocks is stored in different storage locations in the first memory;
  • the three-dimensional coordinate information is classified and stored according to the different map blocks to which it belongs. Since the distance between the three-dimensional coordinate information belonging to the same map block is not much different, the corresponding three-dimensional coordinate information can be obtained in units of map blocks to update the historical three-dimensional map.
  • the three-dimensional map processing efficiency is further improved on the basis of reducing a large number of random accesses to the first memory.
  • the 3D map processing device uses the 3D coordinates The information updates the historical three-dimensional map.
  • the three-dimensional map processing apparatus sequentially reads three-dimensional coordinate information from a storage location pointed to by one of the map blocks in the first memory, and generates the map block according to the three-dimensional coordinate information belonging to the same map block
  • the corresponding three-dimensional map please refer to FIG.
  • the three-dimensional map processing device can sequentially fuse the three-dimensional maps corresponding to different map blocks with the historical three-dimensional map in sequence; this embodiment realizes that the corresponding three-dimensional coordinate information is acquired in units of map blocks to generate a three-dimensional map to update the historical three-dimensional map.
  • the three-dimensional map can further improve the processing efficiency of the three-dimensional map on the basis of reducing a large number of random accesses to the first memory.
  • the three-dimensional map corresponding to the map block is generated by using the three-dimensional coordinate information belonging to the same map block in the multiple depth images. If the number of the depth images is too large, then The data volume of the three-dimensional coordinate information belonging to the same map block in multiple depth images is also large, and it may not be possible to process it at one time, then the three-dimensional map processing device may use the three-dimensional coordinate information belonging to the same map block in a single depth image as a unit.
  • the three-dimensional map processing device can sequentially read the three-dimensional coordinate information corresponding to each depth image from the storage location pointed to by one of the map blocks in the first memory, and according to the The three-dimensional coordinate information of the same map block generates a three-dimensional map corresponding to the map block, and then fuses the three-dimensional maps corresponding to different map blocks in the multiple depth images with the historical three-dimensional map to obtain a fused three-dimensional map; this implementation For example, it is realized to obtain the corresponding three-dimensional coordinate information in units of map blocks to generate a three-dimensional map to update the historical three-dimensional map, so as to further improve the three-dimensional map processing efficiency on the basis of reducing a large number of random accesses to the first memory; In a single depth image, the corresponding three-dimensional coordinate information is obtained in units of the same map block to generate a three-dimensional map corresponding to the map block, and the three-dimensional map corresponding to the map block is smaller than that generated by using the randomly distributed three-dimensional coordinate information corresponding to the
  • the three-dimensional map processing apparatus when storing the three-dimensional coordinate information corresponding to the depth image, firstly encapsulates the three-dimensional coordinate information belonging to the same map block into a data packet, and stores it in the first memory, wherein the three-dimensional coordinate information belonging to different
  • the data packets of the map block can be continuously stored in the first memory; then obtain index information indicating the storage location of the data packet, and store the index information in the storage location pointed to by the map block in the first memory,
  • the index information of data packets belonging to different map blocks is stored in different storage locations respectively pointed to by different map blocks in the first memory.
  • the data packets encapsulating most of the data are continuously stored in the first memory, only the index information with a relatively small amount of data only needs to be stored in the storage spaces corresponding to different map blocks, because there is no need to store different map blocks.
  • the space is determined to be too large, so as to achieve the purpose of saving the storage space of the first memory and improve the utilization rate of the storage resources.
  • Table 1 shows the storage format of the three-dimensional coordinate information corresponding to one pixel in the depth image; the three-dimensional coordinate information belonging to the same map block can be encapsulated into data packets according to preset quantity rules
  • Table 2 shows the storage format of the data packet.
  • a data packet is set to include 3D coordinate information corresponding to 64 pixels, and the 3D coordinate information corresponding to the 64 pixels belongs to the same map block.
  • Table 3 which is the storage format of the index information.
  • the index information of the data packets belonging to different map blocks is stored in different storage locations respectively pointed to by different map blocks in the first memory. middle.
  • the three-dimensional map processing device reads the three-dimensional coordinate information from the first memory, it first reads the index information of the data packets belonging to different map blocks from the storage locations respectively pointed to by different map blocks in the first memory, Then, the data packet is read from the first memory according to the index information, the three-dimensional coordinate information is obtained by decapsulation, and the historical three-dimensional map is updated using the three-dimensional coordinate information.
  • the three-dimensional map processing device may generate a three-dimensional map corresponding to the map block according to the three-dimensional coordinate information belonging to the same map block; and fuse the three-dimensional maps corresponding to different map blocks with the historical three-dimensional map to obtain a fusion Post 3D map.
  • This embodiment realizes that the corresponding 3D coordinate information is obtained in units of map blocks to generate a 3D map to update the historical 3D map, which further improves the 3D map processing efficiency on the basis of reducing a large number of random accesses to the first memory.
  • the method used is to back-project all the depth information in a single depth map to a three-dimensional space and mark the three-dimensional space as obstacles. Specifically, it is to obtain the depth map.
  • the three-dimensional coordinate information of all pixels is then stored in the memory, and then the three-dimensional coordinate information of each pixel is sequentially obtained from the memory to update the three-dimensional map, that is, the three-dimensional map is generated by using the three-dimensional coordinate information of a single pixel in turn, and then the generated three-dimensional map is used.
  • an embodiment of the present application further provides a third three-dimensional map processing method.
  • the method is applied to the three-dimensional map processing device, and the method includes:
  • step S301 a depth image is read from the first memory; any pixel in the depth image corresponds to a depth value.
  • step S302 according to the depth value of each pixel in the depth image, obtain three-dimensional coordinate information corresponding to the pixel.
  • step S303 the map block to which the three-dimensional coordinate information corresponding to each pixel in the depth image belongs is determined.
  • step S304 the three-dimensional coordinate information belonging to different map blocks is stored in different storage locations in the first memory.
  • step S305 the three-dimensional coordinate information is sequentially read from the storage location pointed to by one of the map blocks in the first memory, and a three-dimensional map corresponding to the map block is generated according to the three-dimensional coordinate information belonging to the same map block.
  • the three-dimensional map processing device further includes a second memory provided inside, the first memory is provided outside the three-dimensional map processing device, and the three-dimensional map processing device reads the The depth image may be cached in the second memory, and the three-dimensional coordinate information belonging to the same map tile read from the first memory may be cached in the second memory. Since the second memory is provided inside the three-dimensional map processing device, the access speed of the three-dimensional map processing device to the second memory is faster than the access speed to the first memory.
  • the three-dimensional map processing device is an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), the first memory is a DDR memory, and the second memory is an SRAM.
  • ASIC Application Specific Integrated Circuit
  • the three-dimensional map processing apparatus after reading the depth image from the first memory, the three-dimensional map processing apparatus obtains the three-dimensional coordinate information corresponding to the pixel according to the depth value of each pixel in the depth image, and then determines the The map block to which the three-dimensional coordinate information corresponding to each pixel in the depth image belongs.
  • the storage capacity of the second memory is smaller than that of the first memory. The memory cannot cache too much data, so the three-dimensional map processing device stores the three-dimensional coordinate information belonging to different map blocks in different storage locations in the first memory, and then the three-dimensional map processing device sequentially starts from the first memory.
  • a storage location pointed to by one of the map blocks in a memory reads the three-dimensional coordinate information, and generates a three-dimensional map corresponding to the map block according to the three-dimensional coordinate information belonging to the same map block; this embodiment realizes that the pixels in the depth image correspond to
  • the 3D coordinate information is stored according to the different map blocks to which it belongs. Since the distances between the 3D coordinate information belonging to the same map block are not much different, the corresponding 3D coordinate information can be obtained in units of map blocks to update the historical 3D map.
  • the corresponding three-dimensional coordinate information is obtained in a single depth image belonging to the same map block as a unit to generate the map block corresponding to the
  • the three-dimensional map corresponding to the map block is smaller than the three-dimensional map generated by using the randomly distributed three-dimensional coordinate information corresponding to the entire depth map, which is beneficial to reduce the bandwidth consumption of the second memory.
  • the three-dimensional coordinate information corresponding to the pixel includes the three-dimensional coordinate corresponding to the pixel and the display radius of the target point indicated by the three-dimensional coordinate; the depth value of the pixel is positively correlated with the display radius.
  • the three-dimensional map processing device is further configured to read a historical three-dimensional map from the first memory, and then fuse the three-dimensional maps corresponding to different map blocks with the historical three-dimensional map, and obtain a fused three-dimensional map. and store the fused three-dimensional map in the first memory; this embodiment implements real-time updating of the historical three-dimensional map.
  • the three-dimensional map corresponding to the map block may be stored in the first memory.
  • the map block is determined as a result of dividing the historical three-dimensional map according to a preset direction.
  • the three-dimensional map processing device divides the historical three-dimensional map according to a preset direction in advance, and determines a plurality of map blocks, and the map blocks represent the spatial area of the preset range where the target point indicated by the three-dimensional coordinate information is located, or,
  • the map tiles indicate three-dimensional coordinates within a certain range. It can be understood that, in this embodiment, the preset direction and the number of the map blocks are not limited, and specific settings can be made according to actual application scenarios.
  • the three-dimensional map processing device when storing the three-dimensional coordinate information corresponding to the depth image, the three-dimensional map processing device first encapsulates the three-dimensional coordinate information belonging to the same map block into a data packet, and stores it in the first memory, wherein the three-dimensional coordinate information belonging to different
  • the data packets of the map block can be continuously stored in the first memory; then obtain index information indicating the storage location of the data packet, and store the index information in the storage location pointed to by the map block in the first memory,
  • the index information of data packets belonging to different map blocks is stored in different storage locations respectively pointed to by different map blocks in the first memory.
  • the data packets that encapsulate most of the data are stored continuously in the first memory, only the index information with a relatively small amount of data only needs to be stored in the storage spaces corresponding to different map blocks, because there is no need for different map blocks.
  • the storage space is determined to be too large, so as to achieve the purpose of saving the storage space of the first memory and improve the utilization rate of storage resources.
  • the three-dimensional map processing device When the three-dimensional map processing device reads the three-dimensional coordinate information from the first memory, it first sequentially reads the index information of the data packets belonging to the map block from the storage location pointed to by one of the map blocks in the first memory. , and then read the data packet from the first memory according to the index information, decapsulate the three-dimensional coordinate information to obtain the three-dimensional coordinate information, and then use the three-dimensional coordinate information to update the historical three-dimensional map. Specifically, the The three-dimensional map processing device can generate a three-dimensional map corresponding to the map block according to the three-dimensional coordinate information belonging to the same map block; and then fuse the three-dimensional maps corresponding to different map blocks with the historical three-dimensional map to obtain a fused three-dimensional map.
  • This embodiment realizes that the corresponding 3D coordinate information is acquired in units of map blocks to generate a 3D map to update the historical 3D map, which further improves the 3D map processing efficiency on the basis of reducing a large number of random accesses to the first memory.
  • an embodiment of the present application further provides a three-dimensional map processing apparatus 100 , including:
  • the coordinate information generation module 11 is used to read a plurality of depth images from the first memory 200, and any pixel in the depth image corresponds to a depth value; according to the depth value of each pixel in each depth image, Acquire three-dimensional coordinate information corresponding to the pixel.
  • the map update module is used to read the historical three-dimensional map from the first memory 200 after obtaining the three-dimensional coordinate information corresponding to all the pixels in the multiple depth images;
  • the three-dimensional coordinate information updates the historical three-dimensional map.
  • the three-dimensional coordinate information corresponding to the pixel includes the three-dimensional coordinate corresponding to the pixel and the display radius of the target point indicated by the three-dimensional coordinate.
  • the display radius is determined based on a range corresponding to the depth value of the pixel; different ranges correspond to different display radii.
  • the display radius is determined according to the depth value of the pixel and a preset corresponding relationship, and the preset corresponding relationship indicates that different depth values correspond to different display radii.
  • the depth value of the pixel is positively correlated with the display radius.
  • the apparatus 100 further includes a block module 13, and the block module 13 is configured to divide the historical three-dimensional map according to a preset direction to determine a plurality of map blocks.
  • the block module 13 is further configured to: determine the map block to which the three-dimensional coordinate information corresponding to each pixel in the depth image belongs; store the three-dimensional coordinate information belonging to different map blocks in different storage in the first memory 200 Location.
  • the map update module is further configured to: read the three-dimensional coordinate information corresponding to the plurality of depth images from different storage locations pointed to by different map blocks in the first memory 200, and use the three-dimensional coordinate information to update the history 3D map.
  • the map update module includes a map generation sub-module 121 and a map update sub-module 122 .
  • the map generation sub-module 121 is configured to: sequentially read the three-dimensional coordinate information from the storage location pointed to by one of the map blocks in the first memory 200, and generate the corresponding map block according to the three-dimensional coordinate information belonging to the same map block. 3D map.
  • the map update sub-module 122 is used for: reading a historical three-dimensional map from the first memory 200; and fusing the three-dimensional maps corresponding to different map blocks with the historical three-dimensional map to obtain a combined three-dimensional map.
  • the block module 13 is specifically configured to: encapsulate the three-dimensional coordinate information belonging to the same map block into a data packet, and store it in the first memory 200; obtain a data packet indicating the storage location of the data packet. index information, and store the index information in the storage location pointed to by the map block in the first memory 200 .
  • data packets belonging to different map tiles are stored in the first memory 200 consecutively.
  • the index information of data packets belonging to different map blocks is stored in different storage locations respectively pointed to by different map blocks in the first memory 200 .
  • the map update module is specifically configured to: read index information of data packets belonging to different map blocks from storage locations pointed to by different map blocks in the first memory 200; The data packet is read in the first memory 200, and the three-dimensional coordinate information is obtained by decapsulation.
  • the map update module includes a map generation sub-module 121 and a map update sub-module 122 .
  • the map generating sub-module 121 is configured to: generate a three-dimensional map corresponding to the map block according to the three-dimensional coordinate information belonging to the same map block.
  • the map update sub-module 122 is used for: fusing the three-dimensional maps corresponding to different map blocks with the historical three-dimensional map to obtain a three-dimensional map after fusion.
  • the three-dimensional coordinate information belonging to the same map block read from the first memory 200 is cached in the second memory, wherein the read and write speed for the second memory is faster than that for all the map blocks.
  • the read and write speed of the first memory 200 is described.
  • the storage capacity of the second memory is smaller than the storage capacity of the first memory 200 .
  • the multiple depth images are obtained by photographing the same target by multiple photographing devices 100 at different positions; and/or, the multiple depth images are photographed by one photographing device 100 at different times on the same target get.
  • the map update module includes a map generation sub-module 121 and a map update sub-module 122 .
  • the map generating sub-module 121 is configured to: generate a three-dimensional map by using the three-dimensional coordinate information corresponding to all pixels in the plurality of depth images.
  • the map update sub-module 122 is used for: reading a historical three-dimensional map from the first memory 200; and fusing the three-dimensional map with the historical three-dimensional map to obtain a combined three-dimensional map after fusion.
  • the three-dimensional map, the historical three-dimensional map and the fused three-dimensional map are represented in the form of a three-dimensional grid map.
  • the coordinate information generating module 11 is further configured to store the acquired three-dimensional coordinate information corresponding to all pixels of the depth images in the first memory 200 .
  • the map generation sub-module 121 is further configured to use the three-dimensional coordinate information corresponding to all pixels in the multiple depth images after reading the three-dimensional coordinate information corresponding to all the pixels in the multiple depth images from the first memory 200 . Generate a 3D map.
  • the map generation sub-module 121 is specifically configured to: establish the three-dimensional map, determine the corresponding grid in the three-dimensional map according to the three-dimensional coordinate information corresponding to the pixels; The target point indicated by the coordinate information is projected onto the corresponding grid in the three-dimensional map, and the attribute value of the grid is updated; the attribute value represents the probability that an obstacle exists at the position indicated by the grid.
  • the three-dimensional map processing apparatus 100 includes a second memory installed inside, each grid in the three-dimensional map has a corresponding storage location in the second memory, and the storage location stores The attribute value of the raster.
  • the map generation sub-module 121 is specifically configured to: superimpose the attribute value at the storage location corresponding to the grid in the second memory with a preset value.
  • the map generation sub-module 121 includes a map reading unit 1221 and a map fusion unit 1222 .
  • the map reading unit 1221 is configured to: read a historical three-dimensional map from the first memory 200; and determine the three-dimensional map and a grid corresponding to the same three-dimensional coordinate information in the historical three-dimensional map.
  • the map fusion unit 1222 is configured to: superimpose attribute values of grids corresponding to the same 3D coordinate information to obtain the fused 3D map.
  • the map fusion unit 1222 is further configured to: if the attribute value of the grid in the fused three-dimensional map is greater than the upper threshold, modify the attribute value corresponding to the grid to the upper threshold ; if the attribute value in the grid in the fused three-dimensional map is less than the lower threshold, modify the attribute value corresponding to the grid to the lower threshold.
  • the map fusion unit 1222 is further configured to: store the fused three-dimensional map in the first memory 200 .
  • an embodiment of the present application further provides a three-dimensional map processing apparatus 100 , including:
  • the coordinate information generation module 11 is used to read the depth image from the first memory 200; any pixel in the depth image corresponds to a depth value; according to the depth value of each pixel in the depth image, obtain the depth image The three-dimensional coordinate information corresponding to the pixel.
  • the block module 13 is configured to determine the map block to which the three-dimensional coordinate information corresponding to each pixel in the depth image belongs;
  • the map generation module 14 is configured to sequentially read three-dimensional coordinate information from a storage location pointed to by one of the map blocks in the first memory 200, and generate a three-dimensional map corresponding to the map block according to the three-dimensional coordinate information belonging to the same map block.
  • the apparatus 100 further includes: a map update module 12, configured to read a historical three-dimensional map from the first memory 200; The historical 3D map is fused to obtain a fused 3D map.
  • the map block is determined as a result of dividing the historical three-dimensional map according to a preset direction.
  • the block module 13 is specifically configured to: encapsulate the three-dimensional coordinate information belonging to the same map block into a data packet, and store it in the first memory 200; obtain a data packet indicating the storage location of the data packet. index information, and store the index information in the storage location pointed to by the map block in the first memory 200 .
  • data packets belonging to different map tiles are stored in the first memory 200 consecutively.
  • the index information of data packets belonging to different map blocks is stored in different storage locations respectively pointed to by different map blocks in the first memory 200 .
  • the map generation module 14 is specifically configured to: sequentially read the index information of the data packets belonging to the map block from the storage location pointed to by one of the map blocks in the first memory 200; according to the index The information is read from the first memory 200 and the data packet is decapsulated to obtain the three-dimensional coordinate information.
  • the three-dimensional coordinate information belonging to the same map block read from the first memory 200 is cached in the second memory, wherein the access speed for the second memory is better than that for the second memory.
  • the access speed of the first memory 200 is better than that for the second memory.
  • the storage capacity of the second memory is smaller than the storage capacity of the first memory 200 .
  • non-transitory computer-readable storage medium such as a memory including instructions, executable by a processor of an apparatus to perform the above-described method.
  • the non-transitory computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like.
  • a non-transitory computer-readable storage medium when the instructions in the storage medium are executed by the processor of the terminal, enable the terminal to execute the above method.
  • an embodiment of the present application further provides a movable platform, including the above-mentioned three-dimensional map processing device.
  • the movable platform includes at least one of the following: an unmanned aerial vehicle, an unmanned vehicle, a mobile robot, an unmanned watercraft, or a gimbal.
  • the movable platform is further mounted with one or more photographing devices, and the photographing devices are used for taking depth images.

Landscapes

  • Engineering & Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Automation & Control Theory (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Processing Or Creating Images (AREA)

Abstract

一种三维地图处理方法、装置及存储介质，所述方法包括：从第一存储器中读取多张深度图像；所述深度图像中的任一像素对应有深度值；根据每一张深度图像中的每个像素的深度值，获取所述像素对应的三维坐标信息；在得到所述多张深度图像中所有像素对应的三维坐标信息后，从所述第一存储器中读取历史三维地图；利用所述多张深度图像中所有像素对应的三维坐标信息更新所述历史三维地图。本实施例有利于降低存储器的带宽损耗以及提高处理速度，从而满足实时性需求。

Description

三维地图处理方法、装置、可移动平台及存储介质 技术领域
本申请涉及三维重建技术领域，具体而言，涉及一种三维地图处理方法、装置、可移动平台及存储介质。
背景技术
在计算机视觉中，三维重建是指根据单视图或者多视图的图像重建三维信息的过程。比如会基于深度图进行三维重建，由于单张深度图获取的信息有限，在生成三维地图时通常会使用多张深度图进行融合。相关技术中利用深度图生成三维地图时，使用的方法是将单张深度图内所有深度信息反投影到三维空间并对该三维空间进行障碍信息标记，然后重复这一动作，依次利用其它深度图对同一三维空间进行更新，最终得到三维地图。该过程在每一次利用深度图对同一三维空间进行更新均需要从内存中读取三维地图，由于三维地图的数据量较大，多次从内存中读取三维地图的过程需要占用很大的内存带宽，在更新完成后还需将更新后的三维地图写入内存中，同时多次读写三维地图也会带来很大处理延迟，很难满足实时性的要求。
发明内容
有鉴于此，本申请的目的之一是提供一种三维地图处理方法、装置、可移动平台及存储介质。
第一方面，本申请实施例提供了一种三维地图处理方法，包括：
从第一存储器中读取多张深度图像；所述深度图像中的任一像素对应有深度值；
根据每一张深度图像中的每个像素的深度值，获取所述像素对应的三维坐标信息；
在得到所述多张深度图像中所有像素对应的三维坐标信息后，从所述第一存储器中读取历史三维地图；
利用所述多张深度图像中所有像素对应的三维坐标信息更新所述历史三维地图。
第二方面，本申请实施例提供了一种三维地图处理方法，包括：
从第一存储器中读取深度图像；所述深度图像中的任一像素对应有深度值；
根据所述深度图像中的每个像素的深度值，获取所述像素对应的三维坐标信息；
确定所述深度图像中各个像素对应的三维坐标信息所属地图块；
将属于不同地图块的三维坐标信息存储至所述第一存储器中的不同存储位置；
依次从所述第一存储器中其中一个地图块指向的存储位置读取三维坐标信息，并根据属于同一地图块的三维坐标信息生成所述地图块对应的三维地图。
第三方面，本申请实施例提供了一种三维地图处理装置，包括：
坐标信息生成模块，用于从第一存储器中读取多张深度图像，所述深度图像中的任一像素对应有深度值；根据每一张深度图像中的每个像素的深度值，获取所述像素对应的三维坐标信息；
地图更新模块，用于在得到所述多张深度图像中所有像素对应的三维坐标信息后，从所述第一存储器中读取历史三维地图；利用所述多张深度图像中所有像素对应的三维坐标信息更新所述历史三维地图。
第四方面，本申请实施例提供了一种三维地图处理装置，包括：
坐标信息生成模块，用于从第一存储器中读取深度图像；所述深度图像中的任一像素对应有深度值；根据所述深度图像中的每个像素的深度值，获取所述像素对应的三维坐标信息；
分块模块，用于确定所述深度图像中各个像素对应的三维坐标信息所属地图块；将属于不同地图块的三维坐标信息存储至所述第一存储器中的不同存储位置；
地图生成模块，用于依次从所述第一存储器中其中一个地图块指向的存储位置读取三维坐标信息，并根据属于同一地图块的三维坐标信息生成所述地图块对应的三维地图。
第五方面，本申请实施例提供了一种计算机可读存储介质，其上存储有计算机程序，当所述计算机程序被三维地图处理装置执行时用于实现第一方面或第二方面任意一项所述的方法。
第六方面，本申请实施例提供了一种可移动平台，包括如第三方面或者第四方面任一所述的装置。
本申请实施例所提供的一种三维地图处理方法、装置、可移动平台及存储介质，只需读取一次历史三维地图，有利于降低存储器的带宽损耗，利用多张深度图一次性更新的过程也有利于提高处理速度，从而满足实时性需求。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1、图5以及图8是本申请实施例提供的三维地图处理方法的不同流程示意图；
图2、图3、图9～图15是本申请实施例提供的三维地图处理装置的不同结构示意图；
图4是本申请实施例提供的基于深度图生成三维地图的示意图；
图6是本申请实施例提供的划分地图块的示意图；
图7是本申请实施例提供了生成其中一个分块对应的三维地图的示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
考虑到相关技术中利用深度图生成三维地图时，使用的方法是将单张深度图内所有深度信息反投影到三维空间并对该三维空间进行障碍信息标记，然后重复这一动作，依次利用其它深度图对同一三维空间进行更新，最终得到三维地图。该过程在每一次利用深度图对同一三维空间进行更新均需要从内存中读取三维地图，在读取的三维地图的基础上进行更新，由于三维地图的数据量较大，多次从内存中读取三维地图的过程需要占用很大的内存带宽，在更新完成后还需将更新后的三维地图写入内存中，同时多次读写三维地图也会带来很大处理延迟，很难满足实时性的要求。
在一个例子中，比如在实时的SLAM(simultaneous localization and mapping，同步定位与建图)系统上会不断使用深度图重复上述方式来刷新三维地图，使得三维地图以一定的帧率进行更新，来保证三维地图的实时性，但上述方式需要多次读写三维地图，存在很大的处理延迟，难以满足实时性的要求。所述SLAM系统用于解决SLAM (simultaneous localization and mapping，同步定位与建图)问题，SLAM问题可以描述为：将一个机器人放入未知环境中的未知位置，是否有办法让机器人一边移动一边逐步描绘出此环境完全的地图，所谓完全的地图(a consistent map)是指不受障碍行进到房间可进入的每个角落。
基于此，本申请实施例提供了一种三维地图处理方法、装置及可移动平台，同时读取多张深度图并获取所述多张深度图像中所有像素对应的三维坐标信息，在从第一存储器中读取历史三维地图之后，利用所述多张深度图像中所有像素对应的三维坐标信息一次性更新所述历史三维地图，从而实现只需从第一存储器中读取一次所述历史三维地图，有利于降低所述第一存储器的带宽损耗，同时一次性更新的过程也有利于提高处理速度，从而满足实时性需求。
本申请实施例的所述三维地图处理方法可以应用于三维地图处理装置中，所述三维地图处理装置可以是具有数据处理能力的电子设备，如电脑、服务器、云端服务器或者终端、可移动平台(例如无人飞行器、无人驾驶车辆或者移动机器人等)等；也可以是具有数据处理能力的计算机芯片或者集成电路，例如中央处理单元(Central Processing Unit，CPU)、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)或者现成可编程门阵列(Field-Programmable Gate Array，FPGA)等。
在一实施例中，当所述三维地图处理装置为具有数据处理能力的计算机芯片或者集成电路时，所述三维地图处理装置可以安装在可移动平台上，所述可移动平台包括但不限于无人飞行器、无人驾驶车辆、移动机器人、无人驾驶船只或者云台。
示例性地，当所述三维地图处理装置安装于所述无人飞行器时，所述无人飞行器可以根据所述三维地图处理装置实时更新的三维地图进行航测、导航、避障或者定位等。
示例性地，当所述三维地图处理装置安装于所述无人驾驶车辆时，所述无人驾驶车辆可以根据所述三维地图处理装置实时更新的三维地图进行自动驾驶、避障等。
示例性地，所述可移动平台上还可以安装有一个或多个拍摄装置(如深度相机)，所述拍摄装置用于采集所述三维地图处理装置所需的深度图像。
示例性地，所述可移动平台在获取所述三维地图处理装置实时更新的三维地图之后，可以将所述三维地图发送给所述可移动平台关联的终端，以在所述终端上显示所述三维地图。
本申请实施例的所述三维地图处理方法可以应用但不限于地图导航场景、车辆驾 驶场景、地点查询场景、勘测场景、监控场景或者救援场景等。在一个例子中，在自动驾驶场景下，可以采用本申请实施例提供的所述三维地图处理方法生成以及实时更新三维地图，从而汽车可以基于三维地图进行自动驾驶以及避障等。
请参阅图1，图1为本申请实施例提供的一种三维地图处理方法的流程示意图，所述方法可应用于三维地图处理装置上，其中，所述三维地图处理装置需要从第一存储器中读取多张深度图像以及历史三维地图；在一个例子中，请参阅图2，所述第一存储器200可以设置于所述三维地图处理装置100外部，比如所述三维地图处理装置100为专用集成电路(Application Specific Integrated Circuit，ASIC)，所述第一存储器200为DDR内存；在另一个例子中，所述第一存储器也可以设置于所述三维地图处理装置内部，比如所述三维地图处理装置为电脑，所述第一存储器为设置于电脑上的内存储器。所述方法包括：
在步骤S101中，从第一存储器中读取多张深度图像；所述深度图像中的任一像素对应有深度值。
在步骤S102中，根据每一张深度图像中的每个像素的深度值，获取所述像素对应的三维坐标信息。
在步骤S103中，在得到所述多张深度图像中所有像素对应的三维坐标信息后，从所述第一存储器中读取历史三维地图。
在步骤S104中，利用所述多张深度图像中所有像素对应的三维坐标信息更新所述历史三维地图。
可以理解的是，本申请实施例对于所述多张深度图像的获取不做任何限制，可依据实际应用场景进行具体设置；在一个例子中，所述多张深度图像可以由所述三维地图处理装置上搭载的拍摄装置(如深度相机)实时拍摄得到；在另一个例子中，所述多张深度图像可以是所述三维地图处理装置从其他终端获取。
其中，所述多张深度图像可以由处于不同位置的多个拍摄装置(如深度相机)对同一目标拍摄得到，实现从不同方向获取所述同一目标及其周围的信息，使得基于所述多张深度图像融合后得到的三维地图所包含的所述目标周围的信息(比如障碍信息)更加全面；和/或(和/或表示两者或者两者之一)，所述多张深度图像可以由一个拍摄装置(如深度相机)在不同时刻对同一目标拍摄得到，由于不同时刻获取的所述同一目标的信息也有所差别，则基于所述多张深度图像融合后得到的三维地图能够更好过滤噪声，具有更好的鲁棒性。
所述多张深度图可以存储于所述第一存储器中，以便在进行三维图像处理时，由 所述三维图像处理装置读取。在一个例子中，所述第一存储器可以设置与所述三维地图处理装置外部。或者，所述第一存储器也可以设置于所述三维地图处理装置的内部，本申请实施例对此不做任何限制。所述第一存储器可以是内存储器如DDR(Double Data Rate，双倍速率)内存等，可依据实际应用场景进行具体设置。
在一实施例中，请参阅图3，所述三维地图处理装置100还包括有设置于内部的第二存储器300，所述第二存储器300用于缓存(或者说暂时性保存)所述三维地图处理装置在进行数据处理过程中的一些数据。在进行三维地图处理时，从所述第一存储器读取的多张深度图可以缓存(或者说暂时性保存)在所述第二存储器中。
在进行三维地图处理时，所述三维地图处理装置首先从所述第一存储器中读取所述多张深度地图，所述深度图像中的任一像素对应有深度值，然后根据每一张深度图像中的每个像素的深度值，获取所述像素对应的三维坐标信息。
在一种实现方式中，在所述第二存储器存储容量不足的情况下，所述三维地图处理装置可以每次从第一存储器读取至少一张深度图像，其中，读取的深度图像的数量适应于所述第二存储器的存储容量，然后所述三维地图处理装置根据读取的所述至少一张深度图像中的每个像素的深度值，获取所述像素对应的三维坐标信息，然后将所述至少一张深度图像对应的所有像素的三维坐标信息存储至所述第一存储器中，接着所述三维地图处理装置继续从第一存储器中读取未处理的深度图像进行与上述步骤相同的步骤，直到得到所述多张深度图像中所有像素对应的三维坐标信息。
在另一种实现方式中，在所述第二存储器存储容量足够的情况下，所述三维地图处理装置可以一次性从第一存储器读取所述多张深度图像，然后根据每一张深度图像中的每个像素的深度值，获取所述像素对应的三维坐标信息并缓存在所述第二存储器中，直到得到所述多张深度图像中所有像素对应的三维坐标信息。
其中，所述三维坐标信息包括所述像素对应的三维坐标信息包括所述像素对应的三维坐标以及所述三维坐标指示的目标点的显示半径；所述目标点表示在点云中所述三维坐标指示的点，所述目标点用于显示在三维地图中。在一个例子中，请参阅表1，三维坐标信息(用map_info表示)存储格式如表1所示，idx_x、idx_y、idx_z分别表示目标点在三维地图中的坐标，比如使用8比特表示，tab_info表示目标点的显示半径信息，比如使用2比特表示，一个目标点占用26bit位宽。
表1
Figure PCTCN2020112700-appb-000001
其中，所述三维坐标指示的目标点的显示半径可以根据所述像素的深度值来确定。在一种实现方式中，可以预先设置所述深度值的不同范围对应不同的显示半径，则所述显示半径基于所述像素的深度值对应的范围所确定，所述三维地图处理装置在确定所述显示半径时，首先确定所述像素的深度值所属范围，根据所述深度值所属范围获取相应的显示半径作为所述三维坐标指示的目标点的显示半径。在另一种实现方式中，可以预先设置一预设对应关系，所述预设对应关系指示不同深度值对应不同的显示半径，所述三维地图处理装置在确定所述显示半径时，根据所述像素的深度值从所述预设对应关系中获取相应的显示半径作为所述三维坐标指示的目标点的显示半径。
考虑到所述像素的深度值越大，表明所述像素对应的三维坐标指示的目标点的距离越远，我们知道，距离越远越无法确定一个精确的空间位置，相应的，基于所述三维坐标以及所述目标点确定的三维地图中的位置的准确率也会有所下降，因此，可以设置所述像素的深度值与所述显示半径呈正相关关系，即所述像素的深度值越小，则所述像素对应的三维坐标指示的目标点的显示半径也越小，可以这么说，距离越近可以确定一个越精确的空间位置，所述显示半径可以相应设置得小一些；反之，所述像素的深度值越大，所述像素对应的三维坐标指示的目标点的显示半径也越大，可以这么说，距离越远确定的空间位置越粗糙，则所述显示半径需要设置大一些，从而指示更大的空间位置范围，以避免空间位置确定出现误差。
相关技术中利用深度图生成三维地图时，使用的方法是将单张深度图内所有深度信息反投影到三维空间并对该三维空间进行障碍标记，具体来说，是获取该深度图中所有像素的三维坐标信息然后存储到内存中，接着从内存中依次获取各个像素的三维坐标信息实现对三维地图的更新，即依次用单个像素的三维坐标信息生成三维地图，然后用生成的三维地图来更新历史三维地图；由于深度图像中的每个像素对应的三维坐标信息指示的目标点在三维空间中是随机分布的，比如图4所示，像素P1对应的三维坐标信息指示的目标点与像素P2对应的三维坐标信息指示的目标点分布在三维地图中的不同位置，且两者距离相差较远，因此如果以所述深度图中的像素对应的三维坐标信息进行顺序处理实现对三维地图的更新，可能会存在对内存大量的随机访问，造成访问内存效率低的问题。
基于此，在第一种实现方式中，考虑到第二存储器的存储容量有限的问题，在获取到每一张深度图像中的每个像素对应的三维坐标信息后，所述三维地图处理装置可以将每一张深度图像对应的所有三维坐标信息存储至所述第一存储器中。在一个例子中，所述第一存储器设置于所述三维地图处理装置的外部，所述第二存储器设置于所 述三维地图处理装置的内部。在得到所述多张深度图像中所有像素对应的三维坐标信息并且将所述多张深度图像对应的三维坐标信息均存储至所述第一存储器之后，所述三维地图处理装置从第一存储器中读取所述多张深度图像中所有像素对应的三维坐标信息以及所述历史三维地图，然后所述三维地图处理装置利用所述多张深度图像中所有像素对应的三维坐标信息更新所述历史三维地图。具体而言，所述三维地图处理装置可以依次从所述第一存储器中读取至少一张深度图像中所有像素对应的三维坐标信息(读取的深度图像的数量适应于所述第二存储器的内存)，以生成每张深度图像对应的三维地图，接着将所有深度图像对应的三维地图与所述历史三维地图进行融合，比如可以将每张深度图像对应的三维地图依次与所述历史三维地图进行融合，从而获得融合后的三维地图；本实施例实现将以每张深度图为单位获取所述深度图中所有像素对应的三维坐标信息并缓存在第二存储器中，在第二存储器中实现三维地图的融合更新，而不是依次获取各个像素的三维坐标信息实现对三维地图的更新，从而减少了对第一存储器的大量的随机访问；而且本实施例中，考虑到所述第二存储器存储容量有限的问题，且从所述第一存储外读取三维坐标信息所需的带宽远远小于读取所述三维地图所需的带宽，因此将所述深度图像对应的三维坐标信息存储至所述第一存储器中，在更新所述历史三维地图时，实现只需从第一存储器中读取一次所述历史三维地图，有利于降低所述第一存储器的带宽损耗，同时一次性更新所述历史三维地图的过程也有利于提高处理速度，从而满足实时性需求。
在第二种实现方式中，在所述第二存储器有足够的存储容量的情况下，在获取到每一张深度图像中的每个像素对应的三维坐标信息后，可以将所述三维坐标信息缓存于所述第二存储器中。在一个例子中，请参阅图3，所述第一存储器200设置于所述三维地图处理装置100的外部，所述第二存储器300设置于所述三维地图处理装置100的内部。在得到所述多张深度图像中所有像素对应的三维坐标信息后，所述三维地图处理装置从所述第一存储器中读取历史三维地图，所述历史三维地图为基于历史深度图像生成的，所述历史三维地图也会缓存于所述第二存储器300中，其中，针对于所述第二存储器的读写速度优于针对于所述第一存储器的读写速度；然后所述三维地图处理装置在所述第二存储器中，利用所述多张深度图像中所有像素对应的三维坐标信息更新所述历史三维地图。具体而言，所述三维地图处理装置利用所述多张深度图像中所有像素对应的三维坐标信息生成三维地图，接着将所述三维地图与所述历史三维地图进行融合，获得融合后的三维地图。本实施例实现将所述多张深度图像对应的三维坐标信息缓存在所述第二存储器中，由于所述第二存储器安装于所述三维地图处理 装置内部，使得针对于所述第二存储器的读写速度优于针对于所述第一存储器的读写速度，在第二存储器中实现三维地图的融合更新，也减少了对第一存储器的大量的随机访问；而且本实施例利用所述多张深度图像中所有像素对应的三维坐标信息一次性更新所述历史三维地图，从而实现只需从第一存储器中读取一次所述历史三维地图，有利于降低所述第一存储器的带宽损耗，同时一次性更新的过程也有利于提高处理速度，从而满足实时性需求。
在一个例子中，请参阅图4，本申请实施例利用N(N为大于1的整数)张h(高)*w(宽)的深度图来生成三维地图，每张深度图中的任一像素对应有深度值，根据所述深度值可以获取所述像素对应的三维坐标信息，根据所述三维坐标信息生成三维地图，所述三维地图可以用来更新历史三维地图，比如所述深度图有像素p1和像素p2，根据像素p1的深度值获取其三维坐标信息以及根据像素p2的深度值获取其三维坐标信息，在根据N张深度图的所有像素生成对应的三维坐标信息生成的三维地图中，图中示出像素p1对应的目标点P1’和像素p2对应的目标点P2’在三维地图中的位置，所述目标点表示在点云中所述三维坐标信息指示的点，所述目标点用于显示在三维地图中，所述三维地图可以用来更新历史三维地图。
其中，所述三维地图、所述历史三维地图以及所述融合后的三维地图均以三维栅格地图形式表示。则在利用所述多张深度图像中所有像素对应的三维坐标信息生成三维地图时，所述三维地图处理装置建立所述三维地图，所述三维地图包括有若干个栅格，所述三维地图处理装置根据所述像素对应的三维坐标信息确定所述三维地图中对应的栅格，并将所述像素对应的三维坐标信息指示的目标点投射到所述三维地图中对应的栅格上，所述目标点表示在点云中所述三维坐标信息指示的点，然后根据所述三维坐标信息更新所述对应的栅格的属性值；所述属性值表征所述栅格指示的位置存在障碍物的概率。其中，在所述第二存储器资源有限的情况下，所述三维地图可以根据每张深度图像对应的三维坐标信息生成所述深度图像对应的三维地图，则每张深度图像分别对应有三维地图；在所述第二存储资源足够的情况下，所述三维地图处理装置可以根据多张深度图像对应的三维坐标信息生成一个三维地图，则多张深度图像对应一个三维地图。
在一实施例中，所述三维地图中的每个栅格在所述第二存储器中对应有存储位置，所述存储位置处存储所述栅格的属性值；所述三维地图处理装置在建立所述三维地图时，可以为所述三维地图中的每个栅格初始化一个属性值，并将初始化的属性值存储至所述栅格在第二存储器中对应的存储位置上。可理解的是，本申请对于初始化的属 性值的具体取值不做任何限制，可依据实际应用场景进行具体设置。则所述三维地图处理装置在更新所述栅格的属性值时，可以根据所述三维坐标信息确定所述三维地图中对应的栅格，将所述第二存储器中与该栅格对应的存储位置处的属性值叠加预设值。
最后，在进行融合时，所述三维地图处理装置确定所述三维地图以及所述历史三维地图中对应相同三维坐标信息的栅格，将对应相同三维坐标信息的栅格的属性值叠加，获得所述融合后的三维地图。在一种实现方式中，若所述融合后的三维地图中的栅格的属性值大于上限阈值，将所述栅格对应的属性值修改为所述上限阈值；若所述融合后的三维地图中的栅格中的属性值小于下限阈值，将所述栅格对应的属性值修改为所述下限阈值。本实施例设置了上限阈值和下限阈值，将所述三维地图的属性值控制在一个范围内，防止异常属性值的影响导致融合后的三维地图不准确。
在第三种实现方式中，为了在减少对第一存储器的大量的随机访问的基础上，进一步提高三维地图处理效率，请参阅图5，本申请实施例还提供了另一种三维地图处理方法的流程示意图，所述方法可以由所述三维地图处理装置来执行，所述三维地图处理装置预先按照预设方向对所述历史三维地图进行划分，确定多个地图块，所述地图块指示三维坐标信息指示的目标点所在的预设范围的空间区域，或者说，所述地图块指示指定范围内的三维坐标，从而按照所述地图块将所述深度图中所有像素对应的三维坐标信息进行分类存储；所述方法包括：
在步骤S201中，从第一存储器中读取多张深度图像；所述深度图像中的任一像素对应有深度值。
在步骤S202中，根据每一张深度图像中的每个像素的深度值，获取所述像素对应的三维坐标信息。
在步骤S203中，确定所述深度图像中各个像素对应的三维坐标信息所属地图块，并将属于不同地图块的三维坐标信息存储至所述第一存储器中的不同存储位置。
在步骤S204中，在得到所述多张深度图像中所有像素对应的三维坐标信息后，从所述第一存储器中读取历史三维地图。
在步骤S205中，从所述第一存储器中不同地图块指向的不同存储位置读取所述多张深度图像对应的三维坐标信息，并使用所述三维坐标信息更新所述历史三维地图。
本实施例中，实现将所述深度图像中的像素对应的三维坐标信息按照所属的不同地图块分类存储，由于属于相同地图块的三维坐标信息之间的距离相差不大，后续可以以地图块为单位获取相应的三维坐标信息来更新所述历史三维地图，实现在减少对第一存储器的大量的随机访问的基础上，进一步提高三维地图处理效率。
在一实施例中，所述三维地图处理装置预先按照预设方向对所述历史三维地图进行划分，确定多个地图块，所述地图块表示指示三维坐标信息指示的目标点所在的预设范围的空间区域，或者说，所述地图块指示某个范围内的三维坐标。可以理解的是，本实施例对于所述预设方向以及所述地图块的数量不做任何限制，可依据实际应用场景进行具体设置，比如请参阅图6，所述预设方向可以是所述历史三维地图沿着三维坐标轴的方向，如沿着三维坐标轴X轴的方向确定5个地图块，分别为地图块01、地图块02、地图块03、地图块04和地图块05，不同地图块指示不同范围内的三维坐标，换句话说，不同范围内的三维坐标指示的目标点处于不同的地图块中。
在一实施例中，所述第一存储器安装于所述三维地图处理装置外部，所述三维地图处理装置包括有安装于内部的第二存储器，针对于所述第二存储器的读写速度优于针对于所述第一存储器的读写速度并且所述第二存储器的单位容量的价格高于第一存储器，且为了降低成本，所述第二存储器的存储容量小于所述第一存储器的存储容量。在一个例子中，所述第一存储器可以是DDR存储器，所述第二存储器可以是SRAM。
所述三维地图处理装置从所述第一存储器中读取多张深度图像，并将所述多张深度图像缓存于所述第二存储器中，其中，在所述第二存储器的存储资源有限的情况下，所述三维地图处理装置可以一次从第一存储器读取一张深度图像缓存至所述第二存储器；在所述第二存储器的存储资源足够的情况下，所述三维地图处理装置可以一次从所述第一存储器中读取多张深度图像缓存至所述第二存储器。
所述深度图像中的任一像素对应有深度值，所述三维地图处理装置根据每一张深度图像中的每个像素的深度值，获取所述像素对应的三维坐标信息，然后确定所述深度图像中各个像素对应的三维坐标信息所属地图块，并将属于不同地图块的三维坐标信息存储至所述第一存储器中的不同存储位置；本实施例实现将所述深度图像中的像素对应的三维坐标信息按照所属的不同地图块分类存储，由于属于相同地图块的三维坐标信息之间的距离相差不大，后续可以以地图块为单位获取相应的三维坐标信息来更新所述历史三维地图，实现在减少对第一存储器的大量的随机访问的基础上，进一步提高三维地图处理效率。
接着，在得到所述多张深度图像中所有像素对应的三维坐标信息后，从所述第一存储器中读取历史三维地图以及从所述第一存储器中不同地图块指向的不同存储位置读取所述多张深度图像对应的三维坐标信息，所述历史三维地图以及所述多张深度图像对应的三维坐标信息会缓存在所述第二存储器中，所述三维地图处理装置使用所述三维坐标信息更新所述历史三维地图。
在一种实现方式中，所述三维地图处理装置依次从所述第一存储器中其中一个地图块指向的存储位置读取三维坐标信息，并根据属于同一地图块的三维坐标信息生成所述地图块对应的三维地图，请参阅图7，为生成所述地图块对应的三维地图的示意图；然后将不同地图块分别对应的三维地图与所述历史三维地图进行融合，获得融合后的三维地图，比如所述三维地图处理装置可以依次将不同地图块分别对应的三维地图与所述历史三维地图顺序进行融合；本实施例实现以地图块为单位获取相应的三维坐标信息生成三维地图来更新所述历史三维地图，实现在减少对第一存储器的大量的随机访问的基础上，进一步提高三维地图处理效率。
进一步地，考虑到所述第二存储器容量有限的问题，使用多张深度图像中属于同一地图块的三维坐标信息生成所述地图块对应的三维地图，如果所述深度图像的数量太多，则多张深度图像中属于同一地图块的三维坐标信息的数据量也较大，可能无法一次性处理，则所述三维地图处理装置可以以单张深度图像中属于同一地图块的三维坐标信息为单位进行处理，具体来说，所述三维地图处理装置可以依次从所述第一存储器中其中一个地图块指向的存储位置读取每张深度图像对应的三维坐标信息，并根据每张深度图像中属于同一地图块的三维坐标信息生成所述地图块对应的三维地图，然后将多张深度图像中不同地图块分别对应的三维地图与所述历史三维地图进行融合，获得融合后的三维地图；本实施例实现以地图块为单位获取相应的三维坐标信息生成三维地图来更新所述历史三维地图，实现在减少对第一存储器的大量的随机访问的基础上，进一步提高三维地图处理效率；并且由于以单张深度图像中属于同一地图块为单位获取相应的三维坐标信息生成所述地图块对应的三维地图，所述地图块对应的三维地图小于利用整张深度图对应的随机分布的三维坐标信息生成的三维地图，有利于降低第二存储器的带宽损耗。
在一实施例中，考虑到将属于不同地图块的三维坐标信息存储至所述第一存储器中的不同存储位置时，需要基于不同地图块确定在所述第一存储器中的不同存储空间，由于在开始处理前不知道属于某个地图块的三维坐标信息的数据量有多少，因此只能假设所有的三维坐标信息所指示的目标点均可能落到同一个地图块内，因此需要在第一存储器中确定与所述地图块的数量相应的空间，假设N个空间，N为大于1的整数，每个空间的大小必须满足存储所有三维坐标信息，但所有空间的平均利用率只有1/N，造成了第一存储器存储空间的浪费。
因此，所述三维地图处理装置在存储所述深度图像对应的三维坐标信息时，首先将属于同一地图块的三维坐标信息封装成数据包，并存储至所述第一存储器中，其中， 属于不同地图块的数据包在所述第一存储器中可以连续存储；然后获取指示所述数据包存储位置的索引信息，并将所述索引信息存储至第一存储器中所述地图块指向的存储位置，属于不同地图块的数据包的索引信息存储在所述第一存储器中不同地图块分别指向的不同存储位置中。本实施例中，由于封装大部分数据的数据包在第一存储器中连续存储，而在不同地图块对应的存储空间中只需存储数据量比较小的索引信息，因为无需为不同地图块的存储空间确定太大的容量，从而达到节省第一存储器存储空间的目的，提高存储资源的利用率。
在一个例子中，请参阅表1，表1示出了所述深度图像中一个像素对应的三维坐标信息的存储格式；可以按照预设数量规则将属于同一地图块的三维坐标信息封装成数据包，比如请参阅表2，表2示出了数据包的存储格式，比如设置一个数据包包括64个像素对应的三维坐标信息，该64个像素对应的三维坐标信息均属于同一地图块。对于各个数据包对应的索引信息，请参阅表3，为所述索引信息的存储格式，属于不同地图块的数据包的索引信息存储在所述第一存储器中不同地图块分别指向的不同存储位置中。
表2
Figure PCTCN2020112700-appb-000002
表3
Figure PCTCN2020112700-appb-000003
则所述三维地图处理装置从所述第一存储器中读取三维坐标信息时，首先从所述第一存储器中不同地图块分别指向的存储位置读取属于不同地图块的数据包的索引信 息，然后根据所述索引信息从所述第一存储器中读取所述数据包，解封得到所述三维坐标信息，然后使用所述三维坐标信息更新所述历史三维地图。具体而言，所述三维地图处理装置可以根据属于同一地图块的三维坐标信息生成所述地图块对应的三维地图；将不同地图块分别对应的三维地图与所述历史三维地图进行融合，获得融合后的三维地图。本实施例实现以地图块为单位获取相应的三维坐标信息生成三维地图来更新所述历史三维地图，实现在减少对第一存储器的大量的随机访问的基础上，进一步提高三维地图处理效率。
考虑到相关技术中利用深度图生成三维地图时，使用的方法是将单张深度图内所有深度信息反投影到三维空间并对该三维空间进行障碍标记，具体来说，是获取该深度图中所有像素的三维坐标信息然后存储到内存中，接着从内存中依次获取各个像素的三维坐标信息实现对三维地图的更新，即依次用单个像素的三维坐标信息生成三维地图，然后用生成的三维地图来更新历史三维地图；由于深度图像中的每个像素对应的三维坐标信息指示的目标点在三维空间中是随机分布的，因此如果以所述深度图中的像素对应的三维坐标信息进行顺序处理实现对三维地图的更新，会存在对DDR大量的随机访问，造成访问内存效率低的问题。
基于此，请参阅图8，本申请实施例还提供了第三种三维地图处理方法，所述方法应用于所述三维地图处理装置上，所述方法包括：
在步骤S301中，从第一存储器中读取深度图像；所述深度图像中的任一像素对应有深度值。
在步骤S302中，根据所述深度图像中的每个像素的深度值，获取所述像素对应的三维坐标信息。
在步骤S303中，确定所述深度图像中各个像素对应的三维坐标信息所属地图块。
在步骤S304中，将属于不同地图块的三维坐标信息存储至所述第一存储器中的不同存储位置。
在步骤S305中，依次从所述第一存储器中其中一个地图块指向的存储位置读取三维坐标信息，并根据属于同一地图块的三维坐标信息生成所述地图块对应的三维地图。
其中，所述三维地图处理装置还包括有设置于内部的第二存储器，所述第一存储器设置于所述三维地图处理装置的外部，所述三维地图处理装置从第一存储器读取的所述深度图像可以缓存在所述第二存储器中，以及从所述第一存储器中读取的属于同一地图块的三维坐标信息缓存在第二存储器中。由于所述第二存储器设置于所述三维地图处理装置内部，则所述三维地图处理装置针对于所述第二存储器的访问速度优于 针对于所述第一存储器的访问速度。在一个例子中，所述三维地图处理装置为专用集成电路(Application Specific Integrated Circuit，ASIC)，所述第一存储器为DDR内存，所述第二存储器为SRAM。
本实施例中，所述三维地图处理装置从第一存储器中读取深度图像后，根据所述深度图像中的每个像素的深度值，获取所述像素对应的三维坐标信息，然后确定所述深度图像中各个像素对应的三维坐标信息所属地图块，考虑到所述第二存储器的存储容量有限的问题，所述第二存储器的存储容量小于所述第一存储器的存储容量，所述第二存储器无法缓存过多的数据量，因此所述三维地图处理装置将属于不同地图块的三维坐标信息存储至所述第一存储器中的不同存储位置，接着所述三维地图处理装置依次从所述第一存储器中其中一个地图块指向的存储位置读取三维坐标信息，并根据属于同一地图块的三维坐标信息生成所述地图块对应的三维地图；本实施例实现将所述深度图像中的像素对应的三维坐标信息按照所属的不同地图块分类存储，由于属于相同地图块的三维坐标信息之间的距离相差不大，后续可以以地图块为单位获取相应的三维坐标信息来更新所述历史三维地图，实现在减少对第一存储器的大量的随机访问的基础上，进一步提高三维地图处理效率；而且由于以单张深度图像中属于同一地图块为单位获取相应的三维坐标信息生成所述地图块对应的三维地图，所述地图块对应的三维地图小于利用整张深度图对应的随机分布的三维坐标信息生成的三维地图，有利于降低第二存储器的带宽损耗。
其中，所述像素对应的三维坐标信息包括所述像素对应的三维坐标以及所述三维坐标指示的目标点的显示半径；所述像素的深度值与所述显示半径呈正相关关系。
在一实施例中，所述三维地图处理装置还用于从所述第一存储器中读取历史三维地图，然后将不同地图块分别对应的三维地图与所述历史三维地图进行融合，获得融合后的三维地图，并将融合后的三维地图存储至所述第一存储器中；本实施例实现对所述历史三维地图的实时更新。
在一种实现方式中，在无需更新所述历史三维地图的情况下，可以将所述地图块对应的三维地图存储至所述第一存储器中。
其中，所述地图块为按照预设方向对所述历史三维地图进行划分的结果确定的。所述三维地图处理装置预先按照预设方向对所述历史三维地图进行划分，确定多个地图块，所述地图块表示三维坐标信息指示的目标点所在的预设范围的空间区域，或者说，所述地图块指示某个范围内的三维坐标。可以理解的是，本实施例对于所述预设方向以及所述地图块的数量不做任何限制，可依据实际应用场景进行具体设置。
在一实施例中，考虑到将属于不同地图块的三维坐标信息存储至所述第一存储器中的不同存储位置时，需要基于不同地图块确定在所述第一存储器中的不同存储空间，由于在开始处理前不知道属于某个地图块的三维坐标信息的数据量有多少，因此只能假设所有的三维坐标信息所指示的目标点均可能落到同一个地图块内，因此需要在第一存储器中确定与所述地图块的数量相应的空间，假设N个空间，N为大于1的整数，每个空间的大小必须满足存储所有三维坐标信息，但所有空间的平均利用率只有1/N，造成了第一存储器存储空间的浪费。
因此，所述三维地图处理装置在存储所述深度图像对应的三维坐标信息时，首先将属于同一地图块的三维坐标信息封装成数据包，并存储至所述第一存储器中，其中，属于不同地图块的数据包在所述第一存储器中可以连续存储；然后获取指示所述数据包存储位置的索引信息，并将所述索引信息存储至第一存储器中所述地图块指向的存储位置，属于不同地图块的数据包的索引信息存储在所述第一存储器中不同地图块分别指向的不同存储位置中。本实施例中，由于封装要大部分数据的数据包在第一存储器中连续存储，而在不同地图块对应的存储空间中只需存储数据量比较小的索引信息，因为无需为不同地图块的存储空间确定太大的容量，从而达到节省第一存储器存储空间的目的，提高存储资源的利用率。
当所述三维地图处理装置从所述第一存储器中读取三维坐标信息时，首先依次从所述第一存储器中其中一个地图块指向的存储位置读取属于该地图块的数据包的索引信息，然后根据所述索引信息从所述第一存储器中读取所述数据包，解封得到所述三维坐标信息，然后使用所述三维坐标信息更新所述历史三维地图，具体而言，所述三维地图处理装置可以根据属于同一地图块的三维坐标信息生成所述地图块对应的三维地图；然后将不同地图块分别对应的三维地图与所述历史三维地图进行融合，获得融合后的三维地图。本实施例实现以地图块为单位获取相应的三维坐标信息生成三维地图来更新所述历史三维地图，实现在减少对第一存储器的大量的随机访问的基础上，进一步提高三维地图处理效率。
相应的，请参阅图9，本申请实施例还提供了一种三维地图处理装置100，包括：
坐标信息生成模块11，用于从第一存储器200中读取多张深度图像，所述深度图像中的任一像素对应有深度值；根据每一张深度图像中的每个像素的深度值，获取所述像素对应的三维坐标信息。
地图更新模块，用于在得到所述多张深度图像中所有像素对应的三维坐标信息后， 从所述第一存储器200中读取历史三维地图；利用所述多张深度图像中所有像素对应的三维坐标信息更新所述历史三维地图。
在一实施例中，所述像素对应的三维坐标信息包括所述像素对应的三维坐标以及所述三维坐标指示的目标点的显示半径。
在一实施例中，所述显示半径基于所述像素的深度值对应的范围所确定；不同范围对应不同的显示半径。
在一实施例中，所述显示半径根据所述像素的深度值以及预设对应关系确定，所述预设对应关系指示不同深度值对应不同的显示半径。
在一实施例中，所述像素的深度值与所述显示半径呈正相关关系。
在一实施例中，请参阅图10，所述装置100还包括分块模块13，所述分块模块13用于按照预设方向对所述历史三维地图进行划分，确定多个地图块。
以及，所述分块模块13还用于：确定所述深度图像中各个像素对应的三维坐标信息所属地图块；将属于不同地图块的三维坐标信息存储至所述第一存储器200中的不同存储位置。
所述地图更新模块还用于：从所述第一存储器200中不同地图块指向的不同存储位置读取所述多张深度图像对应的三维坐标信息，并使用所述三维坐标信息更新所述历史三维地图。
在一实施例中，请参阅图11，所述地图更新模块包括有地图生成子模块121和地图更新子模块122。
所述地图生成子模块121用于：依次从所述第一存储器200中其中一个地图块指向的存储位置读取三维坐标信息，并根据属于同一地图块的三维坐标信息生成所述地图块对应的三维地图。
所述地图更新子模块122用于：从所述第一存储器200中读取历史三维地图；将不同地图块分别对应的三维地图与所述历史三维地图进行融合，获得融合后的三维地图。
在一实施例中，所述分块模块13具体用于：将属于同一地图块的三维坐标信息封装成数据包，并存储至所述第一存储器200中；获取指示所述数据包存储位置的索引信息，并将所述索引信息存储至第一存储器200中所述地图块指向的存储位置。
在一实施例中，属于不同地图块的数据包在所述第一存储器200中连续存储。
在一实施例中，属于不同地图块的数据包的索引信息存储在所述第一存储器200中不同地图块分别指向的不同存储位置中。
在一实施例中，所述地图更新模块具体用于：从所述第一存储器200中不同地图块分别指向的存储位置读取属于不同地图块的数据包的索引信息；根据所述索引信息从所述第一存储器200中读取所述数据包，解封得到所述三维坐标信息。
在一实施例中，所述地图更新模块包括有地图生成子模块121和地图更新子模块122。
所述地图生成子模块121用于：根据属于同一地图块的三维坐标信息生成所述地图块对应的三维地图。
所述地图更新子模块122用于：将不同地图块分别对应的三维地图与所述历史三维地图进行融合，获得融合后的三维地图。
在一实施例中，从所述第一存储器200中读取的属于同一地图块的三维坐标信息缓存在第二存储器中，其中，针对于所述第二存储器的读写速度优于针对于所述第一存储器200的读写速度。
在一实施例中，所述第二存储器的存储容量小于所述第一存储器200的存储容量。
在一实施例中，所述多张深度图像由处于不同位置的多个拍摄装置100对同一目标拍摄得到；和/或，所述多张深度图像由一个拍摄装置100在不同时刻对同一目标拍摄得到。
在一实施例中，请参阅图12，所述地图更新模块包括有地图生成子模块121和地图更新子模块122。
所述地图生成子模块121用于：利用所述多张深度图像中所有像素对应的三维坐标信息生成三维地图。
所述地图更新子模块122用于：从所述第一存储器200中读取历史三维地图；将所述三维地图与所述历史三维地图进行融合，获得融合后的融合后的三维地图。
在一实施例中，所述三维地图、所述历史三维地图以及所述融合后的三维地图以三维栅格地图形式表示。
在一实施例中，所述坐标信息生成模块11还用于将获取的所述多张深度图像的所有像素对应的三维坐标信息存储至所述第一存储器200中。
所述地图生成子模块121还用于从所述第一存储器200读取所述多张深度图像的所有像素对应的三维坐标信息后，利用所述多张深度图像中所有像素对应的三维坐标信息生成三维地图。
在一实施例中，所述地图生成子模块121具体用于：建立所述三维地图，根据所述像素对应的三维坐标信息确定所述三维地图中对应的栅格；将所述像素对应的三维 坐标信息指示的目标点投射到所述三维地图中对应的栅格上，并更新该栅格的属性值；所述属性值表征所述栅格指示的位置存在障碍物的概率。
在一实施例中，所述三维地图处理装置100包括安装于内部的第二存储器，所述三维地图中的每个栅格在所述第二存储器中对应有存储位置，所述存储位置处存储所述栅格的属性值。
在更新该栅格的属性值时，所述地图生成子模块121具体用于：将所述第二存储器中与该栅格对应的存储位置处的属性值叠加预设值。
在一实施例中，请参阅图13，所述地图生成子模块121包括地图读取单元1221和地图融合单元1222。
所述地图读取单元1221用于：从所述第一存储器200中读取历史三维地图；确定所述三维地图以及所述历史三维地图中对应相同三维坐标信息的栅格。
所述地图融合单元1222用于：将对应相同三维坐标信息的栅格的属性值叠加，获得所述融合后的三维地图。
在一实施例中，所述地图融合单元1222还用于：若所述融合后的三维地图中的栅格的属性值大于上限阈值，将所述栅格对应的属性值修改为所述上限阈值；若所述融合后的三维地图中的栅格中的属性值小于下限阈值，将所述栅格对应的属性值修改为下限阈值。
在一实施例中，所述地图融合单元1222还用于：将所述融合后的三维地图存储至所述第一存储器200。
相应的，请参阅图14，本申请实施例还提供了一种三维地图处理装置100，包括：
坐标信息生成模块11，用于从第一存储器200中读取深度图像；所述深度图像中的任一像素对应有深度值；根据所述深度图像中的每个像素的深度值，获取所述像素对应的三维坐标信息。
分块模块13，用于确定所述深度图像中各个像素对应的三维坐标信息所属地图块；将属于不同地图块的三维坐标信息存储至所述第一存储器200中的不同存储位置。
地图生成模块14，用于依次从所述第一存储器200中其中一个地图块指向的存储位置读取三维坐标信息，并根据属于同一地图块的三维坐标信息生成所述地图块对应的三维地图。
在一实施例中，请参阅图15，所述装置100还包括：地图更新模块12，用于从所述第一存储器200中读取历史三维地图；将不同地图块分别对应的三维地图与所述历 史三维地图进行融合，获得融合后的三维地图。
在一实施例中，所述地图块为按照预设方向对所述历史三维地图进行划分的结果确定的。
在一实施例中，所述分块模块13具体用于：将属于同一地图块的三维坐标信息封装成数据包，并存储至所述第一存储器200中；获取指示所述数据包存储位置的索引信息，并将所述索引信息存储至第一存储器200中所述地图块指向的存储位置。
在一实施例中，属于不同地图块的数据包在所述第一存储器200中连续存储。
在一实施例中，属于不同地图块的数据包的索引信息存储在所述第一存储器200中不同地图块分别指向的不同存储位置中。
在一实施例中，所述地图生成模块14具体用于：依次从所述第一存储器200中其中一个地图块指向的存储位置读取属于该地图块的数据包的索引信息；根据所述索引信息从所述第一存储器200中读取所述数据包，解封得到所述三维坐标信息。
在一实施例中，从所述第一存储器200中读取的属于同一地图块的三维坐标信息缓存在第二存储器中，其中，针对于所述第二存储器的访问速度优于针对于所述第一存储器200的访问速度。
在一实施例中，所述第二存储器的存储容量小于所述第一存储器200的存储容量。
对于装置实施例而言，由于其基本对应于方法实施例，所以相关之处参见方法实施例的部分说明即可。以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。本领域普通技术人员在不付出创造性劳动的情况下，即可以理解并实施。
需要说明的是，在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
在示例性实施例中，还提供了一种包括指令的非临时性计算机可读存储介质，例 如包括指令的存储器，上述指令可由装置的处理器执行以完成上述方法。例如，非临时性计算机可读存储介质可以是ROM、随机存取存储器(RAM)、CD-ROM、磁带、软盘和光数据存储设备等。
一种非临时性计算机可读存储介质，当存储介质中的指令由终端的处理器执行时，使得终端能够执行上述方法。
相应地，本申请实施例还提供了一种可移动平台，包括上述的三维地图处理装置。
在一实施例中，所述可移动平台包括以下至少一项：无人飞行器、无人驾驶车辆、移动机器人、无人驾驶船只或者云台。
在一实施例中，所述可移动平台还安装有一个或多个拍摄装置，所述拍摄装置用于拍摄深度图像。
以上对本申请实施例所提供的方法和装置进行了详细介绍，本文中应用了具体个例对本申请的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本申请的方法及其核心思想；同时，对于本领域的一般技术人员，依据本申请的思想，在具体实施方式及应用范围上均会有改变之处，综上所述，本说明书内容不应理解为对本申请的限制。

Claims (68)

  1. 一种三维地图处理方法，其特征在于，包括：
    从第一存储器中读取多张深度图像；所述深度图像中的任一像素对应有深度值；
    根据每一张深度图像中的每个像素的深度值，获取所述像素对应的三维坐标信息；
    在得到所述多张深度图像中所有像素对应的三维坐标信息后，从所述第一存储器中读取历史三维地图；
    利用所述多张深度图像中所有像素对应的三维坐标信息更新所述历史三维地图。
  2. 根据权利要求1所述的方法，其特征在于，所述像素对应的三维坐标信息包括所述像素对应的三维坐标以及所述三维坐标指示的目标点的显示半径。
  3. 根据权利要求2所述的方法，其特征在于，所述显示半径基于所述像素的深度值对应的范围所确定；不同范围对应不同的显示半径。
  4. 根据权利要求2所述的方法，其特征在于，所述显示半径根据所述像素的深度值以及预设对应关系确定，所述预设对应关系指示不同深度值对应不同的显示半径。
  5. 根据权利要求2至4任意一项所述的方法，其特征在于，所述像素的深度值与所述显示半径呈正相关关系。
  6. 根据权利要求1至5任意一项所述的方法，其特征在于，还包括：按照预设方向对所述历史三维地图进行划分，确定多个地图块；
    在所述根据每一张深度图像中的每个像素的深度值，获取所述像素对应的三维坐标信息之后，还包括：
    确定所述深度图像中各个像素对应的三维坐标信息所属地图块；
    将属于不同地图块的三维坐标信息存储至所述第一存储器中的不同存储位置；
    所述利用所述多张深度图像中所有像素对应的三维坐标信息更新所述历史三维地图包括：
    从所述第一存储器中不同地图块指向的不同存储位置读取所述多张深度图像对应的三维坐标信息，并使用所述三维坐标信息更新所述历史三维地图。
  7. 根据权利要求6所述的方法，其特征在于，所述从所述第一存储器中不同地图块指向的不同存储位置读取所述多张深度图像对应的三维坐标信息，并使用所述三维坐标信息更新所述历史三维地图，包括：
    依次从所述第一存储器中其中一个地图块指向的存储位置读取三维坐标信息，并根据属于同一地图块的三维坐标信息生成所述地图块对应的三维地图；
    将不同地图块分别对应的三维地图与所述历史三维地图进行融合，获得融合后的 三维地图。
  8. 根据权利要求6所述的方法，其特征在于，所述将属于不同地图块的三维坐标信息存储至所述第一存储器中的不同存储位置，包括：
    将属于同一地图块的三维坐标信息封装成数据包，并存储至所述第一存储器中；
    获取指示所述数据包存储位置的索引信息，并将所述索引信息存储至第一存储器中所述地图块指向的存储位置。
  9. 根据权利要求8所述的方法，其特征在于，属于不同地图块的数据包在所述第一存储器中连续存储。
  10. 根据权利要求8所述的方法，其特征在于，属于不同地图块的数据包的索引信息存储在所述第一存储器中不同地图块分别指向的不同存储位置中。
  11. 根据权利要求8所述的方法，其特征在于，所述从所述第一存储器中的不同存储位置读取所述多张深度图像对应的所述三维坐标信息，包括：
    从所述第一存储器中不同地图块分别指向的存储位置读取属于不同地图块的数据包的索引信息；
    根据所述索引信息从所述第一存储器中读取所述数据包，解封得到所述三维坐标信息。
  12. 根据权利要求11所述的方法，其特征在于，所述使用所述三维坐标信息更新所述历史三维地图，还包括：
    根据属于同一地图块的三维坐标信息生成所述地图块对应的三维地图；
    将不同地图块分别对应的三维地图与所述历史三维地图进行融合，获得融合后的三维地图。
  13. 根据权利要求7或11所述的方法，其特征在于，从所述第一存储器中读取的属于同一地图块的三维坐标信息缓存在第二存储器中，其中，针对于所述第二存储器的读写速度优于针对于所述第一存储器的读写速度。
  14. 根据权利要求13所述的方法，其特征在于，所述第二存储器的存储容量小于所述第一存储器的存储容量。
  15. 根据权利要求1所述的方法，其特征在于，所述多张深度图像由处于不同位置的多个拍摄装置对同一目标拍摄得到；和/或，
    所述多张深度图像由一个拍摄装置在不同时刻对同一目标拍摄得到；和/或，所述多张深度图像中所有像素对应的三维坐标信息缓存在第二存储器中。
  16. 根据权利要求1所述的方法，其特征在于，所述利用所述多张深度图像中所 有像素对应的三维坐标信息更新所述历史三维地图，包括：
    利用所述多张深度图像中所有像素对应的三维坐标信息生成三维地图；
    将所述三维地图与所述历史三维地图进行融合，获得融合后的三维地图。
  17. 根据权利要求16所述的方法，其特征在于，所述三维地图、所述历史三维地图以及所述融合后的三维地图均以三维栅格地图形式表示。
  18. 根据权利要求17所述的方法，其特征在于，所述利用所述多张深度图像中所有像素对应的三维坐标信息生成三维地图，包括：
    建立所述三维地图，根据所述像素对应的三维坐标信息确定所述三维地图中对应的栅格；
    将所述像素对应的三维坐标信息指示的目标点投射到所述三维地图中对应的栅格上，并更新该栅格的属性值；所述属性值表征所述栅格指示的位置存在障碍物的概率。
  19. 根据权利要求18所述的方法，其特征在于，所述三维地图中的每个栅格在第二存储器中对应有存储位置，所述存储位置处存储所述栅格的属性值；
    所述更新该栅格的属性值，包括：
    将所述第二存储器中与该栅格对应的存储位置处的属性值叠加预设值。
  20. 根据权利要求17所述的方法，其特征在于，所述将所述三维地图与所述历史三维地图进行融合，获得融合后的新三维地图，包括：
    确定所述三维地图以及所述历史三维地图中对应相同三维坐标信息的栅格；
    将对应相同三维坐标信息的栅格的属性值叠加，获得所述融合后的三维地图。
  21. 根据权利要求20所述的方法，其特征在于，所述获得所述新三维地图还包括：
    若所述融合后的三维地图中的栅格的属性值大于上限阈值，将所述栅格对应的属性值修改为所述上限阈值；
    若所述融合后的三维地图中的栅格中的属性值小于下限阈值，将所述栅格对应的属性值修改为所述下限阈值。
  22. 根据权利要求13、14或19所述的方法，其特征在于，所述方法应用于三维地图处理装置上，所述第一存储器设置于所述三维地图处理装置外部，所述第二存储器设置于所述三维地图处理装置内部。
  23. 一种三维地图处理方法，其特征在于，包括：
    从第一存储器中读取深度图像；所述深度图像中的任一像素对应有深度值；
    根据所述深度图像中的每个像素的深度值，获取所述像素对应的三维坐标信息；
    确定所述深度图像中各个像素对应的三维坐标信息所属地图块；
    将属于不同地图块的三维坐标信息存储至所述第一存储器中的不同存储位置；
    依次从所述第一存储器中其中一个地图块指向的存储位置读取三维坐标信息，并根据属于同一地图块的三维坐标信息生成所述地图块对应的三维地图。
  24. 根据权利要求23所述的方法，其特征在于，还包括：
    从所述第一存储器中读取历史三维地图；
    将不同地图块分别对应的三维地图与所述历史三维地图进行融合，获得融合后的三维地图。
  25. 根据权利要求24所述的方法，其特征在于，所述地图块为按照预设方向对所述历史三维地图进行划分的结果确定的。
  26. 根据权利要求23所述的方法，其特征在于，所述将属于不同地图块的三维坐标信息存储至所述第一存储器中的不同存储位置，包括：
    将属于同一地图块的三维坐标信息封装成数据包，并存储至所述第一存储器中；
    获取指示所述数据包存储位置的索引信息，并将所述索引信息存储至第一存储器中所述地图块指向的存储位置。
  27. 根据权利要求26所述的方法，其特征在于，属于不同地图块的数据包在所述第一存储器中连续存储。
  28. 根据权利要求26所述的方法，其特征在于，属于不同地图块的数据包的索引信息存储在所述第一存储器中不同地图块分别指向的不同存储位置中。
  29. 根据权利要求26所述的方法，其特征在于，所述依次从所述第一存储器中其中一个地图块指向的存储位置读取三维坐标信息，包括：
    依次从所述第一存储器中其中一个地图块指向的存储位置读取属于该地图块的数据包的索引信息；
    根据所述索引信息从所述第一存储器中读取所述数据包，解封得到所述三维坐标信息。
  30. 根据权利要求23所述的方法，其特征在于，从所述第一存储器中读取的属于同一地图块的三维坐标信息缓存在第二存储器中，其中，针对于所述第二存储器的访问速度优于针对于所述第一存储器的访问速度。
  31. 根据权利要求30所述的方法，其特征在于，所述第二存储器的存储容量小于所述第一存储器的存储容量。
  32. 根据权利要求30所述的方法，其特征在于，所述方法应用于三维地图处理装 置上，所述第一存储器设置于所述三维地图处理装置外部，所述第二存储器设置于所述三维地图处理装置内部。
  33. 一种三维地图处理装置，其特征在于，包括：
    坐标信息生成模块，用于从第一存储器中读取多张深度图像，所述深度图像中的任一像素对应有深度值；根据每一张深度图像中的每个像素的深度值，获取所述像素对应的三维坐标信息；
    地图更新模块，用于在得到所述多张深度图像中所有像素对应的三维坐标信息后，从所述第一存储器中读取历史三维地图；利用所述多张深度图像中所有像素对应的三维坐标信息更新所述历史三维地图。
  34. 根据权利要求33所述的装置，其特征在于，所述像素对应的三维坐标信息包括所述像素对应的三维坐标以及所述三维坐标指示的目标点的显示半径。
  35. 根据权利要求34所述的装置，其特征在于，所述显示半径基于所述像素的深度值对应的范围所确定；不同范围对应不同的显示半径。
  36. 根据权利要求34所述的装置，其特征在于，所述显示半径根据所述像素的深度值以及预设对应关系确定，所述预设对应关系指示不同深度值对应不同的显示半径。
  37. 根据权利要求34至36任意一项所述的装置，其特征在于，所述像素的深度值与所述显示半径呈正相关关系。
  38. 根据权利要求33至37任意一项所述的装置，其特征在于，还包括分块模块，所述分块模块用于按照预设方向对所述历史三维地图进行划分，确定多个地图块；
    以及，所述分块模块还用于：确定所述深度图像中各个像素对应的三维坐标信息所属地图块；将属于不同地图块的三维坐标信息存储至所述第一存储器中的不同存储位置；
    所述地图更新模块还用于:从所述第一存储器中不同地图块指向的不同存储位置读取所述多张深度图像对应的三维坐标信息，并使用所述三维坐标信息更新所述历史三维地图。
  39. 根据权利要求38所述的装置，其特征在于，所述地图更新模块包括有地图生成子模块和地图更新子模块；
    所述地图生成子模块用于：依次从所述第一存储器中其中一个地图块指向的存储位置读取三维坐标信息，并根据属于同一地图块的三维坐标信息生成所述地图块对应的三维地图；
    所述地图更新子模块用于：从所述第一存储器中读取历史三维地图；将不同地图块分别对应的三维地图与所述历史三维地图进行融合，获得融合后的三维地图。
  40. 根据权利要求38所述的装置，其特征在于，所述分块模块具体用于：将属于同一地图块的三维坐标信息封装成数据包，并存储至所述第一存储器中；获取指示所述数据包存储位置的索引信息，并将所述索引信息存储至第一存储器中所述地图块指向的存储位置。
  41. 根据权利要求40所述的装置，其特征在于，属于不同地图块的数据包在所述第一存储器中连续存储。
  42. 根据权利要求40所述的装置，其特征在于，属于不同地图块的数据包的索引信息存储在所述第一存储器中不同地图块分别指向的不同存储位置中。
  43. 根据权利要求40所述的装置，其特征在于，所述地图更新模块具体用于：从所述第一存储器中不同地图块分别指向的存储位置读取属于不同地图块的数据包的索引信息；根据所述索引信息从所述第一存储器中读取所述数据包，解封得到所述三维坐标信息。
  44. 根据权利要求43所述的装置，其特征在于，所述地图更新模块包括有地图生成子模块和地图更新子模块；
    所述地图生成子模块用于：根据属于同一地图块的三维坐标信息生成所述地图块对应的三维地图；
    所述地图更新子模块用于：将不同地图块分别对应的三维地图与所述历史三维地图进行融合，获得融合后的三维地图。
  45. 根据权利要求39或44所述的装置，其特征在于，从所述第一存储器中读取的属于同一地图块的三维坐标信息缓存在第二存储器中，其中，针对于所述第二存储器的读写速度优于针对于所述第一存储器的读写速度。
  46. 根据权利要求45所述的装置，其特征在于，所述第二存储器的存储容量小于所述第一存储器的存储容量。
  47. 根据权利要求33所述的装置，其特征在于，所述多张深度图像由处于不同位置的多个拍摄装置对同一目标拍摄得到；和/或，
    所述多张深度图像由一个拍摄装置在不同时刻对同一目标拍摄得到。
  48. 根据权利要求33所述的装置，其特征在于，所述地图更新模块包括有地图生成子模块和地图更新子模块；
    所述地图生成子模块用于：利用所述多张深度图像中所有像素对应的三维坐标信 息生成三维地图；
    所述地图更新子模块用于：从所述第一存储器中读取历史三维地图；将所述三维地图与所述历史三维地图进行融合，获得融合后的融合后的三维地图。
  49. 根据权利要求48所述的装置，其特征在于，所述三维地图、所述历史三维地图以及所述融合后的三维地图以三维栅格地图形式表示。
  50. 根据权利要求49所述的装置，其特征在于，
    所述坐标信息生成模块还用于将获取的所述多张深度图像的所有像素对应的三维坐标信息存储至所述第一存储器中；
    所述地图生成子模块还用于从所述第一存储器读取所述多张深度图像的所有像素对应的三维坐标信息后，利用所述多张深度图像中所有像素对应的三维坐标信息生成三维地图。
  51. 根据权利要求49所述的装置，其特征在于，
    所述地图生成子模块具体用于：建立所述三维地图，根据所述像素对应的三维坐标信息确定所述三维地图中对应的栅格；将所述像素对应的三维坐标信息指示的目标点投射到所述三维地图中对应的栅格上，并更新该栅格的属性值；所述属性值表征所述栅格指示的位置存在障碍物的概率。
  52. 根据权利要求51所述的装置，其特征在于，所述装置包括安装于内部的第二存储器，所述三维地图中的每个栅格在所述第二存储器中对应有存储位置，所述存储位置处存储所述栅格的属性值；
    在更新该栅格的属性值时，所述地图生成子模块具体用于：将所述第二存储器中与该栅格对应的存储位置处的属性值叠加预设值。
  53. 根据权利要求49所述的装置，其特征在于，所述地图生成子模块包括地图读取单元和地图融合单元；
    所述地图读取单元用于：从所述第一存储器中读取历史三维地图；确定所述三维地图以及所述历史三维地图中对应相同三维坐标信息的栅格；
    所述地图融合单元用于：将对应相同三维坐标信息的栅格的属性值叠加，获得所述融合后的三维地图。
  54. 根据权利要求53所述的装置，其特征在于，所述地图融合单元还用于：若所述融合后的三维地图中的栅格的属性值大于上限阈值，将所述栅格对应的属性值修改为所述上限阈值；若所述融合后的三维地图中的栅格中的属性值小于下限阈值，将所述栅格对应的属性值修改为下限阈值。
  55. 根据权利要求54所述的装置，其特征在于，所述地图融合单元还用于：将所述融合后的三维地图存储至所述第一存储器。
  56. 一种三维地图处理装置，其特征在于，包括：
    坐标信息生成模块，用于从第一存储器中读取深度图像；所述深度图像中的任一像素对应有深度值；根据所述深度图像中的每个像素的深度值，获取所述像素对应的三维坐标信息；
    分块模块，用于确定所述深度图像中各个像素对应的三维坐标信息所属地图块；将属于不同地图块的三维坐标信息存储至所述第一存储器中的不同存储位置；
    地图生成模块，用于依次从所述第一存储器中其中一个地图块指向的存储位置读取三维坐标信息，并根据属于同一地图块的三维坐标信息生成所述地图块对应的三维地图。
  57. 根据权利要求56所述的装置，其特征在于，还包括：
    地图更新模块，用于从所述第一存储器中读取历史三维地图；将不同地图块分别对应的三维地图与所述历史三维地图进行融合，获得融合后的三维地图。
  58. 根据权利要求57所述的装置，其特征在于，所述地图块为按照预设方向对所述历史三维地图进行划分的结果确定的。
  59. 根据权利要求56所述的装置，其特征在于，所述分块模块具体用于：将属于同一地图块的三维坐标信息封装成数据包，并存储至所述第一存储器中；获取指示所述数据包存储位置的索引信息，并将所述索引信息存储至第一存储器中所述地图块指向的存储位置。
  60. 根据权利要求59所述的装置，其特征在于，属于不同地图块的数据包在所述第一存储器中连续存储。
  61. 根据权利要求59所述的装置，其特征在于，属于不同地图块的数据包的索引信息存储在所述第一存储器中不同地图块分别指向的不同存储位置中。
  62. 根据权利要求59所述的装置，其特征在于，所述地图生成模块具体用于：依次从所述第一存储器中其中一个地图块指向的存储位置读取属于该地图块的数据包的索引信息；根据所述索引信息从所述第一存储器中读取所述数据包，解封得到所述三维坐标信息。
  63. 根据权利要求56所述的装置，其特征在于，从所述第一存储器中读取的属于同一地图块的三维坐标信息缓存在第二存储器中，其中，针对于所述第二存储器的访 问速度优于针对于所述第一存储器的访问速度。
  64. 根据权利要求63所述的装置，其特征在于，所述第二存储器的存储容量小于所述第一存储器的存储容量。
  65. 一种可移动平台，其特征在于，包括如权利要求33至64任意一项所述的三维地图处理装置。
  66. 根据权利要求65所述的可移动平台，其特征在于，所述可移动平台包括以下至少一项：无人飞行器、无人驾驶车辆、移动机器人、无人驾驶船只或者云台。
  67. 根据权利要求65所述的可移动平台，其特征在于，所述可移动平台还安装有一个或多个拍摄装置，所述拍摄装置用于拍摄深度图像。
  68. 一种计算机可读存储介质，其特征在于，其上存储有计算机程序，当所述计算机程序被三维地图处理装置执行时用于实现权利要求1至32任意一项所述的方法。
PCT/CN2020/112700 2020-08-31 2020-08-31 三维地图处理方法、装置、可移动平台及存储介质 WO2022041257A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/112700 WO2022041257A1 (zh) 2020-08-31 2020-08-31 三维地图处理方法、装置、可移动平台及存储介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/112700 WO2022041257A1 (zh) 2020-08-31 2020-08-31 三维地图处理方法、装置、可移动平台及存储介质

Publications (1)

Publication Number Publication Date
WO2022041257A1 true WO2022041257A1 (zh) 2022-03-03

Family

ID=80354388

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/112700 WO2022041257A1 (zh) 2020-08-31 2020-08-31 三维地图处理方法、装置、可移动平台及存储介质

Country Status (1)

Country Link
WO (1) WO2022041257A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114723894A (zh) * 2022-04-28 2022-07-08 北京百度网讯科技有限公司 一种三维坐标获取方法、装置及电子设备
CN115406434A (zh) * 2022-08-17 2022-11-29 南京领鹊科技有限公司 室内机器人的导航地图自主更新方法、设备和存储介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005337863A (ja) * 2004-05-26 2005-12-08 Fuji Photo Film Co Ltd 地図更新システム
CN105571588A (zh) * 2016-03-10 2016-05-11 赛度科技（北京）有限责任公司 一种无人机三维空中航路地图构建及其航路显示方法
CN105678754A (zh) * 2015-12-31 2016-06-15 西北工业大学 一种无人机实时地图重建方法
CN106688017A (zh) * 2016-11-28 2017-05-17 深圳市大疆创新科技有限公司 生成点云地图的方法、计算机系统和装置
US10366508B1 (en) * 2016-08-29 2019-07-30 Perceptin Shenzhen Limited Visual-inertial positional awareness for autonomous and non-autonomous device
CN110799804A (zh) * 2017-06-30 2020-02-14 深圳市大疆创新科技有限公司 地图生成系统和方法
CN111174799A (zh) * 2019-12-24 2020-05-19 Oppo广东移动通信有限公司 地图构建方法及装置、计算机可读介质、终端设备

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005337863A (ja) * 2004-05-26 2005-12-08 Fuji Photo Film Co Ltd 地図更新システム
CN105678754A (zh) * 2015-12-31 2016-06-15 西北工业大学 一种无人机实时地图重建方法
CN105571588A (zh) * 2016-03-10 2016-05-11 赛度科技（北京）有限责任公司 一种无人机三维空中航路地图构建及其航路显示方法
US10366508B1 (en) * 2016-08-29 2019-07-30 Perceptin Shenzhen Limited Visual-inertial positional awareness for autonomous and non-autonomous device
CN106688017A (zh) * 2016-11-28 2017-05-17 深圳市大疆创新科技有限公司 生成点云地图的方法、计算机系统和装置
CN110799804A (zh) * 2017-06-30 2020-02-14 深圳市大疆创新科技有限公司 地图生成系统和方法
CN111174799A (zh) * 2019-12-24 2020-05-19 Oppo广东移动通信有限公司 地图构建方法及装置、计算机可读介质、终端设备

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114723894A (zh) * 2022-04-28 2022-07-08 北京百度网讯科技有限公司 一种三维坐标获取方法、装置及电子设备
CN114723894B (zh) * 2022-04-28 2023-10-13 北京百度网讯科技有限公司 一种三维坐标获取方法、装置及电子设备
CN115406434A (zh) * 2022-08-17 2022-11-29 南京领鹊科技有限公司 室内机器人的导航地图自主更新方法、设备和存储介质
CN115406434B (zh) * 2022-08-17 2024-05-07 南京领鹊科技有限公司 室内机器人的导航地图自主更新方法、设备和存储介质

Similar Documents

Publication Publication Date Title
WO2021227359A1 (zh) 一种无人机投影方法、装置、设备及存储介质
EP3664443B1 (en) Panoramic image generation method and device
WO2020119684A1 (zh) 一种3d导航语义地图更新方法、装置及设备
JP2022509302A (ja) 地図生成方法、運転制御方法、装置、電子機器及びシステム
WO2022000260A1 (zh) 地图的更新方法、装置、可移动平台及存储介质
WO2020043081A1 (zh) 定位技术
WO2022041257A1 (zh) 三维地图处理方法、装置、可移动平台及存储介质
JP2020507853A (ja) ３次元点群の再構成のための方法および装置
CN110927708A (zh) 智能路侧单元的标定方法、装置及设备
US20200349727A1 (en) Obstacle detection method, mobile platform, and computer readable storage medium
US11798181B2 (en) Method and system for location detection of photographs using topographic techniques
CN111930877B (zh) 地图路牌的生成方法及电子设备
CN113496503B (zh) 点云数据的生成及实时显示方法、装置、设备及介质
CN113378605B (zh) 多源信息融合方法及装置、电子设备和存储介质
WO2022199195A1 (zh) 地图更新方法、系统、车载终端、服务器及存储介质
CN115457354A (zh) 融合方法、3d目标检测方法、车载设备及存储介质
CN114662587A (zh) 一种基于激光雷达的三维目标感知方法、装置及系统
CN113269168B (zh) 障碍物数据处理方法、装置、电子设备和计算机可读介质
US11353579B2 (en) Method for indicating obstacle by smart roadside unit
WO2021104308A1 (zh) 全景深度测量方法、四目鱼眼相机及双目鱼眼相机
CN112634372A (zh) 一种实时的双目相机矫正方法、装置及存储介质
CN110827340B (zh) 地图的更新方法、装置及存储介质
CN115346386B (zh) 交通灯数据的生成方法和装置
CN117635663B (zh) 一种目标车辆视频跟踪方法及电子设备
WO2022188554A1 (zh) 控制运动体移动的方法以及相关设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20950926

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20950926

Country of ref document: EP

Kind code of ref document: A1