GB2618278A

GB2618278A - Dense 3-D occupancy mapping

Info

Publication number: GB2618278A
Application number: GB2312319.3A
Authority: GB
Inventors: J Tarrio Juan; F Alcantarilla Pablo
Original assignee: Slamcore Ltd
Current assignee: Slamcore Ltd
Priority date: 2021-01-29
Filing date: 2022-01-27
Publication date: 2023-11-01
Also published as: GB202312319D0; WO2022162075A1; GB2603179A; GB202101281D0

Abstract

A computer-implemented method is provided, for building and updating a 3-D occupancy map of occupied and unoccupied spaces in a 3-D environment. A plurality of overlapping sub-maps are defined. Each sub-map describes a different volumetric region of the 3-D environment. Two or more of the sub-maps are updated based on a 2-D depth image and associated pose estimate.

Claims

1. A computer-implemented method for building and updating a 3-D occupancy map of occupied and unoccupied spaces in a 3-D environment, the method comprising: defining (210) a plurality of sub-maps (10, 11 , 12, 13, 14) of the 3-D occupancy map, each sub-map describing a different volumetric region of the 3-D environment, each sub-map having associated with it a sub-map pose defining a position and orientation of the respective volumetric region within the 3-D occupancy map, wherein the volumetric regions overlap with one another; obtaining (220) a set of frames characterising the environment, each frame comprising a 2-D depth image and a pose estimate describing a position and orientation at which the depth image was captured; and updating (300) two or more of the sub-maps based on one of the 2-D depth images and its associated pose estimate.

2. The method of claim 1 , further comprising: obtaining (230) updated pose information; and in response, updating (240) one or more of the sub-map poses, based on the received updated pose information.

3. The method of claim 2, wherein receiving the updated pose information comprises obtaining (230) updated pose estimates for one or more of the frames, and the method comprises updating (240) one or more of the sub-map poses based on the updated pose estimates.

4. The method of any one of claims 1-3, wherein each sub-map comprises an octree for storing occupancy information, each octree comprising a hierarchy of nodes, each node being associated with a voxel, wherein each leaf node of the octree includes an occupancy indication.

5. The method of claim 2 or claim 3, wherein each sub-map comprises an octree for storing occupancy information, each octree comprising hierarchy of nodes, each node being associated with a voxel, wherein each leaf node of the octree contains an occupancy indication indicating whether the respective voxel is occupied or unoccupied, 43 and wherein updating the at least one sub-map based on a given 2-D depth image comprises: projecting (310) each voxel of the sub-map into the 2-D depth image, to define a projected area within the 2-D depth image; and updating (400) each voxel based on depth values of the 2-D depth image that are located within the respective projected area.

6. The method of claim 5, wherein updating (400) each voxel comprises: defining (410) a first interval of depths encompassed by the voxel; defining (420) a second interval of depth values in the depth image that are located within the voxelâ s projected area; calculating (430), based on the first interval and the second interval, an expected variation in a value of the occupancy indication within the voxel and if (435) the expected variation is greater than a threshold, sub-dividing (446) the voxel into eight sub-voxels.

7. The method of claim 5 or claim 6, wherein updating at least one voxel comprises updating (456) the occupancy indication of the voxel, based on the depth values that are located within the voxelâ s projected area, wherein the occupancy indication is updated using an inverse sensor model, which determines the probability of occupation along a ray, given a queried point on the ray and a depth value in the depth image.

8. The method of any one of claims 5-7, further comprising, before updating (400) each voxel, pre-computing (260) a pooling structure based on the 2-D depth image, wherein the pooling structure comprises one of: a pyramidal min-max pooling structure, describing the minimum and maximum depth values for each of a plurality of regions in the depth image, the regions including regions at different resolutions, the regions at each scale being arranged in a regular array; a mean-variance integral image pooling structure, comprising integral images computed for the zeroth, first and second moments of depth; and a smooth-max integral image pooling structure, comprising integral images computed for positive and negative alpha-softmax functions of depth. 44

9. The method of any one of claims 5-8, wherein updating the at least one submap comprises pruning (460) a group of eight child nodes from the octree if their occupancy can be indicated sufficiently accurately by a single occupancy indication associated with the parent node of the group of eight child nodes.

10. The method of any one of claims 2-3 or 5-9, further comprising: before updating (300) the at least one sub-map, fusing (250) two or more of the 2-D depth images into a combined depth image, and updating (300) the at least one sub-map based on the combined depth image.

11 . The method of claim 10, wherein the fusing comprises: re-projecting (252) depth values from one or more of the 2-D depth images, based on the pose estimates associated with the frames, to produce one or more reprojected depth images; and calculating (256) average depth values based at least in part on the re-projected depth images.

12. The method of claim 13, wherein calculating the average depth values comprises, for each pixel of the combined depth image: computing (254) a histogram of depth values, based on the re-projected depth images; and calculating (256) the average depth value based at least in part on the mode of the histogram.

13. The method of any one of the preceding claims, further comprising: maintaining a list of currently active sub-maps; and defining (210) a new sub-map if occupancy information derived from the 2-D depth image extends outside all of the currently active sub-maps.

14. The method of any one of the preceding claims, further comprising: identifying (510), among the plurality of sub-maps, a first sub-map and a second sub-map, wherein the second sub-map overlaps spatially with the first sub-map; merging (520) the second sub-map into the first sub-map; and discarding (530) the second sub-map.

15. The method of claim 14, further comprising: identifying (522) neighbouring sub-maps of the first sub-map; and before discarding the second sub-map, merging (524) the second sub-map into the neighbouring sub-maps.

16. A computer program comprising computer program code configured to cause one or more physical computing devices to perform all the steps of the method of any one of the preceding claims when said computer program is run on the one or more physical computing devices.

17. A mobile device configured to implement the method of any one of claims 1-15.

18. A mobile device (100) configured to build and update a 3-D occupancy map of occupied and unoccupied spaces in a 3-D environment, the mobile device comprising: a memory (110), for storing the 3-D occupancy map; and one or more processors (140), configured to: define (210) a plurality of sub-maps of the 3-D occupancy map, each sub-map describing a different volumetric region of the 3-D environment, each sub-map having associated with it a sub-map pose defining a position and orientation of the respective volumetric region within the 3-D occupancy map, wherein the volumetric regions overlap with one another; wherein the mobile device further comprises: at least one camera (120), configured to capture (222) visual data, comprising a set of frames characterising the environment; and an inertial measurement unit (134), configured to generate (224) inertial data, wherein the inertial measurement unit comprises at least one or any combination of two or more of: an accelerometer (132), a gyroscope (134) and a compass, and wherein the one or more processors (140) are configured to: calculate (226) a pose estimate of the mobile device for each frame, based on the visual data and the inertial data; produce (228) a 2-D depth image for each frame; and update (300) two or more of the sub-maps based on one of the 2-D depth images and its associated pose estimate.

19. The mobile device of claim 18, wherein the one or more processors are further configured to: obtain (230) updated pose information; and in response, update (240) one or more of the sub-map poses, based on the updated pose information.

20 The mobile device of claim 18 or claim 19, wherein the at least one camera comprises a stereo camera pair (120) configured to capture stereo image pairs, and the one or more processors are configured to produce (228) the 2-D depth images based on the stereo image pairs.

21. The mobile device of any one of claims 18 to 20, wherein the one or more processors (140) are configured to: calculate (230) an updated pose estimate for each frame; and update (240) one or more of the sub-map poses based on the updated pose estimates.

22. The mobile device of any one of claims 18-21, wherein the mobile device is comprised in a handheld device, a robot, or a vehicle.