WO2024011557A1 - 地图构建方法、装置及存储介质 - Google Patents

地图构建方法、装置及存储介质 Download PDF

Info

Publication number
WO2024011557A1
WO2024011557A1 PCT/CN2022/105886 CN2022105886W WO2024011557A1 WO 2024011557 A1 WO2024011557 A1 WO 2024011557A1 CN 2022105886 W CN2022105886 W CN 2022105886W WO 2024011557 A1 WO2024011557 A1 WO 2024011557A1
Authority
WO
WIPO (PCT)
Prior art keywords
probability
voxel
map
pixel
distance
Prior art date
Application number
PCT/CN2022/105886
Other languages
English (en)
French (fr)
Inventor
王雷
陈熙
Original Assignee
深圳市正浩创新科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市正浩创新科技股份有限公司 filed Critical 深圳市正浩创新科技股份有限公司
Priority to PCT/CN2022/105886 priority Critical patent/WO2024011557A1/zh
Priority to CN202280004114.6A priority patent/CN115917607A/zh
Publication of WO2024011557A1 publication Critical patent/WO2024011557A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/20—Instruments for performing navigational calculations
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects

Definitions

  • the present application relates to the technical field of map construction, and in particular to a map construction method, device and storage medium.
  • autonomous mobile devices can use three-dimensional probabilistic maps for obstacle avoidance to ensure normal movement and mobile safety of autonomous mobile devices.
  • the construction of general three-dimensional probability maps is mainly based on Bayesian filtering. Due to sensor noise and the complexity of the environment, there are perception errors, and the Bayesian filtering method cannot adapt to the dynamic changes of the environment, making the constructed three-dimensional probability map less accurate, resulting in the problem of using the three-dimensional probability map for mobile devices.
  • the Bayesian filtering method cannot adapt to the dynamic changes of the environment, making the constructed three-dimensional probability map less accurate, resulting in the problem of using the three-dimensional probability map for mobile devices.
  • the Bayesian filtering method cannot adapt to the dynamic changes of the environment, making the constructed three-dimensional probability map less accurate, resulting in the problem of using the three-dimensional probability map for mobile devices.
  • the Bayesian filtering method cannot adapt to the dynamic changes of the environment, making the constructed three-dimensional probability map less accurate, resulting in the problem of using the three-dimensional probability map for mobile devices.
  • a map construction method, device and storage medium are provided.
  • the embodiment of this application provides a map construction method, including:
  • the image data is collected by the image acquisition device on the mobile device.
  • the obstacle information includes the distance from each pixel belonging to the obstacle to the image acquisition device, The position information of each pixel and the first probability that each pixel belongs to an obstacle;
  • the voxel corresponding to each pixel point is determined, and a three-dimensional probability map of the current moment is generated according to the voxel, wherein the three-dimensional probability map of the current moment is The probability value carried by each voxel in the map is the second probability;
  • the probability value of each voxel in the three-dimensional probability map at the current moment is multiplied by the probability value of the corresponding voxel in the updated three-dimensional probability map at the previous moment, and the current moment is updated according to the multiplication result.
  • the updated three-dimensional probability map at the current moment is used as the target three-dimensional probability map at the current moment.
  • Embodiments of the present application also provide a map construction device, which includes a processor, a memory, a computer program stored on the memory and executable by the processor, and a computer program for implementing the processor and the A data bus is used for connection and communication between the memories, and when the computer program is executed by the processor, any map construction method as provided in the specification of this application is implemented.
  • Embodiments of the present application also provide a storage medium for computer-readable storage.
  • the storage medium stores one or more programs.
  • the one or more programs can be executed by one or more processors to implement the following: Any map construction method provided in this application specification.
  • Figure 1 is a schematic flowchart of a map construction method provided by an embodiment of the present application.
  • Figure 2 is a schematic flowchart of a sub-step of the map construction method in Figure 1;
  • Figure 3 is a schematic flowchart of a sub-step of the map construction method in Figure 1;
  • Figure 4 is a schematic flowchart of a sub-step of the map construction method in Figure 1;
  • Figure 5 is a schematic diagram of a scene of obstacle avoidance from a mobile device in an embodiment of the present application.
  • Figure 6 is a schematic flowchart of a sub-step of the map construction method in Figure 1;
  • Figure 7 is a schematic structural block diagram of a map construction device provided by an embodiment of the present application.
  • Embodiments of the present application provide a map construction method, device and storage medium.
  • the map construction method can be applied to self-mobile devices, including sweepers, lawn mowers, food delivery machines, etc.
  • the map construction method can also be applied to a server or terminal device.
  • the server can be an independent server or a server cluster composed of multiple servers. It can also provide cloud services, cloud databases, cloud computing, cloud functions, Cloud servers for basic cloud computing services such as cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.
  • Terminal devices can be remote control devices, mobile phones, tablets, laptops, desktop computers, personal digital assistants, wearable devices, etc.
  • Figure 1 is a schematic flowchart of a map construction method provided by an embodiment of the present application.
  • the map construction method includes steps S101 to S106.
  • Step S101 Obtain obstacle information in the image data at the current moment.
  • the image data is collected by the image acquisition device on the mobile device, and the image data at the current moment is the image data collected by the image acquisition device on the mobile device at the current moment.
  • the screening method can use the RANSAC algorithm (Random Sample Consensus, random sample consistency), plane model segmentation algorithm (Plane model segmentation), Euclidean cluster extraction (Euclidean Cluster Extraction), color-based region growing segmentation (Color-based region growing segmentation) and Conditional Euclidean Clustering (conditional Euclidean group clustering generation), etc., are not limited here.
  • the obstacle information includes the distance from each pixel belonging to the obstacle to the image acquisition device, the position information of each pixel and the first probability that each pixel belongs to the obstacle.
  • the image acquisition device may include an RGB camera and/or a depth camera.
  • the first probability of a pixel in the obstacle information represents the possibility that the pixel belongs to the obstacle, for example, when the first probability is less than the set threshold, it means that the possibility of the pixel belonging to the obstacle is low. . That is to say, there are some pixels in the obstacle information that may not actually be obstacles. That is, the first probability in the obstacle information is only the initial judgment that the pixels belong to obstacles. The process of further judging the pixels as obstacles is still Steps S102 to S106 need to be combined.
  • step S101 includes: sub-steps S1011 to sub-step S1012.
  • Sub-step S1011 when the image data is an RGB image, semantic recognition is performed on each pixel in the RGB image to obtain a semantic recognition result;
  • Sub-step S1012 Obtain pixels whose semantic labels are obstacle labels, and use the obtained semantic label probability of the pixel as the first probability.
  • the semantic recognition result includes the semantic label and the semantic label probability of each pixel in the RGB image.
  • the semantic label describes the type of the pixel
  • the semantic label probability describes the probability that the pixel belongs to the type corresponding to the semantic label.
  • Semantic labels include obstacle labels and non-obstacle labels. Obstacle labels are used to describe the type of pixels as obstacles, and non-obstacle labels are used to describe the type of pixels as non-obstacles.
  • the RGB image is input into a preset semantic segmentation model for processing to obtain a semantic segmentation image corresponding to the RGB image.
  • the semantic segmentation image includes the semantic label and semantic label probability of each pixel in the RGB image.
  • the preset semantic segmentation model is a pre-trained neural network model.
  • the neural network model includes but is not limited to a convolutional neural network (Convolutional Neural Network). Neural Network, CNN), fully convolutional neural network (Fully Convolutional Networks, FCN), deep convolutional neural network (Dynamic Convolution Neural Network, DCNN).
  • step S101 includes: sub-steps S1013 to sub-step S1014.
  • the image data is a depth image
  • convert the depth image to obtain point cloud data
  • Sub-step S1014 Extract the point cloud belonging to the obstacle and the point cloud probability of each point in the point cloud from the point cloud data, and use the point cloud probability as the first probability.
  • the cloud and the point cloud probability of each point in the point cloud can determine all pixels corresponding to the point cloud belonging to the obstacle as pixels belonging to the obstacle.
  • the point cloud probability of each point in the point cloud is used as the first probability that the corresponding pixel point belongs to the obstacle.
  • the point cloud probability represents the probability that the point in the point cloud belonging to the obstacle belongs to the obstacle.
  • the position information of each pixel in the depth image in the camera coordinate system is obtained, as well as the distance between each pixel in the depth image and the image acquisition device; according to each pixel in the depth image The position information of the point in the camera coordinate system, and the distance between each pixel in the depth image and the image acquisition device, determine the three-dimensional position information of each pixel in the depth image in the camera coordinate system; obtain the image acquisition The internal parameter matrix and external parameter matrix of the device are used, and the three-dimensional position information of each pixel in the depth image in the camera coordinate system is converted into the three-dimensional position information in the world coordinate system based on the internal parameter matrix and the external parameter matrix, and the depth image correspondence is obtained. point cloud data.
  • Step S102 Obtain the second probability that each pixel belongs to an obstacle based on the first probability and distance of each pixel.
  • the second probability that the pixel point belongs to the obstacle can be determined more accurately, so as to improve the probability that the pixel point belongs to the obstacle.
  • the accuracy of the probability by comprehensively considering the first probability that the pixel point belongs to the obstacle and the distance from the pixel point to the image acquisition device, the second probability that the pixel point belongs to the obstacle can be determined more accurately, so as to improve the probability that the pixel point belongs to the obstacle. The accuracy of the probability.
  • the first probability of the pixel may include a semantic label probability of the pixel and/or a point cloud probability. For example, multiply the distance probability of the pixel by the semantic label probability to obtain the second probability that the pixel belongs to the obstacle; or multiply the distance probability of the pixel by the point cloud probability to obtain the second probability that the pixel belongs to the obstacle. Probability; or, multiply the distance probability of the pixel, the semantic label probability and the point cloud probability to obtain the second probability that the pixel belongs to an obstacle.
  • step S102 includes: sub-steps S1021 to sub-step S1022.
  • Sub-step S1021 Determine the distance probability of each pixel based on the distance from each pixel to the image acquisition device;
  • Sub-step S1022 Multiply the first probability and the distance probability to obtain the second probability that each pixel belongs to an obstacle.
  • the distance probability of a pixel represents the probability that the corresponding pixel belongs to the obstacle during the movement of the mobile device, that is, the change of the distance from the collection device on the mobile device to the obstacle.
  • the pixel The distance probability of a point is inversely proportional to the distance from the pixel to the image acquisition device. That is, the farther the distance between the pixel point and the image acquisition device is, the smaller the distance probability of the pixel point is, and the closer the distance between the pixel point and the image acquisition device is, the greater the distance probability of the pixel point is.
  • the distance probability of a pixel is inversely proportional to the distance from the pixel to the image acquisition device, when the distance probability is used to generate a three-dimensional probability map, the weight of the obstacle point cloud in the three-dimensional probability map can increase as the distance increases. Small. In this way, the self-moving device can observe changes in obstacles in the three-dimensional probability map, and can accurately avoid obstacles, reducing the risk of collision between the self-moving device and obstacles.
  • the self-mobile device 11 moves from the location point 21 to the location point 22 , but the self-mobile device 11 needs to turn to move to the location point 22 .
  • the gray rectangle shown in Figure 5 is the self-mobile device 11 when it collides with the obstacle 12. .
  • the solution of this application is based on the distance probability obtained from the distance between the image acquisition device and the obstacle, combined with the first probability of the pixel belonging to the obstacle, to further determine the possibility that the pixel belongs to the obstacle, that is, it reflects the relative distance between the obstacle 12 and the obstacle 12 . due to changes in the location of the mobile device 11.
  • Use distance probability to generate a three-dimensional probability map so that the weight of the obstacle point cloud in the three-dimensional probability map can increase with the closer the distance.
  • the mobile device can use the three-dimensional probability map obtained previously to determine the obstacles when turning.
  • the position of the object 12 is determined so that the mobile device 11 can avoid the obstacle 12 .
  • the reciprocal of the distance from the pixel point to the image acquisition device is determined as the distance probability of the pixel point.
  • a preset inverse proportional constant can also be obtained, and the preset inverse proportional constant is divided by the distance from the pixel point to the image acquisition device to obtain the distance probability of the pixel point.
  • the preset inverse proportional constant can be set based on actual conditions, and this is not specifically limited in the embodiments of the present application. For example, assuming that the distance from the pixel to the image acquisition device is d, and the preset inverse proportional constant is k, then the distance probability of the pixel can be expressed as k/d.
  • step S102 includes: sub-steps S1023 to sub-step S1025.
  • Sub-step S1023 Determine the distance probability of each pixel according to the distance from each pixel to the image acquisition device;
  • Sub-step S1024 Obtain the semantic weight coefficient and the point cloud weight coefficient
  • Sub-step S1025 Determine the second probability based on the semantic label probability, point cloud probability, distance probability, semantic weight coefficient and point cloud weight coefficient of the pixel.
  • the semantic weight coefficient is used to describe the accuracy of the semantic label probability of the pixel.
  • the larger the semantic weight coefficient the higher the accuracy of the semantic label probability of the pixel.
  • the smaller the semantic weight coefficient the higher the accuracy of the semantic label probability of the pixel.
  • the point cloud weight coefficient is used to describe the accuracy of the point cloud probability of a pixel.
  • the larger the point cloud weight coefficient the higher the accuracy of the point cloud probability of the pixel.
  • the smaller the point cloud weight coefficient the higher the accuracy of the point cloud probability of the pixel.
  • the accuracy of the point cloud probability is lower.
  • the image quality index of the RGB image is obtained, and the image quality index of the RGB image is determined as the first confidence level of the semantic label probability of the pixel; wherein, the image quality index of the RGB image is used to describe the Image quality; determine the semantic weight coefficient according to the first confidence level; obtain the image quality index of the depth image, and determine the image quality index of the depth image as the second confidence level of the point cloud probability of the pixel; where, the image of the depth image
  • the quality index is used to describe the image quality of the depth image; based on the second confidence level, the point cloud weight coefficient is determined.
  • the first confidence level is positively correlated with the semantic weight coefficient
  • the second confidence level is positively correlated with the point cloud weight coefficient.
  • the methods of obtaining the image quality index may include but are not limited to Full Reference Image Quality Assessment (FR-IQA), semi-reference image quality assessment (Reduced Reference Image Quality Assessment, RR-IQA) and no-reference image Quality assessment (No Reference Image Quality Assessment, NR-IQA) and other algorithms are not limited here.
  • FR-IQA Full Reference Image Quality Assessment
  • RR-IQA semi-reference image quality assessment
  • NR-IQA no-reference image Quality assessment
  • other algorithms are not limited here.
  • the first mapping relationship table and the second mapping relationship table are obtained. Query the first mapping relationship table to obtain the semantic weight coefficient corresponding to the first confidence level, and query the second mapping relationship table to obtain the point cloud weight coefficient corresponding to the second confidence level.
  • the first mapping relationship table includes a mapping relationship between confidence and semantic weight coefficients
  • the second mapping relationship table includes a mapping relationship between confidence and point cloud weight coefficients.
  • the semantic weight coefficient and the point cloud weight coefficient are negatively correlated, that is, the larger the semantic weight coefficient is, the smaller the point cloud weight coefficient is, and the smaller the semantic weight coefficient is, the larger the point cloud weight coefficient is.
  • the semantic weight coefficient is increased and the point cloud weight coefficient is decreased.
  • the semantic weight coefficient is lowered and the point cloud weight coefficient is raised.
  • the semantic tag probability, the distance probability and the semantic weight coefficient are multiplied together to obtain the first multiplied result; the point cloud probability, the distance probability and the point cloud weight coefficient are multiplied together to obtain the multiplied result.
  • the second result add the first result and the second result to obtain the second probability.
  • P the point cloud probability
  • the point cloud weight coefficient is k 1
  • the distance probability is P 2
  • the semantic label probability is P 3
  • the semantic weight coefficient is k 2
  • P is the second probability
  • Step S103 Determine the voxel corresponding to each pixel point based on the second probability and position information of each pixel point, and generate a three-dimensional probability map at the current moment based on the voxel point.
  • the three-dimensional probability map is a three-dimensional grid map.
  • the probability value carried by each voxel is the second probability of the corresponding pixel point. Based on the position information of the pixel point, it can be The corresponding voxel of the pixel point in the three-dimensional grid map is determined, and then the second probability value of the pixel point is configured as the probability value carried by the voxel.
  • the position information of the pixel point includes the three-dimensional position information of the pixel point in camera coordinates.
  • the method of determining the voxel corresponding to the pixel point may be: obtaining the image acquisition device The internal parameter matrix and the external parameter matrix, and according to the internal parameter matrix and the external parameter matrix, the three-dimensional position information of the pixel point in the camera coordinates is converted into the three-dimensional position information in the world coordinate system; according to the converted three-dimensional position information, and based on the eight
  • the three-dimensional map creation tool of the cross tree generates the voxels corresponding to the pixel points, that is, the three-dimensional probability map of this application is a three-dimensional grid map composed of voxels.
  • the position of the voxel in the three-dimensional probability map is represented by the voxel grid index, which is calculated based on the length, width and resolution ratio of the three-dimensional probability map.
  • the probability value of the voxel is configured as the second probability of the pixel point, and the voxel grid stores the second probability of the corresponding pixel point.
  • Step S104 Obtain the three-dimensional probability map at the previous moment, and update the probability value of each voxel in the three-dimensional probability map at the previous moment according to the preset adjustment parameters.
  • the three-dimensional probability map at the previous moment is constructed in the same way as the three-dimensional probability map at the current moment.
  • the preset adjustment parameters can be set by the user based on the actual situation. This is not specifically limited in the embodiment of the present application.
  • the method of updating the probability value of each voxel in the three-dimensional probability map at the previous moment according to the preset adjustment parameter may be: comparing the probability value of each voxel in the three-dimensional probability map at the previous moment with the preset adjustment parameter.
  • the residence time of each voxel in the three-dimensional probability map of the previous moment is obtained, and the residence time represents the duration from the generation moment of the voxel to the current moment; the residence time of each voxel in the three-dimensional probability map of the previous moment is obtained.
  • the probability value is multiplied by the preset adjustment parameter; the multiplied probability value of each voxel is divided by the corresponding residence time of each voxel, and the result of the division is used as the updated probability value of each voxel. .
  • the preset adjustment parameter is A
  • the probability value of a voxel B in the three-dimensional probability map at the previous moment is p 1
  • the generation time of the voxel is t 1
  • the current time is t 2 , so after voxel B is updated
  • Step S105 Multiply the probability value of each voxel in the three-dimensional probability map at the current moment by the probability value of the corresponding voxel in the updated three-dimensional probability map at the previous moment, and update the three-dimensional probability value at the current moment according to the multiplication result.
  • the probability value of each voxel in the probability map is
  • the three-dimensional probability map at the current moment includes voxel C 1 , voxel C 2 , voxel C 3 , voxel C 4 , voxel C 5 and voxel C 6 , and voxel C 1 , voxel C 2
  • the probability values of voxel C 3 , voxel C 4 , voxel C 5 and voxel C 6 are respectively
  • the updated three-dimensional probability map at the last moment includes voxel D 1 , voxel D 2 , voxel D 3 , voxel D 4 , voxel D 5 and voxel D 6 , and voxel D 1 , voxel D 2.
  • the probability values of voxel D 3 , voxel D 4 , voxel D 5 and voxel D 6 are respectively and Since voxel C 1 corresponds to voxel D 1 , voxel C 2 corresponds to voxel D 2 , voxel C 3 corresponds to voxel D 3, voxel C 4 corresponds to voxel D 4 , and voxel C 5 corresponds to Voxel D 5 corresponds to voxel C 6 to voxel D 6 , then in the updated three-dimensional probability map at the current moment, voxel C 1 , voxel C 2 , voxel C 3 , voxel voxel C 4 , The probability values of voxel C 5 and voxel C 6 are respectively and
  • the probability value of the target voxel is multiplied by the preset probability value, and the three-dimensional probability map at the current moment is updated according to the multiplication result.
  • the target voxel and the voxel in the updated three-dimensional probability map at the previous moment are voxels that do not correspond to each other.
  • the preset probability value can be set based on actual conditions, which is not specifically limited in the embodiments of this application. For example, the preset probability value is 1 or 0.85.
  • the three-dimensional probability map at the current moment includes voxel C 1 , voxel C 2 , voxel C 3 , voxel C 4 , voxel C 5 and voxel C 6 , and the updated The three-dimensional probability map at the last moment includes voxel D 1 , voxel D 2 , voxel D 3 and voxel D 4 , voxel C 1 corresponds to voxel D 1 , voxel C 2 corresponds to voxel D 2 Correspondingly, voxel C 3 corresponds to voxel D 3 , and voxel C 4 corresponds to voxel D 4 .
  • the distance between each voxel in the three-dimensional probability map at the current moment and the self-mobile device is obtained; when the distance between any voxel and the self-mobile device is greater than a preset distance threshold, the distance is greater than the preset distance.
  • the voxels corresponding to the distance threshold are deleted from the three-dimensional probability map at the current moment to obtain the updated three-dimensional probability map at the current moment; the probability value of each voxel in the updated three-dimensional probability map at the current moment is compared with the updated The probability values of the corresponding voxels in the three-dimensional probability map at the previous moment are multiplied, and the probability value of each voxel in the three-dimensional probability map at the current moment is updated based on the multiplication result.
  • the size of the three-dimensional probability map can be reduced to prevent the three-dimensional probability map from being too large and wasting storage space, thereby improving storage space utilization.
  • the distance between each voxel in the three-dimensional probability map at the current moment and the mobile device can be obtained by converting the distance from the pixel point corresponding to each voxel in the three-dimensional probability map at the current moment to the image acquisition device.
  • the preset distance threshold can be set by the user according to actual conditions, and this is not specifically limited in the embodiments of this application.
  • Step S106 Use the updated three-dimensional probability map at the current time as the target three-dimensional probability map at the current time.
  • the three-dimensional probability map can be established and updated in the same way, so that the three-dimensional probability map can be It changes with the movement of the mobile device, ensuring that the mobile device can know the changes of obstacles based on the changing three-dimensional probability map, so that it can safely avoid obstacles.
  • Embodiments of the present invention provide a map construction method, device and storage medium, by determining the first probability that each pixel belonging to an obstacle belongs to an obstacle and the distance from each pixel belonging to an obstacle to an image acquisition device.
  • the second probability of each pixel belonging to the obstacle Since the second probability is related to the distance from the pixel to the image acquisition device, the second probability of each pixel belonging to the obstacle and each pixel belonging to the obstacle are used.
  • the position information of the pixel points can generate a three-dimensional probability map at the current moment that can represent the change in the position of the obstacle, and after updating the probability value of each voxel in the three-dimensional probability map at the previous moment, it can be compared with the three-dimensional probability map at the current moment.
  • the probability value of each voxel is multiplied, so that the three-dimensional probability map at the current moment can be updated according to the multiplication result, so that the updated three-dimensional probability map can more accurately describe the changes in obstacles, so that the mobile device can avoid obstacles At this time, the changes in obstacles can be observed using a three-dimensional probability map, so that obstacles can be accurately avoided and the risk of collision between the mobile device and obstacles is reduced.
  • FIG. 7 is a schematic structural block diagram of a map construction device provided by an embodiment of the present application.
  • the map construction device 200 includes a processor 201 and a memory 202.
  • the processor 201 and the memory 202 are connected through a bus 203, such as an I2C (Inter-integrated Circuit) bus.
  • I2C Inter-integrated Circuit
  • the processor 201 is used to provide computing and control capabilities to support the operation of the entire map construction device.
  • the processor 201 can be a central processing unit (Central Processing Unit, CPU).
  • the processor 201 can also be other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC). ), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • the general processor may be a microprocessor or the processor may be any conventional processor.
  • the memory 202 may be a Flash chip, a read-only memory (ROM, Read-Only Memory) disk, an optical disk, a U disk or a mobile hard disk, etc.
  • ROM Read-Only Memory
  • the memory 202 may be a Flash chip, a read-only memory (ROM, Read-Only Memory) disk, an optical disk, a U disk or a mobile hard disk, etc.
  • FIG. 7 is only a block diagram of a partial structure related to the embodiment of the present application, and does not constitute a limitation on the map construction device to which the embodiment of the present application is applied.
  • Particular map construction devices may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements.
  • the processor 201 is used to run a computer program stored in the memory 202, and implement any of the map construction methods provided by the embodiments of the present application when executing the computer program.
  • the processor 201 is configured to run a computer program stored in the memory 202, and implement any of the map construction methods provided by the embodiments of this application when executing the computer program.
  • Embodiments of the present application also provide a storage medium for computer-readable storage.
  • the storage medium stores one or more programs.
  • the one or more programs can be executed by one or more processors to implement the following: Any map construction method provided in the embodiments of this application.
  • the storage medium may be an internal storage unit of the map construction device described in the previous embodiment, such as a hard disk or memory of the map construction device.
  • the storage medium may also be an external storage device of the map construction device, such as a plug-in hard drive, a Smart Media Card (SMC), or a Secure Digital (SD) equipped on the map construction device. Card, Flash Card, etc.
  • SMC Smart Media Card
  • SD Secure Digital
  • Such software may be distributed on computer-readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media).
  • computer storage media includes volatile and nonvolatile media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. removable, removable and non-removable media.
  • Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disk (DVD) or other optical disk storage, magnetic cassettes, tapes, disk storage or other magnetic storage devices, or may Any other medium used to store the desired information and that can be accessed by a computer.
  • communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media .

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Automation & Control Theory (AREA)
  • Image Analysis (AREA)

Abstract

一种地图构建方法，包括：根据每个像素点的第一概率和距离，得到每个像素点属于障碍物的第二概率，根据第二概率和位置信息，确定每个像素点对应的体素，根据体素生成当前时刻的三维概率地图，根据预设调整参数更新上一时刻的三维概率地图中每个体素的概率值，将当前时刻的三维概率地图中每个体素的概率值与更新后的上一时刻的三维概率地图中所对应体素的概率值进行相乘，并根据相乘结果更新当前时刻三维概率地图每个体素的概率值。

Description

地图构建方法、装置及存储介质 技术领域
本申请涉及地图构建的技术领域，尤其涉及一种地图构建方法、装置及存储介质。
背景技术
这里的陈述仅提供与本申请有关的背景信息，而不必然地构成示例性技术。
目前，自移动设备可以使用三维概率地图进行避障，以保证自移动设备的正常移动和移动安全性。然而，一般的三维概率地图的构建主要是贝叶斯滤波的方式。由于传感器噪声和环境的复杂性，存在感知误差，而贝叶斯滤波的方式无法适应环境的动态变化，使得所构建的三维概率地图的准确性较低，导致自移动设备在使用三维概率地图进行避障时，存在与障碍物碰撞的风险。因此，如何提高三维概率地图的准确性，以降低自移动设备与障碍物碰撞的风险是目前亟待解决的问题。
发明内容
根据本申请的各种实施例，提供了一种地图构建方法、装置及存储介质。
本申请实施例提供一种地图构建方法，包括：
获取当前时刻的图像数据中的障碍物信息，所述图像数据由自移动设备上的图像采集装置采集，所述障碍物信息包括属于障碍物的每个像素点到所述图像采集装置的距离、每个像素点的位置信息以及每个所述像素点属于障碍物的第一概率；
根据每个所述像素点的所述第一概率和所述距离，得到每个所述像素点属于障碍物的第二概率；
根据每个所述像素点的第二概率和位置信息，确定每个所述像素点对应的体素，并根据所述体素生成当前时刻的三维概率地图，其中，所述当前时刻的三维概率地图中每个体素携带的概率值为所述第二概率；
获取上一时刻的三维概率地图，并根据预设调整参数更新上一时刻的三维概率地图中每个体素的概率值；
将所述当前时刻的三维概率地图中每个体素的概率值与更新后的上一时刻的三维概率地图中所对应体素的概率值进行相乘，并根据相乘结果更新所述当前时刻的三维概率地图中每个体素的概率值；
将更新后的当前时刻的三维概率地图作为当前时刻的目标三维概率地图。
本申请实施例还提供一种地图构建装置，所述地图构建装置包括处理器、存储器、存储在所述存储器上并可被所述处理器执行的计算机程序以及用于实现所述处理器和所述存储器之间的连接通信的数据总线，其中所述计算机程序被所述处理器执行时，实现如本申请说明书提供的任一项地图构建方法。
本申请实施例还提供一种存储介质，用于计算机可读存储，所述存储介质存储有一个或者多个程序，所述一个或者多个程序可被一个或者多个处理器执行，以实现如本申请说明书提供的任一项地图构建方法。
本申请的一个或多个实施例的细节在下面的附图和描述中提出。本申请的其他特征、目的和优点将从说明书、附图以及权利要求书变得明显。
附图说明
为了更清楚地说明本申请实施例技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本申请实施例提供的一种地图构建方法的流程示意图；
图2是图1中的地图构建方法的一子步骤流程示意图；
图3是图1中的地图构建方法的一子步骤流程示意图；
图4是图1中的地图构建方法的一子步骤流程示意图；
图5是本申请实施例中的自移动设备避障的一场景示意图；
图6是图1中的地图构建方法的一子步骤流程示意图；
图7是本申请实施例提供的一种地图构建装置的结构示意框图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
附图中所示的流程图仅是示例说明，不是必须包括所有的内容和操作/步骤，也不是必须按所描述的顺序执行。例如，有的操作/步骤还可以分解、组合或部分合并，因此实际执行的顺序有可能根据实际情况改变。
应当理解，在此本申请说明书中所使用的术语仅仅是出于描述特定实施例的目的而并不意在限制本申请。如在本申请说明书和所附权利要求书中所使用的那样，除非上下文清楚地指明其它情况，否则单数形式的“一”、“一个”及“该”意在包括复数形式。
本申请实施例提供一种地图构建方法、装置及存储介质。其中，该地图构 建方法可应用于自移动设备中，自移动设备包括扫地机、割草机、送餐机等。该地图构建方法还可以应用于服务器或终端设备中，该服务器可以是独立的服务器，也可以是由多个服务器组成的服务器集群，也可以是提供云服务、云数据库、云计算、云函数、云存储、网络服务、云通信、中间件服务、域名服务、安全服务、内容分发网络(Content Delivery Network，CDN)、以及大数据和人工智能平台等基础云计算服务的云服务器。终端设备可以是遥控设备、手机、平板电脑、笔记本电脑、台式电脑、个人数字助理和穿戴式设备等。
下面结合附图，对本申请的一些实施例作详细说明。在不冲突的情况下，下述的实施例及实施例中的特征可以相互组合。
请参阅图1，图1是本申请实施例提供的一种地图构建方法的流程示意图。
如图1所示，该地图构建方法包括步骤S101至步骤S106。
步骤S101、获取当前时刻的图像数据中的障碍物信息。
本申请实施例中，图像数据由自移动设备上的图像采集装置采集得到，当前时刻的图像数据为由自移动设备上的图像采集装置在当前时刻采集到的图像数据。
从图像数据中筛选出可能属于障碍物的像素点，筛选方式可以采用RANSAC算法(Random Sample Consensus，随机样本一致性)、平面模型分割算法(Plane model segmentation)、欧几里德聚类提取(Euclidean Cluster Extraction)、基于彩色信息区域蔓延分割(Color-based region growing segmentation)和Conditional Euclidean Clustering(有条件的欧几里德群聚类生成)等，此处不做限定。
获取属于障碍物的像素点的障碍物信息，障碍物信息包括属于障碍物的每个像素点到图像采集装置的距离、每个像素点的位置信息以及每个像素点属于障碍物的第一概率。其中，图像采集装置可以包括RGB相机和/或深度相机。
需要说明的是，由于障碍物信息中的像素点的第一概率代表了像素点属于障碍物的可能性，例如当第一概率小于设定阈值时，说明像素点属于障碍物的可能性较低。也就是说障碍物信息中存在部分实际可能不属于障碍物的像素点，即障碍物信息中第一概率仅仅是对像素点属于障碍物的初次判断，对像素点进一步判断为障碍物的过程还需结合步骤S102-步骤S106。
在一实施例中，如图2所示，步骤S101包括：子步骤S1011至子步骤S1012。
子步骤S1011，当图像数据为RGB图像时，对RGB图像中的每个像素点进行语义识别，得到语义识别结果；
子步骤S1012、获取语义标签为障碍物标签的像素点，并将获取到的像素点的语义标签概率作为第一概率。
本申请实施例中，所述语义识别结果包括RGB图像中的每个像素点的语义标签以及语义标签概率，语义标签描述像素点的类型，语义标签概率描述像素 点属于语义标签对应类型的概率。语义标签包括障碍物标签和非障碍物标签，障碍物标签用于描述像素点的类型为障碍物，非障碍物标签用于描述像素点的类型为非障碍物。
在一实施例中，当图像数据为RGB图像时，将RGB图像输入预设的语义分割模型进行处理，得到RGB图像对应的语义分割图像。其中，语义分割图像包括RGB图像中的每个像素点的语义标签以及语义标签概率，预设的语义分割模型为预先训练好的神经网络模型，神经网络模型包括但不限于卷积神经网络(Convolutional Neural Network，CNN)、全卷积神经网络(Fully Convolutional Networks，FCN)、深度卷积神经网络(Dynamic Convolution Neural Network，DCNN)。
在一实施例中，如图3所示，步骤S101包括：子步骤S1013至子步骤S1014。
子步骤S1013、当图像数据为深度图像时，对深度图像进行转换，得到点云数据；
子步骤S1014、从点云数据中提取属于障碍物的点云以及点云中的每个点的点云概率，并将点云概率作为第一概率。
本申请实施例中，由于点云数据中的点与深度图像中的像素点一一对应，这样通过点云数据中的点与深度图像中的像素点之间的对应关系，以及障碍物的点云以及点云中的每个点的点云概率，可以将属于障碍物的点云所对应的全部像素点确定为属于障碍物的像素点。并将点云中的每个点的点云概率作为对应像素点属于障碍物的第一概率，点云概率表示属于障碍物的点云中的点属于障碍物的概率。
在一实施例中，获取深度图像中的每个像素点在相机坐标系下的位置信息，以及深度图像中的每个像素点与图像采集装置之间的距离；根据深度图像中的每个像素点在相机坐标系下的位置信息，以及深度图像中的每个像素点与图像采集装置之间的距离，确定深度图像中的每个像素点在相机坐标系下的三维位置信息；获取图像采集装置的内参矩阵和外参矩阵，并根据内参矩阵、外参矩阵将深度图像中的每个像素点在相机坐标系下的三维位置信息转换为世界坐标系下的三维位置信息，得到深度图像对应的点云数据。
步骤S102、根据每个像素点的第一概率和距离，得到每个像素点属于障碍物的第二概率。
本申请实施例中，通过综合考虑像素点属于障碍物的第一概率以及像素点到图像采集装置的距离，能够更加准确地确定像素点属于障碍物的第二概率，以提高像素点属于障碍物的概率的准确性。
在一实施例中，像素点的第一概率可以包括像素点的语义标签概率和/或点云概率。例如，将像素点的距离概率与语义标签概率相乘，得到像素点属于障碍物的第二概率；或者，将像素点的距离概率与点云概率相乘，得到像素点属 于障碍物的第二概率；或者，将像素点的距离概率、语义标签概率与点云概率相乘，得到像素点属于障碍物的第二概率。
在一实施例中，如图4所示，步骤S102包括：子步骤S1021至子步骤S1022。
子步骤S1021、根据每个像素点到图像采集装置的距离，确定每个像素点的距离概率；
子步骤S1022、将第一概率和距离概率进行相乘，得到每个像素点属于障碍物的第二概率。
本申请实施例中，像素点的距离概率表示在自移动设备移动过程中，也就是自移动设备上的采集装置到障碍物的距离变化过程中，所对应的像素点属于障碍物的概率，像素点的距离概率与像素点到图像采集装置的距离成反比例关系。也就是像素点到图像采集装置的距离越远，像素点的距离概率越小，像素点到图像采集装置的距离越近，像素点的距离概率越大。由于像素点的距离概率与像素点到图像采集装置的距离成反比例关系，在使用距离概率来生成三维概率地图时，使得障碍物点云在三维概率地图中的权重能随着距离越远概率越小。这样，自移动设备在三维概率地图中能观测到障碍物的变化，进而能够准确地避开障碍物，降低了自移动设备与障碍物进行碰撞的风险。
例如，如图5所示，自移动设备11从位置点21向位置点22移动，而自移动设备11需要拐弯才能移动到位置点22。而位置点22的周围存在障碍物12，由于自移动设备11存在感知误差，而一般采用的贝叶斯滤波的避障方式无法适应环境的动态变化，使得自移动设备11在拐弯的时候观测不到近处的障碍物12。即自移动设备11无法感知到拐弯处的障碍物12，很容易引起自移动设备11和障碍物12发生碰撞，如图5中所示的灰色长方形为与障碍物12碰撞时的自移动设备11。而本申请方案根据图像采集装置到障碍物的距离所得到的距离概率，并结合属于障碍物的像素点的第一概率，来进一步确定像素点属于障碍物的可能性，即反映障碍物12相对于自移动设备11的位置变化。使用距离概率来生成三维概率地图，使得障碍物点云在三维概率地图中的权重能随着距离越近概率越大，这样自移动设备能够使用前面获取到的三维概率地图来判断拐弯时的障碍物12的位置，从而使得自移动设备11避开障碍物12。
在一实施例中，将像素点到图像采集装置的距离的倒数确定为像素点的距离概率。在其他的实施例中，也可以获取预设反比例常数，并用预设反比例常数除以像素点到图像采集装置的距离，得到像素点的距离概率。其中，预设反比例常数可以基于实际情况进行设置，本申请实施例对此不做具体限定。例如，设像素点到图像采集装置的距离为d，预设反比例常数为k，则像素点的距离概率可以表示为k/d。
在一实施例中，如图6所示，步骤S102包括：子步骤S1023至子步骤S1025。
子步骤S1023、根据每个像素点到图像采集装置的距离，确定每个像素点的 距离概率；
子步骤S1024、获取语义权重系数以及点云权重系数；
子步骤S1025、根据像素点的语义标签概率、点云概率、距离概率、语义权重系数以及点云权重系数，确定第二概率。
本申请实施例中，语义权重系数用于描述像素点的语义标签概率的准确性，语义权重系数越大，则表示像素点的语义标签概率的准确性越高，而语义权重系数越小，则表示像素点的语义标签概率的准确性越低。点云权重系数用于描述像素点的点云概率的准确性，点云权重系数越大，则表示像素点的点云概率的准确性越高，而点云权重系数越小，则表示像素点的点云概率的准确性越低。通过综合考虑像素点的语义标签概率、点云概率、距离概率、语义权重系数以及点云权重系数，可以提高像素点属于障碍物的概率的准确性。
在一实施例中，获取RGB图像的图像质量指数，并将RGB图像的图像质量指数确定为像素点的语义标签概率的第一置信度；其中，RGB图像的图像质量指数用于描述RGB图像的图像质量；根据第一置信度，确定语义权重系数；获取深度图像的图像质量指数，并将深度图像的图像质量指数确定为像素点的点云概率的第二置信度；其中，深度图像的图像质量指数用于描述深度图像的图像质量；根据第二置信度，确定点云权重系数。第一置信度与语义权重系数呈正相关关系，第二置信度与点云权重系数呈正相关关系。
其中，获取图像质量指数的方式可以包括但不限于全参考图像质量评估(Full Reference Image Quality Assessment,FR-IQA)、半参考图像质量评估(Reduced Reference Image Quality Assessment，RR-IQA)和无参考图像质量评估(No Reference Image Quality Assessment，NR-IQA)等算法，此处不做限定。
示例性的，获取第一映射关系表和第二映射关系表。查询第一映射关系表，获取第一置信度对应的语义权重系数，并查询第二映射关系表，获取第二置信度对应的点云权重系数。其中，第一映射关系表包括置信度与语义权重系数之间的映射关系，第二映射关系表包括置信度与点云权重系数之间的映射关系。
在一实施例中，语义权重系数和点云权重系数呈负相关，即语义权重系数越大，则点云权重系数越小，而语义权重系数越小，则点云权重系数越大。示例性的，在像素点的语义标签概率的准确性高，而像素点的点云概率的准确性低的情况下，调高语义权重系数，并调低点云权重系数。在像素点的语义标签概率的准确性低，而像素点的点云概率的准确性高的情况下，调低语义权重系数，并调高点云权重系数。
在一实施例中，将语义标签概率、距离概率和语义权重系数进行相乘，得到相乘后的第一结果；将点云概率、距离概率和点云权重系数进行相乘，得到相乘后的第二结果；将第一结果和第二结果进行相加，得到第二概率。例如，设点云概率为P 1，点云权重系数为k 1，距离概率为P 2，语义标签概率为P 3，语 义权重系数为k 2，P为第二概率，则P＝k 1×P 1×P 2+k 2×P 2×P 3。
步骤S103、根据每个像素点的第二概率和位置信息，确定每个像素点对应的体素，并根据体素生成当前时刻的三维概率地图。
本申请实施例中，三维概率地图为三维栅格地图，在当前时刻的三维概率地图中，每个体素携带的概率值为各自对应的像素点的第二概率，基于像素点的位置信息，可以确定像素点在三维栅格地图中的对应体素，再将该像素点的第二概率值配置为体素携带的概率值。
在一实施例中，像素点的位置信息包括像素点在相机坐标下的三维位置信息，根据像素点的第二概率和位置信息，确定像素点对应的体素的方式可以为：获取图像采集装置的内参矩阵和外参矩阵，并根据内参矩阵和外参矩阵，将像素点在相机坐标下的三维位置信息转换为世界坐标系下的三维位置信息；根据转换得到的三维位置信息，以及基于八叉树的三维地图创建工具，生成该像素点对应的体素，即本申请的三维概率地图维为由体素构成的三维栅格地图。体素在三维概率地图的位置以体素栅格索引表示，体素栅格索引根据三维概率地图的长宽和分辨比率计算得到。将该体素的概率值配置为该像素点的第二概率，体素珊格存放对应像素点的第二概率。
步骤S104、获取上一时刻的三维概率地图，并根据预设调整参数更新上一时刻的三维概率地图中每个体素的概率值。
本申请实施例中，上一时刻的三维概率地图与当前时刻的三维概率地图的构建方式相同，预设调整参数可以由用户基于实际情况进行设置，本申请实施例对此不做具体限定。
在一实施例中，根据预设调整参数更新上一时刻的三维概率地图中每个体素的概率值的方式可以为：将上一时刻的三维概率地图中每个体素的概率值与预设调整参数进行相乘，并将上一时刻的三维概率地图中每个体素的概率值更新为相乘得到的每个体素的概率值。例如，设预设调整参数为A，上一时刻的三维概率地图中的一个体素B的概率值为p 1，则p 2＝A×p 1，因此，将体素B的概率值更新为p 2。
在一实施例中，获取上一时刻的三维概率地图中每个体素的滞留时长，该滞留时长表示体素的生成时刻到当前时刻的时长；将上一时刻的三维概率地图中每个体素的概率值与预设调整参数进行相乘；将相乘得到的每个体素的概率值与对应的每个体素的滞留时长进行相除，将相除得到的结果作为每个体素更新后的概率值。例如，预设调整参数为A，上一时刻的三维概率地图中的一个体素B的概率值为p 1，体素的生成时刻为t 1，当前时刻为t 2，因此体素B更新后的概率值可以表示为p 3＝(p 1×A)/(t 2-t 1)。
步骤S105、将当前时刻的三维概率地图中每个体素的概率值与更新后的上一时刻的三维概率地图中所对应体素的概率值进行相乘，并根据相乘结果更新 当前时刻的三维概率地图中每个体素的概率值。
例如，设当前时刻的三维概率地图包括体素C 1、体素C 2、体素C 3、体素C 4、体素C 5和体素C 6，且体素C 1、体素C 2、体素C 3、体素体素C 4、体素C 5和体素C 6的概率值分别为
Figure PCTCN2022105886-appb-000001
和
Figure PCTCN2022105886-appb-000002
更新后的上一时刻的三维概率地图包括体素D 1、体素D 2、体素D 3、体素D 4、体素D 5和体素D 6，且体素D 1、体素D 2、体素D 3、体素体素D 4、体素D 5和体素D 6的概率值分别为
Figure PCTCN2022105886-appb-000003
和
Figure PCTCN2022105886-appb-000004
由于体素C 1与体素D 1对应，体素C 2与体素D 2对应，体素C 3与体素D 3对应，体素C 4与体素D 4对应，体素C 5与体素D 5对应，体素C 6与体素D 6对应，则更新后的当前时刻的三维概率地图中体素C 1、体素C 2、体素C 3、体素体素C 4、体素C 5和体素C 6的概率值分别为
Figure PCTCN2022105886-appb-000005
Figure PCTCN2022105886-appb-000006
和
Figure PCTCN2022105886-appb-000007
在一实施例中，在当前时刻的三维概率地图中存在目标体素的情况下，将目标体素的概率值与预设概率值进行相乘，并根据相乘结果更新当前时刻的三维概率地图中的目标体素的概率值。其中，目标体素与更新后的上一时刻的三维概率地图中的体素为互不对应的体素。其中，预设概率值可以基于实际情况进行设置，本申请实施例对此不做具体限定。例如，预设概率值为1或者0.85。
例如，设预设概率值为P′，当前时刻的三维概率地图包括体素C 1、体素C 2、体素C 3、体素C 4、体素C 5和体素C 6，而更新后的上一时刻的三维概率地图包括体素D 1、体素D 2、体素D 3和体素D 4，体素C 1与体素D 1对应，体素C 2与体素D 2对应，体素C 3与体素D 3对应，体素C 4与体素D 4对应。而更新后的上一时刻的三维概率地图中没有体素C 5和体素C 6所对应的体素，则更新后的当前时刻的三维概率地图中体素C 1、体素C 2、体素C 3、体素体素C 4、体素C 5和体素C 6的概率值分别为
Figure PCTCN2022105886-appb-000008
和
Figure PCTCN2022105886-appb-000009
在一实施例中，获取当前时刻的三维概率地图中每个体素到自移动设备之间的距离；在任一体素到自移动设备之间的距离大于预设距离阈值时，将该距离大于预设距离阈值所对应的体素从当前时刻的三维概率地图中删除，以得到更新后的当前时刻的三维概率地图；将更新后的当前时刻的三维概率地图中每个体素的概率值与更新后的上一时刻的三维概率地图中所对应体素的概率值进行相乘，并根据相乘结果更新当前时刻的三维概率地图中每个体素的概率值。通过删除当前时刻的三维概率地图中的距离大于预设距离阈值所对应的体素，可以降低三维概率地图的大小，避免三维概率地图太大，浪费存储空间，以提高存储空间的利用率。
其中，可以通过当前时刻的三维概率地图中每个体素所对应的像素点到图像采集装置的距离转换得到当前时刻的三维概率地图中每个体素到自移动设备之间的距离。预设距离阈值可以由用户按照实际情况进行设置，本申请实施例对此不做具体限定。
步骤S106、将更新后的当前时刻的三维概率地图作为当前时刻的目标三维 概率地图。
通过将更新后的当前时刻的三维概率地图作为当前时刻的目标三维概率地图，使得在确定下一时刻的三维概率地图时，可以按照相同的方式建立与更新三维概率地图，使得三维概率地图能够随着自移动设备的移动而发生变化，保证自移动设备可以基于变化的三维概率地图，知晓障碍物的变化，从而能够安全的避开障碍物。
本发明实施例提供一种地图构建方法、装置及存储介质，通过根据属于障碍物的每个像素点属于障碍物的第一概率以及属于障碍物的每个像素点到图像采集装置的距离，确定属于障碍物的每个像素点的第二概率，由于第二概率与像素点到图像采集装置的距离有关，因此在使用属于障碍物的每个像素点的第二概率以及属于障碍物的每个像素点的位置信息，可以生成能够表示障碍物位置变化的当前时刻的三维概率地图，并且在对上一时刻的三维概率地图中每个体素的概率值进行更新后，与当前时刻的三维概率地图中的每个体素的概率值进行相乘，从而可以根据相乘结果更新当前时刻的三维概率地图，使得更新后的三维概率地图能够更加准确地描述障碍物的变化，这样自移动设备在避障时，能够使用三维概率地图观测到障碍物的变化，从而可以准确地进行避障，降低了自移动设备与障碍物进行碰撞的风险。
请参阅图7，图7是本申请实施例提供的一种地图构建装置的结构示意性框图。
如图7所示，地图构建装置200包括处理器201和存储器202，处理器201和存储器202通过总线203连接，该总线比如为I2C(Inter-integrated Circuit)总线。
具体地，处理器201用于提供计算和控制能力，支撑整个地图构建装置的运行。处理器201可以是中央处理单元(Central Processing Unit，CPU)，该处理器201还可以是其他通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现场可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。其中，通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
具体地，存储器202可以是Flash芯片、只读存储器(ROM，Read-Only Memory)磁盘、光盘、U盘或移动硬盘等。
本领域技术人员可以理解，图7中示出的结构，仅仅是与本申请实施例方案相关的部分结构的框图，并不构成对本申请实施例方案所应用于其上的地图构建装置的限定，具体的地图构建装置可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。
其中，所述处理器201用于运行存储在存储器202中的计算机程序，并在 执行所述计算机程序时实现本申请实施例提供的任意一种所述的地图构建方法。
在一实施例中，所述处理器201用于运行存储在存储器202中的计算机程序，并在执行所述计算机程序时实现本申请实施例提供的任意一种所述的地图构建方法。
需要说明的是，所属领域的技术人员可以清楚地了解到，为了描述的方便和简洁，上述描述的地图构建装置的具体工作过程，可以参考前述地图构建方法实施例中的对应过程，在此不再赘述。
本申请实施例还提供一种存储介质，用于计算机可读存储，所述存储介质存储有一个或者多个程序，所述一个或者多个程序可被一个或者多个处理器执行，以实现如本申请实施例说明书提供的任一项地图构建方法。
其中，所述存储介质可以是前述实施例所述的地图构建装置的内部存储单元，例如所述地图构建装置的硬盘或内存。所述存储介质也可以是所述地图构建装置的外部存储设备，例如所述地图构建装置上配备的插接式硬盘，智能存储卡(Smart Media Card，SMC)，安全数字(Secure Digital，SD)卡，闪存卡(Flash Card)等。
本领域普通技术人员可以理解，上文中所公开方法中的全部或某些步骤、系统、装置中的功能模块/单元可以被实施为软件、固件、硬件及其适当的组合。在硬件实施例中，在以上描述中提及的功能模块/单元之间的划分不一定对应于物理组件的划分；例如，一个物理组件可以具有多个功能，或者一个功能或步骤可以由若干物理组件合作执行。某些物理组件或所有物理组件可以被实施为由处理器，如中央处理器、数字信号处理器或微处理器执行的软件，或者被实施为硬件，或者被实施为集成电路，如专用集成电路。这样的软件可以分布在计算机可读介质上，计算机可读介质可以包括计算机存储介质(或非暂时性介质)和通信介质(或暂时性介质)。如本领域普通技术人员公知的，术语计算机存储介质包括在用于存储信息(诸如计算机可读指令、数据结构、程序模块或其他数据)的任何方法或技术中实施的易失性和非易失性、可移除和不可移除介质。计算机存储介质包括但不限于RAM、ROM、EEPROM、闪存或其他存储器技术、CD-ROM、数字多功能盘(DVD)或其他光盘存储、磁盒、磁带、磁盘存储或其他磁存储装置、或者可以用于存储期望的信息并且可以被计算机访问的任何其他的介质。此外，本领域普通技术人员公知的是，通信介质通常包含计算机可读指令、数据结构、程序模块或者诸如载波或其他传输机制之类的调制数据信号中的其他数据，并且可包括任何信息递送介质。
应当理解，在本申请说明书和所附权利要求书中使用的术语“和/或”是指相关联列出的项中的一个或多个的任何组合以及所有可能组合，并且包括这些组合。需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者系统不 仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者系统所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者系统中还存在另外的相同要素。
上述本申请实施例序号仅仅为了描述，不代表实施例的优劣。以上所述，仅为本申请的具体实施例，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到各种等效的修改或替换，这些修改或替换都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以权利要求的保护范围为准。

Claims (11)

  1. 一种地图构建方法，包括：
    获取当前时刻的图像数据中的障碍物信息，所述图像数据由自移动设备上的图像采集装置采集，所述障碍物信息包括属于障碍物的每个像素点到所述图像采集装置的距离、每个像素点的位置信息以及每个所述像素点属于障碍物的第一概率；
    根据每个所述像素点的所述第一概率和所述距离，得到每个所述像素点属于障碍物的第二概率；
    根据每个所述像素点的第二概率和位置信息，确定每个所述像素点对应的体素，并根据所述体素生成当前时刻的三维概率地图，其中，所述当前时刻的三维概率地图中每个体素携带的概率值为所述第二概率；
    获取上一时刻的三维概率地图，并根据预设调整参数更新所述上一时刻的三维概率地图中每个体素的概率值；
    将所述当前时刻的三维概率地图中每个体素的概率值与更新后的上一时刻的三维概率地图中所对应体素的概率值进行相乘，并根据相乘结果更新所述当前时刻的三维概率地图中每个体素的概率值；
    将更新后的当前时刻的三维概率地图作为当前时刻的目标三维概率地图。
  2. 根据权利要求1所述的地图构建方法，其中，所述获取当前时刻的图像数据中的障碍物信息，包括：
    当所述图像数据为RGB图像时，对所述RGB图像中的每个像素点进行语义识别，得到语义识别结果，所述语义识别结果包括每个像素点的语义标签以及语义标签概率，所述语义标签描述像素点的类型，所述语义标签概率描述像素点属于语义标签对应类型的概率；
    获取所述语义标签为障碍物标签的像素点，并将获取到的所述像素点的所述语义标签概率作为所述第一概率，所述障碍物标签用于描述像素点的类型为障碍物。
  3. 根据权利要求1所述的地图构建方法，其中，所述获取当前时刻的图像数据中的障碍物信息，包括：
    当所述图像数据为深度图像时，对所述深度图像进行转换，得到点云数据；
    从所述点云数据中提取属于障碍物的点云以及所述点云中的每个点的点云概率，并将所述点云概率作为所述第一概率，其中，所述点云概率表示点云中的点属于障碍物的概率。
  4. 根据权利要求2或3所述的地图构建方法，其中，所述根据每个所述像素点的所述第一概率和所述距离，得到每个所述像素点属于障碍物的第二概率，包括：
    根据每个所述像素点到所述图像采集装置的距离，确定每个所述像素点的距离概率，其中，所述距离概率表示在距离变化过程中像素点属于障碍物的概率，所述距离概率与所述距离成反比例关系；
    将所述第一概率和所述距离概率进行相乘，得到每个所述像素点属于障碍物的第二概率。
  5. 根据权利要求1所述的地图构建方法，其中，所述图像数据包括RGB图像和深度图像，所述第一概率包括所述RGB图像中每个属于障碍物的像素点的语义标签概率和所述深度图像中每个属于障碍物的像素点的点云概率，所述根据每个所述像素点的所述第一概率和所述距离，得到每个所述像素点属于障碍物的第二概率，包括：
    根据每个所述像素点到所述图像采集装置的距离，确定每个所述像素点的距离概率；
    获取语义权重系数以及点云权重系数；
    根据所述像素点的所述语义标签概率、所述点云概率、所述距离概率、所述语义权重系数以及所述点云权重系数，确定第二概率。
  6. 根据权利要求5所述的地图构建方法，其中，所述获取语义权重系数以及点云权重系数，包括：
    获取RGB图像的图像质量指数，并将RGB图像的图像质量指数确定为像素点的语义标签概率的第一置信度；其中，RGB图像的图像质量指数用于描述RGB图像的图像质量；
    根据第一置信度，确定语义权重系数；
    获取深度图像的图像质量指数，并将深度图像的图像质量指数确定为像素点的点云概率的第二置信度；其中，深度图像的图像质量指数用于描述深度图像的图像质量；
    根据第二置信度，确定点云权重系数。
  7. 根据权利要求5所述的地图构建方法，其中，所述根据所述像素点的所述语义标签概率、所述点云概率、所述距离概率、所述语义权重以及所述点云权重系数，确定第二概率，包括：
    将所述语义标签概率、所述距离概率和所述语义权重系数进行相乘，得到相乘后的第一结果；
    将所述点云概率、所述距离概率和所述点云权重系数进行相乘，得到相乘后的第二结果；
    将所述第一结果和所述第二结果进行相加，得到所述第二概率。
  8. 根据权利要求1-7中任一项所述的地图构建方法，其中，在将所述当前时刻的三维概率地图中每个体素的概率值与更新后的上一时刻的三维概率地图中所对应体素的概率值进行相乘之前，所述方法还包括：
    获取当前时刻的三维概率地图中每个体素到所述自移动设备之间的距离；
    在任一体素到自移动设备之间的距离大于预设距离阈值时，将所述体素从所述当前时刻的三维概率地图中删除，以得到更新后的当前时刻的三维概率地图。
  9. 根据权利要求1所述的地图构建方法，其中，所述根据预设调整参数更新所述上一时刻的三维概率地图中每个体素的概率值，包括：
    获取所述上一时刻的三维概率地图中每个体素的滞留时长，所述滞留时长表示所述体素的生成时刻到当前时刻的时长；
    将所述上一时刻的三维概率地图中每个体素的概率值与预设调整参数进行相乘；
    将相乘得到的每个所述体素的概率值与对应的每个体素的滞留时长进行相除，将相除得到的结果作为每个所述体素更新后的概率值。
  10. 一种地图构建装置，所述地图构建装置包括处理器、存储器、存储在所述存储器上并可被所述处理器执行的计算机程序以及用于实现所述处理器和所述存储器之间的连接通信的数据总线，其中所述计算机程序被所述处理器执行时，实现如权利要求1至9中任一项所述的地图构建方法。
  11. 一种存储介质，用于计算机可读存储，所述存储介质存储有一个或者多个程序，所述一个或者多个程序可被一个或者多个处理器执行，以实现权利要求1至9中任一项所述的地图构建方法。
PCT/CN2022/105886 2022-07-15 2022-07-15 地图构建方法、装置及存储介质 WO2024011557A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2022/105886 WO2024011557A1 (zh) 2022-07-15 2022-07-15 地图构建方法、装置及存储介质
CN202280004114.6A CN115917607A (zh) 2022-07-15 2022-07-15 地图构建方法、装置及存储介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/105886 WO2024011557A1 (zh) 2022-07-15 2022-07-15 地图构建方法、装置及存储介质

Publications (1)

Publication Number Publication Date
WO2024011557A1 true WO2024011557A1 (zh) 2024-01-18

Family

ID=86494088

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/105886 WO2024011557A1 (zh) 2022-07-15 2022-07-15 地图构建方法、装置及存储介质

Country Status (2)

Country Link
CN (1) CN115917607A (zh)
WO (1) WO2024011557A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116468351B (zh) * 2023-06-16 2023-11-07 深圳市磅旗科技智能发展有限公司 一种基于大数据的智慧物流管理方法、系统以及存储介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108897836A (zh) * 2018-06-25 2018-11-27 广州视源电子科技股份有限公司 一种机器人基于语义进行地图构建的方法和装置
CN111982124A (zh) * 2020-08-27 2020-11-24 华中科技大学 基于深度学习的玻璃场景下三维激光雷达导航方法及设备
US20210026361A1 (en) * 2019-07-26 2021-01-28 Deka Products Limited Partnership System and Method for Free Space Estimation
CN112419501A (zh) * 2020-12-10 2021-02-26 中山大学 一种地空异构协同地图构建方法
CN112419461A (zh) * 2020-11-16 2021-02-26 北京理工大学 一种协同无人系统联合语义建图方法
CN112424720A (zh) * 2018-07-20 2021-02-26 索尼公司 代理、存在概率图创建方法、代理动作控制方法和程序
CN114495064A (zh) * 2022-01-28 2022-05-13 浙江大学 一种基于单目深度估计的车辆周围障碍物预警方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108897836A (zh) * 2018-06-25 2018-11-27 广州视源电子科技股份有限公司 一种机器人基于语义进行地图构建的方法和装置
CN112424720A (zh) * 2018-07-20 2021-02-26 索尼公司 代理、存在概率图创建方法、代理动作控制方法和程序
US20210026361A1 (en) * 2019-07-26 2021-01-28 Deka Products Limited Partnership System and Method for Free Space Estimation
CN111982124A (zh) * 2020-08-27 2020-11-24 华中科技大学 基于深度学习的玻璃场景下三维激光雷达导航方法及设备
CN112419461A (zh) * 2020-11-16 2021-02-26 北京理工大学 一种协同无人系统联合语义建图方法
CN112419501A (zh) * 2020-12-10 2021-02-26 中山大学 一种地空异构协同地图构建方法
CN114495064A (zh) * 2022-01-28 2022-05-13 浙江大学 一种基于单目深度估计的车辆周围障碍物预警方法

Also Published As

Publication number Publication date
CN115917607A (zh) 2023-04-04

Similar Documents

Publication Publication Date Title
US10346996B2 (en) Image depth inference from semantic labels
KR102358554B1 (ko) 시맨틱 분할 모델을 위한 훈련 방법 및 장치, 전자 기기, 저장 매체
AU2016201908A1 (en) Joint depth estimation and semantic labeling of a single image
US20240078680A1 (en) Image segmentation method, network training method, electronic equipment and storage medium
CN114998595B (zh) 弱监督语义分割方法、语义分割方法及可读存储介质
CN113506368B (zh) 地图数据融合方法、装置、电子设备、介质和程序产品
WO2024011557A1 (zh) 地图构建方法、装置及存储介质
WO2022143366A1 (zh) 图像处理方法、装置、电子设备、介质及计算机程序产品
CN110633717A (zh) 一种目标检测模型的训练方法和装置
CN113887447A (zh) 对象分类模型的训练方法、对象分类预测方法及装置
KR20210065901A (ko) 이미지에서의 키 포인트 위치의 인식 방법, 장치, 전자기기 및 매체
CN112052884A (zh) 一种基于局部边缘特征增强的点云分类方法及系统
CN110633716A (zh) 一种目标对象的检测方法和装置
CN114792355A (zh) 虚拟形象生成方法、装置、电子设备和存储介质
CN113837194A (zh) 图像处理方法、图像处理装置、电子设备以及存储介质
CN116824609B (zh) 文档版式检测方法、装置和电子设备
CN116259064B (zh) 表格结构识别方法、表格结构识别模型的训练方法及装置
WO2022160897A1 (zh) 双目视差估计方法、模型训练方法以及相关设备
CN116740160A (zh) 一种复杂交通场景中的毫秒级多平面实时提取方法及装置
CN116596750A (zh) 一种点云处理方法、装置、电子设备及存储介质
CN114119990A (zh) 用于图像特征点匹配的方法、装置及计算机程序产品
CN113870428A (zh) 场景地图生成方法、相关装置及计算机程序产品
CN112180910A (zh) 一种移动机器人障碍物感知方法和装置
CN116030206A (zh) 地图生成方法、训练方法、装置、电子设备以及存储介质
WO2023134068A1 (zh) 数字识别模型的训练方法、装置、设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22950677

Country of ref document: EP

Kind code of ref document: A1