WO2020248851A1 - 库位的检测方法及装置、存储介质和电子装置 - Google Patents

库位的检测方法及装置、存储介质和电子装置 Download PDF

Info

Publication number
WO2020248851A1
WO2020248851A1 PCT/CN2020/093657 CN2020093657W WO2020248851A1 WO 2020248851 A1 WO2020248851 A1 WO 2020248851A1 CN 2020093657 W CN2020093657 W CN 2020093657W WO 2020248851 A1 WO2020248851 A1 WO 2020248851A1
Authority
WO
WIPO (PCT)
Prior art keywords
dimensional
preset
storage location
obstacle
voxel
Prior art date
Application number
PCT/CN2020/093657
Other languages
English (en)
French (fr)
Inventor
王建利
张晓红
宗柏青
赵君峤
张兴连
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Publication of WO2020248851A1 publication Critical patent/WO2020248851A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G06T2207/10012—Stereo images

Definitions

  • This application relates to the field of data processing, and in particular to a method and device for detecting storage locations, a storage medium and an electronic device.
  • the automatic parking system is a system that finds storage spaces and controls vehicles to automatically plan the route to park in.
  • This system can greatly improve the automation level of the current auxiliary parking system, and solve the "last mile" problem of parking.
  • User travel comfort provides technical guarantee, so it has become the current research hotspot of unmanned driving.
  • the system needs to detect accurate storage location and obstacle perception to provide prerequisites for driving decision planning.
  • accurate three-dimensional location detection will greatly improve the automatic parking system Security.
  • the location detection in the prior art is often limited to two-dimensional location detection due to the selection or method of sensors, and it is easy to have a limited detection range and low detection accuracy due to sensor distance, angular resolution, field of view, etc. .
  • the embodiments of the present application provide a method and device for detecting a warehouse location, a storage medium, and an electronic device.
  • a method for detecting storage locations includes: establishing three-dimensional voxel probabilities based on the relative positions of ToF cameras flying at multiple times from different directions and the RGB images of the multiple ToF cameras Map; Determine the semantics of the preset obstacle according to the three-dimensional voxel probability map and the vertical height distribution characteristics of the preset obstacle; According to the preset storage location model and the determined semantic extraction of the preset obstacle, stop can be The two-dimensional storage location of the storage location is combined with the three-dimensional voxel probability map to determine the three-dimensional storage location corresponding to the two-dimensional storage location.
  • a storage location detection device including: a establishing module for flying relative positions between ToF cameras at multiple times from different directions and RGB images of the multiple ToF cameras A three-dimensional voxel probability map is established; a determining module is used to determine the semantics of the preset obstacle according to the three-dimensional voxel probability map and the vertical height distribution characteristics of the preset obstacle; a processing module is used to determine the semantics of the preset obstacle according to the preset storage location The model and the determined semantics of the preset obstacle extract a two-dimensional location of a dockable location, and combine the three-dimensional voxel probability map to determine a three-dimensional location corresponding to the two-dimensional location.
  • a storage medium in which a computer program is stored, wherein the computer program is configured to execute the steps in any one of the foregoing method embodiments when running.
  • an electronic device including a memory and a processor, the memory is stored with a computer program, and the processor is configured to run the computer program to execute any of the above Steps in the method embodiment.
  • Fig. 1 is a flowchart of a method for detecting a warehouse location according to an embodiment of the present application
  • Fig. 2 is a schematic diagram of the arrangement of multiple ToF devices according to an embodiment of the present application.
  • Fig. 3 is a schematic diagram of a single-frame splicing point cloud according to an embodiment of the present application
  • FIG. 4 is a schematic diagram of a three-dimensional voxel probability fusion map according to an embodiment of the present application.
  • Fig. 5 is a flow chart of a warehouse location detection according to an embodiment of the present application.
  • Fig. 6 is a schematic diagram of a detection direction according to an embodiment of the present application.
  • Fig. 7 is a schematic diagram of a two-dimensional location detection result according to an embodiment of the present application.
  • Fig. 8 is a schematic diagram of a three-dimensional storage location detection result according to an embodiment of the present application.
  • Fig. 9 is a structural block diagram of a storage location detection device according to an embodiment of the present application.
  • FIG. 1 is a flowchart of a storage location detection method according to an embodiment of the present application. As shown in FIG. 1, the process includes the following steps:
  • Step S102 establishing a three-dimensional voxel probability map based on the relative positions of ToF cameras flying at multiple times from different directions and the RGB images of the multiple ToF cameras;
  • Step S104 Determine the semantics of the preset obstacle according to the three-dimensional voxel probability map and the vertical height distribution characteristics of the preset obstacle;
  • Step S106 Extract the two-dimensional storage location of the parkable storage location according to the preset storage location model and the determined semantics of the preset obstacle, and determine the three-dimensional storage location corresponding to the two-dimensional storage location in combination with the three-dimensional voxel probability map.
  • a three-dimensional voxel probability map is established based on the relative positions of multiple time-flight ToF cameras from different directions and the RGB images of the multiple ToF cameras; and then based on the three-dimensional voxel probability map and presets
  • the vertical height distribution feature of the obstacle determines the semantics of the preset obstacle; finally, the two-dimensional storage location of the dockable storage location is extracted according to the preset location model and the determined semantics of the preset obstacle, and combined with the three-dimensional voxel probability map
  • the three-dimensional location corresponding to the two-dimensional location is determined, thereby solving the problem that the location can only be detected by the two-dimensional location detection method, and improving the efficiency of automatic parking.
  • the three-dimensional voxel is established based on the relative positions of multiple time-flight ToF cameras from different directions and the RGB images of the multiple ToF cameras.
  • the probability map method can be implemented in the following ways:
  • Step S102-11 obtaining a wide-angle point cloud formed by stitching point clouds within a preset range from the relative position calibration results between multiple time-flying ToF cameras in different directions;
  • Step S102-12 Determine the camera self-motion between adjacent frames in the RGB image based on the feature point method to determine the motion change of the three-dimensional volume between the previous and next frames;
  • Step S102-13 Establish the three-dimensional voxel probability map based on the local point cloud and the global position in the wide-angle point cloud, and update the voxel probability value in the three-dimensional voxel probability map.
  • the method of determining the self-motion of the camera between adjacent frames in the RGB image based on the feature point method involved in the above step S102-12 to determine the motion change of the three-dimensional body between the previous and subsequent frames can be implemented in the following manner:
  • Step S1 Acquire RGB images of adjacent frames through multiple ToF cameras, and extract feature points in the RGB images of adjacent frames based on the scale-invariant feature transformation SIFT algorithm;
  • Step S2 matching the extracted feature points according to the proximity algorithm
  • Step S3 Calculate changes in the positions of adjacent cameras according to the matching results of the feature points to obtain a change matrix, and use the change matrix as a visual odometer.
  • step S1 to step S3 in a specific application scenario for RGB images of adjacent frames, the SIFT feature points of the images are extracted respectively and the nearest neighbor algorithm is used for matching. Since the feature point matching result is the basis for calculating the camera position change between adjacent frames, in the matching result, matches that are greater than N times the minimum feature distance are deleted to ensure the reliability of the feature point matching result.
  • the change matrix T is used to express the position transformation of the camera, which is composed of a rotation matrix R and a translation vector t.
  • the rotation matrix R is calculated and determined by the rotation angles ⁇ , ⁇ , ⁇ of the x, y, and z axes, and the translation vector t is composed of the position difference ( ⁇ x, ⁇ y, ⁇ z) of the origin of the two coordinate systems.
  • the reprojection error is minimized to calculate the camera external parameter change matrix. Since the position transformation is calculated using a three-dimensional point projection method, the scale of the translation vector can be guaranteed to be the same. Finally obtain the optimal parameter estimate versus Form the transformation matrix T as the result of the visual odometer.
  • the method of updating the voxel probability value in the three-dimensional voxel probability map involved in step S102-13 of this embodiment may be: after the voxel has been assigned In this case, the voxel probability value is subsequently updated according to the new point cloud results of each frame, where the update method is:
  • L_new is the updated grid probability value
  • L_old is the grid probability value before update
  • p(n) is the probability value divided by the point cloud of the current frame
  • l_min and l_max are the probability limit thresholds.
  • Step S104-11 setting three height intervals for detection, where the three height intervals include at least one of the following: ground detection interval, low-level obstacle detection interval, and high-level obstacle detection interval;
  • Step S104-12 performing through filtering on the three height intervals to obtain a set of voxels in a range corresponding to the three height intervals;
  • Step S104-13 Determine the distribution interval and geometric characteristics of the preset obstacle according to the voxel sets corresponding to the different height intervals, and perform semantic judgment on the obstacle to determine the semantics of the preset obstacle.
  • each voxel records the probability of being an obstacle, so the probability threshold ⁇ obstacle is taken as the basis for obstacle classification, and the obstacle perception in the three-dimensional map is extracted result.
  • the probability threshold ⁇ obstacle is taken as the basis for obstacle classification, and the obstacle perception in the three-dimensional map is extracted result.
  • three height intervals (unit: meters) are set:
  • the voxels in the detection interval of high and low obstacles are projected to a two-dimensional horizontal plane.
  • There are major obstacles in the garage environment such as pillars, vehicles, walls, etc., but the distribution intervals and geometric characteristics of obstacles with different semantics are obviously different.
  • the walls appear on both high and low floors and have obvious linear characteristics.
  • the lower floors all appear and are rectangular, and the vehicles only appear in a rectangular shape on the lower floors. Therefore, the common obstacles in the scene can be quickly semantically judged according to the differences in distribution and geometric characteristics, and the two-dimensional distribution of the corresponding semantic obstacles can be obtained.
  • Combining the three-dimensional voxel probability map to determine the three-dimensional location corresponding to the two-dimensional location can be achieved in the following ways:
  • Step S106-11 searching for a two-dimensional storage location that represents an empty storage location according to the distribution result of the two-dimensional preset obstacles on the horizontal plane in the preset storage location model;
  • Step S106-12 based on the planar position of the two-dimensional storage location, use the preset length as the step length to grow toward the top surface of the garage to form a three-dimensional bounding box;
  • Step S106-13 When the number of voxels of the obstacle contained in the three-dimensional bounding box exceeds a preset threshold, it is determined that the obstacle is touched and recorded as the maximum vertical height value of the storage location;
  • Step S106-14 taking the height value in the ground segmentation result from the maximum height value as the ground height value, and taking the ground height value as the height limit value of the three-dimensional storage location;
  • step S106-15 the two-dimensional storage location and the height limit value are formed into a three-dimensional storage location.
  • This optional embodiment provides a new solution based on multiple ToF (time of flight) cameras instead of conventional in-vehicle ultrasonic radar and surround view cameras, which can provide automatic parking systems with real-time three-dimensional environment perception in a local area. Location detection to ensure driving safety during parking.
  • ToF time of flight
  • the invention proposes to deploy multiple ToF cameras with different orientations to jointly observe the scene, and pre-calibrate the results of the relative positions between the devices to combine the multiple ToF cameras
  • the point clouds are spliced into a wide-angle point cloud, and then the self-motion of the camera is estimated using the visual odometer, real-time 3D voxel probability mapping of the local scene of the indoor parking lot is performed, and multi-frame point clouds are merged through the update of the probability.
  • the semantic results such as walls, pillars, and vehicles are allocated to the obstacles, combined with the assumptions of the location model and the three-dimensional
  • the top surface and ground structure extracted from the map information provide a real-time detection method for three-dimensional storage locations, and assist automatic parking by providing localized three-dimensional environmental perception and storage location detection information.
  • Step S202 arranging multiple ToF cameras with different orientations, and stitching a small range point cloud into a wide-angle point cloud based on the relative position calibration result between the cameras;
  • Step S204 using the RGB image of the ToF camera, use the feature point method to estimate the 6-DOF self-motion of the camera device, and calculate the three-dimensional rigid body motion change between the previous and next frames;
  • Step S206 3D voxel probability mapping is performed from the local point cloud and the global position, and multi-frame point cloud data is merged by updating the voxel probability value to eliminate dynamic obstacles and reduce the influence of noise points;
  • Step S208 by setting different height intervals, the interval through filtering voxels are obtained, and the vertical height distribution characteristics of common obstacles and structures in the parking lot are used to quickly distinguish the obstacle semantics and extract the ground structure;
  • Step S210 based on the location model assumptions and the location standard parameters, extract the two-dimensional location of the dockable vacancy, and combine the three-dimensional grid map to search for a limited height in the vertical direction of the corresponding location to detect the three-dimensional location;
  • step S212 a suitable parking location is selected in combination with vehicle parameters to obtain the target location and the passable range perception result.
  • the ToF camera is limited by the horizontal field of view, and it is difficult to perceive the wide-angle range around the vehicle body.
  • multiple ToF cameras with different orientations are arranged to jointly observe the scene.
  • the point clouds of multiple ToF cameras are stitched into a wide-angle point cloud.
  • Fig. 2 is a schematic diagram of the arrangement of multiple ToF devices according to the embodiment of the present application. The way the camera is fixed on the platform is shown in Fig. 2. Realize the joint observation of surround view, and there is a certain common field of view between adjacent cameras to ensure the continuity of stitching. When connecting multiple Kinect-V2 cameras, ensure that the bandwidth of the USB3.0 interface and the control bus is sufficient.
  • Kinect-V2 contains color and infrared cameras to obtain RGB and depth images respectively (comprising RGB-D data).
  • the original depth data contains a lot of noise.
  • the bilateral filtering method is used to filter noise points while maintaining clearer clarity.
  • the boundary information Due to different camera positions, image registration is first performed according to camera parameters. Then the point cloud recovered from the depth map and camera parameters is converted to the assumed coordinate system according to the pre-calibrated inter-camera change parameters, and stitched into a wide-angle point cloud.
  • the stitched point cloud is shown in Figure 3.
  • 3 is a schematic diagram of a single-frame splicing point cloud according to an embodiment of the present application.
  • step S204 in order to perceive a wider range, combining the characteristics of the ToF camera data, using the RGB images taken at different times, using the feature point method to estimate the camera self-motion between adjacent frames, and build a visual odometer.
  • SIFT Scale-Invariant Feature Transform
  • feature point matching For RGB images of adjacent frames, the SIFT feature points of the image are extracted and the nearest neighbor algorithm is used for matching. Since the feature point matching result is the basis for calculating the camera position change between adjacent frames, in the matching result, matches that are greater than N times the minimum feature distance are deleted to ensure the reliability of the feature point matching result.
  • calculate the transformation matrix use the transformation matrix T to express the position transformation of the camera, which is composed of the rotation matrix R and the translation vector t.
  • the rotation matrix R is calculated and determined by the rotation angles ⁇ , ⁇ , ⁇ of the x, y, and z axes, and the translation vector t is composed of the position difference ( ⁇ x, ⁇ y, ⁇ z) of the origin of the two coordinate systems.
  • the reprojection error is minimized to calculate the camera external parameter change matrix. Since the position transformation is calculated using a three-dimensional point projection method, the scale of the translation vector can be guaranteed to be the same. Finally obtain the optimal parameter estimate versus Form the transformation matrix T as the result of the visual odometer.
  • the three-dimensional voxel map uses probability to describe the possibility of the voxel state as an obstacle point. Within the limited range of probability, the higher the probability value, the greater the probability of the voxel as an obstacle.
  • a hierarchical data structure is realized. The smallest voxel boundary size represents the resolution of the map, and the voxel indicates whether the area is occupied and Record the probability value of the obstacle.
  • the inserted point cloud after the position transformation is classified into the nearest intersecting voxel, and the ray is constructed from the current position center and each point cloud, and the ray terminal corresponds to In the state of hitting the obstacle, the voxel passing through the middle of the ray corresponds to the state of not hitting.
  • the voxels in the scanning range are divided into two sets of hit and miss. Initialize the unobserved voxels, and assign the probability values to p hit and p miss .
  • the update method is: if the voxel has been assigned, then the voxel probability value is updated according to the new point cloud results of each frame.
  • the update method is:
  • L new is the mesh probability value after update
  • L old is the mesh probability value before update
  • p(n) is the probability value divided by the point cloud of the current frame
  • l min and l max are the probability limit thresholds, which guarantees Voxel status change update times.
  • the update strategy guarantees from the mathematical model that the voxel map can increase the probability value of the target fixed obstacle, thereby eliminating the ghosting of the dynamic obstacle, and eliminating the influence of spatial noise points on the mapping.
  • FIG. 4 is a schematic diagram of the 3D voxel probability fusion map according to an embodiment of the present application.
  • each voxel records the probability of being an obstacle, so the probability threshold ⁇ obstacle is taken as the basis for obstacle classification, and the result of obstacle perception in the three-dimensional map is extracted.
  • the probability threshold ⁇ obstacle is taken as the basis for obstacle classification, and the result of obstacle perception in the three-dimensional map is extracted.
  • three height intervals (unit: meters) are set:
  • the voxels in the detection interval of high and low obstacles are projected to a two-dimensional horizontal plane.
  • There are major obstacles in the garage environment such as pillars, vehicles, walls, etc., but the distribution intervals and geometric characteristics of obstacles with different semantics are obviously different.
  • the walls appear on both high and low floors and have obvious linear characteristics.
  • the lower floors all appear and are rectangular, and the vehicles only appear in a rectangular shape on the lower floors. Therefore, the common obstacles in the scene can be quickly semantically judged according to the differences in distribution and geometric characteristics, and the two-dimensional distribution of the corresponding semantic obstacles can be obtained.
  • Fig. 5 is a flow chart of the location detection according to an embodiment of the present application, searching for a vacant location area that characterizes the location according to the horizontal two-dimensional obstacle distribution result.
  • the design of parking spaces is often vertical or parallel to the wall.
  • the direction of the parking lot road is usually vertical or parallel to the storage space, and the columns are often distributed along the road. Therefore, the two-dimensional location detection method is based on the following assumptions:
  • FIG. 6 is a schematic diagram of the detection direction according to an embodiment of the present application.
  • the two-dimensional inspection image is rotated with the center of the vehicle as the origin, and the longitudinal axis of the image is parallel to the direction line, and then the smallest envelope rectangle of various obstacles is projected onto the detection direction, and the overlapping area rectangles are merged.
  • the rectangular space between adjacent obstacles is greater than the minimum standard for parking spaces, for example, the parallel type is 6.0 meters, the vertical type is 2.4 meters, and the empty rectangular area is used as the parking area.
  • FIG. 7, is a schematic diagram of the detection result of the two-dimensional storage location according to an embodiment of the present application.
  • the sensor data is fully utilized, and the data is merged by establishing a local probability grid map, which expands the sensing range and reduces the impact of various noises; realizes real-time environmental perception And the three-dimensional location detection results provide a basis for driving decision-making for the automatic parking system.
  • the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, of course, it can also be implemented by hardware, but in many cases the former is Better implementation.
  • the technical solution of this application essentially or the part that contributes to the existing technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM/RAM, magnetic disk, The optical disc) includes several instructions to enable a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the method described in each embodiment of the present application.
  • a storage location detection device is also provided, and the device is used to implement the above-mentioned embodiments and preferred implementations, and those that have been explained will not be repeated.
  • the term "module” can implement a combination of software and/or hardware with predetermined functions.
  • the devices described in the following embodiments are preferably implemented by software, hardware or a combination of software and hardware is also possible and conceived.
  • FIG. 9 is a structural block diagram of a storage location detection device according to an embodiment of the present application.
  • the device includes: a establishing module 902 for flying relative positions between ToF cameras at multiple times from different directions and The RGB images of the multiple ToF cameras establish a three-dimensional voxel probability map; the determining module 904 is coupled to the establishing module 902, and is used to determine the preset obstacle according to the three-dimensional voxel probability map and the vertical height distribution characteristics of the preset obstacle.
  • the processing module 906, coupled with the determination module 904 is used to extract the two-dimensional location of the dockable location according to the preset location model and the determined semantics of the preset obstacle, and combine the three-dimensional voxel probability map Determine the three-dimensional location corresponding to the two-dimensional location.
  • the establishing module may include: an acquiring unit, configured to acquire a wide-angle point cloud formed by stitching point clouds within a preset range from the relative position calibration results between multiple time-flight ToF cameras with different orientations; and a determining unit , For determining the self-motion of the camera between adjacent frames in the RGB image based on the feature point method to determine the motion change of the three-dimensional body between the previous and subsequent frames; a processing unit for determining the local point cloud and the global position in the wide-angle point cloud The three-dimensional voxel probability map is established, and the voxel probability values in the three-dimensional voxel probability map are updated.
  • the determining unit may include: a first processing unit, configured to obtain RGB images of adjacent frames through a plurality of ToF cameras, and respectively extract features in the RGB images of adjacent frames based on the scale-invariant feature transformation SIFT algorithm Point; matching unit, used to match the extracted feature points according to the proximity algorithm; the second processing unit, used to calculate the changes in the positions of adjacent cameras according to the matching results of the feature points to obtain the change matrix, and use the change matrix as the vision Odometer.
  • a first processing unit configured to obtain RGB images of adjacent frames through a plurality of ToF cameras, and respectively extract features in the RGB images of adjacent frames based on the scale-invariant feature transformation SIFT algorithm Point
  • matching unit used to match the extracted feature points according to the proximity algorithm
  • the second processing unit used to calculate the changes in the positions of adjacent cameras according to the matching results of the feature points to obtain the change matrix, and use the change matrix as the vision Odometer.
  • the processing unit is further configured to subsequently update the probability value of the voxel according to the new point cloud result of each frame when the voxel has been assigned; wherein the update method is:
  • L new is the mesh probability value after update
  • L old is the mesh probability value before update
  • p(n) is the probability value divided by the point cloud of the current frame
  • l min and l max are the probability limit thresholds.
  • the determination module 908 includes: a setting unit for setting three height intervals for detection, wherein the three height intervals include at least one of the following: ground detection interval, bottom obstacle detection interval, and high Obstacle detection interval; the third processing unit is used to perform through filtering on the three height intervals to obtain the voxel set within the corresponding range of the three height intervals; the fourth processing unit is used to according to the voxels corresponding to different height intervals
  • the collection determines the distribution interval and geometric characteristics of the preset obstacles to make a semantic judgment on the obstacles to determine the semantics of the preset obstacles.
  • the processing module 910 includes: a searching unit for searching for a two-dimensional storage location that characterizes a vacant storage location according to the distribution result of a two-dimensional preset obstacle in the horizontal plane in the preset storage location model; Based on the plane position of the two-dimensional storage location, the preset length is used as the step length to grow toward the top surface of the garage to form a three-dimensional bounding box; the sixth processing unit is used for the number of voxels of obstacles contained in the three-dimensional bounding box When the preset threshold is exceeded, the obstacle is determined to be touched and recorded as the maximum height value in the vertical direction of the location; the seventh processing unit is used to take the height value in the ground segmentation result from the maximum height value as the ground The height value, and the ground height value is used as the height limit value of the three-dimensional storage location; the eighth processing unit is used to form a three-dimensional storage location with the two-dimensional storage location and the height limit value.
  • each of the above modules can be implemented by software or hardware.
  • it can be implemented in the following manner, but not limited to this: the above modules are all located in the same processor; or, the above modules are combined in any combination The forms are located in different processors.
  • the embodiment of the present application also provides a storage medium in which a computer program is stored, wherein the computer program is configured to execute the steps in any of the foregoing method embodiments when running.
  • the storage medium may be configured to store a computer program for executing the following steps:
  • S2 Determine the semantics of the preset obstacle according to the three-dimensional voxel probability map and the vertical height distribution characteristics of the preset obstacle;
  • S3 Extract the two-dimensional storage location of the dockable storage location according to the preset storage location model and the determined semantics of the preset obstacle, and determine the three-dimensional storage location corresponding to the two-dimensional storage location in combination with the three-dimensional voxel probability map.
  • the storage medium may include, but is not limited to: U disk, Read-Only Memory (Read-Only Memory, ROM for short), Random Access Memory (RAM for short), mobile hard disk, magnetic disk Or various media that can store computer programs, such as an optical disk.
  • the embodiment of the present application also provides an electronic device, including a memory and a processor, the memory is stored with a computer program, and the processor is configured to run the computer program to execute the steps in any of the foregoing method embodiments.
  • the electronic device may further include a transmission device and an input-output device, wherein the transmission device is connected to the aforementioned processor, and the input-output device is connected to the aforementioned processor.
  • the processor may be configured to execute the following steps through a computer program:
  • S2 Determine the semantics of the preset obstacle according to the three-dimensional voxel probability map and the vertical height distribution characteristics of the preset obstacle;
  • S3 Extract the two-dimensional storage location of the dockable storage location according to the preset storage location model and the determined semantics of the preset obstacle, and determine the three-dimensional storage location corresponding to the two-dimensional storage location in combination with the three-dimensional voxel probability map.
  • modules or steps of this application can be implemented by a general computing device, and they can be concentrated on a single computing device or distributed in a network composed of multiple computing devices.
  • they can be implemented with program codes executable by the computing device, so that they can be stored in the storage device for execution by the computing device, and in some cases, can be executed in a different order than here.

Abstract

一种库位的检测方法及装置、存储介质和电子装置，该方法包括：基于由不同朝向的多个时间飞行ToF相机间的相对位置以及所述多个ToF相机的RGB图像建立三维体素概率地图（S102）；根据三维体素概率地图和预设障碍物的垂直高度分布特征确定预设障碍物的语义（S104）；根据预设库位模型和确定的预设障碍物的语义提取可停靠的库位的二维库位，并结合三维体素概率地图确定与二维库位对应的三维库位（S106）。

Description

库位的检测方法及装置、存储介质和电子装置
相关申请的交叉引用
本申请基于申请号为201910502737.6、申请日为2019年6月11日的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此以引入方式并入本申请。
技术领域
本申请涉及数据处理领域，具体而言，涉及一种库位的检测方法及装置、存储介质和电子装置。
背景技术
自动泊车系统是通过寻找库位并控制车辆自动规划路径泊车入位的系统，该系统能够大幅提高当前辅助泊车系统的自动化水平，并为解决泊车的“最后一公里”难题、提升用户出行舒适度提供技术保证，因此成为当前无人驾驶的研究热点。系统需探测精确的库位位置与障碍物感知为驾驶决策规划提供先决条件。其中，在复杂室内立体停车环境中，由于垂直区域方向上的复杂结构(如管道、横梁等)限制了不同高度、车型的适停性，故准确的三维库位检测将大幅提升自动泊车系统的安全性。
而现有技术中的库位检测由于传感器的选择或方法限制，往往局限于二维库位检测，且容易受传感器距离、角度分辨率、视场等原因导致检测范围不广以及检测精度不高。
针对相关技术中的上述问题，目前尚未存在有效的解决方案。
发明内容
本申请实施例提供了一种库位的检测方法及装置、存储介质和电子装置。
根据本申请的一个实施例，提供了一种库位的检测方法，包括：基于由不同朝向的多个时间飞行ToF相机间的相对位置以及所述多个ToF相机的RGB图像建立三维体素概率地图；根据所述三维体素概率地图和预设障碍物的垂直高度分布特征确定所述预设障碍物的语义；根据预设库位模型和确定的所述预设 障碍物的语义提取可停靠的库位的二维库位，并结合所述三维体素概率地图确定与所述二维库位对应的三维库位。
根据本申请的一个实施例，提供了一种库位的检测装置，包括：建立模块，用于基于由不同朝向的多个时间飞行ToF相机间的相对位置以及所述多个ToF相机的RGB图像建立三维体素概率地图；确定模块，用于根据所述三维体素概率地图和预设障碍物的垂直高度分布特征确定所述预设障碍物的语义；处理模块，用于根据预设库位模型和确定的所述预设障碍物的语义提取可停靠的库位的二维库位，并结合所述三维体素概率地图确定与所述二维库位对应的三维库位。
根据本申请的又一个实施例，还提供了一种存储介质，所述存储介质中存储有计算机程序，其中，所述计算机程序被设置为运行时执行上述任一项方法实施例中的步骤。
根据本申请的又一个实施例，还提供了一种电子装置，包括存储器和处理器，所述存储器中存储有计算机程序，所述处理器被设置为运行所述计算机程序以执行上述任一项方法实施例中的步骤。
附图说明
此处所说明的附图用来提供对本申请的进一步理解，构成本申请的一部分，本申请的示意性实施例及其说明用于解释本申请，并不构成对本申请的不当限定。在附图中：
图1是根据本申请实施例的库位的检测方法流程图；
图2是根据本申请实施例的多ToF设备布置示意图；
图3是根据本申请实施例的单帧拼接点云示意图；
图4是根据本申请实施例的三维体素概率融合地图的示意图；
图5是根据本申请实施例的库位的检测流程图；
图6是根据本申请实施例的检测方向示意图；
图7是根据本申请实施例的二维库位探测结果示意图；
图8是根据本申请实施例的三维库位检测结果示意图；
图9是根据本申请实施例的库位的检测装置的结构框图。
具体实施方式
下文中将参考附图并结合实施例来详细说明本申请。需要说明的是，在不冲突的情况下，本申请中的实施例及实施例中的特征可以相互组合。
需要说明的是，本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。
实施例1
在本实施例中提供了一种库位的检测方法，图1是根据本申请实施例的库位的检测方法流程图，如图1所示，该流程包括如下步骤：
步骤S102，基于由不同朝向的多个时间飞行ToF相机间的相对位置以及所述多个ToF相机的RGB图像建立三维体素概率地图；
步骤S104，根据三维体素概率地图和预设障碍物的垂直高度分布特征确定预设障碍物的语义；
步骤S106，根据预设库位模型和确定的预设障碍物的语义提取可停靠的库位的二维库位，并结合三维体素概率地图确定与二维库位对应的三维库位。
通过上述步骤S102至步骤S106，基于由不同朝向的多个时间飞行ToF相机间的相对位置以及所述多个ToF相机的RGB图像建立三维体素概率地图；进而根据三维体素概率地图和预设障碍物的垂直高度分布特征确定预设障碍物的语义；最后根据预设库位模型和确定的预设障碍物的语义提取可停靠的库位的二维库位，并结合三维体素概率地图确定与二维库位对应的三维库位，从而解决了只能够通过二维库位检测的方式对库位进行检测的问题，提高了自动泊车的效率。
在本实施例的可选实施方式中，对于本实施例步骤S102中涉及到的基于由不同朝向的多个时间飞行ToF相机间的相对位置以及所述多个ToF相机的RGB图像建立三维体素概率地图的方式，可以通过如下方式来实现：
步骤S102-11，获取由不同朝向的多个时间飞行ToF相机间的相对位置标定结果将预设范围内的点云拼接成的广角点云；
步骤S102-12，基于特征点法确定所述RGB图像中相邻帧间相机自运动以确定前后帧间三维体的运动变化；
步骤S102-13，基于所述广角点云中的局部点云以及全局位置建立所述三维 体素概率地图，并对所述三维体素概率地图中的体素概率值进行更新。
进一步地，上述步骤S102-12中涉及到的基于特征点法确定所述RGB图像中相邻帧间相机自运动以确定前后帧间三维体的运动变化的方式，可以通过如下方式来实现：
步骤S1，通过多个ToF相机获取相邻帧的RGB图像，并基于尺度不变特征变换SIFT算法分别提取相邻帧RGB图像中的特征点；
步骤S2，根据邻近算法对提取出的特征点进行匹配；
步骤S3，根据特征点的匹配结果计算相邻相机位置的变化以得到变化矩阵，并将变化矩阵作为视觉里程计。
对于步骤S1至步骤S3在具体的应用场景中可以是：对于相邻帧的RGB图像，分别提取图像的SIFT特征点并使用最近邻算法进行匹配。由于特征点匹配结果是计算相邻帧间相机位置变化的基础，因此在匹配结果中，删除大于N倍最小特征距离的匹配，以确保特征点匹配结果的可靠性。使用变化矩阵T表达相机的位置变换，其由旋转矩阵R以及平移向量t组成。其中旋转矩阵R由x,y,z轴的旋转角α,β,γ计算确定，平移向量t由两坐标系原点的位置差值(Δx,Δy,Δz)组成。通过使用深度信息恢复的多对匹配特征点的三维坐标与图像坐标，最小化重投影误差计算相机外参变化矩阵。由于使用三维点投影的方式计算位置变换，可保证平移向量的尺度相同。最终获得最优参数估值
Figure PCTCN2020093657-appb-000001
与
Figure PCTCN2020093657-appb-000002
组成变换矩阵T，作为视觉里程计的结果。
在本实施例的另一个可选实施方式中，对本实施例步骤S102-13中涉及到的对三维体素概率地图中的体素概率值进行更新的方式可以是：在体素已被赋值的情况下，后续根据新的每帧点云结果对体素概率值进行更新，其中，更新的方式为：
L(n)＝log[p(n)/(1-p(n))]
L_new(n)＝max(min(L_old(n)+L(n),l_max),l_min)
其中L_new为更新后的网格概率值，L_old为更新前的网格概率值，p(n)为由当帧点云划分的概率值，l_min与l_max为概率限定阈值。
在本实施例的再一个可选实施方式中，对于本实施例步骤S104中涉及到的根据三维体素概率地图和预设障碍物的垂直高度分布特征确定预设障碍物的语 义的方式，可以包括：
步骤S104-11，设置用于进行探测的三个高度区间，其中，三个高度区间包括以下至少之一：地面探测区间、底层障碍物探测区间、高层障碍物探测区间；
步骤S104-12，对三个高度区间进行直通滤波获取与三个高度区间对应范围内的体素集合；
步骤S104-13，根据与不同高度区间对应的体素集合确定预设障碍物的分布区间及几何特征对障碍物进行语义判断，以确定预设障碍物的语义。
对于上述步骤S104-11至步骤S104-13，在具体的应用场景中可以是：每个体素记录着作为障碍物的概率，故取概率阈值θ obstacle作为障碍分类依据，提取三维地图中障碍感知的结果。同时根据设备距离地面水平高度h 0，设置三个高度区间(单位：米)：
Figure PCTCN2020093657-appb-000003
对三个高度区间进行直通滤波，获得对应范围内的体素集合。提取地面探测区间的体素中心点构建点云，并使用RANSAC(Random sample consensus)方法提取地面平面参数记录地面高度值。对高、低层障碍物探测区间体素，正投影至二维水平面。车库环境中存在主要障碍物如柱子、车辆、墙面等，而不同语义的障碍物其分布区间及几何特征相差明显，如墙面在高、低层均有出现且线性特征明显，柱子在高、低层均有出现且呈矩形，车辆只在低层以矩形形状出现。故可根据分布与几何特征差异快速对场景内常见障碍物进行语义判断，并获得对应语义障碍物的二维分布。
在本实施例的另一个可选实施方式中，对于本实施例步骤S106中涉及到的根据预设库位模型和确定的预设障碍物的语义提取可停靠的库位的二维库位，并结合三维体素概率地图确定与二维库位对应的三维库位的方式，可以通过如下方式来实现：
步骤S106-11，根据预设库位模型中水平面二维预设障碍物的分布结果查找表征空位库位的二维库位；
步骤S106-12，基于二维库位的平面位置，以预设长度作为步长向车库顶面 方向增长，组成三维包围盒；
步骤S106-13，在三维包围盒内包含的障碍物的体素个数超过预设阈值的情况下，确定触碰到障碍物，并记录为库位竖直方向上最大高度值；
步骤S106-14，从最大高度值中取地面分割结果中的高度值作为地面高度值，并将地面高度值作为三维库位的限高值；
步骤S106-15，将二维库位与限高值组成三维库位。
下面结合本申请的可选实施例对本申请进行举例说明；
本可选实施例提供了一种基于多ToF(time of flight，飞行时间)相机替代常规的车载超声波雷达与环视相机的全新方案，可为自动泊车系统提供局部范围内实时的三维环境感知与库位探测，从而保证停车过程的驾驶安全。为突破ToF相机有限的水平视场角局限，以及模仿车载环视相机布置方案，发明提出布设多个不同朝向的ToF相机对场景进行联合观测，并通过设备间相对位置预先标定结果，将多ToF相机点云拼接成广角点云，随后利用视觉里程计估计相机自运动，对室内停车场的局部场景进行实时三维体素概率建图，通过概率的更新融合多帧点云。在局部地图上对不同高度区间的体素进行水平俯视投影，根据停车场常见障碍物的垂直高度分布特征，对障碍物分配墙面、柱子、车辆等语义结果，结合库位模型假设与根据三维地图信息提取的顶面、地面结构，提出对三维库位的实时检测方法，通过提供局部范围内的三维环境感知与库位探测信息，辅助自动泊车。
本可选实施例所采用的技术方案的方法步骤包括：
步骤S202，布设多个不同朝向的ToF相机，由相机间相对位置标定结果将小范围点云拼接成广角点云；
步骤S204，利用ToF相机的RGB图像，使用特征点法对相机设备的6自由度自运动进行估计，计算前后帧间的三维刚体运动变化；
步骤S206，由局部点云及全局位置进行三维体素概率建图，通过更新体素概率值融合多帧点云数据，消除动态障碍并降低噪声点的影响；
步骤S208，通过设置不同高度区间，获得区间直通滤波体素，并由停车场常见障碍物与结构的垂直高度分布特征，快速区分障碍物语义并提取地面结构；
步骤S210，基于库位模型假设与库位标准参数，提取可停靠空位的二维位置，结合三维格网地图在对应位置的垂直方向上的搜索限定高度，探测三维库位；
步骤S212，结合车辆参数，选取适停库位，得到目标库位位置与可通行范围感知结果。
对于上述步骤S202，ToF相机受限于水平视场角，难以对车身周围广角范围进行感知。通过模仿车载环视相机布置方案，布设多个不同朝向的ToF相机对场景进行联合观测。由设备间相对位置预先标定结果，将多ToF相机点云拼接成广角点云。
因此，在本可选实施例中优选采用微软生产的Kinect-V2作为ToF相机作为实验仪器，图2是根据本申请实施例的多ToF设备布置示意图，相机固定于平台方式如图2所示，实现环视联合观测，相邻相机间存在一定的公共视场保证拼接连续性。多Kinect-V2相机连接时，需保证USB3.0接口与控制总线的带宽足够。
其中，Kinect-V2含彩色、红外摄像头，分别获取RGB与深度图像(组成RGB-D数据)，原始深度数据中含有较多噪声，采用双边滤波法，目的是过滤噪声点的同时保持更为清晰的边界信息。由于相机位置不同，首先根据相机参数进行图像配准。随后将深度图与相机参数恢复的点云，根据预先标定的相机间变化参数，将所有点云转换至假定坐标系下，拼接成广角点云，拼接后的点云如图3所示，图3是根据本申请实施例的单帧拼接点云示意图。
对于步骤S204，为对更广的范围进行感知，结合ToF相机数据特点，利用不同时刻拍摄的RGB图像，使用特征点法估计相邻帧间相机自运动，搭建视觉里程计。为适应光照环境的复杂变化，选用较为鲁棒的SIFT(Scale-Invariant Feature Transform)算法提取RGB图像上的特征点并进行匹配。
首先，特征点匹配：对于相邻帧的RGB图像，分别提取图像的SIFT特征点并使用最近邻算法进行匹配。由于特征点匹配结果是计算相邻帧间相机位置变化的基础，因此在匹配结果中，删除大于N倍最小特征距离的匹配，以确保特征点匹配结果的可靠性。
然后，进行变换矩阵的计算：使用变化矩阵T表达相机的位置变换，其由旋转矩阵R以及平移向量t组成。其中旋转矩阵R由x,y,z轴的旋转角α,β,γ计 算确定，平移向量t由两坐标系原点的位置差值(Δx,Δy,Δz)组成。通过使用深度信息恢复的多对匹配特征点的三维坐标与图像坐标，最小化重投影误差计算相机外参变化矩阵。由于使用三维点投影的方式计算位置变换，可保证平移向量的尺度相同。最终获得最优参数估值
Figure PCTCN2020093657-appb-000004
与
Figure PCTCN2020093657-appb-000005
组成变换矩阵T，作为视觉里程计的结果。
对于上述步骤S206，三维体素地图使用概率的方式描述体素状态作为障碍点的可能性。在概率限定范围内，概率值越高代表体素作为障碍的概率越大。通过将空间沿x,y,z三主轴方向不断划分子体素，实现了分层的数据结构，其中最小的体素边界尺寸代表了地图的分辨率，体素标识了该区域是否被占据且记录障碍的概率值。结合此空间划分特性，使用八叉树对空间体素元素进行管理，可对内部节点实现高效地查询、插入、删除、修改等操作。
首先，确定网格状态：根据每帧的拼接广角点云与运动估计，将位置变换后的插入点云归于最近的相交体素内，由当前位置中心与每个点云构建射线，射线终端对应于击中障碍物状态，射线中间经过的体素对应于未击中状态，据此将扫描范围内的体素划分为hit与miss两个集合。对未观测过的体素初始化，概率值分配为p hit、p miss。
然后进行概率更新，其更新的方式为：若体素已被赋值，后续根据新的每帧点云结果对体素概率值进行更新，更新方式为：
Figure PCTCN2020093657-appb-000006
L new(n)＝max(min(L old(n)+L(n),l max),l min)
其中L new为更新后的网格概率值，L old为更新前的网格概率值,p(n)为由当帧点云划分的概率值，l min与l max为概率限定阈值，保证了体素状态变化更新次数。更新策略从数理模型上保证了体素地图能增大目标固定障碍的概率值，从而消除动态障碍的重影，同时消除空间噪声点对建图的影响。
需要说明的是，建图过程中保留距离当前位置一定距离阈值内的节点，通过空间限定范围，减少了节点的数量，保证建图更新的效率，同时去除建图过程中，误差累积过大的区域，保证检测精度，三维体素概率融合建图结果如图4所示，图4是根据本申请实施例的三维体素概率融合地图的示意图。
对于上述步骤S208，每个体素记录着作为障碍物的概率，故取概率阈值θ obstacle作为障碍分类依据，提取三维地图中障碍感知的结果。同时根据设备距离地面水平高度h 0，设置三个高度区间(单位：米)：
Figure PCTCN2020093657-appb-000007
对三个高度区间进行直通滤波，获得对应范围内的体素集合。提取地面探测区间的体素中心点构建点云，并使用RANSAC(Random sample consensus)方法提取地面平面参数记录地面高度值。对高、低层障碍物探测区间体素，正投影至二维水平面。车库环境中存在主要障碍物如柱子、车辆、墙面等，而不同语义的障碍物其分布区间及几何特征相差明显，如墙面在高、低层均有出现且线性特征明显，柱子在高、低层均有出现且呈矩形，车辆只在低层以矩形形状出现。故可根据分布与几何特征差异快速对场景内常见障碍物进行语义判断，并获得对应语义障碍物的二维分布。
对于上述步骤S210，基于图5，图5是根据本申请实施例的库位的检测流程图，根据水平面二维障碍物分布结果寻找表征库位的空位区域。在标准的地下车库环境中，车位的设计常垂直或平行于墙面，另一方面，停车场道路方向通常垂直或平行于库位，而柱子常沿道路方向分布。故二维库位的检测方法中基于如下假设：
(1)若局部停车场环境中无墙面，则在柱子沿线方向上检测库位，如图6中的虚线投影方向，其中，图6是根据本申请实施例的检测方向示意图。
(2)若局部停车场环境中存在墙面，则在墙面垂直方向上检测库位，如图6中实线投影方向。
为方便检测，将二维检测图以车辆中心为原点，旋转至图像纵轴与方向线平行，随后将各类障碍物的最小包络矩形投影至检测方向上，并合并重叠区域矩形。取相邻障碍物矩形间距大于停车位最小标准，例如平行式6.0米，垂直式2.4米，的空位矩形区域作为可停车区域。旋转后二维库位探测效果如图7所示，图7是根据本申请实施例的二维库位探测结果示意图。
在已有二维库位位置基础上，基于二维库位的平面位置，以△D作为步长向 车库顶面方向增长，组成三维包围盒。若三维包围盒内包含的障碍体素个数超过设定阈值时视作触碰到障碍物，记录库位竖直方向上方最大高度值h ceiling；取地面分割结果中的高度值作为地面高度h floor，确定三维库位的限高值。最终取水平面二维库位与竖直限定高度组成三维包围盒，作为可停三维库位，三维库位检测结果如图8所示，图8是根据本申请实施例的三维库位检测结果示意图。
通过上述步骤S202至步骤S212可知，在本可选实施例中充分利用传感器数据，通过建立局部概率格网地图的方式融合数据，扩大了感知范围并降低各类噪声影响；实现了实时的环境感知与三维库位探测结果，为自动泊车系统提供驾驶决策基础。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到根据上述实施例的方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端设备(可以是手机，计算机，服务器，或者网络设备等)执行本申请各个实施例所述的方法。
实施例2
在本实施例中还提供了一种库位的检测装置，该装置用于实现上述实施例及优选实施方式，已经进行过说明的不再赘述。如以下所使用的，术语“模块”可以实现预定功能的软件和/或硬件的组合。尽管以下实施例所描述的装置较佳地以软件来实现，但是硬件，或者软件和硬件的组合的实现也是可能并被构想的。
图9是根据本申请实施例的库位的检测装置的结构框图，如图9所示，该装置包括：建立模块902，用于基于由不同朝向的多个时间飞行ToF相机间的相对位置以及所述多个ToF相机的RGB图像建立三维体素概率地图；确定模块904，与建立模块902耦合连接，用于根据三维体素概率地图和预设障碍物的垂直高度分布特征确定预设障碍物的语义；处理模块906，与确定模块904耦合连接，用于根据预设库位模型和确定的预设障碍物的语义提取可停靠的库位的二维库 位，并结合三维体素概率地图确定与二维库位对应的三维库位。
在一实施例中，建立模块可以包括：获取单元，用于获取由不同朝向的多个时间飞行ToF相机间的相对位置标定结果将预设范围内的点云拼接成的广角点云；确定单元，用于基于特征点法确定所述RGB图像中相邻帧间相机自运动以确定前后帧间三维体的运动变化；处理单元，用于基于所述广角点云中的局部点云以及全局位置建立所述三维体素概率地图，并对所述三维体素概率地图中的体素概率值进行更新。
在一实施例中，确定单元可以包括：第一处理单元，用于通过多个ToF相机获取相邻帧的RGB图像，并基于尺度不变特征变换SIFT算法分别提取相邻帧RGB图像中的特征点；匹配单元，用于根据邻近算法对提取出的特征点进行匹配；第二处理单元，用于根据特征点的匹配结果计算相邻相机位置的变化以得到变化矩阵，并将变化矩阵作为视觉里程计。
在一实施例中，处理单元，还用于在体素已被赋值的情况下，后续根据新的每帧点云结果对体素概率值进行更新；其中，更新的方式为：
Figure PCTCN2020093657-appb-000008
L new(n)＝max(min(L old(n)+L(n),l max),l min)
其中L new为更新后的网格概率值，L old为更新前的网格概率值，p(n)为由当帧点云划分的概率值，l min与l max为概率限定阈值。
在一实施例中，确定模块908包括：设置单元，用于设置用于进行探测的三个高度区间，其中，三个高度区间包括以下至少之一：地面探测区间、底层障碍物探测区间、高层障碍物探测区间；第三处理单元，用于对三个高度区间进行直通滤波获取与三个高度区间对应范围内的体素集合；第四处理单元，用于根据与不同高度区间对应的体素集合确定预设障碍物的分布区间及几何特征对障碍物进行语义判断，以确定预设障碍物的语义。
在一实施例中，处理模块910包括：查找单元，用于根据预设库位模型中水平面二维预设障碍物的分布结果查找表征空位库位的二维库位；第五处理单元，用于基于二维库位的平面位置，以预设长度作为步长向车库顶面方向增长，组成三维包围盒；第六处理单元，用于在三维包围盒内包含的障碍物的体素个 数超过预设阈值的情况下，确定触碰到障碍物，并记录为库位竖直方向上最大高度值；第七处理单元，用于从最大高度值中取地面分割结果中的高度值作为地面高度值，并将地面高度值作为三维库位的限高值；第八处理单元，用于将二维库位与限高值组成三维库位。
需要说明的是，上述各个模块是可以通过软件或硬件来实现的，对于后者，可以通过以下方式实现，但不限于此：上述模块均位于同一处理器中；或者，上述各个模块以任意组合的形式分别位于不同的处理器中。
实施例3
本申请的实施例还提供了一种存储介质，该存储介质中存储有计算机程序，其中，该计算机程序被设置为运行时执行上述任一项方法实施例中的步骤。
在一实施例中，存储介质可以被设置为存储用于执行以下步骤的计算机程序：
S1，基于由不同朝向的多个时间飞行ToF相机间的相对位置以及所述多个ToF相机的RGB图像建立三维体素概率地图；
S2，根据三维体素概率地图和预设障碍物的垂直高度分布特征确定预设障碍物的语义；
S3，根据预设库位模型和确定的预设障碍物的语义提取可停靠的库位的二维库位，并结合三维体素概率地图确定与二维库位对应的三维库位。
在一实施例中，存储介质可以包括但不限于：U盘、只读存储器(Read-Only Memory，简称为ROM)、随机存取存储器(Random Access Memory，简称为RAM)、移动硬盘、磁碟或者光盘等各种可以存储计算机程序的介质。
本申请的实施例还提供了一种电子装置，包括存储器和处理器，该存储器中存储有计算机程序，该处理器被设置为运行计算机程序以执行上述任一项方法实施例中的步骤。
在一实施例中，电子装置还可以包括传输设备以及输入输出设备，其中，该传输设备和上述处理器连接，该输入输出设备和上述处理器连接。
在一实施例中，处理器可以被设置为通过计算机程序执行以下步骤：
S1，基于由不同朝向的多个时间飞行ToF相机间的相对位置以及所述多个ToF相机的RGB图像建立三维体素概率地图；
S2，根据三维体素概率地图和预设障碍物的垂直高度分布特征确定预设障碍物的语义；
S3，根据预设库位模型和确定的预设障碍物的语义提取可停靠的库位的二维库位，并结合三维体素概率地图确定与二维库位对应的三维库位。
在一实施例中，具体示例可以参考上述实施例及可选实施方式中所描述的示例，本实施例在此不再赘述。
显然，本领域的技术人员应该明白，上述的本申请的各模块或各步骤可以用通用的计算装置来实现，它们可以集中在单个的计算装置上，或者分布在多个计算装置所组成的网络上，可选地，它们可以用计算装置可执行的程序代码来实现，从而，可以将它们存储在存储装置中由计算装置来执行，并且在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤，或者将它们分别制作成各个集成电路模块，或者将它们中的多个模块或步骤制作成单个集成电路模块来实现。这样，本申请不限制于任何特定的硬件和软件结合。
以上所述仅为本申请的优选实施例而已，并不用于限制本申请，对于本领域的技术人员来说，本申请可以有各种更改和变化。凡在本申请的原则之内，所作的任何修改、等同替换、改进等，均应包含在本申请的保护范围之内。

Claims (14)

  1. 一种库位的检测方法，其中，包括：
    基于由不同朝向的多个时间飞行ToF相机间的相对位置以及所述多个ToF相机的RGB图像建立三维体素概率地图；
    根据所述三维体素概率地图和预设障碍物的垂直高度分布特征确定所述预设障碍物的语义；
    根据预设库位模型和确定的所述预设障碍物的语义提取可停靠的库位的二维库位，并结合所述三维体素概率地图确定与所述二维库位对应的三维库位。
  2. 根据权利要求1所述的方法，其中，所述基于由不同朝向的多个时间飞行ToF相机间的相对位置以及所述多个ToF相机的RGB图像建立三维体素概率地图，包括：
    获取由不同朝向的多个时间飞行ToF相机间的相对位置标定结果将预设范围内的点云拼接成的广角点云；
    基于特征点法确定所述RGB图像中相邻帧间相机自运动以确定前后帧间三维体的运动变化；
    基于所述广角点云中的局部点云以及全局位置建立所述三维体素概率地图，并对所述三维体素概率地图中的体素概率值进行更新。
  3. 根据权利要求2所述的方法，其中，所述获取所述多个ToF相机的RGB图像，并使用特征点法估计相邻帧间相机自运动以确定前后帧间三维体的运动变化，包括：
    通过所述多个ToF相机获取相邻帧的RGB图像，并基于尺度不变特征变换SIFT算法分别提取相邻帧RGB图像中的特征点；
    根据邻近算法对提取出的特征点进行匹配；
    根据特征点的匹配结果计算相邻相机位置的变化以得到变化矩阵，并将所述变化矩阵作为视觉里程计。
  4. 根据权利要求2所述的方法，其中，所述对所述三维体素概率地图中的体素概率值进行更新，包括：
    在体素已被赋值的情况下，后续根据新的每帧点云结果对体素概率值进行更新，其中，更新的方式为：
    Figure PCTCN2020093657-appb-100001
    L new(n)＝max(min(L old(n)+L(n)，,l max),l min)
    其中L new为更新后的网格概率值，L old为更新前的网格概率值，p(n)为由当帧点云划分的概率值，l min与l max为概率限定阈值。
  5. 根据权利要求1所述的方法，其中，所述根据所述三维体素概率地图和预设障碍物的垂直高度分布特征确定所述预设障碍物的语义，包括：
    设置用于进行探测的三个高度区间，其中，所述三个高度区间包括以下至少之一：地面探测区间、底层障碍物探测区间、高层障碍物探测区间；
    对所述三个高度区间进行直通滤波获取与所述三个高度区间对应范围内的体素集合；
    根据与所述不同高度区间对应的体素集合确定所述预设障碍物的分布区间及几何特征对障碍物进行语义判断，以确定所述预设障碍物的语义。
  6. 根据权利要求1所述的方法，其中，所述根据预设库位模型和确定的所述预设障碍物的语义提取可停靠的库位的二维库位，并结合所述三维体素概率地图确定与所述二维库位对应的三维库位，包括：
    根据预设库位模型中水平面二维预设障碍物的分布结果查找表征空位库位的二维库位；
    基于所述二维库位的平面位置，以预设长度作为步长向车库顶面方向增长，组成三维包围盒；
    在所述三维包围盒内包含的障碍物的体素个数超过预设阈值的情况下，确定触碰到障碍物，并记录为库位竖直方向上最大高度值；
    从所述最大高度值中取地面分割结果中的高度值作为地面高度值，并将所述地面高度值作为三维库位的限高值；
    将所述二维库位与所述限高值组成所述三维库位。
  7. 一种库位的检测装置，其中，包括：
    建立模块，用于基于由不同朝向的多个时间飞行ToF相机间的相对位置以及所述多个ToF相机的RGB图像建立三维体素概率地图；
    确定模块，用于根据所述三维体素概率地图和预设障碍物的垂直高度分布特征确定所述预设障碍物的语义；
    处理模块，用于根据预设库位模型和确定的所述预设障碍物的语义提取可停靠的库位的二维库位，并结合所述三维体素概率地图确定与所述二维库位对应的三维库位。
  8. 根据权利要求7所述的装置，其中，所述建立模块包括：
    获取单元，用于获取由不同朝向的多个时间飞行ToF相机间的相对位置标定结果将预设范围内的点云拼接成的广角点云；
    确定单元，用于基于特征点法确定所述RGB图像中相邻帧间相机自运动以确定前后帧间三维体的运动变化；
    处理单元，用于基于所述广角点云中的局部点云以及全局位置建立所述三维体素概率地图，并对所述三维体素概率地图中的体素概率值进行更新。
  9. 根据权利要求8所述的装置，其中，所述确定单元包括：
    第一处理单元，用于通过所述多个ToF相机获取相邻帧的RGB图像，并基于尺度不变特征变换SIFT算法分别提取相邻帧RGB图像中的特征点；
    匹配单元，用于根据邻近算法对提取出的特征点进行匹配；
    第二处理单元，用于根据特征点的匹配结果计算相邻相机位置的变化以得到变化矩阵，并将所述变化矩阵作为视觉里程计。
  10. 根据权利要求8所述的装置，其中，
    所述处理单元，还用于在体素已被赋值的情况下，后续根据新的每帧点云结果对体素概率值进行更新；其中，更新的方式为：
    Figure PCTCN2020093657-appb-100002
    L new(n)＝max(min(L old(n)+L(n),l max),l min)
    其中L new为更新后的网格概率值，L old为更新前的网格概率值，p(n)为由当帧点云划分的概率值，l min与l max为概率限定阈值。
  11. 根据权利要求7所述的装置，其中，所述确定模块包括：
    设置单元，用于设置用于进行探测的三个高度区间，其中，所述三个高度区间包括以下至少之一：地面探测区间、底层障碍物探测区间、高层障碍物探测区间；
    第三处理单元，用于对所述三个高度区间进行直通滤波获取与所述三个高度区间对应范围内的体素集合；
    第四处理单元，用于根据与所述不同高度区间对应的体素集合确定所述预设障碍物的分布区间及几何特征对障碍物进行语义判断，以确定所述预设障碍物的语义。
  12. 根据权利要求7所述的装置，其中，所述处理模块包括：
    查找单元，用于根据预设库位模型中水平面二维预设障碍物的分布结果查找表征空位库位的二维库位；
    第五处理单元，用于基于所述二维库位的平面位置，以预设长度作为步长向车库顶面方向增长，组成三维包围盒；
    第六处理单元，用于在所述三维包围盒内包含的障碍物的体素个数超过预设阈值的情况下，确定触碰到障碍物，并记录为库位竖直方 向上最大高度值；
    第七处理单元，用于从所述最大高度值中取地面分割结果中的高度值作为地面高度值，并将所述地面高度值作为三维库位的限高值；
    第八处理单元，用于将所述二维库位与所述限高值组成所述三维库位。
  13. 一种存储介质，其中，所述存储介质中存储有计算机程序，其中，所述计算机程序被设置为运行时执行所述权利要求1至6任一项中所述的方法。
  14. 一种电子装置，包括存储器和处理器，其中，所述存储器中存储有计算机程序，所述处理器被设置为运行所述计算机程序以执行所述权利要求1至6任一项中所述的方法。
PCT/CN2020/093657 2019-06-11 2020-06-01 库位的检测方法及装置、存储介质和电子装置 WO2020248851A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910502737.6 2019-06-11
CN201910502737.6A CN112070798A (zh) 2019-06-11 2019-06-11 库位的检测方法及装置、存储介质和电子装置

Publications (1)

Publication Number Publication Date
WO2020248851A1 true WO2020248851A1 (zh) 2020-12-17

Family

ID=73658591

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/093657 WO2020248851A1 (zh) 2019-06-11 2020-06-01 库位的检测方法及装置、存储介质和电子装置

Country Status (2)

Country Link
CN (1) CN112070798A (zh)
WO (1) WO2020248851A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140232569A1 (en) * 2013-02-21 2014-08-21 Apple Inc. Automatic identification of vehicle location
CN205230369U (zh) * 2015-12-24 2016-05-11 北京万集科技股份有限公司 一种基于tof相机的停车位检测系统
CN106056643A (zh) * 2016-04-27 2016-10-26 武汉大学 一种基于点云的室内动态场景slam方法及系统
CN107036594A (zh) * 2017-05-07 2017-08-11 郑州大学 智能电站巡检智能体的定位与多粒度环境感知技术
CN108122412A (zh) * 2016-11-26 2018-06-05 沈阳新松机器人自动化股份有限公司 用于监控机器人检测车辆乱停的方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140232569A1 (en) * 2013-02-21 2014-08-21 Apple Inc. Automatic identification of vehicle location
CN205230369U (zh) * 2015-12-24 2016-05-11 北京万集科技股份有限公司 一种基于tof相机的停车位检测系统
CN106056643A (zh) * 2016-04-27 2016-10-26 武汉大学 一种基于点云的室内动态场景slam方法及系统
CN108122412A (zh) * 2016-11-26 2018-06-05 沈阳新松机器人自动化股份有限公司 用于监控机器人检测车辆乱停的方法
CN107036594A (zh) * 2017-05-07 2017-08-11 郑州大学 智能电站巡检智能体的定位与多粒度环境感知技术

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
ZHAO, JUNJIAO ET AL.: "Three-dimensional Parking Slot Detection Using Mapping and Structural Semantics Based on Multiple Time-of-flight Cameras", JOURNAL OF TONGJI UNIVERSITY (NATURAL SCIENCE), vol. 47, no. 4, 6 May 2019 (2019-05-06), pages 1 - 6, XP055764445, ISSN: 0253-374X *

Also Published As

Publication number Publication date
CN112070798A (zh) 2020-12-11

Similar Documents

Publication Publication Date Title
CN110163930B (zh) 车道线生成方法、装置、设备、系统及可读存储介质
CN108763287B (zh) 大规模可通行区域驾驶地图的构建方法及其无人驾驶应用方法
US9483703B2 (en) Online coupled camera pose estimation and dense reconstruction from video
CN111376895B (zh) 一种环视泊车感知方法、装置、自动泊车系统和车辆
US9042639B2 (en) Method for representing surroundings
US20190370989A1 (en) Method and apparatus for 3-dimensional point cloud reconstruction
JP7209115B2 (ja) 複数の相対的に接近する様に動いているリジッドなオブジェクトの検出、３ｄ再現および追跡
WO2020082777A1 (zh) 车位检测方法及装置
CN111213153A (zh) 目标物体运动状态检测方法、设备及存储介质
JP2008262333A (ja) 路面判別装置および路面判別方法
CN114454875A (zh) 一种基于强化学习的城市道路自动泊车方法及系统
CN111507158A (zh) 利用语义分割检测停车区域的方法和装置
WO2021017211A1 (zh) 一种基于视觉的车辆定位方法、装置及车载终端
CN114972758A (zh) 一种基于点云弱监督的实例分割方法
CN113763438B (zh) 一种点云配准方法、装置、设备及存储介质
WO2020248851A1 (zh) 库位的检测方法及装置、存储介质和电子装置
WO2020248118A1 (zh) 点云处理方法、系统、设备及存储介质
CN115164919B (zh) 基于双目相机的空间可行驶区域地图构建方法及装置
CN112150538A (zh) 一种在三维地图构建过程中车辆位姿的确定方法和装置
CN116508071A (zh) 用于注释汽车雷达数据的系统和方法
JP2000222563A (ja) 障害物検出装置および障害物検出装置を搭載した移動体
Habib et al. Integration of lidar and airborne imagery for realistic visualization of 3d urban environments
JPWO2020244717A5 (zh)
KR102540634B1 (ko) 프로젝션 기반 컬러맵 생성 방법 및 이를 실행하기 위하여 기록매체에 기록된 컴퓨터 프로그램
KR102540629B1 (ko) 교통 시설물에 대한 학습 데이터 생성 방법 및 이를 실행하기 위하여 기록매체에 기록된 컴퓨터 프로그램

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20822683

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20822683

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 19.05.2022)

122 Ep: pct application non-entry in european phase

Ref document number: 20822683

Country of ref document: EP

Kind code of ref document: A1