CN115328205B - Flying vehicle takeoff and landing decision planning method based on three-dimensional target detection - Google Patents
Flying vehicle takeoff and landing decision planning method based on three-dimensional target detection Download PDFInfo
- Publication number
- CN115328205B CN115328205B CN202211115814.0A CN202211115814A CN115328205B CN 115328205 B CN115328205 B CN 115328205B CN 202211115814 A CN202211115814 A CN 202211115814A CN 115328205 B CN115328205 B CN 115328205B
- Authority
- CN
- China
- Prior art keywords
- field
- flying
- repulsive
- data
- gravitational field
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/10—Simultaneous control of position or course in three dimensions
- G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Traffic Control Systems (AREA)
- Navigation (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
Description
技术领域Technical Field
本发明属于自动驾驶技术领域,尤其涉及一种基于三维目标检测的飞行汽车起飞着陆决策规划方法。The present invention belongs to the technical field of autonomous driving, and in particular relates to a take-off and landing decision-making planning method for a flying car based on three-dimensional target detection.
背景技术Background Art
美国福特汽车创始人亨利·福特曾预言,飞机和汽车的结合体即将面世。随着城市交通拥堵日益严重,而以飞行汽车为代表的城市空中交通是新的解决方案。多家制造商将2025年前后定为重要的时间节点,飞行汽车将实现商业化。Henry Ford, the founder of Ford Motor Company, once predicted that a combination of airplanes and cars would soon be available. As urban traffic congestion becomes increasingly serious, urban air traffic represented by flying cars is a new solution. Many manufacturers have set around 2025 as an important time node for the commercialization of flying cars.
随之产生两个问题:Two questions arise:
1)如何在车水马龙的道路上,让具有初速度的飞行汽车从空中安全汇入车流中;1) How to make a flying car with initial velocity merge safely into the traffic flow on a busy road;
2)如何让具有初速度的飞行汽车从车水马龙的地面安全的起飞。2) How to make a flying car with initial velocity take off safely from the busy ground.
从空中汇入车流中的难度比从车流飞到空中难度大得多,起飞场是着陆场的简化形式。It is much more difficult to merge into traffic from the air than to fly from traffic into the air. The take-off field is a simplified form of the landing field.
为了解决上述问题,需要解决两个问题环境感知和路径规划:In order to solve the above problems, two issues need to be solved: environmental perception and path planning:
普通的摄像头对于目标的检测只能分辨出目标类别,对于目标在三维空间的位置感知误差较大。激光雷达进行三维目标检测具有天然的距离优势,能够直接获取到各个点的位置信息,但其缺乏图像丰富的视觉信息。Ordinary cameras can only distinguish the target category when detecting targets, and the position perception error of the target in three-dimensional space is large. LiDAR has a natural distance advantage in three-dimensional target detection and can directly obtain the position information of each point, but it lacks rich visual information of images.
不同类型的传感器各有优劣,单一传感器无法实现精确高效的检测.为此,将具有互补特性的多种传感器融合以增强感知能力,图像和点云融合的三维目标检测方法既能弥补图像的深度不足也能弥补点云缺乏视觉信息的不足。Different types of sensors have their own advantages and disadvantages. A single sensor cannot achieve accurate and efficient detection. To this end, multiple sensors with complementary characteristics are fused to enhance perception capabilities. The three-dimensional target detection method that fuses images and point clouds can make up for the lack of depth in images as well as the lack of visual information in point clouds.
目前的路径规划算法分为两大类,以A*算法为代表的全局路径规划的算法,和以人工势场法为例的局部避障算法。两种算法各有优劣,A*算法可以求得全局最优解从而避免无人机陷入局部最优解,但是A*算法需要提前获知整个地图的信息且算法随着地图的增大解算时间也会延长;而人工势场法可以快速针对障碍物位置信息做出响应,方法可靠性高,不依赖环境的先验信息和障碍物形状,不受障碍物的外形影响,但是会陷入局部最优;具体来说,人工势场法的基本原理,在飞行过程中生成虚拟的两个势力场:引力场、斥力场。然后,在两个势力场联合的作用下,根据各个势力场的模型不同产生不同的作用力,飞行汽车在这些力的作用下进行安全起飞、着陆。The current path planning algorithms are divided into two categories, the global path planning algorithms represented by the A* algorithm, and the local obstacle avoidance algorithms represented by the artificial potential field method. Both algorithms have their own advantages and disadvantages. The A* algorithm can obtain the global optimal solution to prevent the drone from falling into the local optimal solution, but the A* algorithm needs to know the information of the entire map in advance and the algorithm will take longer to solve as the map increases; while the artificial potential field method can quickly respond to the obstacle position information, the method is highly reliable, does not rely on the prior information of the environment and the shape of the obstacle, and is not affected by the shape of the obstacle, but it will fall into the local optimum; specifically, the basic principle of the artificial potential field method is to generate two virtual force fields during the flight: the gravitational field and the repulsive field. Then, under the joint action of the two force fields, different forces are generated according to the different models of each force field, and the flying car takes off and lands safely under the action of these forces.
发明内容Summary of the invention
本发明的目的在于克服现有技术缺陷,提出了一种基于三维目标检测的飞行汽车起飞着陆决策规划方法。The purpose of the present invention is to overcome the defects of the prior art and propose a flying car take-off and landing decision planning method based on three-dimensional target detection.
为了实现上述目的,本发明提出了一种基于三维目标检测的飞行汽车起飞着陆决策规划方法,基于部署在飞行汽车上的摄像头和激光雷达实现,所述方法包括:In order to achieve the above-mentioned object, the present invention proposes a flying car takeoff and landing decision planning method based on three-dimensional target detection, which is implemented based on a camera and a laser radar deployed on the flying car. The method comprises:
步骤1)根据实时采集的图像和三维点云数据对指定道路进行环境感知,建立动态三维地图;Step 1) Perform environmental perception on a designated road based on real-time collected images and three-dimensional point cloud data to create a dynamic three-dimensional map;
步骤2)当待决策规划的飞行汽车为飞行状态,转至步骤3);当待决策规划的飞行汽车为行驶状态,转至步骤5);Step 2) When the flying car to be planned and decided is in the flying state, go to step 3); when the flying car to be planned and decided is in the driving state, go to step 5);
步骤3)根据动态三维地图,结合飞行汽车数据和行驶在单行路的汽车情况,建立斥力场,选取行驶汽车后某点或地面上某点建立引力场;Step 3) Based on the dynamic three-dimensional map, combined with the flying car data and the situation of cars driving on the one-way road, a repulsive field is established, and a point behind the driving car or a point on the ground is selected to establish a gravitational field;
步骤4)由斥力场和引力场形成实时变化的合力场,驱动飞行汽车进行动态着陆,判断飞行汽车和目标位置间的距离是否满足安全距离,判断为是,控制飞行汽车着陆,并转至步骤7);判断为否,控制飞行汽车上升,并转至步骤3)重新建立引力场;Step 4) The repulsive field and the gravitational field form a real-time changing resultant force field to drive the flying car to perform dynamic landing, and determine whether the distance between the flying car and the target position meets the safety distance. If it is determined to be yes, the flying car is controlled to land and the process goes to step 7); if it is determined to be no, the flying car is controlled to ascend and the process goes to step 3) to re-establish the gravitational field;
步骤5)根据动态三维地图,结合飞行汽车数据和行驶在单行路的汽车情况,建立斥力场,选取空中安全位置建立引力场;Step 5) Based on the dynamic three-dimensional map, combined with the flying car data and the situation of cars traveling on the one-way road, a repulsive field is established, and a safe position in the air is selected to establish a gravitational field;
步骤6)由斥力场和引力场形成实时变化的合力场,驱动飞行汽车起飞,并转至步骤7);Step 6) The repulsive field and the gravitational field form a real-time changing resultant force field to drive the flying car to take off, and then go to step 7);
步骤7)决策规划结束。Step 7) Decision planning is completed.
作为上述方法的一种改进,所述步骤1)包括:As an improvement of the above method, the step 1) comprises:
步骤1-1)将采集的图像经卷积神经网络处理,经过图像金字塔特征提取,得到与初始图像大小相同的图像特征图;Step 1-1) The collected image is processed by a convolutional neural network and image pyramid feature extraction is performed to obtain an image feature map of the same size as the initial image;
步骤1-2)将获取的三维点云和图像特征图经联合标定,得到在图像范围内的点云,并得到对应的图像特征;Step 1-2) The acquired three-dimensional point cloud and image feature map are jointly calibrated to obtain a point cloud within the image range and obtain corresponding image features;
步骤1-3)将融合的点云图像数据按照点云的分布进行体素网格化,得到体素化数据;Step 1-3) voxel-gridding the fused point cloud image data according to the distribution of the point cloud to obtain voxelized data;
步骤1-4)对体素化数据进行筛选,去掉空的网格,将长宽高按秩序排列成一维,得到处理后的体素化数据,Step 1-4) Filter the voxelized data, remove the empty grids, and arrange the length, width, and height in order into one dimension to obtain the processed voxelized data.
步骤1-5)将处理后的体素化数据输入数据编码网络,得到特征图;Step 1-5) Input the processed voxelized data into the data encoding network to obtain a feature map;
步骤1-6)将特征图通过单阶段的目标检测网络,得到指定道路三维目标的坐标及长宽高信息;Step 1-6) The feature map is passed through a single-stage target detection network to obtain the coordinates and length, width and height information of the three-dimensional target on the specified road;
步骤1-7)根据检测到的每个三维目标的坐标及长宽高信息,建立动态三维地图。Step 1-7) Create a dynamic three-dimensional map based on the coordinates and length, width and height information of each detected three-dimensional target.
作为上述方法的一种改进,所述步骤1-5)的数据编码网络包括:全连接层和VoxelNet;所述数据编码网络的处理包括:As an improvement of the above method, the data encoding network of step 1-5) includes: a fully connected layer and VoxelNet; the processing of the data encoding network includes:
处理后的体素化数据包括若干个非空格子,每个格子包括若干个点,从每个格子提取出一个点表示这个格子,一个高度方向选出一个格子表示这个高度,得到L长、W宽的特征图大小,C为特征图特征数,通过升维扩充特征。The processed voxelized data includes several non-empty grids, each grid includes several points, a point is extracted from each grid to represent the grid, and a grid is selected in a height direction to represent the height, and a feature map size of L length and W width is obtained. C is the feature number of the feature map, and the features are expanded by increasing the dimension.
作为上述方法的一种改进,所述步骤3)包括:As an improvement of the above method, the step 3) comprises:
步骤3-1)提取待决策规划飞行汽车的数据,经分析处理后建立斥力场;Step 3-1) extracting the data of the flying car to be planned and analyzing and processing it to establish a repulsive field;
作用在飞行汽车上的斥力场Uri(X)和斥力Fr(X)满足下式:The repulsive field U ri (X) and repulsive force F r (X) acting on the flying car satisfy the following equation:
其中,kr表示正比例系数,X表示待决策规划的飞行汽车位置,Xi表示障碍物i的位置,ηi(X,Xi)表示X与Xi之间的距离,ηi(X,Xi)表示X与Xi之间的距离,η0,i表示第i个障碍物斥力场作用的最大距离;Wherein, kr represents the positive proportional coefficient, X represents the position of the flying car to be planned, Xi represents the position of obstacle i, ηi (X, Xi ) represents the distance between X and Xi , ηi (X, Xi ) represents the distance between X and Xi , and η0 ,i represents the maximum distance of the repulsive field of the i-th obstacle;
其中,N表示障碍物总数,Fri(X)表示障碍物i的斥力,满足下式:Where N represents the total number of obstacles, and Fri (X) represents the repulsive force of obstacle i, satisfying the following formula:
其中,表示ηi梯度;in, represents η i gradient;
步骤3-2)结合待决策规划飞行汽车的前后方车辆情况,建立引力场;Step 3-2) Establish a gravitational field based on the vehicle conditions in front and behind the flying car to be planned;
作用在飞行汽车上的引力场Ua(X)和引力Fa(X)满足下式:The gravitational field U a (X) and gravitational force F a (X) acting on the flying car satisfy the following equation:
其中,ρ表示引力正比例系数,Xg表示目标位置,η(X,Xg)表示飞行汽车和目标位置的距离,表示η梯度。Among them, ρ represents the gravitational proportional coefficient, X g represents the target position, η(X,X g ) represents the distance between the flying car and the target position, represents the gradient of η.
作为上述方法的一种改进,所述步骤4)判断飞行汽车和目标位置间的距离是否满足安全距离;包括:As an improvement of the above method, the step 4) determining whether the distance between the flying car and the target position meets the safety distance comprises:
当前车和后车之间的距离大于等于安全距离时,满足安全距离,否则不满足安全距离。When the distance between the front vehicle and the rear vehicle is greater than or equal to the safety distance, the safety distance is met, otherwise the safety distance is not met.
作为上述方法的一种改进,所述步骤5)包括:As an improvement of the above method, the step 5) comprises:
步骤5-1)提取行驶在单行路的汽车数据以及待决策规划的飞行汽车的飞行数据,经分析处理后建立斥力场;Step 5-1) extracting the data of cars traveling on a one-way road and the flight data of the flying car to be planned and decided, and establishing a repulsive field after analysis and processing;
作用在飞行汽车上的斥力场Uri(X)和斥力Fr(X)满足下式:The repulsive field U ri (X) and repulsive force F r (X) acting on the flying car satisfy the following equation:
其中,kr表示正比例系数,X表示待决策规划的飞行汽车位置,Xi表示障碍物i的位置,ηi(X,Xi)表示X与Xi之间的距离,ηi(X,Xi)表示X与Xi之间的距离,η0,i表示第i个障碍物斥力场作用的最大距离;Wherein, kr represents the positive proportional coefficient, X represents the position of the flying car to be planned, Xi represents the position of obstacle i, ηi (X, Xi ) represents the distance between X and Xi , ηi (X, Xi ) represents the distance between X and Xi , and η0 ,i represents the maximum distance of the repulsive field of the i-th obstacle;
步骤5-2)选取空中安全位置建立引力场;Step 5-2) Select a safe position in the air to establish a gravitational field;
作用在飞行汽车上的引力场Ua(X)和引力Fa(X)满足下式:The gravitational field U a (X) and gravitational force F a (X) acting on the flying car satisfy the following equation:
其中,ρ表示引力正比例系数,Xg表示目标位置,η(X,Xg)表示飞行汽车和目标位置的距离,表示η梯度。Among them, ρ represents the gravitational proportional coefficient, X g represents the target position, η(X,X g ) represents the distance between the flying car and the target position, represents the gradient of η.
一种基于三维目标检测的飞行汽车起飞着陆决策规划系统,基于部署在飞行汽车上的摄像头和激光雷达实现,其特征在于,所述系统包括:动态三维地图构建模块、状态判断模块、飞行斥引力场建立模块、着陆控制模块、行驶斥引力场建立模块和起飞控制模块;A flying car takeoff and landing decision-making planning system based on three-dimensional target detection is implemented based on a camera and a laser radar deployed on the flying car, characterized in that the system includes: a dynamic three-dimensional map construction module, a state judgment module, a flight repulsion gravitational field establishment module, a landing control module, a driving repulsion gravitational field establishment module and a takeoff control module;
所述动态三维地图构建模块,用于根据实时采集的图像和三维点云数据对指定道路进行环境感知,建立动态三维地图;The dynamic three-dimensional map building module is used to perform environmental perception on the designated road based on the real-time collected images and three-dimensional point cloud data to build a dynamic three-dimensional map;
所述状态判断模块,用于当待决策规划的飞行汽车为飞行状态,转至飞行斥引力场建立模块;当待决策规划的飞行汽车为行驶状态,转至行驶斥引力场建立模块;The state judgment module is used to switch to the flight repulsive gravitational field establishment module when the flying car to be decided and planned is in the flying state; and to switch to the driving repulsive gravitational field establishment module when the flying car to be decided and planned is in the driving state;
所述飞行斥引力场建立模块,用于根据动态三维地图,结合飞行汽车数据和行驶在单行路的汽车情况,建立斥力场,选取行驶汽车后某点或地面上某点建立引力场;The flight repulsive gravitational field establishment module is used to establish a repulsive field based on the dynamic three-dimensional map, combined with the flying car data and the situation of cars traveling on a one-way road, and select a point behind the traveling car or a point on the ground to establish the gravitational field;
所述着陆控制模块,用于由斥力场和引力场形成实时变化的合力场,驱动飞行汽车进行动态着陆,判断飞行汽车和目标位置间的距离是否满足安全距离,判断为是,控制飞行汽车着陆,决策规划结束;判断为否,控制飞行汽车上升,并转至飞行斥引力场建立模块重新建立引力场;The landing control module is used to form a real-time changing resultant force field by the repulsive field and the gravitational field, drive the flying car to perform dynamic landing, and judge whether the distance between the flying car and the target position meets the safety distance. If it is judged to be yes, the flying car is controlled to land and the decision-making planning is ended; if it is judged to be no, the flying car is controlled to ascend and the gravitational field is reestablished in the flight repulsive gravitational field establishment module;
所述行驶斥引力场建立模块,用于根据动态三维地图,结合飞行汽车数据和行驶在单行路的汽车情况,建立斥力场,选取空中安全位置建立引力场;The driving repulsive gravitational field establishment module is used to establish a repulsive field based on the dynamic three-dimensional map, combined with the flying car data and the situation of cars driving on a one-way road, and select a safe position in the air to establish a gravitational field;
所述起飞控制模块,用于由斥力场和引力场形成实时变化的合力场,驱动飞行汽车起飞,决策规划结束。The take-off control module is used to form a real-time changing resultant force field from the repulsive field and the gravitational field to drive the flying car to take off, and the decision-making planning is completed.
与现有技术相比,本发明的优势在于:Compared with the prior art, the advantages of the present invention are:
1、不同类型的传感器各有优劣,单一传感器无法实现精确高效的检测.本发明将具有互补特性的多种传感器融合以增强感知能力,图像和点云融合的三维目标检测方法既能弥补图像的深度不足也能弥补点云缺乏视觉信息的不足;1. Different types of sensors have their own advantages and disadvantages. A single sensor cannot achieve accurate and efficient detection. The present invention fuses multiple sensors with complementary characteristics to enhance perception capabilities. The three-dimensional target detection method of image and point cloud fusion can make up for the lack of depth in the image and the lack of visual information in the point cloud.
2、根据图像和点云融合的三维目标检测方法提供的车辆坐标及车辆长宽高信息建立动态三维地图,空间中车辆的坐标及车辆长宽高信息实时体现在动态三维地图中;2. A dynamic 3D map is established based on the vehicle coordinates and length, width and height information provided by the 3D target detection method of image and point cloud fusion. The coordinates and length, width and height information of the vehicle in space are reflected in the dynamic 3D map in real time;
3、基于动态三维地图,建立引力场和斥力场,飞行汽车在实时变化的势场作用下进行着陆和起飞。3. Based on the dynamic three-dimensional map, the gravitational field and repulsive field are established, and the flying car lands and takes off under the action of the real-time changing potential field.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
图1是本发明基于三维目标检测的飞行汽车起飞着陆决策规划方法着陆控制决策流程图;FIG1 is a landing control decision flow chart of a takeoff and landing decision planning method for a flying car based on three-dimensional target detection according to the present invention;
图2本发明基于三维目标检测的飞行汽车起飞着陆决策规划方法起飞控制决策流程图。FIG2 is a takeoff control decision flow chart of the takeoff and landing decision planning method of a flying car based on three-dimensional target detection of the present invention.
具体实施方式DETAILED DESCRIPTION
为解决上述问题,本发明提供一种基于三维目标检测的飞行汽车起飞着陆决策规划方法。利用三维目标检测的数据建立的动态三维地图,使飞行汽车在动态势场的作用下,让具有初速度的飞行汽车从空中安全汇入车流中,或者让具有初速度的飞行汽车从车流中进行起飞。To solve the above problems, the present invention provides a flying car takeoff and landing decision planning method based on three-dimensional target detection. Using the dynamic three-dimensional map established by the data of three-dimensional target detection, the flying car can safely merge into the traffic flow from the air with an initial velocity, or take off from the traffic flow with an initial velocity under the action of the dynamic potential field.
下面结合附图和实施例对本发明的技术方案进行详细的说明。The technical solution of the present invention is described in detail below with reference to the accompanying drawings and embodiments.
实施例1Example 1
本发明的实施例1提出了一种基于三维目标检测的飞行汽车起飞着陆决策规划方法。通过相机和激光雷达获取道路信息后,使得飞行汽车在城市中安全的从空中汇入到机动车道中。Embodiment 1 of the present invention proposes a flying car takeoff and landing decision planning method based on three-dimensional target detection. After obtaining road information through cameras and laser radars, the flying car can safely merge from the air into the motor vehicle lane in the city.
基于三维目标检测动态着陆场的方法主要包括以下步骤,The method for detecting dynamic landing sites based on three-dimensional targets mainly includes the following steps:
如图1所示,动态着陆场具体实施步骤如下:As shown in Figure 1, the specific implementation steps of the dynamic landing field are as follows:
步骤1:摄像头采集的图像经卷积神经网络处理,经过图像金字塔提取得到与初始图像大小相同的图像特征图。Step 1: The image captured by the camera is processed by a convolutional neural network and extracted through an image pyramid to obtain an image feature map of the same size as the initial image.
图像(h,w,3)表示长为w像素,宽为h像素,每个像素拥有RGB3个通道;图像特征图(h,w,nc)表示长为w像素,宽为h像素,每个像素拥有nc个通道;The image (h, w, 3) means that the length is w pixels, the width is h pixels, and each pixel has 3 RGB channels; the image feature map (h, w, n c ) means that the length is w pixels, the width is h pixels, and each pixel has n c channels;
步骤2:激光雷达获取的三维点云经联合标定后的坐标变换矩阵,坐标变换到图像坐标系进行投影,每个在图像大小范围内的点云都获取对应像素点的特征,最终得到在图像范围内的点云,并且获取了对应的图像特征Step 2: The coordinate transformation matrix of the 3D point cloud acquired by the lidar is jointly calibrated and transformed to the image coordinate system for projection. Each point cloud within the image size range obtains the features of the corresponding pixel points, and finally obtains the point cloud within the image range and obtains the corresponding image features.
三维点云(n,4)表示一帧获取到n个点云,每个点云有(x,y,z,r)四个通道,其中(x,y,z)为点云的三维坐标,r为点云的反射率;图像特征(nr,4+nc)表示nr个在图像范围内的点,4+nc表示点云原有的4个(x,y,z,r)特性再拼接上对应点的图像特征nc The 3D point cloud (n, 4) means that n point clouds are acquired in one frame, and each point cloud has four channels (x, y, z, r), where (x, y, z) is the 3D coordinate of the point cloud and r is the reflectivity of the point cloud; the image feature ( nr , 4+ nc ) means nr points within the image range, and 4+ nc means the original 4 (x, y, z, r) features of the point cloud are spliced with the image feature nc of the corresponding point
步骤3:将融合的点云图像数据按照点云的分布进行体素网格化,得到体素化数据;Step 3: voxel-grid the fused point cloud image data according to the distribution of the point cloud to obtain voxelized data;
体素化后的数据(L,W,H,N,4+nc)表示长边L个格子,宽边W个格子,高度H个格子,每个格子内N个点云图像数据,每个数据4+nc个特征通道,并且N有最大限值,若格子内超过N个点,则根据点的xyz坐标,按照距离原点的距离进行升序排列,取前N个点。The voxelized data (L, W, H, N, 4+n c ) represents L grids on the long side, W grids on the wide side, H grids on the height, N point cloud image data in each grid, 4+n c feature channels for each data, and N has a maximum limit. If there are more than N points in the grid, then according to the xyz coordinates of the point, according to the distance from the origin Arrange in ascending order and take the first N points.
步骤4:对体素化数据进行筛选,去掉空的网格,将长宽高按秩序排列成一维,得到的体素化数据。Step 4: Filter the voxelized data, remove the empty grids, and arrange the length, width, and height in order into one dimension to obtain the voxelized data.
筛选后的体素化数据(K,N,4+nc表示K个非空格子,每个格子N个数据点,每个数据4+nc个特征通道。The filtered voxelized data (K, N, 4 + n c represents K non-empty grids, N data points per grid, and 4 + n c feature channels per data.
步骤5:将处理后的体素化数据输入数据编码网络,得到特征图。Step 5: Input the processed voxelized data into the data encoding network to obtain the feature map.
数据编码网络包括全连接层、VoxelNet;数据编码网络在每个格子提取出一个点表示这个格子,一个高度方向选出一个格子表示这个高度,即得到L长、W宽的特征图大小,C为特征图特征数,通过升维扩充特征。The data encoding network includes a fully connected layer and VoxelNet. The data encoding network extracts a point in each grid to represent the grid, and selects a grid in a height direction to represent the height, that is, a feature map size of L length and W width is obtained. C is the number of feature maps, and features are expanded by increasing the dimension.
步骤6:将特征图通过单阶段的目标检测网络,最后输出层直接输出三维目标的(x,y,z,ry,l,w,h,s)。Step 6: Pass the feature map through a single-stage target detection network, and the final output layer directly outputs the three-dimensional target (x, y, z, ry, l, w, h, s).
三维目标的(x,y,z,ry,l,w,h,s)表示目标的(x,y,z)坐标其朝向与原点坐标系x轴的夹角ry,目标的长宽高(l,w,h),以及置信分数s。The (x, y, z, ry, l, w, h, s) of a three-dimensional target represents the (x, y, z) coordinates of the target, the angle ry between its orientation and the x-axis of the origin coordinate system, the length, width and height of the target (l, w, h), and the confidence score s.
步骤7根据检测到汽车三维目标的(x,y,z,ry,l,w,h,s),建立动态三维地图Step 7: Create a dynamic 3D map based on the detected 3D vehicle target (x, y, z, ry, l, w, h, s).
步骤8提取行驶在单行路的汽车的数据,对行驶在单行路的汽车数据进行分析处理后建立斥力势场。Step 8 extracts the data of the car traveling on the one-way road, and establishes a repulsive potential field after analyzing and processing the data of the car traveling on the one-way road.
作用在飞行汽车上的斥力场和斥力:The repulsive field and repulsive force acting on the flying car:
其中kr>0,表示正比例系数,X表示飞行汽车位置,Xi表示障碍物i的位置,η0,i表示第i个障碍物斥力场作用的最大距离,表示梯度Where k r >0, represents the positive proportional coefficient, X represents the position of the flying car, Xi represents the position of obstacle i, η 0,i represents the maximum distance of the repulsive field of the i-th obstacle, Represents the gradient
步骤9决策系统选取行驶汽车后某点或者地面上某点建立引力场Step 9: The decision system selects a point behind the moving car or a point on the ground to establish a gravitational field.
作用在飞行汽车上的引力场和引力:Gravitational field and gravitational force acting on a flying car:
其中,ρ>0,表示引力正比例系数,X表示飞行汽车位置,Xg表示目标的位置,η(X,Xg)表示飞行汽车和目标的距离。表示梯度Among them, ρ>0, represents the gravity proportional coefficient, X represents the position of the flying car, Xg represents the position of the target, and η(X, Xg ) represents the distance between the flying car and the target. Represents the gradient
步骤10引力场和斥力场形成实时变化的合力场,飞行汽车在实时变化的合力场的作用下进行动态着陆。Step 10: The gravitational field and the repulsive field form a resultant force field that changes in real time, and the flying car performs dynamic landing under the action of the resultant force field that changes in real time.
步骤11若存在η(X1,X2)<安全距离,飞行汽车在引力场和斥力场的作用下上升到安全位置,返回步骤8重新进行引力场的建立。若η(X1,X2)≥安全距离,飞行汽车在引力场和斥力场的作用下安全着陆。Step 11: If η(X 1 , X 2 ) < safe distance, the flying car rises to a safe position under the action of the gravitational field and the repulsive field, and returns to step 8 to re-establish the gravitational field. If η(X 1 , X 2 ) ≥ safe distance, the flying car lands safely under the action of the gravitational field and the repulsive field.
X1表示前车的位置,X2表示后车的位置,η(X1,X2)表示前车和后车直接的距离。 X1 represents the position of the front vehicle, X2 represents the position of the rear vehicle, and η( X1 , X2 ) represents the distance between the front vehicle and the rear vehicle.
如图2所示,动态起飞场具体实施步骤如下:As shown in Figure 2, the specific implementation steps of the dynamic take-off field are as follows:
步骤1-7与着陆场一致;Steps 1-7 are consistent with the landing site;
步骤8提取行驶在单行路的汽车和空中飞车的数据,对行驶在单行路的汽车和空中飞车数据进行分析处理后建立斥力势场。Step 8 extracts the data of cars traveling on the one-way road and cars flying in the air, and establishes a repulsive potential field after analyzing and processing the data of cars traveling on the one-way road and cars flying in the air.
作用在飞行汽车上的斥力场和斥力:The repulsive field and repulsive force acting on the flying car:
其中kr>0,表示正比例系数,X表示飞行汽车位置,Xi表示障碍物i的位置,η0,i表示第i个障碍物斥力场作用的最大距离,表示梯度,需要说明,对于起飞和降落斥力场公式相同,但是kr取值不同。Where k r >0, represents the positive proportional coefficient, X represents the position of the flying car, Xi represents the position of obstacle i, η 0,i represents the maximum distance of the repulsive field of the i-th obstacle, Represents the gradient. It should be noted that the formula for the repulsive field is the same for takeoff and landing, but the values of k r are different.
步骤9决策系统选取空中安全位置建立引力场;Step 9: The decision system selects a safe position in the air to establish a gravitational field;
作用在飞行汽车上的引力场和引力:Gravitational field and gravitational force acting on a flying car:
其中,ρ>0,表示引力正比例系数,X表示飞行汽车位置,Xg表示目标的位置,η(X,Xg)表示飞行汽车和目标的距离。表示梯度Among them, ρ>0, represents the gravity proportional coefficient, X represents the position of the flying car, Xg represents the position of the target, and η(X, Xg ) represents the distance between the flying car and the target. Represents the gradient
步骤10引力场和斥力场形成实时变化的合力场,飞行汽车在实时变化的合力场的作用下进行动态起飞。Step 10: The gravitational field and the repulsive field form a resultant force field that changes in real time, and the flying car takes off dynamically under the action of the resultant force field that changes in real time.
实施例2Example 2
本发明的实施例2提出了一种基于三维目标检测的飞行汽车起飞着陆决策规划系统,基于部署在飞行汽车上的摄像头和激光雷达实现,所述系统包括:动态三维地图构建模块、状态判断模块、飞行斥引力场建立模块、着陆控制模块、行驶斥引力场建立模块和起飞控制模块;采用实施例1的方法实现,Embodiment 2 of the present invention proposes a flying car takeoff and landing decision-making planning system based on three-dimensional target detection, which is implemented based on a camera and a laser radar deployed on the flying car. The system includes: a dynamic three-dimensional map construction module, a state judgment module, a flight repulsion field establishment module, a landing control module, a driving repulsion field establishment module and a takeoff control module; the method of embodiment 1 is used to implement it.
所述动态三维地图构建模块,用于根据实时采集的图像和三维点云数据对指定道路进行环境感知,建立动态三维地图;The dynamic three-dimensional map building module is used to perform environmental perception on the designated road based on the real-time collected images and three-dimensional point cloud data to build a dynamic three-dimensional map;
所述状态判断模块,用于当待决策规划的飞行汽车为飞行状态,转至飞行斥引力场建立模块;当待决策规划的飞行汽车为行驶状态,转至行驶斥引力场建立模块;The state judgment module is used to switch to the flight repulsive gravitational field establishment module when the flying car to be decided and planned is in the flying state; and to switch to the driving repulsive gravitational field establishment module when the flying car to be decided and planned is in the driving state;
所述飞行斥引力场建立模块,用于根据动态三维地图,结合飞行汽车数据和行驶在单行路的汽车情况,建立斥力场,选取行驶汽车后某点或地面上某点建立引力场;The flight repulsive gravitational field establishment module is used to establish a repulsive field based on the dynamic three-dimensional map, combined with the flying car data and the situation of cars traveling on a one-way road, and select a point behind the traveling car or a point on the ground to establish the gravitational field;
所述着陆控制模块,用于由斥力场和引力场形成实时变化的合力场,驱动飞行汽车进行动态着陆,判断飞行汽车和目标位置间的距离是否满足安全距离,判断为是,控制飞行汽车着陆,决策规划结束;判断为否,控制飞行汽车上升,并转至飞行斥引力场建立模块重新建立引力场;The landing control module is used to form a real-time changing resultant force field by the repulsive field and the gravitational field, drive the flying car to perform dynamic landing, and judge whether the distance between the flying car and the target position meets the safety distance. If it is judged to be yes, the flying car is controlled to land and the decision-making planning is ended; if it is judged to be no, the flying car is controlled to ascend and the gravitational field is reestablished in the flight repulsive gravitational field establishment module;
所述行驶斥引力场建立模块,用于根据动态三维地图,结合飞行汽车数据和行驶在单行路的汽车情况,建立斥力场,选取空中安全位置建立引力场;The driving repulsive gravitational field establishment module is used to establish a repulsive field based on the dynamic three-dimensional map, combined with the flying car data and the situation of cars driving on a one-way road, and select a safe position in the air to establish a gravitational field;
所述起飞控制模块,用于由斥力场和引力场形成实时变化的合力场,驱动飞行汽车起飞,决策规划结束The take-off control module is used to form a real-time changing resultant force field by the repulsive field and the gravitational field to drive the flying car to take off, and the decision-making planning is completed.
最后所应说明的是,以上实施例仅用以说明本发明的技术方案而非限制。尽管参照实施例对本发明进行了详细说明,本领域的普通技术人员应当理解,对本发明的技术方案进行修改或者等同替换,都不脱离本发明技术方案的精神和范围,其均应涵盖在本发明的权利要求范围当中。Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention is described in detail with reference to the embodiments, it should be understood by those skilled in the art that any modification or equivalent replacement of the technical solutions of the present invention does not depart from the spirit and scope of the technical solutions of the present invention and should be included in the scope of the claims of the present invention.
Claims (7)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202211115814.0A CN115328205B (en) | 2022-09-14 | 2022-09-14 | Flying vehicle takeoff and landing decision planning method based on three-dimensional target detection |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202211115814.0A CN115328205B (en) | 2022-09-14 | 2022-09-14 | Flying vehicle takeoff and landing decision planning method based on three-dimensional target detection |
Publications (2)
Publication Number | Publication Date |
---|---|
CN115328205A CN115328205A (en) | 2022-11-11 |
CN115328205B true CN115328205B (en) | 2023-04-14 |
Family
ID=83930946
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202211115814.0A Active CN115328205B (en) | 2022-09-14 | 2022-09-14 | Flying vehicle takeoff and landing decision planning method based on three-dimensional target detection |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN115328205B (en) |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117664142B (en) * | 2024-02-01 | 2024-05-17 | 山东欧龙电子科技有限公司 | Flying car path planning method based on three-dimensional map |
CN119045335B (en) * | 2024-08-30 | 2025-06-24 | 爱仕恩动力科技(江苏)有限公司 | An adaptive telescopic control system for driving wheels |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102749847A (en) * | 2012-06-26 | 2012-10-24 | 清华大学 | Cooperative landing method for multiple unmanned aerial vehicles |
CN109521794A (en) * | 2018-12-07 | 2019-03-26 | 南京航空航天大学 | A kind of multiple no-manned plane routeing and dynamic obstacle avoidance method |
CN112288667B (en) * | 2020-11-02 | 2024-06-07 | 智驾汽车科技(宁波)有限公司 | Three-dimensional target detection method based on fusion of laser radar and camera |
CN113359810B (en) * | 2021-07-29 | 2024-03-15 | 东北大学 | A multi-sensor based UAV landing area identification method |
CN114995501B (en) * | 2022-06-13 | 2024-11-29 | 中国电子科技集团公司第五十四研究所 | Unmanned cluster-oriented distributed dynamic path planning method |
-
2022
- 2022-09-14 CN CN202211115814.0A patent/CN115328205B/en active Active
Also Published As
Publication number | Publication date |
---|---|
CN115328205A (en) | 2022-11-11 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2022206942A1 (en) | Laser radar point cloud dynamic segmentation and fusion method based on driving safety risk field | |
EP3651064B1 (en) | Deep learning for object detection using pillars | |
CN115328205B (en) | Flying vehicle takeoff and landing decision planning method based on three-dimensional target detection | |
WO2021259344A1 (en) | Vehicle detection method and device, vehicle, and storage medium | |
CN112698302B (en) | Sensor fusion target detection method under bumpy road condition | |
KR20210058696A (en) | Sequential fusion for 3d object detection | |
CN113791621B (en) | Automatic steering tractor and airplane docking method and system | |
CN110648389A (en) | 3D reconstruction method and system for city street view based on cooperation of unmanned aerial vehicle and edge vehicle | |
WO2018020954A1 (en) | Database construction system for machine-learning | |
CN106896353A (en) | A kind of unmanned vehicle crossing detection method based on three-dimensional laser radar | |
CN105892489A (en) | Multi-sensor fusion-based autonomous obstacle avoidance unmanned aerial vehicle system and control method | |
EP3933439A1 (en) | Localization method and localization device | |
CN118819185A (en) | Intelligent logistics distribution path planning system for drones | |
CN110210384B (en) | A real-time extraction and representation system of road global information | |
CN117440908A (en) | Method and system for pedestrian action prediction based on graph neural network in autonomous driving system | |
JP6578589B2 (en) | Vehicle control device, vehicle control method, and program | |
WO2017038291A1 (en) | Device for automatically producing environmental map | |
CN116661497A (en) | Intelligent aerocar | |
CN107316457A (en) | Judge whether road traffic condition meets the method for automatic driving | |
CN116429145B (en) | Automatic docking navigation method and system for unmanned vehicle and garbage can under complex scene | |
CN112513876B (en) | A kind of road surface extraction method and device for map | |
US12179756B2 (en) | Algorithm to generate planning-based attention signals | |
CN116863687A (en) | Quasi-all-weather traffic safety passing guarantee system based on vehicle-road cooperation | |
US20230252903A1 (en) | Autonomous driving system with air support | |
JP2019185113A (en) | Vehicle control device, vehicle control method, and program |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |