CN115328205B - Flying vehicle takeoff and landing decision planning method based on three-dimensional target detection - Google Patents
Flying vehicle takeoff and landing decision planning method based on three-dimensional target detection Download PDFInfo
- Publication number
- CN115328205B CN115328205B CN202211115814.0A CN202211115814A CN115328205B CN 115328205 B CN115328205 B CN 115328205B CN 202211115814 A CN202211115814 A CN 202211115814A CN 115328205 B CN115328205 B CN 115328205B
- Authority
- CN
- China
- Prior art keywords
- flying
- field
- automobile
- repulsive
- gravitational field
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 29
- 238000001514 detection method Methods 0.000 title claims abstract description 27
- 238000010586 diagram Methods 0.000 claims description 6
- 230000008447 perception Effects 0.000 claims description 5
- 238000012216 screening Methods 0.000 claims description 4
- 230000001174 ascending effect Effects 0.000 claims description 3
- 238000013527 convolutional neural network Methods 0.000 claims description 3
- 230000007613 environmental effect Effects 0.000 claims 1
- 230000009471 action Effects 0.000 description 8
- 230000007812 deficiency Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000000295 complement effect Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 2
- 230000007547 defect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 230000002035 prolonged effect Effects 0.000 description 1
- 238000002310 reflectometry Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/10—Simultaneous control of position or course in three dimensions
- G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Traffic Control Systems (AREA)
- Navigation (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
The invention discloses a flying automobile takeoff and landing decision planning method based on three-dimensional target detection, which is realized based on a camera and a laser radar which are deployed on a flying automobile, and comprises the following steps: establishing a dynamic three-dimensional map according to the real-time acquired image and the three-dimensional point cloud data; when the flying automobile is in a flying state, respectively establishing a repulsive field and a gravitational field to form a real-time changing resultant force field according to the dynamic three-dimensional map, driving the flying automobile to dynamically land, controlling the flying automobile to land when a safe distance is met, and finishing decision planning; otherwise, controlling the flying automobile to ascend, and reestablishing the gravitational field; and when the flying automobile is in a running state, respectively establishing a repulsive force field and a gravitational field to form a real-time changing resultant force field according to the dynamic three-dimensional map, driving the flying automobile to take off, and finishing the decision planning.
Description
Technical Field
The invention belongs to the technical field of automatic driving, and particularly relates to a flying automobile takeoff and landing decision planning method based on three-dimensional target detection.
Background
The american ford automobile founder henne Lifu specifically predicted that a combination of an airplane and an automobile will come into play. With the increasing congestion of urban traffic, the urban air traffic represented by flying cars is a new solution. Several manufacturers will define 2025 years ago as an important time node and the hovercar will be commercialized.
Two problems ensue:
1) How to enable an aerocar with an initial speed to safely merge into a traffic flow from the air on a road of a water horse;
2) How to make an aerocar with an initial speed safely take off from the ground of a train water horse.
The difficulty of merging traffic from the air into the air is much greater than flying from the traffic into the air, and the takeoff field is a simplified form of the landing field.
To solve the above problems, two problems need to be solved:
the common camera can only distinguish the target type for the detection of the target, and has a large position perception error of the target in a three-dimensional space. The laser radar has natural distance advantage for three-dimensional target detection, can directly acquire position information of each point, but lacks visual information with rich images.
Therefore, a plurality of sensors with complementary characteristics are fused to enhance the perception capability, and the three-dimensional target detection method by fusing the image and the point cloud can make up for the deficiency of the depth of the image and the deficiency of the lack of visual information of the point cloud.
The current path planning algorithms are divided into two major categories, namely an algorithm for global path planning represented by an a-x algorithm and a local obstacle avoidance algorithm exemplified by an artificial potential field method. The two algorithms have advantages and disadvantages respectively, the A-x algorithm can obtain a global optimal solution so as to avoid the unmanned aerial vehicle from falling into a local optimal solution, but the A-x algorithm needs to obtain information of the whole map in advance and the resolving time of the A-x algorithm is prolonged along with the increase of the map; the artificial potential field method can quickly respond to the position information of the obstacle, has high reliability, does not depend on the prior information of the environment and the shape of the obstacle, is not influenced by the appearance of the obstacle, but falls into local optimum; specifically, the basic principle of the artificial potential field method is to generate two virtual potential fields during flight: gravitational field, repulsive field. And then, under the combined action of the two potential force fields, different acting forces are generated according to different models of the potential force fields, and the aerocar safely takes off and lands under the action of the forces.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a flying vehicle takeoff and landing decision planning method based on three-dimensional target detection.
In order to achieve the purpose, the invention provides a flying automobile takeoff and landing decision planning method based on three-dimensional target detection, which is realized based on a camera and a laser radar deployed on a flying automobile, and comprises the following steps:
step 1) carrying out environment perception on a specified road according to a real-time collected image and three-dimensional point cloud data, and establishing a dynamic three-dimensional map;
step 2) when the hovercar to be planned by decision-making is in a flight state, turning to step 3); when the flying automobile to be planned by the decision-making is in a running state, turning to the step 5);
step 3) establishing a repulsive field according to the dynamic three-dimensional map by combining the data of the flying automobile and the condition of the automobile running on a one-way road, and selecting a certain point behind the running automobile or a certain point on the ground to establish a gravitational field;
step 4), forming a real-time changing resultant force field by the repulsive force field and the gravitational field, driving the hovercar to dynamically land, judging whether the distance between the hovercar and the target position meets the safety distance, if so, controlling the hovercar to land, and turning to the step 7); if not, controlling the aerocar to ascend, and turning to the step 3) to reestablish the gravitational field;
step 5) according to the dynamic three-dimensional map, combining the data of the aerocar and the situation of the car running on a one-way road, establishing a repulsive field, and selecting an air safety position to establish a gravitational field;
step 6), forming a real-time changing resultant force field by the repulsive force field and the gravitational field, driving the aerocar to take off, and turning to the step 7);
and 7) finishing decision planning.
As a modification of the above method, the step 1) includes:
step 1-1), processing the acquired image by a convolutional neural network, and extracting image pyramid characteristics to obtain an image characteristic diagram with the same size as the initial image;
step 1-2) carrying out joint calibration on the obtained three-dimensional point cloud and the image characteristic map to obtain a point cloud in an image range and obtain corresponding image characteristics;
step 1-3) carrying out voxel meshing on the fused point cloud image data according to the distribution of the point cloud to obtain voxel data;
step 1-4) screening the voxelized data, removing empty grids, arranging the length, the width and the height into one dimension according to a sequence to obtain the processed voxelized data,
step 1-5) inputting the processed voxelized data into a data coding network to obtain a characteristic diagram;
step 1-6) passing the feature map through a single-stage target detection network to obtain the coordinate, length, width and height information of the three-dimensional target of the specified road;
and 1-7) establishing a dynamic three-dimensional map according to the detected coordinate, length, width and height information of each three-dimensional target.
As an improvement of the above method, the data encoding network of step 1-5) comprises: a full connection layer and a VoxelNet; the processing of the data encoding network includes:
the processed voxelized data comprises a plurality of non-empty grids, each grid comprises a plurality of points, one point is extracted from each grid to represent the grid, one grid is selected from one height direction to represent the height, the size of a feature map with the length L and the width W is obtained, C is a feature number of the feature map, and features are expanded through ascending dimensions.
As a modification of the above method, the step 3) includes:
step 3-1) extracting data of the aerocar to be planned by decision, and establishing a repulsive force field after analysis and processing;
repulsive force field U acting on aerocar ri (X) and repulsive force F r (X) satisfies the following formula:
wherein k is r Representing a direct proportionality coefficient, X representing the position of the hovercar to be planned by the decision, X i Indicates the position, η, of the obstacle i i (X,X i ) Denotes X and X i A distance therebetween, η i (X,X i ) Denotes X and X i A distance therebetween, η 0，i Represents the maximum distance acted by the repulsive force field of the ith obstacle;
wherein N represents the total number of obstacles, F ri (X) represents a repulsive force of the obstacle i, satisfying the following formula:
wherein,
expression η i A gradient;
step 3-2) combining the conditions of the front and rear vehicles of the flying vehicle to be planned by decision-making to establish a gravitational field;
gravitational field U acting on aerocar a (X) and gravitational force F a (X) satisfies the following formula:
where ρ represents a gravity direct proportionality coefficient, X g Representing the target position, η (X, X) g ) Indicating the distance between the flying car and the target location,
representing the η gradient.
As an improvement of the above method, the step 4) judges whether the distance between the hovercar and the target position meets a safety distance; the method comprises the following steps:
and when the distance between the front vehicle and the rear vehicle is greater than or equal to the safety distance, the safety distance is met, otherwise, the safety distance is not met.
As a modification of the above method, the step 5) includes:
step 5-1) extracting automobile data running on a one-way road and flight data of an aerocar to be planned by decision, and establishing a repulsion field after analysis and processing;
repulsive force field U acting on aerocar ri (X) and repulsive force F r (X) satisfies the following formula：
Wherein k is r Expressing a direct proportionality coefficient, X expressing the position of the hovercar to be planned by decision, X i Indicates the position, η, of the obstacle i i (X,X i ) Denotes X and X i A distance therebetween, η i (X,X i ) Denotes X and X i A distance therebetween, η 0，i Represents the maximum distance acted by the repulsive force field of the ith obstacle;
step 5-2) selecting an air safety position to establish a gravitational field;
gravitational field U acting on aerocar a (X) and gravitational force F a (X) satisfies the following formula:
where ρ represents a gravitational direct proportionality coefficient, X g Representing the target position, η (X, X) g ) Indicating the distance between the flying car and the target location,
representing the η gradient.
A flying automobile takeoff and landing decision planning system based on three-dimensional target detection is realized based on a camera and a laser radar which are deployed on a flying automobile, and is characterized by comprising: the system comprises a dynamic three-dimensional map building module, a state judging module, a flight repulsive gravitational field building module, a landing control module, a driving repulsive gravitational field building module and a take-off control module;
the dynamic three-dimensional map building module is used for sensing the environment of the specified road according to the image and the three-dimensional point cloud data collected in real time and building a dynamic three-dimensional map;
the state judgment module is used for switching to a flight repulsive gravitational field establishment module when the flying automobile to be planned is in a flight state; when the hovercar to be planned by decision-making is in a running state, the hovercar is transferred to a running repulsive gravitational field establishing module;
the flight repulsion gravitational field establishing module is used for establishing a repulsion field according to the dynamic three-dimensional map by combining the data of the flying automobile and the condition of the automobile running on a one-way road, and selecting a certain point behind the running automobile or a certain point on the ground to establish a gravitational field;
the landing control module is used for forming a real-time changing resultant force field by a repulsive field and a gravitational field, driving the flying automobile to dynamically land, judging whether the distance between the flying automobile and the target position meets a safety distance, if so, controlling the flying automobile to land and finishing decision planning; if not, controlling the flying automobile to ascend, and transferring to a flying repulsive gravitational field establishing module to reestablish a gravitational field;
the driving repulsive gravitational field establishing module is used for establishing a repulsive gravitational field according to the dynamic three-dimensional map by combining the flying automobile data and the automobile condition driven on a one-way road, and selecting an air safety position to establish a gravitational field;
and the takeoff control module is used for forming a real-time changing resultant force field by the repulsion field and the gravitational field, driving the aerocar to take off and finishing decision planning.
Compared with the prior art, the invention has the advantages that:
1. the invention fuses a plurality of sensors with complementary characteristics to enhance perception capability, and the three-dimensional target detection method by fusing the image and the point cloud can not only make up the depth deficiency of the image but also make up the deficiency of the point cloud lack of visual information;
2. establishing a dynamic three-dimensional map according to vehicle coordinates and vehicle length, width and height information provided by a three-dimensional target detection method of image and point cloud fusion, wherein the coordinates of a vehicle in a space and the vehicle length, width and height information are embodied in the dynamic three-dimensional map in real time;
3. and establishing a gravitational field and a repulsive force field based on the dynamic three-dimensional map, and landing and taking off the flying automobile under the action of the real-time changing potential field.
Drawings
FIG. 1 is a landing control decision flow chart of a flying vehicle takeoff and landing decision planning method based on three-dimensional target detection according to the invention;
FIG. 2 is a takeoff control decision flow chart of the flying vehicle takeoff and landing decision planning method based on three-dimensional target detection.
Detailed Description
In order to solve the problems, the invention provides a flying automobile takeoff and landing decision planning method based on three-dimensional target detection. And a dynamic three-dimensional map established by utilizing the data detected by the three-dimensional target enables the flying automobile with the initial speed to safely merge into the traffic flow from the air or enables the flying automobile with the initial speed to take off from the traffic flow under the action of the dynamic potential field.
The technical solution of the present invention is described in detail below with reference to the accompanying drawings and examples.
Example 1
The embodiment 1 of the invention provides a flying car takeoff and landing decision planning method based on three-dimensional target detection. After the road information is acquired through the camera and the laser radar, the flying automobile is safely converged into a motor vehicle lane from the air in a city.
The method for detecting the dynamic landing field based on the three-dimensional target mainly comprises the following steps,
as shown in fig. 1, the dynamic landing site is implemented by the following steps:
step 1: and processing the image acquired by the camera through a convolutional neural network, and extracting the image feature map with the same size as the initial image through an image pyramid.
Image (h, w, 3) represents w pixels long and h pixels wide, each pixel having 3 channels of RGB; image feature map (h, w, n) c ) Representing pixels w pixels long and h pixels wide, each pixel having n c A channel;
step 2: coordinate transformation matrix after three-dimensional point cloud obtained by laser radar is subjected to combined calibration, coordinate transformation is carried out on the three-dimensional point cloud to an image coordinate system for projection, each point cloud in the image size range obtains characteristics of corresponding pixel points, finally point clouds in the image range are obtained, and corresponding image characteristics are obtained
The three-dimensional point cloud (n, 4) represents that n point clouds are obtained in one frame, each point cloud has four channels (x, y, z and r), wherein (x, y and z) are three-dimensional coordinates of the point clouds, and r is the reflectivity of the point clouds; image feature (n) r ,4+n c ) Represents n r A point within the image, 4+n c Image feature n representing original 4 (x, y, z, r) characteristics of point cloud and then splicing corresponding points c
And 3, step 3: carrying out voxel meshing on the fused point cloud image data according to the distribution of the point cloud to obtain voxel data;
voxelized data (L, W, H, N, 4+n) c ) Representing L grids on the long side, W grids on the wide side, H grids on the height, N point cloud image data in each grid, and 4+n data c A characteristic channel, and N has a maximum limit, if N points are exceeded in the grid, according to the xyz coordinates of the points, according to the distance from the origin
And performing ascending arrangement, and taking the first N points.
And 4, step 4: and screening the voxelized data, removing empty grids, and arranging the length, the width and the height into a one-dimensional array according to a sequence to obtain the voxelized data.
Post-screening voxelization data (K, N, 4+n) c Represents K non-spacebars, N data points per grid, data 4+n c A feature channel.
And 5: and inputting the processed voxelized data into a data coding network to obtain a feature map.
The data coding network comprises a full connection layer and a VoxelNet; the data coding network extracts a point in each grid to represent the grid, selects a grid in a height direction to represent the height, and obtains the size of a characteristic diagram with the length of L and the width of W, wherein C is the characteristic number of the characteristic diagram, and the characteristic is expanded through dimension increasing.
Step 6: and (x, y, z, ry, l, w, h and s) of the three-dimensional target are directly output by the final output layer through the single-stage target detection network.
The (x, y, z, ry, l, w, h, s) of the three-dimensional object represents the (x, y, z) coordinates of the object oriented at an angle ry to the x-axis of the origin coordinate system, the length, width, height (l, w, h) of the object, and the confidence score s.
Step 7, establishing a dynamic three-dimensional map according to (x, y, z, ry, l, w, h, s) of the detected automobile three-dimensional target
And 8, extracting data of the automobile running on the one-way road, and establishing a repulsive force potential field after analyzing and processing the data of the automobile running on the one-way road.
Repulsive force field and repulsive force acting on the flying car:
wherein k is r >0, denotes the direct proportionality coefficient, X denotes the hovercar position, X i Indicates the position, η, of the obstacle i 0，i Represents the maximum distance over which the ith obstacle repulsive force field acts,
representing a gradient
Step 9, the decision system selects a certain point behind the running automobile or a certain point on the ground to establish a gravitational field
Gravitational field and gravitational force acting on the flying car:
where ρ is>0, represents the gravitational direct proportionality coefficient, X represents the hovercar position, X g Indicating the position of the target, eta (X, X) g ) Indicating the distance between the flying car and the target.
Representing a gradient
And step 10, forming a real-time changing resultant force field by the gravitational field and the repulsive force field, and dynamically landing the hovercar under the action of the real-time changing resultant force field.
Step 11 if η (X) exists 1 ，X 2 )<And (4) a safe distance, namely, the flying automobile ascends to a safe position under the action of the gravitational field and the repulsive field, and returns to the step 8 to establish the gravitational field again. If eta (X) 1 ，X 2 ) The safe distance is larger than or equal to the safe distance, and the flying automobile safely lands under the action of a gravitational field and a repulsive field.
X 1 Indicating the position of the preceding vehicle, X 2 Indicating the position of the rear vehicle, eta (X) 1 ，X 2 ) Indicating the direct distance between the front and rear vehicles.
As shown in fig. 2, the specific implementation steps of the dynamic takeoff field are as follows:
the steps 1-7 are consistent with the landing field;
and 8, extracting data of the automobile and the air galloping vehicle running on the one-way road, analyzing and processing the data of the automobile and the air galloping vehicle running on the one-way road, and establishing a repulsive force potential field.
Repulsive force field and repulsive force acting on the flying automobile:
wherein k is r >0, represents a direct scaling factor, X represents the hovercar position, X i Indicates the position, η, of the obstacle i 0，i Represents the maximum distance over which the ith obstacle repulsive field acts,
expressing the gradient, it is to be noted that the formula is the same for the repulsive fields for takeoff and landing, but k r The values are different.
9, selecting an air safety position by a decision system to establish a gravitational field;
gravitational field and gravitational force acting on a flying automobile:
where ρ is>0, represents the gravitational direct proportionality coefficient, X represents the hovercar position, X g Indicating the position of the target, eta (X, X) g ) Indicating the distance between the flying car and the target.
Representing a gradient
And step 10, forming a real-time changing resultant force field by the gravitational field and the repulsive force field, and dynamically taking off the aerocar under the action of the real-time changing resultant force field.
Example 2
The embodiment 2 of the invention provides a flying automobile takeoff and landing decision planning system based on three-dimensional target detection, which is realized based on a camera and a laser radar which are deployed on a flying automobile, and comprises: the system comprises a dynamic three-dimensional map building module, a state judging module, a flight repulsive gravitational field building module, a landing control module, a driving repulsive gravitational field building module and a take-off control module; the method is adopted to realize the method of the embodiment 1,
the dynamic three-dimensional map building module is used for sensing the environment of the specified road according to the image and the three-dimensional point cloud data collected in real time and building a dynamic three-dimensional map;
the state judgment module is used for switching to the flying repulsive gravitational field establishing module when the flying automobile to be planned by decision is in a flying state; when the flying automobile to be planned is in a driving state, the flying automobile is switched to a driving repulsive gravitational field establishing module;
the flight repulsive gravitational field establishing module is used for establishing a repulsive field according to the dynamic three-dimensional map by combining the data of the flying automobile and the condition of the automobile running on a one-way road, and selecting a certain point behind the running automobile or a certain point on the ground to establish a gravitational field;
the landing control module is used for forming a real-time changing resultant force field by a repulsive field and a gravitational field, driving the flying automobile to dynamically land, judging whether the distance between the flying automobile and the target position meets a safety distance, if so, controlling the flying automobile to land and finishing decision planning; if not, controlling the flying automobile to ascend, and transferring to a flying repulsive gravitational field establishing module to reestablish a gravitational field;
the driving repulsive gravitational field establishing module is used for establishing a repulsive gravitational field according to the dynamic three-dimensional map by combining the flying automobile data and the automobile condition driven on a one-way road, and selecting an air safety position to establish a gravitational field;
the takeoff control module is used for forming a real-time changing resultant force field by the repulsive force field and the gravitational field, driving the aerocar to take off and finishing decision planning
Finally, it should be noted that the above embodiments are only used for illustrating the technical solutions of the present invention and are not limited. Although the present invention has been described in detail with reference to the embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (7)
1. A flying car takeoff and landing decision-making planning method based on three-dimensional target detection is realized based on a camera and a laser radar which are deployed on a flying car, and comprises the following steps:
step 1) carrying out environmental perception on a specified road according to a real-time acquired image and three-dimensional point cloud data, and establishing a dynamic three-dimensional map;
step 2) when the hovercar to be planned by decision-making is in a flight state, turning to step 3); when the flying automobile to be planned by the decision-making is in a running state, turning to the step 5);
step 3) establishing a repulsive field according to the dynamic three-dimensional map by combining the data of the flying automobile and the condition of the automobile running on a one-way road, and selecting a certain point behind the running automobile or a certain point on the ground to establish a gravitational field;
step 4), forming a real-time changing resultant force field by the repulsive force field and the gravitational field, driving the hovercar to dynamically land, judging whether the distance between the hovercar and the target position meets the safety distance, if so, controlling the hovercar to land, and turning to the step 7); if not, controlling the aerocar to ascend, and turning to the step 3) to reestablish the gravitational field;
step 5) establishing a repulsive field according to the dynamic three-dimensional map by combining the data of the aerocar and the situation of the car running on a one-way road, and selecting an air safety position to establish a gravitational field;
step 6), forming a real-time changing resultant force field by the repulsive force field and the gravitational field, driving the aerocar to take off, and turning to the step 7);
and 7) finishing decision planning.
2. The flying vehicle takeoff and landing decision planning method based on three-dimensional target detection as claimed in claim 1, wherein the step 1) comprises:
step 1-1), processing the acquired image by a convolutional neural network, and extracting image pyramid characteristics to obtain an image characteristic diagram with the same size as the initial image;
step 1-2) carrying out combined calibration on the obtained three-dimensional point cloud and the image characteristic graph to obtain a point cloud in an image range and obtain corresponding image characteristics;
step 1-3) carrying out voxel meshing on the fused point cloud image data according to the distribution of the point cloud to obtain voxel data;
step 1-4) screening the voxelized data, removing empty grids, arranging the length, the width and the height into a dimension according to a sequence to obtain the processed voxelized data,
step 1-5) inputting the processed voxelized data into a data coding network to obtain a characteristic diagram;
step 1-6) passing the feature map through a single-stage target detection network to obtain the coordinate, length, width and height information of the three-dimensional target of the specified road;
and 1-7) establishing a dynamic three-dimensional map according to the detected coordinate, length, width and height information of each three-dimensional target.
3. The flying vehicle takeoff and landing decision-making planning method based on three-dimensional target detection as claimed in claim 2, wherein the data coding network of the step 1-5) comprises: a full connection layer and a VoxelNet; the processing of the data encoding network includes:
the processed voxelized data comprises a plurality of non-empty grids, each grid comprises a plurality of points, one point is extracted from each grid to represent the grid, one grid is selected from one height direction to represent the height, the size of a feature map with the length L and the width W is obtained, C is a feature number of the feature map, and features are expanded through ascending dimensions.
4. The flying vehicle takeoff and landing decision-making planning method based on three-dimensional target detection as claimed in claim 1, wherein the step 3) comprises:
step 3-1) extracting data of the aerocar to be planned by decision, and establishing a repulsive force field after analysis and processing;
repulsive force field U acting on aerocar ri (X) and repulsive force F r (X) satisfies the following formula:
wherein k is r Expressing a direct proportionality coefficient, X expressing the position of the hovercar to be planned by decision, X i Indicates the position, η, of the obstacle i i (X,X i ) Denotes X and X i A distance therebetween, η i (X,X i ) Represents X and X i A distance therebetween, η 0，i Represents the maximum distance acted by the repulsive force field of the ith obstacle;
wherein N represents the total number of obstacles, F ri (X) represents a repulsive force of the obstacle i, satisfying the following formula:
wherein,
expression η i A gradient;
step 3-2) establishing a gravitational field by combining the conditions of the front and rear vehicles of the flying vehicle to be planned by decision;
gravitational field U acting on aerocar a (X) and gravitational force F a (X) satisfies the following formula:
where ρ represents a gravitational direct proportionality coefficient, X g Representing the target position, η (X, X) g ) Indicating the distance between the flying car and the target location,
representing the η gradient.
5. The flying automobile takeoff and landing decision planning method based on three-dimensional target detection as claimed in claim 1, wherein the step 4) is to determine whether the distance between the flying automobile and the target position meets a safe distance; the method comprises the following steps:
and when the distance between the front vehicle and the rear vehicle is greater than or equal to the safety distance, the safety distance is met, otherwise, the safety distance is not met.
6. The flying vehicle takeoff and landing decision-making planning method based on three-dimensional target detection as claimed in claim 1, wherein the step 5) comprises:
step 5-1) extracting the data of the automobile running on a one-way road and the flight data of the aerocar to be planned by decision, and establishing a repulsive field after analysis and processing;
repulsive force field U acting on aerocar ri (X) and repulsive force F r (X) satisfies the following formula:
wherein k is r Expressing a direct proportionality coefficient, X expressing the position of the hovercar to be planned by decision, X i Indicates the position, η, of the obstacle i i (X,X i ) Denotes X and X i A distance therebetween, η i (X,X i ) Denotes X and X i A distance therebetween, η 0，i Represents the maximum distance over which the ith obstacle repulsive field acts;
step 5-2) selecting an air safety position to establish a gravitational field;
gravitational field U acting on aerocar a (X) and gravitational force F a (X) satisfies the following formula:
where ρ represents a gravitational direct proportionality coefficient, X g Representing the target position, η (X, X) g ) Indicating the distance between the flying car and the target location,
representing the η gradient.
7. A flying car take-off and landing decision-making planning system based on three-dimensional target detection is realized based on a camera and a laser radar which are deployed on a flying car, and is characterized by comprising the following components: the system comprises a dynamic three-dimensional map building module, a state judging module, a flight repulsive gravitational field building module, a landing control module, a driving repulsive gravitational field building module and a take-off control module;
the dynamic three-dimensional map building module is used for sensing the environment of the specified road according to the image and the three-dimensional point cloud data collected in real time and building a dynamic three-dimensional map;
the state judgment module is used for switching to a flight repulsive gravitational field establishment module when the flying automobile to be planned is in a flight state; when the flying automobile to be planned is in a driving state, the flying automobile is switched to a driving repulsive gravitational field establishing module;
the flight repulsive gravitational field establishing module is used for establishing a repulsive field according to the dynamic three-dimensional map by combining the data of the flying automobile and the condition of the automobile running on a one-way road, and selecting a certain point behind the running automobile or a certain point on the ground to establish a gravitational field;
the landing control module is used for forming a real-time changing resultant force field by the repulsive force field and the gravitational field, driving the flying automobile to dynamically land, judging whether the distance between the flying automobile and the target position meets the safety distance, if so, controlling the flying automobile to land, and finishing decision planning; if not, controlling the flying automobile to ascend, and transferring to a flying repulsive gravitational field establishing module to reestablish a gravitational field;
the driving repulsive gravitational field establishing module is used for establishing a repulsive gravitational field according to the dynamic three-dimensional map by combining the flying automobile data and the automobile condition driven on a one-way road, and selecting an air safety position to establish a gravitational field;
and the takeoff control module is used for forming a real-time changing resultant force field by the repulsion field and the gravitational field, driving the aerocar to take off and finishing decision planning.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211115814.0A CN115328205B (en) | 2022-09-14 | 2022-09-14 | Flying vehicle takeoff and landing decision planning method based on three-dimensional target detection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211115814.0A CN115328205B (en) | 2022-09-14 | 2022-09-14 | Flying vehicle takeoff and landing decision planning method based on three-dimensional target detection |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115328205A CN115328205A (en) | 2022-11-11 |
| CN115328205B true CN115328205B (en) | 2023-04-14 |
Family
ID=83930946
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211115814.0A Active CN115328205B (en) | 2022-09-14 | 2022-09-14 | Flying vehicle takeoff and landing decision planning method based on three-dimensional target detection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115328205B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117664142B (en) * | 2024-02-01 | 2024-05-17 | 山东欧龙电子科技有限公司 | Method for planning flight vehicle path based on three-dimensional map |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102749847A (en) * | 2012-06-26 | 2012-10-24 | 清华大学 | Cooperative landing method for multiple unmanned aerial vehicles |
| CN109521794A (en) * | 2018-12-07 | 2019-03-26 | 南京航空航天大学 | A kind of multiple no-manned plane routeing and dynamic obstacle avoidance method |
| CN112288667B (en) * | 2020-11-02 | 2024-06-07 | 智驾汽车科技(宁波)有限公司 | Three-dimensional target detection method based on fusion of laser radar and camera |
| CN113359810B (en) * | 2021-07-29 | 2024-03-15 | 东北大学 | Unmanned aerial vehicle landing area identification method based on multiple sensors |
| CN114995501A (en) * | 2022-06-13 | 2022-09-02 | 中国电子科技集团公司第五十四研究所 | Unmanned cluster-oriented distributed dynamic path planning method |
-
2022
- 2022-09-14 CN CN202211115814.0A patent/CN115328205B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN115328205A (en) | 2022-11-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2022206942A1 (en) | Laser radar point cloud dynamic segmentation and fusion method based on driving safety risk field | |
| EP3651064B1 (en) | Deep learning for object detection using pillars | |
| CN110362077B (en) | Unmanned vehicle emergency hedge decision making system, method and medium | |
| CN112212872B (en) | End-to-end automatic driving method and system based on laser radar and navigation map | |
| CN112650220B (en) | Automatic vehicle driving method, vehicle-mounted controller and system | |
| KR20210058696A (en) | Sequential fusion for 3d object detection | |
| WO2018020954A1 (en) | Database construction system for machine-learning | |
| US11861784B2 (en) | Determination of an optimal spatiotemporal sensor configuration for navigation of a vehicle using simulation of virtual sensors | |
| CN113792566A (en) | Laser point cloud processing method and related equipment | |
| CN110210384B (en) | Road global information real-time extraction and representation system | |
| CN115328205B (en) | Flying vehicle takeoff and landing decision planning method based on three-dimensional target detection | |
| WO2023092451A1 (en) | Method and apparatus for predicting drivable lane | |
| CN112513876B (en) | Road surface extraction method and device for map | |
| CN112660128A (en) | Apparatus for determining lane change path of autonomous vehicle and method thereof | |
| CN115840404B (en) | Cloud control automatic driving system based on automatic driving special road network and digital twin map | |
| US20230056589A1 (en) | Systems and methods for generating multilevel occupancy and occlusion grids for controlling navigation of vehicles | |
| US20230252903A1 (en) | Autonomous driving system with air support | |
| Tang et al. | Road detection for autonomous truck in mine environment | |
| CN116030445A (en) | Automatic driving real-time three-dimensional target detection method combining point cloud shape characteristics | |
| CN117389288A (en) | Robot obstacle avoidance method and system based on depth image | |
| CN117036374A (en) | Laser radar point cloud segmentation and motion planning method for automatic driving | |
| WO2022246851A1 (en) | Aerial survey data-based testing method and system for autonomous driving perception system, and storage medium | |
| US11544899B2 (en) | System and method for generating terrain maps | |
| CN115985109B (en) | Unmanned mine car environment sensing method and system | |
| CN118798643A (en) | Anti-collision early warning method and system for fan blade transportation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |