CN112558608B - Vehicle-mounted machine cooperative control and path optimization method based on unmanned aerial vehicle assistance - Google Patents
Vehicle-mounted machine cooperative control and path optimization method based on unmanned aerial vehicle assistance Download PDFInfo
- Publication number
- CN112558608B CN112558608B CN202011460346.1A CN202011460346A CN112558608B CN 112558608 B CN112558608 B CN 112558608B CN 202011460346 A CN202011460346 A CN 202011460346A CN 112558608 B CN112558608 B CN 112558608B
- Authority
- CN
- China
- Prior art keywords
- vehicle
- unmanned aerial
- aerial vehicle
- information
- path
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000005457 optimization Methods 0.000 title claims abstract description 26
- 238000004891 communication Methods 0.000 claims abstract description 41
- 230000000007 visual effect Effects 0.000 claims abstract description 32
- 230000006855 networking Effects 0.000 claims abstract description 5
- 238000012545 processing Methods 0.000 claims description 10
- 101001093748 Homo sapiens Phosphatidylinositol N-acetylglucosaminyltransferase subunit P Proteins 0.000 claims description 8
- 230000004927 fusion Effects 0.000 claims description 7
- 230000008569 process Effects 0.000 claims description 7
- 230000004888 barrier function Effects 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 230000033001 locomotion Effects 0.000 claims description 6
- 230000001133 acceleration Effects 0.000 claims description 5
- 230000006399 behavior Effects 0.000 claims description 4
- 238000005265 energy consumption Methods 0.000 claims description 4
- 238000000605 extraction Methods 0.000 claims description 4
- 238000009499 grossing Methods 0.000 claims description 4
- 238000001514 detection method Methods 0.000 claims description 3
- 230000009471 action Effects 0.000 claims description 2
- 238000004458 analytical method Methods 0.000 claims description 2
- 230000006870 function Effects 0.000 claims description 2
- 239000013589 supplement Substances 0.000 claims description 2
- 101150036841 minJ gene Proteins 0.000 claims 1
- 230000008447 perception Effects 0.000 abstract description 5
- 238000010586 diagram Methods 0.000 description 6
- 230000008901 benefit Effects 0.000 description 5
- 239000013598 vector Substances 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000004347 surface barrier Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/0011—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots associated with a remote control arrangement
- G05D1/0022—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots associated with a remote control arrangement characterised by the communication link
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
- G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0257—Control of position or course in two dimensions specially adapted to land vehicles using a radar
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0276—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle
- G05D1/0278—Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle using satellite positioning signals, e.g. GPS
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/10—Simultaneous control of position or course in three dimensions
- G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- Aviation & Aerospace Engineering (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Electromagnetism (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention relates to a vehicle-machine cooperative control and path optimization method based on unmanned aerial vehicle assistance, and belongs to the field of intelligent transportation and vehicle networking. The vehicle-mounted machine cooperative control comprises an unmanned aerial vehicle and a vehicle; the unmanned aerial vehicle comprises a wireless communication module, a visual sensor module, a GPS positioning module, an air pressure sensor and an Internet of vehicles V2X communication module; the vehicle comprises a wireless communication module, a visual sensor module, a GPS positioning module, a radar, a vehicle-mounted image server and a vehicle networking V2X communication module. The invention combines the close range of the vehicle and the remote traffic state perception of the unmanned aerial vehicle, plans a more reasonable path for the vehicle driving, and improves the driving efficiency and the safety of the vehicle.
Description
Technical Field
The invention belongs to the field of unmanned aerial vehicle control, intelligent transportation and vehicle networking, and relates to a vehicle-mounted machine cooperative control and path optimization method based on unmanned aerial vehicle assistance.
Background
At present, the quantity of vehicles kept continuously increases, and traffic jam becomes an urgent problem to be solved. In the increasingly severe traffic problem solving process, the development of electronic information technology provides new possibility for solving the problems in traffic transportation. In the prior art, the internet is used for providing traffic environment information for users, but the provided traffic environment information has certain hysteresis and is not timely enough to reflect emergencies.
Meanwhile, the unmanned aerial vehicle technology has been developed greatly in recent years, and a vision sensor carried by the unmanned aerial vehicle can acquire wider traffic environment information in real time. Therefore, if the unmanned aerial vehicle acquires wider traffic environment information with higher real-time property under the control of the vehicle and transmits the real-time traffic information to the vehicle through the wireless network, the vehicle plans the path of the vehicle together according to the acquired traffic environment information and the traffic environment information acquired by the unmanned aerial vehicle.
Disclosure of Invention
In view of this, the present invention provides a vehicle-mounted device cooperative control and path optimization method based on unmanned aerial vehicle assistance.
In order to achieve the purpose, the invention provides the following technical scheme:
a vehicle-machine cooperative control and path optimization method based on unmanned aerial vehicle assistance is disclosed, wherein a vehicle-machine cooperative control system comprises an unmanned aerial vehicle and a vehicle;
the unmanned aerial vehicle comprises a wireless communication module, a visual sensor module, a GPS positioning module, an air pressure sensor and an Internet of vehicles V2X communication module;
the vehicle comprises a wireless communication module, a visual sensor module, a GPS positioning module, a radar, a vehicle-mounted image server and a vehicle networking V2X communication module;
the vehicle-mounted image server is used for processing images acquired by the unmanned aerial vehicle and the vehicle, wherein the unmanned aerial vehicle acquires a top view of a front traffic road state, the vehicle acquires a plan view, and structural information of the road and contour information of obstacles are obtained through fusion of the two views;
the unmanned aerial vehicle is provided with a wireless communication module and a V2X communication module, and establishes communication connection with the vehicle through two modes, namely a wireless network and an LTE-V/DSRC;
the first mode is used for sending a control command to the unmanned aerial vehicle by the vehicle and feeding back remote road state image information to the vehicle by the unmanned aerial vehicle;
the second mode is used for sharing the state information of the unmanned aerial vehicle and the vehicle; the unmanned aerial vehicle vision camera is positioned in front of the unmanned aerial vehicle, the angle of the camera is adjustable, and the camera has two degrees of freedom, namely horizontal degree and vertical degree of freedom; the visual image acquired by the unmanned aerial vehicle camera is sent to a vehicle-mounted image server through a UDP datagram; the unmanned aerial vehicle is provided with a GPS communication module for estimating the current position of the unmanned aerial vehicle; the unmanned aerial vehicle is provided with an IMU and a barometer sensor for estimating the attitude of the unmanned aerial vehicle.
Optionally, the information obtained by the unmanned aerial vehicle through the visual sensor is sent to a vehicle in a communication range in real time, and traffic environment information at a far position is collected and sent to the vehicle under the control of the vehicle;
the vehicle control unmanned aerial vehicle's removal to the traffic information in a certain region of pertinence acquisition, acquires traffic environment information through radar and vision sensor, and can handle traffic environment information, through handling the traffic environment information that self gathered with control vehicle safety go forward and through handling the traffic environment information that is gathered by unmanned aerial vehicle in order to carry out route planning and optimization.
Optionally, the path optimization method includes the following steps:
s1: the method comprises the steps that a vehicle sends a take-off instruction to an unmanned aerial vehicle, the unmanned aerial vehicle acquires remote road state information through the visual angle of a visual sensor after taking off and feeds the remote road state information back to the vehicle, and the vehicle starts to execute a track planning algorithm of the vehicle after receiving the road state information sensed by the unmanned aerial vehicle;
s2: the unmanned aerial vehicle acquires GPS position, speed and direction angle information of the vehicle through V2X communication, keeps flying in front of the vehicle and collects road state information, and continuously provides a remote road state image for the vehicle;
s3: the vehicle-mounted image server acquires the visual information of the unmanned aerial vehicle and the vehicle-mounted visual information through a wireless network and a vehicle-mounted local area network respectively, and obtains lane structure information, barrier outline and position information through feature extraction and template matching;
s4: the vehicle-mounted control platform acquires road state information sensed by the vehicle-mounted laser radar through a vehicle-mounted local area network, converts coordinate axes and converts data under a laser radar spherical coordinate system into a rectangular coordinate system;
s5: based on the visual images and the laser radar data in the steps S3 and S4, time and space reference alignment is carried out, and multi-source data fusion is carried out based on D-S theory to obtain structural information of the road and position and contour information of the road surface barrier;
s6: predicting and planning the running path of the vehicle by adopting an LSTM and a fast random search tree algorithm, and obtaining the error of the running track by comparing the predicted running path with the expected running path;
s7: optimizing the vehicle driving path on the basis of the track error in the step S6;
s8: the vehicle sends the instruction of returning a journey to unmanned aerial vehicle, and unmanned aerial vehicle is according to vehicle GPS position and roof landing platform sign, through visual image locking landing point position to relative position and flying height through PID control regulation unmanned aerial vehicle and car, finally realize vehicle trajectory planning and optimization algorithm.
Optionally, step S2 specifically includes:
the vehicle is provided with a GPS module and a V2X communication module, and shares position, speed and direction angle information with the unmanned aerial vehicle through V2X communication; after the unmanned aerial vehicle acquires the information, the unmanned aerial vehicle obtains a control error by combining the position, the speed and the direction angle information of the unmanned aerial vehicle, and the control error is used as the input of the tracking controller, so that the unmanned aerial vehicle moves along with the vehicle, and the remote road state image information is continuously provided for the vehicle.
Optionally, step S4 specifically includes:
obtaining distance information of obstacles in front of a traffic road through a laser radar; the vehicle-mounted control platform obtains the front obstacle information in real time through the online analysis of the radar point cloud data, wherein the conversion relation from the laser radar measuring point coordinate to the rectangular coordinate system is as follows:
wherein the front of the radar is in the x direction, the left is in the y direction, the upper is in the z direction, d is the distance measured by the laser radar, alpha is the included angle between each scanning surface and the horizontal plane, and theta is the rotating angle of the laser radar.
Optionally, step S5 specifically includes:
transmitting far/near road structure and barrier information obtained after processing by the vehicle-mounted image server to a vehicle-mounted control platform through a vehicle-mounted local area network; and the control platform is combined with the laser radar data to further match the radar detection points with the visual images in a rectangular coordinate system, so that the real position of the point in the three-dimensional space is restored, and the object reconstruction in the three-dimensional space is completed.
Optionally, step S6 specifically includes:
generating a smooth track meeting the vehicle running constraint, simultaneously adopting two methods of path prediction and planning to predict the path of the vehicle and plan the path of the vehicle, obtaining the error of the running track according to the difference value of the two methods, and taking the error as one of the bases for optimizing the vehicle running path.
Optionally, step S7 specifically includes:
the path optimization algorithm uses the state information of the vehicle, and comprises the following steps: the position x, the speed v and the acceleration a, the curvature gamma of the running track, the running time t and the energy consumption w are performance constraint indexes, and the following optimal control model is established:
wherein J i And (4) optimizing the driving path planned in the step (S6) according to the optimal control model, wherein the vehicle drives along the optimal path under the action of the controller.
Optionally, step S8 specifically includes:
the unmanned aerial vehicle positions the vehicle through the vehicle GPS position information; then, the unmanned aerial vehicle hovers over the vehicle, and the image information of the roof landing point is acquired through the airborne camera, so that the position of the roof landing point is accurately positioned; the images acquired by the unmanned aerial vehicle are transmitted to the vehicle-mounted image server through a high-bandwidth wireless network, and the vehicle-mounted image server feeds back the results to the unmanned aerial vehicle after processing the images; the unmanned aerial vehicle adjusts the position of the unmanned aerial vehicle based on the result and lands on the roof; the vehicle provides the electric quantity for unmanned aerial vehicle and supplements.
The invention has the beneficial effects that: 1. under the complicated traffic environment, the characteristic that the unmanned aerial vehicle has a wide field of vision is utilized, and the road state of a target area far away from a vehicle can be detected. Simultaneously, on-vehicle vision sensor and lidar can perception road state closely. Therefore, the perception range of the vehicle can be expanded by combining the long-distance road state visual information perceived by the unmanned aerial vehicle and the short-distance road state information perceived by the vehicle through the visual sensor and the laser radar, and the pre-judging capability and safety of vehicle running are enhanced.
2. Through the combination of unmanned aerial vehicle and vehicle, for unmanned aerial vehicle provides reliable removal electric quantity supply platform, because unmanned aerial vehicle's duration is short, can not carry out the large-scale task that requires high to the time length, and the vehicle has the ability of removing the power supply, can provide long-time duration support for unmanned aerial vehicle, improves unmanned aerial vehicle's operation scope, and unmanned aerial vehicle and vehicle have formed the complementary relation of an advantage.
3. The unmanned aerial vehicle and the vehicle are combined, so that the vehicle can acquire remote road state information in real time, a data basis of path planning is provided for the vehicle through the vision of the unmanned aerial vehicle vision sensor and the vehicle and the perception data fusion of the laser radar sensor, optimal control and track prediction are combined, the generated road track is optimized, and the efficiency and the safety of vehicle running can be improved.
Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims thereof.
Drawings
For the purposes of promoting a better understanding of the objects, aspects and advantages of the invention, reference will now be made to the following detailed description taken in conjunction with the accompanying drawings in which:
FIG. 1 is a general architecture diagram of a vehicle-mounted cooperative control and path optimization system based on unmanned aerial vehicle assistance according to a preferred embodiment of the present invention;
FIG. 2 is a data flow diagram of a vehicle travel path planning and optimization method based on unmanned aerial vehicle assistance;
FIG. 3 is a flowchart of the steps executed in the method for planning and optimizing a vehicle driving path based on unmanned aerial vehicle assistance;
FIG. 4 is a schematic diagram of vehicle lidar angle data;
fig. 5 is a schematic diagram of coordinates of an unmanned aerial vehicle body;
FIG. 6 is a flow chart of the relationship between threads of the UAV and the execution of control instructions;
fig. 7 is a flowchart of the unmanned aerial vehicle fixed point landing algorithm.
Detailed Description
The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present invention in a schematic way, and the features in the following embodiments and examples may be combined with each other without conflict.
Wherein the showings are for the purpose of illustrating the invention only and not for the purpose of limiting the same, and in which there is shown by way of illustration only and not in the drawings in which there is no intention to limit the invention thereto; to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of an actual product; it will be understood by those skilled in the art that certain well-known structures in the drawings and descriptions thereof may be omitted.
The same or similar reference numerals in the drawings of the embodiments of the present invention correspond to the same or similar components; in the description of the present invention, it should be understood that if there is an orientation or positional relationship indicated by terms such as "upper", "lower", "left", "right", "front", "rear", etc., based on the orientation or positional relationship shown in the drawings, it is only for convenience of description and simplification of description, but it is not an indication or suggestion that the referred device or element must have a specific orientation, be constructed in a specific orientation, and be operated, and therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes, and are not to be construed as limiting the present invention, and the specific meaning of the terms may be understood by those skilled in the art according to specific situations.
Please refer to fig. 1 to 7, which illustrate a vehicle-mounted device cooperative control and path optimization method based on the assistance of an unmanned aerial vehicle.
A vehicle-machine cooperative control and path optimization method based on unmanned aerial vehicle assistance is shown in fig. 1 as a system overall architecture diagram of an example of the method. The system is composed of a vehicle and an unmanned aerial vehicle, wherein the unmanned aerial vehicle and the vehicle are communicated through a wireless communication module and a DSRC module and cooperate with each other, so that the whole system can stably operate. The vehicle is equipped with wireless communication module, DSRC communication module, radar and vision sensor, and wherein wireless communication module is used for receiving the traffic information that is sent by unmanned aerial vehicle, and DSRC communication module is used for sending unmanned aerial vehicle control command, and radar and vision sensor obtain near the traffic information of current position jointly to keep away the barrier when the vehicle goes. The vehicle can realize storing national electronic maps, and the electronic maps contain longitude and latitude information of roads and non-roads for the navigation of the vehicle and the unmanned aerial vehicle. The unmanned aerial vehicle is equipped with a wireless communication module, a DSRC communication module and a vision sensor, wherein the wireless communication module is used for sending collected traffic conditions, the DSRC communication module is used for receiving control instructions sent by vehicles, and the vision sensor can collect video traffic conditions of a target area from the air.
As shown in fig. 2, a data flow diagram of a vehicle driving path planning and optimizing method based on unmanned aerial vehicle assistance is that, firstly, remote road traffic state information acquired by an unmanned aerial vehicle vision sensor is overlook information under a rectangular coordinate system, specifically includes road structure information and road obstacle information, and is transmitted to a vehicle-mounted image server through a wireless communication module; meanwhile, the vehicle also acquires the nearby road structure and the barrier information on the road through a vehicle-mounted laser radar and a vehicle-mounted vision sensor, and aligns with the space-time reference through coordinate system conversion; then, carrying out multisource data fusion of unmanned aerial vehicle and vehicle perception to obtain numerically quantized road structure information (width and curvature) and distance and contour information of obstacles on the road; further obtaining errors of the expected running path and the actual possible running path of the vehicle through a path planning and vehicle running track prediction algorithm; and finally, inputting the track error, the performance constraint indexes (time and path curvature) and the running path error into a path optimization algorithm to obtain the optimal running path of the vehicle.
The vehicle driving path planning and optimizing method based on unmanned aerial vehicle assistance executes steps, referring to an execution step flow chart of fig. 3, and the specific example is as follows:
step 1: in the vehicle driving process, the vehicle judges that the front road state is abnormal through the vehicle-mounted sensor, but the specific situation of the far-distance traffic state cannot be known due to the limited sensing distance of the vehicle-mounted sensor, at the moment, the vehicle sends a take-off instruction to the unmanned aerial vehicle, the unmanned aerial vehicle flies to a position area appointed by the vehicle after taking off, remote road state information is obtained through a wide visual angle of the high-altitude visual sensor and is fed back to the vehicle through the wireless transmission module, and the vehicle starts to execute a track planning algorithm of the vehicle after receiving the road state information sensed by the unmanned aerial vehicle.
Step 2: in the process that the unmanned aerial vehicle explores traffic conditions in the front, the vehicle may move, at the moment, the unmanned aerial vehicle acquires the GPS position, speed and direction angle information of the vehicle through V2X communication, so that the unmanned aerial vehicle can move along the road along with the movement of the vehicle in a communication range, and can continuously provide road traffic information at a far distance for the vehicle, wherein the information is original traffic image information and can provide warning for the vehicle after being processed by the vehicle-mounted image server.
And 3, step 3: after the vehicle-mounted image server acquires the visual information of the unmanned aerial vehicle and the vehicle-mounted visual information through a wireless network and a vehicle-mounted local area network respectively, feature point extraction is carried out by adopting an HOG method, the image is grayed and subjected to gradient calculation, and feature vector normalization is further carried out to obtain a final feature vector. And comparing the feature vector with template vectors in a feature library, calculating the Euclidean distance of the feature vector, judging the best matching when the distance is the minimum, judging the type of the obstacle on the lane, and obtaining the size of the obstacle through a contour extraction algorithm. Through the judgment of the type of the obstacle, the vehicle can obtain the warning information in advance.
And 4, step 4: the vehicle-mounted control platform acquires road state information sensed by the vehicle-mounted laser radar through a vehicle-mounted local area network, performs coordinate axis conversion, converts data under a laser radar spherical coordinate system into a rectangular coordinate system, and defines laser radar angles as shown in FIG. 4, wherein a specific conversion formula is as follows:
wherein the radar place ahead is the x direction, and the left is the y direction, and the top is the z direction, and d is laser radar measuring distance, and alpha is the contained angle between scanning face and the horizontal plane, and theta is laser radar's rotation angle.
And 5: based on the visual image and the laser radar data in the steps 3 and 4, because the data acquisition frequencies of different sensors are different, the radar point cloud and the road image information acquired by the unmanned aerial vehicle and the vehicle are aligned in a space-time reference manner, firstly, the 3D radar point cloud data is mapped into a 2D radar point map through a formula (2),
wherein p = (x, y, z) T For a point in three-dimensional space, (r, c) is its position projection in 2D space, and θ and α represent the horizontal and vertical angles, as shown in fig. 4, Δ θ and Δ α are the horizontal and vertical accuracies of the laser beam splitter. And further, carrying out multi-source data fusion by adopting a D-S theory, combining a 2D projection image of laser radar point cloud and a color image extracted by a vehicle-mounted camera, assigning the color of the image to a radar point cloud image according to depth information acquired by the laser radar, and further obtaining the structural information of the road and the position and contour information of the road surface barrier.
Step 6: considering factors such as road structure, road surface obstacles, vehicle motion behavior constraints and vehicle current state information, predicting and planning a vehicle running path by adopting an LSTM algorithm and a fast random search tree algorithm respectively, and for predicting a vehicle running track, firstly processing time sequence data of vehicle running:
wherein T is processed data, s is original state data (position, speed and acceleration) of the vehicle, n is a data smoothing parameter which can be adjusted according to needs, then processed time series data is used as an LSTM model to be input, model parameters are trained to obtain a prediction model of vehicle driving behaviors, and the model is adopted to predict vehicle driving tracks in the vehicle driving process; for planning the vehicle running path, a fast random search tree algorithm is adopted, random search is carried out according to the current position of the vehicle, the road structure, the position of an obstacle and other information to obtain a better running path, and track smoothing processing is carried out on the running path to enable the running path to meet the motion rule of vehicle running; and obtaining the error of the running track by comparing the predicted running path with the expected running path.
And 7: on the basis of the step 6, optimizing the vehicle running path by adopting an optimization method (4) according to the error of the running track,
wherein J i Is an objective function that is related to the state of vehicle travel, time constraints and energy consumption constraints, x is the vehicle position, v is the vehicle speed and a is the vehicle acceleration, γ is the vehicle travel trajectory curvature, t is the vehicle travel time and w is the energy consumption of the vehicle. And taking the optimized vehicle running path as the input of the vehicle-mounted control platform, and controlling the vehicle to run according to the output of the vehicle control algorithm.
And 8: the coordinate of the unmanned aerial vehicle is defined as shown in fig. 5, that is, the front of the unmanned aerial vehicle is the positive direction of the y axis, the right is the positive direction of the x axis, and the lower is the positive direction of the z axis. Unmanned aerial vehicle's camera is located under unmanned aerial vehicle, and has two degrees of freedom of level and vertical motion, when carrying out the roof descending algorithm, unmanned aerial vehicle camera orientation is under, and unmanned aerial vehicle's spatial position coordinate and the spatial position coordinate of camera can be regarded as approximately the coincidence promptly. The edge distortion of the camera is not considered, according to experimental data, the distance between the number of the image pixels and the measured object is approximately in a linear relation, namely the ratio of the number of the pixel points reduced to the number of the distance increased is a constant, and the constant is related to the physical structure of the camera and can be obtained through experimental calibration.
The relationship between the threads of the unmanned aerial vehicle control program and the execution flow of the control instruction are shown in fig. 6, wherein a specific roof landing algorithm flow chart is shown in fig. 7, the vehicle sends a return flight instruction to the unmanned aerial vehicle, the unmanned aerial vehicle locks the landing point position through a visual image according to the vehicle GPS position and the roof landing platform identification received by V2X communication, and the roof landing algorithm is started to be executed. The unmanned aerial vehicle transmits the image detected by the camera to the vehicle-mounted server, the server obtains longitudinal deviation delta y and transverse deviation delta x of landing points and an image center through a target detection algorithm, the unmanned aerial vehicle is transmitted to the unmanned aerial vehicle through the wireless transmission module in the form of the number of pixel points, the unmanned aerial vehicle converts the linear relation between the pixel points and the length into a distance error, the relative position and the flying height of the unmanned aerial vehicle and a vehicle are adjusted through PID control, the vehicle finally lands on the roof, and the vehicle track planning and optimization algorithm is terminated.
Finally, although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that various changes and modifications may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (7)
1. A vehicle-machine cooperative control and path optimization method based on unmanned aerial vehicle assistance is characterized in that: the vehicle machine cooperative control system comprises an unmanned aerial vehicle and a vehicle:
the unmanned aerial vehicle comprises a wireless communication module, a visual sensor module, a GPS positioning module, an air pressure sensor and an Internet of vehicles V2X communication module;
the vehicle comprises a wireless communication module, a visual sensor module, a GPS positioning module, a radar, a vehicle-mounted image server and a vehicle networking V2X communication module;
the vehicle-mounted image server is used for processing images acquired by the unmanned aerial vehicle and the vehicle, wherein the unmanned aerial vehicle acquires a top view of a front traffic road state, the vehicle acquires a plan view, and structural information of the road and contour information of obstacles are obtained through fusion of the two views;
the unmanned aerial vehicle is provided with a wireless communication module and a V2X communication module, and establishes communication connection with the vehicle through two modes, namely a wireless network and an LTE-V/DSRC;
the method comprises the steps that a vehicle sends a control command to an unmanned aerial vehicle through a wireless network, and the unmanned aerial vehicle feeds back remote road state image information to the vehicle;
sharing state information of the unmanned aerial vehicle and the vehicle through LTE-V/DSRC; the unmanned aerial vehicle vision camera is positioned in front of the unmanned aerial vehicle, the angle of the camera is adjustable, and the camera has two degrees of freedom, namely horizontal degree and vertical degree of freedom; the visual image acquired by the unmanned aerial vehicle camera is sent to a vehicle-mounted image server through a UDP datagram; the unmanned aerial vehicle is provided with a GPS communication module for estimating the current position of the unmanned aerial vehicle; the unmanned aerial vehicle is provided with an IMU and a barometer sensor and used for estimating the attitude of the unmanned aerial vehicle;
the path optimization method specifically comprises the following steps:
s1: the method comprises the steps that a vehicle sends a take-off instruction to an unmanned aerial vehicle, the unmanned aerial vehicle acquires remote road state information through the visual angle of a visual sensor after taking off and feeds the remote road state information back to the vehicle, and the vehicle starts to execute a track planning algorithm of the vehicle after receiving the road state information sensed by the unmanned aerial vehicle;
s2: the unmanned aerial vehicle acquires GPS position, speed and direction angle information of the vehicle through V2X communication, keeps flying in front of the vehicle and collects road state information, and continuously provides a remote road state image for the vehicle;
s3: the vehicle-mounted image server acquires the visual information of the unmanned aerial vehicle and the vehicle-mounted visual information through a wireless network and a vehicle-mounted local area network respectively, and obtains lane structure information, barrier outline and position information through feature extraction and template matching;
s4: the vehicle-mounted control platform acquires road state information sensed by the vehicle-mounted laser radar through a vehicle-mounted local area network, performs coordinate axis conversion, and converts data under a laser radar spherical coordinate system into a rectangular coordinate system;
s5: based on the visual image and the laser radar data in the steps S3 and S4, time and space reference alignment is carried out, multi-source data fusion is carried out based on D-S theory, and structural information of a road and position and contour information of a road surface obstacle are obtained;
s6: predicting and planning the running path of the vehicle by adopting an LSTM and a fast random search tree algorithm, and obtaining the error of the running track by comparing the predicted running path with the planned running path, wherein the method specifically comprises the following steps:
the method comprises the steps that factors of road structures, road surface obstacles, vehicle motion behavior constraints and current state information of vehicles are considered, and the driving paths of the vehicles are predicted and planned by adopting an LSTM algorithm and a fast random search tree algorithm respectively;
for the prediction of the vehicle running track, firstly, the time series data of the vehicle running is processed as follows:
wherein T represents processed data; s represents raw state data of the vehicle, the raw state data including position, speed, and acceleration; n represents a data smoothing parameter, and is adjusted according to the requirement;
secondly, inputting the processed time series data as an LSTM model, training model parameters to obtain a prediction model of vehicle running behaviors, and predicting the vehicle running track by using the model in the vehicle running process;
for planning the vehicle running path, a fast random search tree algorithm is adopted, random search is carried out according to the current position of the vehicle, the road structure and the position information of the obstacle, a better running path is obtained, and track smoothing processing is carried out on the better running path, so that the better running path meets the motion rule of vehicle running;
obtaining the error of the driving track by comparing the predicted driving path with the expected driving path;
s7: optimizing the vehicle running path on the basis of the track error in the step S6;
s8: the vehicle sends the instruction of returning a journey to unmanned aerial vehicle, and unmanned aerial vehicle is according to vehicle GPS position and roof landing platform sign, through visual image locking landing point position to relative position and flying height through PID control regulation unmanned aerial vehicle and car, finally realize vehicle orbit planning and optimization algorithm.
2. The method for cooperative control and path optimization of the vehicle-mounted machine based on unmanned aerial vehicle assistance, according to claim 1, is characterized in that:
the unmanned aerial vehicle sends information obtained by the visual sensor to vehicles in a communication range in real time, collects traffic environment information at a far position under the control of the vehicles and sends the traffic environment information to the vehicles;
the vehicle controls the unmanned aerial vehicle to move so as to pertinently acquire traffic information of a certain area, acquire traffic environment information through the radar and the vision sensor, process the traffic environment information, control the vehicle to safely drive forwards by processing the traffic environment information acquired by the vehicle and process the traffic environment information acquired by the unmanned aerial vehicle so as to plan and optimize a path.
3. The vehicle-mounted machine cooperative control and path optimization method based on unmanned aerial vehicle assistance as claimed in claim 1, wherein: the step S2 specifically comprises the following steps:
the vehicle is provided with a GPS module and a V2X communication module, and the vehicle shares position, speed and direction angle information with the unmanned aerial vehicle through V2X communication; after the unmanned aerial vehicle acquires the information, the unmanned aerial vehicle obtains a control error by combining the position, the speed and the direction angle information of the unmanned aerial vehicle, and the control error is used as the input of the tracking controller, so that the unmanned aerial vehicle moves along with the vehicle, and the remote road state image information is continuously provided for the vehicle.
4. The vehicle-mounted machine cooperative control and path optimization method based on unmanned aerial vehicle assistance as claimed in claim 1, wherein: the step S4 specifically comprises the following steps:
obtaining distance information of obstacles in front of a traffic road through a laser radar; the vehicle-mounted control platform obtains the front obstacle information in real time through the online analysis of the radar point cloud data, wherein the conversion relation from the laser radar measuring point coordinate to the rectangular coordinate system is as follows:
wherein the front of the radar is in the x direction, the left is in the y direction, the upper is in the z direction, d is the distance measured by the laser radar, alpha is the included angle between each scanning surface and the horizontal plane, and theta is the rotating angle of the laser radar.
5. The method for cooperative control and path optimization of the vehicle-mounted machine based on unmanned aerial vehicle assistance, according to claim 1, is characterized in that: the step S5 specifically comprises the following steps:
transmitting far/near road structure and barrier information obtained after processing by the vehicle-mounted image server to a vehicle-mounted control platform through a vehicle-mounted local area network; and the control platform is combined with the laser radar data to further match the radar detection points with the visual images in the rectangular coordinate system, so that the real position of the midpoint in the three-dimensional space is restored, and the object reconstruction in the three-dimensional space is completed.
6. The vehicle-mounted machine cooperative control and path optimization method based on unmanned aerial vehicle assistance as claimed in claim 1, wherein: the step S7 specifically includes:
the method comprises the following steps of establishing an optimal control model by taking a position x, a speed v and an acceleration a, a running track curvature gamma, a running time t and energy consumption w as performance constraint indexes:
minJ i ＝f(x i ,v i ,a i ,γ i ,w i )
wherein J i Is an objective function, and according to the optimal control model, the driving path planned in the step S6 is optimized, and the vehicle is along the most under the action of the controllerAnd (5) traveling along a good path.
7. The vehicle-mounted machine cooperative control and path optimization method based on unmanned aerial vehicle assistance as claimed in claim 1, wherein: the step S8 is specifically:
the unmanned aerial vehicle positions the vehicle through the vehicle GPS position information; then, the unmanned aerial vehicle hovers over the vehicle, and the image information of the roof landing point is acquired through the airborne camera, so that the position of the roof landing point is accurately positioned; image collected by unmanned aerial vehicle passes through high band
The wide wireless network transmits the image to the vehicle-mounted image server, and the vehicle-mounted image server feeds back the result to the unmanned aerial vehicle after processing the image;
the unmanned aerial vehicle adjusts the position of the unmanned aerial vehicle based on the result and lands on the roof; the vehicle provides the electric quantity for unmanned aerial vehicle and supplements.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011460346.1A CN112558608B (en) | 2020-12-11 | 2020-12-11 | Vehicle-mounted machine cooperative control and path optimization method based on unmanned aerial vehicle assistance |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011460346.1A CN112558608B (en) | 2020-12-11 | 2020-12-11 | Vehicle-mounted machine cooperative control and path optimization method based on unmanned aerial vehicle assistance |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112558608A CN112558608A (en) | 2021-03-26 |
| CN112558608B true CN112558608B (en) | 2023-03-17 |
Family
ID=75062523
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011460346.1A Active CN112558608B (en) | 2020-12-11 | 2020-12-11 | Vehicle-mounted machine cooperative control and path optimization method based on unmanned aerial vehicle assistance |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112558608B (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113189989B (en) * | 2021-04-21 | 2022-07-01 | 东风柳州汽车有限公司 | Vehicle intention prediction method, device, equipment and storage medium |
| CN113359700B (en) * | 2021-05-08 | 2022-12-20 | 安徽泗州拖拉机制造有限公司 | Intelligent operation system of unmanned tractor based on 5G |
| WO2022246851A1 (en) * | 2021-05-28 | 2022-12-01 | 深圳市大疆创新科技有限公司 | Aerial survey data-based testing method and system for autonomous driving perception system, and storage medium |
| CN113433975B (en) * | 2021-08-01 | 2022-08-23 | 陈军 | Method and system for taking off and landing unmanned aerial vehicle airport by unmanned aerial vehicle |
| CN113625752A (en) * | 2021-08-06 | 2021-11-09 | 航天时代飞鹏有限公司 | Vehicle-mounted six-rotor unmanned aerial vehicle control method and device based on satellite navigation positioning |
| CN114319187B (en) * | 2021-12-02 | 2023-05-16 | 北京国家新能源汽车技术创新中心有限公司 | Intelligent network-connected automobile control gateway barrier gate system |
| CN114239319B (en) * | 2021-12-30 | 2023-11-10 | 中国民用航空总局第二研究所 | Unmanned aerial vehicle route acquisition system |
| CN114783184A (en) * | 2022-04-19 | 2022-07-22 | 江苏大学 | Beyond-the-horizon sensing system based on information fusion of vehicle, road and unmanned aerial vehicle |
| CN115484692B (en) * | 2022-09-08 | 2024-03-22 | 山东新一代信息产业技术研究院有限公司 | Method, equipment and medium for cooperative communication of unmanned aerial vehicle and robot cluster |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108803617A (en) * | 2018-07-10 | 2018-11-13 | 深圳大学 | Trajectory predictions method and device |
| CN109059944A (en) * | 2018-06-06 | 2018-12-21 | 上海国际汽车城（集团）有限公司 | Motion planning method based on driving habit study |
| CN109920246A (en) * | 2019-02-22 | 2019-06-21 | 重庆邮电大学 | It is a kind of that local paths planning method is cooperateed with binocular vision based on V2X communication |
| CN111459159A (en) * | 2020-03-16 | 2020-07-28 | 江苏理工学院 | Path following control system and control method |
| EP3696734A1 (en) * | 2019-02-15 | 2020-08-19 | Siemens Aktiengesellschaft | Tensor-train recurrent neural network supported autonomous and assisted driving |
| CN111702754A (en) * | 2020-05-14 | 2020-09-25 | 国网安徽省电力有限公司检修分公司 | Robot obstacle avoidance trajectory planning method based on simulation learning and robot |
Family Cites Families (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103489328B (en) * | 2013-10-16 | 2016-07-06 | 重庆邮电大学 | A kind of intelligent parking lot reverse car seeking method |
| CN107065924A (en) * | 2017-03-15 | 2017-08-18 | 普宙飞行器科技（深圳）有限公司 | The vehicle-mounted landing system of unmanned plane, can vehicle-mounted landing unmanned plane and landing method |
| US20190184561A1 (en) * | 2017-12-15 | 2019-06-20 | The Regents Of The University Of California | Machine Learning based Fixed-Time Optimal Path Generation |
| CN108765284A (en) * | 2018-05-04 | 2018-11-06 | 哈尔滨理工大学 | A kind of autonomous driving vehicle vehicle-mounted unmanned aerial vehicle image processing method and device |
| CN208224845U (en) * | 2018-06-05 | 2018-12-11 | 南京工程学院 | A kind of unmanned vehicle navigation system of omnibearing visual angle |
| CN108791491A (en) * | 2018-06-12 | 2018-11-13 | 中国人民解放军国防科技大学 | Vehicle lateral tracking control method based on self-evaluation learning |
| CN109032131B (en) * | 2018-07-05 | 2021-07-27 | 东南大学 | Dynamic overtaking obstacle avoidance method applied to unmanned automobile |
| CN110377039B (en) * | 2019-07-26 | 2021-07-30 | 吉林大学 | Vehicle obstacle avoidance track planning and tracking control method |
| CN111169479B (en) * | 2020-01-14 | 2021-12-07 | 中国第一汽车股份有限公司 | Cruise control method, device and system, vehicle and storage medium |
| CN111300372A (en) * | 2020-04-02 | 2020-06-19 | 同济人工智能研究院(苏州)有限公司 | Air-ground cooperative intelligent inspection robot and inspection method |
-
2020
- 2020-12-11 CN CN202011460346.1A patent/CN112558608B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109059944A (en) * | 2018-06-06 | 2018-12-21 | 上海国际汽车城（集团）有限公司 | Motion planning method based on driving habit study |
| CN108803617A (en) * | 2018-07-10 | 2018-11-13 | 深圳大学 | Trajectory predictions method and device |
| EP3696734A1 (en) * | 2019-02-15 | 2020-08-19 | Siemens Aktiengesellschaft | Tensor-train recurrent neural network supported autonomous and assisted driving |
| CN109920246A (en) * | 2019-02-22 | 2019-06-21 | 重庆邮电大学 | It is a kind of that local paths planning method is cooperateed with binocular vision based on V2X communication |
| CN111459159A (en) * | 2020-03-16 | 2020-07-28 | 江苏理工学院 | Path following control system and control method |
| CN111702754A (en) * | 2020-05-14 | 2020-09-25 | 国网安徽省电力有限公司检修分公司 | Robot obstacle avoidance trajectory planning method based on simulation learning and robot |
Non-Patent Citations (2)
| Title |
|---|
| 基于深度强化学习的三维路径规划算法;黄东晋等;《计算机工程与应用》;20200325(第15期);全文 * |
| 智能汽车换道避障路径规划与跟踪方法;裴红蕾;《中国安全科学学报》;20180915(第09期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112558608A (en) | 2021-03-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112558608B (en) | Vehicle-mounted machine cooperative control and path optimization method based on unmanned aerial vehicle assistance | |
| US11480435B2 (en) | Map generation systems and methods | |
| CN109164809B (en) | Autonomous following control system and method for vehicle formation | |
| US11548533B2 (en) | Perception and motion prediction for autonomous devices | |
| CN111045425B (en) | Auxiliary teleoperation driving method for ground unmanned vehicle | |
| CN105318888A (en) | Unmanned perception based unmanned aerial vehicle route planning method | |
| CN105759829A (en) | Laser radar-based mini-sized unmanned plane control method and system | |
| US11726501B2 (en) | System and method for perceptive navigation of automated vehicles | |
| CN113791619B (en) | Airport automatic driving tractor dispatching navigation system and method | |
| US20190066522A1 (en) | Controlling Landings of an Aerial Robotic Vehicle Using Three-Dimensional Terrain Maps Generated Using Visual-Inertial Odometry | |
| US11754415B2 (en) | Sensor localization from external source data | |
| CN112068574A (en) | Control method and system for unmanned vehicle in dynamic complex environment | |
| CN117848356A (en) | Techniques for collaborative mapping between unmanned aerial vehicles and ground vehicles | |
| Shit | Precise localization for achieving next-generation autonomous navigation: State-of-the-art, taxonomy and future prospects | |
| EP4148385A1 (en) | Vehicle navigation positioning method and apparatus, and base station, system and readable storage medium | |
| RU195749U1 (en) | Intelligent vision system for an unmanned aerial vehicle for solving navigation problems, building a three-dimensional map of the surrounding space and obstacles, and autonomous patrolling | |
| EP4285083A1 (en) | Methods and system for generating a lane-level map for an area of interest for navigation of an autonomous vehicle | |
| WO2021153176A1 (en) | Autonomous movement device, autonomous movement control method, and program | |
| WO2022141240A1 (en) | Determining vehicle positions for autonomous driving based on monocular vision and semantic map | |
| Kim et al. | Development of a vision-based recognition and position measurement system for cooperative missions of multiple heterogeneous unmanned vehicles | |
| CN114281109A (en) | Multi-machine cooperation control system guided by unmanned aerial vehicle | |
| Lee et al. | Infrastructure Node-based Vehicle Localization for Autonomous Driving | |
| Cordes et al. | Vehicle lane merge visual benchmark | |
| US20230237793A1 (en) | False track mitigation in object detection systems | |
| US20230415739A1 (en) | Systems and methods for controlling longitudinal acceleration based on lateral objects |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |