CN113821044A - Bridge detection unmanned aerial vehicle autonomous navigation and stability control method based on reinforcement learning - Google Patents
Bridge detection unmanned aerial vehicle autonomous navigation and stability control method based on reinforcement learning Download PDFInfo
- Publication number
- CN113821044A CN113821044A CN202110760963.1A CN202110760963A CN113821044A CN 113821044 A CN113821044 A CN 113821044A CN 202110760963 A CN202110760963 A CN 202110760963A CN 113821044 A CN113821044 A CN 113821044A
- Authority
- CN
- China
- Prior art keywords
- unmanned aerial
- aerial vehicle
- bridge
- control
- bridge detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 103
- 238000000034 method Methods 0.000 title claims abstract description 58
- 230000002787 reinforcement Effects 0.000 title claims abstract description 36
- 238000013528 artificial neural network Methods 0.000 claims abstract description 21
- 230000009471 action Effects 0.000 claims description 31
- 238000007689 inspection Methods 0.000 claims description 20
- 230000001133 acceleration Effects 0.000 claims description 10
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 230000005484 gravity Effects 0.000 claims description 6
- 230000006399 behavior Effects 0.000 claims description 5
- 238000013519 translation Methods 0.000 claims description 4
- 230000008859 change Effects 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 3
- 238000012549 training Methods 0.000 claims description 3
- 238000013459 approach Methods 0.000 claims description 2
- 230000004069 differentiation Effects 0.000 claims description 2
- 230000010354 integration Effects 0.000 claims description 2
- RZVHIXYEVGDQDX-UHFFFAOYSA-N 9,10-anthraquinone Chemical compound C1=CC=C2C(=O)C3=CC=CC=C3C(=O)C2=C1 RZVHIXYEVGDQDX-UHFFFAOYSA-N 0.000 abstract description 7
- 230000008034 disappearance Effects 0.000 abstract 1
- 239000011159 matrix material Substances 0.000 description 10
- 238000005259 measurement Methods 0.000 description 10
- 230000008569 process Effects 0.000 description 6
- 230000008901 benefit Effects 0.000 description 4
- 238000013461 design Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 238000010276 construction Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 3
- 239000000126 substance Substances 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 229910000831 Steel Inorganic materials 0.000 description 2
- 238000010168 coupling process Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000004807 localization Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 239000010959 steel Substances 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000004971 IR microspectroscopy Methods 0.000 description 1
- 206010033799 Paralysis Diseases 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000009194 climbing Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 150000001875 compounds Chemical class 0.000 description 1
- 230000001143 conditioned effect Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 230000001186 cumulative effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 230000003292 diminished effect Effects 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 230000005489 elastic deformation Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000012060 immune response imaging Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000011514 reflex Effects 0.000 description 1
- 239000011150 reinforced concrete Substances 0.000 description 1
- 230000003014 reinforcing effect Effects 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
- G05D1/08—Control of attitude, i.e. control of roll, pitch, or yaw
- G05D1/0808—Control of attitude, i.e. control of roll, pitch, or yaw specially adapted for aircraft
- G05D1/0816—Control of attitude, i.e. control of roll, pitch, or yaw specially adapted for aircraft to ensure stability
- G05D1/0825—Control of attitude, i.e. control of roll, pitch, or yaw specially adapted for aircraft to ensure stability using mathematical models
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
- G05D1/10—Simultaneous control of position or course in three dimensions
- G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
- G05D1/106—Change initiated in response to external conditions, e.g. avoidance of elevated terrain or of no-fly zones
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Remote Sensing (AREA)
- Radar, Positioning & Navigation (AREA)
- Aviation & Aerospace Engineering (AREA)
- Algebra (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Mathematical Physics (AREA)
- Pure & Applied Mathematics (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
The invention discloses a bridge detection unmanned aerial vehicle autonomous navigation and stability control method based on reinforcement learning, which comprises the following steps: firstly, performing dynamic modeling on a quad-rotor unmanned aerial vehicle for detecting a bridge based on a Newton-Euler method; step two, considering the surrounding environment information of the bridge to be detected and detection constraint conditions, and providing the unmanned aerial vehicle autonomous obstacle avoidance flying implementation based on neural network reinforcement learning; step three, adopting binocular vision and IMU combined autonomous navigation under the assistance of a bridge building information model; and fourthly, attitude control is carried out on the inner ring control of the system by adopting an active disturbance rejection algorithm, position control is carried out on the outer ring control by adopting PID control, and stable control of the bridge detection unmanned aerial vehicle under the interference of strong wind is realized. The problem of can't stabilize flight control under locating signal disappearance, the strong wind interference under the bridge to unmanned aerial vehicle bridge detection technique exists is solved.
Description
Technical Field
The invention belongs to the technical field of bridge detection, and particularly relates to a bridge detection unmanned aerial vehicle autonomous navigation and stability control method based on reinforcement learning.
Background
The construction of the traffic infrastructure in China is rapidly developed, the bridge construction is rapidly increased at the speed of 2 ten thousand seats per year on average, and by the end of 2018, 9 ten thousand seats and 5 thousand seats of a large-span bridge are built in China, wherein 105 seats of the large-span bridge with the main span of more than 400 meters are built in China. The existing bridge safety problem becomes the core problem of national economy and national safety of the people.
Currently, most bridge detection operations in China still adopt 3 traditional technical modes:
(1) bridge detection vehicle: the method is a subject means of current bridge detection, and the method conveys detection personnel to the vicinity of an observed object by means of a bridge detection vehicle, a climbing vehicle or a support for proximity detection and measurement, and has the defects of time and labor waste and large interference on normal traffic.
(2) Bridge bottom detection channel: the bridge is constructed in the same period as the bridge, the influence of detection work on traffic is small, and the detection range is limited. Meanwhile, due to the limitation of the age, the detection usually ages and loses the effect within the design age.
(3) The comprehensive bridge detection vehicle comprises: mainly rely on the check out test set that the car carried on to ultrasonic wave, vibration etc. hand penetrate the bridge floor and detect the bridge, and the advantage is little to traffic interference, but its detection ripples penetrability is limited, and the accuracy is difficult to guarantee, can't realize the detection to bridge, bridge column. In terms of detection efficiency.
In a word, the traditional bridge detection mode has the characteristics of high cost, strong specialization and the like, and is not suitable for being applied as a daily detection technical means by the management department. In addition, the detection personnel adopting the mode are usually positioned in the high altitude of tens of meters, are greatly influenced by wind power and bridge vibration, belong to high-risk operation and have high potential safety hazard.
Bridge detection unmanned aerial vehicle receives the extensive concern in bridge maintenance industry in recent years with advantages such as its mobility is strong, small, efficient, use cost is lower, modularization maintenance is safeguarded conveniently, safe risk is low. Specifically speaking, compare in traditional detection means, unmanned aerial vehicle bridge detects the advantage as follows:
(1) the technical level is as follows: the unmanned aerial vehicle mounting measuring equipment works in the air, has good maneuvering performance, can reach a blind area which is difficult to reach by the traditional equipment, and makes up for the conventional inspection dead angle and short plates;
(2) and (3) operating layer: the unmanned aerial vehicle has the advantages of simple structure, light weight, small volume and easy transportation and maintenance, and meanwhile, the unmanned aerial vehicle can realize quick assembly and disassembly and is convenient to operate;
(3) and (4) economic aspect: compared with the traditional professional detection equipment, the unmanned detection equipment has low cost, does not need to close traffic in the detection process, and does not influence the conventional driving order;
(4) safety aspect: unmanned aerial vehicle fungible measurement personnel carry out high altitude construction, and measurement personnel control at safe position unmanned aerial vehicle can, unmanned aerial vehicle potential safety hazard.
The complete unmanned aerial vehicle bridge detection system comprises an unmanned aerial vehicle, a data transmission system, a task load system, a ground station system, an analysis processing system and the like. Bridge detection unmanned aerial vehicle can carry all kinds of check out test set, like global positioning system module (GPS), Inertial Measurement Unit (IMU), distance sensor (TOF), high definition camera device etc. data transmission system is used for the transmission of system control signal, detection data. The ground station system is used for monitoring the flight of the unmanned aerial vehicle in real time, and the analysis processing system is responsible for analyzing, diagnosing and quantifying the disease degree of the collected data and evaluating the bridge.
Bridge detection unmanned aerial vehicle has compensatied that traditional bridge detection exists blind area, expense height, the big drawback of inspection personnel's safety risk to its good mobility, security and the wide industry of receiving concerns of economic nature, but it also has the technical bottleneck, mainly includes:
(1) loss of the underbridge localization signal: for the bridge detection unmanned aerial vehicle, due to the shielding of the bridge structure, particularly when the unmanned aerial vehicle is detected under a large-span and wide-width bridge, the communication loss of the GPS equipment of the unmanned aerial vehicle during detection is easily caused, and the system is paralyzed because the signal cannot be received; moreover, the bridge is mainly of a reinforced concrete structure or a steel structure, and the strong magnetic field generated by a reinforcing mesh in the structure seriously influences the performance of the magnetic compass of the unmanned aerial vehicle, so that the accuracy and the robustness of the system are reduced;
(2) multi-source interference in complex environments: in the long-distance flight of unmanned aerial vehicle bridge detection, there is multisource interference to simultaneously come from external environment, sensor noise and model error. In the long-distance bridge detection process, the asymmetric high-frequency vibration of the engine passes through various interference sources such as cold/hot air mass, strong gust, small-scale turbulent flow and the like to act simultaneously, and the precision of the unmanned aerial vehicle navigation and control system is influenced.
Autonomous navigation and control in unstructured environments: under the complex environment, especially the strong wind field near the bridge easily produces great interference to unmanned aerial vehicle bridge detection, has not only reduced unmanned aerial vehicle's detection efficiency, has also increased unmanned aerial vehicle and has bumped the wall risk. Meanwhile, the large-span complex bridge structure form, especially the steel truss and other space structures, provides higher requirements for the obstacle avoidance capacity of the bridge detection unmanned aerial vehicle.
Disclosure of Invention
The invention aims to provide a bridge detection unmanned aerial vehicle autonomous navigation and stable control method based on reinforcement learning, and aims to solve the problems that positioning signals under a bridge are lost and stable flight control cannot be realized under strong wind interference in an unmanned aerial vehicle bridge detection technology. The invention analyzes an autonomous navigation and flight control method of a four-rotor unmanned aerial vehicle in bridge detection, provides a closed-loop system comprising motion planning, navigation and control, and has a model structure shown in attached figure 1.
The invention adopts the following technical scheme: a bridge detection unmanned aerial vehicle autonomous navigation and stability control method based on reinforcement learning comprises the following steps:
the method comprises the following steps: performing dynamic modeling on a quad-rotor unmanned aerial vehicle for detecting the bridge based on a Newton-Euler method;
step two: considering the information of the surrounding environment of the bridge to be detected and detection constraint conditions, providing the method for realizing autonomous obstacle avoidance flight of the unmanned aerial vehicle based on neural network strong learning;
step three: adopting binocular vision and IMU combined autonomous navigation under the assistance of a bridge building information model;
step four: the inner ring control of the system adopts an active disturbance rejection algorithm to perform attitude control, and the outer ring control adopts PID control to perform position control, so that the bridge detection unmanned aerial vehicle can be stably controlled under the interference of strong wind.
Further, the specific method of the step one is as follows:
in order to determine the flying position of the unmanned aerial vehicle, a body coordinate system E-XYZ is established, the origin point of the body coordinate system is the center of mass of the unmanned aerial vehicle during takeoff, the positive direction of an X axis is the heading positive direction of the unmanned aerial vehicle head, the positive direction of a Y axis is the horizontal leftward direction of the unmanned aerial vehicle, and the positive direction of a Z axis is the vertical upward direction of the unmanned aerial vehicle;
in order to determine the posture of the quad-rotor unmanned aerial vehicle, a body coordinate system B-X 'Y' Z 'is established, the origin point of the coordinate system is the mass center of the unmanned aerial vehicle, the positive direction of an X' axis is the direction of a machine head, the positive direction of a Z 'axis is vertical to the plane of the body and upwards, and the positive direction of the Y' axis is determined by a right-hand criterion;
the dynamics model of a quad-rotor drone is then as follows:
wherein m is the mass of the unmanned aerial vehicle, and x, y and z are translation positions from the central point of the four rotors; phi is a roll angle of the bridge detection unmanned aerial vehicle, and the angular speed of the bridge detection unmanned aerial vehicle is p; theta is a bridge detection unmanned aerial vehicle pitch angle; psi is bridge detection unmanned aerial vehicle yaw angle, and angular velocity is r, n3For translational drag coefficient, n3x,n3y,n3zThe translational drag coefficients of X-axis, Y-axis and Z-axis, g is the gravitational acceleration, IXX,IYY,IZZMoment of inertia of X, Y and Z axes, H is angular momentum, and Ω ═ pqr]TIs angular velocity, HRFor angular momentum rate of change, l is the distance of bridge detection unmanned aerial vehicle's barycenter to central point.
Further, the method for reinforcement learning based on the neural network in the step two specifically comprises the following steps:
step 1: initializing a neural network and parameters used in operation;
step 2: initializing the state of an unmanned aerial vehicle for bridge detection;
and step 3: obtaining current state information s of unmanned aerial vehiclet；
And 4, step 4: inputting the current state information into the neural network in the step 1, and selecting an action according to the obtained Q value;
and 5: executing the action to enable the unmanned aerial vehicle to reach a new state st+1Simultaneously obtaining a feedback enhanced signal value; if collision occurs, returning to the initial position of the unmanned aerial vehicle and restarting;
step 6: training a neural network;
and 7: and repeating the steps 3-6 until the learning is finished.
Further, the method for avoiding obstacles autonomously in the second step comprises the following steps:
adopting Boltzmann distribution to realize random selection of actions in the initial stage of the unmanned aerial vehicle, wherein the probability of selecting a certain action is as follows:
in the formula, T is a virtual problem, and the randomness of selection is stronger as T is increased;
as the learning progresses, the Q value gradually approaches to the expected state-action value, and an action is selected according to a greedy strategy, that is, the action corresponding to the maximum Q value is selected;
s is the input of the reinforcement learning system for receiving the environment state, and a is the corresponding behavior action output by the system.
Further, the specific method of the autonomous navigation algorithm in step three is as follows:
according to a frame model of the bridge detection unmanned aerial vehicle system navigation, conversion from a camera coordinate system to a world coordinate system in the unmanned aerial vehicle binocular vision navigation system under the assistance of a building information model is given, then three-dimensional coordinates of characteristic points are determined, and finally initial speed, zero offset of a gyroscope and a gravity direction required by nonlinear optimization are determined.
Further, the specific method of the step four is as follows: based on real-time wind field estimation of the bridge bottom, an inner ring of the bridge inspection unmanned aerial vehicle control system adopts an active disturbance rejection algorithm to perform attitude control, and an outer ring adopts a PID (proportion integration differentiation) method to perform position control, so that rapid and stable control under strong wind field interference is realized.
The invention has the beneficial effects that: (1) the unmanned aerial vehicle binocular vision robust SLAM navigation method under the assistance of the BIM model is adopted, and autonomous navigation and positioning under the condition of unmanned aerial vehicle satellite positioning signal loss can be realized; (2) the invention is based on real-time wind field estimation, the inner ring adopts an active disturbance rejection algorithm to carry out attitude control, and the outer ring adopts a PID method to carry out position control, thereby realizing rapid and stable flight control under the interference of a strong wind field.
Drawings
FIG. 1 is a schematic structural diagram of an unmanned aerial vehicle bridge detection closed-loop system model of the bridge detection unmanned aerial vehicle autonomous navigation and stability control method based on reinforcement learning;
FIG. 2 is a schematic diagram of a coordinate system of a bridge inspection unmanned aerial vehicle based on the bridge inspection unmanned aerial vehicle autonomous navigation and stability control method of reinforcement learning;
FIG. 3 is a structural diagram of an unmanned aerial vehicle bridge detection binocular vision inertial navigation system of the bridge detection unmanned aerial vehicle autonomous navigation and stability control method based on reinforcement learning;
fig. 4 is a structural diagram of an attitude controller of a bridge inspection unmanned aerial vehicle based on an active disturbance rejection control algorithm in the method for autonomous navigation and stability control of the bridge inspection unmanned aerial vehicle based on reinforcement learning.
Detailed Description
The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.
The invention provides a bridge detection unmanned aerial vehicle autonomous navigation and stability control method based on reinforcement learning, which specifically comprises the following steps:
the method comprises the following steps: performing dynamic modeling on the bridge detection unmanned aerial vehicle based on a Newton-Euler method;
step two: the method comprises the following steps that (1) an unmanned aerial vehicle autonomous obstacle avoidance is detected on the basis of a bridge of neural network reinforcement learning;
step three: designing a binocular vision and IMU combined autonomous navigation algorithm under BIM;
step four: and designing a stable control scheme for the bridge detection unmanned aerial vehicle under the interference of a strong wind field.
In some embodiments, in the step one, a dynamic model of the bridge detection unmanned aerial vehicle is required to be performed:
considering that the bridge detection unmanned aerial vehicle is a nonlinear, strong-coupling and under-actuated system, and meanwhile, the unmanned aerial vehicle has complex external interference in the bridge detection process, in order to establish a practical mathematical model and facilitate control algorithm design, the invention makes the following assumptions:
(1) the structure of the machine body and the rotor wing are both regarded as rigid bodies, and the elastic deformation and the vibration of the machine body are ignored;
(2) the four motors and the propellers are symmetrically arranged, and other parameters except positive and negative polarities are the same;
(3) the mass distribution of the machine body is uniform, and the mass center coincides with the appearance center.
Unmanned aerial vehicle is at bridge testing process main atress: the bridge detects lift, air resistance etc. that the gravity of unmanned aerial vehicle body, rotor produced. The main moments are: aerodynamic moment generated by lift force of the rotor wing, counter-torque force generated by rotation, air resistance, resistance moment generated by friction force and the like.
Next, a body coordinate system and a ground coordinate system are established, wherein:
in order to determine the flying position of the unmanned aerial vehicle, a body coordinate system E-XYZ is established, the origin point of the body coordinate system is the center of mass of the unmanned aerial vehicle during takeoff, the positive direction of an X axis is the heading positive direction of the unmanned aerial vehicle head, the positive direction of a Y axis is the horizontal leftward direction of the unmanned aerial vehicle, and the positive direction of a Z axis is the vertical upward direction of the unmanned aerial vehicle.
In order to determine the attitude of the unmanned aerial vehicle, a body coordinate system B-X 'Y' Z 'is established, the origin of the coordinate system is the mass center of the unmanned aerial vehicle, the positive direction of an X' axis is the direction of a machine head, the positive direction of a Z 'axis is vertical to the plane of the body and upwards, and the positive direction of the Y' axis is determined by a right-hand criterion.
As shown in fig. 2. In the attached figure 2, phi is a roll angle of the bridge detection unmanned aerial vehicle rotating around an O-X' axis, and the angular speed is p; theta is a pitch angle of the bridge detection unmanned aerial vehicle rotating around the O-Y' axis; psi is the yaw angle that bridge detection unmanned aerial vehicle revolved around O-Z' axle, and its angular velocity is r. The transformation matrix from the body coordinate system to the inertial coordinate system is:
the lift generated by the rotor is proportional to the square of the rotating speed, and the specific relation of the air resistance to the square of the rotating speed is as follows:
the gravity of the bridge detection unmanned aerial vehicle and the lift force and the air resistance of the four rotors are considered, and the following can be obtained through a Newton-Euler equation:
wherein X is [ X, y, z ]]TFor translational position from the centre point of the four rotors, n3For translational drag coefficient, g is [0,0, g ═]TIs the acceleration of gravity. Will rotate the matrix
Substituting into the equation above, one can obtain:
wherein n is3x,n3y,n3zThe translational drag coefficients of the X axis, the Y axis and the Z axis are respectively.
The bridge detection unmanned aerial vehicle is regarded as a rigid body, and the external force received by the four rotors can be obtained by the product of the angular speed of the four rotors and the rotational inertia of the counter shaft of the four rotors according to the rigid body mechanics. The moment of inertia is set as:
wherein, IXX,IYY,IZZThe moment of inertia of the X-axis, the Y-axis and the Z-axis, respectively, can be used to obtain the angular momentum of the quadrotor, as shown in the following formula:
let H be angular momentum, Ω ═ pqr]TIs angular velocity, HRThe change rate of angular momentum, M is the total external moment, and M is omega × H + HR，
The total torque obtained from the above formula is
After conversion, the following results are obtained:
the rotation angular speed omega of the four rotors is [ pqr ═ p q r-]TThe attitude angular velocity of the rotary wing can be obtained as
Further to find out
Wherein the content of the first and second substances,
the bridge detection unmanned aerial vehicle is in single posture of a roll channel, a pitch channel and a yaw channel, and the moment corresponding to the roll channel is M1The moment corresponding to the pitching channel is M2The moment corresponding to the yaw channel is M3.-IRq(-ω1+ω2-ω3+ω4) And IRp(-ω1+ω2-ω3+ω4) Detection of the spin effect of unmanned aerial vehicles for bridges, IRIs the moment of inertia of each motor. Wherein the content of the first and second substances,
in the formula: l is the distance from the center of mass of the bridge detection unmanned aerial vehicle to the central point.
By integrating the above formulas, the obtained attitude motion equation of the bridge detection unmanned aerial vehicle is as follows:
in summary, the available dynamic model of the bridge detection unmanned aerial vehicle is as follows:
in some embodiments, in the second step, the specific content of autonomous obstacle avoidance for the bridge detection based on neural network reinforcement learning is as follows:
bridge detection unmanned aerial vehicle need consider a large amount of reality situations and constraint condition in the testing process, mainly include:
(1) and (3) environment information: the method comprises the following steps of (1) including bridge site terrain information, bridge wind field information and other possible emergency situations;
(2) constraint conditions are as follows: the method comprises the steps of detecting the longest endurance time and the flight speed of the unmanned aerial vehicle, the safe distance between the unmanned aerial vehicle and obstacles such as piers and guy cables, a real-time wind field, the photographing distance of bridge cracks and the like by a bridge inspection.
The invention provides a laser radar perception and decision method based on neural network reinforcement learning by comprehensively considering the surrounding environment information of a bridge to be detected and detection constraint conditions. The method combines the sensing capability of a convolutional neural network and the decision-making capability of reinforcement learning together in a general form, and realizes direct output control from sensing input of the surrounding environment of a bridge to the flight action of the bridge detection unmanned aerial vehicle in an end-to-end learning mode, wherein the optimal decision-making strategy is obtained by learning in the accumulated return of interaction between the maximum bridge detection unmanned aerial vehicle and a dynamic model.
2.1, reinforcement Q learning:
the reinforcement learning system receives the input of the environment state as s, and outputs corresponding behavior as a according to an internal reasoning mechanism. The environment changes to a new state s' under the action of system action a. The system receives the input of the new state of the environment and simultaneously obtains the progressive reward punishment feedback r of the environment to the system. For reinforcement learning systems, the goal is to learn a behavior strategy τ: s → a that maximizes the cumulative number of environmental rewards available for the actions selected by the system. In the learning process, the basic principle of the reinforcement learning technology is as follows: if an action by the system results in a positive reward in the environment, the system's tendency to generate the action later is heightened, whereas the system's tendency to generate the action is diminished. This is similar to the mechanism of biological conditioned reflex.
Q learning is a widely used type of reinforcement learning that utilizes a function, expressed as follows:
Q(st,at)←rt+γ(rt+γrt+1+γ2rt+2+…)，
in the formula atIs whenAt time t is an action selected from action set a. Since the purpose of the system is to maximize the total prize value, it is useful
Substituted rt+γrt+1+γ2rt+2+ …, the expression is obtained
At the moment t, the unmanned aerial vehicle selects an action a according to the current state, and then the Q value is updated according to the following expression
b is the action selected at the t +1 state.
2.2, the bridge inspection unmanned aerial vehicle keeps away the barrier based on reinforcement learning of neural network:
the invention realizes the obstacle avoidance of the bridge inspection unmanned aerial vehicle by adopting the neural network-based reinforcement learning. The specific implementation steps of the neural network based reinforcement learning are as follows:
step 1: initializing a neural network and parameters used in operation;
step 2: initializing the state of a bridge detection unmanned aerial vehicle;
and step 3: obtaining current state information s of unmanned aerial vehiclet；
And 4, step 4: inputting the state information into a neural network, and selecting an action according to the obtained Q value;
and 5: the execution action enables the bridge detection unmanned aerial vehicle to reach a new state st+1And simultaneously obtaining a feedback enhanced signal value. And if the collision happens, returning to the initial position of the bridge inspection unmanned aerial vehicle and restarting.
Step 6: training a neural network;
and 7: and repeating the steps 3-6 until the learning is finished.
2.3 behavior selection strategy:
the invention adopts Boltzmann distribution to realize the random selection of actions in the initial stage of the bridge inspection unmanned aerial vehicle, and the probability of selecting a certain action is as follows:
where T is a virtual problem, the more random the selection as T increases.
As the learning progresses, the Q value slowly trends towards the desired state-action value, where the action is selected according to a greedy strategy, i.e. the action corresponding to the largest Q value is selected,
in some embodiments, in step three, a binocular vision and IMU combined self-piloting algorithm under BIM assistance is designed, and the specific contents are as follows:
aiming at the navigation and positioning problem under the condition of unmanned aerial vehicle bridge detection satellite positioning signal loss, the invention provides binocular vision robust SLAM navigation of a bridge detection unmanned aerial vehicle under the assistance of BIM. And the autonomous navigation of the bridge detection unmanned aerial vehicle is realized by adopting IMU and binocular vision combined navigation. The proposed navigation framework is shown in fig. 3.
Next, according to a frame model of system navigation, the invention provides the conversion from a camera coordinate system to a world coordinate system in a binocular vision navigation system under the assistance of BIM; and determining the three-dimensional coordinates of the characteristic points, and finally determining the initial speed, the zero offset of the gyroscope and the gravity direction required by the non-linear optimization.
3.1, IMU measurement model:
the IMU generally consists of a three-axis gyroscope and a three-axis accelerometer, and on the premise of not considering the rotation of the earth, the IMU is modeled as follows:
in the formula, the superscript B represents that the output of the IMU is data on the carrier system;
and
representing the IMU angle and acceleration measurements at time t, ωB(t) and aB(t) actual values of the angle and acceleration of the IMU at time t, bω(t)And ba(t)Zero offset, n, representing angular velocity and acceleration of carrier motionωAnd naRepresenting white gaussian noise present at the time of measurement.
3.2, combined navigation of binocular vision and inertia:
the invention adopts a loose coupling method to fuse vision and inertial navigation, namely, the two systems are considered to be independent, and a fusion strategy is adopted with an IMU (inertial measurement Unit) measured value after the pose of a camera is obtained. Its core states are:
wherein the subscript iw represents the conversion of the world coordinate system to the IMU coordinate system,
representing the position of the world coordinate system origin relative to the IMU coordinate system, v, q represent velocity and attitude,
and
representing gyroscope and accelerometer bias. The final equation of motion is as follows:
the state error vector is expressed according to the above equation:
wherein the content of the first and second substances,
representing the difference between the estimated value and the true value,
the post-fusion system states are the IMU and visual odometer state sets, expressed as:
the camera attitude measurement model is as follows:
in the formula (I), the compound is shown in the specification,
is a rotational transformation from the world to the IMU coordinate system,
and pcwA rotation matrix and a translation vector representing the world to camera system,
and
is noise. Measurement errorExpressed as:
linearizing the mixture to obtain:
the update equation is then derived by EKF:
the subscript k + m/k of the above formula means that the time point k is a prediction of k + m, and
the state quantity is represented. The corresponding past state can be found based on the framework:
the covariance matrix corresponding to the above equation is:
in the formula
Representing the embodied system dynamic matrix. The delta difference, covariance matrix is expressed as:
in the formula RrA covariance matrix representing the pose of the visual SLAM,
including two corresponding jacobian matrices, then its corresponding kalman gain is:
the system optimal estimation equation and the update equation of the covariance matrix are as follows:
to this end, post-fusion state estimates have been available.
In some embodiments, in step four, a stable control scheme for detecting interference of the unmanned aerial vehicle in the strong wind field by the bridge inspection bridge is designed, specifically including the following contents:
according to the dynamic model and the navigation scheme of the bridge inspection unmanned aerial vehicle, the attitude controller and the position controller are designed, the inner ring control adopts the active disturbance rejection algorithm to carry out attitude control, and the outer ring control adopts the PID control to carry out position control, so that the anti-jamming capability and the robustness of the bridge inspection unmanned aerial vehicle are improved.
4.1, unmanned aerial vehicle inner ring attitude controller design is examined to bridge based on active disturbance rejection control:
the active disturbance rejection algorithm consists of a tracking differentiator, an extended state observer and a nonlinear error feedback controller.
The bridge inspection unmanned aerial vehicle has 6 degrees of freedom, and attitude control based on active disturbance rejection control comprises 3 channels of pitching, rolling and yawing. As shown in fig. 4, an active disturbance rejection controller is applied to the 3 channels, wherein ADRC1, ADRC2, ADRC3 constitute pitch, roll and yaw loops, respectively. Wherein
θ, ψ are sensor return values.
4.2, designing a PID controller for the outer ring position of the bridge inspection unmanned aerial vehicle:
after the attitude controller is designed by adopting the active disturbance rejection algorithm, the position controller is designed. To reduce the computational load on the microprocessor, the position controller of the outer loop is designed using PID control.
When the position is controlled, the PID control is adopted to enable the four rotors to quickly reach the expected position, the tracking target position of the four rotors can be quickly adjusted when large errors occur, and stable flight of the four rotors can be well achieved when small errors occur. In the design of the inertial frame position controller, translation in three directions of an X axis, a Y axis and a Z axis is included.
As can be seen from the above equation, the altitude loop (Z-axis) is independent of the lateral control loop, so that the total lift of the four rotors can be controlled in the altitude loop, while the lateral loop requires different rotor lifts for control. It is necessary to determine a desired roll angle control amount theta according to the position flight expectationrDesired pitch angle control amount
And vertical control quantity collective pitch deltacol. Through analysis, the expected value of the line speed on the machine system can be obtained through a conversion matrix under an inertial coordinate system and a machine body coordinate system, and a specific formula is as follows:
wherein p isd,qd,rdThe desired linear speed in the four-rotor body coordinate system,
is a transformation matrix from a ground coordinate system to a body coordinate system.
In the formula, axIs a control amount of a desired linear acceleration in the x-axis direction, ayIs a control amount of desired linear acceleration in the y-axis direction, azIs a control amount of a desired linear acceleration in the z-axis direction.
Through the derivation of the formula, the expected line acceleration control quantity can be obtained, and therefore the expected value of the attitude of the bridge detection unmanned aerial vehicle is obtained. It is possible to derive the desired roll angle control amount thetarDesired pitch angle control amount
And vertical control quantity collective pitch deltacol. Through simplification, the method can obtain:
so far, the whole controller is designed.
For making bridge detection unmanned aerial vehicle can be when the bridge detects that can safe and reliable independently fly to the unknown region simultaneously carry out initiative search bridge surface damage region. The autonomous flight of the unmanned aerial vehicle depends on high-precision flight control methods, stable flight state estimation, stable and reliable map building methods, accurate motion strategies, motion planning and other technologies. The invention analyzes the autonomous navigation and flight control method of the bridge detection unmanned aerial vehicle in bridge detection, and provides a closed-loop system comprising motion planning, navigation and control.
Considering the requirements of omnidirectional maneuverability operation and hovering shooting, a large rotor unmanned aerial vehicle is adopted as a flying platform; the method comprises the steps that a binocular vision simultaneous localization and mapping (SLAM) scheme based on a BIM (building information model) model and cooperation identification assistance is used for solving the navigation problem of the unmanned aerial vehicle under the satellite signal loss; the bridge detection unmanned aerial vehicle autonomous navigation and stable flight control are realized on the basis of active disturbance rejection control based on real-time wind field estimation and autonomous maneuver decision based on neural network reinforcement learning.
Claims (6)
1. Bridge detection unmanned aerial vehicle autonomous navigation and stability control method based on reinforcement learning is characterized by comprising the following steps:
the method comprises the following steps: performing dynamic modeling on a quad-rotor unmanned aerial vehicle for detecting the bridge based on a Newton-Euler method;
step two: considering the information of the surrounding environment of the bridge to be detected and detection constraint conditions, providing the unmanned aerial vehicle autonomous obstacle avoidance flying implementation based on neural network reinforcement learning;
step three: adopting binocular vision and IMU combined autonomous navigation under the assistance of a bridge building information model;
step four: the inner ring control of the system adopts an active disturbance rejection algorithm to perform attitude control, and the outer ring control adopts PID control to perform position control, so that the bridge detection unmanned aerial vehicle can be stably controlled under the interference of strong wind.
2. The bridge inspection unmanned aerial vehicle autonomous navigation and stability control method based on reinforcement learning of claim 1, wherein the specific method of the first step is as follows:
in order to determine the flying position of the unmanned aerial vehicle, a body coordinate system E-XYZ is established, the origin point of the body coordinate system is the center of mass of the unmanned aerial vehicle during takeoff, the positive direction of an X axis is the heading positive direction of the unmanned aerial vehicle head, the positive direction of a Y axis is the horizontal leftward direction of the unmanned aerial vehicle, and the positive direction of a Z axis is the vertical upward direction of the unmanned aerial vehicle;
in order to determine the posture of the quad-rotor unmanned aerial vehicle, a body coordinate system B-X 'Y' Z 'is established, the origin point of the coordinate system is the mass center of the unmanned aerial vehicle, the positive direction of an X' axis is the direction of a machine head, the positive direction of a Z 'axis is vertical to the plane of the body and upwards, and the positive direction of the Y' axis is determined by a right-hand criterion;
the dynamics model of a quad-rotor drone is then as follows:
wherein m is the mass of the unmanned aerial vehicle, and x, y and z are translation positions from the central point of the four rotors; phi is a roll angle of the bridge detection unmanned aerial vehicle, and the angular speed of the bridge detection unmanned aerial vehicle is p; theta is a bridge detection unmanned aerial vehicle pitch angle; psi is the yaw angle of the bridge detection unmanned aerial vehicle, and the angular velocity is r, n3For translational drag coefficient, n3x,n3y,n3zThe translational drag coefficients of X-axis, Y-axis and Z-axis, g is the gravitational acceleration, IXX,IYY,IZZMoment of inertia of X, Y and Z axes, H is angular momentum, and Ω ═ pqr]TIs angular velocity, HRFor angular momentum rate of change, l is the distance of bridge detection unmanned aerial vehicle's barycenter to central point.
3. The bridge inspection unmanned aerial vehicle autonomous navigation and stability control method based on reinforcement learning of claim 1 or 2, wherein the method based on neural network reinforcement learning in the second step is specifically:
step 1: initializing a neural network and parameters used in operation;
step 2: initializing the state of an unmanned aerial vehicle for bridge detection;
and step 3: obtaining current state information s of the unmanned aerial vehiclet；
And 4, step 4: inputting the current state information into the neural network in the step 1, and selecting an action according to the obtained Q value;
and 5: performing an action such that the drone reaches a new state st+1Simultaneously obtaining a feedback enhanced signal value; if collision occurs, returning to the initial position of the unmanned aerial vehicle and restarting;
step 6: training a neural network;
and 7: and repeating the steps 3-6 until the learning is finished.
4. The bridge detection unmanned aerial vehicle autonomous navigation and stability control method based on reinforcement learning of claim 3, wherein the autonomous obstacle avoidance method in the second step is as follows:
adopting Boltzmann distribution to realize random selection of actions in the initial stage of the unmanned aerial vehicle, wherein the probability of selecting a certain action is as follows:
in the formula, T is a virtual problem, and the randomness of selection is stronger as T is increased;
as the learning progresses, the Q value gradually approaches to the expected state-action value, and an action is selected according to a greedy strategy, that is, the action corresponding to the maximum Q value is selected;
s is the input of the reinforcement learning system for receiving the environment state, and a is the corresponding behavior action output by the system.
5. The bridge detection unmanned aerial vehicle autonomous navigation and stability control method based on reinforcement learning of claim 1 or 2, wherein the specific method of the autonomous navigation algorithm in step three is as follows:
according to a frame model of the bridge detection unmanned aerial vehicle system navigation, conversion from a camera coordinate system to a world coordinate system in an unmanned aerial vehicle binocular vision navigation system under the assistance of a building information model is given, then three-dimensional coordinates of characteristic points are determined, and finally initial speed, zero offset of a gyroscope and a gravity direction required by nonlinear optimization are determined.
6. The bridge detection unmanned aerial vehicle autonomous navigation and stability control method based on reinforcement learning of claim 1 or 2, wherein the concrete method of the fourth step is: based on real-time wind field estimation of the bridge bottom, an inner ring of the bridge inspection unmanned aerial vehicle control system adopts an active disturbance rejection algorithm to perform attitude control, and an outer ring adopts a PID (proportion integration differentiation) method to perform position control, so that rapid and stable control under strong wind field interference is realized.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110760963.1A CN113821044A (en) | 2021-07-06 | 2021-07-06 | Bridge detection unmanned aerial vehicle autonomous navigation and stability control method based on reinforcement learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110760963.1A CN113821044A (en) | 2021-07-06 | 2021-07-06 | Bridge detection unmanned aerial vehicle autonomous navigation and stability control method based on reinforcement learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113821044A true CN113821044A (en) | 2021-12-21 |
Family
ID=78924157
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110760963.1A Pending CN113821044A (en) | 2021-07-06 | 2021-07-06 | Bridge detection unmanned aerial vehicle autonomous navigation and stability control method based on reinforcement learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113821044A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114578861A (en) * | 2022-04-29 | 2022-06-03 | 北京航空航天大学 | Unmanned aerial vehicle flight control strategy design method utilizing gust environment |
| CN114859971A (en) * | 2022-05-07 | 2022-08-05 | 北京卓翼智能科技有限公司 | Intelligent unmanned aerial vehicle for monitoring wind turbine |
| CN115016291A (en) * | 2022-07-13 | 2022-09-06 | 西北工业大学 | Aircraft anti-interference attitude control system and method based on wind field information |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017177533A1 (en) * | 2016-04-12 | 2017-10-19 | 深圳市龙云创新航空科技有限公司 | Method and system for controlling laser radar based micro unmanned aerial vehicle |
| CN109444171A (en) * | 2018-09-18 | 2019-03-08 | 山东理工大学 | Integrated Bridges Detection based on unmanned plane |
| CN109597425A (en) * | 2018-10-18 | 2019-04-09 | 中国航空无线电电子研究所 | Navigation of Pilotless Aircraft and barrier-avoiding method based on intensified learning |
| CN109911188A (en) * | 2019-03-18 | 2019-06-21 | 东南大学 | The bridge machinery UAV system of non-satellite navigator fix environment |
-
2021
- 2021-07-06 CN CN202110760963.1A patent/CN113821044A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017177533A1 (en) * | 2016-04-12 | 2017-10-19 | 深圳市龙云创新航空科技有限公司 | Method and system for controlling laser radar based micro unmanned aerial vehicle |
| CN109444171A (en) * | 2018-09-18 | 2019-03-08 | 山东理工大学 | Integrated Bridges Detection based on unmanned plane |
| CN109597425A (en) * | 2018-10-18 | 2019-04-09 | 中国航空无线电电子研究所 | Navigation of Pilotless Aircraft and barrier-avoiding method based on intensified learning |
| CN109911188A (en) * | 2019-03-18 | 2019-06-21 | 东南大学 | The bridge machinery UAV system of non-satellite navigator fix environment |
Non-Patent Citations (3)
| Title |
|---|
| 管一凡: "融合双目视觉和惯性的定位技术研究", 《中国优秀硕士学位论文电子期刊》 * |
| 郝超: "一种基于自抗扰算法的四旋翼无人机控制系统研究", 《中国优秀硕士学位论文电子期刊》 * |
| 陈润泽等: "旋翼无人机视觉/惯性组合导航技术研究", 《电子测量技术》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114578861A (en) * | 2022-04-29 | 2022-06-03 | 北京航空航天大学 | Unmanned aerial vehicle flight control strategy design method utilizing gust environment |
| CN114578861B (en) * | 2022-04-29 | 2022-08-16 | 北京航空航天大学 | Unmanned aerial vehicle flight control strategy design method utilizing gust environment |
| CN114859971A (en) * | 2022-05-07 | 2022-08-05 | 北京卓翼智能科技有限公司 | Intelligent unmanned aerial vehicle for monitoring wind turbine |
| CN115016291A (en) * | 2022-07-13 | 2022-09-06 | 西北工业大学 | Aircraft anti-interference attitude control system and method based on wind field information |
| CN115016291B (en) * | 2022-07-13 | 2023-11-10 | 西北工业大学 | Wind field information-based anti-interference attitude control system and method for aircraft |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113945206B (en) | Positioning method and device based on multi-sensor fusion | |
| CN113821044A (en) | Bridge detection unmanned aerial vehicle autonomous navigation and stability control method based on reinforcement learning | |
| CN109911188B (en) | Bridge detection unmanned aerial vehicle system in non-satellite navigation and positioning environment | |
| Paris et al. | Dynamic landing of an autonomous quadrotor on a moving platform in turbulent wind conditions | |
| Chee et al. | Control, navigation and collision avoidance for an unmanned aerial vehicle | |
| CN102809377B (en) | Aircraft inertia/pneumatic model Combinated navigation method | |
| CN109460052B (en) | Control method of combinable aircraft | |
| Kim et al. | Accurate modeling and robust hovering control for a quad-rotor VTOL aircraft | |
| CN110362100B (en) | Large storage tank autonomous inspection method based on quad-rotor unmanned aerial vehicle | |
| CN104236548A (en) | Indoor autonomous navigation method for micro unmanned aerial vehicle | |
| CN105759829A (en) | Laser radar-based mini-sized unmanned plane control method and system | |
| CN106643737A (en) | Four-rotor aircraft attitude calculation method in wind power interference environments | |
| CN109683629B (en) | Unmanned aerial vehicle electric power overhead line system based on combination navigation and computer vision | |
| CN104460685A (en) | Control system for four-rotor aircraft and control method of control system | |
| CN104062977A (en) | Full-autonomous flight control method for quadrotor unmanned aerial vehicle based on vision SLAM | |
| CN105094138A (en) | Low-altitude autonomous navigation system for rotary-wing unmanned plane | |
| CN103365296A (en) | Nonlinear output feedback flight control method for quad-rotor unmanned aerial vehicle | |
| CN104932512A (en) | Quadrotor posture control method based on MIMO nonlinear uncertain backstepping approach | |
| CN103217981A (en) | Four-rotor aircraft speed control method based on integral variable structure control | |
| CN108766035B (en) | Unmanned aerial vehicle terrain matching flight control system under guidance of point density | |
| Cho et al. | Autonomous ship deck landing of a quadrotor UAV using feed-forward image-based visual servoing | |
| RU195749U1 (en) | Intelligent vision system for an unmanned aerial vehicle for solving navigation problems, building a three-dimensional map of the surrounding space and obstacles, and autonomous patrolling | |
| Trunov | Transformation of operations with fuzzy sets for solving the problems on optimal motion of crewless unmanned vehicles | |
| CN114942648B (en) | Autonomous stabilization method for special unmanned aerial vehicle for bridge detection in complex wind field | |
| CN113359809B (en) | Bridge detection unmanned aerial vehicle autonomous positioning method based on RBFNN assistance |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20211221 |