CN112965517B - Unmanned aerial vehicle inspection safety obstacle avoidance system and method based on binocular vision fusion laser radar and electromagnetic field detection - Google Patents
Unmanned aerial vehicle inspection safety obstacle avoidance system and method based on binocular vision fusion laser radar and electromagnetic field detection Download PDFInfo
- Publication number
- CN112965517B CN112965517B CN202110132282.0A CN202110132282A CN112965517B CN 112965517 B CN112965517 B CN 112965517B CN 202110132282 A CN202110132282 A CN 202110132282A CN 112965517 B CN112965517 B CN 112965517B
- Authority
- CN
- China
- Prior art keywords
- obstacle
- sensor
- distance
- information
- electromagnetic field
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000005672 electromagnetic field Effects 0.000 title claims abstract description 75
- 238000001514 detection method Methods 0.000 title claims abstract description 65
- 230000004927 fusion Effects 0.000 title claims abstract description 55
- 238000000034 method Methods 0.000 title claims abstract description 20
- 238000007689 inspection Methods 0.000 title claims abstract description 15
- 238000007781 pre-processing Methods 0.000 claims abstract description 43
- 230000005540 biological transmission Effects 0.000 claims abstract description 30
- 230000004888 barrier function Effects 0.000 claims abstract description 12
- RZVHIXYEVGDQDX-UHFFFAOYSA-N 9,10-anthraquinone Chemical compound C1=CC=C2C(=O)C3=CC=CC=C3C(=O)C2=C1 RZVHIXYEVGDQDX-UHFFFAOYSA-N 0.000 claims description 25
- 239000004020 conductor Substances 0.000 claims description 6
- 238000005259 measurement Methods 0.000 claims description 6
- 230000005684 electric field Effects 0.000 claims description 3
- DMBHHRLKUKUOEG-UHFFFAOYSA-N diphenylamine Chemical compound C=1C=CC=CC=1NC1=CC=CC=C1 DMBHHRLKUKUOEG-UHFFFAOYSA-N 0.000 abstract description 2
- 238000012544 monitoring process Methods 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 238000005267 amalgamation Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/10—Simultaneous control of position or course in three dimensions
- G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
Landscapes
- Engineering & Computer Science (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Radar Systems Or Details Thereof (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention relates to an unmanned aerial vehicle inspection safety obstacle avoidance system and method based on binocular vision fusion laser radar and electromagnetic field detection, wherein the unmanned aerial vehicle inspection safety obstacle avoidance system comprises a detection module, an information preprocessing module, a multi-sensor information fusion module and an obstacle avoidance decision module. The detection module contains binocular vision camera, laser radar, millimeter wave radar, electromagnetic field detection sensor, ultrasonic sensor, big dipper positioning system BDS who loads on unmanned aerial vehicle, and information preprocessing module and multi-sensor information fusion module realize that unmanned aerial vehicle detects and independently keeps away the barrier at the obstacle of complicated electromagnetic environment through multi-sensor data looks fusion. The safe obstacle avoidance system and the safe obstacle avoidance method can improve the level of the unmanned aerial vehicle in inspecting and monitoring the surrounding environment, effectively avoid obstacles in time and ensure the safety of the unmanned aerial vehicle and the safety of a power transmission line.
Description
Technical Field
The invention relates to the technical field of automatic control of unmanned aerial vehicles, in particular to an unmanned aerial vehicle inspection safety obstacle avoidance system and method based on binocular vision fusion laser radar and electromagnetic field detection.
Background
Unmanned aerial vehicle electric power patrols and examines and has efficient, convenient to use, advantage such as with low costs, has solved the tradition and has patrolled and examined a great deal of problem. Unmanned aerial vehicle electric power is patrolled and examined inevitably and can be met some known and unknown barriers, and has regular electromagnetic field distribution around the transmission line, and the distance is too close can produce adverse effect to unmanned aerial vehicle body safety, control, communication and picture biography. The research on a safe obstacle avoidance system and method of an unmanned aerial vehicle in a complex environment is an urgent problem to be solved for guaranteeing the safety of the unmanned aerial vehicle and the safety of a power transmission line.
Unmanned aerial vehicle can carry on multiple sensor, and cooperation carries out many units to the data collection and fuses, and information is mutually testified and is supplemented, reduces the uncertainty, improves and keeps away barrier efficiency and credibility, in time effectively accomplishes the barrier and avoids.
Disclosure of Invention
The invention aims to provide an unmanned aerial vehicle routing inspection safety obstacle avoidance system and method based on binocular vision fusion laser radar and electromagnetic field detection.
In order to solve the technical problems, the technical scheme adopted by the invention is as follows:
unmanned aerial vehicle based on binocular vision fuses laser radar and electromagnetic field detection patrols and examines safe obstacle avoidance system, its characterized in that: the system comprises a detection module, an information preprocessing module, a multi-sensor information fusion module and an obstacle avoidance decision module;
the detection module comprises a binocular vision camera, a laser radar, a millimeter wave radar, an electromagnetic field detection sensor, an ultrasonic sensor and a Beidou positioning system BDS which are loaded on the unmanned aerial vehicle;
the binocular camera is used for calculating a depth image based on binocular vision and acquiring barrier distance information in real time;
the laser radar detects the obstacle by adopting a laser beam and is used for detecting the distance and speed information of the obstacle;
the millimeter wave radar detects the obstacle by adopting millimeter waves and is used for detecting the distance and speed information of the obstacle;
the high-precision electromagnetic field detection sensor is used for acquiring the electromagnetic field intensity of the position in real time, comparing the electromagnetic field intensity with a power transmission conductor electromagnetic field distribution model and converting the electromagnetic field intensity into distance data of the position;
the ultrasonic sensor is used for detecting the distance between obstacles by sending and receiving ultrasonic waves returned by the obstacles;
the BDS is used for obtaining longitude, latitude, height and speed information of the position where the unmanned aerial vehicle is located;
the information preprocessing module is used for preprocessing data information received by the binocular vision camera, the laser radar, the millimeter wave radar, the electromagnetic field detection sensor, the ultrasonic sensor and the Beidou positioning system BDS according to the sensor types;
the multi-sensor information fusion module is used for carrying out information data fusion on the sensor information data transmitted by the information preprocessing module;
and the obstacle avoidance decision module is connected with the unmanned aerial vehicle flight control system and used for receiving the information of the multi-sensor information fusion module, generating an obstacle avoidance strategy and controlling the unmanned aerial vehicle flight control system to realize obstacle avoidance.
An unmanned aerial vehicle routing inspection safety obstacle avoidance method based on binocular vision fusion laser radar and electromagnetic field detection is characterized in that: the method comprises the following steps of obtaining information fusion through multiple sensors, judging and executing the flight decision of the unmanned aerial vehicle, and specifically comprises the following steps:
and 4, the specific steps of the multi-sensor information fusion module for realizing multi-sensor information fusion are as follows:
step 4.1, preprocessing the information obtained by the binocular camera to obtain the distance D between the aircraft and the obstacle1；
Step 4.2, preprocessing the information obtained by the ultrasonic sensor to obtain the distance D between the aircraft and the obstacle2；
Step 4.3, preprocessing the information obtained by the laser radar to obtain the distance D between the aircraft and the obstacle3；
Step 4.4, preprocessing the information obtained by the millimeter radar to obtain the distance D between the aircraft and the obstacle4；
Step 4.5, comparing the current position P acquired by the BDS with the three-dimensional model data of the power transmission line space stored in the flight control to obtain the shortest distance D between the unmanned aerial vehicle and the obstacle5；
Step 4.6, comparing the electromagnetic field intensity data E obtained by the electromagnetic field detection sensor with the transmission line electromagnetic field distribution model stored in flight control, and converting the electromagnetic field intensity data E into the distance D between the aircraft and the obstacle6；
Step 4.7, defining a multi-sensor fusion coefficient knWherein n is the same as {1,2,3,4,5,6}, and finally calculating the aircraft and the transmission powerThe distance value Dis of the line device is:
wherein k is1，k2，k3，k4，k5，k6Is not less than 0, and satisfies k1+k2+k3+k4+k5+k6＝1。
if D isis>D, keeping the current state of the aircraft, and continuing to execute the task;
if D isisIf not, calculating a normal vector which passes through the point P and points to the outer side of the point S, and flying according to a vector path to avoid an obstacle;
if D isis<And D, calculating a normal vector which passes through the point P and points to the outer side of the safety boundary closest to the point P, and flying according to a vector path to avoid the obstacle.
The detection module comprises six groups of sensors including two binocular cameras, two laser radars, two millimeter wave radars, two ultrasonic sensors, an electromagnetic field detection sensor and a Beidou positioning system BDS. The distance between the two groups of binocular cameras and the obstacle is L1iAnd L2i(ii) a The distance between the two groups of ultrasonic sensors and the obstacle is L3iAnd L4i(ii) a The distance between the two groups of laser radar measurements and the obstacle is L5i、L6i(ii) a The distance between the two groups of laser radar measurements and the obstacle is L7iAnd L8i。
In step 3, the information preprocessing module preprocesses the data acquired by the multiple sensors as follows:
step 3.1, distance D to the obstacle obtained by the binocular camera1Taking a left camera L and a right camera L in a binocular camera1iAnd L2iThe minimum value, namely:
D1＝Min(L1i，L2i)
when D is present1When the distance between the two cameras is larger than or equal to 2D, the data output of the binocular camera is infinite, frequent interference in flight control is avoided, and the operating efficiency of an obstacle avoidance system is improved;
step 3.2, distance D between the ultrasonic sensor and the obstacle is obtained2Get left and right camera L3iAnd L4iThe minimum value, namely:
D2＝Min(L3i，L4i)
when D is present2When the distance between the two adjacent ultrasonic sensors is larger than or equal to 2D, the data output of the ultrasonic sensors is infinite, frequent intervention of flight control is avoided, and the operating efficiency of an obstacle avoidance system is improved;
step 3.3, distance D to the obstacle obtained by the laser radar3Get 2 laser radar range information L5iAnd L6iThe minimum value, namely:
D3＝Min(L5i，L6i)
when D is present3When the distance between the laser radar and the obstacle avoidance system is larger than or equal to 2D, the data output of the laser radar is infinite, frequent intervention of flight control is avoided, and the operation efficiency of the obstacle avoidance system is improved;
step 3.4, obtaining the distance D from the millimeter wave radar to the obstacle4Get 2 millimeter wave radar distance information L7iAnd L8iThe minimum value, namely:
D4＝Min(L7i，L8i)
when D is4When the distance between the millimeter wave radar and the target object is larger than or equal to 2D, the data output of the millimeter wave radar is infinite, frequent intervention in flight control is avoided, and the operating efficiency of an obstacle avoidance system is improved;
step 3.5, comparing the current position P acquired by the BDS with the three-dimensional model data of the power transmission line space stored in the flight control to obtain the shortest distance D between the unmanned aerial vehicle and the obstacle5；
Step 3.6, the electromagnetic field detection sensor can accurately acquire the space electric field distribution of the conducting wire by adopting an analog charge method, and further acquire the distance D between the aircraft and the obstacle6。
Six groups of sensors of the detection module are respectively a binocular vision camera, a laser radar, a millimeter wave radar, an electromagnetic field detection sensor, an ultrasonic sensor and a Beidou positioning system BDS.
When any sensor not less than one group fails, namely no output exists in the sensors in the failure group, the output value of the failure group in the information preprocessing module is infinite, and the fusion coefficient k of the corresponding sensor groupnAnd =0. The original value is added to the fusion coefficient k of other normal group sensors after being averagednForming a new sensor fusion coefficient knStill satisfy k1+k2+k3+k4+k5+k6＝1；
When partial fault sensors exist in any sensor group or multiple sensor groups, the output value of the fault sensor in the information preprocessing module is infinite, the output value of the corresponding sensor group is the minimum value, and the fusion coefficient k of the corresponding sensor groupnAnd is not changed.
The unmanned aerial vehicle inspection safety obstacle avoidance system and method based on binocular vision fusion laser radar and electromagnetic field detection have the following beneficial effects: first, can detect through the barrier of multiunit sensor adaptation complex electromagnetic environment, high accuracy electromagnetic field detection sensor can enough regard as the detection sensor with the barrier distance, can also avoid unmanned aerial vehicle to fly into the strong electromagnetic field region, avoids unmanned aerial vehicle to be out of control because of strong electromagnetic field interference. Secondly, the data that acquire between the multiunit sensor can be through amalgamation coefficient evaluating the distance between accurate unmanned aerial vehicle and the barrier jointly, provides the data basis for keeping away barrier strategy generation. Third, the existence of multiunit data sensor has guaranteed still can to operate when having single or a plurality of sensor to the accurate distance data of exporting between unmanned aerial vehicle and the barrier, can be as the replenishment that data acquisition each other between the multiunit sensor, improved the stability and the reliability of system greatly.
Drawings
Fig. 1 is a schematic structural diagram of an unmanned aerial vehicle inspection safety obstacle avoidance system based on binocular vision fusion laser radar and electromagnetic field detection.
Fig. 2 is a schematic diagram of the position distribution of multiple sensor groups on the unmanned aerial vehicle according to the present invention.
Fig. 3 is a sensor data processing flow chart when a sensor fails in the unmanned aerial vehicle inspection safety obstacle avoidance method based on binocular vision fusion laser radar and electromagnetic field detection.
Detailed Description
The invention is further described below in connection with the drawings and the specific preferred embodiments.
Unmanned aerial vehicle patrols and examines safe obstacle avoidance system based on binocular vision fuses laser radar and electromagnetic field detection: the system comprises a detection module, an information preprocessing module, a multi-sensor information fusion module and an obstacle avoidance decision module;
the detection module comprises a binocular vision camera, a laser radar, a millimeter wave radar, an electromagnetic field detection sensor, an ultrasonic sensor and a Beidou positioning system BDS which are loaded on the unmanned aerial vehicle;
the binocular camera is used for calculating a depth image based on binocular vision and acquiring barrier distance information in real time;
the laser radar detects the obstacle by adopting a laser beam and is used for detecting the distance and speed information of the obstacle;
the millimeter wave radar detects the obstacle by adopting millimeter waves and is used for detecting the distance and speed information of the obstacle;
the high-precision electromagnetic field detection sensor is used for acquiring the electromagnetic field intensity of the position in real time, comparing the electromagnetic field intensity with a power transmission conductor electromagnetic field distribution model and converting the electromagnetic field intensity into distance data of the position;
the ultrasonic sensor is used for detecting the distance between obstacles by sending and receiving ultrasonic waves returned by the obstacles;
the BDS is used for obtaining longitude, latitude, altitude and speed information of the position of the unmanned aerial vehicle;
the information preprocessing module is used for preprocessing data information received by the binocular vision camera, the laser radar, the millimeter wave radar, the electromagnetic field detection sensor, the ultrasonic sensor and the Beidou positioning system BDS according to the category of the sensor;
the multi-sensor information fusion module is used for carrying out information data fusion on the sensor information data transmitted by the information preprocessing module;
and the obstacle avoidance decision module is connected with the unmanned aerial vehicle flight control system and used for receiving the information of the multi-sensor information fusion module, generating an obstacle avoidance strategy and controlling the unmanned aerial vehicle flight control system to realize obstacle avoidance.
In the present embodiment, as shown in fig. 2, the detection module includes a binocular camera composed of a first camera 1 and a second camera 2; the ultrasonic sensor group consists of a first ultrasonic sensor 3 and a second ultrasonic sensor 4; a laser radar sensor group consisting of a first laser radar 5 and a second laser radar 6; a millimeter wave radar sensor group consisting of a first millimeter wave radar 7 and a second laser millimeter wave radar 8; beidou positioning system BDS09 and electromagnetic field detection sensor 10.
Wherein, first camera 1 and second camera 2 arrange in unmanned aerial vehicle the place ahead, are 45 with the dead ahead contained angle, and first ultrasonic sensor 3 and second ultrasonic sensor 4 arrange at the unmanned aerial vehicle rear, are 45 with the dead ahead contained angle, and first lidar 5, second lidar 6 deploy at unmanned aerial vehicle dead ahead, dead behind, the dead left of first millimeter wave radar 7 and second millimeter wave radar 8, right-hand, big dipper positioning system BDS 9 and electromagnetic field detection sensor 10 arrange respectively directly over and under the unmanned aerial vehicle middle part.
The first camera 1 and the second camera 2 respectively measure the distance L between the obstacle1iAnd L2i(ii) a The first ultrasonic sensor 3 and the second ultrasonic sensor 4 respectively measure the distance L between the first ultrasonic sensor and the obstacle3iAnd L4i. The distance between the first laser radar 5 and the obstacle measured by the second laser radar 6 is L5i、L6iThe distance between the first millimeter wave radar 7 and the obstacle measured by the second millimeter wave radar 8 is L7iAnd L8i；
In this embodiment, unmanned aerial vehicle inspection safety obstacle avoidance method based on binocular vision fusion laser radar and electromagnetic field detection is characterized in that: the method comprises the following steps of obtaining information fusion through multiple sensors, judging and executing the flight decision of the unmanned aerial vehicle, and as shown in figure 1, specifically:
step 4.1, preprocessing the information obtained by the binocular camera to obtain the distance D between the aircraft and the obstacle1；
Step 4.2, preprocessing the information obtained by the ultrasonic sensor to obtain the distance D between the aircraft and the obstacle2；
Step 4.3, preprocessing the information obtained by the laser radar to obtain the distance D between the aircraft and the obstacle3；
Step 4.4, preprocessing the information obtained by the millimeter radar to obtain the distance D between the aircraft and the obstacle4；
Step 4.5, comparing the current position P acquired by the BDS with the three-dimensional model data of the power transmission line space stored in the flight control to obtain the shortest distance D between the unmanned aerial vehicle and the obstacle5；
Step 4.6, the electromagnetic field intensity data E obtained by the electromagnetic field detection sensor and the transmission line electromagnetism stored in the flight controlComparing the field distribution models, and converting the electromagnetic field intensity data E into the distance D between the aircraft and the barrier6；
Step 4.7, defining a multi-sensor fusion coefficient knAnd finally calculating the distance value Dis between the aircraft and the power transmission line equipment as follows, wherein n is belonged to {1,2,3,4,5,6 }:
wherein k is1，k2，k3，k4，k5，k6Is not less than 0, and satisfies k1+k2+k3+k4+k5+k6＝1。
if D isis>D, keeping the current state of the aircraft, and continuing to execute the task;
if D isisIf not, calculating a normal vector which passes through the point P and points to the outer side of the point S, and flying according to a vector path to avoid an obstacle;
if D isis<And D, calculating a normal vector which passes through the point P and points to the outer side of the safety boundary closest to the point P, and flying according to a vector path to avoid the obstacle.
The detection module comprises six groups of sensors including two binocular cameras, two laser radars, two millimeter wave radars, two ultrasonic sensors, an electromagnetic field detection sensor and a Beidou positioning system BDS. The distance between the two groups of binocular cameras and the obstacle is L1iAnd L2i(ii) a The distance between the two groups of ultrasonic sensors and the obstacle is L3iAnd L4i(ii) a The distance between the two groups of laser radar measurements and the obstacle is L5i、L6i(ii) a The distance between the two groups of laser radar measurements and the obstacle is L7iAnd L8i。
In this embodiment, in step 3, the information preprocessing module performs the following preprocessing on the data acquired by the multiple sensors:
in step 3, the information preprocessing module preprocesses the data acquired by the multiple sensors as follows:
step 3.1, distance D to the obstacle obtained by the binocular camera1Left and right cameras L in binocular camera1iAnd L2iThe minimum value, namely:
D1＝Min(L1i，L2i)
when D is1When the distance between the two cameras is larger than or equal to 2D, the data output of the binocular camera is infinite, frequent interference in flight control is avoided, and the operating efficiency of an obstacle avoidance system is improved;
step 3.2, distance D between the ultrasonic sensor and the obstacle is obtained2Get left and right camera L3iAnd L4iThe minimum value, namely:
D2＝Min(L3i，L4i)
when D is present2When the distance between the two adjacent ultrasonic sensors is larger than or equal to 2D, the data output of the ultrasonic sensors is infinite, frequent intervention of flight control is avoided, and the operating efficiency of an obstacle avoidance system is improved;
step 3.3, distance D to the obstacle obtained by the laser radar3Get 2 laser radar range information L5iAnd L6iThe minimum value, namely:
D3＝Min(L5i，L6i)
when D is present3When the data output of the laser radar is larger than or equal to 2D, the data output of the laser radar is infinite, frequent intervention flight control is avoided, and the operating efficiency of an obstacle avoidance system is improved;
step 3.4, obtaining the distance D from the millimeter wave radar to the obstacle4Get 2 millimeter wave radar distance information L7iAnd L8iThe minimum value, namely:
D4＝Min(L7i，L8i)
when D is4When the distance between the millimeter wave radar and the target object is larger than or equal to 2D, the data output of the millimeter wave radar is infinite, frequent intervention in flight control is avoided, and the operating efficiency of an obstacle avoidance system is improved;
step 3.5, the current position P acquired by the Beidou positioning system BDS is stored in flight controlComparing the three-dimensional model data of the power transmission line space to obtain the shortest distance D between the unmanned aerial vehicle and the obstacle5；
Step 3.6, the electromagnetic field detection sensor can accurately acquire the space electric field distribution of the conducting wire by adopting an analog charge method, and further acquire the distance D between the aircraft and the obstacle6。
Six groups of sensors of the detection module are respectively a binocular vision camera, a laser radar, a millimeter wave radar, an electromagnetic field detection sensor, an ultrasonic sensor and a Beidou positioning system BDS.
When any sensor not less than one group fails, namely no output exists in the sensors in the failure group, the output value of the failure group in the information preprocessing module is infinite, and the fusion coefficient k of the corresponding sensor groupnAnd =0. The original value is added to the fusion coefficient k of other normal group sensors after being averagednForming a new sensor fusion coefficient knStill satisfy k1+k2+k3+k4+k5+k6＝1；
When partial fault sensors exist in any sensor group or multiple sensor groups, the output value of the fault sensor in the information preprocessing module is infinite, the output value of the corresponding sensor group is the minimum value, and the fusion coefficient k of the corresponding sensor groupnAnd is not changed.
The above are only preferred embodiments of the present invention, and the scope of the present invention is not limited to the above examples, and all technical solutions that fall under the spirit of the present invention belong to the scope of the present invention. It should be noted that modifications and embellishments within the scope of the invention may be made by those skilled in the art without departing from the principle of the invention.
Claims (3)
1. An unmanned aerial vehicle inspection safety obstacle avoidance method based on binocular vision fusion laser radar and electromagnetic field detection is characterized in that: the method comprises the following steps of obtaining information fusion through multiple sensors, judging and executing the flight decision of the unmanned aerial vehicle, and specifically comprises the following steps:
step 1, downloading a three-dimensional model of a power transmission line space and an electromagnetic field distribution model of a power transmission conductor to an unmanned aerial vehicle through ground station software;
step 2, starting a binocular camera, a laser radar, a millimeter wave radar, an electromagnetic field detection sensor, an ultrasonic sensor and a Beidou positioning system BDS, wherein the binocular camera, the laser radar, the millimeter wave radar, the electromagnetic field detection sensor, the ultrasonic sensor and the Beidou positioning system BDS receive data information in real time and send the data information to an information preprocessing module, the real-time position of an aircraft is P, and the boundary of a three-dimensional model of a power transmission line space of ground station software is an S domain; the detection module consists of six groups of sensors including two binocular cameras, two laser radars, two millimeter wave radars, two ultrasonic sensors, an electromagnetic field detection sensor and a Beidou positioning system BDS; the distance between the two groups of binocular cameras and the obstacle is L1iAnd L2i(ii) a The distance between the two groups of ultrasonic sensors and the obstacle is L3iAnd L4i(ii) a The distance between the two groups of laser radar measurements and the obstacle is L5i、L6i(ii) a The distance between the two groups of millimeter wave radar measurements and the obstacle is L7iAnd L8i；
Step 3, the information preprocessing module respectively preprocesses information of the binocular camera, the laser radar, the millimeter wave radar, the electromagnetic field detection sensor, the ultrasonic sensor and the Beidou positioning system BDS according to sensor categories, and submits the preprocessed information to the multi-sensor information fusion module; the information preprocessing module preprocesses the data acquired by the multiple sensors as follows:
step 3.1, distance D to the obstacle obtained by the binocular camera1Left and right cameras L in binocular camera1iAnd L2iThe minimum value, namely:
D1＝Min(L1i,L2i)
when D is1When the distance between the two cameras is larger than or equal to 2D, the data output of the binocular camera is infinite, frequent interference in flight control is avoided, and the operating efficiency of an obstacle avoidance system is improved;
step 3.2, obtaining the distance D from the ultrasonic sensor to the obstacle2Get left and right camera L3iAnd L4iThe minimum value, namely:
D2＝Min(L3i，L4i)
when D is present2When the distance between the two adjacent ultrasonic sensors is larger than or equal to 2D, the data output of the ultrasonic sensors is infinite, frequent intervention of flight control is avoided, and the operating efficiency of an obstacle avoidance system is improved;
step 3.3, distance D to the obstacle obtained by the laser radar3Get 2 laser radar range information L5iAnd L6iThe minimum value, namely:
D3＝Min(L5i,L6i)
when D is present3When the data output of the laser radar is larger than or equal to 2D, the data output of the laser radar is infinite, frequent intervention flight control is avoided, and the operating efficiency of an obstacle avoidance system is improved;
step 3.4, obtaining the distance D from the millimeter wave radar to the obstacle4Get 2 millimeter wave radar distance information L7iAnd L8iThe minimum value, namely:
D4＝Min(L7i，L8i)
when D is present4When the distance between the millimeter wave radar and the target object is larger than or equal to 2D, the data output of the millimeter wave radar is infinite, frequent intervention in flight control is avoided, and the operating efficiency of an obstacle avoidance system is improved;
step 3.5, comparing the current position P acquired by the BDS with the three-dimensional model data of the power transmission line space stored in the flight control to obtain the shortest distance D between the unmanned aerial vehicle and the obstacle5；
Step 3.6, the electromagnetic field detection sensor can accurately acquire the space electric field distribution of the conducting wire by adopting an analog charge method, and further acquire the distance D between the aircraft and the obstacle6；
Step 4, the specific steps of the multi-sensor information fusion module for realizing the multi-sensor information fusion are as follows:
step 4.1, preprocessing the information obtained by the binocular camera to obtain the distance D between the aircraft and the obstacle1；
Step 4.2, preprocessing the information obtained by the ultrasonic sensor to obtain the distance D between the aircraft and the obstacle2；
Step 4.3, preprocessing the information obtained by the laser radar to obtainDistance D between aircraft and obstacle3；
Step 4.4, preprocessing the information obtained by the millimeter radar to obtain the distance D between the aircraft and the obstacle4；
Step 4.5, comparing the current position P acquired by the BDS with the three-dimensional model data of the power transmission line space stored in the flight control to obtain the shortest distance D between the unmanned aerial vehicle and the obstacle5；
Step 4.6, comparing the electromagnetic field intensity data E obtained by the electromagnetic field detection sensor with the transmission line electromagnetic field distribution model stored in flight control, and converting the electromagnetic field intensity data E into the distance D between the aircraft and the obstacle6；
Step 4.7, defining a multi-sensor fusion coefficient knAnd finally calculating the distance value Dis between the aircraft and the power transmission line equipment as follows, wherein n is belonged to {1,2,3,4,5,6 }:
wherein k is1,k2,k3,k4,k5,k6Is not less than 0, and satisfies k1+k2+k3+k4+k5+k6＝1；
Step 5, comparing the distance value Dis between the aircraft and the power transmission line equipment with a safe distance D to generate an obstacle avoidance strategy, wherein the safe distance D is determined by an operation rule, and the specific steps are as follows:
if D isis>D, keeping the current state of the aircraft, and continuing to execute the task;
if D isisIf not, calculating a normal vector which passes through the point P and points to the outer side of the point S, and flying according to a vector path to avoid an obstacle;
if D isis<And D, calculating a normal vector which passes through the point P and points to the outer side of the safety boundary closest to the point P, and flying according to a vector path to avoid the obstacle.
2. The unmanned aerial vehicle inspection safety obstacle avoidance method based on binocular vision fusion laser radar and electromagnetic field detection as claimed in claim 1, wherein: six groups of sensors of the detection module are respectively a binocular vision camera, a laser radar, a millimeter wave radar, an electromagnetic field detection sensor, an ultrasonic sensor and a Beidou positioning system BDS;
when any sensor not less than one group fails, namely no output exists in the sensors in the failure group, the output value of the failure group in the information preprocessing module is infinite, and the fusion coefficient k of the corresponding sensor groupn=0; the original value is added to the fusion coefficient k of other normal group sensors after being averagednForming a new sensor fusion coefficient knStill satisfy k1+k2+k3+k4+k5+k6＝1；
When partial fault sensors exist in any sensor group or multiple sensor groups, the output value of the fault sensor in the information preprocessing module is infinite, the output value of the corresponding sensor group is the minimum value, and the fusion coefficient k of the corresponding sensor groupnAnd is not changed.
3. An unmanned aerial vehicle inspection safety obstacle avoidance system adopting the unmanned aerial vehicle inspection safety obstacle avoidance method based on binocular vision fusion laser radar and electromagnetic field detection, which is characterized in that: the system comprises a detection module, an information preprocessing module, a multi-sensor information fusion module and an obstacle avoidance decision module;
the detection module comprises a binocular vision camera, a laser radar, a millimeter wave radar, an electromagnetic field detection sensor, an ultrasonic sensor and a Beidou positioning system BDS which are loaded on the unmanned aerial vehicle;
the binocular camera is used for calculating a depth image based on binocular vision and acquiring barrier distance information in real time;
the laser radar detects the obstacle by adopting a laser beam and is used for detecting the distance and speed information of the obstacle;
the millimeter wave radar detects the obstacle by adopting millimeter waves and is used for detecting the distance and speed information of the obstacle;
the electromagnetic field detection sensor is used for acquiring the electromagnetic field intensity of the position in real time, comparing the electromagnetic field intensity with the transmission conductor electromagnetic field distribution model and converting the electromagnetic field intensity into distance data of the position;
the ultrasonic sensor is used for sending and receiving ultrasonic waves returned by the obstacle and detecting the distance between the obstacles;
the BDS is used for obtaining longitude, latitude, altitude and speed information of the position of the unmanned aerial vehicle;
the information preprocessing module is used for preprocessing data information received by the binocular vision camera, the laser radar, the millimeter wave radar, the electromagnetic field detection sensor, the ultrasonic sensor and the Beidou positioning system BDS according to the sensor types;
the multi-sensor information fusion module is used for carrying out information data fusion on the sensor information data transmitted by the information preprocessing module;
and the obstacle avoidance decision module is connected with the unmanned aerial vehicle flight control system and used for receiving the information of the multi-sensor information fusion module, generating an obstacle avoidance strategy and controlling the unmanned aerial vehicle flight control system to realize obstacle avoidance.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110132282.0A CN112965517B (en) | 2021-01-31 | 2021-01-31 | Unmanned aerial vehicle inspection safety obstacle avoidance system and method based on binocular vision fusion laser radar and electromagnetic field detection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110132282.0A CN112965517B (en) | 2021-01-31 | 2021-01-31 | Unmanned aerial vehicle inspection safety obstacle avoidance system and method based on binocular vision fusion laser radar and electromagnetic field detection |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112965517A CN112965517A (en) | 2021-06-15 |
| CN112965517B true CN112965517B (en) | 2022-11-01 |
Family
ID=76272314
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110132282.0A Active CN112965517B (en) | 2021-01-31 | 2021-01-31 | Unmanned aerial vehicle inspection safety obstacle avoidance system and method based on binocular vision fusion laser radar and electromagnetic field detection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112965517B (en) |
Families Citing this family (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113640802B (en) * | 2021-07-30 | 2024-05-17 | 国网上海市电力公司 | Robot space positioning method and system based on multiple fusion sensors |
| CN113465598B (en) * | 2021-08-04 | 2024-02-09 | 北京云恒科技研究院有限公司 | Inertial integrated navigation system suitable for unmanned aerial vehicle |
| CN113655531B (en) * | 2021-08-10 | 2022-09-13 | 浙江大学 | Peripheral target detection positioning device and method based on high-frequency electromagnetic field |
| CN113791074A (en) * | 2021-08-12 | 2021-12-14 | 中国建筑股份有限公司 | Unmanned aerial vehicle bridge crack inspection system and method based on multi-sensor fusion |
| CN113835444A (en) * | 2021-10-11 | 2021-12-24 | 广州穿越千机创新科技有限公司 | Ground magnetic field automatic detection system and method for formation unmanned aerial vehicle |
| CN114003036A (en) * | 2021-10-28 | 2022-02-01 | 广州赛特智能科技有限公司 | Robot obstacle avoidance control method, device, equipment and medium |
| CN114237277A (en) * | 2021-11-09 | 2022-03-25 | 国网江苏省电力有限公司盐城供电分公司 | Unmanned aerial vehicle keeps away barrier system based on millimeter wave radar and nearly electric induction |
| CN114115278A (en) * | 2021-11-26 | 2022-03-01 | 东北林业大学 | Obstacle avoidance system based on FPGA (field programmable Gate array) for forest fire prevention robot during traveling |
| CN113955087B (en) * | 2021-12-21 | 2022-06-07 | 山东欧龙电子科技有限公司 | Manned vehicle intelligence flight control system |
| CN114047787B (en) * | 2021-12-23 | 2024-04-05 | 普宙科技有限公司 | Unmanned aerial vehicle obstacle avoidance system, unmanned aerial vehicle obstacle avoidance method, unmanned aerial vehicle obstacle avoidance equipment and storage medium |
| CN114697165B (en) * | 2022-03-09 | 2023-12-22 | 杭州市保密技术测评中心(杭州市专用通信与保密技术服务中心) | Signal source detection method based on unmanned aerial vehicle vision and wireless signal fusion |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103135550B (en) * | 2013-01-31 | 2015-05-20 | 南京航空航天大学 | Multiple obstacle-avoidance control method of unmanned plane used for electric wire inspection |
| CN105892489B (en) * | 2016-05-24 | 2019-09-10 | 国网山东省电力公司电力科学研究院 | A kind of automatic obstacle avoiding UAV system and control method based on Multi-sensor Fusion |
| CN107783545B (en) * | 2016-08-25 | 2021-04-27 | 大连楼兰科技股份有限公司 | Obstacle avoidance system of post-disaster rescue rotor unmanned aerial vehicle based on OODA (object oriented data acquisition) ring multi-sensor information fusion |
| CN107783549B (en) * | 2016-08-25 | 2020-12-08 | 大连楼兰科技股份有限公司 | Single-rotor-wing plant protection unmanned aerial vehicle obstacle avoidance system based on multi-sensor information fusion technology |
| CN108319982A (en) * | 2018-02-06 | 2018-07-24 | 贵州电网有限责任公司 | A kind of power-line patrolling unmanned plane Fusion obstacle detection method |
| CN111324143A (en) * | 2020-02-28 | 2020-06-23 | 广东电网有限责任公司 | Unmanned aerial vehicle autonomous patrol obstacle avoidance system, method and computer equipment |
-
2021
- 2021-01-31 CN CN202110132282.0A patent/CN112965517B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN112965517A (en) | 2021-06-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112965517B (en) | Unmanned aerial vehicle inspection safety obstacle avoidance system and method based on binocular vision fusion laser radar and electromagnetic field detection | |
| EP3903293A1 (en) | Crowdsourced detection, identification and sharing of hazardous road objects in hd maps | |
| CN107817488A (en) | The unmanned plane obstacle avoidance apparatus and barrier-avoiding method merged based on millimetre-wave radar with vision | |
| CN212135234U (en) | A flight auxiliary device for transmission line patrols and examines unmanned aerial vehicle | |
| CN109444916B (en) | Unmanned driving drivable area determining device and method | |
| CN112693466A (en) | System and method for evaluating performance of vehicle environment perception sensor | |
| CN110873879A (en) | Device and method for deep fusion of characteristics of multi-source heterogeneous sensor | |
| CN108549084A (en) | A kind of target detection based on sparse two-dimensional laser radar and Attitude estimation method | |
| CN108958284B (en) | Unmanned aerial vehicle obstacle avoidance system and method | |
| CN113566833A (en) | Multi-sensor fusion vehicle positioning method and system | |
| EP3531224A1 (en) | Environment-adaptive sense and avoid system for unmanned vehicles | |
| CN112106111A (en) | Calibration method, calibration equipment, movable platform and storage medium | |
| CN113160327A (en) | Method and system for realizing point cloud completion | |
| CN114020002A (en) | Method, device and equipment for inspecting fan blade by unmanned aerial vehicle, unmanned aerial vehicle and medium | |
| WO2023083198A1 (en) | Echo signal processing method and apparatus, device, and storage medium | |
| CN113791074A (en) | Unmanned aerial vehicle bridge crack inspection system and method based on multi-sensor fusion | |
| CN114578344A (en) | Target sensing method, device and system suitable for rainy environment | |
| CN113281738A (en) | Multi-sensor fusion target detection system and detection method | |
| CN113759947B (en) | Airborne flight obstacle avoidance assisting method, device and system based on laser radar | |
| CN109885091B (en) | Unmanned aerial vehicle autonomous flight control method and system | |
| CN215642397U (en) | Unmanned aerial vehicle crowd flight trajectory control system | |
| CN113790718A (en) | Unmanned ship-based information sharing system | |
| WO2021056283A1 (en) | Systems and methods for adjusting a vehicle pose | |
| CN109708659B (en) | Distributed intelligent photoelectric low-altitude protection system | |
| CN112119361A (en) | Returning control method and device for movable platform and movable platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |