CN109960278A - A kind of bionical obstruction-avoiding control system of unmanned plane based on LGMD and method - Google Patents
A kind of bionical obstruction-avoiding control system of unmanned plane based on LGMD and method Download PDFInfo
- Publication number
- CN109960278A CN109960278A CN201910281845.5A CN201910281845A CN109960278A CN 109960278 A CN109960278 A CN 109960278A CN 201910281845 A CN201910281845 A CN 201910281845A CN 109960278 A CN109960278 A CN 109960278A
- Authority
- CN
- China
- Prior art keywords
- lgmd
- layers
- neuron
- potential
- ffi
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 201000009342 Limb-girdle muscular dystrophy Diseases 0.000 title claims abstract description 119
- 238000000034 method Methods 0.000 title claims abstract description 30
- 238000004891 communication Methods 0.000 claims abstract description 21
- 238000013528 artificial neural network Methods 0.000 claims abstract description 18
- 210000002569 neuron Anatomy 0.000 claims description 160
- 239000012528 membrane Substances 0.000 claims description 45
- 230000008569 process Effects 0.000 claims description 12
- 230000004888 barrier function Effects 0.000 claims description 9
- 230000005611 electricity Effects 0.000 claims description 8
- 230000005764 inhibitory process Effects 0.000 claims description 8
- 230000001537 neural effect Effects 0.000 claims description 7
- 230000002401 inhibitory effect Effects 0.000 claims description 6
- 238000013507 mapping Methods 0.000 claims description 6
- 210000005036 nerve Anatomy 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 claims description 4
- 230000003993 interaction Effects 0.000 claims description 2
- 238000001914 filtration Methods 0.000 claims 1
- 238000001514 detection method Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000011160 research Methods 0.000 description 3
- RZVHIXYEVGDQDX-UHFFFAOYSA-N 9,10-anthraquinone Chemical compound C1=CC=C2C(=O)C3=CC=CC=C3C(=O)C2=C1 RZVHIXYEVGDQDX-UHFFFAOYSA-N 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000003062 neural network model Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 241000238631 Hexapoda Species 0.000 description 1
- 210000003484 anatomy Anatomy 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000002243 primary neuron Anatomy 0.000 description 1
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B13/00—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion
- G05B13/02—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
- G05B13/0265—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric the criterion being a learning criterion
- G05B13/027—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric the criterion being a learning criterion using neural networks only
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B13/00—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion
- G05B13/02—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
- G05B13/04—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric involving the use of models or simulators
- G05B13/042—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric involving the use of models or simulators in which a parameter or coefficient is automatically adjusted to optimise the performance
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B13/00—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion
- G05B13/02—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
- G05B13/04—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric involving the use of models or simulators
- G05B13/048—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric involving the use of models or simulators using a predictor
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
- G05D1/0088—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
- G05D1/10—Simultaneous control of position or course in three dimensions
- G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
Abstract
The bionical obstruction-avoiding control system of a kind of unmanned plane based on LGMD provided by the invention, including flight subsystem, light stream sensor, driving motor, embedded LGMD detector, camera, wireless communication module and earth station PC；The light stream sensor, embedded LGMD detector are electrically connected with the flight subsystem；The wireless communication module, driving motor are electrically connected with the flight subsystem output end；The embedded LGMD detector input terminal is connect with the camera output end signal；The wireless communication module and earth station PC wireless communication connect.The present invention also provides a kind of bionical avoidance obstacle method of unmanned plane based on LGMD, by building LGMD neural network, and view field image is split, direction in space selection and scene prediction are realized during unmanned plane during flying, realize that unmanned plane is real-time in circumstances not known, the flight of efficient avoidance.
Description
Technical field
The present invention relates to unmanned plane during flying device technical fields, more particularly to a kind of unmanned plane based on LGMD is bionical
Obstruction-avoiding control system further relates to a kind of bionical avoidance obstacle method of the unmanned plane based on LGMD.
Background technique
Unmanned plane has broad application prospects in many scenes such as geographical measurement, agricultural aviation, danger souding, and pacifies
Full property is all emphasis concerned by people all the time, especially under complex environment.Traditional unmanned plane is carried out using GPS, light stream
Path planning, and the method for combining the detection of the sensors such as ultrasonic wave, infrared ray and laser carries out collision detection, however, such side
Method is largely dependent upon the complexity of barrier material, texture and background, can only use in simple and specific environment.
And in recent years, the hot spot of domestic and foreign scholars' research efficiently and has flexibly been formed because of it based on the barrier-avoiding method of biological vision.Currently,
Foreign scholar navigates to insect visual and avoidance mechanism has done a large amount of Bioexperiment, and carries out further investigation summary, is formed
A large amount of referential research achievements.To the anatomy experiment of locust vision system, it was demonstrated that the big giant's motion detector of leaflet
(Lobula Giant Movement Detector, LGMD) is the primary neurons for completing anti-collision warning function, Rind F.C.
4 layers of classical LGMD are thus proposed with Bramwell D.I. et al. and input neural network structure, using excited suppressor mode
Come that detection object is close, opens the research for carrying out anti-collision warning by LGMD network.Yue S. et al. introduces new neuron
To enhance the ability of LGMD anti-collision warning, it was demonstrated that validity and robustness of the LGMD network for anti-collision warning, and moved
Plant the anti-collision warning ability studied under the conditions of group on microrobot.However application study of the LGMD on unmanned plane at present
It is less, and avoidance direction selection aspect lacks unknown scene and predict still using randomly choosing, for this purpose, present invention proposition
A kind of unmanned aerial vehicle control system and avoidance direction selection and scene prediction method based on LGMD improves unmanned plane avoidance
Flexibility and efficiency.
Summary of the invention
The present invention is to overcome above-mentioned traditional unmanned plane collision checking method, is existed dependent on barrier material, texture and back
The complexity of scape, can only the technological deficiency used in simple and specific environment, improve unmanned plane avoidance flexibility and effect
Rate provides a kind of bionical obstruction-avoiding control system of the unmanned plane based on LGMD.
A kind of bionical avoidance obstacle method of the unmanned plane based on LGMD is also proposed based on the system.
In order to solve the above technical problems, technical scheme is as follows:
A kind of bionical obstruction-avoiding control system of unmanned plane based on LGMD, including flight subsystem, light stream sensor, drive
Dynamic motor, embedded LGMD detector, camera, wireless communication module and earth station PC；Wherein:
The light stream sensor, embedded LGMD detector are electrically connected with the flight subsystem, carry out letter
Breath interaction；
The wireless communication module, driving motor are electrically connected with the flight subsystem output end；
The embedded LGMD detector input terminal is connect with the camera output end signal；
The wireless communication module and earth station PC wireless communication connect.
Wherein, the embedded LGMD detector is provided with LGMD neural network, the video letter of the camera acquisition
Breath is calculated by the LGMD neural network, is obtained avoidance obstacle instruction by LGMD neural network, is exported to the flight
Control subsystem realizes the avoidance obstacle of unmanned plane.
Wherein, the LGMD neural network includes P layers of neuron, E layers of neuron, I layers of neuron, S layers of neuron, G layers
Neuron, LGMD neuron and feedforward inhibit FFI neuron；Wherein:
The P layers of neuron obtains the view field image information of input video, responds to frame difference, obtains P layers of neuron
Film potential；
Excited film potential of the P layers of membrane potential of neurons directly as the E layers of neuron；
The I layers of neuron receives the output of the P layers of neuron previous frame, and carries out local inhibition, and be inhibited film
Current potential；
The S layers of neuron converges the output in I layers of neuron corresponding position field, by inhibiting film potential to institute
It states E layers of neuron to be inhibited, obtains S layers of membrane potential of neurons；
The G layers of neuron is used to enhance the colliding object extracted under complex background, is calculated according to S layers of membrane potential of neurons
Obtain G layers of membrane potential of neurons；
The LGMD neuron diagonally divides view field image information, obtains 4 azimuth informations；According to G layers of nerve
First film potential and 4 azimuth informations are calculated, and the film potential of 4 orientation C-LGMD is obtained, by the film of 4 orientation C-LGMD
Current potential is added, and obtains LGMD membrane potential of neurons；
The feedforward inhibits FFI neuron directly to obtain view field image information from the P layers of neuron, to LGMD nerve
First film potential, which is formed, to be inhibited, to obtain avoidance obstacle instruction output to the flight subsystem.
Wherein, the system also includes inertial sensor, the inertial sensor is integrated in the flight subsystem
On, it is electrically connected with flight subsystem.
Wherein, the system also includes laser sensor, the laser sensor and light stream sensor input terminal electricity
Property connection.
In above scheme, the core of the flight subsystem is STM32F407V；The inertial sensor main body is
MPU6050 chip；The light stream sensor main body is Pix4Flow chip；The main body of the wireless communication module is
nRF24L01；The inertial sensor for recording UAV Attitude information, the light stream sensor as horizontal plane position with
Velocity feedback device, the camera for acquiring video information in real time；The flight subsystem is visited according to embedded LGMD
The instruction for surveying device output, calculates the corresponding PWM value of the driving motor, is finally output to four driving motors of unmanned plane,
Realize avoidance obstacle；The wireless communication module returning real-time data gives earth station PC.
A kind of bionical avoidance obstacle method of unmanned plane based on LGMD, comprising the following steps:
S1: real time video collection is done by camera, obtains input video；
S2: embedded LGMD detector obtains the view field image information of input video, obtains by LGMD neural computing
It is instructed to avoidance obstacle, exports the avoidance obstacle for realizing unmanned plane to flight subsystem.
Wherein, the detailed process of the LGMD neural computing avoidance obstacle instruction are as follows:
S21:P layers of neuron obtain the view field image information of input video, respond to frame difference, obtain P layers of neuron
Film potential Pf(x, y), specifically:
Pf(x, y)=Lf(x, y)-Lf-1(x, y)； 1)
Wherein: f indicates that the f frame in video sequence, (x, y) are position of the pixel in network layer, Lf(x, y) is
The pixel value of the view field image of input；
Input of the output of S22:P layers of neuron as E layers of neuron, I layers of neuron, E layers of neuron directly receive P layers
Neuron output, and I layers of neuron receive the output of P layers of neuron previous frame, and carry out local inhibition, are embodied as:
Ef(x, y)=Pf(x, y)； 2)
If(x, y)=∑i∑jPf-1(x+i, y+j) wi(i, j) (if i=j, j ≠ 0)； 3)
Wherein: Ef(x, y) is excited film potential, i.e. E layers of membrane potential of neurons；If(x, y) is to inhibit film potential, i.e. I layers of mind
Through first film potential；wi(i, j) is that part inhibits weight；I, j are not 0 simultaneously；
S23:S layers of neuron converge the output in I layers of neuron corresponding position field, by inhibiting film potential to the E
Layer neuron is inhibited, and S layers of membrane potential of neurons are obtained, specifically:
Sf(x, y)=Ef(x, y)-If(x, y) WI； 5)
Wherein: Sf(x, y) is S layers of membrane potential of neurons；WITo inhibit weight matrix；
S24:G layers of neuron are used to enhance the colliding object extracted under complex background, are calculated according to S layers of membrane potential of neurons
G layers of membrane potential of neurons are obtained, specifically:
Wherein: Gf(x, y) is G layers of membrane potential of neurons；[we] it is convolution kernel；R is convolution nuclear radius, r=1；
Threshold value T is setdeFilter weaker excitement, specifically:
Wherein:For filtered G layers of membrane potential of neurons, CdeFor weak coefficient, it is [0,1]；TdeTo filter threshold
Value；
S25: when view field image information reaches LGMD neuron, view field image information is diagonally divided, obtains two
Coordinate in diagonal line y-axis is embodied as:
Wherein: Diag1, Diag2 are respectively the y-coordinate that two diagonal lines correspond to x；W is the width of image；H is the height of image
Degree；To obtain the film potential of 4 orientation C-LGMD, specifically:
Wherein: ULGMD, DLGMD, LLGMD, RLGMDThe respectively film potential in 4 orientation in image upper and lower, left and right；By 4 orientation
The film potential of C-LGMD is added, and obtains LGMD membrane potential of neurons Kf, specifically:
By KfThe range of [0,255] is normalized and is mapped in, specifically:
Wherein, ncellFor the total pixel quantity of image；
According to 4 orientation C-LGMD film potentials in entire image proportion, the film electricity after mapping in 4 orientation is obtained
Position, specifically:
Wherein,Respectively after the orientation mapping of 4, image upper and lower, left and right
Film potential；Work as κfMore than its threshold value Ts, then a LGMD peak pulse can be generatedSpecifically:
If continuous ntsPulse is no less than nsp, then judgement will collide, it is embodied as:
S26: the feedforward inhibits FFI neuron directly to obtain view field image information from the P layers of neuron, specific to indicate
Are as follows:
Wherein, FfInhibit the film potential of FFI neuron for feedforward；TFFIFor preset threshold；Work as FfMore than threshold value TFFI, LGMD
Membrane potential of neurons is suppressed immediately；
S27: direction selection is carried out: when feedforward inhibits FFI film potential FfGreater than threshold value TFFI, all avoidance instruction ignores；When
Cfinal=TURE, and feedover and inhibit FFI film potential FfLess than threshold value TFFIWhen, there is barrier in judgement, by comparing 4 orientationThe size of film potential, using the smallest orientation of film potential in 4 orientation as most safe
Avoidance direction；
S28: flying scene prediction: work as Cfinal=FALSE and feedforward inhibition FFI film potential FfLess than threshold value TFFI, unmanned plane
Normal flight, the film potential in 4 orientation by acquiring FFI, predicts the unknown flying scene of unmanned plane；
S29: the signal that step S25, S27, S28 are obtained forms avoidance obstacle instruction as avoidance obstacle signal.
Wherein, the process of the step S28 flying scene prediction specifically:
Scene set prediction threshold value TFFII, when N frame FFI forward is averaged film potentialLess than threshold value TFFIIWhen, unmanned plane normally flies
Row；When N frame FFI forward is averaged film potentialGreater than threshold value TFFII, it is less than threshold value TFFIWhen, pass through comparison
Size, the smallest orientation of film potential is found out, to predict under frontal scene, the following least side of heading barrier
Position；
Wherein, N frame FFI is averaged film potential forwardIt is embodied as:
Film potential average value in 4 orientationIt respectively indicates are as follows:
In formula,N frame film potential average value before expression 4, the upper and lower, left and right FFI orientation.
Compared with prior art, the beneficial effect of technical solution of the present invention is:
The system and method for the bionical obstruction-avoiding control system of a kind of unmanned plane based on LGMD provided by the invention, by building
LGMD neural network, and view field image is split, direction in space selection and scene are realized during unmanned plane during flying
Prediction realizes that unmanned plane is real-time in circumstances not known, the flight of efficient avoidance.
Detailed description of the invention
Fig. 1 is the structure connection diagram of system of the present invention；
Fig. 2 is LGMD neural network schematic diagram；
Fig. 3 is the flow diagram of the method for the invention；
Fig. 4 is that view field image divides schematic diagram；
Wherein: 1, flight subsystem；2, light stream sensor；3, driving motor；4, embedded LGMD detector；5,
Camera；6, wireless communication module；7, earth station PC；8, laser sensor.
Specific embodiment
The attached figures are only used for illustrative purposes and cannot be understood as limitating the patent；
In order to better illustrate this embodiment, the certain components of attached drawing have omission, zoom in or out, and do not represent actual product
Size；
To those skilled in the art, it is to be understood that certain known features and its explanation, which may be omitted, in attached drawing
's.
The following further describes the technical solution of the present invention with reference to the accompanying drawings and examples.
Embodiment 1
As shown in Figure 1, a kind of bionical obstruction-avoiding control system of unmanned plane based on LGMD, including flight subsystem 1, light
Flow sensor 2, driving motor 3, embedded LGMD detector 4, camera 5, wireless communication module 6 and earth station PC7；Wherein:
The light stream sensor 2, embedded LGMD detector 4 are electrically connected with the flight subsystem 1, are carried out
Information exchange；
The wireless communication module 6, driving motor 3 are electrically connected with 1 output end of flight subsystem；
Embedded 4 input terminal of LGMD detector is connect with 5 output end signal of camera；
The wireless communication module 6 and earth station PC7 wireless communication connects.
More specifically, the embedded LGMD detector 4 is provided with LGMD neural network, what the camera 5 acquired
Video information is calculated by the LGMD neural network, is obtained avoidance obstacle instruction by LGMD neural network, is exported to institute
Flight subsystem 1 is stated, realizes the avoidance obstacle of unmanned plane.
More specifically, as shown in Fig. 2, the LGMD neural network includes P layers of neuron, E layers of neuron, I layers of nerve
Member, S layers of neuron, G layers of neuron, LGMD neuron and feedforward inhibit FFI neuron；Wherein:
The P layers of neuron obtains the view field image information of input video, responds to frame difference, obtains P layers of neuron
Film potential；
Excited film potential of the P layers of membrane potential of neurons directly as the E layers of neuron；
The I layers of neuron receives the output of the P layers of neuron previous frame, and be inhibited film potential；
The S layers of neuron converges the output in I layers of neuron corresponding position field, by inhibiting film potential to institute
It states E layers of neuron to be inhibited, obtains S layers of membrane potential of neurons；
The G layers of neuron is used to enhance the colliding object extracted under complex background, is calculated according to S layers of membrane potential of neurons
Obtain G layers of membrane potential of neurons；
The LGMD neuron diagonally divides view field image information, obtains 4 azimuth informations；According to G layers of nerve
First film potential and 4 azimuth informations are calculated, and the film potential of 4 orientation C-LGMD is obtained, by the film of 4 orientation C-LGMD
Current potential is added, and obtains LGMD membrane potential of neurons；
The feedforward inhibits FFI neuron directly to obtain view field image information from the P layers of neuron, to LGMD nerve
First film potential, which is formed, to be inhibited, to obtain avoidance obstacle instruction output to the flight subsystem 1.
More specifically, the system also includes inertial sensor, the inertial sensor is integrated in flight control
In system, it is electrically connected with flight subsystem.
More specifically, the system also includes laser sensor 8, the laser sensor 8 and the light stream sensor 2 are defeated
Enter end to be electrically connected, be used cooperatively with the light stream sensor 2, the survey for being mainly used for unmanned plane is high, fixed high.
In the specific implementation process, the core of the flight subsystem 1 is STM32F407V；The inertial sensor
Main body is MPU6050 chip；2 main body of light stream sensor is Pix4Flow chip；The main body of the wireless communication module 6 is
nRF24L01；The inertial sensor for recording UAV Attitude information, the light stream sensor 2 as horizontal plane position and
Velocity feedback device, the camera 5 for acquiring video information in real time；The flight subsystem 1 is according to embedded LGMD
The instruction that detector 4 exports, calculates the corresponding PWM value of the driving motor 3, is finally output to four drivings electricity of unmanned plane
Machine 3 realizes avoidance obstacle；6 returning real-time data of wireless communication module gives earth station PC7.
Embodiment 2
More specifically, on the basis of embodiment 1, a kind of bionical avoidance obstacle method of the unmanned plane based on LGMD is provided,
The following steps are included:
S1: real time video collection is done by camera 5, obtains input video；
S2: embedded LGMD detector 4 obtains the view field image information of input video, obtains by LGMD neural computing
It is instructed to avoidance obstacle, exports the avoidance obstacle for realizing unmanned plane to flight subsystem 1.
More specifically, as shown in figure 3, the detailed process of LGMD neural computing avoidance obstacle instruction are as follows:
S21:P layers of neuron obtain the view field image information of input video, respond to frame difference, obtain P layers of neuron
Film potential Pf(x, y), specifically:
Pf(x, y)=Lf(x, y)-Lf-1(x, y)； 1)
Wherein: f indicates that the f frame in video sequence, (x, y) are position of the pixel in network layer, Lf(x, y) is
The pixel value of the view field image of input；
Input of the output of S22:P layers of neuron as E layers of neuron, I layers of neuron, E layers of neuron directly receive P layers
Neuron output, and I layers of neuron receive the output of P layers of neuron previous frame, and carry out local inhibition, are embodied as:
Ef(x, y)=Pf(x, y)； 2)
If(x, y)=∑i∑jPf-1(x+i, y+j) wi(i, j) (if i=j, j ≠ 0)； 3)
Wherein: Ef(x, y) is excited film potential, i.e. E layers of membrane potential of neurons；If(x, y) is to inhibit film potential, i.e. I layers of mind
Through first film potential；wi(i, j) is that part inhibits weight；I, j are not 0 simultaneously, and it is only it that the present embodiment, which locally inhibits weight matrix,
A kind of middle form.
S23:S layers of neuron converge the output in I layers of neuron corresponding position field, by inhibiting film potential to the E
Layer neuron is inhibited, and S layers of membrane potential of neurons are obtained, specifically:
Sf(x, y)=Ef(x, y)-If(x, y) WI； 5)
Wherein: Sf(x, y) is S layers of membrane potential of neurons；WITo inhibit weight matrix；
S24:G layers of neuron are used to enhance the colliding object extracted under complex background, are calculated according to S layers of membrane potential of neurons
G layers of membrane potential of neurons are obtained, specifically:
Wherein: Gf(x, y) is G layers of membrane potential of neurons；[we] it is convolution kernel, the present embodiment convolution kernel and its radius are only
One form of them；R is convolution nuclear radius, r=1；
Threshold value T is setdeFilter weaker excitement, specifically:
Wherein:For filtered G layers of membrane potential of neurons, CdeFor weak coefficient, it is [0,1]；TdeTo filter threshold
Value；
S25: when view field image information reaches LGMD neuron, view field image information is diagonally divided, such as Fig. 4 institute
Show, the coordinate obtained in two diagonal line y-axis is embodied as:
Wherein: Diag1, Diag2 are respectively the y-coordinate that two diagonal lines correspond to x；W is the width of image；H is the height of image
Degree；To obtain the film potential of 4 orientation C-LGMD, specifically:
Wherein: ULGMD, DLGMD, LLGMD, RLGMDThe respectively film potential in 4 orientation in image upper and lower, left and right；By 4 orientation
The film potential of C-LGMD is added, and obtains LGMD membrane potential of neurons Kf, specifically:
By KfThe range of [0,255] is normalized and is mapped in, specifically:
Wherein, ncellFor the total pixel quantity of image；
According to 4 orientation C-LGMD film potentials in entire image proportion, the film electricity after mapping in 4 orientation is obtained
Position, specifically:
Wherein,Respectively after the orientation mapping of 4, image upper and lower, left and right
Film potential；Work as κfMore than its threshold value Ts, then a LGMD peak pulse can be generatedSpecifically:
If continuous ntsPulse is no less than nsp, then judgement will collide, it is embodied as:
S26: the feedforward inhibits FFI neuron directly to obtain view field image information from the P layers of neuron, specific to indicate
Are as follows:
Wherein, FfInhibit the film potential of FFI neuron for feedforward；TFFIFor preset threshold；Work as FfMore than threshold value TFFI, LGMD
Membrane potential of neurons is suppressed immediately；
S27: direction selection is carried out: when feedforward inhibits FFI film potential FfGreater than threshold value TFFI, all avoidance instruction ignores；When
Cfinal=TURE, and feedover and inhibit FFI film potential FfLess than threshold value TFFIWhen, there is barrier in judgement, by comparing 4 orientationThe size of film potential, using the smallest orientation of film potential in 4 orientation as most safe
Avoidance direction；
In the specific implementation process, using the smallest orientation of film potential in 4 orientation as safest avoidance direction, and with
The output function U that C-LGMD film potential, pace of change and the acceleration in the avoidance direction constructf, the speed of avoidance is provided, is moved
Decision neuron collects LGMD neural network model calculated result, and exports as embedded LGMD detector, provides corresponding
Avoidance signal.
In the specific implementation process, it when generation avoidance signal, and detectsFilm potential is minimum, then the direction is most
Excellent avoidance direction, the output function U constructed in conjunction with the film potential, pace of change and accelerationf, provide the speed of avoidance.It is special
In the case of when barrier just in the picture between when, i.e. the equal situation of 4 orientation film potentials is then preferentially flown to the left；If going out
Show 2 or 3 orientation film potentials are identical, then preferentially to the opposite direction avoidance in the most strong direction of film potential.
More specifically, the output function is expressed as:
Wherein: k1, k2, k3For proportionality coefficient；
S28: flying scene prediction: work as Cfinal=FALSE and feedforward inhibition FFI film potential FfLess than threshold value TFFI, unmanned plane
Normal flight predicts the unknown flying scene of unmanned plane by acquiring 4 orientation film potentials of FFI；
S29: the signal that step S25, S27, S28 are obtained forms avoidance obstacle instruction as avoidance obstacle signal, and defeated
Out to flight subsystem.
More specifically, the process of the step S28 flying scene prediction specifically:
Scene set prediction threshold value TFFII, when N frame FFI forward is averaged film potentialLess than threshold value TFFIIWhen, unmanned plane normally flies
Row；When N frame FFI forward is averaged film potentialGreater than threshold value TFFII, it is less than threshold value TFFIWhen, pass through comparison
Size, the smallest orientation of film potential is found out, to predict under frontal scene, the following least side of heading barrier
Position；
Wherein, N frame FFI is averaged film potential forwardIt is embodied as:
Film potential average value in 4 orientationIt respectively indicates are as follows:
In formula,N frame film potential average value before expression 4, the upper and lower, left and right FFI orientation.
In the specific implementation process, heretofore described LGMD neural network model [1], by 5 layers of neuronal cell, i.e.,
P layers of neuron, E layers of neuron, I layers of neuron, S layers of neuron, G layers of neuron and LGMD neuron, feedforward inhibit FFI mind
Formed through member, it is very sensitive to quickly close object, essence be due to object close to when the edge constantly expanded stimulate and make
The excitement obtained in whole network increases sharply, and causes the film potential of LGMD neuron to steeply rise, this variation has network
Volume anti-collision warning function；By the way that view field image is divided into 4 orientation: upper and lower, left and right, the LGMD of 4 orientation competition is formed
Neuron, i.e. C-LGMD respond by comparing the LGMD avoidance of different direction, obtain optimal avoidance direction, realize unmanned plane
More efficient more flexible avoidance.
In the specific implementation process, the present invention records the image information in C-LGMD4 orientation in real time And then first derivative v is asked to the image information in each orientationU、vD、vL、vRAnd second dervative αU、
αD、αL、αR, picture signal variation is respectively represented in the velocity and acceleration in each orientation.Meanwhile adding up the FFI of nearly N frame forward
Film potential simultaneously acquires average valueWith the value in 4 orientation
Obviously, the above embodiment of the present invention be only to clearly illustrate example of the present invention, and not be pair
The restriction of embodiments of the present invention.For those of ordinary skill in the art, may be used also on the basis of the above description
To make other variations or changes in different ways.There is no necessity and possibility to exhaust all the enbodiments.It is all this
Made any modifications, equivalent replacements, and improvements etc., should be included in the claims in the present invention within the spirit and principle of invention
Protection scope within.
[1]Yue S,Rind F C.Collision detection in complex dynamic scenes using
an LGMD-based visual neural network with feature enhancement[J].IEEE
Transactions on Neural Networks,2006,17(3):705-716。
Claims (8)
1. a kind of bionical obstruction-avoiding control system of unmanned plane based on LGMD, it is characterised in that: including flight subsystem (1),
Light stream sensor (2), driving motor (3), embedded LGMD detector (4), camera (5), wireless communication module (6) and ground
It stands PC (7)；Wherein:
The light stream sensor (2), embedded LGMD detector (4) are electrically connected with the flight subsystem (1), into
Row information interaction；
The wireless communication module (6), driving motor (3) are electrically connected with the flight subsystem (1) output end；
Embedded LGMD detector (4) input terminal is connect with the camera (5) output end signal；
The wireless communication module (6) and earth station PC (7) wireless communication connect.
2. the bionical obstruction-avoiding control system of a kind of unmanned plane based on LGMD according to claim 1, it is characterised in that: in institute
It states embedded LGMD detector (4) and is provided with LGMD neural network, described in the video information process that the camera (5) acquires
LGMD neural network is calculated, and is obtained avoidance obstacle instruction by LGMD neural network, is exported to the flight subsystem
(1), the avoidance obstacle of unmanned plane is realized.
3. the bionical obstruction-avoiding control system of a kind of unmanned plane based on LGMD according to claim 2, it is characterised in that: described
LGMD neural network includes P layers of neuron, E layers of neuron, I layers of neuron, S layers of neuron, G layers of neuron, LGMD neuron
Inhibit FFI neuron with feedforward；Wherein:
The P layers of neuron obtains the view field image information of input video, responds to frame difference, obtains P layers of neuron membrane electricity
Position；
Excited film potential of the P layers of membrane potential of neurons directly as the E layers of neuron；
The I layers of neuron receives the output of the P layers of neuron previous frame, and carries out local inhibition, and be inhibited film potential；
The S layers of neuron converges the output in I layers of neuron corresponding position field, by inhibiting film potential to the E
Layer neuron is inhibited, and S layers of membrane potential of neurons are obtained；
The G layers of neuron is used to enhance the colliding object extracted under complex background, is calculated according to S layers of membrane potential of neurons
G layers of membrane potential of neurons；
The LGMD neuron diagonally divides view field image information, obtains 4 azimuth informations；According to G layers of neuron membrane
Current potential and 4 azimuth informations are calculated, and the film potential of 4 orientation C-LGMD is obtained, by the film potential of 4 orientation C-LGMD
It is added, obtains LGMD membrane potential of neurons；
The feedforward inhibits FFI neuron directly to obtain view field image information from the P layers of neuron, to LGMD neuron membrane electricity
Position, which is formed, to be inhibited, so that obtaining avoidance obstacle instruction output gives the flight subsystem (1).
4. the bionical obstruction-avoiding control system of a kind of unmanned plane based on LGMD according to claim 3, it is characterised in that: also wrap
Inertial sensor is included, the inertial sensor is integrated on the flight subsystem (1), with flight subsystem (1)
It is electrically connected.
5. the bionical obstruction-avoiding control system of a kind of unmanned plane based on LGMD according to claim 4, it is characterised in that: also wrap
It includes laser sensor (8), the laser sensor (8) and the light stream sensor (2) input terminal are electrically connected.
6. a kind of method for applying a kind of bionical obstruction-avoiding control system of unmanned plane based on LGMD as claimed in claim 5,
It is characterized in that: the following steps are included:
S1: real time video collection is done by camera (5), obtains input video；
S2: embedded LGMD detector (4) obtains the view field image information of input video, obtains by LGMD neural computing
Avoidance obstacle instruction, exports the avoidance obstacle that unmanned plane is realized to flight subsystem (1).
7. a kind of bionical avoidance obstacle method of unmanned plane based on LGMD according to claim 6, it is characterised in that: described
The detailed process of LGMD neural computing avoidance obstacle instruction are as follows:
S21:P layers of neuron obtain the view field image information of input video, respond to frame difference, obtain P layers of neuron membrane electricity
Position Pf(x, y), specifically:
Pf(x, y)=Lf(x, y)-Lf-1(x, y)； 1)
Wherein: f indicates that the f frame in video sequence, (x, y) are position of the pixel in network layer, Lf(x, y) is input
The pixel value of view field image；
Input of the output of S22:P layers of neuron as E layers of neuron, I layers of neuron, E layers of neuron directly receive P layers it is neural
Member output, and I layers of neuron receive the output of P layers of neuron previous frame, and carry out local inhibition, are embodied as:
Ef(x, y)=Pf(x, y)； 2)
If(x, y)=∑i∑jPf-1(x+i, y+j) wi(i, j) (if i=j, j ≠ 0)； 3)
Wherein: Ef(x, y) is excited film potential, i.e. E layers of membrane potential of neurons；If(x, y) is to inhibit film potential, i.e. I layers of neuron
Film potential；wi(i, j) is that part inhibits weight；I, j are not 0 simultaneously；
S23:S layers of neuron converge the output in I layers of neuron corresponding position field, by inhibiting film potential to the E layers of mind
Inhibited through member, obtain S layers of membrane potential of neurons, specifically:
Sf(x, y)=Ef(x, y)-If(x, y) WI； 5)
Wherein: Sf(x, y) is S layers of membrane potential of neurons；WITo inhibit weight matrix；
S24:G layers of neuron are used to enhance the colliding object extracted under complex background, are calculated according to S layers of membrane potential of neurons
G layers of membrane potential of neurons, specifically:
Wherein: Gf(x, y) is G layers of membrane potential of neurons；[we] it is convolution kernel；R is convolution nuclear radius, r=1；
Threshold value T is setdeFilter weaker excitement, specifically:
Wherein:For filtered G layers of membrane potential of neurons, CdeFor weak coefficient, it is [0,1]；TdeFor filtering threshold；
S25: when view field image information reaches LGMD neuron, view field image information is diagonally divided, it is diagonal to obtain two
Coordinate in line y-axis is embodied as:
Wherein: Diag1, Diag2 are respectively the y-coordinate that two diagonal lines correspond to x；W is the width of image；H is the height of image；From
And the film potential of 4 orientation C-LGMD is obtained, specifically:
Wherein: ULGMD, DLGMD, LLGMD, RLGMDThe respectively film potential in 4 orientation in image upper and lower, left and right；By 4 orientation C-
The film potential of LGMD is added, and obtains LGMD membrane potential of neurons Kf, specifically:
By KfThe range of [0,255] is normalized and is mapped in, specifically:
Wherein, ncellFor the total pixel quantity of image；
According to 4 orientation C-LGMD film potentials in entire image proportion, the film potential after mapping in 4 orientation, tool are obtained
Body are as follows:
Wherein,Film electricity respectively after the orientation mapping of 4, image upper and lower, left and right
Position；Work as κfMore than its threshold value Ts, then a LGMD peak pulse can be generatedSpecifically:
If continuous ntsPulse is no less than nsp, then judgement will collide, it is embodied as:
S26: the feedforward inhibits FFI neuron directly to obtain view field image information from the P layers of neuron, is embodied as:
Wherein, FfInhibit the film potential of FFI neuron for feedforward；TFFIFor preset threshold；Work as FfMore than threshold value TFFI, LGMD nerve
First film potential is suppressed immediately；
S27: direction selection is carried out: when feedforward inhibits FFI film potential FfGreater than threshold value TFFI, all avoidance instruction ignores；Work as Cfinal
=TURE, and feedover and inhibit FFI film potential FfLess than threshold value TFFIWhen, there is barrier in judgement, by comparing 4 orientationThe size of film potential, using the smallest orientation of film potential in 4 orientation as most safe
Avoidance direction；
S28: flying scene prediction: work as Cfinal=FALSE and feedforward inhibition FFI film potential FfLess than threshold value TFFI, unmanned plane is normal
Flight, the film potential in 4 orientation by acquiring FFI, predicts the unknown flying scene of unmanned plane；
S29: the signal that step S25, S27, S28 are obtained forms avoidance obstacle instruction as avoidance obstacle signal.
8. a kind of bionical avoidance obstacle method of unmanned plane based on LGMD according to claim 7, it is characterised in that: described
The process of step S28 flying scene prediction specifically:
Scene set prediction threshold value TFFII, when N frame FFI forward is averaged film potentialLess than threshold value TFFIIWhen, unmanned plane normally flies
Row；When N frame FFI forward is averaged film potentialGreater than threshold value TFFII, it is less than threshold value TFFIWhen, pass through comparison
Size, the smallest orientation of film potential is found out, to predict under frontal scene, the following least side of heading barrier
Position；
Wherein, N frame FFI is averaged film potential forwardIt is embodied as:
Film potential average value in 4 orientationIt respectively indicates are as follows:
In formula,Indicate preceding N frame film potential average value in the orientation of 4, the upper and lower, left and right FFI.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910281845.5A CN109960278B (en) | 2019-04-09 | 2019-04-09 | LGMD-based bionic obstacle avoidance control system and method for unmanned aerial vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910281845.5A CN109960278B (en) | 2019-04-09 | 2019-04-09 | LGMD-based bionic obstacle avoidance control system and method for unmanned aerial vehicle |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109960278A true CN109960278A (en) | 2019-07-02 |
| CN109960278B CN109960278B (en) | 2022-01-28 |
Family
ID=67025945
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910281845.5A Active CN109960278B (en) | 2019-04-09 | 2019-04-09 | LGMD-based bionic obstacle avoidance control system and method for unmanned aerial vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109960278B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110908399A (en) * | 2019-12-02 | 2020-03-24 | 广东工业大学 | Unmanned aerial vehicle autonomous obstacle avoidance method and system based on light weight type neural network |
| CN111831010A (en) * | 2020-07-15 | 2020-10-27 | 武汉大学 | Unmanned aerial vehicle obstacle avoidance flight method based on digital space slice |
| CN114217621A (en) * | 2021-12-15 | 2022-03-22 | 中国科学院深圳先进技术研究院 | Robot collision sensing method and sensing system based on bionic insect vision |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2010129907A2 (en) * | 2009-05-08 | 2010-11-11 | Scientific Systems Company Inc. | Method and system for visual collision detection and estimation |
| CN107170289A (en) * | 2017-06-01 | 2017-09-15 | 岭南师范学院 | One kind is based on optics multi-vision visual vehicle rear-end collision early warning system |
| US9984326B1 (en) * | 2015-04-06 | 2018-05-29 | Hrl Laboratories, Llc | Spiking neural network simulator for image and video processing |
| CN108475058A (en) * | 2016-02-10 | 2018-08-31 | 赫尔实验室有限公司 | Time to contact estimation rapidly and reliably is realized so as to the system and method that carry out independent navigation for using vision and range-sensor data |
| CN208126205U (en) * | 2018-04-28 | 2018-11-20 | 上海工程技术大学 | A kind of unmanned flight's device of automatic obstacle-avoiding |
-
2019
- 2019-04-09 CN CN201910281845.5A patent/CN109960278B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2010129907A2 (en) * | 2009-05-08 | 2010-11-11 | Scientific Systems Company Inc. | Method and system for visual collision detection and estimation |
| US9984326B1 (en) * | 2015-04-06 | 2018-05-29 | Hrl Laboratories, Llc | Spiking neural network simulator for image and video processing |
| CN108475058A (en) * | 2016-02-10 | 2018-08-31 | 赫尔实验室有限公司 | Time to contact estimation rapidly and reliably is realized so as to the system and method that carry out independent navigation for using vision and range-sensor data |
| CN107170289A (en) * | 2017-06-01 | 2017-09-15 | 岭南师范学院 | One kind is based on optics multi-vision visual vehicle rear-end collision early warning system |
| CN208126205U (en) * | 2018-04-28 | 2018-11-20 | 上海工程技术大学 | A kind of unmanned flight's device of automatic obstacle-avoiding |
Non-Patent Citations (6)
| Title |
|---|
| CHENG HU: "A bio-inspired embedded vision system for autonomous micro-robots: the LGMD case；Cheng Hu", 《 IEEE TRANSACTIONS ON COGNITIVE AND DEVELOPMENTAL SYSTEMS》 * |
| JIANNAN ZHAO: "A Bio-inspired Collision Detector for Small Quadcopter", 《 2018 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN)》 * |
| LLEWYN SALT: "Obstacle avoidance with LGMD neuron: towards a neuromorphic UAV implementation", 《 2017 IEEE INTERNATIONAL SYMPOSIUM ON CIRCUITS AND SYSTEMS (ISCAS)》 * |
| QINBING FU: "Collision selective LGMDs neuron models research benefits from a vision-based autonomous micro robot", 《 2017 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS)》 * |
| SHIGANG YUE: "Collision detection in complex dynamic scenes using an LGMD-based visual neural network with feature enhancement，", 《IEEE TRANSACTIONS ON NEURAL NETWORKS》 * |
| 张国鹏: "基于生物视觉的碰撞预警传感器", 《传感器与微系统》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110908399A (en) * | 2019-12-02 | 2020-03-24 | 广东工业大学 | Unmanned aerial vehicle autonomous obstacle avoidance method and system based on light weight type neural network |
| CN111831010A (en) * | 2020-07-15 | 2020-10-27 | 武汉大学 | Unmanned aerial vehicle obstacle avoidance flight method based on digital space slice |
| CN114217621A (en) * | 2021-12-15 | 2022-03-22 | 中国科学院深圳先进技术研究院 | Robot collision sensing method and sensing system based on bionic insect vision |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109960278B (en) | 2022-01-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109960278A (en) | A kind of bionical obstruction-avoiding control system of unmanned plane based on LGMD and method | |
| CN107817820A (en) | A kind of unmanned plane autonomous flight control method and system based on deep learning | |
| CN113359810A (en) | Unmanned aerial vehicle landing area identification method based on multiple sensors | |
| Iida | Biologically inspired visual odometer for navigation of a flying robot | |
| Bian et al. | A novel monocular-based navigation approach for UAV autonomous transmission-line inspection | |
| US11561544B2 (en) | Indoor monocular navigation method based on cross-sensor transfer learning and system thereof | |
| CN107352032A (en) | A kind of monitoring method and unmanned plane of flow of the people data | |
| Rodríguez-Gomez et al. | Asynchronous event-based clustering and tracking for intrusion monitoring in UAS | |
| JP2013020644A (en) | Hybrid control device | |
| Thakoor et al. | BEES: Exploring mars with bioinspired technologies | |
| CN105404894A (en) | Target tracking method used for unmanned aerial vehicle and device thereof | |
| CN114474061A (en) | Robot multi-sensor fusion positioning navigation system and method based on cloud service | |
| CN116343330A (en) | Abnormal behavior identification method for infrared-visible light image fusion | |
| Huber et al. | On robots and flies: Modeling the visual orientation behavior of flies | |
| CN106155082B (en) | A kind of unmanned plane bionic intelligence barrier-avoiding method based on light stream | |
| DE112019007390T5 (en) | Method, device and computer program for tracking moving objects | |
| Iida et al. | Navigation in an autonomous flying robot by using a biologically inspired visual odometer | |
| Liu et al. | A novel trail detection and scene understanding framework for a quadrotor UAV with monocular vision | |
| CN111611869A (en) | End-to-end monocular vision obstacle avoidance method based on serial deep neural network | |
| Park et al. | A track-based human movement analysis and privacy protection system adaptive to environmental contexts | |
| Moeys et al. | Pred18: Dataset and further experiments with davis event camera in predator-prey robot chasing | |
| CN107225571A (en) | Motion planning and robot control method and apparatus, robot | |
| Dijkshoorn et al. | Amsterdam Oxford Joint Rescue Forces− Team Description Paper− Virtual Robot competition− Rescue Simulation League− RoboCup 2011 | |
| Hu et al. | Coping with multiple visual motion cues under extremely constrained computation power of micro autonomous robots | |
| Lewis | Detecting surface features during locomotion using optic flow |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20231211 Address after: 230000 floor 1, building 2, phase I, e-commerce Park, Jinggang Road, Shushan Economic Development Zone, Hefei City, Anhui Province Patentee after: Dragon totem Technology (Hefei) Co.,Ltd. Address before: 524037 No.29 Cunjin Road, Chikan District, Zhanjiang City, Guangdong Province Patentee before: LINGNAN NORMAL University |
|
| TR01 | Transfer of patent right |