CN113247246B - Flapping wing aircraft cruise control system based on asynchronous multiple cameras - Google Patents
Flapping wing aircraft cruise control system based on asynchronous multiple cameras Download PDFInfo
- Publication number
- CN113247246B CN113247246B CN202110722807.6A CN202110722807A CN113247246B CN 113247246 B CN113247246 B CN 113247246B CN 202110722807 A CN202110722807 A CN 202110722807A CN 113247246 B CN113247246 B CN 113247246B
- Authority
- CN
- China
- Prior art keywords
- flapping wing
- infrared camera
- wing aircraft
- control
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64C—AEROPLANES; HELICOPTERS
- B64C33/00—Ornithopters
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
- G05D1/10—Simultaneous control of position or course in three dimensions
- G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
Abstract
The invention discloses a flapping wing aircraft cruise control system based on asynchronous multiple cameras, which comprises an asynchronous multi-view vision module, a control module and a flapping wing aircraft; a plurality of reflecting balls are fixed on the flapping wing aircraft, the asynchronous multi-view vision module comprises a plurality of infrared cameras, and the position and posture information of the flapping wing aircraft is calculated by capturing images of the reflecting balls; the control module comprises an upper computer and issues a control command to the flapping wing air vehicle according to the obtained position and attitude information of the flapping wing air vehicle and a designed cruise control method. The invention solves the asynchronous problem of multiple cameras, reduces the construction condition of a stereoscopic vision system, realizes the closed-loop control of the flapping-wing aircraft, and has wide application range.
Description
Technical Field
The invention relates to the technical field of machine vision and ornithopter control, in particular to an ornithopter cruise control system based on asynchronous multiple cameras.
Background
In recent years, stereoscopic vision has been widely used in the fields of robot positioning, three-dimensional measurement, and scene modeling. Most current research is focused on synchronized binocular/multi-view camera systems, but these methods cannot be directly used for three-dimensional reconstruction of unsynchronized binocular/multi-view cameras. The reasons for non-synchronization are mainly two kinds: firstly, the problem of asynchronous shooting caused by different camera frame rates; secondly, the time delay problem when transmitting and receiving the camera data can not ensure the time corresponding relation of each frame. The problem of non-synchronization is solved, the application range of stereoscopic vision can be widened, a wired data transmission mode is changed into a wireless data transmission mode, and a stereoscopic vision system is simplified. In addition, cameras with different frequencies can also be used for constructing a stereoscopic vision system, and the requirement of the system on the property of hardware is reduced.
In addition, the flapping wing aircraft is taken as a current research hotspot, flies in a flying mode simulating birds or insects in nature, has good concealment, and has wide application prospect in the military field. In addition, the low power consumption of the flapping wing air vehicle also makes the flapping wing air vehicle have certain development prospect in the civil field. At present, the control method of the flapping wing air vehicle is mainly manual remote control, and the mode needs high concentration of an operator. Therefore, in order to conveniently and quickly carry out the experimental verification of the control algorithm of the flapping wing aircraft indoors, it is very meaningful to design a cruise control system of the flapping wing aircraft based on asynchronous multiple cameras.
Disclosure of Invention
The invention aims to provide a cruise control system of a flapping wing aircraft based on asynchronous multiple cameras, which solves the asynchronous problem of the multiple cameras, reduces the construction conditions of a stereoscopic vision system, realizes closed-loop control of the flapping wing aircraft, and controls the flapping wing aircraft to complete a cruise task.
To solve the above technical problem, an embodiment of the present invention provides the following solutions:
a flapping wing aircraft cruise control system based on asynchronous multiple cameras comprises an asynchronous multi-view vision module, a control module and a flapping wing aircraft;
a plurality of reflecting balls are fixed on the flapping wing aircraft, the reflecting balls are infrared reflecting marker balls, and the flapping wing aircraft takes off in a manual control mode;
the asynchronous multi-view vision module comprises a plurality of infrared cameras, a circle of infrared light source is arranged around each infrared camera, the infrared cameras capture images of the infrared cameras by reflecting infrared light, and the asynchronous multi-view vision module calculates the position and posture information of the flapping wing aircraft according to the captured images of the plurality of reflective balls;
the control module comprises an upper computer and is used for switching a control mode into a cruise mode and issuing a control command to the flapping wing air vehicle according to the obtained position and attitude information of the flapping wing air vehicle and a designed cruise control method.
Preferably, the asynchronous multi-view vision module comprises a plurality of infrared cameras, a plurality of reflective balls fixed on the flapping wing aircraft, a router and a stereo vision platform; the flapping wing aircraft comprises a flight control plate, a steering engine, a motor and a wireless serial port module A; the control module comprises an upper computer and a wireless serial port module B;
the infrared camera and the router are in communication in a wireless mode through a TCP/IP protocol, the router and the stereoscopic vision platform are in communication in a wireless mode through a TCP/IP protocol, the stereoscopic vision platform is in communication with the upper computer, the stereoscopic vision platform is connected with a display interface, the upper computer is in communication with the flight control board through the wireless serial port module A and the wireless serial port module B, and the steering engine and the motor are connected with the flight control board.
Preferably, the number of the light reflecting balls is not less than 4.
Preferably, a Linux kernel is embedded in the infrared camera in the asynchronous multi-view vision module, and the Linux kernel can record a timestamp corresponding to each captured image, and package and send the image and the timestamp of each frame to the stereoscopic vision platform;
the infrared camera is also used for preprocessing the image, obtaining the pixel coordinates of the light reflecting balls in the image according to an extraction algorithm and transmitting the pixel coordinates of the light reflecting balls to the stereoscopic vision platform;
the stereoscopic vision platform is used for issuing clock information to different infrared cameras to synchronize clocks.
Preferably, the stereoscopic vision platform is further configured to:
taking a first infrared camera with lower frequency as a reference, searching two frames of images closest to the first infrared camera in a second infrared camera through a timestamp, performing approximate processing on the motion of a reflective ball between the two frames of images, regarding the infrared camera as uniform linear motion when the frequency of the infrared camera is greater than a preset frequency, and performing interpolation through the two frames of image information:
wherein the content of the first and second substances,
is a time stamp corresponding to a frame in the first infrared camera, and is satisfied by finding the time stamp in an image captured by the second infrared camera
The adjacent two frame images of the image of the adjacent two frames,
and
respectively corresponding to the N frame and the (N + 1) frame image information of the second infrared camera
And
(ii) a The image information refers to the pixel coordinates of each reflective ball in each infrared camera, and the image information is interpolated and fitted into one infrared camera and the first infrared camera by the above formula
Second infrared camera reflective ball information corresponding to time image information time
。
Preferably, before performing the interpolation, the stereoscopic platform is further configured to:
calculating the number of the reflective balls in the image frame corresponding to the moment in the first infrared camera, and if the number of the reflective balls is not equal to the set number, ignoring the image information of the frame and carrying out synchronous processing on the next frame of image; otherwise, carrying out the next operation;
and searching two adjacent frames of images which are closest to each other in the image sequence of the second infrared camera according to the timestamp, similarly judging whether the number of the reflective balls in the two frames of images is equal to the set number, and if the number of the reflective balls in the two frames of images is equal to the set number, performing interpolation fitting.
Preferably, the stereoscopic vision platform is further configured to:
after a certain frame of image of the reference infrared camera is processed, mark point three-dimensional reconstruction is carried out on the obtained fitting matching image frame to obtain position information of each mark point, and rigid body pose estimation is carried out on each mark point to obtain position and attitude information of the flapping wing aircraft.
Preferably, the control module switches the control mode to the cruise mode, and specifically includes:
setting the circle center and the radius of a cruise track circle;
calculating and judging whether the common view field of the infrared camera meets the requirement of the set track, if so, carrying out the next step, otherwise, resetting the circle center and the radius of the cruise track circle;
and after the setting is successful, the manual control mode is switched off, and the upper computer controls the flapping wing aircraft.
Preferably, the flight control panel is used for analyzing a control instruction transmitted by an upper computer and outputting a PWM value to control the rotating speed of the motor and the angle of the steering engine; the rotating speed of the motor is used for controlling the flapping frequency of the flapping wing air vehicle so as to adjust the flying height; the steering engine angle is used for controlling the tail wing angle of the flapping wing air vehicle, so that the steering of the flapping wing air vehicle is controlled.
Preferably, the control module adopts two control rings to carry out cruise flight control on the flapping wing aircraft, wherein the two control rings are a height ring and a position ring respectively; the height ring adjusts the flapping frequency of the flapping wing air vehicle through the PID controller, the position ring calculates the required acceleration through an outer ring L1 guidance law, and then the tail wing phase difference of the flapping wing air vehicle is controlled through an inner ring PID controller.
The technical scheme provided by the embodiment of the invention has the beneficial effects that at least:
(1) the flapping wing aircraft cruise control system based on the asynchronous multiple cameras solves the asynchronous problem of the multiple cameras, reduces the building condition of a stereoscopic vision system, and is suitable for a wireless communication scheme and a wired communication scheme of the cameras and an upper computer. Moreover, the asynchronous multi-view vision module does not require the frame rate of the camera, and is suitable for cameras with different frame rates.
(2) The invention provides an autonomous cruise control scheme of an ornithopter, which is characterized in that infrared reflecting marker balls are placed on the ornithopter, and a non-synchronous multi-view vision module reconstructs the spatial position of each reflecting ball according to a solid geometry principle; then calculating the flight attitude of the flapping wing aircraft by a rigid body attitude estimation algorithm, and displaying the flight attitude of the flapping wing aircraft by an interface; meanwhile, the pose information is input into an upper computer control module, and the required control quantity of the motor and the steering engine of the flapping wing aircraft is obtained through calculation according to a set cruise control method; finally, the control signal is sent to the flapping wing air vehicle through the wireless serial port, the flapping wing air vehicle completes the adjustment of the corresponding motor and the steering engine according to the control signal, and the adjustment of the height and the position is realized, so that the closed-loop control of the flapping wing air vehicle is performed, and the cruising task of the flapping wing air vehicle is realized.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a schematic structural diagram of a cruise control system of an ornithopter based on asynchronous multiple cameras, provided by an embodiment of the invention;
FIG. 2 is a pictorial illustration of a cruise control system for an ornithopter according to an embodiment of the present invention;
fig. 3 is a flowchart of an asynchronous binocular stereo vision algorithm provided by an embodiment of the present invention;
FIG. 4 is a flowchart of an asynchronous multi-view stereo vision algorithm provided by an embodiment of the present invention;
FIG. 5 is a flow chart of an flapping wing aircraft cruise altitude ring control algorithm provided by an embodiment of the present invention;
FIG. 6 is a flowchart of an ornithopter cruise position loop control algorithm according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
An embodiment of the invention provides an ornithopter cruise control system based on asynchronous multiple cameras, which comprises an asynchronous multi-view vision module, a control module and an ornithopter, as shown in fig. 1;
the flapping wing air vehicle is fixed with a plurality of reflective balls, wherein the reflective balls are infrared reflective marker balls, and the flapping wing air vehicle takes off in a manual control mode;
the asynchronous multi-view vision module comprises a plurality of infrared cameras, a circle of infrared light source is arranged around each infrared camera, the infrared cameras capture images of the infrared cameras by the aid of the reflective balls through reflecting infrared light, and the asynchronous multi-view vision module calculates position and posture information of the flapping wing aircraft according to the captured images of the reflective balls;
the control module comprises an upper computer and is used for switching the control mode into a cruise mode and issuing a control command to the flapping wing air vehicle according to the obtained position and attitude information of the flapping wing air vehicle and the designed cruise control method.
Specifically, the asynchronous multi-view vision module comprises a plurality of infrared cameras, a plurality of reflective balls fixed on the flapping wing aircraft, a router and a stereoscopic vision platform; the flapping wing aircraft comprises a flight control plate, a steering engine, a motor and a wireless serial port module A; the control module comprises an upper computer and a wireless serial port module B;
the infrared camera and the router are communicated in a wireless mode through a TCP/IP protocol, the router and the stereoscopic vision platform are communicated in a wireless mode through a TCP/IP protocol, the stereoscopic vision platform and the upper computer are communicated in a wired or wireless mode, the stereoscopic vision platform is connected with the display interface, the upper computer and the flight control board are communicated through the wireless serial port module A and the wireless serial port module B, and the steering engine and the motor are connected with the flight control board.
The application process of the system is as follows:
s1: a plurality of reflecting balls are fixed on the flapping wing air vehicle;
s2: manually controlling the flapping wing aircraft to take off in a manual control mode;
s3: acquiring the position and attitude information of the flapping wing aircraft through an asynchronous multi-view vision module;
s4: the upper computer is switched to a cruise mode, signals of a manual remote controller are cut off, the upper computer obtains position and attitude information of the ornithopter through the asynchronous multi-view vision module, and control instructions are issued to the ornithopter according to a designed cruise control method;
s5: the flapping wing air vehicle receives and executes the control command, if the control mode is not switched, the flapping wing air vehicle always carries out autonomous cruise flight, and when the control mode is switched to the manual control mode, the flapping wing air vehicle can be manually controlled.
FIG. 2 is a schematic diagram of an asynchronous multi-camera-based ornithopter cruise control system according to an embodiment of the present invention. The controlled main body of the invention is particularly an ornithopter, which is designed aiming at the special flight mode of the ornithopter, the ornithopter of the invention provides flight lift force by a motor, and two steering engines respectively control the swinging of a left empennage and a right empennage to generate phase difference, thereby realizing the steering of the ornithopter.
The flapping wing aircraft part is represented by a flight control system taking a flight control panel as a core. In order to simplify the flapping wing aircraft part, only a wireless communication module is carried on the flight control panel and used for receiving control commands.
The flight control panel in this embodiment uses the STM32 chip as a processing unit, and the wireless communication module, the motor and the steering engine are controlled by the flight control panel. The wireless communication module receives the control instruction transmitted from the upper computer wireless communication module, the flight control panel is responsible for analyzing the control instruction and converting the control instruction into PWM control signals of the motor and the steering engine to control the rotation of the motor and the steering engine of the flapping wing aircraft, and closed-loop control is realized by continuously receiving the control instruction transmitted by the upper computer and executing the control instruction.
The control module mainly has the function that the control mode of the flapping wing air vehicle is switched through the upper computer, when the flapping wing air vehicle is positioned in a public visual field of the infrared camera, the circle center and the radius of the track circle are correctly set through selecting a cruise mode, and the flapping wing air vehicle is autonomously controlled. When the task is completed, the upper computer is switched to a manual control mode, and the flapping wing air vehicle recovers the manual control mode.
Specifically, the switching process of the control module includes:
setting the circle center and the radius of a cruise track circle;
calculating and judging whether the common view field of the infrared camera meets the requirement of the set track, if so, carrying out the next step, otherwise, resetting the circle center and the radius of the cruise track circle;
and after the setting is successful, the manual control mode is switched off, and the upper computer controls the flapping wing aircraft.
When the frame rates of the infrared cameras are different, the asynchronous multi-view vision module cannot ensure that the received images of different infrared cameras are at the same moment when receiving image information captured by the infrared cameras, and when the frame rates of the infrared cameras are different greatly, the position of a reflective ball is solved by a stereoscopic vision method to generate a large error, which greatly affects the control accuracy of the flapping wing aircraft; secondly, if the infrared cameras are not triggered by using the synchronous trigger, the images of the infrared cameras received by the asynchronous multi-view vision module are not captured at the same time; finally, there may be some lack of a frame of image due to some trigger that does not initiate the capture of an image by the infrared camera.
In order to solve the problems, the invention designs an asynchronous multi-view vision module, and records the time of each frame of image captured by each infrared camera. Specifically, the Linux kernel is embedded in the infrared camera used by the asynchronous multi-view vision module, a timestamp corresponding to each captured image can be recorded through the system, and the image information and the timestamp of each frame are sent to the stereoscopic vision platform. In addition, the infrared camera can also preprocess the image, can directly obtain the pixel coordinates of each reflective ball in the image according to a corresponding extraction algorithm, and directly transmits the pixel coordinates of each reflective ball to the stereoscopic vision platform, so that the data transmission amount is reduced, and the transmission efficiency is improved. The used transmission protocol is TCP/IP protocol, which can ensure that all information is transmitted to the stereoscopic vision platform without error.
Fig. 3 is a flowchart of an asynchronous binocular stereo vision algorithm provided by an embodiment of the present invention, which is used for processing an asynchronous problem of a binocular camera. Because the clock of each infrared camera may have errors, the stereoscopic vision platform is first required to send clock information to different infrared cameras to synchronize the clocks.
And when the clocks of the infrared cameras are synchronized, image information starts to be captured, packaged with the time stamp and sent to the stereoscopic vision platform.
The stereoscopic vision platform uses a first infrared camera with lower frequency as a reference, two frames of images closest to the first infrared camera are searched in a second infrared camera through a timestamp, approximate processing is carried out on the motion of a reflective ball between the two frames of images, when the frequency of the infrared camera is greater than a preset frequency (namely the frequency of the infrared camera is large enough), the infrared camera is regarded as uniform linear motion, and interpolation is carried out through the two frames of image information:
wherein the content of the first and second substances,
is a time stamp corresponding to a frame in the first infrared camera, and is satisfied by finding the time stamp in an image captured by the second infrared camera
The adjacent two frame images of the image of the adjacent two frames,
and
respectively corresponding to the N frame and the (N + 1) frame image information of the second infrared camera
And
(ii) a The image information refers to the pixel coordinates of each reflective ball in each infrared camera, and the image information is interpolated and fitted into one infrared camera and the first infrared camera by the above formula
Second infrared camera reflective ball information corresponding to time image information time
。
In practice, there is a case where the infrared reflective ball exceeds the field of view of a certain infrared camera, and the infrared camera may lose information of the reflective ball, so that further improvement is needed for the interpolation method.
Preferably, the number of the reflective balls arranged on the flapping wing aircraft is not less than 4, and n is equal to 4.
First, in the first infrared camera
And if the number of the reflective balls in the image frame corresponding to the moment is not equal to 4, directly neglecting the image information of the frame and carrying out synchronous processing on the next frame of image. Otherwise, the next operation is carried out.
According to time stamp
And searching two adjacent frames of images which are closest to each other in the image sequence of the second infrared camera, similarly judging whether the number of the reflective balls in the two frames of images is equal to 4 or not, and if the number of the reflective balls in the two frames of images is equal to 4, carrying out interpolation fitting on the reflective balls according to the method.
Before interpolation, it is necessary to pre-process the coordinates of image points of the image frames that meet the conditions. The image points of the image frame are sorted, i.e. matched with the actual points. By designing the placing positions of the light reflecting balls, the corresponding pixel coordinates of the four light reflecting balls in the image frame are distinguished according to a matching method, and the pixel coordinates are sequentially arranged so as to facilitate the calculation of interpolation.
If the number of the light reflecting balls in only one image of two adjacent images with the closest time in the second infrared camera is equal to 4, the frame is taken as the image in the first infrared camera
A synchronous image corresponding to the time image; and if the number of the reflecting balls in the two frames is not more than 4, neglecting the image information of the frame of the first infrared camera and processing the next frame.
And after processing a certain frame of image of the reference infrared camera, performing mark point three-dimensional reconstruction on the obtained fitting matching image frame to obtain position information of each mark point, and performing rigid body pose estimation on each mark point to obtain position and attitude information of the flapping wing aircraft.
The flight radius of the flapping wing aircraft is large, the view fields of the two infrared cameras cannot meet the requirement of cruise flight of the flapping wing aircraft, so that the number of the infrared cameras needs to be increased, the size of a public view field formed by all the infrared cameras needs to meet the flight area of the flapping wing aircraft, and particularly, the public view field refers to the area which can be captured by at least two infrared cameras.
Fig. 4 is a flowchart of an asynchronous multi-view stereo vision algorithm provided by an embodiment of the present invention, which deals with the asynchronous problem of multiple cameras. Referring to the synchronization scheme of the two cameras, as the clock of each infrared camera may have errors, the stereoscopic vision platform issues clock information to different infrared cameras to synchronize the clocks.
And when the clocks of the infrared cameras are synchronized, image information starts to be captured, packaged with the time stamp and sent to the stereoscopic vision platform.
For the acquired image frame processing, the infrared camera with the lowest frequency in all the infrared cameras is firstly determined, and the image frame captured by the infrared camera is used as a reference. Similarly, the image frames of the reference infrared camera are selected according to the time sequence, the number of image points in the image frames of the reference infrared camera is not required to meet 4 when multiple cameras are used, two images with the closest time are directly searched in the image sequences of all the infrared cameras, the fitted or approximate matched image frames are obtained from the rest infrared cameras by analogy of the processing modes of the two cameras, as long as the image matched frames corresponding to the time stamps can be fitted in at least two infrared cameras, the three-dimensional reconstruction of the mark points can be carried out according to the stereoscopic vision principle, and when the image matched frames corresponding to the time stamps are fitted in more infrared cameras, the reconstruction precision can be improved. In addition, when at least two matching frames are not available in all the infrared cameras, asynchronous image processing of the next frame of the reference infrared camera image sequence is performed. The purpose of the reference infrared camera at this point is to provide a time stamp for reference.
And after processing a certain frame of image of the reference infrared camera, performing mark point three-dimensional reconstruction on the obtained fitting matching image frame to obtain position information of each mark point, and performing rigid body pose estimation on each mark point to obtain position and attitude information of the flapping wing aircraft.
Furthermore, a flight control panel of the flapping wing aircraft is used for analyzing a control instruction transmitted by an upper computer and outputting a PWM (pulse width modulation) value to control the rotating speed of the motor and the angle of the steering engine; the rotating speed of the motor is used for controlling the flapping frequency of the flapping wing air vehicle so as to adjust the flying height; the steering engine angle is used for controlling the tail wing angle of the flapping wing air vehicle, so that the steering of the flapping wing air vehicle is controlled.
The control module adopts two control rings to carry out cruise flight control on the flapping wing aircraft, wherein the two control rings are a height ring and a position ring respectively; the height ring adjusts the flapping frequency of the flapping wing aircraft through the PID controller, the position ring calculates the required acceleration through an outer ring L1 guidance law, and the tail wing phase difference of the flapping wing aircraft is controlled through an inner ring PID controller.
Specifically, FIG. 5 is a schematic view of the altitude control loop for cruise flight of an ornithopter. When the flapping wing aircraft enters a cruising flight mode, the controller consists of two control rings. Firstly, for the control of the height of the flapping wing aircraft, the height information of the flapping wing aircraft is obtained in real time through an asynchronous multi-view vision module, then an error value is obtained by making a difference with the expected height, the current required throttle amount is calculated by adopting a PID control method, a motor PWM output value is obtained through a conversion relation, an upper computer sends a control signal to the flapping wing aircraft through a wireless serial port module, the rotating speed of a motor is controlled through the motor PWM, and then the flapping frequency of the flapping wing aircraft is controlled, so that the height of the aircraft is adjusted and is kept near the preset height.
The output quantity of the PID controller of the height control ring is as follows:
wherein
In order to be a height error,
，
，
proportional, integral and differential parameters, respectively.
The output of the throttle amount is:
wherein the content of the first and second substances,
for the currently calculated throttle amount,
the throttle amount at the previous moment.
When the height of the flapping wing aircraft is controlled, as shown in fig. 6, a position control ring needs to be further designed. Particularly, the control loop projects the pose information of the aircraft on a height plane of a preset circular track, and then performs corresponding control, namely, the control loop controls the flapping wing aircraft to fly around a circle on the plane.
The position information of the flapping wing aircraft is obtained through the asynchronous multi-view vision module. And obtaining the approximate speed at a certain moment through the results of two adjacent stereo reconstructions. Similarly, acceleration can also be obtained. The position and the speed are projected to a preset flight track plane, the acceleration required to be set is obtained by using an L1 guidance law, the adjustment of the acceleration is controlled by the phase difference of the empennages of the flapping wing aircraft, the acceleration is converted into the phase difference of the two empennages, and then the PWM value of the steering engine corresponding to the two empennages is obtained.
The flapping wing aircraft provided by the embodiment of the invention has the advantages that the initial positions of the two tail wings are obliquely arranged, the initial positions are mainly used for controlling the steering of the aircraft, and the relation between the acceleration and the phase difference of a steering engine is controlled through the inner ring PID controller. The upper computer sends the PWM values of the two steering engines to the flapping wing aircraft through the wireless serial port module, and the centripetal acceleration of the flapping wing aircraft is adjusted to enable the aircraft to track a circular track on a tracking plane.
Wherein the acceleration obtained according to the L1 guidance law is:
wherein the content of the first and second substances,
in order to be the speed of the vehicle,
as is the distance between the actual position and the desired position,
to set the radius of the trajectory.
Wherein the content of the first and second substances,
and
the two parameters to be adjusted in the guidance law for L1 represent the period and the damping ratio, respectively.
The output quantity of the position control loop PID controller is as follows:
wherein
In order to be an acceleration error,
，
，
respectively being a proportionality coefficient, an integral coefficient anda differential parameter.
The output of the tail phase difference is:
wherein the content of the first and second substances,
for the phase difference of the tail currently calculated,
the phase difference at the previous time. The phase difference is evenly distributed to the two steering engines and then converted into corresponding PWM values, so that the steering of the flapping wing air vehicle is controlled.
In the above embodiment, the flight mode of the flapping wing aircraft can be completed by controlling the altitude ring and the position ring.
In summary, in the embodiment of the present invention, the asynchronous problem of the multiple cameras is solved by using a method of matching and fitting image points by using timestamps, and the position and attitude information of the flapping wing aircraft is obtained by using the stereoscopic vision principle after capturing the pixel coordinates of the infrared reflective sphere; and finally, control signals are sent to the flapping wing aircraft through the wireless serial port module to realize cruise flight.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (5)
1. A flapping wing aircraft cruise control system based on asynchronous multiple cameras is characterized by comprising an asynchronous multi-view vision module, a control module and a flapping wing aircraft;
a plurality of reflecting balls are fixed on the flapping wing aircraft, the reflecting balls are infrared reflecting marker balls, and the flapping wing aircraft takes off in a manual control mode;
the asynchronous multi-view vision module comprises a plurality of infrared cameras, a circle of infrared light source is arranged around each infrared camera, the infrared cameras capture images of the infrared cameras by reflecting infrared light, and the asynchronous multi-view vision module calculates the position and posture information of the flapping wing aircraft according to the captured images of the plurality of reflective balls;
the control module comprises an upper computer and is used for switching a control mode into a cruise mode and issuing a control instruction to the flapping wing aircraft according to the obtained position and attitude information of the flapping wing aircraft and a designed cruise control method;
the asynchronous multi-view vision module also comprises a router and a stereoscopic vision platform; the flapping wing aircraft comprises a flight control plate, a steering engine, a motor and a wireless serial port module A; the control module comprises an upper computer and a wireless serial port module B;
the infrared camera and the router are in communication in a wireless mode through a TCP/IP protocol, the router and the stereoscopic vision platform are in communication in a wireless mode through a TCP/IP protocol, the stereoscopic vision platform is in communication with the upper computer, the stereoscopic vision platform is connected with a display interface, the upper computer and the flight control board are in communication with the wireless serial port module A and the wireless serial port module B through the wireless serial port module A, and the steering engine and the motor are connected with the flight control board;
the infrared camera in the asynchronous multi-view vision module is embedded with a Linux kernel, so that a timestamp corresponding to each captured image can be recorded, and the image and the timestamp of each frame are packaged and sent to the stereoscopic vision platform;
the infrared camera is also used for preprocessing the image, obtaining the pixel coordinates of the light reflecting balls in the image according to an extraction algorithm and transmitting the pixel coordinates of the light reflecting balls to the stereoscopic vision platform;
the stereoscopic vision platform is used for issuing clock information to different infrared cameras to synchronize clocks; the stereoscopic vision platform is further configured to:
taking a first infrared camera with lower frequency as a reference, searching two frames of images closest to the first infrared camera in a second infrared camera through a timestamp, performing approximate processing on the motion of a reflective ball between the two frames of images, regarding the infrared camera as uniform linear motion when the frequency of the infrared camera is greater than a preset frequency, and performing interpolation through the two frames of image information:
wherein the content of the first and second substances,
is a time stamp corresponding to a frame in the first infrared camera, and is satisfied by finding the time stamp in an image captured by the second infrared camera
The adjacent two frame images of the image of the adjacent two frames,
and
respectively corresponding to the N frame and the (N + 1) frame image information of the second infrared camera
And
(ii) a The image information refers to the pixel coordinates of each reflective ball in each infrared camera, and the image information is interpolated and fitted into one infrared camera and the first infrared camera by the above formula
Second infrared camera reflective ball information corresponding to time image information time
；
Prior to performing the interpolation, the stereoscopic vision platform is further to:
in the first infrared camera
If the number of the reflective balls in the image frame corresponding to the moment is not equal to the set number, neglecting the image information of the frame, and carrying out synchronous processing on the next frame of image; otherwise, carrying out the next operation;
according to time stamp
Searching two adjacent frames of images which are closest to each other in the image sequence of the second infrared camera, similarly judging whether the number of the reflective balls in the two frames of images is equal to the set number, and if the number of the reflective balls in the two frames of images is equal to the set number, performing interpolation fitting;
the stereoscopic vision platform is further configured to:
after a certain frame of image of the reference infrared camera is processed, mark point three-dimensional reconstruction is carried out on the obtained fitting matching image frame to obtain position information of each mark point, and rigid body pose estimation is carried out on each mark point to obtain position and attitude information of the flapping wing aircraft.
2. The ornithopter cruise control system based on unsynchronized multiple cameras according to claim 1, characterized in that the number of said light-reflecting balls is not less than 4.
3. The cruise control system for an ornithopter based on unsynchronized multiple cameras according to claim 1, characterized in that said control module switches the control mode to cruise mode, in particular comprising:
setting the circle center and the radius of a cruise track circle;
calculating and judging whether the common view field of the infrared camera meets the requirement of the set track, if so, carrying out the next step, otherwise, resetting the circle center and the radius of the cruise track circle;
and after the setting is successful, the manual control mode is switched off, and the upper computer controls the flapping wing aircraft.
4. The flapping wing aircraft cruise control system based on asynchronous multiple cameras of claim 1, wherein the flight control panel is used for analyzing a control command transmitted by an upper computer and outputting a PWM value to control the rotation speed of a motor and the angle of a steering engine; the rotating speed of the motor is used for controlling the flapping frequency of the flapping wing air vehicle so as to adjust the flying height; the steering engine angle is used for controlling the tail wing angle of the flapping wing air vehicle, so that the steering of the flapping wing air vehicle is controlled.
5. The cruise control system for an ornithopter based on asynchronous multiple cameras as claimed in claim 4, wherein said control module employs two control loops for cruise flight control of the ornithopter, the two control loops being a height loop and a position loop; the height ring adjusts the flapping frequency of the flapping wing air vehicle through the PID controller, the position ring calculates the required acceleration through an outer ring L1 guidance law, and then the tail wing phase difference of the flapping wing air vehicle is controlled through an inner ring PID controller.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110722807.6A CN113247246B (en) | 2021-06-29 | 2021-06-29 | Flapping wing aircraft cruise control system based on asynchronous multiple cameras |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110722807.6A CN113247246B (en) | 2021-06-29 | 2021-06-29 | Flapping wing aircraft cruise control system based on asynchronous multiple cameras |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113247246A CN113247246A (en) | 2021-08-13 |
| CN113247246B true CN113247246B (en) | 2021-10-01 |
Family
ID=77190036
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110722807.6A Active CN113247246B (en) | 2021-06-29 | 2021-06-29 | Flapping wing aircraft cruise control system based on asynchronous multiple cameras |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113247246B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113965285B (en) * | 2021-09-14 | 2022-10-18 | 上海交通大学 | Cross-system multi-sensor time synchronization and calibration method based on NTP protocol |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101733746A (en) * | 2009-12-22 | 2010-06-16 | 哈尔滨工业大学 | Autonomously identifying and capturing method of non-cooperative target of space robot |
| CN101957642B (en) * | 2010-11-08 | 2012-06-27 | 昆山五昌新精密电子工业有限公司 | PIC card for non-synchronous extending system compatible with PIC interface |
| CN105416575B (en) * | 2015-12-14 | 2016-08-17 | 西北工业大学 | Apply in the bionical Landing Gear System of flapping wing aircraft and rising and dropping control method |
| CN106718851A (en) * | 2017-01-23 | 2017-05-31 | 东莞理工学院 | A kind of micro-robot of autonomous agriculture pollination |
| EP3775996A1 (en) * | 2018-04-09 | 2021-02-17 | INESC TEC - Instituto de Engenharia de Sistemas e Computadores, Tecnologia e Ciência | Hybrid imaging system for underwater robotic applications |
| CN111891349A (en) * | 2020-08-03 | 2020-11-06 | 西北工业大学太仓长三角研究院 | Four-rotor-flapping-wing hybrid layout aircraft |
-
2021
- 2021-06-29 CN CN202110722807.6A patent/CN113247246B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN113247246A (en) | 2021-08-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105242685B (en) | A kind of accompanying flying unmanned plane system and method | |
| CN107807661B (en) | Track control indoor quad-rotor unmanned aerial vehicle formation demonstration verification platform and method | |
| CN104828256B (en) | A kind of intelligent multi-control flight capture apparatus and flight control method thereof | |
| CN105629980B (en) | A kind of one camera oblique photograph 3 d modeling system | |
| US10852723B2 (en) | Unmanned aerial vehicle swarm photography | |
| WO2021008628A1 (en) | Aircraft control method and aircraft | |
| CN205353774U (en) | Accompany unmanned aerial vehicle system of taking photo by plane of shooing aircraft | |
| CN105759833A (en) | Immersive unmanned aerial vehicle driving flight system | |
| WO2017166002A1 (en) | Hovering control method and system for unmanned aerial vehicle, and unmanned aerial vehicle | |
| CN201217501Y (en) | Suspending type aviation camera shooting self-determination aircraft system | |
| KR101896654B1 (en) | Image processing system using drone and method of the same | |
| CN105242684A (en) | Unmanned plane aerial photographing system and method of photographing accompanying aircraft | |
| CN204697171U (en) | A kind of intelligent multi-control flight capture apparatus | |
| CN107504969A (en) | Four rotor-wing indoor air navigation aids of view-based access control model and inertia combination | |
| CN205015741U (en) | Companion flies unmanned aerial vehicle system of taking photo by plane and control cabinet and unmanned aerial vehicle thereof | |
| CN105182992A (en) | Unmanned aerial vehicle control method and device | |
| CN206804018U (en) | Environmental data server, unmanned vehicle and alignment system | |
| CN110498039A (en) | A kind of intelligent monitor system based on bionic flapping-wing flying vehicle | |
| WO2021098453A1 (en) | Target tracking method and unmanned aerial vehicle | |
| CN104061904B (en) | A kind of method of definite gap shape and area | |
| CN108733064A (en) | A kind of the vision positioning obstacle avoidance system and its method of unmanned plane | |
| CN106094876A (en) | A kind of unmanned plane target locking system and method thereof | |
| CN106043723A (en) | Swinging oblique photography system and method of fixed-wing unmanned aerial vehicle | |
| WO2020233682A1 (en) | Autonomous circling photographing method and apparatus and unmanned aerial vehicle | |
| CN113247246B (en) | Flapping wing aircraft cruise control system based on asynchronous multiple cameras |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |