CN111045455A - Visual correction method for flight course angle error of indoor corridor of micro unmanned aerial vehicle - Google Patents
Visual correction method for flight course angle error of indoor corridor of micro unmanned aerial vehicle Download PDFInfo
- Publication number
- CN111045455A CN111045455A CN201911408447.1A CN201911408447A CN111045455A CN 111045455 A CN111045455 A CN 111045455A CN 201911408447 A CN201911408447 A CN 201911408447A CN 111045455 A CN111045455 A CN 111045455A
- Authority
- CN
- China
- Prior art keywords
- aerial vehicle
- unmanned aerial
- corridor
- angle
- line
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 27
- 238000012937 correction Methods 0.000 title claims abstract description 25
- 230000000007 visual effect Effects 0.000 title claims abstract description 24
- 238000001514 detection method Methods 0.000 claims abstract description 23
- 238000012545 processing Methods 0.000 claims abstract description 13
- 238000001914 filtration Methods 0.000 claims abstract description 12
- 238000004364 calculation method Methods 0.000 claims abstract description 10
- 230000004927 fusion Effects 0.000 claims abstract description 7
- 230000009466 transformation Effects 0.000 claims abstract description 5
- 238000005259 measurement Methods 0.000 claims description 12
- 230000003287 optical effect Effects 0.000 claims description 11
- 230000014509 gene expression Effects 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 claims description 6
- 230000000877 morphologic effect Effects 0.000 claims description 5
- 150000001875 compounds Chemical class 0.000 claims description 4
- 230000007797 corrosion Effects 0.000 claims description 3
- 238000005260 corrosion Methods 0.000 claims description 3
- 238000003384 imaging method Methods 0.000 claims description 3
- 238000007500 overflow downdraw method Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 4
- RZVHIXYEVGDQDX-UHFFFAOYSA-N 9,10-anthraquinone Chemical compound C1=CC=C2C(=O)C3=CC=CC=C3C(=O)C2=C1 RZVHIXYEVGDQDX-UHFFFAOYSA-N 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 230000002401 inhibitory effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/10—Simultaneous control of position or course in three dimensions
- G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
Landscapes
- Engineering & Computer Science (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
The invention provides a visual correction method for flight course angle errors of an indoor corridor of a micro unmanned aerial vehicle, which comprises corridor line detection, deviation corridor line parameter calculation and course angle correction; the unmanned aerial vehicle acquires a front corridor image through the front-view camera, and the corridor detection is realized by processing the image through Hough transformation to obtain a linear equation of a corridor line in a pixel coordinate system; and calculating the angle and distance information of the unmanned aerial vehicle deviating from the corridor line by using the data of the ground height and the pitch angle given by the unmanned aerial vehicle navigation system, the linear equation of the corridor line in the pixel plane and the internal parameters obtained by calibrating the camera through the triangular relation. And correcting the course angle of the unmanned aerial vehicle by utilizing the angle of the unmanned aerial vehicle deviating from the corridor line and adopting Kalman filtering as a data fusion tool. The invention realizes course error correction by utilizing the angle and the distance of the unmanned aerial vehicle deviating from the corridor line, avoids the continuous divergence of course angle errors along time, ensures that the unmanned aerial vehicle flies linearly along the central line of the corridor, and avoids collision with the wall.
Description
Technical Field
The invention relates to the technical field of unmanned aerial vehicles, in particular to a visual correction method for flight course angle errors of indoor corridors of a micro unmanned aerial vehicle.
Background
Along with the continuous development of science and technology, the continuous rapid development of miniature unmanned aerial vehicle application field, miniature unmanned aerial vehicle is also more and more extensive at indoor application. When the micro unmanned aerial vehicle flies indoors, the course angle cannot be corrected by using the magnetometer due to the complex indoor magnetic field environment; due to the limitation of the volume, the weight and the power consumption of the micro unmanned aerial vehicle, a high-precision visual navigation (such as VIO and SLAM) scheme with high requirements on computing resources and power consumption cannot be adopted. Therefore, when the micro unmanned aerial vehicle flies indoors, the course angle can drift continuously because of no proper correction method.
In the prior art, certain achievements are obtained for the research of indoor flight positioning, such as: chinese patent CN201410350665.5 discloses an indoor positioning method for a micro unmanned aerial vehicle, which adopts an RGB-D camera and an inertia measurement unit to realize indoor positioning. Although the calculated amount is much smaller than that of a Visual Odometer (VO), the method still has characteristic point detection, matching and transmission, and the calculated amount is still not small for a micro unmanned aerial vehicle with the weight below 150 g; in the scheme, magnetometer data are used for course correction, so that the flight error in the indoor environment is large; and the RGB-D camera still is difficult to bear to the unmanned aerial vehicle of weight below 150g in volume, weight and consumption.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: aiming at overcoming the defects in the prior art, the invention provides a visual correction method for flight course angle errors of an indoor corridor of a micro unmanned aerial vehicle, which adopts a visual detection method with small calculation amount to realize the detection of the parameters of a narrow corridor and is used for correcting the course angle during corridor flight when the micro unmanned aerial vehicle flies in an indoor corridor area which is at a right angle with each other, thereby ensuring the precision of the course angle during indoor flight of the micro unmanned aerial vehicle.
The technical scheme adopted for solving the technical problems is as follows: a visual correction method for flight course angle errors of an indoor corridor of a micro unmanned aerial vehicle comprises corridor line detection, corridor line deviation angle and distance calculation and course angle correction, and specifically comprises the following steps:
s1: acquiring a corridor scene image in front of the unmanned aerial vehicle; the acquisition of the scene image of the corridor in front of the unmanned aerial vehicle can be directly acquired by photographing with a forward-looking camera installed at the front end of the unmanned aerial vehicle.
S2: processing the acquired image in an image processing computer on the unmanned aerial vehicle, and realizing corridor line detection by using Hough transformation so as to obtain a linear equation of the corridor line in a pixel coordinate system in a pixel plane;
s3: according to a linear equation of the corridor line in the pixel plane, the flying height and the pitch angle of the unmanned aerial vehicle given by the unmanned aerial vehicle navigation system and f in the parameters in the forward-looking camera are utilizedxAnd fyCalculating the angle and the distance of the unmanned aerial vehicle deviating from the corridor line through the triangular relation; wherein f in the camera intrinsic parametersxAnd fyThe unit is pixel, is the product of zoom magnification and focal length, and is obtained by camera calibration.
S4: and correcting a course angle and estimating a zero offset of a Z gyroscope of a yaw axis by using an angle of the unmanned aerial vehicle deviating from the corridor line and adopting Kalman filtering.
In the flight process of the unmanned aerial vehicle in the indoor corridor, the angle and the distance of the unmanned aerial vehicle deviating from the corridor line are very critical parameters, and the navigation system corrects the course error of the unmanned aerial vehicle by using the parameters, so that the problem that the course angle error is continuously dispersed along with the time under the assistance of a non-magnetometer is avoided; the flight control system is used for controlling the unmanned aerial vehicle to fly linearly along the center line of the corridor after obtaining the parameters, and collision between the unmanned aerial vehicle and the wall is avoided.
Further, the straight line equation of the corridor line in the pixel plane in step S2 is calculated as follows:
step S2.1: performing binarization processing on the gallery scene image obtained in the step S1 by using a self-adaptive local binary threshold detection method to obtain gallery basic information and realize gallery preliminary detection;
step S2.2: carrying out image morphological processing on the binarized image obtained in the step S2.1;
step S2.3: detecting a white connected domain in the processed image, and selecting a wall surface area from the shape and size of the white connected domain;
step S2.4: in the non-wall area that chooses, utilize the Hough transform to realize corridor straight line detection, through angle, length to the straight line that detects and with the regional position relation of wall, find out the contained angle line of wall and ground, accomplish corridor line's straight line detection, it is to obtain corridor line's linear equation expression under the pixel coordinate system:
v＝mu+n (3)
wherein u and v are coordinates of the u axis and the v axis of the pixel coordinate system, respectively, m is the slope of the straight line, and n is the intercept of the straight line on the v axis.
Further, the image morphological processing comprises one or more of expansion, corrosion and noise removal.
Further, the calculation process of the angle and distance of the unmanned aerial vehicle deviating from the corridor line in the step S3 specifically includes the following steps:
the height h of the unmanned aerial vehicle from the ground and the pitch angle theta of the unmanned aerial vehicle are obtained through an unmanned aerial vehicle navigation system, and the camera depression angle α meets the requirement that α is equal to-theta as the height of the camera from the ground is equal to the height of the unmanned aerial vehicle from the ground, so that the height of the camera from the ground and the camera depression angle α are determined;
let P be any point on the corridor line, and the coordinates of the point P in the camera coordinate system are (a', b, c), which can be obtained from the trigonometric relationship:
wherein β is the angle of the unmanned aerial vehicle deviating from the corridor line, d is the distance between the unmanned aerial vehicle and the corridor line, and a is the absolute value of a';
p' is the corresponding pixel point of P point in the image, and the pixel coordinate is (u, v), then there is according to camera aperture principle of imaging then:
in the formula (f)xAnd fyParameters related to the focal length in the parameter matrix in the forward-looking camera are obtained through camera calibration;
the equation of a straight line in equation (3) can be expressed as:
fyb/c＝mfxa/c+n (5)
substituting formulae (1) and (2) into formula (5), eliminating a and b, retaining only c, and obtaining after finishing:
since the point P is an arbitrary point on the straight line of the throat, the above equation holds for an arbitrary value of c, and then the following two expressions are obtained:
the angle β and distance d of the drone from the corridor line can be calculated from equation (7) above:
the final output polarity of the angle β at which the drone deviates from the corridor line is defined as β, which is output as a positive value when the optical axis of the forward-looking camera is oriented toward the corridor line on the left side of the corridor, and β, which is output as a negative value when the optical axis of the forward-looking camera is oriented toward the corridor line on the right side of the corridor.
In the step S4, a measured value of the unmanned aerial vehicle course is extracted by the angle deviating from the corridor line in the unmanned aerial vehicle corridor flight, the unmanned aerial vehicle course angle and the zero deviation of the yaw axis Z gyroscope are taken as states, Kalman filtering is adopted to realize data fusion, and the specific steps are as follows:
when the unmanned aerial vehicle is electrified, the direction of the unmanned aerial vehicle is coincident with a corridor line, the initial direction of the unmanned aerial vehicle is 0 degree, namely the initial direction of the unmanned aerial vehicle is taken as the north direction, the east, west and south directions of an indoor map are redefined by taking the direction as a reference, namely a pseudo east, west, south and north direction is defined for the indoor map, an unmanned aerial vehicle navigation system navigates by using the coordinate system, the heading psi calculated by an AHRS algorithm in the unmanned aerial vehicle navigation system is defined as the north, east and west, and is in the range of 0-2 pi, when the unmanned aerial vehicle flies in the corridor, the heading psi of the unmanned aerial vehicle is corrected by using the angle β deviating from the corridor line calculated by vision, the data fusion method adopts Kalman filtering, and the state:
in the formula (I), the compound is shown in the specification,
is the system state, ωzIs the output of the Z gyro, epsilonzIs Z gyro constant zero offset, wωzRandom white noise output by the Z gyroscope;
the measurement equation is as follows:
z＝ψc+v (10)
wherein z represents a measurement; v represents the measurement noise; psicAnd determining the heading angle obtained by visual calculation according to the angle β of the unmanned aerial vehicle deviating from the corridor line and the corridor direction, wherein the east-west-south-north direction of the corridor direction is determined according to the corridor direction of the unmanned aerial vehicle when the unmanned aerial vehicle is electrified and initialized as the north direction:
when the unmanned plane flies to the north along the north-south corridor, β is more than 0, psic＝2π-β；β＜0,ψc＝-β；
When the unmanned aerial vehicle flies south to south along the north-south corridor: psic＝π-β；
When the drone is flying east-west along the east-west corridor:
when the drone is flying westward along the east-west corridor:
taking the course angle obtained by the AHRS algorithm as one-step prediction of the course angle in the Kalman course correction algorithm;
the kalman filter matrix is as follows:
Φ≈I+FT；Γ＝GT (12)
obtaining a visual course angle psicAfter the unmanned aerial vehicle heading psi and the Z gyro zero-bias epsilon are corrected, data fusion can be completed by Kalman filtering to obtain corrected unmanned aerial vehicle heading psi and Z gyro zero-bias epsilonz(ii) a And recalculating the quaternion by using the corrected unmanned aerial vehicle course psi and the pitch angle theta and roll angle gamma obtained in the unmanned aerial vehicle AHRS algorithm, and feeding back to the AHRS algorithm. Quaternion recalculation as per equation (13):
wherein q is0、q1、q2、q3Representing an attitude quaternion.
The invention has the beneficial effects that: the invention provides a visual correction method for flight course angle error of an indoor corridor of a micro unmanned aerial vehicle, which is suitable for course angle error correction in an indoor corridor flight stage of the unmanned aerial vehicle. Detecting corridor lines by a visual method, calculating the angle and the distance of the unmanned aerial vehicle deviating from the corridor lines, correcting course angle errors by using the angle of the unmanned aerial vehicle deviating from the corridor lines, inhibiting the divergence of the course errors and avoiding the divergence of the course angle errors in the whole indoor autonomous flight process of the unmanned aerial vehicle; the angle and the distance of the unmanned aerial vehicle deviating from the corridor line can be directly output to the flight control system, the unmanned aerial vehicle can be guided to fly linearly along the corridor channel, and collision with the wall is avoided.
Drawings
The invention is further illustrated by the following figures and examples.
Fig. 1 is a schematic view of the installation of a front camera of the unmanned aerial vehicle.
Fig. 2 is a schematic view of a forward looking camera measuring the angle and distance of the drone from the corridor line.
Fig. 3 is a schematic diagram of coordinate system definition in corridor line deviation angle and distance extraction of the corridor flying unmanned aerial vehicle.
Fig. 4 is a schematic diagram of the definition of a pixel coordinate system with the center of a pixel plane as an origin.
Fig. 5 is a schematic view of the flight of an indoor corridor of an unmanned aerial vehicle.
Detailed Description
The present invention will now be described in detail with reference to the accompanying drawings. This figure is a simplified schematic diagram, and merely illustrates the basic structure of the present invention in a schematic manner, and therefore it shows only the constitution related to the present invention.
As shown in fig. 1-5, the visual correction method for flight course angle error of indoor corridor of micro unmanned aerial vehicle of the present invention includes a visual detection method for corridor line, a calculation method for corridor line deviation angle and distance, and course angle correction, and specifically includes the following steps:
s1: shooting a scene image of a corridor in front of the unmanned aerial vehicle by a front-view camera of the unmanned aerial vehicle;
s2: an image processing computer on the unmanned aerial vehicle processes an image acquired by the forward-looking camera, corridor detection is realized by utilizing Hough transformation, so that a linear equation of a corridor line under a pixel coordinate system in a pixel plane is obtained, and the calculation process is as follows:
step S2.1: performing binarization processing on the gallery scene image obtained in the step S1 by using a self-adaptive local binary threshold detection method to obtain gallery basic information and realize gallery preliminary detection;
step S2.2: carrying out image morphological processing such as expansion, corrosion, denoising and the like on the binarized image obtained in the step S2.1;
step S2.3: detecting a white connected domain in the processed image, and selecting a wall surface area from the white connected domain by limiting the shape and the size of the white connected domain;
step S2.4: in the selected non-wall area, corridor linear detection is realized by utilizing Hough transformation, and the included angle line between the wall and the ground is found out by limiting the angle, the length and the position relation of the detected straight line and the wall area, so that the linear detection of the corridor line is completed.
Step S3: after a linear equation of the corridor line in the pixel plane is obtained, the linear equation is utilizedFlying height and pitch angle of unmanned aerial vehicle given by unmanned aerial vehicle navigation system and f in internal parameters of forward-looking cameraxAnd fyThrough the triangle relation, calculate the angle and the distance that unmanned aerial vehicle deviates from the corridor line, specifically include following step:
because the unmanned aerial vehicle belongs to the low head flight before flying, make its optical axis and unmanned aerial vehicle body coordinate system to the preceding parallel when the forward looking camera is installed, the angle of pitch and flying height when flying before miniature unmanned aerial vehicle are shown as figure 1, h is the height of camera apart from ground, also be the height that unmanned aerial vehicle apart from ground, α is the camera angle of pitch figure 2 is angle and distance schematic diagram when unmanned aerial vehicle deviates the corridor, β is the angle that unmanned aerial vehicle deviates the corridor line, d is the distance of unmanned aerial vehicle and corridor line.
As shown in FIG. 3, the origin O of the camera coordinate system is the optical center, ZcThe axis coinciding with the optical axis and pointing in front of the camera, XcAxis directed to right side of camera body, YcThe height h between the unmanned aerial vehicle and the camera from the ground is given by an unmanned aerial vehicle navigation system, the depression angle α of the camera is calculated by a pitch angle theta given by the unmanned aerial vehicle navigation system, α is-theta, β is the angle of the unmanned aerial vehicle deviating from a corridor line, d is the distance between the unmanned aerial vehicle and the corridor line, and for the sake of no loss of generality, no matter whether the optical axis of the camera deviates to the left side or the right side of the corridor, the P is set as any point on the corridor line, the coordinate of the P point in a camera coordinate system is (a ', b, c), obviously when the optical axis of the camera deviates to the left side corridor, the P point is the point on the left side corridor line, a ' is greater than 0, a is the absolute value of a ', and the triangular relation can be obtained:
as shown in fig. 4, a pixel coordinate system is established at the center of the pixel plane imaged by the camera, and the origin is the center O' of the pixel plane, and the horizontal and vertical axes are u and v, respectively. The expression of the straight-line equation in the pixel coordinate system by the corridor line detected in step S2 is:
v＝mu+n (3)
wherein u and v are coordinates of the u axis and the v axis of the pixel coordinate system, respectively, m is the slope of the straight line, and n is the intercept of the straight line on the v axis. P' is the corresponding pixel point of P point in the image, and the pixel coordinate is (u, v), then there is according to camera aperture principle of imaging then:
u＝fxa/c，v＝fyb/c (4)
in the formula (f)xAnd fyAnd parameters related to the focal length in the parameter matrix in the forward-looking camera are obtained through camera calibration. The equation of a straight line in equation (3) can be expressed as:
fyb/c＝mfxa/c+n (5)
substituting formulae (1) and (2) into formula (5), eliminating a and b, retaining only c, and obtaining after finishing:
since the point P is an arbitrary point on the straight line of the throat, the above equation holds for an arbitrary value of c, and then the following two expressions can be obtained:
the angle β and distance d of the drone from the corridor line can be calculated from equation (7) above:
the final output polarity of the angle β at which the drone deviates from the corridor line is defined as β, which is output as a positive value when the optical axis of the forward-looking camera is oriented toward the corridor line on the left side of the corridor, and β, which is output as a negative value when the optical axis of the forward-looking camera is oriented toward the corridor line on the right side of the corridor.
Step S4: after calculating the angle and the distance of the deviation corridor line in the flight of the unmanned aerial vehicle corridor, correcting the course angle by adopting Kalman filtering, extracting a measurement value of the course of the unmanned aerial vehicle from the angle of the deviation corridor line in the flight of the unmanned aerial vehicle corridor, taking the zero offset of the course angle of the unmanned aerial vehicle and a yaw axis Z gyroscope as a state, and realizing data fusion by adopting Kalman filtering, wherein the method specifically comprises the following steps:
the three-dimensional attitude angle and the attitude quaternion of the unmanned aerial vehicle are obtained by an unmanned aerial vehicle AHRS algorithm, wherein the horizontal attitude angle cannot drift due to correction of accelerometer data, so that the unmanned aerial vehicle is ensured to stably fly in the horizontal attitude, but the heading angle cannot be corrected through magnetometer data under an indoor environment, the heading angle can continuously drift due to zero deviation of a Z gyroscope, and other heading-related measurement data are needed to correct the heading angle error.
Taking fig. 5 as an example, an unmanned aerial vehicle flies in the map, the direction of the unmanned aerial vehicle when being powered on is coincident with a corridor line, the heading angle of the initial direction is 0 degree, that is, the initial direction of the unmanned aerial vehicle is taken as the north direction, the east-west-south direction of an indoor map is redefined by taking the direction as a reference, that is, a pseudo east-south-north direction is defined for the indoor map, an unmanned aerial vehicle navigation system navigates by using the coordinate system, the heading psi calculated by an AHRS algorithm in the unmanned aerial vehicle navigation system is defined as north-east-west-east-positive, and within the range of 0-2 pi, the east-south-north directions mentioned later in the invention all refer to pseudo east-south-north directions, the heading psi of the unmanned aerial vehicle is corrected by using the angle β deviating from the corridor line calculated by vision when the unmanned aerial vehicle flies in the corridor, the:
in the formula (I), the compound is shown in the specification,
is the system state, ωzIs the output of the Z gyro, epsilonzIs Z gyro constant zero offset, wωzRandom white noise output by the Z gyro.
The measurement equation is as follows:
z＝ψc+v (10)
wherein z represents a measurement; v represents the measurement noise; psicTo look atThe east-west-north direction of the corridor orientation is determined according to the corridor direction of the orientation when the unmanned aerial vehicle is electrified and initialized as the north direction:
when the unmanned plane flies to the north along the north-south corridor, β is more than 0, psic＝2π-β；β＜0,ψc＝-β；
When the unmanned aerial vehicle flies south to south along the north-south corridor: psic＝π-β；
When the drone is flying east-west along the east-west corridor:
when the drone is flying westward along the east-west corridor:
and taking the course angle obtained by the AHRS algorithm as one-step prediction of the course angle in the Kalman course correction algorithm.
The kalman filter matrix is as follows:
Φ≈I+FT；Γ＝GT (12)
obtaining a visual course angle psicAfter the unmanned aerial vehicle heading psi and the Z gyro zero-bias epsilon are corrected, data fusion can be completed by Kalman filtering to obtain corrected unmanned aerial vehicle heading psi and Z gyro zero-bias epsilonz. And recalculating the quaternion by using the corrected unmanned aerial vehicle course psi and the pitch angle theta and roll angle gamma obtained in the unmanned aerial vehicle AHRS algorithm, and feeding back to the AHRS algorithm. The quaternion is recalculated as in equation (13).
Wherein q is0、q1、q2、q3Representing an attitude quaternion.
The algorithm is suitable for correcting the course angle error of the unmanned aerial vehicle in the indoor corridor flight stage. Detecting corridor lines by a visual method, calculating the angle and the distance of the unmanned aerial vehicle deviating from the corridor lines, correcting course angle errors by using the angle of the unmanned aerial vehicle deviating from the corridor lines, inhibiting the divergence of the course errors and avoiding the divergence of the course angle errors in the whole indoor autonomous flight process of the unmanned aerial vehicle; the angle and the distance of the unmanned aerial vehicle deviating from the corridor line can be directly output to the flight control system, the unmanned aerial vehicle can be guided to fly linearly along the corridor channel, and collision with the wall is avoided.
In light of the foregoing description of preferred embodiments in accordance with the invention, it is to be understood that numerous changes and modifications may be made by those skilled in the art without departing from the scope of the invention. The technical scope of the present invention is not limited to the contents of the specification, and must be determined according to the scope of the claims.
Claims (5)
1. A visual correction method for flight course angle errors of indoor corridors of a micro unmanned aerial vehicle is characterized by comprising the following steps: the method comprises the steps of corridor line detection, corridor line deviation angle and distance calculation and course angle correction, and specifically comprises the following steps:
s1: acquiring a corridor scene image in front of the unmanned aerial vehicle;
s2: processing the acquired image, and realizing corridor line detection by utilizing Hough transformation to obtain a linear equation of the corridor line under a pixel coordinate system in a pixel plane;
s3: according to a linear equation of the corridor line in the pixel plane, the flying height and the pitch angle of the unmanned aerial vehicle given by the unmanned aerial vehicle navigation system and f in the parameters in the forward-looking camera are utilizedxAnd fyCalculating the angle and the distance of the unmanned aerial vehicle deviating from the corridor line through the triangular relation;
s4: and correcting a course angle and estimating a zero offset of a Z gyroscope of a yaw axis by using an angle of the unmanned aerial vehicle deviating from the corridor line and adopting Kalman filtering.
2. The visual correction method for the flight course angle error of the indoor corridor of the micro unmanned aerial vehicle as claimed in claim 1, characterized in that: in step S2, the straight line equation of the corridor line in the pixel plane is calculated as follows:
step S2.1: performing binarization processing on the gallery scene image obtained in the step S1 by using a self-adaptive local binary threshold detection method to obtain gallery basic information and realize gallery preliminary detection;
step S2.2: carrying out image morphological processing on the binarized image obtained in the step S2.1;
step S2.3: detecting a white connected domain in the processed image, and selecting a wall surface area from the white connected domain by limiting the shape and the size of the white connected domain;
step S2.4: in the non-wall area that chooses, utilize the Hough transform to realize corridor straight line detection, through angle, length to the straight line that detects and with the regional position relation of wall, find out the contained angle line of wall and ground, accomplish corridor line's straight line detection, it is to obtain corridor line's linear equation expression under the pixel coordinate system:
v＝mu+n (3)
wherein u and v are coordinates of the u axis and the v axis of the pixel coordinate system, respectively, m is the slope of the straight line, and n is the intercept of the straight line on the v axis.
3. The visual correction method for the flight course angle error of the indoor corridor of the micro unmanned aerial vehicle as claimed in claim 2, characterized in that: the image morphological processing comprises one or more of expansion, corrosion and noise removal.
4. The visual correction method for the flight course angle error of the indoor corridor of the micro unmanned aerial vehicle as claimed in claim 2, characterized in that: the calculation process of the angle and distance of the unmanned aerial vehicle deviating from the corridor line in the step S3 specifically comprises the following steps:
the height h of the unmanned aerial vehicle from the ground and the pitch angle theta of the unmanned aerial vehicle are obtained through an unmanned aerial vehicle navigation system, and the camera depression angle α meets the requirement that α is equal to-theta as the height of the camera from the ground is equal to the height of the unmanned aerial vehicle from the ground, so that the height of the camera from the ground and the camera depression angle α are determined;
let P be any point on the corridor line, and the coordinates of the point P in the camera coordinate system are (a', b, c), which can be obtained from the trigonometric relationship:
wherein β is the angle of the unmanned aerial vehicle deviating from the corridor line, d is the distance between the unmanned aerial vehicle and the corridor line, and a is the absolute value of a';
p' is the corresponding pixel point of P point in the image, and the pixel coordinate is (u, v), then there is according to camera aperture principle of imaging then:
u＝fxa/c，v＝fyb/c (4)
in the formula (f)xAnd fyParameters related to the focal length in the parameter matrix in the forward-looking camera are obtained through camera calibration;
the equation of a straight line in equation (3) can be expressed as:
fyb/c＝mfxa/c+n (5)
substituting formulae (1) and (2) into formula (5), eliminating a and b, retaining only c, and obtaining after finishing:
since the point P is an arbitrary point on the straight line of the throat, the above equation holds for an arbitrary value of c, and then the following two expressions are obtained:
the angle β and distance d of the drone from the corridor line can be calculated from equation (7) above:
the final output polarity of the angle β at which the drone deviates from the corridor line is defined as β, which is output as a positive value when the optical axis of the forward-looking camera is oriented toward the corridor line on the left side of the corridor, and β, which is output as a negative value when the optical axis of the forward-looking camera is oriented toward the corridor line on the right side of the corridor.
5. The visual correction method for the flight course angle error of the indoor corridor of the micro unmanned aerial vehicle as claimed in claim 1 or 4, characterized in that: in the step S4, a measured value of the unmanned aerial vehicle course is extracted by the angle deviating from the corridor line in the unmanned aerial vehicle corridor flight, the unmanned aerial vehicle course angle and the zero deviation of the yaw axis Z gyroscope are taken as states, Kalman filtering is adopted to realize data fusion, and the specific steps are as follows:
when the unmanned aerial vehicle is electrified, the direction of the unmanned aerial vehicle is coincident with a corridor line, the initial direction of the unmanned aerial vehicle is 0 degree, namely the initial direction of the unmanned aerial vehicle is taken as the north direction, the east, west and south directions of an indoor map are redefined by taking the direction as a reference, namely a pseudo east, west, south and north direction is defined for the indoor map, an unmanned aerial vehicle navigation system navigates by using the coordinate system, the heading psi calculated by an AHRS algorithm in the unmanned aerial vehicle navigation system is defined as the north, east and west, and is in the range of 0-2 pi, when the unmanned aerial vehicle flies in the corridor, the heading psi of the unmanned aerial vehicle is corrected by using the angle β deviating from the corridor line calculated by vision, the data fusion method adopts Kalman filtering, and the state:
in the formula (I), the compound is shown in the specification,
is the system state, ωzIs the output of the Z gyro, epsilonzIs Z gyro constant zero offset, wωzRandom white noise output by the Z gyroscope;
the measurement equation is as follows:
z＝ψc+v (10)
in the formula (I), the compound is shown in the specification,z represents a measurement; v represents the measurement noise; psicAnd determining the heading angle obtained by visual calculation according to the angle β of the unmanned aerial vehicle deviating from the corridor line and the corridor direction, wherein the east-west-south-north direction of the corridor direction is determined according to the corridor direction of the unmanned aerial vehicle when the unmanned aerial vehicle is electrified and initialized as the north direction:
when the unmanned plane flies to the north along the north-south corridor, β is more than 0, psic＝2π-β；β＜0,ψc＝-β；
When the unmanned aerial vehicle flies south to south along the north-south corridor: psic＝π-β；
When the drone is flying east-west along the east-west corridor:
when the drone is flying westward along the east-west corridor:
taking the course angle obtained by the AHRS algorithm as one-step prediction of the course angle in the Kalman course correction algorithm;
the kalman filter matrix is as follows:
Φ≈I+FT；Γ＝GT (12)
obtaining a visual course angle psicAfter the unmanned aerial vehicle heading psi and the Z gyro zero-bias epsilon are corrected, data fusion can be completed by Kalman filtering to obtain corrected unmanned aerial vehicle heading psi and Z gyro zero-bias epsilonz(ii) a Recalculating quaternion by using the corrected unmanned aerial vehicle course psi and the pitch angle theta and roll angle gamma obtained in the unmanned aerial vehicle AHRS algorithm, and feeding back to the AHRS algorithm; quaternion recalculation as per equation (13):
wherein q is0、q1、q2、q3Representing an attitude quaternion.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911408447.1A CN111045455A (en) | 2019-12-31 | 2019-12-31 | Visual correction method for flight course angle error of indoor corridor of micro unmanned aerial vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911408447.1A CN111045455A (en) | 2019-12-31 | 2019-12-31 | Visual correction method for flight course angle error of indoor corridor of micro unmanned aerial vehicle |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111045455A true CN111045455A (en) | 2020-04-21 |
Family
ID=70242895
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911408447.1A Pending CN111045455A (en) | 2019-12-31 | 2019-12-31 | Visual correction method for flight course angle error of indoor corridor of micro unmanned aerial vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111045455A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111931776A (en) * | 2020-10-16 | 2020-11-13 | 江西小马机器人有限公司 | Deep learning-based double-pointer instrument reading method |
| CN112003658A (en) * | 2020-06-05 | 2020-11-27 | 鹏城实验室 | Receiver pitch angle estimation method, storage medium and terminal device |
| CN113838097A (en) * | 2021-09-29 | 2021-12-24 | 成都新潮传媒集团有限公司 | Camera lens angle deviation detection method and device and storage medium |
| CN114661061A (en) * | 2022-02-14 | 2022-06-24 | 天津大学 | GPS-free micro unmanned aerial vehicle flight control method based on visual indoor environment |
| CN117470127A (en) * | 2023-12-27 | 2024-01-30 | 济宁市质量计量检验检测研究院(济宁半导体及显示产品质量监督检验中心、济宁市纤维质量监测中心) | Unmanned aerial vehicle building wall surface detection system and method based on coordinate correction |
-
2019
- 2019-12-31 CN CN201911408447.1A patent/CN111045455A/en active Pending
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112003658A (en) * | 2020-06-05 | 2020-11-27 | 鹏城实验室 | Receiver pitch angle estimation method, storage medium and terminal device |
| CN112003658B (en) * | 2020-06-05 | 2022-07-01 | 鹏城实验室 | Receiver pitch angle estimation method, storage medium and terminal device |
| CN111931776A (en) * | 2020-10-16 | 2020-11-13 | 江西小马机器人有限公司 | Deep learning-based double-pointer instrument reading method |
| CN113838097A (en) * | 2021-09-29 | 2021-12-24 | 成都新潮传媒集团有限公司 | Camera lens angle deviation detection method and device and storage medium |
| CN113838097B (en) * | 2021-09-29 | 2024-01-09 | 成都新潮传媒集团有限公司 | Camera lens angle deviation detection method, device and storage medium |
| CN114661061A (en) * | 2022-02-14 | 2022-06-24 | 天津大学 | GPS-free micro unmanned aerial vehicle flight control method based on visual indoor environment |
| CN114661061B (en) * | 2022-02-14 | 2024-05-17 | 天津大学 | GPS-free visual indoor environment-based miniature unmanned aerial vehicle flight control method |
| CN117470127A (en) * | 2023-12-27 | 2024-01-30 | 济宁市质量计量检验检测研究院(济宁半导体及显示产品质量监督检验中心、济宁市纤维质量监测中心) | Unmanned aerial vehicle building wall surface detection system and method based on coordinate correction |
| CN117470127B (en) * | 2023-12-27 | 2024-03-08 | 济宁市质量计量检验检测研究院(济宁半导体及显示产品质量监督检验中心、济宁市纤维质量监测中心) | Unmanned aerial vehicle building wall surface detection system and method based on coordinate correction |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111045455A (en) | Visual correction method for flight course angle error of indoor corridor of micro unmanned aerial vehicle | |
| US10515458B1 (en) | Image-matching navigation method and apparatus for aerial vehicles | |
| CN107180215B (en) | Parking lot automatic mapping and high-precision positioning method based on library position and two-dimensional code | |
| CN109520497B (en) | Unmanned aerial vehicle autonomous positioning method based on vision and imu | |
| US8213706B2 (en) | Method and system for real-time visual odometry | |
| CN107727079A (en) | The object localization method of camera is regarded under a kind of full strapdown of Small and micro-satellite | |
| Wang et al. | Polarized light compass-aided visual-inertial navigation under foliage environment | |
| CN108845335A (en) | Unmanned aerial vehicle ground target positioning method based on image and navigation information | |
| CN111426320B (en) | Vehicle autonomous navigation method based on image matching/inertial navigation/milemeter | |
| CN105352495A (en) | Unmanned-plane horizontal-speed control method based on fusion of data of acceleration sensor and optical-flow sensor | |
| CN111024091A (en) | Three-dimensional attitude algorithm for indoor flight of vision-assisted micro unmanned aerial vehicle | |
| CN106352897B (en) | It is a kind of based on the silicon MEMS gyro estimation error of monocular vision sensor and bearing calibration | |
| Williams et al. | Feature and pose constrained visual aided inertial navigation for computationally constrained aerial vehicles | |
| CN112819711B (en) | Monocular vision-based vehicle reverse positioning method utilizing road lane line | |
| CN110736457A (en) | combination navigation method based on Beidou, GPS and SINS | |
| Wang et al. | Monocular vision and IMU based navigation for a small unmanned helicopter | |
| CN109344970B (en) | Vision target-based dynamic reasoning method on unmanned aerial vehicle | |
| Hosen et al. | Vision-aided nonlinear observer for fixed-wing unmanned aerial vehicle navigation | |
| CN112686149B (en) | Vision-based near-zone autonomous landing method of fixed-wing unmanned aerial vehicle | |
| Li et al. | Combined RGBD-inertial based state estimation for MAV in GPS-denied indoor environments | |
| CN113340272A (en) | Ground target real-time positioning method based on micro-group of unmanned aerial vehicle | |
| Tehrani et al. | Horizon-based attitude estimation from a panoramic vision sensor | |
| Fan et al. | Bio-inspired multisensor navigation system based on the skylight compass and visual place recognition for unmanned aerial vehicles | |
| Fusini et al. | Experimental validation of a uniformly semi-globally exponentially stable non-linear observer for gnss-and camera-aided inertial navigation for fixed-wing uavs | |
| Indelman et al. | Navigation aiding based on coupled online mosaicking and camera scanning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |