CN105450950A - Method for removing jitter from aerial video of unmanned aerial vehicle - Google Patents
Method for removing jitter from aerial video of unmanned aerial vehicle Download PDFInfo
- Publication number
- CN105450950A CN105450950A CN201510888639.2A CN201510888639A CN105450950A CN 105450950 A CN105450950 A CN 105450950A CN 201510888639 A CN201510888639 A CN 201510888639A CN 105450950 A CN105450950 A CN 105450950A
- Authority
- CN
- China
- Prior art keywords
- angle
- straight line
- road
- histogram
- aerial images
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
- H04N25/61—Noise processing, e.g. detecting, correcting, reducing or removing noise the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"
Abstract
The invention discloses a method for removing jitter from an aerial video of an unmanned aerial vehicle. The method comprises the following steps: a step 1: preprocessing a road aerial image; a step 2: extracting a straight line based on the aerial image; a step 3: calculating a straight line direction histogram; and a step 4: correcting and removing jitter from the aerial image. According to the method disclosed by the invention, by means of the property that most straight lines extracted by detecting the road aerial image of the unmanned aerial vehicle are parallel to the road direction, a relative angle histogram of the straight lines is established, the angle corresponding to the maximum peak point of the histogram is detected to obtain the road direction in the aerial image, the image is rotated according to the angle to adjust the road to the horizontal direction, in order to remove jitter from the aerial image. The method disclosed by the invention is based on the image processing technology and is used for intelligently sensing the road direction and rotating the aerial image to remove the jitter from the road aerial image of the unmanned aerial vehicle.
Description
Technical field
The invention belongs to technical field of image processing, relate to a kind of unmanned plane to take photo by plane video jitter removing method, particularly relate to a kind of lines detection method based on Image Edge-Detection and Hough transformation with based on the histogrammic road direction detection method of straight line angle and the bearing calibration based on road direction image rotating, to realize the object of image debounce.
Background technology
The fields such as unmanned plane because have mobility strong, looking away, flight path not by the advantage such as landform restriction, and is widely used in surveying and drawing, take photo by plane, traffic monitoring.The application of diaxon and three-axis stabilization The Cloud Terrace eliminates Aerial Images due to UAV Attitude and adjusts, the flating problem that external condition (fitful wind) etc. causes, but at a lot of application scenarios, as unmanned plane expressway traffic monitoring, because unmanned plane does not have the ability of Intelligent Recognition road, the flight direction of advance of unmanned plane and the angle moment of road direction is caused to change, thus cause the problem of road shake in Aerial Images, no matter this shake is by picture control traffic situation to ground staff, or all bring very large obstacle to carrying out traffic parameter extraction based on image processing method.Therefore, invention Intelligent road Aerial Images jitter removing method just seems and is even more important.
At present, the image jitter removing method of Aerial Images mainly contains mechanical debounce and method for registering debounce two kinds of modes.Hardware debounce, be installed on by image capture device and have on the mechanical underprop increasing steady function, what this mode was eliminated is the flating that Aerial Images causes due to UAV Attitude adjustment, external condition (fitful wind) etc.Method for registering images, obtains the motion of image background by the characteristic point in tracking image, thus eliminates shake by affine transformation method, and this kind of debounce needs to choose basic registration frame in advance.In the unmanned plane traffic monitoring application of reality, user is interested is road area, and what need acquisition is stable road image, for this application scenarios, because existing jitter removing method cannot perception road area, therefore road Aerial Images jitter problem cannot be eliminated targetedly.
Summary of the invention
Intelligence the problem that in unmanned plane road Aerial Images, road is shaken cannot be eliminated for conventional images jitter removing method, the present invention proposes based on the histogrammic image jitter removing method of road direction, adopt and carry out road Aerial Images debounce based on the histogrammic method of road direction.The present invention is based on road direction histogrammic road Aerial Images jitter removing method, first by road direction histogram Detection and Extraction road direction, then based on road direction image rotating, realizes road Aerial Images debounce.Image jitter removing method of the present invention can Intellisense road direction, thus realizes the irrealizable image debounce of existing method.
The present invention enters a little with brand-new research, and proposing one can blanket road Aerial Images debounce way, is realized by following step:
Step 1: road Aerial Images preliminary treatment
Solution frame is carried out to unmanned plane video of taking photo by plane, obtains single frames RGB color road Aerial Images, and RGB color image is converted to gray-scale map;
Step 2: extract straight line based on Aerial Images
Adopt the gray-scale map in the process of Canny edge detection operator in step to obtain edge contour figure, based on Hough transformation Edge detected profile diagram, obtain straight line;
Step 3: calculated line direction histogram
The angle of the straight line that calculating detects in step, then calculated line relative angle histogram.Extract the angle that peak point maximum in histogram is corresponding, this angle is the direction of road in this frame Aerial Images.
Step 4: Aerial Images corrects debounce
The size of the angle obtained in the upper step that turned clockwise by road Aerial Images, unanimously can be adjusted to horizontal direction by Aerial Images towards different roads, thus realizes road Aerial Images debounce.
The invention has the advantages that:
(1) feature that the present invention's utilize unmanned plane to take photo by plane straight line that road image Detection and Extraction go out is mostly parallel with road direction, set up straight line relative angle histogram, by detecting angle corresponding to the maximum peak point of histogram, the direction of road in Aerial Images can be obtained, be horizontal direction according to this angle image rotating by Road adjustment, realize Aerial Images debounce;
(2) the present invention is based on image processing techniques, rotate Aerial Images by Intellisense road direction, thus realize the debounce of unmanned plane road Aerial Images;
(3) the present invention can adapt to the Aerial Images debounce under different kinds of roads scene, has robustness well, fast operation and do not need external data support (as GIS map data), has multiple spot innovation.
Accompanying drawing explanation
Fig. 1 is the gray-scale map of Aerial Images;
Fig. 2 is based on Canny edge detection operator Edge detected profile diagram;
Fig. 3 is based on Hough transformation detection of straight lines figure;
Fig. 4 is image coordinate system and straight line angle direction definition figure;
Fig. 5 is the relative histogram of rectilinear direction;
Fig. 6 is based on road direction level correction figure;
Fig. 7 is method flow diagram of the present invention.
Embodiment
Below in conjunction with drawings and Examples, the present invention is described in further detail.
The invention provides a kind of based on the histogrammic Aerial Images jitter removing method of road direction, first described jitter removing method carries out preliminary treatment to the road Aerial Images extracted and is converted to gray-scale map; Then detect gray-scale map based on Canny edge detection operator and obtain edge contour figure, then edge profile diagram carries out Hough transformation detection of straight lines; Calculate the angle of straight line calculated line direction relative angle histogram that detect, extract the angle of the maximum straight line corresponding to peak value of relative angle histogram, namely obtain road direction; Then based on the road direction detected, original Aerial Images is carried out instantaneous pin rotation, road can be rotated to be horizontal direction, realize Aerial Images debounce.Above-mentioned based on the histogrammic Aerial Images jitter removing method of road direction, as shown in Figure 7, concrete treatment step is as follows for flow process:
Step 1: road Aerial Images preliminary treatment
To road take photo by plane video carry out solution frame obtain RGB color image, be gray-scale map by RGB color-map representation, as shown in Figure 1.
Step 2: extract straight line based on Aerial Images
After obtaining gray level image, next by Canny edge detection operator, gray-scale map is processed, obtain the edge contour figure of two-value, as shown in Figure 2, Hough transformation is carried out to the profile diagram of Fig. 2, detects and obtain rectilinear, as shown in Figure 3.
Step 3: calculated line direction histogram
The angle of every bar straight line in calculating chart 1, wherein the definition of straight line angle as shown in Figure 4, wherein O (0, the 0) initial point that is image pixel coordinates, and with O (0,0) for starting point, being to the right the row reference axis of image, is downwards the row-coordinate axle of image.To any straight line i in Fig. 3, two end points is respectively P1 and P2, and wherein the pixel coordinate of P1 and P2 is respectively (c
i_1, r
i_1) and (c
i_2, r
i_2),
for the angle of straight line i and horizontal direction,
the angle turned over when representing and a horizontal linear is rotated counterclockwise parallel with straight line i, the angle of arbitrary straight line i
computational methods as shown in the formula shown in (1):
Wherein,
for integer, during calculating, adopt the method process rounded up, and
Based on the straight line angle calculated, calculate relative angle histogram, its detailed step is as follows:
(1) the quantity n of the straight line detected in Fig. 3 is confirmed;
(2) 180 minizone θ are set
1~ θ
180, wherein: interval is: θ
1=[0 °, 1 °), θ
2=[1 °, 2 °) ..., θ
i=[(i-1) °, i °) ..., θ
180=[179 °, 180 °);
(3) for the n bar straight line detected in Fig. 3 (n confirms in step (1) to obtain), straight line angle is added up, if angle is θ
istraight line have m bar, then think angle θ
ithe number of times occurred is m time, uses h (θ here
i) represent straight line angle θ
ithe number of times occurred, i.e. the frequency.
(4) to the frequency h (θ that the straight line angle of statistics in step (3) occurs
i) be normalized, calculate each straight line angle θ
irelative frequency H (the θ occurred
i), computational methods are H (θ
i)=h (θ
i)/n.Frequency h (θ
i) normalized object be simplify calculate, reduce value.
(5) draw relative rectilinear angular histogram, as shown in Figure 5, histogrammic transverse axis represents rectilinear direction angle, its domain of definition be [0 °, 180 °), the longitudinal axis represents the relative frequency that straight line angle occurs, its codomain is [0,1].In histogram as shown in Figure 5, each angle θ
ithe height value of corresponding straight line is the relative frequency value H (θ calculated in step (4)
i), the relative frequency value corresponding to the straight line that in Fig. 5, height value is maximum is called maximum histogram peak.
(6) peak-peak H (θ in Fig. 5
k) through type H (θ
k)=Max{H (θ
1), H (θ
2) ..., H (θ
i) ..., H (θ
180) calculate, wherein θ
kstraight line angle corresponding to maximum histogram peak.
Angle θ
kbe the direction of road in this two field picture.The Cleaning Principle of road direction is: in Aerial Images, outline line and the road direction of road and peripheral structure are parallel, therefore the angle that the relative frequency of Fig. 3 cathetus angle appearance is maximum is the direction of road, and this principle is all applicable to general linear road.
Step 4: Aerial Images corrects debounce
Turn clockwise road Aerial Images θ
kdegree, the road in image can be corrected to horizontal direction, the image after correction as shown in Figure 6.
By step 1 is repeated to step 4, the object of video debounce of can taking photo by plane based on road direction correction realization.
Claims (2)
1. unmanned plane is taken photo by plane video jitter removing method, comprises following step:
Step 1: road Aerial Images preliminary treatment
Solution frame is carried out to unmanned plane video of taking photo by plane, obtains single frames RGB color road Aerial Images, and RGB color image is converted to gray-scale map;
Step 2: extract straight line based on Aerial Images
Adopt Canny edge detection operator to obtain gray-scale map edge contour figure, based on Hough transformation Edge detected profile diagram, obtain straight line;
Step 3: calculated line direction histogram
The angle of calculated line, then calculated line relative angle histogram, extract the angle θ that peak point maximum in histogram is corresponding
k, this angle is the direction of road in this frame Aerial Images;
Step 4: Aerial Images corrects debounce
By road Aerial Images dextrorotation gyration θ
k, be unanimously adjusted to horizontal direction by Aerial Images towards different roads, realize road Aerial Images debounce.
2. unmanned plane according to claim 1 is taken photo by plane video jitter removing method, and described step 3 is specially:
If the upper left corner of edge contour figure is the initial point O (0 of image pixel coordinates, 0), with O (0,0) be starting point, being to the right the row reference axis of image, is downwards the row-coordinate axle of image, if any straight line i, two end points is respectively P1 and P2, and wherein the pixel coordinate of P1 and P2 is respectively (c
i_1, r
i_1) and (c
i_2, r
i_2),
for the angle of straight line i and horizontal direction,
the angle turned over when representing and a horizontal linear is rotated counterclockwise parallel with straight line i, the angle of arbitrary straight line i
computational methods as shown in the formula shown in (1):
Wherein,
for integer, round up during calculating, and
Based on the straight line angle calculated, calculate relative angle histogram, be specially:
(1) set the straight line quantity of acquisition in step 2 as n;
(2) 180 minizone θ are set
1~ θ
180, wherein: interval is: θ
1=[0 °, 1 °), θ
2=[1 °, 2 °) ..., θ
i=[(i-1) °, i °) ..., θ
180=[179 °, 180 °);
(3) for n bar straight line, straight line angle is added up, if angle is θ
istraight line have m bar, then think angle θ
ithe number of times occurred is m time, adopts h (θ
i) represent straight line angle θ
ithe number of times occurred, i.e. the frequency;
(4) to the frequency h (θ that the straight line angle obtained in step (3) occurs
i) be normalized, calculate each straight line angle θ
irelative frequency H (the θ occurred
i), computational methods are H (θ
i)=h (θ
i)/n;
(5) draw relative rectilinear angular histogram, histogrammic transverse axis represents rectilinear direction angle, its domain of definition be [0 °, 180 °), the longitudinal axis represents the relative frequency that straight line angle occurs, its codomain is [0,1]; Relative frequency value corresponding to the straight line that in relative angle histogram, height value is maximum is histogram peak;
(6) histogram peak-peak H (θ
k) through type H (θ
k)=Max{H (θ
1), H (θ
2) ..., H (θ
i) ..., H (θ
180) calculate, wherein θ
kstraight line angle corresponding to histogram peak;
Angle θ
kbe the direction of road in this two field picture.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510888639.2A CN105450950B (en) | 2015-12-07 | 2015-12-07 | Unmanned plane video jitter removing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510888639.2A CN105450950B (en) | 2015-12-07 | 2015-12-07 | Unmanned plane video jitter removing method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105450950A true CN105450950A (en) | 2016-03-30 |
| CN105450950B CN105450950B (en) | 2018-07-27 |
Family
ID=55560680
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510888639.2A Active CN105450950B (en) | 2015-12-07 | 2015-12-07 | Unmanned plane video jitter removing method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105450950B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107705301A (en) * | 2017-09-29 | 2018-02-16 | 南京中设航空科技发展有限公司 | A kind of highway graticule damage testing method based on unmanned plane highway map picture |
| CN110320934A (en) * | 2018-03-27 | 2019-10-11 | 杭州海康机器人技术有限公司 | Cloud platform control method, device, holder and mobile device |
| CN111309048A (en) * | 2020-02-28 | 2020-06-19 | 重庆邮电大学 | Method for detecting autonomous flight along road by combining multi-rotor unmanned aerial vehicle with road |
| CN113838313A (en) * | 2021-11-29 | 2021-12-24 | 中国民用航空总局第二研究所 | Obstacle identification method for course beacon channel clearance jitter |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6819779B1 (en) * | 2000-11-22 | 2004-11-16 | Cognex Corporation | Lane detection system and apparatus |
| CN101608924A (en) * | 2009-05-20 | 2009-12-23 | 电子科技大学 | A kind of method for detecting lane lines based on gray scale estimation and cascade Hough transform |
| CN101639893A (en) * | 2008-07-31 | 2010-02-03 | 东软集团股份有限公司 | Method and device for identifying road direction |
| CN101714252A (en) * | 2009-11-26 | 2010-05-26 | 上海电机学院 | Method for extracting road in SAR image |
| CN103500322A (en) * | 2013-09-10 | 2014-01-08 | 北京航空航天大学 | Automatic lane line identification method based on low-altitude aerial images |
| CN103810462A (en) * | 2012-11-14 | 2014-05-21 | 中国科学院沈阳自动化研究所 | High voltage transmission line detection method based on linear targets |
| CN103971081A (en) * | 2013-01-25 | 2014-08-06 | 株式会社理光 | Multi-lane detection method and system |
-
2015
- 2015-12-07 CN CN201510888639.2A patent/CN105450950B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6819779B1 (en) * | 2000-11-22 | 2004-11-16 | Cognex Corporation | Lane detection system and apparatus |
| CN101639893A (en) * | 2008-07-31 | 2010-02-03 | 东软集团股份有限公司 | Method and device for identifying road direction |
| CN101608924A (en) * | 2009-05-20 | 2009-12-23 | 电子科技大学 | A kind of method for detecting lane lines based on gray scale estimation and cascade Hough transform |
| CN101714252A (en) * | 2009-11-26 | 2010-05-26 | 上海电机学院 | Method for extracting road in SAR image |
| CN103810462A (en) * | 2012-11-14 | 2014-05-21 | 中国科学院沈阳自动化研究所 | High voltage transmission line detection method based on linear targets |
| CN103971081A (en) * | 2013-01-25 | 2014-08-06 | 株式会社理光 | Multi-lane detection method and system |
| CN103500322A (en) * | 2013-09-10 | 2014-01-08 | 北京航空航天大学 | Automatic lane line identification method based on low-altitude aerial images |
Non-Patent Citations (2)
| Title |
|---|
| 唐亮、谢维信、黄建军、黄敬雄: "一种新的道路描述子：对称边缘方向直方图", 《电子学报》 * |
| 张道兵、张慧、张正、刘波、王宏琦: "一种稳健的道路主方向提取算法", 《光子学报》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107705301A (en) * | 2017-09-29 | 2018-02-16 | 南京中设航空科技发展有限公司 | A kind of highway graticule damage testing method based on unmanned plane highway map picture |
| CN107705301B (en) * | 2017-09-29 | 2021-04-13 | 南京中设航空科技发展有限公司 | Highway marking damage detection method based on unmanned aerial vehicle aerial highway image |
| CN110320934A (en) * | 2018-03-27 | 2019-10-11 | 杭州海康机器人技术有限公司 | Cloud platform control method, device, holder and mobile device |
| CN110320934B (en) * | 2018-03-27 | 2022-11-22 | 杭州海康机器人技术有限公司 | Cloud deck control method and device, cloud deck and mobile device |
| CN111309048A (en) * | 2020-02-28 | 2020-06-19 | 重庆邮电大学 | Method for detecting autonomous flight along road by combining multi-rotor unmanned aerial vehicle with road |
| CN111309048B (en) * | 2020-02-28 | 2023-05-26 | 重庆邮电大学 | Method for detecting autonomous flight along road by combining multi-rotor unmanned aerial vehicle with road |
| CN113838313A (en) * | 2021-11-29 | 2021-12-24 | 中国民用航空总局第二研究所 | Obstacle identification method for course beacon channel clearance jitter |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105450950B (en) | 2018-07-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10515271B2 (en) | Flight device and flight control method | |
| JP7213809B2 (en) | Video-based positioning and mapping method and system | |
| CN105488501B (en) | The method of license plate sloped correction based on rotation projection | |
| US9619709B2 (en) | Visual localization of unmanned aerial vehicles based on marker detection and processing | |
| CN105450950A (en) | Method for removing jitter from aerial video of unmanned aerial vehicle | |
| Rengarajan et al. | From bows to arrows: Rolling shutter rectification of urban scenes | |
| CN107895375B (en) | Complex road route extraction method based on visual multi-features | |
| CN110163912A (en) | Two dimensional code pose scaling method, apparatus and system | |
| TWI636428B (en) | Image analysis method, camera and image capturing system thereof | |
| CN104268853A (en) | Infrared image and visible image registering method | |
| CN111368797A (en) | Target real-time ranging method based on road end monocular camera | |
| CN111161154A (en) | Real-time and rapid orthoscopic splicing system and method for videos of unmanned aerial vehicle | |
| CN114037970A (en) | Sliding window-based lane line detection method, system, terminal and readable storage medium | |
| CN103514587B (en) | Ship-based image-stabilizing method based on sea-sky boundary detecting | |
| CN106778766B (en) | Positioning point-based rotating number identification method and system | |
| CN114719873A (en) | Low-cost fine map automatic generation method and device and readable medium | |
| Hussnain et al. | Automatic feature detection, description and matching from mobile laser scanning data and aerial imagery | |
| CN105335717B (en) | Face identification system based on the analysis of intelligent mobile terminal video jitter | |
| CN115205825B (en) | Traffic sign detection and identification method based on improved YOLOV5 driving video sequence image | |
| CN108986162B (en) | Dish and background segmentation method based on inertial measurement unit and visual information | |
| CN103456026A (en) | Method for detecting ground moving object under road landmark constraints | |
| JP6492603B2 (en) | Image processing apparatus, system, image processing method, and program | |
| CN103530891B (en) | A kind of Extended target tenacious tracking point extracting method based on the minimum incircle of head | |
| CN103810692A (en) | Method for video tracking of video monitoring equipment and video monitoring equipment | |
| Li et al. | Road markings feature extraction from omni-directional image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |