CN107222662A - A kind of electronic image stabilization method based on improved KLT and Kalman filtering - Google Patents
A kind of electronic image stabilization method based on improved KLT and Kalman filtering Download PDFInfo
- Publication number
- CN107222662A CN107222662A CN201710563620.XA CN201710563620A CN107222662A CN 107222662 A CN107222662 A CN 107222662A CN 201710563620 A CN201710563620 A CN 201710563620A CN 107222662 A CN107222662 A CN 107222662A
- Authority
- CN
- China
- Prior art keywords
- motion
- motion vector
- image
- frame image
- reference frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/14—Picture signal circuitry for video frequency region
- H04N5/21—Circuitry for suppressing or minimising disturbance, e.g. moiré or halo
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/14—Picture signal circuitry for video frequency region
- H04N5/144—Movement detection
- H04N5/145—Movement estimation
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Studio Devices (AREA)
Abstract
The invention discloses a kind of electronic image stabilization method based on improved KLT and Kalman filtering.Method includes：Estimation of motion vectors, motion filtering and motion compensation.The estimation of motion vectors is to utilize Shi Tomasi algorithms detection reference frame image feature angle point, the Lucas Kanade optical flow algorithm tracking and matching angle points of pyramid model, RANSAC algorithm removes Mismatching point, least square fitting global motion vector；The motion filtering separates video camera intended motion using Kalman filtering algorithm and unintentionally shaken；The motion compensation compensates motion vector of the video sequence current frame image relative to reference frame image using transformation parameter and image affine Transform Model.Method therefor of the present invention is simple, steady as precision is high, processing is time-consuming short, simultaneously, video camera is preferably resolved due to video sequence jitter problems caused by unnecessary external interference such as platform shakes, and this method has versatility, the occasions such as hand-held picture pick-up device, video monitoring, military monitoring can be used in.
Description
Technical field
The present invention relates to video camera field of video image processing, various video cameras, such as video monitoring, hand are can be applied to
Video camera, space flight and aviation monitoring, military monitoring etc. are held, infrared video camera electronic steady image can be preferably applied to.
Background technology
Electronic steady image is always the key content of video camera image capturing system, video camera in imaging process due to
The shake of platform and external interference cause the video sequence that camera is obtained in imaging process with shake or even distorted, to scene
Observation generates strong influence, while also limit high-level visual task, such as the inspection of special exercise target in video
Survey and track.
The electronic image stabilization method of video camera leaves many difficult points：Existing electronic steady image such as BMA is commonly available to
Only in the case of translational motion vector, and to noise is more sensitive and sequence of frames of video to being obtained in dynamic scene
Timing easily causes matching error.Although Gray Projection method algorithm speed requires the half-tone information that image has than very fast
Change abundant, otherwise cannot accurately change obvious Gray scale projection curve, cause electronic steady image effect poor.For red
Outer video camera, the noise of video image of acquisition is larger, half-tone information change is not enriched, therefore be not suitable for using BMA and
Gray Projection method realizes electronic steady image function.
For problem above, this method uses the electronic image stabilization method based on improved KLT and Kalman filtering, KLT light
The present frame and the motion vector of reference frame image for the acquisition video sequence that tracing can be more accurate are flowed, precision is high, can be with
Sub-pixel is reached, while suitable for existing the complicated scenes such as translation, rotation, scaling.
The content of the invention
The purpose of the present invention, is to provide for a kind of realtime electronic image stabilizing method, for solving video camera because platform is trembled
Dynamic and video sequence shake and aberration problems caused by external interference.
To achieve the above object, the present invention is adopted the following technical scheme that：
1st, this method implements the step of electronic steady image three：1) estimation of motion vectors, 2) motion filtering, 3) motion compensation.
2nd, described method for estimating motion vector is as follows, and reference frame is detected first with Shi-Tomasi Corner Detection Algorithms
Feature angle point in image, then improves the spy that Lucas-Kanade optical flow algorithm tracking and matchings are detected using pyramid model
Angle point is levied, recycles the RANSAC algorithm based on statistical method to reject the Mismatching point pair produced in matching process,
Finally reference frame is brought into using the matching double points of acquisition and in current frame image transformation model, obtain one group of overdetermined equation, utilize
Least square fitting obtains the global motion vector between image sequence；
3rd, described motion filtering method is as follows, using Kalman filtering algorithm separate infrared video camera intended motion and
Unintentionally shake, so as to retain the intended motion of camera, and to step 1) obtained global motion vector parameter is modified
To the motion vector parameter of amendment.
4th, described motion compensation process is as follows, utilizes step 3) in obtained correction motion vector parameters and video sequence
Current frame image and reference frame image between affine Transform Model video sequence motion vector is compensated, obtain stabilization
Video sequence.
It is an advantage of the current invention that effectively solving many problems left in video camera electronic steady image processing procedure：Place
Manage speed slow, it is impossible to accomplish in real time, image texture does not clearly result in steady as low precision, it is impossible to solve to ask in the presence of rotation etc. between image
Topic.Meanwhile, this method does not need additional process chip, and cost is low, is easy to implement.
Brief description of the drawings
Fig. 1 is detailed implementation figure of the present invention.
Fig. 2 is steady as result for specific implementation example of the invention, and figure a is steady as preceding video sequence, wherein before figure (1) is steady picture
The 30th frame, figure (2) is to be steady as the 54th preceding frame, and figure (3) is to be steady as the 78th preceding frame；B is schemed to be steady as backsight frequency sequence, wherein
It is steady the 30th frame as after to scheme (1), and figure (2) is steady the 54th frame as after, and figure (3) is steady the 78th frame as after.
Embodiment
The present invention is further retouched with reference to specific embodiment, the i.e. electronic steady image of refrigeration mode infrared video camera
State, only illustrated to explain the present invention with the suitability example of the present invention herein, but be not intended as the restriction of the present invention.
Refrigeration mode infrared video camera is made up of infrared detector, analog dither platform, data collecting system, power supply.
In specific embodiment, unintentionally shaken using shaker deck simulation, so that output video sequence shake can be caused, utilize the present invention
Method solves this problem.Method is as shown in figure 1, by estimation of motion vectors, three part groups of motion filtering and motion compensation
Into.
First, using a certain frame in infrared video sequence as reference frame, after being pre-processed to image, Shi- is utilized
Tomasi Corner Detection Algorithms extract the feature angle point in reference frame image, then using improved based on pyramid model
The feature angle point that Lucas-Kanade optical flow algorithm tracking and matchings are detected, recycles RANSAC algorithms to reject in matching process and produces
Raw Mismatching point pair, current frame image and reference frame image are obtained finally according to matching double points using least square fitting
Global motion vector；
Secondly, separate infrared video camera intended motion using Kalman filtering algorithm and unintentionally shake, so as to retain
The intended motion of camera, the shake that is not intended to here is the unnecessary shake that analog dither platform is produced.
Finally, the current frame image in the final correction motion vector parameters and video sequence obtained afterwards after filtering is utilized
Affine Transform Model between reference frame image is compensated to video sequence motion vector, the video sequence stablized.
Electronic steady image mean error is 0.1318pixel in this example, and root-mean-square value is that 0.0457. speed is 34 frame per second.Steady picture result
As shown in Figure 2.
The foregoing is only a preferred embodiment of the present invention, but protection scope of the present invention be not limited thereto,
Any one skilled in the art the invention discloses technical scope in, according to system of the present invention and in fact
Equal change and modification that applying method is done, should all be included within the scope of the present invention.Therefore, protection model of the invention
Enclose is defined by scope of the claims.
Claims (1)
1. a kind of electronic image stabilization method based on improved KLT and Kalman filtering, by 1) estimation of motion vectors, 2) moves filter
Ripple, 3) three steps of motion compensation are constituted, it is characterised in that：
1), described method for estimating motion vector is as follows, and reference frame figure is detected first with Shi-Tomasi Corner Detection Algorithms
Feature angle point as in, then improves the feature that Lucas-Kanade optical flow algorithm tracking and matchings are detected using pyramid model
Angle point, recycles the RANSAC algorithm based on statistical method to reject the Mismatching point pair produced in matching process, most
Reference frame is brought into in current frame image transformation model, one group of overdetermined equation being obtained, using most using the matching double points of acquisition afterwards
Small square law fitting obtains the global motion vector between image sequence；
2), described motion filtering method is as follows, and infrared video camera intended motion and nothing are separated using Kalman filtering algorithm
The shake of meaning, so as to retain the intended motion of camera, and to step 1) obtained global motion vector parameter is modified and obtains
The motion vector parameter of amendment；
3), described motion compensation process is as follows, utilizes step 2) in obtained correction motion vector parameters and video sequence
Affine Transform Model between current frame image and reference frame image is compensated to video sequence motion vector, is stablized
Video sequence.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710563620.XA CN107222662A (en) | 2017-07-12 | 2017-07-12 | A kind of electronic image stabilization method based on improved KLT and Kalman filtering |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710563620.XA CN107222662A (en) | 2017-07-12 | 2017-07-12 | A kind of electronic image stabilization method based on improved KLT and Kalman filtering |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107222662A true CN107222662A (en) | 2017-09-29 |
Family
ID=59952273
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710563620.XA Pending CN107222662A (en) | 2017-07-12 | 2017-07-12 | A kind of electronic image stabilization method based on improved KLT and Kalman filtering |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107222662A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109045407A (en) * | 2018-08-31 | 2018-12-21 | 四川省肿瘤医院 | A kind of real-time transfusion monitoring system of gravity type stabilization based on ZigBee-network |
| CN109302545A (en) * | 2018-11-15 | 2019-02-01 | 深圳市炜博科技有限公司 | Video image stabilization method, device and computer readable storage medium |
| CN109743495A (en) * | 2018-11-28 | 2019-05-10 | 深圳市中科视讯智能系统技术有限公司 | Video image electronic stability augmentation method and device |
| CN109887011A (en) * | 2017-12-06 | 2019-06-14 | 中国科学院宁波材料技术与工程研究所 | Video image stabilization method and device |
| CN109922258A (en) * | 2019-02-27 | 2019-06-21 | 杭州飞步科技有限公司 | Electronic image stabilization method, device and the readable storage medium storing program for executing of in-vehicle camera |
| CN110660086A (en) * | 2019-06-17 | 2020-01-07 | 珠海全志科技股份有限公司 | Motion control method and system based on optical flow algorithm |
| CN112287819A (en) * | 2020-10-28 | 2021-01-29 | 武汉三力通信有限责任公司 | High-speed multi-channel real-time image stabilizing method for video recording equipment |
| CN113949812A (en) * | 2021-10-21 | 2022-01-18 | 浙江大立科技股份有限公司 | Electronic image stabilization method based on partitioned Kalman motion prediction |
| CN114584785A (en) * | 2022-02-07 | 2022-06-03 | 武汉卓目科技有限公司 | Real-time image stabilizing method and device for video image |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080273751A1 (en) * | 2006-10-16 | 2008-11-06 | Chang Yuan | Detection and Tracking of Moving Objects from a Moving Platform in Presence of Strong Parallax |
| CN102231792A (en) * | 2011-06-29 | 2011-11-02 | 南京大学 | Electronic image stabilization method based on characteristic coupling |
| CN104853064A (en) * | 2015-04-10 | 2015-08-19 | 海视英科光电（苏州）有限公司 | Electronic image-stabilizing method based on infrared thermal imager |
| CN106878612A (en) * | 2017-01-05 | 2017-06-20 | 中国电子科技集团公司第五十四研究所 | A kind of video stabilizing method based on the optimization of online total variation |
-
2017
- 2017-07-12 CN CN201710563620.XA patent/CN107222662A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080273751A1 (en) * | 2006-10-16 | 2008-11-06 | Chang Yuan | Detection and Tracking of Moving Objects from a Moving Platform in Presence of Strong Parallax |
| CN102231792A (en) * | 2011-06-29 | 2011-11-02 | 南京大学 | Electronic image stabilization method based on characteristic coupling |
| CN104853064A (en) * | 2015-04-10 | 2015-08-19 | 海视英科光电（苏州）有限公司 | Electronic image-stabilizing method based on infrared thermal imager |
| CN106878612A (en) * | 2017-01-05 | 2017-06-20 | 中国电子科技集团公司第五十四研究所 | A kind of video stabilizing method based on the optimization of online total variation |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109887011A (en) * | 2017-12-06 | 2019-06-14 | 中国科学院宁波材料技术与工程研究所 | Video image stabilization method and device |
| CN109045407A (en) * | 2018-08-31 | 2018-12-21 | 四川省肿瘤医院 | A kind of real-time transfusion monitoring system of gravity type stabilization based on ZigBee-network |
| CN109302545B (en) * | 2018-11-15 | 2021-06-29 | 深圳万兴软件有限公司 | Video image stabilization method and device and computer readable storage medium |
| CN109302545A (en) * | 2018-11-15 | 2019-02-01 | 深圳市炜博科技有限公司 | Video image stabilization method, device and computer readable storage medium |
| CN109743495A (en) * | 2018-11-28 | 2019-05-10 | 深圳市中科视讯智能系统技术有限公司 | Video image electronic stability augmentation method and device |
| CN109743495B (en) * | 2018-11-28 | 2021-02-09 | 深圳市中科视讯智能系统技术有限公司 | Electronic stability augmentation method and device for video image |
| CN109922258A (en) * | 2019-02-27 | 2019-06-21 | 杭州飞步科技有限公司 | Electronic image stabilization method, device and the readable storage medium storing program for executing of in-vehicle camera |
| CN109922258B (en) * | 2019-02-27 | 2020-11-03 | 杭州飞步科技有限公司 | Electronic image stabilizing method and device for vehicle-mounted camera and readable storage medium |
| CN110660086A (en) * | 2019-06-17 | 2020-01-07 | 珠海全志科技股份有限公司 | Motion control method and system based on optical flow algorithm |
| CN110660086B (en) * | 2019-06-17 | 2022-01-04 | 珠海全志科技股份有限公司 | Motion control method and system based on optical flow algorithm |
| CN112287819A (en) * | 2020-10-28 | 2021-01-29 | 武汉三力通信有限责任公司 | High-speed multi-channel real-time image stabilizing method for video recording equipment |
| CN113949812A (en) * | 2021-10-21 | 2022-01-18 | 浙江大立科技股份有限公司 | Electronic image stabilization method based on partitioned Kalman motion prediction |
| CN114584785A (en) * | 2022-02-07 | 2022-06-03 | 武汉卓目科技有限公司 | Real-time image stabilizing method and device for video image |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107222662A (en) | A kind of electronic image stabilization method based on improved KLT and Kalman filtering | |
| Aguilar et al. | Real-time model-based video stabilization for microaerial vehicles | |
| Aghaei et al. | PV power plant inspection by image mosaicing techniques for IR real-time images | |
| US10573018B2 (en) | Three dimensional scene reconstruction based on contextual analysis | |
| US10911680B2 (en) | Method and system of geolocation and attitude correction for mobile rolling shutter cameras | |
| CN107343145A (en) | A kind of video camera electronic image stabilization method based on robust features point | |
| WO2016030305A1 (en) | Method and device for registering an image to a model | |
| CN110956661B (en) | Method for calculating dynamic pose of visible light and infrared camera based on bidirectional homography matrix | |
| CN110677578A (en) | Mixed image stabilization method and device based on bionic eye platform | |
| US11042984B2 (en) | Systems and methods for providing image depth information | |
| Kejriwal et al. | A hybrid filtering approach of digital video stabilization for UAV using kalman and low pass filter | |
| CN107749069B (en) | Image processing method, electronic device and image processing system | |
| Ding et al. | Minimal solutions for panoramic stitching given gravity prior | |
| CN111712857A (en) | Image processing method, device, holder and storage medium | |
| Gu et al. | Real-time image mosaicing system using a high-frame-rate video sequence | |
| Vlahović et al. | Deep learning in video stabilization homography estimation | |
| Florez et al. | Video stabilization taken with a snake robot | |
| Dale et al. | Target tracking, moving target detection, stabilisation and enhancement of airborne video | |
| Contreras et al. | FPGA implementation of global vision for robot soccer as a smart camera | |
| Sörös et al. | Multiframe visual-inertial blur estimation and removal for unmodified smartphones | |
| Diskin et al. | UAS exploitation by 3D reconstruction using monocular vision | |
| Rasheed et al. | AirMatch: An automated mosaicing system with video preprocessing engine for multiple aerial feeds | |
| Verma et al. | FAST Based Video Stabilization with Preserved Intentional Motion and Smear Removal | |
| Shankarpure et al. | Real-Time Frame-to-Frame Jitter Removing Video Stabilization Technique | |
| Bernardo et al. | Onboard video stabilization for low cost small rpas surveillance applications |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20170929 |
|
| WD01 | Invention patent application deemed withdrawn after publication |