CN109688329B - Anti-shake method for high-precision panoramic video - Google Patents
Anti-shake method for high-precision panoramic video Download PDFInfo
- Publication number
- CN109688329B CN109688329B CN201811584520.6A CN201811584520A CN109688329B CN 109688329 B CN109688329 B CN 109688329B CN 201811584520 A CN201811584520 A CN 201811584520A CN 109688329 B CN109688329 B CN 109688329B
- Authority
- CN
- China
- Prior art keywords
- image
- video
- shake
- panoramic
- jitter
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 43
- 238000012545 processing Methods 0.000 claims abstract description 19
- 238000000605 extraction Methods 0.000 claims abstract description 11
- 238000001914 filtration Methods 0.000 claims description 15
- 238000009499 grossing Methods 0.000 claims description 15
- 238000012937 correction Methods 0.000 claims description 9
- 230000004927 fusion Effects 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 6
- 239000013598 vector Substances 0.000 claims description 6
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims description 3
- 230000003287 optical effect Effects 0.000 claims description 3
- 230000000737 periodic effect Effects 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 3
- 238000004321 preservation Methods 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 230000000007 visual effect Effects 0.000 abstract description 4
- 238000012544 monitoring process Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000009434 installation Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
- H04N23/681—Motion detection
- H04N23/6811—Motion detection based on the image signal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Studio Devices (AREA)
- Image Processing (AREA)
Abstract
The invention provides an anti-shake method for a high-precision panoramic video, which comprises the following steps: s1, inputting a video image; s2, splicing and fusing the input video images; s3, performing foreground extraction, jitter judgment, characteristic point motion estimation and image processing on the spliced image; and S4, outputting the panoramic stable video. The anti-shake method for the high-precision panoramic video obtains the high-precision panoramic stable image through the anti-shake algorithm, improves the peak signal-to-noise ratio of the output video, and has stable and reliable visual effect.
Description
Technical Field
The invention belongs to the field of security video monitoring, and particularly relates to an anti-shake method for a high-precision panoramic video.
Background
The panoramic video monitoring camera aiming at the wide-area visual angle is more and more favored by customers due to the characteristics of wide shooting visual angle, ultrahigh definition and the like. The method is very suitable for scenes with numerous observation points and complex environment, such as people flow gathering places like squares, stations, supermarkets and the like, and can highlight the advantage of omnibearing 360-degree dead-corner-free monitoring.
Because the installation environment is more and more complicated, receive wind-force and the influence of vibration, can produce the phenomenon of video shake, should add the anti-shake function, improve the ease for use of equipment. The panoramic video surveillance camera has a plurality of video acquisition input ends, and the structure is complicated, and the image is 360 degrees covers, and traditional electron anti-shake algorithm can't be directly suitable for.
Disclosure of Invention
In view of this, the present invention is directed to an anti-shake method for a high-precision panoramic video, so as to solve the problem that the conventional surveillance camera does not have an electronic anti-shake algorithm.
In order to achieve the purpose, the technical scheme of the invention is realized as follows:
an anti-shake method for a high-precision panoramic video, comprising:
s1, inputting a video image;
s2, splicing and fusing the input video images;
s3, performing foreground extraction, jitter judgment, characteristic point motion estimation and image processing on the spliced image;
and S4, outputting the panoramic stable video.
Further, in step S1, the video image input is a video image input by a plurality of sensors received by the multi-view video image input module.
Further, in step S2, the video image splicing and fusing method includes:
s201, filtering and denoising images input by each sensor;
s202, distortion correction before splicing is carried out on the image input by the sensor;
s203, adjusting image brightness information by using a mean value estimation method to perform exposure compensation due to different exposure parameters of the images acquired by the sensors;
and S204, splicing the panoramic image by using a fusion algorithm.
Further, in step S202, the distortion correction method is as follows:
and correcting the distortion of the image by using affine transformation by taking the corner points of the acquired image as characteristic points.
Further, in step S204, image stitching is performed by using a fusion algorithm according to the feature points obtained by distortion correction.
Further, in step S3, the foreground extracting method is to divide the panoramic image into a foreground and a background by the foreground extracting module, and obtain the foreground of the panoramic image that is focused by the person.
The method for judging the jitter comprises the steps of carrying out frequency domain analysis on an image according to the extracted foreground and background through a jitter judging module, and judging that the video has jitter if the image has periodic high-frequency components; if the foreground moves and the background also moves, preprocessing the background movement according to the movement consistency principle;
the characteristic point motion estimation method is to obtain the characteristic points of the panoramic image through a characteristic point motion estimation module, perform optical flow calculation on image pixel points, and estimate motion vectors to be used as a motion compensation path.
Further, in step S3, the image processing includes a shake filtering smoothing process, where the shake filtering smoothing process is to smooth the motion trajectory of the image by using a kalman filter through a shake filtering smoothing module, so as to filter out a high-frequency component caused by image shake.
Further, after the jitter filtering smoothing processing, jitter compensation and feature frame replacement processing are required;
the method for the shake compensation and the characteristic frame replacement processing comprises the steps of carrying out reverse compensation on a motion vector image through a shake compensation and characteristic frame replacement module according to an original track and a smooth expected track to obtain an anti-shake video; in this way, under the conditions of discontinuity and object distortion of the video frame after the anti-shake processing, corresponding pixels of the characteristic frame are adopted for the discontinuous video frame to replace the discontinuous pixels; and for the distorted pixel points, eliminating distortion by adopting a smoothing mode of edge extraction, feature contour preservation and weighted average.
Compared with the prior art, the anti-shake method for the high-precision panoramic video has the following advantages:
the anti-shake method for the high-precision panoramic video obtains the high-precision panoramic stable image through the anti-shake algorithm, improves the peak signal-to-noise ratio of the output video, and has stable and reliable visual effect.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate an embodiment of the invention and, together with the description, serve to explain the invention and not to limit the invention. In the drawings:
fig. 1 is a flowchart of an anti-shake method for a high-precision panoramic video according to an embodiment of the present invention.
Detailed Description
It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict.
In the description of the present invention, it is to be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", and the like, indicate orientations or positional relationships based on those shown in the drawings, and are used only for convenience in describing the present invention and for simplicity in description, and do not indicate or imply that the referenced devices or elements must have a particular orientation, be constructed and operated in a particular orientation, and thus, are not to be construed as limiting the present invention. Furthermore, the terms "first", "second", etc. are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first," "second," etc. may explicitly or implicitly include one or more of that feature. In the description of the present invention, "a plurality" means two or more unless otherwise specified.
In the description of the present invention, it should be noted that, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meaning of the above terms in the present invention can be understood by those of ordinary skill in the art through specific situations.
The present invention will be described in detail below with reference to the embodiments with reference to the attached drawings.
As shown in fig. 1, an anti-shake method for a high-precision panoramic video includes:
s1, inputting a video image;
s2, splicing and fusing the input video images;
s3, performing foreground extraction, jitter judgment, characteristic point motion estimation and image processing on the spliced image;
and S4, synthesizing the stable panoramic video stream by the panoramic stable video output module.
In step S1, the video image input is the video images input by the sensors received by the multi-view video image input module.
In step S2, the method for splicing and fusing the video images includes:
s201, filtering and denoising images input by each sensor;
s202, distortion correction before splicing is carried out on the image input by the sensor;
s203, adjusting image brightness information by using a mean value estimation method to perform exposure compensation due to different exposure parameters of the images acquired by the sensors;
and S204, splicing the panoramic image by using a fusion algorithm.
In step S202, the distortion correction method is as follows:
and correcting the distortion of the image by using affine transformation by taking the corner points of the acquired image as characteristic points.
And step S204, carrying out image splicing by using a fusion algorithm according to the characteristic points obtained by distortion correction.
In step S3, the foreground extraction method is to divide the panoramic image into a foreground and a background by the foreground extraction module, and obtain the foreground of the panoramic image that is focused by the person.
The method for judging the jitter comprises the steps of carrying out frequency domain analysis on an image according to the extracted foreground and background through a jitter judging module, and judging that the video has jitter if the image has periodic high-frequency components; if the foreground moves and the background also moves, preprocessing the background movement according to the movement consistency principle;
the characteristic point motion estimation method is to obtain the characteristic points of the panoramic image through a characteristic point motion estimation module, perform optical flow calculation on image pixel points, and estimate motion vectors to be used as a motion compensation path.
In step S3, the image processing includes a shake filtering smoothing process, where the shake filtering smoothing process is to smooth the motion trajectory of the image by using a kalman filter through a shake filtering smoothing module, and filter out high-frequency components caused by image shake.
After the jitter filtering smoothing processing, jitter compensation and characteristic frame replacement processing are required; the method for the shake compensation and the characteristic frame replacement processing comprises the steps of carrying out reverse compensation on a motion vector image through a shake compensation and characteristic frame replacement module according to an original track and a smooth expected track to obtain an anti-shake video; in this way, under the conditions of discontinuity and object distortion of the video frame after the anti-shake processing, corresponding pixels of the characteristic frame are adopted for the discontinuous video frame to replace the discontinuous pixels; and for the distorted pixel points, eliminating distortion by adopting a smoothing mode of edge extraction, feature contour preservation and weighted average.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (6)
1. An anti-shake method for a high-precision panoramic video, comprising:
s1, inputting a video image;
s2, splicing and fusing the input video images;
s3, performing foreground extraction, jitter judgment, characteristic point motion estimation and image processing on the spliced image;
s4, outputting the panoramic stable video;
in the step S3, the image processing includes a shake filtering smoothing process, where the shake filtering smoothing process is to smooth the motion trajectory of the image by using a kalman filter through a shake filtering smoothing module, and filter out high-frequency components caused by image shake;
after the jitter filtering smoothing processing, jitter compensation and characteristic frame replacement processing are required;
the method for the shake compensation and the characteristic frame replacement processing comprises the steps of carrying out reverse compensation on a motion vector image through a shake compensation and characteristic frame replacement module according to an original track and a smooth expected track to obtain an anti-shake video; in this way, under the conditions of discontinuity and object distortion of the video frame after the anti-shake processing, corresponding pixels of the characteristic frame are adopted for the discontinuous video frame to replace the discontinuous pixels; and for the distorted pixel points, eliminating distortion by adopting a smoothing mode of edge extraction, feature contour preservation and weighted average.
2. The anti-shake method for high-precision panoramic video according to claim 1, characterized in that: in step S1, the video image input is the video images input by the sensors received by the multi-view video image input module.
3. The anti-shake method for high-precision panoramic video according to claim 1, characterized in that: in step S2, the method for splicing and fusing the video images includes:
s201, filtering and denoising images input by each sensor;
s202, distortion correction before splicing is carried out on the image input by the sensor;
s203, adjusting image brightness information by using a mean value estimation method to perform exposure compensation due to different exposure parameters of the images acquired by the sensors;
and S204, splicing the panoramic image by using a fusion algorithm.
4. The anti-shake method for high-precision panoramic video according to claim 3, wherein in step S202, the distortion correction method is as follows:
and correcting the distortion of the image by using affine transformation by taking the corner points of the acquired image as characteristic points.
5. The anti-shake method for high-precision panoramic video according to claim 4, characterized in that: and step S204, carrying out image splicing by using a fusion algorithm according to the characteristic points obtained by distortion correction.
6. The anti-shake method for high-precision panoramic video according to claim 1, wherein in step S3, the foreground extraction method is to divide the panoramic image into a foreground and a background by a foreground extraction module, and obtain the attention foreground of the panoramic image;
the method for judging the jitter comprises the steps of carrying out frequency domain analysis on an image according to the extracted foreground and background through a jitter judging module, and judging that the video has jitter if the image has periodic high-frequency components; if the foreground moves and the background also moves, preprocessing the background movement according to the movement consistency principle;
the characteristic point motion estimation method is to obtain the characteristic points of the panoramic image through a characteristic point motion estimation module, perform optical flow calculation on image pixel points, and estimate motion vectors to be used as a motion compensation path.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811584520.6A CN109688329B (en) | 2018-12-24 | 2018-12-24 | Anti-shake method for high-precision panoramic video |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811584520.6A CN109688329B (en) | 2018-12-24 | 2018-12-24 | Anti-shake method for high-precision panoramic video |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109688329A CN109688329A (en) | 2019-04-26 |
| CN109688329B true CN109688329B (en) | 2020-12-11 |
Family
ID=66188930
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811584520.6A Active CN109688329B (en) | 2018-12-24 | 2018-12-24 | Anti-shake method for high-precision panoramic video |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109688329B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110175011B (en) * | 2019-05-06 | 2022-06-03 | 长春理工大学 | Panoramic image seamless splicing method |
| CN111062881A (en) * | 2019-11-20 | 2020-04-24 | RealMe重庆移动通信有限公司 | Image processing method and device, storage medium and electronic equipment |
| CN114079725B (en) * | 2020-08-13 | 2023-02-07 | 华为技术有限公司 | Video anti-shake method, terminal device, and computer-readable storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104184961A (en) * | 2013-05-22 | 2014-12-03 | 辉达公司 | Mobile device and system used for generating panoramic video |
| CN104732542A (en) * | 2015-03-27 | 2015-06-24 | 安徽省道一电子科技有限公司 | Image processing method for panoramic vehicle safety system based on multi-camera self calibration |
| CN105005964A (en) * | 2015-06-30 | 2015-10-28 | 南京师范大学 | Video sequence image based method for rapidly generating panorama of geographic scene |
| CN107154022A (en) * | 2017-05-10 | 2017-09-12 | 北京理工大学 | A kind of dynamic panorama mosaic method suitable for trailer |
| CN107274346A (en) * | 2017-06-23 | 2017-10-20 | 中国科学技术大学 | Real-time panoramic video splicing system |
| CN207166600U (en) * | 2017-01-23 | 2018-03-30 | 兹曼软件技术(北京)有限公司 | A kind of panorama camera |
| CN108038820A (en) * | 2017-11-14 | 2018-05-15 | 深圳岚锋创视网络科技有限公司 | A kind of method, apparatus and panorama camera for realizing bullet time shooting effect |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5493942B2 (en) * | 2009-12-15 | 2014-05-14 | ソニー株式会社 | Imaging apparatus and imaging method |
-
2018
- 2018-12-24 CN CN201811584520.6A patent/CN109688329B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104184961A (en) * | 2013-05-22 | 2014-12-03 | 辉达公司 | Mobile device and system used for generating panoramic video |
| CN104732542A (en) * | 2015-03-27 | 2015-06-24 | 安徽省道一电子科技有限公司 | Image processing method for panoramic vehicle safety system based on multi-camera self calibration |
| CN105005964A (en) * | 2015-06-30 | 2015-10-28 | 南京师范大学 | Video sequence image based method for rapidly generating panorama of geographic scene |
| CN207166600U (en) * | 2017-01-23 | 2018-03-30 | 兹曼软件技术(北京)有限公司 | A kind of panorama camera |
| CN107154022A (en) * | 2017-05-10 | 2017-09-12 | 北京理工大学 | A kind of dynamic panorama mosaic method suitable for trailer |
| CN107274346A (en) * | 2017-06-23 | 2017-10-20 | 中国科学技术大学 | Real-time panoramic video splicing system |
| CN108038820A (en) * | 2017-11-14 | 2018-05-15 | 深圳岚锋创视网络科技有限公司 | A kind of method, apparatus and panorama camera for realizing bullet time shooting effect |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109688329A (en) | 2019-04-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP5179398B2 (en) | Image processing apparatus, image processing method, and image processing program | |
| US7773115B2 (en) | Method and system for deblurring digital camera images using reference image and motion estimation | |
| CN109688329B (en) | Anti-shake method for high-precision panoramic video | |
| US8508605B2 (en) | Method and apparatus for image stabilization | |
| US8054881B2 (en) | Video stabilization in real-time using computationally efficient corner detection and correspondence | |
| US8488010B2 (en) | Generating a stabilized video sequence based on motion sensor data | |
| JP4377932B2 (en) | Panorama image generating apparatus and program | |
| US9202263B2 (en) | System and method for spatio video image enhancement | |
| US20060244836A1 (en) | Motion stabilization | |
| CN105611116B (en) | A kind of global motion vector method of estimation and monitor video digital image stabilization method and device | |
| CN103517041A (en) | Real-time full-view monitoring method and device based on multi-camera rotating scanning | |
| US8085986B2 (en) | Image processing apparatus and method for processing images more naturally and sharply | |
| EP3782359B1 (en) | Method of combining content from multiple frames and electronic device therefor | |
| JP5210198B2 (en) | Image processing apparatus, image processing method, and image processing program | |
| EP2680567A1 (en) | Video noise reduction | |
| Lee et al. | Color shift model-based image enhancement for digital multifocusing based on a multiple color-filter aperture camera | |
| KR101202642B1 (en) | Method and apparatus for estimating global motion using the background feature points | |
| CN113391644A (en) | Unmanned aerial vehicle shooting distance semi-automatic optimization method based on image information entropy | |
| US20110085026A1 (en) | Detection method and detection system of moving object | |
| CN114429191A (en) | Electronic anti-shake method, system and storage medium based on deep learning | |
| TW201523516A (en) | Video frame stabilization method for the moving camera | |
| JP5539565B2 (en) | Imaging apparatus and subject tracking method | |
| CN108460747B (en) | Sub-aperture synthesis unblocking method of light field camera | |
| CN110784642A (en) | Image processing apparatus, control method thereof, storage medium, and imaging apparatus | |
| KR100734847B1 (en) | Apparatus and method for taking a clear image in slow shutter speed |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |