CN105721776A - Sports camera device with digital image stabilization function and digital image stabilization method - Google Patents

Abstract

According to a sports camera device with a digital image stabilization function and a digital image stabilization method provided by the present invention, an FPGA installed with a gyroscope in a camera watermarks each image, and image stabilization processing is performed on a video at a playing end. An image stabilization algorithm comprises two parts: preliminary image stabilization based on gyroscope information and precise image stabilization based on gradient block matching. The device required by the method provided by the present invention is simple and is low in cost. The algorithm part is not complex in calculation. The image stabilization effect is good. Various videos shot by sports cameras can be well processed.

Description

A kind of Flying Camera machine with Digital image stabilization function and Digital image stabilization method

Technical field

The present invention relates to video image digital processing field, be specifically related to a kind of Flying Camera machine with Digital image stabilization function and Digital image stabilization method.

Background technology

Recently as the development of camera technique, the field such as motion cameras is widely used to outdoor activity, life records, goes sightseeing, Professional Photography, performance recording.In numerous applications, the work platforms residing for motion cameras is generally all unstable, can cause that the image information obtained is also unstable.As often caused shake because of the irregular movement of video camera by the video signal obtained such as hand-held, vehicle-mounted.This shake very easily causes the fatigue of observer's eyes, has a strong impact on further effective exploitation and the utilization of image information.How these video signals are converted into high-quality video of stablizing and there is highly important theory significance and practical value.

Steady acting on by it as technology may be generally divided into three kinds with principle, i.e. the steady picture of machinery, photorefractive crystals and electronic steady image.The steady picture of machinery and photorefractive crystals all must use some special equipments, such as mechanical gyro, servosystem, optical element etc. so that image stabilization system has the features such as cost height, volume are big, is restricted in a lot of application scenarios.And electronic steady image has low cost, small size, construction cycle advantages of higher short, portable, it it is the study hotspot of steady picture technical field at present.

Electronic steady image is image to carry out estimation by obtaining the motion vector of sequence image, and by a kind of technology that Digital Image Processing compensates.According to the difference obtaining image motion vector method, electronic steady image is broadly divided into two kinds: a kind of is that the motion vector utilizing sensor detection video camera is then converted to image motion vector；Another kind is to completely dispense with sensor, directly utilizes Video stabilization and obtains image motion vector, then carries out image compensation.The method of sensor detection, is utilize the inertance elements such as gyroscope directly the quantity of motion of video camera to be detected, not by the impact of picture quality and information, it is not necessary to large amount of complex ask for computing, there is detection speed fast, the advantage that the scope of application is wide.But high-precision gyroscope cost is high, volume is big；And often there is the problem that certainty of measurement is not high in the micro-mechanical gyroscope that volume is little, cost is low.And Video stabilization detection method is directly the image information after imaging to be processed, it is not necessary to auxiliary equipment, its core calculations is that consecutive frame image is mated, and conventional method has: Block Matching Algorithm, Bit-plane matching, Gray Projection method and characteristic matching method.Block Matching Algorithm, Bit-plane matching and Gray Projection method that wherein matching way is relatively simple generally can only estimate translational motion, and when image is deposited when rotated, matching effect is poor.Characteristic matching method is for the effect of steady picture, and matching algorithm is typically more complicated, and amount of calculation is relatively larger, it is difficult to realize in real-time system.

Motion cameras in order to easy to use, often small volume, lighter in weight, be not suitable for high-precision gyroscope is installed；And captured video is normally used for real time processing system, for instance the detection of target and the system of tracking.This needs digital image stabilization method in real time video to be processed, it is impossible to adopt complex Video stabilization.Existing steady all it is not suitable for this application of motion cameras as technology.

Summary of the invention

The present invention is directed to the deficiencies in the prior art, a kind of Flying Camera machine with Digital image stabilization function and Digital image stabilization method are proposed, required matching requirements is simple, less costly, Video stabilization part computation complexity is little, steady better as effect, it is possible to process the various videos of motion cameras shooting preferably.

A kind of Flying Camera machine with Digital image stabilization function of the present invention, it is characterised in that comprise Flying Camera module and playing module: comprise gyroscope in described Flying Camera module, by gyroscope to each image plus movable information；Described playing module comprises steady as intercutting module, and steady picture intercuts module and video image carried out steady as processing, and exports stable video image.

Concrete, described Flying Camera module is made up of cmos sensor, the FPGA installing gyroscope and compression module, after cmos sensor input picture, watermark is stamped to each image again through a FPGA installing described gyroscope, watermark information is the angle information of gyroscope output, then image is compressed, the video after output squeezing.

Concrete, described gyroscope is the micro-mechanical gyroscope of low cost.

Concrete, described playing module also comprises the playout software of routine, uses the playout software of routine that video image is decompressed.

A kind of Digital image stabilization method of the Flying Camera machine with Digital image stabilization function, it is characterised in that: the method comprises two steps, and one is based on the preliminary steady picture of gyroscope information, and two Block-matching being based on gradient information carry out accurately steady picture.

Concrete, the described preliminary steady picture based on gyroscope information comprises the steps:

(1) after the video shot by motion cameras being decompressed, it is thus achieved that video in adjacent kth frame and kth-1 two field picture, this two frame is set to reference frame and present frame；

(2) the gyroscope angle information of current frame image namely watermark are (ω^k _x,ω^k _y), it is the kth frame image translation information that is equivalent to kth-1 two field picture, then kth frame image is equivalent to the translation information of the 1st two field picture and is:Being translated by kth frame image, translation vector isWherein f is focal length, namely obtains the frame of video of preliminarily stabilised.

Concrete, the described Block-matching based on gradient information carries out accurately steady as comprising the steps:

(1) current frame image after carrying out preliminarily stabilised and reference frame image are calculated, respectively obtain the edge gradient image of correspondence；

(2) present frame gradient image is divided into much it is sized to the subimage into M × N, randomly choose 10 subimages, on reference frame gradient image, then calculate the Block-matching image of correspondence respectively；

(3) by mating the image motion vector to calculating each subimage, there are 10 motion vectors, these 10 motion vectors are obtained through median filtering operation the motion vector μ of present frame；

(4) start each two field picture is calculated all by the above process the motion vector of two consecutive frames from the 1st frame of video, ifFor the kth frame motion vector relative to kth-1 frame, carry out cumulative obtaining the present frame global motion amount to the 1st frame ${\mathrm{\μ}}_1^k=\underset{i=1}{\overset{k-1}{\Σ}}{\mathrm{\μ}}_i^{i+1};$

(5) adopt method of least square to motion vectorCarrying out curve fitting, motion vector is smoothed, the vector after note is smooth is main motion amountThe motion compensation quantity of present frameComputing formula is as follows:According to gained motion compensation quantityPresent frame is compensated, obtains final stable video.

Assembly of the invention requires simple, adopts the micro-mechanical gyroscope of low cost, less costly, and the algorithm part computation complexity in the Digital image stabilization method of the present invention is little, steady better as effect, it is possible to process the various videos of motion cameras shooting preferably.

Accompanying drawing explanation

Fig. 1 is the Flying Camera machine schematic diagram with Digital image stabilization function that the present invention proposes；

Fig. 2 is the flow chart of the Digital image stabilization method that the present invention proposes.

Detailed description of the invention

Below in conjunction with accompanying drawing, the present invention is described in further details.

Flying Camera machine with Digital image stabilization function is as shown in Figure 1, after cmos sensor input picture, watermark is stamped to each image by the FPGA of an installation gyroscope, watermark information is the angle information of gyroscope output, then pass through compression module image is compressed, the video after output squeezing.By wifi when playback terminal plays out, first by traditional playout software, video being decompressed, what then pass through that the present invention proposes steady carries out steady as processing as intercutting software to the image after decompression, finally plays out stable video image.

It is the unstable video image after decompression that the steady picture that the present invention proposes intercuts software input, by Video stabilization, video image is processed, finally exports stable video image.

Video stabilization therein specifically comprises the following steps that

Assuming that motion cameras imaging is a desirable pin-hole model, then 1 Dot (x in camera coordinate system_a,y_a,z_a), by perspective projection, this point is projected to imaging plane, the coordinate of this subpoint dot be (x, y), then both sides relation is as follows:

$\left\{\begin{array}{c}x=f\frac{x_a}{z_a}\\ y=f\frac{y_a}{z_a}\end{array}\right.---\left(1\right)$

Wherein f is focal length.

When motion cameras rotates θ along x-axis_xTime, then the coordinate in some Dot camera coordinate system after rotation is for making Dot ' (x '_a,y′_a,z′_a), both coordinates have following relation:

Order point Dot ' (x '_a,y′_a,z′_a) it is dot ' (x ', y ') at the subpoint of imaging plane, bring formula (2) into formula (1), due to θ_xOnly small, it can be assumed that (y_a/f)θ_xMuch smaller than 1, may finally obtain:

X '=x, y '=y+f θ_x(3)

Similarly, motion cameras rotates θ along y-axis_y, can obtain:

X '=x+f θ_y, y '=y (4)

When motion cameras rotates around x-axis and y-axis simultaneously, coordinate transform formula is as follows:

$\left\{\begin{array}{c}{x}^{\′}=x+f\·{\mathrm{\θ}}_y\\ y^{\′}=u+f\·{\mathrm{\θ}}_x\end{array}\right.---\left(5\right)$

From formula 5 it will be seen that motion cameras can represent with the translation parameters of video camera around the rotation of x-axis and y-axis.

Under theoretical case, the translation parameters of motion cameras accurately can be obtained by the angle information that gyroscope exports.But in practical situation, due to the precision problem in sampling time and gyroscope itself, still there is certain error in the output result of gyroscope.In order to improve the precision of steady picture, the angle information (ω that first gyroscope is exported by we_x,ω_y) as initial estimate, image is carried out translation transformation.Specifically comprise the following steps that

After the video shot by motion cameras is decompressed, it is thus achieved that video in adjacent kth frame and kth-1 two field picture be set to reference frame and present frame.The gyroscope angle information of current frame image namely watermark are (ω^k _x,ω^k _y), it is the kth frame image translation information that is equivalent to kth-1 two field picture.So kth frame image is equivalent to the translation information of the 1st two field picture and is:Being translated by kth frame image, translation vector isNamely the frame of video of preliminarily stabilised is obtained.

Due to precision problem, still there is the shake that some amplitudes are little in the frame of video so processed, next carries out once precise and stable process again.The present invention adopts a kind of block matching method based on gradient information that video image carries out accurately steady picture.Classical block matching method adopts the half-tone information of image, it is easy to be subject to the interference of noise.Therefore we adopt sobel operator, extract the edge gradient information of image, as the object of coupling.

First the current frame image (being designated as D) after carrying out preliminarily stabilised is calculated, obtains edge gradient image (being designated as DT),

$DT=\sqrt{D_x{(x,y)}^2+D_y{(x,y)}^2}---\left(6\right)$

Wherein $D_x=\left[\begin{array}{ccc}-1& 0& 1\\ -2& 0& 2\\ -1& 0& 1\end{array}\right]*D$ The gradient information being horizontally oriented, $D_y=\left[\begin{array}{ccc}1& 2& 1\\ 0& 0& 0\\ -1& -2& -1\end{array}\right]*D$ The gradient information being vertically oriented.

In like manner the reference frame image (being designated as C) after preliminarily stabilised is calculated, obtains edge gradient image (being designated as CT).

Image DT is divided into and is much sized to the subimage into M × N, randomly choose 10 subimage DT_s1,…,DT_s10, then calculate corresponding Block-matching image at image CT respectively.With subimage DT_s1For example, left upper apex coordinate is (x₁,y₁), so with a window being sized to M × N in the enterprising line slip of image CT, owing to having made initial steady picture, the scope of slip scan can somewhat take smaller, here we make hunting zone be 10, namely the left upper apex of sliding window is only with (x₁,y₁) centered by, length be 10 field in slide.Slip each time all obtains the subimage of a M × N on image CT, is designated as CT_s1, adopt minimum absolute difference and criterion SAD to calculate two width subimage DT_s1And CT_s1Matching degree:

$SAD(m,n)=\underset{x=1}{\overset{M}{\Σ}}\underset{y=1}{\overset{N}{\Σ}}|{\mathrm{DT}}_{s1}(x,y)-{\mathrm{CT}}_{s1}(x,y)|---\left(7\right)$

Wherein m, n represent that sliding window is relative to coordinate (x respectively₁,y₁) deviation post, therefore | m | < 10, | n | < 10；

Calculating all corresponding matching degrees of sliding, as SAD, (m, when n) reaching minimum, now slip is the best match position of two width images, and corresponding side-play amount is designated as (m₁,n₁), it is subimage DT_s1Striked motion vector.

Calculate the motion vector of remaining nine width subimages after the same method respectively, there are 10 motion vectors.These 10 motion vectors are obtained through median filtering operation the motion vector μ of present frame.

Start each two field picture is calculated all by the above process the motion vector of two consecutive frames from the 1st frame of video, ifFor the kth frame motion vector relative to kth-1 frame, carry out cumulative obtaining the present frame global motion amount to the 1st frame

Owing to video there is also shake by a small margin, so the motion vector asked for is not smooth.The present invention adopts method of least square to motion vectorCarry out curve fitting, it is possible to effectively motion vector is smoothed.Vector after note is smooth is main motion amountThe motion compensation quantity of present frameComputing formula is as follows:

${\mathrm{\&mu;}}_b^t={\mathrm{\&mu;}}_1^t-{\mathrm{\&mu;}}_z^t---\left(8\right)$

According to gained motion compensation quantityPresent frame is compensated, it is possible to obtain final stable video.

The motion cameras device and method with Digital image stabilization function that the present invention proposes, at video camera end equipped with gyroscope, obtains the movable information of video camera, then steady video is carried out steady as processing as intercutting software, finally obtains stable video image.It is simple that the present invention proposes the matching requirements needed for method, less costly, and algorithm part computation complexity is little, steady better as effect, it is possible to process the various videos of motion cameras shooting preferably.

Above a kind of camera system with Digital image stabilization function provided by the invention and Digital image stabilization method being described in detail, description above is only intended to help to understand method and the core concept thereof of the present invention；Simultaneously for those skilled in the art, according to the thought of the present invention, it may also be made that some simple deduction or replace, all should be considered as belonging to the protection domain that claims of present invention submission are determined.

Claims (7)

1. the Flying Camera machine with Digital image stabilization function, it is characterised in that comprise Flying Camera module and playing module: comprise gyroscope in described Flying Camera module, by gyroscope to each image plus movable information；Described playing module comprises steady as intercutting module, and steady picture intercuts module and video image carried out steady as processing, and exports stable video image.

2. a kind of Flying Camera machine with Digital image stabilization function according to claim 1, it is characterized in that: described Flying Camera module is made up of cmos sensor, the FPGA installing gyroscope and compression module, after cmos sensor input picture, watermark is stamped to each image again through a FPGA installing described gyroscope, watermark information is the angle information of gyroscope output, then image is compressed, the video after output squeezing.

3. a kind of Flying Camera machine with Digital image stabilization function according to claim 2, it is characterised in that: described gyroscope is the micro-mechanical gyroscope of low cost.

4. a kind of Flying Camera machine with Digital image stabilization function according to claim 3, it is characterised in that: described playing module also comprises the playout software of routine, uses the playout software of routine that video image is decompressed.

5. the Digital image stabilization method of the arbitrary described Flying Camera machine with Digital image stabilization function of Claims 1-4, it is characterized in that: comprise two steps, the one preliminary steady picture being based on gyroscope information, two Block-matching being based on gradient information carry out accurately steady picture.

6. a kind of Digital image stabilization method according to claim 5, it is characterised in that the described preliminary steady picture based on gyroscope information comprises the steps:

(1) after the video shot by motion cameras being decompressed, it is thus achieved that video in adjacent kth frame and kth-1 two field picture, this two frame is set to reference frame and present frame；

(2) the gyroscope angle information of current frame image namely watermark are (ω^k _x,ω^k _y), it is the kth frame image translation information that is equivalent to kth-1 two field picture, then kth frame image is equivalent to the translation information of the 1st two field picture and is:Being translated by kth frame image, translation vector isNamely the frame of video of preliminarily stabilised is obtained.

7. a kind of Digital image stabilization method according to claim 5, it is characterised in that the described Block-matching based on gradient information carries out accurately steady as comprising the steps:

(1) current frame image after carrying out preliminarily stabilised and reference frame image are calculated, respectively obtain the edge gradient image of correspondence；

(2) present frame gradient image is divided into much it is sized to the subimage into M × N, randomly choose 10 subimages, on reference frame gradient image, then calculate the Block-matching image of correspondence respectively；

(3) by mating the image motion vector to calculating each subimage, there are 10 motion vectors, these 10 motion vectors are obtained through median filtering operation the motion vector μ of present frame；

(4) start each two field picture is calculated all by the above process the motion vector of two consecutive frames from the 1st frame of video, ifFor the kth frame motion vector relative to kth-1 frame, carry out cumulative obtaining the present frame global motion amount to the 1st frame ${\mathrm{\&mu;}}_1^k=\underset{i=1}{\overset{k-1}{\&Sigma;}}{\mathrm{\&mu;}}_i^{i+1};$

(5) adopt method of least square to motion vectorCarrying out curve fitting, motion vector is smoothed, the vector after note is smooth is main motion amountThe motion compensation quantity of present frameComputing formula is as follows:According to gained motion compensation quantityPresent frame is compensated, obtains final stable video.