CN102073993B - Camera self-calibration-based jittering video deblurring method and device - Google Patents
Camera self-calibration-based jittering video deblurring method and device Download PDFInfo
- Publication number
- CN102073993B CN102073993B CN2010106126772A CN201010612677A CN102073993B CN 102073993 B CN102073993 B CN 102073993B CN 2010106126772 A CN2010106126772 A CN 2010106126772A CN 201010612677 A CN201010612677 A CN 201010612677A CN 102073993 B CN102073993 B CN 102073993B
- Authority
- CN
- China
- Prior art keywords
- image
- blurred
- blurred image
- video
- camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The invention provides a camera self-calibration-based jittering video deblurring method and a camera self-calibration-based jittering video deblurring device. The method comprises the following steps: A, calculating the initial point diffuse function and deblurred image of a blurred image; B, performing self-calibration on the deblurred image to obtain the internal and external parameters of a camera; C, calculating a depth map according to the deblurred image and the internal and external parameters; D, estimating the intra-frame motion of the camera according to the initial point diffuse function and the depth map; E, optimizing intra-frame motion and the deblurred image according to a probability model to obtain the final intra-frame motion and the final deblurred image; and F, executing the steps from A to E circularly to obtain the deblurred video of a jittering video. In the method, depth-associated video image sequence deblurring is realized by a camera self-calibration technique, so the video image sequence has a better deblurring effect and the image deblurring efficiency is improved.
Description
Technical field
The present invention relates to computer vision and digital video image process field, particularly a kind of shake video deblurring method and device based on camera self-calibration.
Background technology
The deblurring of shake video is a kind of video image processing technology.Along with the development of picture pick-up device, the price of various apparatuss for making a video recording reduces significantly, and individual picture pick-up device and various hand-held picture pick-up device are popularized in a large number, cause a large amount of appearance of vedio data.And because image blurring the becoming that the motion of video camera causes when taking reduces one of video image quality principal element, therefore, many image deblurring algorithms are suggested, to be used to repair fuzzy image.Existing algorithm adopts the consistent point spread function hypothesis in space mostly, and the fuzzy core of promptly supposing on the image to be had a few is identical.Yet in fact the point spread function of each point is relevant with the degree of depth of motion and this some place corresponding scenery of video camera in the time shutter on the image.Extreme example is that the scenery in the infinite distance can't thicken because of the translation motion of video camera.Therefore, adopt overall consistent point spread function to make and have, even can cause the violent fringe region of change in depth ringing to occur than some regional deblurring poor effect in the image of big depth range.Therefore, how can especially to shake video image deblurring, and how to make the deblurring better effects if of image become current social problem demanding prompt solution to video image.
Summary of the invention
The object of the invention is intended to solve at least one of above-mentioned technological deficiency.
The present invention be directed to the deblurring poor effect of existing shake video deblurring method, and a kind of shake video image deblurring method and the device based on camera self-calibration that propose.
For achieving the above object; One aspect of the present invention proposes a kind of shake video image deblurring method based on camera self-calibration, and may further comprise the steps: A. is according to the initial point spread function and the de-blurred image of blurred picture in the said shake video of blind deconvolution algorithm computation; B. said de-blurred image is carried out from demarcating to obtain the inside and outside parameter of video camera; C. according to the depth image of said de-blurred image and the said de-blurred image of the inside and outside calculation of parameter of said video camera; D. estimate the intraframe motion of said video camera according to said initial point spread function and said depth image; E. optimize said intraframe motion and said de-blurred image according to probability model, to obtain final intraframe motion and final de-blurred image; And F. circulation carries out A and finishes dealing with until all two field pictures of said shake video to E, obtains the deblurring video of said shake video.
In an embodiment of the present invention, said steps A further comprises: said blurred picture is carried out down-sampling to reduce the blur level of said blurred picture; According to the sparse constraint condition of convolution kernel, adopt the Richardson-Lucy algorithm that said blurred picture is carried out the blind deconvolution computing, to obtain said de-blurred image and convolution kernel; Said convolution kernel is carried out up-sampling, and as initial value said blurred picture is carried out the blind deconvolution computing, with blurred picture after being optimized and convolution kernel with said convolution kernel.
In an embodiment of the present invention, said step B further comprises: according to the gaussian filtering equation said de-blurred image is carried out filtering, to obtain the denoising image; Detect the unique point of said denoising image according to KLT feature point tracking algorithm; The inside and outside parameter of said denoising image is demarcated in employing based on the quadric camera self-calibration algorithm of absolute antithesis.
In an embodiment of the present invention, said step C further comprises: the ID figure that calculates said de-blurred image according to adjacent front and back two two field pictures of belief propagation algorithm and said de-blurred image; Adopt the average drifting algorithm that said de-blurred image is carried out color and cut apart, said ID figure is carried out match according to carve information; Adopt a bundle collection adjustment algorithm that said ID figure is optimized, to guarantee the consistance of the degree of depth.
In an embodiment of the present invention, said step D further comprises: D1. estimates the motion of said video camera in the time shutter according to said initial point spread function; D2. the point spread function that obtains the pixel of said de-blurred image according to motion and the said depth image of said video camera in the said time shutter; D3. the execution in step that circulates D1 until the operation of accomplishing each pixel in the said de-blurred image, obtains initial video camera intraframe motion to step D2.
In an embodiment of the present invention, said step e further comprises: E1. sets up the Bayesian probability model according to the gradient of said de-blurred image and the prior distribution of noise; E2. adopt belief propagation algorithm that de-blurred image is optimized; E3. adopt the Levenberg-Marquard algorithm that said intraframe motion is optimized; E4. the execution in step that circulates E2 and step e 3 pre-determined numbers are to obtain final camera motion model and final de-blurred image.
In an embodiment of the present invention, saidly according to the gaussian filtering equation said de-blurred image is carried out filtering, wherein, said gaussian filtering equation is:
Wherein, w is said denoising image, and x, y are the coordinate of said denoising image, and A is the normalization coefficient of said gaussian filtering equation, and σ is the standard deviation of said gaussian filtering equation.
In an embodiment of the present invention, said Bayesian probability model is:
p(I
l，E(t)，D|I
b)∝p(I
b|I
l，E(t)，D)p(E(t)|I
l，D)p(I
l，D)，
Wherein, I
lBe said final de-blurred image, E (t) is the t intraframe motion parameter of said video camera constantly, and D is the depth information of said depth image, I
bBe said blurred picture.
The present invention proposes a kind of shake video image deblurring device based on camera self-calibration on the other hand, and comprising: computing module is used for initial point spread function and de-blurred image according to blind deconvolution algorithm computation blurred picture; Demarcating module is used for said de-blurred image is carried out from demarcating to obtain the inside and outside parameter of video camera; The depth image generation module is used for the depth image according to said de-blurred image and the said de-blurred image of the inside and outside calculation of parameter of said video camera; The intraframe motion generation module is used for estimating according to said initial point spread function and said depth image the intraframe motion of said video camera; And optimal module, be used for optimizing said intraframe motion and said de-blurred image, to obtain final intraframe motion and final de-blurred image according to probability model.
Aspect that the present invention adds and advantage part in the following description provide, and part will become obviously from the following description, or recognize through practice of the present invention.
Description of drawings
Above-mentioned and/or additional aspect of the present invention and advantage are from obviously with easily understanding becoming the description of embodiment below in conjunction with accompanying drawing, wherein:
Fig. 1 is the process flow diagram based on the shake video deblurring method of camera self-calibration of the embodiment of the invention; And
Fig. 2 is the structural drawing based on the shake video deblurring device of camera self-calibration of the embodiment of the invention.
Embodiment
Describe whole embodiment of the present invention below in detail, the example of said embodiment is shown in the drawings, and wherein identical from start to finish or similar label is represented identical or similar elements or the element with identical or similar functions.Be exemplary through the embodiment that is described with reference to the drawings below, only be used to explain the present invention, and can not be interpreted as limitation of the present invention.
The present invention be directed to a kind of shake video deblurring method and device that existing method proposes deblurring poor effect with big depth range image based on camera self-calibration.Understand for the method to the embodiment of the invention has more clearly, below just combine accompanying drawing that the workflow and the principle of work of the method and apparatus of the embodiment of the invention are done detailed description.
As shown in Figure 1, be the process flow diagram based on the shake video deblurring method of camera self-calibration of the embodiment of the invention.In specific embodiment of the present invention, this method may further comprise the steps:
Step S101, A. is according to the initial point spread function and the de-blurred image of blind deconvolution algorithm computation blurred picture.
Particularly, in one embodiment of the invention, at first this blurred picture is carried out down-sampling, purpose is to reduce the blur level of this blurred picture, and for next using this image of blind deconvolution algorithm process to reduce difficulty.Then; After this image is carried out deblurring, adopt the Richardson-Lucy algorithm that this blurred picture is carried out the blind deconvolution computing, thereby obtain image and convolution kernel after the deblurring of this blurred picture; Here need to prove; When the utilization blind deconvolution was tried to achieve this convolution kernel, prerequisite was for to use this constraint condition of sparse constraint to convolution kernel, that is: each pixel on this blurred picture is to have a part of weighting summation in the corresponding picture rich in detail surrounding pixel to obtain.
Step S102, B. carries out from demarcating to obtain the inside and outside parameter of video camera said de-blurred image.
Particularly; In one embodiment of the invention, 1, at first, the image after using the gaussian filtering equation to this deblurring carries out filtering to reduce the influence of picture noise to the camera self-calibration method that next carries out; More particularly, this gaussian filtering equation is:
In this equation, w is said denoising image, and x, y are the coordinate of said denoising image, and A is the normalization coefficient of said gaussian filtering equation, and σ is the standard deviation of said gaussian filtering equation, through this equation this de-blurred image is carried out denoising.
2, then, adopt KLT feature point tracking algorithm to detect the unique point of this denoising image and follow the tracks of the unique point in this denoising image.
3, adopt the ransac algorithm that the fundamental matrix of this denoising image and its each two field picture of front and back is carried out Robust Estimation then; And with the unique point deletion that is classified as exterior point in the estimation procedure; Because these matched feature points do not satisfy that the fundamental matrix calculate describes to utmost point geometrical constraint, so be considered to error matching points.
4, subsequently; Robust features point matching relationship according to obtaining in 3 carries out projective reconstruction to this denoising image, restructuring procedure more detailed step be: at first confirm two width of cloth initialization views, set up world coordinate system; Rebuild according to the triangle projection relation then and obtain 3 corresponding dimension space coordinates of matched feature points; And circulation adds new picture, all joins in the projective reconstruction until all pictures, thereby image scene has been carried out reconstruct.
5, adopt linear calibration's algorithm directly to calibrate the transformation matrix that projective space is rebuild to tolerance.With absolute antithesis quadric surface Ω
*Be decomposed into SS
T, wherein S is 4 * 3 companion matrix, can will be converted into the linear restriction to S to the linear restriction of confidential reference items matrix K according to K=PS like this, can prove SS in theory
TThe Ω that obtains
*Have orthotropicity, and its order is 3.
6, S is mended rows of vectors, the 4 rank square formations that obtain are reversible, and this square formation contrary is exactly can this projection be rebuild a transformation matrix that tolerance is rebuild of conversion.
7, the tolerance that conversion is obtained is rebuild the video camera projection matrix under the meaning, carries out RQ and decomposes, and the confidential reference items matrix K that obtains video camera respectively is the rotation matrix R translation vector T of video camera and the essential matrix E=[t] that calculates video camera with taking each two field picture
*R (the whole attitude informations that wherein comprise video camera) accomplishes camera self-calibration.In a preferred embodiment of the invention, the intrinsic parameter of said video camera is 5, and outer parameter is 6.
Step S103, C. is according to the depth image of said de-blurred image and the said de-blurred image of the inside and outside calculation of parameter of said video camera.
Particularly, in one embodiment of the invention, at first adopt adjacent front and back two two field pictures of belief propagation algorithm and this de-blurred image to calculate the ID figure that changes de-blurred image.Then adopt the average drifting algorithm that this de-blurred image is carried out color and cut apart, and the pairing ID figure of this de-blurred image is carried out match according to carve information.At last, have consistance, in specific embodiment of the present invention, adopt bundle collection adjustment algorithm that the depth image of all de-blurred image is carried out combined optimization in order to ensure the degree of depth of each two field picture.
Step S104, D. estimate the intraframe motion of said video camera according to said initial point spread function and said depth image.
Particularly, in one embodiment of the invention, this step S104 comprises:
Step 1 is estimated the motion of this video camera in the time shutter according to the initial global point spread function that obtains among the step S101;
Step 2; The depth image that obtains among motion in the time shutter that in step 1, obtains according to this video camera and the step S103 obtains the point spread function of each pixel of corresponding de-blurred image; Compare with initial global point spread function, the point spread function of each pixel is more accurate.
Step 3, circulation execution in step 1 until the operation of accomplishing each pixel in this de-blurred image, obtains the motion of initial video camera in this frame to step 2.
More particularly, can regard on the de-blurred image corresponding point of each pixel in three-dimensional scenic on the point spread function as along with the movement locus of motion imaging on the CCD imaging plane of video camera in the time shutter.Can be designated as x '=PSF (x, d, t); Wherein, x representes this pixel coordinate on de-blurred image, and d representes this and puts the corresponding degree of depth; T express time, x ' be illustrated in t constantly on the de-blurred image pixel coordinate be x, the degree of depth is the pixel coordinate of d imaging on the video camera imaging plane.According to the video camera projection model, can try to achieve corresponding video camera at t moment kinematic parameter by the initial point spread function with method of addition.Wherein, in a preferred embodiment of the invention, this projection model is the pinhole imaging system model.Certainly; Those skilled in the art will know that; This pinhole imaging system model only as concrete an application of one embodiment of the invention, also can use other projection model, as: the quadrature imaging model; Affine projection models etc., these conversion based on inventive concept all should be classified protection scope of the present invention as with changing.
At first, on de-blurred image, choose on every side the unique point of texture-rich (can select the bigger point of SIFT proper vector mould value) point as a reference, in a preferred embodiment of the invention, the number of RP is more than or equal to 9.
Then, according to before the initial point spread function that obtains, suppose that t coordinate constantly is x ', make Δ x=x '-x, can try to achieve the increment Delta E of this moment video camera video camera attitude E corresponding according to following equation with respect to de-blurred image:
x′
TΔEx＝-Δx
TEx，
Adopt the RANSAC algorithm to utilize following formula can try to achieve the increment Delta E of video camera in t relative movement parameters constantly.
At last, adopt the SVD decomposition algorithm, video camera is obtained rotation matrix R and translation vector T in t actual motion parameter E '=E+ Δ E decomposition constantly.
Step S 105, and E. optimizes said intraframe motion and said de-blurred image according to probability model, to obtain final intraframe motion and final de-blurred image.Particularly, in one embodiment of the invention, step S105 may further comprise the steps:
Step 1 is set up the Bayesian probability model according to the gradient of this de-blurred image and the prior distribution of noise;
Step 2 adopts belief propagation algorithm that this de-blurred image is optimized according to the Bayesian probability model that obtains in the step 1;
Step 3 adopts the Levenberg-Marquard algorithm that this intraframe motion is optimized according to said Bayesian probability model;
Step 4, circulation execution in step 2 and step 3 pre-determined number, thus obtain final camera motion model and final de-blurred image.
More particularly, set up the Bayesian probability model according to the gradient of this de-blurred image and the prior distribution of noise.Wherein, the representation of this Bayesian probability model is shown below: p (I
l, E (t), D|I
b) ∝ p (I
b| I
l, E (t), D) p (E (t) | I
l, D) p (I
l, D) 1)
In this Bayesian probability model, I
lBe said final de-blurred image, E (t) is the t intraframe motion parameter of said video camera constantly, and D is the depth information of said depth image, I
bBe said blurred picture.1) levoform (p (I in the formula
l, E (t), D|I
b)) be illustrated in given blurred picture I
bCondition under picture rich in detail to be asked, video camera intraframe motion and scene depth probability distribution, according to the maximum a posteriori criterion, require this 1) formula obtains maximization, calculate for ease, to 1) formula gets negative logarithm.Right formula (p (I
b| I
l, E (t), D) p (E (t) | I
l, D) p (I
l, D)) in first p (I
b| I
l, E (t) D) is defined as:
In the following formula
Wherein,
Because the motion E (t) and the I of video camera
l, D is separate, thus second p of following formula right-hand member (E (t) | I
l, D) become p (E (t)), can suppose even distribution in the reality, can ignore this influence.The 3rd p (I of following formula right-hand member
l, D) being degree of depth constraint, can decompose as follows:
-log(p(I
l，D))＝-log(p(D|I
l))-log(p(I
l))，
Wherein-log (p (D|I
l)) be the depth map energy function, can use the depth map derivation algorithm that it is optimized ,-log (p (I
l)) prior probability distribution of presentation video gradient, be that piecewise function has following form:
Wherein, k, a, b represent the parameter of prior distribution, can get k=0.5～5 for general pattern, a=1 * 10
-4～1 * 10
-3, b=1～10;
Adopt belief propagation algorithm that de-blurred image is optimized I
l
Adopt the Levenberg-Marquard algorithm that kinematic parameter between camera frame is optimized;
Two steps operation on the loop iteration, convergence obtains final camera motion model and de-blurred image.
Step S106, the F. circulation is carried out A and is finished dealing with until all two field pictures of said shake video to E, obtains the deblurring video of said shake video.
Another aspect of the present invention also proposes a kind of shake video deblurring device based on camera self-calibration, and is as shown in Figure 2, is the structural drawing based on the shake video deblurring device of camera self-calibration of the embodiment of the invention.In specific embodiment of the present invention, should comprise computing module 201, demarcating module 202, depth image generation module 203, intraframe motion generation module 204 and optimal module 205 based on the shake video deblurring device 200 of camera self-calibration.Wherein, Computing module 201 act as initial point spread function and de-blurred image according to blind deconvolution algorithm computation blurred picture; Acting as of demarcating module 202 carried out from demarcating to obtain the inside and outside parameter of video camera said de-blurred image; Depth image generation module 203 act as depth image according to said de-blurred image and the said de-blurred image of the inside and outside calculation of parameter of said video camera; Intraframe motion generation module 204 act as the intraframe motion of estimating said video camera according to said initial point spread function and said depth image; Said intraframe motion and said de-blurred image are optimized in acting as according to probability model of optimal module 205, to obtain final intraframe motion and final de-blurred image.
Particularly; In one embodiment of the invention; Computing module 201 at first carries out down-sampling to reduce the blur level of said blurred picture to said blurred picture, then according to the sparse constraint condition of convolution kernel, adopts the Richardson-Lucy algorithm that said blurred picture is carried out the blind deconvolution computing; To obtain said de-blurred image and convolution kernel; At last said convolution kernel is carried out up-sampling, and as initial value said blurred picture is carried out the blind deconvolution computing, with blurred picture after being optimized and convolution kernel with said convolution kernel.
In one embodiment of the invention; Demarcating module 202 at first carries out filtering according to the gaussian filtering equation to said de-blurred image; To obtain the denoising image; Detect the unique point of said denoising image then according to KLT feature point tracking algorithm, adopt the inside and outside parameter of demarcating said denoising image based on the quadric camera self-calibration algorithm of absolute antithesis at last.
In one embodiment of the invention; Depth image generation module 203 at first calculates the ID figure of said de-blurred image according to adjacent front and back two two field pictures of belief propagation algorithm and said de-blurred image; Adopting the average drifting algorithm that said de-blurred image is carried out color then cuts apart; Said ID figure is carried out match, adopt a bundle collection adjustment algorithm that said ID figure is optimized at last, to guarantee the consistance of the degree of depth according to carve information.
In one embodiment of the invention; Intraframe motion generation module 204 comprises step 1, estimates the motion of said video camera in the time shutter, step 2 according to said initial point spread function; The point spread function that obtains the pixel of said de-blurred image according to motion and the said depth image of said video camera in the said time shutter; Step 3, circulation execution in step 1 until the operation of accomplishing each pixel in the said de-blurred image, obtains initial video camera intraframe motion to step 2.
In one embodiment of the invention; Optimal module 205 comprises step 1, sets up Bayesian probability model, step 2 according to the gradient of said de-blurred image and the prior distribution of noise; Adopt belief propagation algorithm that de-blurred image is optimized according to said Bayesian probability model; Step 3 adopts the Levenberg-Marquard algorithm that said intraframe motion is optimized step 4 according to said Bayesian probability model; Circulation execution in step 2 and step 3 pre-determined number are to obtain final camera motion model and final de-blurred image.
Shake video deblurring method and device through embodiment of the invention proposition based on camera self-calibration; This method is through camera self-calibration technical calibration camera parameters; And then try to achieve depth image; And according to the three-dimensional coordinate of each pixel in the image and each pixel of motion calculation point spread function separately of video camera, thereby realized the relevant video image deblurring of the degree of depth, than compared the better blur effect that becomes according to the method deblurring of entire frame image overall point spread function in the past; And this method realizes simple, and this installs easy operating.
Although illustrated and described embodiments of the invention; For those of ordinary skill in the art; Be appreciated that under the situation that does not break away from principle of the present invention and spirit and can carry out multiple variation, modification, replacement and modification that scope of the present invention is accompanying claims and be equal to and limit to these embodiment.
Claims (14)
1. the shake video deblurring method based on camera self-calibration is characterized in that, may further comprise the steps:
A. according to the initial point spread function and the de-blurred image of blurred picture in the said shake video of blind deconvolution algorithm computation;
B. said de-blurred image is carried out from demarcating to obtain the inside and outside parameter of video camera;
C. according to the depth image of said de-blurred image and the said de-blurred image of the inside and outside calculation of parameter of said video camera;
D. estimate the intraframe motion of said video camera according to said initial point spread function and said depth image;
E. optimize said intraframe motion and said de-blurred image according to probability model, to obtain final intraframe motion and final de-blurred image; And
F. circulation execution A finishes dealing with until all two field pictures of said shake video to E, obtains the deblurring video of said shake video.
2. the shake video deblurring method based on camera self-calibration as claimed in claim 1 is characterized in that said steps A further comprises:
Said blurred picture is carried out down-sampling to reduce the blur level of said blurred picture;
According to the sparse constraint condition of convolution kernel, adopt the Richardson-Lucy algorithm that said blurred picture is carried out the blind deconvolution computing, to obtain said de-blurred image and convolution kernel;
Said convolution kernel is carried out up-sampling, and as initial value said blurred picture is carried out the blind deconvolution computing, with blurred picture after being optimized and convolution kernel with said convolution kernel.
3. the shake video deblurring method based on camera self-calibration as claimed in claim 1 is characterized in that said step B further comprises:
According to the gaussian filtering equation said de-blurred image is carried out filtering, to obtain the denoising image;
Detect the unique point of said denoising image according to KLT feature point tracking algorithm;
The inside and outside parameter of said video camera is demarcated in employing based on the quadric camera self-calibration algorithm of absolute antithesis.
4. the shake video deblurring method based on camera self-calibration as claimed in claim 1 is characterized in that said step C further comprises:
Calculate the ID figure of said de-blurred image according to adjacent front and back two two field pictures of belief propagation algorithm and said de-blurred image;
Adopt the average drifting algorithm that said de-blurred image is carried out color and cut apart, said ID figure is carried out match according to carve information;
Adopt a bundle collection adjustment algorithm that said ID figure is optimized.
5. the shake video deblurring method based on camera self-calibration as claimed in claim 1 is characterized in that said step D further comprises:
D1. estimate the motion of said video camera in the time shutter according to said initial point spread function;
D2. the point spread function that obtains the pixel of said de-blurred image according to motion and the said depth image of said video camera in the said time shutter;
D3. the execution in step that circulates D1 until the operation of accomplishing each pixel in the said de-blurred image, obtains initial video camera intraframe motion to step D2.
6. the shake video deblurring method based on camera self-calibration as claimed in claim 1 is characterized in that said step e further comprises:
E1. set up the Bayesian probability model according to the gradient of said de-blurred image and the prior distribution of noise;
E2. adopt belief propagation algorithm that de-blurred image is optimized according to said Bayesian probability model;
E3. adopt the Levenberg-Marquardt algorithm that said intraframe motion is optimized according to said Bayesian probability model;
E4. the execution in step that circulates E2 and step e 3 pre-determined numbers are to obtain final camera motion model and final de-blurred image.
7. the shake video deblurring method based on camera self-calibration as claimed in claim 3 is characterized in that, saidly according to the gaussian filtering equation said de-blurred image is carried out filtering, and wherein, said gaussian filtering equation is:
Wherein, w is said denoising image, and x, y are the coordinate of said denoising image, and A is the normalization coefficient of said gaussian filtering equation, and σ is the standard deviation of said gaussian filtering equation.
8. the shake video deblurring method based on camera self-calibration as claimed in claim 6 is characterized in that, said Bayesian probability model is:
p(I
l，E(t)，D|I
b)∝p(I
b|I
l，E(t)，D)p(E(t)|I
l，D)p(I
l，D)，
Wherein, I
lBe said final de-blurred image, E (t) is the t intraframe motion parameter of said video camera constantly, and D is the depth information of said depth image, I
bBe said blurred picture.
9. the shake video deblurring device based on camera self-calibration is characterized in that, comprising:
Computing module is used for initial point spread function and de-blurred image according to the said shake video of blind deconvolution algorithm computation blurred picture;
Demarcating module is used for said de-blurred image is carried out from demarcating to obtain the inside and outside parameter of video camera;
The depth image generation module is used for the depth image according to said de-blurred image and the said de-blurred image of the inside and outside calculation of parameter of said video camera;
The intraframe motion generation module is used for estimating according to said initial point spread function and said depth image the intraframe motion of said video camera; And
Optimal module is used for optimizing said intraframe motion and said de-blurred image according to probability model, to obtain final intraframe motion and final de-blurred image.
10. the shake video deblurring device based on camera self-calibration as claimed in claim 9; It is characterized in that; Said computing module carries out down-sampling reducing the blur level of said blurred picture to said blurred picture, and according to the sparse constraint condition of convolution kernel, adopts the Richardson-Lucy algorithm that said blurred picture is carried out the blind deconvolution computing; To obtain said de-blurred image and convolution kernel; And said convolution kernel carried out up-sampling, and as initial value said blurred picture is carried out the blind deconvolution computing with said convolution kernel, with blurred picture after being optimized and convolution kernel.
11. the shake video deblurring device based on camera self-calibration as claimed in claim 9; It is characterized in that; Said demarcating module carries out filtering according to the gaussian filtering equation to said de-blurred image; Obtaining the denoising image, and detect the unique point of said denoising image, and adopt the inside and outside parameter of demarcating said video camera based on the quadric camera self-calibration algorithm of absolute antithesis according to KLT feature point tracking algorithm.
12. the shake video deblurring device based on camera self-calibration as claimed in claim 9; It is characterized in that; Said depth image generation module calculates the ID figure of said de-blurred image according to adjacent front and back two two field pictures of belief propagation algorithm and said de-blurred image; And adopt the average drifting algorithm that said de-blurred image is carried out color to cut apart; Said ID figure being carried out match, and adopt a bundle collection adjustment algorithm that said ID figure is optimized, to guarantee the consistance of the degree of depth according to carve information.
13. the shake video deblurring device based on camera self-calibration as claimed in claim 9; It is characterized in that; Said intraframe motion generation module is estimated the motion of said video camera in the time shutter according to said initial point spread function, and the point spread function that obtains the pixel of said de-blurred image according to motion and the said depth image of said video camera in the said time shutter; And the circulation execution obtains initial video camera intraframe motion until the operation of accomplishing each pixel in the said de-blurred image.
14. the shake video deblurring device based on camera self-calibration as claimed in claim 9 is characterized in that the execution of said optimal module comprises the steps:
Step 1 is set up the Bayesian probability model according to the gradient of said de-blurred image and the prior distribution of noise;
Step 2 adopts belief propagation algorithm that de-blurred image is optimized according to said Bayesian probability model;
Step 3 adopts the Levenberg-Marquardt algorithm that said intraframe motion is optimized according to said Bayesian probability model;
Step 4, circulation execution in step 2 and step 3 pre-determined number are to obtain final camera motion model and final de-blurred image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010106126772A CN102073993B (en) | 2010-12-29 | 2010-12-29 | Camera self-calibration-based jittering video deblurring method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2010106126772A CN102073993B (en) | 2010-12-29 | 2010-12-29 | Camera self-calibration-based jittering video deblurring method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102073993A CN102073993A (en) | 2011-05-25 |
| CN102073993B true CN102073993B (en) | 2012-08-22 |
Family
ID=44032523
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2010106126772A Active CN102073993B (en) | 2010-12-29 | 2010-12-29 | Camera self-calibration-based jittering video deblurring method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102073993B (en) |
Families Citing this family (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102436639B (en) * | 2011-09-02 | 2013-12-04 | 清华大学 | Image acquiring method for removing image blurring and image acquiring system |
| CN102509294B (en) * | 2011-11-08 | 2013-09-25 | 清华大学深圳研究生院 | Single-image-based global depth estimation method |
| CN102609977B (en) * | 2012-01-12 | 2014-07-16 | 浙江大学 | Depth integration and curved-surface evolution based multi-viewpoint three-dimensional reconstruction method |
| CN102663718B (en) * | 2012-03-19 | 2015-06-24 | 清华大学 | Method and system for deblurring of gloablly inconsistent image |
| CN103426144B (en) * | 2012-05-17 | 2016-05-11 | 佳能株式会社 | For making the method and apparatus of the image deblurring with perspective distortion |
| CN102750679B (en) * | 2012-06-28 | 2014-07-09 | 西安电子科技大学 | Blind deblurring method for image quality evaluation |
| CN103413277B (en) * | 2013-08-19 | 2017-04-05 | 南京邮电大学 | Blind camera shake deblurring method based on L0 sparse priors |
| CN104932868B (en) * | 2014-03-17 | 2019-01-15 | 联想(北京)有限公司 | A kind of data processing method and electronic equipment |
| CN104954631B (en) * | 2014-03-25 | 2018-02-27 | 腾讯科技（深圳）有限公司 | A kind of method for processing video frequency, device and system |
| CN104867111B (en) * | 2015-03-27 | 2017-08-25 | 北京理工大学 | A kind of blind deblurring method of non-homogeneous video based on piecemeal fuzzy core collection |
| CN106683174B (en) * | 2016-12-23 | 2020-10-27 | 南京拓恒无人系统研究院有限公司 | Three-dimensional reconstruction method and device of binocular vision system and binocular vision system |
| CN106920229B (en) * | 2017-01-22 | 2021-01-05 | 北京奇艺世纪科技有限公司 | Automatic detection method and system for image fuzzy area |
| CN107369135A (en) * | 2017-06-22 | 2017-11-21 | 广西大学 | A kind of micro imaging system three-dimensional point spread function space size choosing method based on Scale invariant features transform algorithm |
| CN107590781B (en) * | 2017-08-17 | 2020-11-27 | 天津大学 | Self-adaptive weighted TGV image deblurring method based on original dual algorithm |
| CN109151281A (en) * | 2018-09-26 | 2019-01-04 | 中国计量大学 | A kind of pixel aperture offset camera obtaining depth information |
| CN113992848A (en) * | 2019-04-22 | 2022-01-28 | 深圳市商汤科技有限公司 | Video image processing method and device |
| US11127119B1 (en) * | 2020-03-17 | 2021-09-21 | GM Global Technology Operations LLC | Systems and methods for image deblurring in a vehicle |
| CN111445404A (en) * | 2020-03-23 | 2020-07-24 | 上海数迹智能科技有限公司 | Phase deblurring method based on dual-frequency sum probability model |
| CN113077395B (en) * | 2021-03-26 | 2023-10-24 | 东北大学 | Deblurring method for large-size sample image under high-power optical microscope |
| CN113222863B (en) * | 2021-06-04 | 2024-04-16 | 中国铁道科学研究院集团有限公司 | Video self-adaptive deblurring method and device based on high-speed railway operation environment |
| CN115396644B (en) * | 2022-07-21 | 2023-09-15 | 贝壳找房(北京)科技有限公司 | Video fusion method and device based on multi-section external reference data |
| CN115047890B (en) * | 2022-08-17 | 2022-11-01 | 北京大学 | Unmanned ship control method, unmanned ship control device and computer-readable storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP3251127B2 (en) * | 1993-08-05 | 2002-01-28 | ソニー・ユナイテッド・キングダム・リミテッド | Video data processing method |
| US7379612B2 (en) * | 2004-12-16 | 2008-05-27 | The Regents Of The University Of California, Santa Cruz | Dynamic reconstruction of high-resolution video from color-filtered low-resolution video-to-video super-resolution |
| CN101406041A (en) * | 2006-05-08 | 2009-04-08 | 三菱电机株式会社 | Method for reducing blur in an image of a scene and method for deblurring an image of a scene |
| CN101727663A (en) * | 2008-10-13 | 2010-06-09 | 索尼株式会社 | Method and system for image deblurring |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8064717B2 (en) * | 2005-10-28 | 2011-11-22 | Texas Instruments Incorporated | Digital camera and method |
-
2010
- 2010-12-29 CN CN2010106126772A patent/CN102073993B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP3251127B2 (en) * | 1993-08-05 | 2002-01-28 | ソニー・ユナイテッド・キングダム・リミテッド | Video data processing method |
| US7379612B2 (en) * | 2004-12-16 | 2008-05-27 | The Regents Of The University Of California, Santa Cruz | Dynamic reconstruction of high-resolution video from color-filtered low-resolution video-to-video super-resolution |
| CN101406041A (en) * | 2006-05-08 | 2009-04-08 | 三菱电机株式会社 | Method for reducing blur in an image of a scene and method for deblurring an image of a scene |
| CN101727663A (en) * | 2008-10-13 | 2010-06-09 | 索尼株式会社 | Method and system for image deblurring |
Non-Patent Citations (3)
| Title |
|---|
| Amit Agrawal等.Invertible Motion Blur in Video.《Appeared in ACM Transactions on Graphics》.2009,第28卷(第3期), * |
| JP特许第3251127号B2 2002.01.28 |
| 李沛秦 等.一种面向目标区域的快速去模糊算法.《信号处理》.2010,第26卷(第8期), * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102073993A (en) | 2011-05-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102073993B (en) | Camera self-calibration-based jittering video deblurring method and device | |
| EP2291999B1 (en) | Image deconvolution using color priors | |
| CN102436639B (en) | Image acquiring method for removing image blurring and image acquiring system | |
| US9412151B2 (en) | Image processing apparatus and image processing method | |
| Shan et al. | High-quality motion deblurring from a single image | |
| US10007990B2 (en) | Generating composite images using estimated blur kernel size | |
| CN112132958B (en) | Underwater environment three-dimensional reconstruction method based on binocular vision | |
| US20140354886A1 (en) | Device, system, and method of blind deblurring and blind super-resolution utilizing internal patch recurrence | |
| US9253415B2 (en) | Simulating tracking shots from image sequences | |
| CN111415310B (en) | Image processing method and device and storage medium | |
| US9342873B1 (en) | Tile-based optical flow | |
| KR101671391B1 (en) | Method for deblurring video using modeling blurred video with layers, recording medium and device for performing the method | |
| Hervieu et al. | Stereoscopic image inpainting: distinct depth maps and images inpainting | |
| US20140105515A1 (en) | Stabilizing and Deblurring Atmospheric Turbulence | |
| US20130016239A1 (en) | Method and apparatus for removing non-uniform motion blur using multi-frame | |
| CN103544681A (en) | Non-uniform motion blurred image restoration method | |
| JP5012967B2 (en) | Image processing apparatus and method, and program | |
| Singh et al. | Weighted least squares based detail enhanced exposure fusion | |
| US9860441B1 (en) | Tile-based digital image correspondence | |
| CN114640885B (en) | Video frame inserting method, training device and electronic equipment | |
| JP2009111921A (en) | Image processing device and image processing method | |
| CN103839233A (en) | Restoration method for blurred image caused by camera shaking | |
| US8712179B2 (en) | Reconstruction of deforming surfaces by canceling ambient occlusion and refining 3-D shape | |
| JP2006195790A (en) | Lens distortion estimation apparatus, lens distortion estimation method, and lens distortion estimation program | |
| US7742660B2 (en) | Scale-space self-similarity image processing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |