CN109491079A - Total focus imaging system based on rotary coding aperture - Google Patents
Total focus imaging system based on rotary coding aperture Download PDFInfo
- Publication number
- CN109491079A CN109491079A CN201811519373.4A CN201811519373A CN109491079A CN 109491079 A CN109491079 A CN 109491079A CN 201811519373 A CN201811519373 A CN 201811519373A CN 109491079 A CN109491079 A CN 109491079A
- Authority
- CN
- China
- Prior art keywords
- depth
- aperture
- rotary coding
- estimation
- coding aperture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 79
- 238000000034 method Methods 0.000 claims abstract description 57
- 230000008569 process Effects 0.000 claims abstract description 23
- 238000001228 spectrum Methods 0.000 claims abstract description 17
- 238000011156 evaluation Methods 0.000 claims description 34
- 238000005457 optimization Methods 0.000 claims description 34
- 238000011084 recovery Methods 0.000 claims description 22
- 230000002068 genetic effect Effects 0.000 claims description 18
- 230000004075 alteration Effects 0.000 claims description 9
- 230000003287 optical effect Effects 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000012512 characterization method Methods 0.000 claims description 5
- 238000013178 mathematical model Methods 0.000 claims description 5
- 230000000694 effects Effects 0.000 abstract description 5
- 210000003739 neck Anatomy 0.000 abstract description 3
- 230000004438 eyesight Effects 0.000 abstract description 2
- 230000007246 mechanism Effects 0.000 abstract description 2
- 230000006870 function Effects 0.000 description 58
- 239000011159 matrix material Substances 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 2
- 230000007812 deficiency Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000012804 iterative process Methods 0.000 description 2
- 238000012634 optical imaging Methods 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 239000000571 coke Substances 0.000 description 1
- 238000009792 diffusion process Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000035800 maturation Effects 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 230000004304 visual acuity Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/0012—Optical design, e.g. procedures, algorithms, optimisation routines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/12—Computing arrangements based on biological models using genetic models
- G06N3/126—Evolutionary algorithms, e.g. genetic algorithms or genetic programming
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30168—Image quality inspection
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Biophysics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Physiology (AREA)
- Data Mining & Analysis (AREA)
- Optics & Photonics (AREA)
- Genetics & Genomics (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Computational Linguistics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Processing (AREA)
Abstract
The invention belongs to computer vision technique necks, and in particular to a kind of total focus imaging system based on rotary coding aperture.Compared with prior art, technical characterstic and effect of the invention: 1) have odd encoder aperture system using different aperture be imaged when and need to switch over aperture, switching mechanism needs to accommodate the different aperture of two panels and is switched at high speed, the challenge beaten very much to design with integrated belt；And rotary coding aperture imaging system of the invention only needs a piece of coding aperture, switching is more simple and convenient, and it is more stable effectively to obtain image.2) mode of rotary coding aperture rotated three times makes the discrete performance of the frequency spectrum of aperture more preferable, therefore, becomes apparent from the discrete degree of the corresponding point spread function of each depth level.In the calculating process of estimating depth, the better discrete performance of frequency spectrum makes the performance of performance of the estimated result of depth in the accuracy of identification and the resolution ratio of depth more excellent.
Description
Technical field
The invention belongs to computer vision technique necks, and in particular to a kind of total focus imaging system based on rotary coding aperture
System.
Background technique
Requirement of the modern society for the image image quality of digital imaging device is higher and higher, and it is expected that obtain all clear clear
Image.However, optical imaging system all can not thoroughly solve the problems, such as defocus blur till now.Traditional optical imaging device
Basic imaging principle is that concave-convex lens changes direction to light, is designed like equipment with this characteristic.Therefore subject
The object plane of body has certain selectivity to the distance between object lens, if subject is flat in the coke for being located at imaging system
On face, then imaging plane obtains clearest picture；, whereas if shooting object has a certain distance far from focal plane therewith,
The image that so imaging plane obtains will thicken.Before introducing coding aperture technology, this method is solved the problems, such as mostly
It is to need using additional device, improves focusing system characteristic to reach higher focusing precision.
Color image can bring more fully image restoration information to be applied to such as image with the combination of corresponding depth map
It edits, in the fields such as image rendering and augmented reality based on depth.However, depth information is compared to the light that can use maturation
For learning the luminosity information that equipment obtains, it is difficult to obtain.Therefore, depth estimation method based on blurred picture out of focus due to
Requirement to optical device is lower and the characteristics such as robustness with higher and the neck such as be widely used in the depth recovery of image
In domain.
Aperture technology is encoded as a kind of emerging calculating and images technology, passes through the point of modulation imaging system in itself
Spread function come realize the depth estimation method based on digital camera with improve the depth of field of visual light imaging expand performance.Wherein,
In parameter, the i.e. shape of aperture in three of camera imaging, imaging focal length and camera lens are to realize to the distance of imaging sensor
The important indicator that depth information restores.Coding aperture technology mainly carries out coding reconstruct work to the lens aperture of camera, improves
The discrete information of depth included in its point spread function obtains accurate depth estimation result, and by the depth map of image
The recovery of information realization total focus clear image.In forefathers based in fuzzy estimation of Depth work out of focus, list encodes aperture,
Encode aperture to or colored aperture in, there are estimation of Depth to contradict with total focus image restoration, practical application complexity or
The problems such as color image information is distorted.
Summary of the invention
(1) technical problems to be solved
The technical problem to be solved by the present invention is how to provide a kind of rotary coding aperture mode, it is desirable that its depth obtained
It is accurate to spend information, precision is high, total focus restored image quality is promoted, meanwhile, the usage mode of imaging system is convenient, stability
It is high.
(2) technical solution
In order to overcome the deficiencies of the prior art, the present invention provides a kind of total focus imaging method based on rotary coding aperture,
This method proposes the Performance evaluation criterion of rotary coding aperture imaging system first, and uses genetic algorithm according to the evaluation criterion
And coordinate descent optimizes rotary coding aperture, then realizes estimation of Depth and total focus figure by the rotary coding aperture of optimization
As recovery operation.
Wherein, this method comprises:
Step 1: the Performance evaluation criterion of proposition rotary coding aperture imaging system is applied to estimation of Depth recovery and entirely
Focusedimage restores:
The step 1 comprises the following specific steps that:
(1) fuzzy imaging model out of focus:
Wherein, f represents the blurred picture out of focus that imaging system is shot, f0Represent original clear image, kdRepresent imaging
Point spread function of the system at different depth d, η~N (0, σ2) represent white Gaussian noise；
(2) imaging model out of focus of rotary coding aperture system three times:
Fi=F0·Ki d+Ni, i=1,2,3
Wherein, this model is the expression of model in a frequency domain in (1): FiIndicate that rotary coding aperture imaging system is shot
The blurred picture out of focus arrived, F0Indicate original image, Ki dIndicate the point spread function of different depth, NiIndicate white Gaussian noise, i
Then indicate number of revolutions, i.e., rotation can generate blurred pictures out of focus different three times three times.This model corresponds to rotary coding light
The mathematical model of resulting three blurred pictures out of focus of shooting of coil imaging system；
(3) solve the problems, such as maximum a posteriori probability (Maximum A Posteriori) to realize using energy function minimum value
Performance optimization to rotary coding aperture；
Wherein,The jamtosignal (Noise to Signal Ratios) of representative image priori, can be expressed as
σ2/ A, wherein A can be used 1/f rule, i.e. the average power spectra of multiple natural images indicates,
ξ indicates frequency.Mean to make the difference between ambiguous estimation image and original blurred picture minimum using energy function minimum value,
I.e. so that estimation procedure precision highest, depth recovery result are most accurate；
(4) estimation clear image is obtained using wiener warp area method；I.e. to a set estimating depthUse energy
Function first solves estimation clear imageExistUnder conditions of, it can obtain,
Wherein,It is the complex conjugate of K,Refer in set estimating depthLocate corresponding imaging system point spread function
Frequency domain representation；
(5) formula in step (4) and step (2) is brought into step (3), reconstructs energy function, collated simplification can obtain
One about estimating depthHeat-supplied function,
Wherein, d*It indicates to be located at the fuzzy core that imaging system corresponds to standard depth, d indicates each level of visible images
Locate the fuzzy core of depth；
(6) under conditions of noise level is σ, d is located to rotary coding aperture*The estimation of Depth at place carries out performance evaluation
It can obtain,
Wherein, D={ c1d*,c2d*,…,cld*The size of the corresponding point spread function fuzzy core of each depth level is represented,
{ciIt can be taken as { 0.1,0.15 ..., 1.5 }, step-length 0.05；
(7) Performance evaluation criterion that can obtain rotary coding aperture result above is normalized:
The energy function variation of the bigger characterization evaluation estimation of Depth of R value is more precipitous, so that estimation of Depth performance is for image
The robustness of noise or color texture is stronger；
The rotary coding aperture for designing and genetic algorithm optimization being used to obtain low resolution；It is improved and is revolved using coordinate descent
Turn the resolution ratio of coding aperture；
Estimation of Depth restores and total focus image restoration process, is to carry out depth to it by wiener deconvolution and aberration method
Degree restores and the work of total focus image restoration；
Step 2: designing and obtain using genetic algorithm optimization the rotary coding aperture of low resolution；
For resolution ratio is the aperture of N × N, possibility up to 2N×NKind, calculation amount is very big, therefore can not be direct
Process is in optimized selection using high-resolution；
Step 3: the resolution ratio of rotary coding aperture is improved using coordinate descent；
In order to reduce the influence of optical diffraction, continue the performance for encoding aperture, using coordinate descent to the performance of aperture
Re-optimization is carried out, optimization rotary coding aperture is obtained；
Step 4: estimation of Depth restores and total focus image restoration process；
After obtaining visible images using rotary coding aperture imaging system, by wiener deconvolution and aberration method to it
Carry out the work of depth recovery and total focus image restoration.
(3) beneficial effect
Compared with prior art, technical characterstic of the invention and effect:
1) have when odd encoder aperture system is imaged using different apertures and need to switch over aperture, switching mechanism
It needs to accommodate the different aperture of two panels and is switched at high speed, the challenge beaten very much to design with integrated belt；And it is of the invention
Rotary coding aperture imaging system only need a piece of coding aperture, switch more simple and convenient, it is more stable to obtain image
Effectively.
2) mode of rotary coding aperture rotated three times makes the discrete performance of the frequency spectrum of aperture more preferable, therefore, in each depth
The discrete degree of the corresponding point spread function of degree level becomes apparent from.In the calculating process of estimating depth, better frequency spectrum is discrete
Performance makes the performance of performance of the estimated result of depth in the accuracy of identification and the resolution ratio of depth more excellent.
3) on the basis of more accurate depth map is estimated in acquisition, total focus is restored using wiener warp area method and is clearly schemed
The effect of picture can be more preferably.Compared to the mode of previous coding aperture, the frequency spectrum complementation performance of rotary coding aperture imaging system
More excellent, the frequency coverage that frequency spectrum is imaged is bigger, and it is more that this retains the spectrum information of visible images, also implies that recovery
The effect of clear image is more preferable, improves the quality of restored image.
4) since restored method of the invention is simple, easy to accomplish, runing time is short, therefore, ambiguity solution can be restored
Performance, which is directly integrated into the hardware of digital camera, allows camera or other imaging devices directly to carry out estimation of Depth and figure
As the work restored, reduce dependence of this system to external equipment.
Detailed description of the invention
Fig. 1 is the result that the present invention optimizes rotary coding aperture using genetic algorithm and coordinate descent, wherein resolution ratio
For 11 × 11 aperture the result is that using genetic algorithm generate low resolution optimize aperture, be later coordinate descent improve light
Enclose resolution ratio as a result, 47 × 47 be rotary coding aperture used in the present invention；
Fig. 2 is the spectral characteristic of rotary coding aperture；
Fig. 3 is the result for carrying out estimation of Depth recovery to variant depth model using rotary coding aperture system；
Wherein, (a) is discrete stair, ladder, continuous inclined-plane, circular conical surface threedimensional model, the attachment of model surface half respectively
Complex texture (left side), half adhere to simple textures (right side).
It (b) is the corresponding original depth-map of above four models respectively.
(c) the estimating depth figure for corresponding respectively to four models of aperture pair is encoded.
(d) the estimating depth figure for corresponding respectively to four models of circular iris is rotated.
(e) the estimating depth figure for corresponding respectively to four models of rotary coding aperture.
Following table be encode aperture to, rotation circular iris, rotary coding aperture each estimating depth figure and archetype depth
Spend RMSE (root-mean-square error) statistical result between figure.
Fig. 4 is to be restored using rotary coding aperture system to the estimation of Depth of 3 dimension scenes and total focus clear image
As a result.
(1a) (1b) is the clear figure and depth map of the original scene of indoor and outdoor scene respectively.
(2a) (2b) is the total focus restored image and estimating depth figure of the coding aperture pair of corresponding scene respectively.
(3a) (3b) is the total focus restored image and estimating depth figure of the rotary coding aperture of corresponding scene respectively.
Following table is the RMSE (root-mean-square error) and estimating depth figure of total focus restored image and the clear figure of original scene
With the statistical result of the PSNR (Y-PSNR) of original scene depth map.
Specific embodiment
To keep the purpose of the present invention, content and advantage clearer, with reference to the accompanying drawings and examples, to of the invention
Specific embodiment is described in further detail.
In order to overcome the deficiencies of the prior art, the present invention provides a kind of total focus imaging method based on rotary coding aperture,
This method proposes the Performance evaluation criterion of rotary coding aperture imaging system first, and uses genetic algorithm according to the evaluation criterion
And coordinate descent optimizes rotary coding aperture, then realizes estimation of Depth and total focus figure by the rotary coding aperture of optimization
As recovery operation.
Wherein, this method comprises:
Step 1: the Performance evaluation criterion of proposition rotary coding aperture imaging system is applied to estimation of Depth recovery and entirely
Focusedimage restores:
The step 1 comprises the following specific steps that:
(1) fuzzy imaging model out of focus:
Wherein, f represents the blurred picture out of focus that imaging system is shot, f0Represent original clear image, kdRepresent imaging
Point spread function of the system at different depth d, η~N (0, σ2) represent white Gaussian noise；
(2) imaging model out of focus of rotary coding aperture system three times:
Fi=F0·Ki d+Ni, i=1,2,3
Wherein, this model is the expression of model in a frequency domain in (1): FiIndicate that rotary coding aperture imaging system is shot
The blurred picture out of focus arrived, F0Indicate original image, Ki dIndicate the point spread function of different depth, NiIndicate white Gaussian noise, i
Then indicate number of revolutions, i.e., rotation can generate blurred pictures out of focus different three times three times.This model corresponds to rotary coding light
The mathematical model of resulting three blurred pictures out of focus of shooting of coil imaging system；
(3) solve the problems, such as maximum a posteriori probability (Maximum A Posteriori) to realize using energy function minimum value
Performance optimization to rotary coding aperture；
Wherein,The jamtosignal (Noise to Signal Ratios) of representative image priori, can be expressed as
σ2/ A, wherein A can be used 1/f rule, i.e. the average power spectra of multiple natural images indicates,
ξ indicates frequency.Mean to make the difference between ambiguous estimation image and original blurred picture minimum using energy function minimum value,
I.e. so that estimation procedure precision highest, depth recovery result are most accurate；
(4) estimation clear image is obtained using wiener warp area method；I.e. to a set estimating depthUse energy
Function first solves estimation clear imageExistUnder conditions of, it can obtain,
Wherein,It is the complex conjugate of K,Refer in set estimating depthLocate corresponding imaging system point spread function
Frequency domain representation；
(5) formula in step (4) and step (2) is brought into step (3), reconstructs energy function, collated simplification can obtain
One about estimating depthHeat-supplied function,
Wherein, d*It indicates to be located at the fuzzy core that imaging system corresponds to standard depth, d indicates each level of visible images
Locate the fuzzy core of depth；
(6) under conditions of noise level is σ, d is located to rotary coding aperture*The estimation of Depth at place carries out performance evaluation
It can obtain,
Wherein, D={ c1d*,c2d*,…,cld*The size of the corresponding point spread function fuzzy core of each depth level is represented,
{ciIt can be taken as { 0.1,0.15 ..., 1.5 }, step-length 0.05；
(7) Performance evaluation criterion that can obtain rotary coding aperture result above is normalized:
The energy function variation of the bigger characterization evaluation estimation of Depth of R value is more precipitous, so that estimation of Depth performance is for image
The robustness of noise or color texture is stronger；
The rotary coding aperture for designing and genetic algorithm optimization being used to obtain low resolution；It is improved and is revolved using coordinate descent
Turn the resolution ratio of coding aperture；
Estimation of Depth restores and total focus image restoration process, is to carry out depth to it by wiener deconvolution and aberration method
Degree restores and the work of total focus image restoration；
Step 2: designing and obtain using genetic algorithm optimization the rotary coding aperture of low resolution；
For resolution ratio is the aperture of N × N, possibility up to 2N×NKind, calculation amount is very big, therefore can not be direct
Process is in optimized selection using high-resolution；
Step 3: the resolution ratio of rotary coding aperture is improved using coordinate descent；
In order to reduce the influence of optical diffraction, continue the performance for encoding aperture, using coordinate descent to the performance of aperture
Re-optimization is carried out, optimization rotary coding aperture is obtained；
Step 4: estimation of Depth restores and total focus image restoration process；
After obtaining visible images using rotary coding aperture imaging system, by wiener deconvolution and aberration method to it
Carry out the work of depth recovery and total focus image restoration.
Embodiment 1
The present embodiment adopts the technical scheme that, the depth estimation method based on rotary coding aperture imaging system.It proposes
The Performance evaluation criterion of rotary coding aperture imaging system, and it is excellent using genetic algorithm and coordinate descent according to the evaluation criterion
Change rotary coding aperture, after by the rotary coding aperture system of optimization realize that estimation of Depth and total focus image restoration work.
When rotary coding aperture imaging system carries out estimation of Depth to visible images, need by the corresponding point diffusion of each depth level
Function, the i.e. otherness of point spread function at different depth are the main indicator parameters of resolution depth.Therefore, realize that depth is quasi-
The critical issue really estimated is that the point spread function of rotary coding aperture imaging system has outstanding discrete performance, corresponds to
In frequency spectrum, it is meant that the zero crossing that the frequency spectrum of rotary coding aperture needs to have more.The corresponding point spread function of different depth
Frequency spectrum zero crossing having different positions, when these zero crossings are discrete as much as possible, the estimation of depth information will be got over
Add precisely.Include the following steps:
1) propose that the Performance evaluation criterion of rotary coding aperture imaging system is applied to estimation of Depth recovery and total focus
Image restoration:
(1) fuzzy imaging model out of focus:
Wherein, f represents the blurred picture out of focus that imaging system is shot, f0Represent original clear image, kdRepresent imaging
Point spread function of the system at different depth d, η~N (0, σ2) represent white Gaussian noise.
(2) imaging model out of focus of rotary coding aperture system three times:
Fi=F0·Ki d+Ni, i=1,2,3
Wherein, this model is the expression of model in a frequency domain in (1).FiIndicate that rotary coding aperture imaging system is shot
The blurred picture out of focus arrived, F0Indicate original image, Ki dIndicate the point spread function of different depth, NiIndicate white Gaussian noise, i
Then indicate number of revolutions, i.e., rotation can generate blurred pictures out of focus different three times three times.This model corresponds to rotary coding light
The mathematical model of resulting three blurred pictures out of focus of shooting of coil imaging system.
(3) solve the problems, such as maximum a posteriori probability (Maximum A Posteriori) to realize using energy function minimum value
Performance optimization to rotary coding aperture.
Wherein,The jamtosignal (Noise to Signal Ratios) of representative image priori, can be expressed as
σ2/ A, wherein A can be used 1/f rule, i.e. the average power spectra of multiple natural images indicates,
ξ indicates frequency.Mean to make the difference between ambiguous estimation image and original blurred picture minimum using energy function minimum value,
I.e. so that estimation procedure precision highest, depth recovery result are most accurate.
(4) estimation clear image is obtained using wiener warp area method.I.e. to a set estimating depthUse energy
Function first solves estimation clear imageExistUnder conditions of, it can obtain,
Wherein,It is the complex conjugate of K,Refer in set estimating depthLocate corresponding imaging system point spread function
Frequency domain representation.
(5) formula in step (4) and step (2) is brought into step (3), reconstructs simultaneously readjusting and simplifying energy function, it was demonstrated that
Process is as follows:
(51) in given rotary coding aperture K1,K2,K3, standard depth obscures size d*And noise σ (is in frequency domain
N under conditions of), the energy function E for obscuring d corresponding to estimating depth is writeable are as follows:
(52) by the F in step (2)iIn step (4)It brings into the energy function of (51), can obtain:
(53) under the conditions of thinking that picture signal and noise signal are independent from each other, the formula in (52) can be arranged
Are as follows:
(54) for convenient for resolution, orderDue to NiIt is independent from each other white Gaussian
Noise signal N (0, σ2), therefore, E (Ni)=0, var (Ni)=σ2, E (NiNj)=0.Then energy function may be expressed as:
(55) according to 1/f rule, F is defined0Power spectrum be desired for A, meanwhile,C=σ2/
A.Therefore, energy function can arrange are as follows:
(56) collated, final energy formula identity result can be obtained:
Wherein, d*It indicates to be located at the fuzzy core that imaging system corresponds to standard depth, d indicates each level of visible images
Locate the fuzzy core of depth.
(6) under conditions of noise level is σ, d is located to rotary coding aperture*The estimation of Depth at place carries out performance evaluation
It can obtain,
Wherein, D={ c1d*,c2d*,…,cld*The size of the corresponding point spread function fuzzy core of each depth level is represented,
{ciIt can be taken as { 0.1,0.15 ..., 1.5 }, step-length 0.05.
(7) Performance evaluation criterion that can obtain rotary coding aperture result above is normalized:
The energy function variation of the bigger characterization evaluation estimation of Depth of R value is more precipitous, so that estimation of Depth performance is for image
The robustness of noise or color texture is stronger.
2) design and obtain using genetic algorithm optimization the rotary coding aperture of low resolution.It is N × N's for resolution ratio
For aperture, possibility up to 2N×NKind, calculation amount is very big, therefore can not be directly in optimized selection using high-resolution
Journey.
3) resolution ratio of rotary coding aperture is improved using coordinate descent.In order to reduce the influence of optical diffraction, continue
The performance for encoding aperture carries out re-optimization using performance of the coordinate descent to aperture, obtains optimization rotary coding aperture.
4) estimation of Depth is restored and total focus image restoration process.Visible light is obtained using rotary coding aperture imaging system
After image, the present invention carries out the work of depth recovery and total focus image restoration by wiener deconvolution and aberration method to it.
Embodiment 2
In the present embodiment,
1) propose that the Performance evaluation criterion of rotary coding aperture imaging system is applied to estimation of Depth recovery and total focus
Image restoration:
(1) fuzzy imaging model out of focus:
Wherein, f represents the blurred picture out of focus that imaging system is shot, f0Represent original clear image, kdRepresent imaging
Point spread function of the system at different depth d, η~N (0, σ2) white Gaussian noise is represented, N indicates Gaussian Profile, σ table herein
Show the standard deviation of Gaussian noise.
(2) imaging model out of focus of rotary coding aperture system three times:
Fi=F0·Ki d+Ni, i=1,2,3
Wherein, this model is the expression of model in a frequency domain in (1).FiIndicate that rotary coding aperture imaging system is shot
The blurred picture out of focus arrived, F0Indicate original image, Ki dIndicate the point spread function at different depth d, NiIndicate Gauss white noise
Sound, i then indicate number of revolutions, i.e., rotation can generate blurred pictures out of focus different three times three times.This model corresponds to rotation and compiles
The mathematical model of resulting three blurred pictures out of focus of shooting of code aperture imaging system.
(3) using energy function E minimum value solve the problems, such as maximum a posteriori probability (Maximum A Posteriori) with
It realizes and the performance of rotary coding aperture is optimized.
Wherein,The jamtosignal (Noise to Signal Ratios) of representative image priori, can be expressed as
σ2/ A, wherein A can be used 1/f rule, i.e. the average power spectra of multiple natural images indicates,
ξ indicates frequency, μ (F0) indicate the natural image integrated.Min expression is minimized, and ∑ indicates adduction, | | * | | indicate modulus.Make
Mean to make the difference between ambiguous estimation image and original blurred picture minimum with energy function minimum value, i.e., so that estimating
Cheng Jingdu highest, depth recovery result are most accurate.
(4) estimation clear image is obtained using wiener warp area method.I.e. to a set estimating depthUse energy
Function first solves estimation clear imageExistUnder conditions of, it can obtain,
Wherein,It is the complex conjugate of K,Refer in set estimating depthLocate corresponding imaging system point spread function
Frequency domain representation.
(5) formula in step (4) and step (2) is brought into step (3), reconstructs simultaneously readjusting and simplifying energy function, it was demonstrated that
Process is as follows:
(51) in given rotary coding aperture K1,K2,K3, standard depth obscures size d*And noise σ (is in frequency domain
N under conditions of), the energy function E for obscuring d corresponding to estimating depth is writeable are as follows:
Wherein, N1,N2,N3It refers to corresponding respectively to the visible images that rotary coding aperture system shoots acquisition three times
Frequency domain in white Gaussian noise.
(52) by the F in step (2)iIn step (4)It brings into the energy function of (51), can obtain:
(53) under the conditions of thinking that picture signal and noise signal are independent from each other, the formula in (52) can be arranged
Are as follows:
(54) for convenient for resolution, orderDue to NiIt is independent from each other white Gaussian
Noise signal N (0, σ2), therefore, E (Ni)=0, var (Ni)=σ2, E (NiNj)=0.Herein, var means to seek variance, then can
Flow function may be expressed as:
(55) according to 1/f rule, F is defined0Power spectrum be desired for A, meanwhile,C=σ2/
A.Therefore, energy function can arrange are as follows:
(56) collated, final energy formula identity result can be obtained:
Wherein, d*It indicates to be located at the fuzzy core that imaging system corresponds to standard depth, d indicates each level of visible images
Locate the fuzzy core of depth,It indicates in a frequency domain, positioned at the point spread function of # depth when corresponding to the * times rotation.
(6) under conditions of noise level is σ, d is located to rotary coding aperture*The estimation of Depth at place carries out performance evaluation
It can obtain,
Wherein, D={ c1d*,c2d*,…,cld*The size of the corresponding point spread function fuzzy core of each depth level is represented,
{ciIt is one group of depth level parameter, it can be taken as { 0.1,0.15 ..., 1.5 }, step-length 0.05, d ∈ D/d*It indicates not including mark
Quasi- Depth Blur core d*Depth set.
(7) result above is normalized and simplifies the Performance evaluation criterion that can obtain rotary coding aperture:
The energy function variation of the bigger characterization evaluation estimation of Depth of R value is more precipitous, so that estimation of Depth performance is for image
The robustness of noise or color texture is stronger.
2) design and obtain using genetic algorithm optimization the rotary coding aperture of low resolution.It is N × N's for resolution ratio
For aperture, possibility up to 2N×NKind, calculation amount is very big, therefore is not available high-resolution and process is in optimized selection.
But meanwhile high-resolution is the key that guarantee aperture not by optical diffraction jamming performance.To solve this problem, the present invention takes
First entire optimization process is carried out using the mode for promoting resolution ratio to higher standard after low resolution.Genetic algorithm optimization step is such as
Under:
(1) algorithm relevant parameter input initialization:
D={ c1d*,c2d*,…,cld*, indicate the depth layer grade resolving power performance parameter of rotary coding aperture system, d*
=7；
Indicate that rotary coding aperture rotates the initial value of angle three times；
G=0, g indicate the number of iterations of genetic algorithm, and primary iteration number is 0；
The set of S=4000 initial random binary sequences (only including 0,1) is generated, the length of each sequence is L=
N*N, N=11；
(2) iterative process (carries out loop iteration process, G=60 from g=1:G；)
(21) selection (the optimum option process in genetic algorithm simulates the natural selection process of biological heredity process):
For each binary sequence b, the aperture matrix k for being combined into that a size is N × N is reformed；Then by matrix k
According to rotation angle, θiIt is the aperture matrix k that rotation obtains corresponding to each rotation anglei(i=1,2,3).Then, using rotary coding
Aperture Performance evaluation criterion is calculated, and selects Z=400 wherein optimal result as the optimal of this time iteration result
Solution set.Finally, carrying out rotary coding aperture performance evaluation mark using rotation orientation optimization angle set H to optimal solution set Z
Quasi- calculates again, and the rotation angle for choosing optimal rotation angle set as next iteration inputs θi.Wherein, the composition of H
Rotation angle element composition is as follows: fixing first rotation angle is θ1=0 °；From 0 ° to 355 °, every 5 ° of intervals take an angle
One angle set J of value composition；θ2,θ3Unduplicated takes two angle values from angle set J；Then, by various θ1,θ2,
θ3Angle combinations value alternatively angle element be added rotation orientation optimization angle set H in.
(22) the following calculating process for repeating this step, until the quantity of arrangement set is restored to S from Z:
Intersect:
From arbitrarily choosing in the optimal set Z in step (21) and two sequences are replicated, then, by two sequence step-by-steps
After alignment, with Probability p1=0.2 carries out the exchange of place value, and obtains two new subsequences.
Variation:
For two new sequences that previous step obtains, with Probability p2The value of each of=0.05 pair of each new sequence into
Row overturning.
(3) the performance evaluation value of all sequences remained is calculated, and obtains the optimal result of coding aperture.
By the iteration optimization of genetic algorithm, shown in 11 × 11 aperture of rotary coding aperture optimum results as shown in figure 1,
Its corresponding optimization rotates angle result are as follows: θi={ 0 °, 110 °, 235 ° }, i=1,2,3
3) resolution ratio of rotary coding aperture is improved using coordinate descent.In order to reduce the influence of optical diffraction, and after
Sequel the performance of yard aperture, carry out re-optimization using performance of the coordinate descent to aperture, after obtain optimization rotary coding aperture.
Algorithm steps are as follows:
(1) rotary coding aperture is gradually successively improved from low resolution to high score first using bicubic interpolation mode
Resolution (such as the first suboptimization improves resolution ratio to 13 × 13 from 11 × 11)
(2) optimize rotary coding aperture performance using coordinate descent:
(21) two-value is carried out to each element of encoded light cycle matrix along horizontal (x) and vertical two dimension directions (y)
Overturning, and to newly-generated rotary coding aperture in-service evaluation criterion calculation aperture performance number.Then, this mistake is constantly repeated
Journey, until each element of traversal rotary coding aperture matrix, retains the wherein optimal result of aperture performance number.I.e. each time
Iteration result only changes the element value of an encoded light cycle matrix.
(22) iterative process in step (21) is repeated, until the evaluation criterion index convergence of coding aperture, aperture is no longer
Until changing.
(3) step (1) (2) are repeated, until with the raising of resolution ratio, the evaluation criterion index convergence of rotary coding aperture,
Until coding aperture is no longer substantially change.The resolution ratio for encoding aperture, which is finally improved by the 11 × 11 of low resolution, to be converged to
High-resolution 47 × 47.
It mentions high-resolution rotary coding aperture optimum results and is differentiated as shown in Fig. 2, corresponding to the optimization of rotary coding aperture
The process that rate steps up.
4) estimation of Depth is restored and total focus image restoration process.Visible light is obtained using rotary coding aperture imaging system
After image, the present invention carries out the work of depth recovery and total focus image restoration by wiener deconvolution and aberration method to it.
(1) the estimation clear image of different depth level is obtained using wiener warp area method firstThen aberration is used
Function e (x, y) calculates original blurred picture out of focus and estimates that clear image and corresponding each depth level point are expanded with each depth level
Each depth level for dissipating convolution of functions estimates the residual result of blurred picture out of focus.That is:
Wherein, arg indicates that the neighborhood window of part, IFT indicate inverse Fourier transform.
Within the scope of depth level, the corresponding depth level of minimum value of the capture difference function in each local neighborhood is should
The estimation of Depth information in region.Then, reconstruct depth results are averaged in the local window of a very little, rotation can be obtained
Turn the depth map of the visible images of coding aperture system photographs.
(2) total focus clear image is restored using acquired depth map.It is sought for corresponding to estimating depth out of focus fuzzy
The point spread function of the corresponding depth level of image each section, then using the resilient clear image out of wiener warp area method.It is i.e. complete
FocusedimageRecuperation it is as follows:
To the estimation of Depth recovery of the visible images of this clearly demarcated rotary coding aperture system and total focus figure in experiment
As reconstitution properties effect is tested, and compared with the coding aperture performance of pertinent literature, concrete outcome such as Fig. 3,
Shown in Fig. 4.
The above is only a preferred embodiment of the present invention, it is noted that for the ordinary skill people of the art
For member, without departing from the technical principles of the invention, several improvement and deformations can also be made, these improvement and deformations
Also it should be regarded as protection scope of the present invention.
Claims (2)
1. a kind of total focus imaging method based on rotary coding aperture, which is characterized in that this method proposes rotary coding first
The Performance evaluation criterion of aperture imaging system, and compiled according to the evaluation criterion using genetic algorithm and coordinate descent optimization rotation
Then code aperture realizes that estimation of Depth and total focus image restoration work by the rotary coding aperture of optimization.
2. as described in claim 1 based on the total focus imaging method of rotary coding aperture, which is characterized in that this method packet
It includes:
Step 1: proposing that the Performance evaluation criterion of rotary coding aperture imaging system is applied to estimation of Depth recovery and total focus
Image restoration:
The step 1 comprises the following specific steps that:
(1) fuzzy imaging model out of focus:
Wherein, f represents the blurred picture out of focus that imaging system is shot, f0Represent original clear image, kdRepresent imaging system
Point spread function at different depth d, η~N (0, σ2) represent white Gaussian noise；
(2) imaging model out of focus of rotary coding aperture system three times:
Fi=F0·Ki d+Ni, i=1,2,3
Wherein, this model is the expression of model in a frequency domain in (1): FiIndicate what rotary coding aperture imaging system was shot
Blurred picture out of focus, F0Indicate original image, Ki dIndicate the point spread function of different depth, NiIndicate white Gaussian noise, i then table
Show number of revolutions, i.e., rotation can generate blurred pictures out of focus different three times three times.This model correspond to rotary coding aperture at
As the mathematical model of resulting three blurred pictures out of focus of shooting of system；
(3) solve the problems, such as maximum a posteriori probability (Maximum A Posteriori) to realize to rotation using energy function minimum value
Turn the performance optimization of coding aperture；
Wherein,The jamtosignal (Noise to Signal Ratios) of representative image priori, can be expressed as σ2/ A,
Wherein, A can be used 1/f rule, i.e. the average power spectra of multiple natural images indicates,ξ is indicated
Frequency.Mean to make the difference between ambiguous estimation image and original blurred picture minimum using energy function minimum value, even if
Estimation procedure precision highest is obtained, depth recovery result is most accurate；
(4) estimation clear image is obtained using wiener warp area method；I.e. to a set estimating depthUse energy function
First solve estimation clear imageExistUnder conditions of, it can obtain,
Wherein,It is the complex conjugate of K,Refer in set estimating depthLocate the frequency domain of corresponding imaging system point spread function
It indicates；
(5) formula in step (4) and step (2) is brought into step (3), reconstructs energy function, collated simplification can obtain one
About estimating depthHeat-supplied function,
Wherein, d*It indicates to be located at the fuzzy core that imaging system corresponds to standard depth, d indicates deep at each level of visible images
The fuzzy core of degree；
(6) under conditions of noise level is σ, d is located to rotary coding aperture*The estimation of Depth at place, which carries out performance evaluation, to be obtained,
Wherein, D={ c1d*,c2d*,…,cld*Represent the size of the corresponding point spread function fuzzy core of each depth level, { ciCan
It is taken as { 0.1,0.15 ..., 1.5 }, step-length 0.05；
(7) Performance evaluation criterion that can obtain rotary coding aperture result above is normalized:
The energy function variation of the bigger characterization evaluation estimation of Depth of R value is more precipitous, so that estimation of Depth performance is for picture noise
Or the robustness of color texture is stronger；
The rotary coding aperture for designing and genetic algorithm optimization being used to obtain low resolution；Rotation is improved using coordinate descent to compile
The resolution ratio of code aperture；
Estimation of Depth restores and total focus image restoration process, is that carry out depth to it by wiener deconvolution and aberration method extensive
Multiple and total focus image restoration work；
Step 2: designing and obtain using genetic algorithm optimization the rotary coding aperture of low resolution；
For resolution ratio is the aperture of N × N, possibility up to 2N×NKind, calculation amount is very big, therefore can not directly use
Process is in optimized selection in high-resolution；
Step 3: the resolution ratio of rotary coding aperture is improved using coordinate descent；
In order to reduce the influence of optical diffraction, continues the performance for encoding aperture, carried out using performance of the coordinate descent to aperture
Re-optimization obtains optimization rotary coding aperture；
Step 4: estimation of Depth restores and total focus image restoration process；
After obtaining visible images using rotary coding aperture imaging system, it is carried out by wiener deconvolution and aberration method
The work of depth recovery and total focus image restoration.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811519373.4A CN109491079A (en) | 2018-12-12 | 2018-12-12 | Total focus imaging system based on rotary coding aperture |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811519373.4A CN109491079A (en) | 2018-12-12 | 2018-12-12 | Total focus imaging system based on rotary coding aperture |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109491079A true CN109491079A (en) | 2019-03-19 |
Family
ID=65709961
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811519373.4A Pending CN109491079A (en) | 2018-12-12 | 2018-12-12 | Total focus imaging system based on rotary coding aperture |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109491079A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113747052A (en) * | 2020-05-27 | 2021-12-03 | 深圳清华大学研究院 | Focusing tracking system and method |
| CN114615427A (en) * | 2022-02-19 | 2022-06-10 | 复旦大学 | Wavefront coding scene data set enhancement method based on small samples |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105469416A (en) * | 2015-12-25 | 2016-04-06 | 天津大学 | Rotary coded aperture imaging system based depth estimation method |
-
2018
- 2018-12-12 CN CN201811519373.4A patent/CN109491079A/en active Pending
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105469416A (en) * | 2015-12-25 | 2016-04-06 | 天津大学 | Rotary coded aperture imaging system based depth estimation method |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113747052A (en) * | 2020-05-27 | 2021-12-03 | 深圳清华大学研究院 | Focusing tracking system and method |
| CN113747052B (en) * | 2020-05-27 | 2023-05-12 | 深圳清华大学研究院 | Focusing tracking system and method |
| CN114615427A (en) * | 2022-02-19 | 2022-06-10 | 复旦大学 | Wavefront coding scene data set enhancement method based on small samples |
| CN114615427B (en) * | 2022-02-19 | 2023-11-28 | 复旦大学 | Wavefront coding scene data set enhancement method based on small samples |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ancuti et al. | Ntire 2019 image dehazing challenge report | |
| Strecke et al. | Accurate depth and normal maps from occlusion-aware focal stack symmetry | |
| CN109255831A (en) | The method that single-view face three-dimensional reconstruction and texture based on multi-task learning generate | |
| CN105806259B (en) | A kind of method for three-dimensional measurement based on the projection of two-value grating defocus | |
| CN106408524A (en) | Two-dimensional image-assisted depth image enhancement method | |
| CN105704371B (en) | A kind of light field refocusing method | |
| CN110443883A (en) | A kind of individual color image plane three-dimensional method for reconstructing based on dropblock | |
| CN107135388A (en) | A kind of depth extraction method of light field image | |
| CN110225260B (en) | Three-dimensional high dynamic range imaging method based on generation countermeasure network | |
| CN109491079A (en) | Total focus imaging system based on rotary coding aperture | |
| CN109658443A (en) | Stereo vision matching method and system | |
| CN109118544A (en) | Synthetic aperture imaging method based on perspective transform | |
| CN107093194A (en) | A kind of sub-aperture image-pickup method and system | |
| CN108668069A (en) | A kind of image background weakening method and device | |
| CN107610219A (en) | The thick densification method of Pixel-level point cloud that geometry clue perceives in a kind of three-dimensional scenic reconstruct | |
| CN111369443B (en) | Zero-order learning super-resolution method of light field cross-scale | |
| CN104661013B (en) | A kind of virtual viewpoint rendering method based on spatial weighting | |
| CN106096516A (en) | The method and device that a kind of objective is followed the tracks of | |
| Sui et al. | Accurate 3D reconstruction of dynamic objects by spatial-temporal multiplexing and motion-induced error elimination | |
| Chen et al. | GM-NeRF: Learning generalizable model-based Neural Radiance Fields from multi-view images | |
| CN113837947B (en) | Processing method for obtaining optical coherence tomography large focal depth image | |
| De Luigi et al. | Scannerf: a scalable benchmark for neural radiance fields | |
| Chandramouli et al. | A generative model for generic light field reconstruction | |
| CN107220945A (en) | The restored method of the pole blurred picture of multiple degeneration | |
| Li et al. | Image reflection removal using end‐to‐end convolutional neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190319 |