CN107622477A - A kind of RGBW images joint demosaicing and deblurring method - Google Patents
A kind of RGBW images joint demosaicing and deblurring method Download PDFInfo
- Publication number
- CN107622477A CN107622477A CN201710687202.1A CN201710687202A CN107622477A CN 107622477 A CN107622477 A CN 107622477A CN 201710687202 A CN201710687202 A CN 201710687202A CN 107622477 A CN107622477 A CN 107622477A
- Authority
- CN
- China
- Prior art keywords
- msub
- mrow
- fuzzy core
- image
- rgbw
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Image Processing (AREA)
Abstract
The present invention be directed to existing method, first carries out demosaicing processing to RGBW images, then carries out that image deblurring processing existing PSF estimated accuracies are poor, easily produce the problem of ringing, proposition while demosaicing and deblurring method.Comprise the concrete steps that:Image fuzzy core is estimated on RGBW single channel images, demosaicing interpolation is carried out to image fuzzy core, demosaicing is carried out to RGBW images, deblurring is carried out to demosaicing image, obtains picture rich in detail.Compared with prior art, PSF estimated accuracies, colour imaging quality of the lifting RGBW imaging sensors under unzoned lens system can be significantly improved.
Description
Technical field
The present invention relates to digital image processing techniques field, refers in particular to a kind of RGBW images joint demosaicing and deblurring side
Method.
Background technology
In modern optical system, picture quality can reduce because of optical parallax, and the overwhelming majority has spherical mirror knot
Single convex lens of structure all can be by the influence such as aberration, spherical aberration, coma.In order to solve this predicament, existing optical imagery
System is mainly to make up the aberration of single eyeglass by the combined lens of complexity, for example, slr camera camera lens may include it is tens of
Individual independent single eyeglass or lens set.But the design of complex combination camera lens is while image quality is improved, undoubtedly also significantly
The cost of lens design manufacture is added, and the volume and weight of camera lens is consequently increased.Therefore, how simple mirror is being eliminated
Piece group aberration, while ensureing image quality, reduce the design and manufacture cost of camera lens, make it lighter, be future optical into
As the development trend of system.Unzoned lens system has a potential prospect in many scientific domains, for example, unmanned plane, remotely sensed image with
And medical imaging.
In recent years, with the fast development of image restoration technology, more and more ripe, certain in camera lens the methods of image deblurring
A little aberrations and the eyeglass of Modified geometrical distortion of eliminating can be calculated camera work replacement, therefore, unzoned lens imaging by deblurring etc.
And the combination for calculating camera work is increasingly becoming a new research direction.
General to obtain coloured image using the imaging sensor with color filter array (CFA), most common of which is
Bayer forms CFA, Bayer are made up of green (g), red (r), blue (b) pixel.RGBW forms are on this basis by increasing white
(w) pixel, to strengthen low-light-level imaging performance.In RGBW format-patterns, r, g, b, w tetra- is contained in 2*2 adjacent pixel
Pixel.
For unzoned lens imaging system, because shooting image exist it is larger fuzzy, it is necessary to complete RGBW images to
Deblurring processing is carried out to image after the conversion of RGB images.But the demosaicing flow changed in RGBW images to RGB image
In, easily induce one different interchannel picture noises, follow-up image is obscured kernel estimates inaccuracy, so as to influence image deblurring
Quality, for example produce ringing.Therefore, the present invention proposes directly first to carry out fuzzy core estimation on RGBW format-patterns, then
Operated by demosaicing, obtain real RGB color channel blur core, deblurring is carried out to the RGB image after demosaicing,
To obtain the picture rich in detail of high quality.
The content of the invention
The defects of to overcome prior art to exist, the present invention provide a kind of RGBW format-patterns demosaicing and deblurring side
Method, it is characterised in that:Comprise the following steps:
S1:According to RGBW imaging formats, y is extractedRGBWR, g, b, w passage pixel in image, form 4 width single channel images:
yR、yG、yB、yW, wherein yR、yG、yB、yWImage size be yRGBW1/4;
S2:Using blind convolution method of estimation estimation image yR、yG、yB、yWFuzzy core:kr、kg、kb、kwSo that:
Wherein, xr、xg、xb、xwIt is y respectivelyR、yG、yB、yWCorresponding picture rich in detail, nr、ng、nb、nwIt is picture noise, mould
Pasting core size is:R*R;
S3:The matrix M of 2R*2R sizes is created, according to RGBW imaging formats, by kr、kg、kb、kwMiddle element look at r, g,
B, w pixel values are inserted in matrix M successively, M is turned into a RGBW form fuzzy core, i.e.,:Contained in M in 2*2 elements r,
G, the fuzzy core value of b, w passage;
S4:Matrix M look at RGBW format-patterns, using RGBW form demosaicing methods, matrix M is converted to RGB
Triple channel matrix M';
S5:G channel datas in M' are taken, as image yRGBFuzzy core estimated matrix, it is clear to operate to obtain by deconvolution
Image xRGB。
Preferably, in step S5, the fuzzy core of r, g, b channel data in M' respectively as r, g, b passage pixel is taken, to figure
As yRGBR, g, b passage carry out deconvolution operation, obtain picture rich in detail xRGB。
Fuzzy core corresponding to image is estimated using the Krishnan blind convolution proposed in step sl, wherein blind convolution is asked
The object function of the ambiguity solution nuclear issue can be expressed as:
Wherein, k represents fuzzy core, also known as point spread function PSF;X represents picture rich in detail;What y represented to obtain in S1 obscures
Image;Represent convolution operation;Section 1It is data fit term, represents blurred picture and picture rich in detail after convolution
Matching degree;Section 2It is x business of the norm with two norms, is the bound term to picture rich in detail x;Section 3 μ | | k
||1The total variant prioris to fuzzy core k are represented, are the bound term to fuzzy core k;λ and μ is data fit term
With the weight coefficient of bound term;It is the conservation of energy to fuzzy core and nonnegativity restriction.
Formula (5) is carried out by iterative, solution procedure in multiscale space.Detailed process is as follows:Based on down-sampled
A picture size is established by being to fine image pyramid, the ratio of each two image layer roughlyWith 3 × 3 Gauss
The initial value of function or delta function as fuzzy core, the number of plies of the image layer of image pyramid corresponding to current image block
Determined by the ratio of the size and the size of initial fuzzy core of the fuzzy core set;By successively in different levels metric space
Solution formula (5), progressive alternate try to achieve final fuzzy core, in each layer of metric space, first by last layer subdimension space
Middle tried to achieve fuzzy core tries to achieve potential picture rich in detail, then potential picture rich in detail as initial value with reference to blurred picture
Next metric space is substituted into as known terms with fuzzy core, then obtains fuzzy core and picture rich in detail, until last yardstick is empty
Between.
The RGBW format-patterns demosaicing methods, 3*3,5*5 pixel size template can be used, it is real by linear interpolation
It is existing.
Present invention has the advantages that:Image demosaicing is first carried out different from existing method, then mould is carried out to RGB image
Paste, the inventive method, simultaneously considers image demosaicing and deblurring, can effectively lift fuzzy kernel estimates precision, lifts RGB
Image definition.
Brief description of the drawings
Nothing;
Embodiment
In order that the purpose of the present invention, technical scheme and beneficial effect are more clearly understood, with reference to embodiment, to this
Invention is further elaborated.It should be noted that specific embodiment described herein is only to explain the present invention, and do not have to
It is of the invention in limiting.
Embodiment 1:
S1:According to RGBW imaging formats, y is extractedRGBWR, g, b, w passage pixel in image, form 4 width single channel images:
yR、yG、yB、yW, wherein yR、yG、yB、yWImage size be yRGBW1/4;
S2:Using blind convolution method of estimation estimation image yR、yG、yB、yWFuzzy core:kr、kg、kb、kwSo that:
Wherein, xr、xg、xb、xwIt is y respectivelyR、yG、yB、yWCorresponding picture rich in detail, nr、ng、nb、nwIt is picture noise, mould
Pasting core size is:R*R;
S3:The matrix M of 2R*2R sizes is created, according to RGBW imaging formats, by kr、kg、kb、kwMiddle element look at r, g,
B, w pixel values are inserted in matrix M successively, M is turned into a RGBW form fuzzy core, i.e.,:Contained in M in 2*2 elements r,
G, the fuzzy core value of b, w passage;
S4:Matrix M look at RGBW format-patterns, using RGBW form demosaicing methods, matrix M is converted to RGB
Triple channel matrix M';
S5:G channel datas in M' are taken, as image yRGBFuzzy core estimated matrix, it is clear to operate to obtain by deconvolution
Image xRGB。
The RGBW format-patterns demosaicing methods, 3*3,5*5 pixel size template can be used, it is real by linear interpolation
It is existing.
Embodiment 2:
Difference from Example 1 is, in step s 5, takes in M' r, g, b channel data respectively as r, g, b passage
The fuzzy core of pixel, to image yRGBR, g, b passage carry out deconvolution operation, obtain picture rich in detail xRGB。
Embodiment 3:
Fuzzy core corresponding to image is estimated using the Krishnan blind convolution proposed in step sl, wherein blind convolution is asked
The object function of the ambiguity solution nuclear issue can be expressed as:
Wherein, k represents fuzzy core, also known as point spread function PSF;X represents picture rich in detail;yRepresent to obtain in S1 obscures
Image;Represent convolution operation;Section 1It is data fit term, represents blurred picture and picture rich in detail after convolution
Matching degree;Section 2It is x business of the norm with two norms, is the bound term to picture rich in detail x;Section 3 μ | | k
||1The total variant prioris to fuzzy core k are represented, are the bound term to fuzzy core k;λ and μ is data fit term
With the weight coefficient of bound term;It is the conservation of energy to fuzzy core and nonnegativity restriction.
Formula (5) is carried out by iterative, solution procedure in multiscale space.Detailed process is as follows:Based on down-sampled
A picture size is established by being to fine image pyramid, the ratio of each two image layer roughlyWith 3 × 3 Gauss
The initial value of function or delta function as fuzzy core, the number of plies of the image layer of image pyramid corresponding to current image block
Determined by the ratio of the size and the size of initial fuzzy core of the fuzzy core set;By successively in different levels metric space
Solution formula (5), progressive alternate try to achieve final fuzzy core, in each layer of metric space, first by last layer subdimension space
Middle tried to achieve fuzzy core tries to achieve potential picture rich in detail, then potential picture rich in detail as initial value with reference to blurred picture
Next metric space is substituted into as known terms with fuzzy core, then obtains fuzzy core and picture rich in detail, until last yardstick is empty
Between.
Claims (4)
1. a kind of RGBW images joint demosaicing and deblurring method, it is characterised in that:Comprise the following steps:
S1:According to RGBW imaging formats, y is extractedRGBWR, g, b, w passage pixel in image, form 4 width single channel images:yR、yG、
yB、yW, wherein yR、yG、yB、yWImage size be yRGBW1/4;
S2:Using blind convolution method of estimation estimation image yR、yG、yB、yWFuzzy core:kr、kg、kb、kwSo that:
<mrow>
<msub>
<mi>y</mi>
<mi>R</mi>
</msub>
<mo>=</mo>
<msub>
<mi>k</mi>
<mi>r</mi>
</msub>
<mo>&CircleTimes;</mo>
<msub>
<mi>x</mi>
<mi>r</mi>
</msub>
<mo>+</mo>
<msub>
<mi>n</mi>
<mi>r</mi>
</msub>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>1</mn>
<mo>)</mo>
</mrow>
</mrow>
<mrow>
<msub>
<mi>y</mi>
<mi>G</mi>
</msub>
<mo>=</mo>
<msub>
<mi>k</mi>
<mi>g</mi>
</msub>
<mo>&CircleTimes;</mo>
<msub>
<mi>x</mi>
<mi>g</mi>
</msub>
<mo>+</mo>
<msub>
<mi>n</mi>
<mi>g</mi>
</msub>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>2</mn>
<mo>)</mo>
</mrow>
</mrow>
<mrow>
<msub>
<mi>y</mi>
<mi>B</mi>
</msub>
<mo>=</mo>
<msub>
<mi>k</mi>
<mi>b</mi>
</msub>
<mo>&CircleTimes;</mo>
<msub>
<mi>x</mi>
<mi>b</mi>
</msub>
<mo>+</mo>
<msub>
<mi>n</mi>
<mi>b</mi>
</msub>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>3</mn>
<mo>)</mo>
</mrow>
</mrow>
<mrow>
<msub>
<mi>y</mi>
<mi>W</mi>
</msub>
<mo>=</mo>
<msub>
<mi>k</mi>
<mi>w</mi>
</msub>
<mo>&CircleTimes;</mo>
<msub>
<mi>x</mi>
<mi>w</mi>
</msub>
<mo>+</mo>
<msub>
<mi>n</mi>
<mi>w</mi>
</msub>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>4</mn>
<mo>)</mo>
</mrow>
</mrow>
Wherein, xr、xg、xb、xwIt is y respectivelyR、yG、yB、yWCorresponding picture rich in detail, nr、ng、nb、nwIt is picture noise, fuzzy core
Size is:R*R;
S3:The matrix M of 2R*2R sizes is created, according to RGBW imaging formats, by kr、kg、kb、kwMiddle element look at r, g, b, w pixel
Value is inserted in matrix M successively, M is turned into a RGBW form fuzzy core, i.e.,:R, g, b, w are contained in M in 2*2 elements to lead to
The fuzzy core value in road;
S4:Matrix M look at RGBW format-patterns, using RGBW form demosaicing methods, matrix M is converted to RGB threeways
Road matrix M';
S5:G channel datas in M' are taken, as image yRGBFuzzy core estimated matrix, operate to obtain picture rich in detail by deconvolution
xRGB。
2. according to claim 1, fuzzy core corresponding to image is estimated using the Krishnan blind convolution proposed in step sl,
The object function that wherein blind the solution of convolution obscures the nuclear issue can be expressed as:
<mrow>
<munder>
<mi>min</mi>
<mrow>
<mi>x</mi>
<mo>,</mo>
<mi>y</mi>
</mrow>
</munder>
<mi>&lambda;</mi>
<mo>|</mo>
<mo>|</mo>
<mi>x</mi>
<mo>&CircleTimes;</mo>
<mi>k</mi>
<mo>-</mo>
<mi>y</mi>
<mo>|</mo>
<msubsup>
<mo>|</mo>
<mn>2</mn>
<mn>2</mn>
</msubsup>
<mo>+</mo>
<mfrac>
<mrow>
<mo>|</mo>
<mo>|</mo>
<mi>x</mi>
<mo>|</mo>
<msub>
<mo>|</mo>
<mn>1</mn>
</msub>
</mrow>
<mrow>
<mo>|</mo>
<mo>|</mo>
<mi>x</mi>
<mo>|</mo>
<msub>
<mo>|</mo>
<mn>2</mn>
</msub>
</mrow>
</mfrac>
<mo>+</mo>
<mi>&mu;</mi>
<mo>|</mo>
<mo>|</mo>
<mi>k</mi>
<mo>|</mo>
<msub>
<mo>|</mo>
<mn>1</mn>
</msub>
<mi>s</mi>
<mo>.</mo>
<mi>t</mi>
<mo>.</mo>
<mi>k</mi>
<mo>></mo>
<mn>0</mn>
<mo>,</mo>
<munder>
<mi>&Sigma;</mi>
<mi>i</mi>
</munder>
<msub>
<mi>k</mi>
<mi>i</mi>
</msub>
<mo>=</mo>
<mn>1</mn>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>5</mn>
<mo>)</mo>
</mrow>
</mrow>
Wherein, k represents fuzzy core, also known as point spread function PSF;X represents picture rich in detail;Y represents the blurred picture obtained in S1;Represent convolution operation;Section 1It is data fit term, represents the matching of blurred picture and picture rich in detail after convolution
Degree;Section 2It is x business of the norm with two norms, is the bound term to picture rich in detail x;Section 3 μ | | k | |1Table
Show the total variant prioris to fuzzy core k, be the bound term to fuzzy core k;λ and μ is data fit term and constraint
The weight coefficient of item;It is the conservation of energy to fuzzy core and nonnegativity restriction.
3. according to claim 1 and 2, in step sl, by iterative formula (5), iterative process is in multiple dimensioned sky
Between carry out, detailed process is as follows:A picture size is established by roughly to fine image pyramid, each two based on down-sampled
The ratio of image layer isUsing 3 × 3 Gaussian function or delta function as the initial value of fuzzy core, current image block pair
The number of plies of the image layer for the image pyramid answered is determined by the ratio of the size and the size of initial fuzzy core of the fuzzy core set;
Final fuzzy core is tried to achieve by solution formula (5) in different levels metric space, progressive alternate successively, in each layer of yardstick
Space, first using the fuzzy core tried to achieve in last layer subdimension space as initial value, tried to achieve potentially with reference to blurred picture
Picture rich in detail, potential picture rich in detail and fuzzy core are then substituted into next metric space as known terms, then obtain fuzzy core
And picture rich in detail, until last metric space.
4. according to claim 1, in step S5, r, g, b channel data obscuring respectively as r, g, b passage pixel in M' are taken
Core, to image yRGBR, g, b passage carry out deconvolution operation, obtain picture rich in detail xRGB。
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710687202.1A CN107622477A (en) | 2017-08-08 | 2017-08-08 | A kind of RGBW images joint demosaicing and deblurring method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710687202.1A CN107622477A (en) | 2017-08-08 | 2017-08-08 | A kind of RGBW images joint demosaicing and deblurring method |
Publications (1)
Publication Number | Publication Date |
---|---|
CN107622477A true CN107622477A (en) | 2018-01-23 |
Family
ID=61089047
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201710687202.1A Pending CN107622477A (en) | 2017-08-08 | 2017-08-08 | A kind of RGBW images joint demosaicing and deblurring method |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN107622477A (en) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109544462A (en) * | 2018-09-28 | 2019-03-29 | 北京交通大学 | License plate image deblurring method based on adaptively selected fuzzy core |
CN110290370A (en) * | 2019-07-05 | 2019-09-27 | 上海富瀚微电子股份有限公司 | Image processing method and device |
WO2022011506A1 (en) * | 2020-07-13 | 2022-01-20 | 深圳市汇顶科技股份有限公司 | Image processing method and image processing apparatus |
CN117806036A (en) * | 2024-03-01 | 2024-04-02 | 中国科学院光电技术研究所 | Achromatic method of monolithic diffraction lens system based on compressed sensing |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20030169353A1 (en) * | 2002-03-11 | 2003-09-11 | Renato Keshet | Method and apparatus for processing sensor images |
CN106709879A (en) * | 2016-12-08 | 2017-05-24 | 中国人民解放军国防科学技术大学 | Spatial variation point diffusion function smoothing method based on simple lens calculating imaging |
-
2017
- 2017-08-08 CN CN201710687202.1A patent/CN107622477A/en active Pending
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20030169353A1 (en) * | 2002-03-11 | 2003-09-11 | Renato Keshet | Method and apparatus for processing sensor images |
CN106709879A (en) * | 2016-12-08 | 2017-05-24 | 中国人民解放军国防科学技术大学 | Spatial variation point diffusion function smoothing method based on simple lens calculating imaging |
Non-Patent Citations (2)
Title |
---|
RUIWEN ZHEN 等: "Joint Deblurring and Demosaicking of CFA Image Data with Motion Blur", 《 CONFERENCE ON VISUAL INFORMATION PROCESSING AND COMMUNICATION V》 * |
周同同: "基于相机抖动的模糊图像的盲复原实现", 《中国优秀硕士学位论文全文数据库 信息科技辑 ISSN 1674-0246》 * |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109544462A (en) * | 2018-09-28 | 2019-03-29 | 北京交通大学 | License plate image deblurring method based on adaptively selected fuzzy core |
CN110290370A (en) * | 2019-07-05 | 2019-09-27 | 上海富瀚微电子股份有限公司 | Image processing method and device |
WO2022011506A1 (en) * | 2020-07-13 | 2022-01-20 | 深圳市汇顶科技股份有限公司 | Image processing method and image processing apparatus |
CN117806036A (en) * | 2024-03-01 | 2024-04-02 | 中国科学院光电技术研究所 | Achromatic method of monolithic diffraction lens system based on compressed sensing |
CN117806036B (en) * | 2024-03-01 | 2024-05-17 | 中国科学院光电技术研究所 | Achromatic method of monolithic diffraction lens system based on compressed sensing |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111127336B (en) | Image signal processing method based on self-adaptive selection module | |
CN103327220B (en) | With green channel for the denoising method guided on low-light (level) Bayer image | |
CN107622477A (en) | A kind of RGBW images joint demosaicing and deblurring method | |
TWI737979B (en) | Image demosaicer and method | |
CN103905802B (en) | A kind of demosaicing methods based on P pattern colour filter array and device | |
CN108734668B (en) | Image color recovery method and device, computer readable storage medium and terminal | |
CN111882485B (en) | Hierarchical feature feedback fusion depth image super-resolution reconstruction method | |
CN106709879B (en) | A kind of spatial variations point spread function smoothing method that picture is calculated as based on unzoned lens | |
CN105122793B (en) | Image processing device, image capture device, and image processing program | |
WO2020126023A1 (en) | Image processor | |
CN105869112B (en) | A kind of edge keeps the high dynamic range image tone mapping method of minimum | |
CN107967668A (en) | A kind of image processing method and device | |
CN113129391B (en) | Multi-exposure fusion method based on multi-exposure image feature distribution weight | |
CN108171657B (en) | Image interpolation method and device | |
CN103268596A (en) | Method for reducing image noise and enabling colors to be close to standard | |
CN111563866B (en) | Multisource remote sensing image fusion method | |
CN106454144A (en) | Correction method for over-exposure of Google glasses image | |
CN103747222B (en) | A kind of method that color artifact of image suppresses and device | |
CN113689335B (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
CN113052765B (en) | Panoramic image splicing method based on optimal grid density model | |
CN117274060B (en) | Unsupervised end-to-end demosaicing method and system | |
CN101815220B (en) | Method for correcting image color distortion | |
CN102903091B (en) | Method for stitching image in digital image processing apparatus | |
CN106162133A (en) | Color interpolation method based on adaptive directed filtering | |
CN106780378A (en) | A kind of blind convolved image restored method that two lenses lens have been corrected for aberration |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
CB02 | Change of applicant information | ||
CB02 | Change of applicant information |
Address after: 610021 Longquanyi Jingkai District, Chengdu City, Sichuan Province, 309 Dingfeng Power Port, 12 buildings, 7 units 501 Applicant after: Chengdu Seiko Hua Yao Technology Co., Ltd. Address before: 610021 Longquanyi Jingkai District, Chengdu City, Sichuan Province, 309 Dingfeng Power Port, 12 buildings, 7 units 501 Applicant before: Chengdu Huayao Precision Machinery Manufacturing Co. Ltd. |
|
WD01 | Invention patent application deemed withdrawn after publication | ||
WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20180123 |