CN102663719B - Bayer-pattern CFA image demosaicking method based on non-local mean - Google Patents

Abstract

The invention discloses a bayer-pattern CFA image demosaicking method based on a non-local mean, mainly solving the problem of poor effect of interpolation on a small fringe part of an image in the prior art, comprising the following steps:(1) inputting a Bayer-pattern CFA image; (2) estimating brightness; (3) adopting an image block pixel by pixel as a current to-be-demosaicked image block; (4) extracting three image block sets; (5) calculating weights of the image blocks; (6) performing weighted average on the similar image blocks; (7) performing interpolation on the current to-be-demosaicked image block; (8) determining whether interpolation on all pixels is completed, if yes, executing step (9), if not shifting to step (3); (9) correcting an edge; (10) correcting a color; and (11) outputting a color image. By adopting the method provided in the invention small fringe area information of an image can well restored, and false color effects can be effectively inhibited; and the method is especially suitable for a CFA image with more textures.

Description

Bayer type CFA image demosaicing method based on non-local mean

Technical field

The invention belongs to technical field of image processing, further relate to the Bayer type CFA image demosaicing method based on non-local mean in image-recovery technique field.The present invention can be used for recovering the full color information of CFA image in patrilineal line of descent with only one son in each generation sense chip camera, thereby makes up the loss of the image color information of bringing due to minimizing camera hardware cost; Meanwhile, the raw data that also can be used in computing machine, camera being preserved in " Raw Mode " mode is processed, thereby obtains high-quality coloured image.

Background technology

Patrilineal line of descent with only one son in each generation sense camera adopts single CCD or CMOS sensing chip, catches image by cover a color filter array (color filer array, CFA) before sensor.The CFA image that this mode is caught is a kind of mosaic image, each pixel only has a color value in three primary colours, two other color value needs to carry out interpolation according to its neighborhood information, and this color interpolation technology is called as " demosaicing " (demosaicing).

Existing CFA image demosaicing technology can be divided into linear interpolation techniques and non-linear interpolation technology simply.Linear interpolation is the simplest also the most representative bilinear interpolation and two cube interpolation.The advantage of bilinear interpolation and two cube interpolation techniques is that its realization is simple, and speed is fast; But its shortcoming is also apparent: in image, can produce serious artificial trace (as color fringing, i.e. false colour effect), and especially more outstanding at the HFS of image.Relatively, non-linear interpolation technology is more complicated, and owing to having considered the interchannel correlativity of RGB, its interpolation successful is better than linear interpolation techniques.This class technology has a lot, for example, Wenmain Lu and Yap-peng Tan be " Color filter array demosaicing:new method and performance measures " (IEEE Trans.Image Process. in article, vol.12, no.10, pp.1194-1210, Oct.2003) the middle demosaicing method proposing.Although non-linear interpolation technology can obtain high-quality color image, its complexity is high, and calculation cost is large.

Patented claim " high-quality gradient for coloured image demosaicing is proofreaied and correct the linear interpolation " (applying date: on 03 15th, 2005 that Microsoft proposes, application number: 200510055929.5, publication number: CN1722852) in a kind of gradient calibration linear interpolation method and system for coloured image demosaicing disclosed.First the method and system estimate the value of required color by existing linear interpolation techniques as bilinear interpolation technology, then carry out calculation correction item by the gradient of calculating required color on current pixel, last this interpolation of linear combination and correction term are omitted color-values to produce in pixel.The method directly affects by gradient and proofreaies and correct the color-values of being estimated by existing interpolation technique.But weak point is: only utilized gradient to proofread and correct the result of bilinear interpolation, although had significant improvement than bilinear interpolation, still not ideal to image detail section processes effect, false colour effect is still serious.

Patent " the CFA image demosaicing method based on edge direction the interpolation " (applying date: on 07 22nd, 2011 that Xian Electronics Science and Technology University proposes, application number: 201110207617.7, publication number: CN102254301) a kind of demosaicing method based on edge direction interpolation of middle proposition.The method is walked always interpolation according to the edge of image, and the interpolation effect of the marginal portion of image is significantly improved.The weak point of the method is: the method is to judge edge direction by gradient, can obtain good interpolation result for large edge, but cannot judge edge direction for tiny edge, so the method is undesirable to fine edge and texture region interpolation result.

Summary of the invention

The present invention is directed to the fine edge of existing demosaicing algorithm to image and the defect of texture region interpolation poor effect, a kind of Bayer type CFA image demosaicing method based on non-local mean has been proposed, the present invention has adopted the thought of non-local mean, by fragmental image processing, in the neighborhood of current image block, search for similar image piece, utilize the structural similarity of image to carry out interpolation to current image block, the interpolation result of the HFS of image is increased significantly, effectively suppressed false colour effect.

For achieving the above object, the present invention includes following key step:

(1) input one width is treated the Bayer type CFA image of demosaicing;

(2) adopt brightness-formula estimated brightness;

(3) treat that in input the image block of getting 5 × 5 pixel sizes by pixel in the Bayer type CFA image of demosaicing is as the current image block for the treatment of demosaicing;

(4) in the neighborhood of current 33 × 33 pixel sizes for the treatment of demosaicing image block, extract three groups of image set of blocks;

(5) weight of computed image piece

5a) adopt gaussian kernel range formula to calculate each image block and the current gaussian kernel distance for the treatment of demosaicing image block in three groups of image set of blocks;

5b) in every group of image set of blocks in three groups of image set of blocks, by gaussian kernel distance is ascending, all image blocks of this group are sorted, get front 5 image blocks as similar image piece, give up remaining image piece;

5c) adopt weight calculation formula to calculate the weight of 5 similar image pieces in every group of image set of blocks;

(6) adopt weighted mean formula to be weighted on average the similar image piece in three groups of image set of blocks;

(7) value in weighted image piece is according to pixels inserted into current treating in demosaicing image block in locus correspondingly;

(8) judged whether the interpolation of all pixels, if complete, execution step (9), otherwise proceed to step (3);

(9) with edges by trimming correction method to interpolation after image carry out edge correction;

(10) Color correction

10a) adopt color space conversion formula that revised edge image is transformed into yuv space from rgb space;

10b) in yuv space, the chrominance matrix U to image and V carry out respectively the medium filtering of 3 × 3 windows;

10c) image is transformed into rgb space from yuv space, and the color value that does not need interpolation in each pixel is replaced with to original color value corresponding in CFA image;

10d) judge whether the medium filtering of tertiary colo(u)r(s) degree matrix U and V, if complete, execution step (11), otherwise proceed to step 10a);

(11) output coloured image.

The present invention has the following advantages compared with prior art:

First, the present invention adopts the thought of non-local mean, utilize the redundancy of image self to search for similar image piece, and calculate weight according to the brightness of accurately estimating, utilize the structural similarity of similar image piece to carry out interpolation to current image block, the colour effect that can avoid prior art to cause across edge interpolation, makes the present invention be particularly useful for processing the more image of texture.

Second, the present invention is directed to the defect of the fine edge REGION INTERPOLATION poor effect of prior art to image, image after non-local mean interpolation is carried out to edge correction, the interpolation effect of the HFS of image is significantly improved, improved the interpolation effect to fine edge;

The 3rd, the revised image of edge of the present invention has carried out Color correction, two chrominance matrixs of image are carried out to medium filtering three times, improved prior art and easily cause the shortcoming of fraction pixel chromaticity distortion in Interpolation Process, effectively suppressed the appearance of false color.

Brief description of the drawings

Fig. 1 is process flow diagram of the present invention;

Fig. 2 is prior art and demosaicing simulated effect figure of the present invention.

Embodiment

Below in conjunction with accompanying drawing 1, the step that the present invention is realized is described in further detail:

Step 1, input one width is treated the CFA image I of demosaicing _sthe CFA image of wherein treating demosaicing is the CFA image of Bayer pattern, and in image, each pixel only exists a color value in red, green, blue three primary colours, and this color value does not need interpolation, two other color value disappearance, need to adopt demosaicing method to carry out interpolation.

Step 2, estimated brightness, computing formula is as follows:

$L=A{(H^T{\mathrm{\σ}}^{-1}H+{\mathrm{\λ}}_1{M}_1^T{M}_1+{\mathrm{\λ}}_2{M}_2^T{M}_2)}^{-1}{H}^T{\mathrm{\σ}}^{-1}{I}_s$

Wherein, the luminance matrix that L is required estimation, A is the transition matrix that RGB coloured image is converted into luminance matrix, and H is the decimation factor that RGB color picture sampling becomes Bayer form A FA image, and T is matrix transpose operator, and subscript-1 is matrix inversion operation symbol, σ, λ ₁and λ ₂for constant, its value is respectively: σ=0.00001, λ ₁=0.0008, λ ₂=0.02, M ₁and M ₂for filtering matrix,

i ₃be 3 × 3 size unit matrixes, S ₁for Hi-pass filter [0.2 ,-0.5,0.65 ,-0.5,0.2],

represent Kronecker (Kronecker) operator, M ₂obtained by following formula computing:

$M_2=\left(\begin{array}{ccc}1.547& -0.577& -0.577\\ -0.577& 1.547& -0.577\\ -0.577& -0.577& 1.547\end{array}\right)\⊗S_2$

S ₂for Hi-pass filter [0.5,1 ,-0.5], I _sfor the CFA image of input.

Step 3, in input picture, from the pixel of the 17th row the 17th row, the pixel being listed as to the 17th row the reciprocal the 17th reciprocal, individual element is got size as the image block of 5 × 5 pixels is as the current image block for the treatment of demosaicing centered by this pixel;

Step 4 is extracted three groups of image set of blocks in the current neighborhood for the treatment of demosaicing image block 33 × 33 pixel sizes

4a) centered by the central pixel point of the current image block for the treatment of demosaicing, get the search window of a size as 33 × 33 pixels;

Pixel 4b) being listed as from the 3rd row the 4th in search window, get a pixel as central pixel point with column direction every a pixel in the row direction respectively, centered by this pixel, get the image block of 5 × 5 pixel sizes, altogether obtain 15 × 14 totally 210 image blocks, using these image blocks as first group of image set of blocks;

Pixel 4c) being listed as from the 4th row the 3rd in search window, get a pixel as central pixel point with column direction every a pixel in the row direction respectively, centered by this pixel, get the image block of 5 × 5 pixel sizes, altogether obtain 14 × 15 totally 210 image blocks, using these image blocks as second group of image set of blocks;

Pixel 4d) being listed as since the 4th row the 4th in search window, get a pixel as central pixel point with column direction every a pixel in the row direction respectively, centered by this pixel, get the image block of 5 × 5 pixel sizes, altogether obtain 14 × 14 totally 196 image blocks, using these image blocks as the 3rd group of image set of blocks;

Step 5, computed image piece weight

5a) adopt gaussian kernel range formula to calculate each image block and the current gaussian kernel distance for the treatment of demosaicing image block in three groups of image set of blocks, computing formula is as follows:

$d_{k,i}={\left|\right|G\·(y_{k,i}^L-y_0^L)\left|\right|}_2^2$

Wherein, d _{k, i}i image block and the current distance for the treatment of between demosaicing image block in k group image set of blocks, k is image block set numbers k=1, the index of image block in 2,3, i presentation video set of blocks, represent two norms square, G is gaussian kernel matrix, represent point multiplication operation,

be the vector representation of i image block brightness in k group image set of blocks, the brightness of subscript L presentation video piece,

for the current vector representation for the treatment of the brightness of demosaicing image block.

5b) in every group of image set of blocks in three groups of image set of blocks, by gaussian kernel distance is ascending, all image blocks of this group are sorted, get front 5 image blocks as similar image piece, give up remaining image piece;

5c) adopt weight calculation formula to calculate the weight of 5 similar image pieces in every group of image set of blocks, computing formula is as follows:

$w_{k,j}=\frac{1}{Z(k,j)}\exp (-\frac{d_{k,j}}{h^2}),$ $Z(k,j)=\underset{j}{\mathrm{\Σ}}\exp (-\frac{d_{k,j}}{h^2})$

Wherein, w _{k, j}be the weight of j similar image piece in k group image set of blocks, k is similar image set of blocks sequence number k=1, and 2,3, j represents the index j=1 of 5 similar image pieces in similar image set of blocks, and 2,3,4,5, Z (k, j) is normalization coefficient, d _{k, j}be j similar image piece and the current gaussian kernel distance for the treatment of between demosaicing image block in k group image set of blocks, constant h is smoothing parameter, and value is 10, and ∑ represents cumulative operation.

Step 6, adopts weighted mean formula to be weighted on average the similar image piece in three groups of image set of blocks, obtains three image blocks after weighting, and their vector representation is

with

${\stackrel{\‾}{y}}_k=\underset{j}{\mathrm{\Σ}}{w}_{k,j}{y}_{k,j}$

Wherein, be image block after the weighting of k group similar image set of blocks, k is similar image set of blocks sequence number k=1,2,3, and ∑ represents cumulative operation, the index j=1 of 5 similar image pieces in j presentation video set of blocks, 2,3,4,5, w _{k, j}be the weight of j similar image piece in k group similar image set of blocks, y _{k, j}it is the vector representation of j similar image piece in k group similar image set of blocks.

Step 7, utilizes three weighted image pieces to treat that to current demosaicing image block carries out interpolation, the current demosaicing image block y that treats ₀with weighted image piece

be 5 × 5 image blocks, by weighted image piece

in value be according to pixels inserted into correspondingly the current demosaicing image block y that treats in locus ₀in, obtain the three-dimensional matrice of 5 × 5 × 4, wherein each pixel has a red value, two green values and a blue valve.In this three-dimensional matrice, on the same space position, the final green value of each pixel is got the average of the two, obtains a coloured image piece, thereby has completed the interpolation to current center image piece.

Step 8, has judged whether the interpolation of all pixels, if complete execution step (9), otherwise proceeds to step (3);

Step 9, edge correction

9a) according to the horizontal and vertical gradient of whole green channel after following formula calculating interpolation:

$\left\{\begin{array}{c}{\mathrm{\ΔH}}_g(i,j)=|G(i,j-1)-G(i,j+1)|+\frac{1}{2}|2G(i,j)-G(i,j-1)-G(i,j+1)|\\ {\mathrm{\ΔV}}_g(i,j)=|G(i-1,j)-G(i+1,j)|+\frac{1}{2}|2G(i,j)-G(i-1,j)-G(i+1,j)|\end{array}\right.$

Wherein, Δ H _g(i, j) is the horizontal gradient value of green channel pixel (i, j) after interpolation, and (i, j) is pixel location index, and G (i, j) is the green value of green channel pixel (i, j) after interpolation, Δ V _g(i, j) is the VG (vertical gradient) value of green channel pixel (i, j) after interpolation;

9b), for the pixel of known red value in CFA image, the green value inserting in these pixels is revised according to following formula:

$\tilde{G}(i,j)=r(i,j)+\left\{\begin{array}{cc}\frac{w_2(g(i-1,j)-R(i-1,j))+w_7(g(i+1,j)-R(i+1,j))}{w_2+w_7},& {\mathrm{\&Delta;H}}_g(i,j)>{\mathrm{\&Delta;V}}_g(i,j)\\ \frac{w_4(g(i,j-1)-R(i,j-1))+w_5(g(i,j+1)-R(i,j+1))}{w_4+w_5},& {\mathrm{\&Delta;H}}_g(i,j)<{\mathrm{\&Delta;V}}_g(i,j)\\ \frac{\underset{k=1}{\overset{8}{\mathrm{\&Sigma;}}}{w}_k(g(i+m_k,j+n_k)-R(i+m_k,j+n_k))}{\underset{k=1}{\overset{8}{\mathrm{\&Sigma;}}}{w}_k},& {\mathrm{\&Delta;H}}_g(i,j)={\mathrm{\&Delta;V}}_g(i,j)\end{array}\right.$

Wherein,

$w_k=\frac{1}{{[1+{(r(i,j)-r(i+m_k,j+n_k))}^2]}^{1/2}}$

for the revised green value of pixel (i, j), (i, j) is pixel location index, and r (i, j) is the red value that pixel (i, j) is known, w _kfor weights, k is w _k, m _kand n _kin value sequence number, k=1,2...8, g (i-1, j) is the green value that pixel (i-1, j) is known, the red value inserting in R (i-1, j) pixel (i-1, j), Δ H _g(i, j) is the horizontal gradient value of green channel pixel (i, j), Δ V _g(i, j) is the VG (vertical gradient) value of green channel pixel (i, j), and ∑ represents cumulative operation, m _kand n _kfor integer, m _k∈ 2 ,-2 ,-2,0,0,2,2,2}, n _k∈ 2,0,2 ,-2,2 ,-2,0,2};

9c), for the pixel of known blue valve in CFA image, the green value inserting in these pixels is revised according to following formula:

$\tilde{G}(i,j)=b(i,j)+\left\{\begin{array}{cc}\frac{w_2(g(i-1,j)-B(i-1,j))+w_7(g(i+1,j)-B(i+1,j))}{w_2+w_7},& {\mathrm{\&Delta;H}}_g(i,j)>{\mathrm{\&Delta;V}}_g(i,j)\\ \frac{w_4(g(i,j-1)-B(i,j-1))+w_5(g(i,j+1)-B(i,j+1))}{w_4+w_5},& {\mathrm{\&Delta;H}}_g(i,j)<{\mathrm{\&Delta;V}}_g(i,j)\\ \frac{\underset{k=1}{\overset{8}{\mathrm{\&Sigma;}}}{w}_k(g(i+m_k,j+n_k)-B(i+m_k,j+n_k))}{\underset{k=1}{\overset{8}{\mathrm{\&Sigma;}}}{w}_k},& {\mathrm{\&Delta;H}}_g(i,j)={\mathrm{\&Delta;V}}_g(i,j)\end{array}\right.$

Wherein,

$w_k=\frac{1}{{[1+{(b(i,j)-b(i+m_k,j+n_k))}^2]}^{1/2}}$

for the revised green value of pixel (i, j), (i, j) is pixel location index, and b (i, j) is the blue valve that pixel (i, j) is known, w _kfor weights, k is w _k, m _kand n _kin value sequence number, k=1,2...8, g (i-1, j) is the green value that pixel (i-1, j) is known, B (i-1, j) is the blue valve that pixel (i-1, j) inserts, Δ H _g(i, j) is the horizontal gradient value of green channel pixel (i, j), Δ V _g(i, j) is the VG (vertical gradient) value of green channel pixel (i, j), and ∑ represents cumulative operation, m _kand n _kfor integer, m _k∈ 2 ,-2 ,-2,0,0,2,2,2}, n _k∈ 2,0,2 ,-2,2 ,-2,0,2};

9d) inserted whole red value is revised according to following formula:

$\tilde{R}(i,j)=\tilde{G}(i,j)+\frac{\underset{k=1}{\overset{8}{\mathrm{\&Sigma;}}}{w}_k(R(i+m_k,j+n_k)-\tilde{G}(i+m_k,j+n_k))}{\underset{k=1}{\overset{8}{\mathrm{\&Sigma;}}}{w}_k}$

Wherein,

$w_k=\frac{1}{{[1+{(\tilde{G}(i,j)-\tilde{G}(i+m_k,j+n_k))}^2]}^{1/2}}$

for the revised red value of pixel (i, j),

for the revised green value of pixel (i, j), ∑ represents cumulative operation, w _kfor weights, k is w _k, m _kand n _kin value sequence number, k=1,2...8, R (i+m _k, j+n _k) be pixel (i+m _k, j+n _k) in the red value that inserts, m _kand n _kfor integer, m _k∈ 2 ,-2 ,-2,0,0,2,2,2}, n _k∈ 2,0,2 ,-2,2 ,-2,0,2};

9e) inserted whole blue valve is revised according to following formula:

$\tilde{B}(i,j)=\tilde{G}(i,j)+\frac{\underset{k=1}{\overset{8}{\mathrm{\&Sigma;}}}{w}_k(B(i+m_k,j+n_k)-\tilde{G}(i+m_k,j+n_k))}{\underset{k=1}{\overset{8}{\mathrm{\&Sigma;}}}{w}_k}$

Wherein,

$w_k=\frac{1}{{[1+{(\tilde{G}(i,j)-\tilde{G}(i+m_k,j+n_k))}^2]}^{1/2}}$

for the revised blue valve of pixel (i, j),

for the revised green value of pixel (i, j), ∑ represents cumulative operation, w _kfor weights, k is w _k, m _kand n _kin value sequence number, k=1,2...8, B (i+m _k, j+n _k) by being inserted (i+m in pixel _k, j+n _k) blue valve, m _kand n _kfor integer, m _k∈ 2 ,-2 ,-2,0,0,2,2,2}, n _k∈ 2,0,2 ,-2,2 ,-2,0,2}.

Step 10, Color correction

10a) adopt color space conversion formula that revised edge image is transformed into yuv space from rgb space, color space conversion formula is:

$\left\{\begin{array}{c}Y=0.299R+0.587G+0.114B\\ U=R-Y\\ V=B-Y\end{array}\right.$

Wherein, Y is the luminance matrix of entire image yuv space, and R, G and B are three access matrixs of red, green, blue of entire image rgb space, and U and V are two chrominance matrixs of entire image yuv space.

10b) in yuv space, the chrominance matrix U to image and V carry out respectively the medium filtering of 3 × 3 windows, and the chromatic value on each pixel is got the intermediate value of 9 values in these pixel 3 × 3 windows;

10c) image is transformed into rgb space from yuv space, and the color value that does not need interpolation in each pixel is replaced with to original color value corresponding in CFA image, color space conversion formula is:

$\left\{\begin{array}{c}R=U+Y\\ B=V+Y\\ G=(Y-0.299R-0.114B)/0.587\end{array}\right.$

Wherein, R, G and B are three access matrixs of red, green, blue of entire image rgb space, and Y is the luminance matrix of entire image yuv space, and U and V are two chrominance matrixs of entire image yuv space.

10d) judge whether the medium filtering of tertiary colo(u)r(s) degree matrix U and V, if complete, execution step (11), otherwise proceed to step 10a).

Step 11, output coloured image.

Below in conjunction with accompanying drawing 2, simulated effect of the present invention is described further.

1. emulated data

The test pattern that emulation is used is the 19th width coloured image in Kodak standard coloured image storehouse, and image size is 768 × 512 pixels, has three passages of red, green, blue, and each passage gray level is 256.

2. simulation evaluation index

The method of evaluating CFA image demosaicing effect is that a clear clean image sampling is become to the CFA image of Bayer type, and then carries out demosaicing emulation, obtains the coloured image after demosaicing, finally by evaluating with former figure contrast.

The evaluation of CFA image denoising gram effect is divided into two aspects of subjectivity and objectivity.Be mainly to weigh by the visual characteristic of human eye in the quality of subjective evaluation piece image denoising effect, if result images details keeps, the false look of appearance is less, illustrate that this demosaicing method is effective, otherwise effect is poor.Objectively evaluating the demosaicing effect quality of piece image, the present invention adopts conventional colored Y-PSNR (CPSNR) to weigh.

Making true color image is I, and image size is m × n × 3, and the coloured image after demosaicing is

.Colored Y-PSNR is calculated as follows:

$\mathrm{CPSNR}=10{\log }_{10}\frac{{255}^2}{\frac{1}{3\mathrm{mn}}\underset{k=1}{\overset{3}{\mathrm{\&Sigma;}}}\underset{i=1}{\overset{m}{\mathrm{\&Sigma;}}}\underset{j=1}{\overset{n}{\mathrm{\&Sigma;}}}{(\hat{I}(i,j,k)-I(i,j,k))}^2}$

Wherein, the evaluation index value that CPSNR is required calculating, the line number that m is image, the columns that n is image, the passage index that k is image, the index of the row and column that i and j are pixel, ∑ represents cumulative operation,

for the coloured image after demosaicing, I is original color image.

3. simulation result and analysis

Accompanying drawing 2 is prior art and demosaicing design sketch of the present invention, and wherein, accompanying drawing 2 (a) is former figure; Accompanying drawing 2 (b) is the design sketch of the CFA image demosaicing method based on edge direction interpolation; Accompanying drawing 2 (c) is demosaicing design sketch of the present invention.The image of accompanying drawing 2 (b) derives from patent " the CFA image demosaicing method based on edge direction interpolation " (applying date: on 07 22nd, 2011, application number: 201110207617.7, publication number: CN102254301).

Three width subgraphs in contrast accompanying drawing 2, the smooth region of the CFA image demosaicing method based on edge direction interpolation can found out to image and thick fringe region interpolation effect are fine, but the yellow patch not having in the former figure of a fritter has appearred in the intensive place of hedge in the drawings; The present invention has recovered the information of former figure well, does not occur false color block and distortion, still fine to the interpolation effect of fine edge.Aspect objective evaluation, adopt the computing formula of evaluation index, the CPSNR value that calculates the CFA image demosaicing method of prior art based on edge direction interpolation is 40.75dB, CPSNR value of the present invention is 41.16dB.Objective evaluation index CPSNR of the present invention is higher than the CPSNR of prior art, illustrates that demosaicing effect of the present invention will get well.Contrast two width result figure and evaluation indexes thereof, the effect that can find out the recovery of the present invention to image texture region is very good, overcome false look easily appears in general demosaicing method shortcoming at image fine edge place, be particularly useful for the more CFA image of texture.

Claims (9)

1. the Bayer type CFA image demosaicing method based on non-local mean, comprises the steps:

(1) input one width is treated the Bayer type CFA image of demosaicing;

(2) adopt brightness-formula estimated brightness;

(3) treat that in input the image block of getting 5 × 5 pixel sizes by pixel in the Bayer type CFA image of demosaicing is as the current image block for the treatment of demosaicing;

(4) in the neighborhood of current 33 × 33 pixel sizes for the treatment of demosaicing image block, extract three groups of image set of blocks;

The concrete grammar of described extraction image block set is as follows:

4a) centered by the central pixel point of the current image block for the treatment of demosaicing, get the search window of a size as 33 × 33 pixels;

Pixel 4b) being listed as from the 3rd row the 4th in search window, get a pixel as central pixel point with column direction every a pixel in the row direction respectively, centered by this pixel, get the image block of 5 × 5 pixel sizes, altogether obtain 15 × 14 totally 210 image blocks, using these image blocks as first group of image set of blocks;

Pixel 4c) being listed as from the 4th row the 3rd in search window, get a pixel as central pixel point with column direction every a pixel in the row direction respectively, centered by this pixel, get the image block of 5 × 5 pixel sizes, altogether obtain 14 × 15 totally 210 image blocks, using these image blocks as second group of image set of blocks;

Pixel 4d) being listed as since the 4th row the 4th in search window, get a pixel as central pixel point with column direction every a pixel in the row direction respectively, centered by this pixel, get the image block of 5 × 5 pixel sizes, altogether obtain 14 × 14 totally 196 image blocks, using these image blocks as the 3rd group of image set of blocks;

(5) weight of computed image piece

5a) adopt gaussian kernel range formula to calculate each image block and the current gaussian kernel distance for the treatment of demosaicing image block in three groups of image set of blocks;

5b) in every group of image set of blocks in three groups of image set of blocks, by gaussian kernel distance is ascending, all image blocks of this group are sorted, get front 5 image blocks as similar image piece, give up remaining image piece;

5c) adopt weight calculation formula to calculate the weight of 5 similar image pieces in every group of image set of blocks;

(6) adopt weighted mean formula to be weighted on average the similar image piece in three groups of image set of blocks;

(7) value in weighted image piece is according to pixels inserted into current treating in demosaicing image block in locus correspondingly;

(8) judged whether the interpolation of all pixels, if complete, execution step (9), otherwise proceed to step (3);

(9) with edges by trimming correction method to interpolation after image carry out edge correction;

(10) Color correction

10a) adopt color space conversion formula that revised edge image is transformed into yuv space from rgb space;

10b) in yuv space, the chrominance matrix U to image and V carry out respectively the medium filtering of 3 × 3 windows;

10c) image is transformed into rgb space from yuv space, and the color value that does not need interpolation in each pixel is replaced with to original color value corresponding in CFA image;

10d) judge whether the medium filtering of tertiary colo(u)r(s) degree matrix U and V, if complete, execution step (11), otherwise proceed to step 10a);

(11) output coloured image.

2. the Bayer type CFA image demosaicing method based on non-local mean according to claim 1, it is characterized in that: the Bayer type CFA image described in step (1) is a kind of mosaic image, in image only there is a color value in red, green, blue three primary colours in each pixel, this color value does not need interpolation, two other color value disappearance, need to adopt demosaicing method to carry out interpolation.

3. the Bayer type CFA image demosaicing method based on non-local mean according to claim 1, is characterized in that: the illumination estimate formula described in step (2) is as follows:

$L=A{(H^T{\mathrm{\&sigma;}}^{-1}H+{\mathrm{\&lambda;}}_1{M}_1^T{M}_1+{\mathrm{\&lambda;}}_2{M}_2^T{M}_2)}^{-1}{H}^T{\mathrm{\&sigma;}}^{-1}{I}_S$

Wherein, the luminance matrix that L is required estimation, A is the transition matrix that RGB coloured image is converted into luminance matrix, and H is the decimation factor that RGB color picture sampling becomes Bayer form A FA image, and T is matrix transpose operator, and subscript-1 is matrix inversion operation symbol, σ, λ ₁and λ ₂for constant, its value is respectively: σ=0.00001, λ ₁=0.0008, λ ₂=0.02, M ₁and M ₂for filtering matrix,

, I ₃be 3 × 3 size unit matrixes, S ₁for Hi-pass filter [0.2 ,-0.5,0.65 ,-0.5,0.2],

represent Kronecker (Kronecker) operator, M ₂obtained by following formula computing:

$M_2=\left(\begin{array}{ccc}1.547& -0.577& -0.577\\ -0.577& 1.547& -0.577\\ -0.577& -0.577& 1.547\end{array}\right)\&CircleTimes;S_2$

S ₂for Hi-pass filter [0.5,1 ,-0.5], I _sfor the CFA image of input.

4. the Bayer type CFA image demosaicing method based on non-local mean according to claim 1, it is characterized in that: the method for getting image block by pixel described in step (3) is: from the pixel of the 17th row the 17th row, till the pixel of the 17th row reciprocal the 17th row reciprocal, each pixel is all got to size as the image block of 5 × 5 pixel sizes is as the current image block for the treatment of demosaicing centered by this pixel.

5. the Bayer type CFA image demosaicing method based on non-local mean according to claim 1, is characterized in that: step 5a) described in gaussian kernel range formula be:

$d_{k,i}={\left|\right|G\&CenterDot;(y_{k,i}^L-y_0^L)\left|\right|}_2^2$

Wherein, d _{k, i}i image block and the current distance for the treatment of between demosaicing image block in k group image set of blocks, k is image block set numbers k=1, image block index in 2,3, i presentation video set of blocks,

represent two norms square, G is gaussian kernel matrix, represent point multiplication operation, be the vector representation of i image block brightness in k group image set of blocks, the brightness of subscript L presentation video piece,

for the current vector representation for the treatment of the brightness of demosaicing image block.

6. the Bayer type CFA image demosaicing method based on non-local mean according to claim 1, is characterized in that: step 5c) described in weight calculation formula be:

$w_{k,j}=\frac{1}{Z(k,j)}\exp (-\frac{d_{k,j}}{h^2}),Z(k,j)=\underset{j}{\mathrm{\&Sigma;}}\exp (-\frac{d_{k,j}}{h^2})$

Wherein, w _k,jbe the weight of j similar image piece in k group image set of blocks, k is similar image set of blocks sequence number k=1, and 2,3, j represents the index j=1 of 5 similar image pieces in similar image set of blocks, and 2,3,4,5, Z (k, j) is normalization coefficient, d _k,jbe j similar image piece and the current gaussian kernel distance for the treatment of between demosaicing image block in k group image set of blocks, constant h is smoothing parameter, and value is 10, and ∑ represents cumulative operation.

7. the Bayer type CFA image demosaicing method based on non-local mean according to claim 1, is characterized in that: the weighted mean formula described in step (6) is:

${\stackrel{\&OverBar;}{y}}_k=\underset{j}{\mathrm{\&Sigma;}}{w}_{k,j}{y}_{k,j}$

Wherein,

be image block after the weighting of k group similar image set of blocks, k is similar image set of blocks sequence number k=1,2,3, and ∑ represents cumulative operation, the index j=1 of 5 similar image pieces in j presentation video set of blocks, 2,3,4,5, w _k,jbe the weight of j similar image piece in k group similar image set of blocks, y _k,jit is the vector representation of j similar image piece in k group similar image set of blocks.

8. the Bayer type CFA image demosaicing method based on non-local mean according to claim 1, it is characterized in that: described in step (7), to the current method for the treatment of demosaicing image block interpolation be: the value in weighted image piece is according to pixels inserted into current treating in demosaicing image block in position correspondingly, obtain the three-dimensional matrice of 5 × 5 × 4, wherein each pixel has a red value, two green values and a blue valve; The average that the final green value of each pixel is got two green values, obtains the coloured image piece after interpolation.

9. the Bayer type CFA image demosaicing method based on non-local mean according to claim 1, it is characterized in that: the edges by trimming correction method described in step (9) is: to the pixel inserting in green channel calculated level and VG (vertical gradient) respectively, if horizontal gradient is less than VG (vertical gradient), the green value of this pixel is carried out to horizontal neighbors weighting correction, if horizontal gradient is greater than VG (vertical gradient), the green value of this pixel is carried out to vertical neighborhood weighting correction, if horizontal gradient equals VG (vertical gradient), the green value of this pixel is carried out to horizontal and vertical neighborhood weighting correction, red value to interpolating pixel point and blue valve all carry out horizontal and vertical neighborhood weighting correction.

Images