CN100459718C - Method and device for decoding mosaic of color filter array picture - Google Patents
Method and device for decoding mosaic of color filter array picture Download PDFInfo
- Publication number
- CN100459718C CN100459718C CNB2004100973248A CN200410097324A CN100459718C CN 100459718 C CN100459718 C CN 100459718C CN B2004100973248 A CNB2004100973248 A CN B2004100973248A CN 200410097324 A CN200410097324 A CN 200410097324A CN 100459718 C CN100459718 C CN 100459718C
- Authority
- CN
- China
- Prior art keywords
- color
- pixel
- value
- object pixel
- processing window
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Abstract
The method includes following steps: inputting a great number of image samples at data training stage in order to make statistic analysis; comparing with corresponding true color image, the optimal result is recorded in database; inquiring at data base in data application stage, and rebuilding up color filter array image. By using the invention, the result of relieving mosaic is fine, and the requirement for hardware is low.
Description
Technical field
The present invention is relevant for image processing (image processing) method and apparatus, particularly relevant for a kind of in digital image collection system (digital image acquisition system), color filter array (color filter array checks colors, CFA) pixel in the image (pixel) is separated the method and apparatus of mosaic (demosaicking).
Background technology
In recent years, the fast development of digital image acquisition technology, people generally utilize digital camera (digital stillcamera, DSC), (digital video DV) takes pictures with the image capturing system of scanner etc. and writes down the process of life digital camera.Yet write down a number word image and need pass through many handling procedures, comprise white balance adjustment (white balance adjustment), gamma correction (gamma correction), image compression (compression) etc., one of them very important part is interpolation (interpolation), the reconstruction (reconstruction) of color filtration array image, perhaps can be described as and separates mosaic.
The color filtration array has many patterns (pattern), modal array is bayer color filter array (bayer CFA), filter out red (red), green (green) and blue (blue) three kinds of colors, other has the color filtration array of other sample to filter out viridescent (cyan), pinkish red (magenta), yellow (yellow) and green four kinds of colors.Below, be without loss of generality, be that example is done explanation with the bayer color filter array.
Each pixel (pixel) of a coloured image needs three kinds of base color at least, could make up back its color originally.General computer picture all can use red, green usually and blue three-color as base color.If present the real phenomena of a scenery, need to use three inductors to go to write down the value of at least three colors of each pixel respectively at least.In order to reduce hardware size and cost consideration, most image capturing system only uses single the inductor that has the color filtration array.But the color filtration array only allows to have only on each pixel a kind of color can be recorded on the inductor, and this represents that this image capturing system must rebuild (reconstruct) other color that each pixel lacked.This program is the so-called mosaic of separating.
Proposed bayer color filter array sample since 1976, be suggested with regard to a variety of methods of separating mosaic are arranged.Be generally divided into two classes, the first kind is not done any detection, and same set of mathematical formulae cover is used each pixel.Second class is to detect between this pixel and pixel on every side according to some geometrical patterns whether the example edition of edge (edge) is arranged, if having, can this pixel be applied mechanically the different mosaic methods of separating at the difference of edge example edition, calculates other color that this pixel lacks.
The first kind is not done any detection.The simplest method is exactly the interpolation of bilinearity (bilinear), has object pixel around utilizing and lacks other color pixel group, and doing on average according to color its numerical value promptly is the value of other color of object pixel.Though this method is calculated very fast and is easy to and carries out, the bilinear interpolation bad accessory when much separating mosaic of can deriving, for example: the color of distortion and make the edge blurryization between object in the image.
The method of Pei be proposed 2000 Christian eras (" Effective color interpolation in CCDcolor filter array using signal correlation; " IEEE Image Processing), utilize Kr in the real world to equal the difference of G and R, the value that Kb equals the difference of G and B is that constant is a theoretical foundation, derives the mathematical formulae of two colors in addition that is short of.
In No. 4642678 patent documentation of the U.S., Cok has proposed a kind of between different Color plane (color plane), simple spectrum (spectrum) association phenomenon, in a topography zone, it is red/and green, indigo plant/green color ratio (color ratios) almost is very close phenomenon.This observe phenomena was applied in a lot of methods afterwards, designed the auxiliary color that is lacked of calculating by other planes of color.Except color ratio, a lot of methods are also utilized similar idea, for example utilize difference between different colours (that is red green difference, or bluish-green difference).Yet above-mentioned these methods can't be handled finely for edge very sharp keen in the image and trickle part.
In 2002, people such as Gunturk (" Color plane interpolation using alternatingprojections; " IEEE Trans.Image Processing) a kind of efficient method is proposed, utilize originally and limit set (constraint set) and the previous spectrum association of carrying on the color filtration array image, (alternating projection) estimates two limitations set by alternating projection, utilizes limitations set to calculate other color that each pixel is short of again.In 2003, people such as Lu (" On new method andperformance measures for color filter array; " IEEE Trans.Image Processing) another kind of method is proposed, intermediate point pixel and relation are on every side divided, down, a left side, right four direction is considered, if the difference of this pixel and this direction same color pixel is bigger, then the association on the space is more little, derives the mathematical formulae of other color that is short of with this.In addition, the method for Lu is also used and is adjusted mediant filtration method (adaptive median filtering) and do reprocessing.
Disclose in No. 20030215159 at american documentation literature, people such as Okuno disclose a kind of pixel interpolation devices, comprise an interpolation pattern table (interpolation pattern table), this table output one interpolation bearing data is with appointment interpolation direction, and each inter polated pixel data computing is according to the pixel data that drops on the specified interpolation direction of interpolation bearing data.
Second class detects the edge example edition.In 1986, Cok (in No. 4630307 patent documentation of the U.S.) discloses a kind of pattern recognition (pattern recognition, PR) the mosaic method of separating at first provides a plurality of different interpolation formulas producing suitable interpolation signal value, and finishes corresponding a plurality of several picture feature.Then, have or not between identification object pixel and its surrounding pixel to meet some several picture features, for example edge, line (line) and corner (corner).If meet one of them several picture feature, then use its interpolation formula that is suitable for to produce the interpolation signal value.All incongruent words then use easy method such as bilinearity interpolation to estimate other color value that is lacked.In general, by exploring the association between the pixel of surrounding, space, these methods all can help the color interpolation effect along the image border, surpass the algorithm that strides across the edge.For example the method for the marginal classification that proposed of people's (No. 5373322 patent documentation of the U.S.) such as Larochen based on marginal classification, picks out preferable calculated direction, other color value that obtains being lacked.
In addition, in No. 5629734 patent documentation of the U.S., people such as Hamilton have proposed a kind of device of separating mosaic, utilize Laplce's second order value (Laplacian second-order values) and Grad (gradient values) to be used as the instrument of rim detection, to obtain preferable interpolation orientation (preferredorientation for interpolation).
In addition, some adopts mixed mode to do the action of separating mosaic, reaches to have sharper keen edge and reduce the picture quality of separating the bad accessory that mosaic causes.In 1999, Kimmel (" Demosaicing:image reconstruction from color CCD samples; " IEEETrans.Image Processing) use a kind of skill of edge guiding to go other color that obtains being lacked, its practice is by some edge identifications, the ratio proportion of interpolation four color of pixel values or color around it also uses the program of counter diffusion (inverse diffusion) to suppress to separate the bad accessory that mosaic causes simultaneously.In calendar year 2001, people such as Li (" New edge-directed interpolation; " IEEE Trans.ImageProcessing) discloses a kind of method that detects the edge example edition, conciliate covariance (covariance) between high resolution graphic behind the mosaic by the color filtration array image of low resolution, utilize dual (duality) and edge guiding its how much, and the interplanar difference of different colours does interpolation, calculates other color value that is lacked.
Separate the method for mosaic or install too numerous to enumerate, its main purpose be exactly have an optimum picture quality separate the mosaic picture, but how to spend the cost of minimal hardware expenditure, be only the most practical method and apparatus of separating mosaic.
Summary of the invention
The present invention is for realizing a kind of mosaic method of separating of reaching optimum picture quality and cost minimal hardware expenditure cost.Its main purpose provides a kind of in digital image collection system, and the color filter array picture of checking colors is separated the method for mosaic.
The method comprises two stages, and one is data training stage (data training phase), and another is data application stage (data practice phase).The data training stage mainly comprises the following step: (a1) prepare a plurality of color filtration array image samples and corresponding full-color (full-color) image thereof, each pixel in each full-color image comprises at least three kinds of color actual values.(a2) to each target (target) pixel in each color filtration array image sample, taking out a corresponding processing window (concentrative window) and quantizing (quantize) these all pixels of handling in the window becomes one group of sample index (pattem index).(a3) according to color of pixel value and corresponding sample index in the corresponding processing window of each object pixel, use many feasible coefficient combinations of group (possible coefficient set) to draw each color reconstructed value that each object pixel is lacked, to calculate each object pixel again each the color reconstructed value that lacked and error (error) value between the actual value.(a4) determine the best (optimal) coefficient combination that each group sample index is corresponding with its each color that is lacked, and export a database (database) to.
In application stage, mainly comprise the following step in data: (b1) import color filter array picture of the same colour, at each object pixel, all pixels of taking out in a corresponding processing window and the quantification treatment window become one group of sample index.(b2) according to each color that each object pixel lacked, import its corresponding sample index database so far, to obtain the optimum coefficient combination of each color correspondence that each object pixel lacked, according to the color of pixel value in the corresponding processing window of object pixel, calculate each color reconstructed value that each object pixel lacks again.
Wherein, this quantification action in the step (a2) is that the brightness that will handle the pixel of all same colors in the window is respectively compared mutually, again with after the big or small binarization of the relative brightness of each pixel with at least one bit representation; This processing window comprises object pixel in the interior regional extent that limited pixel contained in the step (a2) around this object pixel; In the step (a3) with step (b2) in pixel in the corresponding processing window of this object pixel be whole pixels in the regional extent that contained of corresponding processing window; In the step (a3) with step (b2) in the pixel of corresponding processing window of this object pixel be part pixel in the regional extent that contained of corresponding processing window; In the step (a3) with step (b2) in this color value be the primitive color value; In the step (a3) with step (b2) in this color value be the primitive color value, the color reconstructed value of one or more that collocation reconstructs earlier in the process of reconstruction; In the step (a3) in this processing window the reconstructed value of the color that object pixel lacked be the combination of feasible coefficient with the corresponding processing window of object pixel in the function of primitive color value of pixel; In the step (a3) in this processing window the reconstructed value of the color that object pixel lacked be the function of the color value that reconstructs earlier in the primitive color value of the pixel in the corresponding processing window of the combination of feasible coefficient, object pixel and the process of reconstruction; Step (a4) but in this decision action be that the feasible coefficient sets of Select Error value minimum is combined into optimum coefficient combination from these many group row coefficients; This error amount in the step (a3) is a distance measure of the difference between two images of expression.
Another object of the present invention provides the above-mentioned device of separating mosaic.This device comprises a sampler, a quantizer, an Error Calculation and selector, an archival memory and an image reconstructor.Sampler to each object pixel in each color filtration array image, is got a corresponding processing window according to a plurality of color filtration array images of input, takes out the primitive color value of handling all pixels in the window again.The primitive color value of all pixels in quantizer reception and the quantification treatment window produces one group of sample index then.Error Calculation and selector, receive the primitive color value of all pixels in the processing window and the sample index that quantizer produces, and according to the full-color image of importing that corresponds to this color filtration array image, the feasible coefficient combination of the many groups of Error Calculation and selector utilization is to draw each color reconstructed value that each object pixel is lacked, calculate again each color reconstructed value and actual value between error amount, among many groups of feasible coefficient combination cumulative errors values corresponding, determine and produce this group sample index to lack the corresponding optimum coefficient combination of each color then with object pixel with it.
Archival memory receives and to calculate the sample index that produces with selector with memory error and lack the corresponding optimum coefficient of each color with it and makes up, and according to the sample index of quantizer generation, the optimum coefficient that the generation object pixel lacks each color correspondence makes up.The optimum coefficient that image reconstructor is then lacked each color correspondence according to the object pixel that is produced by archival memory makes up the color of pixel value in the corresponding processing window with object pixel, produces the reconstructed value of each color that each object pixel lacks.
Wherein, the pixel in the corresponding processing window of this object pixel of this image reconstructor institute basis is the interior whole pixels of regional extent that corresponding processing window is contained; The partial pixel of the corresponding processing window of this object pixel of this image reconstructor institute basis is the interior part pixel of regional extent that corresponding processing window is contained; But this Error Calculation and selector are from these many group row coefficients, and the feasible coefficient sets of Select Error value minimum is combined into optimum coefficient combination; This error amount is a distance measure of the difference between two images of expression; This archival memory is a recordable media; This archival memory is a random access memory or a read-only memory or a flash card or a CD; This color value of this image reconstructor institute foundation is the primitive color value; This image reconstructor is also according to this primitive color value, and the color reconstructed value that collocation has produced is to rebuild uncreated other color reconstructed value.
The characteristic of maximum of the present invention is to utilize the hardware cost that lacks than prior art to implement, and obtains approaching effect.And the tranining database of statistics is to store with storage device (storage device).The user can change statistical, also can adjust according to circumstances or new database more.In addition, database is not limited to unique one group, can adjust required database according to circumstances, very flexible and variation.
Detailed description and claims of now cooperating following accompanying drawing, embodiment, will on address other purpose of the present invention and advantage and be specified in after.
Description of drawings
Fig. 1 be the present invention in digital image collection system, the color filter array picture of checking colors is separated the flow chart of mosaic method.
Fig. 2 A is according to the present invention, color filtration array image and the graph of a relation of handling window.
Fig. 2 B is sample index, the green feasible coefficient combination and error amount relation table of object pixel.
Fig. 2 C is sample index, the red feasible coefficient combination and error amount relation table of object pixel.
Fig. 3 is a structure chart of separating the mosaic device according to of the present invention.
Fig. 4 A～Fig. 4 B is difform processing window.
Fig. 5 separates the PSNR value that the mosaic method is finished for comparing five kinds of differences.
Fig. 6 illustrates and uses five kinds of differences to separate the required hsrdware requirements table of mosaic method.
Wherein, description of reference numerals is as follows:
101 prepare a plurality of color filtration array image samples and corresponding full-color image thereof, and each pixel in each full-color image comprises at least three kinds of color actual values;
102 to each object pixel in each color filtration array image sample, and taking out a corresponding processing window and quantizing these all pixels of handling in the window becomes one group of sample index;
103 according to color of pixel value and corresponding sample index in the corresponding processing window of each object pixel, use the feasible coefficient combination of many groups to draw each color reconstructed value that each object pixel is lacked, to calculate each object pixel again each the color reconstructed value that lacked and the error amount between the actual value;
The optimum coefficient combination that each group sample index of 104 decisions is corresponding with its each color that is lacked, and export a database to;
105 inputs color filter array picture of the same colour at each object pixel, takes out a corresponding processing window and all pixels of quantizing in this processing window become one group of sample index;
106 according to each color that each object pixel lacked, import its corresponding sample index database so far, to obtain the optimum coefficient combination of each color correspondence that each object pixel lacked, according to the color of pixel value in the corresponding processing window of object pixel, calculate each color reconstructed value that each object pixel lacks again.
201 color filtration array images 202 are handled window
The red pixel that 203 object pixels 204 are handled in the window
205 handle the blue pixel that the green pixel 206 in the window is handled in the window
The set 208 original sample index of the luminance bit of 207 all pixels
The 212 green feasible coefficient combinations of 211,221 sample index
The 222 red feasible coefficient combinations of 213,223 error amounts
300 devices of separating mosaic of the present invention
301 samplers, 302 quantizers
Embodiment
Fig. 1 be the present invention in digital image collection system, the color filter array picture of checking colors is separated the flow chart of mosaic method.The present invention divides two stages: data training stage 100 and data application stage 110.The input great amount of images sample statistics that performs an analysis in the data training stage 100, and contrast its corresponding full-color image, optimum is recorded in database.Query Database in the data application stage 110 is rebuild the color filtration array image.
With reference to figure 1, in the training stage, comprise the following step in data: at first, prepare a plurality of color filtration array image samples and corresponding full-color image thereof, each pixel in each full-color image comprises at least three kinds of color actual values, shown in step 101.Then, to each object pixel in each color filtration array image sample, taking out a corresponding processing window and quantizing these all pixels of handling in the window becomes one group of sample index, shown in step 102.Then, according to primitive color value of the pixel in the corresponding processing window of each object pixel (the color reconstructed value that reconstructs earlier in can also primitive color value collocation process of reconstruction) and corresponding sample index, use the feasible coefficient combination of many groups to draw each color reconstructed value that each object pixel is lacked, each color reconstructed value that calculates each object pixel again and lacked and the error amount between the actual value are shown in step 103.At last, determine the optimum coefficient combination that each group sample index is corresponding with its each color that is lacked, and export a database to, shown in step 104.
The present invention comprises following two steps in the data application stage.In step 105, import color filter array picture of the same colour, at each object pixel, take out in a corresponding processing window and the quantification treatment window all pixels and make and become one group of sample index.In step 106, according to each color that each object pixel lacked, import its corresponding sample index database so far, to obtain the optimum coefficient combination of each color correspondence that each object pixel lacked, according to the primitive color value of the pixel in the corresponding processing window of object pixel (the color reconstructed value that reconstructs earlier in can also primitive color value collocation process of reconstruction), calculate the color value that each object pixel lacks again.
Fig. 2 A is color filtration array image and the graph of a relation of handling window.Fig. 2 B is sample index, the green feasible coefficient combination and error amount relation table of object pixel 203 among Fig. 2 A.Fig. 2 C is sample index, the red feasible coefficient combination and error amount relation table of object pixel 203 among Fig. 2 A.
With reference to figure 2A, in step 102, in color filtration array image 201, appoint and get the fixing processing window 202 of a block size, in handling window 202 scopes, to handle the pixel (as label 204,205, shown in 206) of all same colors in the window 202 respectively, compare brightness (intensity) size of its relativity mutually, again each amount of pixels is turned at least one luminance bit.With luminance bit is that a position is an example, and 204 size between red pixel in the comparison process window 202 then can be quantified as red pixel R1, R2, R3 and R4 R1=R2=R3=L (being logic low value)=0, R4=H (being logic-high value)=1.205 size between green pixel in the comparison process window 202 then can be quantified as G1=G2=L=0 with green pixel G1, G2, G3 and G4, G3=G4=H=1.206 size then can be quantified as B1=B2=B5=L=0 with blue pixel B1, B2, B3, B4 and B5, B3=B4=H=1 between comparison process window 202 Smalt pixels.This set 207 of handling the luminance bit of all pixels in window promptly is one group of original sample index 208 (promptly 0000000110111).
According to the present invention, the reconstructed value of handling the color that object pixel lacked in the window is the function that feasible coefficient makes up the primitive color value of the pixel in the corresponding processing window with object pixel, that is the reconstructed value=F of the color that object pixel lacked (feasible coefficient combination, the primitive color value of the pixel in the corresponding processing window of object pixel), the pixel of wherein handling window can only contain partly pixel, or is considered as partly the coefficient of pixel and forces to be made as zero.It is on duty with corresponding feasible coefficient combination below to be that the reconstructed value of the color that lacked with object pixel equals the primitive color of the pixel in the corresponding processing window, explains divided by the coefficient summation again.
In step 103, because the color that object pixel 203 is lacked is green and red, in the relation table of Fig. 2 B, according to a plurality of green reconstructed value (demonstration in the table) that object pixel 203 corresponding sample index (000001011) (this example is omitted B1, B2, B3 and B4) are used the feasible coefficient combination of many groups 212 (only using the combination of G1, G2, G3 and G4 in this example) to draw object pixel respectively to be lacked, calculate the error amount 213 between the green actual value of corresponding pixel among green reconstructed value and the full-color figure again.The rest may be inferred, in the relation table of Fig. 2 C, according to a plurality of red reconstructed value (demonstration in the table) that object pixel 203 corresponding sample index (000001011) use the feasible coefficient combination of many groups 222 (only using the combination of R1, R2, R3 and R4 in this example) to be lacked to draw object pixel, calculate the error amount 223 between the red actual value of corresponding pixel among red reconstructed value and the full-color figure again.Error amount is wherein represented the distance measure (possible distance metric) of two differences between image.Mean square deviation (mean squared error) value or root-mean-square deviation (root mean squared error) value etc. for example.
In step 104, select the optimum coefficient combination of minimum error values, therefore, the corresponding green optimum coefficient of this sample index (000001011) is combined as (1100), and red optimum coefficient is combined as (2110), and is recorded in a database.
In step 105, if the object pixel in the color filtration array image of input is originally as blue and its corresponding sample index (000001011).In step 106, be used as input with sample index (000001011) and remove Query Database, the pixel that obtains in the corresponding processing window of object pixel multiply by corresponding coefficient, again divided by the coefficient summation, the green reconstructed value that can obtain object pixel equals 1/2*G1+1/2*G2 and red reconstructed value equals 1/2*R1+1/4*R2+1/4*R3 (with reference to figure 2A).
Below with an example, describe the steps flow chart among Fig. 1 of the present invention in detail.The data training stage:
The a large amount of color filtration array image samples prepared in step 101 and corresponding full-color image thereof are the statistics usefulness that performs an analysis.In step 102, the sample index after the quantification is to use the foundation of feasible coefficient combination.In step 103, basically take the fixedly method of colourity interpolation (constant-hue-basedinterpolation), calculate the pixel that all lack the first color C1 earlier, its pixel that has first color value up and down multiply by various feasible coefficient combinations, again divided by the resulting first color reconstructed value C1r of coefficient summation, contrast the mean square deviation that the first color actual value of corresponding pixel among the full-color figure is calculated this pixel.Record object pixel and the luminance bit of 8 pixel groups (being sample index) simultaneously on every side make up resulting mean square deviation with various different feasible coefficients.
Then, at the pixel that lacks the second color C2, have the K of second colored pixels around calculating originally
C2The difference of C1r and C2 just, multiply by various feasible coefficient combinations, again divided by the coefficient summation, obtain the difference of the C1 and the C2 of object pixel, contrast original C1 of corresponding pixel among the full-color figure again or lack the C1r that the back rebuilds and obtain the second color reconstructed value C2r, and make comparisons, calculate mean square deviation with the difference of the second color actual value of full-color image.Simultaneously record object pixel and the luminance bit of 8 pixel groups (being sample index) on every side, with various different feasible be that array closes resulting mean square deviation.As at the pixel that lacks the 3rd color C3, have the K of the 3rd colored pixels around calculating originally
C3The difference of C1 and C3 just, multiply by various feasible coefficient combinations, again divided by the coefficient summation, obtain the difference of the C1r and the C3 of object pixel, contrast original C1 of corresponding pixel among the full-color figure again or lack the C1r that the back rebuilds and obtain the 3rd color reconstructed value C3r, and make comparisons, calculate mean square deviation with the difference of the 3rd color actual value of full-color image.Record object pixel and the luminance bit of 8 pixel groups (being sample index) simultaneously on every side make up resulting mean square deviation with various different feasible coefficients.
By the time after importing great amount of images and having done statistics, in step 104, with object pixel and the luminance bit of 8 pixel groups (being sample index) on every side, the numerical value of optimum coefficient combination of corresponding its accumulative total mean square deviation minimum is put into database.
The data application stage:
In step 105, import color filter array picture of the same colour, at each object pixel, taking out a corresponding processing window and quantizing these all pixels of handling in the window becomes one group of sample index.
In step 106, sample index is used as the input of database, Query Database, and its pixel that has first color value up and down multiply by optimum coefficient combination corresponding in the database, again divided by the coefficient summation, the first color reconstructed value C1r that can obtain being short of.Then, sample index is used as the input of database, Query Database, the difference of the C1r and the C2 of the second color value pixel will be had around the object pixel originally, multiply by optimum coefficient corresponding in the database, divided by the coefficient summation, obtain the difference of C1r and C2, and the second color reconstructed value C2r that object pixel lacked equals C1 (r)-(C1-C2) again.At last, sample index is used as the input of database, Query Database, the difference of the C1r and the C3 of the 3rd color value pixel will be had around the object pixel originally, multiply by optimum coefficient combination corresponding in the database, divided by the coefficient summation, obtain the difference of C1 and C3, and the 3rd color reconstructed value C3r that object pixel lacked equals C1 (r)-(C1-C3) again.
Therefore, according to the present invention, handling the color value that object pixel lacked in the window also is feasible coefficient combination, the function of the color value that reconstructs earlier in the pixel process of reconstruction in the primitive color value of the pixel in the corresponding processing window of object pixel and the corresponding processing window of object pixel, wherein handling the pixel of window can optionally choose partly important, or force to be made as the coefficient of pixel partly zero, that is the reconstructed value=F of the color that object pixel lacked (feasible coefficient combination, the primitive color value of the pixel in the corresponding processing window of object pixel, the color value that reconstructs earlier in the pixel process of reconstruction in the corresponding processing window of object pixel), the pixel of wherein handling window can only contain partly pixel, or is considered as partly the coefficient of pixel and forces to be made as zero.
Fig. 3 is a structural representation of separating the mosaic device of the present invention.With reference to figure 3, the invention provides a kind of in digital image collection system, the color filter array picture of checking colors is separated the device 300 of mosaic, and this device comprises a sampler 301, a quantizer 302, an Error Calculation and selector 303, an archival memory 304 and an image reconstructor 305.In data in the training stage: import a plurality of color filtration array images 306 to sampler 301, sampler 301 is got a corresponding processing window and is taken out the primitive color value of handling all pixels in the window each object pixel in each color filtration array image sample.Quantizer 302 receptions and quantification produce one group of sample index again by the primitive color value of all pixels in the processing window of sampler 301 outputs.
Error Calculation and selector 303, reception is by the primitive color value of all pixels in the processing window of sampler 301 taking-ups and the sample index of quantizer 302 generations, simultaneously, the full-color image 307 that corresponds to color filtration array image 306 according to input, the feasible coefficient combination of the many groups of Error Calculation and selector utilization is to draw each color reconstructed value that each object pixel is lacked, calculate the error amount between each color reconstructed value and the actual value again, continue the error amount of accumulative total great amount of samples image, among many groups of feasible coefficient combination error amounts corresponding with it, decision and the generation sample index optimum coefficient corresponding with each color that object pixel is lacked makes up then.Archival memory 304 receives this group of sample index that produces with this Error Calculation of storage and selector and lacks the corresponding optimum coefficient combination of each color with object pixel.
In data in the application stage: import a plurality of color filtration array images 306 to sampler 301, sampler 301 is got a corresponding processing window and is taken out the primitive color value of handling all pixels in the window each object pixel in each color filtration array image.Quantizer 302 receptions and quantification produce one group of sample index again by the primitive color value of all pixels in the processing window of sampler 301 outputs.Archival memory 304 produces the optimum coefficient combination that object pixel lacks each color correspondence according to the sample index that quantizer 302 produces.The optimum coefficient of each color correspondence that object pixel lacked that 305 bases of image reconstructor are produced by archival memory 304 makes up the color of pixel value in the corresponding processing window with object pixel, produce the reconstructed value of each color that each object pixel lacks, to rebuild the full-color image 308 of reconstruction of color filtration array image 306.
Wherein, archival memory 304 is a recordable media (record medium), for example, a random access memory (random access memory) or a read-only memory (read-only memory) or a flash card (flash card) or a CD (compact disc).
In addition, rebuild full-color image 308 among Fig. 3 to the dotted line key of image reconstructor No. 305, a kind of color reconstructed value that expression has produced, can select (optional) whether will be fed back to image reconstructor 305, adjust and rebuild uncreated other color reconstructed value again, this more increases the elasticity of account form of the present invention.
Fig. 4 A～Fig. 4 B is difform processing window.Wherein, the shape of handling window can be a different shape, for example square (shown in Fig. 4 A), rhombus or groined type (shown in Fig. 4 B) or the like.Different color filtration array images can correspond to difform processing window.In addition, among a kind of color filtration array, when handling different color (or even different scarce looks), can also correspond to different processing windows.
According to the present invention, the image pattern of data training also can appropriateness be screened: promptly optionally image pattern is added staqtistical data base.The auto-correlation (auto-correlation) of all processing windows is lower than a certain critical value (threshold) in a new image pattern, represent that this sample image is in the process of statistics, a lot of same group of sample index occur and corresponded to several feasible coefficient combinations, its mean square deviation numerical value is all similar, just the situation of disperseing very much.Or the cross-correlation of this sample image and other image (cross-correlation) is lower than a certain critical value, judges that then this sample image is a unique image, also do not add staqtistical data base.Because generally speaking this image all is that abundant or mixed and disorderly content are arranged, even this type of image is not added statistics, and the direct cover of database before is used in this type of image, effect is not bad after can not causing this type of image reorganization yet, because this type of picture material is abundant, human eye is difficult for discovering its slight change when watching.
The screening mechanism of another one lost generation (iterative) is as follows: in the process of statistics, if there have any sample image to cause PSNR (the peak signal-to-noise ratio) value of whole reconstructed image to reduce to be a lot, then this sample image does not add staqtistical data base yet.
Fig. 5 relatively uses five kinds to separate the PSNR value that the mosaic method is finished.With reference to figure 5, do five kinds of PSNR values comparisons of separating the mosaic method according to 12 pictures that the photograph CD of Kodak is provided, these five kinds of methods comprise Gunturk, Lu, BP, Pei and method used in the present invention.Wherein, (back projection, meaning BP) is, when each pixel tentatively obtains three color values, does further reprocessing, allows some noises or slide fastener shape effect in the image can reduce to minimum to annotate and comment on back projection among the figure.Utilize the BP mode, calculate Kr, the Kb value of each pixel of nine palace lattice, the value of nine palace lattice intermediate pixels is Kr ', the Kb ' value after surrounding pixel is averaged, utilize the color and Kr ', the Kb ' value of the intermediate pixel of sample image itself originally simultaneously, remove counter two color values in addition that intermediate pixel is short of originally that push away, can be compared with other two color values that tentatively obtain more near its actual value.Generally speaking, the value of PSNR is high more, represents the approaching more former figure of its figure.The mosaic effect of separating of the present invention as seen from Figure 5 adds that than other two kinds of Pei BP, PR add that BP is better near Gunturk, Lu.
Fig. 6 is to use five kinds of different required hsrdware requirements tables of the mosaic method of separating.Hsrdware requirements of the present invention as seen from Figure 6 are between Gunturk, Lu and Pei, PR.
By the demand of above-mentioned effect and hardware relatively, the present invention produces as can be seen separates the method that the mosaic picture effect presses on towards Gunturk and Lu, and the demand of hardware only exceeds than PR and Pei, also hardwareization easily.In addition, because the present invention all makes database to all pixel relationship to add up, database is not limited to unique one group, and adjusts required database according to circumstances, for example claps the personage, landscape can be applied mechanically different databases.Moreover the mode that the database of statistics can random access memory is stored, if the user has better statistical, all can be at any time new database more.Database not only can static state produce, and also can dynamically set up or dynamically adjustment of static state generation back adding.
Yet the above only is preferred embodiment of the present invention, when not limiting scope of the invention process with this.Promptly the equalization of doing according to claims of the present invention generally changes and modifies, and all should still belong in the scope that patent of the present invention contains.
Claims (17)
1. the method that the color filter array picture of checking colors in digital image collection system is separated mosaic is characterized in that, this method comprises a data training stage and a data application stage,
These data comprise the following step in the training stage:
(a1) prepare a plurality of color filtration array image samples and corresponding full-color image thereof, each pixel in each full-color image comprises at least three kinds of color actual values;
(a2) to each object pixel in each color filtration array image sample, take out a corresponding processing window, and all pixels that quantize in this processing window become one group of sample index, wherein quantizing action is that the brightness that will handle the pixel of all same colors in the window is respectively compared mutually, again with after the big or small binarization of the relative brightness of each pixel with at least one bit representation;
(a3) according to color of pixel value and corresponding sample index in the corresponding processing window of this object pixel respectively, use the feasible coefficient combination of many groups to draw each color reconstructed value that respectively this object pixel was lacked, again calculating each color reconstructed value that respectively this object pixel lacked and the error amount between the actual value; And
(a4) determine the optimum coefficient combination that each group sample index is corresponding with its each color that is lacked, and export a database to, but wherein the decision action is that the feasible coefficient sets of Select Error value minimum is combined into optimum coefficient combination from these many group row coefficients;
This data application stage comprises the following step:
(b1) import color filter array picture of the same colour,, take out a corresponding processing window and all pixels of quantizing in this processing window become one group of sample index at each object pixel; And
(b2) according to each color that each object pixel lacked, import its corresponding sample index to this database, to obtain the optimum coefficient combination of each color correspondence that each object pixel lacked, according to the color of pixel value in the corresponding processing window of this object pixel, calculate each color reconstructed value that each object pixel lacks again.
2. the method that the color filter array picture of checking colors in digital image collection system as claimed in claim 1 is separated mosaic, it is characterized in that, this processing window comprises object pixel in the interior regional extent that limited pixel contained in the step (a2) around this object pixel.
3. the method that the color filter array picture of checking colors in digital image collection system as claimed in claim 1 is separated mosaic, it is characterized in that, in the step (a3) with step (b2) in pixel in the corresponding processing window of this object pixel be whole pixels in the regional extent that contained of corresponding processing window.
4. the method that the color filter array picture of checking colors in digital image collection system as claimed in claim 1 is separated mosaic, it is characterized in that, in the step (a3) with step (b2) in the pixel of corresponding processing window of this object pixel be part pixel in the regional extent that contained of corresponding processing window.
5. the method that the color filter array picture of checking colors in digital image collection system as claimed in claim 1 is separated mosaic is characterized in that, in the step (a3) with step (b2) in this color value be the primitive color value.
6. the method that the color filter array picture of checking colors in digital image collection system as claimed in claim 1 is separated mosaic, it is characterized in that, in the step (a3) with step (b2) in this color value be the primitive color value, the color reconstructed value of one or more that collocation reconstructs earlier in the process of reconstruction.
7. the method that the color filter array picture of checking colors in digital image collection system as claimed in claim 1 is separated mosaic, it is characterized in that, in the step (a3) in this processing window the reconstructed value of the color that object pixel lacked be the combination of feasible coefficient with the corresponding processing window of object pixel in the function of primitive color value of pixel.
8. the method that the color filter array picture of checking colors in digital image collection system as claimed in claim 7 is separated mosaic, it is characterized in that, in the step (a3) in this processing window the reconstructed value of the color that object pixel lacked be the function of the color value that reconstructs earlier in the primitive color value of the pixel in the corresponding processing window of the combination of feasible coefficient, object pixel and the process of reconstruction.
9. the method that the color filter array picture of checking colors in digital image collection system as claimed in claim 1 is separated mosaic is characterized in that, this error amount in the step (a3) is a distance measure of the difference between two images of expression.
10. the color filter array picture of checking colors in digital image collection system is separated the device of mosaic, it is characterized in that comprising:
One sampler according to a plurality of color filtration array images of input, to each object pixel in each color filtration array image, is got a corresponding processing window, obtains the primitive color value of all pixels in this processing window;
One quantizer, receive and quantize the primitive color value of these all pixels, produce one group of sample index again, wherein quantizing action is that the brightness that will handle the pixel of all same colors in the window is respectively compared mutually, again with after the big or small binarization of the relative brightness of each pixel with at least one bit representation;
One Error Calculation and selector, receive primitive color value and this group of sample index of these all pixels, and according to the full-color image of importing that corresponds to this color filtration array image, utilize the feasible coefficient combination of many groups to draw each color reconstructed value that respectively this object pixel was lacked, calculate the error amount between each color reconstructed value and the actual value again, then among the feasible coefficient combination of these many groups error amount corresponding with it, determine and produce this group of sample index to lack the corresponding optimum coefficient combination of each color with this object pixel, wherein but this Error Calculation and selector are from these many group row coefficients, and the feasible coefficient sets of Select Error value minimum is combined into optimum coefficient combination;
One archival memory, reception lacks this corresponding optimum coefficient combination of each color with this group of sample index of storage and its, and according to this group of sample index, produces the optimum coefficient combination that this object pixel lacks each color correspondence; And
One image reconstructor according to the color of pixel value in the corresponding processing window of the optimum coefficient combination that is lacked each color correspondence by this object pixel and object pixel, produces the reconstructed value of each color that each object pixel lacks.
11. the color filter array picture of checking colors in digital image collection system as claimed in claim 10 is separated the device of mosaic, it is characterized in that the pixel in the corresponding processing window of this object pixel of this image reconstructor institute basis is the interior whole pixels of regional extent that corresponding processing window is contained.
12. the color filter array picture of checking colors in digital image collection system as claimed in claim 10 is separated the device of mosaic, it is characterized in that the partial pixel of the corresponding processing window of this object pixel of this image reconstructor institute basis is the interior part pixel of regional extent that corresponding processing window is contained.
13. the color filter array picture of checking colors in digital image collection system as claimed in claim 10 is separated the device of mosaic, it is characterized in that, this error amount is a distance measure of the difference between two images of expression.
14. the color filter array picture of checking colors in digital image collection system as claimed in claim 10 is separated the device of mosaic, it is characterized in that, this archival memory is a recordable media.
15. the color filter array picture of checking colors in digital image collection system as claimed in claim 10 is separated the device of mosaic, it is characterized in that, this archival memory is a random access memory or a read-only memory or a flash card or a CD.
16. the color filter array picture of checking colors in digital image collection system as claimed in claim 10 is separated the device of mosaic, it is characterized in that, this color value of this image reconstructor institute foundation is the primitive color value.
17. the color filter array picture of checking colors in digital image collection system as claimed in claim 16 is separated the device of mosaic, it is characterized in that, this image reconstructor is also according to this primitive color value, and the color reconstructed value that collocation has produced is to rebuild uncreated other color reconstructed value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2004100973248A CN100459718C (en) | 2004-11-26 | 2004-11-26 | Method and device for decoding mosaic of color filter array picture |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2004100973248A CN100459718C (en) | 2004-11-26 | 2004-11-26 | Method and device for decoding mosaic of color filter array picture |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1780404A CN1780404A (en) | 2006-05-31 |
| CN100459718C true CN100459718C (en) | 2009-02-04 |
Family
ID=36770486
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB2004100973248A Expired - Fee Related CN100459718C (en) | 2004-11-26 | 2004-11-26 | Method and device for decoding mosaic of color filter array picture |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN100459718C (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102630018B (en) * | 2012-03-27 | 2014-07-23 | 上海算芯微电子有限公司 | Method and device for demosaicking |
| WO2014007041A1 (en) * | 2012-07-06 | 2014-01-09 | 富士フイルム株式会社 | Image processing device and method, and imaging device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020063789A1 (en) * | 2000-11-30 | 2002-05-30 | Tinku Acharya | Color filter array and color interpolation algorithm |
| CN1398108A (en) * | 2001-07-18 | 2003-02-19 | 三洋电机株式会社 | Picture signal processor |
| US20030052981A1 (en) * | 2001-08-27 | 2003-03-20 | Ramakrishna Kakarala | Digital image system and method for implementing an adaptive demosaicing method |
| US20030117507A1 (en) * | 2001-12-21 | 2003-06-26 | Nasser Kehtarnavaz | Color filter array interpolation |
| US20040161145A1 (en) * | 2003-02-18 | 2004-08-19 | Embler Gary L. | Correlation-based color mosaic interpolation adjustment using luminance gradients |
| CN1529495A (en) * | 2003-09-27 | 2004-09-15 | 哈尔滨工业大学 | Method for filtering image by using neural network technique |
-
2004
- 2004-11-26 CN CNB2004100973248A patent/CN100459718C/en not_active Expired - Fee Related
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020063789A1 (en) * | 2000-11-30 | 2002-05-30 | Tinku Acharya | Color filter array and color interpolation algorithm |
| CN1398108A (en) * | 2001-07-18 | 2003-02-19 | 三洋电机株式会社 | Picture signal processor |
| US20030052981A1 (en) * | 2001-08-27 | 2003-03-20 | Ramakrishna Kakarala | Digital image system and method for implementing an adaptive demosaicing method |
| US20030117507A1 (en) * | 2001-12-21 | 2003-06-26 | Nasser Kehtarnavaz | Color filter array interpolation |
| US20040161145A1 (en) * | 2003-02-18 | 2004-08-19 | Embler Gary L. | Correlation-based color mosaic interpolation adjustment using luminance gradients |
| CN1529495A (en) * | 2003-09-27 | 2004-09-15 | 哈尔滨工业大学 | Method for filtering image by using neural network technique |
Also Published As
| Publication number | Publication date |
|---|---|
| CN1780404A (en) | 2006-05-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101053249B (en) | Image processing method, image storage method, image processing device and image file format | |
| CN111127336B (en) | Image signal processing method based on self-adaptive selection module | |
| US9467704B2 (en) | Adaptive ratio images in HDR image representation | |
| CN101779472B (en) | Image data generating apparatus and method | |
| CN104702926B (en) | Video camera | |
| US7292725B2 (en) | Demosaicking method and apparatus for color filter array interpolation in digital image acquisition systems | |
| CN102572443B (en) | For isochronous audio in image-signal processing system and the technology of video data | |
| Chang et al. | Effective use of spatial and spectral correlations for color filter array demosaicking | |
| CN101675454B (en) | Adopt the edge mapping of panchromatic pixels | |
| CN102420995A (en) | System and method for processing images using predetermined tone reproduction curves | |
| US20060152596A1 (en) | Noise cleaning sparsely populated color digital images | |
| CN102034113A (en) | A method to measure local image similarity based on the l1 distance measure | |
| CN103268596B (en) | A kind of method for reducing picture noise and making color be near the mark | |
| CN103327220B (en) | With green channel for the denoising method guided on low-light (level) Bayer image | |
| CN104410786B (en) | The control method of image processing equipment and image processing equipment | |
| CN107454285A (en) | Image noise estimation method and device and image extraction device | |
| CN112508812A (en) | Image color cast correction method, model training method, device and equipment | |
| CN100459718C (en) | Method and device for decoding mosaic of color filter array picture | |
| WO2022141515A1 (en) | Video encoding method and device and video decoding method and device | |
| US20070035634A1 (en) | System and method for reduction of chroma aliasing and noise in a color-matrixed sensor | |
| Wu et al. | Color demosaicking with sparse representations | |
| Vandewalle et al. | Joint demosaicing and super-resolution imaging from a set of unregistered aliased images | |
| Pan et al. | Joint demosaicking and denoising for CFA and MSFA images using a mosaic-adaptive dense residual network | |
| CN106408647A (en) | Image shading adding method and device | |
| US20060104537A1 (en) | System and method for image enhancement |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20090204 Termination date: 20161126 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |