WO2001026359A1 - Demosaicing using wavelet filtering for digital imaging device - Google Patents

Demosaicing using wavelet filtering for digital imaging device Download PDF

Info

Publication number
WO2001026359A1
WO2001026359A1 PCT/US2000/027470 US0027470W WO0126359A1 WO 2001026359 A1 WO2001026359 A1 WO 2001026359A1 US 0027470 W US0027470 W US 0027470W WO 0126359 A1 WO0126359 A1 WO 0126359A1
Authority
WO
WIPO (PCT)
Prior art keywords
demosaicing
mosaiced
interpolation
filtration
image
Prior art date
Application number
PCT/US2000/027470
Other languages
French (fr)
Inventor
Bo Tao
Original Assignee
Sony Electronics Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Electronics Inc. filed Critical Sony Electronics Inc.
Priority to AU78603/00A priority Critical patent/AU7860300A/en
Publication of WO2001026359A1 publication Critical patent/WO2001026359A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/10Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N2209/00Details of colour television systems
    • H04N2209/04Picture signal generators
    • H04N2209/041Picture signal generators using solid-state devices
    • H04N2209/042Picture signal generators using solid-state devices having a single pick-up sensor
    • H04N2209/045Picture signal generators using solid-state devices having a single pick-up sensor using mosaic colour filter
    • H04N2209/046Colour interpolation to calculate the missing colour values

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Color Television Image Signal Generators (AREA)

Abstract

A digital camera (10) is provided having a lens (14) for focusing light from an image through a RGB mosaiced filter (28) onto a solid state photosensor array (24) containing a matrix of CCD's (26). The CCD's provide a single channel mosaiced image output. A demosaicing process uses wavelet filtering to provide super-resolution imaging for demosaicing of the mosaiced image to an image output. In particular, a quincunx wavelet filter is used for interpolation for one color component, and one-dimensional wavelet filters are used for interpolation for other color components. The interpolated color components are then combined to form the demosaiced image for further processing in the digital camera (10).

Description

DEMOSAICING USING WAVELET FILTERING FOR DIGITAL IMAGING DEVICE
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates generally to digital imaging devices, and more specifically, to digital video cameras and still cameras using a single-channel, solid-state imaging sensor array.
2. Background of the Invention
Solid-state digital video and still cameras are becoming more and more popular. Most of the current digital cameras use a single-channel, solid-state photosensor array, and many of the digital still cameras on the market now are single channel charge coupled device cameras, or 1-
CCD cameras. In such cameras, the solid-state photosensors are CCD's arranged in an array in a matrix fashion with each CCD defining an element in the matrix called a "pixel". By placing a mosaiced color filter over the photosensor array, a single channel image is created where each pixel has a single color component. For example for a red-green-blue (RGB) system, a pixel has a single red (R), green (G), or blue (B) color component of a full color image. This forms a single channel mosaiced color image. The single channel mosaiced color image has to be demosaiced subsequently so that a full RGB (three-channel) image is created where each pixel contains the three RGB color components.
Each single-channel digital camera has its own unique demosaicing method. Although a number of demosaicing methods have been proposed, many do not take into account the filtering and sampling process performed by the optical lens and the CCD. Thus, they often generate images that are not sharp and/or which have aliasing effects. Others lack proper modeling of the filtering and sampling process and/or use interpolation filters that either do not have a good frequency response, are not mathematically interpolating, or are not smooth. Thus there is a need for a demosaicing method which properly accounts for the sampling and filtering process performed by the optical lens and CCD, and thus which provides a higher resolution image.
SUMMARY OF THE INVENTION
According to one embodiment of the present invention, a method is provided for processing single- channel mosaiced image information from a sensor and then demosaicing the information using a wavelet filter to provide a super-resolution demosaiced image for further image processing. The super-resolution imaging may also be used for any application involving data interpolation.
In accordance with another embodiment of the present invention, a demosaicing method is provided in which a mosaiced image is received in a plurality of colors and separated into a plurality of single color images. The plurality of single color images are interpolated using wavelet filters, and the interpolated single color images are combined into a demosaiced color image. According to one embodiment of the present invention, one-dimensional wavelet filters are used for the interpolation in the demosaicing process. Alternatively, according to a different embodiment, quincunx wavelet filters may be used for such interpolation.
In one aspect of the present invention, the filters are thresholded by setting to zero coefficients which are smaller than a predetermined threshold.
Another embodiment of the present invention provides a digital camera having a lens for focusing light from a scene through a RGB mosaiced filter onto a solid state photosensor array containing a matrix of CCD's, which provide a single channel mosaiced image output. A demosaicing process uses a wavelet filter to provide super-resolution imaging for demosaicing of the mosaiced image to provide an image output.
The present invention further provides for generating high-resolution image/video from a low-resolution one and may be used to interpolate an NTSC or CCIR-601 signal into a HDTV signal. The above and additional advantages of the present invention will become apparent to those skilled in the art from a reading of the following detailed description when taken in conjunction with the accompanying drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 shows a solid state digital camera incorporating one embodiment of the present invention;
FIG. 2 shows a representation of a mosaiced image in a Bayer pattern; FIG. 3 shows a representation of a full RGB image output after demosaicing; FIG. 4 shows a flow chart of one embodiment of a imaging process including demosaicing used in the solid state digital camera of FIG. 1;
FIG. 5 shows a flow chart of the demosaicing process according to one embodiment of the present invention;
FIG. 6 is a representation of a series of separated mosaiced image planes each of which is output from the separation process according to one embodiment of the present invention; FIG. 7 is a representation of a series of separated demosaiced image planes each of which is input to the combination process according to one embodiment of the present invention;
FIG. 8 shows a flow chart of an alternate embodiment of the filtering process of the imaging process of FIG. 4; and,
FIG. 9 is a representation of a series of separate demosaiced image planes produced by the alternate filtering of FIG. 8.
DETAILED DESCRIPTION OF THE INVENTION
Referring now to FIG. 1, a solid state digital camera 10 is shown having a camera body 12 and an attached lens system 14 with a lens filter 15. Although a single lens reflex camera is shown it will be appreciated by those skilled in the art that the present invention could be implemented for other still digital imaging devices as well as motion digital imaging devices.
The camera 10 has a mirror 16 which pivots on a pivot 18 to initially direct light both from an image as well as ambient light through the lens filter 15 and the lens system 14 up to a prism 20 and out through an eyepiece 22.
When a picture is being taken, the mirror 16 is pivoted up so as to allow light to strike a recording medium 24. In a digital still camera (DSC), the digital recording medium 24 is a matrix of photosensors, such as analog output, single channel, charge-coupled devices (CCD's) 26. Each of the CCD's 26 represents one picture element, or pixel, of the picture and the full array is called the "mosaic" pattern. Although cyan, magenta, yellow, and green filters can be used, in the embodiment shown in Figure 1, each of the CCD's 26 has a green (G), red (R), or blue (B) filter over it, generally designated as a mosaic filter 28. After processing, the image falling on these CCD's 26 is digitally recorded, either electronically or magnetically, for later playback. A processing unit 30 incorporating an embodiment of the present invention processes the signals from the CCD's 26 though to the image output.
One example of a mosaiced image 50 may be seen with reference to United States Patent No. 3,971,065 entitled "Color Imaging Array", granted to Bryce E. Bayer, such pattern being commonly referred to as a "Bayer pattern". Figure 2 shows a portion of a representation of part of a mosaiced image 50 in a Bayer pattern, wherein columns "i" (52) show only columns one through six, and rows "j" (54) show only rows 1 through 6. As seen in Figure 2, row 1 starts with a green "G" pixel and a red "R" pixel, and then alternates between the two, while row 2 starts with a blue "B" pixel and a green "G" pixel and then alternates between the two. As will be appreciated from Figure 2, this pattern is repeated for the subsequent rows.
In order to reproduce a full RGB image 60 as shown in Figure 3, the RGB colors of a mosaiced image such as the mosaiced image 50 of Figure 2 must be determined. This is effected through a "demosaicing" process.
Figure 4 shows a flow chart 70 of the solid state digital camera 10 including the demosaicing process of the present invention. Light from an image passes through the mosaic filter 28 onto the CCD's 26, which photo-electrically convert the light falling on them into proportional analog electrical signals at step 71. The analog signals are then converted into digital signals by an analog to digital (A/D) conversion step 72. The digital signals are then output for demosaicing at demosaicing step 74. After demosaicing processing at step 74, the output is subjected to image processing 76, which includes a series of conventional image processing steps, including white balancing and compression. The resulting image output 78 may be provided to user devices such as a recording unit or display device. It will be evident to those skilled in the art, that various mechanisms or devices for performing the conventional image processing steps of the aforementioned method are known and the invention is not limited to any such one mechanism or device.
Referring now to FIG. 5, a flow chart of one embodiment of the demosaicing process 74 of the present invention is shown. According to such embodiment, after the analog signals are converted to digital signals, the thus converted signals are provided to a separation unit 80, which processes the signal to provide separated image planes. In a preferred embodiment such planes are G-R-B planes. G-R-B signals representative of the planes are then subjected to G, R, and B interpolation at interpolation units 82, 84, and 86, respectively. After interpolation, each of the G,
R, and B interpolated signals are then provided to combination unit 88, which provides a demosaiced signal output for image processing 76.
One embodiment of the separation unit 80 wherein the output has G-R-B planes may be seen in Figure 6. As seen therein, the output from separation unit 80 is a series of three mosaiced image planes 100, 102, and 104. In such embodiment, the image plane 100 preferably includes a first row starting with a green (G) pixel and then alternating between zero-value (0) and green (G) pixels, while the second row starts with a zero-value (0) pixel and then alternates between green
(G) and zero-value (0) pixels. Image plane 102 preferably includes a first row starting with a zero-value (0) pixel and then alternating between red (R) and zero-value (0) pixels, while the second row includes all zero-value (0) pixels. Finally, image plane 104 preferably includes a first row starting with all zero-value (0) pixels, while the second row starts with a blue (B) pixel and then alternates between zero-value(O) and blue (B) pixels. In each of the image planes 100, 102, and 104, the first and second row patterns are then preferably repeated in rows three and four, five and six, and so on.
Once the image planes have been created by separation unit 80, as stated previously, they are provided to each of the interpolating units 82, 84, 86. In a preferred embodiment, image planes 100, 102 and 104, are respectively input to the G, R, and B interpolation units 82, 84, and 86. The outputs from such units are shown in Figure 7, and comprise mosaiced image planes shown at 110, 112, and 114, respectively.
Operation of the imaging process including an embodiment of the demosaicing process of the present invention will now be discussed with reference to Figures 1 and 2. When light from an image enters the camera 10 through a lens filter 15 and lens system 14, it is reflected by the mirror 16 which is in the down position as shown in Figure 1. The reflected light travels upwards into prism 20 and then again reflected through prism 20 to exit out eyepiece 22, where a user sees the exact image "seen" by lens filter 15 and the lens system 14.
When a photograph is taken, the mirror 16 is caused to pivot upward out of the way so that instead of being reflected, the light will pass through a mosaic filter 28 to strike a photosensor matrix array 26, which, again, preferably include single-channel solid-state CCD's. After a single- channel image is created from the photosensor array, each pixel has one color component, R, G, or B, and a mosaiced image 50, preferably in the form of the Bayer pattern of Figure 2 is formed. The mosaiced image 50 will then be digitized by the in A/D conversion step 72. According to the present invention a demosaicing process is subsequently applied to generate the missing color components. For example, referring to Figure 2, for the pixel at the upper-left comer at column 1, row 1 (bold-faced and italicized), only the G component is created from the photo sensor array 26. Thus, the missing R and B components have to be generated using a demosaicing process. Similarly, for the pixel at column 3, row 4 (bold-faced and italicized), only the R component is created, and the missing G and B components must be generated through demosaicing. The output is subsequently subject to demosaicing step 74 where the mosaiced image 50 is demosaiced and the full RGB image 60 is generated. The full RGB image 60 after demosaicing is shown in Fig. 3.
It will be appreciated that the demosaicing step 74 plays an important role in the digital camera 10, as it has a direct impact on image quality. It is important that the filter used in the demosaicing process have a good frequency response, be interpolating, and be smooth. It was determined that "wavelet filters" provided the aforementioned desired characteristics.
While wavelet filters have been used in the fields of signal analysis and synthesis, particularly with respect to sound and image signals, for the most part such filters have been limited to use in image compression. As known to those skilled in the art, as used in such image compression processes, "wavelet transforms" provide representations of any arbitrary signals as a superposition of wavelets. These wavelets are themselves functions which are, in turn, generated from a single function through dilations and translations and then passed through filters to allow decomposition of a signal into different levels (each of which is further decomposed with a resolution adapted to the particular level). (A detailed discussion of the operation and creation of wavelet transforms will not be presented herein as these functions are understood to those skilled in the art.)
One embodiment of the demosaicing process 74 of the present invention is shown in Fig. 5. The mosaiced output from A/D conversion unit 72 (FIG. 2) is then subjected to the demosaicing process 74. Initially, the mosaiced output is separated by the separation unit 80 into the three separated mosaiced image planes 100, 102, and 104, respectively as shown in Fig. 6. (Again, although these planes are shown as R, G, and B planes, it will be understood that cyan, magenta, yellow, and green planes may also be used). The mosaiced image planes 100, 102 and 104 are then provided as inputs to G, R, and B interpolation units 82, 84, and 86, respectively (FIG. 5). The outputs from these interpolation units 82, 84, and 86, respectively, are shown in Fig. 7 as G, R, and B mosaiced planes 110, 112 and 114, respectively. These three mosaiced planes 110, 112 and 114 are subsequently combined in the combination unit 88, the output demosaiced RGB signal of which is subjected to image processing 76. Description of embodiments of the interpolation units 82, 84, and 86 will now be discussed in more detail. In one embodiment, the G interpolation unit 82 uses a two-dimensional quincunx wavelet filter, although it will be appreciated to those skilled in the art that other quincunx wavelet filters may be used as well. The coefficients of one embodiment of such filter may be seen in Table I below:
Table I
Figure imgf000008_0002
The interpolated data may be given by the following equation: + m,; +«)/2l5 x V2
Figure imgf000008_0001
where F, represents this quincunx filter, Xg represents the G mosaiced image plane 100 at pixel position (i,j), and Yg is the demosaiced output 110 as shown in Fig. 7.
The R interpolation unit 84 preferably uses a one-dimensional wavelet filter, although, again, it will be appreciated to those skilled in the art that other quincunx wavelet filters may be used. One such filter for the R interpolation unit may have the following coefficients: ^[-0.045636 -0.028772 0.295636 0.557543 0.295636 -0.028772 -0.045636]
It is necessary to perform two filtering processes, once in the horizontal direction and once in the vertical direction. However, the order in which these processes is performed is arbitrary. In the following example, the vertical filtering is completed first. When filtering is applied in the vertical direction at pixel position (i,j), the filtered output is given by the equation:
Zr( j) = ∑F2(m) x Xr(i + m,j) x 2 χ [2
where: Xr represents the R mosaiced image plane 102.
After Zr has been computed as above, each row of the image pattern (e.g., the Bayer pattern of
Fig. 2) is flipped. Then the following equation is then used to derive Z where Zr represent this new signal with pixel index i=0, ..., N-l, j=0, ..., N-l
Zr(i,j) = Zr(i,N -\ - j)
Horizontal filtering is then performed on Zr and the resulting output of the R interpolation unit 84 is given by the equation:
Yr(i,j) = ∑F2(n) x Zr(i,j + n) x f2 . n = -3
Each row of Ϋr is flipped to get the output Y'r. Thus: Yr (i, j) = Yr (i, N - \ - j) . This output, Y which is the R demosaiced image plane 112, is shown in Fig. 7.
The B interpolation unit 86 preferably uses the same filter F2 as the R interpolation unit 84. The only difference between these two interpolation units is that in the B interpolation unit 86, the two flipping processes are performed during vertical filtering, while no flipping is needed for horizontal filtering.
An alternative embodiment of this invention can be realized by thresholding the filter F i.e., setting those coefficients of F2 smaller than a selected threshold. This can reduce the computational cost. One embodiment of such a thresholded filter, Flt is shown in Table II below with the threshold set at 15:
Figure imgf000010_0001
Table II
The G interpolation unit 82 then performs the following summation: γ i > ) = Σ ∑F(m,ή)χ Xg(i + m,j + n)l2'5 χ 2lk nι=-4.ι=-4 with k given by: k = ∑ ∑Ft(m,n) . nι=-4 «=-4
It will be appreciated that other thresholds may be used.
After processing in combination unit 88, the fully demosaiced RGB image 60 output is then provided to the image processing unit 76. In the image processing unit 76, the RGB image will go through the series of conventional image processing steps, including white balancing and compression, and finally be output as the image output 78.
Referring now to FIG. 8, an alternate embodiment of the filtering process included in the imaging process step of FIG. 4 is shown. In such embodiment, the image output from the demosaicing step 74 is then input into low-pass and high pass branches 151 and 153. In the low- pass branch 151, the input is provided for filtering in F3' block 154, the output of which is then provided to a "reduce by 2" block 156 for removal of every other pixel. The output from "reduce by 2" block 156 is then provided to the Q block 158 to for quantitization. After quantitization, the output is provided to an "increase by 2" block 160 for addition of every other pixel (which is preferably zero in this embodiment). The output of the "increase by 2" block 160 is then provided to F3 block 162 for front-end weighted averaging, the output of which is then provided to a sum block 164 which, in turn, provides the image output 78.
The image output from the demosaicing step 74 is input into the high-pass branch 153 and is filtered in H3' block 170. The output of H3' block 170 is then provided to a "reduce by 2" block 172 for removal of every other pixel, after which it is passed to the Q block 174 to be quantized. After quantization in Q block 174, the output is provided to an "increase by 2" block 176 for addition of every other pixel (which is preferably zero in this embodiment). The output of the "increase by 2" block 176 is provided to H3 block 178 for front-end weighted averaging and then to the sum block 164 to be added with the F3 block 162 output.
Referring now to FIG. 9, therein is shown a series of two separate demosaiced image planes 200 and 202 produced in the alternate filtering of the present invention. In operation in the low-pass branch 151, the image input from demosaicing step 74 is weight averaged by wavelet filtering in F3' block 154. Alternating pixels are removed in "decrease by 2" block 156 and then the real number value of pixels is quantized into one of a discrete number of values in the quantize block 158. The quantized output then has alternating zeros inserted by the "increase by 2" block 160. The image data is then weight averaged by wavelet filtering in F3 block 162 and provided to sum block 164.
In operation in the high-pass branch 153, the image input from demosaicing step 74 is weight averaged by wavelet filtering in H3 λ block 170. Alternating pixels are removed in "decrease by 2" block 172 and then the real number value of pixels is quantized into one of a discrete number of values in the quantize block 174. The quantized output then has alternating zeros inserted by the "increase by 2" block 176. The image data is then weight averaged by wavelet filtering in H3 block 178 and provided to the sum block 164 where it is summed with the output of the F3 block 162 for the image output 78.
When using the compression system shown in the image processing flow chart 150, the F2 filtering in the demosaicing step 74 is replaced with F3 filtering for R, G, and B planes in the R, G, and B interpolations 82, 84, and 86. For R and B, the filtering process is the same as before. For G, the plane is separated into two planes, filtered, summed, and divided by two. That is, the G plane 100 is separated into the two G planes 200 and 202 , and each is filtered using F3 filtering.
No flipping in either the horizontal or vertical direction is required. The two separate G planes 200 and 202 are summed and divided by two to give an image which is used as the G plane 100. In this alternate embodiment, the compression algorithm used in the image processing step 76 uses known wavelet compression. In such a situation, the wavelet filtering used in the demosaicing step 74 should be chosen such that the high frequency band signals in the wavelet compression will be small and close to zero. With small coefficients, efficient compression will be obtained.
For example, when a pair of analysis wavelet filters, F3' and H3', are used in wavelet compression, there is a corresponding synthesis filter pair, F3 and H3. The compression system 150 (shown only one-dimensional and for one stage of the entire system) indicates that F2 should be replaced by F3. Thus, the R, G, and B planes are demosaiced by using F3. In the demosaicing of the G plane 100, it is separated into the two planes 200 and 202, as shown in FIG. 9. For either plane 200 or 202, the same filtering process is applied as for the R and B planes except that no flipping is needed in either horizontal or vertical direction. The filtering results of the two separate G planes 200 and 202 are then combined by summing in the sum block 164 and dividing the sum by two. This average value is then used as the G plane 110.
As described above, the present invention provides super-resolution imaging for any application involving data interpolation and more particularly for generating high-resolution image/video from a low-resolution image/video. One application is in the interpolation of an NTSC or CCIR-601 signal into a HDTV signal.
While the invention has been described in conjunction with a specific best mode, it is to be understood that many alternatives, modifications, and variations will be apparent to those skilled in the art in light of the aforegoing description. Accordingly, it is intended to embrace all such alternatives, modifications, and variations which fall within the spirit and scope of the included claims. All matters set forth herein or shown in the accompanying drawings are to be interpreted in an illustrative and non-limiting sense.

Claims

1. A demosaicing method comprising the steps of: receiving patterned mosaiced images in a plurality of colors; separating the patterned mosaiced images into a plurality of single color images; interpolation of each of the plurality of single color images using a filter; and combining the plurality of interpolated single color images into a demosaiced color image.
2. The demosaicing method as claimed in claim 1 wherein said step of: interpolation includes a step of filtration using a wavelet filter for filtration of at least one of the plurality of single color images.
3. The demosaicing method as claimed in claim 1 wherein said step of: interpolation includes a step of filtration using a thresholded quincunx wavelet filter.
4. The demosaicing method as claimed in claim 1 wherein said step of: interpolation includes a step of filtration using a single-dimensional wavelet filter for filtration of at least one of the plurality of single color images.
5. The demosaicing method as claimed in claim 1 including the step of: image processing after said step of combining, said step of image processing using a predetermined filtering; and said step of interpolation uses the same filtering.
6. A demosaicing method in a digital camera comprising the steps of: focusing light from a scene through a lens; filtering the light into mosaiced red (R), green (G), and blue (B) lights; converting the mosaiced R, G, and B lights into patterned mosaiced images in R, G, and B; separating the patterned mosaiced images into R, G, and B mosaiced images; interpolation of the R, G, and B mosaiced image using respective R, G, and B wavelet filters; and providing a demosaiced RGB image output from interpolation of the R, G, and B mosaiced images.
7. The demosaicing method in a digital camera as claimed in claim 6 wherein said step of: demosaicing includes a step of separating the patterned mosaiced images into separate R,B, and G color planes; demosaicing includes a step of interpolation of the separate R, G, and B color planes; and, demosaicing includes a step of combining said separate R, G, and B color planes after said step of interpolation.
8. The demosaicing method in a digital camera as claimed in claim 7 wherein said step of: interpolation includes a step of filtration on the R color plane using a one-dimensional wavelet filter; and, interpolation includes a step of filtration on the B color image using one-dimensional wavelet filter.
9. The demosaicing method in a digital camera as claimed in claim 8 wherein said step of: filtration of the R color plane using said one-dimensional wavelet filter is performed in a horizontal direction and in a vertical direction; and, filtration of the B color plane using said one-dimensional wavelet filter is performed at least in a vertical direction.
10. The demosaicing method in a digital camera as claimed in claim 9 including a step of: flipping the output of the B color plane filtration twice after the vertical direction filtration.
11. The demosaicing method in a digital camera as claimed in claim 10 wherein said step of: interpolation includes a step of filtration on the G color plane using a quincunx two- dimensional wavelet filter.
12. The demosaicing method in a digital camera as claimed in claim 11 wherein said step of: demosaicing uses thresholded quincunx two-dimensional wavelet filtering whereby the filter coefficients of said quincunx two-dimensional wavelet filtering below a predetermined value are set to zero.
13. The demosaicing method in a digital camera as claimed in claim 12 including a step of: image processing using a wavelet compression filtering; and, said step of demosaicing uses wavelet filtering similar to said wavelet compression filtering.
14. The demosaicing method as claimed in claim 14 including the step of: image processing after said step of combining, said step of image processing using a predetermined filtering; and said step of interpolation uses the same filtering.
15. A digital camera comprising: a lens for focusing light from a scene; a mosaiced filter for converting the light into a plurality of mosaiced colors; a photosensor array associated with said mosaiced filter and having a plurality of elements for receiving each of the plurality of mosaiced colors and providing a plurality of outputs as patterned mosaiced images; a demosaicing unit operatively connected to said photosensor array for receiving and demosaicing the patterned mosaiced images into a demosaiced image, said demosaicing unit including a wavelet filter used in the demosaicing of the mosaiced image; and an image output device operatively connected to said demosaicing unit to output said demosaiced image.
16. The digital camera as claimed in claim 16 wherein: said demosaicing unit includes a separation unit for separating the patterned mosaiced images into separate color images; said demosaicing unit includes a plurality of interpolation units for interpolating the separate color images; and, said demosaicing unit includes a combination unit for combining said separate color images after interpolation.
17. The digital camera as claimed in claim 17 wherein one of said plurality of interpolation units includes a one-dimensional wavelet filter.
18. The digital camera as claimed in claim 18 wherein: said one-dimensional wavelet filter is capable of performing a filtration in a horizontal direction and a filtration in a vertical direction; and, said interpolation unit is capable of manipulating the output of a filtration after a filtration.
19. The digital camera as claimed in claim 19 wherein said interpolation unit is capable of manipulating the output of a filtration a plurality of times.
20. The digital camera as claimed in claim 20 wherein one of said plurality of interpolation units includes a quincunx wavelet filter used in the demosaicing of the mosaiced image.
21. The digital camera as claimed in claim 21 wherein said quincunx wavelet filter is thresholded whereby the filter coefficients of said quincunx wavelet filter below a predetermined value are set to zero.
22. A digital camera comprising: a lens for focusing light from an image; a mosaiced filter for converting the light into mosaiced red (R), green (G), and blue (B) lights; a photosensor array associated with said mosaiced filter and having a plurality of elements for receiving each of the mosaiced R, G, and B lights and providing a plurality of outputs as patterned mosaiced images; a demosaicing unit operatively connected to said photosensor array for receiving and demosaicing the patterned mosaiced images into a demosaiced image, said demosaicing unit including a wavelet filter used in the demosaicing of the mosaiced image; and, an image output device operatively connected to said demosaicing unit to output said demosaiced image.
23. The digital camera as claimed in claim 23 wherein: said demosaicing unit includes a separation unit for separating the patterned mosaiced images into separate R, G, and B color images; said demosaicing unit includes a plurality of interpolation units for interpolating the separate R, G, and B color images; and, said demosaicing unit includes a combination unit for combining said separate R, G, and B color images.
24. The digital camera as claimed in claim 24 wherein: one of said plurality of interpolation units includes a one-dimensional wavelet filter for filtration on the R color image; and, one of said plurality of interpolation units includes a one-dimensional wavelet filter of filtration on the B color image.
25. The digital camera as claimed in claim 25 wherein: said one-dimensional wavelet filter is capable of performing a filtration in a horizontal direction and a filtration in a vertical direction; and, said interpolation unit is capable of flipping the output of the R color image filtration after the horizontal direction filtration and after the vertical direction filtration.
26. The digital camera as claimed in claim 26 wherein said interpolation unit is capable of flipping the output of the B color image filtration twice after the vertical direction filtration.
27. The digital camera as claimed in claim 27 wherein one of said plurality of interpolation units includes a quincunx wavelet filter used in the filtration of the G color image filtration.
28. The digital camera as claimed in claim 28 wherein said quincunx wavelet filter is thresholded whereby the filter coefficients of said quincunx wavelet filter below a predetermined value are set to zero.
29. The demosaicing method as claimed in claim 29, further including the step of image processing unit connected after interpolation unit, interpolation unit using the same filtering as said image processing unit.
PCT/US2000/027470 1999-10-05 2000-10-05 Demosaicing using wavelet filtering for digital imaging device WO2001026359A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU78603/00A AU7860300A (en) 1999-10-05 2000-10-05 Demosaicing using wavelet filtering for digital imaging device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US41235199A 1999-10-05 1999-10-05
US09/412,351 1999-10-05

Publications (1)

Publication Number Publication Date
WO2001026359A1 true WO2001026359A1 (en) 2001-04-12

Family

ID=23632661

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2000/027470 WO2001026359A1 (en) 1999-10-05 2000-10-05 Demosaicing using wavelet filtering for digital imaging device

Country Status (2)

Country Link
AU (1) AU7860300A (en)
WO (1) WO2001026359A1 (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2002075654A2 (en) * 2001-03-20 2002-09-26 Hewlett-Packard Company System and method for asymmetrically demosaicing raw data images using color discontinuity equalization
EP1262917A1 (en) * 2001-05-31 2002-12-04 Agilent Technologies, Inc. (a Delaware corporation) System and method for demosaicing raw data images with compression considerations
EP1271928A2 (en) * 2001-06-19 2003-01-02 Agilent Technologies, Inc. (a Delaware corporation) System and method for processing demosaiced images to reduce color aliasing artifacts
EP1372335A1 (en) * 2002-06-12 2003-12-17 STMicroelectronics S.r.l. Processing method for obtaining a high-resolution digital image
EP1615168A1 (en) * 2004-07-09 2006-01-11 STMicroelectronics S.r.l. Colour interpolation in DWT domain

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5412741A (en) * 1993-01-22 1995-05-02 David Sarnoff Research Center, Inc. Apparatus and method for compressing information
US5778106A (en) * 1996-03-14 1998-07-07 Polaroid Corporation Electronic camera with reduced color artifacts

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5412741A (en) * 1993-01-22 1995-05-02 David Sarnoff Research Center, Inc. Apparatus and method for compressing information
US5778106A (en) * 1996-03-14 1998-07-07 Polaroid Corporation Electronic camera with reduced color artifacts

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
LEE ET AL.: "A novel interpolation scheme for quincunx-subsampled images", PROCEEDINGS OF THE 1999 IEEE INTERNATIONAL SYMPOSIUM ON CIRCUITS AND SYSTEMS, vol. 4, May 1999 (1999-05-01), pages IV-147 - IV-150, XP002935006 *
YASUOKA ET AL.: "2 Dimensional recursive orthogonal wavelet transformation", 1994 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING, vol. IV, April 1994 (1994-04-01), pages IV-113 - IV-116, XP002935007 *

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2002075654A2 (en) * 2001-03-20 2002-09-26 Hewlett-Packard Company System and method for asymmetrically demosaicing raw data images using color discontinuity equalization
WO2002075654A3 (en) * 2001-03-20 2003-09-12 Hewlett Packard Co System and method for asymmetrically demosaicing raw data images using color discontinuity equalization
EP1262917A1 (en) * 2001-05-31 2002-12-04 Agilent Technologies, Inc. (a Delaware corporation) System and method for demosaicing raw data images with compression considerations
US7155066B2 (en) 2001-05-31 2006-12-26 Agilent Technologies, Inc. System and method for demosaicing raw data images with compression considerations
EP1271928A2 (en) * 2001-06-19 2003-01-02 Agilent Technologies, Inc. (a Delaware corporation) System and method for processing demosaiced images to reduce color aliasing artifacts
EP1271928A3 (en) * 2001-06-19 2004-04-21 Agilent Technologies, Inc. (a Delaware corporation) System and method for processing demosaiced images to reduce color aliasing artifacts
US6819801B2 (en) 2001-06-19 2004-11-16 Agilent Technologies, Inc. System and method for processing demosaiced images to reduce color aliasing artifacts
EP1372335A1 (en) * 2002-06-12 2003-12-17 STMicroelectronics S.r.l. Processing method for obtaining a high-resolution digital image
US7257275B2 (en) 2002-06-12 2007-08-14 Stmicroelectronics S.R.L. Processing method for obtaining a high resolution digital image
EP1615168A1 (en) * 2004-07-09 2006-01-11 STMicroelectronics S.r.l. Colour interpolation in DWT domain

Also Published As

Publication number Publication date
AU7860300A (en) 2001-05-10

Similar Documents

Publication Publication Date Title
US6809765B1 (en) Demosaicing for digital imaging device using perceptually uniform color space
JP5108172B2 (en) Image data size conversion processing apparatus, electronic still camera, and image data size conversion processing recording medium
US5808674A (en) Producing and improved digital image from digital signals corresponding to pairs of photosites
CN101924947B (en) Image processing device, image processing method, and imaging apparatus
US6075889A (en) Computing color specification (luminance and chrominance) values for images
US6847397B1 (en) Solid-state image sensor having pixels shifted and complementary-color filter and signal processing method therefor
US7512267B2 (en) Learning device and learning method for image signal processing
JP5280448B2 (en) Image processing apparatus, image processing method, image processing program, and semiconductor integrated circuit
US7236191B2 (en) Method and system for image processing with pixel interpolation using second order gradients
EP0732859A2 (en) Adaptive colour plane interpolation in an electronic single-sensor colour camera
JPH04284087A (en) Electronic still camera
EP0469836B1 (en) Image signal processing apparatus
EP0869683B1 (en) Image pickup device
EP0436012A1 (en) A compression method and apparatus for single-sensor color imaging systems
GB2306074A (en) CCD camera with two CCDs having mutually different colour filter arrays
TWI236839B (en) Image processor, image processing method, recording medium, and program
JP4182566B2 (en) Digital camera and computer-readable recording medium
WO2002085034A1 (en) Method and apparatus for sensing and interpolating color image data
EP2800376B1 (en) Imaging device, method for controlling imaging device, and control program
WO2007082289A2 (en) Color filter array with neutral elements and color image formation
WO2001026359A1 (en) Demosaicing using wavelet filtering for digital imaging device
CN104580942B (en) Data processing equipment
JP4133179B2 (en) Image signal processing device
JP4687750B2 (en) Digital camera and image signal processing storage medium
JP2003111092A (en) Data conversion circuit

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AL AM AT AU AZ BA BB BG BR BY CA CH CN CR CU CZ DE DK DM EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG UZ VN YU ZA ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

122 Ep: pct application non-entry in european phase
NENP Non-entry into the national phase

Ref country code: JP