Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N5/00—Details of television systems
  • H04N5/14—Picture signal circuitry for video frequency region
  • H04N5/21—Circuitry for suppressing or minimising disturbance, e.g. moiré or halo
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N9/00—Details of colour television systems
  • H04N9/64—Circuits for processing colour signals
  • H04N9/646—Circuits for processing colour signals for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters

Definitions

• the present invention relates to a noise filtering method and thereby a high-definition image restoring technique from a blurred color image captured under an environment of extremely low-level illumination.
• the present invention relates to an image processing technique to eliminate the color blurring and signal- dependent Poisson noise of images captured under an extremely low-level illumination wherein the edge boundaries as well as the detailed information of captured images are well preserved .
• an image- capturing device such as a CCD (charge coupled device) camera or a digital video camera under a condition of extremely low-level illumination
• the quality of the captured image tends to be deteriorated because the energy density of the captured image is lower than that of background noise of the image - capturing device .
• a specially designed image- capturing apparatus such as an IR (infrared) input device or a photo- amplifier should be employed for the enhancement of the quality of images .
• the color blurring is quite frequently observed in images captured under low-level illumination wherein the chromaticity of a spot is totally different from that of the vicinity.
• the color blurring is mitigated under relatively bright illumination. However, when the light illumination is not sufficient, the problem of the color blurring becomes severe .
• each channel of the array comprising the color filter in a CCD sensor is uniformly processed irrespective of the different characteristics of each ' channel .
• a signal processing without taking the brightness of illumination into account influences the relative ratio of the colors of each pixel, which causes a local color blurring as a consequence .
• the captured image under low-level illumination suffers from the signal- dependent Poisson noise in terms of intensity as well as the aforementioned color-blurring problem .
• FIG. 1 is a schematic diagram illustrating the captured image the quality of which is degraded due to the noise under low- level illumination in accordance with the prior art .
• the captured image looks brighter than what it should be due to operation of an automatic gain control (AGC) .
• AGC automatic gain control
• FIG. 1 more carefully, we can observe the blurring of colors in terms of red (R) , blue (B) , and green (G) all over the image.
• the Poisson noise in a pixel unit can also be observed at several spots where without the color blurring.
• DVR digital video recorder
• the technical limit of the MPEG scheme in compressing the image captured under low-level illumination is that since the scattered occurrence of a color spot (which is called as ""color blurring 1 ') could be recognized as a movement of an object in time frame by an MPEG processor, the captured image cannot be effectively compressed and thereby the size of the data storage media should inevitably large .
• the color spot namely, blurring of color
• the images captured under low-level illumination occurs in a randomly scattered manner at each time frame, it is erroneously regarded as a movement of an object in temporal domain during the MPEG compression, which thereby causes the degradation of the MPEG compression rate.
• the temporal filtering scheme in accordance with the prior art employs the concept of motion compensation. Therefore, the traditional temporal filtering scheme requires a huge amount of calculation time (CPU intensive) for the post processing.
• the conventional temporal filtering scheme performs a filtering process by tracing the trajectory of a moving object at each time frame, the calculation time for the estimation of the trajectory becomes too enormous to be implemented in real time.
• the noise filtering technique in a temporal domain relies on a scheme that the motion of an object in a color image is detected only in terms of the brightness (namely, intensity) .
• the traditional technique of detecting the motion in terms of difference in brightness should have a technical limit for the application in a DVR under low-level illumination.
• the prior art has a shortcoming in that the Poisson noise present in the intensity region of an image cannot be eliminated even if the color blurring can be efficiently eliminated in case the prior art is applied in a temporal domain.
• an edge adaptive filtering technique can be utilized.
• the traditional edge adaptive filtering technique has a shortcoming because it cannot eliminate the color blurring.
• the color blurred pixels Since the color blurring in a spatial domain has a large correlation among the neighboring pixels, the color blurred pixels, the color blurring of which is regarded as noise in case of the filtering, is treated as pixels in the neighborhood. As a consequence, the filtered image also includes a color blurring.
• the present invention discloses a technique to eliminate the color blurring and the signal dependent noise of the image captured under low-level illumination, comprising steps of (a) sensing the degree of motion through calculating the difference in brightness and chromaticity among the pixels comprising a frame under consideration and the pixels of a reference frame; (b) calculating a intensity weight -function from the calculated difference in intensity of step (a) and thereafter estimating a chromaticity weight - function from the calculated difference in chromaticity in step (a) ; (c) performing a temporal filtering only for a predefined number of pixels wherein the degree of motion calculated at step (b) is less than a predefined threshold, on each of R, G, and B channels, respectively; ( d) transforming the RGB image into the YUV format; (e) sensing the degree of edge sharpness through estimating the difference in brightness between the central pixels constituting a frame of the image and a predefined number of neighboring pixels; ( f ) cal culat ing the intensity weight - function
• FIG. 1 is a schematic diagram illustrating au exemplary image of deteriorated quality due to the noise generated under low- level illumination according to the prior art.
• FIG. 2 is a schematic diagram illustrating a method of eliminating the noise and restoring the image in spatio-temporal domain in accordance with the present invention.
• FIGS. 3A through 3B are schematic diagrams illustrating embodiments of a spatio- temporal noise elimination method in accordance with the present invention.
• the noise elimination method in accordance with the present invention can effectively eliminate the color blurring and signal -dependent noise in a simultaneous manner even with preserving the edge sharpness and the details of the image under extremely low-level illumination.
• the present invention discloses a mot ion- adapt ive temporal filtering technique in time axis for eliminating the color blurring as well as filtering the Poisson noise even with preserving the edge boundary.
• the present invention has a feature in that the temporal filtering step is preceded to the spatial filtering step in an effort to effectively eliminate the color blurring.
• the noise elimination and image - restoring method in accordance with the present invention has a feature in a sense that the filtering process is performed for each of R, G, and B channels while the prior art relies only on the intensity component for the color image filtering.
• the present invention performs an independent filtering process for each of R, G, and B channels in order to take both the intensity and the chromaticity into account .
• FIG. 2 is a schematic diagram illustrating an adaptive noise elimination technique and an image restoring method in spatio-temporal domain in accordance with the present invention.
• the mot ion- adapt ive temporal filtering 120 starts with the detection of motion among the frames as a pixel unit through vector-order statistics of the color image . Since the difference in brightness (i.e. light intensity) of an object is not sufficient for the detection of motion under low-level illumination, the prior art has a shortcoming for the application in practice.
• the present invention has a characteristic of taking differences both in the intensity and in the chromaticity in order to detect the motion of an object with accuracy .
• the detection of motion is performed both at intensity weight function block 100 and at chromaticity weighting function block 130 for temporal filtering 100 of FIG. 2.
• W is the intensity weighting function
• W ( is the chromaticity weighting function
• Y 10, 11, and 12 is the deteriorated vector color image .
• y R 10 is the deteriorated R- channel image while y G 11 and y B 12 are the deteriorated G-channel and B-channel images, respectively.
• t x is a reference frame and t 2 is another frame in temporal filtering.
• a function /(•) is a monotonically decreasing function with a functional value between O and 1.
• /(•) has a small value in an interval between 0 and 1, and thereby a small weight is assigned if there exists relatively a large difference in intensity or chromaticity between a frame in processing and a reference frame .
• sigmoid function and on-off function can be utilized.
• T is a threshold that determines the degree of motion
• ⁇ is a coefficient that determines the slope of the function.
• the spatio-temporal filtering technique with motion compensation in accordance with the prior art relies on a method of tracing the motion accurately and estimating the average along the trace of motion.
• the present invention discloses a technique of sensing the motion of an object with weighting function 110 and 130, and performing R, G, and B filtering at pixels wherein no motion has been detected.
• T is a support in a temporal filter and can be 3 ⁇ 9 frames as a preferred embodiment.
• the weighted filtering in accordance with the present invention effectively eliminates the noise due to motion, while the (R, G, and B) channel filtering can eliminate the color blurring.
• an LLMMSE (local linear minimum mean square error) filter can be utilized in the intensity component (Y component) of the image.
• the spat io- filtering 700 in accordance with the present invention effectively eliminates the Poisson noise with preserving the edge sharpness through estimating a suitable local mean 400 and local variance 500 from the non- stationary characteristics of the image.
• the above process can be represented by the estimation of local mean 400 and local variance 500 through the spatio weighting function 300 in spatio filtering block 700.
• T N is a support in spatio domain and W, is a weighting function in intensity domain for representing the edge sharpness.
• the estimation of a local mean through the weighting function in accordance with the invention is performed with respect to the pixels of large correlation (the pixels located on the same side with reference to the edge) rather than those of little correlation (the pixels located on the opposite side with reference to the edge) .
• the estimation of a local mean restores the image with good edge boundary, while the estimation of a local variance through the weight function makes it possible to remove the noise at the edge region with keeping the fine region preserved in the image .
• the LLMSE filter for the local statistics in accordance with the present invention can be designed such that it is suitable for the elimination of the Poisson noise .
• the intensity component of the image that has experienced the spatio filtering in intensity domain is combined with the original chromaticity component prior to the spatio filtering, thereafter being transformed into RGB format .
• FIGS. 3A through 3D are schematic diagrams illustrating the preferred embodiments of the present invention in comparison with the prior art .
• a CCD camera- captured image is depicted for the illustration of the color blurring and Poisson noise.
• FIG. 3B represents an exemplary image restored by eliminating the noise in accordance with the prior art .
• the color blurring has not been effectively removed because the prior art takes only the intensity component into account .
• FIG. 3B exhibits that the Poisson noise present in the intensity region has not been eliminated as yet, either.
• FIG. 3C is a picture of image that has been restored by eliminating the noise with the conventional spatio filtering technique.
• FIG. 3C it is noted that the prior art cannot effectively eliminate the color blurring even if the Poisson noise has been removed to some extent. Furthermore, FIG. 3C exhibits that the edge boundary of the image has been seriously damaged.
• FIG. 3D is a picture illustrating the image wherein the noise has been eliminated by the spatio-temporal filtering technique in accordance with the present invention.
• FIG. 3D shows that the color blurring and Poisson noise generated under low-level illumination have been effectively eliminated in accordance with the present invention.
• the present invention makes it possible to restore the image captured under low-level illumination to the one of high image quality through eliminating the color blurring and the Poisson noise even with preserving the edge sharpness of an object.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Picture Signal Circuits (AREA)
• Image Processing (AREA)
• Processing Of Color Television Signals (AREA)
• Image Analysis (AREA)

Priority Applications (2)

Applications Claiming Priority (2)

Publications (1)

Family

ID=19711556

Family Applications (1)

Country Status (6)

Cited By (3)

Families Citing this family (43)

Citations (8)

Family Cites Families (15)

• 2001
  • 2001-06-29 KR KR10-2001-0038280A patent/KR100405150B1/ko not_active IP Right Cessation
• 2002
  • 2002-06-26 EP EP02743917A patent/EP1400109A4/en not_active Withdrawn
  • 2002-06-26 JP JP2003511534A patent/JP2004522372A/ja not_active Ceased
  • 2002-06-26 WO PCT/KR2002/001216 patent/WO2003005705A1/en not_active Application Discontinuation
  • 2002-06-26 US US10/344,477 patent/US20030189655A1/en not_active Abandoned
  • 2002-06-26 CN CNA028021436A patent/CN1466844A/zh active Pending

Patent Citations (8)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events