KR102138330B1 - Method and Apparatus for removing aliasing in Multi spectral Filter Array image - Google Patents

Abstract

As a preferred embodiment of the present invention, a method of removing aliasing from an MFA image includes: receiving an MFA image composed of a luminance information signal and a color information signal (Chrominance signal); And removing aliasing between the signal representing the brightness information and the signal representing the color information by using a high frequency component of the signal representing the color information.

Description

Method and Apparatus for removing aliasing in Multi spectral Filter Array image}

The present invention relates to MFA input image processing, and more specifically, to remove an aliasing component caused by sub-sampling of each channel in an MFA image.

The image obtained from the multi-spectral filter array (hereinafter referred to as "MFA") aims to acquire natural color and detail information of the NIR or enhanced sensitivity at night.

However, when using the MFA, images of each channel can be simultaneously acquired, but the resolution of the images of each channel is limited, and there is a problem in that aliasing occurs due to sampling of each channel.

KR 2005-0051437

In the case of an MFA image using a multi-band filter arrangement, images of R, G, B, and N channels can be simultaneously acquired, but there is a problem that aliasing occurs due to sampling of each channel. In addition, even when considering the MFA image in terms of Luminiance (brightness) and Chrominance (color), there is a problem that aliasing occurs due to overlapping of the Luminiance signal and the Chrominance signal.

In a preferred embodiment of the present invention, after analyzing the MFA signal and removing aliasing through the cancellation of the high frequency component of the chrominance signal for aliasing that occurs due to the mixing of the two signals in the luminance signal portion and the chrominance signal portion, the luminance signal is compensated. We want to acquire MFA images with minimal aliasing.

As a preferred embodiment of the present invention, a method of removing aliasing from an MFA image includes: receiving an MFA image composed of a luminance information signal and a color information signal (Chrominance signal); And removing aliasing between the signal representing the brightness information and the signal representing the color information by using a high frequency component of the signal representing the color information.

As another preferred embodiment of the present invention, the anti-aliasing device of the MFA image performs Fourier signal analysis of the received MFA input image, so that the received MFA image has a signal representing one brightness information and three colors. A Level Balancing unit, which is expressed as a signal representing information, and removes an aliasing signal generated between the signal representing the brightness information and the signal representing the three color information; And an aliasing minimization unit that compensates the MFA image by performing weighting between the MFA image from which the aliasing signal is removed and the signal representing the one brightness information.

In an exemplary embodiment of the present invention, aliasing is eliminated by canceling the aliasing caused by the mixing of the two signals in the luminance signal portion and the chrominance signal portion by canceling the aliasing by canceling the high frequency component of the chrominance signal, and then performing luminance signal compensation. There is an effect that the part where the wrong color information appears and the part where the high-frequency information distortion occurs are removed and the edge information is recovered.

1 is a block diagram schematically showing a camera system as a preferred embodiment of the present invention.
2 shows an example of aliasing in the Fourier region.
3 is a preferred embodiment of the present invention, showing a block diagram for minimizing aliasing of the MFA input image and restoring high-frequency information.
4 shows an example of aliasing in the Fourier region.
5 is a preferred embodiment of the present invention, showing an example of removing aliasing from the Level Balancing unit.
6 is a preferred embodiment of the present invention, and shows an example in which the luminance signal is compensated through a weighted sum method in an aliasing minimization unit.
7 to 8 show an example of compensating the luminance signal through a weighted sum method.

The terminology used herein is for describing the embodiments and is not intended to limit the present invention. In the present specification, the singular form also includes the plural form unless otherwise specified in the phrase. As used herein, "comprises" and/or "comprising" refers to the components, steps, operations and/or elements mentioned above, the presence of one or more other components, steps, operations and/or elements. Or do not exclude additions.

Unless otherwise defined, all terms (including technical and scientific terms) used in the present specification may be used as meanings commonly understood by those skilled in the art to which the present invention pertains. In addition, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively, unless specifically defined.

1 is a block diagram schematically showing a camera system as a preferred embodiment of the present invention.

The present invention can be applied to a digital camera system using a sensor having an RGB+NIR structure.

Referring to FIG. 1, the camera system 1 includes an image sensor 10, an image processing device 30, and a display device 50. The camera system 1 may be a video shooting system such as a digital camera, a camcorder, or a surveillance camera, or may be mounted on a computer, a PDA (Personal Digital Assistant), a PMP (Portable Multimedia Player), a mobile phone, or the like.

The image sensor 10 uses a photoelectric conversion element such as a Charge Coupled Device (CCD) or a Complementary Metal Oxide Semiconductor (CMOS). The image sensor 10 includes a pixel array in which a plurality of pixels are formed to convert an optical signal into an electrical image signal by a photoelectric conversion element.

On the image sensor 10, a multi-spectral filter array that passes light in visible and invisible bands is disposed. The MFA includes a color filter that transmits light components in the visible light band ranging from 400 nm to 700 nm, such as RGB, and a near infrared filter that transmits light components in the near-infrared region of 700 nm to 1100 nm in the invisible band.

The color filters (RCF, GCF, BCF) of the MFA and the near infrared filter (NIRF) are arranged to correspond to each pixel of the image sensor 10, so that each pixel of the image sensor 10 is a color channel signal R passing through the MFA , G, B) and near infrared channel signals (NIR, hereinafter "N"). A lens (not shown) for receiving an optical signal may be provided at the front end of the image sensor 10.

2 shows an example of aliasing in the Fourier region.

As shown in Equation 1, the MFA signal is composed of a luminance signal and three color information signals.

를 대입한 후, 푸리에 변환을 수행하면 수학식 2와 같이 된다. In Equation 1 -1=

After assigning, and performing the Fourier transform, Equation 2 is obtained.

Luminance 신호와 수평, 수직, 대각선 방향으로 Π, -Π 로 반복적으로 복제되어 나타나는 3개의 Chrominance신호로 이루어져 있는 것을 확인할 수 있다. In the Fourier domain, the MFA signal is located at (0,0).

It can be seen that it consists of Luminance signal and three Chrominance signals that are repeatedly duplicated in the horizontal, vertical and diagonal directions with Π and -Π

신호는 수평, 수직 방향으로 각각 Π만큼, 즉 대각선 방향으로 shift 되어 있고,

신호는 수평 방향으로 Π만큼, 그리고

신호는 수직 방향으로 Π만큼 shift되어 있다.

The signal is shifted by Π in the horizontal and vertical directions, that is, in the diagonal direction,

The signal is equal to Π in the horizontal direction, and

The signal is shifted by Π in the vertical direction.

As shown in Equation 2, the signal of the MFA pattern image appears together with a luminance signal containing detailed information of the image and a chrominance signal representing the chromaticity signal, and aliasing occurs due to the mixing of the two signals.

The aliasing that occurs when the luminance signal and the chrominance signal are mixed is classified according to which signal is mixed in which part.

Referring to FIG. 2, aliasing 210 generated by mixing the luminance signal and chrominance signal is caused by mixing the first aliasing 230 generated when the chrominance signal is mixed around the luminance signal and the luminance signal mixed around the chrominance signal. It can be divided into aliasing (220).

In a preferred embodiment of the present invention, a method of removing aliasing generated in an MFA image as shown in FIG. 2 is disclosed.

To this end, the MFA image is represented by one Luminance (brightness information) signal (hereinafter, L1) and three Chrominance (color information) signals (hereinafter, C1, C2, C3). In addition, a method of removing aliasing caused by overlap between L1 signals and C1, C2, and C3 signals using a level balancing method is disclosed.

As a preferred embodiment of the present invention, the level balancing method refers to a method of removing high-frequency components of C1, C2, and C3 signals (color information) from MFA images composed of L1, C1, C2, and C3 signals. For this, refer to the embodiment of FIG. 5.

Thereafter, in the preferred embodiment of the present invention, the remaining aliasing part is removed by the level balancing method, and the remaining aliasing part is removed by the signal compensation method using a weighted polymerization method between the L1 signal and the C1, C2, and C3 signals. For this, refer to the embodiment of FIG. 6.

As a preferred embodiment of the present invention, the weighting method means that when the size of the L1 signal is larger than a preset size, a weight higher than a preset weight value k is assigned to the L1 signal value, and when the L1 signal value is small, the L1 signal value is preset. It refers to a method of adding weights lower than the weight value k and combining them with C1, C2, and C3 signals.

Through the above process, in one preferred embodiment of the present invention, an MFA image with minimal aliasing can be obtained.

3 is a preferred embodiment of the present invention, showing a block diagram for minimizing aliasing of the MFA input image and restoring high-frequency information.

In one preferred embodiment of the present invention, in order to minimize aliasing of the MFA input image, the Level Balancing unit 320 receives the MFA input image 310 and performs Fourier signal analysis of the received MFA input image 310. .

Thereafter, in the level balancing unit 320, the high-frequency component (FIG. 5, S420) of the chrominance signal (FIG. 5, S420) in the luminance signal portion in aliasing generated between the Luminance signal (FIG. 5, S410) and the Chrominance signal (FIG. 5, S420). 5, 410), and the aliasing minimization unit (FIGS. 3 and 330) replaces the aliasing portion not removed by the level balancing unit 320 with a high-resolution luminance signal to minimize aliasing and improve resolution.

Specifically, the Level Balancing unit 320 analyzes Aliasing components through Fourier signal analysis in the same manner as in Equation 2. In this case, it can be assumed that the high-frequency components between channels are similar. Subsequently, the high frequency component of the chrominance signal in the luminance signal portion is removed from aliasing (FIGS. 4 and 400) that occurs due to sampling in the MFA input image (FIGS. 5 and 410).

The features of the level balancing unit (FIGS. 3 and 320) will be described with reference to FIGS. 4 to 5 as follows.

4 shows an example of analyzing the aliasing 400 generated between the Luminance signal S410 and the Chrominance signal S420 representing color information through Fourier signal analysis indicating brightness information.

FIG. 5 shows the aliasing 400 generated between the Luminance signal S410 and the Chrominance signal S420 of FIG. 4 in the aliasing minimization unit 330, and removes the high frequency component 410 of the chrominance signal S420 in the luminance signal portion. An example is shown.

In a preferred embodiment of the present invention, the Level Balancing unit (FIGS. 3 and 320) removes the high-frequency component (FIGS. 5 and 410) of the chrominance signal (FIGS. 5 and S420) in the luminance signal (FIGS. 5 and S410 ). . In this case, the chrominance signal (FIG. 5, S420) penetrates the luminance signal (FIG. 5, S410) and removes aliasing, thereby improving the resolution by correcting the edge in the detail region of the high-frequency component. .

In a preferred embodiment of the present invention, aliasing generated in the luminance signal (FIGS. 4 and 5, S410) is removed from the level balancing unit (FIGS. 3 and 320), and then aliased in the same manner as the embodiment shown in FIG. The chrominance signals (FIGS. 4 and 6, S420) are minimized by compensating the luminance signals (FIGS. 4 and 6, S410) through the minimizing unit (FIGS. 3 and 330).

The aliasing minimization unit (FIGS. 3 and 330) aliases generated due to overlap of luminance signals (FIGS. 4 and S410) and chrominance signals (FIGS. 4 and S420) due to subsampling in the MFA input image (FIGS. 3 and 310). 4, 400), the aliasing (FIG. 6, 420) part not removed in the level balancing unit (FIG. 3, 320) is replaced with a high-resolution luminance signal to remove aliasing (FIG. 6, 420) and improve resolution. .

That is, there is an effect of increasing the resolution by minimizing aliasing and restoring the edge by restoring the luminance signal without aliasing in the region where aliasing occurs in the MFA image.

The aliasing minimization unit (FIGS. 3 and 330) will be described with reference to FIGS. 7 to 8 as follows.

In a preferred embodiment of the present invention, the MFA image f _MFA' (i) in which aliasing of the high frequency component 410 of the chrominance signal (FIGS. 4 and 5, S420) is primarily removed in the level balancing unit (FIGS. 3 and 320) ,j) (Figs. 7, 710), after weighting each of the chrominance signal representing the color information and the luminance signal representing the brightness information, the weighted sum is calculated to compensate the MFA image.

As a preferred embodiment of the present invention, the method of compensating the MFA image by the weighted polymerization method detects aliasing of the remaining chrominance signals C1, C2, and C3 even after passing through the level balancing unit (FIGS. 3 and 320). Thereafter, when the signal value 720 of the luminance signal L1 is greater than the detected aliasing signals of the chrominance signals C1, C2, and C3, a weight w(i,j) higher than a preset weight value k is assigned to the L1 signal value. When the L1 signal value is small, a weight w(i,j) lower than the preset weight value k is assigned and summed with the C1, C2, and C3 signals. As a preferred embodiment, referring to an example shown in FIG. 8, a preset weight value k may be set to 0.5 (810).

This is expressed by Equation (3).

In this case, the weight value w(i,j) represents the form of an S-curve as shown in FIG. 8, and a value between 0 and 1 can be set. In FIG. 8, the x-axis represents w(i,j), and the y-axis represents values between 0 and 1.

8 shows an example of applying an S curve to a weight value. The S curve can be expressed as Equation (4).

In one preferred embodiment of the present invention, MFA images with minimized aliasing for which luminance signals are compensated can be obtained through weighting through Equations 3 to 4.

In the above, preferred embodiments of the present invention have been illustrated and described, but the present invention is not limited to the specific embodiments described above, and it is usually in the technical field to which the present invention belongs without departing from the gist of the present invention claimed in the claims. Of course, various modifications can be made by those having knowledge of, and these modifications should not be individually understood from the technical spirit or prospect of the present invention.

Claims (6)

Receiving an MFA image consisting of a luminance signal and a luminance signal from the level balancing unit;
Removing an aliasing between the signal representing the brightness information and the signal representing the color information by using the high frequency component of the signal representing the color information in the level balancing unit;
Compensating the MFA image by weighting each of the signal representing the color information and the signal representing the brightness information in the MFA image in which the aliasing is removed by the aliasing minimization unit, and calculating a weighted sum; Including,
Compensating the MFA image, after detecting the aliasing of the signal representing the color information, when the magnitude of the signal representing the brightness information is greater than the magnitude of the aliasing signal of the signal representing the color information, the When a signal value indicating brightness information is given a weight higher than a preset weight value, and the magnitude of the signal indicating brightness information is smaller than the magnitude of the aliasing signal of the signal indicating color information, it is lower than the preset weight value. A method for removing aliasing in an MFA image, characterized by compensating the MFA image by performing a weighted sum calculation by applying a weight. delete The method of claim 1, wherein removing the aliasing
A method for removing aliasing in an MFA image, characterized in that a high frequency component of the signal representing the color information is removed from the aliasing signal generated between the signal representing the brightness information and the signal representing the color information. By performing Fourier signal analysis of the received MFA input image, the received MFA image is represented by a signal representing one brightness information and a signal representing three color information, and a signal representing the one brightness information and the three A level balancing unit that removes an aliasing signal generated between signals representing color information; And
It includes; an aliasing minimization unit that compensates the MFA image by weighting between the MFA image from which the aliasing signal is removed and the signal representing the one brightness information; and
The level balancing unit removes a high frequency component of the signal representing the three color information from the aliasing signal generated between the signal representing the one brightness information and the signal representing the three color information, and the aliasing minimization unit After detecting the aliasing of the signal representing the three color information remaining in the MFA image from which the aliasing signal is removed through the level balancing unit, the three color information in which the magnitude of the signal representing the one brightness information is detected When the signal of the signal representing is greater than the magnitude of the aliasing signal, a weight higher than a preset weight value is assigned to the signal value representing the one brightness information, and when the signal value representing the one brightness information is small, the preset value is An anti-aliasing device for MFA images, wherein weighting is performed by assigning a weight lower than a weight value. delete delete

Images