JP2003087809A - Image processing apparatus and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus and an image processing method for enhancing the resolution of an imaging signal by using a luminance component signal. SOLUTION: The image processing apparatus includes; a medium tone image generating means 5a for using an imaging signal of other pixel having color information other than a color of a pixel of a received digital imaging signal D1 to interpolated three primary color signals to the pixel so as to generate a medium tone image signal D2; a luminance component calculation means 5b for generating an image luminance component signal D3 from the medium tone image signal D2; an attached color component calculation means 5e for generating an attached color component signal D5 from the digital imaging signal D1; and a luminance color composite means 5f for composing the image luminance component signal D3 of each pixel obtained by the luminance component calculation means 5b with the attached color component signal D5 of each pixel obtained by the attached color component calculation means 5e to generate a color image signal D6 with respect to each pixel of the image.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus and an image processing method used for a digital still camera using a solid-state image sensor.

[0002]

2. Description of the Related Art In recent years, CCD (Ch
A large number of solid-state image pickup devices such as image pickup devices are widely used.

As a method of increasing the resolution of an image obtained from a television camera, a video camera, an electronic camera or the like, there is a method of properly interpolating a pixel signal missing from a solid-state image pickup element and improving a method of analyzing the image pickup signal. . If an appropriate analysis is not performed in this analysis method, a false signal may be generated, colors, contours, and brightness may be erroneously created, resulting in an image different from the captured image.

As an image pickup signal analysis method, an image pickup signal processing circuit (Japanese Patent Laid-Open No. 9-84031) can be cited. In this image pickup signal processing circuit, a signal obtained from a G sampling section for obtaining a green sampling output signal and a pixel in the vicinity of a pixel corresponding to a non-sampling engine in the green sampling output signal is used as an interpolation signal for the green sampling output signal. A signal combining unit for combining with the sampling output signal is provided. According to this image pickup signal processing circuit, a high-resolution green primary color signal can be formed based on the image pickup signal.

[0005]

However, the resolution of the image pickup signal obtained in the above-mentioned image pickup signal processing circuit is limited, and it is not possible to meet the needs of the user who desires an image having a higher resolution.

On the other hand, it is said that the human eye is less sensitive to color difference resolution than luminance. In order to make maximum use of this property, it is sufficient to construct a means for calculating a luminance component signal by some means and obtaining a color image signal to which a color difference component is added while keeping the luminance component.

Therefore, an object of the present invention is to provide an image processing apparatus and an image processing method which do not generate a false signal by using a luminance component signal.

[0008]

The first feature of the present invention is to:
The present invention relates to an image processing device that outputs an image from a color filter in which a plurality of pixels having any one of the three primary colors are arrayed, interpolates based on an image pickup signal having one color information for each pixel, and processes an image. That is, the image processing device according to the first aspect of the present invention interpolates signals of three primary colors with respect to a pixel by using an image pickup signal of another pixel having color information other than the color of the pixel (a), Intermediate image creating means for creating an intermediate image signal, (b) brightness component calculating means for creating a brightness component signal of the image from the intermediate image signal, and (c) additional color for creating an additional color component signal of the image from the image pickup signal. Component calculation means, (d)
The luminance component signal of each pixel obtained by the luminance component calculating means and the additional color component signal of each pixel obtained by the additional color component calculating means are combined to create a color image signal for each pixel of the image. And a luminance color synthesizing means.

Here, in the luminance color synthesizing means, each pixel of the image is filtered from the additional color component signal by using signals of pixels other than the pixel, and the additional color component signal excluding the luminance component signal is obtained. You can create it.

According to the image processing apparatus of the first aspect of the present invention, the luminance component calculating unit calculates the luminance component signal of the color image signal, and the additional component obtained by the additional color component calculating unit is added to the luminance component signal. By adding and synthesizing color component signals, it is possible to provide an image processing apparatus that makes the most of the property that "human eyes are less sensitive to color difference resolution than luminance".

The brightness component signal of the color image signal obtained by the brightness color synthesizing means may be the same as the brightness component signal obtained by the brightness component signal calculating means. If the luminance component signal better reflects the subject, this
An image closer to the subject can be obtained.

Further, a contour enhancement means for enhancing the contour of the image with respect to the luminance component signal may be further provided. In this case, the luminance image signal of the color image signal obtained by the luminance color synthesizing means is preferably the same as the luminance component signal obtained by the contour enhancing means.

A noise removing means for removing noise from the luminance component signal may be further provided. In this case, the luminance image signal of the color image signal obtained by the luminance color synthesizing means is preferably the same as the luminance component signal obtained by the noise removing means.

Noise removal means for removing noise from the luminance component signal and contour enhancement means for enhancing the contour of the image may be further provided. In this case, it is preferable that the luminance image signal of the color image signal obtained by the luminance color synthesizing means is the same as the luminance component signal obtained by the noise removing means and the edge enhancing means.

As described above, noise removal means for removing noise from the luminance component signal or contour enhancement means for enhancing the contour of the image may be provided. By increasing the accuracy of the luminance component signal, the resolution of the image obtained by the image processing device of the present invention can be increased. Increasing the accuracy of the luminance component signal has the advantage of being easier to implement than increasing the accuracy of the color image signal.

The second feature of the present invention is that interpolation is performed based on an image pickup signal which is output from a color filter in which a plurality of pixels having any one of the three primary colors are arranged and one pixel has one color information. Then, the present invention relates to an image processing method used in an image processing apparatus for processing an image. That is, in the image processing method according to the second feature of the present invention, (a) an image pickup signal of another pixel having color information other than the color of the pixel is used to interpolate signals of three primary colors to the pixel, Intermediate image creating step of creating an intermediate image signal, (b) brightness component calculating step of creating an image brightness component signal from the intermediate image signal, (c) additional color creating an additional color component signal of the image from the image pickup signal The luminance component signal of each pixel obtained in the component calculation step, (d) luminance component calculation step, and the additional color component signal of each pixel obtained in the additional color component calculation step are combined, and for each pixel of the image And a luminance color synthesizing step for creating a color image signal.

Further, in the luminance color synthesizing step, each pixel of the image is filtered from the additional color component signal by using signals of pixels other than the pixel to generate an additional color component signal excluding the luminance component signal. You may.

The luminance component signal of the color image signal obtained in the luminance color synthesizing step is preferably the same as the luminance component signal obtained in the luminance component signal calculating step.

A noise removing step for removing noise from the luminance component signal may be further provided. In this case, it is preferable that the luminance image signal of the color image signal obtained in the luminance color combining step is the same as the luminance component signal obtained in the noise removing step.

A contour emphasizing step for emphasizing the contour of the image with respect to the luminance component signal may be further provided. in this case,
The luminance image signal of the color image signal obtained by the luminance color synthesizing step is preferably the same as the luminance component signal obtained by the contour enhancing step.

A noise removal step of removing noise from the luminance component signal and a contour enhancement step of enhancing the contour of the image may be further provided. In this case, it is preferable that the luminance image signal of the color image signal obtained by the luminance color combining step is the same as the luminance component signal obtained by the noise removal step and the edge enhancement step.

[0022]

BEST MODE FOR CARRYING OUT THE INVENTION Next, a best mode for carrying out the present invention will be described with reference to the drawings. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals. However, the block diagram and the like showing the system configuration are schematic.

FIG. 1 is a functional block diagram of an electronic camera which is an image pickup apparatus equipped with an image processing apparatus according to the first embodiment of the present invention.

Lens section 1, color filter 2 and CCD
A photographing unit is constituted by 3. The lens unit 1 constitutes an optical system for forming an image of a subject on an image forming surface, and although not shown,
A control signal is supplied from the microcomputer 7, and an automatic aperture control operation and an automatic focus control operation are performed. CCD (Charge Cou
The pled device 3 is an image sensor (image sensor), has a predetermined number of pixels of the image sensor, and is provided at the position of the image forming surface of the lens unit 1. Color filter 2
Is provided between the lens unit 1 and the CCD 3. As shown in FIG. 2, the color filter 2 has an array called a Bayer array in which a plurality of pixels having any one color of red (R), green (G) and blue (B) are arranged in a checkered pattern. The light incident on the color filter 2 transmits only one of red (R), green (G) and blue (B), and the CCD 3
Is incident on. Each pixel of the color filter 2 is
It corresponds to the image pickup device of the CCD 3. Further, the CCD 3 photoelectrically converts an image obtained by being incident from the lens unit 1 through the color filter 2 to generate and output an image signal. That is, the image signal obtained from the CCD 3 is a signal of any one of red (R), green (G) and blue (B).
The image signal output from the CCD 3 is input to the A / D conversion unit 4, and the A / D conversion unit 4 generates a digital photographing signal of 10 bits per sample. A / D converter 4
Has a structure of a correlated double sampling circuit and performs noise removal, waveform shaping, compensation of defective pixels, and the like. The arrangement of the red (R), green (G), and blue (B) of the image data at this point is the arrangement corresponding to the color filter 2.

The digital image pickup signal output from the A / D converter 4 is input to the camera signal processor 5. The camera signal processing unit 5 is an image processing device according to the present invention. or,
The camera signal processing unit 5 includes a digital clamp circuit, a luminance signal processing circuit, a color signal processing circuit, a contour correction circuit, a defect compensation circuit, an automatic aperture control circuit, an automatic focus control circuit, an automatic white balance correction circuit, and the like. The camera signal processing unit 5 generates and outputs a digital image signal in the form of a component signal including a luminance signal and a color difference signal converted from an RGB signal which is a digital photographing signal output from the A / D conversion unit 5. . Although not shown, the digital photographing signal output from the camera signal processing unit 5 is output to a storage device or the like and stored in the storage device.

The timing generator 6 generates a read timing pulse for the CCD 3 and, at the same time, generates an A / D signal.
The clock of the conversion unit 4 and the clock taken into the camera signal processing unit 5 are generated. Further, a horizontal synchronizing signal HD and a vertical synchronizing signal VD are generated for performing synchronous processing.
These signals generated from the timing generator 6 are transferred to the CCD unit 3, the A / D conversion unit 4, the camera signal processing unit 5 and the like.
Is supplied to. The microcomputer 7 controls the timing generator 6 and the camera signal processing unit 5.

FIG. 3 is a diagram showing processing means and data flow in the camera signal processing unit 5 (image processing apparatus) according to the embodiment of the present invention.

The camera signal processing section 5 comprises an intermediate image creating means 5a, a luminance component calculating means 5b, a noise removing means 5c, a contour emphasizing means 5d, an additional color component calculating means 5e, and a luminance color synthesizing means 5f.

The intermediate image forming means 5a photoelectrically converts the image obtained by being incident from the lens section 1 through the color filter 2 and uses the digital image pickup signal D1 inputted through the A / D conversion section 4. Then, an intermediate image is created and an intermediate image signal D2 is output. The intermediate image creating means 5a uses the digital image pickup signal of another pixel having color information other than the color of the target pixel to interpolate the signals of the three primary colors for this pixel. By performing this for all the pixels, the intermediate image signal D2 is created. In the intermediate image creating means 5a, the intermediate image signal D2 is emphasized with emphasis on the luminance component signal.
Are preferably created.

The luminance component calculating means 5b is provided with an intermediate image signal D
Of 2, the luminance component signal D3 of each pixel is calculated. Here, the brightness of the brightness component signal D3 is defined as follows.

Y = kr * R + kg * G + kb * B where kr, kg and kb are constants, and kr + kg + k
Let b = 1 be a constant. Furthermore, kr = 1/4, kg
= 1/2 and kb = 1/4 are preferable. Thus, by adopting the numerical values represented by the squares of 1/2 and 1/2 as kr, kg, and kb, when the numerical values are treated as 2 bits, it can be realized by left shift, The circuit chip of the camera signal processing unit 5 can be simplified.

The noise removing means 5c is means for removing noise through a low pass filter. The contour emphasizing means 5d is means for emphasizing the contour of the photographed subject.
Specifically, as shown in FIG. 4, for example, when a contour is present in a portion where a low-luminance low-dark signal transitions to a high-luminance high-luminance signal, a signal before and after the contour is emphasized by means for emphasizing it. is there. The brightness component signal D3 output from the brightness component calculating unit 5b is output as an adjusted brightness component signal D4 via the noise removing unit 5c and the contour enhancing unit 5d.

On the other hand, the additional color component calculation means 5e outputs R from the digital image pickup signal D1 to each pixel.
The (red), G (green) and B (blue) signals are interpolated to create an additional color component signal D5.

The adjusted luminance component signal D4 output from the contour emphasizing means 5d and the additional color component signal D5 output from the additional color component calculating means 5e are the luminance color synthesizing means 5f.
Entered in. The luminance color synthesizing unit 5f synthesizes the adjusted luminance component signal D4 and the additional color component signal D5 to create a color image signal D6 for each pixel of the image.

The luminance color synthesizing means 5f calculates the color signals R ', G', B'of the color image signal D6.
Here, the luminance color synthesizing unit 5f may be filtered to reduce false colors. The filter is, for example, as shown in FIG. 5, and is a signal having a size that is ⅛ times the signal of each pixel in the surroundings of the corresponding pixel whose signal is calculated by the luminance color synthesizing unit 5f Is calculated, and the signal obtained by each sum is used as the signal of the corresponding pixel.

The luminance color synthesizing means 5f calculates the color signals R ', G', B'based on the following formula.

[R-G] and [B-G] are filtered as follows, and color signals obtained as a result of filtering as shown in FIG. 5 are designated as R ', G', and B '. .

R'-G '= Σ {Wi · [Ri-Gi]} B'-G' = Σ {Mi · [Bi-Gi]} where i is the pixel to be processed and its vicinity , And Wi and Mi are filters as shown in FIG. 5, and are constants that satisfy ΣWi = ΣMi = 1.

As a result, the following color signals R ', G'and B'are obtained.

[0040]

## EQU1 ## R '= Y + (1-kr) .SIGMA. {Wi.Ri}-
(1-kr) ・ Σ {Wi ・ Gi} + kb ・ Σ {Mi ・ G
i} −kb · Σ {Mi · Bi} G ′ = Y−kr · Σ {Wi · Ri} + kr · Σ {Wi ·
Gi} + kb · Σ {Mi · Gi} −kb · Σ {Mi · B
i} B ′ = Y−kr · Σ {Wi · Ri} + kr · Σ {Wi ·
Gi} + (1-kb) · Σ {Mi · Gi} + (1-k
b) .SIGMA. {Mi.Bi} In this way, the luminance component signal D4 and the additional color component signal D
With 5 as input, the color image signal D6 represented by R ', G'and B'can be calculated.

As a result, the luminance component signal in the color image signal D6 and the adjusted luminance component signal D4 can be made equal.

According to the embodiment of the present invention, since the color difference is filtered, the resolution for the color difference is reduced. This suppresses spurious signals.

Further, according to the embodiment of the present invention, since the luminance is not changed by the luminance color synthesizing means, the resolution of the luminance component is not affected. As described above, according to the present invention, it is possible to perform image processing that does not generate a false signal by taking advantage of the property that “the resolution of the color difference is insensitive to the human eye compared to the luminance”.

(Other Embodiments) As described above, the embodiments of the present invention have been described, but it should be understood that the description and drawings forming a part of this disclosure limit the present invention. Absent. From this disclosure, various alternative embodiments, examples and operational techniques will be apparent to those skilled in the art.

In the embodiment of the present invention, the noise removal and the edge enhancement are performed on the luminance component signal, but it may be performed on the color image signal, the additional color component signal or the intermediate image signal. Noise removal and contour enhancement may be performed on appropriate signals output in the embodiment of the present invention. In this case, it is preferable that the luminance component signal in the signal input to the luminance color synthesizing unit and the luminance component signal in the color image signal output from the luminance color synthesizing unit are equal.

Further, the color filter used in the present invention does not have to have the Bayer array. The shape of the color filter, the arrangement of colors, and the like may be arbitrary.

Further, although the filter as shown in FIG. 5 is used in the luminance color synthesizing means, it is not limited to this filter. For example, the corresponding pixel may be added as 1/2 and other peripheral pixels may be added as 1/16. Also, the filter is 3 *
Instead of the size of 3, a filter of 5 * 5 or 7 * 7 may be used.

As described above, it goes without saying that the present invention includes various embodiments not described here. Therefore, the technical scope of the present invention is defined only by the matters specifying the invention according to the scope of claims reasonable from the above description.

[0049]

According to the present invention, it is possible to provide an image processing apparatus and an image processing method which do not generate a false signal by using a luminance component signal.

[Brief description of drawings]

FIG. 1 is a functional block diagram of an electronic camera that is one type of an image capturing apparatus according to an embodiment of the present invention.

FIG. 2 is a diagram showing a color filter of the image capturing apparatus according to the embodiment of the present invention.

FIG. 3 is a diagram showing a processing means and a data flow in the camera signal processing unit according to the embodiment of the present invention.

FIG. 4 is a diagram illustrating a contour enhancement method by a contour enhancement unit according to an embodiment of the present invention.

FIG. 5 is a diagram illustrating a filter by the luminance color synthesizing unit according to the embodiment of the present invention.

[Explanation of symbols]

1 lens part 2 color filters 3 CCD 4 A / D converter 5 Camera signal processor 5a Intermediate image creating means 5b Luminance component calculation means 5c Noise removal means 5d contour enhancement means 5e Additional color component calculation means 5f luminance color synthesizing means 6 Timing generator 7 Microcomputer

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) H04N 1/46 H04N 5/21 B 5C079 1/60 9/68 103A 5/208 101: 00 5/21 1 / 40 D 9/68 103 1/46 Z // H04N 101: 00 1/40 101C 101D F term (reference) 5B057 BA11 CA01 CA08 CA16 CB01 CB08 CB16 CE02 CE03 CE06 CE08 CE16 5C021 PA17 PA92 XB04 YA01 YC00 5C065 AA01 AA03 BB13 CC02 DD02 DD17 GG13 GG15 GG21 5C066 AA01 AA11 CA06 CA07 EA03 EA11 EC02 EE02 GA01 GA05 GA32 GB03 HA02 JA02 KA12 KE16 5C077 LL19 MM03 MP07 MP08 PP03 PP23 PP32 PP34 RR19 TT09 5C079 HB01 LA15 LA40 JA12 NA40

Claims (12)

[Claims] 1. An image which is output from a color filter in which a plurality of pixels having any one of the three primary colors are arranged, and which is interpolated based on an image pickup signal having one color information in one pixel to process the image. A processing device, which uses the imaging signal of another pixel having color information other than the color of the pixel, interpolates signals of three primary colors for the pixel, and creates an intermediate image signal Means, a luminance component calculation means for generating a luminance component signal of the image from the intermediate image signal, an additional color component calculation means for generating an additional color component signal of the image from the image pickup signal, and the luminance component calculation The luminance component signal of each pixel obtained by the means and the additional color component signal of each pixel obtained by the additional color component calculation means are combined, and a color image signal is obtained for each pixel of the image. The image processing apparatus characterized by comprising a luminance color synthesizing means for creating. 2. The luminance color synthesizing means filters each pixel of the image from the additional color component signal by using a signal of a pixel other than the pixel to add an additional color excluding the luminance component signal. The image processing apparatus according to claim 1, wherein a component signal is created. 3. The brightness component signal of the color image signal obtained by the brightness color synthesizing means is the same as the brightness component signal obtained by the brightness component signal calculating means. The image processing device according to item 1. 4. A noise removing unit that removes noise from the luminance component signal, wherein the luminance image signal of the color image signal obtained by the luminance color combining unit is the luminance obtained by the noise removing unit. The image processing apparatus according to claim 1, wherein the image processing apparatus is the same as the component signal. 5. The image processing apparatus further comprises contour enhancing means for enhancing the contour of the image with respect to the luminance component signal, wherein the luminance image signal of the color image signal obtained by the luminance color combining means is obtained by the contour enhancing means. 2. The luminance component signal obtained is the same as the obtained luminance component signal.
Alternatively, the image processing device according to item 2. 6. A noise removing means for removing noise from the luminance component signal, and a contour enhancing means for enhancing the contour of the image, further comprising: a color image signal of the color image signal obtained by the luminance color synthesizing means. The image processing device according to claim 1 or 2, wherein the luminance image signal is the same as the luminance component signal obtained by the noise removing means and the edge enhancing means. 7. An image which is output from a color filter in which a plurality of pixels having any one of the three primary colors is arranged, and which is interpolated based on an image pickup signal having one color information in one pixel to process the image. An image processing method used in a processing device, wherein a signal of three primary colors is interpolated for the pixel using the image pickup signal of another pixel having color information other than the color of the pixel, and an intermediate image signal An intermediate image creating step of creating an image, a brightness component calculating step of creating a brightness component signal of the image from the intermediate image signal, and an additional color component calculating step of creating an additional color component signal of the image from the imaging signal And combining the luminance component signal of each pixel obtained in the luminance component calculation step with the additional color component signal of each pixel obtained in the additional color component calculation step. And, an image processing method characterized by comprising a luminance color synthesizing step of generating a color image signal for each pixel of the image. 8. In the luminance color synthesizing step, each pixel of the image is filtered from the additional color component signal by using a signal of a pixel other than the pixel, and an additional color excluding the luminance component signal is added. The image processing method according to claim 7, wherein a component signal is created. 9. The luminance component signal of the color image signal obtained in the luminance color synthesizing step is the same as the luminance component signal obtained in the luminance component signal calculating step. The image processing method described in. 10. A noise removing step of removing noise from the luminance component signal, wherein the luminance image signal of the color image signal obtained in the luminance color combining step is the luminance obtained in the noise removing step. 9. The image processing method according to claim 7, wherein the image processing method is the same as the component signal. 11. A contour enhancing step of enhancing the contour of the image with respect to the luminance component signal, the luminance image signal of the color image signal obtained by the luminance color combining step is obtained by the contour enhancing step. 9. The image processing method according to claim 7, wherein the image processing method is the same as the obtained luminance component signal. 12. A noise removal step of removing noise from the luminance component signal, and an edge enhancement step of enhancing the edge of the image are further provided, and the color image signal of the color image signal obtained by the luminance color combination step is included. 9. The image processing method according to claim 7, wherein the luminance image signal is the same as the luminance component signal obtained by the noise removal step and the contour enhancement step.