JP2003110856A - Image processing device and method therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device and an image processing method capable of improving the degree of resolution by forming a smooth signal. SOLUTION: The image processing device for processing an image outputs images from a color filter 2 in which a plurality of pixels having either one color among a plurality of colors and interpolates them based on a digital image pickup signal D1 having a color information with respect to one pixel. The device is provided with a direction judging means 5a for detecting a correlative direction D2 of either a vertical or a horizontal direction from the digital image pickup signal D1, an interpolating means 5b for outputting an interpolating signal D3 by obtaining the color information of the pixel employing the pixel having the color information, and a profile emphasizing means 5c for effecting profile emphasizing with respect to the interpolating signal D3 to output a regulated interpolating signal D4.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus and an image processing method used for a digital still camera using a solid-state image sensor.

[0002]

2. Description of the Related Art In recent years, CCD (Ch
A large number of solid-state image pickup devices such as image pickup devices are widely used.

As a method of increasing the resolution of an image obtained from a television camera, a video camera, an electronic camera or the like, there is a method of properly interpolating a pixel signal missing from a solid-state image pickup element and improving a method of analyzing the image pickup signal. . In this analysis method, if proper analysis is not performed, false color will occur and the image will be different from the captured image.

Currently, when an image is obtained from a television camera, a video camera, an electronic camera or the like, the light from the subject is CC through a color filter as shown in FIG.
It is incident on a solid-state image sensor such as D. In the color filter shown in FIG. 6, pixels having R (red), G (green), and B (blue) colors are in a checkered array called a Bayer array.

The image pickup signal obtained from this color filter is R (red), G (green), B for one pixel.
If you want to get the image, because it has one of the colors (blue)
The G (green) signal is extracted and processed as a video signal. Therefore, it is necessary to interpolate the R (red) and B (blue) pixels as the missing parts. Especially in the case of Bayer array, G
Since the number of R (red) and B (blue) pixels is smaller than that of (green) pixels, the interpolation method for R (red) and B (blue) pixels is important. This interpolation method depends on the resolution of the photographing device, and various methods have been considered.

On the other hand, one of the methods for increasing the resolution is a method for emphasizing the contour of the subject in the interpolated signal.

[0007]

However, when the contour of a subject is emphasized, if the signal before the contour emphasis is not smoothly formed, the contour emphasis may generate an image signal that does not exist originally. .

FIG. 7 shows an example of a signal when the contour enhancement processing is performed on a signal in which the signal before the contour enhancement is not smoothly formed. In the contour enhancement method, a signal obtained by adding a second-order difference value to the signal before contour enhancement is used as the value of the signal after contour enhancement. The signal amount at the coordinate position i is represented as R (i). Using this expression, the signal amount of the first-order difference of the coordinate position i [first-order difference] (i) = R (i) −R (i−1). Also, the signal amount of the second-order difference [inverse sign of the second-order difference]
(I) =-R (i + 1) + 2R (i) -R (i-1). In this case, the amount of signal after edge enhancement [edge enhancement]
(I) = R (i) + {-R (i + 1) + 2R (i) -R
(I-1)}. FIG. 7 shows the signal amount after the edge enhancement in this way. Referring to the figure, the contour enhancement processing exhibits a behavior different from the behavior of the signal before the contour enhancement, and the contour enhancement processing generates an image signal that does not originally exist.

On the other hand, FIG. 8 shows an example of a signal when the contour enhancement processing is performed on a signal on which the signal before the contour enhancement is smoothly formed. In FIG. 8, the same contour enhancement processing as in FIG. 7 is performed. In FIG. 8, the signal before contour enhancement is formed more smoothly than in FIG. 7. When the contour enhancement process is performed on this signal, the same behavior as that of the signal before the contour enhancement is exhibited, and the contour of the subject is enhanced by the contour enhancement process, and the resolution can be improved.

As described above, the resolution can be increased by performing the edge enhancement processing after forming the smooth signal.

Therefore, an object of the present invention is to provide an image processing apparatus capable of emphasizing an actually existing contour by suppressing the generation of an image signal which does not originally exist in the contour enhancement by forming a smooth signal. It is to provide an image processing method.

[0012]

The first feature of the present invention is to:
The present invention relates to an image processing device that outputs an image from a color filter in which a plurality of pixels having any one of a plurality of colors are arranged, interpolates based on an image pickup signal having one color information in one pixel, and processes an image. . That is, according to the image processing apparatus of the present invention, based on the correlation direction obtained by (a) the direction determining means for detecting the correlation direction in which the change of the image capturing signal is small, and (b) the correlation direction obtained by the direction determining means. This is an image processing apparatus including an interpolating unit that obtains the certain color information of the pixel using the pixel having the information. At this time, (b)
When the pixel already has the certain color information, the interpolating unit corrects the certain color information of the pixel using the pixel and the pixel having the certain color information other than the pixel. When the pixel does not have the certain color information, it is preferable to perform the interpolation by using pixels other than the pixel having the certain color information. The method of correcting / interpolating the certain color information used at this time is preferably changed depending on the type of the pixel. As the type of the pixel, for example, (1)
If the pixel has the certain color information, (2) if a pixel next to the pixel in the horizontal direction has the certain color information,
(3) A pixel adjacent to the pixel in the vertical direction has the certain color information, and (4) a pixel adjacent to the pixel in the diagonal direction has the certain color information.

Further, (a) the direction determining means may detect the correlation direction in either the vertical direction or the horizontal direction. In this case, (a) when the direction determining means cannot detect the correlation direction, (b) the interpolating means uses the pixels having certain color information existing around the pixel to detect the existence of the pixel. Color information may be obtained.

Alternatively, (a) the direction determining means is a vertical direction,
The correlation direction may be detected in any one of the horizontal direction, the right diagonal direction, and the left diagonal direction. In this case, (a) when the direction determining means cannot detect the correlation direction,
(B) The interpolating means may obtain the certain color information of the pixel using the pixel having the certain color information existing around the pixel.

A second feature of the present invention is based on an image pickup signal which is output from a color filter in which a plurality of pixels having any one of a plurality of colors are arranged and which has one color information in one pixel. The present invention relates to an image processing method in which an image is interpolated and an image is processed. That is, according to the image processing method of the present invention, based on (a) a direction determining step of detecting a correlation direction in which there is little change in the imaging signal, (b) a certain direction based on the correlation direction obtained by the direction determining means. And an interpolation step of obtaining the certain color information of the pixel using the pixel having the information. Further, in the (b) interpolation step, if the pixel already has the certain color information,
The certain color information is corrected using the pixel and the pixel having the certain color information other than the pixel, and when the pixel does not have the certain color information, the certain color information other than the pixel is corrected. Interpolation may be performed by using the owned pixels.

Further, (a) the direction determining method may detect the correlation direction in either the vertical direction or the horizontal direction. In this case, when the correlation direction cannot be detected in the (a) direction determining step, the (b) interpolation step uses pixels having certain color information existing around the pixel,
The certain color information of the pixel may be obtained.

Alternatively, (a) the direction determining method is the vertical direction,
The correlation direction may be detected in any one of the horizontal direction, the right diagonal direction, and the left diagonal direction. In this case, (a) when the correlation direction cannot be detected in the direction determination step,
(B) In the interpolation step, a pixel having certain color information existing around the pixel may be used to obtain the certain color information of the pixel.

[0018]

BEST MODE FOR CARRYING OUT THE INVENTION Next, a best mode for carrying out the present invention will be described with reference to the drawings. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals. However, the block diagram and the like showing the system configuration are schematic.

FIG. 1 is a functional block diagram of an electronic camera which is a photographing device equipped with an image processing device according to an embodiment of the present invention.

Lens part 1, color filter 2 and CCD
A photographing unit is constituted by 3. The lens unit 1 constitutes an optical system for forming an image of a subject on an image forming surface, and although not shown,
A control signal is supplied from the microcomputer 7, and an automatic aperture control operation and an automatic focus control operation are performed. CCD (Charge Cou
The pled device 3 is an image sensor (image sensor), has a predetermined number of pixels of the image sensor, and is provided at the position of the image forming surface of the lens unit 1. Color filter 2
Is provided between the lens unit 1 and the CCD 3. As shown in FIG. 6, the color filter 2 has an array called a Bayer array in which a plurality of pixels having any one color of R (red), G (green) and B (blue) are arranged in a checkered pattern. The light incident on the color filter 2 transmits only light of one of R (red), G (green) and B (blue), and the CCD 3
Is incident on. Each pixel of the color filter 2 is
It corresponds to the image pickup device of the CCD 3. Further, the CCD 3 photoelectrically converts the image obtained by being incident from the lens unit 1 via the color filter 2 to generate and output an image pickup signal. That is, the image pickup signal obtained from the CCD 3 is a set of signals having any one color of R (red), G (green) and B (blue) for one pixel. The image pickup signal output from the CCD 3 is input to the A / D conversion unit 4, and the A / D conversion unit 4 generates a digital image pickup signal of 10 bits per sample. The A / D conversion unit 4 has a structure of a correlated double sampling circuit, and performs noise removal, waveform shaping, compensation of defective pixels, and the like. R of image data at this point
The arrangement of (red), G (green), and B (blue) is the arrangement corresponding to the color filter 2.

The digital image pickup signal output from the A / D converter 4 is input to the camera signal processor 5. The camera signal processing unit 5 is an image processing device according to the present invention. or,
The camera signal processing unit 5 includes a digital clamp circuit, a luminance signal processing circuit, a color signal processing circuit, a contour correction circuit, a defect compensation circuit, an automatic aperture control circuit, an automatic focus control circuit, an automatic white balance correction circuit, and the like. The camera signal processing unit 5 generates and outputs a digital image signal in the form of a component signal composed of a luminance signal and a color difference signal converted from the RGB signal which is the digital imaging signal output from the A / D conversion unit 4. Here, by performing color information interpolation processing, all information of R (red), G (green), and B (blue) can be obtained. Although not shown, the digital image pickup signal output from the camera signal processing unit 5 for each pixel is output to a storage device or the like and stored in the storage device.

The timing generator 6 generates a read timing pulse for the CCD 3 and, at the same time, generates an A / D signal.
The clock of the conversion unit 4 and the clock taken into the camera signal processing unit 5 are generated. Further, a horizontal synchronizing signal HD and a vertical synchronizing signal VD are generated for performing synchronous processing.
These signals generated from the timing generator 6 are transferred to the CCD unit 3, the A / D conversion unit 4, the camera signal processing unit 5 and the like.
Is supplied to. The microcomputer 7 controls the timing generator 6 and the camera signal processing unit 5.

FIG. 2 is a diagram showing a processing means and a data flow included in the camera signal processing unit 5 (image processing apparatus) according to the embodiment of the present invention. The camera signal processing unit 5 includes not only these means but also various means such as a brightness calculating means and a noise removing means, which are not shown here, as a matter of course.

The camera signal processing section 5 includes a direction determining means 5
a, an interpolating means 5b and a contour emphasizing means 5c.

The direction determining means 5a uses the digital image pickup signal D
From 1, the direction in which there is little change in the image (contour direction) in the vicinity of each pixel is detected, and this is output as the correlation direction D2. The direction determining means 5a outputs the correlation direction D2 which is either the vertical direction or the horizontal direction.

The interpolating means 5b obtains the certain color information of the pixel by using the pixel having certain color information included in the digital image pickup signal D1 based on the correlation direction D2 obtained by the direction determining means 5a. The interpolation signal D3 is output. This time,
If the pixel already has the certain color information, the interpolating means 5b corrects the certain color information of the pixel by using the pixel and the pixel having the certain color information other than the pixel. If the pixel does not have the certain color information, interpolation is performed using pixels other than the pixel having the certain color information. The interpolator 5b first obtains the signal of the pixel, and corrects the signal of the pixel by using the pixel in the vertical direction in the correlation direction D2. In this way, the interpolation means 5b
Performs interpolation so that the interpolation signal D3 becomes a smooth signal.

The contour emphasizing means 5c performs contour emphasizing processing on the interpolation signal D3, and outputs an adjusted interpolation signal D4.

Next, detailed processing of the interpolating means 5b will be described. FIG. 3 is a diagram showing pixels in the embodiment of the present invention. Here, for a certain color, if the pixel has the color information, the pixel is shaded. Further, regarding the digital image pickup signal D1 obtained by passing through the color filter 2 having the Bayer array as shown in FIG. 6, in association with the positions of the color filter 2 and the CCD 3, as shown in FIG. P11, P12,
Name it P21 .... These pixels are classified as follows. (1) When the pixel has the certain color information, the pixel is an o pixel, and (2) When a pixel horizontally adjacent to the pixel has the certain color information, the pixel is an h pixel. , (3) When a pixel adjacent to the pixel in the vertical direction has the certain color information, the pixel is a v pixel, and (4) When a pixel adjacent to the pixel in the diagonal direction has the certain color information. , The pixel is classified as x pixel. As a result of classifying each pixel in FIG. 3 in this way, each pixel is expressed as shown in FIG.

Next, the interpolation method in the embodiment of the present invention will be described for each of the o pixel, h pixel, v pixel, and x pixel. This method is based on Bayer array R (red), B
It is effective to perform processing on the colors existing in the pixels of every two pixels and every two lines, such as (blue).

(Correction Method for O-Pixel) When correcting a pixel having certain color information, a pixel existing in the direction perpendicular to the contour direction with the pixel at the center is used to calculate the signal of the pixel. Make a correction. As a result, the signal of the pixel can be prevented from protruding and the pixel can be smoothed so as to adapt to the signals of the surrounding pixels.

With respect to P33 in FIG. 3, an example of calculation for correction will be given. The corrected value of P33 is defined as P'33.

(A) When the contour direction is the vertical direction P'33 = 3 / 4.P33 + 1/8 (P13 + P53) (b) When the contour direction is the horizontal direction P'33 = 3 / 4.P33 + 1/8 (P31 + P35) ) (Interpolation method in h pixel) When interpolating a pixel adjacent to a pixel having certain color information in the horizontal direction, a signal value of the pixel is obtained using a signal in the contour direction with the pixel as the center, A pixel existing in a direction perpendicular to the contour direction is used to correct the signal of this pixel. As a result, the signal of the pixel can be prevented from protruding and the pixel can be smoothed so as to adapt to the signals of the surrounding pixels.

With respect to P23 in FIG. 3, an example of calculation when performing interpolation will be described. The value of P23 after interpolation is set to P'23.

(A) When the contour direction is the vertical direction P'23 = 1 / 2.P13 + 1 / 2.P33 (b) When the contour direction is the horizontal direction P'23 = 1/4 (P13 + P33) +1/4 ( P22
+ P24) = 3/8 (P13 + P33) +1/16 (P
11 + P15 + P31 + P35) (Interpolation method for v pixel) When interpolating a pixel adjacent to a pixel having certain color information in the vertical direction, a signal value of the pixel is obtained using a signal in the contour direction centered on the pixel. , The pixel existing in the direction perpendicular to the contour direction is used to correct the signal of this pixel. As a result, the signal of the pixel can be prevented from protruding and the pixel can be smoothed so as to adapt to the signals of the surrounding pixels.

With respect to P32 in FIG. 3, an example of calculation when performing interpolation will be described. The value of P32 after interpolation is P′32.

(A) When the contour direction is the vertical direction P'32 = 1/4 (P31 + P33) +1/4 (P22
+ P42) = 3/8 (P31 + P33) +1/16 (P
11 + P13 + P51 + P53) (b) When the contour direction is the horizontal direction P′32 = 1/2 · P31 + 1/2 · P33 (Interpolation method for x pixel) When interpolating a pixel adjacent to a pixel having certain color information in the diagonal direction, The signal of the pixel is obtained by using the surrounding signals centered on the pixel.

With respect to P22 of FIG. 3, an example of calculation when performing interpolation will be described. The value of P22 after interpolation is P'22.

(A) When the contour direction is the vertical direction P'22 = 1/4 (P11 + P13 + P31 + P33) (b) When the contour direction is the horizontal direction P'22 = 1/4 (P11 + P13 + P31 + P33) FIG. It is a figure showing a signal and a signal after interpolation of an embodiment of the invention. The signal before interpolation is indicated by a black circle,
It is shown by the dotted line. The signals after interpolation are indicated by white circles, and the approximation of these white circles is indicated by the solid line.
With reference to FIG. 5, it can be seen that the image processing method according to the embodiment of the present invention smoothes the signal behavior (the manner in which the image signal changes).

In the embodiment of the present invention, the coefficient of each signal is 1/4, 1/16, or the like. By employing these values, these calculations can be performed with a simple circuit.

(First Modification) As a modification of the present invention,
A case will be described in which the direction determining means 5a calculates an optimum direction from the right diagonal direction and the left diagonal direction in addition to the vertical direction and the horizontal direction.

(Correction Method for O-Pixel) When a pixel having a certain color information is to be corrected, the correction is performed by using a pixel having the pixel at the center and existing in a direction perpendicular to the contour direction. As a result, the signal of the pixel can be prevented from protruding and the pixel can be smoothed so as to adapt to the signals of the surrounding pixels.

An example of calculation when P33 in FIG. 3 is corrected will be described. The corrected value of P33 is defined as P'33.

(A) When the contour direction is the vertical direction P'33 = 3 / 4.P33 + 1/8 (P13 + P53) (b) When the contour direction is the horizontal direction P'33 = 3 / 4.P33 + 1/8 (P31 + P35) ) (C) When the contour direction is diagonally right upward P'33 = A1 · P33 + A2 (P31 + P13 + P5
3 + P35) + A3 (P11 + P55) (d) When the contour direction is diagonally upward to the left P′33 = A1 · P33 + A2 (P31 + P13 + P5
3 + P35) + A3 (P51 + P15) In the above (c) and (d), A1 + 4A2 + 2A
It is preferable that 3 = 1 and A1> 4A2 and A1> 2A3. Accordingly, the value of the signal of the pixel can be corrected by using the pixel existing in the direction perpendicular to the contour direction.

(Interpolation method for h pixel) When interpolating a pixel adjacent to a pixel having certain color information in the horizontal direction,
The signal value of the pixel is obtained using the signal in the contour direction centered on the pixel, and the signal of the pixel is corrected using the pixel existing in the direction perpendicular to the contour direction. . As a result, the signal of the pixel can be prevented from protruding and the pixel can be smoothed so as to adapt to the signals of the surrounding pixels.

With respect to P23 in FIG. 3, an example of calculation when performing interpolation will be described. The value of P23 after interpolation is set to P'23.

(A) When the contour direction is the vertical direction P'23 = 1/2 * P13 + 1/2 * P33 (b) When the contour direction is the horizontal direction P'23 = 1/4 (P13 + P33) +1/4 ( P22
+ P24) = 3/8 (P13 + P33) +1/16 (P
11 + P15 + P31 + P35) (c) When the contour direction is diagonally right upward P'23 = B1 (P13 + P33) + B2 (P31 + P
15) + B3 (P11 + P35) (d) When the contour direction is diagonally upward to the left P'23 = B1 (P13 + P33) + B2 (P11 + P
35) + B3 (P31 + P15) In the above (c) and (d), 2B1 + 2B2 + 2
It is preferable that B3 = 1 and B1> B2 and B1> B3. With this, for the value of the signal of the pixel,
The correction can be performed using pixels existing in the direction perpendicular to the contour direction.

(Interpolation method for v pixel) When interpolating a pixel adjacent to a pixel having certain color information in the vertical direction,
The signal value of the pixel is obtained using the signal in the contour direction centered on the pixel, and the signal of the pixel is corrected using the pixel existing in the direction perpendicular to the contour direction. . As a result, the signal of the pixel can be prevented from protruding and the pixel can be smoothed so as to adapt to the signals of the surrounding pixels.

With respect to P32 in FIG. 3, an example of calculation when performing interpolation will be described. The value of P32 after interpolation is P′32.

(A) When the contour direction is the vertical direction P'32 = 1/4 (P31 + P33) +1/4 (P22
+ P42) = 3/8 (P31 + P33) +1/16 (P
11 + P13 + P51 + P53) (b) When the contour direction is horizontal P'32 = 1/2 * P31 + 1/2 * P33 (c) When the contour direction is diagonally right upward P'32 = B1 (P31 + P33) + B2 (P13 + P
51) + B3 (P11 + P53) (d) When the contour direction is diagonally upward to the left P'32 = B1 (P31 + P33) + B2 (P11 + P
53) + B3 (P13 + P51) In the above (c) and (d), 2B1 + 2B2 + 2
It is preferable that B3 = 1 and B1> B2 and B1> B3. With this, for the value of the signal of the pixel,
The correction can be performed using pixels existing in the direction perpendicular to the contour direction.

(Interpolation method for x pixel) When interpolating a pixel adjacent to a pixel having a certain color information in an oblique direction,
The signal value of the pixel is obtained using the signal in the contour direction centered on the pixel, and the signal of the pixel is corrected using the pixel existing in the direction perpendicular to the contour direction. . As a result, the signal of the pixel can be prevented from protruding and the pixel can be smoothed so as to adapt to the signals of the surrounding pixels.

With respect to P22 in FIG. 3, an example of calculation when performing interpolation will be described. The value of P22 after interpolation is P'22.

(A) When the contour direction is the vertical direction P'22 = 1/4 (P11 + P13 + P31 + P33) (b) When the contour direction is the horizontal direction P'22 = 1/4 (P11 + P13 + P31 + P33) (c) The contour direction is right In case of diagonally upward direction P'22 = C1 (P31 + P13) + C2 (P11 + P
33) (d) When the contour direction is diagonally upward to the left P'22 = C1 (P11 + P33) + C2 (P31 + P
13) In the above (c) and (d), 2C1 + 2C2 = 1
And it is preferable that C1> C2. Thereby, the value of the temporary signal calculated using the pixels in the contour direction can be corrected by using the pixels existing in the direction perpendicular to the contour direction.

By performing the interpolation process in this way, the same effect as shown in FIG. 5 can be obtained also in the first modification of the embodiment of the present invention.

(Second Modification) As a modification of the present invention,
The case where the direction determination means 5a cannot determine the direction or the case where the direction determination means 5a is not provided will be described.

(Interpolation Method for O-Pixel) When a pixel having certain color information is to be corrected, the pixel having the pixel as the center is used for the correction. This prevents the signal of the pixel concerned from protruding and adjusts to the signals of the surrounding pixels.
The pixel can be smoothed.

With respect to P33 in FIG. 3, an example of calculation for correction will be given. The corrected value of P33 is defined as P'33.

P'33 = A4 / P33 + A5 (P31 +
P13 + P53 + P35) + A6 (P11 + P51 + P
15 + P55) where A4 + 4A5 + 4A6 = 1, A4> 4A5,
It is preferable that A4> 4A6.

(Interpolation method for h pixel) When interpolating a pixel adjacent to a pixel having certain color information in the horizontal direction,
The value of the signal of the pixel is calculated, and the signal of the pixel is corrected using pixels other than the pixel having the pixel as the center. As a result, the signal of the pixel can be prevented from protruding and the pixel can be smoothed so as to adapt to the signals of the surrounding pixels.

With respect to P23 in FIG. 3, an example of calculation when performing interpolation will be described. The value of P23 after interpolation is set to P'23.

P'23 = B4 (P13 + P33) 3 + B
5 (P11 + P31 + P15 + P35) However, it is preferable that 2B4 + 4A5 = 1 and B4> 2B5.

(Interpolation method for v pixel) When interpolating a pixel adjacent to a pixel having certain color information in the vertical direction,
The value of the signal of the pixel is calculated, and the signal of the pixel is corrected using pixels other than the pixel having the pixel as the center. As a result, the signal of the pixel can be prevented from protruding and the pixel can be smoothed so as to adapt to the signals of the surrounding pixels.

With respect to P32 in FIG. 3, an example of calculation when interpolation is performed will be described. The value of P32 after interpolation is P′32.

P'32 = B4 (P31 + P33) 3 + B
5 (P11 + P31 + P51 + P53) However, it is preferable that 2B4 + 4A5 = 1 and B4> 2B5.

(Interpolation method for x pixel) When interpolating a pixel adjacent to a pixel having certain color information in the diagonal direction,
Using pixels other than the pixel centered on the pixel,
The signal of this pixel is interpolated. As a result, the signal of the pixel can be prevented from protruding and the pixel can be smoothed so as to adapt to the signals of the surrounding pixels.

With respect to P22 of FIG. 3, an example of calculation when performing interpolation will be described. The value of P22 after interpolation is P'22.

P'22 = (1/4) · (P11 + P13
+ P31 + P33) By performing the interpolation processing in this manner, in the second modification of the embodiment of the present invention, the pixel of the pixel is prevented from protruding and the pixel of the pixel is smoothly smoothed so as to adapt to the signals of the surrounding pixels. can do.

(Other Embodiments) As described above, the embodiments of the present invention have been described, but it should be understood that the description and drawings forming a part of this disclosure limit the present invention. Absent. From this disclosure, various alternative embodiments, examples and operational techniques will be apparent to those skilled in the art.

Used in the embodiment of the present invention,
The coefficient of each signal is not limited to this.

In the embodiment of the present invention, the direction determining means detects the contour direction, but the contour direction may be determined in advance. Alternatively, (1) when the contour direction is the vertical direction or the horizontal direction, (2) when the contour direction is the vertical direction, the horizontal direction, the diagonally right upward direction, or the diagonally left upward direction,
(3) The contour direction is the vertical direction, the horizontal direction, or when the distinction cannot be made. (4) The contour direction is the vertical direction, the lateral direction, the diagonally right upper direction, the diagonally left upper direction, or the combination which cannot be distinguished. Therefore, the contour direction may be determined. In this case, which combination of (1) to (4) to use may be determined in advance.

Further, in the embodiment of the present invention, the color filter is composed of pixels of three colors, but the color filter may be composed of pixels of two colors or four or more colors.

Further, the color filter used in the present invention does not have to have the Bayer array. The shape of the color filter, the arrangement of colors, and the like may be arbitrary.

As described above, it goes without saying that the present invention includes various embodiments not described here. Therefore, the technical scope of the present invention is defined only by the matters specifying the invention according to the scope of claims reasonable from the above description.

[0073]

According to the present invention, by forming a smooth signal, it is possible to suppress the generation of an image signal that does not originally exist in contour enhancement, and to enhance the contour that actually exists. An image processing method can be provided.

[Brief description of drawings]

FIG. 1 is a functional block diagram of an electronic camera that is one type of an image capturing apparatus according to an embodiment of the present invention.

FIG. 2 is a diagram showing a processing means and a data flow in the camera signal processing unit according to the embodiment of the present invention.

FIG. 3 is a diagram showing pixels in the embodiment of the present invention.

FIG. 4 is a diagram showing a result of classifying each pixel of FIG. 3;

FIG. 5 is a diagram showing a signal before interpolation and a signal after interpolation according to the embodiment of the present invention.

FIG. 6 is a diagram showing a color filter of the image capturing apparatus according to the embodiment of the present invention.

FIG. 7 is an example of a signal when contour enhancement processing is performed on a signal in which a signal before contour enhancement is not smoothly formed.

FIG. 8 is an example of a signal when contour enhancement processing is performed on a signal on which a signal before contour enhancement is smoothly formed.

[Explanation of symbols]

1 lens part 2 color filters 3 CCD 4 A / D converter 5 Camera signal processor 5a Direction determining means 5b Interpolation means 5c Contour enhancement means 6 Timing generator 7 Microcomputer

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) H04N 9/07 H04N 1/46 ZF term (reference) 5B057 BA02 CA01 CA08 CA12 CA16 CB01 CB08 CB12 CB16 CC01 CE03 CE16 CH08 5C065 AA03 BB00 CC01 DD01 5C077 LL02 LL08 MP08 PP03 PP32 PP37 PP47 PQ12 SS01 TT09 5C079 HB01 JA23 LA01 LA02 LA15 LA28 MA11 NA04

Claims (12)

[Claims] 1. An image is processed by interpolating based on an imaging signal output from a color filter in which a plurality of pixels having any one of a plurality of colors are arranged and having one color information in one pixel. In the image processing device, the direction determining means for detecting a correlation direction in which the change in the image capturing signal is small from the image capturing signal, and based on the correlation direction obtained by the direction determining means,
An image processing apparatus comprising: an interpolating unit that obtains the certain color information of the pixel using the pixel having the certain color information. 2. The interpolation means, when the pixel already has the certain color information, uses the pixel and the pixel having the certain color information other than the pixel to determine the existence of the certain pixel. The image according to claim 1, wherein the color information is corrected, and when the pixel does not have the certain color information, the pixel having the certain color information other than the pixel is used to perform the interpolation. Processing equipment. 3. The image processing apparatus according to claim 1, wherein the direction determining means detects a correlation direction in either the vertical direction or the horizontal direction. 4. The direction determining means, when the correlation direction cannot be detected, the interpolating means uses pixels having the certain color information existing around the pixel, The image processing apparatus according to claim 3, wherein the certain color information is obtained. 5. The direction determining means includes a vertical direction, a horizontal direction,
The image processing apparatus according to claim 1 or 2, wherein the correlation direction in any one of the right diagonal direction and the left diagonal direction is detected. 6. The direction determining means, when the correlation direction cannot be detected, the interpolating means uses pixels having the certain color information existing around the pixel, The image processing apparatus according to claim 5, wherein the certain color information is obtained. 7. An image is processed by interpolating based on an image pickup signal output from a color filter in which a plurality of pixels having any one of a plurality of colors are arranged, and having one color information in one pixel. An image processing method, wherein, from the image pickup signal, a direction determination step of detecting a correlation direction in which the change in the image pickup signal is small, based on the correlation direction obtained by the direction determination means,
An image processing method comprising: using a pixel having certain color information, and interpolating the certain color information of the pixel. 8. The interpolation step uses the pixel and the pixel having the color information other than the pixel if the pixel already has the color information, The image according to claim 7, wherein the color information is corrected, and when the pixel does not have the certain color information, the pixel having the certain color information other than the pixel is used to perform the interpolation. Processing method. 9. The image processing method according to claim 7, wherein the direction determining step detects a correlation direction in one of a vertical direction and a horizontal direction. 10. In the direction determining step, if the correlation direction cannot be detected, the interpolation step uses pixels having the certain color information existing around the pixel, The image processing method according to claim 9, wherein the certain color information is obtained. 11. The image processing according to claim 7, wherein the direction determining step detects a correlation direction in any one of a vertical direction, a horizontal direction, an oblique right direction, and an oblique left direction. Method. 12. When the correlation direction cannot be detected in the direction determining step, the interpolation step uses pixels having the certain color information existing around the pixel, The image processing method according to claim 11, wherein the certain color information is obtained.