JP2000102024A - Image signal processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enhance resolution in an oblique direction as well as in horizontal and vertical directions by providing an interpolation means that interpolates a read signal from a photographing means with neighboring pixels in other colors to the processing unit in the case of processing the read signal from the photographing means consisting of an array of pixels to which complementary color group coating is applied. SOLUTION: A 1-line delay circuit 2 and a resolution enhancement detection interpolation circuit 9 receive an image signal from a CCD consisting of pixels in a lattice arrangement to which a 2×4 chequerwise pattern coating of a complementary color group of yellow, cyan, magenta, green colors is applied. An output of a delay circuit 2 is given to delay circuits 3-8, where the output is sequentially delayed by one-line each, the interpolation circuit 9 interpolates a signal in the same line based on a result of detection of correlation of signals resulting from synchronizing 8 lines to generate an interpolation signal of 4 lines. This interpolation signal is processed by a V filter 10, a 1/4H filter 11, a 1/2H filter 12, a vertical direction low pass filter 13 multipliers 14-16 and adders 17, 18 and a gamma correction circuit 19 corrects the processed signal to obtain a luminance signal.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image signal processing apparatus for performing pixel interpolation on a read signal from an image pickup device such as a CCD having a complementary color coating.

[0002]

2. Description of the Related Art Conventionally, a complementary color camera system provided with an imaging device such as a CCD coated with a complementary color system such as yellow (Ye), cyan (Cy), magenta (Mg), and green (G) has been known. , Red (R), blue (B), green (G) primary color camera system provided with an imaging device such as CCD coated with primary color system coating, the sensitivity of each pixel is higher, and each pixel color component Therefore, it is characterized that relatively good pixels can be obtained without improving the resolution.

FIG. 11 shows an example of a configuration of a luminance signal processing at the time of two-pixel mixed reading in a conventional complementary color camera system. In the example of the complementary color camera system shown in FIG. 11, a so-called complementary checkered coating is applied to the imaging device, and an image signal from an imaging device such as a CCD is subjected to filtering processing in the time axis direction to perform high-frequency filtering of the image signal. Signal processing for compensating components (hereinafter, referred to as aperture control processing) is performed.

In FIG. 11, an input terminal 101 is supplied with spatially discrete complementary color image signals of Ye, Cy, Mg, and G from an imaging device such as a CCD. This complementary color image signal is output to the 1/4 V filter 104 and 1
The output of the one-line delay circuit 102 is sent to the one-line delay circuit 1.
03 is sent.

[0005] One-line delay circuits 102 and 103
Delays each input image signal by one line,
Each delayed output is sent to a 1 / 4V filter 104. As a result, the 信号 V filter 104 is supplied with an image signal that has been synchronized for three lines.

In addition, the one-line delay circuit 102
The image signal delayed by the line is also sent to the horizontal low-pass filter 107.

The horizontal low-pass filter 107 extracts a luminance signal by extracting a low-frequency component of the image signal delayed by one line in the one-line delay circuit 102, and
This luminance signal is added to an adder 112, a 1 / 4H filter 10
5 and 1/2 H filters.

The ４V filter 104 extracts a high-frequency component of 垂直 of the sampling frequency in the vertical direction from the image signal of three lines, and sends the frequency component to the multiplier 108. The ＨH filter 105 outputs the luminance signal extracted from the horizontal low-pass filter 107
A high-frequency component in the horizontal direction 周波 数 of the sampling frequency is extracted, and the frequency component is sent to the multiplier 109. 1 / 2H
The filter 106 extracts a high-frequency component of サ ン プ リ ン グ of the sampling frequency in the horizontal direction from the luminance signal extracted from the horizontal low-pass filter 107, and sends the frequency component to the multiplier 110.

The multiplier 108 includes a 1/4 V filter 104
Is multiplied by a predetermined aperture control gain and the output signal is added to an adder 11.
Send to 1. The multiplier 109 multiplies the frequency component in the horizontal 帯 域 band from the Ｈ H filter 105 by a predetermined aperture control gain, and sends the output signal to the adder 111. The multiplier 110 is provided with the ＨH filter 10
A predetermined aperture control gain is multiplied by the frequency component of the band in the horizontal direction ６ from 6, and the output signal is sent to the adder 111.

In the adder 111, multipliers 108 to 110
Are added and output as an aperture control signal. The aperture control signal from the adder 111 is sent to the adder 112.

The adder 112 adds the aperture control signal and the luminance signal, whereby the adder 112 outputs a luminance signal in which high-frequency components have been compensated (aperture control processing). This adder 1
The luminance signal output from 12 is sent to a gamma (γ) correction circuit 113.

The gamma correction circuit 113 performs gamma processing on the luminance signal after the aperture control processing, and outputs the gamma corrected luminance signal from an output terminal 114.

In the luminance signal processing at the time of two-pixel mixture readout of the conventional complementary color camera system shown in FIG. 11, a luminance signal is extracted by applying a horizontal filter to the input image signal, and the luminance signal is synchronized. Only high-frequency components (vertical 1/4 of sampling frequency, high-frequency components of 1/2 and 1/4 of the sampling frequency) are extracted from the horizontal and vertical signals through a filter, multiplied by an appropriate gain, and added. A desired aperture control signal (high frequency compensation signal) is generated, and the aperture control signal is added to the luminance signal.

FIG. 12 shows a configuration example of luminance signal processing at the time of independent reading of all pixels in a conventional complementary color camera system. Note that, also in the example of FIG. 12, the imaging device is provided with a complementary color checkerboard coating, and an aperture control process is performed.

In FIG. 12, complementary color image signals of Ye, Cy, Mg, and G, which are spatially discrete, are supplied to an input terminal 121 from an imaging device such as a CCD. This complementary color image signal is supplied to the V filter 125, the vertical low-pass filter 128, and the one-line delay circuit 1
22.

The output of the one-line delay circuit 122 is 1
The signal is further sent to the one-line delay circuit 124 via the line delay circuit 123. Each of the one-line delay circuits 122 to 124 delays the input image signal by one line, and sends each delayed output to the V filter 125 and the vertical low-pass filter 128. Thereby, V
The filter 125 and the vertical low-pass filter 128 are supplied with an image signal synchronized with four lines.

The signal of the vertical low-pass filter 128 is further passed through a horizontal low-pass filter 129, whereby a luminance signal is extracted from an image signal obtained by synchronizing four lines. This luminance signal is output to the ４H filter 126 and the ＨH filter 12.
7, and sent to the adder 134.

The V filter 125 extracts a high-frequency component in the vertical direction of the sampling frequency from the image signal obtained by synchronizing the four lines, and sends the frequency component to the multiplier 130. The ＨH filter 126 extracts a high-frequency component of サ ン プ リ ン グ of the sampling frequency in the horizontal direction from the luminance signal,
The frequency component is sent to the multiplier 131. The ＨH filter 127 extracts a high-frequency component in the horizontal direction の of the sampling frequency from the luminance signal, and sends the frequency component to the multiplier 132.

The multiplier 130 multiplies the vertical frequency component from the V filter 125 by a predetermined aperture control gain, and sends the output signal to the adder 133. The multiplier 131 multiplies the frequency component in the horizontal １ ／ band from the Ｈ H filter 126 by a predetermined aperture control gain, and sends the output signal to the adder 133. Multiplier 132 multiplies the frequency component in the horizontal ２ band from Ｈ H filter 127 by a predetermined aperture control gain, and outputs the output signal to adder 133.
Send to

In the adder 133, multipliers 130 to 132
Are added and output as an aperture control signal. The aperture control signal from the adder 133 is sent to the adder 134.

In the adder 134, the aperture control signal and the luminance signal are added, whereby the adder 134 outputs a luminance signal in which high-frequency components have been compensated (aperture control processing). This adder 1
The luminance signal output from 34 is sent to a gamma (γ) correction circuit 135.

The gamma correction circuit 135 performs gamma processing on the luminance signal after the aperture control processing, and outputs the gamma corrected luminance signal from an output terminal 136.

In the luminance signal processing at the time of independent reading of all pixels of the conventional complementary color camera system shown in FIG. 12, a luminance signal is extracted by applying a filter in the vertical and horizontal directions to an input image signal, and the image signal is synchronized. From the horizontal and vertical signals obtained, only high-frequency components (vertical, horizontal 1/2 and 1/4 high-frequency components of the sampling frequency) are extracted through a filter, multiplied by an appropriate gain, and added to obtain a desired value. Is generated, and this aperture control signal is added to the luminance signal.

As described above, in the conventional complementary color camera system, the aperture control signal is generated in parallel with the luminance signal in both of the two-pixel mixed read mode and the all-pixel read mode, and before the gamma (γ) correction. The aperture control signal is added to the luminance signal to raise the high frequency component.

[0025]

However, in the case of the above-mentioned conventional complementary color camera system, the resolution in the oblique direction is low, and particularly in a still image, the image quality tends to be conspicuous as deterioration. That is, for example, there is a problem that a phenomenon in which an oblique line becomes stair-like is easily seen.

The present invention has been made in view of such a situation, and an object of the present invention is to provide an image signal processing apparatus capable of increasing not only the resolution in the horizontal and vertical directions but also the resolution in the oblique direction. Aim.

[0027]

An image signal processing apparatus according to the present invention is an image signal processing apparatus for processing a read signal from an image pickup means in which pixels coated with a complementary color system are arranged, and The above-mentioned problem is solved by having an interpolating means for performing pixel interpolation of other neighboring colors for each of the readout signals.

[0028]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will be described with reference to the drawings.

FIG. 1 shows a first embodiment of an image signal processing apparatus according to the present invention.
e) shows a schematic configuration of a complementary color camera system for processing an image signal from a CCD imaging device having a 2 × 4 checkered coating of complementary colors of cyan (Cy), magenta (Mg), and green (G). Note that, in the complementary color camera system of FIG. 1, an example is described in which an aperture control process is performed to compensate for a high-frequency component of a luminance signal when reading out all pixels.

In FIG. 1, an input terminal 1 is supplied with a complementary color image signal from a CCD having a complementary color checkered coating as shown in FIG. This complementary color image signal is sent to the one-line delay circuit 2 and the resolution improvement detection interpolation circuit 9.

The output of the one-line delay circuit 2 is 1
The signals are sequentially sent to the line delay circuits 3, 4, 5, 6, 7, and 8. Each of the one-line delay circuits 2 to 8 delays the input image signal by one line, and sends each delayed output to the resolution improvement detection interpolation circuit 9. As a result, the image signal synchronized with eight lines is supplied to the resolution improvement detection interpolation circuit 9.

As will be described in detail later, the resolution improvement detection interpolation circuit 9 detects the correlation of the signal synchronized for eight lines in order to improve the resolution, and based on the detection result, the signal is synchronized for the eight lines. The signals (for example, the signals of the Ye and Cy pixels and the signals of the Mg and G pixels) in the same line of the extracted signals are interpolated to generate an interpolation signal of four lines. In other words, in the complementary color camera system of the present embodiment, the 1-line delay circuits 2 to 8 for 8 lines are required in order for the resolution improvement detection interpolation circuit 9 to perform interpolation for 4 lines. The 4-line interpolation signal output from the resolution improvement detection interpolation circuit 9 is sent to the vertical low-pass filter 13 and the V filter 10.

The vertical low-pass filter 13 extracts a luminance signal by extracting a low-frequency component in the vertical direction from the 4-line interpolation signal from the resolution improvement detection interpolation circuit 9. That is, in the vertical low-pass filter 13, a line obtained by interpolating the signal of the Cy pixel into the Ye pixel and the line interpolating the signal of the G pixel into the Mg pixel by the resolution improvement detection interpolation circuit 9 in the coating pattern of FIG. Are added to generate a luminance signal of Ye + Cy + Mg + G = Y, and similarly, a line obtained by interpolating the signal of the Ye pixel to the Cy pixel by the resolution improvement detection interpolation circuit 9 and the signal of the Mg pixel to the G pixel A luminance signal of Ye + Cy + Mg + G = Y is generated by adding two lines of the interpolated lines. This luminance signal is supplied to an adder 18, a 1 / 4H filter 1
It is sent to the 1 and 1/2 H filters 12.

The V filter 10 extracts a high-frequency component in the vertical direction from the interpolation signal of four lines, and sends the frequency component to the multiplier 14. The ＨH filter 11 extracts a high-frequency component in the horizontal direction の of the sampling frequency from the luminance signal, and sends the frequency component to the multiplier 15. 1 /
The 2H filter 12 extracts a high-frequency component in the horizontal direction の of the sampling frequency from the luminance signal, and sends the frequency component to the multiplier 16.

The multiplier 14 multiplies the vertical frequency component from the V filter 10 by a predetermined aperture control gain, and sends the output signal to the adder 17. Multiplier 1
5 multiplies the frequency component in the horizontal 帯 域 band from the Ｈ H filter 11 by a predetermined aperture control gain, and sends the output signal to the adder 17. Multiplier 16
Multiplies the frequency component in the horizontal １ ／ band from the Ｈ H filter 12 by a predetermined aperture control gain, and sends the output signal to the adder 17.

The adder 17 adds the output signals of the multipliers 14 to 16 and outputs the result as an aperture control signal. The aperture control signal from the adder 17 is sent to the adder 18.

The adder 18 adds the aperture control signal and the luminance signal, whereby the adder 18 outputs a luminance signal in which high-frequency components have been compensated (aperture control processing). The luminance signal output from the adder 18 is sent to a gamma (γ) correction circuit 19.

In the gamma correction circuit 19, the luminance signal after the aperture control processing is gamma-processed, and the luminance signal after the gamma correction is output from the output terminal 20.

In the luminance signal processing at the time of independent reading of all pixels of the complementary color camera system of the embodiment shown in FIG.
The resolution improvement detection interpolation circuit 9 interpolates the eight lines of the signal to generate a four-line interpolation signal, applies a vertical filter to the four-line interpolation signal, and extracts a luminance signal. , A desired aperture control by extracting only high-frequency components (high-frequency components of vertical and horizontal 1/2 and 1/4) by passing a 4-line interpolation signal through a filter, multiplying them by an appropriate gain, and adding them up A signal (high frequency compensation signal) is generated, and the aperture control signal is added to the luminance signal.

That is, in the complementary color camera system of the embodiment shown in FIG. 1, signals in the same line are interpolated (Ye
Since the pixel and the Cy pixel and the Mg pixel and the G pixel are interpolated), a luminance signal can be obtained without a horizontal low-pass filter. In other words, when the complementary color checkered coating as shown in FIG. 2 is applied to the imaging device,
Since the luminance signal (Y) can be generated by adding (Ye + Cy + Mg + G) the signals of the Ye pixel, the Cy pixel, the Mg pixel, and the G pixel, the complementary color camera system according to the present embodiment includes:
The luminance signal is obtained by passing the interpolation signal of four lines through the vertical low-pass filter 13. The horizontal and vertical aperture control signals can be generated by a method similar to the conventional method.

Here, the most characteristic point of the embodiment of the present invention resides in that a signal in the same line is interpolated, and line addition is performed by a subsequent filter to generate a luminance signal. As a comparison with the present embodiment, for example, in a camera system of the R, G, B primary color system, the color data of all pixels is assigned to one pixel so that the R pixel and the B pixel are interpolated into the G pixel. Is interpolated.

Next, correlation detection and interpolation processing in the resolution improvement detection interpolation circuit 9 of the complementary color camera system of FIG. 1 will be described below.

FIGS. 3 and 4 show a pixel array handled by the resolution improvement detection interpolation circuit 9.

The resolution improvement detection interpolation circuit 9 interpolates the Cy pixel for the Ye pixel and the Cy pixel for the line composed of the Ye pixel and the Cy pixel in the pixel rows shown in FIGS. To interpolate Ye pixels. For a line composed of a G pixel and a Mg pixel, the G pixel is interpolated for the G pixel and the G pixel is interpolated for the Mg pixel.

When performing such interpolation processing,
The resolution improvement detection interpolation circuit 9 detects the correlation of each pixel and performs an interpolation process according to the correlation detection result.

A simple processing example will be described below as an example of actual correlation detection.

For example, as shown in FIG. 3, when a G pixel is interpolated with respect to an Mg pixel, first, a value (level) obtained by calculating | G1-G4 | Compared with the value (level) obtained by calculating G2-G3 |, the horizontal correlation is large if the former level is large, the vertical correlation is large if the latter level is large, and the oblique correlation is large if the levels are almost equal. Judge as big.

For example, as shown in FIG. 4, when interpolating Cy pixels with respect to Ye pixels, first, | (Cy1 + Cy3 + Cy5) / 3- (Cy2 + Cy4) for correlation detection.
+ Cy6) / 3 | is compared with the value (level) calculated as | (Cy1 + Cy2) / 2− (Cy5 + Cy6) / 2 |, and if the former level is larger, it is horizontal. If the correlation is large and the latter level is large, it is determined that the vertical correlation is large, and if they are substantially equal, it is determined that the correlation in the oblique direction is large.

Based on the result of the above correlation detection, for example, when the G pixel is interpolated with respect to the Mg pixel, if the horizontal correlation is large, the interpolation is performed by calculating (G2 + G3) / 2, and the vertical is calculated. If the correlation in the direction is large, interpolation is performed by calculating (G1 + G4) / 2, while if the correlation in the oblique direction is large, interpolation is performed by calculating (G1 + 2 × G2 + 2 × G3 + G4) / 6.

Similarly, based on the result of the above-described correlation detection, for example, when the Cy pixel is interpolated with respect to the Ye pixel, if the correlation in the horizontal direction is large, the interpolation is performed by calculating (Cy3 + Cy4) / 2. If the correlation in the vertical direction is large, (Cy1 + Cy2 + Cy3 + Cy4 + Cy5 + Cy6)
/ 6, and if the correlation in the oblique direction is large, (Cy2 + Cy5) / 2 or (Cy1 + C
Interpolation is performed by calculating y6) / 2.

Note that the resolution improvement detection interpolation circuit 9 calculates the vertical and horizontal correlation ratios more finely at the time of correlation detection, so that not only the three directions of the vertical direction, the horizontal direction, and the oblique direction, but also the finer directions. Interpolation control is also possible.

However, in the above-described interpolation processing based on the correlation detection, when the correlation is detected when the G pixel is interpolated with respect to the Mg pixel, the correlation is determined by calculating only one pixel. For example, there is a high possibility that an erroneous detection will be made when noise is present on the vehicle. In addition, when the Cy pixel is interpolated with respect to the Ye pixel, if the correlation in the vertical direction is strong, the interpolation is performed on three pixels in the horizontal direction, which is disadvantageous in terms of resolution. The same applies to the case where the Ye pixel is interpolated for the Cy pixel and the case where the Mg pixel is interpolated for the G pixel. Therefore, it is necessary to take these factors into account in the interpolation processing based on the correlation detection.

By the way, the spectral characteristics of the Ye, Cy, Mg, and G pixels in the complementary color checkered coating pattern of FIG. 2 are as shown in FIG. 5, which can be seen from the spectral characteristic examples shown in FIG. Thus, it can be seen that the sensitivity (response) for each wavelength is higher in the line of the Ye pixel and the Cy pixel than in the line of the G pixel and the Mg pixel. In contrast, Mg pixels have low sensitivity to green light,
Pixels have low sensitivity to red and blue light. From these, it can be seen that in the line of the Mg pixel and the G pixel, the resolution can be sufficiently corrected only by interpolating the G pixel for the Mg pixel and the Mg pixel for the G pixel. .

In the case of such interpolation processing, (Ye + C
When generating a luminance signal of (y + Mg + G) = Y, two pixels in the horizontal direction and the vertical direction must always be added.
Therefore, in such a case, in order to generate a luminance signal, not only a vertical low-pass filter but also a horizontal low-pass filter is required.

FIG. 6 shows an example of the configuration of a complementary color camera system according to the second embodiment of the present invention in a case where interpolation is performed only on lines of Mg and G pixels as described above. Note that, in FIG. 6, the same reference numerals are given to the respective components that perform substantially the same operation as the respective components in FIG.

That is, the difference between the complementary color camera system of the second embodiment shown in FIG. 6 and the system shown in FIG. 1 is that the resolution improvement detection interpolation circuit 21 of FIG.
Instead of performing interpolation for all the pixels shown in FIGS. 3 and 4 as in the resolution improvement detection interpolation circuit 9 of FIG. 1, only the G pixel and Mg pixel lines of the complementary checkered coating pattern of FIG. And a point that a horizontal low-pass filter 22 is further provided downstream of the vertical low-pass filter 13 in order to generate a luminance signal. Therefore, in the case of the configuration shown in FIG. 6, the 1 / 4H filter 11, the 1 / 2H filter 12, and the adder 18 are added to the horizontal low-pass filter 22.
Is sent. Note that the method of interpolating the G pixel and the Mg pixel in the resolution improvement detection interpolation circuit 21 of FIG. 6 is the same as the interpolation process of the G pixel and the Mg pixel in the resolution improvement detection interpolation circuit 9 of FIG.

Next, a third embodiment of the present invention for dealing with erroneous detection when noise is present in a pixel signal among points to be considered in the above-described correlation detection and interpolation processing. Will be described. Since the system configuration of the third embodiment is the same as the system configuration of FIG. 1, the illustration is omitted. However, in the case of the third embodiment, the processing content of the resolution improvement detection interpolation circuit is different from that of the first embodiment, and therefore, in the following description, the resolution improvement detection interpolation in the first embodiment will be described. In order to distinguish it from the circuit 9, a number 23 (that is, the resolution improvement detection circuit 23) is used as an instruction code of the resolution improvement detection circuit.

The pixel rows for correlation detection handled by the resolution improvement detection interpolation circuit 23 of the third embodiment will be described with reference to FIGS.

The resolution improvement detection interpolation circuit 2 of the present embodiment
3, for example, G pixels and M in the range indicated by the arrow in FIG.
The signal obtained by adding the g pixel is subjected to correlation detection as a signal for correlation detection, or, for example, Y in a range indicated by an arrow in FIG.
Correlation detection is performed using a signal obtained by adding each pixel of e, Cy, Mg, and G as a signal for correlation detection. Hereinafter, performing the correlation detection by adding the G pixel and the Mg pixel as shown in FIG. 7 is referred to as a first correlation detection method, and adding each pixel of Ye, Cy, Mg, and G as shown in FIG. Performing the correlation detection will be referred to as a second correlation detection method. In the third embodiment, the actual interpolation pixels and interpolation calculations are the same as those in the simple horizontal, vertical, and oblique directions described in the first embodiment.

Here, in the third embodiment,
Both the first and second correlation detection methods handle Mg pixels and G pixels, and the spectral characteristics of correlation detection signals obtained by adding the signals of these Mg pixels and G pixels match. That is, in the embodiment described above, the G pixel used at the time of correlation detection,
Since the spectral characteristics of the signals of the Mg pixels are different, the level at the time of selecting the interpolation processing in the horizontal, vertical, and oblique directions is accordingly changed to the G pixel,
Although different settings had to be made for the Mg pixels, in the third embodiment, since the G pixels and the Mg pixels were added, it was necessary to set the judgment level for correlation detection separately. Absent. In other words, in the third embodiment, as shown in FIGS. 7 and 8, the spectral characteristics of the correlation detection signal at the time of determining the correlation of each pixel are made uniform,
It is possible to easily set the correlation detection level.

For example, taking the first correlation detection method as an example,
When selecting the interpolation processing only in the horizontal and vertical directions, the selection is performed as follows as an example. For example, G1Ja = (Mg5 + 2 × G1 + Mg1) / 4 G2Ja = (Mg6 + 2 × G2 + Mg2) / 4 G3Ja = (Mg2 + 2 × G3 + Mg3) / 4 G4Ja = (Mg7 + 2 × G4 + Mg4) / 4 At this time, for correlation detection, | G2Ja−G3Ja | − | G1Ja−G4Ja |, and if the value of the calculation result is a positive value, Ghokan = (G1 + G4) / 2 is used to perform interpolation in the vertical direction. If the value of the calculation result for detection is a negative value, horizontal interpolation is performed by the operation of Ghokan = (G2 + G3) / 2.

Further, in the case of performing not only horizontal and vertical interpolation but also oblique interpolation using the average value of the four pixel values of Mg and G, the following is performed. For example, one interpolated value (level) in the oblique direction is set as Jlev, and if the relationship | G1Ja−G4Ja | − | G2Ja−G3Ja | <−Jlev holds, the calculation of Ghokan = (G1 + G4) / 2 is performed. If the relationship | G1Ja−G4Ja | − | G2Ja−G3Ja |> Jlev holds, the horizontal direction interpolation is performed by the calculation of Ghokan = (G2 + G3) / 2. If not, Ghokan = (2 * (G2 + G3) + (G1 + G4)) / 6
(Average) is calculated as diagonal interpolation.

As described above, the third embodiment performs the correlation detection using the correlation detection signal obtained by adding the signals of the Mg pixel and the G pixel, and selects the interpolation processing according to the detection result. FIG. 9 shows a specific configuration example of the resolution improvement detection interpolation circuit 23 of FIG.

In FIG. 9, an adder 31 adds signals of G2 and G3 pixels, and an adder 32 adds signals of G1 and G4 pixels. The addition output of the adder 31 is ２
The added output from the adder 32 is sent to the adder 42 and the 倍 -fold calculator 46. The １ ／ multiplication unit 44 calculates the addition output of the adder 31 as 1
The doubling calculator 41 doubles the added output of the adder 31, and the 倍 calculator 46 doubles the added output of the adder 32. The adder 42 adds the output of the doubling calculator 41 and the output of the adder 32, and divides the added output by 1/6.
It is sent to the arithmetic unit 45. The 1/6 arithmetic unit 45 multiplies the addition output of the adder 42 by 1/6.倍 multiplication unit 44, 1 /
The outputs of the sixfold calculator 45 and the 1/2 calculator 46 are sent to the selector 47, respectively.

In the adder 33, G1, Mg1, Mg
The signals of the five pixels are added and the result of the addition is sent to the subtractor 37. The adder 34 adds the signals of the G4, Mg4 and Mg7 pixels and sends the addition result to the subtractor 37. Adder 35
Then, the signals of the G2, Mg2, and Mg6 pixels are added, and the addition result is sent to a subtractor 38.
2 and the signal of the Mg3 pixel are added, and the addition result is sent to the subtractor 38. The subtractor 37 calculates the difference between the addition outputs of the adders 33 and 34 and sends the difference result to the absolute value converter 39. The subtractor 38 calculates the difference between the addition outputs of the adders 35 and 36 and calculates the difference result. It is sent to the absolute value converter 40. The absolute value generator 39 calculates the absolute value of the subtraction output of the subtractor 37 and sends the absolute value output to the subtractor 43. The absolute value generator 40 calculates the absolute value of the subtraction output of the subtractor 38 and calculates the absolute value. The output is sent to the subtractor 43. The subtracter 43 calculates the difference between the absolute value outputs of the absolute value converters 39 and 40 and sends the difference result to the comparator 48. The value Jlev set as the oblique interpolation value (level) is sent to the comparator 48, and the comparison result is sent to the selector 47.

In the selector 47, according to the comparison result from the comparator 48, the 倍 multiplication unit 44 and the ６ multiplication unit 45
And any one of the outputs of the 1/2 arithmetic unit 46 are selected, and the selected output is output as the interpolation value G.

As shown in the example of FIG. 9, if the resolution improvement detection interpolation circuit 23 is configured to select the interpolation direction from the level of the difference between the correlation in the horizontal direction and the correlation in the vertical direction, a small-scale circuit can be used. Interpolation can be realized.

As another example, in order to perform not only three interpolations in the horizontal direction, the vertical direction, and the diagonal direction as in the above example, but also to perform a finer interpolation, the obtained correlations in the horizontal direction and the vertical direction are obtained. A method of determining the data ratio and determining the addition ratio in the vertical direction and the horizontal direction from the result may be considered.

FIG. 10 shows an example of the configuration of a resolution improvement detection circuit in the case where finer interpolation is performed as described above. Note that among the components in FIG. 10, the same components as those in FIG. 9 are denoted by the same reference numerals, and description thereof will be omitted.

In FIG. 10, the addition output from the adder 31 is sent to a multiplier 53 as a horizontal interpolation value Gh, and the addition output of the adder 32 is sent to a multiplier 54 as a vertical interpolation value Gv. .

The absolute value output D from the absolute value generator 39
iffV is sent to the divider 52 and the adder 51, and the absolute value
The absolute value output DiffH from 0 is sent to the adder 51. The adder 51 adds the absolute value output DiffV and the absolute value output DiffH, and sends the added output (DiffV + DiffH) to the divider 52. The divider 52 outputs an absolute value output D from the absolute value generator 39.
iffV is divided by the addition output (DiffV + DiffH) from the adder 51, and the divided value α = (DiffV / (DiffV + Diff)
H)) and outputs the value of 1-α. The division value α is a multiplier 53
, And the value (1−α) is sent to the multiplier 54.

In the multiplier 53, the interpolation value G from the adder 31 is calculated.
h is multiplied by the division value α from the divider 52, and the multiplied output (α × Gh) is sent to the adder 55. The multiplier 54 interpolates the interpolated value Gv from the adder 32 and the value from the divider 52. (1
-Α) and sends the multiplied output (1-α) Gv to the adder 55. In the adder 55, the multiplication output (α × Gh) from the multiplier 53 and the multiplication output (1-α) from the multiplier 54
Add Gv. The addition output of this adder 55 (α × Gh +
(1-α) Gv) is output as the interpolation value G.

It should be noted that the pixel selection in the second correlation detection method also applies to Ye, Cy, Mg, G shown in FIG.
If the determination and processing are performed in the same manner as in the above example from the addition result of each pixel, interpolation detection and interpolation processing can be realized.

As described above, in the complementary color camera system according to the present embodiment, in response to a read signal from an image pickup device such as a CCD having a pixel array provided with a complementary color system coating, pixel interpolation of other nearby colors is performed. This makes it possible to improve the resolution of the luminance signal. In addition, when performing pixel interpolation for other colors in the vicinity, the resolution can be improved by performing pixel interpolation on some pixels instead of all pixels.

In the complementary color camera system of the present embodiment, the Ye, Cy, Mg, G (2 × 4) complementary color checker pixel array is used for all pixel independent read signals from the imaging device, and for the Mg, G pixels. The resolution can be improved by interpolating the pixels only once. In addition, when performing pixel interpolation, accurate interpolation detection can be performed by performing interpolation detection using not only information on the color to be interpolated but also pixel information on some other colors. Further, when performing pixel interpolation, by performing interpolation detection using all pixel information of Ye, Cy, Mg, and G, accurate error detection can be performed without error detection.

In the complementary color camera system of this embodiment, when performing interpolation detection on a read signal from an imaging device having a complementary color checkerboard pixel array, interpolation is performed using the magnitude relation of the pixel level differences in the horizontal and vertical directions. By determining the method, accurate interpolation can be performed with a small circuit scale.

As described above, according to the complementary color camera system of the present embodiment, by processing the image pickup device having the complementary color checkerboard pixel array, two pixels are mixed from one complementary color checker CCD and all pixels are mixed. Optimal signal processing can be performed for each of the two readout methods. For example, in the case where signal processing is performed by two-pixel mixed readout for a moving image and signal processing is performed for all pixels independently for a still image, optimal signal processing can be realized by one system.

[0078]

As is apparent from the above description, in the present invention, the resolution of the luminance signal can be improved by interpolating the neighboring signals with the readout signal from the image pickup means. Therefore, it is possible to increase the resolution not only in the horizontal and vertical directions but also in the diagonal direction. In addition, when performing pixel interpolation of other neighboring colors with respect to the readout signal from the imaging unit, the resolution can be improved by performing pixel interpolation on some of the pixels.

Further, in the present invention, the image pickup means has a checkerboard-like coating of complementary colors of yellow, cyan, magenta, and green for each pixel arranged in a grid pattern. By performing pixel interpolation only on the magenta and green pixels for the independent readout signal, the resolution can be improved.
Alternatively, when pixel interpolation is performed on a readout signal independent of all pixels from the imaging unit, interpolation detection is performed by using information of pixels of a plurality of other colors simultaneously with information of a color to be interpolated. Good interpolation detection can be performed. Alternatively, when pixel interpolation is performed on all pixel-independent readout signals from the imaging unit, interpolation detection is performed using all pixel information of yellow, cyan, magenta, and green, so that there is no error detection and high accuracy. Interpolation detection can be performed.

Further, in the present invention, when performing interpolation detection, the interpolation method is determined by using the magnitude relation between the pixel level differences in the horizontal direction and the vertical direction, so that accurate interpolation can be performed with a small circuit scale. .

From the above, according to the present invention, it is possible to optimally cope with the signal processing for each of the two-pixel mixed readout method from the image pickup means or both readout methods for all pixels. In the case of (1), signal processing is performed by two-pixel mixed reading, and, for example, in the case of a still image, signal processing can be performed by all-pixel independent reading, and optimal signal processing can be realized by one system.

[Brief description of the drawings]

FIG. 1 is a block circuit diagram illustrating a schematic configuration of a complementary color camera system according to first and third embodiments of the present invention.

FIG. 2 is a diagram illustrating an example of an all-pixel independent readout coating pattern.

FIG. 3 is a diagram illustrating a pixel row handled by a resolution improvement detection interpolation circuit, which is used to explain correlation detection when a G pixel is interpolated with respect to a Mg pixel.

FIG. 4 is a diagram illustrating a pixel row handled by a resolution improvement detection interpolation circuit, which is used for explaining correlation detection when a Cy pixel is interpolated with a Ye pixel.

FIG. 5 is a characteristic diagram showing an example of spectral characteristics of a complementary color coating.

FIG. 6 is a block circuit diagram illustrating a schematic configuration of a complementary color camera system according to a second embodiment of the present invention.

FIG. 7 is a diagram used for describing a first correlation detection method.

FIG. 8 is a diagram used to explain a second correlation detection method.

FIG. 9 is a diagram of a resolution improvement detection interpolation circuit according to a third embodiment, which performs correlation detection using a correlation detection signal obtained by adding signals of Mg pixels and G pixels, and selects an interpolation process according to the detection result. FIG. 3 is a circuit diagram illustrating a specific configuration example.

FIG. 10 is a circuit diagram showing a configuration example of a resolution improvement detection circuit when performing not only three interpolations in the horizontal, vertical, and oblique directions but also finer interpolations.

FIG. 11 is a block circuit diagram showing a configuration example of luminance signal processing at the time of two-pixel mixed reading in a conventional complementary color camera system.

FIG. 12 is a block circuit diagram showing a configuration example of luminance signal processing at the time of independent reading of all pixels in a conventional complementary color camera system.

[Explanation of Signs] 1 input terminal, 2 to 8 1 line delay circuit,
9,21,23 Resolution improvement detection interpolation circuit, 10 V
Filter, 11 1 / 4H filter, 121 / 2H
Filter, 13 Vertical low-pass filter, 14
~ 16 multiplier, 17,18 adder, 19 gamma correction circuit, 20 output terminal, 22 horizontal low-pass filter

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 5C065 AA01 BB10 BB12 BB48 CC01 DD02 DD17 EE05 EE07 EE08 GG01 GG02 GG13 GG21 GG22 GG23

Claims (7)

[Claims] 1. An image signal processing apparatus for processing a read signal from an image pickup means comprising pixels arrayed with complementary color-based coatings, the read signal from the image pickup means being provided with other colors adjacent thereto. An image signal processing device comprising an interpolating unit for interpolating pixels. 2. The apparatus according to claim 1, wherein said interpolation means performs pixel interpolation on some of the pixels when performing pixel interpolation of neighboring other colors with respect to the readout signal from said imaging means. The image signal processing device according to any one of the preceding claims. 3. The image pickup means includes a checkerboard-like coating of complementary colors of yellow, cyan, magenta, and green for each pixel arranged in a grid pattern. 2. The image signal processing apparatus according to claim 1, wherein pixel interpolation is performed only on magenta and green pixels for the readout signal independent of all pixels. 4. The image pickup means includes a checkerboard-like complementary color coating of yellow, cyan, magenta, and green applied to each pixel arranged in a lattice. 2. A pixel according to claim 1, wherein when performing pixel interpolation on the readout signal independent of all pixels, information on a color to be interpolated and information on a plurality of other color pixels are used at the same time. Image signal processing device. 5. The image pickup means includes a checkerboard-shaped complementary color coating of yellow, cyan, magenta, and green applied to each pixel arranged in a grid pattern. 2. The image signal processing apparatus according to claim 1, wherein when performing pixel interpolation on the readout signal independent of all pixels, interpolation detection is performed using all pixel information of yellow, cyan, magenta, and green. 6. An image signal according to claim 4, wherein said interpolation means determines an interpolation method using a magnitude relation between pixel level differences in horizontal and vertical directions when performing said interpolation detection. Processing equipment. 7. The image signal according to claim 5, wherein, when performing the interpolation detection, the interpolation means determines an interpolation method using a magnitude relationship between pixel level differences in a horizontal direction and a vertical direction. Processing equipment.