JPH06205422A - Color image pickup device - Google Patents

Abstract

PURPOSE:To pick up also a still frame picture with excellent resolution and less color moire by providing a specific optical low pass filter to the device, reading required information from a memory and applying digital signal processing to the information. CONSTITUTION:The image pickup device is provided with an image pickup element 1 having an offset sampling structure, with an optical member separating an incident ray into two rays apart by a distance D in a direction with an angle theta to the scanning direction of the image pickup element 1 clockwise or counter-clockwise, with an optical filter 50 whose distance D satisfies the condition in equation, where Ph is a horizontal pitch and Pv is a vertical pitch of the image pickup element 1, and with a memory 4 storing once an output from the image pickup element 1, and a picture is obtained by reading required information from the memory 4 to apply digital signal processing to the information. Thus, a color carrier generated in horizontal and vertical directions is reduce and color moire caused by the color carrier in horizontal and vertical directions is reduced.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a color image pickup apparatus having an image pickup device in which a plurality of pixels are arranged two-dimensionally.

[0002]

2. Description of the Related Art Due to recent rapid advances in semiconductor technology,
In the field of video signal processing, digital signal processing is becoming a favorite. This is because digital signal processing is advantageous over analog signal processing in terms of IC integration, higher precision, ease of various adjustments, and good S / N ratio.

For this reason, the image pickup device has various advantages such as small size, light weight and high reliability, and various digital systems have been developed. Among them, an image pickup device for a video camera or an electronic still camera has a color filter arrangement as shown in FIGS. 28 and 29 from the viewpoint of high resolution, good S / N ratio, and ease of signal processing. CCD imagers are most commonly used today.

When a still image is frame-photographed by an electronic still camera or the like, it is necessary to use an image pickup device having a color filter arrangement in which different colors are repeated every two pixel periods in the vertical direction as shown in FIG. This is because the pixel signals formed by the image sensor are alternately read from the odd field and the even field, and therefore the outputs of both fields must be the same. In the figure, Ph represents the horizontal pitch of the image sensor, and Pv represents the vertical pitch.

On the other hand, the image pickup device as shown in FIG. 29 is not arranged in a color filter such that different colors are repeated every two pixel periods in the vertical direction. That is, this image sensor has a color filter that repeats different colors in a two-pixel cycle in the horizontal direction. And
The filter array includes lines in which magenta Mg and green Gr are repeated and lines in which cyan Cy and yellow Ye are repeated. Also, Mg and Gr have an offset sampling arrangement, and Cy and Ye
Is a square sampling array. On the contrary, Mg and Gr are square sampling arrays, and Cy and Y are
In some cases, e is an offset sampling array.

Therefore, when a signal is read from the image pickup device as shown in FIG. 29, a method called pixel mixed read (or field read) is usually used. This is a method of mixing and reading out the photoelectrically converted signal charges by two rows. According to this method, the signal charge is read out as A _n-1 , A _n , A _{n + 1} , ... In the A field. In the B field, B
_It is read as _n-1 , B _n , B _{n + 1} , ....

As a result, (Mg +
Cy) and (Ye + Gr), and (Cy + Gr) and (M
g + Ye) pixel signals are alternately output. For convenience sake, these four signals will be referred to as C ₁ = Mg + Cy C ₂ = Ye + Gr C ₃ = Cy + Gr C ₄ = Mg + Ye.

At this time, the set of spatial barycentric positions when these four signals are obtained can be considered as a sampling structure by C ₁ , C ₂ , C ₃ , and C ₄ as shown in FIG. . Therefore, the image sensor of FIG. 29 can be considered in the same manner as an image sensor having a color filter arrangement in which different colors are repeated every two pixel periods in the vertical direction when performing pixel mixture reading. Then, a color image is obtained by forming a luminance signal and a color signal of each field based on this pixel signal.

[0009]

However, as shown in FIG.
8. When an image sensor having a color filter arrangement as shown in FIG. 30 is used, in order to derive a luminance signal and a color signal, two row data skipped by one row (for example, 1 and 3 for odd fields are used). Row data, 2 or 4 rows for even fields) must be used.

Therefore, the information of two spatially distant points in the vertical direction is added, and the correlation in the vertical direction is reduced. As a result, the resolution in the vertical direction is deteriorated, and the amount of false color (color moire) generated is increased. In particular, when a still image is photographed by an electronic still camera or the like, when pixel mixture reading is performed, only an image in one field can be obtained, so that a so-called frame image with high resolution cannot be obtained.

Next, the cause of the occurrence of color moire as described above will be described in detail with reference to FIGS. 31 and 32.
In the figure, f _H indicates a horizontal frequency, and f _V indicates a vertical frequency.

FIG. 31 is a spatial frequency plane (f _H , f _V ) of an image pickup device having a color filter arrangement as shown in FIG.
It is a figure which shows the characteristic of the carrier above. Also, FIG.
FIG. 2 is a diagram showing carrier characteristics on the spatial frequency plane (f _H , f _V ) based on the sampling structure as shown in FIG. 31 and 32, the arrow indicates M
g, Gr, Cy, Ye color carriers, and C ₁ ,
The carriers of the respective colors C ₂ , C ₃ , and C ₄ are represented, and the directions of the arrows represent the phase relationship of the respective colors.

Here, the carrier in which the phases of all four colors are aligned is a carrier generated for achromatic light, and this carrier becomes a carrier of a luminance signal (hereinafter referred to as "luminance carrier"). In addition, the other carriers that are not aligned in phase cancel each other out completely with respect to achromatic light and disappear, but they do not cancel each other out at the time of generation of the color difference signal, and they appear as color moiré. Carrier (hereinafter referred to as "color carrier").

According to FIG. 31, the frequency (0,
It can be seen that color carriers are generated at 1/2 Pv). This color carrier causes color moire when a subject having an achromatic fine pattern is photographed. Also,
According to FIG. 32, a relatively low frequency in the vertical direction (0, 1
It can be seen that color carriers are generated at / 4 Pv).
This is 240 TV for NTSC standard video camera.
It is equivalent to a book. This color carrier causes color moire at a relatively low frequency in the vertical direction.

The generation of this color moire is generally suppressed by an optical low-pass filter placed in front of the image pickup device. On the other hand, the use of this optical low-pass filter makes it possible to reduce the horizontal resolution. Cause deterioration of. Therefore, in the conventional video camera, it is necessary to design an optical low-pass filter by trade-off between resolution and color moiré caused by a color carrier to determine the image quality.

This problem will be described in detail with reference to FIGS. 33 to 37. FIG. 33 is a frequency characteristic diagram showing the generation position of the carrier component in the vicinity of the base band of the image sensor shown in FIGS. 28 and 30.

According to this, the luminance carrier has a frequency (±
It occurs at the position of 1 / Ph, 0). Both image sensors have a color filter horizontal repetition period of 1
Since it is / 2Ph, the color carrier has a frequency (± 1 /
It can be seen that it occurs at the position of 2Ph, 0).

By the way, according to the sampling theorem, the luminance signals obtained from these image pickup devices should be able to have a pass band up to a frequency of 1/2 Ph. Therefore, in order to prevent the aliasing distortion of the luminance signal, it is ideal that the optical low-pass filter cuts all frequency components above the Nyquist frequency f _H = 1 / 2Ph to zero.

However, as described above, f _H = 1 / 2Ph
Since a color carrier is generated at the position of, if an optical low-pass filter having the frequency characteristics as described above is used,
Color moiré occurs due to aliasing distortion of the color difference signals. Therefore, the frequency characteristic of the optical low-pass filter, as shown in FIG. 34, the generation of color carrier f _H =
The pass band must be set lower than the frequency of 1/2 Ph by the band of the color difference signal. Therefore, it is possible to obtain only a resolution lower than the theoretical resolution limit.

Next, FIG. 35 shows a schematic structure of a conventional optical low-pass filter. In FIG. 35, an optical low-pass filter 110 includes a birefringent plate 111 that divides an incident light beam into two in the horizontal direction by a distance Ph.
It is composed of a phase plate 112 for converting linearly polarized light into circularly polarized light, and a birefringent plate 113 for splitting an incident light ray into two in the horizontal direction by a distance Ph / 2.

The transfer characteristic (MTF characteristic) of this optical low-pass filter is H ₁ (f _H , f _V ) = | cos (π / 2 · Phf _H ) · cos (πPhf _H ) | (1) It is expressed by

A graph of this is shown in FIG.
FIG. 37 shows the characteristic on the spatial frequency plane (f _H , f _V ). In FIG. 37,
Dotted lines 114a, 114b, 115a, 115b are resonance frequencies trapped by the optical low pass filter 110. From this, the optical low-pass filter 110 is
It can be seen that the carrier frequency f _H = 1 / Ph of the luminance signal and the carrier frequency f _H = 1 / 2Ph of the color difference signal are trapped.

Further, in the optical low-pass filter 110 having the above-mentioned characteristics, as can be seen from the equation (1) or FIG. 36, the carrier frequency f _H = 1 / of the color difference signal.
At a frequency of about 85% of 2Ph, the MTF is already -15.
It dropped to db, and even if f _H = 1 / 2Ph, MT
F is 0. Therefore, in the conventional image sensor using such an optical low pass filter 110,
Only a resolution lower than the resolution limit f _H = ½Ph could be obtained.

Further, as is apparent from FIG. 36, f _H =
Since the frequency components of 1/2 Ph or more are not sufficiently suppressed, there is also a problem that due to this, color moiré occurs due to aliasing distortion of the color carrier.

In recent years, in order to improve image quality, a two-plate type color image pickup apparatus using two image pickup elements, and a single-plate type color image pickup using an image pickup element capable of obtaining a high image quality comparable thereto. Although the device is adopted, these color image pickup devices also have the problem of occurrence of color moiré as described above and the problem that sufficient resolution cannot be obtained.

The present invention has been made in view of the above problems, and an object of the present invention is to provide a color image pickup device having a high resolution, a small amount of color moire, and a still frame image.

[0027]

A color image pickup apparatus according to the present invention has four kinds of color filters, in which color filters of different colors are repeated every two pixel periods in the horizontal direction and in the vertical direction. Is an image pickup device having an offset sampling structure in which a color filter of a different color is repeated with a one-pixel offset in a horizontal direction in a two-pixel cycle, and an incident light beam is directed to a scanning direction of the image pickup device. It has an optical member that separates into two light rays separated by a distance D in a direction that forms an angle θ in the clockwise or counterclockwise direction, and the horizontal pitch of the image pickup element is P.
h, when the vertical pitch is Pv, the distance D is configured to satisfy the condition of 1.6PhPv / | Phsinθ + 2Pvcosθ | ≦ D ≦ 2.4PhPv / | Phsinθ + 2Pvcosθ | (0 ≦ θ ≦ π / 2) And a memory for temporarily storing the output from the image sensor, and an image is obtained by reading out necessary information from the memory and performing digital signal processing. It is a thing.

Another feature of the present invention is that the output from the image pickup device is applied to an achromatic subject in the above-mentioned 4
A gain adjusting unit that adjusts the gain so that the output from each type of color filter is constant, and in all rows, the sum of the coefficients of any two columns is equal to the sum of the remaining two coefficients, and is in a 3 × 4 matrix. By performing the matrix calculation of
It has a signal forming means for forming RGB signals from the outputs of the four types of color filters.

Further, the color image pickup device of the present invention has four kinds of color filters, in which color filters of different colors are repeated in the horizontal direction at a period of two pixels, and two pixels in the vertical direction. An image sensor having an offset sampling structure in which color filters of different colors are repeated by offsetting one pixel in the horizontal direction in the cycle, and the sum of the coefficients of arbitrary two columns remains in all rows. A signal forming means for forming RGB signals from the outputs of the four types of color filters by performing a matrix operation of 3 rows and 4 columns equal to the sum of two coefficients may be provided. .

Still another feature of the present invention is that in a color image pickup apparatus having an image pickup device having a plurality of pixels arranged two-dimensionally in the horizontal direction and the vertical direction, a desired image pickup device is used for the image pickup device. The number of horizontal lines required to obtain an image is set to twice, and the horizontal lines are the first horizontal line on which the first color filter is arranged, the second color filter and the third color filter. And a second horizontal line arranged every other pixel are alternately arranged, and the signal output from the pixel to which the second color filter is arranged and the first color filter in the same row as the signal output from the pixel. Form a first difference signal by taking the difference from the signal output from the selected pixel and the signal output from the pixel on which the third color filter is arranged, and the same row as that. By taking the difference between a certain signal the first color filter is output from the pixels arranged to form a second difference signal, said first difference signal and the second
It is characterized in that the color difference signal is formed based on the difference signal.

Further, the present invention comprises a first image pickup device having a first color filter and a second image pickup device having second and third color filters, and the first image pickup device described above. In a color image pickup apparatus configured to obtain a desired image by using a device and a second image pickup device, a signal output from a pixel in the second image pickup device in which a second color filter is arranged, and a signal output from the signal The first difference signal is formed by taking the difference from the signal output from the pixel in which the first color filter in the first image pickup element is arranged in the equivalent position on the surface, and the second difference signal is formed. Third in the image sensor of
Difference between the signal output from the pixel in which the color filter is arranged and the signal output from the pixel in which the first color filter is arranged in the first image sensor at the position equivalent to it May be used to form the second difference signal, and the color difference signal may be formed based on the first difference signal and the second difference signal.

At this time, the luminance signal forms a luminance signal by synthesizing the signal output from the pixel provided with the first color filter with the first difference signal and the second difference signal. . Further, it may be formed based on the signal output from the pixel in which the first color filter for the luminance signal is arranged.

[0033]

According to the present invention configured as described above, the four types of color filters arranged in the image pickup device are arranged in the horizontal direction at 2Ph and the vertical direction at 2Pv.
Arranging in an array with an offset sampling structure offset by h reduces horizontal and vertical color carriers, which reduces horizontal and vertical color moire caused by these color carriers. At the same time, high resolution can be obtained up to the limit resolution.

Further, two optical low-pass filters are provided, which are separated by a distance D satisfying a certain condition in a direction in which the incident light beam makes an angle of θ in the clockwise or counterclockwise direction with respect to the scanning direction of the image sensor. By using the optical member that separates into the light rays, sufficient MTF characteristics can be obtained in a region below the limit resolution, and thus high resolution up to the limit resolution can be obtained. Further, since the frequency components above the frequency of the limit resolution are sufficiently suppressed, the color moire due to the aliasing distortion of the color carrier is reduced.

With respect to the output from the image pickup device, the gains are adjusted so that the outputs from the four types of color filters are constant, and the matrix calculation performed when forming the RGB signals from those outputs is performed. Performing a sum of the coefficients of any two columns in all rows equal to the sum of the remaining two coefficients with a matrix of 3 rows and 4 columns further reduces the color carriers generated in the vertical direction. This further reduces color moiré in the vertical direction.

In the case of a multi-plate color image pickup device using a plurality of image pickup devices or a single plate type color image pickup device using an image pickup device having a horizontal line number twice as many as usual, each pixel of these image pickup devices is Are different types of first to third color filters, and the signals respectively obtained from the pixels having the second color filter and the third color filter, and the positions at which the respective pixels are equivalent on the imaging surface. And forming a color difference signal based on the two kinds of difference signals by taking the difference from the signal obtained from the pixel in which the first color filter is arranged. Results in less horizontal and vertical color carriers, which results in horizontal and vertical color moiré caused by these color carriers. As well as reduced, high up to a limit resolution resolution can be obtained.

At this time, the luminance signal is formed by correcting the signal obtained from the pixel in which the first color filter is arranged so as to combine the above two kinds of difference signals to form a luminance signal. A good luminance signal can be obtained. Further, if a color filter having the same spectral characteristic as the luminosity factor is used as the first color filter and the luminance signal is formed based on the signal obtained from the pixel in which the first color filter is arranged, the processing is performed. Can be simplified.

[0038]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing a first embodiment of the color image pickup apparatus of the present invention. In FIG. 1, reference numeral 1 denotes an image pickup device for converting light from an optical low-pass filter 50, which will be described later, into an electrical signal, and four types of color filters, which will be described later, are arranged.

An image pickup signal is read from the image pickup device 1 for each pixel. The image pickup signal read from the image pickup device 1 is subjected to processing such as CDS (removal of reset noise and clock noise of output signal of the image pickup device 1) and AGC (adaptation of input signal level) in the preprocessing circuit 2. After that, the A / D converter 3 performs A / D conversion at a timing synchronized with the read clock. This A / D converter 3 preferably has a linear characteristic for color processing performed later,
Considering the quantization error, A /
Those that perform D conversion are desirable.

The A / D converted signal is processed later by 2
It is once written in the buffer memory 4 for the dimension signal processing. Then, the signals required for processing are read out from the buffer memory 4.

The signals corresponding to the luminance signals are sequentially read from the buffer memory 4 in the order corresponding to the pixel array of the image pickup device 1 and input to the low pass filter 5. The signal input to the low-pass filter 5 is subjected to a predetermined band limitation here, and then subjected to a predetermined process by the γ conversion unit 14 and the enhancer 15 to be a luminance signal Y _H including a high frequency component. The luminance signal Y _H including this high frequency component is
Of the high pass filter (HP
F) 21 may be used.

The luminance signal Y _H including the high frequency component thus obtained is combined with the low frequency component Y _L of the luminance obtained by the method described later in the luminance signal forming circuit 16 to obtain the luminance signal Y. Is output.

On the other hand, the signals corresponding to the color signals Mg, Gr, Cy and Ye are read from the buffer memory 4 in the same manner as described above and are synchronized by the four interpolation filters 6, 7, 8 and 9, respectively. The color signals are Mg, Gr, Cy, and Ye. Subsequently, these four types of color signals are input to the RGB conversion section 10 and converted into three primary color signals of R, G, and B. It should be noted that this conversion is performed by the matrix operation shown below.

[0044]

[Equation 1]

Here, the matrix A is M of the image sensor 1.
Spectral characteristics of g, Gr, Cy, Ye Mg (λ), Gr
(Λ), Cy (λ), Ye (λ) are the ideal spectral characteristics R (λ), G of R, G, B defined by the NTSC standard.
(Λ), 3 rows 4 optimized to approach B (λ)
It is a matrix of columns.

Next, in the white balance section 11, RGB
White balance is achieved by converting the three R, G, and B signals formed by the conversion unit 10 into the forms αR, G, and βB, respectively. This white balance adjustment is performed by the white balance sensor 2 as shown in FIG.
Alternatively, it may be automatically performed based on the color temperature information obtained by using 2.

Subsequently, the RGB signals are subjected to γ conversion by table conversion in the γ conversion unit 12, and then the color difference matrix unit 1
Input to 3. The color difference matrix unit 13 performs color difference conversion conforming to the NTSC standard as described below to obtain the above-described low-frequency component Y _{L of} luminance and two color difference signals R
-Y and BY are generated.

[0048]

[Equation 2]

The color difference signals R-Y and B-Y are output after being subjected to predetermined band limitation by the low pass filters 18 and 19, respectively. Also, the low-frequency component Y of brightness
_L is a luminance signal Y _H including a high frequency component after being subjected to a predetermined band limitation by the low pass filter 17, as described above.
And the luminance signal forming circuit 16 synthesizes it with each other and outputs it as a luminance signal Y. Then, each of these signals is thereafter D / A converted by a D / A converter (not shown) to be an analog signal.

[0050] Synthesis according to the luminance signal forming circuit _{16, Y = Y H -δ (LPF17} ) * Y H + δY L (0 ≦ δ ≦ 1) intended to be carried out by the conversion formula shown in (4) is there. Here, (LPF17) is a low-pass filter having the same band as the low-pass filter 17, and (LPF17) * Y _H is the same band limitation as the low-pass filter 17 applied to the luminance signal Y _H including high-frequency components. It means that there is.

In general, such a conversion is performed because the low-frequency component Y _L of the luminance has a correct luminance in terms of the spectral characteristics, but the S / N ratio is not necessarily good, and the high-frequency component is included. This is because the luminance signal Y _H has a good S / N ratio on the contrary, but the luminance is not always correct due to the spectral characteristics.

Here, δ is the S / N of the low frequency component Y _L of the luminance.
It may be fixed at 1 if the ratio is good. In this case, the processing in the luminance signal forming circuit 16 is equivalent to replacing the low-frequency component Y _L entirety of brightness the same band part of the low-frequency component Y _L of the luminance of the luminance signal Y _H including a high frequency component To do.

If the spectral characteristic of the luminance signal Y _H including the high frequency component is sufficiently close to the characteristic of the original luminance, δ may be fixed to 0. In this case, the luminance signal forming circuit 16
Luminance signal Y including high frequency components without any processing
_This corresponds to the case where _H is directly output as the luminance signal Y. Note that the value of δ may be changed according to the brightness and color information of the subject in consideration of the S / N ratio and color reproducibility.

FIGS. 3 and 4 are views showing an embodiment of the color filter array arranged in the image pickup device 1 described above.
This color filter array is composed of Mg, Gr, Cy, Ye.
It is composed of 4 colors. And 2 for the horizontal direction
Repeat different colors in Ph, 2P in the vertical direction
The array has an offset sampling structure in which v is offset by Ph in the horizontal direction and different colors are repeated.

The color filter array need not necessarily be as shown in FIGS. 3 and 4, for example,
Alternatively, Mg and Cy or Gr and Ye may be replaced with each other. Furthermore, not limited to these four colors, for example,
You may make it arrange | position by the combination of colors, such as red Rd, blue Bl, and white Wh, and 4 colors arbitrarily selected from said 4 colors.

FIG. 5 is a frequency characteristic diagram showing the generation position of the carrier component in the vicinity of the base band of the image sensor 1 in which the color filter is arranged in this way. According to this, the luminance carrier has a horizontal frequency (± 1 / Ph,
0) and the color carrier is at frequency (± 1 / 2P
It can be seen that it occurs at the positions of h, ± 1/4 Pv) and (0, ± 1/2 Pv).

A more detailed representation of this frequency characteristic is shown in FIGS. 6 and 7. 6 and 7 are frequency characteristic diagrams of the image sensor 1 having the color filter arrangements shown in FIGS. 3 and 4, respectively.

Comparing these frequency characteristics with the frequency characteristics of the conventional image sensor shown in FIGS. 31 and 32, in the present embodiment, the vertical frequency (0, 1 /
At the position of 4Pv, the horizontal frequency (1 / 2Ph,
It can be seen that no color carrier is generated at the position 0). Therefore, color moire caused by folding back from these color carriers is less likely to occur. Furthermore, since the color carrier does not occur at the position of the frequency (1 / 2Ph, 0), it is possible to obtain a resolution of up to f _H = 1 / 2Ph a fundamental resolution limit.

Next, FIG. 8 shows a schematic structure of the optical low-pass filter 50 described above. In FIG. 8, the optical low-pass filter 50 has a horizontal angle of 45 °.
The first optical member composed of the birefringent plate 51 that divides the incident light ray into two by a _first distance d ₁ in the direction of and the two incident light rays by the second distance d ₂ in the horizontal direction. A second optical member composed of a birefringent plate 52 which is divided into two parts, and a birefringent plate 53 which divides an incident ray into two parts by a _first distance d ₁ in the direction of −45 ° with respect to the horizontal direction. It is composed of a third optical member. These birefringent plates 51 to 5
Due to the action of 3, the incident light ray follows the locus shown in FIG.

This optical low-pass filter 50 has a relationship of 1.6PhPv / | Phsin θ + 2Pvcos θ | ≦ d ₁ ≦ 2.4PhP v / | Phsin θ + 2Pvcos θ | (0 ≦ θ ≦ π / 2) with respect to the first distance d _1. It is configured to satisfy the condition represented by the inequality (5).

That is, if the first distance d ₁ is inequality (5)
If the lower limit is exceeded, it is not possible to suppress the aliasing distortion due to the color carrier, especially the occurrence of color moire. On the contrary, if the upper limit is exceeded, sufficient resolution cannot be obtained. Therefore, in order to prevent color moiré and obtain sufficient resolution, the first distance d ₁ needs to satisfy the inequality (5).

By the way, the transfer characteristic (MTF characteristic) of the optical low-pass filter 50 is expressed by the following equation. H ₂ (f _H , f _V ) = | cos (πd ₂ f _H ) · cos (π / √2 · d ₁ (f _H + f _V )) · cos (π / √2 · d ₁ (f _H −f _V )) | ... (6)

Here, for example, in the NTSC system, the image pickup device is used when the aspect ratio of the display screen is 3: 4, and the number of effective pixels is 640 in the horizontal direction and 4 in the vertical direction.
In the image pickup device of about 80, the following relationship is established. Ph = Pv (7) Therefore, in the present embodiment, the first distance d ₁ and the second distance d ₂ are set as follows: d ₁ = 2√2 / 3 · Ph, d ₂ = Ph / 2 ···・ (8)

The MTF characteristic satisfying the condition of the equation (8) is represented by a graph as shown in FIG. The characteristics on the spatial frequency plane (f _H , f _V ) are expressed as shown in FIG. In FIG. 11, dotted lines 54a, 54
b, 55a, 55b, 56a and 56a are resonance frequencies trapped by the optical low pass filter 50. From this, it is understood that the optical low-pass filter 50 traps all carrier frequencies of both the luminance and color difference signals.

Further, as can be seen from FIG. 10, since the frequency component of the resolution limit frequency f _H = 1 / 2Ph or more is sufficiently suppressed, the aliasing distortion of luminance does not occur. Therefore, this can reduce color moire. Further, equations (6), (7), (8) and FIG.
As is apparent from, MTF, in the frequency region of f _H ≦ 1 / 2Ph, so does not fall beyond the -15 dB, it is possible to secure the resolution to approximately the resolution limit frequency f _H = 1 / 2Ph.

On the other hand, the sampling frequency of the luminance signal, which is widely used at present, is 4 which is the carrier frequency f _SC of the color difference signal.
In a double image pickup device, the following relationships are almost established. Ph = 1.2Pv (9) At this time, if the first distance d ₁ and the second distance d ₂ are d ₁ = Ph, d ₂ = Ph / 2 (10) The same effect as the above effect can be obtained.

As described above, according to this embodiment, on the one hand, due to the characteristics of the optical low-pass filter 50, it is possible to obtain a high resolution up to the limit resolution. On the other hand, on the other hand, the action of the color filter array of the image pickup device 1 and Due to the characteristics of the optical low pass filter 50, color moire can be sufficiently suppressed. Therefore, when designing the optical low-pass filter 50, it is not necessary to consider the trade-off between resolution and color moiré, and it is possible to design with priority on resolution. Therefore, the optical low-pass filter 50 can be made thin and compact, so that the entire imaging optical system can be made compact.

Next, the third embodiment will be described with reference to FIG. 12, parts corresponding to those in FIG. 1 are designated by the same reference numerals, and their basic operations are the same as those in FIG. 1, so detailed description thereof will be omitted here.

In this embodiment, a gain adjusting circuit 20 is arranged in front of the A / D converter 3, and the gain adjusting circuit 20 allows the gain of the image pickup signal obtained at each pixel of Mg, Gr, Cy and Ye. Is characterized in that it is adjusted to be uniform with respect to an achromatic subject. Note that this gain adjustment may be performed on the analog signal before A / D conversion as shown in FIG. 12, but if the quantization accuracy of the A / D converter 3 is sufficient, the A / D conversion is performed. It may be performed after the conversion.

By performing such gain adjustment,
Not only the unnecessary modulation component is not superimposed on the luminance signal, but also the following improvement can be brought to the color signal.

Now, in the conversion equation shown in equation (2),
Let the matrix A be A = (a _ij ) (i = 1, 2, 3; j = 1, 2, 3, 4) (11).

On the other hand, by the gain adjusting circuit 20, Mg,
Gain of each pixel of Gr, Cy, and Ye to an achromatic subject
Since it is adjusted to be uniform with respect to Mg, Gr,
The baseband components of Cy and Ye are in the frequency space.
Function φ (f_H, F_V), Mg (f_H, F_V) = Gr (f_H, F_V) = Cy (f_H, F_V) = Ye (f _H , F_V) = Φ (f_H, F_V) ... (12) can be represented.

Here, the baseband components of each color are interpolated
Two-dimensional filters such as filters 6, 7, 8, 9 etc.
Assuming that the bandwidth is limited, it is shown in FIGS. 6 and 7.
Carrier component at the frequency (0,1 / 2Pv)
Are Mg (f_H, F_V) = Gr (f_H, F_V) = − Φ (f_H, F_V), Cy (f _H , F_V) = Ye (f_H, F_V) = Φ (f_H, F_V), Mg (f_H, F_V) = Gr (f_H, F_V) = Φ (f_H, F_V), Cy (f_H , F_V) = Ye (f_H, F_V) = − Φ (f_H, F_V) (13)

Therefore, the carrier component of the RGB signal at this point is R (f _H , f _V ) = ± (−a ₁₁ −a) according to the equations (2), (11), (12) and (13). ₁₂ + a ₁₃ + a ₁₄ ) φ (f _H , f _V ), G (f _H , f _V ) = ± (−a ₂₁ −a ₂₂ + a ₂₃ + a ₂₄ ) φ (f _H , f _V ), B (f _H , F _V ) = ± (−a ₃₁ −a ₃₂ + a ₃₃ + a ₃₄ ) φ (f _H , f _V ) ... (14)

According to this, in each row of the matrix A, if the sum of the coefficients in the first and second columns is equal to the sum of the coefficients in the third and fourth columns, that is, a _i1 + a _i2 = a _i3 + A _i4 (i = 1,2,3) (15) If the relationship holds, the carrier component of the RGB signal at this point (0,1 / 2Pv) disappears and no color carrier occurs at this point. . Similarly, if the relation of the expression (15) is established, the point (0, 1 / 2P
The carrier component at a point (0, -1 / 2Pv) symmetrical with respect to v) at the origin also disappears, and no color carrier occurs at this point.

That is, by constructing the matrix A in the RGB conversion section 10a so as to satisfy the condition shown in the expression (15), the carrier component in the vertical direction of the color signal can be eliminated, so that it does not occur. It is possible to reduce color moire that occurs in a subject with a finely colored pattern.

When the gain adjustment by the gain adjusting circuit 20 can be accurately performed at each color temperature, it is not necessary to perform the white balance adjustment in the white balance section 11.

In this embodiment, the output luminance signal Y and the color difference signal R are output regardless of the presence or absence of the gain adjusting circuit 20.
-Y and BY may be output non-interlaced or interlaced. When output in non-interlaced format, it is advantageous for image compression with a card camera and is suitable for computer input. When output by interlace, it can be used in a video camera or a still video camera. At this time, a D / A converter for converting the output signal into an analog signal is required at the output end.

In addition, the above-mentioned interpolation processing, band limitation, RGB
Each processing such as conversion processing is a linear processing.
Therefore, the result is the same even if the order of these processes is changed, so that the order of these processing units may be changed according to the hardware used. Therefore, what is equivalent to the block diagram described above only by changing the order is included in the present invention.

As described above, according to the first to third embodiments, the color filter arranged in the image pickup device 1 has the offset sampling arrangement, and the optical low-pass filter 50 is constituted so as to satisfy a certain condition. .
Furthermore, the matrix calculation performed when forming the RGB signals from the signals output from the image sensor 1 is performed in three rows in which the sum of the coefficients of arbitrary two columns in all rows is made equal to the sum of the remaining two coefficients. By using the matrix of four columns, it is possible to reduce the color carriers generated in the horizontal direction and the vertical direction, and it is possible to reduce the color moire in the horizontal direction and the vertical direction caused by these color carriers. Furthermore, since color moiré that occurs at the resolution limit can be eliminated, it is possible to obtain a high resolution up to the resolution limit.

By the way, the improvement of the image quality in the color image pickup device has been rapidly advanced in recent years by the introduction of the high image quality formats such as S-VHS and high band 8 m / m. Corresponding to this, it is necessary to improve the image quality in the camera section, but the limit is approached in the single-plate color image pickup apparatus using only one image pickup element. Therefore, in some models, a two-plate type color image pickup device using two image pickup elements is currently used.

FIG. 38 is a diagram showing a structural example of this two-plate type color image pickup device. In FIG. 38, the incident light taken in by the lens 121 is separated into two optical paths by the prism 122, and is imaged on the luminance signal image pickup device 123 and the color signal image pickup device 124.

The signal taken out from the image pickup device 123 for the luminance signal is subjected to a predetermined processing by the luminance signal processing circuit 125, and is guided to the encoder 127. On the other hand, the signal extracted from the color signal imaging device 124 is the color signal processing circuit 12.
It is separated into R, G, and B in 6 and guided to the encoder 127. Then, these luminance signal and chrominance signal are converted into an NTSC signal by the encoder 127 and output to the output terminal 12
It is taken out from 8.

Conventionally, the luminance signal image pickup device 123 and the color signal image pickup device 124 are, for example, as shown in FIG.
Those having the color filter arrangements shown in (a) and (b) were used, respectively. As is apparent from this, the image sensor 123 for the luminance signal is provided with a green G color filter over the entire surface. Further, in the color signal imaging device 124, red R and blue B color filters are arranged in a vertical stripe shape.

In the two-plate type color image pickup device, since these two image pickup elements 123 and 124 are separately installed, the characteristics of each image pickup element 123 and 124 can be optimized. Furthermore, since the information amount is twice as large as that of the single-plate color image pickup device, it is possible to obtain high image quality which cannot be obtained by the single-plate color image pickup device.

However, in this two-plate color image pickup apparatus, the optical system becomes extremely complicated, and two drive circuits are required to correspond to the two image pickup elements 123 and 124. Therefore, in the color image pickup device aiming at small size, light weight and low cost, the conventional two-plate type color image pickup device is large, heavy and expensive.

On the other hand, as an image pickup device capable of obtaining the same high image quality as that of the two-plate type color image pickup device, even a single-plate type color image pickup device is disclosed in, for example, Japanese Patent Laid-Open No. Hei 2 (1999) -211.
There is one such as shown in Japanese Patent Publication No. 183677.

As shown in FIG. 40, the total number of horizontal lines is twice the desired number of scanning lines, and a horizontal line group for luminance signals indicated by Y and color signals indicated by R and B are provided. Alternate with the horizontal lines of. Then, the signals of these two types of lines are read out as one set, and predetermined signal processing is performed. The color filter indicated by Y is a color filter designed in advance so that its spectral characteristic matches the visual sensitivity.

However, even when the two-plate type color image pickup device or the single-plate type color image pickup device as described above is used, the above-mentioned problem of color moire and the problem that a sufficient resolution cannot be obtained. there were. That is, the frequency characteristics of the image sensor shown in FIGS. 39 and 40 are both as shown in FIG. As is clear from this, the horizontal frequency (± 1 / 2Ph, 0), (± 1
A color carrier is generated at the position of / Ph, 0). Therefore, the color moiré occurs due to the action of the color carrier, and the frequency component that should be originally resolvable cannot be resolved.

The fourth to eighth embodiments for solving the above-mentioned problems in the case of using such a single plate type color image pickup device or a two plate type color image pickup device will be described below. The fourth and fifth embodiments show a single-plate color image pickup device, and the sixth to eighth embodiments show a two-plate color image pickup device. In addition, in the configuration diagrams showing the respective embodiments, portions denoted by the same reference numerals mean the same configuration portions.

FIG. 13 is a block diagram showing the fourth embodiment. In FIG. 13, an image pickup device 61 is an image pickup device having twice as many scanning lines as desired, and has color filters arranged as shown in FIG. In this image pickup device 61, a G filter is arranged on an odd line,
R filters and B filters are alternately arranged on even-numbered lines. Here, among the G filters of the odd lines, those in the same column as the R filters in the even lines are designated as G _1, and those in the same column as the B filters are designated as G ₂ .

The image pickup signal output from the image pickup device 61 is
The color separation units 63 and 64 separate the G ₁ signal and the G ₂ signal, and the R signal and the B signal, respectively. Note that in this color separation processing, the odd-numbered line and the even-numbered line in FIG.
The processing can be simplified.

Appropriate gains for white balance are applied to the separated signals in white balance units 65 and 66, respectively, and γ conversion is performed in subsequent γ conversion units 67 and 68. This white balance processing may be performed only on the R and B signals. In this case, the white balance gain cannot be applied to the G ₁ and G ₂ signals.

In the adder 69, the difference between the G ₁ signal and the R signal output from the γ conversion units 67 and 68 is calculated, and the difference signal R
-G ₁ is formed. Further, in the adder 70, the difference between the G ₂ signal output from the γ conversion units 67 and 68 and the B signal is calculated to form a difference signal B−G ₂ .

These difference signals R-G ₁ and B-G ₂ are input to the color difference signal processing circuit 72 and converted into color difference signals R-Y and B-Y by the conversion formulas shown below. continue,
Color difference signals R-Y and B-Y having desired frequency bands are generated by a low-pass filter (not shown) or the like.

[0096]

[Equation 3]

Here, (1 / 2P on the spatial frequency plane
It is assumed that an achromatic subject at the position h, 0) is sampled by the image sensor 61. This subject is vertical stripes having a period of 2Ph. Since R = G ₁ and B = G ₂ for such a subject, the difference signal R output from the adders 69 and 70 is
Both -G _1, B-G ₂ is zero.

Therefore, the color difference signals RY and BY output from the color difference signal processing circuit 72 are also zero and are not output. This means that the color carrier disappears at the position of the frequency (1 / 2Ph, 0). In other words, the R signal carrier and the G ₁ signal carrier have the same phase on the frequency (1 / 2Ph, 0), and the B signal carrier and the G ₂ signal carrier have the same phase. It is a phase. Therefore, these difference signals R-G ₁ and B-G
_Since carriers at ₂ in this frequency can be extinguished, color carriers are not generated.

Difference signal R output from the adders 69 and 70
-G ₁ and B-G ₂ are also input to the brightness correction signal forming circuit 73. The brightness correction signal forming circuit 73 is composed of a constant multiplier and an adder for the difference signals R-G ₁ and B-G ₂ . That is, in the luminance correction signal forming circuit 73, the difference signals R-G ₁ and B-G _{2 are} converted from the difference signals R-G ₁ and B-G ₂ by the conversion formula S = C ₁ (R-G ₁ ) + C ₂ (B-G ₂ ) ... (17) A brightness correction signal S is formed. Note that C ₁ and C ₂ are constants, and can be set to C ₁ = 0.30 and C ₂ = 0.11.

On the other hand, G ₁ output from the γ conversion unit 67,
The G ₂ signal is input to the luminance signal processing circuit 71, and appropriate processing such as low-pass filtering and enhancement is performed. Then, the brightness correction signal S formed as described above is added by the adder 74. By thus adding the brightness correction signal S to the G ₁ and G ₂ signals, the spectral characteristics of the brightness signal are corrected, and a good brightness signal matching the visual sensitivity can be obtained.

As described above, according to this embodiment, it is possible to eliminate the color carrier at the frequency f _H = 1 / 2Ph, so that it is possible to obtain a color image pickup device having a small amount of color moire and a high resolution. You can

Next, the fifth embodiment will be described. FIG. 15 is a block diagram showing the configuration of this embodiment. Figure 1
5, the image pickup device 62 is an image pickup device having a line twice as many as the desired number of scanning lines, and has a color filter arranged as shown in FIG. The image sensor 62 includes
Y filters, which are designed in advance so that the spectral characteristics match the visual sensitivity, are arranged on the odd lines, and R filters and B filters are alternately arranged on the even lines. Here, among the Y filters of the odd lines, the R of the even lines is used.
The one in the same row as the filter is designated as Y _1, and the one in the same row as the B filter is designated as Y ₂ .

In this embodiment, the Y ₁ signal and Y separated by the color separation units 75 and 64 are used as in the fourth embodiment.
_{From the two} signals and the R and B signals, adders 69, 7
The difference signals R-Y ₁ and BY ₂ are formed by 0. Since the respective difference signals formed here are the color difference signals RY and BY, the color difference signal processing circuit 76 does not have to perform the conversion processing into the color difference signals as shown in the equation (16). Further, since the Y ₁ signal and the Y ₂ signal which have been adjusted to the luminosity in advance are input to the luminance signal processing circuit 71, (17)
It is not necessary to perform the formation processing of the brightness correction signal S as shown in the equation. Therefore, in this embodiment, the processing can be simplified as compared with the fourth embodiment.

Also in the present embodiment having such a configuration, the color carrier at the frequency f _H = 1 / 2Ph can be eliminated in the same manner as in the fourth embodiment, so that the amount of color moire generated is small and the resolution is high. A high color imaging device can be obtained.

Next, a sixth embodiment will be described. FIG. 17 is a block diagram showing the configuration of this embodiment. Figure 1
7, the image pickup elements 81 and 82 are respectively arranged in FIG.
The color filters are arranged as shown in (a) and (b). That is, the image sensor 81 is provided with a G filter for luminance signals on one surface. On the other hand, the image sensor 8
In FIG. 2, R filters and B filters for color signals are arranged in a vertical stripe pattern. Here, the image sensor 8
Among the G filters of No. 1, those at the position equivalent to the R filter group of the image sensor 82 on the image pickup surface are designated as G _1, and those at the position equivalent to the B filter group are designated as G ₂ .

In this embodiment, the two image pickup devices 8 are thus provided.
1 and 82, and other parts are the same as those in the fourth embodiment shown in FIG. The operation of each part is performed in the same manner as in the fourth embodiment. Therefore, also in this embodiment, the frequency f _H = 1 /
The color carrier at 2Ph can be eliminated, the amount of color moire generated is small, and a color image pickup device with high resolution can be obtained.

Next, the seventh embodiment will be described. FIG. 19 is a block diagram showing the configuration of this embodiment. Figure 1
9, the image pickup devices 83 and 84 are respectively shown in FIG.
The color filters are arranged as shown in (a) and (b). That is, the image sensor 83 is provided with a G filter for the luminance signal on one surface. On the other hand, the image sensor 8
4 has 2 R and B filters for color signals
The lines are arranged in horizontal stripes. Here, of the G filters of the image sensor 83, the image sensor 84
G at the position equivalent to the R filter group of
₁ is set to the position equivalent to the B filter group as G ₂
I am trying.

In the image pickup device having such a filter arrangement, color carriers are generated at the positions shown in FIG.

The image pickup devices 83 and 84 are scanned every other horizontal line by the interlace method, and the G ₁ and G ₂ signals and the R and B signals are read out from them. These signals are output to the white balance unit 6 respectively.
After predetermined processing by 5, 66 and γ conversion units 67, 68, 1H delay circuits 85, 86 separate G ₁ signal and G ₂ signal, and R signal and B signal, respectively.

The G ₁ signal and the R signal, and the G ₂ signal and the B signal thus separated are added to the adder 6
At 9, 70, the difference signals R-G ₁ and B-G ₂ are formed. Then, the color difference signal processing circuit 72 causes the color difference signals R-Y, B
-Y is formed, and a desired band is limited by a low-pass filter (not shown) or the like.

If an achromatic subject at the frequency (0, 1 / 4Pv) is sampled by the image pickup devices 83 and 84, R = G ₁ , B for such a subject. = G ₂ holds. Therefore, the difference signals R-G ₁ , B-G ₂ ,
As a result, the color difference signals R-Y and B-Y become zero and are not output.

This means that the color carrier disappears at the position of frequency (0, 1/4 Pv). In other words, the carrier of the R signal and the carrier of the G ₁ signal on the frequency (0, 1 / 4Pv) have the same phase, and the carrier of the B signal and the carrier of the G ₂ signal have the same phase. It is a phase. Therefore, these difference signals R-
Since the carriers of G ₁ and B-G ₂ at this frequency can be extinguished, no color carrier is generated.

The processing by the luminance signal processing circuit 71, the luminance correction signal forming circuit 73 and the adder 74 is the same as that of the above-mentioned fourth embodiment. As a result, the spectral characteristic of the luminance signal is corrected, and a good luminance signal that matches the luminosity can be obtained.

When scanning the image pickup devices 83 and 84 in the non-interlaced mode, the filter arrangement of the image pickup device 84 is such that the R filter group and the B filter group are repeated every line as shown in FIG. 21 (b). The arrangement may be horizontal stripes. In this case as well, G of the image sensor 83
The arrangement of ₁ filter and G ₂ filter is shown in Fig. 21 (a).
As shown in (4), it is determined corresponding to the filter arrangement of the image sensor 84. According to this arrangement, the color carrier is
It occurs at the position as shown in FIG.

In this case, by operating the device in the same manner as in the case of reading the signal by the above-mentioned interlace method, the carrier at the position of the frequency (0,1 / 2Pv) can be eliminated, and a good image can be obtained. Can be obtained. Furthermore, it is possible to obtain an interlaced image by using a memory that stores signals for one frame.

Furthermore, the filters of the image pickup devices 83 and 84 may be arranged as shown in FIGS. 22 (a) and 22 (b). Here, in the image pickup device 84, the R filter and the B filter are arranged in an offset shape as shown in FIG. Corresponding to the arrangement, the G ₁ filter and the G ₂ filter of the image pickup device 83 are also arranged in an offset shape as shown in FIG. According to this arrangement, the color carrier is generated at the position shown in FIG.

In this case as well, by performing the same processing as described above, the color carrier at the position of the frequency (± 1 / 2Ph, 1 / 2Pv) can be eliminated, and a high quality image can be obtained.

In this embodiment, one horizontal signal period (1
Since the difference signal R-G ₁ and the difference signal B-G ₂ are switched for each H), the conversion process shown by the equation (16) in the color difference signal processing circuit 72 and the luminance correction signal forming circuit 73.
In order to obtain the desired color difference signal and luminance correction signal S by the processing of forming the luminance correction signal S represented by the equation (17) in (1), processing such as switching the input signal to each circuit every 1H is required.

Next, the eighth embodiment will be described. FIG. 26 is a block diagram showing the configuration of this embodiment. Figure 2
6, the image pickup devices 87 and 88 are respectively shown in FIG.
The color filters are arranged as shown in (a) and (b). That is, the image sensor 87 is provided with a Y filter for luminance signals on one surface. On the other hand, the image sensor 8
In FIG. 8, R filters and B filters for color signals are arranged in a vertical stripe pattern. Here, the image sensor 8
Among the G filters of _{No. 7} , those at the position equivalent to the R filter group of the image sensor 88 on the image pickup surface are designated as Y _1, and those at the position equivalent to the B filter group are designated as Y ₂ .

In this embodiment, the two image pickup devices 8 are thus provided.
7 and 88, and other parts are the same as in the fifth embodiment shown in FIG. The operation of each part is performed in the same manner as in the fifth embodiment. Therefore, also in this embodiment, the frequency f _H = 1 /
The color carrier at 2Ph can be eliminated, the amount of color moire generated is small, and a color image pickup device with high resolution can be obtained. Further, in the case of the present embodiment, the process of forming the luminance signal can be simplified as compared with the sixth embodiment and the seventh embodiment.

In this embodiment, the image pickup device 88
The case where the color filter shown in FIG. 27B is used as the color filter arranged in FIG. 27 has been described, but the color filter is not limited to this, and the color filters shown in FIG. 20B, FIG. 21B, and FIG.
Any of the color filters shown in (b) may be used. In this case, the Y ₁ signal and the Y ₂
The signal separation processing is shown in FIG.
The same processing as the separation processing of the G ₁ signal and the G ₂ signal in (a) and FIG. 22 (a) may be performed.

In the fourth to eighth embodiments described above, the color moire is small and the resolution is small for both the moving image pickup device such as a video camera and the still image pickup device such as an electronic still camera. It is possible to obtain a high quality image.

[0123]

As described above, according to the present invention, the color filter array arranged in the image pickup device repeats different colors with a period of two pixels in the horizontal direction and two pixels in the vertical direction. Since the array has an offset sampling structure in which different colors are repeated by offsetting only one pixel in the horizontal direction in the cycle, it is possible to reduce the number of color carriers generated in the horizontal direction and the vertical direction. It is possible to reduce the horizontal and vertical color moire caused by. Furthermore, since color moiré that occurs at the resolution limit can be eliminated, it is possible to obtain a high resolution up to the resolution limit.

Further, two optical low-pass filters are provided, which are separated from each other by a distance D satisfying a certain condition in the direction in which the incident light ray makes an angle of θ in the clockwise or counterclockwise direction with respect to the scanning direction of the image sensor. Since it is configured by using an optical member that separates into the light rays of the above, a sufficient MTF characteristic can be obtained in a region below the resolution limit, a high resolution up to the limit resolution can be obtained, and a color carrier generated at a frequency above the limit resolution It is possible to reduce the color moire due to the aliasing distortion. Therefore, when designing an optical low-pass filter, it is not necessary to consider the trade-off between resolution and color moiré, it is possible to design with priority on resolution, and it is possible to capture high-resolution still frame images. it can.

With respect to the output from the image pickup device, the gain is adjusted so that the outputs from the four types of color filters are constant, and the matrix calculation performed when forming the RGB signals from those outputs is performed. Since the sum of the coefficients of any two columns in all rows is made equal to the sum of the remaining two coefficients by using a matrix of 3 rows and 4 columns, the number of color carriers generated in the vertical direction is further reduced. As a result, the color moiré in the vertical direction can be further reduced, and an excellent image can be obtained.

When the color image pickup device is a multi-plate color image pickup device using a plurality of image pickup devices or a single plate type color image pickup device using an image pickup device in which the number of horizontal lines is twice the normal number, Different types of first to third color filters are arranged in each pixel of the image pickup device, and a signal obtained from each pixel in which the second color filter and the third color filter are arranged, and each pixel And two types of difference signals are formed from the signals obtained from the pixels in which the first color filters are arranged at equivalent positions on the image pickup surface, and the color difference signals are formed based on these difference signals. It is possible to reduce the color carrier generated in the horizontal direction and the vertical direction, thereby reducing the color moire and obtaining a high-resolution image. .

Further, the difference signal is used to appropriately correct the signal obtained from the pixel in which the first color filter is arranged to form the luminance signal.
An image with a good / N ratio can be obtained. Further, as the first color filter, a color filter for a luminance signal, which is configured in consideration of the above-mentioned correction amount in advance, is used, and the luminance signal is formed based on the signal obtained by this, so that a high-quality image is obtained. Of course, the luminance signal forming process can be simplified.

Further, since these processes are performed digitally, there are advantages in that the performance is uniform and stable.

[Brief description of drawings]

FIG. 1 is a block diagram showing a first embodiment of a color image pickup apparatus of the present invention.

FIG. 2 is a block diagram showing a second embodiment of the color image pickup apparatus of the present invention.

FIG. 3 is a diagram showing a filter arrangement of an image pickup element used in the first and second embodiments.

FIG. 4 is a diagram showing a filter arrangement of an image sensor in the first and second embodiments.

FIG. 5 is a frequency characteristic diagram of image pickup elements used in the first and second embodiments.

FIG. 6 is a frequency characteristic diagram of image pickup devices used in the first and second embodiments.

FIG. 7 is a frequency characteristic diagram of the image sensor used in the first and second embodiments.

FIG. 8 is a configuration diagram of an optical low-pass filter of the present invention.

FIG. 9 is a diagram showing an optical path by the optical low pass filter of the present invention.

FIG. 10: MTF of the optical low pass filter of the present invention
It is a characteristic diagram.

FIG. 11 is a frequency characteristic diagram of the optical low-pass filter of the present invention.

FIG. 12 is a block diagram showing a third embodiment of the color image pickup apparatus of the present invention.

FIG. 13 is a block diagram showing a fourth embodiment of the color image pickup apparatus of the present invention.

FIG. 14 is a diagram showing a filter arrangement of an image sensor used in a fourth embodiment.

FIG. 15 is a block diagram showing a fifth embodiment of the color image pickup apparatus of the present invention.

FIG. 16 is a diagram showing a filter arrangement of an image sensor used in a fifth embodiment.

FIG. 17 is a block diagram showing a sixth embodiment of the color image pickup apparatus of the present invention.

FIG. 18 is a diagram showing a filter arrangement of an image pickup element used in a sixth embodiment.

FIG. 19 is a block diagram showing a seventh embodiment of the color image pickup apparatus of the present invention.

FIG. 20 is a diagram showing a filter arrangement of an image sensor used in a seventh embodiment.

FIG. 21 is a diagram showing a filter arrangement of an image sensor used in a seventh embodiment.

FIG. 22 is a diagram showing a filter arrangement of an image sensor used in a seventh embodiment.

FIG. 23 is a frequency characteristic diagram of the image sensor used in the seventh embodiment.

FIG. 24 is a frequency characteristic diagram of the image sensor used in the seventh embodiment.

FIG. 25 is a frequency characteristic diagram of the image sensor used in the seventh embodiment.

FIG. 26 is a block diagram showing an eighth embodiment of the color image pickup apparatus of the present invention.

FIG. 27 is a diagram showing the filter arrangement of the image sensor used in the eighth embodiment.

FIG. 28 is a diagram showing a filter arrangement of an image pickup element used in a conventional color image pickup apparatus.

FIG. 29 is a diagram showing a filter arrangement of an image pickup element used in a conventional color image pickup apparatus.

FIG. 30 is a diagram showing a sampling structure when pixel mixture reading is performed on the image sensor of FIG. 29;

FIG. 31 is a frequency characteristic diagram based on the filter arrangement of FIG. 28.

32 is a frequency characteristic diagram based on the filter arrangement of FIG. 30.

FIG. 33 is a frequency characteristic diagram based on the filter arrangements of FIGS. 28 and 30.

FIG. 34 is a characteristic diagram showing a pass band of a conventional optical low pass filter.

FIG. 35 is a configuration diagram of a conventional optical low-pass filter.

FIG. 36 is an MTF characteristic diagram of a conventional optical low pass filter.

FIG. 37 is a frequency characteristic diagram of a conventional optical low-pass filter.

FIG. 38 is a block diagram showing a schematic configuration of a conventional two-plate type color imaging device.

FIG. 39 is a diagram showing a filter arrangement of an image pickup element used in a conventional two-plate type color image pickup apparatus.

FIG. 40 is a diagram showing a filter arrangement of image pickup elements used in a single-plate type color image pickup device for obtaining image quality equivalent to that of a conventional two-plate type color image pickup device.

41 is a frequency characteristic diagram based on the filter arrangements of FIGS. 39 and 40. FIG.

[Explanation of symbols]

1 Image sensor 3 A / D converter 4 Buffer memory 10, 10a RGB converter 20 Gain adjustment circuit 50 Optical low pass filter 51, 52, 53 Birefringent plate 54a, 54b, 55a, 55b, 56a, 56b Optical low pass filter Trajectories of frequencies trapped by 61, 62, 81, 82, 83, 84, 87, 88 Image sensor 63, 64, 75 Color separation unit 69, 70 Adder 71 Luminance signal processing circuit 72, 76 Color difference signal processing circuit 73 Luminance Correction signal forming circuit 74 Adder

Claims (7)

[Claims] 1. A color filter having four types of color filters, wherein color filters of different colors are repeated every two pixel periods in the horizontal direction and every two pixel periods in the vertical direction.
And an image sensor with an offset sampling structure in which the color filters of different colors are repeated by horizontally offsetting one pixel, and the incident light beam is rotated clockwise or counterclockwise with respect to the scanning direction of the image sensor. When an optical member for separating two light beams separated by a distance D in a direction forming an angle of θ is provided and the horizontal pitch of the image pickup element is Ph and the vertical pitch thereof is Pv, the distance D Is 1.6PhPv / | Phsinθ + 2Pvcosθ | ≦ D ≦ 2.4PhPv / | Phsinθ + 2Pvcosθ | (0 ≦ θ ≦ π / 2) An image is provided by including a memory for storing once, and reading out necessary information from the memory to perform digital signal processing. The color image pickup apparatus characterized by being obtained manner. 2. Gain adjusting means for adjusting the output from the image pickup device so that the outputs from the four types of color filters are constant for an achromatic subject, and in all rows, Signal forming means for forming RGB signals from the outputs of the four types of color filters by performing a matrix operation of 3 rows and 4 columns in which the sum of the coefficients of any two columns is equal to the sum of the remaining two coefficients. The color image pickup apparatus according to claim 1, further comprising: 3. A color filter having four kinds of color filters, wherein color filters of different colors are repeated in a horizontal direction at a 2-pixel cycle, and in a vertical direction at a 2-pixel cycle.
And an image sensor having an offset sampling structure in which color filters of different colors are repeated by offsetting one pixel in the horizontal direction, and the sum of the coefficients of any two columns in all rows is the remaining two coefficients. By performing a matrix operation of 3 rows and 4 columns equal to the sum of
A color image pickup apparatus comprising: a signal forming unit that forms RGB signals from outputs of the four types of color filters. 4. A color image pickup apparatus comprising an image pickup device having a plurality of pixels arranged two-dimensionally in a horizontal direction and a vertical direction, wherein the image pickup device has a number of horizontal lines required to obtain a desired image. 2 times the horizontal line, and the horizontal line is the first horizontal line on which the first color filter is arranged, the second color filter and the third horizontal line.
Second horizontal line in which every other pixel is arranged alternately, the signal output from the pixel in which the second color filter is arranged and the first color in the same column as the signal output from the pixel in which the second color filter is arranged. The first difference signal is formed by taking the difference from the signal output from the pixel in which the filter is arranged, and is in the same column as the signal output from the pixel in which the third color filter is arranged. A second difference signal is formed by taking a difference from the signal output from the pixel on which the first color filter is arranged, and a color difference signal is generated based on the first difference signal and the second difference signal. A color imaging device characterized by being formed. 5. A first color filter provided with a first color filter.
Image pickup device and a second image pickup device provided with a second color filter and a third color filter, and a desired image is obtained using the first image pickup device and the second image pickup device. In the color image pickup device described above, a signal output from a pixel in the second image pickup element in which the second color filter is arranged and a signal output from a pixel in the first image pickup element at a position equivalent to the signal on the image pickup surface. The first difference signal is formed by taking the difference from the signal output from the pixel in which the first color filter is arranged, and from the pixel in which the third color filter in the second image sensor is arranged. The second difference signal is obtained by taking the difference between the output signal and the signal output from the pixel in which the first color filter in the first image sensor is arranged at a position equivalent to the output signal. To form Serial color imaging device being characterized in that so as to form a color difference signal based on the first difference signal and the second difference signal. 6. A luminance signal is formed by synthesizing a signal output from a pixel provided with the first color filter with the first difference signal and the second difference signal. The color image pickup apparatus according to claim 4, wherein the color image pickup apparatus is a color image pickup apparatus. 7. The luminance signal is formed on the basis of a signal output from a pixel in which the first color filter provided for the luminance signal is arranged. Color imaging device.