JPH04278792A - Color image pickup device - Google Patents

Abstract

PURPOSE:To obtain the excellent image by forming color information from a difference signal R-G1 obtained by synchronizing only a signal G1 by a picture element of the same line as a chrominance signal R in a chrominance signal G, and a difference signal R-G2 obtained by synchronizing only a signal G2 by a picture element of the same line as a chrominance signal B in the signal G. CONSTITUTION:In an image pickup optical system, an optical type LPF 1 for separating an incident beam into two pieces of beams separated by a distance D in the direction if an angle of theta against the scanning direction is provided. Also, an image pickup element 101 is provided with a color filter array of a Bayer array. In such a state, based on a first, a second and a third chrominance signals G, R and B outputted from picture elements corresponding to a first, a second and a third color filters, respectively, color information is formed a difference signal R-G1 obtained by synchronizing only a signal G1 by the picture element of the same line as the picture element of the signal R in the signal G, and taking a difference to the signal R, and a difference signal R-G2 obtained by synchronizing only a signal G2 by the picture element of the same line as the picture element of the signal G in the signal G, and taking a difference to the signal B.

Description

[Detailed description of the invention]

0001

[Industrial Field of Application] The present invention relates to a color imaging device equipped with an imaging device in which a plurality of light receiving elements (pixels) are two-dimensionally arranged.
The present invention relates to a color imaging device that can output images with a good N ratio.

0002

2. Description of the Related Art FIGS. 19, 20, and 21 are diagrams showing examples of arrangement configurations of color filters of conventionally known color solid-state imaging devices. FIG. 19 shows a configuration called a so-called stripe filter in which a red light transmitting filter R, a green light transmitting filter G, and a blue light transmitting filter B are vertically arranged in a stripe shape. On the other hand, FIGS. 20 and 21 have a configuration called a so-called mosaic filter, and in FIG. 20, the green light transmitting filter G is vertically striped.
Red light transmission filters R and blue light transmission filters B are arranged horizontally between the G filters in every two rows and two columns, and in FIG. 21, magenta light transmission filters Mg,
Green light transmission filter Gr, cyan light transmission filter Cy,
The yellow light transmitting filter Ye has eight color filters each consisting of two pixels in the horizontal direction and four pixels in the vertical direction, which are arranged in the order shown in the figure.

[0003] However, these image pickup devices having color filter arrays have the following problems. In other words, in an image sensor equipped with a color filter having the configuration shown in FIG. 19, a color signal carrier is generated at a frequency that is 1/3 of the sampling frequency, so it is possible to resolve up to a frequency that is 1/2 of the sampling frequency, which is the Nyquist frequency. The resolution is inferior.

[0004] In an image sensor equipped with a color filter having the structure shown in FIG. 20, since the R filter and B filter with different bands are arranged vertically, color moire tends to occur in the vertical direction, especially in chromatic images. An ugly scene appears.

An image sensor equipped with a color filter having the configuration shown in FIG. 21 is composed of complementary color filters with a wide band, so color moire is less likely to occur than an image sensor equipped with a color filter having the configuration shown in FIG. Since the color signal is formed from the difference signal between the output signals of the pixels, the S/N ratio of the color signal is poor, and furthermore, when the output signal is quantized and digitally processed,
This is undesirable because the quantization error of the color signal increases.

Furthermore, in an image sensor equipped with a color filter having the configuration shown in FIGS. 20 and 21, the sampling frequency is 1.
Since a color signal carrier is generated at a frequency of /2, it is not possible to resolve frequencies up to 1/2 of the sampling frequency, which is the Nyquist frequency.

In contrast, US Patent No. 39710
There is an image sensor having a color filter array called a Bayer array, which is disclosed in Japanese Patent No. 65. As shown in FIGS. 22(a) and 22(b), if the horizontal pitch of the image sensor is PH and the vertical pitch is PV, then the green light transmission filter G (FIG. 22(a)) or the luminance signal The transmission filter Y (Figure (b)) has a pitch of 2PH in the horizontal direction and a pitch of PV in the vertical direction.
The red light transmitting filter R and the blue light transmitting filter B are arranged in an offset sampling structure offset by 2 PH in the horizontal direction and 2 PH in the vertical direction.
The PV is arranged in a rectangular grid sampling structure. It is known that when an image sensor having such a Bayer array is used, a good image with less moire and a good S/N ratio can be obtained.

[0008]

However, even if an image sensor having the Bayer array is used, the following problems occur. That is, FIGS. 23(a) and 23(b) show the positions of signal carriers generated in the image sensor of the color filter shown in FIGS. 22(a) and 22(b), respectively, on a two-dimensional frequency plane (fH).
, fV ) is a characteristic diagram of the first quadrant when expressed above.

In the color filter image sensor shown in FIG. 22(a), a luminance signal is formed by directly switching the output signal from each pixel, and the color filter shown in FIG. 22(b) is In the image sensor of the filter, Y
A luminance signal is generated using only signals from pixels in which filters are arranged.

In either case, (1/2P
It can be seen that color signal carriers are generated at (H,0) and (0,1/2PV). In other words, even in the case of an image sensor having a Bayer array, the sampling frequency is 1
Since a color signal carrier is generated at a frequency of /2, it is not possible to resolve frequencies up to 1/2 of the sampling frequency, which is the Nyquist frequency.

The present invention has been made in view of these problems, and an object of the present invention is to provide a color imaging device that can obtain images with good resolution, less moire, and a good S/N ratio. It is something.

0012

Means for Solving the Problems In the present invention, in order to achieve the above object, a color imaging device is configured as shown in (1) and (2) below. (1) A color imaging device that converts a subject image into an electrical signal having brightness information and color information, which has the following a, b, c,
A color imaging device comprising the components of d.

a. An image sensor in which pixels are arranged in a rectangular grid with a pitch PH in the horizontal direction and a pitch PV in the vertical direction.

b. a first color filter provided corresponding to the pixel and having an offset sampling structure offset by PH in the horizontal direction with a pitch of 2PH in the horizontal direction and a pitch in the vertical direction; A color filter array comprising a second color filter and a third color filter having a rectangular grid sampling structure with a pitch of 2 PV.

c. The imaging optical system includes an optical member that separates an incident light beam into two light beams separated by a distance D in a direction making an angle θ clockwise or counterclockwise with respect to the scanning direction of the image pickup device. An optical low-pass filter, 0.8PH PV / | PH sin θ + PV cos θ |
≦D≦1.2PH PV /｜PHsinθ+PVco
An optical low-pass filter that satisfies the condition sθ | where 0≦θ≦π/2.

d. Based on the first color signal, second color signal, and third color signal output from the pixels corresponding to the first color filter, second color filter, and third color filter, the first color signal is Of the color signals, only the signals from pixels in the same column or row as the pixels of the second color signal are synchronized, and a signal obtained by taking the difference between the second color signal and the third color signal of the first color signal is synthesized. A color information forming means that synchronizes only signals from pixels in the same column or row as a pixel of a color signal, and forms the color information from a signal obtained by taking a difference from a third color signal.

(2) A color imaging device that converts a subject image into an electrical signal having brightness information and color information, and includes the following components a, b, c, and d.

a. An image sensor in which pixels are arranged in a rectangular grid with a pitch PH in the horizontal direction and a pitch PV in the vertical direction.

b. a first color filter provided corresponding to the pixel and having an offset sampling structure offset by PH in the horizontal direction with a pitch of 2PH in the horizontal direction and a pitch in the vertical direction; A color filter array comprising a second color filter and a third color filter having a rectangular grid sampling structure with a pitch of 2 PV.

c. The imaging optical system includes an optical member that separates an incident light beam into two light beams separated by a distance D in a direction making an angle θ clockwise or counterclockwise with respect to the scanning direction of the image pickup device. An optical low-pass filter, 0.8PH PV / | PH sin θ + PV cos θ |
≦D≦1.2PH PV /｜PHsinθ+PVco
An optical low-pass filter that satisfies the condition sθ | where 0≦θ≦π/2.

d. Based on the first color signal, second color signal, and third color signal output from the pixels corresponding to the first color filter, second color filter, and third color filter, the first color signal is Of the color signals, only the signals from pixels in the same column as the pixels of the second color signal are synchronized, and a signal obtained by taking the difference between the second color signal and the third color signal of the first color signal is generated. The first color information is formed by synchronizing only the signals from the pixels in the same column as the pixel, and from the signal obtained by taking the difference from the third color signal.
and synchronizing only the signals from the pixels in the same row as the pixels of the second color signal among the first color signals,
The signal obtained by taking the difference from the second color signal and the signal from the pixels of the first color signal in the same row as the pixels of the third color signal are synchronized, and the difference from the third color signal is taken. the first color information forming means and the first color information forming means according to the frequency component of the subject image in the scanning direction or in a direction orthogonal thereto; color information forming means for switching between the second color information forming means;

[0022]

[Operation] With the configuration (1) above, the carrier of the color difference signal for a black and white object located at (1/2PH, 0) or (0,1/2PV) on the two-dimensional frequency space disappears, and (±1 /2PH, ±1/2PV) carriers of color difference signals are suppressed. With the configuration in (2) above, 2
(1/2PH ,0) and (0,
The carrier of the color difference signal for the black and white subject at 1/2 PV) disappears, and
) is suppressed.

[0023]

[Examples] The present invention will be explained in detail below using examples. FIG. 1 is a block diagram of a "color imaging device" which is a first embodiment of the present invention. The image sensor (sensor) 101 is shown in FIG.
2(a) is provided with R, G, and B filters (filter array) in a Bayer array. The image signal read out pixel by pixel from the image sensor 101 is separated into R, G, and B signals by the color separation unit 102, and then the gains of the R, G, and B signals are adjusted by the white balance unit 111 to the white balance sensor ( The white balance is adjusted based on the color temperature information obtained from the AWB) 120, and then the γ conversion unit 112
The signal is then converted into an A/D signal by an A/D (analog-digital) converter 103.

The brightness signal is transmitted through the switch circuit (SWY) 126
The signals are switched in order to be read out, and are extracted as a high frequency component YH of the luminance signal by a bandpass filter (BPF) 116. The high frequency component YH of this luminance signal is added to the low frequency component YL of the luminance signal obtained by a method described later in an adder 117, and is D/A converted by a D/A (digital-analog) converter 118 and output. Ru.

On the other hand, among the outputs of the A/D converter 103, G
The G1 (γ power) signal and the G2 (γ power) signal are placed in the positions shown in FIG. 2 by the switch (SW) 128.
signal). This is possible by switching the switch 128, for example, every horizontal scanning period. The thus separated G1 (γ power) signal and G2 (γ power) signal are input to interpolation filters 106, 107, 108, 109 together with the R (γ power) signal and B (γ power) signal.
Simultaneous signals R (γ power), G1 (γ power), G
2 (γ power) and B (γ power). Note that interpolation filter 1
In 06 to 109, not only synchronization by interpolation but also linear processing such as two-dimensional low-pass filtering and edge enhancement is performed. Since these processes are linear processes, the order may be changed with processes such as addition processing and matrix processing, which will be described later.

The synchronized R (γ power) signal, G1 (γ
The adder 129 outputs the R (γ power) − G1 (γ power) signal.
The B (γ power) signal and the G2 (γ power) signal become B (γ power) − G2 (γ power) signal in the adder 130.
These are input to the color difference signal matrix processing section 113,

[0027]

[Math 1]

The following conversion is performed, and the color difference signals R-Y,B
-Y is generated.

Here, on the frequency space (1/2PH, 0
) is captured by the image sensor 101. This object has vertical stripes with a period of 2PH, and for such an object, R (γ power) = G1 (γ power)
, B (γ power) = G2 (γ power), so the adder 12
R (γ power) − G1 (γ power) output from 9,130
The signal B (to the power of γ) - the signal B (to the power of γ) are both zero. Therefore, the color difference signals R-Y and B-Y output from the color difference matrix processing section 113 also become zero and are not output. This means that the carrier of the color difference signal at the frequency (1/2PH, 0) disappears. To interpret it another way, R (γ power) on frequency (1/2PH, 0)
The carrier of the signal and the carrier of the G1 (γ power) signal are in the same phase, and the carrier of the B (γ power) signal and the G2 (γ power) signal carrier are in phase.
The signal carriers are in the same phase, so these difference signals R (γ power) - G1 (γ power), B (γ power) - G2
Since carriers at this frequency of (γ power) can be eliminated, carriers of color difference signals are not generated. For the same reason, (-1/2PH
, 0), no color difference signal carrier is generated. These color difference signals are converted into D/A converters 114 and 115.
A is converted and output. Furthermore, interpolation filters 106 to
The low-frequency component of the luminance signal from the output of
(γ power) + βG2 (γ power)] +0.11B (γ power)...
(2) However, it is generated by α+β=0.59, and as mentioned above, the high-frequency component of luminance YH
are added by the adder 117, and the D/A converter 118 adds the
A is converted and output. Note that generally the color difference signals RY,
B-Y, the low frequency component YL of luminance has a sufficiently narrow band compared to the luminance signal Y, so the interpolated and synchronized R (γ power), G1
(γ power), G2 (γ power), B (γ power) signal adders 129, 130, color difference matrix processing unit 113, luminance signal generation circuit 117, etc. process by thinning out the luminance signal Y. It may be performed using a clock slower than the processing.

Next, the optical low-pass filter shown in FIG. 1 will be explained. FIG. 3 shows the configuration of the optical low-pass filter 1 in this embodiment. In the figure, an optical low-pass filter 300 includes an optical member 301 that splits an incident light beam into two light beams separated by a distance D1 in an angle direction of θ1 counterclockwise with respect to the scanning direction, and an optical member 301 that splits an incident light beam into two light beams separated by a distance D1. Distance D in the direction of θ2 clockwise with respect to the direction
It is composed of an optical member 302 that splits the beam into two beams separated by 2. The optical member 301 is composed of a birefringent plate 303 whose projection of the optical axis onto a plane parallel to the image plane forms an angle of θ1 counterclockwise in the scanning direction.
It is composed of a quarter-wave plate 304 that converts linearly polarized light into circularly polarized light, and a birefringent plate 305 whose projection onto a plane parallel to the image plane of the optical axis forms an angle of θ2 clockwise in the scanning direction. All of them satisfy the following conditions. 0.8PHPV/｜PHsinθ1 +PVcos
θ1｜≦D1≦ 1.2PHPV/｜PHsinθ1
+PVcos θ1｜

...(2)
0≦θ1 ≦π/2 0.8PHPV/｜PHsinθ2 +PVcos
θ2｜≦D2≦ 1.2PHPV/｜PHsinθ2
+PVcos θ2｜

...(3)
0≦θ2 ≦π/2 When the values of D1 and D2 exceed the lower limit of inequalities (2) and (3), aliasing distortion increases, and when the upper limit is exceeded, the resolution decreases, and in either case, it is generally preferable. do not have. In addition, if the application is limited and only frequency components having a specific direction are a problem, the optical members 301 and 302
Either one of them may be used, and in this case, the quarter-wave plate 304 constituting the optical member 302 is not necessarily necessary, and the birefringent plate 305 alone may be used. FIG. 4 shows the spatial frequency characteristics of this optical low-pass filter 300. In the same figure,

[0031]

[Math 2]

The case of [0032] is shown below. At this time, as shown by the dotted line, on the spatial frequency plane (fH, fV), (±1/2
All the color difference signal carriers at (PH, ±1/2 PV) are trapped, and the color difference signal carriers at (0, ±1/2 PV) are also sufficiently suppressed, making it possible to obtain a good image with little aliasing distortion. can. Also, θ1 =
When θ2 = π/4, the optical low-pass filter 1 is
It may be configured as 500 in FIG. That is, the birefringent plate 503 has its optical axis projected onto a plane parallel to the image plane at an angle of π/4 counterclockwise with respect to the scanning direction, and the birefringent plate 503 has a projection of its optical axis onto a plane parallel to the image plane at an angle of π/4 clockwise with respect to the scanning direction. An optical member 501 composed of a birefringent plate 504 having its optical axis projected onto a plane parallel to the image plane, and an optical member 501 comprising a birefringent plate 504 whose optical axis is projected onto a plane parallel to the image plane in the same direction as the scanning direction. Birefringent plate 505 with
An optical low-pass filter 500 is constituted by an optical member 502 composed of the following. Birefringent plates 503, 504, 5
The separation distance of 05 is respectively

[0033]

[Math 3]

[0034] With this configuration, due to the polarization effect of the birefringent plate, the first optical member 501 separates the incident light beam into two light beams separated by a distance D1, and the direction thereof is π/clockwise with respect to the scanning direction. In the end, the optical low-pass filter shown in Fig. 3 becomes θ1 = θ2.
It has the same spatial frequency characteristics as when = π/4.

Furthermore, the optical low-pass filter 1 may have the configuration shown in FIG. That is, there is an optical member 601 made of a birefringent plate that separates light beams in a direction of π/4 counterclockwise with respect to the scanning direction, and an optical member 601 made of a birefringent plate that separates light rays in a direction of π/4 counterclockwise with respect to the scanning direction.
The optical member 602 is composed of a birefringent plate that separates light rays in the /2 direction, and the optical member 603 is made of a birefringence plate that separates light rays in the π/4 direction clockwise with respect to the scanning direction. The beam separation widths of the optical members 601, 602, and 603 are respectively D1, D2, and D3,

[0036]

[Math 4]

[0037] Then, the optical low-pass filter 600
The spatial frequency characteristic of is shown in FIG. That is, on the spatial frequency plane (fH, fV), (±1/2PH
, ±1/2 PV ) and (0, ±1/2 PV ) are all trapped, making it possible to satisfactorily suppress aliasing distortion.

As explained above, in this embodiment, since the Bayer array color filter is used, there is little moire and the S/N ratio is good. The resolution is good because it uses signal processing means.

Next, a second embodiment of the present invention will be explained. The image sensor 101 is provided with the color filter shown in FIG. 22(a) as in the first embodiment, and the overall configuration is also the same as that shown in FIG. However, the G (γ power) signal output from the A/D converter 103 is separated by the switch 128 into a G1 (γ power) signal and a G2 (γ power) signal located at the positions shown in FIG. Here, on the frequency space (0, 1/2
Assume that a black and white subject at PV) is captured by the image sensor 101. This object has horizontal stripes with a period of 2 PV, and for such an object, adders 129 and 13
R (γ power) − G1 (γ power) signal output from 0, B
(γ power)−G2 (γ power) signal both become zero. Therefore, the color difference signals R-Y and B-Y output from the color difference matrix processing section 113 also become zero and are not output. This means that
This means that the carrier of the color difference signal at the frequency (0,1/2 PV) disappears. To interpret it another way, the carrier of the R (γ power) signal and the carrier of the G1 (γ power) signal on the frequency (0, 1/2 PV) are in phase, and the carrier of the B
The carrier of the (γ power) signal and the carrier of the G2 (γ power) signal are in the same phase, so their difference signals R (γ power) - G1 (γ power), B (γ power) - G2 (γ power) Because the carrier at this frequency can be annihilated,
No carrier of the color difference signal is generated. For the same reason, no color difference signal carrier is generated at (0, -1/2 PV) which is symmetrical about the fH axis. Note that when recording the output signal obtained from the processing block diagram shown in FIG. 1 in analog form,
Although D/A converters 118, 114, and 115 are necessary,
Some kind of magnetic medium, magneto-optical medium, E2 PROM (el
electrically erasable PROM)
It is not necessary to include it when digitally recording on a computer or other media. FIG. 9 shows the spatial frequency characteristics when the optical low-pass filter shown in FIG. 3 or FIG. 5 described above is used in this imaging device. At this time, on the spatial frequency plane (fH, fV) (
All carriers of the color difference signal at (±1/2PH, ±1/2PV) are trapped, and also (±1/2PH
, 0) is also sufficiently suppressed, and a good image with little aliasing distortion can be obtained.

Furthermore, the optical low-pass filter may have the configuration shown in FIG. That is, an optical member 151 made of a birefringent plate that separates light rays in a direction of π/4 counterclockwise with respect to the scanning direction, an optical member 152 made of a birefringence plate that separates light rays parallel to the scanning direction, The optical member 153 is composed of a birefringent plate that separates light rays in the direction of π/4 clockwise with respect to the direction. optical member 151,
The beam separation widths of 152 and 153 are D1 and D2, respectively.
,D3,

[0041]

[Math 5]

[0042] Then, the optical low-pass filter 150
The spatial frequency characteristic of is shown in FIG. In other words, on the spatial frequency plane (fH, fV) (±1/2PH
, ±1/2 PV ) and (±1/2 PH , 0) are all trapped, and aliasing distortion can be suppressed well.

Next, a third embodiment of the present invention will be explained. FIG. 12 is a block diagram of the "color imaging device" of this embodiment. Light from the subject is passed through an optical low-pass filter 1 by an imaging optical system (not shown) to an image sensor (sensor) 1.
01. The optical low-pass filter 1 has the configuration shown in FIG. 3, FIG. 5, or FIG. 6, and suppresses aliasing distortion well as described above. The image sensor (sensor) 101 is provided with a Bayer array YRB filter shown in FIG. 22(b). The image signal read out pixel by pixel from the image sensor 101 is separated into Y, R, B by the color separation unit 102.
After being separated into signals, the white balance unit 111
The gains of the R and B signals are white balanced adjusted based on the color temperature information obtained from the white balance sensor 120, then γ converted by the γ converter 112, and then A/D converted by the A/D converter 103. be done. The luminance signal is Y(γ)
After the signal is two-dimensionally interpolated with an offset sampling structure by the interpolation filter 205, it is D/A converted by the D/A converter 118 and output. Note that the interpolation filter 205 performs not only synchronization by interpolation, but also processes such as two-dimensional low-pass filtering and edge emphasis. On the other hand, Y (γ power) of the output of the A/D converter 103 is
) 128, Y1 (
Y2 (γ power) signal and Y2 (γ power) signal. This is possible by switching the switch 128, for example, every horizontal scanning period. Y1 separated in this way (γ power)
The signal, Y2 (γ power) signal is the R (γ power) signal, B (γ power) signal,
interpolation filters 206, 207, 208 together with the
, 209 and are each synchronized signals R (γ power)
, Y1 (γ power), Y2 (γ power), and B (γ power). Note that the interpolation filters 206 to 209 perform not only synchronization by interpolation, but also linear processing such as two-dimensional low-pass filtering and edge enhancement. Since these processes are linear processes, the order may be switched with processes such as addition processing, which will be described later. The synchronized R (γ power) and Y1 (γ power) signals become R-Y signals in the adder 129, and B (γ power)
The signal and the Y2 (γ-th power) signal are converted into a BY signal by an adder 130, and color difference signals RY and BY are generated.

Here, on the frequency space (1/2PH, 0)
Assume that a black and white subject at 2 is captured by the image sensor 101. This object has vertical stripes with a period of 2PH, and for such an object, the RY signal and BY signal output from the adders 129 and 130 are both zero. Therefore, the color difference signals R-Y and B-Y become zero and are not output. This means that the carrier of the color difference signal at the frequency (1/2PH, 0) disappears. To interpret it another way, R (γ power) on frequency (1/2PH, 0)
The carrier of the signal and the carrier of the Y1 (γ power) signal are in the same phase, and the carrier of the B (γ power) signal and the carrier of the Y2 (γ power) signal are in phase.
The carriers of the signals are in the same phase, and therefore the carriers of these difference signals R-Y, B-Y at this frequency can be eliminated, so that no carrier of the color difference signal is generated. These color difference signals are D/A converted by subsequent D/A converters 114 and 115 and output. For the same reason, no color difference signal carrier is generated at (-1/2PH, 0) which is symmetrical about the fV axis. Note that generally the color difference signal R
-Y, B-Y have sufficiently narrow bands compared to the luminance signal Y, so they are interpolated and synchronized R (γ power), Y1 (γ power), Y2
(γ power) and B (γ power) signals in the adders 129, 130, etc. may be performed using a slower clock than the processing of the luminance signal Y by thinning out or the like. In this embodiment, unlike the first and second embodiments, the spatial frequency plane (fH
, fV), carriers of luminance signals instead of color difference signals are generated at positions (±1/2PH, ±1/2PV), but optical low-pass filters 300, 500 and 600 has frequency characteristics as shown by the dotted lines in FIGS. 4 and 7, traps these points, and suppresses aliasing distortion well.

Next, a fourth embodiment of the present invention will be described. The image sensor 101 is provided with the color filter shown in FIG. 22(b) as in the third embodiment, and the overall configuration is also the same as that shown in FIG. 12. However, the Y (γ power) signal output from the A/D converter 103 is switched between Y1 (γ power) and Y2 (
γ power) signals.

Here, on the frequency space (0,1/2PV)
Assume that a black and white subject at 2 is captured by the image sensor 101. This object has horizontal stripes with a period of 2 PV, and for such an object, the RY signal and BY signal output from the adders 129 and 130 are both zero. Therefore, the color difference signals R-Y and B-Y become zero and are not output. This means that the carrier of the color difference signal at the frequency (0,1/2 PV) disappears. To interpret it another way, R (γ power) on the frequency (0, 1/2 PV)
The carrier of the signal and the carrier of the Y1 (γ power) signal are in the same phase, and the carrier of the B (γ power) signal and the carrier of the Y2 (γ power) signal are in phase.
The carriers of the signals are in the same phase, and therefore the carriers of these difference signals R-Y, B-Y at this frequency can be eliminated, so that no carrier of the color difference signal is generated. Note that when analog recording the output signal obtained from the processing block diagram shown in FIG. 12, D/A converters 118, 114, and 115 are necessary, but digital recording on some magnetic medium, magneto-optical medium, E2 PROM, etc. You don't have to include it if you do. As the optical low-pass filter 1, one shown in FIG. 3, FIG. 5, or FIG. 10 is used, and the spatial frequency characteristics suppress aliasing distortion well as shown in FIGS. 9 and 11.

Furthermore, the color difference signal formation may be performed as follows. That is, as shown in FIG. 15, the separation of the G (γ power) signal by the switch 128 may be switched between the timing shown in FIG. 2 and the timing shown in FIG. 8, depending on the luminance signal of the subject. This example will be described as a fifth embodiment. The determination circuit 131, which will be described later, determines whether the timing shown in FIG. 2 or the timing shown in FIG. 8 is selected. Similarly, as shown in FIG. 16, the Y (γ power) signal switch 20
The separation according to No. 3 may be switched between the timing shown in FIG. 13 and the timing shown in FIG. 14 depending on the luminance signal of the subject. This will be explained as a sixth embodiment. Which timing to select is determined by a determination circuit 204, which will be described later. Next, the configuration and operation of the determination circuits 131 (see FIG. 15) and 204 (see FIG. 16) will be explained. FIG. 17 is a diagram showing an example of the configuration of this determination circuit. Here, the luminance signal output from the switch circuit 126 (see FIG. 15) or the interpolation filter 205 (see FIG. 16) is applied with a horizontal band pass filter 31 to extract high frequency components in the horizontal direction. This is input to the comparison circuit 32 and compared with a certain threshold level (ref). If it is determined that the high-frequency component in the horizontal direction is larger than the threshold level (ref), the timing shown in FIG. 2 for extinguishing the carrier of the horizontal color difference signal is selected; The timing shown in FIG. 8 for extinguishing carriers is selected.

Further, the determination circuits 131 and 204 are as shown in FIG.
The configuration shown in 8 may also be used. That is, the luminance signal output from the switch circuit 126 or the interpolation filter 205 is applied to the vertical band pass filter 61 to extract the high frequency component in the vertical direction. This is input to a comparison circuit 62 and compared with a certain threshold level (ref). If it is determined that the high frequency component in the vertical direction is larger than the threshold level (ref), the timing shown in FIG. 8 for extinguishing the carrier of the vertical color difference signal is selected; The timing of FIG. 2 is selected to cause the carrier of the signal to disappear. In this manner, in the fifth and sixth embodiments, it is possible to change carriers to be annihilated depending on the frequency components of the subject in the horizontal and vertical directions, so that it is possible to obtain good images with less aliasing distortion. Note that if the optical low-pass filter 1 has the configurations shown in FIGS. 3, 5, 6, and 10, aliasing distortion can be suppressed well.

[0049]

[Effects of the Invention] As explained above, according to the present invention,
By providing a Bayer array filter array in the image sensor and applying appropriate signal processing, it is possible to provide a color imaging device that can obtain images with good resolution, less moire, and a good S/N ratio.

[Brief explanation of the drawing]

[Figure 1] Block diagram of the first embodiment

[Fig. 2] Explanatory diagram of the first embodiment

[Figure 3] Configuration diagram of the optical low-pass filter used in the first embodiment

[Fig. 4] Explanatory diagram of the optical low-pass filter shown in Fig. 3

[
Figure 5: Configuration diagram of an optical low-pass filter used in a modification of the first embodiment

[Fig. 6] Configuration diagram of an optical low-pass filter used in a modification of the first embodiment.

FIG. 7 is an explanatory diagram of the optical low-pass filter shown in FIG. 6.

[
Figure 8: Explanatory diagram of the second embodiment

[Fig. 9] Explanatory diagram of the optical low-pass filter shown in Figs. 3 and 5

FIG. 10 is a configuration diagram of an optical low-pass filter used in a modification of the second embodiment.

FIG. 11 is an explanatory diagram of the optical low-pass filter shown in FIG. 10.

[Figure 12] Block diagram of the third embodiment

[Fig. 13] Explanatory diagram of the third embodiment

[Fig. 14] Explanatory diagram of the fourth embodiment

[Fig. 15] Block diagram of the fifth embodiment

[Fig. 16] Block diagram of the sixth embodiment

FIG. 17 is a diagram showing an example of the configuration of a determination circuit.

FIG. 18 is a diagram showing an example of the configuration of a determination circuit.

[Figure 19] Diagram showing an example of arrangement of color filters

[Figure 20] Diagram showing an example of arrangement of color filters

[Figure 21] Diagram showing an example of arrangement of color filters

FIG. 22 is a diagram showing an example of a Bayer array color filter arrangement.

FIG. 23 is a diagram showing the positions of signal carriers according to the color filter in FIG. 22;

[Explanation of symbols]

1. Optical low-pass filter 101 Image sensor 106-109 Interpolation filter 128 Switch 129,130 Adder

Claims (2)

[Claims] Claim 1: A color imaging device that converts a subject image into an electrical signal having brightness information and color information, comprising:
, b, c, and d. a. An image sensor in which pixels are arranged in a rectangular grid with a pitch PH in the horizontal direction and a pitch PV in the vertical direction. b. PH in the horizontal direction with a pitch of 2 PH in the horizontal direction and a pitch PV in the vertical direction provided corresponding to the pixels.
a first color filter having an offset sampling structure offset by a horizontal pitch of 2PH,
A color filter array comprising a second color filter and a third color filter having a rectangular grid sampling structure with a vertical pitch of 2 PV. c. The imaging optical system includes an optical member that separates an incident light beam into two light beams separated by a distance D in a direction making an angle θ clockwise or counterclockwise with respect to the scanning direction of the image pickup device. Optical low pass filter, 0.8P
H PV / | PH sin θ + PV cos θ | ≦D≦1
．． 2PH PV / | PH sin θ + PV cos θ | Optical low-pass filter that satisfies the condition 0≦θ≦π/2. d. Based on the first color signal, second color signal, and third color signal output from the pixels corresponding to the first color filter, second color filter, and third color filter, the first color signal is Of the color signals, only the signals from pixels in the same column or row as the pixels of the second color signal are synchronized, and a signal obtained by taking the difference between the second color signal and the third color signal of the first color signal is synthesized. A color information forming means that synchronizes only signals from pixels in the same column or row as a pixel of a color signal, and forms the color information from a signal obtained by taking a difference from a third color signal. 2. A color imaging device that converts a subject image into an electrical signal having brightness information and color information, the device comprising:
, b, c, and d. a. An image sensor in which pixels are arranged in a rectangular grid with a pitch PH in the horizontal direction and a pitch PV in the vertical direction. b. PH in the horizontal direction with a pitch of 2 PH in the horizontal direction and a pitch PV in the vertical direction provided corresponding to the pixels.
a first color filter having an offset sampling structure offset by a horizontal pitch of 2PH,
A color filter array comprising a second color filter and a third color filter having a rectangular grid sampling structure with a vertical pitch of 2 PV. c. The imaging optical system includes an optical member that separates an incident light beam into two light beams separated by a distance D in a direction making an angle θ clockwise or counterclockwise with respect to the scanning direction of the image pickup device. Optical low pass filter, 0.8P
H PV / | PH sin θ + PV cos θ | ≦D≦1
．． 2PH PV / | PH sin θ + PV cos θ | Optical low-pass filter that satisfies the condition 0≦θ≦π/2. d. Based on the first color signal, second color signal, and third color signal output from the pixels corresponding to the first color filter, second color filter, and third color filter, the first color signal is Of the color signals, only the signals from pixels in the same column as the pixels of the second color signal are synchronized, and a signal obtained by taking the difference between the second color signal and the third color signal of the first color signal is generated. a first color signal forming means that synchronizes only signals from pixels in the same column as the pixel and forms the color information from a signal obtained by taking a difference from a third color signal; Only the signals from the pixels in the same row as the pixels of the second color signal are synchronized, and the signal obtained by taking the difference from the second color signal and the signal from the pixels in the same row as the third color signal of the first color signal are combined. a second color information forming means that synchronizes only signals from pixels and forms the color information from a signal obtained by taking a difference from a third color signal, and is arranged in the scanning direction of the subject image or perpendicular thereto; Color information forming means that switches between the first color information forming means and the second color information forming means according to a frequency component in a direction.