JP3535623B2 - Color imaging device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a color image pickup apparatus provided with an image pickup device having a plurality of pixels (photoelectric conversion means) arranged two-dimensionally.

[0002]

2. Description of the Related Art There is an image pickup device disclosed in Japanese Patent Laid-Open No. 6-205422, which has a high resolution, a small amount of color moire, and is capable of shooting a still frame image. This is because color filters having a color array of offset sampling structure as shown in FIG. 11 are arranged, and horizontal M pixels ×
An image sensor capable of sequentially and independently reading signals of all pixels of vertical N pixels (N rows × M columns) without interlacing, that is, an image sensor of a so-called all-pixel reading method (progressive scan method) is used, This output signal is processed by a circuit as shown in FIG. 12 to obtain a high quality image signal.

In the circuit of FIG. 12, a light beam that has entered an image pickup device 1 such as a CCD image sensor through an optical low-pass filter 50 is converted into an electric signal by the image pickup device 1, and then a preprocessing circuit 2 and an A / D converter It is sent to the buffer memory 4 via the conversion circuit 3. And Mg (magenta),
R, G, and B colors are converted by an RGB conversion circuit 10 through interpolation filters 6, 7, 8, and 9 for each color of G (green), Cy (cyan), and Ye (yellow), and a white balance is further obtained. The signal is processed by the adjustment circuit 11, the γ conversion circuit 12, the color difference matrix circuit 13, and the low-pass filters 17, 18, and 19 to be output as the low-frequency luminance signal Y _L and the color difference signals RY and BY. . The signal sent from the buffer memory 4 to the low pass filter 5 is
The signal is processed by the γ conversion circuit 14 and the enhancement circuit 15 and output as a high-frequency luminance signal Y _H. Then, the low band luminance signal Y _L and the high band luminance signal Y _H are processed by the luminance signal forming circuit 16 and output as the luminance signal H.

[0004]

However, in the above-mentioned conventional example, an image for one screen cannot be formed unless the signals of all the pixels of the image sensor 1 are read out. Since it takes / 30 seconds, a so-called frame moving image could not be obtained. Further, a method of doubling the transfer speed of the image sensor 1 in order to shorten the read time, a method of using two horizontal transfer units of the image sensor, and the like have been devised, but in order to increase the transfer speed, It is necessary to improve the performance of the semiconductor, and if the number of horizontal transfer parts is two, the structure of the element and the transfer method become complicated, and in any case, the cost is increased.

Further, as a conventional reading method, there is a method of reading by field accumulation, that is, a method of adding and reading signals of two rows in a vertical transfer section. However, in a color filter as shown in FIG. In this case, there is a problem that a color image signal, for example, RGB primary color signal, or a signal such as a luminance signal or a color difference signal cannot be formed from the read signal.

Therefore, the first object of the present invention is to
For each pixel that converts an optical signal from a subject into a charge signal, one of four color filters having different spectral characteristics is arranged, and a plurality of the color filters are arranged in the order of a first color filter and a second color filter. The first row in which the rows are alternately arranged in the row direction, the third color filter, and the fourth row.
A second row in which a plurality of the color filters are alternately arranged in the row direction in the order of the color filter, a second color filter, and a plurality of the color filters in the row direction in the order of the first color filter. And a fourth row in which a plurality of the color filters are alternately arranged in the row direction in the order of a fourth color filter and a third color filter. A color image pickup apparatus having an image pickup element in which a continuous arrangement is repeatedly arranged in the column direction in the order of rows to fourth rows, and with a relatively simple structure, the resolution is good, the color moire is small, and the still image is stationary. It is an object of the present invention to provide a color imaging device capable of capturing a frame image and further capturing a moving image of a frame.

A second object of the invention of the present application is to provide a color image pickup apparatus capable of shortening the readout time by thinning out lines and capable of photographing a moving image of a frame.

[0008]

In order to achieve the above object, the color image pickup device of the present invention has a different spectral characteristic for each pixel for converting an optical signal from a subject into a charge signal.
A first row in which any one of the two color filters is arranged, and a plurality of the color filters are alternately arranged in the row direction in the order of a first color filter and a second color filter; and a third color filter, A second row in which the plurality of color filters are alternately arranged in the row direction in the order of the fourth color filter, and a plurality of the color filters in the row direction in the order of the second color filter and the first color filter. A third row and a fourth color filter alternately arranged in
A fourth row in which a plurality of the color filters are alternately arranged in the row direction in the order of a third color filter, and the first row to the fourth row are arranged in this order in the column direction. In a color image pickup device having an image pickup device in which the above-mentioned arrangements are repeated, addition of signals from the pixels in which the first color filter and the third color filter are arranged, the second color filter, and the A first addition for adding the first row and the second row so as to add signals from the pixels in which a fourth color filter is arranged; the first color filter and the first addition; No. 4 color filter is arranged to add the signals from the pixels, and the second color filter and the third color filter are arranged to arrange the signals from the pixels. Line and said fourth It is characterized by a second adder for adding, with a control circuit that controls to perform sequentially alternately.

In one aspect of the present invention, the image pickup device is provided for each column of the pixels, and a vertical transfer unit for transferring a signal from the pixel in a column direction and a line for transferring the signal from the vertical transfer unit are provided. And a horizontal transfer means for performing addition of signals from the two pixels as well as transfer in the direction.

In another aspect of the color image pickup apparatus of the present invention, one of four color filters having different spectral characteristics is arranged for each pixel that converts an optical signal from a subject into a charge signal. A first row in which a plurality of the color filters are alternately arranged in the order of a first color filter and a second color filter; and a plurality of the color filters in the order of a third color filter and a fourth color filter. Second rows alternately arranged, second color filters, third rows in which a plurality of the color filters are alternately arranged in the order of the first color filter, fourth color filters, third A fourth row in which a plurality of the color filters are alternately arranged in the order of the color filters of
In a color image pickup apparatus including an image pickup device in which a continuous array is repeatedly arranged in the column direction in the order of the rows from the fourth row to the fourth row, a signal output from the image pickup element from the pixel is output in a predetermined number of rows. Thinning-out, addition of signals from the pixels in which the first color filter and the third color filter are arranged to the signals from the pixels read out by thinning out, and the second color filter A first addition for adding two adjacent predetermined rows after thinning so as to add signals from the pixels in which the fourth color filter is arranged; the first color filter and the first addition; 4 after the thinning is performed so as to add signals from the pixels where the color filters are arranged and to add signals from the pixels where the second color filter and the third color filter are arranged. A second adder for adding the constant two adjacent rows, is characterized by comprising a control circuit that controls to perform sequentially alternately.

In one aspect of the present invention, the image pickup device is provided for each column of the pixels, and a vertical transfer unit for transferring a signal from the pixel in a column direction and a signal for transferring the signal from the vertical transfer unit are provided. And horizontal transfer means for transferring in the direction,
The control circuit controls so that signals from the pixels in the thinned rows are transferred faster than signals from the pixels in the non-thinned rows.

In one aspect of the present invention, the signals from the pixels in a plurality of adjacent rows to be thinned are added by the horizontal transfer means.

In one aspect of the present invention, the addition of the signals from the two pixels is performed by delaying the signal output from the image pickup device and the signal output from the image pickup device by a delay unit for a predetermined time. And is performed by adding and.

The image pickup device can read out data of all pixels independently.

[0015]

DETAILED DESCRIPTION OF THE INVENTION

(First Embodiment) FIG. 1 shows a block diagram of an image pickup apparatus according to a first embodiment of the present invention. In FIG. 1, 150
Is an optical low-pass filter, 101 is an image pickup device such as a CCD for converting a subject image into an electric signal, and 102 is a circuit for preprocessing an output signal of the image pickup device 101, including a correlated double sampling circuit (CDS circuit) and an auto gain. It includes a control circuit (AGC circuit) and the like. 103 is an A / D for converting the input analog signal into a digital signal
It is a conversion circuit.

Reference numeral 104 is a buffer memory for storing the input signals, 106, 107, 108 and 109 are interpolation filters for simultaneously synchronizing the input image data, and 110 is the input four color signals ( Mg, G, C
y, Ye) is a first RGB conversion circuit for converting three primary color signals of R, G, B. 120 is a color separation circuit that performs color separation from the input signal, 121 is an addition circuit, 122 is a subtraction circuit, 123 is a line memory, and 124 is the input 3
A second RGB conversion circuit for converting one signal into three primary color signals.

A switch circuit 130 is a circuit for selecting and outputting either the signal output from the block 140 surrounded by the dotted line or the signal output from the block 141. Reference numeral 105 is a low-pass filter that limits the input signal to a predetermined band, 114 is a γ conversion circuit, 115 is an enhancement circuit, and 116 is a high-frequency luminance signal Y _H output from the enhancement circuit 115 and a low-pass filter 117. The luminance signal forming circuit forms the luminance signal Y from the output signal Y _L , and the luminance signal Y is output from 116. These low pass filters 10
5, the γ conversion circuit 114, the enhancement circuit 115, and the luminance signal formation circuit 116 form a block 142.

Reference numeral 111 is a white balance circuit, 112 is a γ conversion circuit, 113 is a color difference matrix circuit for generating a luminance signal and a color difference signal from input primary color signals, 117, 1
Reference numerals 18 and 119 denote low-pass filters, 117 is a low-frequency luminance signal Y _L , 118 is a color difference signal RY, 1
A color difference signal BY is output from 19. The white balance circuit 111, the γ conversion circuit 112, the color difference matrix circuit 113, and the low-pass filters 117 and 11
Reference numerals 8 and 119 form a block 143.

Further, 132 is a drive circuit for generating timing pulses for driving the image pickup device 101, and 131 is a control circuit for controlling the drive circuit 132 and the switch circuit 130.

FIG. 2 is a block diagram showing the arrangement of the image pickup device 101 shown in FIG. 1. The image pickup device has horizontal M pixels × vertical N pixels, for example 510 pixels × 492 pixels or 768 pixels ×
A 492-pixel CCD image sensor or the like, in which the color filter shown in FIG. 11 is arranged.

In FIG. 2, 201 is a photosensitive portion (pixel) formed of a photoelectric conversion element such as a photodiode, 202 is a set of four vertical transfer CCDs 212, 213 arranged along each vertical line of the photosensitive portion 201. A vertical transfer unit consisting of 214 and 215, and 203 a pair of horizontal transfer CCDs 216 and 217 electrically connected to the vertical transfer unit 202.
The vertical transfer unit 202 and the horizontal transfer unit 203 are shielded from light. 204 is a charge detection circuit, 205 is an output signal output terminal, 206, 207, 2
08 and 209 are drive pulses φV ₁ to the vertical transfer unit 202
φV ₄ input terminal for each drive pulse φV ₁
~ Φ V ₄ is the vertical transfer CCD 212 of the vertical transfer unit 202,
213, 214 and 215, respectively. 21
0 and 211 are drive pulses φH ₁ and φ of the horizontal transfer unit 203.
H ₂ input terminal for each drive pulse φH ₁ ,
φH ₂ is the horizontal transfer CCDs 216 and ₂ of the horizontal transfer unit 203
17 respectively.

Next, the first drive mode (all pixel read mode) of the image pickup device 101 in the present embodiment will be described with reference to FIGS. Here, FIG. 3 is a diagram showing a first drive mode in the present embodiment, that is, a state of drive pulses of the image pickup device 101 at the time of reading all pixels, and FIG. 4 is a diagram showing charges in each transfer CCD at that time. It is a figure for explaining the mode of transfer.

The charges accumulated in all the photosensitive portions 201 are
By the signal charge read pulse (not shown) within the vertical blanking period, the photosensitive unit 201 and the vertical transfer unit 202 simultaneously.
Read out. The signal charges read to the vertical transfer unit 202 are stored in the horizontal blanking period, that is, HBLK in FIG.
Are vertically transferred in the Low period (periods a to k). The state of each transfer pulse and the state of each transfer CCD at this time are shown in FIGS. 3 to 4, and the state of the transfer CCD at each time point shown in the periods a to k of FIG. 3 is shown in FIG. It is shown in (a) to (k). In FIG.
V ₁ , V ₂ , V ₃ , and V ₄ are the potential states of the vertical transfer CCDs 212 to 215 of the vertical transfer units 202 in FIG. 2, and H ₁ is the horizontal transfer CCD 21 of the horizontal transfer unit 203.
In the state of the potential of No. 6, the hatched lines are the charges of a predetermined row.

Further, in FIG. 3, each drive pulse φV ₁
~ When φV ₄ , φH ₁ , and φH ₂ become High, transfer CC
The potential of D becomes low, and the potential of the transfer CCD becomes high when the drive pulse becomes low. Vertical drive pulses φV _{1 to} φV ₄ and horizontal drive pulses φH ₁ and φH
_{As 2} is sequentially supplied from the period a as shown in FIG. 3, the potential state of each transfer CCD changes as shown in FIG. As a result, as is apparent from FIG. 4, the signal charges of the vertical transfer unit 202 are sequentially transferred in the vertical direction, and the signal charges of the predetermined nth row are transferred to the horizontal transfer CCD 216 (H ₁ ) of the horizontal transfer unit 203. Will be transferred.

Further, as shown in FIG. 3, during the period l, that is, the video signal period, by supplying the driving pulses of opposite phases to the vertical driving pulses φH ₁ and φH ₂ , the signal charges are transferred to the horizontal transfer section 203. The signal is transferred in the horizontal direction, converted into a voltage by the charge detection circuit 204, and sequentially output as a time-series video signal from the output terminal 205 to the outside. By repeating this operation for 492 lines, for example, the signals of all pixels can be read. At this time, it takes 1/30 second to read an image for one screen.

Next, a method of processing the video signal read from the image sensor 101 as described above will be described with reference to FIG. The drive circuit 132 controlled by the control circuit 131 uses the drive pulse φV ₁ as shown in FIG.
~ ΦV ₄ , φH ₁ , φH ₂ are supplied to the image sensor 101. As a result, the image sensor 101 is driven in the all-pixel reading mode, and the video signal is read. Image sensor 10
The video signal read from 1 is C in the preprocessing circuit 102.
A / D conversion circuit 103 is processed by DS, AGC, etc.
Then, the analog signal is converted into a digital signal, is temporarily stored in the buffer memory 104 for the two-dimensional processing performed later, and the signal necessary for the processing is read from the buffer memory 104.

The signal Y ₁ corresponding to the luminance signal is supplied to the buffer memory 10 in the order corresponding to the pixel array of the image sensor 101.
The data are sequentially read out from No. 4, and input to the switch circuit 130. The switch circuit 130 causes the control circuit 13 to output the signal supplied from the block 140 to a circuit in the subsequent stage.
Controlled by 1. The signal supplied to the low-pass filter 105 via the switch circuit 130 is subjected to predetermined processing by the γ conversion circuit 114 and the enhancement circuit 115 after being subjected to predetermined band limitation, and a luminance signal Y _H including a high frequency component. And is supplied to the luminance signal forming circuit 116. At this time, the luminance signal forming circuit 116 is simultaneously supplied with the low-frequency luminance signal Y _L generated by the method described later,
These are combined and output as a luminance signal Y.

On the other hand, the signals corresponding to the color signals Mg, G, Cy and Ye are similarly read out from the buffer memory 104 and four interpolation filters 106, 107 and 108,
Each of the signals is synchronized at 109 and is converted to R at the RGB conversion circuit 110.
It is converted into three primary color signals of G and B. Note that this conversion is performed by the matrix calculation shown below.

[0029]

[Equation 1]

Here, the matrix A is the image pickup device 101.
Of Mg, G, Cy, Ye spectral characteristics of Mg (λ), G
(Λ), Cy (λ) and Ye (λ) are spectral characteristics R (λ), G (λ) and B of R, G and B defined by the NTSC standard.
It is a matrix with 3 rows and 4 columns optimized so as to approach (λ).

The three primary color signals R ₁ , G ₁ , and B ₁ thus output from the RGB conversion circuit 110 are supplied to the white balance circuit 111 via the switch circuit 130 to be white-balanced and γ-converted. Γ in the circuit 112
After the conversion, the color difference matrix circuit 113 is supplied. The color difference matrix circuit 113 performs the following calculations to reduce the luminance signal Y _L and the color difference signals RY and B.
-Generate Y.

[0032]

[Equation 2]

The signals output from the color difference matrix circuit 113 are band-limited by the low-pass filters 117, 118, and 119, and then the low-frequency luminance signal Y _L is supplied to the luminance signal forming circuit 116. Further, the color difference signal R
-Y and BY are output. The video signal thus obtained is subjected to predetermined processing and then recorded on a recording medium (not shown) or output to the outside.

According to the above method, it takes 1/30 seconds to read an image for one screen, as described above, so that only a still image or a moving image every 1/30 seconds can be obtained. Therefore, in order to obtain a moving image of a frame, it is necessary to drive the image sensor 101 in the second drive mode as described below.

5 and 6, in order to obtain the second driving mode in this embodiment, that is, to obtain a moving image of a frame,
The drive pulse supplied to the image sensor 101 when performing row addition reading is shown. Further, FIGS. 7 and 8 are diagrams for explaining the state of charge transfer when the image sensor 101 is driven by the drive pulses shown in FIGS. 5 and 6, respectively. These FIGS. 2, 5-8
The two-row addition reading in the image sensor 101 will be described using.

First, as in the case of reading all pixels, the charges accumulated in all the photosensitive portions 201 are simultaneously transferred from the photosensitive portion 201 to the vertical transfer portion 202 by a signal charge read pulse (not shown) within the vertical blanking period. Read out.
Next, the signal charges read to the vertical transfer unit 202 are transferred in the vertical direction during the horizontal blanking period. For that purpose, first, drive pulses φV _{1 to} φ shown in FIG.
V ₄ , φH ₁ , and φH ₂ are supplied from the drive circuit 132 to the image sensor 1
01 is supplied.

FIG. 7 shows the state of charge transfer in the transfer CCD at this time. 7A to 7J are shown in FIG.
(A) to (j), and the charges in the nth row are transferred to the horizontal transfer CCD 216 (H ₁ ) of the horizontal transfer unit 203. Here, the n-th row corresponds to the N-th row in FIG. When the drive pulse as shown in FIG. 5 is further applied, the charges are transferred as shown in (k) to (s) of FIG. 7, and the charges on the (n−1) th row are transferred to the horizontal transfer CCD 216 of the horizontal transfer unit 203. Will be transferred to. As a result, the charges on the n-th row and the charges on the (n-1) -th row which have been previously transferred to the horizontal transfer CCD 216 are added. This is transferred in the horizontal direction in the period t of FIG. 5 and output from the output terminal 205. That is, the signal output at this time is as shown in FIG.
The signal obtained by adding the signal of the Nth row of 1 and the signal of the (N-1) th row is output as a signal for one line.

When the reading of the signals for one line is completed and the next horizontal blanking period starts, the drive pulses φV _{1 to} φV ₄ , φH ₁ and φH ₂ shown in FIG. 6 are supplied to the image pickup element 101. . FIG. 8 shows the state of electric charges in the transfer CCD at this time.

In FIG. 8, the operation is the same as that in FIG. 7 from the period (a) to the period (j), and the description thereof will be omitted. Period (k)
The horizontal drive pulse φH ₁ changes from High to L at the start time of
The charges transferred to the horizontal transfer CCD 216 (H ₁ ) are transferred to the horizontal transfer CCD 217 (H ₂ ) by changing φH ₂ to low from Low to High. Further, when φH ₁ becomes High again and φH ₂ becomes Low at the start time of the period (l), the charges are further transferred in the horizontal direction, and the charges of the horizontal transfer CCD 217 are transferred to the horizontal transfer CCD 216 of the adjacent group. . As a result, the charges transferred to the horizontal transfer unit 203 are shifted by one pixel in the horizontal direction. Therefore, the charges in the horizontal transfer CCD 216 (H ₁ ) in the state of (l) of FIG. 8 are the charges of the nth row of the adjacent column.

Hereinafter, drive pulses φV _{1 to} φ as shown in FIG.
When V ₄ , φH ₁ , and φH ₂ are supplied to the image sensor 101,
The state becomes from (m) to (u) of FIG. 8, and (n-1)
The charges on the row move vertically and the horizontal transfer CCD 216
And is added to the electric charge of the nth row previously transferred to the horizontal transfer CCD 216. This added signal is
It is transferred in the horizontal direction during the period v in FIG. 6 and is output from the output terminal 205. That is, the signal output at this time is
The signal obtained by shifting the signal of the (N-2) th row in FIG. 11 by one pixel in the horizontal direction (leftward in FIG. 11) and the signal of the (N-3) th row (that is, diagonally adjacent to each other). Signal obtained by adding the signals of the two pixels) is output as a signal for one line.

Thereafter, the drive pulses shown in FIGS. 5 and 6 are alternately supplied to the image pickup device 101 every horizontal period, so that the electric charges for one screen accumulated in the image pickup device 101 are stored in two rows. Are output in the state of being added one by one. 2 here
Since the output is added row by row, the time required to read out the charges of the entire screen is half that in the first drive mode, and the reading can be performed in one field period, that is, 1/60 seconds, so that It is possible to shoot.

When the signals read from the image pickup device 101 by the above operation are summarized, the signals read from the Nth row and the (N-1) th row in FIG. 11 are added and read as they are, so [G + Ye], [Mg + Cy], [G + Ye], [Mg + Cy] ... (3). The signals read from the (N−2) th row and the (N−3) th row are added and read after the (N−2) th row is horizontally shifted by one pixel, so that [G + Cy]. , [Mg + Ye], [G + Cy], [Mg + Ye] ... (4), and a signal is output by repeating these.

Further, in the next field, the combination of rows to be added is changed, and the rows (N-1) and (N-2) are
The charges are added and read in the combination of the (N-3) th row and the (N-4) th row .... Therefore, when the charges of the (N-1) th row and the (N-2) th row are added, the image sensor 1
The drive pulse shown in FIG. 6 is supplied to 01, and (N-1)
After shifting the charges of the row by one pixel, the charges of the (N−2) th row are added. Further, when the charges of the (N-3) th row and the (N-4) th row are added, the drive pulse shown in FIG. 5 is supplied, and the charges of the (N-3) th row and the (N-4) th row are added as they are. To do. By repeating this alternately, the image for one screen is read.

Therefore, the signals read in this field are [Mg + Cy], [G + Ye], [Mg + Cy], [G + Ye] ... (5) in the (N-1) th row and the (N-2) th row. , (N-3) row and (N-4) row, [G + Cy], [Mg + Ye], [G + Cy], [Mg + Ye] ... (6), and the signal is output by repeating this.

As described above, according to the second drive mode in the present embodiment, the combination of rows for adding charges is changed for each field, and the signal is read out every 1/60 seconds, so that a frame moving image is displayed. You can get a statue.

Next, in the second drive mode, the image sensor 101
A method of processing the video signal read from will be described with reference to FIG.

The drive circuit 132 controlled by the control circuit 131 uses the drive pulse φV ₁ as shown in FIGS.
~ ΦV ₄ , φH ₁ , φH ₂ are supplied to the image sensor 101. As a result, the image sensor 101 is driven in the two-row addition read mode, and the video signal is read. The video signal read from the image sensor 101 is processed by the preprocessing circuit 102.
Then, the signal is subjected to processing such as CDS and AGC, and is supplied to the switch circuit 130 as a signal Y ₂ corresponding to the luminance signal and further to the color separation circuit 120 as a signal for separating the color signal.

The switch circuit 130 is controlled by the control circuit 131 so as to output the signal supplied from the block 141 to the subsequent circuit, and supplies the Y ₂ signal to the low pass filter 105. The subsequent operation is the same as the first
Since the operation is the same as that in the drive mode, description thereof will be omitted.

Further, the color separation circuit 120 separates the color signals from the signals sequentially output as signals from the image pickup device 101, and further performs the synchronization processing. That is, when the sequential signals as shown in the equation (3) are input, they are separated into two signals of S ₁ = [G + Ye] S ₂ = [Mg + Cy].

Each color signal is output from the image pickup device 101.
Since the signal of is a sequential signal, S₁, S ₂, S₁, S₂......
The color is reproduced as it is output every 1 pixel.
I can't. Therefore, interpolate the missing parts of the signal.
There is a need to. As a method of interpolation,
Pre-interpolation that interpolates with the pixel before the pixel
Interpolation with average value This is done using average value interpolation. Color separation
The signals separated by the circuit 120 are added by the adding circuit 121.
And further subtracted by the subtraction circuit 122. Adder circuit 1
21, the signals output from the subtraction circuit 122 are respectively
It becomes like the following formulas (7) and (8).   S₁+ S₂= [G + Ye] + [Mg + Cy] = 2R + 3G + 2B                                                       …… (7)   S₂-S₁= [Mg + Cy]-[G + Ye] = 2B-G                                                       …… (8)

Similarly, when the signal read from the image pickup element 101 is as shown in equation (4), S ₁ = [G + Cy] S ₂ = [Mg + Ye] S ₁ + S ₂ = [G + Cy] + [Mg + Ye] ] = 2R + 3G + 2B (9) S ₂ −S ₁ = [Mg + Ye] − [G + Cy] = 2R−G (10)

The signals output from the addition circuit 212 and the subtraction circuit 122 are supplied to the second RGB conversion circuit 124. At the same time, the signal output from the subtraction circuit 122 is
The signal is supplied to the line memory 123 and delayed by one horizontal scanning period, and the delayed signal is supplied to the second RGB conversion circuit 124.
Is supplied to. Therefore, when the signals represented by the formulas (9) and (10) are supplied to the second RGB conversion circuit 124, the signals represented by the formula (8) are supplied to the second RGB conversion circuit 124 at the same time. Will be.

In the second RGB conversion circuit 124,
Calculation is performed and the signal R of the three primary colors₂, G ₂, B₂Generate
It   G₂= {(S₁+ S₂)-(S₂-S₁)-(S₂-S₁) '} / 5       = {2R + 3G + 2B- (2R-G)-(2B-G)} / 5                                                         …… (11)   R₂= {(S₂-S₁) + G} / 2       = {(2R-G) + G} / 2 (12)   B₂= {(S₂-S₁) '+ G} / 2       = {(2B-G) + G} / 2 (13) However, (S₂-S₁) 'Is from the line memory 123
It is assumed that the signal delayed by one line is supplied.

Further, G on the next line₂The signal is
R is calculated by the same calculation as the formula (11). ₂, B₂Signal is it
Each can be obtained by the following calculation.   R₂= {(S₂-S₁) '+ G} / 2       = {(2R-G) + G} / 2 (14)   B₂= {(S₂-S₁) + G} / 2       = {(2B-G) + G} / 2 (15)

The primary color signals R ₂ , G ₂ , and B ₂ generated by the second RGB conversion circuit 124 as described above are supplied to the switch circuit 130. Also in the case of the next field, equation (3) such as equations (5) and (6)
Since the same color signal as in (4) is output, RGB primary color signals can be generated in the same manner as in the case of the preceding field. However, since the order of outputting the color signals of the signal obtained by the equation (5) is opposite to that in the case of the equation (3), it is necessary to change the order of separation in the color separation circuit 120.

R ₂ , G supplied to the switch circuit 130
₂ and B ₂ signals are selected by the switch circuit 130,
It is supplied to the white balance control circuit 111. Subsequent operations are the same as in the first drive mode, so description will be omitted. In this way, the moving image of the frame can be obtained as an output signal.

As described above, the color image pickup apparatus according to the present embodiment has four types of color filters, and the color filters of different colors are repeated every two pixel periods in the horizontal direction and the vertical direction. In contrast, in an image pickup apparatus including an image pickup element capable of reading all pixels having an offset sampling structure in which a color filter of a different color is repeated in a 2-pixel cycle and offset by one pixel in the horizontal direction, When capturing a still image of a frame, signals of all pixels are independently read out, and when capturing a moving image of a frame, signals of two vertically adjacent rows are added and read out. , A mode in which signals of two pixels adjacent in the vertical direction are added, and a mode in which signals in one row are horizontally shifted by one pixel and signals in the next row are added, It is configured to perform repeated and a mode for adding signals of two pixels adjacent to the ie oblique direction.

With this structure, it is possible to provide an image pickup device capable of shooting a still frame image having a good resolution and a small amount of color moire with a relatively simple structure, and at the same time capable of shooting a moving image of a frame. .

(Second Embodiment) Next, the second embodiment of the present invention will be described.
The embodiment will be described. FIG. 9 shows the second embodiment of the present invention.
It is a block diagram of the imaging device of the embodiment of. In FIG. 9, the same reference numerals as those in FIG. 1 indicate the same constituent elements, and the blocks 140, 141, 142,
The internal structure of 143 is omitted.

In FIG. 9, 301 is a line memory capable of storing the input signal for one line, and 302
Is an adder circuit, and 303 is a buffer memory.

Next, the operation of the image pickup apparatus of this embodiment will be described. There are two driving modes of the image sensor in the present embodiment. The first drive mode is the same as the first drive mode in the first embodiment, that is, the case of reading all pixels, and the method of processing the signal output from the image sensor is also the same, so the description will be omitted. Omit it. This first drive mode is performed when capturing a still image of a frame. Then, in the second drive mode, line thinning-out is performed and signals are read out without using the signals of all the lines among the signals accumulated in the pixels of the image sensor 101. Here, as an example thereof, a case will be described in which two lines are thinned out to three lines, that is, one line is read out and two lines are thinned out repeatedly to be read out.

In this second driving mode, the charges accumulated in all the photosensitive portions 201 of FIG. 2 are simultaneously transferred from the photosensitive portion 201 to the vertical transfer portion 202 by a signal charge read pulse (not shown) within the vertical blanking period. Read out. The signal charges read to the vertical transfer unit 202 are
3 is transferred in the vertical direction by the drive pulse shown in FIG. 3, the lowermost charge is transferred to the horizontal transfer CCD 216, and further read out in the horizontal direction to output terminal 205.
Are sequentially read. The charges on the next line are similarly transferred in the vertical direction, and the charges at the bottom are transferred horizontally to the CCD 216.
Transferred to.

Here, in order to thin out the charges on this line, the horizontal transfer section 203 is transferred at high speed in the horizontal direction and output from the output terminal 205. At this time, the transfer speed of the horizontal transfer unit 203 is set to be several times as high as a normal read speed. Further, the charges of the next line are similarly read at high speed. This operation is repeated to read the signals of the entire screen. By thus transferring the charges of the thinned lines in the horizontal transfer unit 203 at high speed, it is possible to shorten the read time of the signal charges of the entire screen.

As a method of reading out the charges of the thinned lines, the driving pulses as shown in FIG. 5 are used to add the charges of the two thinned lines by the horizontal transfer CCD 216 and then transfer and read them in the horizontal direction. By doing so, the read time can be further shortened.

A method of processing the signal read from the image sensor 101 as described above will be described. Image sensor 1
The signal read from 01 is processed by the preprocessing circuit 102. At this time, the signal read from the image sensor 101 at the normal transfer speed is processed by the preprocessing circuit 102, but the signal read at high speed from the image sensor 101 is not processed by the preprocessing circuit 102 and discarded. Thinning out by dropping.

Color signals read out from the image pickup device 101 having the color filter array shown in FIG. 11 by thinning out lines by this method are output in the order shown in FIG. The signal output in this way is processed by the preprocessing circuit 102.
The analog signal is converted into a digital signal by the A / D conversion circuit 103 and is supplied to the line memory 301 and the addition circuit 302. First, when the signal on the n-th row in FIG. 10 is supplied from the A / D conversion circuit 103 to the line memory 301, this signal is stored, and the signal on the next (n-1) -th row is the A / D conversion circuit. At the same time as being output from 103, it is read from the line memory 301. Therefore, at this time, the adder circuit 302 outputs the signal of the nth row in FIG.
1) The signals of the row are supplied at the same time and they are added. Therefore, the signal output from the adder circuit 302 is a signal obtained by adding the signals of pixels adjacent to each other in the same column in the vertical direction.

Further, regarding the (n-2) th line and the (n-3) th line, the signal of the (n-2) th line is first stored in the line memory 301, and the next (n-3) th line is stored. Signal is A / D
The signal reading from the line memory 301 is started from the timing one pixel before the timing output from the conversion 103, and the signal is supplied to the addition circuit 302. By doing so, the adder circuit 302 adds the signal obtained by shifting the signal on the (n−2) th row by one pixel and the signal on the (n−3) th row. A signal obtained by adding the signals of the pixels adjacent in the diagonal direction is output.

Therefore, the signal output by adding the signal of the nth row and the signal of the (n-1) th row is given by the following equation (1)
6), and the signal of the (n-2) th row and (n-3)
The signal output by adding the signal of the row is Equation (17)
become that way. [G + Cy], [Mg + Ye], [G + Cy], [Mg + Ye] ... (16) [G + Ye], [Mg + Cy], [G + Ye], [Mg + Cy] ... (17)

The signal thus output is temporarily stored in the buffer memory 303. At this time, the image sensor 1
As described above, the signal output from 01 includes a line that is read at a normal speed and a line that is read at a high speed for performing line thinning, and the output signal is not continuous in time. Therefore, equation (16) (1
The signal represented by 7) is once stored in the buffer memory 303, read out as a continuous signal from here, and supplied to the circuit block 141 in the subsequent stage. Circuit block 1
The signals supplied to 41 are those shown in (16) and (17) above, and are the same color signals as (3) and (4) in the first embodiment. Subsequent processing is the same as that of the first embodiment, so description thereof will be omitted.

In this embodiment, in this way, it is possible to perform reading by thinning out lines, and further, a moving image of a frame can be obtained as an output signal.

In the description of this embodiment, the thinning out of two lines to three lines has been described, but the present invention is not limited to this thinning out, and the signals of the two thinned lines remaining are added. Therefore, the above formula (16) (1
It goes without saying that the thinning-out can be performed as long as the color signal shown in 7) is obtained as an output.

Also in the above-described first embodiment, the addition of signals is not performed by the image sensor 101, but the addition is performed by using the line memory 301 and the addition circuit 302 as in the present embodiment. May be.

As described above, the color image pickup apparatus according to the present embodiment has four types of color filters, and the color filters of different colors are repeated in the horizontal direction in a two-pixel cycle, and the vertical direction is used. In a color image pickup apparatus having an image pickup element capable of reading all pixels having an offset sampling structure in which a color filter of two pixels is offset in the horizontal direction and the color filters of different colors are repeated. When capturing a still image of a frame, signals of all pixels are independently read out, and when capturing a moving image of a frame, line thinning is performed to read out pixel signals. At the time of thinning, the signal of the thinning line is transferred to the horizontal transfer unit at high speed and is discarded, and the signal of the line to be used is read at the normal speed. The signals read out in this manner are subjected to external addition processing of signals for two rows. In the addition, a mode in which signals of two pixels adjacent in the vertical direction are added, a mode in which a signal for one row is horizontally shifted by one pixel, and a signal for the next row is added,
That is, the mode of adding the signals of two pixels adjacent in the diagonal direction is repeatedly performed.

By doing so, it is possible to shoot a still frame image with good resolution and little color moire, and at the same time, it is possible to shorten the readout time by thinning out the lines and also to shoot a frame moving image. An imaging device can be provided.

[0075]

As described above, according to the present invention,
For each pixel that converts an optical signal from a subject into a charge signal, one of four color filters having different spectral characteristics is arranged, and a plurality of the color filters are arranged in the order of a first color filter and a second color filter. The first row in which the rows are alternately arranged in the row direction, the third color filter, and the fourth row.
A second row in which a plurality of the color filters are alternately arranged in the row direction in the order of the color filter, a second color filter, and a plurality of the color filters in the row direction in the order of the first color filter. And a fourth row in which a plurality of the color filters are alternately arranged in the row direction in the order of a fourth color filter and a third color filter. A still frame image with good resolution and little color moire can be taken when shooting is performed using an image sensor in which rows and rows are sequentially arranged in a row direction to a fourth row. That is, it is possible to capture a moving image of a frame with a relatively simple structure.

According to another feature of the present invention, one of four color filters having different spectral characteristics is arranged for each pixel for converting an optical signal from a subject into a charge signal, and the first color filter is provided. A first row in which the plurality of color filters are alternately arranged in the row direction in the order of a filter and a second color filter; and a plurality of the color filters in the order of a third color filter and a fourth color filter. Second rows alternately arranged in a row direction, second color filters,
A third row in which the plurality of color filters are alternately arranged in the row direction in the order of the first color filter, a fourth color filter, and a plurality of the color filters in the row direction in the order of the third color filter. And a fourth row alternately arranged, in the order of the first row to the fourth row, when performing imaging using an imaging element that is repeatedly arranged in a continuous column direction, It is possible to shoot still frame images with good resolution and less color moire,
Readout time can be shortened by thinning out lines,
Furthermore, it becomes possible to shoot a moving image of a frame.

[Brief description of drawings]

FIG. 1 is a block diagram of an image pickup apparatus according to a first embodiment of this invention.

FIG. 2 is a schematic diagram showing a configuration of an image pickup element used in the image pickup apparatus according to the first embodiment of the present invention.

FIG. 3 is a diagram showing a timing at the time of reading all pixels in the image pickup apparatus according to the first embodiment of the present invention.

FIG. 4 is a diagram for explaining charge transfer at the time of reading all pixels in the imaging device according to the first embodiment of the present invention.

FIG. 5 is a timing chart at the time of 2-pixel addition reading in the image pickup apparatus according to the first embodiment of the present invention.

FIG. 6 is a timing chart at the time of 2-pixel addition reading in the image pickup apparatus according to the first embodiment of the present invention.

FIG. 7 is a diagram for explaining charge transfer at the time of 2-pixel addition reading in the image pickup device according to the first embodiment of the present invention.

FIG. 8 is a diagram for explaining charge transfer at the time of 2-pixel addition reading in the image pickup apparatus according to the first embodiment of the present invention.

FIG. 9 is a block diagram of an image pickup apparatus according to a second embodiment of the present invention.

FIG. 10 is a diagram illustrating the order of signals read when line thinning is performed in the second embodiment of the present invention.

FIG. 11 is a diagram showing a color array of color filters of the image pickup apparatus of the present invention and the conventional example.

FIG. 12 is a block diagram of a conventional imaging device.

[Explanation of symbols]

101 image sensor 102 Preprocessing circuit 103 A / D conversion circuit 104 buffer memory 105 low-pass filter 106, 107, 108, 109 Interpolation filter 110 RGB conversion circuit 111 White balance adjustment circuit 112, 114 γ conversion circuit 113 color difference matrix circuit 115 Enhance circuit 116 Luminance signal forming circuit 117, 118, 119 Low-pass filter 120 color separation circuit 121 Adder circuit 122 Subtraction circuit 123 line memory 124 RGB conversion circuit 130 switch circuit 131 control circuit 132 drive circuit

Claims (7)

(57) [Claims] 1. One of four color filters having different spectral characteristics is arranged for each pixel for converting an optical signal from a subject into a charge signal, and a first color filter and a second color filter are arranged in this order. A first row in which the plurality of color filters are alternately arranged in the row direction, and a second row in which the plurality of color filters are alternately arranged in the row direction in the order of a third color filter and a fourth color filter. Of the second color filter and the first color filter, the third row in which the plurality of color filters are alternately arranged in the row direction, and the fourth color filter and the third color filter. A plurality of the color filters are arranged in order in a row direction, and a fourth row is alternately arranged. The first row to the fourth row are arranged in this order, and a continuous arrangement is repeatedly arranged in the column direction. Image sensor In the above color imaging device, the addition of signals from the pixels in which the first color filter and the third color filter are arranged, and the arrangement in which the second color filter and the fourth color filter are arranged A first addition for adding the first row and the second row so as to add signals from pixels; and the pixel in which the first color filter and the fourth color filter are arranged. The third row and the fourth row are added so that the signals from the pixels from which the second color filter and the third color filter are arranged are added. A color image pickup apparatus comprising a control circuit for controlling the second addition and the second addition so as to be alternately performed. 2. The image pickup device is provided for each column of the pixels, and transfers vertical signals from the pixels in a column direction; and transfers vertical signals from the vertical transfer unit in a row direction, The color image pickup apparatus according to claim 1, further comprising a horizontal transfer unit that adds signals from the two pixels. 3. One of four color filters having different spectral characteristics is arranged for each pixel that converts an optical signal from a subject into a charge signal, and the first color filter and the second color filter are arranged in this order. A first row in which the plurality of color filters are alternately arranged, a second row in which the plurality of color filters are alternately arranged in the order of a third color filter and a fourth color filter, and a second row A third row in which the plurality of color filters are alternately arranged in the order of the color filter and the first color filter, and a plurality of the color filters in the order of a fourth color filter and a third color filter. And a fourth row arranged in a row, the color image pickup apparatus including an image pickup element in which a continuous array in the column direction is repeatedly arranged in the order of the first row to the fourth row, The first color filter and the third color filter are arranged with respect to the signals from the pixels read out by thinning out the signals from the pixels output from the image pickup device by a predetermined number of rows. In order to add signals from the pixels and signals from the pixels in which the second color filter and the fourth color filter are arranged, predetermined adjacent two rows after thinning are performed. First addition for addition, addition of signals from the pixels in which the first color filter and the fourth color filter are arranged, arrangement of the second color filter and the third color filter, And a control circuit for controlling so as to sequentially and alternately perform a second addition for adding predetermined adjacent two rows after thinning so as to add signals from the pixels. Color Imaging device. 4. The image pickup device is provided for each column of the pixels, and has vertical transfer means for transferring signals from the pixels in a column direction, and horizontal transfer for transferring signals from the vertical transfer means in a row direction. 4. The control circuit controls so that a signal from the pixel in a thinned row is transferred faster than a signal from the pixel in a non-thinned row. Color imaging device. 5. The color image pickup apparatus according to claim 4, wherein the signals from the pixels in a plurality of adjacent thinned rows are added by the horizontal transfer unit. 6. The addition of the signals from the two pixels includes a signal output from the image sensor and a signal obtained by delaying the signal output from the image sensor by a delay unit for a predetermined time.
6. The color image pickup apparatus according to claim 1, wherein the color image pickup apparatus is performed by adding. 7. The image pickup device is capable of independently reading the data of all pixels.
7. The color imaging device according to any one of items 6 to 6.