JPH02299374A - Smear elimination method for pseudo frame electronic shutter - Google Patents

Abstract

PURPOSE:To obtain a picture without flicker or line crawl by multiplying a coefficient proportional to a shutter speed with a picture element signal of a line not including the smear component and subtracting the multiplied result from a picture element signal including the smear component. CONSTITUTION:A green G or a white W picture element signal G relevant to an even number field includes a smear component and the result is expressed in equation I, where a smear component Qs is added to a true picture element signal G*. A signal G* not including the smear component is obtained by equation II, in which Ts is a shutter speed and a coefficient is alphas. Thus, the smear elimination calculation based on the picture element signal of red R and blue B by using the coefficient alphas is implemented according to equations III, IV (k is a coefficient). That is, a smear elimination circuit 7 calculates each picture element data corresponding to one frame picture stored in a frame memory 6 according to a proper equation among the equations II, III, IV. The picture element data after the processing is stored again in the memory 6. A luminance signal Y and color difference signals R-Y, B-Y are formed based on the picture element data after the smear elimination processing.

Description

Detailed Description of the Invention [Industrial Application Field] The present invention relates to a pseudo-frame electronic shutter using a frame interline transfer type charge-coupled solid-state imaging device (FIT-CCD), and in particular, to The present invention relates to a method for removing smear components mixed into output pixel signals.

[Conventional technology]

Figure 5 shows the F applied to the pseudo-frame electronic shutter.
The configuration of the IT-CCD is conceptually shown.

That is, the FIT-CCD has a light-receiving area in which a plurality of light-receiving elements (indicated by ; in the figure) are arranged in a matrix along the horizontal scanning direction X and the vertical scanning direction y. 1, and furthermore, a plurality of vertical charge transfer paths 11 to 1.4 are formed adjacent to each column of light receiving elements arranged along the vertical scanning direction y. The light-receiving element indicated by a symbol in the figure is a photoelectric conversion element such as a photodiode provided with a color mosaic filter for generating a green (G) color signal with respect to incident light, and will be described later. Arranged in even fields. Also, the light-receiving element indicated by the symbol is red (R) for the incident light.
A photoelectric conversion element such as a photodiode equipped with a color mosaic filter for generating color signals, symbol:;
The light-receiving element indicated by :;: has a blue (B) color with respect to the incident light.
) is formed of photoelectric conversion elements such as photodiodes equipped with color mosaic filters for generating color signals, and is arranged in odd-numbered fields (described later), and red and blue filters are arranged in the horizontal and vertical scanning directions. They are arranged so that they are completely staggered. Such a filter arrangement is called a horizontal G stripe R/B point sequential arrangement.

On the other hand, the vertical charge transfer path! , ~2M, gate electrodes (not shown) for transferring signal charges in the vertical scanning direction y are formed on the surface in a predetermined arrangement, and drive signals such as a so-called four-phase drive system are applied to these gate electrodes. The structure is such that the transfer operation in the vertical scanning direction y is performed by applying the voltage.

An accumulation region 2 is formed following the light receiving region 1, and each vertical charge transfer path i, to 1 . It is equipped with a vertical charge transfer path connected to the Note that the vertical charge transfer paths in the light receiving area 1 and the M loading/unloading area 2 are not cut off for each area, but rather extend across each area, so that one point 1 in the figure The ¥ lines simply indicate the boundaries of each area.

A horizontal charge transfer path 3 is formed at the output end of the storage region 2 to transfer signal charges transferred from the vertical charge transfer path of the storage region 2 in the horizontal scanning direction. An output amplifier 4 is formed and the pixel signal S is outputted from the output amplifier 4. -I: Has a structure for output.

Further, as shown in the figure, assuming that the light receiving area 1 is formed in N columns along the horizontal scanning direction X and M rows in the vertical scanning direction y, a total of NxM light receiving elements, the light receiving area l
The vertical charge transfer path 1~! continues from the storage region 2 to the horizontal charge transfer path 3! 8 are formed, and the charge transfer pixels of each vertical charge transfer path 21 to N in the light receiving area l are M.
/2 stages, each vertical charge transfer path 11 to f in storage region 2
N charge transfer pixels are set in M/2 stages, and charge transfer pixels of the horizontal charge transfer path 3 are set in at least N stages. Also, odd-numbered rows in the vertical charge transfer direction y (y=1.
3.5...M-1) light-receiving element is odd field, even row (y=2.46...M)
) corresponds to the even field. Furthermore, the upper surface of the vertical charge transfer path in the storage region 2 is shielded from light to prevent it from entering.

Next, an imaging operation involving a pseudo-frame electronic shutter operation by the FIT-CCD having such a structure will be explained based on FIG. 6.

Fig. 6 is an explanatory diagram showing the operation of an electronic still camera etc. when capturing a still image over time, and shows the - times of image capturing operation, and (A) of the same figure shows the operation of an electronic still camera etc. when capturing a still image. Odd field readout for reading out red and blue pixel signals from a light receiving element; Figure (B) shows the timing for even field readout for reading out green pixel signals from a light receiving element in which green filters are arranged. .

First, in synchronization with pressing the shutter release button of the camera, the signal charges of the light-receiving electrons corresponding to the odd field are transferred to the adjacent vertical charge transfer path 2° in a very short period of time from time t1 to time t2. ~2. Move to.

Next, in an extremely short period of time ~L4, the signal charges of the light-receiving elements corresponding to even fields are transferred to the adjacent vertical charge transfer paths! ,~1. and mixes it with the signal charge of the light receiving element corresponding to the odd field.

In this way, after the signal charges corresponding to the odd field and the signal charges corresponding to the even field are transferred to the vertical charge transfer paths in the light receiving area 1, the vertical charge transfer paths 21 to 2N are transferred between times t and ~t. By transferring the signal charges in the vertical scanning direction y at high speed (for example, about 400 μs) and simultaneously causing the horizontal charge transfer path 3 to read out the signal charges, all the signal charges related to the odd and even fields are swept out. .

Exposure time τ of the light receiving element corresponding to the odd field
, is the period from immediately after the unnecessary signal charge is transferred to the vertical charge transfer path at time L2 to just before time , and the exposure time τ6 of the light receiving element corresponding to the even field is the period when the unnecessary signal charge is vertically transferred at time L6. The period is from immediately after the charge is transferred to the charge transfer path to just before time L+1, which will be described later, and the exposure time can be changed by expanding or contracting the periods from time L2 to t7 and from t4 to t11.

Next, a video signal reading operation after exposure will be explained.

First, for an extremely short period of time ~L, the signal charges of the light receiving elements corresponding to the odd field are transferred to predetermined transfer pixels of the adjacent vertical charge transfer paths 11 to 11 to P and 4, respectively. Next, these signal charges are timed, ~t
The data is transferred to the storage region 2 in a relatively short period of 1O (approximately 400 μs).

Next, with the signal charge transfer operation of the storage region 2 stopped, the signal charges of the light-receiving elements corresponding to even fields are transferred to the adjacent vertical charge transfer paths for an extremely short time from time t to tlO. ,~! 8 predetermined charge transfer pixels.

In this way, odd field signal charges are transferred to the storage region 2.
The signal charges of one even field are transferred to the vertical charge transfer path 2 in the light receiving area. After moving to ~lN, at time t+3~
In the period 114, first, the signal charges in the accumulation region 2 are sequentially transferred to the horizontal charge transfer path 3 side, and each time the signal charges for one row are transferred, the horizontal charge transfer path 3 transfers each signal charge in time series. By repeating this vertical charge transfer and horizontal charge transfer operation, all signal charges in the accumulation region 2 are read out, and pixel signals corresponding to all these signal charges are stored as odd field pixel signals in, for example, a semiconductor memory. .

Next, during the period from time t+s to t16, the vertical charge transfer path in the light receiving region 1! ,~1. The signal charge corresponding to the even field, which was temporarily stopped at , is transferred to the storage region 2, and then, at time t+? All the signal charges corresponding to the even fields of the storage region 2 are read out during the period from ~til+. Note that the signal readout operation from time 19 LIT to t18 is the same as the readout operation from time t13 to t14, and is stored in another semiconductor memory or the like as an even field pixel signal.

In this way, the period of time fl+"""j+. becomes the basic operation for capturing a still image for one frame.
By using the FIT-CCD, there is no need to provide a mechanical shutter in the imaging optical system, and the exposure time τ and τ8 can be varied and controlled for each field image, that is, the shutter time can be varied and controlled using pseudo-electronic A shutter can be realized.

[Problems to be Solved by the Invention] However, such a conventional pseudo-electronic shutter has a problem in that smear generated by external incident light causes a deterioration in image quality.

The principle of generation of this smear will be explained with reference to FIG. 7. The vertical charge transfer path 2. is provided, and further a vertical charge transfer path 2 is provided.
A gate electrode GT is formed on the upper surface of J via a gate oxide film (not shown), and noise charges Ql generated in the semiconductor substrate BA due to light strong enough to pass through the light receiving element PD, Noisy charges qz generated due to light incident on the gate oxide film after being diffusely reflected, and
Noisy charges ql and the like generated due to light passing through the gate electrode GT become smear components.

Furthermore, pixel signals related to even fields are much more affected by smear than pixel signals related to odd fields.

That is, the signal charge corresponding to the odd field is for a short period τC (τc = about 400 μs) at time ti"t+ in FIG.
is transferred to the accumulation region 2 and is affected by smear for only this short period of time, whereas in the signal readout of the even field, the signal charge corresponding to the even field is received during the period from time t12 to t16 in FIG. From area 1 to accumulation area 2
It will be affected by smear during the period τ until the data is transferred to. In particular, the noise charge q1 shown in FIG. 7 moves relatively slowly within the substrate and forms a vertical charge transfer path! .

The vertical charge transfer path for a long time τ4! Even field signal charges that stop at , are more affected by smear. normal NT
When the signal is read out using the SC method, τ=1/60 (second), so the difference (ratio) in the amount of smear is about 42 times.

And, if the smear components for each field are different, NTSC
When playing back images in an interlaced format that is compatible with standard television formats, flickering occurs because the brightness of each field is different, and when a still image is formed by hard copy, each frame If the pixel signals for 100 minutes are reproduced in a non-interlaced manner, line crawl will occur, resulting in a decrease in image quality.

The present invention has been made in view of such problems,
An object of the present invention is to provide a pseudo-frame electronic shutter that can realize clear images without causing flicker or line crawl in both non-interlaced and interlaced image reproduction.

[Means for Solving the Problems] First, the present invention will be explained in detail. The FIT-COD having the configuration shown in FIG. 5 is applied to a pseudo electronic shutter, and the signal charges generated in each light-receiving element in the light-receiving area are read out in accordance with field readout similar to that shown in FIG. 6.

Furthermore, the color filters provided in the light receiving element are green (G) arranged in even-numbered rows in the vertical scanning direction y, as shown in FIG.
) stripe filters and so-called horizontal G stripe R/B point sequential color filters (Bayer (including color filters in the array), and the reverse array of these, that is, red (R) in even numbered rows.
A horizontal G stripe R/B point sequential color filter, etc., in which mosaic filters of and blue (B) are arranged alternately and green (G) stripe filters are arranged in odd-numbered rows, is applied.

According to the present invention, a luminance signal corresponding to each light-receiving element is formed by a predetermined calculation method based on pixel data corresponding to one frame outputted from a FIT-CCD by field readout. That is, the first field (
The multiplication value obtained by multiplying the pixel signal corresponding to the light-receiving element of the light-receiving element of the second field adjacent to the light-receiving element (the field in which the first field is read) is multiplied by a predetermined coefficient proportional to the shutter speed. The difference between the pixel signal and the pixel signal corresponding to the light receiving element arranged in the second
By creating new pixel signals corresponding to the light receiving elements arranged in the field, smear components included in the pixel signals of the second field are removed.

FIGS. 1 and 2 are explanatory diagrams for explaining the basic principle of smear removal in more detail. For convenience of explanation, red (R) and blue (B) light receiving elements are arranged in the first field, and green (G) or white (white) elements are arranged in the second field.
A case will be explained in which the light receiving elements of W) are arranged.

First, in FIG. 1, the smear component (noise charge) generated in each light receiving element in response to a change in shutter speed is proportional to the shutter speed. In other words, if the amount of exposure applied to the CCD is ideally controlled to a constant value, the faster the shutter speed is set and the image is taken, the greater the intensity of the incident light will be, and the smear component will be proportional to the shutter speed. It will be done. The present invention attempts to remove smear components by applying such a relationship.

Therefore, when the exposure amount is constant, if the pixel signal generated in the light-receiving element is QA, the shutter speed (corresponding to the time the shutter opens) is Ts, and the amount of smear is Q, then T's is The faster the speed, the higher the light intensity, so smear fitQ.

and Ts are in a proportional relationship, Q, - constant ・・・・・・・・・・・・ (1) Qs
cc Ts . . . The relational expression (2) holds true.

Regarding the pixel signal QA, first, the red (R) and blue (B) pixel signals corresponding to the odd field (first field) can ignore the influence of the smear component, so R-constant...・・・・・・・・・ (3)B
= constant ・・・・・・・・・・・・ (4). On the other hand, even field (second field)
Since the green (G) or white (W) pixel signal C that corresponds to , contains a smear component as described above, the smear component Q is included in the true pixel signal G".

It is the sum of

G = C”+ Q8・・・・・・・・・・・・
(5) Furthermore, by substituting the above equation (2) into the above equation (5), c=c''+α, XTs...
(6), and by modifying the above formula (6), G'' that does not include the smear component is G'' = G - α, XTs... (
7). Here, α is an appropriate coefficient,
Therefore, the actually obtained line or white pixel signal (G)
By subtracting a value proportional to the shutter speed Ts from , a line or white pixel signal without a smear component can be obtained. In the present invention, this coefficient α is used as a red (R) or blue (B) pixel signal G” = G−kxRxTs---
-(8) or G"=G-, xBxTs...-old++ A smear removal operation based on (9) is performed. Note that k is an appropriate coefficient.

In this way, as is clear from the above equations (8) and (9), the present invention assigns a value proportional to the shutter speed Ts to the red and blue pixel signals corresponding to the first field, and applies a value proportional to the shutter speed Ts to the red and blue pixel signals corresponding to the first field. The basic principle is to eliminate the smear component by subtracting it from the value G of the pixel signal corresponding to Niremen[. Note that pixel signals of mutually adjacent light-receiving elements are used as pixel signals for performing these calculations.

For example, to remove the smear component included in the green signal G(i+l+J) in row y=i-)-1 and column x=j in FIG. 2(A), the above equation (8) or (9 ), based on G”(
i+1. j)=G(i+1.j) −k
j) XT s or G”(i+1.j) −〇(i+1.j) −k X
Processing is performed by the calculation of B(i+2.j)×Ts. Also, for example, y=i+1 row, x=j-1 in FIG. 2(A)
In order to remove the smear component included in the green signal G(1+11J+1) of the -1 column, similarly based on the above formula (7) or (8), G ''(i+1.jN) = G(i+Lj
+1) -k X R(i+2.j+1) X T s
Or, G"(i+1.j+1)=G(i+l,j+1)-k
Processing is performed by xB(i,j+1)xTs operation. Then, by performing similar processing on other pixel signals, pixel signals with the smear component removed as shown in Figure 2 (B) from the pixel signals arranged as shown in Figure 2 (A) are formed. do.

In addition, in the principle explanation based on the above equations (13 to (9)), red and blue light receiving elements are placed in the first field where the first signal is read out, and green or white light receiving elements are placed in the second field where the signal is read out next. Although we have explained the case where the light receiving elements are arranged, it is possible to arrange the light receiving elements in the opposite arrangement, that is, placing the green or white light receiving elements in the first field, and then placing the red and blue light receiving elements in the second field where the signal is read out. The principle of the present invention can also be applied to the case where they are arranged, and in this case, the relationships corresponding to the above equations (8) and (9) may be reversed.

Furthermore, smear occurs due to the time difference caused by field readout even in a pseudo-frame electronic shutter that produces a TI image of black and white images without providing a color filter. This smear component is removed by replacing it with the pixel signal of G) and applying the processing of the present invention to each pixel signal.

[For production]

According to the present invention based on such a processing method, a pixel signal of a field that does not include a smear component is multiplied by a coefficient that is inversely proportional to the shutter speed, and this multiplied value is subtracted from a pixel signal that includes a smear component. Therefore, it is possible to form a pixel signal from which smear components have been removed, and a clear image without flicker or line crawl can be provided.

〔Example〕

Hereinafter, an electronic still camera to which the signal processing method of the present invention is applied will be described as an example with reference to the drawings.

In this example, the FIT-CCD applied to the pseudo electronic shutter of the electronic still camera has a structure similar to that shown in FIG. It is assumed that the field reading is performed according to the field reading shown in FIG.

First, to explain the configuration, in FIG. 3, the pixel signal S0 output from the FIT-CCD by shutter operation similar to the conventional one is transmitted through a preamplifier, a white balance circuit,
Signal output system 5 with built-in T correction circuit, A/D converter, etc.
The digital image data is converted into digital image data and stored in the frame memory 6.

The frame memory 6 holds pixel data for one frame by storing pixel data corresponding to odd fields and pixel data corresponding to even fields in a predetermined storage area in a non-interlaced manner corresponding to the pixel arrangement. .

7 is a smear removal circuit, which converts each pixel data corresponding to one frame stored in the frame memory 6 into the above equations (7) to
(9) Perform the arithmetic processing based on the appropriate second one, and store the processed pixel data in the frame memory 6 again.

A matrix circuit 8 generates a well-known luminance signal Y and a color difference signal RY based on the pixel data after smear removal processing stored in the frame memory 6.

Form B-Y.

FIG. 4 shows the internal configuration of the smear removal circuit 7.

That is, 9 is a variable multiplication circuit that multiplies pixel signals R and B that do not include smear components (in this example, the case is shown in which red and blue light receiving elements are arranged in the first field) by the shutter speed Ts. [Formula (7), (8) above]
), and outputs the multiplication value data D. IO is a subtraction circuit, which subtracts the multiplication value data D from the green pixel signal G, and outputs GD as true green data G with no smear component. Data R/B corresponds to mutually adjacent rows. Also, the above formula (8),
The coefficient k in (9) may be switched for each pixel signal of the red light-receiving element and the blue light-receiving element.

Further, the smear removal circuit shown in FIG. 4 may be an analog circuit or a digital circuit, or may be configured to perform arithmetic processing using a computer program.

〔Effect of the invention〕

As explained above, according to the present invention, a pixel signal of a line that does not include a smear component is multiplied by a coefficient proportional to the shutter speed, and this multiplied value is subtracted from a pixel signal that includes a smear component. It becomes possible to form a pixel signal with smear components removed, and a clear image without flicker or line crawl can be provided.

In addition, smear removal should be performed not only for pseudo-frame electronic shutters for color imaging in which color filters are arranged, but also for pseudo-frame electronic shutters that form luminance signals by performing addition operations for photographing black and white images. I can do it.

Furthermore, for example, the processing method is not limited by the positional relationship of the red, blue, and green (or white) light-receiving elements;
As described above, the smear component can be easily removed by simply multiplying the pixel signal of a line that does not include the smear component by a coefficient proportional to the shunter speed, and then subtracting this multiplied value from the pixel signal that includes the smear component. Because it is possible to
It has an extremely wide range of applications and exhibits excellent effects.

[Brief explanation of drawings]

1 and 2 are detailed explanatory diagrams of the present invention; FIG. 3 is an explanatory diagram of the configuration of one embodiment; FIG. 4 is a block diagram showing the configuration of the smear removal circuit in FIG. 3; is an explanatory diagram of the configuration of FIT-CCD; Figure 6 is an explanatory diagram of the FI
Timing chart for explaining the pseudo electronic shutter operation of the T-CCD; FIG. 7 is an explanatory diagram for explaining the cause of smear generation. Symbols in the figure: 1; Light receiving area 2; Accumulation area 3; Horizontal transfer path 4; Output amplifier 5; Signal output system 6; Frame memory 7; Smear removal circuit 8; Matrix circuit 9; Variable multiplication circuit lO; Subtraction circuit Agent: Patent attorney (8107) Kiyotaka Sasaki (3 others) No. 5 Figure 1: Ill filter i: Red filter
-1 ; Blue filter 2nd figure Frame data before processing tm1 1st figure Shutter speed (seconds) Frame data after processing (B)

Claims (1)

[Claims] In a smear removal method for a pseudo-frame electronic shutter that removes smear components contained in pixel signals output from a pseudo-frame electronic shutter using a charge-coupled solid-state imaging device using a frame interline transfer method, A multiplication value obtained by multiplying the pixel signals corresponding to the light receiving elements arranged in one horizontal line by a predetermined coefficient proportional to the shutter speed, and a multiplication value obtained by multiplying the pixel signals corresponding to the light receiving elements arranged in the second horizontal line adjacent to the light receiving elements A method for removing smear from a pseudo frame electronic shutter, characterized in that a difference from a corresponding pixel signal is used as a new pixel signal corresponding to a light receiving element arranged in the second horizontal line.