JP2000023040A - Solid-state image pickup device and system on-chip solid- state image pickup device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent image quality deterioration by suppressing fluorescent lamp flicker occurring in a striped horizontal pattern shape when a photograph is taken with a MOS image pickup device under fluorescent lamp lighting. SOLUTION: An output waveform of a photometric element 103 and electronic shutter value that is an output of an electronic shutter setting part 104 are inputted to a control value generating part 105 and gain control value is produced. And, fluorescent lamp flicker is suppressed by such a manner that a gain control part 106 controls the gain of a video signal that is successively read from an image pickup device 101 by a horizontal scanning part 203 in every horizontal line.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a solid-state image pickup device using an XY address type image pickup device and a system-on-chip (hereinafter referred to as SOC) type solid-state image pickup device integrated into one chip. The present invention relates to an improvement in so-called flicker in which the brightness and hue of an imaging screen change under an electric lamp that is AC-lit such as a fluorescent lamp that blinks.

[0002]

2. Description of the Related Art When an image is taken by a television camera under illumination of a fluorescent lamp, interference occurs due to a difference between a vertical synchronization frequency of the television camera and a blinking frequency of the fluorescent lamp, which is called a fluorescent lamp flicker. The phenomenon occurs. C
In a camera using a CD-type imaging device, charge is accumulated in units of -frame or -field. For this reason, the influence of flicker occurs between frames, and its correction is relatively simple and has already been put to practical use.

[0003] Taking a progressive IT-type CCD image sensor as an example, the IT-type CCD image sensor has an analog frame memory on an element. The entire screen of the signal exposed in the same period by the photodiode is transferred to the vertical transfer CCD on the element, and is read out one horizontal line at a time. For this reason, when shooting is performed by using an NTSC camera using an IT-type CCD image sensor with 50 Hz alternating-current lighting, the luminance and hue of the entire screen change at a cycle of approximately three fields.

As a method of correcting such flicker in an IT type CCD image sensor, a method of setting an exposure time to 1/100 second in an electronic shutter mode to equalize an exposure time for each field, and a method of almost three flickers. Utilizing the fact that it occurs in the field cycle, the current luminance and hue fluctuations are predicted from the video signal three fields before, so that the average value of the video signal of each field is constant, and a correction value is generated to generate flicker. Suppression techniques are used.

On the other hand, the MOS type imaging apparatus is of an XY address scanning type in which a MOS transistor is provided as a switching element for each pixel, and the pixels are sequentially scanned and read out by XY addresses. Since such a MOS type imaging device can be driven by a single power supply voltage, consumes low power, and can be manufactured by the same process as a normal MOS type logic IC, unlike a CCD type imaging device. There is an advantage that the manufacturing cost is low.

However, in this MOS type imaging device, the exposure period sequentially moves by one horizontal read clock cycle for each pixel. For this reason, the timings at which all the pixels are exposed are different. Therefore, there is a difference in the amount of fluorescent lamp illumination that each pixel integrates during the accumulation period, and this appears as flicker in the frame. FIG. 24 shows a flicker generation model in a MOS imaging device having a structure having the same exposure timing in the scanning direction for simplification. The flicker occurs due to a difference between the blinking cycle of the fluorescent lamp and the exposure timing. Therefore, C
Unlike CD type imaging device, power line cycle is 50H
It occurs in both cases of z and 60 Hz.

[0007] Here, assuming a MOS type image pickup device that collectively reads one horizontal line at a time, in this case, since the exposure period on the horizontal line is the same, the horizontal stripe fluorescent lamp flickers in the vertical direction. appear. As one means for correcting the fluorescent lamp flicker, as shown in FIG. A method is conceivable in which the exposure time is set to one cycle of the flicker cycle of the fluorescent lamp or an integral multiple thereof. When the accumulation time is set in this manner, even when the fluorescent lamp is flickering, the integrated amounts of the fluorescent lamp components in all the pixels are the same, so that the fluorescent lamp flicker can be suppressed.

[0008] However, as shown in FIG.
If the accumulation time is different from this, as shown in FIG. 24, the integrated value of the amount of light received for each line differs, and a horizontal stripe fluorescent lamp flicker occurs. As a correction method when the exposure time is arbitrarily set, a periodic variation pattern of the illumination light amount is extracted from the output of the image sensor or a video signal obtained by signal processing the output, and a flicker correction value is generated from the variation pattern. Thus, there is a method of controlling the gain of the luminance signal or the color signal for each location on the screen to correct flicker. Alternatively, a method has been proposed in which a flicker waveform is stored as a template in a memory, and flickering of a fluorescent lamp is detected by an external sensor, and correction is performed by synchronizing the template with the image sensor output.

However, it is generally very difficult to detect a flicker waveform in a frame from a video signal, except when photographing a white wall or the like. In order to extract a flicker component in a frame from a video signal, a method of removing a component other than the flicker component using an analog or digital low-pass filter or an analog or digital low-pass filter is used. However, a pattern in the same frequency band as the flicker component is often mixed in the photographing subject, which hinders accurate flicker waveform extraction. In particular, when the angle of view of the imaging device is fixed, the moving cycle of flicker within the screen is approximately three frames. For this reason, even if the average is taken between the frames, the flicker waveform comes at almost the same place every three frames, so that the influence of the picture cannot be removed.

[0010] The flicker waveform of the fluorescent lamp is used as a template, and it is known that the blinking waveform of the fluorescent lamp is unique to each fluorescent lamp lighting device. For this reason, even if a very large number of flicker waveforms are investigated and a database is generated in order to construct a flicker correction system, it is not possible to cope with future fluorescent lighting devices.

As described above, since the extraction of the flicker waveform cannot be easily predicted in the MOS type image pickup device as in the case of the CCD type, there is no effective correction means so far, and the fluorescent lamp flicker remains, resulting in image quality. However, there is a problem in that the metal is deteriorated.

[0012]

As described above, in the prior art, X-Y which performs reading with different accumulation periods on a line-by-line basis.
When an image is picked up under fluorescent lamp illumination using a MOS type image pickup device based on the address method, the exposure amount differs for each location on the image pickup surface, and there is a problem that so-called fluorescent lamp flicker occurs.

In view of the above circumstances, the present invention sufficiently suppresses horizontal stripe fluorescent lamp flicker in which luminance and hue change in the vertical direction when imaging is performed for an arbitrary storage time under illumination of a fluorescent lamp to improve image quality. X that can be improved
An object of the present invention is to provide a solid-state imaging device and an SOC type imaging device of a -Y address system.

[0014]

According to the present invention, there is provided a solid-state imaging device comprising: an area sensor section in which pixels for performing a photoelectric conversion to generate a pixel signal are two-dimensionally arranged; A first scanning unit that sequentially selects a line in units of a first direction in a first direction, a second scanning unit that sequentially selects the pixel signal in units of a line of the first direction in a second direction, An X-Y address type image sensor having a signal processing unit that performs predetermined signal processing on the pixel signal scanned by the second scanning unit and read from the area sensor unit and outputs a video signal; A photometric element that performs photoelectric conversion to output a light quantity waveform, a control value generation that generates a control value for controlling a gain of the video signal based on the exposure period of the imaging element and the output light quantity waveform. And the control value generation There based on the generated control values, and further comprising a gain control unit for controlling a gain of said video signal.

Here, the solid-state imaging device further includes an analog-to-digital conversion unit that digitizes the video signal output from the imaging device, and the gain control unit converts the digitized video signal into a required signal. Signal processing may be performed to obtain a digital video signal in a signal format, and the gain of the digital video signal may be controlled based on the control value.

The control value generating section integrates the light quantity waveform output by the photometric element over an effective charge accumulation period for each line of the area sensor section, and performs a required process on the obtained integrated value. The control value may be generated.

The solid-state imaging device further includes a signal level measurement unit for measuring an average signal level of the video signal output from the imaging device, and the control value generation unit is configured to measure the signal level by the signal level measurement unit. The control value may be generated using the averaged signal level.

Further, the control value generation section generates the control value for each line, and outputs the control value to the gain control section before the video signal of the corresponding line is output from the image pickup device. May be given.

The solid-state imaging device according to the present invention may further include a reference table for preliminarily holding an amplitude of a flicker component included in the video signal output from the area sensor unit. The output light quantity waveform is integrated over the effective charge accumulation period for each line of the area sensor unit, and the amplitude of the obtained integrated waveform matches the amplitude of the flicker component included in the video signal held by the reference table. The gain can also be controlled so that

A low-pass is provided with the light amount waveform output from the photometric element, performs a process of passing only a signal band depending on a power supply cycle in the light amount waveform, and outputs the signal band to the control value generation unit. A filter may be further provided.

As the photometric element, a plurality of photometric elements in which different color filters are arranged can be provided.

Here, the number of the photometric elements is the same as the number of types of color filters provided in the pixels for obtaining color image information in the area sensor section, and the color filters arranged in the photometric elements are The color may be the same as the color filter arranged in the area sensor unit or a color complementary thereto.

In this case, the control value generation section separates the light amount waveform output from the photometric element for each color, generates the control device from the light amount waveform for each color, and separates the control device. The white balance can also be corrected according to the ratio of the output level for each color.

The light amount waveform output from the photometric element is applied to the control value generation section to generate the control value, and the control of the light amount input to the area sensor section or the accumulation of the area sensor section is performed. It may be used for controlling time.

The image processing apparatus further comprises a read area setting section for setting a pixel area from which the pixel signal is read from the area sensor section in accordance with a command input from the outside, wherein the control value generation section includes a pixel set by the read area setting section. The control value may be generated based on a region.

The SOC type solid-state image pickup device according to the present invention has an area sensor section in which pixels for generating a pixel signal by performing photoelectric conversion are two-dimensionally arranged, and the area sensor section is defined as a line in a first direction. A first scanning unit that sequentially selects in the first direction, a second scanning unit that sequentially selects the pixel signals in units of lines in the first direction in a second direction, and the first scanning unit.
And a signal processing unit that performs predetermined signal processing on the pixel signal scanned by the second scanning unit and read from the area sensor unit, and outputs a video signal.
An image sensor of an address method, an interface unit to which a light amount waveform is externally given, an exposure period of the image sensor,
Based on the light amount waveform given to the interface unit, a control value generation unit that generates a control value for controlling the gain of the video signal, based on the control value generated by the control value generation unit, A gain control unit for controlling the gain of the video signal, the image sensor, the interface unit,
The control value generation unit and the gain control unit are formed on the same chip.

Alternatively, the SOC type solid-state imaging device according to the present invention includes an area sensor unit in which pixels that generate a pixel signal by performing photoelectric conversion are two-dimensionally arranged, and the area sensor unit is formed of a line in a first direction. A first scanning unit for sequentially selecting a unit in the first direction as a unit, a second scanning unit for sequentially selecting the pixel signal in a second direction in units of the line in the first direction, and the first and second units A signal processing unit that performs predetermined signal processing on the pixel signal read from the area sensor unit by scanning by the scanning unit and outputs a video signal; A photometric element that performs a conversion to output a light amount waveform, an exposure period of the image sensor, and a control value generation unit that generates a control value that controls a gain of the video signal based on the output light amount waveform. , The control value raw A gain control unit that controls the gain of the video signal based on the control value generated by the unit, wherein the imaging element, the photometric element, the control value generation unit, and the gain control unit are formed on the same chip. It is characterized by having been done.

[0028]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

The XY address type solid-state imaging device according to the first embodiment of the present invention, as shown in FIG.
Image sensor 101, timing generator 107, photometric device 1
03, electronic shutter setting unit 104, control value generation unit 10
5. The gain control unit 106 is provided.

The image sensor 101 has an area sensor unit 201 in which pixels that receive light, perform photoelectric conversion, generate signal charges, and output the signals are arranged in a matrix, and an analog pixel signal output from the area sensor unit 201. Is transferred in the vertical direction and temporarily held, and an analog signal processing unit 204 that performs processing such as noise reduction, a vertical scanning unit 202 that vertically scans the area sensor unit 201, and a horizontal scanning unit that scans the area sensor unit 201 in the horizontal direction Scans the analog signal processing unit 20
4 for outputting the signal held in 4 as a video signal, and a vertical / horizontal scanning signal for controlling the operations of the vertical scanning unit 202 and the horizontal scanning unit 203 given the timing signal output from the timing generation unit 107. Generator 2
05.

The electronic shutter setting section 104 receives an output from the analog signal processing section 204 as a feedback signal, sets an electronic shutter value, and outputs the electronic shutter value to the timing generation section 107 and the control value generation section 105.

The timing generator 107 generates a clock signal for defining the timing for scanning and reading the pixel signal generated in the area sensor 201 in the vertical and horizontal directions based on the given electronic shutter value. It is.

The photometric element 103 is arranged around the image sensor 101, measures a light amount waveform around the image sensor 101, and outputs a measurement result to the control value generator 105.

The control value generation unit 105 is supplied with the electronic shutter value output from the electronic shutter setting unit 104, the clock signal output from the timing generation unit 107, and the light amount waveform output from the photometric element 103, and performs processing as described later. And outputs a control value signal for controlling the gain.

The gain control section 106 receives the control value signal and controls the gain of the output from the image sensor 101.

The solid-state imaging device according to the present embodiment having such a configuration operates as follows. In the area sensor unit 201, the pixel signals obtained by the photoelectric conversion are selected by the vertical scanning unit 202 in units of one vertical line, transferred to the analog signal processing unit 204, and temporarily held. After undergoing processing such as noise reduction in the analog signal processing unit 204, the signals are sequentially selected in units of one horizontal line by the horizontal scanning unit 203 and output as video signals to the outside of the image sensor 101. In this way, after the image information of the area sensor unit 201 is transferred and held in the vertical direction, it is output for each horizontal line and the video signal for one screen is read. Here, the analog signal processing unit 204 may perform processing such as gamma correction and amplification in addition to noise reduction.

The photometric element 103 measures a light quantity waveform around the image pickup element 101, and this waveform is used as a control value generation unit 10.
5 is output. As described above with reference to FIG.
The flicker of the S-type imaging device is caused by flickering of the illumination due to AC lighting and a shift in the accumulation period of the signal charges of the area sensor unit 201. Therefore, by integrating the light amount waveform measured by the photometry element 103 based on the electronic shutter value set by the electronic shutter setting unit 104, the same waveform as the vertical component of the horizontal stripe flicker appearing in the image is obtained by the control value generation unit. 1
05 can be estimated.

For the estimation of the flicker waveform, FIG.
Several schemes as shown in (c) are possible. In the first method, as shown in FIG.
The integration of the output of the photometric element 103 is started in accordance with the timing at which the first horizontal line of the first horizontal line starts accumulating signal charges, and after integrating for the accumulation period set in this first row, the integrated value is obtained. Save I ₀ in the buffer. Thereafter, the integral of the output of the photometric device 103 waits until the timing of the accumulation start of the second row defined by the electronic shutter value starts to store the integrated value I ₁ in the buffer. By repeating such an operation, the flicker waveform can be estimated. In this method, measurement of flicker values of a plurality of rows cannot be performed in parallel. Therefore, it is necessary to prepare flicker waveform data before performing flicker correction.

In the second method, as shown in FIG. 2B, the output from the photometric element 103 is temporarily stored in a buffer, and the flicker phase θ _t at the time of starting accumulation in each row of the image sensor. from only the data accumulation period a _t which is determined by the current of the electronic shutter value performs integral operation is read from the buffer, and sequentially outputs each line flicker component.

In the third method, as shown in FIG. 2C, the output from the photometric element is temporarily stored in a buffer, and data of a period determined by the electronic shutter value in each row is read from the buffer and integrated. after determining a plurality of the integrated value and calculate the average _{Σ x i n (x) /} x. According to this method, it is not possible to estimate the flicker component in real time, but it is possible to improve the estimation accuracy of the waveform.

Using the flicker waveform estimated by any of the above methods, the control value generation unit 105 generates flicker suppression data for correcting fluctuations in signal level due to flicker. This data is the gain control unit 1
06 to control the gain. Control value generator 1
At 06, as shown in FIG. 3, it is necessary to send the flicker suppression control value for each line to the gain control unit 106 before the horizontal read period in which the video signal is read from the image sensor, and finish the gain setting. . FIG. 2 (b)
When the flicker waveform is measured by the second method described with reference to the above, the generation of the flicker correction value and the setting of the gain control value need to be completed between the vertical reading and the horizontal reading.

The gain control unit 106 includes a control value generation unit 105
By changing the gain of the video signal read from the image sensor 101 according to the gain value for each line set in the above, flickers in the form of horizontal stripes are removed.

Here, FIG. 4 shows an example of specific processing blocks included in the control value generation unit 105. The control generation unit 105 includes an integrator 301, a signal level measurement unit 309,
It has a level correction unit 302, a signal processing unit 307, and a level inversion unit 303.

The integrator 301 of the control value generator 105 includes:
As described above, the light amount waveform around the imaging device 101 or the solid-state imaging device measured by the photometry device 103, the electronic shutter setting value set by the electronic shutter setting unit 104,
The clock signal output from the timing generator 107 is input. The video signal output from the image sensor 101 is input to the signal level measurement unit 309 to measure the signal level.

The integrator 301 integrates the photometric element waveform and estimates the flicker waveform. Thereafter, the signal level measuring unit 309 measures the average signal level of the video signal,
Level correction section 302 adjusts the gain of the integrated waveform. This gain adjustment is due to factors not under the control of the solid-state imaging device,
For example, the correction is performed to correct a shift between a video signal level caused by a difference in the amount of incident light due to a shift in a set position between an image pickup surface and a photometric element and an integrated photometric element signal level. Although the signal processing unit 307 is not always necessary, the signal processing unit 30
7 to perform auxiliary signal processing, for example, to cope with a decrease in peripheral light amount by a lens, so-called shading, and to perform averaging with a waveform measured in a previous frame to increase the accuracy of an estimated waveform. Generation of a flicker component appearing therein may be performed.

Alternatively, before starting the photographing, a white surface is photographed in advance, and the flicker waveform in the actual image and the photometry element 103 are measured.
It is also possible to measure the difference between the level of the output and the output waveform of the integral calculation and store the difference in a table, and correct the integrated signal level in the signal processing unit 307 in the control value generation unit 105.

After performing such processing, the level inverting section 303 outputs the flicker waveform F as shown in FIG.
(t) so that the flicker suppression waveform S
By generating (t) and using this S (t), the gain control unit 106 suppresses flicker. At this time, F (t) and S
The relationship of F (t) × S (t) = constant does not hold between (t).

As described above, according to the present embodiment, in a solid-state imaging device based on the XY address system in which reading is performed with different exposure periods on a line-by-line basis, a photometric element provided near the imaging device is used to control the fluorescent lamp. By measuring the flicker waveform, generating a control value that cancels out this waveform, and controlling the gain of the video signal output from the exposure element,
It is possible to suppress horizontal stripe-like flicker in which luminance and hue change in the vertical direction, and prevent image quality from deteriorating.

The second embodiment of the present invention has a configuration as shown in FIG. 6 and is characterized in that a flicker sensor 308 is provided outside the image pickup apparatus. In the configuration shown in FIG. 1 or FIG. 4 as the first embodiment, the control value generation unit 105 has a function of integrating the output of the photometric element 103. However, in the present embodiment, the integrator 301 and the photometric element 103 are combined, and a flicker sensor 308 that can be given an electronic shutter value from the electronic shutter setting unit 104 of the solid-state imaging device is provided as an external element. This embodiment is the same as the first embodiment except that an integrator 301 for integrating the output of the photometric element 103 is provided in the flicker sensor 308, and the description of the operation is omitted.

The third embodiment of the present invention is characterized in that a digital signal processing unit 401 for estimating a flicker component is provided as shown in FIG. After the video signal output from the image sensor 101 is digitized by the A / D converter 402b, the digital signal processing unit 401
Is input to The digital signal processing unit 401 performs processing such as color correction, resolution conversion, and compression on the video signal as necessary. In the present embodiment, the digital signal processing unit 4
01, the output of the digitized photometric element 103 is A /
A flicker correction unit 404 that performs flicker component estimation, generation of a correction value, and flicker correction by controlling the amplitude of a video signal by using the digital value obtained by the D converter 402a and the electronic shutter value. Has been realized.

FIG. 8 shows a configuration of a solid-state imaging device according to a fourth embodiment of the present invention. This embodiment is characterized in that a low-pass filter (LPF) 305 is inserted after the photometric element 103 in the first embodiment. In recent years, fluorescent lamps have become the mainstream of inverter types that use an alternating current of 50 or 60 Hz instead of using the alternating current of the lamp as it is, and increase the frequency of the alternating current of the lamp to about 50 kHz by an inverter circuit. The inverter circuit used in such a fluorescent lamp first rectifies the AC of the electric lamp and converts it to DC to form a high-frequency lighting waveform. However, since the rectification is not sufficient, it is known that the waveform has amplitude modulation as shown in FIG. There is no problem when the amplitude of the modulation component is sufficiently small. However, when this amplitude is about several percent, it appears as flicker that can be detected by human eyes. Therefore, in the present embodiment, the high-frequency component is removed from the output of the photometric element 103 using the LPF 305. When a flicker component is still recognized in the output of the LPF 305, flicker correction is performed by the control value generation unit 105 and the gain control unit 106 at the subsequent stage in the same manner as in the first embodiment.

In the first to fourth embodiments, the number of photometric elements 103 is not particularly mentioned. But,
The persistence characteristics of the phosphor used in the fluorescent tube differ depending on the color. For this reason, the flicker waveform appearing in the video differs for each color. To correct this difference, prepare the necessary number of photometric elements with color filters, or calculate all the colors from the output of one photometric element. It is necessary to estimate the flicker waveform of the component.

The fifth embodiment of the present invention includes three photometric elements 701 to 703 as shown in FIG. Here, it is assumed that green, blue, and red color filters are used in the area sensor unit 201. In such a case, green, blue, and red color filters are added to the three photometric elements 70.
1 to 703 to obtain a fluorescent lamp waveform of each color.

The control value generation unit 105 corrects the signal level of each color with respect to the video signal output from the analog signal processing unit 204 using the sensitivity correction table 304, and then sends the corrected signal level to the built-in signal level measurement unit. The gain correction is performed based on the signal level average value measured by the above. Further, the control value generation unit 105 generates a control signal for performing flicker correction by performing required signal processing, and outputs the control signal to the gain control unit 106.

Here, the sensitivity correction table 304 is used as follows. The signal level of each color is generally different from the output level of the area sensor unit 201 due to the structure of the photometric elements 701 to 703, basic characteristics, or variations in characteristics among individuals. Therefore, before the start of the photographing, the photometric element 7
The sensitivity ratio between 01 to 703 and the area sensor unit 201 is measured for each color, recorded in the sensitivity correction table 204, and read out and used as needed when correcting the signal level.

When the control signal is supplied to the gain control unit 106, the correction is performed while synchronizing the phase with each color in the analog video signal output from the analog signal processing unit 204 and the line position in the image. In the present embodiment, since the correction is performed on the analog signal, the phase adjustment in the control value generation unit 105 is complicated.

On the other hand, in the sixth embodiment of the present invention shown in FIG. 11, the digital signal processing unit 401 has the flicker correction unit 404. This flicker correction unit 40
In step 4, since the control value generation and the gain control for each pixel color and position are performed, the processing can be performed relatively easily.

In the fifth and sixth embodiments, three color filters of green, blue and red are used in the area sensor section.
Further, three color filters of green, blue and red are similarly used for the photometric element. However, the present invention is not limited to this, and a complementary color red-green color filter may be used for one of the area sensor unit and the photometric element, or a red-green color filter may be used for both.

FIG. 12 shows a configuration of an apparatus for correcting a color image using one photometric element 103 as a seventh embodiment of the present invention. Here, two methods can be considered as a method of correcting a color image with one photometric element 103.

The first method divides a color video signal into luminance and color difference, and detects a fluorescent lamp flicker waveform by a photometric element 103 set to detect a white illumination waveform to correct the luminance signal. I do. However, in this method, the afterglow characteristics of the phosphor used in the fluorescent tube are different for each color. Therefore, even if a white surface is photographed, the valley portion of the flicker waveform, that is, the emission of the fluorescent lamp is not emitted. At the time of disappearing, the image is colored.

Therefore, here, the afterglow characteristics of the colors of the respective phosphors are stored in the characteristic table 306, and the color difference components are corrected at the valleys of the flicker waveform. The fluorescent material used in the fluorescent tube has almost the same characteristics as those used by any fluorescent tube manufacturer. Therefore, it is easy to measure the characteristics in advance and store the characteristics in the characteristic table 306.

The second method estimates a flicker waveform of another color from a flicker waveform of a single color. First, a fluorescent lamp waveform of any color is measured, and a fluorescent lamp waveform of another color is estimated from the waveform and data of the characteristic table 306 in which the emission characteristics of each color of the phosphor are recorded. Is used to correct flicker. Here, the color of the fluorescent lamp waveform is measured. However, since the fluorescent light for home use and office use has a high color density, the amplitude of blue light is larger than other colors, and flicker of this color is conspicuous. Therefore, the blue filter is connected to the photometric element 103.
Preferably used above. Then, it is necessary to estimate a flicker waveform of another color from the measured waveform, and to know in advance how much the waveform of each color has in the image being captured. For example, it is conceivable that a white wall or the like is photographed in advance before photographing, and the flicker amplitude of each color is obtained.

As an eighth embodiment of the present invention, an example will be described in which the photometric element 103 is used not only for flicker correction but also for other purposes.

The present embodiment has a configuration as shown in FIG.
By setting the value to 1, it is possible to set the amount of light suitable for shooting. Alternatively, as in the ninth embodiment of the present invention shown in FIG. 14, the output of the photometric element 103 may be given to the electronic shutter setting unit 104 to set the light amount. Alternatively, by combining the eighth and ninth embodiments, the output of the photometric element 103 may be given to both the lens aperture 501 and the electronic shutter setting unit 104 to set the light amount.

Next, a third embodiment shown in FIG.
Alternatively, FIG. 15 shows an example of the configuration of the digital signal processing unit 401 in the sixth embodiment shown in FIG. The digital signal processing unit 401 includes a white balance correction unit 403 in addition to the flicker correction unit 404 that performs the above-described flicker correction and the video signal processing unit 405 that performs processing of a video signal. That is, the output from the photometry unit 103 is used not only for flicker correction in the flicker correction unit 404 but also for white balance in the white balance correction unit 403.

In recent years, the white balance is corrected by estimating the color balance from a captured image without performing external photometry from the viewpoint of reducing the number of parts. However, in the flicker correction system according to the above-described embodiment using the photometric element 103, the white balance correction can be accurately performed and the overall configuration of the digital signal processing unit 401 is simplified by sharing the photometric element 103 with the flicker correction. be able to.

FIG. 16 shows, as a tenth embodiment of the present invention, an image pickup apparatus having a plurality of resolution modes or an image pickup apparatus which performs image stabilization and flicker correction by changing the cutout position of an image from an area sensor. 1 shows a configuration of an apparatus. Externally provided interface (IF) unit 13
A command is input to the readout area setting unit 108 via “01”, and an image readout area is set according to the command. Information about the set readout area is given to the vertical / horizontal scanning signal generation unit 205 of the image sensor 101, and the timing of cutting out an image is set. at the same time,
Information about the read area is provided to the control value generation unit 105, and a gain control value in the read area is generated using this information as described later.

In recent years, solid-state imaging devices have been widely used up to digital still cameras, PC cameras, and video cameras, and various image formats have been defined for each application. The image format used for still images for PCs is QVGA (320 × 24
0), VGA (640 × 490), SVGA (800 ×
600, 1024 × 768, 1280 × 1024) and the like. In the video conference, QCIF
(176 × 144), CIF (352 × 288), etc.
It has been adopted as a TU standard.

However, instead of using different image sensors for each of these different formats, it is common to use one image sensor for resolution modes that can be included. For example, an image sensor used for the VGA format is also used for the CIF format by using only the central region of the area sensor unit. Alternatively, when the resolution may be further reduced, sub-sampling can be performed to support the QCIF format and the QVGA format.

As described above, when a plurality of resolution modes are realized by one image sensor, the start position of the image in the vertical direction differs for each mode. Therefore, when performing flicker correction, it is necessary to perform phase matching in accordance with the mode.

As shown in FIG. 17A, the estimation of the flicker waveform assumes that the entire area of the area sensor unit 201 of the image sensor 101 is used. Therefore, in a mode in which a part of the imaging area is used, it is necessary to cut out a required flicker waveform as shown in FIG. 17B, or to subsample the flicker waveform in accordance with the subsampling mode.

When the analog signal processing unit 204 performs such flicker waveform cutout processing and flicker waveform sub-sampling processing, as shown in FIG. Part 10
The correction start pulse and the correction end pulse are sent to 5, and the timing for cutting out the waveform is adjusted to perform the correction.

However, such processing can be performed using a digital signal processing unit 401 as shown in the third or sixth embodiment instead of the analog signal processing unit 204. In this case, by sending the read position and the resolution information from the read area setting unit 108 to the digital signal processing unit, it is possible to cope with the selected resolution mode.

For example, when the CIF mode is selected by using an image sensor having a VGA resolution, a predetermined portion of the correction data buffer is read out as shown in FIG. . Also Q
In the case where the CIF mode is selected and a predetermined area in the area sensor unit 201 of the image sensor 101 is sub-sampled, this is indicated as “a case where the predetermined area is cut out and sub-sampled at a low resolution” in FIG. So, the data in the central part is subsampled and read out,
Used for flicker correction.

The means corresponding to a plurality of resolution modes has been described above. However, even when flicker correction is performed in an image pickup apparatus that performs camera shake correction, flicker correction can be similarly performed. Such a device performs image stabilization by switching an image cutout position using an image sensor having an imaging area wider than an imaging region required for image output. Flicker correction can be realized by a configuration similar to that of the device having a plurality of modes described above.

In the first to tenth embodiments described above,
The configuration required for flicker correction is provided in an element of the image sensor 101 different from the area sensor unit 101. But,
As in the SOC type image sensor according to the eleventh embodiment described below, a component for flicker correction can be incorporated on the same chip as the area sensor unit 101.

In recent years, C, which is a type of MOS image pickup device,
In a MOS image sensor, research and development for forming a peripheral circuit on the same chip as the image sensor have been actively conducted. This is different from the CCD type image pickup device.
The OS type image sensor uses the same MO for the area sensor and the peripheral circuit.
This is because it can be manufactured by the manufacturing process of the S transistor, and the commercialization of the one-chip camera is expected in the future.

An example of the SOC type image sensor in which the peripheral circuit is incorporated on the image sensor as described above will be described. 18, 19, 2
0, the eleventh, twelfth, and thirteenth of the present invention, respectively.
In the embodiment, the flicker correction function blocks other than the photometric element 103 are provided as one-chip image pickup elements 1001, 1002,
1003 respectively.

The eleventh embodiment shown in FIG.
This is an example of an SOC type image sensor having an analog signal processing function on the same chip as the image sensor. Analog data output from the photometric element 103 provided outside is input from a dedicated port 1201 and is input to the electronic shutter setting unit 10.
The integrator 301 integrates based on the electronic shutter value set by 4 to estimate a flicker waveform. The estimated flicker waveform is provided to a control value generation unit 105 that generates a correction coefficient and a gain control value, and performs appropriate signal processing to generate correction data. Then, based on the pulse output from the timing generator 107, the gain controller 106 performs flicker correction by changing the gain in synchronization with the video signal output from the analog signal processor 204.

The twelfth embodiment shown in FIG.
This is an example of an SOC type image sensor in which a digital signal processing unit 401 for performing flicker correction is incorporated on the same chip as the image sensor. Waveform data output from the photometric element 103 provided outside is input from the dedicated port 1202 in the form of an analog signal, and is converted into a digital signal by the A / D converter 402b on the chip. Further, the A / D converter 402a provided on the same chip converts the waveform data into a digital signal, and the digital signal processing unit 401 performs flicker correction.

In this embodiment, the analog signal processing unit 20
4 includes an A / D converter 402a that performs A / D conversion on the analog video signal output from the photometer 4 and an A / D converter 402b that performs A / D conversion on the analog data output from the photometric element 103. However, it is not always necessary to provide two A / D converters, and one A / D converter may be shared to reduce the size of the system. In this case, the output data of the photometric element 103 digitized by the common A / D converter is sent to the digital signal processing unit 401, and appropriate signal processing is performed using the electronic shutter setting value to generate flicker correction data. .
Then, based on the pulse from the timing generation unit 107, flicker correction is performed while synchronizing with the digital video signal.

The thirteenth embodiment shown in FIG.
The A / D converter 402b as well as the photometric element 103 is arranged outside the element. That is, after the waveform data output from the photometric element 103 is converted into a digital signal by the A / D converter 402b, the correction processing is performed in the digital signal processing unit 401 inside the element in the same manner as in the eleventh and twelfth embodiments. I do.

The SOC according to the fourteenth embodiment of the present invention
As shown in FIG. 21, the type image sensor is provided outside the image sensor, with a photometric element 103 and an analog signal processor 7 such as an integrator.
04 and a flicker sensor 308 incorporating an A / D converter 402b. Image sensor 1004
The electronic shutter setting value output from the electronic shutter setting unit 104 is input to the flicker sensor 308 via the dedicated port 1204, and the analog signal processing unit 704 integrates the waveform output from the photometric element 103 according to this value and flickers. A / D converter 402b generates a waveform, converts the signal into a digital signal, and inputs the digital signal to image sensor 1004 via dedicated port 1205. The image sensor 1004 is provided with a flicker waveform as a digital signal, and performs flicker correction in the same manner as in the above-described first to fourteenth embodiments.

In this embodiment, the flicker waveform as a digital signal is input to the image sensor 1004 in this manner. However, the present invention is not limited to such a technique, and a flicker waveform may be input to an image sensor in the form of an analog signal, and A / D conversion may be performed inside the element to perform flicker correction.

In the fifteenth embodiment of the present invention, as shown in FIG. 22, all the components for flicker correction including the photometric element 103 are incorporated in the same chip as the image sensor 1005. Also in the present embodiment, the operation of the flicker correction processing is the same as in the other embodiments. However, as in the present embodiment, the photometric element 103
When mounted, the subject is imaged up to the photometric element 103. As a result, the output waveform of the photometric element 103 is affected by the picture of the subject, and it is necessary to take some measures as described below so as not to be affected.

FIGS. 23A to 23C show an example of the structure of a photometric element for reducing the effect of the subject image formed on the image pickup surface and its layout.

In the example shown in FIG. 23A, the size of the photometric element 1101 is
It is arranged larger than 2. In the example shown in FIG. 23B, the lens 11
03 is arranged. Thus, the photometric element 1101
Either increase the size of itself or
By disposing the lens 1103 on 4 and making the opening sufficiently wide, the influence of the pattern formed on the photometric element 1101 can be reduced.

FIG. 23C shows an example in which a plurality of photometric elements 1107 are arranged. Multiple photometric elements 1
By averaging the outputs from 107, it is possible to reduce the influence of the picture.

[0089]

As described above, according to the present invention, the flicker waveform of a fluorescent lamp is measured using a photometric element, and the gain of a video signal output from an image pickup element is controlled, whereby the fluorescent lamp can be illuminated. It is possible to suppress the horizontal stripe fluorescent lamp flicker and realize a high image quality.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a solid-state imaging device according to a first embodiment of the present invention.

FIG. 2 is a graph showing an integration process of an output waveform of a photometric element in the solid-state imaging device.

FIG. 3 is an explanatory diagram showing a timing at which a gain control unit of the solid-state imaging device sets a gain.

FIG. 4 is a block diagram showing a more detailed configuration of a control value generator of the solid-state imaging device.

FIG. 5 is a graph showing a flicker waveform and a flicker correction waveform estimated by the solid-state imaging device.

FIG. 6 is a block diagram illustrating a configuration of a solid-state imaging device including a flicker sensor according to a second embodiment of the present invention.

FIG. 7 is a block diagram illustrating a configuration of a solid-state imaging device that performs flicker correction by a digital signal processing unit according to a third embodiment of the present invention.

FIG. 8 is a block diagram illustrating a configuration of a solid-state imaging device including a low-pass filter according to a fourth embodiment of the present invention.

FIG. 9 is a graph showing a blinking waveform of an inverter type fluorescent lamp.

FIG. 10 is a block diagram illustrating a configuration of a solid-state imaging device including a plurality of photometric elements according to a fifth embodiment of the present invention.

FIG. 11 is a block diagram illustrating a configuration of a solid-state imaging device including a plurality of photometric elements and performing flicker correction by a digital signal processing unit according to a sixth embodiment of the present invention.

FIG. 12 is a block diagram illustrating a configuration of a solid-state imaging device including a characteristic table according to a seventh embodiment of the present invention.

FIG. 13 is a block diagram showing a configuration of a solid-state imaging device that shares a photometric element for flicker correction and light amount control according to an eighth embodiment of the present invention.

FIG. 14 is a block diagram showing a configuration of a solid-state imaging device according to a ninth embodiment of the present invention.

FIG. 15 is a block diagram showing a configuration in a case where an output of a photometric element is used for white balance correction in the digital signal processing unit according to the third or sixth embodiment.

FIG. 16 is a block diagram showing a configuration of a solid-state imaging device having a plurality of resolution modes or a camera shake correction function according to a tenth embodiment of the present invention.

FIG. 17 is an explanatory diagram showing a procedure for generating flicker correction data in the solid-state imaging device.

FIG. 18 shows an S in which a flicker correction element excluding a photometric element according to an eleventh embodiment of the present invention is incorporated on an image sensor.
FIG. 2 is a block diagram illustrating a configuration of an OC-type solid-state imaging device.

FIG. 19 is a block diagram showing a configuration of an SOC type solid-state imaging device in which a digital signal processing unit according to a twelfth embodiment of the present invention is incorporated on an imaging device.

FIG. 20 is a block diagram showing a configuration of an SOC type solid-state imaging device in which a flicker correction element excluding a photometric device and an A / D converter according to a thirteenth embodiment of the present invention is incorporated in the imaging device.

FIG. 21 shows a flicker sensor having a photometric element and an analog signal processing unit according to a fourteenth embodiment of the present invention, and
FIG. 3 is a block diagram showing a configuration of an SOC type solid-state imaging device in which flicker correction elements other than a D converter are incorporated on the imaging device.

FIG. 22 is a block diagram showing a configuration of an SOC type solid-state imaging device in which a flicker correction element according to a fifteenth embodiment of the present invention is incorporated in an imaging device.

FIG. 23 is an explanatory view showing the structure and layout of a photometric element and pixels in the SOC type solid-state imaging device.

FIG. 24 is an explanatory diagram illustrating the principle of generation of flicker in a MOS image sensor.

FIG. 25 is an explanatory diagram showing a relationship between a fluorescent lamp waveform and a signal charge accumulation time in an image sensor.

[Explanation of symbols]

 101, 1106 Image sensor 103, 1101, 1104, 1107 Photometry device 104 Electronic shutter setting unit 105 Control value generation unit 106 Gain control unit 107 Timing generation unit 108 Reading area setting unit 201, 1105 Area sensor unit 202 Vertical scanning unit 203 Horizontal scanning Unit 204 analog signal processing unit 205 vertical / horizontal scanning signal generation unit 301 integrator 302 level correction unit 303 level inversion unit 304 sensitivity correction table 305 low-pass filter 306 characteristic table 307 signal processing unit 308 flicker sensor 309 signal level measurement unit 401 digital signal processing Units 402, 402a, 402b A / D converter 403 White balance correction unit 404 Flicker correction unit 405 Video signal processing unit 501 Lens aperture 701 to 703 Photometric element 705 A Interface unit 1001 to 1003 image sensor 1102 pixels 1103 lens 1201 to 1205 only port 130 IF section

 ──────────────────────────────────────────────────続 き Continuing from the front page (72) Inventor Kazunari Oi 8 Term Shinsugita-cho, Isogo-ku, Yokohama-shi, Kanagawa Prefecture F-term in the Toshiba Yokohama Office 4M118 AA05 AA10 AB01 BA13 BA14 FA06 FA50 GC08 5C022 AB13 AB15 AB20 AB37 AB51 AC42 AC52 AC55 AC69 5C024 AA01 CA07 CA26 DA01 EA08 FA01 FA08 GA01 GA31 HA02 HA09 HA10 HA14 HA23 JA04 5C065 AA01 BB02 BB21 BB41 CC01 CC09 DD15 EE06 GG03 GG18 GG26

Claims (10)

[Claims] 1. An area sensor unit in which pixels that generate a pixel signal by performing photoelectric conversion are arranged two-dimensionally, and the area sensor unit is sequentially selected in a first direction in units of a line in the first direction. 1 scanning unit, and the pixel signal in units of the line in the first direction as a second line.
A second scanning unit for sequentially selecting the pixel signals in a direction, and performing predetermined signal processing on the pixel signals scanned by the first and second scanning units and read from the area sensor unit, and outputting a video signal. An X-Y address type image sensor having a signal processing unit that performs a photoelectric conversion, outputs a light amount waveform by performing photoelectric conversion, an exposure period of the image sensor, and the output light amount waveform based on the light amount waveform. A control value generation unit that generates a control value for controlling the gain of the video signal; and a gain control unit that controls the gain of the video signal based on the control value generated by the control value generation unit. A solid-state imaging device characterized by the above-mentioned. 2. An image processing apparatus according to claim 1, further comprising an analog-to-digital conversion unit that digitizes the video signal output from the image sensor, wherein the gain control unit converts the digitized video signal into a digital video signal having a required signal format. 2. The solid-state imaging device according to claim 1, wherein the signal processing is performed to control the gain of the digital video signal based on the control value. 3. The image processing apparatus according to claim 2, further comprising: a reference table for preliminarily holding an amplitude of a flicker component included in the video signal output from the area sensor unit, wherein the control value generation unit converts the light amount waveform output from the photometric element. The gain is adjusted so that the amplitude of the integrated waveform obtained by integrating over the effective charge accumulation period for each line of the area sensor unit matches the amplitude of the flicker component included in the video signal held by the reference table. The solid-state imaging device according to claim 1, wherein the control is performed. 4. A low-pass receiving the light amount waveform output from the photometric element, performing a process of passing only a signal band depending on a power supply cycle in the light amount waveform, and outputting the signal band to the control value generating unit. The solid-state imaging device according to claim 1, further comprising a filter. 5. The solid-state imaging device according to claim 1, wherein the photometric device includes a plurality of photometric devices in which different color filters are arranged. 6. The control value generation section separates the light amount waveforms output from the photometric elements for each color, generates the control value according to the light amount waveforms for each color, and separates the control values. 6. The solid-state imaging device according to claim 1, wherein the white balance is corrected in accordance with a ratio of an output level of each color. 7. The light amount waveform output from the photometric element is supplied to the control value generation unit to generate the control value, and the control of the light amount input to the area sensor unit or the area sensor unit The solid-state imaging device according to any one of claims 1 to 6, wherein the solid-state imaging device is used for controlling an accumulation time of the solid-state imaging device. 8. A reading area setting section for setting a pixel area from which the pixel signal is read from the area sensor section in accordance with a command input from the outside, wherein the control value generating section is configured so that the reading area setting section sets the pixel area. The solid-state imaging device according to any one of claims 1 to 7, wherein the control value is generated based on the determined pixel region. 9. An area sensor unit in which pixels that generate a pixel signal by performing photoelectric conversion are two-dimensionally arranged, and the area sensor unit is sequentially selected in a first direction in units of a line in the first direction. 1 scanning unit, and the pixel signal in units of the line in the first direction is used as a second unit.
A second scanning unit for sequentially selecting directions, a predetermined signal processing is performed on the pixel signals scanned by the first and second scanning units and read from the area sensor unit, and a video signal is output. An X-Y address type image sensor having a signal processing unit for performing the operation, an interface unit to which a light amount waveform is externally given, an exposure period of the image sensor, and the light amount waveform given to the interface unit. A control value generation unit that generates a control value that controls the gain of the video signal; and a gain control unit that controls the gain of the video signal based on the control value generated by the control value generation unit. A system-on-chip type solid-state imaging device, wherein the imaging device, the interface unit, the control value generation unit, and the gain control unit are formed on the same chip. 10. An area sensor unit in which pixels that generate a pixel signal by performing photoelectric conversion are two-dimensionally arranged, and the area sensor unit is sequentially selected in a first direction in units of a line in the first direction. 1 scanning unit, and the pixel signal in units of the line in the first direction is used as a second unit.
A second scanning unit for sequentially selecting the pixel signals in a direction, and performing predetermined signal processing on the pixel signals scanned by the first and second scanning units and read from the area sensor unit, and outputting a video signal. An X-Y address type image sensor having a signal processing unit that performs a photoelectric conversion, a photometric element that outputs a light amount waveform by performing photoelectric conversion, an exposure period when the image sensor performs imaging, and the output light amount waveform A control value generation unit that generates a control value for controlling the gain of the video signal based on the control value generation unit; and a gain control unit that controls the gain of the video signal based on the control value generated by the control value generation unit. And a system-on-chip type solid-state imaging device, wherein the imaging device, the photometry device, the control value generation unit, and the gain control unit are formed on a same chip.