JP2007336329A - Image sensing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image sensing device reducing an image quality deteriorating component remaining in image signals subjected to flicker correction. <P>SOLUTION: An image sensor 1 outputs image signals according to a storage time, and a flicker correction unit 2 subjects the image signals to flicker correction. When a still image detection unit 6 detects that the image signals shows a still image, a storage time control unit 5 changes the storage time of the image sensor 1. A feature detection unit 3 detects the features of the output signals of the flicker correction unit 2. A correction calculating unit 6 subjects image signals to correction computation after flicker correction, using the output signals of the feature detection unit 3 and storage time setting information. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an imaging apparatus that includes an image sensor and outputs an image signal.

  A conventional imaging apparatus having a CMOS image sensor employs a rolling shutter system that shifts the accumulation time zone of pixel signals for each pixel. Therefore, a striped pattern is generated in an image output from the imaging device when affected by illumination light caused by blinking illumination such as a fluorescent lamp. An imaging apparatus that suppresses the stripe pattern is disclosed in Patent Document 1.

  The conventional imaging device described in Patent Document 1 shown in FIG. 8 includes a CMOS image sensor, a camera 20 that outputs an image signal, a FIFO 21 that stores an image signal for one field, and a current image from the camera 20. Based on the image signal and the image signal delayed by one field from the FIFO 21, an arithmetic unit 22 that calculates an average value of the image signal for each pixel is provided.

  When an image is input while the camera 20 is affected by illumination light from a fluorescent lamp or the like by a 50 Hz power source, if a period that is an integral multiple of the fluorescent lamp cycle is set as the pixel accumulation time, the accumulated light intensity is The pixel signals are equal, and the image signal output from the camera 20 is not affected by the blinking illumination light. However, when the integer multiple relationship is broken, the image signal output from the camera 20 is affected by the blinking illumination light.

FIG. 9 shows the image signals 91 and 92 affected by the blinking illumination light and the image signal 93 averaged by the calculation unit 22. When the signal displayed on the screen changes from the image signal 91 of the first field to the image signal 92 of the second field, a striped pattern flows on the screen. Therefore, the calculation unit 22 performs flicker correction by averaging the image signal 91 of the first field stored in the FIFO 21 and the currently input image signal 92 of the second field, which are 180 degrees different in phase, for each pixel. Apply. Thereby, the image signal output from the calculating part 22 becomes the uniform signal 93, and the striped pattern on the screen is canceled.
JP 2004-88720 A

  In addition, unlike FIG. 9, when an image signal is configured using three fields whose phases are different by 120 degrees, the calculation unit 22 performs flicker correction by averaging the image signals for three fields. FIG. 10 shows the waveforms of the image signal for three fields and the image signal after flicker correction. In the conventional imaging apparatus, when there is distortion in the light emission waveform of blinking illumination such as a fluorescent lamp, the influence of distortion as shown by the shaded portion 105 in FIG. 10 appears in the image signals 101, 102, 103 output from the camera 20. As a result, the influence of distortion remains in the image signal 104 after averaging the image signals 101, 102, and 103 of the three fields. The distortion component remaining in the image signal 104 becomes a fixed thin horizontal stripe on the screen, which causes image quality deterioration.

  The present invention solves the above-described conventional problems, and an object thereof is to provide an imaging apparatus that reduces distortion components remaining in an image signal after flicker correction and suppresses image quality deterioration.

  An image pickup apparatus of the present invention includes an image sensor that outputs an image signal corresponding to an accumulation time, a flicker correction unit that performs flicker correction on the image signal output from the image sensor, and the image signal after flicker correction is a still image. A still image detection unit that detects whether the image is in a state, an accumulation time control unit that controls the accumulation time according to a detection result of the still image detection unit, a feature detection unit that detects a feature of the image signal after flicker correction, A correction calculation unit that performs a correction calculation on the image signal after flicker correction based on the feature of the image signal detected by the feature detection unit and the storage time set by the storage time control unit. With this configuration, the present invention can remove distortion components remaining in the image signal after flicker correction, and can realize image quality deterioration suppression.

  The flicker correction unit may perform flicker correction according to the lighting cycle of the blinking illumination.

  The feature detection unit may calculate an average value within a predetermined horizontal period for the image signal after flicker correction, and output the average value as a feature of the image signal.

  The accumulation time control unit may set the accumulation time of the image sensor according to the detection result of the still image detection unit, and may output information regarding the accumulation time to the correction calculation unit.

  When the still image detection unit determines that the image signal after flicker correction is in a still image state, the accumulation time control unit may newly set an accumulation time different from the current accumulation time.

  The correction calculation unit detects a difference between the output signals of the feature detection units having different accumulation times, and based on the difference, removes a frequency component that causes image quality degradation included in the image signal after flicker correction. good.

  According to the imaging apparatus of the present invention, it is possible to remove a distortion component remaining in an image signal after flicker correction, and to achieve a special effect that image quality deterioration can be suppressed.

  The best mode for carrying out the present invention will be described below with reference to the drawings.

[Configuration of imaging device]
FIG. 1 shows a configuration of an imaging apparatus according to an embodiment of the present invention. The image pickup apparatus according to the present embodiment includes a MOS, and includes an image sensor 1 that outputs an image signal of a moving image or a still image corresponding to an accumulation time, and flicker correction that performs flicker correction on an image signal output from the image sensor 1. 2, the image signal output from the flicker correction unit 2 is input, and the still image detection unit 6 that detects whether the image signal generated by the image sensor 1 is in a still image state or the detection result of the still image detection unit 6 Accordingly, the accumulation time control unit 5 that controls the accumulation time of the image sensor 1, the feature detection unit 3 that detects the characteristics of the image signal output from the flicker correction unit 2, and the feature detection unit 3 and the accumulation time control unit 5 Based on the information, there is a correction calculation unit 4 that performs a correction calculation on the flicker-corrected image signal output from the flicker correction unit 2.

[Influence of blinking illumination light on image sensor]
The image sensor 1 employs a rolling shutter system, and accumulates charges according to the input light amount in the pixels according to the accumulation time set by the accumulation time control unit 5. The amount of light input by the image sensor 1 varies under the influence of illumination light from blinking illumination such as a fluorescent lamp whose light intensity varies periodically.

  FIG. 2 shows the waveform of the light intensity of a 50 Hz fluorescent lamp. A fluorescent lamp with a 50 Hz power source emits light with a light intensity of 100 Hz period. When the image sensor 1 inputs an image affected by a 50 Hz fluorescent lamp, if a period of an integral multiple of the fluorescent lamp's light emission period of 100 Hz is set as the pixel accumulation time, the accumulated light intensity is determined for each pixel. It becomes equal and is not affected by flashing illumination light. However, since the light intensity accumulated in each pixel is different when the integer multiple relationship is broken, the image signal output from the image sensor 1 is an image signal including a stripe pattern that causes flicker.

  FIGS. 3A, 3B, and 3C show image signals of an image having uniform brightness over the entire image when the image sensor 1 accumulates electric charges in a time that is half of the light emission period of the 50 Hz fluorescent lamp. Illustrate. The vertical period in FIGS. 3A, 3B, and 3C is 33 msec. When the image signals of the first field to the third field are displayed on the screen, a horizontal stripe pattern flows.

[Flicker correction]
FIG. 4 shows the waveforms of the image signals 41, 42, and 43 in the first, second, and third fields when the vertical period is set so that the phase is shifted by 120 degrees, and the imaging apparatus of the present embodiment corrects the waveform. The waveform of the output signal 44 output from the calculation unit 4 is shown. In this embodiment, the image sensor 1 sets the vertical period so that the phases of the image signals 41, 42, and 43 in the first, second, and third fields are shifted by 120 degrees, and the image signals 41, 42, and 43 Is output. The flicker correction unit 2 stores the first field image signal input first and the second field image signal input next, and stores the image signal of the third field newly input from the image sensor 1. Flicker correction is performed by calculating an average value for each pixel based on the image signals of the first field and the second field. However, if there is distortion in the light emission waveform of the flashing illumination such as a fluorescent lamp, the image signal output from the flicker correction unit 2 is a frequency component that causes image quality degradation, that is, distortion, as indicated by the image signal 104 in FIG. It becomes an image signal including a thin striped pattern in which components remain. Therefore, the imaging apparatus according to the present embodiment removes the remaining distortion components as described below, and outputs a uniform image signal 44 from the correction calculation unit 4.

[Removal of remaining distortion components]
The still image detection unit 6 stores the image signal of the field and detects whether or not several image data on the screen between the fields have changed, so that after the flicker correction output from the flicker correction unit 2 is performed. Whether the image signal is in a still image state is detected. When the still image detection unit 6 determines that the still image is in the still image state, it outputs still image determination information indicating the still image state to the accumulation time control unit 5.

  When the storage time control unit 5 receives the still image determination information from the still image detection unit 6, the storage time control unit 5 sets a storage time shorter than the currently set predetermined storage time as a distortion component detection value for the image sensor 1. In addition, storage time setting information including information on the newly set storage time for distortion component detection is output to the correction calculation unit 4. Thereafter, when a predetermined time elapses, the accumulation time control unit 5 returns the set value of the accumulation time to the original predetermined value, sets the accumulation time in the image sensor 1, and stores information related to the original accumulation time as the accumulation time setting information. Is output to the correction calculation unit 4. The accumulation time control unit 5 takes into consideration the influence of image change when the set value of the accumulation time is changed to a value for detecting distortion components, and the length of the accumulation time that is shorter than the currently set predetermined accumulation time. To decide.

  FIG. 5 shows an operation flow relating to the setting of the accumulation time according to the present embodiment. The flicker correction unit 2 determines whether the image signal output from the image sensor 1 is affected by the blinking illumination light (S501). If the flickering illumination light is affected, the flicker correction unit 2 performs flicker correction. The still image detection unit 6 determines whether the image signal output from the flicker correction unit 2 is in a still image state (S502). If it is determined that the still image detection unit 6 is in the still image state, the accumulation time control unit 5 changes the set value of the accumulation time from a predetermined value to a value for distortion component detection (S503) and changes to the image sensor 1. Sets the value accumulation time. The accumulation time control unit 5 determines whether a predetermined time has elapsed since the accumulation time was changed (S504). If the predetermined time has elapsed, the accumulation time control unit 5 returns the accumulation time to the original value, that is, the predetermined value before the change (S505). If it is not affected by the blinking illumination light (No in S501) or not in the still image state (No in S502), the setting value of the accumulation time is not changed.

  FIG. 6 shows the detection range of the feature of the image signal in the feature detection unit 3 and the waveform 31 of the detection result. The feature detection unit 3 provides an average value detection range within a predetermined horizontal period of the image signal output from the flicker correction unit 2, and calculates the average value of the image signal in the vertical period direction of the average value detection range in units of horizontal lines. Then, the average value is output to the correction calculation unit 4 as a feature of the image signal. Thus, by providing the average value detection range within a predetermined horizontal period, the feature detection unit 3 does not have to store and process the image signal data of the entire screen. Therefore, the feature detection unit 3 can be realized with a simple circuit.

  FIG. 7 shows an example of the internal configuration of the correction calculation unit 4. The correction calculation unit 4 receives the accumulation time setting information transferred from the accumulation time control unit 5 and obtains a gain A corresponding to a predetermined accumulation time and a gain B corresponding to a distortion component detection value accumulation time. The gain calculation unit 10 to output, the FIFO 11 that stores the output signal of the feature detection unit 3, the multiplier 12 that multiplies the output of the FIFO 11 and the gain B, and the multiplier 13 that multiplies the output signal of the feature detection unit 3 and the gain A. A subtractor 14 that subtracts the output value of the multiplier 12 from the output value of the multiplier 13, and a subtractor 15 that subtracts the output value of the subtractor 14 from the output signal of the flicker correction unit 2.

  The gain A and the gain B are determined based on the fact that the amount of light accumulated differs from the difference in the accumulation time setting value. That is, the signal obtained by multiplying the output signal of the feature detection unit 3 by the gain A when the set value of the accumulation time is the original predetermined value and the set value of the accumulation time stored in the FIFO 11 are used for distortion component detection. In each of the signals obtained by multiplying the output signal of the feature detection unit 3 at the time of value and the gain B, the gain A is set so that the signal level is the same except that it is influenced by the distortion of the flashing illumination. And the gain B are calculated. The shorter the accumulation time, the more light emission distortion components of the flashing illumination remain. This is because the rolling shutter method is close to the method for calculating the moving average of the amount of light, and if the accumulation time is long, the low-pal filter effect is increased and the distortion component is further suppressed.

  When the storage time control unit 5 returns the set value of the storage time from the distortion component detection value to a predetermined value, the subtractor 14 outputs the output of the multiplier 13 when the storage time set value is the original predetermined value. The difference between the value and the output value of the multiplier 12 when the set value of the accumulation time is a distortion component detection value is calculated, and the distortion portion of the image signal due to the light emission distortion of the flashing illumination is extracted. The subtracter 15 subtracts this difference from the output signal of the flicker correction unit 2. As a result, the image signal output from the correction calculation unit 4 is a signal that does not include or reduces the distortion component that causes image quality degradation.

  As described above, when the image signal is in a still image state, a flicker is obtained by generating a signal containing a large amount of light emission distortion components by shortening the set value of the accumulation time and subtracting the distortion components from the original image signal. The distortion component remaining after correction can be removed. Thereby, a striped pattern does not flow on the screen.

  In consideration of the magnitude of the output value of the multiplier 14 subtracted from the output signal of the flicker correction unit 2, a correlation may be taken when calculating the gain A and the gain B. When the correlation is not taken, a multiplier that multiplies the output result of the subtracter 14 by a predetermined gain is provided between the subtractor 14 and the subtractor 15, and the output value of the multiplier is changed to the flicker correction unit 2. May be subtracted from the output signal.

  In this embodiment, the accumulation time to be changed is shorter than the original accumulation time, but the present invention is distorted from the difference between the image signal obtained by changing the set value of the accumulation time and the original image signal. Since this is a method of extracting components, the set value of the accumulation time to be changed may be longer than the original accumulation time. Even when the set value of the accumulation time to be changed is longer than the original accumulation time, the same effect as in the present embodiment can be obtained.

  In the present embodiment, the case where the flicker correction is performed on the image signals of three fields whose phases are different by 120 degrees has been described. However, the number of fields constituting the image is not limited to three. The present invention can be applied even when image signals for two different fields are used. For example, the image sensor 1 may set the vertical period so that the phases are different by 180 degrees and output an image signal, and the flicker correction unit 2 may perform flicker correction using the image signals for two fields. The remaining distortion component can be removed from the image signal after such flicker correction.

  Further, the flicker correction unit 2 determines whether the image signal output from the image sensor 1 is affected by the blinking illumination, and when it is determined that the image signal is affected by the blinking illumination, the flicker is performed. Corrections may be made. For example, the flicker correction unit 2 calculates an average value of luminance of a desired horizontal line for each field with respect to the image signal output from the image sensor 1. The average value of the luminance of the horizontal line in the previous field is stored. When the phase of the image signal is 180 degrees different, the flicker correction unit 2 calculates the average value of the luminance of the desired horizontal line in the current field, and then uses the average value of the previous field and the current field for two fields. The average value is calculated. The flicker correction unit 2 compares the difference between the average value of a desired horizontal line in the current field and the average value of two fields. When the result of the comparison is inverted for each field, the flicker correction unit 2 is output from the image sensor 1. The flicker correction may be performed by determining that the image signal is affected by blinking illumination.

  INDUSTRIAL APPLICABILITY The present invention has an effect of reducing frequency components that cause image quality degradation remaining in an image signal after flicker correction, and is useful as an imaging device or the like.

1 is a block diagram showing a schematic configuration of an imaging apparatus according to the present invention. Waveform diagram of light emission of 50Hz fluorescent lamp It is a figure which shows the image signal at the time of accumulating in the time of 1/2 of a light emission period, (a) is an image signal of the 1st field, (b) is an image signal of the 2nd field, (c) is 3rd. Waveform diagram showing field image signal Waveform diagram of 3 fields of signal and averaged image signal The flowchart which shows the setting method of the accumulation time of this invention The figure which shows the detection state of the feature of the image signal in the feature detection part of this invention The figure which shows an example of an internal structure of the correction | amendment calculating part of this invention A block diagram showing a schematic configuration of a conventional imaging device Waveform diagram of two fields of signal and averaged image signal Waveform diagram of the image signal when the light emission waveform is distorted

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image sensor 2 Flicker correction part 3 Feature detection part 4 Correction calculation part 5 Accumulation time control part 6 Still image detection part 10 Gain calculation part 11 FIFO
20 camera 21 FIFO
22 Calculation unit

Claims (6)

An image sensor that outputs an image signal according to the accumulation time;
A flicker correction unit that performs flicker correction on an image signal output from the image sensor;
A still image detection unit for detecting whether the image signal after flicker correction is in a still image state;
An accumulation time control unit that controls the accumulation time according to a detection result of the still image detection unit;
A feature detector for detecting features of the image signal after flicker correction;
A correction calculation unit that performs a correction calculation on the image signal after flicker correction based on the feature of the image signal detected by the feature detection unit and the storage time set by the storage time control unit;
An imaging device comprising:   The imaging apparatus according to claim 1, wherein the flicker correction unit performs flicker correction according to a lighting cycle of the blinking illumination.   2. The imaging according to claim 1, wherein the feature detection unit calculates an average value in a predetermined horizontal period for the image signal after flicker correction in units of lines and outputs the average value as a feature of the image signal. apparatus.   The accumulation time control unit sets the accumulation time for the image sensor according to a detection result of the still image detection unit, and outputs information on the accumulation time to the correction calculation unit. Item 2. The imaging device according to Item 1.   The accumulation time control unit newly sets an accumulation time different from the current accumulation time when the still image detection unit determines that the image signal after flicker correction is in a still image state. The imaging apparatus according to 1. The correction calculation unit detects a difference between output signals of the feature detection units having different accumulation times, and based on the difference, removes a frequency component that causes image quality degradation included in the image signal after flicker correction. The imaging apparatus according to claim 5, wherein:

Images