JP2001119708A - Flicker detection and correction device, and flicker detection and correction method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately detect flicker even when a luminance level of a video signal is changed due to a motion of an object. SOLUTION: The flicker detection and correction device is provided with an integration means 1 that integrates a pixel level of a video signal for each line, an averaging means 3 that averages the outputs of the integration means 1 with respect to lines at the same picture position in frames or fields, a still part extract means 4 that extracts a still part of the picture by using the output of the integration means 1, a division means 5 that divides the integration result for each line by the integration means 1 by an averaging result for each line by the averaging means 3, and a flicker discrimination means 6 that applies frequency analysis of the result of division by the division means 5 to discriminate the flicker.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a flicker generated in an image captured under illumination light whose brightness fluctuates with a power supply frequency by using an XY address type image sensor such as a MOS type image sensor. The present invention relates to a flicker detection / correction device capable of detecting and correcting.

[0002]

2. Description of the Related Art First, the principle of occurrence of the flicker will be described with reference to FIGS.

FIG. 18 is a diagram for explaining the principle of generation of flicker when the power supply frequency is 50 Hz.

When the frequency as shown in FIG.
When a fluorescent lamp or the like is turned on with an AC power supply of Hz, the illuminating light becomes brightest when the amplitude of the power supply current is the largest. Therefore, as shown in FIG. At 100 Hz).

[0005] The accumulation timing and the output of the image pickup device when the image is picked up by the MOS type image pickup device for 1/30 seconds under the fluorescent lamp whose brightness fluctuates periodically are shown in FIGS.
(D). In this case, as shown in (c), a value obtained by integrating the amount of incident light from the readout points A1 to B1 becomes the output signal of the first line of the image sensor. Similarly, the read point
The integration of the amount of incident light from A2 to B2 becomes the output signal of the second line, and the same result will be obtained until the last line.

At this time, the output of the image pickup device fluctuates as shown in FIG. 4D in accordance with the amount of incident light, and the luminance level fluctuates on the screen, so that it is recognized as flicker. When the frame cycle is 30 Hz, the phases of the brightness of the illumination are aligned in the three frame cycles, so that the luminance level changes every three frame cycles. Further, in the case of the MOS type image sensor, as shown in FIG. 2B, since the accumulation timing differs for each line, this flicker appears in one frame.
This will be recognized as a black stripe pattern on the screen, causing image quality deterioration.

FIG. 19 shows a case where the power supply frequency is 60 Hz.
FIG. 3 is a diagram for explaining the principle of generation of flicker.

[0008] As shown in FIG.
When a fluorescent lamp or the like is turned on with an AC power supply of 0 Hz, the light amount fluctuates at twice the power supply frequency (in this case, 120 Hz) as shown in FIG.

As described above, when an image is taken by a MOS type image pickup device with 1/50 second accumulation under a fluorescent lamp whose brightness fluctuates at a cycle of 120 Hz, as shown in FIG.
A value obtained by integrating the amount of incident light up to B1 is an output signal of the first line of the image sensor. Similarly, the integral of the amount of incident light from the read points A2 to B2 becomes the output signal of the second line, and the same result is obtained from the last line to the last line.

At this time, the output of the image pickup device fluctuates as shown in FIG. 4D according to the amount of incident light, and the luminance level fluctuates on the screen, so that it is recognized as flicker.

When the power supply frequency is 60 Hz, the frame period is an integral multiple of the light amount fluctuation period, so that the luminance level does not fluctuate for each frame as occurs at 50 Hz. However, if the power frequency fluctuates around 60Hz,
The black stripes on the screen appear to move from frame to frame, causing image quality degradation. Further, in the case of the MOS type image pickup device, the accumulation timing differs for each line as in the case of 50 Hz, so that this flicker appears in one frame and is recognized as a black stripe pattern on the screen, and the image quality deteriorates. cause.

Next, the principle of correcting flicker will be described with reference to FIG. Here, (a) to (c) show a case where the power supply frequency is 50 Hz, and (d) to (f) show a case where the power supply frequency is 60 Hz.
FIG.

When the power supply frequency is 50 Hz, the brightness of the illumination fluctuates in 1/100 second as shown in FIG. At this time, as shown in (b), the shutter speed (accumulation time of the image sensor) is set to an integral multiple of 1/100 second (the example is twice, but may be one or three times). . With this setting, the readout timing of the first line (A1 to B1) and the readout timing of the second line (A2
To B2), the incident light amounts are the same. Similarly, the amount of incident light is the same from the third line to the last line. For this reason, as shown in (c), the output of the image sensor becomes constant and no flicker occurs.

When the power supply frequency is 60 Hz, 50
As in the case of Hz, the shutter speed is set to an integral multiple of 1/120 second (1 to 4 times). (E) shows a case where the time is set to 1/60 second, which is twice as large as 1/120 second. As shown in this figure, the read timing of the first line (A1
To B1) and the readout timing of the second line (from A2
In B2), the amount of incident light is the same. Similarly, the amount of incident light is the same from the third line to the last line. For this reason, the output of the image sensor becomes a fixed amount, and no flicker occurs.

That is, flicker is suppressed by setting the shutter speed to be an integral multiple of the power supply cycle and within the frame cycle.

As an image pickup apparatus that performs flicker correction based on such a correction principle, for example, Japanese Patent Publication No.
There was one described in No. 15324. In this imaging device, the presence or absence of flicker is detected by comparing a difference value between the integrated value of the current field and the integrated value of the previous field of the video signal generated by the image sensor with a predetermined threshold value,
The shutter speed is switched according to the presence or absence of the detected flicker.

[0017]

However, the conventional imaging apparatus has the following problems (1) to (4). (1) Since the difference between the current field and the previous field is obtained, it is not possible to detect flicker in a field (or a frame) that occurs when a MOS image sensor is used. (2) Since the difference between the current field and the previous field is calculated, if the luminance level of the video signal fluctuates due to the movement of the subject or the like, it may be erroneously determined to be flicker, and the flicker is accurately detected. Can not do. (3) Flicker that occurs when the power supply frequency is 60 Hz hardly varies from field to field, and therefore cannot be detected by taking the difference between the current field and the previous field. For this reason, it is determined that there is no flicker. (4) Reduce shutter speed to 1/10 for flicker correction
If it is set to 0 second or 1/60 second, the video signal level is saturated when the amount of incident light increases, and no video is displayed.

The present invention has been made in view of such a problem, and even when the luminance level of a video signal fluctuates due to the movement of a subject or the like, flicker generated in a frame or a field can be accurately detected. It is an object of the present invention to provide a flicker detection device capable of detecting a flicker.

It is another object of the present invention to provide a flicker detection device capable of detecting flicker generated when the power supply frequency is 60 Hz.

It is another object of the present invention to provide a flicker detection / correction device capable of realizing flicker correction and avoiding a state in which an image is not displayed when the amount of incident light increases.

[0021]

SUMMARY OF THE INVENTION A flicker detection apparatus according to the present invention comprises an integrating means for integrating a pixel level for each predetermined area in a frame or a field, and an integrating means for each area having the same image position in a plurality of frames or fields. Averaging means for averaging the result of integration, static part extracting means for extracting a static part of the image using the output of the integrating means, and a stationary part extracted by the static part extracting means, Dividing means for dividing the integrated result for each area by the averaging result for each area of the averaging means, and flicker determining means for performing frequency analysis of the division result of the dividing means to determine whether or not flicker exists. Was. With this configuration,
By performing flicker detection on a moving image by using the image information of the still part, even a moving image
Hz and 60 Hz flicker detection can be performed with high accuracy.

A flicker detection / correction device according to the present invention comprises: a flicker detection device according to the present invention; a control signal for controlling a shutter speed of an image pickup means for generating a video signal based on an output of the flicker detection device; Flicker correction control means for generating a control signal for controlling the gain of the video signal. With this configuration, it is possible to realize the correction of the flicker of the moving image and to avoid a state in which the image is not displayed when the amount of incident light increases.

According to the flicker detection method of the present invention, a pixel level of each predetermined area in a frame or a field is integrated, an integration result of each area at the same image position in a plurality of frames or fields is averaged, and the integration result is obtained. The still part of the image is extracted using, the integrated result of each area is divided by the averaging result of each area with respect to the extracted still part, and the division result is subjected to frequency analysis to determine whether there is flicker. Is determined. With this configuration, flicker detection at 50 Hz and 60 Hz can be performed with high accuracy even in a moving image.

A flicker detection / correction method according to the present invention includes controlling a shutter speed of an image pickup apparatus for generating a video signal based on a flicker frequency detected by the flicker detection method according to the present invention, and controlling a level of the video signal. Control. With this configuration, it is possible to realize the correction of the flicker of the moving image and to avoid a state in which the image is not displayed when the amount of incident light increases.

[0025]

Embodiments of the present invention will be described below with reference to the drawings. Note that the present invention is applicable to both frame processing and field processing, but the following description describes the case of frame processing.

(First Embodiment) A flicker detection apparatus according to a first embodiment of the present invention includes an integrating means for integrating the pixel level of a video signal in a frame for each line, and an integrating means for a plurality of past frames. Averaging means for averaging the integration result for each line with respect to lines at the same image position, stationary part extracting means for extracting a static part of the image using the output of the integrating means, and static part extracting means Dividing means for dividing the integration result for each line by the integration means by the averaging result for each line of the averaging means, A flicker determining means for analyzing and determining the presence or absence of flicker.

FIG. 1 is a block diagram showing the configuration of the flicker detection device according to the first embodiment of the present invention. This flicker detection device includes an integrating means 1, a storage means 2 to which the output of the integrating means 1 is input, an averaging means 3 to which the output of the integrating means 1 and the output of the storage means 2 are input, The stationary part extracting means 4 to which the output is input, the output of the integrating means 1, the output of the averaging means 3, and the stationary part extracting means 4
And the flicker determination unit 6 to which the output of the division unit 5 is input.
Here, the integrating means 1, the averaging means 3, the stationary portion extracting means 4, the dividing means 5, and the flicker determining means 5 may be realized by using any of hard logic, DSP, or software processing by a computer.

The integrating means 1 includes a MOS (not shown)
A video signal during an effective scanning period captured by the image sensor is input. This video signal has a brightness of 50Hz or 60Hz
Generated under a light source that fluctuates. The integration means 1 adds or averages the pixels of the video signal during the effective scanning period of one frame for each line. As shown in FIG. 2, the result of adding or averaging the pixel levels of the i-th line of the n-th frame for each line is described as SUM _ni . Therefore, when one frame of the video signal is composed of 480 lines, SUM _{n1 to} SUM _n for i = 1 to 480
UM _n480 is calculated.

The storage means 2 temporarily stores the output of the integrating means 1 for a predetermined number of frames. Averaging means 3
_Are SUM _n-1i output from the integrating means 1 before the SUM _ni is calculated and stored in the storage means 2;
_n-2i and SUM _n-3 , _i are added or averaged. Here, SUM _n−1 , _i , SUM _n−2 , _i , SUM _n−3 , _i are the (n−1) th frame and the (n−2) th frame as shown in FIG.
It is obtained by adding or averaging the pixel levels in the i-th line of the frame and the (n-3) th frame. in this case,
The storage means 2 stores the output of the integrating means 1 for three frames. Here, SUM _n-1i , SUM _n-2 , _i , and SUM
_The result of adding or averaging _n-3 and _i is described as AVE _ni . Here, addition or averaging with the past three frames is performed, but it is sufficient if the addition or averaging is performed for two frames or more.

The stationary part extracting means 4 extracts a stationary part of the image using the output of the integrating means 1. The stationary portion extraction unit 4 receives the addition unit 7 to which the output of the accumulation unit 1 is input, the storage unit 8 to which the output of the addition unit 7 is input, and the output of the addition unit 7 and the output of the storage unit 8. And a stationary part extraction unit 9.

The adder 7 adds the output of the integrating means 1 to the lines of N cycles of the flicker component in the frame. The portion of the image constituted by the lines for the N cycles is referred to as a still portion determination block. Assuming that the leading line number to be added in the j-th still part determination block of the n-th frame is k and the number of lines of N periods of flicker components in the frame is p, the output B of the adding means is
-SUM _nj can be represented by the following equation. B-SUM _nj = SUM _nk + SUM _{nk + 1} + ... + SUM
_{nk + p-1}

FIG. 4 shows the case where N = 1 and j = 1. As shown in FIG. 18, the power source frequency of the light source is 50H
When z and the frame period are 30 Hz, N takes an integer value from 1 to 3. Then, when N = 1 shown in FIG. 4, j takes an integer value from 1 to 3. As described above, the addition result obtained by adding the output of the integration means 1 to the lines of N cycles of the flicker component in the frame indicates that the change component of the luminance level due to the periodic change of the brightness of the light source is the same for any frame. Becomes

The storage unit 8 temporarily stores the output of the adding unit 7 for several frames. The stationary part extraction unit 9 calculates a difference between the addition result B-SUM _nj calculated by the addition unit 7 and the addition result B-SUM _{n-1j of} one frame before read from the storage unit 8, and calculates the difference. When the value is equal to or smaller than a predetermined threshold TH, the block for determining a stationary portion is determined to be a stationary portion. As described above, in B-SUM _nj and B-SUM _n-1j , the change component of the luminance level due to the periodic change of the brightness of the light source is the same, and the difference result corresponds to the change of the subject. Therefore, by comparing the difference result with the threshold value TH, it is possible to determine whether or not it is a stationary portion.

The dividing means 5 applies SUM _ni which is the output of the integrating means 1 and AVE which is the output of the averaging means 3 to the block determined as the stationary part by the stationary part extracting means 4.
_ni and SUM _ni / AVE _ni are calculated. The flicker determining means 6 determines the presence or absence of flicker using the output of the dividing means 5. FIG. 6 shows a configuration example of the flicker determination means 6. The flicker determination means 6 includes a DFT (Discrete Fourier Transform) means 21 to which the output of the division means 5 is input, and a threshold value processing means for performing threshold processing on the output to determine the presence or absence of flicker. 22.

FIG. 7A shows the output S of the dividing means 5.
This is an example of a waveform of UM _ni / AVE _ni . here,
The horizontal axis represents the number of lines, ie, i, and the vertical axis represents the level of the division result, ie, SUM _ni / AVE _ni .

FIG. 7B shows an example of the output of the DFT means 21. Here, the horizontal axis indicates the frequency, and the vertical axis indicates the magnitude of the level of the frequency component. In order to detect the frequency component of 50Hz, the frequency component level when performing DFT calculation for 50Hz is F ₅₀ in the drawing, in order to detect a frequency component of 60Hz, the DFT operation for 60Hz frequency component level when done by an F ₆₀ in FIG.

The threshold value processing means 22 outputs four threshold values, TH _{50 -ON} , TH _{60 -ON} and TH, to the output of the DFT unit 21.
_50-OFF , TH _60-0FF are set in advance. These _{thresholds, TH 50-ON> TH 50} -OFF, the relationship TH _60-ON> TH _60-0FF holds. The threshold value is compared with the above-described 50 Hz frequency component and the 60 Hz frequency component, and the presence or absence of flicker is determined based on the magnitude relationship.

That is, F ₅₀ <TH _50-OFF and F ₆₀ <
When TH _60-0FF , it is determined that there is no flicker, and α × F ₆₀ <F ₅₀
And, F _50> determines that there flicker 50Hz when TH _50-ON, and _{β × F 50 <F 60,} F 60> when TH _60-ON is determined that there is flicker of 60 Hz, if other than the above Judge as unknown.

In the above equation, α is a weighting factor for detecting flicker of 50 Hz, and β is a weighting factor for detecting flicker of 60 Hz. Since all of these coefficients are set to values sufficiently larger than 1, 50 Hz (or 60 Hz)
Is larger than the frequency component of 60 Hz (or 50 Hz) by a predetermined weighting factor,
That is, it is determined that there is a flicker of Hz (or 60 Hz). This reduces the possibility that a change in the luminance level in the frame due to the pattern of the subject is determined as flicker.

As described above, according to the first embodiment of the present invention, the pixel level of each predetermined line in a frame is integrated, and the line at the same image position in a plurality of past frames is calculated. Average the integration results for each line,
A static portion of an image is extracted using the integration result, the integration result for each line is divided by the averaging result for each line, and the extracted result is subjected to frequency analysis. Thus, the presence or absence of flicker is determined, so that flicker detection at 50 Hz and 60 Hz can be performed with high accuracy even when a moving image is captured.

In the above embodiment, the integrating means 1
In the above, the integration is performed for all the lines in the frame, but the integration may be performed for lines thinned out at intervals sufficiently short with respect to the period of the flicker component. In this case, the averaging, division, and flicker detection units subsequent to the integration unit also perform processing on the thinned line signals. With such a configuration, the capacity of the storage unit 2 can be reduced. The averaging means 3 is not limited to the averaging, and the same effect can be obtained by using a cyclic filter or an FIR filter.

(Second Embodiment) In the flicker detection device according to the second embodiment of the present invention, instead of the integrating means 1 integrating the pixel level of the video signal line by line, a flicker component in a frame is used. Are configured to be integrated for each region of which is approximately equal. Except for the content of the integration operation of the integration means 1, the configuration is the same as that of the first embodiment.

FIG. 8 is a diagram for explaining the integration operation of the integration means 1 according to the second embodiment of the present invention. As shown in this figure, the levels of all effective pixels in a region where flicker components in a frame can be considered to be substantially equal (in the figure, the left halves of two adjacent lines) are added or averaged. Here, an area in the frame where the flicker components can be considered to be substantially equal is called a block. Then, a value obtained by adding or averaging the levels of all the effective pixels of the i-th block of the n-th frame is described as SUM _nbi .

FIG. 9 is a diagram for explaining the averaging operation of the averaging means 3 according to the second embodiment of the present invention. As shown in this figure, the averaging means 3 performs SUM _nbi
SUM _{n-1b i} , SUM _n-2bi, and SUM _n-1bi output from the integrating means 1 before being calculated and stored in the storage means 2.
Addition or averaging with UM _n−3 , _bi is performed. Where SU
M _n−1 , _bi , SUM _n−2 , _bi , and SUM _n−3 , _bi are the i-th frame of the (n−1) -th frame, the (n-2) -th frame, and the (n−3) -th frame, respectively, as shown in FIG. Are obtained by adding or averaging the pixel levels in the block of FIG. in this case,
The storage means 2 stores the output of the integrating means 1 for three frames. Here, SUM _n−1bi , SUM _n−2 , _{b i} , and SU
The result of adding or averaging M _n−3 , _i is described as AVE _ni .

FIG. 10 is a diagram for explaining the addition operation of the addition unit 7 according to the second embodiment of the present invention. The adder 7 adds the output of the integrating means 1 to blocks in the image vertical direction corresponding to N periods of flicker components in the frame. The portion of the image composed of the vertical blocks for the N periods is referred to as a still portion determination block.
Assuming that the starting block number to be added in the j-th still part determination block of the n-th frame is m, and the number of blocks in the image vertical direction corresponding to N periods of flicker components in the frame is q, the addition is performed. Means output B-SUM _nbj
Can be represented by the following formula. B-SUM _nbj = SUM _nm + SUM _{nm + 1} + ... + SU
M _{nm + q-1}

FIG. 10 shows the case where N = 1 and j = 1. As in the first embodiment, when the power frequency of the light source is 5
When 0 Hz and the frame period are 30 Hz, N is 1 to 3
Take an integer value up to. When N = 1 shown in FIG. 10, j takes an integer value from 1 to 3. in this way,
The addition result obtained by adding the output of the integrating means 1 to the N cycles of the flicker component in the frame has the same luminance level change component due to the periodic change of the light source brightness in any frame.

FIG. 11 is a diagram for explaining the processing of the stationary part extracting unit 9 according to the second embodiment of the present invention. The stationary part extraction unit 9 calculates a _difference between the addition result B-SUM _nbj calculated by the addition unit 7 and the addition result B-SUM _{n-1 bj of} one frame before read from the storage unit 8,
If the value is equal to or smaller than a predetermined threshold TH, the stationary part determination block is determined to be a stationary part.

Dividing means 5 divides the judgment result of stationary part extracting section 9
Given to. The dividing means 5 adds the integrating means 1 to the block determined as a stationary part by the stationary part extracting means 4.
A is the output SUM _nbi that the output of the averaging means 3
SUM _nbi / AVE _nbi is calculated using VE _nbi .
The flicker determining means 6 determines the presence or absence of flicker using the output of the dividing means 5. The configuration and operation of the flicker determining means 6 are the same as in the first embodiment.

The flicker detection device according to the second embodiment of the present invention is particularly effective for signals picked up by an image pickup device using a color filter. FIG. 12 shows an arrangement of color filters for a single-chip image sensor. Here, (a) shows a complementary color filter array, and (b) shows a part of a Bayer array which is a kind of primary color filter. As shown in these figures, different color filters are attached to the respective pixels of the image sensor.

In the complementary color filter shown in FIG. 12A, a line in which cyan Cy and yellow Ye are alternately arranged by one pixel and a line in which magenta Mg and green G are alternately arranged by one pixel are formed. Are arranged alternately one by one. When adding two lines of the output of the image sensor using this color filter, four pixels surrounded by a dotted line are regarded as one block, and a plurality of the same blocks are integrated. The signal in one block is as follows: Cy + Mg + Ye + G = 2R + 3G + 2B 、 Y, and almost the same signal as the luminance signal Y is obtained. By using a signal obtained by integrating a number of signals close to the luminance signal, flicker detection can be performed using the luminance signal, so that highly accurate flicker detection can be performed.

Next, in the case of the Bayer arrangement shown in FIG. 12B, a line in which red R and green G are alternately arranged by one pixel, and a line in which green G and blue B are alternately arranged by one pixel. Lines are alternately arranged line by line. When adding two lines of the output of the image sensor using this color filter, four pixels surrounded by a dotted line are regarded as one block, and a plurality of the same blocks are integrated. The signals in one block are as follows: R + G + G + B = R + 2G + B ≒ Y, and almost the same signal as the luminance signal Y is obtained. By using a signal obtained by integrating a number of signals close to the luminance signal, flicker detection can be performed using the luminance signal, so that highly accurate flicker detection can be performed.

In the second embodiment of the present invention, the pixel level is integrated for each area in the frame where the flicker components are substantially equal, and the area at the same image position in a plurality of past frames is compared with the area for each area. Averaging the integration result, extracting a static portion of the image using the integration result, and dividing the integration result for each region by the averaging result for each region for the extracted static portion; Since the result is subjected to frequency analysis to determine the presence or absence of flicker, the size of the stationary portion determination block can be reduced as compared with the first embodiment. The accuracy is improved. Also, as in the first embodiment, 50 Hz, 6 Hz
0 Hz flicker detection can be performed with high accuracy.

(Third Embodiment) In the flicker detection device according to the third embodiment of the present invention, the stationary part extracting section 9
Difference means for taking the difference between the current addition result of the addition unit 8 and the past addition result of the block at the same image position; and division means for dividing the output of the difference means by the current addition result or the past addition result And threshold processing means for determining whether or not the block is a stationary part based on a magnitude relationship between the division result and a predetermined threshold. The configuration of the parts other than the stationary part extraction unit 9 is the same as that of the first embodiment.

FIG. 13 is a block diagram showing the configuration of the stationary part extracting unit 9 in the flicker detection device according to the third embodiment of the present invention. The stationary part extracting unit 9 includes an adding unit 7
Means 31 to which the output of the storage unit 8 and the output of the storage unit 8 are input
And a dividing means 32 to which the output of the difference means 31 and the output of the adding section 7 are inputted, and a threshold processing means 33 to which the output of the dividing means 32 is inputted.

The difference means 31 calculates the difference between the current addition result B-SUM _nj of the addition section 7 and the addition result B-SUM _n-1j one frame before read from the storage section 8 and divides the result.
Output to 32. The dividing means 32 divides the output of the difference means 31 by the output of the adding section 7 and outputs the result to the threshold value processing means 33.
The output of the dividing means is | B-SUM _nj -B-SUM _n-1j | / B-SUM _nj .

When the value of the above equation is equal to or smaller than the preset threshold value TH, the threshold value processing means 33 determines that the j-th stationary portion determination block of the n-th frame is a stationary portion. The determination result of the threshold value processing unit 33 is given to the division unit 5.
The operations of the dividing means 5 and the flicker determining means 6 are the same as in the first embodiment.

In the third embodiment of the present invention, the ratio of the amount of change between B-SUM _nj of the current frame and B-SUM _{n-1j of} one frame before is calculated, and threshold processing is performed. So
Even if the level of the target video signal is different, highly accurate still portion extraction can be performed, and as a result, the flicker detection accuracy can be improved.

In the division means 32, the same effect can be obtained by dividing by B-SUM _n-1j which is the output of the storage unit 8 instead of dividing by B-SUM _nj which is the output of the adding unit 7. Can be

(Fourth Embodiment) In the flicker detection device according to the fourth embodiment of the present invention, the stationary part extraction unit 9
Averaging means for averaging the current addition result of the addition unit 7 and the past addition result of the still part determination block at the same image position, and a difference between the current addition result and the averaging result of the averaging means. Difference means for taking, difference means for dividing the difference result of the difference means by the averaging result of the averaging means or the current addition result, and the block is determined by the magnitude relationship between the division result and a predetermined threshold value. Threshold processing means for determining whether or not the part is a stationary part.

FIG. 14 is a block diagram showing the configuration of the stationary part extracting section 9 in the flicker detection device according to the fourth embodiment of the present invention. The stationary part extracting unit 9 includes an adding unit 7
Averaging means 41 to which the output of the storage section 8 and the output of the storage section 8 are input.
And the difference means 42 to which the output of the addition section 7 and the output of the averaging means 41 are inputted, the division means 43 to which the output of the difference means 42 and the output of the addition section 7 are inputted, and the output of the division means 43 to be inputted. And a threshold processing means 33 for performing the processing.

The averaging means 41 calculates the current addition result B-SUM _nj of the addition section 7 and the addition results B-SUM _n-1j and B-SUM _{n of the} previous and next frames read from the storage section 8. _-2j is averaged and output to the difference means 42. _Assuming that the output of the averaging means 41 is B-AVE _nj , B-AVE _nj = (B-SUM _nj + B-SUM _n-1j + B-
SUM _n-2j ) _×× .

The difference means 42 comprises a current addition result B-SUM _nj of the addition section 7 and an average result B-AVE _nj of the averaging means.
And outputs the result to the dividing means 43. Division means 43
Divides the output of the difference means 42 by the output of the adder 7 and outputs the result to the threshold value processing means 44. The output of the dividing means is | B-SUM _nj -B-AVE _nj | / B-SUM _nj .

When the value of the above equation is equal to or less than the preset threshold value TH, the threshold value processing means 44 determines that the j-th stationary portion determination block of the n-th frame is a stationary portion. The judgment result of the threshold value processing means 44 is given to the division means 5.
The operations of the dividing means 5 and the flicker determining means 6 are the same as in the first embodiment.

In the fourth embodiment of the present invention, the ratio of the change amount between the addition result B-SUM _nj of the current frame and the average value B-AVE _nj of the addition results of the past frames is calculated, and the threshold is calculated. Since the value processing is performed, the past frame to be compared with the current frame is stabilized. For this reason, a still portion can be accurately extracted from a moving image.
The accuracy of detecting flicker at 60 Hz is also improved.

In the dividing means 32, B-SUM _nj
The same effect can be obtained by dividing by B-AVE _nj instead of dividing by. Further, the same effect can be obtained even if the averaging means 41 is constituted by a recursive filter or an FIR filter instead of the averaging.

(Fifth Embodiment) In a flicker detection / correction device according to a fifth embodiment of the present invention, an image sensor is provided based on the output of the flicker detection device according to the first to fourth embodiments. And the gain of the video signal generated by the image sensor.

FIG. 15 is a block diagram showing a configuration of an image pickup apparatus provided with a flicker detection / correction apparatus according to the fifth embodiment of the present invention. This flicker detection / correction device is configured as a part of an imaging device.

This image pickup apparatus includes an image pickup means 53 such as a MOS image pickup element, an AGC amplification means 54 for controlling the level of a video signal generated by the image pickup means 53, and an AD for digitizing the output of the AGC amplification means 54. A conversion unit 55, a driving unit 56 for driving the imaging unit 53, a flicker detection unit 51 for detecting flicker from an output of the AD conversion unit 55, a flicker detection output of the flicker detection unit 51, and an output of the AD conversion unit 55. And a flicker correction control unit 52 for generating an AGC gain control signal and a shutter speed control signal for the image sensor 53. Here, the flicker detection means 51 and the flicker correction control means 52 constitute a flicker detection / correction device.

In the image pickup apparatus, an image pickup means 53 picks up an image of a subject under a light source whose brightness periodically changes, and generates a video signal. The imaging means 53 is driven by the driving means 56. The gain of the AGC amplifier 54 is controlled according to an AGC gain control signal described later, and controls the level of the input video signal. The AD converter 55 converts the video signal output from the AGC amplifier 54 into a digital video signal. The flicker detection unit 51 has the same configuration as the flicker detection unit described in the first to ninth embodiments, and performs flicker detection using the digital video signal output from the AD conversion unit 55. The flicker correction control unit 52 creates a shutter speed control signal using the output of the flicker detection unit 51 and the output of the AD conversion unit 55 and supplies the shutter speed control signal to the driving unit 56, and also creates an AGC gain control signal. The signal is supplied to the AGC amplifier 54.

The flicker correction control means 52 performs the following processing according to the flowchart shown in FIG. (1) Initially set to mode = 50 when the power is turned on (step S1 → S2). Thereafter, the following loop operations (2) to (7) are performed. (2) Obtain a video signal level (step S3). (3) The automatic gain control signal and the shutter speed are set according to the video signal level and the mode (step S4). (4) Obtain a flicker detection result (step S5). (5) If 50 Hz flicker is present from the flicker detection result, mode = 50 is set (step S6 → S
8). (6) If the result of the flicker detection indicates that there is a flicker of 60 Hz, mode = 60 is set (steps S6 → S7 →
S9). (7) If there is no flicker or it is unknown from the flicker detection result, the mode is held (steps S6 → S7 → S
10)

Next, a method of setting the AGC gain and the shutter speed will be described with reference to FIG. (A) of this figure shows the set value of the AGC gain according to the light amount, and (b) shows the set value of the shutter speed when mode = 50 according to the light amount. Since the video signal level is proportional to the amount of light at the time of imaging, the shutter speed and AGC are controlled so that the level of the video signal is kept constant even when the amount of light fluctuates.

First, the AGC gain is changed according to the light amount as shown in FIG.
Control is performed as shown in FIG. Here, MIN is the minimum value of the range that the AGC gain can take, and MAX is the maximum value.

When the amount of light is small, the shutter speed is set to the frame frequency (30 Hz in this case) and the power frequency of the light source.
(50 Hz in this case), which is the slowest shutter speed at which flicker does not occur, that is, 3/100 seconds, which is an integer multiple of the power supply frequency and equal to or lower than the frame frequency.

As the amount of light increases, the AGC gain is gradually lowered, and when MIN is taken, the shutter speed is set to an integral multiple of the power supply frequency and faster than the current value (2/100 seconds). At the same time, the AGC gain is M
IN is changed by 3/2 times, which is the reciprocal of the ratio of the change amount of the shutter speed. As described above, by controlling the shutter speed and the AGC gain in conjunction, it is possible to avoid a sudden change in the video level when the shutter speed changes, thereby preventing image quality deterioration.

When the light amount becomes sufficiently large, the shutter speed becomes the fastest speed at which flicker does not occur (1/100 second in the case of 50 Hz), and when the AGC gain becomes MIN, the shutter speed is increased in proportion to the light amount. Go faster. At this time, the AGC gain is fixed at MIN. With this setting, the video signal level does not saturate even when the light amount increases, so that the dynamic range is widened and the video is displayed.

A value of 1/100 second or more (for example, 1/100 second)
(250 seconds), it is preferable to provide a hysteresis so as not to frequently return.

Although the above description is for the shutter speed when the power supply frequency is 50 Hz, the same applies to the case where the power supply frequency is 60 Hz.
/ 120 seconds, 3/120 seconds,...

In FIG. 15, the AGC amplifying means 54 controls the level of the analog video signal.
Instead of the C amplifying means 54, a digital AGC amplifying means may be provided at the subsequent stage of the AD converting means 55 to perform a gain control digitally.

As described above, according to the fifth embodiment of the present invention, the 50 Hz, 60 Hz shown in the first to fourth embodiments is used.
By controlling the shutter speed of the image sensor and the gain of the video signal according to the flicker frequency detected using a flicker detection device capable of accurately detecting the flicker of Hz and the input video signal level, high accuracy is achieved. Flicker correction can be performed. Also, by simultaneously changing the shutter speed, the gain of the video signal is changed by the reciprocal of the change amount, thereby preventing a sudden change in the luminance level due to the change in the shutter speed and preventing the image quality from deteriorating. Becomes Further, even when the amount of incident light increases, it is possible to avoid a state in which an image is not displayed.

In the above embodiments, the averaging means 3 averages a plurality of past frames. However, the same effect can be obtained by averaging including the current frame.

[0081]

As described above, according to the flicker detection apparatus and method of the present invention, flicker is detected by using a still portion of an image, so that even when the luminance level fluctuates due to the movement of a subject or the like. In addition, it is possible to detect an intra-frame flicker generated at the time of imaging using a MOS-type imaging device.

Further, according to the flicker detection / correction apparatus and method of the present invention, an image pickup for generating a video signal in accordance with the flicker frequency detected by the flicker detection apparatus and method of the present invention and the input video signal level. Since the shutter speed of the element and the gain of the video signal are controlled, 5
Flicker at both 0 Hz and 60 Hz can be automatically determined, and flicker correction can be performed. Further, when the amount of incident light increases, the shutter speed is increased and the image is taken, so that even in a bright case, the image can be taken without saturating the video signal level.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a flicker detection device according to a first embodiment of the present invention;

FIG. 2 is a diagram for explaining an operation of an integrating means in FIG. 1;

FIG. 3 is a diagram for explaining an operation of an averaging unit in FIG. 1;

FIG. 4 is a diagram for explaining an operation of an adding unit in FIG. 1;

FIG. 5 is a diagram for explaining a determination process of a stationary part extraction unit in FIG. 1;

FIG. 6 is a diagram showing a configuration example of flicker determination means in FIG. 1;

FIG. 7 is a diagram showing an example of the output of the division means and DFT means in FIG. 1;

FIG. 8 is a diagram for explaining the operation of the integrating means in the flicker detection device according to the second embodiment of the present invention;

FIG. 9 is a diagram for explaining the operation of the averaging means in the flicker detection device according to the second embodiment of the present invention;

FIG. 10 is a diagram for explaining an operation of an adding unit in the flicker detection device according to the second embodiment of the present invention;

FIG. 11 is a diagram illustrating a determination process of a stationary portion extraction unit in the flicker detection device according to the second embodiment of the present invention;

FIG. 12 is a diagram illustrating an example of a color filter used for an image sensor that generates a video signal input to the flicker detection device according to the second embodiment of the present invention;

FIG. 13 is a block diagram illustrating a configuration of a still part extracting unit in the flicker detection device according to the third embodiment of the present invention;

FIG. 14 is a block diagram illustrating a configuration of a still part extracting unit in the flicker detection device according to the fourth embodiment of the present invention;

FIG. 15 illustrates flicker detection and detection according to a fifth embodiment of the present invention.
A block diagram showing a configuration of an imaging device including a correction device,

FIG. 16 is a flowchart showing processing of flicker correction control means in FIG. 14;

FIG. 17 is a view for explaining the operation of the flicker correction control means in FIG. 14;

FIG. 18 is a diagram illustrating the principle of generation of flicker when the power supply frequency is 50 Hz.

FIG. 19 is a diagram for explaining the principle of generation of flicker when the power supply frequency is 60 Hz.

FIG. 20 is a diagram illustrating the principle of correcting flicker.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Integrating means 2 Storage means 3 Averaging means 4 Stationary part extracting means 5, 32, 43 Dividing means 6 Flicker judging means 7 Addition part 8 Storage part 9 Stationary part extracting part 21 DFT means 22, 33, 44 Threshold processing means 31, 42 Difference means 41 Averaging means 51 Flicker detection means 52 Flicker correction control means 53 Imaging means

 ────────────────────────────────────────────────── ─── Continuing on the front page (72) Inventor Shin Shinbe, 3-3-1 Tsunashimahigashi, Kohoku-ku, Yokohama-shi, Kanagawa F-term (reference) in Matsushita Communication Industrial Co., Ltd. 5C021 XA43 YA07 5C022 AA11 AB37 AC42 5C065 AA01 BB21 DD15 DD17 EE05 EE06 EE08 GG17 GG22 GG23 GG24 GG27 GG50

Claims (14)

[Claims] An integrating means for integrating a pixel level for each predetermined area in a frame or a field; an averaging means for averaging the integrated result for each area at the same image position in a plurality of frames or fields; A static part extracting means for extracting a static part of the image using the output of the integrating means, and for the static part extracted by the static part extracting means,
Division means for dividing the integration result for each area of the integration means by the averaging result for each area of the averaging means; and flicker determination means for frequency-analyzing the division result of the division means to determine the presence or absence of flicker. And a flicker detection device. 2. The flicker detection device according to claim 1, wherein the predetermined area is one line. 3. The predetermined area is an area where flicker components in a frame or a field are substantially equal.
The flicker detection device according to the above. 4. The stationary part extracting means according to claim 1, wherein said stationary part extracting means computes the integration result of each area with respect to a stationary part determination block comprising a plurality of areas in the vertical direction of the image corresponding to an integral multiple of a beat cycle of flicker in a frame or a field. Addition unit to add, based on the amount of change between the addition result of the addition unit in the current frame or field, and the addition result of the same image position in the past frame or field,
4. The flicker detection device according to claim 1, further comprising: a still part extracting unit that determines whether the block is a still part. 5. 5. The still part extracting unit includes: a difference unit that calculates a difference between a current addition result of the adding unit and a past addition result of a still part determination block at the same image position; and an output of the difference unit. Division means for dividing by the current addition result or the past addition result; and threshold processing means for determining whether the block is a stationary part based on a magnitude relationship between the division result and a predetermined threshold value, The flicker detection device according to claim 4, comprising: 6. An averaging means for averaging a current addition result of the addition unit and a past addition result of a still part determination block at the same image position, wherein the averaging means calculates an average value of the current addition result. Difference means for taking a difference from the averaging result of the averaging means, division means for dividing the difference result of the difference means by the averaging result of the averaging means or the current addition result; and 5. The flicker detection device according to claim 4, further comprising a threshold value processing unit for determining whether or not the block is a stationary portion based on a magnitude relationship with the threshold value. 7. A flicker detection device according to claim 1, a control signal for controlling a shutter speed of an imaging unit for generating a video signal based on an output of the flicker detection device, and A flicker detection / correction device comprising: a flicker correction control unit that generates a control signal for controlling a gain of the video signal. 8. A pixel level for each predetermined area in a frame or a field is integrated, an integration result for each area at the same image position in a plurality of frames or fields is averaged, and a still image is stored using the integration result. Flicker flicker detection that extracts a portion, divides the integrated result for each area by the averaged result for each area, and performs frequency analysis on the divided result to determine whether or not flicker exists for the extracted stationary part. Method. 9. The method according to claim 8, wherein the predetermined area is one line. 10. The flicker detection method according to claim 8, wherein the predetermined area is an area where flicker components in a frame or a field are substantially equal. 11. An integrated result for each area is added to a stationary part determination block composed of a plurality of areas in the image vertical direction corresponding to an integral multiple of the beat cycle of flicker in a frame or a field, and the current frame or the A static portion is extracted by determining whether the block is a static portion based on the amount of change between the addition result in the field and the addition result of the same image position in the past frame or field. The flicker detection method according to claim 10. 12. A difference between the current addition result and a past addition result of a still portion determination block at the same image position is obtained, and the difference result is divided by the current addition result or the past addition result. 12. The flicker detection method according to claim 11, wherein it is determined whether or not the block is a stationary portion based on a magnitude relationship between a division result and a predetermined threshold value. 13. An average of a current addition result of the addition unit and a past addition result of a still portion determination block at the same image position, and a difference between the current addition result and the averaging result is calculated. 12. The flicker detection according to claim 11, wherein a result is divided by the averaging result or the current addition result, and whether or not the block is a stationary portion is determined based on a magnitude relationship between the division result and a predetermined threshold. Method. 14. A control of a shutter speed of an imaging apparatus for generating a video signal based on a flicker frequency detected by the flicker detection method according to claim 8, and a level of the video signal. Method for detecting and correcting flicker.