JP2009017214A - Flicker detection device, and flicker detection method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately detect a flicker component even when the flicker component has a distribution in an image signal or an intensity distribution in a vertical direction of a current frame or a current field is varied. <P>SOLUTION: A plurality of flicker detection frames are set to a one-picture portion of an image signal which is obtained by an imaging element, at least in a horizontal direction (S302). Flicker components caused as variations in luminance level in a vertical direction are detected per flicker detection frame (S303 to S305). <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a flicker detection apparatus and a flicker detection method, and more particularly to a flicker detection apparatus and a flicker detection method for detecting a flicker component in an image picked up using an XY address type image pickup device.

  In an imaging apparatus using an XY address type solid-state imaging device typified by a CMOS image sensor, when shooting a subject under a fluorescent lamp, moving images are taken in a vertical direction as shown in FIG. May appear. This phenomenon is called fluorescent lamp flicker, and is due to the difference between the commercial power supply frequency and the vertical synchronization frequency of the imaging device.

  Therefore, there is a conventional technique for detecting and correcting a flicker component by detecting a fluctuation component (hereinafter referred to as a flicker component) of a periodic signal level in the vertical direction and multiplying by its inverse gain (see Patent Document 1). ).

Japanese Patent Laid-Open No. 11-122513

  However, the method described in Patent Document 1 is a method for detecting a flicker component using a vertical intensity distribution generated by integrating image signals of current and past frames (or fields) in the horizontal direction. Therefore, there is a problem that when the flicker component is different on the left and right of the screen, the flicker component cannot be detected accurately. This problem may occur when, for example, the left side of the subject is illuminated by a fluorescent lamp and the right side is illuminated by sunlight as shown in FIG.

Another problem is that the detected flicker component depends on the movement of the subject.
FIG. 18 shows an image signal for the latest three frames obtained from the subject, a vertical intensity distribution obtained from the past two frames of the image signal, a vertical intensity distribution of the current frame, and these two intensity distributions. The flicker component obtained by dividing is shown.
(A) shows the case where the subject is stationary, and (b) shows the case where the subject moves in the third frame (current frame).

  With the technique described in Patent Document 1, when the subject is stationary as shown in FIG. 18A, the flicker component can be accurately detected. However, if the subject moves and the vertical intensity distribution of the current frame or field changes as shown in FIG. 18B, the flicker component cannot be detected accurately.

  The present invention has been made in view of such problems of the prior art. The present invention relates to a flicker detection apparatus and flicker capable of accurately detecting a flicker component even when the flicker component has a distribution in an image signal or when the intensity distribution in the vertical direction of the current frame or current field varies. An object is to provide a detection method.

  That is, the above-described object is a flicker detection device that detects a flicker component in an image signal obtained by an image sensor, and each image signal of one pixel or more in at least the horizontal direction of an image signal for one screen. And a flicker detection unit configured to detect a flicker component generated as a change in luminance level in the vertical direction of the image signal in each of the plurality of flicker detection frames. Achieved by the detection device.

  Another object of the present invention is to provide a flicker detection method for detecting a flicker component in an image signal obtained by an image sensor, wherein each image signal for one screen is an image signal of one pixel or more in at least the horizontal direction. And a flicker detection step for detecting a flicker component generated as a change in luminance level in the vertical direction of the image signal in each of the plurality of flicker detection frames. This is also achieved by the detection method.

  As described above, according to the present invention, even when the flicker component has a distribution in the image signal or the intensity distribution in the vertical direction of the current frame or current field varies, the flicker component can be accurately obtained. It can be detected.

(First embodiment)
DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the invention will be described in detail with reference to the accompanying drawings.
FIG. 1 is a block diagram showing a configuration example of a flicker correction apparatus 10 as an example of a flicker detection apparatus according to the first embodiment of the present invention.
In FIG. 1, a flicker detection unit 1 detects a flicker component from an image signal captured by an XY address type image sensor such as a CMOS image sensor. In the present embodiment, it is assumed that image signals are sequentially accumulated by sequentially resetting the image sensor in the vertical direction for each line, and read out by sequentially performing horizontal scanning for each line.

  The flicker correction value generation unit 2 obtains the reverse characteristic (reciprocal number) of the flicker component detected by the flicker detection unit 1 as a flicker correction value, and generates a flicker correction value for each pixel. The multiplier 3 multiplies the flicker correction value obtained by the flicker correction value generation unit 2 and the image signal, and outputs the result as a corrected image signal. The storage unit 9 is a storage device such as a semiconductor memory or a hard disk, and stores the image signal of the current frame or current field. The control unit 11 controls the overall operation of the flicker correction apparatus 10.

FIG. 2 is a block diagram illustrating a configuration example of the flicker detection unit 1 in FIG.
In FIG. 2, the block division unit 100 sets a region (flicker detection frame) for detecting a flicker component in the horizontal direction and the vertical direction with respect to the image signal for one screen. The addition averaging unit 101 calculates the average value of the luminance levels of the pixels for each of the detection areas set by the block dividing unit 100. The subtractor 102, the multiplier 103, the adder 104, and the memory 105 form a so-called cyclic low-pass filter 110 by performing the following calculation.

  Here, ave represents the output of the averaging unit 101, and mout represents the output from the memory 105. mem indicates an output from the adder 104 and indicates a value newly stored in the memory 105. K is a filter coefficient of the cyclic low-pass filter 110.

The divider 106 divides the output of the averaging unit 101 and the output from the memory 105 to calculate and output a flicker component for each flicker detection frame.
That is, in each horizontal detection area (consisting of a plurality of vertical detection areas), a flicker component generated as a change in the luminance level in the vertical direction is detected.

(Operation)
Next, an example of the operation of the flicker correction apparatus of this embodiment will be described with reference to the flowchart of FIG.
First, in S300, an image signal is input to the flicker correction apparatus 10. In step S <b> 301, the input image signal is stored in the storage unit 9. In step S302, the block dividing unit 100 of the flicker detection unit 1 sets flicker detection frames of M blocks in the horizontal direction and N blocks in the vertical direction in the input image signal for one screen.

  In S303, for each flicker detection frame set in S302, the average value of the luminance level is calculated by the addition averaging unit 101 of the flicker detection unit 1. Next, in S304, the low-pass filter processing is performed for each flicker detection frame by the cyclic low-pass filter 110 of the flicker detection unit 1 based on, for example, the calculation of [Equation 1].

  In S305, the image signal averaged for each flicker detection frame in S303 and the image signal subjected to the low-pass filter processing in each flicker detection frame in S304 are divided by the divider 106 of the flicker detection unit 1. As a result, a flicker component is extracted for each flicker detection frame. The extracted flicker component is given from the flicker detection unit 1 to the flicker correction value generation unit 2.

  Next, in S306, the flicker correction value generation unit 2 obtains the reciprocal number (reverse characteristic) of the flicker component given from the flicker detection unit in S305, and generates a flicker correction value corresponding to the pixel position.

  In S307, the control unit 11 sequentially reads out the image signal stored in S301 from the storage unit 9. In step S308, the flicker correction value corresponding to the read image signal is supplied from the flicker correction value generation unit 2 to the multiplier 3, and the two are multiplied to perform flicker correction. The multiplication result of the multiplier 3 is output from the flicker correction apparatus 10 as an image signal after flicker correction.

  In this example, in order to obtain a flicker component, a flicker detection frame of M blocks in the horizontal direction and N blocks in the vertical direction is set. However, the flicker detection frame only needs to include an image signal of at least one pixel. That is, a flicker component may be obtained for each pixel.

  As described above, according to the present embodiment, flicker components are detected in a plurality of regions in both the horizontal direction and the vertical direction of the input image signal. Therefore, even when the flicker component has a distribution in the image signal or when the intensity distribution in the vertical direction of the current frame or current field varies, the flicker component can be detected and corrected with high accuracy.

(Second Embodiment)
Next, a second embodiment of the present invention will be described.
FIG. 4 is a block diagram showing a configuration example of a flicker correction apparatus according to the second embodiment of the present invention.

  In FIG. 4, the same components as those in the first embodiment are denoted by the same reference numerals, and redundant description is omitted. The difference from the first embodiment is that the flicker correction device 10 ′ includes a flicker component evaluation unit 4, and the flicker component evaluation unit 4 controls the entire imaging device (not shown). This is a point connected to the control unit 5.

The flicker component evaluation unit 4 evaluates the reliability of the flicker component detected by the flicker detection unit 1. Then, the flicker component evaluation unit 4 outputs the reliability of the flicker component as the evaluation result and the flicker component detected by the flicker detection unit 1 to the flicker correction value generation unit 2.
The flicker correction value generation unit 2 controls the generation method of the flicker correction value according to the reliability of the flicker component given from the flicker component evaluation unit 4.

FIG. 5 is a block diagram illustrating a configuration example of the flicker component evaluation unit 4 in the present embodiment.
In FIG. 5, the flicker model selection unit 400 selects a flicker model having a high correlation with the flicker component detected by the flicker detection unit 1 among the plurality of flicker models output from the system control unit 5.

  Here, the flicker model is data of a flicker component in a specific flicker state, for example, a model of a specific flicker component having a gentle slope in the horizontal direction and approximated by a sin wave in the vertical direction. It is.

  The difference extraction unit 401 obtains a difference between the flicker component detected by the flicker detection unit 1 and the flicker model selected by the flicker model selection unit 400 and outputs the difference to the reliability determination unit 402. In the present embodiment, since the flicker component is detected for each flicker detection frame, a flicker model is also generated for each flicker detection frame. Specifically, the averaging process can be applied to the flicker model as well as the detection of the flicker component. When the flicker component is detected in units of pixels, the flicker model is also converted into a value in units of pixels.

  The reliability determination unit 402 determines the reliability of the flicker component detected by the flicker detection unit 1 based on the difference obtained by the difference extraction unit 401, and outputs the determination result to the flicker correction value generation unit 2. Here, the reliability of the flicker component means information indicating the probability of flicker that the detected flicker component is a fluctuation component of the periodic signal level in the vertical direction generated by the fluorescent lamp flicker.

  Next, an example of the operation of the flicker correction apparatus in the present embodiment will be described with reference to the flowchart of FIG. In FIG. 6, steps that perform the same operations as in FIG. 3 are given the same reference numerals, and descriptions thereof are omitted.

The process from the input of the image signal (S300) to the detection of the flicker component (S305) is the same as in FIG.
In step S <b> 606, the flicker component evaluation unit 4 acquires, for example, a plurality of flicker models from the system control unit 5. In step S <b> 607, the flicker component evaluation unit 4 calculates the correlation for one screen between each of the plurality of flicker models acquired from the system control unit 5 and the flicker component detected by the flicker detection unit 1 for each flicker detection frame. Calculated as the sum of correlations. Then, the flicker model that shows the highest correlation is selected. The correlation for one screen can be calculated by, for example, the calculation shown in the following [Equation 2].

Here, x and y represent the coordinate positions (1 ≦ x ≦ M, 1 ≦ y ≦ N) of each flicker detection frame. Further, a _xy represents the value of the flicker component of each flicker detection frame of the flicker model, and b _xy represents the value of the flicker component of each flicker detection frame of the flicker component detected by the flicker detection unit 1.

  In S608, the flicker component evaluation unit 4 extracts the difference between the flicker model selected in S607 and the flicker component detected by the flicker detection unit 1. Specifically, for the selected flicker model and the flicker component detected by the flicker detection unit 1, the value obtained by [Equation 2] for each flicker detection frame can be used as a difference.

In step S609, the flicker component evaluation unit 4 determines whether the difference extracted in step S608 is larger than the threshold value Th for each flicker detection frame.
When it is determined in S609 that the difference is equal to or smaller than the threshold value Th, the flicker component evaluation unit 4 determines that the reliability of the flicker component detected by the flicker detection unit 1 is high (S611). Then, the flicker component evaluation unit 4 supplies the reliability determination result to the flicker correction value generation unit 2.
In S612, the flicker correction value generation unit 2 uses the flicker component detected by the flicker detection unit 1, and generates a flicker correction value for the detection frame corresponding to the flicker component.

  On the other hand, if it is determined in S609 that the difference is larger than the threshold value Th, the flicker component evaluation unit 4 determines that the reliability of the flicker component detected by the flicker detection unit 1 is low, and the determination result is used as the flicker correction value generation unit. 2 is notified (S610). Then, the flicker correction value generation unit 2 generates a flicker component that replaces the flicker component determined to have low reliability by using the flicker component determined to have high reliability (S613).

  This interpolated flicker component is detected in a plurality of flicker detection frames closest to the flicker detection frame in which the flicker component determined to have low reliability is detected, and is determined to have high reliability at the time of performing interpolation processing. The flicker component can be generated by interpolation.

In S614, the flicker correction value generation unit 2 generates a flicker correction value using the interpolated flicker component. Note that the operation of the flicker correction value generation unit 2 in S612 and S614 differs only in whether the flicker component to be used is actually detected or generated by interpolation.
In S307 and S308, as in the first embodiment, flicker correction is performed by reading the image signal stored in the storage unit 9 and multiplying by the flicker correction value.

  In this embodiment as well, as in the first embodiment, in order to obtain a flicker component, an image signal for one frame is divided into M × N flicker detection frames of M blocks in the horizontal direction and N blocks in the vertical direction. The flicker component was detected and corrected for each flicker detection frame. However, the flicker detection frame may include an image signal of at least one pixel. That is, a flicker component may be obtained for each pixel. In this case, the processing of S302 and S303 is not necessary.

  There is no particular restriction on the interpolation method in S613. For example, as the distance from the flicker detection frame for generating the interpolated flicker component is closer, a larger weight may be given, or a value detected from the flicker detection frame closer in the horizontal direction than in the vertical direction may be given a larger weight. Basically, the flicker correction value may be generated based on the flicker component determined to have high reliability or the interpolated flicker component determined from the flicker component determined to have high reliability.

  As described above, according to the present embodiment, the reliability of the detected flicker component can be evaluated by comparing the detected flicker component with the flicker model. Then, instead of the flicker component determined to be low in reliability, a flicker component interpolated from surrounding flicker components determined to be high in reliability is used. As a result, a more reliable flicker detection result can be obtained. Therefore, highly accurate flicker correction can be performed.

(Third embodiment)
Next, a third embodiment of the present invention will be described.
FIG. 7 is a block diagram showing a configuration example of a flicker correction apparatus according to the third embodiment of the present invention.
In FIG. 7, the same components as those in the second embodiment are denoted by the same reference numerals, and redundant description is omitted. The difference from the second embodiment is that the flicker correction device 10 ″ includes a flicker characteristic parameter extraction unit 6 and a flicker model generation unit 7, and a system control unit 5 is connected to the flicker component evaluation unit 4. That is not.

The flicker characteristic parameter extraction unit 6 extracts flicker characteristic parameters from the flicker components detected by the flicker detection unit 1 and outputs them to the flicker model generation unit 7. In the present embodiment, the flicker characteristic parameter is a parameter that represents the characteristic of a specific flicker component, and here is the phase, amplitude, and frequency of the specific flicker component.
Further, the flicker characteristic parameter extraction unit 6 outputs a control signal SEL indicating a flicker characteristic parameter extraction result to the flicker correction value generation unit 2.

FIG. 8 is a block diagram illustrating a configuration example of the flicker characteristic parameter extraction unit 6 in the present embodiment.
The amplitude / frequency calculation unit 800 uses the flicker component detected by the flicker component detection unit 1 to calculate the amplitude and frequency for a specific flicker component, for example, according to the procedure shown in FIG.

FIG. 9 is a flowchart for explaining an example of the operation of the amplitude / frequency calculation unit 800.
First, in S900, the amplitude / frequency calculation unit 800 applies the Fourier transform represented by [Equation 3], [Equation 4], and [Equation 5] to the flicker component detected by the flicker detection unit 1.

Here, h represents the detection frame number in the vertical direction (0 ≦ h ≦ (N−1)), and y (h) represents the value of the flicker component detected for the h-th detection frame. Further, f is represents the _{frequency, Y Re (f), Y} Im (f) represents a real part and an imaginary part when the Fourier transform, respectively. Y (f) represents an amplitude component.

In step S901, the amplitude / frequency calculation unit 800 calculates f = 50 in [Equation 3] to [Equation 5], and calculates the ₅₀ Hz amplitude component Y50 in the detected flicker component.
In S902, the amplitude / frequency calculation unit 800 performs calculation with f = 60 in [Expression 3] to [Expression 5], and calculates an amplitude component Y60 of ₆₀ Hz in the detected flicker component.

In S903, the amplitude-frequency calculation unit 800 compares the Y ₅₀ and Y _60. Here, if the direction of Y ₆₀ than Y ₅₀ is large to S904, if Y ₆₀ is was Y ₅₀ or less the processing proceeds to S905.

In S904, the amplitude / frequency calculation unit 800 outputs the amplitude Y ₆₀ and the frequency 60 Hz to the flicker model generation unit 7 as the flicker characteristic parameters. On the other hand, in S905, the amplitude-frequency calculation unit 800, a flicker characteristic parameter, and outputs the amplitude Y _50, a frequency 50Hz to flicker model generating unit 7.

  The calculation of the amplitude and frequency is performed on the entire screen in units of N detection frames arranged in the vertical direction. Therefore, in this embodiment, M sets of amplitudes and frequencies are calculated for one screen.

  Further, the phase calculation unit 801 of the flicker characteristic parameter extraction unit 6 uses the flicker component detected by the flicker component detection unit 1 to calculate the phase for a specific flicker component, for example, according to the procedure shown in FIG.

  FIG. 10 is a flowchart for explaining an example of the operation of the phase calculation unit 801. FIG. 11 is a diagram illustrating an example of the flicker component detected by the flicker detection unit 1 for one vertical column of the flicker detection frame.

The value of the flicker component detected by the flicker detection unit 1 shows a sinusoidal distribution centering on the flicker component value 1 for each column of flicker detection frames arranged in the vertical direction, for example, as shown in FIG.
Therefore, in S1000, the phase calculation unit 801 first connects flicker component values detected from adjacent flicker detection frames with a straight line, and generates an approximate waveform of the flicker component.

  Next, in S1001, the phase calculation unit 801 obtains all the line numbers V1, V2, V3,... Where the approximate waveform of the flicker component and the horizontal line representing the flicker component value = 1 intersect. As described in the first embodiment, the flicker component value is a division result between the output value of the averaging unit 101 and a value obtained by applying a low-pass filter to the output value. Therefore, flicker component value = 1 means that no flicker component is detected.

  Specifically, the flicker component value detected for each flicker detection frame is the nth (1 ≦ n ≦ m) line of the predetermined number of mlines included in the flicker detection frame, such as the first line or the middle. It is regarded as the value in the line. Then, by performing linear interpolation between adjacent flicker component values, it is possible to determine which line the intersection of the approximate waveform of the flicker component and the horizontal line of flicker component value = 1 corresponds to.

Next, in S1002, the phase calculation unit 801 obtains the difference between V1 and V2, and determines whether the difference is greater than the threshold value V _Th1 . When the difference is less than or equal to the threshold value V _{Th1, the} frequency is too high than the flicker component, and it is regarded as noise. That is, the threshold value V _Th1 is provided in order to prevent erroneous detection of the phase of the flicker component due to noise.
It is also possible to set the V1 and difference upper threshold V _Th2 of V2 (V _Th1 <V _Th2) if necessary.

When the difference between V1 and V2 is larger than the threshold value _VTh1 , the phase calculation unit 801 obtains the slopes of the flicker component waveform lines V1 and V2 in S1003.
In step S <b> 1004, the phase calculation unit 801 outputs, to the flicker model generation unit 7, the line number indicating the positive value of the obtained slope as the phase of the flicker characteristic parameter. For example, in the example of FIG. 11, if V2-V1> _VTh1 , the line number V1 is output as the phase (the flicker component phase = 0 line).

In step S <b> 1009, the phase calculation unit 801 sets “0”, which means that extraction has been performed, to the value of the control signal SEL indicating the flicker characteristic parameter extraction result.
On the other hand, if the difference between V1 and V2 is equal to or smaller than the threshold value V _Th1 in S1002, the phase calculation unit 801 determines the presence or absence of the line V3 where the approximate waveform intersects the horizontal line with flicker component value = 1 for the third time (S1005).

  If V3 is present, the phase calculation unit 801 proceeds to S1002 with the second intersection V2 as the first intersection V1 and the third intersection V3 as the second intersection V2 in S1006. In this way, the process is repeated for adjacent pairs of intersections in descending order of line number until a pair of line numbers matching the difference condition is found.

On the other hand, if there is no V3 (that is, unprocessed intersection) in S1005, the phase calculation unit 801 selects line number 0 as the phase in S1007. In step S1008, the phase calculation unit 801 sets “1” indicating that the control signal SEL indicating the extraction result of the flicker characteristic parameter could not be extracted to the flicker correction value generation unit 2 in step S1010. Output.
The flicker correction value generation unit 2 stores the value of the control signal notified from the phase calculation unit 801 in association with the flicker detection frame.

  Similarly to the calculation of the amplitude and the frequency, the calculation of the phase is performed for the entire screen in units of N detection frames arranged in the vertical direction. Therefore, in this embodiment, M phases are calculated for one screen.

  The flicker model generation unit 7 (FIG. 7) uses the flicker characteristic parameters (phase, amplitude, frequency in this case) output from the flicker characteristic parameter extraction unit 6, and performs a flicker model, for example, by calculation as shown in [Formula 6]. Generate Fm.

  Here, A represents the amplitude, and f represents the frequency of the flicker component. In addition, y represents a pixel position (that is, a line number) in the vertical direction, and l represents a phase.

  The flicker model Fm is also generated for the entire screen in units of N detection frames arranged in the vertical direction. Therefore, in this embodiment, M flicker models are generated for one screen.

  Next, FIG. 12 is a flowchart for explaining an example of the operation of the flicker correction apparatus according to this embodiment. In FIG. 12, the same reference numerals are given to the operation steps common to the first and second embodiments, and the overlapping description is omitted.

  The process from the input of the image signal (S300) to the detection of the flicker component (S305) is the same as in FIG. However, in the present embodiment, in order to generate the flicker characteristic parameter, after all the flicker component detection processing for the image signal for one screen is performed, the processing proceeds to S1206.

  In step S1206, the flicker characteristic parameter extraction unit 6 extracts flicker characteristic parameters according to the processing procedure described above with reference to FIGS. In step S <b> 1207, the flicker model generation unit 7 generates a flicker model using an operation shown in [Equation 6].

The subsequent processing is performed for each flicker detection frame.
The difference between the flicker model and the flicker component detected by the flicker component detector 1 is compared, the difference is compared with the threshold value Th, and the processing (S608 to S611) until the reliability of the detected flicker component is determined is the second. This is the same as the embodiment.

  If it is determined in S611 that the reliability is high, in S1212, the flicker correction value generation unit 2 determines whether the value of the control signal SEL notified from the flicker characteristic parameter extraction unit 6 is “0” for the corresponding detection frame. Judgment is made.

  If the value of the control signal SEL is “0”, the flicker correction value generation unit 2 generates a flicker correction value based on the detected flicker component (S612). On the other hand, if the value of the control signal SEL is “1” in S1212, the flicker correction value generation unit 2 generates “1” as the flicker correction value (S1218).

  If the difference is larger than the threshold value Th in S609, it is determined that the reliability of the detected flicker component is low (S610). In this case, as in the second embodiment, the flicker correction value generation unit 2 interpolates surrounding flicker components with high reliability in S613, and replaces the flicker components that have been determined to have low reliability with flicker components. Generate.

  In step S <b> 1214, the flicker correction value generation unit 2 determines whether the value of the control signal SEL notified from the flicker characteristic parameter extraction unit 6 is “0” for the corresponding detection frame.

If the value of the control signal SEL is “0”, the flicker correction value generation unit 2 generates a flicker correction value based on the interpolated flicker component (S614). On the other hand, if the value of the control signal SEL is “1” in S1212, the flicker correction value generation unit 2 generates “1” as the flicker correction value (S1216).
After that, in S307 and S308, as in the first embodiment, flicker correction is performed by reading the image signal stored in the storage unit 9 and multiplying by the flicker correction value.

  In this embodiment as well, as in the first embodiment, in order to obtain a flicker component, an image signal for one frame is divided into M × N flicker detection frames of M blocks in the horizontal direction and N blocks in the vertical direction. The flicker component was detected and corrected for each flicker detection frame. However, the flicker detection frame may include an image signal of at least one pixel. That is, a flicker component may be obtained for each pixel. In this case, the processing of S302 and S303 is not necessary.

  As described above, according to the present embodiment, it is possible to improve the reliability evaluation accuracy of the flicker component by generating the flicker model using the flicker characteristic parameter extracted from the detected flicker component. As a result, a flicker component with higher accuracy can be detected.

(Fourth embodiment)
Next, a fourth embodiment of the present invention will be described.
FIG. 13 is a block diagram showing a configuration example of a flicker correction apparatus according to the fourth embodiment of the present invention.

In FIG. 13, the same components as those in the second embodiment are denoted by the same reference numerals, and redundant description is omitted. The difference from the second embodiment is that the flicker correction apparatus 10 ″ ′ includes a classification unit 8.
For example, the classification unit 8 classifies the flicker detection frame according to the reliability evaluation result from the flicker component evaluation unit 4, and outputs the classification result to the flicker correction value generation unit 2.

  FIG. 14 shows that the image is divided into a total of 121 flicker detection frames of M = 11 and N = 11, and the flicker component obtained for each flicker detection frame is based on the difference from the flicker model acquired from the system control unit 5. It is a figure which shows the reliability evaluated in this way visually. In FIG. 14, the flicker detection frame determined to have high reliability of the detected flicker component is shown in white, and the flicker detection frame determined to have low reliability is shown in black.

  The classification unit 8 generates information focusing on the continuity of the position of the flicker detection frame with low reliability and outputs the information to the flicker correction value generation unit 2. In the example of FIG. 14, three flicker detection frames with low reliability continue in the horizontal direction from the second row and the fifth column, and the flicker detection frames with low reliability start from the first column in the sixth row in the horizontal direction. There are three consecutive 10th lines, and there is only one flicker detection frame with high reliability.

The flicker correction value generation unit 2 performs interpolation by selecting an interpolation method based on the classification information.
In the example of FIG. 14, three flicker detection frames having low reliability are consecutive in the horizontal direction from the second row and the fifth column, but since there are flicker values with high reliability on the left and right, for example, [number 7] is weighted and interpolated.

Here, x and y represent the coordinate position of each flicker detection frame (1 <x <11, y ≦ 11 in the example of FIG. 14). K represents the coordinate position of the flicker detection frame that is in the same row as the target flicker detection frame, is closest to the left side (1 ≦ k <x), and is determined to have high reliability. Further, h represents the coordinate position of the flicker detection frame that is in the same row as the target flicker detection frame, is closest to the right side (x <h ≦ 11), and is determined to have high reliability. Further, a _xy represents the value of the flicker component of the flicker detection frame to be interpolated, and a _ky and a _hy represent the values of the flicker component at the respective positions.

  In the sixth row, since three flicker detection frames having low reliability from the first column are consecutive in the horizontal direction, interpolation is performed with a straight line, for example, [Equation 8].

Here, x and y represent the coordinate position of each flicker detection frame (1 ≦ x ≦ 11, y ≦ 11 in the example of FIG. 14). Further, s and t are in the same row as the target flicker detection frame, and represent the coordinate positions of the flicker detection frame determined to have high reliability. Further, a _xy represents the value of the flicker component of the flicker detection frame to be interpolated, and a _sy and a _ty represent the value of the flicker component at each position.
Further, since there is only one flicker detection frame with high reliability in the 10th line, no interpolation is performed.

  Next, the operation of the flicker correction apparatus in the present embodiment will be described with reference to the flowchart of FIG. 15, steps that perform the same operations as in FIG. 6 are given the same reference numerals, and descriptions thereof are omitted.

In the present embodiment, the process from the input of the image signal to the reliability determination process can be performed in the same manner as in the second embodiment.
In S1512, the classification unit 8 stores the reliability determination result for each flicker detection frame. When the reliability determination result for one screen is stored, classification information is generated and notified to the flicker correction value generation unit 2 (S1513).

Thereafter, processing is performed for each flicker detection frame.
That is, for the flicker detection frame corresponding to the flicker detection value determined to have high reliability, the flicker correction value generation unit 2 generates a correction value using the flicker value detected by the flicker detection unit 1 in S612. .

  On the other hand, for the flicker detection frame corresponding to the flicker detection value determined to be low in reliability, the flicker correction value generation unit 2 performs flicker component interpolation in S613.

At this time, as described above, an interpolation formula based on the classification information notified from the classification unit 8 is used. Then, the flicker correction value generation unit 2 generates a flicker correction value based on the interpolated flicker component (S614).
After that, in S307 and S308, as in the first embodiment, flicker correction is performed by reading the image signal stored in the storage unit 9 and multiplying by the flicker correction value.

Here, an example is shown in which classification is performed in the horizontal direction and interpolation is performed in the horizontal direction. However, the classification unit 8 generates classification information focusing on the continuity in the vertical direction, and the flicker correction value generation unit 2 performs the vertical direction. Weighting may be performed for interpolation. In addition, when the number of flicker detection frames evaluated as having high reliability in one screen is larger than the threshold value Th _true , the correction value is used by using the flicker component of the flicker detection frame evaluated as having low reliability as it is. May be generated.

  As described above, in the present embodiment, when the flicker component of the flicker detection frame evaluated to have low reliability is interpolated and generated, by selecting an interpolation method suitable for each, it is possible to obtain more accuracy. High interpolation flicker components can be generated. Thereby, a more reliable flicker detection result can be obtained. Therefore, flicker correction with high accuracy can be performed.

(Other embodiments)
In the above-described embodiment, when the correction value given to the multiplier 3 is 1.0 or a value that is sufficiently close to 1.0, it can be regarded as synonymous with no correction. Therefore, when the correction value has a value in a range that can be regarded as 1.0, the pixel value read from the past section 9 is output as it is as a corrected pixel value without reading and multiplying the correction value. It may be configured. Specifically, a switch for switching between a path through which the output from the storage unit 9 is given to the multiplier 3 and a path through which the output is directly output is provided. When the correction value is a value that can be regarded as 1.0, a path for directly outputting the pixel value read from the storage unit is selected, and in other cases, a path for supplying the pixel value to the multiplier 3 The control unit 11 may control the switch so as to select.

  The flicker detection apparatus described in the above embodiment can be suitably used for an imaging apparatus using an XY address type imaging element such as a CMOS image sensor. In other words, by applying the flicker detection device according to the embodiment of the present invention to the image signal obtained from the image sensor, it is possible to detect flicker components with high accuracy, and as a result, it is possible to correct favorable flicker components. Become. Then, the image signal after flicker correction may be used to perform known image processing or encoding processing and record it on a recording medium or output it to an external device.

The above-described embodiment can also be realized in software by a computer of a system or apparatus (or CPU, MPU, etc.).
Therefore, the computer program itself supplied to the computer in order to implement the above-described embodiment by the computer also realizes the present invention. That is, the computer program itself for realizing the functions of the above-described embodiments is also one aspect of the present invention.

  The computer program for realizing the above-described embodiment may be in any form as long as it can be read by a computer. For example, it can be composed of object code, a program executed by an interpreter, script data supplied to the OS, but is not limited thereto.

  A computer program for realizing the above-described embodiment is supplied to a computer via a storage medium or wired / wireless communication. Examples of the storage medium for supplying the program include a magnetic storage medium such as a flexible disk, a hard disk, and a magnetic tape, an optical / magneto-optical storage medium such as an MO, CD, and DVD, and a nonvolatile semiconductor memory.

  As a computer program supply method using wired / wireless communication, there is a method of using a server on a computer network. In this case, a data file (program file) that can be a computer program forming the present invention is stored in the server. The program file may be an executable format or a source code.

Then, the program file is supplied by downloading to a client computer that has accessed the server. In this case, the program file can be divided into a plurality of segment files, and the segment files can be distributed and arranged on different servers.
That is, a server apparatus that provides a client computer with a program file for realizing the above-described embodiment is also one aspect of the present invention.

  In addition, a storage medium in which the computer program for realizing the above-described embodiment is encrypted and distributed is distributed, and key information for decrypting is supplied to a user who satisfies a predetermined condition, and the user's computer Installation may be allowed. The key information can be supplied by being downloaded from a homepage via the Internet, for example.

Further, the computer program for realizing the above-described embodiment may use an OS function already running on the computer.
Further, a part of the computer program for realizing the above-described embodiment may be configured by firmware such as an expansion board attached to the computer, or may be executed by a CPU provided in the expansion board. Good.

It is a block diagram which shows the structural example of the flicker correction apparatus in the 1st Embodiment of this invention. It is a block diagram which shows the structural example of the flicker detection part in the 1st Embodiment of this invention. It is a flowchart which shows an example of operation | movement of the flicker correction apparatus in the 1st Embodiment of this invention. It is a block diagram which shows the structural example of the flicker correction apparatus in the 2nd Embodiment of this invention. It is a block diagram which shows the structural example of the flicker component evaluation part in the 2nd Embodiment of this invention. It is a flowchart which shows an example of operation | movement of the flicker correction apparatus in the 2nd Embodiment of this invention. It is a block diagram which shows the structural example of the flicker correction apparatus in the 3rd Embodiment of this invention. It is a block diagram which shows the structural example of the flicker characteristic parameter extraction part in the 3rd Embodiment of this invention. It is a flowchart which shows an example of operation | movement of the amplitude and frequency calculation part in the 3rd Embodiment of this invention. It is a flowchart which shows an example of operation | movement of the phase calculation part in the 3rd Embodiment of this invention. It is a figure which shows an example of the flicker component detected by the flicker detection part. It is a flowchart which shows an example of operation | movement of the flicker correction apparatus in the 3rd Embodiment of this invention. It is a block diagram which shows the structural example of the flicker correction apparatus in the 4th Embodiment of this invention. It is a figure which shows an example of the flicker component evaluated by the flicker component evaluation part. It is a flowchart which shows an example of operation | movement of the flicker correction apparatus in the 4th Embodiment of this invention. It is a figure showing the flicker phenomenon which generate | occur | produces when using a solid-state image sensor of an XY address system. It is a figure showing the flicker phenomenon which occurs when a subject is illuminated with a fluorescent light from the left side and a subject illuminated by sunlight from the right side using an XY address type solid-state imaging device. The image signal for the last three frames obtained from the subject, the vertical intensity distribution obtained from the past two frames of the image signal, the vertical intensity distribution of the current frame, and these two intensity distributions are divided. It is a figure which shows the calculated | required flicker component.

Claims (8)

A flicker detection device for detecting a flicker component from an image signal obtained by an image sensor,
Setting means for setting a plurality of flicker detection frames each including an image signal of at least one pixel in at least the horizontal direction of the image signal for one screen;
A flicker detection device comprising flicker detection means for detecting a flicker component generated as a fluctuation of a luminance level in a vertical direction in the image signal for each of the plurality of flicker detection frames.   2. The flicker detection apparatus according to claim 1, wherein the setting unit sets the plurality of flicker detection frames by dividing the image signal for one screen in a horizontal direction and a vertical direction.   3. The flicker detection apparatus according to claim 1, further comprising an evaluation unit that determines the reliability of the flicker component detected by the flicker detection unit based on a comparison with a predetermined flicker model. Characteristic parameter extraction means for extracting characteristic parameters representing characteristics of a specific flicker component for each flicker detection frame;
Model generation means for generating the flicker model based on the characteristic parameter;
The flicker detection apparatus according to claim 3, wherein the evaluation unit performs the evaluation using the flicker model generated by the model generation unit.   The flicker detection apparatus according to claim 4, wherein the characteristic parameter is a parameter related to a frequency, an amplitude, and a phase of a flicker component.   Interpolation means for interpolating and generating other flicker components determined to be highly reliable by the evaluation means as a flicker component replacing the flicker components determined to be low reliability by the evaluation means is further provided. The flicker detection apparatus according to any one of claims 3 to 5. Further comprising classification means for generating classification information obtained by classifying the determination result by the evaluation means based on a predetermined condition;
The flicker detection apparatus according to claim 6, wherein the interpolation unit performs interpolation according to the classification information. A flicker detection method for detecting a flicker component from an image signal obtained by an image sensor,
A setting step of setting a plurality of flicker detection frames each including an image signal of at least one pixel in at least the horizontal direction of the image signal for one screen;
And a flicker detection step of detecting, in each of the plurality of flicker detection frames, a flicker component generated as a luminance level fluctuation in a vertical direction in the image signal.