JP4724767B2 - Flicker detection method, flicker detection apparatus, and flicker detection program for imaging apparatus - Google Patents

Description

  The present invention, for example, is a method of imaging image data in an imaging device, such as an imaging device configured by a CMOS (Complementary Metal Oxide Semiconductor) for a subject illuminated with a light source whose brightness changes due to a power supply frequency, such as a fluorescent lamp. The present invention relates to a flicker detection method, a flicker detection apparatus, and a flicker detection program for an image pickup apparatus that can accurately detect flicker generated in an image when the image is picked up by an image pickup apparatus that performs scanning in units of pixel rows.

  2. Description of the Related Art Conventionally, in an imaging apparatus such as a digital camera, a subject image is formed on a solid-state imaging device (for example, a CMOS: Complementary Metal-Oxide Semiconductor or a CCD: Charged Coupled Device, hereinafter referred to as an imaging device) via a lens. An image processing apparatus and an image processing method are known in which a subject image is photoelectrically converted by this image sensor to generate an image signal.

  In addition, an image pickup device using a CMOS can use a manufacturing process similar to that of an LSI as compared with an image pickup device using a CCD, so that mass productivity is excellent and various functions such as an amplifier can be provided for each pixel. It is also used mainly because it is excellent in miniaturization and weight reduction.

  However, since the CMOS sensor performs an exposure operation and a signal readout operation by sequentially scanning the imaging pixel portion of the two-dimensional pixel array in units of each pixel row, basically, compared with a CCD sensor that performs simultaneous exposure of all pixels, There is a problem that noise is likely to occur on the output screen due to the influence of the periodic fluctuation of the illumination light.

  For example, in a digital camera with a vertical synchronization frequency of 60 Hz, when shooting is performed under an illumination light source that fluctuates at a frequency of 50 Hz, the level of illumination light varies during exposure scanning for one screen, and the level of each pixel row varies. There is a case where horizontal stripes of light and darkness called flicker occur.

  That is, as shown in FIG. 6, a general fluorescent lamp blinking with a commercial AC power supply repeats blinking at a cycle of 1/100 seconds when the frequency of the power supply is 50 Hz and 1/120 seconds when the frequency of 60 Hz. When such incident light is converted into an electrical signal by an image sensor and read out, as shown in FIG. 7, when a CMOS sensor is used, the exposure timing differs for each line (pixel row) to be read. If the time is not an integral multiple of the light source blinking cycle, the total amount of light incident within the exposure time of each pixel row will be different even within the same frame. For this reason, even in the same frame, a bright portion and a dark portion are generated with a period of 1/100 seconds or 1/120 seconds. Such a phenomenon is flicker.

  In addition, since fluorescent lamps emit white light, a plurality of phosphors, for example, phosphors such as R (red), G (green), and B (blue) are usually used, each having a unique afterglow characteristic. Therefore, when the exposure timing is shifted as described above, not only the brightness change but also the hue change occurs. The light and dark stripes will be colored. In addition, when there is a difference between the illumination light level at the first exposure and the illumination light level at the second exposure, the signal level may change in the time axis direction, resulting in fine noise in pixel units. is there.

  Next, an example of the principle for preventing flicker will be described using FIG. 8 as an example when the light source frequency is 100 Hz.

  When the light source frequency is 100 Hz, the brightness of the illumination fluctuates at a period of (1/100) seconds as shown in (a). At this time, the shutter speed (exposure time of the image sensor) is set to an integer multiple of (1/100) seconds (the figure is an example of 1 time, but may be an integer multiple such as 2 or 3). . With this setting, as shown in FIGS. 8B to 8E, the exposure amount of each line becomes equal regardless of the exposure timing, and flicker can be prevented (for example, Patent Documents 1 and 2). reference).

  However, in order to prevent the occurrence of flicker as described above, it is necessary to accurately detect the presence / absence of flicker and the flicker light source blinking cycle in advance.

  Therefore, integrating means for integrating the pixel levels for each predetermined area in the frame or field, averaging means for averaging the integration results for each area of the same image position in a plurality of frames or fields, and for each area of the integrating means Disclosed is a configuration of a flicker detection device including a division unit that divides an integration result by an averaged result for each area of the averaging unit, and a flicker determination unit that performs frequency analysis on the division result of the division unit and determines presence / absence of flicker. (For example, refer to Patent Document 3).

  In addition, the image is divided in the vertical scanning direction to set a plurality of detection frames, luminance data is detected for each detection frame, and the difference between the luminance data of the two frames before and after each detection frame is taken to flicker component A method of calculating the flicker frequency by detecting the above is disclosed (for example, see Patent Document 4).

  Also, the pixel values are integrated in the horizontal direction of the screen to generate flicker component data that weakens the influence of the background (pattern), and the integrated data is averaged over a plurality of screens to obtain a value corresponding to the background component. And then normalizing the integrated data with this average value to calculate flicker component data from which the influence of the background has been removed, and then performing a Fourier transform on the flicker component data in the vertical direction of the screen to obtain a flicker frequency component Only the spectrum is extracted and the level of the spectrum is compared with a threshold value to determine whether flicker is occurring, that is, whether it is under fluorescent lamp illumination, and whether the fluorescent lamp drive power supply frequency is 50 Hz or 60 Hz. A flicker detection method for determining whether or not is satisfied is disclosed (for example, see Patent Document 5).

JP-A-2005-033616 JP 2000-175105 A JP 2001-119708 A JP 2003-189129 A JP 2001-1111887 A

  However, according to the flicker detection methods described in Patent Documents 3 to 5, there is still room for improvement in terms of simplification of operations and constant accuracy.

  Accordingly, an object of the present invention is to provide a flicker detection method, a flicker detection apparatus, and a flicker detection program for an imaging apparatus capable of accurately detecting flicker generated in a frame or a field with simple calculation and constant accuracy.

The invention according to claim 1, which has been made to achieve the above object, is an image output by scanning a pixel row by scanning a signal to each pixel arranged in a matrix via an image sensor. A flicker detection method for an image pickup apparatus that detects a flicker component generated under a blinking illumination light source based on a signal and obtains a difference in luminance signal between a plurality of temporally continuous frames. A luminance generation step for generating a luminance signal in association with each pixel for each of the frames, and for each of the plurality of frames, one frame is divided into a plurality of regions in a pixel column direction orthogonal to the pixel row and a frame region dividing step of generating a plurality of pixel regions as a unit of predetermined number of pixel rows, each the same region in the plurality of frames, of identity one over the low-pass filter And time direction smoothing step of the luminance signal of the frequency band to generate a smoothed to smooth frames, each respectively to the plurality of regions, obtains the difference of the luminance signal of the latest frame adjacent to the plurality of frames and said smooth frame, A flicker evaluation value generating step for generating a difference pixel area representing a difference value of the luminance signal instead of the luminance signal, and detecting a phase change in the pixel column direction of the difference value at each frame generation cycle. A vector derivation step for deriving a vector of flicker frequency components, an inner product value calculation step for sequentially calculating inner product values between a pair of vectors derived adjacent in time for each period of the frame generation, An inner product value smoothing step for calculating a plurality of inner product values generated in the inner product value calculating step by applying a low pass filter to smooth the inner product value; A difference between the smoothed inner product value and the inner product value calculated in the inner product value calculating step is obtained, and a flicker phase acceleration generating step for generating the difference as a flicker phase acceleration is compared with a predetermined threshold value. And a flicker determination step for determining the presence / absence of flicker.

According to the flicker detection method of the imaging device according to claim 1, a luminance generation step of generating a luminance signal in association with each pixel for each of a plurality of temporally continuous frames, and each of the plurality of frames. A frame region dividing step for dividing each frame into a plurality of regions in a pixel column direction orthogonal to the pixel rows and generating a plurality of regions each having a predetermined number of pixel rows as a unit; and a plurality of frames A smoothing frame and a latest frame adjacent to the plurality of frames for each of the plurality of regions, and a time direction smoothing step for smoothing the luminance signal of the same region by applying a low-pass filter for each same region in And a flicker evaluation value generation step for generating a difference pixel area representing a difference value of the luminance signal instead of the luminance signal, and for each frame generation cycle A vector deriving step for deriving a vector of flicker frequency components by detecting a phase change in the pixel column direction of the difference value, and an inner product value between a pair of vectors derived adjacent in time for each frame generation period An inner product value calculating step for sequentially calculating, an inner product value smoothing step for calculating an inner product value smoothed by applying a low pass filter to a plurality of inner product values generated in the inner product value calculating step, and a smoothed inner product value; The difference between the inner product value calculated in the inner product value calculating step and the flicker phase acceleration generation step for generating the difference as a flicker phase acceleration, and the presence or absence of flicker by comparing the flicker phase acceleration with a predetermined threshold value. By providing the flicker determination step for determining the flicker, it is possible to detect flicker with high accuracy with simple calculation and constant accuracy.

  According to a first aspect of the present invention, there is provided a flicker detection method for an image pickup apparatus, wherein, as in the second aspect of the invention, a summation value of a luminance signal is calculated for each of a plurality of regions divided in the frame region dividing step. It is preferable that a region luminance signal totaling step to be generated is provided, and the total value of the luminance signal is used instead of the luminance signal in the time direction smoothing step and the flicker evaluation value generating step. Thereby, the luminance signal for each region is represented by the total value, and the calculation in the time direction smoothing step and the flicker evaluation value generating step can be simplified.

  Further, according to the flicker detection method of the imaging apparatus according to claim 1 or 2, as in the invention according to claim 3, in the flicker determination step, with respect to the absolute values of the plurality of flicker phase accelerations, By generating a smoothed flicker phase acceleration by applying a low-pass filter in the time direction and comparing the smoothed flicker phase acceleration with the predetermined threshold value to determine the presence or absence of flicker, the presence or absence of flicker is more accurately detected. Can be determined.

Next, according to the fourth aspect of the present invention, based on a video signal output by scanning a pixel row by scanning a signal to each pixel arranged in a matrix through the image sensor, the time is determined. A flicker detection device for an imaging apparatus that takes a difference in luminance signal between a plurality of consecutive frames and detects a flicker component generated under a blinking illumination light source, for each of a plurality of temporally consecutive frames A luminance generation means for generating a luminance signal in association with each pixel, and for each of the plurality of frames, one frame is divided into a plurality of regions in a pixel column direction orthogonal to a pixel row, and a predetermined pixel and a frame region dividing means for generating a plurality of regions of the number of rows as a unit, each the same region in the plurality of frames, raw smooth frame luminance signals of identity one region by applying a low-pass filter smoothes And time direction smoothing unit that, for each respective on of the plurality of regions, obtains the difference of the luminance signal of the latest frame adjacent to the plurality of frames and said smooth frame difference value of the luminance signal in place of said luminance signal A flicker evaluation value generating means for generating a difference pixel region representing the difference, and a vector derivation means for detecting a phase change of the difference value in the pixel column direction for each frame generation period to derive a vector of flicker frequency components An inner product value calculating means for sequentially calculating inner product values between a pair of vectors derived adjacent in time for each frame generation period, and a plurality of inner product values generated by the inner product value calculating means The inner product value smoothing means for calculating the inner product value smoothed by applying a low-pass filter, and the difference between the smoothed inner product value and the inner product value calculated by the inner product value calculating means The flicker phase acceleration generating means for generating the difference as flicker phase acceleration and the flicker determination means for determining the presence or absence of flicker by comparing the flicker phase acceleration with a predetermined threshold value. To do.

According to the flicker detection device of the imaging device according to claim 4, a luminance generation unit that generates a luminance signal in association with each pixel for each of a plurality of temporally continuous frames, and each of the plurality of frames. A frame area dividing unit that divides one frame into a plurality of areas in a pixel column direction orthogonal to a pixel row for each, and generates a plurality of areas each having a predetermined number of pixel rows as a unit; and a plurality of frames A time direction smoothing unit that generates a smooth frame by applying a low pass filter to smooth the luminance signal in the same region for each same region in the region, and the latest frame adjacent to the smooth frame and the plurality of frames for each of the plurality of regions. obtains the difference luminance signal with a flicker evaluation value generation means for generating a differential pixel region showing the difference value of the luminance signal instead of the luminance signal for each period of the frame generation, the pixel row of the difference value A vector deriving means for deriving a vector of flicker frequency components by detecting the phase change of direction, the inner product value for each period of the frame generation, sequentially calculates an inner product value between the pair of vectors derived by temporally adjacent Calculating means; inner product value smoothing means for calculating a smoothed inner product value obtained by applying a low pass filter to a plurality of inner product values generated by the inner product value calculating means; and calculating the smoothed inner product value and the inner product value. Flicker phase acceleration generating means for obtaining a difference from the inner product value calculated by the means and generating the difference as flicker phase acceleration, and flicker for determining the presence or absence of flicker by comparing the flicker phase acceleration with a predetermined threshold. Since the determination means is provided, flicker can be detected with high accuracy with a simple calculation and constant accuracy, as in the first aspect of the invention.

  According to a fourth aspect of the present invention, there is provided a flicker detection apparatus for an image pickup apparatus, wherein, as in the fifth aspect of the present invention, the summation value of the luminance signal is calculated for each of the plurality of areas divided by the frame area dividing means. And a time direction smoothing unit and a flicker evaluation value generating unit configured to use a total value of the luminance signal instead of the luminance signal. Similar to the described invention, the luminance signal for each region is represented by the total value, and the calculation in the time direction smoothing means and the flicker evaluation value generating means can be simplified.

  According to a fourth aspect of the present invention, there is provided a flicker detection device for an image pickup apparatus, as in the sixth aspect of the invention, wherein the flicker determination means is configured to detect the absolute value of the plurality of flicker phase accelerations. By generating a smoothed flicker phase acceleration by applying a low-pass filter in the time direction, and comparing the smoothed flicker phase acceleration with the predetermined threshold value to determine the presence or absence of flicker. Similar to the invention of Item 3, the presence or absence of flicker can be determined with higher accuracy.

Next, according to the seventh aspect of the present invention, based on a video signal output by scanning a pixel row by scanning a signal to each pixel arranged in a matrix through an image sensor, a time is determined. A flicker detection program for an imaging apparatus that takes a difference in luminance signal between a plurality of consecutive frames and detects a flicker component generated under a blinking illumination light source, for each of a plurality of temporally consecutive frames A luminance generation step for generating a luminance signal in association with each pixel; and for each of the plurality of frames, one frame is divided into a plurality of regions in a pixel column direction orthogonal to the pixel row, and a predetermined pixel a frame region dividing step of generating a plurality of regions of the number of rows as a unit, each the same region in the plurality of frames, the luminance signal of identity one region by applying a low-pass filter smoothes Rights And time direction smoothing step of generating the frame, each respectively to the plurality of regions, obtains the difference of the luminance signal of the latest frame adjacent to the plurality of frames and the smooth frame, the luminance signal in place of said luminance signal A flicker evaluation value generating step for generating a difference pixel region representing the difference value of the first and a phase change in the pixel column direction of the difference value for each frame generation cycle to derive a vector of flicker frequency components A vector deriving step, an inner product value calculating step for sequentially calculating inner product values (phase differences) between the pair of vectors derived adjacent in time for each frame generation period, and the inner product value calculating step. An inner product value smoothing step for calculating a smoothed inner product value obtained by applying a low pass filter to the plurality of generated inner product values, and the smoothed inner product value A difference between the inner product value calculated in the inner product value calculating step and a flicker phase acceleration generating step for generating the difference as a flicker phase acceleration is compared with a predetermined threshold value. A flicker determination step for determining presence / absence is executed by a computer.

According to the flicker detection program of claim 7, a luminance generation step of generating a luminance signal in association with each pixel for each of a plurality of temporally continuous frames, and for each of the plurality of frames, A frame region dividing step of dividing one frame into a plurality of regions in a pixel column direction orthogonal to the pixel rows and generating a plurality of regions each having a predetermined number of pixel rows as a unit, and the same region in the plurality of frames A time direction smoothing step for generating a smooth frame by smoothing the luminance signal of the same region by applying a low-pass filter every time, and the luminance of the smooth frame and the latest frame adjacent to the plurality of frames for each of the plurality of regions obtains the difference signal, the flicker evaluation value generation step of generating a difference pixel area representing the difference value of the luminance signal instead of the luminance signal for each period of the frame generation, the difference A vector derivation step of deriving a vector of flicker frequency components by detecting the phase change of the pixel row direction value for each period of the frame generation, the inner product value between the pair of vectors derived by temporally adjacent sequentially Inner product value calculating step for calculating, inner product value smoothing step for calculating a plurality of inner product values generated in the inner product value calculating step by applying a low pass filter to smooth the inner product value, smoothed inner product value and inner product The difference between the inner product value calculated in the value calculation step and the flicker phase acceleration generation step for generating the difference as a flicker phase acceleration is compared with a predetermined threshold value to determine the presence or absence of flicker. By causing the computer to execute the flicker determination step for determination, similar to the invention according to claim 1, simple calculation and constant Can be detected accurately flicker accuracy.

  According to a seventh aspect of the present invention, there is provided a program for detecting flicker in an imaging apparatus, wherein, as in the eighth aspect of the invention, the summation value of the luminance signal is calculated for each of the plurality of regions divided in the frame region dividing step. A region luminance signal totaling step to be generated is provided, and in the time direction smoothing step and the flicker evaluation value generating step, the total value of the luminance signal is used instead of the luminance signal, as in the invention according to claim 2. The luminance signal for each region is represented by the total value, and the calculation in the time direction smoothing step and the flicker evaluation value generating step can be simplified.

  Further, the flicker detection program of the imaging device according to claim 7 or claim 8 is the flicker detection step for the absolute value of the plurality of flicker phase accelerations in the flicker determination step as in the invention according to claim 9. The invention according to claim 3, wherein a flicker phase acceleration smoothed by applying a low-pass filter in a time direction is generated, and the smoothed flicker phase acceleration is compared with the predetermined threshold value to determine the presence or absence of flicker. Similarly to the above, the presence / absence of flicker can be determined with higher accuracy.

The flicker detection method, the flicker detection method, and the flicker detection program of the imaging apparatus according to the present invention generate a luminance signal in association with each pixel for each of a plurality of temporally continuous frames, and then For each of them, one frame is divided into a plurality of regions in the pixel column direction orthogonal to the pixel rows to generate a plurality of pixel regions having a predetermined number of pixel rows as one unit, and then the same in the plurality of frames For each region, a low-pass filter is applied to smooth the luminance signal in the same region to generate a smooth frame, and then the luminance between the smooth frame and the latest frame adjacent to the plurality of frames for each of the plurality of pixel regions. obtains the difference signal to generate a difference pixel area representing the difference value of the luminance signal instead of the luminance signal, and then, for each cycle of frame generation, the pixel column direction of the phase change in the differential value A flicker frequency component vector is detected and the inner product value between a pair of vectors derived adjacent in time is sequentially calculated for each frame generation period, and then a plurality of calculated inner product values are calculated. The inner product value smoothed by applying a low-pass filter is calculated, the difference between the smoothed inner product value and the calculated inner product value is obtained, and this difference is generated as the flicker phase acceleration. By comparing the acceleration with a predetermined threshold value to determine the presence or absence of flicker, flicker can be detected with high accuracy with a simple calculation and constant accuracy.

1 is a block diagram illustrating a configuration of a flicker detection device of an imaging apparatus according to an embodiment of the present invention. It is explanatory drawing of a frame area | region division | segmentation in the flicker detection apparatus of the imaging device of the Example. It is explanatory drawing of vector derivation | leading-out of the flicker frequency component in the flicker detection apparatus of the imaging device of the Example. It is a figure showing the experimental result at the time of calculating | requiring the flicker phase acceleration in the flicker detection apparatus of the imaging device of the Example. 6 is a flowchart illustrating a flicker detection procedure in the imaging apparatus flicker detection apparatus according to the embodiment. It is a figure showing the example of the light quantity change with respect to the period of a light source power supply. It is a figure showing the example of flicker expression at the time of using a CMOS sensor. It is a figure showing the example of the flicker reduction at the time of using a CMOS sensor.

  Next, an embodiment of a flicker detection apparatus, flicker detection method, and flicker detection program for an image pickup apparatus according to the present invention will be described with reference to the drawings.

  As illustrated in FIG. 1, the imaging apparatus 300 detects an flicker based on the imaging unit 100 that outputs an optical image to the CMOS sensor 3 and outputs a digital image signal, and the digital image signal output from the imaging unit 100. The flicker detection device 200 is provided.

  The imaging unit 100 includes an imaging optical system 1 that guides a captured optical image to the CMOS sensor 3, and a mechanical shutter 2 that is provided on the incident optical path and that mechanically opens and closes to switch between passing and blocking of incident light. And a CMOS sensor 3 that photoelectrically converts and amplifies an optical image formed through the imaging optical system 1 and converts it into a digital signal and outputs it.

  The imaging optical system 1 includes an imaging lens 1a that collects subject light and guides it to a CMOS sensor 3, and an iris (aperture) 1b that has a variable aperture diameter and adjusts the amount of incident light. .

  The CMOS sensor 3 has an imaging element 3a that converts an optical image formed through the imaging optical system 1 into an analog electric signal in association with the amount of received light, and a variable gain that amplifies the analog electric signal output from the imaging element 3a. An amplifier (AGC: Automatic Gain Control) 3b, an analog-to-digital converter (ADC) 3c that converts an analog electric signal output from the variable gain amplifier 3b into a digital signal, and the like are included. In addition, the imaging device 3a is configured such that a plurality of photoelectric conversion elements (pixels) are arranged in a matrix, and charge accumulation to each photoelectric conversion element and frame reading are performed in units of pixel rows. Thus, the function of the electronic shutter is controlled.

  Next, the flicker detection device 200 generates a luminance signal in association with each pixel for each temporally continuous frame based on the digital image signal input via the imaging unit 100. Each of the plurality of frames includes a frame region dividing unit 10 that generates a plurality of regions by dividing the plurality of regions into a plurality of regions in a pixel column direction orthogonal to the pixel row.

  Further, the flicker detection apparatus 200 uses the luminance signal for each divided area to generate a flicker evaluation value generation unit 11 that generates an evaluation value of flicker presence / absence and flicker frequency, and a flicker evaluation value generated by the flicker evaluation value generation unit 11. The vector deriving means 12 for deriving the flicker frequency component vector using, the previous frame vector holding means 13 for storing the vector derived by the vector deriving means 12, and generated adjacent in time for each frame generation period Inner product value calculating means 14 for sequentially calculating inner product values between a pair of vectors (vectors stored in the previous frame vector holding means and current frame vectors), and a plurality of inner products calculated by the inner product value calculating means 14 Flicker phase acceleration generating means 15 for generating flicker phase acceleration based on the value, flicker phase acceleration A flicker determination unit 16 that determines the presence or absence of flicker by comparing with a predetermined threshold and detects a light source frequency, a CPU 17, a ROM 18, and the like, and the CPU 17 performs the flicker detection device based on a control program stored in the ROM 18. It is configured to control each of the 200 functions.

  Specifically, as shown in FIG. 2, the frame region dividing unit 10 divides the detection target height H orthogonal to the detection target width in the pixel row direction into N, and the size in the height direction becomes H / N. Generate regions. Further, the frame area dividing unit 10 includes a luminance totaling unit 10a (S300 in FIG. 3) that totals the luminance value for each region, and the evaluation value generating unit 11 calculates the total value for each region totaled by the luminance totaling unit 10a. It inputs to the time direction smoothing means 11a and the difference pixel area generation means 11b.

The evaluation value generating means 11 is a time direction smoothing means 11a (S400 in FIG. 3) that generates a smooth frame by smoothing the luminance signal of the same area by applying a low-pass filter for each same area of the plurality of frames, A difference pixel area generation unit 11b that calculates a difference between the luminance signal between the smooth frame and the latest frame adjacent to the plurality of frames and generates a difference pixel area that represents the difference value of the luminance signal instead of the luminance signal. (S500 in FIG. 3).

  The time direction smoothing means 11a is preferably constituted by an IIR filter (Infinite impulse response filter). Since the IIR filter has a feedback function, the sampling order can be reduced to shorten the calculation processing time, and the quantization error included as noise in the total value calculated by the luminance totaling means 10a can be effectively removed. .

・・・式１
（式１）において、Χ_ｋが周波数成分、χ _ｎ がフーリエ変換されるサンプリングデータである。また、Ｎはサンプリング次数であって、領域分割手段１０で分割された領域の数Ｎに相当する。また、ｋは光源周波数の折り返し周波数である。

The vector deriving means 12 (S600 in FIG. 3) performs an FFT (Fast Fourier Transform) operation using the equation 1 on the flicker evaluation value for each frame, and the flicker frequency of the index i associated with the light source frequency (100 Hz or 120 Hz). The component X _i = a _i + jb _i (so-called vector (a _i , b _i )) is derived.

... Formula 1
In (Expression 1), Χ _k is a frequency component, and χ _n is sampling data subjected to Fourier transform. N is the sampling order and corresponds to the number N of regions divided by the region dividing means 10. Further, k is a folding frequency of the light source frequency.

  Next, the inner product value calculation means 14 calculates the inner product value AB as AB = | A || B | cos θ when vectors derived in two temporally adjacent frames are represented as A and B, respectively. Calculate using.

  Next, the flicker phase acceleration generation unit 15 calculates an inner product value obtained by smoothing a plurality of inner product values generated by the inner product value calculation unit 14 by applying a low pass filter (hereinafter referred to as inner product smoothing value). The smoothing means 15a is constituted by an inner product value difference calculating means 15b for calculating a difference between the inner product smooth value and the inner product value calculated by the inner product value calculating means 14, and the difference obtained by the inner product value difference calculating means 15b is expressed by the flicker phase. Generate as acceleration.

  Next, the flicker determination unit 16 outputs the flicker phase acceleration absolute value acquisition unit 16a that acquires the absolute value of the flicker phase acceleration generated by the flicker phase acceleration generation unit 15 and the flicker phase acceleration absolute value acquisition unit 16a. A time direction smoothing means 16b for generating a flicker phase acceleration smoothed by applying a low-pass filter to the absolute values of the plurality of flicker phase accelerations in the time direction, and comparing the smoothed flicker phase acceleration with a predetermined threshold value. The threshold determination means 16c for determining the presence / absence of flicker and the light source frequency detection means 16d for detecting the light source frequency of flicker when the threshold determination means 16c determines that there is flicker.

  The light source frequency detection unit 16d compares the smoothed flicker phase acceleration absolute values generated by the time direction smoothing unit 16b and associated with the light source frequencies of 100 Hz and 120 Hz, respectively, and the smaller value is obtained. Let the obtained light source frequency be a flicker light source frequency.

Next, the flicker phase acceleration obtained by the inventor through experiments will be described with reference to FIG. In FIG. 4, the horizontal axis is a flicker frequency index (light source frequency index), and the vertical axis is a flicker phase acceleration index. In this experiment, for the flicker evaluation value obtained via the evaluation value generating means 11, the above-described (Equation 1) is used, and the flicker frequency component vector (a _i , b _i ) is associated with the flicker frequency index. The flicker phase acceleration generation means 15 calculates the flicker phase acceleration corresponding to each index. The frequency of the light source actually used at the time of shooting corresponds to index 4.

  As shown in FIG. 4, the flicker phase acceleration of the light source frequency (index 4) actually used appears approximately to zero. The closer the flicker phase acceleration is to 0, the higher the flicker frequency detection accuracy.

  Next, the CPU 17 controls the speed of the mechanical shutter 2 and the charge accumulation and readout time of the image sensor 3a so as to reduce flicker in association with the light source frequency detected via the light source frequency detection means 16d. . That is, when the light source frequency is 100 Hz, the exposure time is an integral multiple of (1/100) seconds, and when the light source frequency is 120 Hz, the exposure time is an integral multiple of (1/120) seconds. To do.

As described above, the flicker detection apparatus 200 of the imaging apparatus according to the present exemplary embodiment derives a flicker frequency component vector by detecting the phase change in the pixel column direction of the difference value in the difference pixel area for each frame generation cycle. Deriving means 12, inner product value calculating means 14 for sequentially calculating inner product values between a pair of vectors derived adjacent in time for each period of frame generation, and a plurality of inner product value calculating means 14 The inner product value smoothing means 15a for calculating the inner product value smoothed by applying a low pass filter to the inner product value, and the difference between the smoothed inner product value and the inner product value calculated by the inner product value calculating means 14, Flicker phase acceleration generation means 15 for generating this difference as flicker phase acceleration, and flicker judgment for determining the presence or absence of flicker by comparing the flicker phase acceleration with a predetermined threshold. Since provided with a means 16, and can accurately detect the flicker occurring in a frame or field. In addition, flicker can be detected with simple calculation and constant accuracy by obtaining the inner product value of a pair of vectors to obtain the phase change amount of flicker.

  Next, the procedure of the flicker detection method and the flicker detection program of the imaging apparatus 300 will be described with reference to FIG. This procedure is executed by the CPU 17 giving a command signal to each functional unit based on a program stored in the ROM 18. Further, S in FIG. 5 represents a step.

  First, this procedure starts when a start signal is input to the flicker detection apparatus 200 and the imaging apparatus 300 by the user.

  Next, in step S <b> 100, the frame video signal is read into the flicker detection device 200 via the imaging unit 100.

  Next, in S200, the luminance generation unit 9 is used to generate a luminance signal in association with each pixel for each temporally continuous frame, and then the process proceeds to S300.

  Next, in S300, the region dividing unit 10 is used to generate a plurality of regions by dividing each of the plurality of frames into a plurality of regions in the pixel column direction orthogonal to the pixel row, and then the process proceeds to S400.

  Next, in S400, the luminance totaling means 10a is used to determine the luminance total value for each region, and then the process proceeds to S500.

  Next, in S500, the time direction smoothing unit 11a is used to apply a low-pass filter to the same area of a plurality of frames to smooth the luminance signal in the same area to generate a smooth frame, and then the process proceeds to S600.

Then, in S600, using a differential pixel area generation unit 11b, each s each of the plurality of regions, obtains the difference of the luminance signal of the latest frame adjacent to the plurality of frames obtained by multiplying the smoothed frame and the low-pass filter, the luminance signal Instead, a difference pixel region representing the difference value of the luminance signal is generated, and then the process proceeds to S700.

Next, in S700, the vector deriving means 12 is used to detect the phase change in the pixel column direction of the difference value in the difference pixel region for each frame generation cycle to derive a flicker frequency component vector, and then the process proceeds to S800. .

  Next, in S800, the inner product value calculation means 14 is used to calculate the inner product value between a pair of vectors derived adjacent in time for each frame generation cycle, and then the process proceeds to S900.

  Next, in S900, the inner product value smoothing means 15a is used to calculate a smoothed inner product value obtained by applying a low pass filter to the plurality of inner product values generated in S900, and then the process proceeds to S1000.

  Next, in S1000, the inner product value difference calculation means 15b is used to calculate the difference between the inner product value smoothed in S900 and the inner product value calculated in 800 to calculate the flicker phase acceleration, and then the process proceeds to S1100.

  Next, in S1100, the time direction smoothing means 16b is used to smooth the flicker phase acceleration by applying a low-pass filter in the time direction to the absolute value of the flicker phase acceleration, and then the flow proceeds to S1200.

  Next, in S1200, the threshold determination unit 16c is used to compare the flicker phase acceleration smoothed in S1100 with a predetermined threshold. If the smoothed flicker phase acceleration is smaller than the threshold (Yes), the process proceeds to S1300. On the other hand, if the smoothed flicker phase acceleration is greater than or equal to the threshold (No), the process moves to S1400.

  Next, in S1300, the light source frequency detection unit 16d is used to detect whether the light source frequency of the flicker is 100 Hz or 120 Hz, and an appropriate exposure time is acquired so as to reduce the flicker in association with the light source frequency. , The process proceeds to S1400.

  Next, in S1400, the CPU 17 is used to generate control signals such as the shutter speed of the mechanical shutter 2, the charge accumulation and charge readout time of the image sensor 3a so as to satisfy the appropriate exposure time acquired in S1300, and this flicker detection The process of the method and flicker detection program ends.

  The luminance generation step of the present invention corresponds to S200, the frame region division step of the present invention corresponds to S300, the region luminance signal aggregation step of the present invention corresponds to S400, and the time direction smoothing step of the present invention corresponds to S500. The flicker evaluation value generation step of the present invention corresponds to S600, the vector derivation step of the present invention corresponds to S700, the inner product value calculation step of the present invention corresponds to S800, and the inner product value smoothing step of the present invention Corresponds to S900, the flicker phase acceleration generation step of the present invention corresponds to S1000, and the flicker determination step of the present invention corresponds to S1100 and S1200.

As described above, the flicker detection method and the flicker detection program of the imaging apparatus 300 according to the present exemplary embodiment detect the flicker by detecting the phase change in the pixel column direction of the difference value of the difference pixel area at every frame generation period in S700. In S800, frequency product vectors are derived, and then, in each frame generation period, inner product values (a so-called phase difference) between a pair of vectors derived adjacent in time are sequentially calculated, and then In S900, a smoothed inner product value is calculated by applying a low pass filter to a plurality of inner product values, and in S1000, the smoothed inner product value and the inner product value calculated in S800 (inner product value calculating step) are calculated. And the difference is generated as flicker phase acceleration, and then, in S1100, a plurality of flicker phase accelerations are smoothed. Then, in S1200, by comparing the flicker phase acceleration blunted with S1100 with a predetermined threshold value, it can be accurately detected a flicker occurring in a frame or field. Further, by obtaining the inner product value of a pair of vectors via S800 and obtaining the amount of flicker phase change, flicker can be detected with simple calculation and constant accuracy.

  As mentioned above, although one Example of this invention was described, this invention is not limited to the said Example, Various aspects can be taken.

  A flicker detection method, a flicker detection apparatus, and a flicker detection program for an image pickup apparatus according to the present invention provide a CMOS (Complementary Metal Oxide Semiconductor) for an object illuminated with a light source whose brightness changes due to a power frequency such as a fluorescent lamp. This is suitable for reliably detecting and reducing flicker generated in an image when an image pickup device that reads out image data in an image pickup device by scanning in units of pixel rows like the image pickup device configured in the above. .

  DESCRIPTION OF SYMBOLS 1 ... Imaging optical system, 1a ... Imaging lens, 1b ... Iris (diaphragm), 2 ... Mechanical shutter, 3 ... CMOS (Complementary Metal-Oxide Semiconductor) sensor, 3a ... Imaging element, 3b ... Variable gain amplifier (AGC: (Automatic Gain Control), 3c... A / D converter, 9... Luminance generation means, 10. Pixel area generation means, 12... Vector derivation means, 13... Previous frame vector holding means, 14... Inner product value calculation means, 15... Flicker phase acceleration generation means, 15 a. ... Flicker determination means, 16a ... Flicker phase acceleration Counter value acquisition means, 16b ... time direction smoothing means, 16c ... threshold value judgment means, 16d ... light source frequency detection means, 17 ... CPU (Central Processing Unit), 18 ... ROM (Read Only Memory), 100 ... imaging unit, 200 ... Imaging device, 300... Flicker detection device.

Claims (9)

The luminance signal difference between multiple frames that are temporally continuous based on the video signal that is output by scanning the pixel row by scanning the signal to each pixel arranged in a matrix via the image sensor. A flicker detection method for an image pickup apparatus for detecting a flicker component generated under a blinking illumination light source,
A luminance generation step for generating a luminance signal in association with each pixel for each of a plurality of temporally continuous frames;
Frame region division for dividing each frame into a plurality of regions in a pixel column direction orthogonal to the pixel row for each of the plurality of frames, and generating a plurality of regions with a predetermined number of pixel rows as one unit Steps,
A time direction smoothing step for generating a smooth frame by applying a low pass filter to smooth the luminance signal of the same region for each same region in the plurality of frames,
For each of the plurality of regions, obtain a difference in luminance signal between the smoothed frame and the latest frame adjacent to the plurality of frames, and a difference pixel region representing a difference value of the luminance signal instead of the luminance signal A flicker evaluation value generation step to be generated;
A vector derivation step for deriving a flicker frequency component vector by detecting a phase change in the pixel column direction of the difference value for each frame generation period;
An inner product value calculating step for sequentially calculating inner product values between a pair of vectors derived adjacent in time for each frame generation period;
An inner product value smoothing step for calculating an inner product value obtained by applying a low pass filter to the inner product values generated in the inner product value calculating step and smoothing the inner product value,
A flicker phase acceleration generation step for obtaining a difference between the smoothed inner product value and the inner product value calculated in the inner product value calculation step, and generating the difference as a flicker phase acceleration;
A flicker determination step for determining the presence or absence of flicker by comparing the flicker phase acceleration with a predetermined threshold;
A flicker detection method for an imaging apparatus, comprising:
For each of the plurality of areas divided in the frame area dividing step, an area luminance signal totaling step for generating a total value of luminance signals is provided,
In the time direction smoothing step and the flicker evaluation value generating step, the total value of the luminance signal is used instead of the luminance signal.
The flicker detection method for an imaging apparatus according to claim 1.
In the flicker determination step, a flicker phase acceleration that is smoothed by applying a low-pass filter to the absolute value of the plurality of flicker phase accelerations in a time direction is generated, and the smoothed flicker phase acceleration is set as the predetermined threshold value. By comparing, determine the presence or absence of flicker,
The flicker detection method for an image pickup apparatus according to claim 1, wherein the flicker detection method is used.
The luminance signal difference between multiple frames that are temporally continuous based on the video signal that is output by scanning the pixel row by scanning the signal to each pixel arranged in a matrix via the image sensor. A flicker detection device for an imaging device that detects flicker components generated under a blinking illumination light source,
A luminance generation means for generating a luminance signal in association with each pixel for each of a plurality of temporally continuous frames;
Frame region division for dividing each frame into a plurality of regions in a pixel column direction orthogonal to the pixel row for each of the plurality of frames, and generating a plurality of regions with a predetermined number of pixel rows as one unit Means,
Time direction smoothing means for generating a smooth frame by smoothing the luminance signal of the same region by applying a low-pass filter for each same region in the plurality of frames;
For each of the plurality of regions, obtain a difference in luminance signal between the smoothed frame and the latest frame adjacent to the plurality of frames, and a difference pixel region representing a difference value of the luminance signal instead of the luminance signal Flicker evaluation value generating means for generating;
Vector derivation means for deriving a flicker frequency component vector by detecting a phase change in the pixel column direction of the difference value for each frame generation period;
Inner product value calculating means for sequentially calculating inner product values between a pair of vectors derived adjacent in time for each frame generation period;
Inner product value smoothing means for calculating an inner product value obtained by applying a low-pass filter to a plurality of inner product values generated by the inner product value calculating means,
Flicker phase acceleration generation means for obtaining a difference between the smoothed inner product value and the inner product value calculated by the inner product value calculation means, and generating the difference as flicker phase acceleration;
Flicker determination means for determining the presence or absence of flicker by comparing the flicker phase acceleration with a predetermined threshold;
An apparatus for detecting flicker in an imaging apparatus.
A region luminance signal totaling unit that generates a total value of luminance signals for each of the plurality of regions divided by the frame region dividing unit,
The time direction smoothing unit and the flicker evaluation value generating unit are configured to use the total value of the luminance signal instead of the luminance signal.
The flicker detection apparatus for an imaging apparatus according to claim 4.
The flicker determination means generates a flicker phase acceleration that is smoothed by applying a low-pass filter to the absolute value of the plurality of flicker phase accelerations in a time direction, and the smoothed flicker phase acceleration is set as the predetermined threshold value. Configured to determine the presence or absence of flicker by comparing,
6. The flicker detection apparatus for an image pickup apparatus according to claim 4, wherein the flicker detection apparatus is used.
The luminance signal difference between multiple frames that are temporally continuous based on the video signal that is output by scanning the pixel row by scanning the signal to each pixel arranged in a matrix via the image sensor. A flicker detection program for an imaging device that detects flicker components generated under a blinking illumination light source,
A luminance generation step for generating a luminance signal in association with each pixel for each of a plurality of temporally continuous frames;
Frame region division for dividing each frame into a plurality of regions in a pixel column direction orthogonal to the pixel row for each of the plurality of frames, and generating a plurality of regions with a predetermined number of pixel rows as one unit Steps,
A time direction smoothing step for generating a smooth frame by applying a low pass filter to smooth the luminance signal of the same region for each same region in the plurality of frames,
For each of the plurality of regions, obtain a difference in luminance signal between the smoothed frame and the latest frame adjacent to the plurality of frames, and a difference pixel region representing a difference value of the luminance signal instead of the luminance signal A flicker evaluation value generation step to be generated;
A vector derivation step for deriving a flicker frequency component vector by detecting a phase change in the pixel column direction of the difference value for each frame generation period;
An inner product value calculating step for sequentially calculating inner product values between a pair of vectors derived adjacent in time for each frame generation period;
An inner product value smoothing step for calculating an inner product value obtained by applying a low pass filter to the inner product values generated in the inner product value calculating step and smoothing the inner product value,
A flicker phase acceleration generation step for obtaining a difference between the smoothed inner product value and the inner product value calculated in the inner product value calculation step, and generating the difference as a flicker phase acceleration;
A flicker determination step for determining the presence or absence of flicker by comparing the flicker phase acceleration with a predetermined threshold;
A program for detecting flicker in an imaging apparatus, which causes a computer to execute the above.
For each of the plurality of areas divided in the frame area dividing step, an area luminance signal totaling step for generating a total value of luminance signals is provided,
In the time direction smoothing step and the flicker evaluation value generating step, the total value of the luminance signal is used instead of the luminance signal.
The computer-executable flicker detection program according to claim 7, wherein the computer executes the program as described above.
In the flicker determination step, a flicker phase acceleration that is smoothed by applying a low-pass filter to the absolute value of the plurality of flicker phase accelerations in a time direction is generated, and the smoothed flicker phase acceleration is set as the predetermined threshold value. By comparing, determine the presence or absence of flicker,
9. The computer-executable flicker detection program according to claim 7, wherein the program is executed by a computer.

Images