JP2004173016A - Flicker eliminating device and its method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a flicker eliminating device capable of eliminating the influence of the flicker of a discharge lamp such as a fluorescent lamp included in an image photographed at a high frame rate. <P>SOLUTION: A high speed image storage part 1 is a camera capable of photographing high speed images, which can photograph three or more frames of images within the lighting cycle of the discharge lamp such as the fluorescent lamp. The high speed image photographed in the high speed image storage part 1 is transferred to a high speed image storage part 2 and stored in a memory over the time which is an integral multiple of the lighting cycle of the discharge lamp. In a flicker elimination part 3, the image from which the influence of the flicker is eliminated is generated by using the image stored in the high speed image storage part 2. A high speed image output part 4 outputs moving image data. <P>COPYRIGHT: (C)2004,JPO

Description

ここで、Ｉ（ｘ，ｙ，ｉ）は現在の画像からｉ枚前の画像の位置（ｘ，ｙ）での輝度値、Ｐは画像全体、または、画像の部分領域の画素の集合、Ｎ（Ｐ）はＰに含まれる画素の数である
＜２−１＞第１の実施例
フリッカー除去部３の第１の実施例について、図３に基づいて説明する。
【００４３】
なお、本実施例の場合には、蛍光灯等の放電灯の点灯周期Ｔ［ｍｓ］内に３枚以上の画像撮影できるものでなくてもよく、蛍光灯等の放電灯の点灯周期Ｔ［ｍｓ］内に１フレーム以上撮影できればよい。例えば、点灯周期Ｔ［ｍｓ］内に２フレームや、（Ｔ×２）［ｍｓ］内に３枚を撮影してもよい。
【００４４】
交流電源によって駆動される蛍光灯等の放電灯は、交流電源の周期の１／２の周期Ｔで点灯する。これにより、高速度画像の輝度値も周期Ｔで変化する。この輝度値変化一周期にかかる撮影される画像の枚数は、前述のように、高速度画像の撮像間隔をｆ［ｍｓ］とすると、Ｔ／ｆ枚である。Ｔ／ｆより大きく、かつ、最も小さい整数をＦとすると、Ｆ枚の画像の平均をとればフリッカーの影響のない画像を得ることができる。すなわち、ノイズ除去のため、フリッカー周期のｎ倍（ｎは１以上の整数）の画像の平均をとって、フリッカーの影響のない画像とする。
【００４５】
高速度画像蓄積部２に蓄積されている、現在の画像からｉ枚前の過去の画像の位置（ｘ，ｙ）の画素値をＩ（ｘ，ｙ，ｉ）とすると、フリッカーの影響のない時系列的に輝度値を平均した平均画像Ｉ’（ｘ，ｙ）は、数２のようになる。
【００４６】
【数２】

数２によって補正された画像は、各画像の輝度値を時系列的に平均した輝度値を有する画素が集合した画像であり、画像中に移動物体がなく、シーンに大きな変化がなければ、上記の数１で説明した平均輝度値であるフリッカー特徴量とほぼ等しくことと等価となる。
【００４７】
図３にこの処理のフローチャートを示す。画像が１枚入力されるたびにこの処理を繰り返す。
【００４８】
ステップ１において、バッファ画像Ｉｎ（ｘ、ｙ）を０に初期化する。
【００４９】
ステップ２において、画像の枚数ｉを処理回数として、ｉを０に初期化する。
【００５０】
ステップ３において、高速度画像蓄積部２からｉ番目の画像Ｉ（ｘ，ｙ，ｉ）を読み込む。
【００５１】
ステップ４において、バッファ画像Ｉｎ（ｘ、ｙ）に画像Ｉ（ｘ，ｙ，ｉ）を足し込む。
【００５２】
ステップ５において、ｉに１を付加する。
【００５３】
ステップ６において、ｉ＜ｎＦであればステップ３に戻り、ｉ＞＝ｎＦであればステップ７に進む。
【００５４】
ステップ７において、（ｎ×Ｆ）個を足し込んだバッファ画像Ｉｎ（ｘ、ｙ）を（ｎ×Ｆ）で割って、すなわち数２を計算して、フリッカーの影響のない時系列で輝度値を平均した平均画像Ｉ’（ｘ，ｙ）を高速度画像出力部４へ出力する。
【００５５】
第１の実施例のフリッカー除去部３では、（ｎ×Ｆ）フレームにわたって輝度値の平均をとるため、静止物体を撮影するのに好適であると共に、その計算方法が簡単であるとの特徴がある。
【００５６】
高速度画像出力部４では、フリッカーが除去された画像を時系列で順番に出力して、動画像として再生する。
【００５７】
この再生される動画像の時刻ｔにおける画像は、時刻ｔの画像から過去の（ｎ×Ｆ）フレームの画像を用いてフリッカーを除去した画像であり、時刻（ｔ＋１）における画像は、時刻（ｔ＋１）の画像から過去の（ｎ×Ｆ）枚の画像を用いてフリッカーを除去した画像である。
【００５８】
＜２−２＞第２の実施例
次に、フリッカー除去部３の第２の実施例について、図４と図５に基づいて説明する。
【００５９】
図４に示すように、フリッカー補正部３は、領域選択部３２１、フリッカー特徴量計算部３２２、フリッカー特徴量選択部３２３、画素修正部３２４より構成される。なお、本実施例の場合には、蛍光灯等の放電灯の点灯周期Ｔ［ｍｓ］内に３枚以上の画像撮影できる高速度画像の撮影が可能なカメラシステムが好適である。
【００６０】
領域選択部３２１によりフリッカー特徴量を計算する画像領域Ｐを選択する。選択する領域は、以下のような実現方法を選択して用いる。
【００６１】
（１）画像全体を選択する、
（２）画像内の一部の領域を選択する、
（３）画像全体にわたってまばらに画素を選択する、
（４）画像内の一部の領域についてまばらに画素を選択する。
【００６２】
ここでまばらに画素を選択するとは、例えば、縦方向、横方向それぞれ２画素おきに画素を選択するということである。
【００６３】
画像内の一部の領域を選択する方法として、画像内の一定の領域を選択する方法と、画像中に映る物体を追跡することにより各画像で異なる領域を選択する方法がある。
【００６４】
画像内の移動物体の追跡方法には、テンプレートマッチング、画像内の各ピクセルの運動情報であるオプティカルフローを用いる方法等がある。
【００６５】
オプティカルフローを用いる場合、以下のように追跡を行う。
【００６６】
まず、追跡を開始する画像において、フリッカー補正を行いたい画像中の注目領域を選択領域として設定する。その後の画像では、選択領域内でオプティカルフローを計算して選択領域内の画素の移動先を取得し、全ての画素の移動先を含むように選択領域を更新する。
【００６７】
選択された画素集合Ｐを用いて、フリッカー特徴量計算部３２２により領域的な輝度値の平均値、すなわち、フリッカー特徴量である輝度値平均値Ｉａ（ｉ）を数１の式を用いて計算する。
【００６８】
フリッカー特徴量選択部３２３では各画像について得られているフリッカー特徴量の中から基準となる基準フリッカー特徴量（輝度値平均値）を選択する。
【００６９】
最終的に得られる画質を考慮して、数３に示すように、最も大きい値の輝度値平均値を基準フリッカー特徴量として選択する。
【００７０】
【数３】

最後に、画素値修正部３２４で、現在の画像のフリッカー特徴量が基準フリッカー特徴量と等しくなるように、数４に示すように画素値を変更する。
【００７１】
【数４】

この第２の実施例のフリッカー除去部３では、移動物体を撮影するのに好適である。
【００７２】
以上の処理を実現するフローチャートを図５に示す。
【００７３】
ステップ１１において、画像の枚数ｉを処理回数として、ｉを０に初期化する。
【００７４】
ステップ１２において、高速度画像蓄積部２からｉ番目の画像Ｉ（ｘ，ｙ，ｉ）を読み込む。
【００７５】
ステップ１３において、数１に示すように、ｉ番目における輝度値平均値Ｉａ（ｉ）を計算する。
【００７６】
ステップ１４において、ｉに１を付加する。
【００７７】
ステップ１５において、ｉ＜（ｎ×Ｆ）であればステップ１２に戻り、ｉ＞＝（ｎ×Ｆ）であればステップ１６に進む。
【００７８】
ステップ１６において、数３に示すように、（ｎ×Ｆ）個の中から最も大きい値の輝度値平均値を基準フリッカー特徴量Ｉｒとして選択する。
【００７９】
ステップ１７において、数４に示すように、フリッカーの影響のない輝度値を平均した平均画像Ｉ’（ｘ，ｙ）を高速度画像出力部４へ出力する。
【００８０】
＜２−３＞第３の実施例
フリッカー除去部３の第３の実施例について図６に基づいて説明する。
【００８１】
フリッカー除去部３の第１の実施例では、（ｎ×Ｆ）枚にわたって輝度値の平均をとるため、動きのある領域ではぼやけた画像になる。そのため、変化が激しい画像では多少画質が劣化する。
【００８２】
これに対してフリッカー除去部３の第２の実施例では、フリッカーによる輝度値の変動が、数４で示されるようなモデルに完全に当てはまらない場合にはフリッカーの影響が完全には除去されずにその影響が残る。
【００８３】
そこで、これを解決するために図６に示すように、第３の実施例ではフリッカー補正部３が、領域選択部３３１、画像変化率計算部３３２、フリッカー補正方法選択部３３３、フリッカー補正部３２４より構成される。
【００８４】
画像変化率計算部３３２は、領域選択部３３１によって選択した画像全体または注目領域に対して画像がどれだけ変化しているかを表す画像変化率を計算する。この画像変化率として、例えば次のような量を用いる。
【００８５】
（１）各ピクセルの運動ベクトルを算出し、そのベクトルの大きさの平均値を画像変化率として用いる。
【００８６】
（２）輝度値の微分値の絶対値和を各画像の注目領域内で計算し、絶対値和の画像間の差を画像変化率として用いる。
【００８７】
（３）輝度値の二次微分の絶対値和を各画像の注目領域内で計算し、絶対値和の画像間の差を画像変化率として用いる。
【００８８】
フリッカー補正方法選択部３３３は、画像変化率が閾値よりも小さければ第１の実施例の方法を、大きければ第２の実施例の方法を選択する。
【００８９】
そして、フリッカー補正部３３４は、選択した各実施例の方法で、上記で説明したようにフリッカーを補正する。
【００９０】
この第３の実施例のフリッカー除去部３を用いると、動きのある領域でもぼやけず、また、静止物体に対しては簡単な計算でフリッカーを除去できる。
【００９１】
【発明の効果】
本発明によれば、高いフレームレートのビデオ映像において、蛍光灯等の放電灯のフリッカーによる画素値の変動を取り除いて高品質な画像を得ることが可能となる。
【図面の簡単な説明】
【図１】本発明の一実施形態の示すブロック図である。
【図２】画像蓄積部の一実施形態の実施例を示すブロック図である。
【図３】フリッカー補正部の第１の実施例に関するフローチャートである。
【図４】フリッカー補正部の第２の実施例に関するブロック図である。
【図５】フリッカー補正部の第２の実施例に関するフローチャートである。
【図６】フリッカー補正部の第３の実施例に関するブロック図である。
【図７】本実施形態の一般環境を示す全体図である。
【符号の説明】
１・・・高速度画像取得部
２・・・高速度画像蓄積部
３・・・フリッカー除去部
４・・・高速度画像出力部
２１・・・書き込みバッファ選択部
２２・・・読み出しバッファ選択部
３２１・・・領域選択部
３２２・・・フリッカー特徴量計算部
３２３・・・フリッカー特徴量選択部
３２４・・・画素修正部
３３１・・・領域選択部
３３２・・・画像変化率計算部
３３３・・・フリッカー補正方法選択部
３３４・・・フリッカー補正部[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a flicker removing apparatus and a method for removing the influence of flicker such as fluorescent light present in an image captured at a high frame rate.
[0002]
[Prior art]
Discharge lamps such as fluorescent lamps and mercury lamps are turned on at a half cycle of the AC power supply. Therefore, when video is shot in a fluorescent lamp illumination environment, flicker occurs and causes deterioration in image quality. For this reason, conventionally, a method of removing flicker has been proposed.
[0003]
A method of changing the charge accumulation time (shutter speed) according to the flicker cycle of the shooting state (see Patent Documents 1 to 4), a method of changing the frame rate (see Patent Document 5), and adaptively changing the gain As in the method (see Patent Document 6), flicker can be removed by changing the photographing conditions.
[0004]
According to a method of storing an image for a specific period according to the flicker cycle and the vertical synchronization of the camera and selecting and outputting only a specific image from the image (see Patent Documents 7 and 8), the output interval of the image is limited. Although limited, it is effective in removing flicker.
[0005]
A flicker elimination method (see Patent Document 9) in which the influence of flicker varies depending on the location in a field, such as a MOS image sensor, has also been proposed.
[0006]
All of the above methods are intended for video at a video rate (approximately 30 Hz), and are difficult to apply to moving images faster than the lighting frequency of a fluorescent lamp (for example, 100 Hz in Kanto) or have low effectiveness. .
[0007]
Since the high-speed image has a high frame rate, it is impossible to change the shooting conditions such as the exposure time so that the influence of flicker does not appear. Also, the method of selecting an image cannot be applied to a video having a high frame rate. In a high-speed image, the influence of flicker does not differ within one field.
[0008]
[Patent Document 1]
JP-A-11-205658
[Patent Document 2]
JP-A-5-191711
[Patent Document 3]
JP-A-9-83945
[Patent Document 4]
JP-A-7-336586
[Patent Document 5]
JP 2002-165141 A
[Patent Document 6]
JP-A-2-260977
[Patent Document 7]
JP-A-9-270950
[Patent Document 8]
JP-A-11-261881
[Patent Document 9]
JP-A-11-122513
[Problems to be solved by the invention]
The present invention provides a flicker removing apparatus and method for removing flicker of a high-speed image whose frame rate is much higher than the lighting cycle of a discharge lamp such as a fluorescent lamp and outputting a high-speed image free from the influence of flicker.
[0018]
[Means for Solving the Problems]
The invention according to claim 1 is an image photographing means capable of photographing a plurality of images during a lighting cycle T of a discharge lamp, and a pixel value of each pixel of a past image photographed over the lighting cycle T or more. Using the image storage unit that holds the image data and the past image taken before the stored reference time, the pixel value is corrected so that the flicker feature amount, which is the average value of the luminance values, is equal between the images. By this means, there is provided flicker removing means for generating an image at a reference time in which the influence of flicker has been removed, and image output means for outputting the image at the reference time from which the flicker has been removed as a moving image in chronological order. It is a flicker removing device characterized by the following.
[0019]
According to a second aspect of the present invention, the flicker elimination means includes a time setting means for setting a time which is an integral multiple of the lighting period T, and the set time based on the captured image of the reference time held by the image storage means. A flicker feature amount calculating unit that calculates a flicker feature amount that is a time-series average value of luminance values for each pixel up to a past image captured at a time closest to the time that has just been traced back; 2. The flicker removing device according to claim 1, further comprising: an image generating unit configured to generate an image at a reference time from which the influence of flicker has been removed by using the flicker feature amount of the pixel as a pixel value.
[0020]
The invention according to claim 3, wherein the flicker removal means selects a selection area for calculating a flicker feature amount for a past image taken before a reference time held by the image storage means. A flicker feature amount calculating unit that calculates a flicker feature amount that is a regional average value of luminance values of the pixels in the selected region with respect to the photographed past image and the photographed reference time image. A flicker feature amount selecting means for selecting, from the calculated flicker feature amount, a reference flicker feature amount serving as a reference for removing the influence of flicker; and the flicker feature amount of the image at the photographed reference time is selected. By correcting the pixel value in the selected area so as to be equal to the reference flicker feature, A pixel value correcting means for generating an image of the reference time effect removal of a flicker removing apparatus according to claim 1, characterized by including the.
[0021]
According to a fourth aspect of the present invention, the flicker removing unit calculates an image change rate which is a change speed of a past image held by the image storage unit; If the value is smaller than the threshold value, an image at a reference time from which the influence of flicker has been removed by the first removing means is generated. If the image change rate is greater than the threshold value, the reference value from which the effect of flicker has been removed by the second removing means. A removing method selecting and executing means for selecting an image of a time to be generated, wherein the first removing means is a time setting means for setting a time which is an integral multiple of the lighting cycle T, and is held by the image accumulating means. The brightness value of each pixel from the image of the reference time taken to the past image taken at the time closest to the time set back by the above set time A flicker feature calculating means for calculating a flicker feature which is a time-series average value, and an image for generating an image at a reference time excluding the influence of flicker, using the calculated flicker feature of each pixel as a pixel value. Generation means, wherein the second removal means selects a selection area for calculating a flicker feature amount for a past image photographed before a reference time held by the image accumulation means. Selecting means for calculating a flicker characteristic amount, which is a regional average value of luminance values of respective pixels in the selected region, with respect to the photographed past image and the photographed reference time image; Calculating means for selecting, from the calculated flicker feature value, a reference flicker feature value serving as a reference for removing the influence of flicker; A flicker characteristic amount selecting unit that corrects a pixel value in the selected area so that a flicker characteristic amount of the image at the photographed reference time becomes equal to the selected reference flicker characteristic amount. 2. The flicker removing device according to claim 1, further comprising: a pixel value correcting unit configured to generate an image at a reference time from which the image is removed.
[0022]
The invention according to claim 5 is the flicker removing device according to claim 3 or 4, wherein the selected area is the entire image or a part of the image.
[0023]
The invention according to claim 6 is an image photographing step of photographing a plurality of images during a lighting cycle T of the discharge lamp, and holding a pixel value of each pixel of a past image photographed over the lighting cycle T or more. Image storage step, by using the past image captured before the held reference time, by correcting the pixel value so that the flicker feature amount that is the average value of the luminance value is equal between the images, A flicker removing step of generating an image at a reference time from which the influence of flicker has been removed; and an image outputting step of outputting the image at the reference time from which the flicker has been removed as a moving image in chronological order. This is a flicker removal method.
[0024]
The invention according to claim 7 is an image photographing function for photographing a plurality of images during the lighting cycle T of the discharge lamp, and holds a pixel value of each pixel of a past image photographed over the lighting cycle T or more. By using an image storage function and a past image taken before the held reference time, by correcting the pixel value so that the flicker feature amount, which is the average value of the luminance values, is equal between the images. A computer realizes a flicker removing function for generating an image at a reference time from which the influence of flicker has been removed, and an image output function for outputting an image at the reference time from which the flicker has been removed as a moving image in chronological order. Is a program for a flicker removal method.
[0025]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, a flicker removing device according to an embodiment of the present invention will be described with reference to FIGS.
[0026]
<1> Basic Configuration of Flicker Removal Apparatus FIG. 1 shows a basic configuration example of the flicker removal apparatus of the present embodiment, and includes a high-speed image acquisition unit 1, a high-speed image storage unit 2, a flicker removal unit 3, It comprises a high-speed image output unit 4. Each function of the high-speed image acquisition unit 1, the high-speed image storage unit 2, the flicker removal unit 3, and the high-speed image output unit 4 is realized by a program stored in a computer. Then, as shown in FIG. 7, as a general environment, a flicker removing device is disposed inside a room illuminated by a fluorescent lamp to obtain an image from which flicker has been removed.
[0027]
<1-1> Configuration of High-Speed Image Acquisition Unit 1 The high-speed image acquisition unit 1 captures high-speed images capable of capturing three or more images within a lighting cycle T [ms] of a discharge lamp such as a fluorescent lamp. A possible camera system. Basically, it is composed of parts of a lens, an aperture, an image sensor, and an AD converter. For example, 300 images can be taken per second. Then, as shown in FIG. 7, the camera system is shot at the subject to obtain a high-speed image.
[0028]
<1-2> Configuration of the high-speed image storage unit 2 The high-speed image storage unit 2 includes a write buffer selection unit 21, a buffer, and a read buffer selection unit 22, as shown in FIG.
[0029]
The high-speed image captured by the high-speed image acquisition unit 1 is transferred to the high-speed image storage unit 2 and stored in the memory by the write buffer selection unit 21 as shown in FIG. That is, it is stored in the memory over a time period of n times (n is an integer) the lighting cycle T [ms] of the discharge lamp.
[0030]
This memory will be described.
[0031]
Assuming that the imaging interval of the high-speed image is f [ms], an integer F larger than T / f and the smallest is the number of images captured during the lighting cycle T of the discharge lamp. Assuming that the capacity of a memory capable of storing the pixel values of one image is m, the memory is a ring buffer including (n × F) image buffers having a capacity m.
[0032]
Then, the image buffer in the memory is selected by the write buffer selection unit 21, the image input from the high-speed image acquisition unit 1 is written, and the read buffer selection unit 22 is offset from the position of the image buffer storing the latest image. By giving a value, an image at that position can be output.
[0033]
<1-3> Flicker removing unit 3
The flicker removing unit 3 generates an image at a reference time from which the influence of flicker has been removed, using the captured image stored in the memory in the high-speed image storage unit 2.
[0034]
Detailed examples will be described later.
[0035]
Note that the reference time corresponds to the current time when an image captured by the high-speed image acquisition unit 1 is viewed in real time, and the high-speed image storage unit 2 calls up an image stored in the past. In the case of reproduction, the past time called corresponds to the reference time. In the following description, it is assumed that the current time corresponds to the reference time for the sake of simplicity.
[0036]
<1-4> High-speed image output unit 4
Finally, the high-speed image output unit 4 reproduces an image at a reference time (for example, the current time) from which the influence of flicker has been removed by a display device or the like in chronological order, and outputs a high-speed moving image.
[0037]
<2> Embodiments of Flicker Removal Unit 3 Hereinafter, three types of embodiments of the flicker removal unit 3 will be described in detail, but before that, the basic items that are the premise of each embodiment will be described.
[0038]
Since the brightness of the high-speed image also changes at the lighting cycle T of the discharge lamp, the average value of the luminance values of the entire image or of the attention area can be used as the image feature amount related to flicker. This is called “flicker feature value”.
[0039]
If the image is corrected so that the luminance value of each image becomes equal to the obtained flicker feature, an image free from the influence of flicker can be obtained.
[0040]
Therefore, the flicker removing unit 3 corrects the pixel value so that the luminance value of each image becomes equal to the average luminance value Ia (i), which is a flicker feature amount.
[0041]
The average luminance value is as follows.
[0042]
(Equation 1)

Here, I (x, y, i) is the luminance value at the position (x, y) of the image i images before the current image, P is the entire image or a set of pixels in a partial region of the image, N (P) is the number of pixels included in P <2-1> First Embodiment A first embodiment of the flicker removal unit 3 will be described with reference to FIG.
[0043]
In the case of this embodiment, it is not necessary that three or more images can be captured within the lighting cycle T [ms] of the discharge lamp such as a fluorescent lamp, and the lighting cycle T [ [ms]. For example, two frames may be shot within the lighting period T [ms], or three frames may be shot within (T × 2) [ms].
[0044]
A discharge lamp such as a fluorescent lamp driven by an AC power supply lights up at a cycle T that is １ ／ of the cycle of the AC power supply. Thereby, the luminance value of the high-speed image also changes in the cycle T. As described above, the number of images to be shot in one cycle of the luminance value change is T / f, where the shooting interval of the high-speed image is f [ms]. Assuming that an integer greater than T / f and the smallest integer is F, an image free of flicker can be obtained by averaging F images. That is, in order to remove noise, an image having n times the flicker cycle (n is an integer of 1 or more) is averaged to obtain an image free from flicker.
[0045]
Assuming that the pixel value at the position (x, y) of the past image i pages before the current image stored in the high-speed image storage unit 2 is I (x, y, i), there is no flicker effect. An average image I ′ (x, y) obtained by averaging the luminance values in time series is as shown in Expression 2.
[0046]
(Equation 2)

The image corrected by Equation 2 is an image in which pixels having luminance values obtained by averaging the luminance values of the images in time series are collected. If there is no moving object in the image and there is no significant change in the scene, This is equivalent to being substantially equal to the flicker feature amount, which is the average luminance value described in Equation (1).
[0047]
FIG. 3 shows a flowchart of this processing. This process is repeated each time one image is input.
[0048]
In step 1, the buffer image In (x, y) is initialized to zero.
[0049]
In step 2, i is initialized to 0 with the number i of images as the number of times of processing.
[0050]
In step 3, the i-th image I (x, y, i) is read from the high-speed image storage unit 2.
[0051]
In step 4, the image I (x, y, i) is added to the buffer image In (x, y).
[0052]
In step 5, 1 is added to i.
[0053]
In step 6, if i <nF, the process returns to step 3, and if i> = nF, the process proceeds to step 7.
[0054]
In step 7, the buffer image In (x, y) obtained by adding (n × F) pixels is divided by (n × F), that is, the expression 2 is calculated, and the luminance value is calculated in a time series without the influence of flicker. Is output to the high-speed image output unit 4.
[0055]
Since the flicker removing unit 3 of the first embodiment averages luminance values over (n × F) frames, the flicker removing unit 3 is suitable for photographing a still object and has a simple calculation method. is there.
[0056]
The high-speed image output unit 4 sequentially outputs the images from which flicker has been removed in chronological order, and reproduces the images as moving images.
[0057]
The image at the time t of the reproduced moving image is an image obtained by removing flicker from the image at the time t using the image of the past (n × F) frame, and the image at the time (t + 1) is the image at the time (t + 1). ) Is an image from which flicker has been removed using the past (n × F) images.
[0058]
<2-2> Second Embodiment Next, a second embodiment of the flicker removing section 3 will be described with reference to FIGS.
[0059]
As shown in FIG. 4, the flicker correction unit 3 includes an area selection unit 321, a flicker feature value calculation unit 322, a flicker feature value selection unit 323, and a pixel correction unit 324. In the case of this embodiment, a camera system capable of capturing high-speed images capable of capturing three or more images within a lighting cycle T [ms] of a discharge lamp such as a fluorescent lamp is preferable.
[0060]
An image area P for which a flicker feature amount is calculated is selected by the area selection unit 321. For the area to be selected, the following realization method is selected and used.
[0061]
(1) Select the entire image,
(2) selecting a partial area in the image,
(3) sparsely selecting pixels over the entire image,
(4) Pixels are sparsely selected for a partial area in the image.
[0062]
Here, sparsely selecting pixels means, for example, selecting pixels every two pixels in the vertical and horizontal directions.
[0063]
As a method of selecting a partial area in an image, there are a method of selecting a certain area in the image and a method of selecting a different area in each image by tracking an object appearing in the image.
[0064]
Methods for tracking a moving object in an image include template matching, a method using an optical flow that is motion information of each pixel in the image, and the like.
[0065]
When optical flow is used, tracking is performed as follows.
[0066]
First, in an image to start tracking, an attention area in an image for which flicker correction is to be performed is set as a selection area. In the subsequent image, the optical flow is calculated in the selected area to obtain the destinations of the pixels in the selected area, and the selected area is updated to include the destinations of all the pixels.
[0067]
Using the selected pixel set P, the flicker feature value calculation unit 322 calculates the average value of the regional brightness values, that is, the brightness value average value Ia (i), which is the flicker feature value, using the equation (1). I do.
[0068]
The flicker feature selection unit 323 selects a reference flicker feature (average luminance value) as a reference from the flicker features obtained for each image.
[0069]
In consideration of the finally obtained image quality, as shown in Expression 3, the largest average luminance value is selected as the reference flicker feature amount.
[0070]
[Equation 3]

Finally, the pixel value correction unit 324 changes the pixel value as shown in Expression 4 so that the flicker feature amount of the current image becomes equal to the reference flicker feature amount.
[0071]
(Equation 4)

The flicker removing unit 3 of the second embodiment is suitable for photographing a moving object.
[0072]
FIG. 5 shows a flowchart for realizing the above processing.
[0073]
In step 11, i is initialized to 0, with the number i of images as the number of times of processing.
[0074]
In step 12, the i-th image I (x, y, i) is read from the high-speed image storage unit 2.
[0075]
In step 13, as shown in Expression 1, an average luminance value Ia (i) at the i-th position is calculated.
[0076]
In step 14, 1 is added to i.
[0077]
In step 15, if i <(n × F), the process returns to step 12, and if i> = (n × F), the process proceeds to step 16.
[0078]
In step 16, as shown in Expression 3, the luminance average value having the largest value from the (n × F) values is selected as the reference flicker feature Ir.
[0079]
In step 17, as shown in Expression 4, an average image I ′ (x, y) obtained by averaging luminance values free from flicker is output to the high-speed image output unit 4.
[0080]
<2-3> Third Embodiment A third embodiment of the flicker removing section 3 will be described with reference to FIG.
[0081]
In the first embodiment of the flicker elimination unit 3, since the average of the luminance values is obtained over (n × F) sheets, a blurred image is obtained in a moving area. For this reason, the quality of an image having a drastic change slightly deteriorates.
[0082]
On the other hand, in the second embodiment of the flicker removing unit 3, when the fluctuation of the luminance value due to flicker does not completely apply to the model as shown in Expression 4, the influence of flicker is not completely removed. The effect remains.
[0083]
Therefore, in order to solve this, as shown in FIG. 6, in the third embodiment, the flicker correction unit 3 includes an area selection unit 331, an image change rate calculation unit 332, a flicker correction method selection unit 333, and a flicker correction unit 324. It is composed of
[0084]
The image change rate calculation unit 332 calculates an image change rate indicating how much the image has changed with respect to the entire image or the attention area selected by the area selection unit 331. For example, the following amount is used as the image change rate.
[0085]
(1) The motion vector of each pixel is calculated, and the average value of the magnitude of the vector is used as the image change rate.
[0086]
(2) The absolute value sum of the differential value of the luminance value is calculated in the attention area of each image, and the difference between the images of the absolute value sum is used as the image change rate.
[0087]
(3) The absolute value sum of the second derivative of the luminance value is calculated in the attention area of each image, and the difference between the images of the absolute value sum is used as the image change rate.
[0088]
The flicker correction method selection unit 333 selects the method of the first embodiment if the image change rate is smaller than the threshold, and selects the method of the second embodiment if the image change rate is larger.
[0089]
Then, the flicker correction unit 334 corrects flicker as described above using the method of each selected embodiment.
[0090]
By using the flicker removing unit 3 of the third embodiment, it is possible to remove the flicker by a simple calculation without blurring even a moving area.
[0091]
【The invention's effect】
ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to remove the fluctuation | variation of a pixel value by the flicker of discharge lamps, such as a fluorescent lamp, and to obtain a high quality image from a video image with a high frame rate.
[Brief description of the drawings]
FIG. 1 is a block diagram showing an embodiment of the present invention.
FIG. 2 is a block diagram illustrating an example of an embodiment of an image storage unit.
FIG. 3 is a flowchart relating to a first embodiment of the flicker correction unit.
FIG. 4 is a block diagram relating to a second embodiment of the flicker correction unit.
FIG. 5 is a flowchart relating to a second embodiment of the flicker correction unit.
FIG. 6 is a block diagram relating to a third embodiment of the flicker correction unit.
FIG. 7 is an overall view showing a general environment of the embodiment.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 ... High speed image acquisition part 2 ... High speed image storage part 3 ... Flicker removal part 4 ... High speed image output part 21 ... Write buffer selection part 22 ... Read buffer selection part 321, an area selection unit 322, a flicker feature amount calculation unit 323, a flicker feature amount selection unit 324, a pixel correction unit 331, an area selection unit 332, an image change rate calculation unit 333, ..Flicker correction method selection section 334 ... Flicker correction section

Claims (7)

Image photographing means capable of photographing a plurality of images during a lighting cycle T of the discharge lamp;
Image storage means for storing a pixel value of each pixel of a past image photographed over the lighting period T or more;
Eliminating the effects of flicker by correcting the pixel values so that the flicker feature, which is the average value of the luminance values, is equal between the images, using the past image captured before the held reference time. Flicker removing means for generating an image at the reference time,
Image output means for outputting the image at the reference time from which the flicker has been removed as a moving image in chronological order,
A flicker removing device comprising: The flicker removing means,
Time setting means for setting a time that is an integral multiple of the lighting cycle T;
The time-series average value of the luminance value for each pixel from the image at the reference time taken by the image storage means to the past image taken at the time closest to the time set back by the set time. Flicker feature value calculating means for calculating a flicker feature value,
Image generation means for generating an image at a reference time from which the influence of flicker has been removed, using the calculated flicker feature amount of each pixel as a pixel value,
The flicker removing device according to claim 1, further comprising: The flicker removing means,
For a past image photographed before the reference time held by the image storage unit, an area selection unit that selects a selection area for calculating a flicker feature amount,
A flicker feature value calculation unit that calculates a flicker feature value that is a regional average value of luminance values of pixels in the selected selection region, for the captured past image and the captured reference time image,
Flicker feature value selecting means for selecting a reference flicker feature value serving as a reference for removing the influence of flicker from the calculated flicker feature value;
By correcting the pixel value in the selected area so that the flicker feature of the captured image at the reference time becomes equal to the selected reference flicker feature, the image at the reference time from which the influence of flicker has been removed is obtained. A pixel value correcting unit to be generated;
The flicker removing device according to claim 1, further comprising: The flicker removing means,
Image change rate calculation means for calculating an image change rate which is a change speed of a past image held by the image storage means,
When the image change rate is smaller than the threshold value, an image at a reference time from which the influence of flicker has been removed by the first removing means is generated. When the image change rate is larger than the threshold value, flicker of the flicker is reduced by the second removing means. Removing method selection performing means for selecting to generate an image of the reference time with the effect removed,
The first removing means includes:
Time setting means for setting a time that is an integral multiple of the lighting cycle T;
The time-series average value of the luminance value for each pixel from the image at the reference time taken by the image storage means to the past image taken at the time closest to the time set back by the set time. Flicker feature value calculating means for calculating a flicker feature value,
Image generation means for generating an image at a reference time from which the influence of flicker has been removed, using the calculated flicker feature amount of each pixel as a pixel value,
With
The second removing means includes:
For a past image photographed before the reference time held by the image storage unit, an area selection unit that selects a selection area for calculating a flicker feature amount,
A flicker feature value calculation unit that calculates a flicker feature value that is a regional average value of luminance values of pixels in the selected selection region, for the captured past image and the captured reference time image,
Flicker feature value selecting means for selecting a reference flicker feature value serving as a reference for removing the influence of flicker from the calculated flicker feature value;
By correcting the pixel value in the selected area so that the flicker feature of the captured image at the reference time becomes equal to the selected reference flicker feature, the image at the reference time from which the influence of flicker has been removed is obtained. A pixel value correcting unit to be generated;
The flicker removing device according to claim 1, further comprising: The flicker removing device according to claim 3, wherein the selected area is the entire image or a part of the image. An image capturing step of capturing a plurality of images during a lighting cycle T of the discharge lamp;
An image accumulating step of holding a pixel value of each pixel of a past image photographed over the lighting cycle T or more,
Eliminating the effects of flicker by correcting the pixel values so that the flicker feature, which is the average value of the luminance values, is equal between the images, using the past image captured before the held reference time. A flicker removing step of generating an image at the reference time
An image output step of outputting an image at the reference time from which the flicker has been removed as a moving image in chronological order,
A method of removing flicker, comprising: An image capturing function for capturing a plurality of images during a lighting cycle T of the discharge lamp;
An image accumulation function for holding a pixel value of each pixel of a past image photographed over the lighting cycle T or more,
Eliminating the effects of flicker by correcting the pixel values so that the flicker feature, which is the average value of the luminance values, is equal between the images, using the past image captured before the held reference time. A flicker removal function that generates an image of the reference time
An image output function of outputting an image at the reference time from which the flicker has been removed as a moving image in chronological order,
A flicker elimination method, wherein the program is realized by a computer.

Images