JP4071650B2 - Image target detection method and apparatus, and image target detection program - Google Patents

Description

【数２】

【外１】

また、しきい値・積分係数計算部２３は、第ｈフレームの積分画像ＩＮ_ｈ（１≦ｈ≦ｎ）の積分係数ｋを、１フレーム前である第ｈ−１フレームの積分画像ＩＮ_ｈの平均値ＡＶに基づいて計算する積分係数計算部４１と、各フレームの積分画像の平均値ＡＶおよび分散σ^２に基づいて、その各フレームの積分画像のしきい値ｔを計算するしきい値計算部４２とを備えている。
【００３６】
積分係数計算部４１は、図３に示すように、平均値・分散計算部４０により求められた平均値ＡＶを外部から入力できる基準値ＳＴ｛例えば、画像全体の輝度（明るさ）調整用の基準値｝から減算する減算部５０と、この減算部５０により得られた増減量を、外部から入力できる可変係数ゲインに基づいて増幅する増幅部５１と、少なくとも時系列画像の１フレーム分の記憶領域を有する画像メモリ５２と、増幅部５１により増幅された第ｈ−１フレーム（１≦ｈ≦ｎ）に対応する係数増減量を１フレーム毎に画像メモリ５２に蓄積する蓄積処理部５５と、この蓄積処理部５５により蓄積された係数増減量を、その係数増減量の次フレームの係数増減量に対して画素毎に積算し、この積算された係数増減量を、次フレーム（第ｈフレーム（１≦ｈ≦ｎ））の係数増減量として出力する積算部５６とを備えている。
【００３７】
次に、本実施形態の画像目標検出装置１の全体動作について説明する。
【００３８】
画像目標検出装置１により、例えばヘリコプターのロータ部分等、回転等の動作により画像上において明滅する部分を目標として、その目標を検出する際、画像目標検出装置１には、例えば、可視光センサや放射線センサ等の画像センサにより時系列的に収集された目標の画像を含む複数（例えば、ｎ枚）の時系列画像Ｉ_１〜Ｉ_ｎが１フレーム毎に順次時間フィルタ２における差分処理部２０および平滑処理部５に入力される。
【００３９】
時間フィルタ２における差分処理部２０に入力された時系列画像Ｉ_１〜Ｉ_ｎ＋１は、画像蓄積処理部３２により１フレーム毎に画像メモリ３１に蓄積される。
【００４０】
今、画像メモリ３１に時系列画像Ｉ_ｈ−１（１≦ｈ≦ｎ）が蓄積された状態において、次フレームの時系列画像（現在の入力画像）Ｉ_ｈが入力されると、差分画像生成出力部３３により、画像メモリ３１に蓄積された時系列画像Ｉ_ｈ−１から次フレームの時系列画像（現在の入力画像）Ｉ_ｈが画素毎に差分（フレーム差分）され、差分画像（出力画像）Ｄ_ｈが生成出力される（図４（ａ）、図４（ｂ）参照）。
【００４１】
このようにして、差分処理部２０に入力された時系列画像Ｉ_１〜Ｉ_ｎは、時間的に隣接する画素間の差分を表す差分画像Ｄ_１〜Ｄ_ｎとして順次絶対値処理部２１に出力される。
【００４２】
絶対値処理部２１では、差分画像生成出力部３３から順次出力される各差分画像Ｄ_１〜Ｄ_ｎにより各差分絶対値画像ＡＢ_１〜ＡＢ_ｎが生成され、生成された各差分絶対値画像ＡＢ_１〜ＡＢ_ｎは、順次積分処理部２２へ出力される。
【００４３】
積分処理部２２に順次入力された差分絶対値画像ＡＢ_１〜ＡＢ_ｎは、しきい値・積分係数計算部４１により計算された積分係数ｋに基づいて、積分処理部２２により画素毎に時間的に積分される。
【００４４】
例えば、差分絶対値画像ＡＢ_ｈが入力されたとき、画像メモリ３５には、図５（ａ）および（ｂ）に示すように、差分絶対値画像ＡＢ_ｈの１フレーム前（第ｈ−１フレーム）までの全ての入力画像ＡＢ_１〜ＡＢ_ｈ−１の積分結果を表す画像｛積分画像ＩＮ_ｈ−１（＝ＡＢ_ｈ−１＋ｋＡＢ_ｈ−２＋ｋ^２ＡＢ_ｈ−３＋・・・＋ｋ^ｈ−２ＡＢ_１）｝が蓄積されている。
【００４５】
一方、しきい値・積分係数計算部２３の平均値・分散計算部４０では、積分処理部２２から出力される各積分画像ＩＮ_１〜ＩＮ_ｈ−１から、上式（１）に従って各積分画像ＩＮ_１〜ＩＮ_ｈ−１の平均値ＡＶが計算され、計算された平均値ＡＶに基づいて各積分画像ＩＮ_１〜ＩＮ_ｈ−１の分散σ^２が上式（２）に基づいて計算されている。
【００４６】
このとき、しきい値・積分係数計算部２３の積分係数計算部４１においては、積分処理結果（各積分画像ＩＮ_１〜ＩＮ_ｈ−１）の平均値ＡＶと予め設定された基準値ＳＴとの差が一定、すなわち、積分処理結果（各積分画像ＩＮ_１〜ＩＮ_ｈ−１）の平均値ＡＶが予め設定された基準値ＳＴに保持されるように、積分処理部２２における積分係数ｋが増幅器５１の係数ゲインにより調整されている。
【００４７】
例えば、積分処理結果の平均値ＡＶが予め設定された基準値ＳＴより大きい場合には、積分係数ｋが小さくなるように係数ゲインが調整され、また、積分処理結果の平均値ＡＶが基準値ＳＴより小さい場合には、積分係数ｋが大きくなるように係数ゲインが調整される。
【００４８】
このようにして、第ｈ−１フレームまでの積分画像ＩＮ_ｈ−１に基づいて次フレーム（第ｈフレーム）の積分画像に対応する積分係数ｋが計算されており、第ｈ−１フレームまでの積分結果を表す画像（ＡＢ_ｈ−１＋ｋＡＢ_ｈ−２＋ｋ^２ＡＢ_ｈ−３＋・・・＋ｋ^ｈ−２ＡＢ_１）に対して、乗算部３７により、計算された積分係数ｋが乗算される。
【００４９】
そして、積分係数ｋの乗算処理により得られた画像ｋ（ＡＢ_ｈ＋ｋＡＢ_ｈ−１＋ｋ^２ＡＢ_ｈ−２＋・・・＋ｋ^ｈ−１ＡＢ_１）に対して、第ｈフレームの差分絶対値画像ＡＢ_ｈが積算部３８により画素毎に積算される。
【００５０】
この結果、第１〜第ｈの差分絶対値画像ＡＢ_１〜ＡＢ_ｈの積分結果を表す画像（ＡＢ_ｈ＋ｋＡＢ_ｈ−１＋ｋ^２ＡＢ_ｈ−２＋・・・＋ｋ^ｈ−１ＡＢ_１）が生成される。
【００５１】
生成された第１〜第ｈの差分絶対値画像ＡＢ_１〜ＡＢ_ｈの積分結果を表す画像（ＡＢ_ｈ＋ｋＡＢ_ｈ−１＋ｋ^２ＡＢ_ｈ−２＋・・・＋ｋ^ｈ−１ＡＢ_１）は、積分画像ＩＮ_ｈとして、２値化処理部３に出力される（図５（ｂ）参照）。
【００５２】
このようにして、第ｎフレームの差分絶対値画像ＡＢ_ｎが入力されると、積分処理部２２の積分処理により、第１〜第ｎの差分絶対値画像ＡＢ_１〜ＡＢ_ｎの積分結果を表す画像（ＡＢ_ｎ＋ｋＡＢ_ｎ−１＋ｋ^２ＡＢ_ｎ−２＋・・・＋ｋ^ｎ−１ＡＢ_１）が生成される。
【００５３】
生成された積分結果を表す画像（ＡＢ_ｎ＋ｋＡＢ_ｎ−１＋ｋ^２ＡＢ_ｎ−２＋・・・＋ｋ^ｎ−１ＡＢ_１）は、第ｎフレームまでの積分画像ＩＮ_ｎ、すなわち、第１〜第ｎの差分絶対値画像ＡＢ_１〜ＡＢ_ｎ分が積算（積分）された積分画像ＩＮ_ｎとして、２値化処理部３に出力される。
【００５４】
一方、しきい値・積分係数計算部２３のしきい値計算部４２では、図３に示すように、平均値・分散計算部４０から出力される各積分画像ＩＮ_１〜ＩＮ_ｎの平均値ＡＶおよび分散σ^２に基づいて、積分画像毎にしきい値ｔが計算されており、計算されたしきい値ｔは、２値化処理部３に送信される。
【００５５】
２値化処理部３では、図６（ａ）および（ｂ）に示すように、例えば、積分画像ＩＮ_ｈが入力されると、この積分画像ＩＮ_ｈを構成する画素の画素値がしきい値ｔ以上の場合には、その画素が目的候補領域であると判断され、その画素の画素値が１に設定される。
【００５６】
この結果、図６（ｂ）に示すように、目的候補領域Ｒ_ｈおよび背景領域Ｒ_ｈａを含む２値化画像ＢＩ_ｈが求められる。
【００５７】
このように、各積分画像ＩＮ_１〜ＩＮ_ｎが２値化処理部３に順次入力されると、２値化処理部３により、目的候補領域Ｒ_１および背景領域Ｒ_１ａを含む２値化画像ＢＩ_１〜目的候補領域Ｒ_ｎおよび背景領域Ｒ_ｎａを含む２値化画像ＢＩ_ｎが順次求められる。
【００５８】
このようにして求められた目標候補領域Ｒ_ｈおよび背景領域Ｒ_ｈａを含む２値化画像ＢＩ_ｈの従来技術との差異について、図７を用いて説明する。
【００５９】
本実施形態においては、上述したように、例えばヘリコプターのロータ部分等、回転等の動作により画像上において明滅する部分を目標としているため、その入力画像Ｉ_ｈの輝度分布は、図７の符号Ｔ１に示すように、周期的に変化している。
【００６０】
この点、従来のように、通常の空間フィルタを用いた２値化処理（しきい値に基づく目標候補領域抽出処理；図７の符号Ｔ２参照）では、図７の符号Ｔ３に示すように、画像上における消滅部分が背景領域と誤認識されることになり、正確な目標候補領域を抽出することができないことが分かる。
【００６１】
これに対して、本実施形態の画像目的検出装置１における時間フィルタ２の処理においては、図７の符号Ｔ１０に示すように、入力画像Ｉ_１〜ＩＩ_ｈに基づいて差分処理部２０により得られた差分画像Ｄ_１〜Ｄ_ｈは、絶対値処理部２１を介して、その−側の信号レベルが＋側に反転されて差分絶対値画像ＡＢ_１〜ＡＢ_ｈが生成される。
【００６２】
そして、生成された差分絶対値画像ＡＢ_１〜ＡＢ_ｈは、積分処理部２２により時間的に積分処理され、積分画像ＩＮ_ｈが生成される。
【００６３】
このとき、積分画像ＩＮ_ｈにおいては、差分絶対値画像ＡＢ_１〜ＡＢ_ｈの各画素の画素値が対応する積分係数ｋにより調整されながら積算（累積）されていくため、例えば、明滅部分、すなわち、周期的に画素値（輝度）が小さい画素領域が存在しても、その明滅部分に対応する画素領域は、図７の符号Ｔ１０に示すように、積分処理部２２による積分処理により、画像上に存在する時（“明”の時）の大きい画素値が累積されていくため、その累積画素値は次第に大きくなっていく。
【００６４】
この結果、図７の符号Ｔ１０およびＴ１１に示すように、所定のしきい値を用いて２値化処理を行った場合においても、目標の明滅部分を目標候補領域として抽出することができる。
【００６５】
一方、画像センサから順次入力された各時系列画像Ｉ_１〜Ｉ_ｎは、平滑処理部５により平滑化され、この平滑化された各時系列画像ＩＳ_１〜ＩＳ_ｎに基づいて、平滑処理部５における平均値・分散計算部４０と同様の処理により、各平滑化時系列画像ＩＳ_１〜ＩＳ_ｎの平均値および分散が計算される。
【００６６】
計算された各平滑化時系列画像ＩＳ_１〜ＩＳ_ｎの平均値および分散に基づいて、平滑処理部５により、しきい値が計算されており、計算されたしきい値に基づいて、２値化処理部１０により、各平滑化時系列画像ＩＳ_１〜ＩＳ_ｎの２値化画像ＢＳ_１〜ＢＳ_ｎが順次生成される。
【００６７】
他方、平滑処理部５により平滑化された各時系列画像ＩＳ_１〜ＩＳ_ｎは、微分処理部６の微分処理によりエッジ（輪郭）強調され、この微分処理された各時系列画像ＩＤ_１〜ＩＤ_ｎに基づいて、微分処理部６における平均値・分散計算部４０と同様の処理により、各時系列画像ＩＤ_１〜ＩＤ_ｎの平均値および分散が計算される。
【００６８】
計算された各時系列画像ＩＤ_１〜ＩＤ_ｎの平均値および分散に基づいて、微分処理部６により、しきい値が計算されており、計算されたしきい値に基づいて、２値化処理部１１により、各時系列画像ＩＤ_１〜ＩＤ_ｎの２値化画像ＤＳ_１〜ＤＳ_ｎが順次生成される。
【００６９】
目標領域判定部１２は、２値化処理部３により生成された各２値化画像ＢＩ_１〜２値化画像ＢＩ_ｎを１フレームずつ順次取り込む。同様に、目標領域判定部１２は、２値化処理部１０により生成された各２値化画像ＢＳ_１〜ＢＳ_ｎを１フレームずつ順次取り込み、さらに、２値化処理部１１により生成された各２値化画像ＤＳ_１〜ＤＳ_ｎを１フレームずつ順次取り込む。
【００７０】
次いで、目標領域判定部１２は、最初に取り込んだ２値化画像ＢＩ_１、２値化画像ＢＳ_１および２値化画像ＤＳ_１を、それぞれ複数画素から成るセグメントに分割し、各セグメントにおける画素値１が互いに連結されている領域に同一のラベルを付ける。
【００７１】
続いて、目標領域判定部１２は、２値化画像ＢＩ_１により得られた目標候補領域となる複数のラベリングされた領域（ラベリング領域）と２値化画像ＢＳ_１および２値化画像ＤＳ_１によりそれぞれ得られたラベリング領域とをそれぞれ比較して、目標候補となる各ラベリング領域の特徴量を表すパラメータ（特徴量パラメータ）Ｐとして、各ラベリング領域の平滑化最大値、微分最大値、差分最大値、移動方向および移動量をそれぞれ抽出する。
【００７２】
続いて、目標領域判定部１２は、得られた各ラベリング領域の特徴量（特徴量パラメータＰ）に基づいて、各ラベリング領域の評価値を計算し、以下、残りの全ての２値化画像ＢＩ_２〜２値化画像ＢＩ_ｎ、２値化画像ＢＳ_２〜ＢＳ_ｎ、２値化画像ＤＳ_２〜ＤＳ_ｎを用いた全ての目標候補領域に対応する全てのラベリング領域の評価値計算が終了するまで、この処理を繰り返し実行する。
【００７３】
そして、全ての目標候補領域に対応する全てのラベリング領域の評価値計算が終了したとき、目標領域判定部１２は、全てのラベリング領域の中で最も評価値の高いラベリング領域を目標として検出する。
【００７４】
この結果、ヘリコプターのロータ部分等、回転等の動作により画像上において明滅する目標を検出することができる。
【００７５】
以上詳述したように、本実施形態によれば、検出目標が例えばヘリコプターのロータ部分等、回転等の動作により画像上において明滅する部分であっても、その明滅部分を含む複数の時系列画像の差分画像を時間的に積分することにより、その明滅部分の周波数成分を目標候補領域として抽出することができる。
【００７６】
したがって、この目標候補領域を用いて目標判定処理を行うことにより、画像上において明滅する部分を目標とした場合であっても、その目標を正確に検出することができる。
【００７７】
特に、本実施形態では、入射光学系として特別なハードウェア構成要素およびその制御系を用いることなく、通常の画像センサにより収集された時系列画像に基づいて明滅部分の周波数成分を目標候補領域として抽出することができる。このため、目標検出装置全体をコンパクトに維持しながら、明滅部分等の目標を容易に検出することができる。
【００７８】
また、本実施形態によれば、積分処理結果の平均値ＡＶが予め設定された基準値ＳＴより大きい場合には、積分係数ｋが小さくなるように係数ゲインを調整し、また、積分処理結果の平均値ＡＶが基準値ＳＴより小さい場合には、積分係数ｋが大きくなるように係数ゲインを調整することにより、積分処理結果（各積分画像ＩＮ_１〜ＩＮ_ｈ−１）の平均値ＡＶを予め設定された基準値ＳＴに保持している。
【００７９】
この制御により、周波数変調する面積（画素領域）が多いとき（例えば、画像の移動等による）であっても、輝度（画素値）の平均値の上昇を抑制することができ、その輝度の平均値の上昇に起因した真の目標以外の目標候補領域を多く抽出する恐れを軽減することができる。この結果、目標候補抽出精度および目標検出精度をそれぞれ向上させることができる。
【００８０】
また、周波数変調する面積（画素領域）の周波数成分が微少であっても、輝度（画素値）の平均値を基準値に近付くまで上昇させることができ、その輝度の平均値の上昇により、微少な周波数成分であっても、目標候補領域として抽出することができる。この結果、目標候補抽出精度および目標検出精度をそれぞれ向上させることができる。
【００８１】
なお、本実施形態においては、目標が例えばヘリコプターのロータ部分等、回転等の動作により画像上において明滅する部分である場合について説明したが、本発明はこの構成に限定されるものではなく、上記明滅部分を目標の一部に含む場合の全体の目標を検出する場合、あるいは明滅部分を含まない目標に対しても、本実施形態の画像目標検出装置を適用することができる。
【００８２】
また、本実施形態においては、時間フィルタ２を構成する差分処理部２０、絶対値処理部２１、積分処理部２２およびしきい値・積分係数計算部２３と２値化処理部３とを、それぞれワイヤードロジック回路により構成したが、本発明はこれに限定されるものではなく、差分処理部２０、絶対値処理部２１、積分処理部２２、しきい値・積分係数計算部２３および２値化処理部３のそれぞれの処理を、画像目標検出用プログラムに従った少なくとも１台のコンピュータの機能により実現してもよい。
【００８３】
さらに、本実施形態においては、画像目標検出装置１を構成する全ての構成要素（図１に示す全てのブロック構成要素）の処理を、プログラムに従った少なくとも１台のコンピュータの機能により実現してもよい。
【００８４】
また、本実施形態において、差分処理部は、入力された時系列画像Ｉ_１〜Ｉ_ｎに基づいて、時間的に隣接する画素間の差分を表す差分画像Ｄ_１〜Ｄ_ｎを生成している。このとき、差分処理部は、最初の時系列画像Ｉ_１が入力された際においては、画像メモリに前フレームのフレーム画像が蓄積されていないため、その差分画像は時系列画像Ｉ_１そのものとなる。
【００８５】
この点において、本実施形態の変形例として、次の時系列画像Ｉ_２が入力されるまで、差分画像出力を行わないようにすることも可能であり、積分画像の平均値が基準値に収束するまでのフレーム画像数を減らすことができる。
【００８６】
【発明の効果】
以上述べたように、請求項１、６および７記載の本発明によれば、差分絶対値画像生成手段から出力された複数の差分絶対値画像を所定の積分係数に基づいて画素毎に時間的に積分しているため、例えば、目標が明滅部分であった場合でも、画像上に存在する際の画素値を積算することができる。
請求項１、６および７記載の本発明によれば、各フレームの積分画像の各画素の画素値および平均値に基づいて、各フレームの積分画像の次フレームの積分画像に対する所定の積分係数を求めているため、例えば所定の基準値に対する平均値の大小に応じて積分係数を調整することにより、平均値が大きい場合に起因した目標候補領域の誤判定を軽減することができ、さらに、平均値が微少な場合に起因した目標候補領域の検出漏れを防止することができる。
【００８７】
この結果、目標が明滅部分であっても、その明滅部分の周波数成分を目標候補領域として抽出することができる。
【００８８】
請求項２記載の本発明によれば、抽出された目標候補領域の画素値を１および背景領域の画素値を０とする２値化画像を生成することにより、生成された２値化画像を用いて目標領域を正確に判定することが可能になる。
【００９０】
請求項３記載の本発明によれば、蓄積された各フレームの差分絶対値画像の各画素に前記所定の積分係数を乗算し、当該所定の積分係数が乗算された各フレームの差分絶対値画像を、当該各フレームの差分絶対値画像の次フレームの差分絶対値画像に対して画素毎に積算し、この積算結果を、次フレームの差分絶対値画像として蓄積するとともに、次フレームの差分絶対値画像に対応する次フレームの積分画像として出力しているため、フレーム毎に正確に積分画像を生成出力することができる。
【００９１】
請求項４および５記載の本発明によれば、目標候補領域となる２値化画像から、その目標候補領域に対応するラベル付けを行い、このラベル付けられたそれぞれの目標候補領域の中から、それぞれの領域における、例えば、平滑化最大値、微分最大値、差分最大値、移動方向および移動量等の特徴量を表すパラメータを抽出し、抽出されたそれぞれの領域の特徴量パラメータに基づいて、対応するそれぞれの目標候補領域を評価し、その評価値の最も高い領域を目標領域として検出している。このため、抽出された特徴量パラメータに基づいて正確な目標領域検出処理を行うことができる。
【図面の簡単な説明】
【図１】本発明の実施の形態に係る画像目標検出装置の概略構成を示す図である。
【図２】本実施形態の時間フィルタを構成する差分処理部、絶対値処理部、積分処理部およびしきい値・積分係数計算部の概略構成を示すブロック図である。
【図３】図２に示す積分係数計算部の概略構成を示すブロック図である。
【図４】（ａ）は、図２に示す差分処理部のブロック図であり、（ｂ）は、図４（ａ）に示す差分処理部の差分処理を、その差分処理部に対して入出力されるフレーム画像の変化により説明するための概念図である。
【図５】（ａ）は、図２に示す積分処理部のブロック図であり、（ｂ）は、図５（ａ）に示す積分処理部の積分処理を、その積分処理部に対して入出力されるフレーム画像の変化により説明するための概念図である。
【図６】（ａ）は、図２に示す時間フィルタ側の２値化処理部のブロック図であり、（ｂ）は、図６（ａ）に示す２値化処理部の２値化処理を、その２値化処理部に対して入出力されるフレーム画像の変化により説明するための概念図である。
【図７】本実施形態の画像目的検出装置により求められた目標候補領域を含む２値化画像と従来技術による目標候補領域との差異について説明するための図である。
【図８】従来の目標検出処理部の概略構成を示すブロック図である。
【符号の説明】
１ 画像目標検出装置
２ 時間フィルタ
５ 平滑処理部
６ 微分処理部
３、１０、１１ ２値化処理部
１２ 目標領域判定部
２０ 差分処理部
２１ 絶対値処理部
２２ 積分処理部
２３ しきい値・積分係数計算部[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image target detection method and apparatus for detecting a target candidate area based on a time-series image including a target image obtained by an image sensor such as a visible light sensor or an infrared radiation sensor. In particular, the present invention relates to an image target detection method and apparatus that can detect a target candidate area that blinks on an image, and an image target detection program.
[0002]
[Prior art]
Conventionally, for example, a target detection process for detecting a target based on a target image such as a moving object obtained by an image sensor is shown in FIG.
[0003]
According to the target detection processing unit 50 shown in FIG. 8, each pixel of the image including the target acquired by the image sensor 51 is averaged with its surrounding pixels by the preprocessing unit 52, and noise is removed. A (smoothed) image is obtained.
[0004]
The smoothed image is subjected to a differentiation process by the feature extraction unit 53, and the edges of a plurality of target candidate portions (regions: target candidate regions) are emphasized. The edge-enhanced image is binarized based on a predetermined threshold value through a spatial filter in the feature extraction unit 53, and a target candidate region is extracted.
[0005]
Then, for example, features such as area, barycentric position and aspect ratio of the extracted target candidate image are evaluated in a composite manner by the target determination unit 54, and a target candidate area corresponding to the target position is determined from the target candidate areas. The
[0006]
Further, as an apparatus for detecting a target such as a moving object, an apparatus using an incident optical unit that captures an infrared image in all directions of 360 ° is known (for example, see Patent Document 1).
[0007]
[Patent Document 1]
JP 2000-346923 A
[0008]
[Problems to be solved by the invention]
For example, when detecting a rotating member such as a rotor portion of a helicopter as a target, the rotating member blinks on the image due to the rotation.
[0009]
When the rotor portion that periodically flickers is detected using the target detection process shown in FIG. 8, the feature extraction unit 53 in the target detection process performs the feature extraction process without considering any frequency components. Therefore, it is difficult to extract the frequency component of the rotor portion, and the target cannot be accurately detected.
[0010]
In this regard, in Patent Document 1, the rotor portion that rotates periodically is detected by rotating the incident optical unit and the derotation prism.
[0011]
However, in the configuration of Patent Document 1, an incident optical unit, a derotation prism, and a member that controls the rotation of both are required as the incident optical system, leading to an increase in scale and complexity of the entire target detection apparatus. .
[0012]
The present invention has been made in view of the above. For example, even when the target changes in frequency, the entire apparatus is not enlarged and complicated, and the image including the target is used. An object of the present invention is to provide an image target detection method and apparatus capable of accurately detecting a target, and an image target detection program.
[0013]
[Means for Solving the Problems]
In order to solve the above-mentioned problem, the present invention is an image target detection device that detects a target candidate area based on a plurality of time-series images including a target image in units of frames. Difference image generation means for calculating a difference for each pixel between temporally adjacent images and generating a plurality of difference images; and calculating a plurality of difference absolute values for each pixel by calculating an absolute value of each of the generated plurality of difference images A difference absolute value image generation unit that generates an image and outputs the generated plurality of difference absolute value images, and a plurality of difference absolute value images output from the difference absolute value image generation unit based on a predetermined integration coefficient The gist of the present invention is that it has integrated image generation means for integrating each time to generate an integrated image.
[0014]
Alternatively, in order to solve the above problem, the present invention performs binarization processing on the generated integral image based on a predetermined threshold value, and sets the pixel value of the target candidate region to 1 and the pixel of the background region Binarization means for generating a binarized image with a value of 0 is further provided.
[0015]
Alternatively, in order to solve the above-described problem, the integrated image generation unit generates and outputs a plurality of frames of integrated images corresponding to the plurality of frames of absolute difference images as the integrated images. The integrated image generating means calculates an average value of pixel values of each pixel of the output integrated image of each frame, and each of the pixel values based on the pixel value of each pixel and the calculated average value. A calculation means for calculating a variance of each pixel in the integral image of the frame; and a means for obtaining a predetermined integration coefficient for the integral image of the next frame of the integral image of each frame based on the calculated average value, The binarizing means further includes means for obtaining the predetermined threshold value based on the calculated average value and variance.
[0016]
Alternatively, in order to solve the above-described problem, the present invention provides that the integral image generation unit accumulates a difference absolute value image of each frame, and a difference absolute value of each frame accumulated by the accumulation processing unit. Each pixel of the image is multiplied by the predetermined integration coefficient, and the difference absolute value image of each frame multiplied by the predetermined integration coefficient is compared with the difference absolute value image of the next frame of the difference absolute value image of the frame. An integration processing unit that integrates each pixel and sends the integration result as a difference absolute value image of the next frame to the accumulation processing unit and outputs it as an integration image of the next frame corresponding to the difference absolute value image of the next frame; The main point is that
[0017]
Alternatively, in order to solve the above-described problem, the present invention divides the binarized image serving as the target candidate region into segments composed of a plurality of pixels, and the same pixel regions 1 are connected to each other. A means for attaching a label, and a parameter representing a feature value in each labeled region is extracted, and each extracted region is evaluated based on the feature parameter, and the region having the highest evaluation value is determined. And a means for detecting the target area.
[0018]
Alternatively, in order to solve the above problems, the present invention is characterized in that the parameter representing the feature amount includes a smoothing maximum value, a differential maximum value, a difference maximum value, a movement direction, and a movement amount in each of the regions. To do.
[0019]
Alternatively, in order to solve the above-described problem, the present invention provides a computer-executable image target detection program for detecting a target candidate region based on a plurality of time-series images in frame units including a target image. A difference image generating unit configured to generate a plurality of difference images by obtaining a difference for each pixel between temporally adjacent images in the plurality of time-series images; and each of the generated plurality of difference images. A difference absolute value image generating means for generating a plurality of difference absolute value images by obtaining an absolute value of each pixel and generating a plurality of difference absolute value images, and a plurality of outputs from the difference absolute value image generating means The difference absolute value image is integrated with respect to time for each pixel based on a predetermined integration coefficient, and the integrated image generating means for generating an integrated image is used as a gist.
[0020]
Alternatively, in order to solve the above problem, the present invention is an image target detection method for detecting a target candidate area based on a plurality of time-series images in frame units including a target image.
Calculating a difference for each pixel between temporally adjacent images in the plurality of time-series images to generate a plurality of difference images; and determining a plurality of absolute values of the generated plurality of difference images for each pixel. Generating a plurality of difference absolute value images, and outputting a plurality of difference absolute value images generated by the difference absolute value image generation step for each pixel based on a predetermined integration coefficient. And a step of generating an integrated image by temporal integration.
[0021]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an image target detection method and apparatus, and an image target detection program according to embodiments of the present invention will be described with reference to the drawings. In the present embodiment, a case will be described in which, for example, a target such as a rotor portion of a helicopter that blinks on an image by an operation such as rotation is detected as a target.
[0022]
FIG. 1 is a diagram showing a schematic configuration of an image target detection apparatus 1 according to an embodiment of the present invention.
[0023]
As shown in FIG. 1, the image target detection device 1 includes a plurality of {(target detection) images including target images (frame images) captured based on light rays (beams) such as visible light and infrared light reflected from the target. When time series images of the required number of calculation frames n (n is an integer of 2 or more)} are sequentially input from the image sensor to the apparatus 1, a target is determined based on the input time series images. It is a device to detect.
[0024]
That is, the image target device 1 detects a target frequency modulation component periodically flickering included in the time-series image based on a plurality of input time-series images, and the time filter 2 detects the target frequency-modulated component. The obtained frequency modulation component is binarized using a predetermined threshold value t, the target candidate region (target candidate region) has a pixel value (luminance value) of 1, and the other background region pixels A binarization processing unit 3 is provided to extract a binarized image having a value of 0.
[0025]
In addition, the image target detection device 1 includes a smoothing unit 5 for removing noise by replacing the central pixel with an average value of each predetermined region for each predetermined pixel region of each input time-series image. The edge of each time-series image is emphasized by differentiating each pixel value of each time-series image smoothed by the smoothing processing unit 5 using a difference with a pixel value in the vicinity of each pixel value. And a differential processing unit 6.
[0026]
The image target detection apparatus 1 also binarizes each time-series image smoothed by the smoothing processing unit 5 using a predetermined threshold value to generate a binarized image. Unit 10 and a binarization processing unit 11 for binarizing the differential image edge-enhanced by the differential processing unit 6 using a predetermined threshold value to generate a binarized image. .
[0027]
Further, the image target detection apparatus 1 converts the binarized image including the target candidate area extracted by the binarization processing unit 3 into the binarized image obtained by the binarization processing unit 10 and the binarization processing unit. 11 is provided with a target area determination unit 12 that determines a target area serving as a detection target from the target candidate areas by comparing with the binarized image obtained in step S11.
[0028]
As shown in FIG. 1, the time filter 2 includes a difference processing unit 20 for obtaining a difference for each pixel between temporally adjacent images in a plurality of input time-series images and generating a plurality of difference images. The absolute value processing unit 21 that calculates the absolute value of each of the plurality of difference images generated by the difference processing unit 20 for each pixel, generates a plurality of difference absolute value images, and outputs the generated plurality of difference absolute value images. And.
[0029]
The time filter 2 includes an integration processing unit 22 that temporally integrates a plurality of difference absolute value images output from the absolute value processing unit 21 for each pixel based on a predetermined integration coefficient k to generate an integrated image. The predetermined integral k and the predetermined threshold value t are calculated based on the integral image generated by the integral processing unit 22, the calculated integral coefficient k is fed back to the integral processing unit 22, and the calculated threshold is calculated. A threshold / integral coefficient calculation unit 23 is provided for sending the value t to the binarization processing unit 3.
[0030]
FIG. 2 is a block diagram showing a schematic configuration of the difference processing unit 20, the absolute value processing unit 21, the integration processing unit 22, and the threshold / integral coefficient calculation unit 23 that constitute the time filter 2 of the present embodiment.
[0031]
As illustrated in FIG. 2, the difference processing unit 20 of the time filter 2 is configured by, for example, a wired logic circuit dedicated to difference processing, and sequentially includes an image memory 31 having a storage area for at least one frame of a time-series image, An image storage processing unit 32 that sequentially stores the input time-series images of a plurality of frames in the image memory 31 frame by frame, and the time-series image I stored in the image memory 31 _h-1 (1 ≦ h (integer) ≦ n; where I ₀ Represents a state in which no image is stored in the image memory 31). _h To the time-series image I stored in the image memory 31. _h-1 That is, the time-series image I one frame before _h-1 The difference image D _h And a differential image generation / output unit 33 that generates and outputs the difference image.
[0032]
Further, the absolute value processing unit 21 is configured by, for example, a wired logic circuit dedicated to absolute value processing, and obtains an absolute value of each pixel constituting each difference image sequentially output from the difference image generation / output unit 33. Each difference absolute value image is generated using each absolute value as the pixel value of each pixel, and the generated difference absolute value images are sequentially output.
[0033]
Further, the integration processing unit 22 is configured by, for example, a wired logic circuit dedicated to integration processing, and includes an image memory 35 having a storage area for at least one frame of a time-series image, and one frame of a difference absolute value image of each frame. An accumulation processing unit 36 for accumulating images in the image memory 35; a multiplication unit 37 for multiplying each pixel of the difference absolute value image of each frame accumulated in the image memory 35 by the accumulation processing unit 36 by a predetermined integration coefficient k; The difference absolute value image of each frame multiplied by the predetermined integration coefficient k by the multiplication unit 37 is integrated for each pixel with respect to the difference absolute value image of the next frame of the difference absolute value image of each frame, and this The integration result is sent to the delay processing unit 36 as a difference absolute value image of the next frame, and an integrated image of the next frame corresponding to the difference absolute value image of the next frame And a integrating section 38 for outputting as.
[0034]
The threshold / integral coefficient calculation unit 23 includes, for example, a wired logic circuit dedicated to threshold and integration coefficient calculation processing. As illustrated in FIG. 2, each integral image output from the integration processing unit 22 ( When the number of pixels is collectively N), the average value AV of each integrated image IN is calculated based on the following formula (1), and each integrated image IN is calculated based on the calculated average value AV. Variance of σ ² Is calculated based on the following formula (2).
[0035]
[Expression 1]

[Expression 2]

[Outside 1]

Further, the threshold / integral coefficient calculation unit 23 calculates the integral image IN of the h-th frame. _h The integral coefficient k of (1 ≦ h ≦ n) is set to the integral image IN of the (h−1) th frame that is one frame before. _h An integration coefficient calculation unit 41 that calculates based on the average value AV of the image, and the average value AV and variance σ of the integral image of each frame ² And a threshold value calculation unit 42 for calculating the threshold value t of the integral image of each frame.
[0036]
As shown in FIG. 3, the integration coefficient calculation unit 41 has a reference value ST {for example, for adjusting the brightness (brightness) of the entire image, which can input the average value AV obtained by the average value / dispersion calculation unit 40 from the outside. Subtraction unit 50 that subtracts from the reference value}, an amplification unit 51 that amplifies the increase / decrease amount obtained by this subtraction unit 50 based on a variable coefficient gain that can be input from the outside, and storage of at least one frame of a time-series image An image memory 52 having a region, an accumulation processing unit 55 for accumulating the coefficient increase / decrease amount corresponding to the h−1th frame (1 ≦ h ≦ n) amplified by the amplification unit 51 in the image memory 52 for each frame, The coefficient increase / decrease amount accumulated by the accumulation processing unit 55 is integrated for each pixel with respect to the coefficient increase / decrease amount of the next frame of the coefficient increase / decrease amount, and the accumulated coefficient increase / decrease amount is added to the next frame (hth frame ( 1 And a integration unit 56 for outputting a coefficient decrease amount of h ≦ n)).
[0037]
Next, the overall operation of the image target detection apparatus 1 of the present embodiment will be described.
[0038]
When the target is detected by the image target detection device 1, for example, a helicopter rotor portion or the like that blinks on the image by an operation such as rotation, the image target detection device 1 includes, for example, a visible light sensor, Multiple (for example, n) time-series images I including target images collected in time series by an image sensor such as a radiation sensor ₁ ~ I _n Are sequentially input to the difference processing unit 20 and the smoothing processing unit 5 in the time filter 2 for each frame.
[0039]
Time-series image I input to the difference processing unit 20 in the time filter 2 ₁ ~ I _{n + 1} Is stored in the image memory 31 frame by frame by the image storage processing unit 32.
[0040]
Now, the time series image I is stored in the image memory 31. _h-1 In a state where (1 ≦ h ≦ n) is accumulated, a time-series image (current input image) I of the next frame _h Is input, the time-series image I stored in the image memory 31 by the difference image generation / output unit 33 is input. _h-1 To the next frame time-series image (current input image) I _h Is different for each pixel (frame difference), and a difference image (output image) D _h Is generated and output (see FIGS. 4A and 4B).
[0041]
In this way, the time-series image I input to the difference processing unit 20 ₁ ~ I _n Is a difference image D representing a difference between temporally adjacent pixels. ₁ ~ D _n Are sequentially output to the absolute value processing unit 21.
[0042]
In the absolute value processing unit 21, each difference image D sequentially output from the difference image generation / output unit 33. ₁ ~ D _n Each difference absolute value image AB ₁ ~ AB _n Are generated, and each generated difference absolute value image AB is generated. ₁ ~ AB _n Are sequentially output to the integration processing unit 22.
[0043]
Difference absolute value image AB sequentially input to the integration processing unit 22 ₁ ~ AB _n Are integrated temporally for each pixel by the integration processing unit 22 based on the integration coefficient k calculated by the threshold / integration coefficient calculation unit 41.
[0044]
For example, the difference absolute value image AB _h Is input to the image memory 35, as shown in FIGS. 5A and 5B, the difference absolute value image AB. _h All input images AB up to one frame before (h-1 frame) ₁ ~ AB _h-1 Image representing the integration result of {integrated image IN _h-1 (= AB _h-1 + KAB _h-2 + K ² AB _h-3 + ... + k ^h-2 AB ₁ )} Is accumulated.
[0045]
On the other hand, in the average value / dispersion calculation unit 40 of the threshold value / integration coefficient calculation unit 23, each integral image IN output from the integration processing unit 22. ₁ ~ IN _h-1 From each integrated image IN according to the above equation (1) ₁ ~ IN _h-1 Average value AV is calculated, and each integral image IN is calculated based on the calculated average value AV. ₁ ~ IN _h-1 Variance of σ ² Is calculated based on the above equation (2).
[0046]
At this time, the integration coefficient calculation unit 41 of the threshold value / integration coefficient calculation unit 23 performs integration processing results (each integrated image IN ₁ ~ IN _h-1 ) Of the average value AV and the reference value ST set in advance is constant, that is, the integration processing result (each integrated image IN ₁ ~ IN _h-1 The integration coefficient k in the integration processing unit 22 is adjusted by the coefficient gain of the amplifier 51 so that the average value AV of) is held at a preset reference value ST.
[0047]
For example, when the average value AV of the integration processing result is larger than the preset reference value ST, the coefficient gain is adjusted so that the integration coefficient k becomes smaller, and the average value AV of the integration processing result becomes the reference value ST. If it is smaller, the coefficient gain is adjusted so that the integral coefficient k becomes larger.
[0048]
In this way, the integral image IN up to the (h-1) th frame. _h-1 The integration coefficient k corresponding to the integration image of the next frame (the h-th frame) is calculated based on the image (AB). _h-1 + KAB _h-2 + K ² AB _h-3 + ... + k ^h-2 AB ₁ ) Is multiplied by the calculated integration coefficient k.
[0049]
Then, an image k (AB) obtained by the multiplication process of the integral coefficient k. _h + KAB _h-1 + K ² AB _h-2 + ... + k ^h-1 AB ₁ ) Difference absolute value image AB of the h-th frame _h Is integrated for each pixel by the integration unit 38.
[0050]
As a result, the first to h-th absolute difference image AB ₁ ~ AB _h An image representing the integration result of (AB _h + KAB _h-1 + K ² AB _h-2 + ... + k ^h-1 AB ₁ ) Is generated.
[0051]
The generated first to h-th difference absolute value images AB ₁ ~ AB _h An image representing the integration result of (AB _h + KAB _h-1 + K ² AB _h-2 + ... + k ^h-1 AB ₁ ) Is the integral image IN _h Is output to the binarization processing unit 3 (see FIG. 5B).
[0052]
In this way, the absolute difference image AB of the nth frame _n Is input by the integration processing of the integration processing unit 22, the first to nth difference absolute value images AB. ₁ ~ AB _n An image representing the integration result of (AB _n + KAB _n-1 + K ² AB _n-2 + ... + k ^n-1 AB ₁ ) Is generated.
[0053]
An image (AB) representing the generated integration result _n + KAB _n-1 + K ² AB _n-2 + ... + k ^n-1 AB ₁ ) Is the integral image IN up to the nth frame. _n That is, the first to nth difference absolute value images AB ₁ ~ AB _n Integrated image IN with minutes integrated (integrated) _n Is output to the binarization processing unit 3.
[0054]
On the other hand, in the threshold value calculation unit 42 of the threshold value / integration coefficient calculation unit 23, as shown in FIG. ₁ ~ IN _n Mean value AV and variance σ ² Based on the above, the threshold value t is calculated for each integral image, and the calculated threshold value t is transmitted to the binarization processing unit 3.
[0055]
In the binarization processing unit 3, as shown in FIGS. 6A and 6B, for example, the integral image IN _h Is input, this integral image IN _h When the pixel value of the pixel constituting the pixel is equal to or greater than the threshold value t, it is determined that the pixel is the target candidate region, and the pixel value of the pixel is set to 1.
[0056]
As a result, as shown in FIG. _h And background region R _ha Binary image BI including _h Is required.
[0057]
Thus, each integral image IN ₁ ~ IN _n Are sequentially input to the binarization processing unit 3, the binarization processing unit 3 causes the target candidate region R to be input. ₁ And background region R _1a Binary image BI including ₁ ~ Target candidate area R _n And background region R _na Binary image BI including _n Are required sequentially.
[0058]
The target candidate area R thus obtained _h And background region R _ha Binary image BI including _h The difference from the prior art will be described with reference to FIG.
[0059]
In the present embodiment, as described above, the target image is a portion that blinks on the image due to an operation such as rotation, such as a rotor portion of a helicopter. _h The luminance distribution changes periodically as indicated by reference numeral T1 in FIG.
[0060]
In this respect, in the conventional binarization process using a normal spatial filter (target candidate area extraction process based on a threshold; see T2 in FIG. 7), as indicated by T3 in FIG. It can be seen that the disappeared portion on the image is erroneously recognized as the background region, and an accurate target candidate region cannot be extracted.
[0061]
On the other hand, in the process of the time filter 2 in the image purpose detection apparatus 1 of the present embodiment, as shown by the reference symbol T10 in FIG. ₁ ~ II _h The difference image D obtained by the difference processing unit 20 based on ₁ ~ D _h Is transmitted through the absolute value processing unit 21 so that the signal level on the − side is inverted to the + side and the difference absolute value image AB is ₁ ~ AB _h Is generated.
[0062]
Then, the generated difference absolute value image AB ₁ ~ AB _h Is integrated over time by the integration processing unit 22 and the integrated image IN _h Is generated.
[0063]
At this time, the integral image IN _h In the difference absolute value image AB ₁ ~ AB _h Since the pixel value of each pixel is integrated (accumulated) while being adjusted by the corresponding integral coefficient k, for example, even if there is a blinking portion, that is, a pixel region having a periodically small pixel value (luminance), In the pixel region corresponding to the blinking portion, as indicated by reference numeral T10 in FIG. 7, a large pixel value when accumulated on the image (when “bright”) is accumulated by integration processing by the integration processing unit 22. Therefore, the accumulated pixel value gradually increases.
[0064]
As a result, as shown by reference numerals T10 and T11 in FIG. 7, even when binarization processing is performed using a predetermined threshold value, a target blinking portion can be extracted as a target candidate region.
[0065]
On the other hand, each time-series image I sequentially input from the image sensor ₁ ~ I _n Is smoothed by the smoothing unit 5 and each smoothed time series image IS ₁ ~ IS _n Based on the above, each smoothed time-series image IS is processed by the same processing as the average value / variance calculation unit 40 in the smoothing processing unit 5. ₁ ~ IS _n The mean and variance of are calculated.
[0066]
Each smoothed time series image IS calculated ₁ ~ IS _n Is calculated by the smoothing processing unit 5 on the basis of the average value and the variance, and each smoothed time-series image IS is calculated by the binarizing processing unit 10 on the basis of the calculated threshold value. ₁ ~ IS _n Binary image BS ₁ ~ BS _n Are generated sequentially.
[0067]
On the other hand, each time-series image IS smoothed by the smoothing processing unit 5 ₁ ~ IS _n Is the edge (contour) emphasis by the differentiation processing of the differentiation processing unit 6, each time-series image ID subjected to the differentiation processing ₁ ~ ID _n Based on the above, each time series image ID is obtained by the same processing as the average value / variance calculation unit 40 in the differentiation processing unit 6. ₁ ~ ID _n The mean and variance of are calculated.
[0068]
Each calculated time-series image ID ₁ ~ ID _n The threshold value is calculated by the differentiation processing unit 6 based on the average value and the variance of the time series images, and each time-series image ID is calculated by the binarization processing unit 11 based on the calculated threshold value. ₁ ~ ID _n Binary image DS ₁ ~ DS _n Are generated sequentially.
[0069]
The target area determination unit 12 includes each binarized image BI generated by the binarization processing unit 3. ₁ ~ Binary image BI _n Are sequentially fetched frame by frame. Similarly, the target area determination unit 12 uses each binarized image BS generated by the binarization processing unit 10. ₁ ~ BS _n Are sequentially captured frame by frame, and each binarized image DS generated by the binarization processing unit 11 ₁ ~ DS _n Are sequentially fetched frame by frame.
[0070]
Next, the target area determination unit 12 first acquires the binarized image BI. ₁ Binary image BS ₁ And binarized image DS ₁ Are divided into segments each consisting of a plurality of pixels, and the same label is attached to regions where pixel values 1 in each segment are connected to each other.
[0071]
Subsequently, the target area determination unit 12 performs the binarized image BI. ₁ A plurality of labeled regions (labeling regions) and binarized images BS to be target candidate regions obtained by ₁ And binarized image DS ₁ Each of the obtained labeling regions is compared with each other, and the smoothing maximum value, differential maximum value, and maximum difference of each labeling region are used as a parameter (feature parameter) P representing the feature amount of each labeling region that is a target candidate. A value, a moving direction, and a moving amount are extracted.
[0072]
Subsequently, the target region determination unit 12 calculates an evaluation value of each labeling region based on the obtained feature amount (feature parameter P) of each labeling region, and hereinafter, all the remaining binarized images BI. ₂ ~ Binary image BI _n Binary image BS ₂ ~ BS _n, Binary image DS ₂ ~ DS _n This process is repeatedly executed until evaluation value calculation for all labeling areas corresponding to all target candidate areas using is completed.
[0073]
When the evaluation value calculation for all labeling regions corresponding to all target candidate regions is completed, the target region determination unit 12 detects the labeling region having the highest evaluation value among all the labeling regions as a target.
[0074]
As a result, it is possible to detect a target that blinks on the image by an operation such as rotation, such as a rotor portion of a helicopter.
[0075]
As described above in detail, according to the present embodiment, even if the detection target is a part that blinks on the image by an operation such as rotation, such as a rotor part of a helicopter, a plurality of time-series images including the blinking part. By integrating the difference image in terms of time, the frequency component of the blinking portion can be extracted as the target candidate region.
[0076]
Therefore, by performing the target determination process using the target candidate area, even if the target is a blinking portion on the image, the target can be accurately detected.
[0077]
In particular, in the present embodiment, the frequency component of the blinking portion is set as the target candidate region based on the time-series image collected by the normal image sensor without using a special hardware component and its control system as the incident optical system. Can be extracted. For this reason, it is possible to easily detect a target such as a blinking portion while keeping the entire target detection device compact.
[0078]
Further, according to the present embodiment, when the average value AV of the integration processing result is larger than the preset reference value ST, the coefficient gain is adjusted so that the integration coefficient k becomes small, and the integration processing result When the average value AV is smaller than the reference value ST, by adjusting the coefficient gain so that the integration coefficient k becomes larger, the integration processing result (each integrated image IN ₁ ~ IN _h-1 ) Is held at a preset reference value ST.
[0079]
This control can suppress an increase in the average value of the luminance (pixel value) even when the frequency modulation area (pixel region) is large (for example, due to image movement). The possibility of extracting many target candidate regions other than the true target due to the increase in value can be reduced. As a result, the target candidate extraction accuracy and the target detection accuracy can be improved.
[0080]
Also, even if the frequency component of the frequency modulation area (pixel area) is very small, the average value of luminance (pixel value) can be increased until it approaches the reference value. Even a small frequency component can be extracted as a target candidate region. As a result, the target candidate extraction accuracy and the target detection accuracy can be improved.
[0081]
In the present embodiment, the case where the target is a portion that blinks on the image by an operation such as rotation, such as a rotor portion of a helicopter, has been described, but the present invention is not limited to this configuration, and the above The image target detection apparatus of the present embodiment can be applied to the case where the entire target is detected when the blinking part is included as part of the target, or the target that does not include the blinking part.
[0082]
Further, in the present embodiment, the difference processing unit 20, the absolute value processing unit 21, the integration processing unit 22, the threshold value / integral coefficient calculation unit 23, and the binarization processing unit 3 that constitute the time filter 2, respectively, Although constituted by a wired logic circuit, the present invention is not limited to this, and a difference processing unit 20, an absolute value processing unit 21, an integration processing unit 22, a threshold / integral coefficient calculation unit 23, and a binarization process Each process of the unit 3 may be realized by the function of at least one computer according to the image target detection program.
[0083]
Furthermore, in the present embodiment, the processing of all the constituent elements (all the block constituent elements shown in FIG. 1) constituting the image target detection apparatus 1 is realized by the function of at least one computer according to the program. Also good.
[0084]
In the present embodiment, the difference processing unit receives the input time-series image I. ₁ ~ I _n A difference image D representing a difference between temporally adjacent pixels based on ₁ ~ D _n Is generated. At this time, the difference processing unit performs the first time-series image I. ₁ Is input, since the frame image of the previous frame is not stored in the image memory, the difference image is the time-series image I. ₁ It becomes itself.
[0085]
In this regard, as a modification of the present embodiment, the following time-series image I ₂ It is also possible not to output the difference image until is inputted, and the number of frame images until the average value of the integral image converges to the reference value can be reduced.
[0086]
【The invention's effect】
As stated above, claim 1, 6 and 7 According to the described invention, since the plurality of difference absolute value images output from the difference absolute value image generating means are temporally integrated for each pixel based on a predetermined integration coefficient, for example, the target is a blinking portion. Even in this case, the pixel values when existing on the image can be integrated.
Claim 1, 6 and 7 According to the described invention, since the predetermined integration coefficient for the integral image of the next frame of the integral image of each frame is obtained based on the pixel value and the average value of each pixel of the integral image of each frame, By adjusting the integration coefficient according to the average value with respect to the reference value, it is possible to reduce the erroneous determination of the target candidate area due to the large average value, and furthermore, due to the small average value. It is possible to prevent omission of detection of the target candidate area.
[0087]
As a result, even if the target is a blinking part, the frequency component of the blinking part can be extracted as a target candidate area.
[0088]
According to the second aspect of the present invention, by generating a binarized image in which the pixel value of the extracted target candidate area is 1 and the pixel value of the background area is 0, the generated binarized image is It is possible to accurately determine the target area by using it.
[0090]
Claim 3 According to the described invention, each pixel of the accumulated difference absolute value image of each frame is multiplied by the predetermined integration coefficient, and the difference absolute value image of each frame multiplied by the predetermined integration coefficient is The difference absolute value image of the next frame of the difference absolute value image of each frame is integrated pixel by pixel, and this integration result is accumulated as the difference absolute value image of the next frame and corresponds to the difference absolute value image of the next frame. Therefore, the integrated image can be generated and output accurately for each frame.
[0091]
Claim 4 and 5 According to the described invention, labeling corresponding to the target candidate area is performed from the binarized image serving as the target candidate area, and from each of the labeled target candidate areas, For example, parameters representing feature quantities such as a smoothing maximum value, a differential maximum value, a difference maximum value, a moving direction, and a moving amount are extracted, and each corresponding target is extracted based on the extracted feature amount parameters of each region. The candidate area is evaluated, and the area with the highest evaluation value is detected as the target area. For this reason, accurate target area detection processing can be performed based on the extracted feature parameter.
[Brief description of the drawings]
FIG. 1 is a diagram showing a schematic configuration of an image target detection apparatus according to an embodiment of the present invention.
FIG. 2 is a block diagram showing a schematic configuration of a difference processing unit, an absolute value processing unit, an integration processing unit, and a threshold / integral coefficient calculation unit that constitute the time filter of the present embodiment.
FIG. 3 is a block diagram showing a schematic configuration of an integral coefficient calculation unit shown in FIG. 2;
4A is a block diagram of the difference processing unit shown in FIG. 2; FIG. 4B is a block diagram of the difference processing unit shown in FIG. It is a conceptual diagram for demonstrating by the change of the frame image output.
5A is a block diagram of the integration processing unit shown in FIG. 2, and FIG. 5B is a diagram illustrating the integration processing of the integration processing unit shown in FIG. It is a conceptual diagram for demonstrating by the change of the frame image output.
6A is a block diagram of a binarization processing unit on the time filter side illustrated in FIG. 2, and FIG. 6B is a binarization process of the binarization processing unit illustrated in FIG. 6A. FIG. 2 is a conceptual diagram for explaining a change in a frame image input to and output from the binarization processing unit.
FIG. 7 is a diagram for explaining a difference between a binarized image including a target candidate area obtained by the image purpose detection device of the present embodiment and a target candidate area according to a conventional technique.
FIG. 8 is a block diagram showing a schematic configuration of a conventional target detection processing unit.
[Explanation of symbols]
1 Image target detection device
2 hour filter
5 Smoothing processing part
6 Differential processing unit
3, 10, 11 Binarization processing unit
12 Target area determination unit
20 Difference processing part
21 Absolute value processing section
22 Integration processing unit
23 Threshold / integral coefficient calculator

Claims (7)

An image target detection device for detecting the target candidate area based on a plurality of time-series images in frame units including a target image,
Difference image generation means for generating a plurality of difference images by obtaining a difference for each pixel between temporally adjacent images in the plurality of time-series images;
A difference absolute value image generation means for generating a plurality of difference absolute value images by obtaining an absolute value of each of the plurality of generated difference images for each pixel, and outputting the generated plurality of difference absolute value images;
A plurality of frames of integral images corresponding to the plurality of frames of difference absolute value images by temporally integrating the plurality of difference absolute value images output from the difference absolute value image generating means for each pixel based on a predetermined integration coefficient. And calculating an average value of pixel values of each pixel of the integrated image of each frame, and calculating the average value of each pixel in the integrated image of each frame based on the pixel value of each pixel and the calculated average value. An integral image generating means for calculating a variance and obtaining a predetermined integral coefficient for the integral image of the next frame of the integral image of each frame based on the average value ;
An image target detection apparatus comprising: An integral image the generated by binarization processing based on a predetermined threshold value, the pixel value of 1 and the background area the pixel values of the candidate region of the target to generate a binary image to 0 Binarization means for obtaining a predetermined threshold value for a next integral image of the integral image of each frame based on the average value and the variance ;
The image target detection apparatus according to claim 1, further comprising: The integral image generation means multiplies the accumulation unit that accumulates the difference absolute value image of each frame and each pixel of the difference absolute value image of each frame accumulated by the accumulation processing unit by the predetermined integration coefficient. Then, the difference absolute value image of each frame multiplied by the predetermined integration coefficient is integrated for each pixel with respect to the difference absolute value image of the next frame of the difference absolute value image of each frame, and this integration result is added to the next frame. claims and sends to the storage processing unit as a difference absolute value image, characterized by comprising an integrating processing unit for outputting as the integral image of the next frame corresponding to the difference absolute value image of the next frame 1 or 2 The image target detection apparatus described. Means for dividing the binarized image serving as the target candidate region into segments composed of a plurality of pixels and attaching the same label to regions where pixel values 1 in each segment are connected to each other;
A parameter representing a feature amount in each labeled region is extracted, each extracted region is evaluated based on the feature amount parameter, and a region having the highest evaluation value is detected as the target region. Means to
The image target detection apparatus according to claim 2, further comprising: 5. The image target detection apparatus according to claim 4 , wherein the parameter representing the feature amount includes a smoothing maximum value, a differential maximum value, a difference maximum value, a movement direction, and a movement amount in each of the regions. A computer-executable image target detection program for detecting the target candidate area based on a plurality of time-series images in frame units including a target image,
The computer,
Difference image generation means for generating a plurality of difference images by obtaining a difference for each pixel between temporally adjacent images in the plurality of time-series images;
A difference absolute value image generation means for generating a plurality of difference absolute value images by obtaining an absolute value of each of the plurality of generated difference images for each pixel, and outputting the generated plurality of difference absolute value images;
Integration of multiple frames corresponding to the difference absolute value images of the plurality of frames in temporal integration for each pixel based on the plurality of differential absolute value image outputted from the difference absolute value image generating means to a predetermined integral coefficient An image is generated, and an average value of pixel values of each pixel of the integral image of each frame is calculated, and each pixel in the integral image of each frame is calculated based on the pixel value of each pixel and the calculated average value Integral image generation means for calculating a predetermined integration coefficient for the integral image of the next frame of the integral image of each frame based on the average value ;
And a program for detecting an image target. An image target detection method for detecting a target candidate area based on a plurality of time-series images in frame units including a target image,
Obtaining a difference for each pixel between temporally adjacent images in the plurality of time-series images, and generating a plurality of difference images;
Obtaining the absolute value of each of the plurality of generated difference images for each pixel to generate a plurality of difference absolute value images, and outputting the generated plurality of difference absolute value images;
A plurality of frames of integral images corresponding to the plurality of frames of difference absolute value images by temporally integrating the plurality of difference absolute value images output in the difference absolute value image generation step for each pixel based on a predetermined integration coefficient. A step of generating
Calculate the average value of the pixel values of each pixel of the integral image of each frame, calculate the variance of each pixel in the integral image of each frame based on the pixel value of each pixel and the calculated average value, Obtaining a predetermined integration coefficient for the integral image of the next frame of the integral image of each frame based on the average value;
An image target detection method comprising:

Images