JP3853574B2 - Moving object detection system - Google Patents

Description

【００２７】
以上のように、この実施の形態１によれば、撮像画像と背景画像とを比較して車両１１を検出するに当たって車両検出手段６は、撮像画像および背景画像のそれぞれの画素毎に周囲の画素との相対輝度差を演算し、この画素毎の相対輝度差をそれぞれに累積加算し、その加算値の差を画像差信号として演算し、更にこの画像差信号同士の差が所定の閾値よりも大きい場合には車両検出と判定して出力するので、撮像画像と背景画像とは各領域毎にそれぞれの画像内での相対輝度差でもって比較されることになり、車両１１のヘッドライトの光などによる画像の輝度変化があったとしてもこの輝度変化を抑制した状態で撮像画像と背景画像とを比較することができ、これに影響されることなく車両１１を好適に検出することができる効果がある。
【００２８】
実施の形態２．
図５はこの発明の実施の形態２による移動体検出システムの構成を示すシステム構成図である。図において、１６は車両検出手段６が車両非検出と判定したタイミングの撮像画像を用いて車両１１の画像を特徴点として抽出するとともにその特徴点の路面からの高さを判定する車両判定手段（特徴点抽出手段、サイズ判定手段）である。なお、この特徴点としては例えば、２つのテールランプ、昼間屋外のナンバープレートなどが挙げられる。
【００２９】
図６はこの発明の実施の形態２による監視用ＩＴＶカメラ１の撮像画像の一例を示す説明図である。図において、（ａ）は時刻ｔ（０）での撮像画像、（ｂ）は時刻ｔ（１）（＝ｔ（０）＋Δｔ：但しΔｔは監視用ＩＴＶカメラ１の撮像周期１周期分）での撮像画像、１７は移動体検出領域１２の上（車両１１の進行方向下流側）に隣接する位置に設定された移動体判定領域、１８は移動体検出領域１２の車両移動方向下流側端辺、１９はそれぞれ車両１１のテールランプである。これ以外の構成は実施の形態１と同様であり説明を省略する。
【００３０】
次に動作について説明する。
図６（ａ）に示す撮像画像では車両検出手段６から車両検出を意味する処理結果が出力され、これが同図（ｂ）に示す撮像画像に変化すると、車両検出手段６から出力される処理結果は車両検出から車両非検出に変化する。車両判定手段１６は、この車両検出手段６の出力の処理結果が車両非検出に変化したタイミングの撮像画像（図６（ｂ））の移動体判定領域１７を用いて、車両１１の特徴点を抽出するとともにその特徴点の路面からの高さを判定する。
【００３１】
図７はこの発明の実施の形態２による車両判定手段１６の処理を示すフローチャートである。図において、ＳＴ７は撮像画像を構成する各画素がマトリックス状に配列されていることを前提として、移動体判定領域１７内の各画素の相対輝度差および背景画像のものとの輝度変化値（＝判定に用いる撮像画像における輝度−背景画像における輝度）を各列毎に累積加算する列別累積加算ステップ、ＳＴ８は各列の累積加算値が周囲の列よりも特に大きい列のまとまり（１乃至複数）を抽出する列抽出ステップ、ＳＴ９は当該各組の列のまとまりそれぞれについてそれぞれの範囲内において現画像に戻って相対輝度差が他の部分よりも高い画素のまとまり（以下小領域と呼ぶ）をテンプレートとして切出す切出ステップ、ＳＴ１０は当該小領域と移動体検出領域１２との撮像画像上の距離に基づいて当該特徴点の路面からの高さを特定する高さ判定ステップである。
【００３２】
そして、このような処理では、累積加算値が大きくなる列において、更にその他の画素よりも相対輝度差が大きい画素の集まりが小領域として抽出されることになり、この実施の形態２のように車両１１をその後方から撮像した場合、最も撮像輝度が明るくなる例えばテールランプなどについてその輪郭に沿った形状の小領域を抽出することができる。
【００３３】
図８はこの発明の実施の形態２による現画像と輝度変化値の列毎の累積加算値との関係を示す説明図である。同図（ａ）は図６（ｂ）と同じ画像、同図（ｂ）はそのときの輝度変化値の列毎の累積加算値グラフである。同図（ｂ）において横軸は画像上の列、縦軸は累積加算値である。そして、同図に示すように、車両を後方から撮像した場合に移動体判定領域１７の各列毎の累積加算値は画像の２つのテールランプを含む列の輝度が他の列の輝度よりも相対的に高くなっており、図７においはこの性質を利用して車両の特徴点を抽出している。また、同図に示すように、輝度変化値の累積加算値が生じる（０よりも大きい値をとる）列の範囲は車両の車幅とほぼ一致する。従って、この累積加算値が生じた列の数に基づいて車幅、ひいては車種を判定することができる。
【００３４】
また、図９はこの小領域と移動体検出領域１２との撮像画像上の距離に基づいて当該小領域の路面からの高さを特定することができる理由を説明するための説明図である。撮像画像としては図６（ｂ）の撮像画像が相当する。図において、２０は車両１１が走行する路面、２１は撮像画像においては車両移動方向下流側端辺１８となる視野方向、２２は特徴点（テールランプ１９）の視野方向、２３は移動体検出領域１２に相当する路面位置である。
【００３５】
そして、同図に示すように図６（ｂ）の撮像タイミングは車両１１の後端が路面位置２３を通過しきったタイミングに相当し、このタイミングにおいては車両移動方向下流側端辺１８の真上に特徴点（テールランプ１９）が位置すると考えることができる。従って、撮像画像上における車両移動方向下流側端辺１８と特徴点（テールランプ１９）との距離を角度θに変換し、そのθに基づいて特徴点の路面２０からの高さｈを求めることができる。これ以外の動作は実施の形態１と同様であり説明を省略する。
【００３６】
以上のように、この実施の形態２によれば、一方向に沿って移動する車両１１を監視用ＩＴＶカメラ１にて撮像し、車両判定手段１６が移動体検出領域１２を車両１１が通過したタイミングの撮像画像を用いて車両１１の後方の最も輝度が高くなる小領域、すなわちテールランプ１９などを特徴点として抽出しているので、移動体判定領域１７に確実に車両１１が存在する状態で特徴点抽出処理を行うことができ、確実に車両１１の特徴点を抽出することができる効果がある。
【００３７】
この実施の形態２によれば、車両１１の移動方向に沿った方向に対して、０度あるいは９０度以外の角度にて撮像するように監視用ＩＴＶカメラ１を設置するとともに、車両判定手段１６が抽出した特徴点と移動体検出領域１２の車両移動方向下流側端辺１８との画像上の距離に基づいて特徴点の路面２０からの高さを判定するので、車両１１の特徴点（テールランプ１９）の高さに基づいて車両１１のサイズを判定することができる。また、累積加算値が生じた列の数に基づいて車幅、ひいては車種を判定することができる。
【００３８】
実施の形態３．
図１０はこの発明の実施の形態３による移動体検出システムの構成を示すシステム構成図である。図において、２４は撮像画像記憶手段４が記憶する撮像画像が入力され、これに基づいて背景画像を生成して背景画像記憶手段５へ出力する背景画像生成部である。
【００３９】
図１１はこの発明の実施の形態３による背景画像生成部２４の構成を示すシステム構成図である。図において、２５は上記撮像画像記憶手段４からの撮像画像を最新画像として記憶する最新画像バッファ（撮像画像記憶手段）、２６はこの最新画像バッファ２５が１つ前に出力した画像（１フレーム前の画像）を直前画像として記憶する直前画像バッファ（撮像画像記憶手段）、２７はこの最新画像と直前画像とが入力され、各画素毎にこれら２つの画像の輝度差が所定の閾値よりも低いか否かを判定し、当該閾値よりも輝度差が小さくなっている期間を示す輝度安定性情報を出力する安定度判定手段、２８は背景画像記憶手段５に記憶されている背景画像のコピーを記憶するコピー記憶手段、２９は上記輝度安定性情報に示される期間が所定の期間以上となると、各画素毎の輝度安定性情報をリセットするとともに当該画素の背景画像における輝度値を最新画像の輝度値に置き換える背景画像生成手段である。これ以外の構成は実施の形態２と同様であり説明を省略する。
【００４０】
次に動作について説明する。
撮像画像記憶手段４に監視用ＩＴＶカメラ１から出力された最新の撮像画像が記憶されると、最新画像バッファ２５にも同一の画像が記憶され、直前画像バッファ２６には当該最新画像バッファ２５が直前に記憶していた直前画像が記憶される。なお、この実施の形態３では、これらバッファの画像の記憶周期は監視用ＩＴＶカメラ１から出力される撮像画像の出力周期と同一として説明するが、撮像画像を適当に間引くような記憶周期であってもよい。また、背景画像記憶手段５およびコピー記憶手段２８には同一の背景画像が記憶されているものとする。
【００４１】
図１２はこのように最新画像バッファ２５および直前画像バッファ２６に撮像時間が相前後する２つの撮像画像が記憶された状態で実施される背景画像更新処理を示すフローチャートである。図において、ＳＴ１１は安定度判定手段２７が撮像画像において或る座標（画素）について最新画像における輝度と直前画像における輝度との輝度差が所定の判定値よりも小さいか否かを判定する輝度差判定ステップ、ＳＴ１２は安定度判定手段２７が輝度安定性情報に含まれる期間に「１」加える期間加算ステップ、ＳＴ１３は安定度判定手段２７が輝度安定性情報に含まれる期間を「０」にリセットする期間リセットステップ、ＳＴ１４は背景画像生成手段２９がこの期間が所定の安定度判定期間よりも長いか否かを判定する安定期間判断ステップ、ＳＴ１５は背景画像生成手段２９が背景画像の上記座標（画素）の輝度を最新画像の輝度に変更する輝度更新ステップ、ＳＴ１６は背景画像生成手段２９が背景画像の上記座標（画素）の輝度をそのままとする輝度未更新ステップ、ＳＴ１７は背景画像の全ての座標（画素）に対して当該処理を繰り返し実行させる繰返制御ステップである。
【００４２】
そして、全ての画素について当該処理が完了したら、背景画像記憶手段５には新たな背景画像が記憶されることになり、コピー記憶手段２８にはこの新たな背景画像がコピー記憶されることになる。これ以外の動作は実施の形態３と同様であり説明を省略する。
【００４３】
以上のように、この実施の形態３によれば、最新画像バッファ２５および直前画像バッファ２６に時間的に相前後する２つの撮像画像を記憶すると共に、安定度判定手段２７が各画素毎に輝度変化量に基づいて安定度を判定し、更に、この安定と判定される期間が所定の期間以上であれば背景画像生成手段２９が新たな背景画像を生成するので、背景画像を更新することができる。しかも、所定の期間において殆ど変化がなかった撮像画像を用いて背景画像を生成するので、車両１１などが含まれる可能性が低い画像を用いて適切に背景画像を生成することができ、しかも、例え含まれていたとしても所定の期間の後にはそれを画像から削除することができるので誤った移動体検出をし続けてしまうことはない。従って、環境の変化に追従した最適な背景画像を用いて２４時間３６５日の監視を行うことができる。
【００４４】
この実施の形態３によれば、背景画像記憶手段５の記憶する画像が更新されたら当該画像を記憶するコピー記憶手段２８を設け、背景画像生成手段２９は、安定度判定情報に基づいて各画素毎に最新画像の輝度と背景画像の輝度とのうちから一方を選択し、これを組み合わせることで新たな背景画像を生成するので、撮像画像や現在の背景画像自体に車両１１などが含まれていたとしても、その影響を削減するように画像を合成し、それを背景画像とすることができる。従って、単に撮像画像を用いた場合や従来の背景画像更新処理に比べても背景画像に含まれる車両１１などを誤って検出してしまうことを抑制することができる効果がある。
【００４５】
実施の形態４．
図１３はこの発明の実施の形態４による移動体検出システムの構成を示すシステム構成図である。図において、３０は車両判定手段１６と同様の処理をして車両１１の特徴点およびその路面２０からの高さを求めた後に更に、それ以降の撮像画像に対して当該特徴点に基づく車両追跡処理を行う車両追跡手段（特徴点抽出手段、サイズ判定手段、追跡手段）である。これ以外の構成は実施の形態３と同様であり説明を省略する。
【００４６】
次に動作について説明する。
図１４はこの発明の実施の形態４による車両追跡手段３０の処理を示すフローチャートである。図において、ＳＴ１８はそれ以降の撮像画像に対しても列別累積加算ステップＳＴ７から切出ステップＳＴ９までの処理を行うと共にその結果得られた各小領域とテンプレートとを比較して例えば相似形状であると推定されるなどのマッチング判定により車両１１の移動先判定を行う移動先判定ステップ、ＳＴ１９は相前後する撮像画像において同一のテンプレートにマッチングすると判定された小領域の画像上の位置変化に基づいて当該画像間における車両１１の移動速度を判定する移動速度判定ステップである。そして、この移動速度情報は特徴点の路面２０からの高さ情報などと共に車両監視センタ３へ出力される。
【００４７】
また、移動速度は例えば以下の手順にて求めることができる。まず、各画像における小領域の撮像位置と車両１１の高さ情報とを用いて当該車両１１の後部のそれぞれの撮像タイミングにおける路面２０上の位置を特定する。次に当該２つの撮像画像における車両後部の位置同士の差を移動距離として求め、更に当該２つの画像の撮像時間間隔（これは監視用ＩＴＶカメラ１の撮像周期に基づいて予め特定できる）で除算することで移動速度を求めることができる。これ以外の動作は実施の形態３と同様であり説明を省略する。
【００４８】
そして、このように車両１１の移動を特徴点の移動として捕捉するとともに、その特徴点の路面２０からの高さを予め特定しておくことで、速度演算に用いる２つの画像における当該車両１１の路面２０上の位置を精度良く特定することができるので、車両１１の移動速度を精度良く求めることができる。
【００４９】
また、このように特徴点を抽出しているので、従来のように影の発生状態変化などの影響を受けてしまうことなく確実に車両１１の移動を捕捉することができ、しかも、車両１１が横方向に移動したとしてもその移動距離を確実に捕捉することができる。
【００５０】
更に、車両１１を後方から捕らえると共に輝度が高い部分を特徴点として抽出しているので、車両１１の後部に必ず２つ設けられているテールランプ１９，１９を特徴点として抽出することができる。このように少なくとも２つの特徴点を抽出することができるので、そのうちの一つの特徴点が例え支柱、電信柱、道路標識、他の移動体などによって隠蔽されてしまったとしても、他の特徴点に基づいてパターンマッチング判定を行い、車両１１を継続して追跡することができる効果がある。
【００５１】
従って、この特徴点の画像上の位置変化に基づく車両１１の追跡位置情報を用いることで、上記ヘッドライトの光による輝度変化、太陽の位置に応じた影の発生状態の変化、夜間照明の位置に応じた影の発生状態の変化などの影響を受けることなく、車両１１の移動距離や移動速度を精度良く得ることができる効果がある。
【００５２】
実施の形態５．
図１５はこの発明の実施の形態５による移動体検出システムの構成を示すシステム構成図である。図において、３１は実施の形態４の車両追跡手段３０と同様の処理を行った後更に、テンプレートマッチング判定にて一致する小領域を特定することができなかったテンプレートがある場合にはそのテンプレートに一致するであろう小領域の位置を推定する車両追跡手段（特徴点抽出手段、サイズ判定手段、追跡手段、推定手段）である。これ以外の構成は実施の形態４と同様であり説明を省略する。
【００５３】
次に動作について説明する。
図１６はこの発明の実施の形態５による車両追跡手段３１の処理を示すフローチャートである。図において、ＳＴ２０はテンプレートマッチング判定にて一致する小領域を特定することができなかったテンプレートがある場合にはそのテンプレートに一致するであろう小領域の位置を推定する推定ステップである。そして、この推定した小領域の推定位置情報は他のマッチングが得られた小領域の情報とともに車両監視センタ３へ出力される。
【００５４】
また、小領域の位置推定は例えば以下の手順にて求めることができる。マッチすると判定された他の特徴点がある場合には、例えば他の特徴点に基づいて得られた車両１１の後部位置情報に基づいて当該位置における撮像画像上の特徴点同士の位置関係（間隔）を演算し、次にその位置関係（間隔）、車両後部の位置情報、特徴点の高さ情報などを用いて当該特徴点の位置を推定すれば良い。また、マッチすると判定された特徴点が１つも無いような場合においては、それ以前の撮像画像における特徴点の位置変化、それに基づいて得られた車両１１の移動速度などの情報を用いて例えば車両１１はこれらの画像間においては直線的に移動しているなどの推定条件の下で今回の撮像画像における車両１１の路面上の位置を推定し、その位置における各特徴点の位置を更に推定すれば良い。これ以外の動作は実施の形態４と同様であり説明を省略する。
【００５５】
以上のように、この実施の形態５によれば、テンプレートマッチング判定においてマッチすると判定されなかった特徴点について、車両追跡手段３１がそのテンプレートに一致するであろう小領域の位置を推定するので、例え全ての特徴点が例えば支柱、電信柱、道路標識、他の移動体などによって隠蔽されてしまったとしてもその位置を推定し、その後の撮像画像に対しても継続して追跡することができる効果がある。
【００５６】
なお、以上の実施の形態では、背景画像を生成するに当たり各画素毎に当該画素の撮像画像における輝度安定性を判定し、所定の期間以上にわたって安定状態にある輝度を用いて更新しているが、他にも、図１７に示すようなやり方で背景画像を生成しても良い。同図において、３２は最新の撮像画像、３３は背景画像、３４はそれぞれ瞬時的な変化が生じている部分、３５は差分画像、３６は反転二値化画像、３７は新しい背景画像である。そして、同図においては、まず、最新の撮像画像３２と背景画像３３との差分画像３５を生成し、この差分画像３５を所定の閾値に基づいて二値化し更に反転して反転二値化画像３６を生成し、更に、最新の撮像画像３２や背景画像３３とともにこの反転二値化画像３６を用いて新しい背景画像３７を生成するものである。
【００５７】
但し、この背景画像の生成方法では同図に示すように、車両１１などの移動体や車両１１のヘッドライトの光などによって瞬時的な輝度変化が生じている部分が背景画像に残ってしまうので、以上の実施の形態に示すように輝度が安定していると判断された輝度のみを順次各画素に適用する更新方法の方が、この瞬時的な輝度変化を取り除いたより適した背景画像を生成することができる効果がある。
【００５８】
【発明の効果】
以上のように、この発明によれば、所定の空間を撮像し撮像画像を出力する撮像手段と、当該撮像手段が過去に出力した撮像画像あるいはそれを加工した画像を背景画像として記憶する背景画像記憶手段と、上記撮像画像と上記背景画像の移動体の侵入側に設定された移動体検出領域において、画素毎に周囲の画素との相対輝度差を演算し、演算した画素毎の相対輝度差を上記移動体検出領域全体に渡って加算し、上記撮像画像と上記背景画像における画素毎の相対輝度差の加算値の差を画像差信号として演算する画像差演算手段と、上記画像差信号と所定の閾値とを比較して移動体の検出判定を行う判定手段とを備えるので、撮像画像と背景画像とは各領域毎にそれぞれの画像内での相対輝度差でもって比較されることになる。従って、車両のヘッドライトの光などによる画像の輝度変化があったとしてもこの輝度変化を抑制した状態で撮像画像と背景画像とを比較することができ、これに影響されることなく移動体を好適に検出することができる効果がある。
【００５９】
この発明によれば、判定手段が移動体を検出と判定した後に非検出と判定したタイミングにおいて、撮像画像と背景画像の移動体検出領域の移動体の進行方向の下流側に設定された移動体判定領域において、上記撮像画像と上記背景画像の輝度変化値を各列毎に累積加算し、累積加算値が周囲の列よりも大きい列のまとまりを抽出し、抽出した列のまとまりにおいて、上記撮像画像における画素毎の相対輝度差が他の画素よりも大きい画素のまとまりを特徴点として切り出す特徴点抽出手段を設けたので、上記移動体判定領域に移動体が存在する状態で特徴点抽出処理を行うことができ、確実に移動体の特徴点を抽出することができる効果がある。
【００６０】
この発明によれば、移動体の移動方向に沿った方向に対して０度あるいは９０度以外の撮像角度となるように撮像手段を設置するとともに、特徴点抽出手段が抽出した特徴点と移動体検出領域の移動体移動方向下流側端辺との画像上の距離に基づいて移動体のサイズを判定するサイズ判定手段を設けたので、移動体のサイズに基づいて移動体の種類などを判別することができる効果がある。
【００６１】
この発明によれば、移動体の複数の特徴点を抽出する特徴点抽出手段と、上記特徴点抽出手段が複数の特徴点を抽出した後の撮像画像に対して、上記特徴点抽出手段と同様の処理を行って小領域を抽出し、この小領域と各特徴点とを比較してマッチング判定を行い、少なくとも１つの特徴点においてマッチすると判定される小領域が存在する場合に、この小領域の撮像画像上の位置に基づいて移動体の追跡位置情報を出力する追跡手段とを設けたので、１つの特徴点が例えば支柱、電信柱、道路標識、他の移動体などによって隠蔽されてしまったとしても、他の特徴点に基づいて移動体を継続して追跡することができる効果がある。
【００６２】
また、相対輝度差に基づいて複数の特徴点を抽出しているので、車両のヘッドライトの光などに影響されることなく、それらの相互の位置関係などに基づいて車両の大きさなどを判定することができる効果もある。
【００６３】
更に、この特徴点の画像上の位置変化に基づく移動体の追跡位置情報を用いることで、上記ヘッドライトの光による輝度変化、太陽の位置に応じた影の発生状態の変化、夜間照明の位置に応じた影の発生状態の変化などの影響を受けることなく、移動体の移動距離や移動速度を得ることができる効果がある。
【００６４】
この発明によれば、移動体の１乃至複数の特徴点を抽出する特徴点抽出手段と、
上記特徴点抽出手段が特徴点を抽出した後の撮像画像に対して、上記特徴点抽出手段と同様の処理を行って小領域を抽出し、この小領域と各特徴点とを比較してマッチング判定を行い、少なくとも１つの特徴点においてマッチすると判定される小領域が存在する場合に、この小領域の撮像画像上の位置に基づいて移動体の追跡位置情報を出力する追跡手段と、上記追跡手段がマッチすると判定できなかった特徴点について、以前の当該特徴点の軌跡あるいはマッチすると判定された他の特徴点がある場合にはそれを利用して推定位置情報を出力する推定手段とを設けたので、例え全ての特徴点が例えば支柱、電信柱、道路標識、他の移動体などによって隠蔽されてしまったとしてもその位置を推定し、継続して追跡することができる効果がある。
【００６５】
また、相対輝度差に基づいて複数の特徴点を抽出しているので、車両のヘッドライトの光などに影響されることなく、それらの相互の位置関係などに基づいて車両の大きさなどを判定することができる効果もある。
【００６６】
更に、この特徴点の画像上の位置変化に基づく移動体の追跡位置情報を用いることで、上記ヘッドライトの光による輝度変化、太陽の位置に応じた影の発生状態の変化、夜間照明の位置に応じた影の発生状態の変化などの影響を受けることなく、移動体の移動距離や移動速度を得ることができる効果がある。
【００６７】
そして、例えば、移動体は車両であり、特徴点抽出手段は当該車両のテールランプを特徴点として抽出するものであればよい。
【００６８】
この発明によれば、撮像手段から出力された撮像画像を２つ以上記憶する撮像画像記憶手段と、この撮像画像記憶手段に記憶された複数の撮像画像同士を比較し、画像の変化の少なさを判定する安定度判定手段と、安定度判定手段において所定の回数連続して安定と判断されたら上記撮像画像記憶手段に記憶されている撮像画像を用いて背景画像を生成する背景画像生成手段とを設け、背景画像記憶手段は当該背景画像を記憶するので、背景画像を逐次更新することができる。
【００６９】
しかも、所定の回数に相当する期間において殆ど変化がなかった撮像画像を用いて背景画像を生成するので、移動体などが含まれる可能性が低い画像を用いて適切に背景画像を生成することができ、しかも、例え含まれていたとしても所定の期間の後にはそれが更新されるので誤った移動体検出をし続けてしまうことはない。従って、環境の変化に追従した最適な背景画像を用いて２４時間３６５日の監視を行うことができる。
【００７０】
この発明によれば、背景画像記憶手段の記憶する画像が更新されたら当該画像を記憶するコピー記憶手段を設け、背景画像生成手段は、安定度判定手段において判定に用いられた撮像画像のうちの１つと当該コピー記憶手段に記憶されている画像とを合成して新たな背景画像を生成するので、撮像画像や現在の背景画像に移動体が含まれていたとしても、その影響を削減するように画像を合成し、それを背景画像とすることができる。従って、単に撮像画像を用いた場合に比べて背景画像に含まれる移動体を誤って検出してしまうことを抑制することができる効果がある。
【図面の簡単な説明】
【図１】 この発明の実施の形態１による移動体検出システムの構成を示すシステム構成図である。
【図２】 この発明の実施の形態１による監視用ＩＴＶカメラの撮像画像の一例を示す説明図である。
【図３】 この発明の実施の形態１による両検出フローを示すフローチャートである。
【図４】 この発明の実施の形態１による相対輝度差演算処理を説明するための説明図である。
【図５】 この発明の実施の形態２による移動体検出システムの構成を示すシステム構成図である。
【図６】 この発明の実施の形態２による監視用ＩＴＶカメラの撮像画像の一例を示す説明図である。
【図７】 この発明の実施の形態２による車両判定手段の処理を示すフローチャートである。
【図８】 この発明の実施の形態２による現画像と輝度変化値の列毎の累積加算値との関係を示す説明図である。
【図９】 小領域と車両検出領域との撮像画像上の距離に基づいて当該小領域の路面からの高さを特定することができる理由を説明するための説明図である。
【図１０】 この発明の実施の形態３による移動体検出システムの構成を示すシステム構成図である。
【図１１】 この発明の実施の形態３による背景画像生成部の構成を示すシステム構成図である。
【図１２】 この発明の実施の形態３による背景画像更新処理を示すフローチャートである。
【図１３】 この発明の実施の形態４による移動体検出システムの構成を示すシステム構成図である。
【図１４】 この発明の実施の形態４による車両追跡手段の処理を示すフローチャートである。
【図１５】 この発明の実施の形態５による移動体検出システムの構成を示すシステム構成図である。
【図１６】 この発明の実施の形態５による車両追跡手段の処理を示すフローチャートである。
【図１７】 他の背景画像生成処理を示す図である。
【図１８】 従来の移動体検出システムの構成を示すシステム構成図である。
【図１９】 従来の移動体検出システムにおいて撮像カメラで撮像した画像の一例である。
【符号の説明】
１ 監視用ＩＴＶカメラ（撮像手段）、２ 画像処理ユニット、３ 車両監視センタ、４ 撮像画像記憶手段、５ 背景画像記憶手段、６ 車両検出手段（画像差演算手段、判定手段）、７ 撮像画像、８ 道路左側端部、９ 道路右側端部、１０ センターライン、１１ 車両、１２ 移動体検出領域、１３ 検出領域、１４，１５ 画素群、１６ 車両判定手段（特徴点抽出手段、サイズ判定手段）、１７ 移動体判定領域、１８ 車両移動方向下流側端辺、１９ テールランプ、２０ 路面、２１ 視野方向、２２ 特徴点の視野方向、２３ 移動体検出領域に相当する路面位置、２４ 背景画像生成部、２５ 最新画像バッファ（撮像画像記憶手段）、２６ 直前画像バッファ（撮像画像記憶手段）、２７ 安定度判定手段、２８ コピー記憶手段、２９ 背景画像生成手段、３０ 車両追跡手段（特徴点抽出手段、サイズ判定手段、追跡手段）、３１ 車両追跡手段（特徴点抽出手段、サイズ判定手段、追跡手段、推定手段）、３２ 最新の撮像画像、３３ 背景画像、３４ 瞬時的な変化が生じている部分、３５ 差分画像、３６ 反転二値化画像、３７ 新しい背景画像。[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a moving body detection system that detects a moving body based on an image captured by an imaging means, and more particularly to a moving body detection system that can suitably detect a moving body even at night or in a tunnel. It is.
[0002]
[Prior art]
FIG. 18 is a system configuration diagram showing a configuration of a conventional moving body detection system disclosed in Japanese Patent Laid-Open No. 9-81888. In the figure, 38 is an imaging camera, 39 is an image input means for storing an image, and 40 is an image processing means.
[0003]
FIG. 19 is an example of an image captured by the imaging camera 38. In the figure, 41 is a captured image, 42 is a road left edge, 43 is a road right edge, 44 is a center line, 45 is a vehicle, 46 is a moving object detection area, 47 is a detection area, 48 is a measurement area, and 49 is This is a moving body moving speed determination area. The vehicle 45 moves along the center line 44 of the road.
[0004]
Next, the operation will be described.
When the image captured by the imaging camera 38 is stored in the image input means 39, the image processing means 40 performs a vehicle 45 detection process on the captured image. First, a change in luminance in the moving object detection area 46 is determined. If a detection area 47 whose luminance has changed as shown in FIG. 19 is generated, it is determined that the vehicle is detected. Further, the size of the vehicle 45, for example, a large vehicle or a small vehicle is determined based on the ratio of the detection region 47 to the moving object detection region 46. In the state where it is determined that the vehicle is a small car, if the luminance changes simultaneously in the measurement region 48, it is determined that the small car is continuously running. Furthermore, the image of the moving body moving speed determination area 49 of a plurality of consecutive captured images is analyzed, and the shadow generated on the rear road surface of each vehicle 45 is detected in each captured image, and based on the position change of the plurality of shadows. The moving speed of each vehicle is detected.
[0005]
[Problems to be solved by the invention]
Since the conventional moving body detection system is configured as described above, the brightness of the image itself is used for comparison. If the brightness change occurs in the captured image due to the light of the headlight of the vehicle 45 or the like, this is erroneously detected. There existed problems, such as catching as a brightness | luminance change by moving bodies, such as the vehicle 45.
[0006]
In particular, in the conventional moving body detection system, the size of the vehicle 45 is determined based on the size of the area where the luminance has changed, or the vehicle is determined based on the change in the position of the shadow generated behind the vehicle 45 on the image. 45 movement speed is detected, so the brightness change due to the light of the headlight, the change of the shadow generation state according to the position of the sun and clouds, the change of the shadow generation state according to the position of the night illumination There are also issues such as being easily affected.
[0007]
The present invention has been made to solve the above-described problems. Even if there is a change in luminance of a captured image due to light of a vehicle headlight or the like, the moving object is preferably used without being affected by the luminance change. It is an object of the present invention to obtain a moving body detection system that can be detected easily.
[0008]
[Means for Solving the Problems]
The moving object detection system according to the present invention images a predetermined space. Shoot Imaging means for outputting an image image, background image storage means for storing a captured image output by the imaging means in the past or an image obtained by processing the image as a background image, and the captured image the above background image In the moving object detection area set on the intrusion side of the moving object, a relative luminance difference with surrounding pixels is calculated for each pixel, and the calculated relative luminance difference for each pixel is added over the entire moving object detection area. Then, the difference between the relative luminance differences for each pixel in the captured image and the background image is calculated as an image difference signal. A moving object by comparing the image difference calculating means with the image difference signal and a predetermined threshold value Inspection And a determination means for performing an output determination.
[0009]
The moving body detection system according to the present invention is: After the judging means judges that the moving body is detected At the timing determined as non-detection, In the moving object detection area set on the downstream side of the moving object in the moving object detection area of the captured image and the background image, the luminance change values of the captured image and the background image are accumulated and added for each column. A group of columns whose added value is larger than the surrounding columns is extracted, and in the extracted column group, a group of pixels in which the relative luminance difference for each pixel in the captured image is larger than other pixels is extracted as a feature point. Feature point extraction means is provided.
[0010]
The moving body detection system according to the present invention is: Direction along the moving direction of the moving object The image pickup means is installed so as to have an image pickup angle other than 0 degree or 90 degrees with respect to the feature point and the moving object detection area extracted by the feature point extraction means The downstream side of the moving body in the moving direction Is provided with size determining means for determining the size of the moving body based on the distance on the image.
[0011]
The moving object detection system according to the present invention includes a feature point extracting means for extracting a plurality of feature points of a moving object, and after the feature point extracting means extracts a plurality of feature points. of For captured images ,the above Perform the same process as the feature point extraction means Extract a small area and compare this small area with each feature point A matching decision is made and a match is found at at least one feature point If there is a small area And tracking means for outputting tracking position information of the moving body based on the position on the captured image.
[0012]
A moving body detection system according to the present invention includes a feature point extracting unit that extracts one or more feature points of a moving body, and the feature point extracting unit that extracts the feature points. of For captured images ,the above Perform the same process as the feature point extraction means Extract a small area and compare this small area with each feature point A matching decision is made and a match is found at at least one feature point If there is a small area Tracking means for outputting tracking position information of the moving body based on the position on the captured image of the above For a feature point that could not be determined to be matched by the tracking means, if there is a previous trajectory of the feature point or another feature point determined to match, an estimation means that outputs estimated position information using the feature point It is provided.
[0013]
In the moving body detection system according to the present invention, the moving body is a vehicle, and the feature point extraction means extracts the tail lamp of the vehicle as a feature point.
[0014]
The moving body detection system according to the present invention compares a captured image storage unit that stores two or more captured images output from the imaging unit with a plurality of captured images stored in the captured image storage unit, and A stability determination unit that determines a small amount of change, and a background that generates a background image using the captured image stored in the captured image storage unit when the stability determination unit determines that the image is stable continuously a predetermined number of times. An image generation unit, and the background image storage unit stores the background image.
[0015]
The moving body detection system according to the present invention includes a copy storage unit that stores an image stored in the background image storage unit when the image stored in the background image storage unit is updated, and the background image generation unit captures the image used for the determination in the stability determination unit. One of the images and the image stored in the copy storage means are combined to generate a new background image.
[0016]
DETAILED DESCRIPTION OF THE INVENTION
An embodiment of the present invention will be described below.
Embodiment 1 FIG.
FIG. 1 is a system configuration diagram showing the configuration of a moving object detection system according to Embodiment 1 of the present invention. This moving body detection system is used to detect the traffic volume and moving speed of an automobile traveling on a road. In the figure, 1 is a monitoring ITV (industrial television) camera (imaging means) that captures a predetermined monitoring space at a constant cycle and outputs a digital captured image, and 2 receives this captured image. An image processing unit 3 that performs various processes based on the captured image is a vehicle monitoring center to which a processing result of the image processing unit 2 and a captured image are input.
[0017]
In the image processing unit 2, 4 is a captured image storage means for storing a captured image input from the monitoring ITV camera 1, and 5 is a background image of a captured image captured by the monitoring ITV camera 1 in the past or an image obtained by processing the captured image. The background image storage means 6 stores the relative luminance difference with the surrounding pixels for each pixel of the captured image and the background image, cumulatively adds the relative luminance difference for each pixel, and the added value Vehicle detection means (image difference calculation means, determination means) for calculating and outputting vehicle detection when the difference between the image difference signals is larger than a predetermined threshold. .
[0018]
FIG. 2 is an explanatory diagram showing an example of a captured image of the monitoring ITV camera 1 according to the first embodiment of the present invention. In the figure, 7 is a captured image, 8 is a road left end, 9 is a road right end, 10 is a center line, 11 is a vehicle, 12 is a moving object detection area, and 13 is a detection area. The vehicle 11 moves on the left side of the road along the center line 10 in the direction of the arrow, and the monitoring ITV camera 1 moves on the road obliquely from above (when the moving direction of the vehicle 11 is horizontal). Is taken at a depression angle greater than 0 degrees and less than 90 degrees. The moving body detection area 12 is set on the side where the vehicle 11 of the captured image 7 enters the imaging space.
[0019]
Next, the operation will be described.
When a captured image is output from the monitoring ITV camera 1, the captured image is temporarily stored in the captured image storage unit 4. The background image storage means 5 stores in advance a background image in a situation where the vehicle 11 does not exist in the monitoring space shown in FIG. As the background image, a captured image in a situation where the vehicle 11 does not actually exist may be used as it is, or an image processed so as to delete the vehicle 11 from the captured image in a situation where the vehicle 11 exists may be used.
[0020]
When the captured image is stored in the captured image storage unit 4 as described above, the vehicle detection unit 6 performs detection determination of the vehicle 11 according to the flowchart shown in FIG. 3 and transmits the determination result and the captured image to the vehicle monitoring center 3. To do. In the figure, ST1 calculates a relative luminance difference with the surrounding pixels for each pixel for each of the captured image and the background image, and adds the relative luminance difference for each pixel over the entire moving object detection area 12. Further, a non-detection calculation step for calculating the difference between the addition values as an image difference signal, ST2 is a detection determination step for determining whether or not the value of the image difference signal is larger than a predetermined threshold A1, and ST3 is for the vehicle 11 Detection determination output step for outputting a detection result, ST4 is a detection time calculation step for calculating an image difference signal by the same processing as the non-detection time calculation step ST1, and ST5 is a value of the image difference signal smaller than a predetermined threshold A2. ST6 is a non-detection determination output step for outputting the detection result of the vehicle 11. Therefore, when the vehicle 11 is imaged, a processing result indicating vehicle detection continues to be output during that period.
[0021]
FIG. 4 is an explanatory diagram for explaining the relative luminance difference calculation processing according to the first embodiment of the present invention. In the figure, (a) is a horizontal edge detection filter, (b) is a vertical edge detection filter, and (c) is an image example. In these figures, each ridge corresponds to each pixel, the numerical values described in each ridge in (a) and (b) are weighting coefficients, and the pixel corresponding to the center is the target pixel. . In (c), the value described in each box is the luminance of each pixel (a numerical value from 0 to 255 that becomes whiter as the value increases). By the way, the image shown in (c) is an image in which the right half is whitish and the left half is dark, and the edges of the image serving as the boundary between these regions are present vertically. Note that the horizontal edge detection filter and the vertical edge detection filter shown in the figure are examples, and other means (other filters) may be used as long as they have the same edge detection function.
[0022]
For example, when the lateral edge detection filter of (a) is applied to the pixel group 14 of (c), the calculation formula is the following formula 1, and the value of “248” is obtained as the lateral edge strength of the target pixel. can get. Similarly, when the vertical edge detection filter of (b) is applied to the pixel group 14 of (c), the calculation formula thereof is the following formula 2, and a value of “590” is obtained as the vertical edge strength of the target pixel. It is done. Further, the relative luminance difference of the target pixel is obtained by the following equation 3 and has a value of “838”.
[0023]
Similarly, for the pixel group 15, the horizontal edge strength of the pixel of interest is “26” based on the following equation 4 and the vertical edge strength of the pixel of interest is “8” based on the following equation 5: The relative luminance difference is “34”. As is apparent from comparing the two target pixels with the balance of the luminance values of the entire image, this relative luminance difference becomes larger at the edge portion of the image such as the vehicle 11 or the boundary between the vehicle 11 and the road, There is a tendency for the value to be smaller in a part where there is no pattern such as a road surface.
[0024]
The vehicle detection means 6 adds the relative luminance difference for each pixel over the entire moving body detection area 12, and determines that the vehicle is detected when the difference between the addition values is larger than a predetermined threshold value. And output. Therefore, for example, when the vehicle 11 enters and as a result, the relative luminance difference increases in some pixels, and this can be detected if the accumulated value exceeds a predetermined threshold.
[0025]
On the other hand, when a part of the captured image becomes bright due to the light of the headlight of the vehicle 11 coming from the opposite direction, the image part and the surrounding area are gradually brightened. The relative luminance difference is a relatively small value, and even if it is accumulated over the entire moving object detection region 12, the threshold value is not exceeded, and the headlight light of the vehicle 11 is erroneously determined as the vehicle 11 There is no end to it.
[0026]

[0027]
As described above, according to the first embodiment, in detecting the vehicle 11 by comparing the captured image and the background image, the vehicle detection means 6 uses the surrounding pixels for each pixel of the captured image and the background image. The relative luminance difference between each pixel is cumulatively added to each other, the difference between the addition values is calculated as an image difference signal, and the difference between the image difference signals is greater than a predetermined threshold value. If it is larger, it is determined that the vehicle is detected and output. Therefore, the captured image and the background image are compared for each region with a relative luminance difference in each image, and the light of the headlight of the vehicle 11 is compared. Even if there is a change in the brightness of the image due to the above, the captured image and the background image can be compared in a state in which the change in the brightness is suppressed, and the vehicle 11 can be suitably detected without being affected by this. Is
[0028]
Embodiment 2. FIG.
FIG. 5 is a system configuration diagram showing the configuration of the moving object detection system according to the second embodiment of the present invention. In the figure, 16 is a vehicle determination means for extracting an image of the vehicle 11 as a feature point using a captured image at a timing when the vehicle detection means 6 determines that the vehicle is not detected, and for determining the height of the feature point from the road surface ( Feature point extraction means, size determination means). In addition, as this feature point, two tail lamps, a daytime outdoor license plate, etc. are mentioned, for example.
[0029]
FIG. 6 is an explanatory diagram showing an example of a captured image of the monitoring ITV camera 1 according to the second embodiment of the present invention. In the figure, (a) is a captured image at time t (0), (b) is time t (1) (= t (0) + Δt: where Δt is one period of the imaging period of the monitoring ITV camera 1). , 17 is a moving body determination area set at a position adjacent to the moving body detection area 12 (downstream in the traveling direction of the vehicle 11), and 18 is a downstream edge of the moving body detection area 12 in the vehicle movement direction. , 19 are tail lamps of the vehicle 11, respectively. The other configuration is the same as that of the first embodiment, and the description is omitted.
[0030]
Next, the operation will be described.
In the captured image shown in FIG. 6 (a), a processing result indicating vehicle detection is output from the vehicle detection means 6, and when this changes to the captured image shown in FIG. 6 (b), the processing result output from the vehicle detection means 6 is output. Changes from vehicle detection to vehicle non-detection. The vehicle determination means 16 uses the moving object determination area 17 of the captured image (FIG. 6B) at the timing when the output processing result of the vehicle detection means 6 changes to vehicle non-detection to determine the feature points of the vehicle 11. While extracting, the height from the road surface of the feature point is determined.
[0031]
FIG. 7 is a flowchart showing the process of the vehicle determination means 16 according to the second embodiment of the present invention. In the figure, ST7 assumes that the pixels constituting the captured image are arranged in a matrix, and the relative luminance difference of each pixel in the moving body determination region 17 and the luminance change value (=) of the background image. ST8 is a column-by-column cumulative addition step of cumulatively adding for each column the luminance in the captured image used for the determination—the luminance in the background image. ST8 is a group of columns (1 to a plurality of columns) in which the cumulative addition value of each column is particularly larger than the surrounding columns. ST9 is a column extraction step for extracting a set of columns, and a group of pixels (hereinafter referred to as a small region) in which each group of columns is returned to the current image within each range and the relative luminance difference is higher than that of other portions. A cutout step ST10 for cutting out as a template specifies the height of the feature point from the road surface based on the distance on the captured image between the small region and the moving object detection region 12. The height determination step.
[0032]
In such processing, a group of pixels having a relative luminance difference larger than that of other pixels is extracted as a small region in the column where the cumulative addition value is large, as in the second embodiment. When the vehicle 11 is imaged from behind, a small region having a shape along the contour of, for example, a tail lamp where the imaging luminance is brightest can be extracted.
[0033]
FIG. 8 is an explanatory diagram showing the relationship between the current image and the cumulative addition value for each column of luminance change values according to Embodiment 2 of the present invention. 6A is the same image as FIG. 6B, and FIG. 4B is a cumulative addition value graph for each column of the luminance change values at that time. In FIG. 5B, the horizontal axis represents the column on the image, and the vertical axis represents the cumulative added value. As shown in the figure, when the vehicle is imaged from the rear, the cumulative addition value for each column of the moving body determination region 17 is relative to the luminance of the column including the two tail lamps in the image relative to the luminance of the other columns. In FIG. 7, the feature points of the vehicle are extracted using this property. Further, as shown in the figure, the range of the column in which the cumulative addition value of the luminance change value occurs (takes a value greater than 0) substantially matches the vehicle width of the vehicle. Therefore, it is possible to determine the vehicle width, and hence the vehicle type, based on the number of columns in which the cumulative addition value has occurred.
[0034]
FIG. 9 is an explanatory diagram for explaining the reason why the height of the small region from the road surface can be specified based on the distance on the captured image between the small region and the moving object detection region 12. The captured image corresponds to the captured image in FIG. In the figure, 20 is the road surface on which the vehicle 11 travels, 21 is the visual field direction that is the downstream edge 18 in the vehicle movement direction in the captured image, 22 is the visual field direction of the feature point (tail lamp 19), and 23 is the moving object detection region 12. Is the road surface position corresponding to.
[0035]
As shown in FIG. 6, the imaging timing in FIG. 6B corresponds to the timing at which the rear end of the vehicle 11 has passed the road surface position 23. At this timing, the position directly above the downstream side edge 18 in the vehicle movement direction. It can be considered that the feature point (tail lamp 19) is located in Therefore, the distance between the downstream side edge 18 in the vehicle movement direction on the captured image and the feature point (tail lamp 19) is converted to an angle θ, and the height h of the feature point from the road surface 20 is obtained based on θ. it can. Other operations are the same as those in the first embodiment, and a description thereof will be omitted.
[0036]
As described above, according to the second embodiment, the vehicle 11 moving along one direction is imaged by the monitoring ITV camera 1, and the vehicle determination means 16 has passed the moving object detection area 12 by the vehicle 11. Since a small region with the highest luminance behind the vehicle 11, that is, the tail lamp 19, is extracted as a feature point using the timing captured image, the feature is obtained when the vehicle 11 is reliably present in the moving object determination region 17. The point extraction process can be performed, and the feature point of the vehicle 11 can be extracted with certainty.
[0037]
According to the second embodiment, the monitoring ITV camera 1 is installed so as to take an image at an angle other than 0 degrees or 90 degrees with respect to the direction along the moving direction of the vehicle 11, and the vehicle determination means 16 Since the height of the feature point from the road surface 20 is determined on the basis of the image distance between the feature point extracted by and the downstream edge 18 in the vehicle movement direction of the moving object detection region 12, the feature point (tail lamp) of the vehicle 11 is determined. The size of the vehicle 11 can be determined based on the height of 19). Further, it is possible to determine the vehicle width and consequently the vehicle type based on the number of columns in which the cumulative addition value has occurred.
[0038]
Embodiment 3 FIG.
FIG. 10 is a system configuration diagram showing the configuration of a moving object detection system according to Embodiment 3 of the present invention. In the figure, reference numeral 24 denotes a background image generation unit that receives a captured image stored in the captured image storage unit 4, generates a background image based on the input image, and outputs the background image to the background image storage unit 5.
[0039]
FIG. 11 is a system configuration diagram showing the configuration of the background image generation unit 24 according to the third embodiment of the present invention. In the figure, reference numeral 25 denotes a latest image buffer (captured image storage means) for storing a captured image from the captured image storage means 4 as a latest image, and reference numeral 26 denotes an image output by the latest image buffer 25 one previous time (one frame before). The latest image and the immediately preceding image are input to the immediately preceding image buffer (captured image storage means) 27 for storing the image of the previous image) as the immediately preceding image, and the luminance difference between these two images is lower than a predetermined threshold value for each pixel. A stability determination unit that outputs luminance stability information indicating a period in which the luminance difference is smaller than the threshold, and 28 is a copy of the background image stored in the background image storage unit 5 The copy storage means 29 for storing resets the luminance stability information for each pixel and displays it in the background image of the pixel when the period indicated by the luminance stability information exceeds a predetermined period. A background image generation means for replacing a kick luminance value to the luminance value of the latest image. The other configuration is the same as that of the second embodiment, and the description is omitted.
[0040]
Next, the operation will be described.
When the latest captured image output from the monitoring ITV camera 1 is stored in the captured image storage unit 4, the same image is stored in the latest image buffer 25, and the latest image buffer 25 includes the latest image buffer 25. The immediately previous image stored immediately before is stored. In the third embodiment, the storage cycle of the images in these buffers is described as being the same as the output cycle of the captured image output from the monitoring ITV camera 1, but the storage cycle is such that the captured image is appropriately thinned out. May be. Further, it is assumed that the same background image is stored in the background image storage unit 5 and the copy storage unit 28.
[0041]
FIG. 12 is a flowchart showing background image update processing that is performed in a state where two captured images with different imaging times are stored in the latest image buffer 25 and the immediately preceding image buffer 26 as described above. In the figure, ST11 is a luminance difference in which the stability determination means 27 determines whether or not the luminance difference between the luminance in the latest image and the luminance in the immediately preceding image is smaller than a predetermined determination value for a certain coordinate (pixel) in the captured image. A determination step, ST12 is a period addition step in which "1" is added to the period in which the stability determination means 27 is included in the luminance stability information, and ST13 is a reset in which the period in which the stability determination means 27 is included in the luminance stability information is "0" ST14 is a period reset step in which the background image generation means 29 determines whether or not this period is longer than a predetermined stability determination period, and ST15 is a step in which the background image generation means 29 determines the coordinates ( A luminance update step of changing the luminance of the pixel) to the luminance of the latest image; ST16 is a step in which the background image generation means 29 sets the coordinates (pixels) of the background image Brightness non-update step in degrees as, ST17 is a repeat control step of repeatedly executing the processing for all the coordinates of the background image (pixel).
[0042]
When the processing is completed for all the pixels, a new background image is stored in the background image storage unit 5, and this new background image is copied and stored in the copy storage unit 28. . Other operations are the same as those in the third embodiment, and a description thereof will be omitted.
[0043]
As described above, according to the third embodiment, the two latest captured images 25 and 26 are stored in the latest image buffer 25 and the immediately preceding image buffer 26, and the stability determination unit 27 determines the luminance for each pixel. The stability is determined based on the amount of change, and if the period determined to be stable is equal to or longer than a predetermined period, the background image generation unit 29 generates a new background image. it can. Moreover, since a background image is generated using a captured image that has hardly changed in a predetermined period, a background image can be appropriately generated using an image that is unlikely to include the vehicle 11, and Even if it is included, it can be deleted from the image after a predetermined period of time, so that it will not continue to detect erroneous moving objects. Therefore, it is possible to perform monitoring for 24 hours 365 days using an optimal background image that follows changes in the environment.
[0044]
According to the third embodiment, when the image stored in the background image storage means 5 is updated, the copy storage means 28 for storing the image is provided, and the background image generation means 29 is provided for each pixel based on the stability determination information. Each time, one of the brightness of the latest image and the brightness of the background image is selected and combined to generate a new background image, so the captured image or the current background image itself includes the vehicle 11 and the like. Even so, it is possible to synthesize an image so as to reduce the influence thereof and to use it as a background image. Therefore, there is an effect that it is possible to suppress erroneous detection of the vehicle 11 or the like included in the background image even when the captured image is simply used or compared with the conventional background image update processing.
[0045]
Embodiment 4 FIG.
FIG. 13 is a system configuration diagram showing the configuration of a moving object detection system according to Embodiment 4 of the present invention. In the figure, reference numeral 30 denotes a vehicle tracking based on the feature point of the subsequent captured image after obtaining the feature point of the vehicle 11 and its height from the road surface 20 by performing the same processing as the vehicle determination means 16. Vehicle tracking means (feature point extraction means, size determination means, tracking means) that performs processing. The other configuration is the same as that of the third embodiment, and the description is omitted.
[0046]
Next, the operation will be described.
FIG. 14 is a flowchart showing the processing of the vehicle tracking means 30 according to the fourth embodiment of the present invention. In the figure, ST18 also performs processing from column-by-column cumulative addition step ST7 to cutout step ST9 for the subsequent captured images, and compares each small region obtained as a result with the template, for example, in a similar shape. A destination determination step for determining the destination of the vehicle 11 by matching determination such as presumed to be present, ST19 is based on the position change on the image of the small area determined to match the same template in the captured images that follow each other. This is a movement speed determination step for determining the movement speed of the vehicle 11 between the images. The moving speed information is output to the vehicle monitoring center 3 together with the height information of the feature points from the road surface 20 and the like.
[0047]
Further, the moving speed can be obtained by the following procedure, for example. First, the position on the road surface 20 at each imaging timing of the rear part of the vehicle 11 is specified using the imaging position of the small area in each image and the height information of the vehicle 11. Next, the difference between the positions of the rear part of the vehicle in the two captured images is obtained as a movement distance, and further divided by the imaging time interval between the two images (this can be specified in advance based on the imaging period of the monitoring ITV camera 1). By doing so, the moving speed can be obtained. Other operations are the same as those in the third embodiment, and a description thereof will be omitted.
[0048]
The movement of the vehicle 11 is captured as the movement of the feature point in this way, and the height of the feature point from the road surface 20 is specified in advance, so that the vehicle 11 in the two images used for the speed calculation is identified. Since the position on the road surface 20 can be specified with high accuracy, the moving speed of the vehicle 11 can be determined with high accuracy.
[0049]
Further, since the feature points are extracted in this way, the movement of the vehicle 11 can be reliably captured without being affected by the change in the state of occurrence of the shadow as in the prior art. Even if it moves in the lateral direction, the movement distance can be reliably captured.
[0050]
Furthermore, since the vehicle 11 is captured from behind and a portion with high luminance is extracted as a feature point, the tail lamps 19 and 19 that are always provided at the rear of the vehicle 11 can be extracted as feature points. Since at least two feature points can be extracted in this way, even if one of the feature points is hidden by a pillar, a telephone pole, a road sign, another moving object, etc., other feature points There is an effect that the pattern matching determination is performed based on the vehicle 11 and the vehicle 11 can be continuously tracked.
[0051]
Accordingly, by using the tracking position information of the vehicle 11 based on the position change of the feature point on the image, the brightness change due to the light of the headlight, the change in the generation state of the shadow according to the position of the sun, the position of the night illumination There is an effect that the moving distance and moving speed of the vehicle 11 can be obtained with high accuracy without being affected by the change in the generation state of the shadow according to the above.
[0052]
Embodiment 5 FIG.
FIG. 15 is a system configuration diagram showing the configuration of a moving object detection system according to Embodiment 5 of the present invention. In the figure, after performing the same processing as that of the vehicle tracking means 30 of the fourth embodiment, if there is a template that could not identify a matching small region in the template matching determination, 31 is included in the template. Vehicle tracking means (feature point extraction means, size determination means, tracking means, estimation means) for estimating the position of a small area that will match. The other configuration is the same as that of the fourth embodiment, and the description is omitted.
[0053]
Next, the operation will be described.
FIG. 16 is a flowchart showing the processing of the vehicle tracking means 31 according to the fifth embodiment of the present invention. In the figure, ST20 is an estimation step for estimating the position of a small region that will match the template when there is a template that could not be identified in the template matching determination. Then, the estimated position information of the estimated small area is output to the vehicle monitoring center 3 together with the information of the small area where other matching is obtained.
[0054]
Further, the position estimation of the small area can be obtained by the following procedure, for example. If there is another feature point determined to match, for example, based on the rear position information of the vehicle 11 obtained based on the other feature point, the positional relationship (interval) between the feature points on the captured image at that position ) And then the position of the feature point may be estimated using the positional relationship (interval), the position information of the rear part of the vehicle, the height information of the feature point, and the like. In the case where there is no feature point determined to match, for example, the vehicle is used by using information such as the position change of the feature point in the previous captured image and the movement speed of the vehicle 11 obtained based on the change. 11 estimates the position on the road surface of the vehicle 11 in the current captured image under an estimation condition such as linear movement between these images, and further estimates the position of each feature point at that position. It ’s fine. Other operations are the same as those in the fourth embodiment, and a description thereof will be omitted.
[0055]
As described above, according to the fifth embodiment, for the feature points that are not determined to be matched in the template matching determination, the vehicle tracking unit 31 estimates the position of the small region that will match the template. Even if all feature points are hidden by, for example, a pillar, a telephone pole, a road sign, or another moving object, the position can be estimated and the subsequent captured image can be continuously tracked. effective.
[0056]
In the above embodiment, when generating the background image, the luminance stability in the captured image of the pixel is determined for each pixel and updated using the luminance in the stable state for a predetermined period or longer. In addition, the background image may be generated in the manner shown in FIG. In the figure, 32 is the latest captured image, 33 is the background image, 34 is a portion where an instantaneous change occurs, 35 is a difference image, 36 is an inverted binarized image, and 37 is a new background image. In the figure, first, a difference image 35 between the latest captured image 32 and the background image 33 is generated, and the difference image 35 is binarized based on a predetermined threshold and further inverted to be an inverted binary image. 36, and a new background image 37 is generated by using the inverted binarized image 36 together with the latest captured image 32 and background image 33.
[0057]
However, in this background image generation method, as shown in the figure, a portion where an instantaneous luminance change occurs due to a moving body such as the vehicle 11 or light of a headlight of the vehicle 11 remains in the background image. As shown in the above embodiment, the update method that sequentially applies only the luminance determined to be stable to each pixel generates a more suitable background image that eliminates this instantaneous luminance change. There is an effect that can be done.
[0058]
【The invention's effect】
As described above, according to the present invention, a predetermined space is imaged. Shoot Imaging means for outputting an image image, background image storage means for storing a captured image output by the imaging means in the past or an image obtained by processing the image as a background image, and the captured image the above background image In the moving object detection area set on the intrusion side of the moving object, a relative luminance difference with surrounding pixels is calculated for each pixel, and the calculated relative luminance difference for each pixel is added over the entire moving object detection area. Then, the difference between the relative luminance differences for each pixel in the captured image and the background image is calculated as an image difference signal. A moving object by comparing the image difference calculating means with the image difference signal and a predetermined threshold value Inspection Since the determination means for performing the output determination is provided, the captured image and the background image are compared with the relative luminance difference in each image for each region. Therefore, even if there is a change in the brightness of the image due to the light of the headlight of the vehicle, the captured image and the background image can be compared in a state in which this change in brightness is suppressed. There exists an effect which can be detected suitably.
[0059]
According to this invention, After the judging means judges that the moving body is detected At the timing determined as non-detection, In the moving object detection area set on the downstream side of the moving object in the moving object detection area of the captured image and the background image, the luminance change values of the captured image and the background image are accumulated and added for each column. A group of columns whose added value is larger than the surrounding columns is extracted, and in the extracted column group, a group of pixels in which the relative luminance difference for each pixel in the captured image is larger than other pixels is extracted as a feature point. Since the feature point extraction means is provided, the feature point extraction process can be performed in a state where the moving object exists in the moving object determination area, and the feature point of the moving object can be reliably extracted.
[0060]
According to this invention, Direction along the moving direction of the moving object The image pickup means is installed so as to have an image pickup angle other than 0 degree or 90 degrees with respect to the feature point and the moving object detection area extracted by the feature point extraction means The downstream side of the moving body in the moving direction Since the size determination means for determining the size of the moving body based on the distance on the image is provided, the type of the moving body can be determined based on the size of the moving body.
[0061]
According to the present invention, the feature point extracting means for extracting a plurality of feature points of the moving body, and the feature point extracting means after extracting the plurality of feature points of For captured images ,the above Perform the same process as the feature point extraction means Extract a small area and compare this small area with each feature point A matching decision is made and a match is found at at least one feature point If there is a small area Tracking means that outputs tracking position information of the moving object based on the position on the captured image of one of the images, so that one feature point is hidden by, for example, a pillar, a telephone pole, a road sign, another moving object, etc. Even so, there is an effect that the moving body can be continuously tracked based on other feature points.
[0062]
In addition, since multiple feature points are extracted based on the relative luminance difference, the size of the vehicle can be determined based on the positional relationship between them without being affected by the light from the vehicle headlights. There is also an effect that can be done.
[0063]
Furthermore, by using the tracking position information of the moving object based on the position change of the feature point on the image, the brightness change due to the light of the headlight, the change in the state of occurrence of the shadow according to the position of the sun, the position of the night illumination Thus, there is an effect that the moving distance and moving speed of the moving body can be obtained without being affected by the change in the generation state of the shadow according to the above.
[0064]
According to this invention, the feature point extracting means for extracting one or more feature points of the moving body;
After the feature point extraction means extracts the feature points of For captured images ,the above Perform the same process as the feature point extraction means Extract a small area and compare this small area with each feature point A matching decision is made and a match is found at at least one feature point If there is a small area Tracking means for outputting tracking position information of the moving body based on the position on the captured image of the above For a feature point that could not be determined to be matched by the tracking means, if there is a previous trajectory of the feature point or another feature point determined to match, an estimation means that outputs estimated position information using the feature point Since it is provided, even if all feature points are concealed by, for example, pillars, telephone poles, road signs, other moving bodies, etc., there is an effect that the position can be estimated and continuously tracked.
[0065]
In addition, since multiple feature points are extracted based on the relative luminance difference, the size of the vehicle can be determined based on the positional relationship between them without being affected by the light from the vehicle headlights. There is also an effect that can be done.
[0066]
Furthermore, by using the tracking position information of the moving object based on the position change of the feature point on the image, the brightness change due to the light of the headlight, the change in the state of occurrence of the shadow according to the position of the sun, the position of the night illumination Thus, there is an effect that the moving distance and moving speed of the moving body can be obtained without being affected by the change in the generation state of the shadow according to the above.
[0067]
For example, the moving body is a vehicle, and the feature point extracting unit may extract the tail lamp of the vehicle as a feature point.
[0068]
According to the present invention, the captured image storage unit that stores two or more captured images output from the imaging unit and the plurality of captured images stored in the captured image storage unit are compared, and the change in the image is small. And a background image generating means for generating a background image using a captured image stored in the captured image storage means when the stability determining means determines that the stability is continuously determined a predetermined number of times. Since the background image storage means stores the background image, the background image can be updated sequentially.
[0069]
In addition, since a background image is generated using a captured image that has hardly changed in a period corresponding to a predetermined number of times, it is possible to appropriately generate a background image using an image that is unlikely to include a moving object or the like. In addition, even if it is included, it is updated after a predetermined period of time, so that erroneous moving object detection is not continued. Therefore, it is possible to perform monitoring for 24 hours 365 days using an optimal background image that follows changes in the environment.
[0070]
According to the present invention, the copy storage means for storing the image when the image stored in the background image storage means is updated is provided, and the background image generation means includes the captured image used for the determination in the stability determination means. Since a new background image is generated by synthesizing one image and the image stored in the copy storage unit, even if a moving object is included in the captured image or the current background image, the influence is reduced. An image can be synthesized with the image as a background image. Therefore, compared to a case where a captured image is simply used, there is an effect that it is possible to suppress erroneous detection of a moving body included in a background image.
[Brief description of the drawings]
FIG. 1 is a system configuration diagram showing a configuration of a moving object detection system according to a first embodiment of the present invention.
FIG. 2 is an explanatory diagram showing an example of a captured image of a monitoring ITV camera according to Embodiment 1 of the present invention.
FIG. 3 is a flowchart showing both detection flows according to the first embodiment of the present invention.
FIG. 4 is an explanatory diagram for explaining a relative luminance difference calculation process according to Embodiment 1 of the present invention;
FIG. 5 is a system configuration diagram showing a configuration of a moving object detection system according to a second embodiment of the present invention.
FIG. 6 is an explanatory diagram showing an example of a captured image of a monitoring ITV camera according to Embodiment 2 of the present invention.
FIG. 7 is a flowchart showing processing of a vehicle determination unit according to Embodiment 2 of the present invention.
FIG. 8 is an explanatory diagram showing a relationship between a current image and a cumulative addition value for each column of luminance change values according to Embodiment 2 of the present invention;
FIG. 9 is an explanatory diagram for explaining the reason why the height of the small region from the road surface can be specified based on the distance on the captured image between the small region and the vehicle detection region.
FIG. 10 is a system configuration diagram showing a configuration of a moving object detection system according to a third embodiment of the present invention.
FIG. 11 is a system configuration diagram showing a configuration of a background image generation unit according to Embodiment 3 of the present invention.
FIG. 12 is a flowchart showing background image update processing according to Embodiment 3 of the present invention;
FIG. 13 is a system configuration diagram showing a configuration of a moving object detection system according to a fourth embodiment of the present invention.
FIG. 14 is a flowchart showing processing of a vehicle tracking unit according to Embodiment 4 of the present invention.
FIG. 15 is a system configuration diagram showing a configuration of a moving object detection system according to a fifth embodiment of the present invention.
FIG. 16 is a flowchart showing processing of a vehicle tracking unit according to Embodiment 5 of the present invention.
FIG. 17 is a diagram showing another background image generation process.
FIG. 18 is a system configuration diagram showing a configuration of a conventional mobile object detection system.
FIG. 19 is an example of an image captured by an imaging camera in a conventional moving body detection system.
[Explanation of symbols]
1 monitoring ITV camera (imaging means), 2 image processing unit, 3 vehicle monitoring center, 4 captured image storage means, 5 background image storage means, 6 vehicle detection means (image difference calculation means, determination means), 7 captured image, 8 Road left edge, 9 Road right edge, 10 Center line, 11 Vehicle, 12 Moving object detection area, 13 Detection area, 14, 15 Pixel group, 16 Vehicle determination means (feature point extraction means, size determination means), DESCRIPTION OF SYMBOLS 17 Moving body determination area | region, 18 Vehicle moving direction downstream edge, 19 Tail lamp, 20 Road surface, 21 Visual field direction, 22 Feature point visual field direction, 23 Road surface position equivalent to moving body detection area, 24 Background image generation part, 25 Latest image buffer (captured image storage means), 26 Immediately preceding image buffer (captured image storage means), 27 Stability determination means, 28 Copy storage means, 29 Background image generation means 30 vehicle tracking means (feature point extraction means, size determination means, tracking means), 31 vehicle tracking means (feature point extraction means, size determination means, tracking means, estimation means), 32 latest captured image, 33 background image, 34 A portion where an instantaneous change occurs, 35 Difference image, 36 Inverted binarized image, 37 New background image.

Claims (8)

Imaging means for outputting a captured IMAGING images a predetermined space,
A background image storage unit that stores a captured image output by the imaging unit in the past or an image obtained by processing the image as a background image;
In the moving object detection areas set in intruder moving body of the captured image and the background image, and calculates the relative brightness difference between the surrounding pixels for each pixel, the moving body relative luminance difference for each calculated pixel Image difference calculating means for performing addition over the entire detection area and calculating a difference between the values of relative luminance differences for each pixel in the captured image and the background image as an image difference signal ;
Moving object detecting system comprising a determining means for performing detection determination of the moving body by comparing the image difference signal with a predetermined threshold value. In the moving body determination area set downstream of the moving body in the moving body detection area of the captured image and the background image at the timing when the determination means determines that the moving body is detected and then non-detection , the above imaging is performed. The luminance change values of the image and the background image are cumulatively added for each column, a group of columns in which the cumulative addition value is larger than the surrounding columns is extracted, and the relative value for each pixel in the captured image is extracted in the group of extracted columns. 2. The moving object detection system according to claim 1, further comprising a feature point extraction unit that extracts a group of pixels having a luminance difference larger than that of other pixels as a feature point. While installing the imaging means so as to have an imaging angle other than 0 degrees or 90 degrees with respect to the direction along the moving direction of the moving body ,
A size determination unit is provided that determines the size of the moving object based on the distance on the image between the feature point extracted by the feature point extracting unit and the downstream edge of the moving object detection region in the moving object moving direction. The moving body detection system according to claim 2. Feature point extracting means for extracting a plurality of feature points of the moving body;
The captured image after the feature point extracting means has extracted a plurality of feature points, and performs the same processing as the feature point extraction means extracts a small area, compared with the respective feature points and the small region performs matching determination Te, when the small area that is determined to match at least one feature point is present, and a tracking means for outputting track position information of the moving object based on the position on the captured image of the small area The mobile body detection system according to claim 1, wherein the mobile body detection system is provided. Feature point extraction means for extracting one or more feature points of the moving body;
The captured image after the feature point extracting means has extracted feature points, and by performing the same processing as the feature point extraction means extracts a small area, compared with the respective feature points and the small region matching a judgment, when the small area that is determined to match at least one feature point is present, a tracking means for outputting a tracking positional information of the movable body based on the position on the captured image of the small area,
For the feature points that could not be determined to match the tracking means, if there is a previous trajectory of the feature points or other feature points determined to match, an estimation means that outputs estimated position information using it The moving body detection system according to claim 1, wherein:   The mobile body detection system according to any one of claims 2 to 5, wherein the mobile body is a vehicle, and the feature point extraction unit extracts a tail lamp of the vehicle as a feature point. Captured image storage means for storing two or more captured images output from the imaging means;
A stability determination unit that compares a plurality of captured images stored in the captured image storage unit and determines a small change in the image;
A background image generating means for generating a background image using a captured image stored in the captured image storage means when the stability determining means determines that the image is stable continuously a predetermined number of times;
2. The moving body detection system according to claim 1, wherein the background image storage means stores the background image. When the image stored in the background image storage means is updated, a copy storage means for storing the image is provided,
The background image generation means generates one new background image by combining one of the captured images used for the determination by the stability determination means and the image stored in the copy storage means. The moving body detection system according to claim 7.

Images