JP3828096B2 - Object tracking device - Google Patents

Description

【００６１】
例えば、テンプレート更新処理（ステップＳ１３）の実行時におけるＴＶカメラ１１の撮像レンズ１２の焦点距離ｆ’が２０ｍｍであり、現在における焦点距離ｆが２４ｍｍであるとすると、ｒ＝２４／２０＝１．２となり、撮像レンズ１２の焦点距離の変化によって対象物体の見かけの大きさが１．２倍になったことになる。すなわち、拡大前と拡大後とでテンプレートの中心位置８２、８４を一致させるようにして、テンプレート８１を１．２倍し、この結果を新たなテンプレート８３にすれば、入力画像中の侵入物体の大きさとテンプレートの侵入物体の大きさとを一致させることができる。
【００６２】
なお、同図の例では、Ｘ−Ｙ直交座標系において、拡大前のテンプレート８１のＸ軸方向の長さＴｘをズーム倍率ｒ倍した結果である（ｒ×Ｔｘ）の長さを拡大後のテンプレート８３のＸ軸方向の長さとしており、同様に、拡大前のテンプレート８１のＹ軸方向の長さＴｙをズーム倍率ｒ倍した結果である（ｒ×Ｔｙ）の長さを拡大後のテンプレート８３のＹ軸方向の長さとしている。また、これに際して、テンプレート８１、８３の中心位置８２、８４が不変であるようにしている。
【００６３】
また、上記した侵入物体の検出処理Ｔ１で侵入物体を検出した直後においては、テンプレート更新処理（ステップＳ１３）が実行されていないことから、テンプレート更新時におけるＴＶカメラ１１の撮像レンズ１２の焦点距離ｆ’が取得されていないため、この場合には、テンプレートの拡大・縮小処理（ステップＳ１０）は実行されない。
【００６４】
また、本例のようにテンプレート拡大・縮小処理（ステップＳ１０）が行われる場合には、テンプレート更新処理（ステップＳ１３）では、処理の実行時に、ワークメモリ２７に記録した焦点距離ｆ’を現在におけるＴＶカメラ１１の撮像レンズ１２の焦点距離ｆを用いて更新するようにする。
【００６５】
ここで、上記図３及び図５を参照して、上記した侵入物体の移動量の検出処理について具体的に説明する。
上記図３には、切り出し器７３を示してある。また、同図（ｅ）には、テンプレート画像５５を示してある。
【００６６】
そして、入力画像５１中に映る侵入物体は、上記したラベリング処理（ステップＳ５）により二値化画像内の画素値“２５５”のかたまりとして得られた侵入物体６３の外接矩形６４に基づいて切り出し器７３で切り出され、これにより、テンプレート画像５５が得られる。テンプレート画像５５中には、侵入物体６１のテンプレート６５が含まれており、これがテンプレートマッチング法による侵入物体の移動量の検出処理における初期のテンプレートとなる。次いで、当該初期テンプレートに基づいて、テンプレートマッチングが実行される。
【００６７】
図５には、テンプレートマッチング法による侵入物体の移動量の検出処理を逐次実行して侵入物体を追跡する処理の流れの一例を示してある。
同図では、テンプレート画像９１を取得した時刻をｔ０と表し、所定の時間間隔（例えば、１００ｍｓ）で逐次入力する入力画像の取得時刻を、入力順に（ｔ０＋１）、（ｔ０＋２）、・・・と表す。
【００６８】
同図（ａ）には、時刻ｔ０におけるテンプレート画像９１を示してある。当該テンプレート画像９１中に、時刻ｔ０におけるテンプレート１０１がある。なお、これらは、それぞれ、上記図３（ｅ）に示したテンプレート画像５５、テンプレート６５と同一のものである。
【００６９】
同図（ｂ）には、時刻（ｔ０＋１）における入力画像９２を示してある。当該入力画像９２において、矩形領域１０２は時刻ｔ０における侵入物体の位置（テンプレート１０１の位置）を表しており、矩形領域１０３はテンプレートマッチングの対象となる領域（探索領域）を表している。
なお、図５（ｂ）では説明を簡単にするため、前述の対象物体の移動量予測に基づいた探索領域の設定方法は用いていないが、移動量予測を用いた場合でも同様に侵入物体の追跡が行える。
【００７０】
そして、テンプレートマッチング処理（ステップＳ１１）１４１を実行すると、テンプレートマッチングの探索領域 １０３の中でテンプレート１０１に最も一致する画像１０４で一致度が最も大きくなり、侵入物体は時刻（ｔ０＋１）において画像１０４の位置に存在していることが分かる。この位置は、時刻ｔ０におけるテンプレート１０１の位置（ｘ０、ｙ０）から見て相対位置（Δｘ、Δｙ）で表される。すなわち、侵入物体は矢印１０５により示される分だけ移動したことが分かる。
【００７１】
そこで、テンプレート更新処理（ステップＳ１３）１４２により、テンプレート１０１に最も一致した画像１０４を時刻（ｔ０＋１）における新たなテンプレートとして更新する。すなわち、同図（ｃ）に示されるように、入力画像９２から侵入物体の位置１０４を切り出し、これをテンプレート画像９３とし、侵入物体の画像１０４を時刻（ｔ０＋１）における新たなテンプレート１１１として更新する。
【００７２】
この処理をＴＶカメラ１１から逐次入力される入力画像に対して適用する。
具体的には、同図（ｄ）に示されるように、時刻（ｔ０＋２）における入力画像９４中にテンプレート１１１の位置１１２に基づいて探索領域１１３を設定し、時刻（ｔ０＋１）におけるテンプレート画像９３中のテンプレート１１１を用いてテンプレートマッチング処理（ステップＳ１１）１４３によって侵入物体の位置１１４を検出する。すると、侵入物体は矢印１１５で示されるように移動したことが分かる。
【００７３】
更に、同図（ｅ）に示されるように、テンプレート更新処理（ステップＳ１３）１４４によって、時刻（ｔ０＋２）におけるテンプレート画像９５及び侵入物体のテンプレート１２１を更新する。
また、同図（ｆ）に示されるように、時刻（ｔ０＋３）における入力画像９６中にテンプレート１２１の位置１２２に基づいて探索領域１２３を設定し、時刻（ｔ０＋２）におけるテンプレート画像９５中のテンプレート１２１を用いてテンプレートマッチング処理（ステップＳ１１）１４５によって侵入物体の位置１２４を検出する。すると、侵入物体は矢印１２５で示されるように移動したことが分かる。
【００７４】
更に、同図（ｇ）に示されるように、テンプレート更新処理（ステップＳ１３）１４６によって、時刻（ｔ０＋３）におけるテンプレート画像９７及び侵入物体のテンプレート１３１を更新する。
そして、同図（ｈ）に示されるように、時刻（ｔ０＋４）における入力画像９８中にテンプレート１３１の位置１３２に基づいて探索領域１３３を設定し、時刻（ｔ０＋３）におけるテンプレート画像９７中のテンプレート１３１を用いてテンプレートマッチング処理（ステップＳ１１）１４７によって侵入物体の位置１３４を検出する。すると、侵入物体は矢印１３５で示されるように移動したことが分かる。
このように、テンプレートマッチングを逐次実行することで、侵入物体を追跡することができる。
【００７５】
ここで、上記したテンプレートマッチング処理（ステップＳ１１）における探索領域や一致度について具体的に説明する。
上記した探索領域の範囲は、例えば、テンプレートに登録された対象物体の入力画像上での動きによって決定される。
具体例として、撮像装置１として１／３インチＣＣＤ（撮像素子サイズ ４．８ｍｍ×３．６ｍｍ）が用いられ、撮像レンズ１２の焦点距離が３２ｍｍであり、対象物体までの距離が３０ｍであるといった条件で撮像すると、ＴＶカメラ１１の横方向の視野は、３０×４．８÷３２＝４．５ｍとなる。このＴＶカメラ１１で、移動速度が時速５ｋｍ／ｈ（約１．３９ｍ／ｓ）の対象物体を、画像サイズ ３２０×２４０画素、入力間隔０．１ｓ（１００ｍｓ）で撮像すると、対象物体の入力画像毎の画像上での移動量は、横方向３２０×１．３９×０．１／４．５≒９．８８画素となる。
【００７６】
また、対象物体がＴＶカメラ１１の方向に向かって移動すると画像上での移動量も大きくなるため、実際の探索領域の範囲は、上記で算出した値の５倍程度の余裕をもって設定する。すなわち、探索領域の横方向の大きさＭｘを５０画素とする。探索領域の縦方向の大きさＭｙは、ＴＶカメラ１１の仰角に依存して、ＴＶカメラ１１の取り付け位置によって変化するため、横方向の大きさのおよそ４０％程度の値とする。従って、探索範囲は、この例では、テンプレートに対して左右Ｍｘ＝５０画素、上下Ｍｙ＝２０画素だけ広げた領域とすればよい。
【００７７】
また、一致度としては、例えば正規化相関値ｒ（Δｘ，Δｙ）を適用することができ、式２のように表される。
【００７８】
【数２】

【００７９】
ここで、ｆ（ｘ，ｙ）は入力画像を表している。また、後述する図６を参照して、ｇ（ｘ，ｙ）はテンプレート画像１５１を表しており、（ｘ０，ｙ０）はテンプレート１６１の左上の座標を表しており、Ｄはテンプレート１６１の大きさを表している。本例では、画像の座標軸としては、画像の左上を原点（０，０）としている。また、上記図３（ｄ）を参照して、Ｄは二値化画像５４で検出された侵入物体の外接矩形６４の大きさに相当し、本例では横５０画素、縦２０画素に相当する。
【００８０】
正規化相関値ｒ（Δｘ，Δｙ）は、−１≦ｒ（Δｘ，Δｙ）≦１の値を取り、入力画像とテンプレートとが全く一致した場合には、“１”となる。
テンプレートマッチングでは、Δｘ、Δｙを探索範囲内で走査させた場合に、すなわち上記した例では−Ｍｘ≦Δｘ≦Ｍｘ、−Ｍｙ≦Δｙ≦Ｍｙと変化させた場合に、正規化相関値ｒ（Δｘ，Δｙ）が最も大きくなる位置（Δｘ，Δｙ）（すなわち、対象物体の移動量）を検出する処理が行われる。
また、前述した対象物体の移動量の予測を行う場合には、Δｘ、ΔｙをΔｘ’―Ｍｘ≦Δｘ≦Δｘ’＋Ｍｘ、Δｙ’―Ｍｙ≦Δｙ≦Δｙ’＋Ｍｙと変化させる。ここで、Δｘ’、Δｙ’は前フレームの対象物体の移動量を表す。
【００８１】
次に、上記図２に示した処理手順の続きを説明する。
カメラ雲台制御処理では、上記した侵入物体の移動量の検出処理Ｔ２におけるテンプレートマッチング処理（ステップＳ１１）によって検出された侵入物体の位置と、入力画像の中心との変位に応じて、カメラ雲台１３の制御を行う（ステップＳ１４）。
【００８２】
ここで、図６を参照して、上記したカメラ雲台制御処理（ステップＳ１４）を具体的に説明する。
一例として、テンプレート画像１５１において、同図に示されるような位置１６１に侵入物体が検出されたとする。この場合、侵入物体の中心位置をテンプレートの中心１６２とすると、テンプレート画像１５１の中心１６３からのＸ軸方向の変位ｄｘ、Ｙ軸方向の変位ｄｙが算出される。
【００８３】
そして、テンプレートの中心位置１６２が入力画像の中心１６３と比べて、所定量Ｓ以上左側（ｄｘ＜−Ｓ）であればカメラ雲台１３を左に回転（パン）させ、所定量Ｓ以上右側（ｄｘ＞Ｓ）であればカメラ雲台１３を右に回転（パン）させる。また、テンプレートの中心位置１６２が入力画像の中心１６３と比べて、所定量Ｓ以上上側（ｄｙ＜−Ｓ）であればカメラ雲台１３を上に傾け（チルト）させ、所定量Ｓ以上下側（ｄｙ＞Ｓ）であれば下に傾け（チルト）させる。
【００８４】
このような所定量Ｓを用いると、侵入物体が画像の中心付近に存在する場合には、カメラ雲台１３を制御する必要がなく、このため、所定量Ｓによってカメラ雲台１３の制御を開始する侵入物体の位置を指定することができる。
なお、左、右、上、下のそれぞれについての所定量Ｓとしては、種々な値が用いられてもよく、例えば、左右上下で同一の値が用いられてもよく、或いは、左右上下でそれぞれ任意な値が用いられてもよい。
【００８５】
一例として、左右上下の所定量Ｓ＝５０という値を用いることができる。
また、例えば、所定量Ｓが小さいほど、侵入物体が少しでも中心から外れたらカメラ雲台１３が制御されてしまって画像が見づらくなってしまう可能性はあるが、上記した所定量Ｓ＝０といった値や、所定量Ｓとして小さい値を用いることも可能である。
【００８６】
また、テンプレート画像１５１の中心１６３に対する侵入物体のＸ軸方向の変位ｄｘ、Ｙ軸方向の変位ｄｙの絶対値に応じてパンや、チルトモータの制御速度を変化させるような制御を行うことも可能である。この場合、例えば、Ｘ軸方向の変位ｄｘ、或いは、Ｙ軸方向の変位ｄｙの絶対値が大きいほど、制御速度を大きくする。
【００８７】
また、本例では、侵入物体の追跡として、カメラ雲台１３の制御を伴った侵入物体の追跡を行っている。これにより、侵入物体をＴＶカメラ１１の視野内に捉えながら、カメラ雲台１３を自動的に制御して、侵入物体を追跡することができる。
【００８８】
次に、上記図２に示した処理手順の続きを説明する。
焦点距離情報取得処理では、ワークメモリ２７に記録した現在の入力画像を取得した時点の撮像レンズ１２の焦点距離ｆを取得する（ステップＳ１５）。
次に、ズーム倍率算出処理では、テンプレートマッチング処理（ステップＳ１１）で得られた侵入物体の移動量（Δｘ，Δｙ）に基づいて、式３によりズーム倍率ｒｆを算出する（ステップＳ１６）。
【００８９】
【数３】

【００９０】
上記した式３において、Ｍｘ、Ｍｙは、テンプレートマッチング法における探索範囲を表している。また、Ｓｘ、Ｓｙは、安定に追跡することが可能な侵入物体の画像上での最大移動量を表しており、例えば、探索範囲の半分程度、すなわち、上記した例ではＳｘ＝２５、Ｓｙ＝１０とする。なお、ＳｘやＳｙの値は、例えば、物体が探索範囲から外れないように探索範囲の半分程度で余裕をもたせるといった程度で、実験などにより設定される。
また、上記した式３では、侵入物体の移動量（Δｘ，Δｙ）＝（０，０）である場合には、つまり侵入物体の移動量がゼロである場合には、ズーム倍率ｒｆ＝１．５としている。
【００９１】
なお、例えば、ズーム倍率ｒｆが所定の値以上になった場合には、ズーム倍率ｒｆを当該所定の値にして、急激にズームアップしないようにすることも可能である。当該所定の値としては、例えば、１．５を用いることができる。この場合、１回のズームアップで最大５０％のズームアップまでが可能となる。
このように、１回のズームアップにおける最大のズーム倍率ｒｆ（上限値）を設定すると、例えば、画像の端近くで検出された物体はズームアップによって画像上の視野の外側に飛び出してしまうといった問題を抑制することができる。
【００９２】
また、例えば、ズーム倍率ｒｆの上限値（ＭＡＸ値）を可変とするような構成を用いることも可能である。このような構成では、例えば、画像の画面のサイズと比べてテンプレートが十分に小さい場合には、ズーム倍率ｒｆの上限値を１．５より大きい値とするようなことが可能である。
【００９３】
また、例えば、ズームアップに上限を設けるような構成を用いることも可能である。具体的には、一例として、画像の画面の高さがテンプレートの高さの１２０％以上となる範囲でしかズームアップを行わないようなことや、画像の画面の幅がテンプレートの幅の１２０％以上となる範囲でしかズームアップを行わないようなことが可能である。これにより、侵入物体の移動量（Δｘ，Δｙ）が小さくて多数回のズームアップが為されてテンプレートが画像の画面のサイズを超えてしまうようなことを防ぐことができ、画像を見易くして、安定した動作を確保することができる。
【００９４】
また、テンプレートと画像の上端、下端、左端、右端までのそれぞれの距離に基づいてズームアップに上限を設けるような構成にすることも可能である。具体的には、一例として、図７に示すようにテンプレート１７２と画像１７１の上端、下端、左端、右端までの距離をそれぞれｄｕ、ｄｂ、ｄｌ、ｄｒとし、テンプレート１７２の上辺、下辺、左辺、右辺がズームアップによって画面の外に超える倍率１２０／（１２０−ｄｕ）、１２０／（１２０−ｄｂ）、１６０／（１６０−ｄｌ）、１６０／（１６０−ｄｒ）の中で負の値を除いて最も小さい倍率をズームアップの倍率の上限とする。ここで、画像サイズとして、幅３２０画素、高さ２４０画素を想定した。
【００９５】
また、対象物体の移動量の予測を行う場合には、前フレームでの対象物体の移動量を（Δｘ’、Δｙ’）とした場合、１２０／｛１２０−（ｄｕ＋Δｙ’）｝、１２０／｛１２０−（ｄｂ−Δｙ’）｝、１６０／｛１６０−（ｄｌ＋Δｘ’）｝、１６０／｛１６０−（ｄｒ−Δｘ’）｝として、上記と同様にして、テンプレートの上辺、下辺、左辺、右辺がズームアップによって画面の外に超える倍率（ズームアップの倍率の上限）を算出する。
【００９６】
なお、例えば、テンプレート１７２と画像１７１の上端、下端までの距離ｄｕ、ｄｂのうちで短い方のみを採用して考慮してズームアップの倍率の上限を算出してもよく、また、テンプレート１７２と画像１７１の左端、右端までの距離ｄｌ、ｄｒのうちで短い方のみを採用して考慮してズームアップの倍率の上限を算出してもよい。
【００９７】
次に、撮像レンズ制御処理では、ズーム後の焦点距離ｆ×ｒｆによって、レンズ制御部２３を介して撮像レンズ１２の焦点距離をｆ×ｒｆに調整する（ステップＳ１７）。これにより、追跡する対象の侵入物体の画像上での移動速度が所定の値以下に抑えられるように、撮像レンズ１２を自動的に調節することができる。当該所定の値としては、上記した例では、横方向が２５であり、縦方向が１０である。
【００９８】
すなわち、追跡する対象の侵入物体の画像上での移動速度が所定の値以上である（又は、所定の値を超える）場合には、撮像レンズ１２の焦点距離を小さくし（つまり、ズーム倍率を小さくする、すなわちズームアウトし）、侵入物体の画像上での移動速度を所定の値未満（又は、所定の値以下）にする。また、追跡する対象の侵入物体の画像上での移動速度が所定の値未満である（又は、所定の値以下である）場合には、撮像レンズ１２の焦点距離を大きくし（つまり、ズーム倍率を大きくする、すなわちズームアップし）、侵入物体の画像上での移動速度が所定の値になるまで侵入物体をズームアップすることができる。
【００９９】
次に、警報・追跡情報表示処理では、例えば、侵入物体を追跡中であることを表す警報を監視員に伝えるため、画像出力部２９を介して監視用の画像モニタ５に侵入物体の情報を表示することや、警報出力部３０を介して警告灯６を点灯させることなどを行う（ステップＳ１８）。ここで、侵入物体の情報としては、例えば、移動量や、移動経路などの情報を用いることができる。
【０１００】
なお、本例では、カメラ雲台１３を制御する処理（ステップＳ１４）と、撮像レンズ１２を制御する処理（ステップＳ１７）の両方が行われており、上記したカメラ雲台制御に関する所定量Ｓと、ズーム倍率ｒｆの上限値などとは、例えば、互いに考慮されて実験などにより設定されるのが好ましい。これにより、例えば、カメラ雲台１３の制御とズーム倍率ｒｆによるズームアップとが同時に両方為されたような場合に、物体が画像の画面の外に出てしまうようなことを抑制することが可能である。
【０１０１】
また、本例では、上記図２に示した処理手順を用いたが、他の例として、カメラ雲台制御処理（ステップＳ１４）をズーム倍率算出処理（ステップＳ１６）と撮像レンズ制御処理（ステップＳ１７）との間に設けて実行するような処理手順を用いることも可能である。このような処理手順では、例えば、ズーム倍率算出処理（ステップＳ１６）で算出されたズーム倍率ｒｆを考慮して、ズーム後の侵入物体の位置をズーム倍率ｒｆから予測して、当該予測結果に基づいて、カメラ雲台１３の制御処理（ステップＳ１４）を行うようなことが可能である。
【０１０２】
以上のように、本例の画像監視装置では、撮像装置１によって得られる画像信号に基づいて監視対象領域内の侵入物体を検出し、当該侵入物体の画像信号上での移動方向及び移動量を検出し、当該移動方向及び当該移動量に基づいて当該撮像装置１を搭載する雲台１３及び当該撮像装置１の撮像レンズ１２を制御しながら当該侵入物体を追跡する物体追跡方法及び物体追跡装置において、当該移動量に基づいて当該撮像装置１の撮像レンズ１２を制御する。
【０１０３】
また、本例の画像監視装置では、上記のような構成において、侵入物体の画像信号上での移動量に基づいて撮像装置１の撮像レンズ１２のズーム倍率ｒｆを算出するズーム倍率算出処理（ステップＳ１６）と、当該ズーム倍率算出処理によって算出されたズーム倍率に基づいて当該撮像装置１の撮像レンズ１２を制御する撮像レンズ制御処理（ステップＳ１７）を備え、これら少なくとも２種の処理が、侵入物体の移動方向及び移動量を検出するための処理手順に備えられている。
【０１０４】
また、本例の画像監視装置では、上記のような構成において、ズーム倍率算出処理（ステップＳ１６）では、侵入物体の画像信号上での移動量を所定の値以下に抑えるようにズーム倍率ｒｆを算出する。
【０１０５】
具体的な一例に係る画像監視装置（以下で、画像監視装置Ａ１と言う）では、少なくとも監視対象とする範囲を撮像する撮像装置１と、当該撮像装置１の視野方向を変えるための当該撮像装置１を搭載する外部信号により制御可能なカメラ雲台１３と、当該撮像装置１の画角を変えるための当該撮像装置１に取り付けられた外部信号により制御可能な撮像レンズ１２と、当該撮像装置１からの画像信号をデジタルの画像信号に変換する画像入力部（インタフェース）２１と、当該画像入力部２１からの画像信号を処理する機能を有して例えば少なくともＭＰＵ２６と画像メモリ２５とプログラムメモリ４とワークメモリ２７を有する画像処理部と、当該画像処理部からのカメラ雲台１３を制御するための制御信号を供給する雲台制御部（インタフェース）２２と、当該画像処理部からの撮像レンズ１２を制御するための制御信号を供給する撮像レンズ制御部（インタフェース）２３を備える。
【０１０６】
そして、当該画像監視装置（画像監視装置Ａ１）では、画像処理部は、撮像装置１により撮像された画像信号から監視対象領域内の侵入物体を検出し、当該侵入物体の画像信号上での移動方向及び移動量を検出し、当該移動方向に基づいて雲台制御部２２を介してカメラ雲台１３を制御して撮像装置１の視野方向を調整し、当該移動量に基づいて撮像レンズ１２のズーム倍率ｒｆの値を算出し、当該ズーム倍率ｒｆの値に基づいて撮像レンズ制御部２３を介して撮像レンズ１２を制御して撮像装置１の画角を調整し、これにより、撮像装置１の撮像視野内に侵入した物体を追跡する。
【０１０７】
また、更に具体的な一例に係る画像監視装置（以下で、画像監視装置Ａ２と言う）では、例えば監視対象とする監視範囲を撮像するＴＶカメラ１１等の画像入力機能１と、当該ＴＶカメラ１１等の画像入力機能１が撮像した画像を入力する画像入力部（インタフェース）２１と、当該画像入力部２１から入力された画像を記憶する画像メモリ２５と、物体認識を行う物体追跡装置の動作のプログラムを記憶しているプログラムメモリ４と、当該プログラムメモリ４に保持されているプログラムにしたがって物体追跡装置を動作させる処理装置２６と、画像メモリ２５に記憶された画像の解析を行うためのワークメモリ２７と、音、可視光、振動、回転運動、上下運動等の少なくとも１つ以上で表し人間または補助動物が感知可能な信号を発生する警告表示機能６と、監視用の画像モニタ６を備える。
【０１０８】
また、当該画像監視装置（画像監視装置Ａ２）では、ワークメモリ２７による解析結果に対応して処理装置２６の指示によって警告表示機能６に警告を表示させる信号を伝達する警報出力部（インタフェース）３０と、ワークメモリ２７による解析結果に対応して処理装置２６の指示によって監視用の画像モニタ５への画像を送る画像出力部（インタフェース）２９と、ワークメモリ２７による解析結果に対応して処理装置２６の指示によってＴＶカメラ１１等の視野方向を制御するカメラ雲台１３と、ワークメモリ２７による解析結果に対応して処理装置２６の指示によってＴＶカメラ１１等の視野方向を制御させる信号を伝達する雲台制御部（インタフェース）２２と、ワークメモリ２７による解析結果に対応して処理装置２６の指示によってＴＶカメラ１１等の画角を制御する撮像レンズ１２と、ワークメモリ２７による解析結果に対応して処理装置２６の指示によってＴＶカメラ１１等の画角を制御させる信号を伝達する撮像レンズ制御部（インタフェース）２３を備える。
【０１０９】
また、当該画像監視装置（画像監視装置Ａ２）では、プログラムメモリに保持されているプログラムが、画像処理機能１により取得され画像メモリ２５に記憶された画像に映る物体を検出する機能と、当該侵入物体の画像信号上での移動方向及び移動量を検出する機能と、当該移動量に基づいて撮像レンズ１２のズーム倍率ｒｆの値を算出する機能と、当該移動方向に基づいて雲台制御部２２を介してカメラ雲台１２を制御して撮像装置１の視野方向を調整する機能と、当該ズーム倍率ｒｆの値に基づいて撮像レンズ制御部２３を介して撮像レンズ１２を制御して撮像装置１の画角を調整する機能を設け、これにより、ＴＶカメラ１１等の撮像視野内に侵入した物体の安定な追跡を実現する。
【０１１０】
従って、本例の画像監視装置では、撮像装置１を用いた監視装置において、撮像装置１の撮像視野内の侵入物体を当該撮像装置１の画像信号の中から自動的に検出し、当該侵入物体の動きを自動的に検出し、当該侵入物体の動きに応じて当該撮像装置１の撮像方向及び画角を調節することができる。
【０１１１】
具体的には、本例の画像監視装置では、撮像レンズ１２の適切なズーム倍率ｒｆを自動的に調整し、監視領域内に侵入した物体を正確に検出して追跡することができ、これにより、信頼性の高い物体追跡を実現することができる。本例の画像監視装置では、例えば、物体追跡処理によって得られた侵入物体の（表示）画像上での移動量に応じて撮像レンズ１２のズーム倍率ｒｆ（撮像レンズ１２の焦点距離）を自動的に調整することで、侵入物体の（表示）画像上での移動量を所定の値以下に抑えながら（所定の値以上としないように）侵入物体をズームアップして追跡することができる。
【０１１２】
このように、本例の画像監視装置では、撮像レンズ１２の適切な焦点距離を自動的に調整することにより、監視領域内に侵入した物体を適切なズーム倍率ｒｆで正確にかつ安定して検出や追跡することができ、画像監視装置の適用範囲を大きく広げることができる。例えば、対象物体をズームアップして撮像する際などに、対象物体の画像上での移動量が大きくなったような場合においても、安定に追跡動作が実行されるように撮像レンズ１２の焦点距離を自動的に調整することができ、信頼性の高い監視システムを簡易に構築することができる。
【０１１３】
ここで、本例では、画像から物体を検出する方法として差分法を例として用い、また、物体の移動量を検出する方法としてテンプレートマッチング法を例として用いたが、例えば、本例と同様に侵入物体の移動量を検出しながら追跡することが可能な方法であれば、撮像レンズ１２のズーム倍率ｒｆを算出して焦点距離を適切に調節することができるため、種々な方法が用いられてもよい。
【０１１４】
なお、本例の画像監視装置では、撮像装置１の機能により撮像手段が構成されており、処理装置２の機能により画像中物体移動量検出手段が構成されており、処理装置２の機能により撮像レンズ制御手段が構成されており、画像モニタ５の機能により画像表示出力手段が構成されている。
【０１１５】
次に、本発明に関する比較例を示す。なお、ここで記載する事項は、必ずしも全てが従来技術であるとは限らない。
本比較例に係る画像監視装置の構成としては、概略的には、上記した本発明の実施例に係る上記図１に示される画像監視装置の構成と同様である。
図８には、差分法やテンプレートマッチング法を用いる本比較例に係る画像監視装置により行われる物体追跡処理の手順の一例を示してある。
同図に示した処理手順は、概略的には、例えば、上記した本発明の実施例に係る上記図２に示される処理手順と比べて、テンプレートマッチング法による侵入物体の移動量の検出処理Ｔ３においてテンプレート拡大・縮小処理（上記図２のステップＳ１０）が行われず、また、焦点距離情報取得処理（上記図２のステップＳ１５）やズーム倍率算出処理（上記図２のステップＳ１６）や撮像レンズ制御処理（上記図２のステップＳ１７）が行われない点を除いては、上記図２に示される処理手順と同様である。
【０１１６】
ここで、本発明に係る物体追跡装置や画像監視装置などの構成としては、必ずしも以上に示したものに限られず、種々な構成が用いられてもよい。なお、本発明は、例えば本発明に係る処理を実行する方法或いは方式や、このような方法や方式を実現するためのプログラムなどとして提供することも可能であり、また、例えば物体監視装置や物体検出装置などの種々な装置やシステムとして提供することも可能である。
また、本発明の適用分野としては、必ずしも以上に示したものに限られず、本発明は、種々な分野に適用することが可能なものである。
【０１１７】
また、本発明に係る物体追跡装置や画像監視装置などにおいて行われる各種の処理としては、例えばプロセッサやメモリ等を備えたハードウエア資源においてプロセッサがＲＯＭ（Read Only Memory）に格納された制御プログラムを実行することにより制御される構成が用いられてもよく、また、例えば当該処理を実行するための各機能手段が独立したハードウエア回路として構成されてもよい。
また、本発明は上記の制御プログラムを格納したフロッピー（登録商標）ディスクやＣＤ（Compact Disc）−ＲＯＭやＤＶＤ（Digital Versatile Disk）−ＲＯＭ等のコンピュータにより読み取り可能な記録媒体や当該プログラム（自体）として把握することもでき、当該制御プログラムを記録媒体からコンピュータに入力してプロセッサに実行させることにより、本発明に係る処理を遂行させることができる。
【０１１８】
【発明の効果】
以上説明したように、本発明に係る物体追跡装置によると、撮像により得られる画像信号に基づいて画像中の物体を追跡するに際して、画像中における物体の移動量を検出し、当該検出結果に基づいて、撮像を行う撮像レンズを制御するようにしたため、当該撮像レンズを効果的に制御することができる。
【図面の簡単な説明】
【図１】 本発明の一実施例に係る画像監視装置の構成例を示す図である。
【図２】 本発明の一実施例に係る画像監視装置により行われる物体追跡処理の手順の一例を示す図である。
【図３】 差分法を用いて侵入物体を検出する処理の一例の概略と、当該侵入物体の画像をテンプレートに登録する処理の一例の概略を示す図である。
【図４】 テンプレートの拡大・縮小の様子の一例を示す図である。
【図５】 テンプレートマッチング法による侵入物体の移動量の検出処理を逐次実行して侵入物体を追跡する処理の流れの一例を示す図である。
【図６】 検出物体の位置に基づいてカメラ雲台を制御する動作の一例を示す図である。
【図７】 テンプレートと画像の上端、下端、左端、右端までのそれぞれの距離に基づいてズームアップに上限を設ける処理の一例を説明するための図である。
【図８】 画像監視装置により行われる物体追跡処理の手順の一例を示す図である。
【符号の説明】
１・・撮像装置、 ２・・処理装置、 ３・・操作装置、
４・・外部記憶装置、 ５・・画像モニタ、 ６・・警告灯、
１１・・ＴＶカメラ、 １２・・撮像レンズ、 １３・・カメラ雲台、
２１・・画像入力部、 ２２・・雲台制御部、 ２３・・レンズ制御部、
２４・・操作入力部、 ２５・・画像メモリ、 ２６・・ＭＰＵ、
２７・・ワークメモリ、 ２８・・外部入出力部、 ２９・・画像出力部、
３０・・警報出力部、 ４１・・ジョイスティック、 ４２、４３・・ボタン、
５１、９２、９４、９６、９８・・入力画像、 ５２・・基準背景画像、
５３・・差分画像、 ５４・・二値化画像、
５５、９１、９３、９５、９７、１５１・・テンプレート画像、
６１、６３・・物体、 ６２・・変化領域、 ６４・・外接矩形、
６５、８１、８３、１０１、１１１、１２１、１３１、１６１、１７２・・テンプレート、
７１・・減算器、 ７２・・二値化器、 ７３・・切り出し器、
８２、８４、１６２・・テンプレートの中心位置、
１０２、１１２、１２２、１３２・・矩形領域、
１０３、１１３、１２３、１３３・・探索領域、
１０４、１１４、１２４、１３４・・物体の位置、
１０５、１１５、１２５、１３５・・矢印、
１４１、１４３、１４５、１４７・・テンプレートマッチング処理、
１４２、１４４、１４６・・テンプレート更新処理、
１６３・・画像の中心位置、 １７１・・画像、[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an object tracking device that tracks an object in an image to be captured, and more particularly to a technique for effectively controlling an imaging lens that performs imaging.
[0002]
[Prior art]
For example, a remote monitoring type monitoring system using an imaging device such as a TV camera (television camera) has been widely used in the past, and many of them are monitored while watching an image displayed on the monitor. It is a so-called manned monitoring system. In a manned monitoring system, it is necessary to monitor in real time an image displayed on the monitor, and to identify intruders such as humans and automobiles entering the monitored area in real time. A heavy burden is placed on the staff.
[0003]
In other words, since the human concentration is limited, in the monitoring system of the manned monitoring system, the occurrence of oversight of the intruding object cannot be ignored, and there is a problem in terms of reliability. In addition, with the explosive spread of surveillance cameras, there are also many scenes where one surveillance person monitors many TV camera images on multiple monitors, and even when multiple TV cameras capture an intruding object at the same time, the intruding object May be missed.
[0004]
Therefore, instead of monitoring by such a person, a camera platform (turning) equipped with a TV camera so that an intruding object is automatically detected by image processing from an image captured by the TV camera and an image of the intruding object is captured. In recent years, a so-called automatic tracking type monitoring system that automatically adjusts the viewing direction and the angle of view by controlling the table to obtain a predetermined notification and alarm processing has been strongly demanded. ing.
[0005]
By the way, in order to realize such a system, it is necessary to use a predetermined monitoring method, detect an object to be monitored that should be regarded as an intruding object from an image signal, and detect a movement of the intruding object.
One example of a monitoring method for detecting such an intruding object is a method called a difference method, which has been widely used. The difference method is a method in which an input image obtained by a TV camera is compared with a reference background image created in advance, that is, an image in which an object to be detected is not captured, a difference in luminance value is obtained for each pixel, and the difference value is obtained. Is detected as an object. In addition, an application example of the difference method has been studied (for example, see Patent Document 1).
[0006]
Furthermore, there is a method called a template matching method as an example of a monitoring method for detecting the amount of movement of an intruding object, which has been widely used conventionally as in the difference method. In the template matching method, an image of an intruding object detected by a difference method or the like is registered as a template, and a position most similar to the template image is detected among images sequentially input (see, for example, Non-Patent Document 1). .) Normally, when tracking a target object using template matching, in order to follow a change in the posture of the target object, an image of the position of the target object detected by the matching process is sequentially updated as a template.
[0007]
[Patent Document 1]
JP-A-9-73541
[Non-Patent Document 1]
Supervised by Hideyuki Tamura, “Introduction to Computer Image Processing”, Soken Publishing, 1985, p. 149-153
[0008]
[Problems to be solved by the invention]
By the way, when monitoring the target object, it is required to zoom in on the intruding object as much as possible. However, if the target object is zoomed up (that is, the focal length of the imaging lens is increased) and monitored, the amount of movement of the target object in the displayed image increases, so that the target image is input from within the region of the sequentially input image. There arises a problem that the object comes off and the tracking of the intruding object cannot be performed stably. Therefore, as an example, setting the focal length of the imaging lens so as to zoom in on the intruding object as much as possible without degrading the tracking performance of the intruding object is important in providing a reliable object tracking device. become.
[0009]
As described above, in an automatic tracking type monitoring system, it is important to zoom in on a target object to be monitored without degrading the reliability of the object tracking function as much as possible. Since the focal length varies depending on the size of the image sensor, it requires skill to set it. Furthermore, if the distance between the TV camera and the target object changes, the zoom magnification must be changed accordingly. There is a problem.
[0010]
The present invention has been made in view of such conventional circumstances, and provides an object tracking device that can effectively control an imaging lens that performs imaging when tracking an object in an image to be captured. For the purpose.
[0011]
[Means for Solving the Problems]
In order to achieve the above object, the object tracking device according to the present invention performs the following processing when tracking an object in an image based on an image signal obtained by an imaging means.
That is, the object movement amount detection means in the image detects the movement amount of the object in the image, and the imaging lens control means detects the imaging lens of the imaging means based on the detection result by the object movement amount detection means in the image. Control.
Therefore, the imaging lens of the imaging means can be effectively controlled by controlling the imaging lens of the imaging means based on the amount of movement of the object in the image.
[0012]
In this specification, the term “tracking” is used for explanation, but for example, the term “tracking” is also a similar term and is included in the present invention.
In this specification, the term “image” is used for explanation. For example, the term “video” is a similar term and is included in the present invention. In addition, for example, an image referred to in the present specification refers to a moving image when it is temporally continuous. For example, as to a still image, an image of one frame in the moving image, Some of the images and still images that are unrelated to moving images.
[0013]
Here, various imaging means may be used, and for example, a camera or the like can be used.
Various image signals may be used.
Various objects may be used as the object.
Further, the object in the image corresponds to, for example, the image portion of the target object existing in the image.
[0014]
Various modes may be used as a mode for tracking an object in an image. For example, when a target object moves, such a mode that the object is included in the image is used. Can do.
Further, as the object movement amount detection means in the image, for example, a means that uses processing by template matching based on the detection result of the object in the image can be used.
[0015]
Further, as the amount of movement of the object in the image detected by the object movement amount detection means in the image, for example, the amount of movement of the object in the image is used instead of the amount of movement of the object in the real world. For example, even if the amount of movement of the object in the real world is the same, the amount of movement of the object in the image may differ depending on how the image is taken. On the other hand, for example, even if the amount of movement of the object in the real world is different, the amount of movement of the object in the image can be the same depending on how the image is taken.
[0016]
Various lenses may be used as the imaging lens of the imaging means, and for example, a zoom lens can be used.
Various modes may be used as a mode for controlling the imaging lens of the imaging unit based on the detection result by the object movement amount detection unit in the image.
[0017]
In the object tracking device according to the present invention, as one configuration example, the imaging lens control unit calculates the zoom magnification of the imaging lens of the imaging unit based on the detection result by the object movement amount detection unit in the image, and based on the calculation result. To control the imaging lens of the imaging means.
Therefore, the imaging lens of the imaging unit can be effectively controlled by controlling the imaging lens of the imaging unit based on the zoom magnification calculated based on the amount of movement of the object in the image.
[0018]
Here, various zoom magnifications may be used. For example, a value that specifies the size of an actual area displayed in one frame of an image is used.
Various methods may be used as the method of calculating the zoom magnification of the imaging lens of the imaging unit based on the detection result by the object movement amount detection unit in the image.
Various methods may be used as a method of controlling the imaging lens of the imaging unit based on the calculation result of the zoom magnification. For example, the imaging lens of the imaging unit is used so as to realize the calculated zoom magnification. A way of moving can be used.
[0019]
In the object tracking device according to the present invention, as one configuration example, the imaging lens control unit has a predetermined amount (hereinafter referred to as a predetermined value) based on a detection result by the object movement amount detection unit in the image. The zoom magnification of the imaging lens of the imaging means is calculated so as to be less than or less than the value P).
Therefore, the imaging lens of the imaging unit can be effectively controlled by calculating the zoom magnification of the imaging lens of the imaging unit so that the amount of movement of the object in the image is less than or less than a predetermined value.
[0020]
Here, as an aspect in which the amount of movement of the object in the image is less than or less than a predetermined value, for example, an aspect in which the amount of movement of the object in the image is less than or equal to a predetermined value is used. Alternatively, an aspect in which the amount of movement of the object in the image is less than a predetermined value may be used.
[0021]
The amount of movement of the object in the image corresponds to, for example, the amount of movement of the object in the frame of the image. For example, the amount of movement of the object can be detected based on the number of pixels constituting the frame.
Various values may be used as the predetermined value (predetermined value P) relating to the amount of movement of the object in the image. For example, to prevent the moving speed of the object in the frame from being too fast. It can be set in consideration.
[0022]
In the object tracking apparatus according to the present invention, as one configuration example, the image display output unit displays and outputs an image using an image signal obtained by the imaging unit. Further, as an example of the configuration, the imaging lens control unit is configured such that the amount of movement of the object in the image displayed and output by the image display output unit is based on a detection result by the object movement amount detection unit in the image. The zoom magnification of the imaging lens of the imaging means is calculated so as to be less than or less than a predetermined value Q).
[0023]
Therefore, the image pickup lens of the image pickup means is effectively controlled by calculating the zoom magnification of the image pickup lens of the image pickup means so that the amount of movement of the object in the displayed and output image is less than or less than a predetermined value. For example, an image that is displayed and output can be easily viewed by a person who sees it.
[0024]
Here, as an aspect in which the amount of movement of the object in the displayed and output image is equal to or less than a predetermined value, for example, the amount of movement of the object in the displayed and output image is equal to or less than the predetermined value. An aspect may be used, or an aspect may be used in which the amount of movement of an object in an image to be displayed and output is less than a predetermined value.
[0025]
Also, calculating the zoom magnification of the imaging lens of the imaging means so that the amount of movement of the object in the displayed and output image is less than or less than a predetermined value is not necessarily performed while the image is displayed and output. Good. For example, the image pickup lens of the image pickup means is configured so that the amount of movement of the object in the image that is displayed and output is assumed to be less than or less than a predetermined value, assuming that no image is actually displayed and output, It is also possible to use a configuration that calculates the zoom magnification of the image, and in this configuration, the image display output means is not necessarily provided.
[0026]
Various image display output means may be used. For example, a display device that displays and outputs an image corresponding to the image signal on the screen can be used. When such a display device is used, the movement amount of the object in the displayed and output image corresponds to, for example, the movement amount of the object on the screen.
Various values may be used as the predetermined value (predetermined value Q) relating to the amount of movement of the object in the displayed and output image. For example, the moving speed of the object in terms of the visibility of the image It can be set taking into consideration that it is not too fast.
[0027]
Below, the structural example which concerns on this invention is shown further.
The object tracking apparatus according to the present invention includes, as one configuration example, an in-image object detecting unit that detects an object in the image, and as one configuration example, detects the object moving direction detection in the image that detects the moving direction of the object in the image. And an imaging direction control means for controlling the imaging direction by the imaging means based on the detection result by the object moving direction detection means in the image.
[0028]
Here, as the object detection means in the image, for example, a means using processing by a difference method can be used.
Further, as the object moving direction detection means in the image, for example, a means that uses processing by template matching based on the detection result of the object in the image can be used.
Various imaging direction control means may be used. For example, a pan head for controlling the orientation of the imaging means may be used.
[0029]
In the object tracking device according to the present invention, as an example of the configuration, it is possible to use an aspect in which an upper limit value and a lower limit value are provided for the zoom magnification. For example, an object is present in a predetermined ratio area in one frame of an image. For example, a mode in which an upper limit value of the zoom magnification at the time of zooming up is provided can be used.
[0030]
In the object tracking apparatus according to the present invention, as one configuration example, when performing processing by template matching, it is possible to use a mode in which the size of the template is adjusted according to a change in zoom magnification.
In the object tracking device according to the present invention, as an example, it is possible to use a mode in which a predetermined relationship is set between the mode of controlling the imaging lens and the mode of controlling the imaging direction.
[0031]
Hereinafter, configuration examples (1) to (4) according to the present invention will be further described.
(1) In an object tracking device that tracks an object in an image based on an image signal obtained by an imaging means,
An object position detection means for detecting the position of the object in the image;
An object movement amount detection means for detecting an object movement amount in the image;
An imaging lens control means for controlling the imaging lens of the imaging means based on the detection result by the object movement amount detection means in the image;
An object tracking device comprising:
[0032]
(2) In the object tracking device according to (1) above,
The imaging lens control means calculates the zoom magnification of the imaging lens of the imaging means based on the detection result by the object position detection means in the image and the detection result by the object movement amount detection means in the image, and based on the calculation result, Control the imaging lens,
An object tracking device.
[0033]
(3) In the object tracking device according to (2) above,
The imaging lens control unit makes the distance between the object position and the upper end, lower end, left end, and right end of the image based on the detection result by the object position detection unit in the image greater than or equal to a predetermined value, and Calculating the zoom magnification of the imaging lens of the imaging means so that the amount of movement of the object in the image is less than or less than a predetermined value based on the detection result by the object movement amount detection means;
An object tracking device.
[0034]
(4) In the object tracking device according to (3) above,
The imaging lens control means determines that the distance between the object position and the upper end, lower end, left end, and right end of the image is a predetermined value or more based on the detection result by the object position detection means in the image and the detection result by the object movement amount detection means in the image. Alternatively, the zoom magnification of the imaging lens of the imaging unit is calculated so that the amount of movement of the object in the image is less than or less than a predetermined value based on the detection result by the object movement amount detection unit in the image. To
An object tracking device.
[0035]
DETAILED DESCRIPTION OF THE INVENTION
An embodiment according to the present invention will be described with reference to the drawings.
FIG. 1 shows a hardware configuration example of an image monitoring apparatus to which an object tracking apparatus according to the present invention is applied.
The image monitoring device of this example includes an imaging device 1, a processing device 2, an operation device 3, an external storage device 4, an image monitor 5, and a warning lamp 6.
[0036]
The image pickup apparatus 1 includes a TV camera 11, an image pickup lens 12 formed of, for example, a zoom lens, and a camera head 13 formed of, for example, a swivel base.
The processing device 2 includes an image input unit 21, a pan head control unit 22, a lens control unit 23, an operation input unit 24, an image memory 25, a micro processing unit (MPU) 26, and a work memory. 27, an external input / output unit 28, an image output unit 29, an alarm output unit 30, and a data bus 31.
[0037]
The operating device 3 includes a joystick 41, a first button 42, and a second button 43.
Note that the image input unit 21, the pan head control unit 22, the lens control unit 23, the operation input unit 24, the external input / output unit 28, the image output unit 29, and the alarm output unit provided in the processing device 2 are provided. Reference numeral 30 denotes an interface (I / F).
[0038]
Specifically, the output of the TV camera 11 is connected to the data bus 31 via the image input unit 21, and the control unit of the imaging lens 12 is connected to the data bus 31 via the lens control unit 23, The camera head 13 on which the TV camera 11 is mounted is connected to the data bus 31 via the head control unit 22, and the output of the operation device 3 is connected to the data bus 31 via the operation input unit 24.
[0039]
The external storage device 4 is connected to the data bus 31 via the external input / output unit 28, and the monitoring image monitor 5 is connected to the data bus 31 via the image output unit 29. Is connected to the data bus 31 via an alarm output unit 31.
The image memory 25, the MPU 26, and the work memory 27 are directly connected to the data bus 31.
[0040]
Here, the TV camera 11 captures an area to be monitored within a predetermined visual field, images the monitored area, and outputs an image signal. For this reason, the TV camera 11 includes an imaging lens 12 and is mounted on a camera head 13. The image signal picked up by the TV camera 11 is stored in the image memory 25 from the image input unit 21 via the data bus 31.
[0041]
The external storage device 4 functions to store programs and data, and the programs and data are read into the work memory 27 through the external input / output unit 28 as necessary. Is stored in the external storage device 4 from the work memory 27.
[0042]
The MPU 26 executes processing according to a program stored in the external storage device 4 and read into the work memory 27 when the processing device 2 is operating, and analyzes the image stored in the image memory 25 in the work memory 27. Then, the MPU 26 controls the imaging lens 12 and the camera pan 13 in accordance with the processing result, changes the imaging field of view of the TV camera 11, and enters the image monitor 5 as necessary. An image obtained as a result of detecting the object is displayed, and the warning lamp 6 is turned on.
[0043]
An example of the procedure of the object tracking process using the difference method and the template matching method performed by the image monitoring apparatus of this example will be shown.
FIG. 2 shows an example of the procedure of such processing.
First, in the initialization process, the external device, the variable, the image memory 25, etc. for executing the object tracking method are initialized (step S1).
[0044]
Next, an intruding object detection process T1 (step S2 to step S6) is performed by a difference method.
That is, in the first image input process, an input image having a horizontal size of 320 pixels and a height of 240 pixels is obtained from the TV camera 11 (step S2).
In the difference processing, a difference in luminance value for each pixel is calculated between the input image obtained in the first image input processing and a reference background image that is created in advance and does not show an intruding object (step S3). .
[0045]
In the binarization process, the pixel value of the pixel whose pixel value (difference value) of the difference image obtained by the difference process is less than the predetermined threshold Th is set to “0”, and the pixel of the pixel that is equal to or greater than the predetermined threshold Th A binarized image is obtained with a value of “255” (step S4). Here, for example, Th = 20 is used as the predetermined threshold Th. Further, the pixel value of one pixel is calculated by 8 bits (“0” to “255”).
[0046]
In the labeling process, a block of pixels having the pixel value “255” in the binarized image obtained by the binarization process is detected so that each can be distinguished by being numbered (step S5).
In the intruding object presence determination process, it is determined that there is an intruding object in the monitoring target area when a group of pixels having the pixel value “255” numbered in the labeling process satisfies a predetermined condition (step S6). . Here, as the predetermined condition, for example, a condition that the width is 20 pixels or more and the height is 50 pixels or more is used.
[0047]
When the intruding object presence determination process determines that an intruding object exists, the process proceeds to the alarm / detection information display process (step S7). When it is determined that no intruding object exists, the above-described process is performed again. The process proceeds to the first image input process (step S2), and the process by the difference method is executed again.
[0048]
Here, with reference to FIG. 3, the above-described intruding object detection processing will be specifically described.
In the figure, an outline of an example of a process for detecting an intruding object using the above-described difference method and an outline of an example of a process for registering an image of the intruding object described later in a template are shown.
FIG. 4A shows the input image 51 obtained by the first image input process (step S2). FIG. 4B shows the input image 51 created in advance and recorded in the image memory 25. The reference background image 52 is shown. Further, a subtractor 71 representing the difference process (step S3) is shown.
[0049]
FIG. 3C shows a difference image 53 obtained by the difference process. Further, a binarizer 72 representing binarization processing (step S4) is shown.
FIG. 4D shows a binarized image 54 obtained by binarization processing.
[0050]
Then, the subtracter 71 calculates a difference in luminance value for each pixel between the input image 51 and the reference background image 52 and outputs a difference image 53. Next, the binarizer 72 performs threshold processing on the difference image 53 with the threshold Th, sets the pixel value of pixels less than the threshold Th to “0”, sets the pixel value of pixels greater than the threshold Th to “255”, A valued image 54 is obtained. Thereby, for example, the humanoid object 61 shown in the input image 51 is calculated as a region (image signal changing region) 62 in which a difference is generated by the subtractor 71, and the image 63 of the intruding object is displayed by the binarizer 72. Detected as
[0051]
Next, the continuation of the processing procedure shown in FIG. 2 will be described.
In the alarm / detection information display processing, for example, in order to transmit a warning indicating that an intruding object has been found to the monitoring staff, information on the intruding object is displayed on the monitoring image monitor 5 via the image output unit 29. Then, the warning lamp 6 is turned on via the alarm output unit 30 (step S7). Here, as the information on the intruding object, for example, information such as the position and the number of people can be used.
[0052]
Next, detection processing T2 (steps S8 to S13) of the amount of movement of the intruding object by the template matching method is performed.
That is, in the template registration process, an image of an intruding object in the input image is cut out based on the circumscribed rectangle of the pixel block having the pixel value “255” numbered in the labeling process (step S5) as a template. Register (step S8).
[0053]
In the second image input process, as in the first image input process (step S2) described above, an input image having a horizontal size of 320 pixels and a height of 240 pixels is obtained from the TV camera 11 (step S9). At that time, the focal length of the imaging lens 12 of the TV camera 11 is set to f and recorded in the work memory 27.
In the template enlargement / reduction processing, the focal length f ′ recorded in the work memory 27, that is, the focal length of the imaging lens 12 of the TV camera 11 and the work memory 27 at the time of execution of the template update processing (step S13) described later, for example, last time. The difference in the size of the tracking target object shown in the template and the input image generated by changing the focal length of the imaging lens 12 is corrected in accordance with the ratio of the recorded focal length f (step S10). In this example, the imaging lens 12 is controlled by an imaging lens control process (step S17) described later, and the focal length changes.
[0054]
In the template matching process, an image having the highest degree of coincidence with the template is detected from the input images obtained in the second image input process. Usually, since it takes a long time to compare the template and the entire input image, an image having the highest degree of coincidence with the template in the search area is detected with a predetermined range as a search area for the template (step S11).
[0055]
Note that the result of the template matching process for the previous frame is recorded, and the search area is set based on the result, so that the search area can be narrowed and the amount of calculation can be reduced. In other words, intruders and vehicles subject to tracking rarely change their direction of movement suddenly, and when the amount of movement of the target object is determined in the template matching process of the previous frame, the amount of movement is the same in the next frame. Expected to be. Therefore, using a template position (x0, y0) and a movement amount (Δx, Δy) of a target object (described later) detected by the template matching process of the previous frame, a predetermined range centered on (x0 + Δx, y0 + Δy) is set as a search region. By doing so (movement amount prediction), template matching processing in which the motion of the target object is predicted can be performed.
[0056]
In the coincidence degree determination process, a coincidence degree r (Δx, Δy), which will be described later, is determined, and when the normalized correlation value represented by Equation 1 described later is used, for example, if the degree of coincidence is 0.7 or more, the coincidence If the degree of coincidence is less than 0.7, the process proceeds to the first image input process (step S2) described above (step S12).
[0057]
Here, the high degree of coincidence means that there is an image similar to the template in the input image, that is, there is an intruding object in the monitoring target area, and its position is viewed from the template position (x0, y0) described later. This means that the position is relatively (Δx, Δy). In this case, the amount of movement of the intruding object is subsequently detected. In addition, the low degree of coincidence means that there is no image similar to the template in the input image, that is, there is no intruding object in the monitoring target area. In this case, the first image input processing is performed. Then, the intruding object is detected again by the difference method.
[0058]
In the template update process, the input image obtained in the second image input process (step S9) described above is cut out based on the newly obtained position of the intruding object and is used as a new template image (step S13). In this way, by updating the template sequentially, the latest image of the intruding object is recorded in the template, and the amount of movement of the intruding object can be detected stably even when the intruding object changes its posture. .
[0059]
Here, the template enlargement / reduction processing (step S10) will be described in detail with reference to FIG. In the example shown in the figure, the case where the template is enlarged is shown, but the same applies to the case where the template is reduced.
FIG. 4A shows an example of an image 81 before the template is enlarged, and FIG. 5B shows an example of an image 83 after the template is enlarged.
The zoom magnification r of the template is expressed as Equation 1.
[0060]
[Expression 1]

[0061]
For example, assuming that the focal length f ′ of the imaging lens 12 of the TV camera 11 at the time of executing the template update process (step S13) is 20 mm and the current focal length f is 24 mm, r = 24/20 = 1. 2 and the apparent size of the target object is increased by 1.2 times due to the change in the focal length of the imaging lens 12. That is, the template 81 is multiplied by 1.2 so that the center positions 82 and 84 of the template match before and after enlargement, and this result is used as a new template 83, so that the intruding object in the input image is displayed. The size and the size of the intruding object of the template can be matched.
[0062]
In the example shown in the figure, in the XY orthogonal coordinate system, the length Tx in the X-axis direction of the template 81 before enlargement is the result of multiplying the zoom magnification by r, and the length (r × Tx) is obtained after enlargement. Similarly, the length of the template 83 is the length in the X-axis direction. Similarly, the length of (y × Ty) obtained by multiplying the length Ty of the template 81 before enlargement in the Y-axis direction by a zoom factor r is enlarged. 83 in the Y-axis direction. At this time, the center positions 82 and 84 of the templates 81 and 83 are not changed.
[0063]
Further, immediately after the intruding object is detected in the intruding object detection process T1, the template update process (step S13) is not executed. Therefore, the focal length f of the imaging lens 12 of the TV camera 11 at the time of updating the template. In this case, the template enlargement / reduction process (step S10) is not executed because 'is not acquired.
[0064]
When the template enlargement / reduction process (step S10) is performed as in this example, the template update process (step S13) uses the focal length f ′ recorded in the work memory 27 at the time of execution of the process as of the present time. Updating is performed using the focal length f of the imaging lens 12 of the TV camera 11.
[0065]
Here, with reference to FIG. 3 and FIG. 5, the above-described detection processing of the amount of movement of the intruding object will be described in detail.
FIG. 3 shows the cutout device 73. Further, FIG. 5E shows a template image 55.
[0066]
Then, the intruding object shown in the input image 51 is extracted based on the circumscribed rectangle 64 of the intruding object 63 obtained as a block of pixel values “255” in the binarized image by the above-described labeling process (step S5). 73, and a template image 55 is obtained. The template image 55 includes a template 65 of the intruding object 61, and this becomes an initial template in the detection processing of the moving amount of the intruding object by the template matching method. Next, template matching is executed based on the initial template.
[0067]
FIG. 5 shows an example of a flow of processing for tracking the intruding object by sequentially executing the detection processing of the moving amount of the intruding object by the template matching method.
In the figure, the time when the template image 91 is acquired is represented as t0, and the acquisition time of the input images sequentially input at a predetermined time interval (for example, 100 ms) is (t0 + 1), (t0 + 2),. To express.
[0068]
FIG. 4A shows a template image 91 at time t0. In the template image 91, there is a template 101 at time t0. These are the same as the template image 55 and the template 65 shown in FIG.
[0069]
FIG. 4B shows an input image 92 at time (t0 + 1). In the input image 92, the rectangular area 102 represents the position of the intruding object (the position of the template 101) at time t0, and the rectangular area 103 represents the area (search area) to be subjected to template matching.
In FIG. 5B, for the sake of simplicity, the search area setting method based on the above-described target object movement amount prediction is not used. However, even when the movement amount prediction is used, the intruding object is similarly detected. Can be tracked.
[0070]
Then, when the template matching process (step S11) 141 is executed, the degree of coincidence becomes the largest in the image 104 that most closely matches the template 101 in the template matching search area 103, and the intruding object is the image 104 at time (t0 + 1). It can be seen that it exists at the position. This position is represented by a relative position (Δx, Δy) when viewed from the position (x0, y0) of the template 101 at time t0. That is, it can be seen that the intruding object has moved by the amount indicated by the arrow 105.
[0071]
Therefore, the template 104 (step S13) 142 updates the image 104 that most closely matches the template 101 as a new template at time (t0 + 1). That is, as shown in FIG. 6C, the position 104 of the intruding object is cut out from the input image 92, and this is used as the template image 93, and the intruding object image 104 is updated as the new template 111 at time (t0 + 1). .
[0072]
This process is applied to input images sequentially input from the TV camera 11.
Specifically, as shown in FIG. 4D, the search area 113 is set based on the position 112 of the template 111 in the input image 94 at the time (t0 + 2), and the template image 93 at the time (t0 + 1) The position 114 of the intruding object is detected by template matching processing (step S11) 143 using the template 111. Then, it can be seen that the intruding object has moved as indicated by the arrow 115.
[0073]
Further, as shown in FIG. 8E, the template update process (step S13) 144 updates the template image 95 and the template 121 of the intruding object at time (t0 + 2).
Also, as shown in FIG. 5F, a search area 123 is set based on the position 122 of the template 121 in the input image 96 at time (t0 + 3), and the template 121 in the template image 95 at time (t0 + 2). The position 124 of the intruding object is detected by template matching processing (step S11) 145 using Then, it can be seen that the intruding object has moved as indicated by the arrow 125.
[0074]
Further, as shown in FIG. 6G, the template update process (step S13) 146 updates the template image 97 and the intruding object template 131 at time (t0 + 3).
Then, as shown in FIG. 9H, a search area 133 is set based on the position 132 of the template 131 in the input image 98 at time (t0 + 4), and the template 131 in the template image 97 at time (t0 + 3). The position 134 of the intruding object is detected by template matching processing (step S11) 147 using Then, it can be seen that the intruding object has moved as indicated by the arrow 135.
In this way, an intruding object can be tracked by sequentially executing template matching.
[0075]
Here, the search area and the degree of matching in the template matching process (step S11) will be specifically described.
The range of the search area described above is determined by, for example, the movement of the target object registered in the template on the input image.
As a specific example, a 1/3 inch CCD (imaging element size: 4.8 mm × 3.6 mm) is used as the imaging device 1, the focal length of the imaging lens 12 is 32 mm, and the distance to the target object is 30 m. When imaging is performed under conditions, the horizontal field of view of the TV camera 11 is 30 × 4.8 ÷ 32 = 4.5 m. When a target object with a moving speed of 5 km / h (about 1.39 m / s) is captured with the TV camera 11 at an image size of 320 × 240 pixels and an input interval of 0.1 s (100 ms), an input image of the target object is obtained. The moving amount on each image is 320 × 1.39 × 0.1 / 4.5≈9.88 pixels in the horizontal direction.
[0076]
Further, since the amount of movement on the image increases as the target object moves in the direction of the TV camera 11, the actual search area range is set with a margin of about five times the value calculated above. That is, the horizontal size Mx of the search area is 50 pixels. The vertical size My of the search area depends on the elevation angle of the TV camera 11 and changes depending on the mounting position of the TV camera 11, so it is set to a value of about 40% of the horizontal size. Therefore, in this example, the search range may be an area that is widened by Mx = 50 pixels on the left and right and My + 20 on the top and bottom with respect to the template.
[0077]
As the degree of coincidence, for example, a normalized correlation value r (Δx, Δy) can be applied, and is expressed as Expression 2.
[0078]
[Expression 2]

[0079]
Here, f (x, y) represents an input image. Referring to FIG. 6 described later, g (x, y) represents the template image 151, (x0, y0) represents the upper left coordinates of the template 161, and D represents the size of the template 161. Represents. In this example, the upper left corner of the image is the origin (0, 0) as the coordinate axis of the image. 3D, D corresponds to the size of the circumscribed rectangle 64 of the intruding object detected in the binarized image 54. In this example, D corresponds to 50 pixels horizontally and 20 pixels vertically. .
[0080]
The normalized correlation value r (Δx, Δy) takes a value of −1 ≦ r (Δx, Δy) ≦ 1, and is “1” when the input image and the template completely match.
In template matching, when Δx and Δy are scanned within the search range, that is, in the above example, when −Mx ≦ Δx ≦ Mx and −My ≦ Δy ≦ My, the normalized correlation value r (Δx , Δy) is detected to detect the position (Δx, Δy) (that is, the amount of movement of the target object).
Further, when the above-described movement amount of the target object is predicted, Δx and Δy are changed to Δx′−Mx ≦ Δx ≦ Δx ′ + Mx and Δy′−My ≦ Δy ≦ Δy ′ + My. Here, Δx ′ and Δy ′ represent the amount of movement of the target object in the previous frame.
[0081]
Next, the continuation of the processing procedure shown in FIG. 2 will be described.
In the camera head control process, the camera head is determined according to the displacement between the position of the intruding object detected by the template matching process (step S11) and the center of the input image in the moving object detection process T2 described above. 13 is controlled (step S14).
[0082]
Here, with reference to FIG. 6, the camera pan head control process (step S14) will be described in detail.
As an example, it is assumed that an intruding object is detected at a position 161 as shown in FIG. In this case, assuming that the center position of the intruding object is the center 162 of the template, the displacement dx in the X-axis direction and the displacement dy in the Y-axis direction from the center 163 of the template image 151 are calculated.
[0083]
Then, if the center position 162 of the template is a predetermined amount S or more on the left side (dx <−S) with respect to the center 163 of the input image, the camera head 13 is rotated (panned) to the left, and the right side (a predetermined amount S or more on the right side) If dx> S), the camera head 13 is rotated (panned) to the right. If the center position 162 of the template is higher than the center 163 of the input image by a predetermined amount S or more (dy <−S), the camera platform 13 is tilted upward, and the predetermined position S or more lower. If (dy> S), tilt down.
[0084]
When such a predetermined amount S is used, there is no need to control the camera platform 13 when an intruding object is present near the center of the image. Therefore, the control of the camera platform 13 is started by the predetermined amount S. The position of the intruding object to be specified can be specified.
Note that various values may be used as the predetermined amount S for each of left, right, upper, and lower. For example, the same value may be used in the left, right, up, and down directions, or in the left, right, up, and down directions, respectively. Any value may be used.
[0085]
As an example, a value of a predetermined amount S = 50 on the left, right, top and bottom can be used.
Further, for example, as the predetermined amount S is smaller, there is a possibility that the camera pan head 13 is controlled and the image becomes difficult to see if the intruding object deviates from the center even a little, but the predetermined amount S = 0 described above. It is also possible to use a small value as the value or the predetermined amount S.
[0086]
It is also possible to perform control such as changing the control speed of the pan and tilt motors according to the absolute values of the displacement dx in the X-axis direction and the displacement dy in the Y-axis direction with respect to the center 163 of the template image 151. It is. In this case, for example, the control speed is increased as the absolute value of the displacement dx in the X-axis direction or the displacement dy in the Y-axis direction is larger.
[0087]
In this example, as the tracking of the intruding object, the tracking of the intruding object with the control of the camera head 13 is performed. Thus, the camera pan head 13 can be automatically controlled to track the intruding object while capturing the intruding object within the field of view of the TV camera 11.
[0088]
Next, the continuation of the processing procedure shown in FIG. 2 will be described.
In the focal length information acquisition process, the focal length f of the imaging lens 12 at the time when the current input image recorded in the work memory 27 is acquired is acquired (step S15).
Next, in the zoom magnification calculation process, the zoom magnification rf is calculated by Equation 3 based on the movement amount (Δx, Δy) of the intruding object obtained in the template matching process (step S11) (step S16).
[0089]
[Equation 3]

[0090]
In Equation 3 above, Mx and My represent search ranges in the template matching method. Sx and Sy represent the maximum movement amount on the image of the intruding object that can be stably tracked. For example, about half of the search range, that is, in the above example, Sx = 25, Sy = 10 is assumed. Note that the values of Sx and Sy are set by experiment or the like, for example, so that an object has a margin at about half of the search range so that the object does not fall out of the search range.
In the above equation 3, when the moving amount of the intruding object (Δx, Δy) = (0, 0), that is, when the moving amount of the intruding object is zero, the zoom magnification rf = 1. Five.
[0091]
For example, when the zoom magnification rf is equal to or greater than a predetermined value, the zoom magnification rf can be set to the predetermined value so that the zoom-up is not suddenly increased. As the predetermined value, for example, 1.5 can be used. In this case, it is possible to zoom up to a maximum of 50% with one zoom-up.
In this way, when the maximum zoom magnification rf (upper limit value) in one zoom-up is set, for example, an object detected near the edge of the image jumps out of the field of view on the image due to the zoom-up. Can be suppressed.
[0092]
Further, for example, a configuration in which the upper limit value (MAX value) of the zoom magnification rf is variable can be used. In such a configuration, for example, when the template is sufficiently smaller than the screen size of the image, the upper limit value of the zoom magnification rf can be set to a value larger than 1.5.
[0093]
In addition, for example, a configuration in which an upper limit is set for zooming up can be used. Specifically, as an example, zooming is performed only in a range where the image screen height is 120% or more of the template height, or the image screen width is 120% of the template width. It is possible to zoom in only within the above range. As a result, it is possible to prevent the amount of movement (Δx, Δy) of the intruding object from being small and zooming in many times, so that the template does not exceed the screen size of the image. , Can ensure stable operation.
[0094]
It is also possible to employ a configuration in which an upper limit is provided for zooming up based on the distance between the template and the upper end, lower end, left end, and right end of the image. Specifically, as an example, as shown in FIG. 7, the distances between the template 172 and the upper end, lower end, left end, and right end of the image 171 are du, db, dl, dr, respectively, and the upper side, lower side, left side, Negative values are excluded from magnifications 120 / (120-du), 120 / (120-db), 160 / (160-dl), and 160 / (160-dr) whose right side exceeds the screen by zooming up. The smallest magnification is the upper limit of zoom-up magnification. Here, the image size is assumed to be 320 pixels wide and 240 pixels high.
[0095]
In addition, when the amount of movement of the target object is predicted, if the amount of movement of the target object in the previous frame is (Δx ′, Δy ′), 120 / {120− (du + Δy ′)}, 120 / { 120− (db−Δy ′)}, 160 / {160− (dl + Δx ′)}, 160 / {160− (dr−Δx ′)}, and the upper side, the lower side, the left side, and the right side of the template in the same manner as described above. Magnification that exceeds the outside of the screen by zooming up (upper limit of zooming magnification) is calculated.
[0096]
Note that, for example, the upper limit of the zoom-up magnification may be calculated by considering only the shorter one of the distances du and db between the template 172 and the upper end and the lower end of the image 171. The upper limit of the zoom-up magnification may be calculated taking into account only the shorter one of the distances dl and dr to the left and right ends of the image 171.
[0097]
Next, in the imaging lens control process, the focal length of the imaging lens 12 is adjusted to f × rf via the lens control unit 23 based on the zoomed focal length f × rf (step S17). Thereby, the imaging lens 12 can be automatically adjusted so that the moving speed on the image of the intruding object to be tracked is suppressed to a predetermined value or less. The predetermined value is 25 in the horizontal direction and 10 in the vertical direction in the above example.
[0098]
That is, when the moving speed on the image of the intruding object to be tracked is equal to or higher than a predetermined value (or exceeds a predetermined value), the focal length of the imaging lens 12 is reduced (that is, the zoom magnification is increased). The moving speed of the intruding object on the image is set to be less than a predetermined value (or less than a predetermined value). Further, when the moving speed on the image of the intruding object to be tracked is less than a predetermined value (or less than the predetermined value), the focal length of the imaging lens 12 is increased (that is, the zoom magnification). Can be zoomed up until the moving speed of the intruding object on the image reaches a predetermined value.
[0099]
Next, in the alarm / tracking information display processing, for example, in order to transmit an alarm indicating that the intruding object is being tracked to the monitor, the intruding object information is sent to the monitoring image monitor 5 via the image output unit 29. Displaying or lighting the warning lamp 6 through the alarm output unit 30 is performed (step S18). Here, as the information on the intruding object, for example, information such as a movement amount and a movement route can be used.
[0100]
In this example, both the process for controlling the camera head 13 (step S14) and the process for controlling the imaging lens 12 (step S17) are performed. The upper limit value of the zoom magnification rf and the like are preferably set by experiment, taking into consideration each other. As a result, for example, when the control of the camera head 13 and the zoom-up with the zoom magnification rf are both performed simultaneously, it is possible to prevent the object from going out of the image screen. It is.
[0101]
In this example, the processing procedure shown in FIG. 2 is used. As another example, the camera head control process (step S14), the zoom magnification calculation process (step S16), and the imaging lens control process (step S17) are performed. It is also possible to use a processing procedure that is provided between and executed. In such a processing procedure, for example, the position of the invading object after zooming is predicted from the zoom magnification rf in consideration of the zoom magnification rf calculated in the zoom magnification calculation processing (step S16), and based on the prediction result. Thus, it is possible to perform control processing of the camera pan head 13 (step S14).
[0102]
As described above, in the image monitoring apparatus of this example, an intruding object in the monitoring target area is detected based on the image signal obtained by the imaging apparatus 1, and the moving direction and moving amount of the intruding object on the image signal are determined. In an object tracking method and an object tracking device for detecting and tracking the intruding object while controlling the pan 13 on which the imaging device 1 is mounted and the imaging lens 12 of the imaging device 1 based on the moving direction and the moving amount. The imaging lens 12 of the imaging apparatus 1 is controlled based on the movement amount.
[0103]
Further, in the image monitoring apparatus of the present example, in the configuration as described above, zoom magnification calculation processing (step for calculating the zoom magnification rf of the imaging lens 12 of the imaging apparatus 1 based on the amount of movement of the intruding object on the image signal) S16) and an imaging lens control process (step S17) for controlling the imaging lens 12 of the imaging apparatus 1 based on the zoom magnification calculated by the zoom magnification calculation process. Are provided in a processing procedure for detecting the moving direction and moving amount.
[0104]
Further, in the image monitoring apparatus of the present example, in the configuration as described above, in the zoom magnification calculation process (step S16), the zoom magnification rf is set so that the amount of movement of the intruding object on the image signal is suppressed to a predetermined value or less. calculate.
[0105]
In an image monitoring apparatus according to a specific example (hereinafter referred to as image monitoring apparatus A1), an imaging apparatus 1 that captures at least a range to be monitored, and the imaging apparatus that changes the viewing direction of the imaging apparatus 1 1, a camera head 13 that can be controlled by an external signal, an imaging lens 12 that can be controlled by an external signal attached to the imaging device 1 for changing the angle of view of the imaging device 1, and the imaging device 1 An image input unit (interface) 21 that converts an image signal from the digital image signal into a digital image signal, and a function of processing the image signal from the image input unit 21, for example, at least an MPU 26, an image memory 25, and a program memory 4. An image processing unit having a work memory 27, and a pan head control unit (in) that supplies a control signal for controlling the camera pan head 13 from the image processing unit. Comprising a face) 22, an imaging lens control unit supplies a control signal for controlling the imaging lens 12 from the image processing section (interface) 23.
[0106]
In the image monitoring apparatus (image monitoring apparatus A1), the image processing unit detects an intruding object in the monitoring target area from the image signal captured by the image capturing apparatus 1, and moves the intruding object on the image signal. The direction and the amount of movement are detected, the camera head 13 is controlled via the head control unit 22 based on the direction of movement to adjust the viewing direction of the image pickup apparatus 1, and the imaging lens 12 is adjusted based on the amount of movement. A value of the zoom magnification rf is calculated, and the angle of view of the imaging device 1 is adjusted by controlling the imaging lens 12 via the imaging lens control unit 23 based on the value of the zoom magnification rf. Track an object that has entered the field of view.
[0107]
Further, in an image monitoring apparatus according to a more specific example (hereinafter referred to as an image monitoring apparatus A2), for example, an image input function 1 such as a TV camera 11 that captures a monitoring range to be monitored, and the TV camera 11 An image input unit (interface) 21 for inputting an image captured by the image input function 1 such as an image memory 25 for storing an image input from the image input unit 21, and an object tracking device for performing object recognition. A program memory 4 storing a program, a processing device 26 for operating the object tracking device in accordance with the program held in the program memory 4, and a work memory for analyzing an image stored in the image memory 25 27 and generates a signal that can be detected by humans or auxiliary animals with at least one of sound, visible light, vibration, rotational movement, vertical movement, etc. That a warning display function 6, comprising an image monitor 6 for monitoring.
[0108]
In the image monitoring device (image monitoring device A2), an alarm output unit (interface) 30 that transmits a signal for displaying a warning on the warning display function 6 according to an instruction from the processing device 26 in response to the analysis result of the work memory 27. An image output unit (interface) 29 for sending an image to the monitoring image monitor 5 according to an instruction from the processing device 26 in response to the analysis result from the work memory 27, and a processing device in response to the analysis result from the work memory 27. The camera head 13 that controls the visual field direction of the TV camera 11 or the like according to the instruction 26 and a signal that controls the visual field direction of the TV camera 11 or the like according to the instruction of the processing device 26 are transmitted corresponding to the analysis result by the work memory 27. In response to an analysis result by the pan head control unit (interface) 22 and the work memory 27, an instruction from the processing device 26 is given. The imaging lens 12 that controls the angle of view of the TV camera 11 and the like, and the imaging lens control that transmits a signal for controlling the angle of view of the TV camera 11 and the like according to an instruction of the processing device 26 in accordance with the analysis result by the work memory 27. A unit (interface) 23 is provided.
[0109]
In the image monitoring apparatus (image monitoring apparatus A2), the program stored in the program memory detects the object appearing in the image acquired by the image processing function 1 and stored in the image memory 25, and the intrusion A function of detecting the movement direction and movement amount of the object on the image signal, a function of calculating the value of the zoom magnification rf of the imaging lens 12 based on the movement amount, and the pan head control unit 22 based on the movement direction The camera head 12 is controlled via the image pickup device 1 to adjust the viewing direction, and the image pickup device 12 is controlled via the image pickup lens control unit 23 based on the zoom magnification rf. A function for adjusting the angle of view is provided, thereby realizing stable tracking of an object that has entered the imaging field of view such as the TV camera 11.
[0110]
Therefore, in the image monitoring apparatus of this example, in the monitoring apparatus using the imaging apparatus 1, an intruding object in the imaging field of the imaging apparatus 1 is automatically detected from the image signal of the imaging apparatus 1, and the intruding object is detected. Can be automatically detected, and the imaging direction and angle of view of the imaging apparatus 1 can be adjusted according to the movement of the intruding object.
[0111]
Specifically, in the image monitoring apparatus of the present example, an appropriate zoom magnification rf of the imaging lens 12 can be automatically adjusted to accurately detect and track an object that has entered the monitoring area. Reliable object tracking can be realized. In the image monitoring apparatus of this example, for example, the zoom magnification rf (focal length of the imaging lens 12) of the imaging lens 12 is automatically set in accordance with the amount of movement of the intruding object on the (display) image obtained by the object tracking process. By adjusting to, the intruding object can be zoomed up and tracked while keeping the moving amount of the intruding object on the (display) image below a predetermined value (so as not to exceed the predetermined value).
[0112]
As described above, in the image monitoring apparatus of the present example, by automatically adjusting an appropriate focal length of the imaging lens 12, an object that has entered the monitoring area is accurately and stably detected with an appropriate zoom magnification rf. Can be traced, and the application range of the image monitoring apparatus can be greatly expanded. For example, when the target object is zoomed in and imaged, the focal length of the imaging lens 12 is such that the tracking operation can be performed stably even when the amount of movement of the target object on the image increases. Can be automatically adjusted, and a highly reliable monitoring system can be easily constructed.
[0113]
In this example, the difference method is used as an example of a method for detecting an object from an image, and the template matching method is used as an example of a method for detecting the amount of movement of an object. Any method that can be tracked while detecting the amount of movement of the intruding object can calculate the zoom magnification rf of the imaging lens 12 and appropriately adjust the focal length. Therefore, various methods are used. Also good.
[0114]
In the image monitoring apparatus of this example, an imaging unit is configured by the function of the imaging device 1, an object movement amount detection unit in the image is configured by the function of the processing device 2, and imaging is performed by the function of the processing device 2. A lens control unit is configured, and an image display output unit is configured by the function of the image monitor 5.
[0115]
Next, comparative examples relating to the present invention will be shown. Note that the matters described here are not necessarily all related to the prior art.
The configuration of the image monitoring apparatus according to this comparative example is generally the same as the configuration of the image monitoring apparatus shown in FIG. 1 according to the above-described embodiment of the present invention.
FIG. 8 shows an example of the procedure of the object tracking process performed by the image monitoring apparatus according to this comparative example using the difference method or the template matching method.
The processing procedure shown in the figure is roughly compared with the processing procedure shown in FIG. 2 according to the embodiment of the present invention described above, for example, the detection processing T3 of the moving amount of the intruding object by the template matching method. Template enlargement / reduction processing (step S10 in FIG. 2) is not performed, and focal length information acquisition processing (step S15 in FIG. 2), zoom magnification calculation processing (step S16 in FIG. 2), and imaging lens control are performed. The processing procedure is the same as that shown in FIG. 2 except that the processing (step S17 in FIG. 2) is not performed.
[0116]
Here, the configurations of the object tracking device and the image monitoring device according to the present invention are not necessarily limited to those described above, and various configurations may be used. The present invention can also be provided as a method or method for executing the processing according to the present invention, a program for realizing such a method or method, and the like, for example, an object monitoring device or an object. It can also be provided as various devices and systems such as a detection device.
The application field of the present invention is not necessarily limited to the above-described fields, and the present invention can be applied to various fields.
[0117]
In addition, as various processes performed in the object tracking device and the image monitoring device according to the present invention, for example, a control program stored in a ROM (Read Only Memory) in a hardware resource including a processor, a memory, etc. A configuration controlled by execution may be used, and for example, each functional unit for executing the processing may be configured as an independent hardware circuit.
The present invention also provides a computer-readable recording medium such as a floppy (registered trademark) disk, a CD (Compact Disc) -ROM, and a DVD (Digital Versatile Disk) -ROM storing the above control program, and the program itself. The processing according to the present invention can be performed by inputting the control program from a recording medium to a computer and causing the processor to execute the control program.
[0118]
【The invention's effect】
As described above, according to the object tracking device of the present invention, when tracking an object in an image based on an image signal obtained by imaging, the amount of movement of the object in the image is detected, and based on the detection result. Thus, since the imaging lens that performs imaging is controlled, the imaging lens can be effectively controlled.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating a configuration example of an image monitoring apparatus according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating an example of a procedure of object tracking processing performed by an image monitoring apparatus according to an embodiment of the present invention.
FIG. 3 is a diagram illustrating an outline of an example of processing for detecting an intruding object using a difference method and an example of processing for registering an image of the intruding object in a template.
FIG. 4 is a diagram illustrating an example of how a template is enlarged or reduced.
FIG. 5 is a diagram illustrating an example of a flow of processing for tracking an intruding object by sequentially executing detection processing of the amount of movement of the intruding object by a template matching method.
FIG. 6 is a diagram illustrating an example of an operation for controlling a camera platform based on the position of a detected object.
FIG. 7 is a diagram for explaining an example of a process for setting an upper limit for zooming in based on the distance between the template and the upper end, lower end, left end, and right end of the image.
FIG. 8 is a diagram illustrating an example of a procedure of object tracking processing performed by the image monitoring apparatus.
[Explanation of symbols]
1 .... Imaging device, 2 .... Processing device, 3 .... Operating device,
4 .... External storage device, 5 .... Image monitor, 6 .... Warning light,
11 .... TV camera, 12 .... Imaging lens, 13 .... Camera head,
21 .. Image input unit 22.. Pan head control unit 23.. Lens control unit
24..Operation input section 25.Image memory 26.MPU
27..Work memory 28.External input / output unit 29.Image output unit
30..Alarm output part, 41..joystick, 42, 43 ... button,
51, 92, 94, 96, 98 ... Input image 52 ... Standard background image
53 ... Differential image, 54 ... Binary image,
55, 91, 93, 95, 97, 151 .. template image,
61, 63 ... Object, 62 ... Change area, 64 ... circumscribed rectangle,
65, 81, 83, 101, 111, 121, 131, 161, 172 .. template,
71 .. Subtractor, 72 .. Binarizer, 73.
82, 84, 162 .. center position of template,
102, 112, 122, 132 .. rectangular area,
103, 113, 123, 133 .. search area,
104, 114, 124, 134 .. position of the object,
105, 115, 125, 135 ... Arrows
141, 143, 145, 147 .. template matching processing,
142, 144, 146 .. template update processing,
163 .. center position of image 171.

Claims (2)

In an object tracking device that tracks an object in an image based on an image signal obtained by an imaging means,
An object position detection means for detecting the position of the object in the image;
An object movement amount detection means for detecting an object movement amount in the image;
Based on the detection result by the object position detection means in the image for one or more of the upper end, the lower end, the left end, and the right end of the image, the distance between the position of the object and the end is set to be a predetermined value or larger. And an imaging lens control means for controlling the imaging lens of the imaging means so that the movement amount of the object in the image is less than or less than a predetermined value based on the detection result by the object movement amount detection means in the image ;
An object tracking device comprising: In an object tracking device that tracks an object in an image based on an image signal obtained by an imaging means,
  An object position detection means for detecting the position of the object in the image;
  An object movement amount detection means for detecting an object movement amount in the image;
  The position of the object and its end based on both the detection result by the object position detection means in the image and the detection result by the object movement amount detection means in the image for one or more of the upper end, the lower end, the left end, and the right end of the image Of the imaging unit so that the distance of the object in the image is less than or less than the predetermined value based on the detection result by the object movement amount detection unit in the image. Imaging lens control means for controlling the imaging lens;
  An object tracking device comprising:

Images