JP3577895B2 - Wide area monitoring device and wide area monitoring system - Google Patents

Description

【００３２】
によりまず類似度Ｓを計算する。ここでＳｍａｘ、ｗ１、ｗ２はあらかじめ設定する定数である。
【００３３】
対応付けは、管理動物体毎に最も類似度Ｓの大きい画像動物体を、各画像処理装置毎に最大一つずつ選択し、対応付けを決定する。なお、Ｓが負の場合には対応付けは行わず、ある画像処理装置においては画像動物体が対応しない場合もある。これは動物体が画像入力装置の撮像エリアの外に出た場合、動物体が静止した場合、管理動物体が誤って生成されたものである場合などに対応する。逆に、同じ画像動物体を複数の管理動物体に対応付けてもかまわないし、管理動物体に対応付けされない画像動物体が存在してもかまわない。前者の場合は動物体が画像入力装置から見て重なり合っている場合に相当し、後者は新たに動物体が発生した場合に相当する。
【００３４】
次に、対応付け決定部３２より、管理動物体とその管理動物体に対応付けされた画像動物体との実空間位置と平均ＲＧＢ値との組を動物体更新部３４に入力するとともに、管理動物体に対応付けされない画像動物体の実空間位置と平均ＲＧＢ値を動物体生成部３３に入力する。なお、一つも画像動物体が対応付けられなかった管理動物体に関する情報はどこにも入力しない。
【００３５】
動物体生成部３３では、管理動物体に対応付けされなかった画像動物体の実空間位置と平均ＲＧＢ値を持つ管理動物体を新たに生成し、動物体情報管理部３０に入力する。ただし、新たに生成する管理動物体は各フレームで一つに限定し、複数の画像動物体が入力された場合にはその他の画像動物体に関する情報は捨てる。これにより、画像入力装置２から４の重複する撮像エリアに新たに動物体が発生しても、複数の管理動物体が生成されることはない。複数の新たな動物体が同時に監視エリアに発生することは少なく、仮にその場合でも次のフレームで無視された動物体に対応する管理動物体は生成されるので、実用上の問題はない。
【００３６】
動物体更新部３４では、管理動物体とその管理動物体に対応付けされた画像動物体の実空間位置と平均ＲＧＢ値の組から現在の動物体の位置を計算して、動物体情報管理部３０に入力する。まず、対応付けられた画像動物体の位置、平均ＲＧＢ値を平均し、さらに、管理動物体の現在推定位置との重み付け平均を計算する。例えば、ある管理動物体に画像処理装置５から７毎に一つずつ画像動物体が対応付けられ、管理動物体の現在推定位置を（Ｘ、Ｙ）、平均ＲＧＢ値を（Ｒ、Ｇ、Ｂ）とし、対応付けられた画像動物体の実空間位置を（ｘｉ、ｙｉ）、平均ＲＧＢ値を（ｒｉ、ｇｉ、ｂｉ）（ｉは１から３でそれぞれ画像処理装置５から７に対応する）とすると、新しい管理動物体の位置Ｘｎｅｗは
【００３７】
【数２】

【００３８】
で計算する。ここでｗ１は０よりも大きく１よりも小さい定数である。同様に、Ｙｎｅｗ、Ｒｎｅｗ、Ｇｎｅｗ、Ｂｎｅｗも計算する。
【００３９】
動物体情報管理部３０は、動物体更新部３４より入力した管理動物体の新しい位置と平均ＲＧＢ値とを記憶する。この時、過去の複数フレームの管理動物体の位置を動物体の情報として管理しておき、動物体の予測位置を動物体情報予測部３１で計算できるようにする。また、管理動物体に画像動物体が一つも対応付けられなかった場合、つまり更新した管理動物体が動物体更新部３４より入力されなかった場合にはその管理動物体をリストから削除する。
【００４０】
また、動物体情報管理部３０は、新たに生成した動物体が動物体生成部３３から入力された場合には、その動物体を新たに動物体のリストに付け加える。
【００４１】
そして動物体管理部３０は、更新または新たに付け加えられた管理動物体のリストを通信ネットワーク３５を介して動物体統合処理装置１１に送信する。
【００４２】
以上のようにして、統合処理装置８は、画像処理装置５から７より入力した画像動物体の実空間位置と平均ＲＧＢ値から、管理動物体を更新、もしくは新たに生成する。
【００４３】
以上のようにして、動物体追跡装置１は、映像入力装置２から４で撮像される範囲に存在する動物体を検出・追跡し、各フレーム毎に現在の動物体の位置とその平均ＲＧＢ値を通信ネットワーク３５を介して動物体統合処理装置１１に送信する。
【００４４】
図４は本実施の形態における動物体統合処理装置１１のブロック図である。以下、その画像フレーム毎の動作を説明する。
【００４５】
動物体情報予測部４１では、各フレーム毎に動物体情報管理部４０より過去に存在した動物体（以下、広域管理動物体と呼ぶ）のリストを読み出し、広域管理動物体の現在の実空間位置を計算し、平均ＲＧＢ値とともに、対応付け決定部４２に入力する。
【００４６】
対応付け決定部４２では、通信ネットワーク３５を介して動物体追跡装置１、９、１０から送信された管理動物体と、動物体情報予測部４１で推定した広域管理動物体とを、動物体追跡装置毎に、実空間位置と平均ＲＧＢ値を用いて対応付ける。例えば、ある広域管理動物体の現在推定位置を（Ｘ、Ｙ）、平均ＲＧＢ値を（Ｒ、Ｇ、Ｂ）とし、ある管理動物体の実空間位置を（ｘ、ｙ）、平均ＲＧＢ値を（ｒ、ｇ、ｂ）とする時、（数１）により、まず類似度Ｓを計算する。ここでＳｍａｘ、ｗ１、ｗ２はあらかじめ設定する定数である。対応付けは、広域管理動物体毎に最も類似度の大きい管理動物体を各動物体追跡装置毎に最大一つずつ選択し、対応付けを決定する。この時、一度対応付けた管理動物体は二度は対応付けず、次に大きい類似度を持つ管理動物体を対応付ける。また、Ｓが負の場合には対応付けは行わず、ある動物体追跡装置では管理動物体が対応しない場合もある。また、広域管理動物体に対応付けされない管理動物体が存在してもよい。
【００４７】
そして対応付け決定部４２は、広域管理動物体と対応付けた管理動物体の実空間位置と平均ＲＧＢ値の組を動物体更新部４４に入力する。なお、一つも管理動物体が対応付けられなかった広域管理動物体も動物体更新部４４に入力する。また、広域管理動物体に対応付けられなかった管理動物体が存在した場合にはその管理動物体の実空間位置と平均ＲＧＢ値とを動物体生成部４３に入力する。
【００４８】
動物体更新部４４では、広域管理動物体とそれに対応付けられた一つまたは複数の管理動物体から現在の広域管理動物体の位置と平均ＲＧＢ値を計算し、動物体情報管理部４０に入力する。例えば、ある広域管理動物体に動物体追跡装置１、９、１０毎に一つずつ管理動物体が対応付けられ、広域管理動物体の現在推定位置を（Ｘ、Ｙ）、平均ＲＧＢ値を（Ｒ、Ｇ、Ｂ）とし、対応付けられた管理動物体の実空間位置を（ｘｉ、ｙｉ）、平均ＲＧＢ値を（ｒｉ、ｇｉ、ｂｉ）（ｉは１から３でそれぞれ動物体追跡装置１、９、１０に対応）とすると、新しい広域管理動物体の位置Ｘｎｅｗは、（数２）で計算する。ここで、ｗ１は０よりも大きく１よりも小さい定数である。同様に、Ｙｎｅｗ、Ｒｎｅｗ、Ｇｎｅｗ、Ｂｎｅｗも計算する。なお、一つの管理動物体にも対応付けられなかった広域管理動物体の実空間位置、平均ＲＧＢ値は動物体情報予測部４１で予測したそのままの値を用いる。
【００４９】
動物体生成部４３では、広域管理動物体に対応付けされなかった管理動物体の実空間位置と平均ＲＧＢ値を持つ広域管理動物体を新たに生成し、動物体情報管理部４０に入力する。まず、動物体追跡装置１に管理動物体が広域管理動物体に対応せずに残っている場合には、その管理動物体の位置と平均ＲＧＢ値を持つ広域管理動物体を新たに生成する。これを仮広域管理動物体と呼ぶ。次に、動物体追跡装置９に管理動物体が広域管理動物体に対応付けられずに残っている場合には、まず、その管理動物体と存在する仮広域管理動物体との間の類似度Ｓを（数１）に準じて計算し、類似度Ｓがあらかじめ設定した閾値を超えている場合には、その管理動物体と仮広域管理動物体との平均の実空間位置と平均ＲＧＢ値とを仮広域管理動物体に再設定する。類似度Ｓが閾値を越えていない場合には、その管理動物体から仮広域管理動物体を新たに生成する。
【００５０】
さらに、動物体追跡装置１０の管理動物体と仮広域管理動物体との対応付けを同様に調べ、対応付けられた場合には仮広域管理動物体の実空間位置と平均ＲＧＢ値を修正し、対応付けられない場合には仮広域管理動物体を新たに生成する。なお、動物体追跡装置１０の管理動物体に対応付けて修正する場合には、単に平均するのではなく、それまでに仮広域管理動物体に対応付けられた管理動物体の数に応じて重みを付けて平均する。そして、最終的に残った仮広域管理動物体を新たに生成した広域管理動物体として動物体情報管理部４０に入力する。
【００５１】
動物体情報管理部４０では、新たに生成した広域管理動物体を動物体生成部４３から受け取り、リストに加えるとともに、動物体変更部４４から入力した更新された広域管理動物体の情報で、既存の広域管理動物体を更新する。なお、広域管理動物体は過去の一定時間の動物体の位置やその他の特徴量を記憶し、動物体情報予測部４１で位置等の特徴量を予測できるようにする。
【００５２】
なお、動物体情報管理部４０で管理している動物体の情報は必要に応じて、表示装置や自動警告装置などの外部に出力する。
【００５３】
以上のようにして、動物体統合処理装置４１は、動物体追跡装置１、９、１０から入力した管理動物体を用いて、広域の監視エリアに存在する動物体を検出した動物体追跡装置に関わらず同一のものは同一のものと認識して、一元的に管理する。
【００５４】
以下、本実施の形態における広域監視装置の特徴について説明する。例えば、映像入力装置２から４は撮像エリアを共有しており、その撮像エリアに３つの動物体が存在し、また、その動物体を含め５つの動物体が広域管理動物体として検出されており、残りの２つの動物体はどの映像入力装置の撮像エリアにも入っていないとする。また、他の動物体追跡装置では全く動物体が検出されていないものとする。
【００５５】
この時、統合処理装置８の動物体情報管理部３０で管理している動物体の数は３つであるから、対応付けの類似度の計算は（画像動物体の数）＊（管理動物体の数）＊（カメラの数）の２７回であり、さらに、動物体統合処理装置１１では（管理動物体の数）＊（広域管理動物体の数）の１５回の類似度の計算が必要で、合計４２回の類似度の計算が必要となる。仮に、全ての画像動物体を広域管理動物体と対応付ける装置を考えると、（画像動物体の数）＊（広域管理動物体の数）＊（カメラの数）の４５回の類似度の計算を一つの計算機で行わなければならない。したがって、本発明によれば、類似度の計算の回数を減らせるだけではなく、類似度の計算を異なる計算機に分散するため、処理を軽減・分散し、もって高速に処理を行うことができる。
【００５６】
また、画像処理装置５から７から出力される９個の画像動物体の情報は、ネットワークに流れず、もって通信量を削減することもできる。さらに、仮に動物体追跡装置１とネットワークの間のインタフェースが故障しても、その他の動物体追跡装置９、１０は動作するため、一部の撮像エリアの監視は不能になるが、他の部分には影響がでない。
【００５７】
さらに、映像入力装置の追加や削除は該当する動物体追跡装置内にのみ影響を与えるため、映像入力装置の変更が容易である。例えば、動物体追跡装置１で新たに映像入力装置を追加したり、映像入力装置を取り除いても、統合処理装置８における処理を変更するだけでよく、動物体追跡装置９および１０、動物体統合処理装置１１の処理を変更する必要はない。
【００５８】
また、例えば３階建ての建物を監視する場合に、各動物体追跡装置を各階に対応させることにより、各映像入力装置を各階毎に階層的に管理し、把握しやすいシステムを構築することができる。
【００５９】
なお、本実施の形態では映像入力装置２から４にカラーＴＶカメラを用いたが、例えば赤外線カメラや、白黒のカメラを用いてもよいし、複数種類のカメラが混在してもよい。この場合にはカメラにより特徴量が異なり（数１）のような一律の類似度を用いることはできないが、例えば、管理動物体には全ての特徴量の属性を保持し、対応する特徴量が画像動物体及び管理動物体に設定されている場合にのみ各特徴量毎の類似度を計算し、その類似度に適当な重みを掛け加算するなどの処理を行い対応付けを行えばよい。
【００６０】
また、画像処理装置５では動物体の検出にフレーム間差分を用いているが、背景差分やオプティカルフロー等を利用して動物体を検出してもよい。
【００６１】
また、画像処理装置５では動物体の検出を動領域により行っているが、例えば人の頭、車のタイヤなど特徴的な部分を抽出したり、動物体を動領域で検出しそれからテンプレートを生成し、そのテンプレートとマッチングする領域を動物体として検出してもよい。
【００６２】
また、画像処理装置５で計算する動領域の特徴量としてＲＧＢ値を用いたが、その他の色の表示系や、領域の実空間での大きさや形状特徴、テクスチャなどの特徴量を用いてもよい。
【００６３】
また、画像処理装置５では各ラベル領域の中心画素から動物体の実空間での位置を計算しているが、その他の領域の特徴点を用いてもよいし、映像入力装置が撮像エリアを共有し、複数の映像入力装置で撮像した特徴点からステレオ計測により実空間位置を計算するなどの方法を用いてもよい。
【００６４】
また、統合処理装置８では、画像動物体と対応付けできなかった管理動物体を削除しているが、ある一定期間残しておき、静止した場合や一時的に検出できなかった場合に対応するようにしてもよい。
【００６５】
また、動物体生成部３３では１フレームに新たに生成する管理動物体は一つに限定していたが、動物体生成部４３で使用した処理のように、別々の画像処理装置から入力した複数の画像動物体がある場合にはそれらを対応付け、複数の管理動物体を同時に生成するようにしてもよい。
【００６６】
また、動物体追跡装置１では３台の映像入力装置を用いたが、１台以上の任意の台数でよく、また動物体追跡装置１、９、１０でそれぞれ異なる台数の映像入力装置を用いてもよい。
【００６７】
また、例えば映像入力装置２を動物体追跡装置９で用いるなど、同じ映像入力装置を異なる動物体追跡装置で共有してもよい。
【００６８】
また、映像入力装置２から４の撮像範囲は重複していても、重複していなくとも、それらが混在していてもよい。
【００６９】
（実施の形態２）
図５は、本発明の実施の形態２による広域監視装置の構成図である。図５において、１００から１０２は映像入力装置で、本実施の形態ではカラーＴＶカメラを用いる。１０３から１０５は、それぞれ映像入力装置１００から１０２により取得したカラー映像をフレーム画像毎に処理して第１の動物体とその特徴量を抽出する画像処理装置、１０６から１０８は、第２の動物体とそれぞれ画像処理装置１０３から１０５から入力した第１の動物体とをその特徴量を利用して対応付ける対応付け装置である。１０９は、映像入力装置１００から１０２の撮像エリアを含む監視エリアに存在する動物体を第２の動物体として管理する動物体管理装置、１１０は動物体管理装置１０９の出力である第２の動物体の情報からその第２の動物体を撮像している可能性のある映像入力装置を映像入力装置１００から１０２の中から選択し、選択した映像入力装置に接続した対応付け装置１０６から１０８に第２の動物体の情報を送信する動物体情報ルーティング装置である。
【００７０】
以下、画像処理装置１０３から１０５、対応付け装置１０６から１０８、動物体管理装置１０９の順に動作を説明した後に、本実施の形態における広域監視装置の動作を説明する。
【００７１】
画像処理装置１０３から１０５は、第１の実施の形態における画像処理装置５と同様の働きをするため、動作の説明は省略する。以降、第１の実施の形態と同様に第１の動物体を画像動物体と呼ぶ。
【００７２】
以下、図５を参照しながら本実施の形態における広域監視装置の動作を説明する。各映像入力装置１００から１０２から入力したカラー映像は画像処理装置１０３から１０５でフレーム毎に処理して、画像動物体を抽出し、各画像動物体の実空間位置と平均ＲＧＢ値を計算し、対応付け装置１０６から１０８にそれぞれ入力する。また、動物体管理装置では、現在管理している管理動物体の現在の時刻における実空間位置と平均ＲＧＢ値を計算して、動物体情報ルーティング装置１１０に入力する。そして動物体情報ルーティング装置１１０では、各管理動物体の予測実空間位置を含む撮像エリアを持つ映像入力装置を映像入力装置１００から１０２の中から選択し、選択した映像入力装置に対応する対応付け装置にのみ、管理動物体の情報を送信する。
【００７３】
対応付け装置１０６から１０８では、各画像処理装置１０３から１０５から入力した画像動物体と、動物体情報ルーティング装置１１０から送信されてきた管理動物体とを、その実空間位置と平均ＲＧＢ値を用いて対応付け、対応付けの結果と画像動物体の情報とを動物体管理装置１０９に送信する。そして動物体管理装置１０９では、対応付けられた画像動物体により管理動物体の現在位置と平均ＲＧＢ値を計算して管理動物体を更新するとともに、管理動物体に対応付けられなかった画像動物体から新たに管理動物体を生成するものである。
【００７４】
図６は本実施の形態における対応付け装置１０６のブロック図である。なお、対応付け装置１０７及び１０８は対応付け装置１０６と同様の動作を行うので説明は省略する。
【００７５】
図６において通信Ｉ／Ｆ部２６を介し、画像処理装置１０３から画像動物体の実空間位置と平均ＲＧＢ値を類似度計算部１２１に入力する。また、通信ネットワーク１２０からは第２の動物体（以降管理動物体と呼ぶ）の現フレームにおける予測実空間位置と平均ＲＧＢ値を類似度計算部１２１に入力する。
【００７６】
類似度計算部１２１では、各管理動物体と各画像動物体との間の類似度Ｓを計算する。類似度Ｓの計算は第１の実施の形態の画像動物体と管理動物体の類似度と同様に（数１）により計算する。そして計算した類似度と管理動物体、画像動物体の情報を対応付け決定部１２２に入力する。
【００７７】
対応付け決定部１２２では、各管理動物体毎に類似度Ｓが最大の画像動物体を最大一つ選択して対応付け、対応付けの結果と画像動物体の情報とを通信ネットワーク１２０を介して動物体管理装置１０９に送信する。この時、類似度Ｓが負の管理動物体と画像動物体は対応付けず、また、一つの画像動物体を複数の管理動物体に対応付けてもよい。
【００７８】
以上のようにして、本実施の形態における対応付け装置１０６は各管理動物体に最大１つの画像動物体をその特徴量により対応付け、その対応付け結果と画像動物体の情報を動物体管理装置１０９に入力する。
【００７９】
図７は本実施の形態における動物体管理装置１０９のブロック図である。対応付け装置１０６から１０８から入力した対応付け結果と管理動物体に対応付けられた画像動物体の情報は、通信ネットワーク１２０を介して動物体更新部１３１に入力する。また、管理動物体の予測実空間位置と平均ＲＧＢ値を動物体情報予測部１３０から動物体更新部１３１に入力する。また、管理動物体と対応付けできなかった画像動物体は動物体生成部１３２に入力する。
【００８０】
動物体更新部１３１では、各管理動物体に対応付けられた画像動物体の実空間位置及び平均ＲＧＢ値の平均を計算し、その平均値を用いて管理動物体の実空間位置と平均ＲＧＢ値の現在値を計算する。例えば、ある管理動物体の予測現在位置を（Ｘ、Ｙ）、平均ＲＧＢ値を（Ｒ、Ｇ、Ｂ）とし、その管理動物体には対応付け装置１０６と１０７から一つずつ画像動物体が対応付けられ、対応付け装置１０８では対応する画像動物体が無く、そして、対応付けられた画像動物体の実空間位置及び平均ＲＧＢ値をそれぞれ（ｘｉ、ｙｉ）、（ｒｉ、ｇｉ、ｂｉ）（ｉは１および２。それぞれ、対応付け装置１０６、１０７に対応する）とする時、管理動物体の実空間位置の現在値Ｘｎｅｗは、
【００８１】
【数３】

【００８２】
により計算する。ここでｗ１は、あらかじめ定めた定数である。同様に、Ｙｎｅｗ、Ｒｎｅｗ、Ｇｎｅｗ、Ｂｎｅｗを計算する。なお、一つも画像動物体が対応付けされなかった管理動物体の現在の実空間位置、平均ＲＧＢ値は、予測した実空間位置と平均ＲＧＢ値をそのまま用いる。動物体更新部１３１は以上のようにして管理動物体の現在の実空間位置と平均ＲＧＢ値を計算して動物体情報管理部１３３に入力する。
【００８３】
動物体生成部１３２では、管理動物体と対応付けされなかった画像動物体から新たに管理動物体を生成して動物体情報管理部１３３に入力する。まず、対応付け装置１０６に画像動物体が管理動物体に対応せずに残っている場合には、その画像動物体の位置と平均ＲＧＢ値を持つ管理動物体を新たに生成する。これを仮管理動物体と呼ぶ。次に、対応付け装置１０７に画像動物体が管理動物体に対応付けられずに残っている場合には、まず、その画像動物体と存在する仮管理動物体との間の類似度Ｓを（数１）に準じて計算し、類似度Ｓがあらかじめ設定した閾値を超えている場合には、その画像動物体と仮管理動物体との平均の実空間位置と平均ＲＧＢ値とを仮管理動物体に設定する。類似度Ｓが閾値を越えていない場合には、その画像動物体から仮管理動物体を新たに生成する。さらに、対応付け装置１０８の画像動物体と仮管理動物体との対応付けを同様に調べ、対応付けられた場合には仮管理動物体の実空間位置と平均ＲＧＢ値を修正し、対応付けられない場合には仮管理動物体を新たに生成する。なお、対応付け装置１０８の画像動物体と対応付けて修正する場合には、単に平均するのではなく、それまでに仮管理動物体に対応付けられた画像動物体の数に応じて重みを付けて平均する。そして、最終的に残った仮管理動物体を新たに生成した管理動物体として動物体情報管理部１３３に入力する。
【００８４】
動物体情報管理部１３３では、新たに生成した管理動物体を動物体生成部１３２から受け取り、リストに加えるとともに、動物体更新部１３１から入力した更新した管理動物体の情報で、既存の管理動物体を更新する。なお、管理動物体は過去の一定時間の動物体の位置を記憶し、動物体情報予測部１３０で位置を予測できるようにする。
【００８５】
なお、動物体情報管理部１３３で管理している動物体の情報は必要に応じて表示装置や自動警告装置などの外部に出力する。
【００８６】
動物体情報予測部１３０では、動物体情報管理部１３３から現在の管理動物体のリストを読み出し、それから次のフレームにおける動物体の実空間位置を予測し、平均ＲＧＢ値とともに動物体情報ルーティング装置１１０と動物体更新部１３１に入力する。
【００８７】
以上のようにして、動物体管理装置１０９は、対応付け装置１０６から１０８から入力した管理動物体と画像動物体との対応付け情報と画像動物体の情報を用いて、広域の監視エリアに存在する動物体を検出した対応付け装置に関わらず同一のものは同一のものと認識して、一元的に管理することができる。
【００８８】
以上、本実施の形態による広域監視装置では、管理動物体と画像動物体の対応付けの計算を対応付け装置１０６から１０８に分散し、動物体管理装置１０９では管理動物体の更新と新たな生成、及び管理動物体の現在位置の予測のみを行うため、処理の負荷が分散される。さらに、動物体情報ルーティング装置により管理動物体の予測現在位置に対応する対応付け装置にのみ管理動物体の情報を送信して対応付けを行うため、無駄な対応付け処理が行われないため処理の負荷が軽減されるとともに、通信量も削減される。
【００８９】
（実施の形態３）
図８は、本発明の実施の形態３による広域監視システムの構成図である。図８において、１４０から１４２は中域監視装置、１４４は中域監視装置１４０から１４２で検出した第１の動物体を統合し、同一の動物体は同一の動物体として第２の動物体として管理する動物体統合処理装置である。１４３は、中域監視装置１４０から１４２と動物体統合処理装置１４４の間の通信を仲介するローカルエリアネットワーク（ＬＡＮ）である。１４５は広域監視装置１４０から１４２と動物体統合処理装置１４４とからなる広域監視装置である。また、１４６及び１４７は広域監視装置１４５と同様の構成の広域監視装置である。１４９は広域監視装置１４５から１４７で検出した第２の動物体を統合し、同一の動物体は同一の動物体として第３の動物体として管理する動物体統合管理装置である。１４８は、広域監視装置１４５から１４７と動物体統合管理装置１４９の間の通信を仲介するワイドエリアネットワーク（ＷＡＮ）である。
【００９０】
以下、本実施の形態における広域監視システムの動作を説明する。中域監視装置１４０から１４２は、実施の形態２の広域監視装置と同様に動作し、多数台の映像入力装置から入力された映像から第１の動物体を抽出し、その第１の動物体の各フレーム時刻における実空間位置と平均ＲＧＢ値をＬＡＮ１４３を介して出力する。そして動物体統合処理装置１４４は、上記第１の動物体の実空間位置と平均ＲＧＢ値をＬＡＮ１４３を介して受け取り、実施の形態１の動物体統合処理装置１１と同様な処理で、中域監視装置１４０から１４２の監視エリアを含むより広域の監視エリアに存在する動物体を、同一の動物体は同一の動物体として同定し第２の動物体として管理する。さらに、動物体統合処理装置１４４は、第２の動物体の各フレーム時刻における実空間位置と平均ＲＧＢ値をＷＡＮ１４８を介して出力する。
【００９１】
次に、動物体統合管理装置１４９は、広域監視装置１４５から１４７からＷＡＮ１４８を介して第２の動物体の実空間位置と平均ＲＧＢ値を受け取り、実施の形態１の動物体統合処理装置１１と同様な処理で、広域監視装置１４５から１４７の監視エリアを含むさらに広域の監視エリアに存在する動物体を、同一の動物体は同一の動物体として同定し第３の動物体として管理する。
【００９２】
以上のようにして本実施の形態における広域監視システムでは、各監視エリアにおける情報を各広域監視装置の内部にとどめて処理の分散と通信量の削減を行い、もって多数台の映像入力装置を用いても処理時間と装置間の通信量が急激に増加しないようにしている。また、広域監視装置を階層的に分けて管理することにより、映像入力装置の追加や広域監視装置の追加等が影響を与える範囲を最小限にし、もってシステムの把握や修正に柔軟に対応できるようにしている。
【００９３】
なお、本実施の形態では、中域監視装置１４０から１４２に第２の実施の形態における広域監視装置を用いているが、第１の実施の形態の広域監視装置を用いてもよいし、その他の形態の広域監視装置を混在して用いてもよい。
【００９４】
また、本実施の形態では、広域監視システムを３層に分けて管理しているが、４層以上の構成としてもよい。
【００９５】
また、本実施の形態では、広域監視装置１４６、１４７を広域監視装置１４５と同様の構成としているが、例えば広域監視装置１４６は第１の実施の形態の広域監視装置にし、広域監視装置１４７は第２の実施の形態の広域監視装置にするという構成としてもよい。
【００９６】
【発明の効果】
以上のように本発明によれば第１に、多数台のビデオカメラなどの映像入力装置を用いても処理時間と装置間の通信量が急激に増加しないという有利な効果が得られる。
【００９７】
第２に、多数台の映像入力装置を階層的に管理し、システムの把握が容易でかつ修正に柔軟に対応できるという有利な効果が得られる。
【図面の簡単な説明】
【図１】本発明の実施の形態１における広域監視装置の構成図
【図２】本発明の実施の形態１における広域監視装置の画像処理装置のブロック図
【図３】本発明の実施の形態１における広域監視装置の統合処理装置のブロック図
【図４】本発明の実施の形態１における広域監視装置の動物体統合処理装置のブロック図
【図５】本発明の実施の形態２における広域監視装置の構成図
【図６】本発明の実施の形態２における広域監視装置の対応付け装置のブロック図
【図７】本発明の実施の形態２における広域監視装置の動物体管理装置のブロック図
【図８】本発明の実施の形態３における広域監視システムの構成図
【図９】従来の移動物体追跡システムの構成図
【図１０】従来の移動物体物体監視装置の主要部分のブロック図
【図１１】従来の分散型監視装置の構成図
【符号の説明】
１ 動物体追跡装置
２、３、４ 映像入力装置
５、６、７ 画像処理装置
８ 統合処理装置
９ 動物体追跡装置
１０ 動物体追跡装置
１１ 動物体統合処理装置
２０ Ａ／Ｄ変換器
２１、２２ 画像メモリ
２３ 差分計算部
２４ ラベリング部
２５ 特徴量計算部
２６ 通信Ｉ／Ｆ
３０、４０ 動物体情報管理部
３１、４１ 動物体情報予測部
３２、４２ 対応付け決定部
３３、４３ 動物体生成部
３４、４４ 動物体更新部
３５ 通信ネットワーク
１００、１０１、１０２ 映像入力装置
１０３、１０４、１０５ 画像処理装置
１０６、１０７、１０８ 対応付け装置
１０９ 動物体管理装置
１１０ 動物体情報ルーティング装置
１２０ 通信ネットワーク
１２１ 類似度計算部
１２２ 対応付け決定部
１３０ 動物体情報予測部
１３１ 動物体更新部
１３２ 動物体生成部
１３３ 動物体情報管理部
１４０、１４１、１４２ 中域監視装置
１４３ ローカルエリアネットワーク
１４４ 動物体統合処理装置
１４５、１４６、１４７ 広域監視装置
１４８ ワイドエリアネットワーク
１４９ 動物体統合管理装置[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a wide area monitoring device and a wide area monitoring system for monitoring a moving object existing in a wide area such as a building, a public facility, a store, a parking lot, and a road.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, as devices for monitoring a moving object such as a public facility, those disclosed in JP-A-6-36031, JP-A-7-49952, and JP-A-8-241480 are known.
[0003]
FIG. 9 is a configuration diagram of a conventional moving object tracking system described in JP-A-6-36031. In FIG. 9, moving object detection devices 401 to 403 each calculate feature amount information such as a size, a shape, and a color unique to the moving object from a video imaged by an image capturing unit such as a television camera, and further predict a moving object. The moving object is identified from the position information, and the position information of the moving object is detected. The tracking calculation processing devices 404 to 405 calculate the predicted position information of the moving object based on the position information of the moving object detected by the moving object detection devices 401 to 403 up to the previous time, and transmit the predicted position information to the moving object detection devices 401 to 403. I do. The tracking management device 406 displays the moving object detected by the moving object detection devices 401 to 403 on the screen based on the position information.
[0004]
FIG. 10 is a block diagram of a main part of a conventional moving object monitoring device described in Japanese Patent Application Laid-Open No. 7-49952. The images captured by the video cameras 410 to 412 are input to the image processing devices 413 to 415, respectively, to obtain the feature amount of the moving object, and input to the integrated processor 416 via the interface unit 418. The integrated processor 416 uses the CPU 419 to make a correspondence between the moving object at the previous time and the moving object at the current time, and plots the motion state of the moving object on the plan view of the created monitoring area in the memory 420; This is displayed on the display device 417.
[0005]
FIG. 11 is a configuration diagram of a conventional distributed monitoring apparatus described in Japanese Patent Application Laid-Open No. 8-241480. The moving images (video) captured by the moving image input units 431 and 437 are input to the moving image processing units 432 and 438, respectively, and the moving regions are extracted by the moving image processing units 432 and 438. The moving regions extracted by the moving image processing units 432 and 438 are input to the moving object detecting unit 433, and the moving object detecting unit 433 reads out information of moving objects detected in the past from the moving object information storing unit 434, The integration process is performed to detect the current moving object. The moving object detection result is evaluated by the moving object detection evaluation means 435 and used for adjusting the moving image processing means 432 and 438 and the moving image input means 431 and 437.
[0006]
[Problems to be solved by the invention]
Each conventional monitoring device described in the prior art associates a moving object or a moving region in one frame of an image with a moving object detected in the past by a feature amount, a predicted position, or the like. When a camera is used and the number of moving objects existing in the monitoring area increases, the number of association processes increases rapidly, and there is a problem that real-time processing becomes difficult. For example, even if the monitoring areas of the cameras do not overlap, the number of moving objects needs to be squared. If the monitoring areas overlap, the number of times of association processing is further multiplied by the number of overlaps.
[0007]
Further, in the system of FIG. 9, a plurality of moving object detection devices and a plurality of tracking operation processing devices are assigned to different CPUs to distribute processing, but all moving object detection devices and tracking operation processing devices are Since information is exchanged, there has been a problem that the amount of communication rapidly increases as the number of cameras increases, and that there is a lack of robustness against an abnormality in a communication path. Regarding the point that the traffic increases with the number of cameras and the lack of robustness, the same applies to the monitoring apparatus shown in FIGS. 10 and 11.
[0008]
In addition, any of the conventional monitoring devices has a two-layer structure including a moving object detection unit and an integrated unit, and therefore, when a new camera is added, the existing unit is greatly affected. Further, since the system is uniform and does not reflect the actual structure of the monitoring target, there is a problem that it becomes difficult to grasp and correct the system as the number of cameras increases.
[0009]
SUMMARY OF THE INVENTION The present invention solves the above-mentioned conventional problems. First, it is an object of the present invention to provide a wide-area monitoring device in which the processing time and the communication volume between the devices do not sharply increase even when a plurality of video input devices such as video cameras are used. It is an object.
[0010]
A second object of the present invention is to provide a wide-area monitoring device that manages a large number of video input devices in a hierarchical manner, allows easy understanding of the system, and can flexibly respond to correction.
[0011]
[Means for Solving the Problems]
In order to solve this problem, the present invention firstly obtains one or a plurality of images, and detects and tracks a moving object from the images, and the N moving object tracking devices A moving object integrated processing device that manages the moving object detected by the moving object integrated processing device, wherein the moving object integrated processing device has a function of identifying the same moving object as the same moving object regardless of the detected moving object tracking device. By integrating the moving objects detected by the moving object tracking device, the moving object information obtained from one or a plurality of images can be kept in each moving object tracking device by integrating the moving objects detected by the moving object tracking device. Dispersion and reduction of communication volume are realized, so that a wide area monitoring device is realized in which the processing time and the communication volume between the devices do not sharply increase even when a large number of video input devices are used. In addition, by dividing the video input device into N moving object tracking devices and managing them, the range affected by the addition of the video input device and the like is minimized, and thus a large number of video input devices are hierarchically managed. In addition, a wide-area monitoring device capable of flexibly responding to grasp and correction of the entire system is realized.
[0012]
The present invention secondly provides L image input devices, L image processing devices that respectively calculate the first moving object and its characteristic amount from images acquired by the L image input devices, And L image correlating devices for associating the first moving object and its characteristic amount, which are the outputs of the image processing device, with the transmitted information of the second moving object, and the L image input means. A moving object management device that manages information on a moving object present in a monitoring area including a moving area and outputs the information as a second moving object information as necessary, and a second animal that is an output of the moving object management device Moving object information routing for selecting the associating device connected to the video input device which may be imaging the second moving object from the body information and transmitting the second moving object information thereto And the moving object management device has the L units. A structure wherein detection and tracking of a moving object are performed by updating information of a second moving object from the first moving object and its characteristic amount associated with the association result from the associating device. By transmitting the information of the second moving object only to the associating device corresponding to the video input device which may be imaged, the communication amount is reduced, and the unnecessary associating is not performed and the associating is performed. The present invention realizes a wide-area monitoring device in which the processing is dispersed and the processing time and the communication volume between the devices do not rapidly increase even when a large number of video input devices are used.
[0013]
Third, the present invention realizes a wide area monitoring system having a wider monitoring area by hierarchically combining a plurality of wide area monitoring apparatuses. A wide-area monitoring system that does not rapidly increase the traffic is realized. In addition, the impact on the entire system due to the addition of video input devices and wide-area monitoring devices is minimized, and a large number of video input devices and wide-area monitoring devices are managed hierarchically, making it possible to flexibly grasp and correct the entire system. Realize a wide area monitoring system that can respond to
[0017]
BEST MODE FOR CARRYING OUT THE INVENTION
Claim 1 of the present inventionThe invention described in (1), L image input devices, L image processing devices that respectively calculate a first moving object and its characteristic amount from images acquired by the L image input devices, and the L image input devices And L image correlating devices for associating the first moving object and its characteristic amount, which are the outputs of the image processing device, with the transmitted information of the second moving object, and the L image input means. A moving object management device that manages information on a moving object present in a monitoring area including a moving area and outputs the information as a second moving object information as necessary, and a second animal that is an output of the moving object management device Moving object information routing for selecting the associating device connected to the video input device which may be imaging the second moving object from the body information and transmitting the second moving object information thereto And the moving object management device is the L units. Detecting and tracking the moving object by updating the information of the second moving object from the first moving object and the feature amount thereof associated with the association result from the associating device; By transmitting the information of the second moving object only to the associating device corresponding to the video input device that is likely to be imaged, it is possible to reduce communication amount and perform useless associating. This has the effect of dispersing the processing of associating a few pieces, so that the processing time and the traffic between the apparatuses do not increase rapidly even when a large number of video input devices are used.
[0018]
Of the present inventionClaim 2Some or all of the inventions described inClaim 1And a moving object integrated processing device that integrates moving objects detected by the plurality of moving object tracking devices, identifies the same moving object, and manages the moving object. The wide area monitoring system is characterized in that the video input devices are divided into several groups, the moving objects are detected and tracked by the wide area monitoring device in each group, and detected by the moving object integrated processing device for each group. By managing the moving objects in all the monitoring areas by judging that the same moving object is the same, information on the moving object obtained from the image of each video input device is retained in the wide area monitoring device and the processing is performed. Dispersion and reduction of the communication amount are performed, so that even if a large number of video input devices are used, the processing time and the communication amount between the devices do not increase rapidly. In addition, by managing the wide area monitoring system by dividing it hierarchically, it is possible to minimize the range affected by the addition of video input devices and the addition of wide area monitoring devices, etc., so that it is possible to respond flexibly to grasping and modifying the system. Has an action.
[0019]
Of the present inventionClaim 3Some or all of the inventions described inClaim 1 or 2And a moving object integrated processing device that integrates moving objects detected by the plurality of moving object tracking devices, identifies the same moving object, and manages the moving object. The wide area monitoring system is characterized in that the video input devices are divided into several groups, the moving objects are detected and tracked by the wide area monitoring device in each group, and detected by the moving object integrated processing device for each group. By managing the moving objects in all the monitoring areas by judging that the same moving object is the same, information on the moving object obtained from the image of each video input device is retained in the wide area monitoring device and the processing is performed. Dispersion and reduction of the communication amount are performed, so that even if a large number of video input devices are used, the processing time and the communication amount between the devices do not increase rapidly. In addition, by managing the wide area monitoring system by dividing it hierarchically, it is possible to minimize the range affected by the addition of video input devices and the addition of wide area monitoring devices, etc., so that it is possible to respond flexibly to grasping and modifying the system. Has an action.
[0020]
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
(Embodiment 1)
FIG. 1 shows a configuration diagram of a wide area monitoring apparatus according to Embodiment 1 of the present invention. In FIG. 1, reference numerals 1, 9, and 10 denote moving object tracking devices that input images and detect and track moving objects from the images. The moving object tracking device 1 includes three video input devices 2 to 4, and image processing devices 5 to 7 connected to the video input devices 2 to 4, respectively, and an integrated processing device 8 connected to the image processing devices 5 to 7. It is configured. In the present embodiment, the video input devices 2 to 4 are realized by a monitoring color TV camera, and the integrated processing device 8 is realized by a personal computer.
[0021]
The moving object tracking devices 9 and 10 have the same configuration as the moving object tracking device 1. Reference numeral 11 denotes a moving object integrated processing device, which acquires information on the detected moving object from the moving object tracking devices 1, 9, and 10, and determines that the same moving object is the same regardless of the detected moving object tracking device. Identify and manage animals throughout the system. In the present embodiment, the moving object integrated processing device 11 is realized by a personal computer, and communication between the moving object tracking devices 1, 9, 10 and the moving object integrated processing device 11 is performed by 35 local area networks (hereinafter referred to as network). Is called).
[0022]
Hereinafter, the operations of the image processing device 5, the integrated processing device 8, the moving object tracking device 1, and the integrated moving object processing device 11 will be described in order of the operation of the wide area monitoring device in the present embodiment. Since the image processing devices 6 and 7 perform the same operations as the image processing device 5 and the moving object tracking devices 9 and 10 perform the same operation as the moving object tracking device 1, the description thereof is omitted.
[0023]
FIG. 2 is a block diagram of the image processing device 5 according to the present embodiment. Hereinafter, the operation for each image frame will be described. The color video captured by the video input device 2 is input to the A / D converter 20, converted into a digital signal, and stored in the image memory 21 as a frame image. The contents stored in the image memory 21 are copied to the image memory 22 before the next frame image is input. That is, the image memory 22 stores the frame image of the previous frame.
[0024]
Next, the difference calculation unit 23 compares each pixel of the images in the image memory 21 and the image memory 22 and assigns 1 to a pixel having a difference exceeding a predetermined threshold and assigns 0 to an image not having the difference. A value image is generated and input to the labeling unit 24.
[0025]
The labeling unit 24 generates a labeling image in which pixels are labeled for each connected component from the difference binary image, and inputs the generated labeling image to the feature amount calculation unit 25.
[0026]
The feature amount calculation unit 25 calculates an average RGB value from the frame image stored in the image memory 21 for each label of the labeling image, and calculates the position of the moving object in the real space from the pixel at the center of each label region. The calculated value is input to the integrated processing device 8 via the communication I / F unit 26.
[0027]
As described above, the image processing apparatus 5 according to the present embodiment extracts a moving object from the input video, calculates its real space position and average RGB values, and inputs the calculated values to the integrated processing apparatus 8.
[0028]
FIG. 3 is a block diagram of the integrated processing device 8 according to the present embodiment. Hereinafter, the operation for each image frame will be described. The real space position and the average RGB value of the moving object (hereinafter, referred to as an image moving object) detected by the image processing devices 5 to 7 input from the communication I / F unit 26 are input to the association determining unit 32.
[0029]
The moving object information predicting unit 31 reads a list of moving objects (hereinafter, referred to as a managed moving object) existing up to the previous frame from the moving object information managing unit 30, and calculates the current estimated position and the average RGB value of the moving object. It is calculated from the past position of the moving object and input to the association determination unit 32.
[0030]
The association determination unit 32 associates the managed moving object with the image moving object for each image processing device using the current estimated position and the average RGB value of the managed moving object, the real space position of the image moving object, and the average RGB value. . For example, the current estimated position of a certain managed object is (X, Y), the average RGB value is (R, G, B), the real space position of a certain image object is (x, y), and the average RGB value is ( r, g, b),
[0031]
(Equation 1)

[0032]
First, the similarity S is calculated. Here, Smax, w1, and w2 are constants set in advance.
[0033]
The association is determined by selecting at most one image moving object having the highest similarity S for each managed moving object for each image processing apparatus. Note that when S is negative, no association is performed, and there is a case where an image moving object does not correspond in a certain image processing apparatus. This corresponds to a case where the moving object comes out of the imaging area of the image input device, a case where the moving object is stationary, and a case where the management moving object is erroneously generated. Conversely, the same image moving object may be associated with a plurality of managed moving objects, or there may be an image moving object that is not associated with the managed moving object. The former case corresponds to the case where the moving objects overlap when viewed from the image input device, and the latter case corresponds to the case where a new moving object is generated.
[0034]
Next, a set of the real space position and the average RGB value of the managed moving object and the image moving object associated with the managed moving object is input to the moving object updating unit 34 from the association determining unit 32, and the management is performed. The real space position and the average RGB value of the image moving object not associated with the moving object are input to the moving object generating unit 33. It should be noted that no information on the managed moving object to which no image moving object is associated is input anywhere.
[0035]
The moving object generation unit 33 newly generates a managed moving object having the real space position and the average RGB value of the image moving object that is not associated with the managed moving object, and inputs the generated managed moving object to the moving object information management unit 30. However, the management object to be newly generated is limited to one in each frame, and when a plurality of image objects are input, information on other image objects is discarded. Thus, even if a new moving object occurs in the overlapping imaging area of the image input devices 2 to 4, a plurality of managed moving objects are not generated. It is unlikely that a plurality of new moving objects will occur in the monitoring area at the same time, and even in such a case, there is no practical problem since a managed moving object corresponding to the moving object ignored in the next frame is generated.
[0036]
The moving object updating unit 34 calculates the current position of the moving object from the set of the real space position and the average RGB value of the managed moving object and the image moving object associated with the managed moving object, and calculates the moving object information managing unit. Enter 30. First, the position of the associated image moving object and the average RGB value are averaged, and a weighted average with the current estimated position of the management moving object is calculated. For example, an image moving object is associated with a certain managed moving object one by one from the image processing device 5 for each 7, and the current estimated position of the managed moving object is (X, Y), and the average RGB value is (R, G, B). ), The real space position of the associated image moving object is (xi, yi), and the average RGB value is (ri, gi, bi) (i is 1 to 3 corresponding to the image processing devices 5 to 7, respectively) Then, the position Xnew of the new managed animal is
[0037]
(Equation 2)

[0038]
Calculate with Here, w1 is a constant larger than 0 and smaller than 1. Similarly, Ynew, Rnew, Gnew, and Bnew are calculated.
[0039]
The moving object information management unit 30 stores the new position of the managed moving object and the average RGB value input from the moving object updating unit 34. At this time, the positions of the management moving objects in a plurality of past frames are managed as moving object information, and the predicted position of the moving object can be calculated by the moving object information prediction unit 31. If no image moving object is associated with the managed moving object, that is, if the updated managed moving object is not input from the moving object updating unit 34, the managed moving object is deleted from the list.
[0040]
When a newly generated moving object is input from the moving object generation unit 33, the moving object information management unit 30 adds the moving object to the moving object list.
[0041]
The moving object management unit 30 transmits the updated or newly added management moving object list to the moving object integrated processing device 11 via the communication network 35.
[0042]
As described above, the integrated processing device 8 updates or newly generates a management moving object from the real space position and the average RGB value of the image moving object input from the image processing devices 5 to 7.
[0043]
As described above, the moving object tracking device 1 detects and tracks the moving object existing in the range imaged by the video input devices 2 to 4, and the current position of the moving object and its average RGB value for each frame. Is transmitted to the moving object integrated processing apparatus 11 via the communication network 35.
[0044]
FIG. 4 is a block diagram of the moving object integrated processing apparatus 11 according to the present embodiment. Hereinafter, the operation for each image frame will be described.
[0045]
The moving object information predicting unit 41 reads out a list of moving objects that existed in the past (hereinafter, referred to as a wide-area managed moving object) from the moving object information managing unit 40 for each frame, and reads the current real space position of the wide-area managed moving object. Is calculated and input to the association determination unit 42 together with the average RGB value.
[0046]
The association determination unit 42 tracks the managed moving object transmitted from the moving object tracking devices 1, 9 and 10 via the communication network 35 and the wide-area managed moving object estimated by the moving object information prediction unit 41. For each device, correspondence is established using the real space position and the average RGB value. For example, the current estimated position of a certain wide-area management object is (X, Y), the average RGB value is (R, G, B), the real space position of a certain management object is (x, y), and the average RGB value is When (r, g, b) is used, first, the similarity S is calculated by (Equation 1). Here, Smax, w1, and w2 are constants set in advance. The association is determined by selecting at most one management object having the highest similarity for each wide-area management object for each moving object tracking device. At this time, the management moving object once associated is not associated twice, and the management moving object having the next highest similarity is associated. When S is negative, the association is not performed, and the management moving object may not correspond to a certain moving object tracking device. In addition, there may be a managed moving object that is not associated with the wide-area managed moving object.
[0047]
Then, the association determination unit 42 inputs the set of the real space position and the average RGB value of the managed moving object associated with the wide-area managed moving object to the moving object updating unit 44. Note that a wide-area management moving object to which no management moving object is associated is also input to the moving object updating unit 44. When there is a managed moving object that is not associated with the wide-area managed moving object, the real space position and the average RGB value of the managed moving object are input to the moving object generating unit 43.
[0048]
The moving body updating unit 44 calculates the current position and average RGB value of the wide-area management moving object from the wide-area management moving object and one or a plurality of management moving objects associated therewith, and inputs them to the moving object information management unit 40. I do. For example, a management object is associated with a certain wide-area management object one by one for each of the moving object tracking devices 1, 9, and 10, the current estimated position of the wide-area management object is (X, Y), and the average RGB value is ( R, G, B), the real space position of the associated managed moving object is (xi, yi), and the average RGB value is (ri, gi, bi) (i is 1 to 3; , 9, 10), the position Xnew of the new wide-area management moving object is calculated by (Equation 2). Here, w1 is a constant larger than 0 and smaller than 1. Similarly, Ynew, Rnew, Gnew, and Bnew are calculated. Note that the real space position and the average RGB value of the wide-area management moving object that is not associated with one management moving object use the values predicted by the moving object information prediction unit 41 as they are.
[0049]
The moving object generation unit 43 newly generates a wide-area management moving object having a real space position and an average RGB value of the managed moving object that has not been associated with the wide-area management moving object, and inputs the generated object to the moving object information management unit 40. First, when the managed moving object remains in the moving object tracking device 1 without corresponding to the wide-area managed moving object, a new wide-area managed moving object having the position of the managed moving object and the average RGB value is newly generated. This is referred to as a provisional wide-area management animal. Next, when the managed moving object remains in the moving object tracking device 9 without being associated with the wide-area managed moving object, first, the similarity between the managed moving object and the existing temporary wide-area managed moving object. S is calculated according to (Equation 1), and when the similarity S exceeds a preset threshold, the average real space position and the average RGB value of the managed moving object and the provisional wide area managed moving object are calculated. Is reset to the temporary wide-area management object. If the similarity S does not exceed the threshold, a temporary wide-area management moving object is newly generated from the management moving object.
[0050]
Furthermore, the correspondence between the management moving object of the moving object tracking device 10 and the temporary wide area management moving object is similarly examined, and if the association is made, the real space position and the average RGB value of the temporary wide area management moving object are corrected, If it is not associated, a temporary wide area management moving object is newly generated. In the case where the correction is performed in association with the managed moving object of the moving object tracking device 10, the weight is not simply averaged, but is weighted according to the number of managed moving objects that have been previously associated with the provisional wide area managed moving object. Add and average. Then, the final remaining temporary wide area management moving object is input to the moving object information management unit 40 as a newly generated wide area management moving object.
[0051]
The moving object information management unit 40 receives the newly generated wide area management moving object from the moving object generation unit 43, adds the newly generated wide area management moving object to the list, and uses the updated information of the wide area management moving object input from the moving object changing unit 44 to the existing wide area management moving object. To update the wide-area management of the animal. The wide-area management moving object stores the position of the moving object and other characteristic amounts at a certain time in the past, and the moving object information prediction unit 41 can predict the characteristic amount such as the position.
[0052]
The moving object information managed by the moving object information management unit 40 is output to an external device such as a display device or an automatic warning device as needed.
[0053]
As described above, the moving object integrated processing device 41 uses the management moving object input from the moving object tracking devices 1, 9, and 10 to the moving object tracking device that detects the moving object existing in the wide monitoring area. Regardless, the same thing is recognized as the same thing and is managed centrally.
[0054]
Hereinafter, features of the wide area monitoring device according to the present embodiment will be described. For example, the video input devices 2 to 4 share an imaging area, there are three moving objects in the imaging area, and five moving objects including the moving object are detected as wide-area management moving objects. Assume that the remaining two moving objects are not in the imaging area of any of the video input devices. It is also assumed that no moving object is detected by another moving object tracking device.
[0055]
At this time, since the number of moving objects managed by the moving object information management unit 30 of the integrated processing apparatus 8 is three, the similarity of the association is calculated by (number of image moving objects) * (managed moving object) (Number of cameras) * (number of cameras) 27 times, and the moving object integrated processing apparatus 11 needs to calculate 15 similarities of (number of managed objects) * (number of wide-area managed objects) Thus, a total of 42 similarity calculations are required. Assuming a device for associating all image moving objects with the wide-area management moving object, it is necessary to calculate 45 similarities of (number of image moving objects) * (number of wide-area management moving objects) * (number of cameras). You have to do it on one computer. Therefore, according to the present invention, not only the number of similarity calculations can be reduced, but also the similarity calculations are distributed to different computers, so that the processing can be reduced and dispersed, and the processing can be performed at high speed.
[0056]
Further, the information of the nine image moving objects output from the image processing devices 5 to 7 does not flow to the network, and thus the communication amount can be reduced. Furthermore, even if the interface between the moving object tracking device 1 and the network fails, the other moving object tracking devices 9 and 10 operate, so that monitoring of a part of the imaging area becomes impossible, but other parts Has no effect.
[0057]
Further, since addition or deletion of the video input device affects only the corresponding moving object tracking device, it is easy to change the video input device. For example, even if a new video input device is added or the video input device is removed from the moving object tracking device 1, it is only necessary to change the processing in the integrated processing device 8, and the moving object tracking devices 9 and 10, the moving object integration device There is no need to change the processing of the processing device 11.
[0058]
Further, for example, when monitoring a three-story building, it is possible to construct a system that manages each video input device hierarchically for each floor by associating each moving object tracking device with each floor, and that is easy to grasp. it can.
[0059]
In this embodiment, a color TV camera is used for the video input devices 2 to 4. However, for example, an infrared camera or a black and white camera may be used, or a plurality of types of cameras may be mixed. In this case, although the feature amounts differ depending on the camera, it is not possible to use a uniform similarity as shown in (Equation 1). However, for example, the management moving object holds all the attributes of the feature amounts, and the corresponding feature amounts are Only when the moving object is set to the image moving object and the management moving object, the similarity may be calculated for each feature amount, and the similarity may be multiplied by an appropriate weight and added to perform the processing.
[0060]
In the image processing apparatus 5, the difference between frames is used for detecting a moving object. However, the moving object may be detected using a background difference, an optical flow, or the like.
[0061]
In the image processing device 5, the moving object is detected by using the moving area. For example, a characteristic portion such as a human head or a car tire is extracted, or the moving object is detected by using the moving area to generate a template. Alternatively, a region that matches the template may be detected as a moving object.
[0062]
Although the RGB value is used as the feature amount of the moving area calculated by the image processing apparatus 5, other display systems of colors, and feature amounts such as the size, shape feature, and texture of the area in the real space may be used. Good.
[0063]
In the image processing device 5, the position of the moving object in the real space is calculated from the center pixel of each label region. However, feature points of other regions may be used, or the image input device may share the imaging area. Alternatively, a method of calculating a real space position by stereo measurement from feature points captured by a plurality of video input devices may be used.
[0064]
In addition, the integrated processing device 8 deletes the management moving object that could not be associated with the image moving object, but keeps it for a certain period of time so as to cope with the case where the moving object is stationary or cannot be detected temporarily. It may be.
[0065]
In the moving object generation unit 33, the number of management moving objects newly generated in one frame is limited to one. However, as in the processing used in the moving object generation unit 43, a plurality of management objects input from separate image processing devices are used. If there are image moving objects, the moving objects may be associated with each other and a plurality of managed moving objects may be simultaneously generated.
[0066]
Although three video input devices are used in the moving object tracking device 1, one or more arbitrary number of video input devices may be used, and the moving object tracking devices 1, 9, and 10 use different numbers of video input devices. Is also good.
[0067]
Further, the same video input device may be shared by different moving object tracking devices, such as using the video input device 2 in the moving object tracking device 9.
[0068]
Further, the imaging ranges of the video input devices 2 to 4 may be overlapping or not overlapping, or may be mixed.
[0069]
(Embodiment 2)
FIG. 5 is a configuration diagram of the wide area monitoring device according to the second embodiment of the present invention. In FIG. 5, reference numerals 100 to 102 denote video input devices, and in this embodiment, a color TV camera is used. Reference numerals 103 to 105 denote image processing devices that process a color image obtained by the video input devices 100 to 102 for each frame image to extract a first moving object and its feature amount, and 106 to 108 denote a second animal. This is an associating device for associating a body with a first animal body input from each of the image processing devices 103 to 105 by using its feature amount. Reference numeral 109 denotes a moving object management device that manages a moving object existing in a monitoring area including the imaging area of the video input devices 100 to 102 as a second moving object. 110 denotes a second animal that is an output of the moving object management device 109. From the body information, a video input device that may have imaged the second moving object is selected from the video input devices 100 to 102, and the corresponding video input device is connected to the associating devices 106 to 108 connected to the selected video input device. A moving object information routing device that transmits information on a second moving object.
[0070]
Hereinafter, the operation of the image processing devices 103 to 105, the association devices 106 to 108, and the moving object management device 109 will be described in this order, and then the operation of the wide area monitoring device according to the present embodiment will be described.
[0071]
Since the image processing apparatuses 103 to 105 have the same function as the image processing apparatus 5 in the first embodiment, the description of the operation is omitted. Hereinafter, the first moving object is referred to as an image moving object, as in the first embodiment.
[0072]
Hereinafter, the operation of the wide area monitoring device according to the present embodiment will be described with reference to FIG. The color video input from each of the video input devices 100 to 102 is processed for each frame by the image processing devices 103 to 105 to extract an image moving object, calculate a real space position and an average RGB value of each image moving object, The information is input to the association devices 106 to 108, respectively. In addition, the moving object management apparatus calculates the real space position and the average RGB value of the management object currently managed at the current time, and inputs the calculated values to the moving object information routing apparatus 110. Then, the moving object information routing device 110 selects a video input device having an imaging area including the predicted real space position of each managed moving object from the video input devices 100 to 102, and associates the video input device with the selected video input device. Only the information of the management moving object is transmitted to the device.
[0073]
In the associating devices 106 to 108, the image moving object input from each of the image processing devices 103 to 105 and the management moving object transmitted from the moving object information routing device 110 are compared using the real space position and the average RGB value. The association, the result of the association, and the information on the image moving object are transmitted to the moving object management device 109. The moving object management device 109 calculates the current position and the average RGB value of the managed moving object based on the associated moving image, updates the managed moving object, and updates the managed moving object that is not associated with the managed moving object. A new managed animal is generated from
[0074]
FIG. 6 is a block diagram of the associating device 106 according to the present embodiment. Note that the associating devices 107 and 108 perform the same operation as the associating device 106, and a description thereof will be omitted.
[0075]
6, the real space position and the average RGB value of the image moving object are input from the image processing device 103 to the similarity calculation unit 121 via the communication I / F unit 26. Also, the predicted real space position and the average RGB value of the second moving object (hereinafter referred to as a management moving object) in the current frame are input to the similarity calculating unit 121 from the communication network 120.
[0076]
The similarity calculation unit 121 calculates a similarity S between each managed moving object and each image moving object. The calculation of the similarity S is performed by (Equation 1) similarly to the similarity between the image moving object and the management moving object in the first embodiment. Then, the calculated similarity and information on the management moving object and the image moving object are input to the association determining unit 122.
[0077]
The association determination unit 122 selects and associates at most one image moving object having the largest similarity S for each management moving object, and associates the result of the association with information on the image moving object via the communication network 120. This is transmitted to the moving object management device 109. At this time, the management moving object having the negative similarity S and the image moving object may not be associated with each other, and one image moving object may be associated with a plurality of management moving objects.
[0078]
As described above, the associating device 106 according to the present embodiment associates at most one image moving object with each managed moving object according to the feature amount, and associates the result of the association with the information of the image moving object with the moving object managing device. Input to 109.
[0079]
FIG. 7 is a block diagram of the moving object management device 109 according to the present embodiment. The association result input from the association devices 106 to 108 and the information on the image moving object associated with the management moving object are input to the moving object updating unit 131 via the communication network 120. The predicted real space position and the average RGB value of the managed moving object are input from the moving object information predicting unit 130 to the moving object updating unit 131. The image moving object that could not be associated with the management moving object is input to the moving object generating unit 132.
[0080]
The moving object update unit 131 calculates the average of the real space position and the average RGB value of the image moving object associated with each managed moving object, and uses the average value to calculate the real space position and the average RGB value of the managed moving object. Calculate the current value of. For example, the predicted current position of a certain managed object is (X, Y) and the average RGB value is (R, G, B). There is no corresponding image moving object in the associating device 108, and the real space position and the average RGB value of the associated image moving object are respectively (xi, yi), (ri, gi, bi) ( i is 1 and 2. The current value Xnew of the real space position of the management moving object is
[0081]
(Equation 3)

[0082]
Is calculated by Here, w1 is a predetermined constant. Similarly, Ynew, Rnew, Gnew, and Bnew are calculated. As the current real space position and the average RGB value of the management moving object to which no image moving object is associated, the predicted real space position and the average RGB value are used as they are. The moving object updating unit 131 calculates the current real space position and the average RGB value of the managed moving object as described above, and inputs them to the moving object information management unit 133.
[0083]
The moving object generation unit 132 newly generates a management moving object from the image moving object that is not associated with the management moving object, and inputs the generated management moving object to the moving object information management unit 133. First, when the image moving object remains in the association device 106 without corresponding to the management moving object, a management moving object having the position of the image moving object and the average RGB value is newly generated. This is called a temporary management animal. Next, when the image moving object remains in the association device 107 without being associated with the management moving object, first, the similarity S between the image moving object and the existing temporary management moving object is determined by ( If the similarity S exceeds a preset threshold value, the average real space position and the average RGB value of the image moving object and the temporary management moving object are calculated according to Expression 1). Set on the body. If the similarity S does not exceed the threshold, a temporary management moving object is newly generated from the image moving object. Furthermore, the association between the image moving object and the temporary management moving object of the association device 108 is similarly examined, and if the association is made, the real space position and the average RGB value of the temporary management moving object are corrected, and the association is made. If not, a new temporary management animal is created. In the case where the correction is made in association with the image moving object of the associating device 108, weighting is performed according to the number of image moving objects that have been previously associated with the temporary management moving object, instead of simply averaging. Average. Then, the finally managed temporary moving object is input to the moving object information management unit 133 as a newly generated managed moving object.
[0084]
The moving object information management unit 133 receives the newly generated managed moving object from the moving object generating unit 132, adds it to the list, and uses the updated managed moving object information input from the moving object updating unit 131 to update the existing managed animal. Update your body. Note that the management moving object stores the position of the moving object at a certain time in the past so that the moving object information prediction unit 130 can predict the position.
[0085]
The moving object information managed by the moving object information management unit 133 is output to a display device, an automatic warning device, or the like, as needed.
[0086]
The moving object information predicting unit 130 reads the current list of managed moving objects from the moving object information managing unit 133, predicts the real space position of the moving object in the next frame, and, together with the average RGB value, the moving object information routing device 110. Is input to the moving object updating unit 131.
[0087]
As described above, the moving object management apparatus 109 uses the association information between the management moving object and the image moving object input from the associating devices 106 to 108 and the information on the image moving object to store the moving object in the wide monitoring area. Regardless of the associating device that has detected the moving object, the same object can be recognized as the same and managed in an integrated manner.
[0088]
As described above, in the wide-area monitoring apparatus according to the present embodiment, the calculation of the association between the managed moving object and the image moving object is distributed to the associating devices 106 to 108, and the moving object managing device 109 updates the managed moving object and newly generates the managed moving object. , And since only the current position of the management moving object is predicted, the processing load is dispersed. Further, since the moving object information routing device transmits information of the managed moving object only to the associating device corresponding to the predicted current position of the managed moving object to perform the association, useless association processing is not performed. The load is reduced, and the traffic is reduced.
[0089]
(Embodiment 3)
FIG. 8 is a configuration diagram of a wide area monitoring system according to Embodiment 3 of the present invention. In FIG. 8, reference numerals 140 to 142 denote the mid-range monitoring devices, and reference numeral 144 denotes the first moving object integrated by the mid-range monitoring devices 140 to 142. The same moving object is the same as the second moving object. It is a moving object integrated processing device to be managed. Reference numeral 143 denotes a local area network (LAN) that mediates communication between the midrange monitoring devices 140 to 142 and the moving object integrated processing device 144. Reference numeral 145 denotes a wide area monitoring device including the wide area monitoring devices 140 to 142 and the moving object integrated processing device 144. Reference numerals 146 and 147 denote wide area monitoring apparatuses having the same configuration as the wide area monitoring apparatus 145. Reference numeral 149 denotes a moving object integrated management device that integrates the second moving objects detected by the wide area monitoring devices 145 to 147, and manages the same moving object as a third moving object as the same moving object. 148 is a wide area network (WAN) that mediates communication between the wide area monitoring devices 145 to 147 and the moving object integrated management device 149.
[0090]
Hereinafter, the operation of the wide area monitoring system according to the present embodiment will be described. The mid-range monitoring devices 140 to 142 operate in the same manner as the wide-area monitoring device according to the second embodiment, extract a first moving object from images input from a large number of image input devices, and extract the first moving object. The real space position and the average RGB value at each frame time are output via the LAN 143. Then, the moving object integrated processing device 144 receives the real space position and the average RGB value of the first moving object via the LAN 143, and executes the same processing as the moving object integrated processing device 11 of the first embodiment to monitor the middle region. The moving object existing in the wider monitoring area including the monitoring area of the devices 140 to 142 is identified as the same moving object, and is managed as the second moving object. Further, the moving object integrated processing device 144 outputs the real space position and the average RGB value of the second moving object at each frame time via the WAN 148.
[0091]
Next, the moving object integrated management device 149 receives the real space position and the average RGB value of the second moving object from the wide area monitoring devices 145 to 147 via the WAN 148, and the integrated moving object processing device 11 of the first embodiment. By the same processing, the moving object existing in the monitoring area of the wider area including the monitoring area of the wide area monitoring devices 145 to 147 is identified as the same moving object with the same moving object, and managed as the third moving object.
[0092]
As described above, in the wide area monitoring system according to the present embodiment, information in each monitoring area is retained inside each wide area monitoring apparatus, processing is dispersed and communication volume is reduced, and thus a large number of video input apparatuses are used. Even so, the processing time and the communication volume between the devices are prevented from increasing rapidly. Also, by managing the wide area monitoring devices in a hierarchical manner, the range affected by the addition of a video input device or the addition of a wide area monitoring device is minimized, so that it is possible to flexibly cope with system grasp and correction. I have to.
[0093]
In the present embodiment, the wide-area monitoring device according to the second embodiment is used for the middle-range monitoring devices 140 to 142, but the wide-area monitoring device according to the first embodiment may be used. The wide-area monitoring device of the above-described embodiment may be used in combination.
[0094]
Further, in this embodiment, the wide area monitoring system is managed in three layers, but may be configured in four or more layers.
[0095]
Further, in the present embodiment, the wide area monitoring devices 146 and 147 have the same configuration as the wide area monitoring device 145. For example, the wide area monitoring device 146 is the wide area monitoring device of the first embodiment, and the wide area monitoring device 147 is The wide area monitoring device according to the second embodiment may be configured.
[0096]
【The invention's effect】
As described above, according to the present invention, first, there is obtained an advantageous effect that the processing time and the communication amount between the devices do not increase rapidly even when a plurality of video input devices such as video cameras are used.
[0097]
Secondly, an advantageous effect is obtained in which a large number of video input devices are hierarchically managed, the system can be easily grasped, and correction can be flexibly performed.
[Brief description of the drawings]
FIG. 1 is a configuration diagram of a wide area monitoring device according to a first embodiment of the present invention.
FIG. 2 is a block diagram of an image processing device of the wide area monitoring device according to the first embodiment of the present invention.
FIG. 3 is a block diagram of an integrated processing device of the wide area monitoring device according to the first embodiment of the present invention;
FIG. 4 is a block diagram of a moving object integrated processing device of the wide area monitoring device according to the first embodiment of the present invention;
FIG. 5 is a configuration diagram of a wide area monitoring device according to a second embodiment of the present invention.
FIG. 6 is a block diagram of an associating device of a wide area monitoring device according to the second embodiment of the present invention;
FIG. 7 is a block diagram of a moving object management device of the wide area monitoring device according to the second embodiment of the present invention.
FIG. 8 is a configuration diagram of a wide area monitoring system according to a third embodiment of the present invention.
FIG. 9 is a configuration diagram of a conventional moving object tracking system.
FIG. 10 is a block diagram of a main part of a conventional moving object monitoring apparatus.
FIG. 11 is a configuration diagram of a conventional distributed monitoring device.
[Explanation of symbols]
1 Moving object tracking device
2,3,4 Video input device
5, 6, 7 image processing device
8 Integrated processing equipment
9 Moving object tracking device
10. Moving object tracking device
11 Integrated animal processing equipment
20 A / D converter
21, 22 Image memory
23 Difference calculator
24 Labeling part
25 Feature Calculator
26 Communication I / F
30, 40 Animal Information Management Department
31, 41 Animal body information prediction unit
32, 42 association determination unit
33, 43 Animal body generation unit
34, 44 Animal Renewal Department
35 Communication Network
100, 101, 102 Video input device
103, 104, 105 Image processing device
106, 107, 108 associating device
109 Animal body management device
110 Animal body information routing device
120 communication network
121 Similarity calculator
122 Correlation Determination Unit
130 Animal Information Prediction Unit
131 Animal Renewal Department
132 Animal body generator
133 Animal Information Management Department
140, 141, 142 Mid-range monitoring device
143 Local Area Network
144 animal body integrated processing device
145, 146, 147 Wide area monitoring device
148 Wide Area Network
149 Animal body integrated management device

Claims (3)

L image input devices, L image processing devices respectively calculating a first moving object and its characteristic amount from images acquired by the L image input devices, and outputs of the L image processing devices And a monitoring apparatus including an L number of associating devices for associating the first moving object and its characteristic amount with the transmitted information of the second moving object, and an area captured by the L image input means. A moving object management device that manages information on moving objects existing in the area and outputs the information as a second moving object as necessary, and a second moving object information from the second moving object information output from the moving object management device. A moving object information routing device that selects the associating device connected to the video input device that may be imaging the second moving object and transmits information of the second moving object thereto; Whether the body management device is the L corresponding devices The correlation results and the first animal body associated and wide area monitoring system, characterized in that the detection and tracking of moving objects by updating the information of the second moving object from its feature quantity. An animal that integrates a moving object tracking device that is part or all of the wide area monitoring device according to claim 1 and a moving object detected by the plurality of moving object tracking devices, identifies the same moving object, and manages the moving object. A wide area monitoring system comprising a body integrated processing device. 3. A moving object tracking device which is a wide area monitoring device according to claim 1 or 2, and a moving object detected by the plurality of moving object tracking devices are integrated to identify the same moving object and to move the moving object. A wide-area monitoring system, comprising:

Images