JP4167777B2 - VIDEO DISPLAY DEVICE, VIDEO DISPLAY METHOD, AND RECORDING MEDIUM CONTAINING PROGRAM FOR DISPLAYING VIDEO - Google Patents

Description

【０１３２】
となり、ズーム画角θの値を求めることができる。
【０１３３】
さらに被写体情報（図５）に、被写体の大きさを表す外接多角形情報を記述することによって、被写体がそのカメラにどれくらいの大きさで撮影されるかを計算することができる。外接多角形情報とは、被写体の形状が複雑であったり曲線部分を含むような場合、その被写体の存在領域を近似的に表すために用いられる、その被写体に外接する２次元または３次元の多角形の情報をいう。
【０１３４】
図１５は、撮影範囲と被写体の関係と撮影状態を示す。
【０１３５】
図１５（ａ）は、撮影範囲の中央に被写体（文字「Ａ」）が存在する場合を示す。文字「Ａ」の外接多角形情報は、図示されるように文字「Ａ」に外接する長方形である。このとき図示されるように、被写体は映像の中央部に存在する。図１５（ｂ）は、撮影範囲の一部（撮影範囲の下方）に被写体（文字「Ａ」）が存在する場合を示す。このとき、撮影された映像の下方に被写体が存在する。なお、実際には被写体の全体は表示されないが、撮影面と被写体との関係を明確にするために被写体の全体を図示している。
【０１３６】
図１５（ａ）および（ｂ）に示すような場合に、被写体がどの程度写っているかを定量的に計算する手順を以下に説明する。例えば図１４に示されるように、まず撮影面を（Ｘ−Ｙ）平面とする。つづいて被写体の近傍に存在しカメラの撮影方向に直交する（ｘ−ｙ）平面を考え、この（ｘ−ｙ）平面に被写体の外接多角形を射影する。ここで、（ｘ−ｙ）平面と撮影方向が交わる点を（ｘ−ｙ）平面の原点とする。この（ｘ−ｙ）平面において、射影した外接多角形の１つの頂点座標を（ｘ０，ｙ０）とすると、その頂点に対応する撮影面（Ｘ−Ｙ平面）上の点の座標（Ｘ０，Ｙ０）は、
【０１３７】
【数２】

【０１３８】
で与えられる。ここで、Ｌは、レンズから被写体が射影された平面（すなわちｘ−ｙ平面）までの距離を表す。（ｘ−ｙ）平面は被写体の近傍に存在するので、被写体の位置情報を、レンズから（ｘ−ｙ）平面までの距離として利用してもよい。このような手順によれば、外接多角形の頂点に対応する撮影面上での座標を得ることができる。座標が得られると、その座標によって囲まれる撮影面上の多角形の面積と、撮影面の面積との比を算出し、撮影面に占める被写体の割合を近似的に求めることができる。
【０１３９】
撮影範囲情報を利用することにより、図１５（ｂ）に例示したように、カメラの撮影範囲と被写体の存在位置との関係で被写体の一部しか撮影されていない場合には、被写体がカメラに写る割合（面積比）と被写体の被写体重要度とを乗算した値をその映像の映像重要度とすれば、被写体重要度の高い被写体がよく写っている映像を高い映像重要度を持つ映像として優先的に表示することができる。
【０１４０】
このように映像重要度を計算すると、撮影範囲が固定されたカメラだけでなく、移動カメラや首振りカメラで撮影した動画であっても、各画像フレーム毎に撮影範囲情報を記述しておくことにより、重要な被写体が写っている動画部分のみを抜き出して優先的に表示することができる。
【０１４１】
本実施の形態で説明した全ての場合において上述した映像の表示態様は一例であり、表示規則を適当に変更することによってより柔軟な映像表示を実現することができる。上述した例（２−２．（ｂ））で表示規則として規定していたのは、表示されるのが重要度の高いＮ個の映像であること、地図上での映像の表示位置は撮影位置の近傍とすること、同じ位置またはかなり近い位置で撮影した映像のグループは撮影時刻の古いものから表示すること、および同一のグループに属する映像のうち、代表映像以外の映像をオペレータの操作により選択的に表示することである。しかし、例えば、閾値以上の映像重要度を有する映像すべてを表示したり、閾値以上でＮ個を超えない映像を表示する等の表示規則が適用でき、オペレータの目的に応じて使い分けることができる。
【０１４２】
さらに、映像重要度の低く表示されない映像を、例えばカメラの形をしたアイコンとして表示された地図上の撮影位置近傍に表示し、オペレータが地図上でそのカメラアイコンを選択して映像表示を指示するようにすれば、どのような映像重要度であっても任意の映像を簡単な操作により見ることができる。
【０１４３】
（実施の形態３）
３−１．本実施の形態における映像表示装置の動作概要
図１６は、本実施の形態における映像表示装置１６０の構成を示すブロック図である。
【０１４４】
本実施の形態における映像表示装置１６０は、映像の重要性に応じて複数の映像を表示する。映像の重要性は、映像に含まれる被写体に設定された被写体重要度に基づいて決定される。映像表示装置１６０は、映像の表示中であっても被写体重要度を変更することができ、その変更を映像の表示にリアルタイムに反映させる。例えば、プラント監視において燃料バルブのセンサからアラームが発生すると、オペレータや運転制御を行う外部モジュールからの指示により、燃料バルブ付近に存在する被写体や燃料系の被写体の被写体重要度を上げ、その被写体を含む映像重要度を上げる。これにより、その近辺や系統的に繋がりのある被写体が優先的に表示される。
【０１４５】
３−２．本実施の形態における映像表示装置の構成および動作
（ａ）本実施の形態における映像表示装置の構成
映像表示装置１６０は、複数のカメラ１１ａ〜１１ｎと、映像情報取得部１２と、被写体情報管理部１３と、被写体情報格納部１４と、映像重要度計算部１５と、表示制御部１６と、表示部１７とを含む。映像表示装置１６０はさらに、入力部１６１および／または外部モジュール１６２とを含む。
【０１４６】
以下では、映像表示装置１６０の構成要素のうち、映像表示装置１０（図１）の構成要素と同一の符号が付された構成要素については、映像表示装置１０（図１）の構成要素の構成および機能と同一であるのでその説明は省略する。したがって、以下では、入力部１６１と外部モジュール１６２とについて説明する。映像表示装置１０（図１）と異なる動作または追加の動作が行われる場合には、その都度説明する。
【０１４７】
入力部１６１は、オペレータが被写体情報を修正、変更して設定するための入力装置である。入力部１６１は、例えばキーボード等であり得る。入力部１６１から変更、設定される被写体情報は被写体情報管理部１３に送信される。
【０１４８】
外部モジュール１６２は、プラント状態を計算するシミュレータ等であり、運転制御を行う。外部モジュール１６２から変更、設定される被写体情報は被写体情報管理部１３に送信される。
【０１４９】
被写体情報管理部１３は、入力部１６１および／または外部モジュール１６２から被写体情報を変更するよう指示を受け取り、さらに新たな被写体情報を受け取る。被写体情報管理部１３は、受け取った新たな被写体情報を対応する前の被写体情報にかえて被写体情報格納部１４に格納する。なお被写体情報の変更指示は、新たな被写体情報と別個に受け取られるのではなく、被写体情報管理部１３が新たな被写体情報を受け取ったときに与えられたとみなしてもよい。
【０１５０】
（ｂ）本実施の形態における映像表示装置の動作および映像表示例
図１７は、映像表示装置１６０（図１６）の動作手順を表すフローチャートである。このフローチャートに示されたステップのうち、被写体重要度に基づいて映像重要度を計算し、映像情報が表す映像を表示するステップ１７０〜１７３は、図４を参照して説明したフローチャートのステップ４０〜４３とそれぞれ同じであるので、その説明は省略する。
【０１５１】
ステップ１７４において、被写体情報が被写体情報管理部１３（図１６）によって変更されたか否かが判定される。
【０１５２】
被写体情報が被写体情報管理部１３（図１６）によって変更されたと判定されると（すなわち入力部１６１（図１６）または外部モジュール１６２（図１６）から被写体重要度を変更するよう指示を受け取った被写体情報管理部１３（図１６）が、所定の被写体情報を受け取った新たな被写体情報に変更すると）、被写体情報管理部１３（図１６）は、新たな被写体情報を被写体情報格納部１４（図１６）に格納する（ステップ１７５）。映像重要度計算部１５（図１６）は、変更された新たな被写体情報に基づいてリアルタイムかつ自動的に映像重要度を変更し、表示する映像を変更する。これはステップ１７５に続いて、ステップ１７０〜１７３が次々実行されることにより実現される。
【０１５３】
被写体情報が被写体情報管理部１３（図１６）によって変更されていないと判定されると、変更の指示があるまで待機し、映像の表示を続行する。なお、被写体情報の変更指示があるまでステップ１７２〜１７４からの処理を繰り返すことにより、映像の表示を次々に行うことができる。これは、図４に示されるステップ４２〜４４およびステップ４７の処理を繰り返し行うことと同じである。
【０１５４】
なお、映像情報が表す映像は、動画であっても静止画であってもよく、さらにはそれらの混在であってもよい。表示する表示規則に基づいて、映像が表示されるのは既に説明したとおりであるが、その表示規則は任意に変更しうる。
【０１５５】
以上のように本実施の形態によれば、映像表示中であっても、被写体情報管理部１３（図１６）は、入力部１６１（図１６）や外部モジュール１６２（図１６）からの要求に従い、被写体情報に含まれる被写体重要度を変更する。表示制御部１６（図１６）は、被写体情報の変更が反映された結果変更された映像重要度に応じて、複数の映像の表示を自動的に変更する。したがって、注目すべき被写体の種類や位置などが状況に応じて変化する場合でも、映像重要度の高い映像が自動的に選択表示され、オペレータは素早く的確な被写体観察ができる。言うまでもなく、実施の形態１で説明した映像表示装置１０（図１）の効果も得ることができる。
【０１５６】
（実施の形態４）
４−１．本実施の形態における映像表示装置の動作概要
実施の形態１〜３では、映像重要度の高い映像から順に時間を変化させ、または複数の映像のうちＮ個の映像を選択して表示した。本実施の形態では、映像重要度に従って、映像の表示サイズを変更し、または映像を表示するウインドウの枠の太さや色を変更する強調表示の度合いを変化させる。
【０１５７】
４−２．本実施の形態における映像の表示
本実施の形態において用いられる映像表示装置は、実施の形態１〜３で説明した映像表示装置のいずれでもよい。映像表示装置１０、９０、１６０（図１、９、１６）は、表示する映像の映像重要度に応じて以下のように映像を表示する。
【０１５８】
図１８は、映像重要度の高い映像ほど表示サイズを大きくする表示画面の例を示す。
【０１５９】
図１８（ａ）は、実施の形態２にかかる映像表示装置１０（図１）を用いて映像を表示させた例を示す。表示画面は１０分割され、映像重要度が最も高い映像と２番目に高い映像とが画面の上半分に、その他の映像が画面の下半分に表示されている。画面の上半分に表示された映像の表示サイズは、他の映像の表示サイズの例えば４倍である。
【０１６０】
図１８（ｂ）は、実施の形態２にかかる映像表示装置９０（図９）を用いて映像を表示させた例を示す。撮影された位置に対応する地図上の位置に表示される映像のうち、映像重要度が大きい順に２つの映像が拡大されて表示され、映像重要度が小さい映像ほど、小さく表示されている。
【０１６１】
図１８（ｃ）は、実施の形態２にかかる映像表示装置９０（図９）を用いて映像を表示させた例を示す。撮影された位置に対応する地図上の位置に表示される映像のうち、映像重要度が大きい映像ほど、映像を表示するウィンドウの枠が太く、色が濃く表示されている。
【０１６２】
図１９は、映像重要度に応じて映像の強調度（表示サイズ）を変更するための表示規則の例を示す。図１８（ａ）〜（ｃ）で用いられた映像表示装置１０、９０、１６０（図１、９、１６）の表示制御部１６（図１、９、１６）は、このような表示規則と映像重要度とを用いることによって、上述のような映像の表示を実現することができる。
【０１６３】
図１９（ａ）は、映像重要度の高い映像の表示サイズをより大きく表示するための表示規則であり、例えば１番目の映像重要度の映像を表示するサイズは４８０×３２０ドット、２番目は４５０×３００ドットなどと規定している。
【０１６４】
図１９（ｂ）は、映像重要度の高い映像の表示ウィンドウの枠を太く、色を変更して表示する規則であり、例えば、１番目はのウィンドウ枠は３ポイントの赤色、２番目は２．５ポイントの橙色などと規定している。色は目立つ色から順に用いられるようにしてもよい。なお、上述した強調表示の表示規則は一例にすぎず、他の例として、重要な映像のウィンドウ枠をブリンク表示する、重要な映像のコントラストや輝度を上げる等、オペレータが映像表示装置１０、９０、１６０（図１、９、１６）を利用する目的に応じてさまざまな表示規則を規定し、変更することができる。
【０１６５】
これらの例においては、映像情報が表す映像は、動画であっても静止画であってもよく、さらにはそれらの混在であってもよい。
【０１６６】
以上のように本実施の形態によれば、表示規則と映像重要度とに基づいて、表示制御部１６（図１、９、１６）が映像の強調表示の度合いを変化させて表示するようにしたので、重要な映像ほど大きく表示する、または重要な映像を表示するウインドウの枠の色を目立つようにする等、被写体の重要性をより意識して観察することができる。
【０１６７】
（実施の形態５）
５−１．本実施の形態における映像表示装置の動作概要
実施の形態１〜４は、映像は静止画または動画のいずれでもよかったが、本実施の形態では、映像は動画である。本実施の形態では、映像重要度が高い映像ほど映像の品質を高くして表示する。
【０１６８】
５−２．本実施の形態における映像の表示
本実施の形態において用いられる映像表示装置は、実施の形態１〜４で説明した映像表示装置のいずれでもよい。映像表示装置１０、９０、１６０（図１、９、１６）は、表示する映像の映像重要度に応じて以下に説明する映像を表示する。
【０１６９】
図２０は、映像重要度の順番にしたがって映像の表示品質を変更するための表示規則の例を示す。
【０１７０】
表示制御部１６（図１、９、１６）は、映像重要度とこの表示規則とに基づいて映像を表示する。例えば、１番目の映像重要度の映像を表示する表示品質は、３０フレーム／秒で解像度が４８０×３２０、２番目は２５フレーム／秒の４５０×３００、…となる。フレームレートが高いほど滑らかな動きを表示することができ、解像度が高いほど映像を緻密に表示することができる。周知のように、動画は、時間的に連続した複数の画像フレームを表示したときに実現される。
【０１７１】
以上のように本実施の形態によれば、表示制御部１６（図１、９、１６）は表示規則および映像重要度に基づいてフレームレートや解像度等の映像品質の度合いを変化させて表示するよう映像の表示を制御する。したがって映像重要度の高い映像（すなわち重要な映像）がより見やすくかつ緻密に表示されるので、オペレータは自然と重要な映像を注視することができる。さらに、比較的映像重要度が低い映像（すなわち比較的重要でない映像）は画像フレームが間引かれて表示されるので、映像表示装置１０、９０、１６０（図１、９、１６）にかかる表示処理の負荷は軽くなり、比較的安価な処理回路を用いても映像表示装置１０、９０、１６０（図１、９、１６）を製造することができる。したがって、映像表示装置１０、９０、１６０（図１、９、１６）の製造コストを低く抑えることができる。
【０１７２】
なお、ＪＰＥＧ（Ｊｏｉｎｔ Ｐｈｏｔｏｇｒａｐｈｉｃ Ｅｘｐｅｒｔｓ Ｇｒｏｕｐ）のようなフレーム間符号化のない符号化方式を用いて圧縮・記録されている場合は、フレームレートに合わせて必要な映像情報だけを効率的に読み出すことができる。さらに本発明による映像表示装置９０（図９）を用いると、映像情報格納部９３（図９）に同じ映像を異なる解像度で、または映像の低周波数成分から高周波数成分を何段階かに分けて符号化（階層的符号化）して、映像情報を格納しておき、表示に際して必要な解像度の映像情報のみを読み出すようにすることができる。これにより、読み出し処理の負荷を軽くすることができる。なお、階層的符号化は公知の技術であり、本発明の本質ではないので説明は省略する。なおＪＰＥＧが静止画に多く利用される符号化であることを鑑みると、本実施の形態の説明は、動画のみではなく静止画にも応用することができる。静止画を利用する場合には、フレームレートに関する条件を無効にし、解像度のみを有効にすればよい。
【０１７３】
本実施の形態における映像表示装置１０、９０、１６０（図１、９、１６）では、その装置が有する映像の読み出し速度、表示性能等の処理能力に応じて、その表示能力を超えないよう映像に表示品質を設定することにより、その装置が有する能力を最大限使った品質のよい複数の映像を表示することができる。
【０１７４】
（実施の形態６）
６−１．本実施の形態における映像表示装置の動作概要
図２１は、本実施の形態における映像表示装置２１０の構成を示すブロック図である。
【０１７５】
本実施の形態における映像表示装置２１０は、実施の形態５で説明したと同様、映像重要度が高い映像ほど映像の品質を高くして表示する。ただし、本実施の形態の映像表示装置２１０は、映像を映像情報として受け取り、格納し、ネットワークを介してその映像情報を配信する映像サーバ２１１と、ネットワークから映像情報を受信し、表示する映像表示端末２１２とを含む一体的な装置である。映像サーバ２１１は、映像表示端末２１２から所定の映像品質の映像情報を送信するよう要求されると、その映像品質で映像情報を映像表示端末２１２に送信する。
【０１７６】
６−２．本実施の形態における映像表示装置の構成および動作
（ａ）本実施の形態における映像表示装置の構成
映像表示装置２１０は、複数のカメラ１１ａ〜１１ｎと、映像サーバ２１１と、映像表示端末２１２とを含む。映像サーバ２１１は、映像情報を格納する映像データベース的な役割を果たし、要求に応じて保持している映像情報を提供する。映像表示端末２１２は必要な映像情報を映像サーバ２１１に要求し、受け取った映像情報を表示する。
【０１７７】
複数のカメラ１１ａ〜１１ｎで撮影された映像は、映像情報として映像サーバ２１１へ送信される。映像サーバ２１１は、複数のカメラ１１ａ〜１１ｎから送信された映像情報を受け取って蓄積し、映像表示端末２１２からの要求に応答して、映像情報を映像表示端末２１２へ送信する。映像表示端末２１２は、映像サーバ２１１へ映像情報の送信を要求し、送信された映像情報を受け取った後、その映像情報が表す映像を表示する。
【０１７８】
映像サーバ２１１は、映像情報取得部１２と映像情報格納部２３と送受信部２１３とを含む。
【０１７９】
以下では、映像表示装置２１０の構成要素のうち、映像表示装置１０（図１）および映像表示装置９０（図９）の構成要素と同一の符号が付された構成要素については、映像表示装置１０（図１）および映像表示装置９０（図９）の構成要素の構成および機能と同一であるのでその説明は省略する。既に説明した動作と異なる動作または追加の動作が行われる場合には、その都度説明する。
【０１８０】
映像情報取得部１２は、映像表示端末２１２から送信された所定の映像情報の送信要求を送受信部２１３を介して受け取り、その映像情報を映像情報格納部９３から読み出す。映像情報取得部１２は、読み出した映像情報を送受信部２１３を介して映像表示端末２１２に送信する。映像情報格納部９３には、同じ映像を表すが異なる表示品質の映像情報が複数格納されている。
【０１８１】
送受信部２１３は、映像情報等のデータを送受信するための機能を有する。送受信部２１３は、映像表示端末２１２から送信された映像情報送信要求その他の情報を受信し、映像情報取得部１２へ送信する。送受信部２１３は、映像情報取得部１２から映像情報その他の情報を受け取り、映像表示端末２１２へ送信する。送受信部２１３は、映像サーバ２１１をネットワーク回線等の情報通信媒体に接続する。
【０１８２】
映像表示端末２１２は、送受信部２１４と、被写体情報管理部１３と、被写体情報格納部１４と、映像重要度計算部１５と、表示制御部１６と、表示部１７とを含む。
【０１８３】
表示制御部１６は、表示規則と映像重要度計算部１５において計算された映像重要度とに基づいて、表示すべき映像の映像品質を決定する。表示制御部１６は、送受信部２１４を介して、その映像品質を有する映像情報の送信を映像サーバ２１１に要求する。映像表示部１６は、送受信部２１４を介して、映像サーバ２１１から送信された映像情報を表示するよう表示を制御する。
【０１８４】
送受信部２１４は、映像情報等のデータを送受信するための機能を有する。送受信部２１４は、表示制御部１６から映像情報送信要求その他の情報を受け取り、映像サーバ２１１へ送信する。送受信部２１４は、映像サーバ２１１から送信された映像情報その他の情報を受信し、表示制御部１６へ送信する。送受信部２１４は、映像表示端末２１２をネットワーク回線等の情報通信媒体に接続する。
【０１８５】
（ｂ）本実施の形態における映像表示装置の動作および映像表示例
図２２は、映像表示装置２１０（図２１）の動作手順を表すフローチャートである。
【０１８６】
被写体情報に基づく映像重要度の計算（ステップ２２０）および計算した映像重要度の格納（ステップ２２１）については、図４のステップ４０および４１と同じであるのでその説明は省略する。
【０１８７】
ステップ２２２では、表示制御部１６（図２１）は、映像重要度および表示規則に基づいて表示すべき映像の表示品質を決定する。表示品質とは、実施の形態５に関連して説明したように、映像重要度に応じて決定される動画のフレームレートおよび解像度（図２０）を表す。映像の表示品質を決定すると、表示制御部１６（図２１）は、その映像の映像情報の送信を映像サーバ２１１（図２１）に要求する。この要求は、表示すべき映像の映像ＩＤおよびその映像の映像品質を送信することによって行われる。送信を要求される映像情報は、１つに限らず、表示可能な数の映像に対応する複数の映像情報であり得る。
【０１８８】
ステップ２２３では、映像サーバ２１１（図２１）が要求された映像情報を読み出し、映像表示端末２１２（図２１）に送信する。映像表示端末２１２（図２１）の表示制御部１６（図２１）は、送信されてきた映像情報を受信する。なお映像情報の受信は静止画であれば１度で終了するが、動画の場合には以下に説明するステップが行われている間にも連続して行われることがあり得る。
【０１８９】
ステップ２２４では、表示制御部１６（図２１）は、表示規則に基づいて受信した映像情報が表す映像を表示する。本実施の形態は映像の表示品質に関しているので、実施の形態１〜４に説明した表示態様と組み合わせることで、実施の形態１〜４で説明したと同じ効果を得ることができる。なお、実施の形態１〜４で説明した映像表示装置には、本実施の形態で説明した映像表示装置２１０（図２１）には示されていない構成要素が含まれている場合がある。その場合であっても、全く同様の構成要素を映像表示装置２１０（図２１）に設ければよい。
【０１９０】
最後に、ステップ２２５では、表示制御部１６（図２１）は、オペレータ等から映像の表示を終了する指示を受け取ったか否かが判定される。この映像の表示を中止する指示は、オペレータ等によって、外部から与えられる。
【０１９１】
映像の表示を終了する指示を受け取っていない場合には、再びステップ２２２からの処理を繰り返し、映像の表示を続行する。例えば、本実施の形態による映像表示装置２１０（図２１）に実施の形態１にかかる映像表示装置１０（図１）と同様の表示パターンで映像の表示を行わせる場合には、スムーズかつ高い解像度で表示された映像重要度の高い映像のグループから、間引かれかつ低い解像度で表示された映像重要度の低い映像のグループへと表示時間を変えながら映像が繰り返し表示されることになる。
【０１９２】
映像の表示を終了する指示を受け取った場合には、映像の表示を終了する。
【０１９３】
以上のように本実施の形態によれば、表示制御部１６（図２１）は表示規則および映像重要度に基づいてフレームレートや解像度等の映像品質の度合いを変化させて表示するよう映像の表示を制御する。したがって映像重要度の高い映像（すなわち重要な映像）がより見やすくかつ緻密に表示されるので、オペレータは自然と重要な映像を注視することができる。さらに、比較的映像重要度が低い映像（すなわち比較的重要でない映像）は画像フレームが間引かれて表示されるので、映像表示装置２１０（図２１）にかかる表示処理の負荷を軽くすることができ、比較的安価な処理回路を用いても映像表示装置２１０（図２１）を製造することができる。したがって、映像表示装置２１０（図２１）の製造コストを低く抑えることができる。表示規則は、目的に応じて適切に変更することができる。
【０１９４】
さらに、映像サーバ２１（図２１）と映像表示端末２１２（図２１）とがネットワークで接続された構成であっても、ネットワーク上は表示に必要な映像品質の情報しか伝送されないため、ネットワーク資源やサーバおよび端末の計算機資源を有効に利用することができる。それらの資源が有する伝送帯域や処理速度等の性能に応じてその性能を超えないよう映像に映像品質を設定することにより、そのネットワークベースの映像表示装置２１０（図２１）が有する能力を最大限使った品質のよい複数の映像を表示することができる。
【０１９５】
本実施の形態では、映像情報格納部９３には同じ映像を表す異なる表示品質の映像情報が複数格納されているとしたが、１つの高解像度映像を格納し、解像度を適宜変換して必要な表示品質の映像にしてもよい。そのためには、読み出した符号化データを、その直交変換領域（ＪＰＥＧの場合は離散コサイン変換した周波数領域）を維持したまま解像度を下げる等の幾何変換を行う公知のアルゴリズムを利用することが好ましい。このアルゴリズムは、映像サーバ２１１（図２１）に高解像度映像をＪＰＥＧ等の圧縮符号化して記録され、その後読み出された符号化データに対して適用される。読み出された符号化データは幾何変換により解像度の低い符号化データに変換され、伝送される。このアルゴリズムによれば、一度イメージデータに復号化して解像度を低下させ、その解像度が低い映像を再度符号化する場合に比べ、処理時間は大幅に短縮される。さらに、同じ映像を表す表示品質の異なる複数の映像情報を格納する領域を設ける必要がなくなり、有効に記憶領域を利用することができる。
【０１９６】
（実施の形態７）
７−１．本実施の形態における映像表示装置の動作概要
本実施の形態では、映像を構成する複数の画像フレームそれぞれに着目する。すなわち、１枚の画像フレームの映像を、例えばそれぞれ１つの被写体を含む複数の領域（部分空間）に分割し、部分空間それぞれを異なる態様で強調して表示する。複数の画像フレームそれぞれに同様の処理を施すことにより、映像に含まれるそれぞれの被写体を強調して表示することできる。
【０１９７】
７−２．本実施の形態における映像の表示
本実施の形態において用いられる映像表示装置は、実施の形態１〜６で説明した映像表示装置のいずれでもよい。映像表示装置１０、９０、１６０、２１０（図１、９、１６、２１）の表示制御部１６は、表示する映像の映像重要度に応じて以下のように映像の表示を制御する。
【０１９８】
図２３は、３つの被写体を撮影するカメラと、本実施の形態による処理を施されて表示された被写体の例を示す。３つの被写体「Ａ」、「Ｂ」および「Ｃ」に設定された被写体重要度は、それぞれ０．８、１．０および０．６とする。撮影された映像は、画像フレームの映像毎に複数の部分領域に仮想的に分割され、被写体「Ａ」、「Ｂ」および「Ｃ」はそれぞれ別個の部分空間に存在する。
【０１９９】
図２３（ａ）は、被写体「Ａ」、「Ｂ」および「Ｃ」が存在する部分領域の映像重要度に応じて、表示される被写体を囲む枠の色を変えて映像を表示させた例を示す。図２３（ａ）では、被写体「Ｂ」が写っている中央の領域を濃い色の枠で囲い、被写体「Ａ」または被写体「Ｃ」が写っているその左右の部分領域をより薄い色の枠で囲む。枠の色は、その部分領域の映像重要度に基づいて決定される。部分領域の映像重要度は、映像重要度計算部１５（図１、９、１６、２１）によって、映像が表示されるその瞬間瞬間の画像フレームの部分領域それぞれについて計算される。映像重要度は、その領域内に存在する被写体の被写体重要度に基づいて計算される。以上のように画像フレームを部分領域に分割し、その部分領域毎に映像重要度を規定したので、部分領域毎に映像重要度に応じた濃さの枠で映像を部分領域毎に強調して表示することができるようになる。
【０２００】
なお本実施の形態では、映像情報が表す映像は、動画であっても静止画であってもよく、さらにはそれらの混在であってもよい。
【０２０１】
図２３（ｂ）は、被写体「Ａ」、「Ｂ」および「Ｃ」が存在する部分領域の映像重要度に応じて、表示される被写体の表示品質を変えて映像を表示させた例を示す。表示品質とは、動画の場合には実施の形態５に関連して説明したように、映像重要度に応じて決定されるフレームレートおよび解像度（図２０）を表す。静止画の場合には、その静止画の解像度等を表す。部分領域毎の映像重要度の決定手順については図２３（ａ）においてした説明と同じである。図２３（ｂ）では、被写体「Ｂ」が存在する中央の領域は一番映像品質を高く、次に被写体「Ａ」が存在する領域、その次は被写体「Ｃ」が存在する領域と映像品質を下げている。部分領域毎に映像の品質を変化させた符号化映像情報を生成することができる符号化方式として、ＪＰＥＧ２０００などが既に開発されているが、公知の技術であるため説明は省略する。映像は、リアルタイムの映像であっても、一旦映像情報格納部９３（図９、２１）に格納された映像であってもよい。
【０２０２】
なお、画像フレームのどの領域にどの被写体が存在するかについては、実施の形態２の変形例において図１４、１５を参照して説明したように、各画像フレームに対して撮影範囲情報と被写体情報に被写体の位置と大きさを示す外接多角形等の情報とを用いて、幾何的に計算することができる。
【０２０３】
以上のように本実施の形態によれば、１枚の映像の中で注目すべき箇所をより明確に表示することができる。
【０２０４】
（実施の形態８）
８−１．本実施の形態における映像表示装置の動作概要
上述した実施の形態においては、「被写体」とは、撮影対象としての建築物、設備、人物などの物体を主として説明した。本実施の形態では、町名や番地等で指定される撮影対象空間の部分空間も、被写体として定義し、被写体情報に記述する。観察対象として重要な町（映像重要度が高い町）等を撮影した映像を優先的に表示すること、またはオペレータが町名または番地で指示することにより優先的に表示するという点では、例えば実施の形態２の映像表示装置９０（図９）と同様の意義を有する。
【０２０５】
８−２．本実施の形態における映像表示装置の動作
本実施の形態において用いられる映像表示装置は、実施の形態１〜７で説明した映像表示装置のいずれでもよい。
【０２０６】
図２４は、空間を被写体として定義した被写体情報の例を示す。被写体情報は、空間を識別するＩＤ、町名等の名称および区、郡等の種類、空間の領域を示す座標、空間を撮影するカメラのカメラＩＤ、被写体重要度などが定義される。映像重要度計算部１５（図１、９、１６、２１）はこの被写体情報に基づいて映像の映像重要度を計算し、表示制御部１６（図１、９、１６、２１）が映像重要度の高い空間を写す画像を優先的に表示するよう、表示を制御する。さらに、映像表示装置１０、９０、１６０、２１０（図１、９、１６、２１）は、オペレータ等により外部から、被写体情報に記述されているパラメータ（名称、種類、領域またはカメラＩＤ等）を受け取ると、その被写体を含む映像を他の映像よりも優先して表示する。これによりオペレータは、任意の被写体が撮影された映像を観察することができる。
【０２０７】
以上のように本実施の形態によれば、被写体として地点や地域といった撮影対象空間における部分空間も含むようにしたので、建物等の物体だけでなく、町名や地図上での位置などで特定される空間を撮影した映像を優先的に表示することができる。
【０２０８】
なお上記説明において、部分空間は町や番地など固定的な地域として説明したが、これに限定されない。例えば、図９に示す地図と連動した映像表示装置９０において、オペレータがマウス等で指定した地図上の地点およびその近傍を動的な部分空間として定義し、この部分空間の映像重要度を上げる。これにより、指定した地点の近傍を写す映像が表示され、オペレータが関心をもつ空間の映像を地図に基づいて表示することができる。この部分空間毎に、実施の形態７に説明したように表示規則にしたがって表示品質を変えて表示することもできる。さらに、例えばマウスを地図上の道路に沿って動かすとその近傍の映像が次々と表示され、地図上でのウォークスルーに対応した映像表示も実現することができる。
【０２０９】
（実施の形態９）
９−１．本実施の形態における映像表示装置の動作概要
図２５は、本実施の形態における映像表示装置２５０の構成を示すブロック図である。
【０２１０】
本実施の形態における映像表示装置２５０は、被写体に発生したイベントをイベント発生検出部２５１で検出し、そのイベントの重要性を示すイベント重要度、イベントの発生時刻等と上述した被写体重要度とに基づいて、映像重要度を求め、映像重要度に基づいて、イベントが発生した位置を撮影していたカメラ（図２６のカメラＡ）、およびイベントが発生した位置の近傍を撮影していたカメラ（図２６のカメラＢ）がイベント発生前後に撮影した映像の表示を行う（図２７）。なお、本実施の形態では、映像は主として静止画を表す。映像が動画である場合については実施の形態１０で説明される。
【０２１１】
９−２．本実施の形態における映像表示装置の構成および動作
（ａ）本実施の形態における映像表示装置の構成
映像表示装置２５０は、複数のカメラ１１ａ〜１１ｎと、映像情報取得部１２と、被写体情報管理部１３と、被写体情報格納部１４と、映像重要度計算部１５と、表示制御部１６と、表示部１７と、映像情報格納部９３と、地図情報格納部９４と、地図情報読み出し部９５と、操作部９６とを含む。映像表示装置２５０はさらに、イベント発生検出部２５１を含む。
【０２１２】
以下では、映像表示装置２５０の構成要素のうち、映像表示装置９０（図９）の構成要素と同一の符号が付された構成要素については、映像表示装置９０（図９）の構成要素の構成および機能と同一であるのでその説明は省略する。したがって、以下ではイベント発生検出部２５１について説明する。他の構成要素について映像表示装置９０（図９）と異なる動作または追加の動作が行われる場合には、その都度説明する。
【０２１３】
本実施の形態における映像表示装置２５０においては、映像情報格納部９３に格納される映像情報には、実際の映像を表す情報と、その映像が撮影された時刻を表す撮影時刻情報とが含まれる。映像情報が格納される手順は、映像情報入力部９１が、映像を表す情報をカメラ１１ａ〜１１ｎから受け取り、その映像を表す情報を映像情報格納部９３に格納する。格納に際しては、映像を表す情報には撮影された時刻を表すタイムスタンプが付される。本実施の形態では、タイムスタンプは、画像フレーム毎、または連続する画像フレームの集合毎に付される。連続する画像フレームの集合毎にタイムスタンプを付す場合には、少なくとも先頭フレームの撮影時刻とフレームレートがタイムスタンプとして付される。以下では、このタイムスタンプを利用して映像情報がどの時刻に撮影された映像を含むかが判断される。
【０２１４】
イベント発生検出部２５１は、イベントの発生を検出するために設けられている。イベントの例としては、プラント設備において発生した温度異常や圧力異常、高速道路で発生した事故である。前者の場合は、イベント発生検出部２５１は温度センサまたは圧力センサ等のセンサ、後者の場合は、イベント発生検出部２５１は電話、非常ボタン等による通報を検出する装置が該当する。イベント発生検出部２５１は、イベントを検出すると、被写体情報管理部１３に対し被写体情報を変更するよう指示し、発生したイベントに関する情報を被写体情報管理部１３に送信し、被写体情報に記述させる。発生したイベントに関するイベント情報は、イベントが発生した被写体のＩＤ、イベントの種類、時刻、重要度等である。
【０２１５】
被写体情報管理部１３は、被写体情報の変更指示と記述すべきイベント情報とを受信すると、被写体情報格納部１４に格納された被写体情報にイベント情報を書き込む。
【０２１６】
図２８は、イベント情報を含む被写体情報の例を示す。
【０２１７】
被写体情報は、被写体情報テーブル（図２８（ａ））とイベント情報テーブル（図２８（ｂ））とから構成される。被写体情報テーブル（図２８（ａ））には、被写体で発生したイベントの数と、イベント情報テーブルを参照するためのポインタとが記述される。イベント情報テーブル（図２８（ｂ））には、イベントのＩＤ、種類、発生時刻、イベントの重要度等が記述される。
【０２１８】
（ｂ）本実施の形態における映像表示装置の動作および映像表示例
図２６は、カメラＡ，ＢおよびＣの位置とイベントＥが発生した位置の位置関係を示す。図２６に示されるように、イベントＥは時刻ｔｅ、位置（ｘｅ， ｙｅ）で発生したことが理解される。このような位置関係にあるカメラＡ，ＢおよびＣとイベントについて、以下どのように映像が表示されるかを説明する。
【０２１９】
本実施の形態における映像表示装置２５０（図２５）の処理手順は、図１１に示す映像表示装置９０（図９）の動作手順と同じである。ただし、「被写体情報と映像管理情報とに基づいて各映像の映像重要度を計算する」（図１１、ステップ１１０）際の映像重要度の計算内容のみが異なる。以下、その計算内容について説明する。
【０２２０】
映像重要度計算部１５（図２５）は、同じカメラによって撮影された映像であれば、イベント発生時刻に近い映像ほど、より大きい映像重要度になるよう計算する。
【０２２１】
映像重要度の計算に際して、映像重要度計算部１５（図２５）はまず、イベントの発生位置を撮影していたカメラの映像撮影時刻と、イベントの発生時刻との時間差ｔを計算する。カメラの映像撮影時刻は映像管理情報（図１２）に記述されており、イベントの発生時刻はイベント情報テーブルに記述されている。映像重要度計算部１５（図２５）は、イベントの発生位置を撮影していたカメラ（図２６のカメラＡ）およびイベントの発生位置の近傍を撮影していたカメラ（図２６のカメラＢ）からの映像について、上述した時間差を計算する。時間差を計算され、表示される対象となる映像は、適当な時間的範囲、例えばイベントが発生した時間の前後３０分に撮影された映像であるとしてもよい。この時間的範囲は任意に調整され得る。
【０２２２】
時間差ｔが計算されると、その時間差ｔをイベント重要度関数Ｗｅ（ｔ）に与え、イベントに対する映像重要度Ｗｅを計算する。
【０２２３】
図２９は、イベント重要度関数Ｗｅ（ｔ）の例を示す。イベント重要度関数はイベント発生時刻の映像重要度を最大値（１．０）として時間差ｔの絶対値に比例して小さくなる関数である。ただし、イベント発生位置の近傍を撮影していたカメラの映像に対しては、映像重要度の最大値は０．５とする。したがって、カメラＡはこのイベントが発生した位置を撮影しており、最大値は１．０である。カメラＢはイベントが発生した位置の近傍を撮影しているので、最大値は０．５である。一方、カメラＣはイベントの発生位置びその近傍を撮影していないので、時間差ｔに関係なく映像重要度は０とされる。
【０２２４】
以上の手順によれば、図２９に示されるように映像重要度を得ることができる。すなわち、イベントの発生時刻（ｔ＝０）を１３時２０分とし、カメラＡがイベント発生位置を、カメラＢがイベント発生位置の近傍を撮影しているとする。撮影時刻は、カメラＡは１３時１０分と１３時２２分、カメラＢは１３時１５分と１３時２５分、カメラＢは１３時２０分である。カメラＡで撮影した画像Ａ１はｔ＝−８（分）、Ｗｅ＝０．３、画像Ａ２はｔ＝２（分）、Ｗｅ＝０．８となる。カメラＢで撮影した画像Ｂ１はｔ＝−５（分）、Ｗｅ＝０．３、画像Ｂ２はｔ＝５（分）、Ｗｅ＝０．３となる。カメラＣで撮影した画像Ｃ１は、ｔ＝０（分）、Ｗｅ＝０．０である。上述した最終的な映像重要度は、イベント重要度関数Ｗｅの値をそのまま用いたが、イベント重要度関数Ｗｅの値と被写体情報に記述された被写体重要度Ｗｏの値の積を映像重要度としてもよい。
【０２２５】
以上のようにして得られた映像重要度によって、映像が表示される態様を説明する。
【０２２６】
図２７は、本実施の形態における、イベントが発生した時に表示される複数の映像の例を示す。イベント発生位置を撮影したカメラＡの２つの映像とともに、その近傍を撮影したカメラＢの２つの映像が表示される。
【０２２７】
まず、イベントの発生位置を撮影していたカメラＡの映像について、イベント発生時刻により近い時刻の映像（１３時２２分に撮影された映像）の映像重要度が最も大きいので（Ｗｅ＝０．８）、その映像が最も大きく表示される。他方の映像は比較的小さく表示される。イベントの発生位置の近傍を撮影していたカメラＢの映像については、いずれの映像も、映像重要度は同じであり（Ｗｅ＝０．３）、かつカメラＡの撮影した映像の映像重要度と同じであるので、いずれも同じサイズで表示される。イベントの発生位置びその近傍を撮影していないカメラＣの映像は、映像重要度が０であるので表示されていない。
【０２２８】
なお、映像重要度に応じて表示品質を変更することもできる。ただし、その説明は既に実施の形態５、６および７で説明したとおりなので省略する。
【０２２９】
以上のように本実施の形態によれば、映像管理情報（図１２）に映像の撮影時刻を記述し、被写体情報にある時刻に発生したイベントに関するイベント情報を記述し、これらに基づいて映像重要度計算部１５（図２５）が映像重要度を計算するので、被写体そのものの重要度だけでなく、被写体に発生したセンサ異常等のイベントの重要度に従い、対応する映像を優先的に表示することができる。これにより、素早く的確にイベントを観察することができる。さらにイベント発生時刻の近傍はより強調して表示するなどにより、映像の重要性を時間的に意識して観察することができる。
【０２３０】
上述したイベント重要度関数は、表示規則として規定され得る。したがって、図２９に示すイベント重要度関数に限らない、他のイベント重要度関数を用いて映像重要度を決定することもできる。
【０２３１】
図３１は、イベント重要度関数の例を示す。
【０２３２】
図３１（ａ）〜（ｃ）は正規分布関数を示す。図３１（ｂ）および（ｃ）はイベントが発生する時刻（ｔ＝０）の前後にピークが存在する。このようなイベント重要度関数は、センサの感度等によりアラーム発生時刻よりも以前あるいは以降に重要なシーンがあるような場合に有効である。
【０２３３】
図３１（ｄ）は、立ち上がりが急でかつ減衰が小さい関数である。この関数は、イベント発生時刻（ｔ＝０）以降が重要な場合に有効である。
【０２３４】
図３１（ｅ）は、ステップ関数である。ステップ関数を利用することで、映像重要度計算の処理を簡単にすることができる。
【０２３５】
図３１（ｆ）は、複数のピークを持つ関数である。この関数は、イベント発生時刻（ｔ＝０）の前後のある時点で重要なシーンがある場合に有効である。
【０２３６】
イベントの種類や観察目的に応じて、図３１（ａ）〜（ｆ）に示す関数および例示していない他の多くの関数を用いることができる。
【０２３７】
また上述した例では、被写体管理情報（図２８）にイベント情報テーブルへのポインタを記述し被写体とイベントを関連付けているが、これに限定されることはない。すなわち、被写体の位置が記述されている被写体情報（図５）とイベント発生地点の位置情報が記述されているイベント情報（図３８）とを参照することにより、被写体の位置とイベントの発生位置とが所定の距離内にあれば、イベントがその被写体で発生したイベントであると推定することができる。このように、被写体とイベントとの関連付けを行うこともできる。
【０２３８】
さらに、撮影時におけるカメラの情報であるカメラ管理情報を用いて、被写体とイベントとが適切に撮影された映像を得ることができる。
【０２３９】
図３０は、カメラ管理情報の例を示す。カメラ管理情報は、カメラＩＤ、カメラの撮影範囲をあらわすカメラの位置、水平および垂直角度、ズーム度が記述されている。
【０２４０】
上述した被写体情報（図６）に記述された被写体の位置情報およびイベント情報（図３８）に記述されたイベントの発生位置情報に加え、さらにカメラ管理情報のカメラの位置および角度情報を利用することにより、イベントがその被写体の近傍で発生したと推定できるのみならず、カメラがその被写体およびイベントを撮影していたか否か、さらにカメラが被写体およびイベントを撮影している場合には、その被写体およびイベントがその映像に占める割合等も判断することができる。どのようにその判断が行われるかの詳細は、本明細書２−２．（ｃ）で説明したので省略する。カメラ管理情報を利用することにより、その被写体およびイベントを撮影していたカメラが複数ある場合などは、その中から最も的確にイベントおよび被写体を捉えているカメラの映像を表示に利用することができる。したがって、映像の表示をより柔軟に行うことができる。
【０２４１】
上記９−１．では、本実施の形態は映像は主として静止画を表すと述べたが、これに限定されることはない。すなわち、映像が動画像であってもよい。その際には、イベント発生時刻との時間差との計算において、撮影開始時刻、撮影終了時刻、撮影時刻中の時刻のいずれを上述した映像の撮影時刻にしてもよい。どの時刻を映像の撮影時刻として採用するかは本発明の本質的な問題ではなく、目的に応じて使い分けることができる。
【０２４２】
（実施の形態１０）
１０−１．本実施の形態における映像表示装置の動作概要
実施の形態９においては、イベントの発生時刻に関連して、撮影時刻の異なる静止画を表示する場合を主として説明した。本実施の形態では、イベントの発生時刻に関連して、複数の動画を撮影時刻に応じて表示する場合を説明する。撮影された動画は、その撮影時刻がイベント発生時刻に近づくと表示サイズが徐々に広げられ（図３２（ａ）および（ｂ））、撮影時刻がイベント発生時刻から経過するにつれ表示サイズが徐々に縮小される（図３２（ｃ））。このとき、複数の動画のそれぞれは、撮影時刻に基づいて同期して再生・表示される（図３２におけるカメラＡおよびＢ近傍に表示された映像）。
【０２４３】
なお本実施の形態において「同期」という語は、複数の映像のそれぞれが、撮影された時刻に基づいて表示される際に用いられる。同期して表示されている複数の映像とは、画像フレームレベルで同じ時刻に撮影された複数の映像であって、その撮影時刻に基づいて再生・表示された複数の映像のことをいう。なお、「同じ撮影時刻」は、必ずしも厳密かつ正確に計測された時刻でなくともよい。映像の同期表示の際には、映像それぞれのフレームレートの差異、表示するハードウェアの性能等によって撮影時刻に厳密に対応する時刻に表示されなくともよく、多少のずれが生じてもよい。
【０２４４】
１０−２．本実施の形態における映像表示装置の動作および映像表示例
本実施の形態における映像表示装置は、実施の形態９で説明した映像表示装置２５０（図２５）と同じである。したがって、映像表示装置２５０（図２５）の具体的な構成については説明を省略する。
【０２４５】
以下、本実施の形態における映像表示装置２５０（図２５）の動作を説明する。映像表示装置２５０（図２５）が動作する状況に関して、カメラの配置およびその撮影範囲、イベントの発生時刻および発生位置は、図２６に示されるとおりとする。映像情報格納部９３（図２５）には、映像情報としてカメラＡ，ＢおよびＣ（図２６）が撮影した動画像が格納されている。
【０２４６】
図３３は、映像表示装置２５０（図２５）の動作手順を表すフローチャートである。
【０２４７】
表示制御部１６（図２５）は、地図情報読み出し部９５（図２５）を介して地図情報格納部９４（図２５）に格納された地図情報を読み込み、読み込んだ地図情報に基づいてその地図情報が表す地図を表示部１７（図２５）に表示する（ステップ３３０）。
【０２４８】
続いて表示制御部１６（図２５）は、オペレータ等が操作部９６（図２５）を介して入力した、映像の再生開始時刻Ｔｓと、再生終了時刻Ｔｅとを受け取る（ステップ３３１）。ここでいう再生開始時刻Ｔｓおよび再生終了時刻Ｔｅは、例えば映像が時刻１３：００から時刻１３：３０まで撮影されており、再生を開始したい時刻が時刻１３：１０であればＴｓ＝１３：１０と、再生を終了したい時刻が時刻１３：２５であればＴｅ＝１３：２５と設定される。
【０２４９】
表示制御部１６（図２５）は、再生時刻ｔをｔ＝Ｔｓに設定する（ステップ３３２）。上記と同様に、再生時刻ｔとは、再生されているその映像が実際に撮影された時刻を表す。したがって、再生開始時刻Ｔｓおよび再生終了時刻Ｔｅについて説明した上記例では、再生時刻ｔは再生開始時刻Ｔｓ（時刻１３：１０）から再生終了時刻Ｔｅ（時刻１３：２５）の間に含まれる時刻をいう。「映像が実際に撮影された時刻」には、再生されている画像フレームそれぞれの撮影時刻を含み得る。
【０２５０】
映像重要度計算部１５（図１５）は、時刻ｔにおける映像の映像重要度を計算する（ステップ３３３）。本実施の形態では表示すべき映像は動画であるため、時刻ｔにおける映像の映像重要度とは、時刻ｔにおける動画のフレームを表す映像の映像重要度を表す。動画を構成する各画像フレームの撮影時刻は、映像管理情報（図１２）に記述されている撮影開始時刻と終了時刻およびフレームレートから求めることができる。映像重要度の計算は、実施の形態９で説明したように行われる。すなわち、時刻ｔにおける画像フレームの映像重要度は、再生時刻とイベント発生時刻との時間差を求め、イベント重要度関数を用いてその時間差に対応する映像重要度の値を求めることができる。
【０２５１】
図３４は、再生時刻とイベント発生時刻との時間差に応じて配列された画像フレームの集合とイベント重要度関数との対応関係を示す。本実施の形態では映像は動画であるため、映像の表示は画像フレームの連続表示により実現される。図３４の時間軸に沿った直方体を時間軸に垂直に切断した面は、その時間における画像フレームを表す。画像フレームそれぞれについて（すなわち画像フレームの撮影時刻ごとに）映像重要度を計算することにより、動画像の映像重要度がリアルタイムで変更されていると考えることができる。なお、イベント重要度関数は実施の形態９で示したイベント重要度関数と同じである。すなわち、イベント重要度関数はイベント発生時刻の映像重要度を最大値（１．０）として時間差ｔの絶対値に比例して小さくなる関数である。ただし、イベント発生位置の近傍を撮影していたカメラの映像に対しては、映像重要度の最大値は０．５とする。したがって、カメラＡはこのイベントが発生した位置を撮影しており、最大値は１．０である。カメラＢはイベントが発生した位置の近傍を撮影しているので、最大値は０．５である。一方、カメラＣはイベントの発生位置びその近傍を撮影していないので、時間差に関係なく映像重要度は０とされる。
【０２５２】
再び図３３を参照して、計算された映像重要度は、映像管理情報の一部として映像管理情報に格納される（ステップ３３４）。
【０２５３】
表示制御部１６（図２５）は、映像情報読み出し部９２（図９）を介して映像情報格納部９３（図９）から時刻ｔにおける映像の映像情報を読み出す（ステップ３３５）。
【０２５４】
表示制御部１６（図２５）は、表示規則と時刻ｔにおける映像重要度とに基づいて表示サイズを決定し、時刻ｔにおける映像を表示部１７（図２５）表示する（ステップ３３６）。
【０２５５】
続いて表示制御部１６は、現在の再生時刻ｔが再生終了時刻Ｔｅに達していないかどうか（すなわち、ｔ＜Ｔｅかどうか）を判定する（ステップ３３６）。
【０２５６】
現在の再生時刻ｔが再生終了時刻Ｔｅに達していない場合には、再生時刻ｔを所定の時間Δｔだけ進める、すなわち、ｔをｔ＋Δｔとして設定する（ステップ３３８）。その後は、ステップ３３３以降の処理を繰り返すことで再生が進行する。
【０２５７】
現在の再生時刻ｔが再生終了時刻Ｔｅに達した場合（すなわち、ｔ≧Ｔｅの場合）には、続いて動画像の再生時刻が変更されたか否か、すなわち再生開始時刻Ｔｓまたは再生終了時刻Ｔｅが変更されたか否かを判定する（ステップ３３９）。
【０２５８】
動画像の再生時刻が変更された場合には、表示制御部１６（図２５）は変更された再生時刻を受け取り、再びステップ３３１からの処理を繰り返す。
【０２５９】
動画像の再生時刻が変更されていない場合には、オペレータが次の再生時刻を入力されるまで、ステップ３３９の判定を繰り返し行う。
【０２６０】
以上、本実施の形態における映像表示装置２５０（図２５）の処理手順について説明した。なお上述した処理は、動画像それぞれが再生時刻に基づいて同期して再生・表示されるように行われる。このような同期再生・表示が行われることを条件とすれば、上述した処理で、複数の動画の各画像フレームの全てについてあるステップの処理が終了して次のステップに移行するとしても、複数の動画の各画像フレームについての処理が独立かつ並行に行われるとしてもよい。
【０２６１】
図３２は、本実施の形態における、イベントの発生に応じて表示される複数の映像の例を示す。イベント発生位置を撮影したカメラＡの動画像とともに、その近傍を撮影したカメラＢの動画像がそれぞれの撮影時刻に基づいて同時に表示される。カメラＣの撮影した映像は上述のように映像重要度が０であるので表示されていない。
【０２６２】
図３２（ａ）は、１３時２０分に発生したイベントに対して、１３時１０分に撮影された映像を表示した状態を示す。ここでは、映像は動画である。カメラＡおよびＢの各画像フレームの映像重要度は、イベントが発生するまで間があるため比較的小さい。したがって表示される映像のサイズは比較的小さい。
【０２６３】
図３２（ｂ）は、イベントの発生時刻（１３時２０分）に撮影された映像を表示した状態を示す。イベントの発生位置を撮影していたカメラＡの映像は、図３２（ａ）に示す映像よりも表示サイズが大きい。表示サイズは、画像フレームの映像重要度の増加に応じて滑らかに拡大する。すなわち、イベント発生時刻に近づくに従ってカメラＡの映像の表示サイズが拡大する。イベントの発生位置の近傍を撮影していたカメラＢの映像についても同様である。なお、同時刻におけるカメラＡの映像の映像重要度は、カメラＢの映像の映像重要度よりも大きいので、より大きなサイズで表示される。
【０２６４】
図３２（ｃ）は、１３時２０分に発生したイベントに対して、１３時２５分に撮影された映像を表示した状態を示す。イベント発生時刻から時間が経過するにつれて、画像フレームの映像重要度は小さくなり、それにしたがって表示サイズは小さくなる。
【０２６５】
図３２（ａ）〜（ｃ）を参照して説明した上記の例では、表示される映像は、それぞれのカメラＡおよびＢで現実に撮影された時刻に基づいて同期して再生・表示されている。
【０２６６】
以上説明したように本実施の形態によれば、撮影時刻毎に動画をなす画像フレームの映像重要度を計算し、表示制御部１６（図２５）が重要度に従って表示サイズを決めて、その撮影時刻に基づいて同期して再生・表示するようにしたので、一つのカメラでは撮影できないイベントの周辺状況を見やすく表示することができる。さらに、例えばカメラが移動した場合、または異なる時間に異なる位置で複数のイベントが発生する場合に対しても、撮影時刻に基づいて映像を表示し、および全体の状況を広い視点で表示することができるので、時間的・空間的に容易に把握することができる。比較的近い位置で複数のイベントが発生した場合を、以下に簡単に説明する。
【０２６７】
図３５は、異なる時刻に発生した２つのイベントとそのイベントを撮影するカメラの位置関係を示す。
【０２６８】
２つのイベントはイベントＥおよびＦである。イベントＦは時刻ｔｅ、位置（ｘｅ，ｙｅ）で発生し、イベントＦは時刻ｔｆ、位置（ｘｆ，ｙｆ）で発生したと仮定する（ｔｅ＜ｔｆ）。さらにカメラＡはイベントＥの発生した位置を撮影し、かつイベントＦの発生した位置の近傍をも撮影している。カメラＢはイベントＦの発生した位置を撮影し、かつイベントＥの発生した位置の近傍をも撮影していたとする。
【０２６９】
図３６は、イベントＥおよびＦが発生したときに用いられるイベント重要度関数の例を示す。
【０２７０】
図３６（ａ）は、カメラＡの映像に対する映像重要度を計算するためのイベント重要度関数を示す。図３６には、イベントＥおよびＦが単独で発生した場合にそれぞれカメラＡに対して用いられるイベント重要度関数がオーバーラップして表示されている。イベントＥの発生位置を撮影していた時間と、イベントＦの発生位置の近傍を撮影していた時間とは、異なるイベント重要度関数が用いられる。これは上述したとおりである。カメラＡの映像は、まずそれぞれのイベントＥおよびＦについて映像重要度を計算され、その後最終的な映像重要度が決定される。最終的な映像重要度は、イベントＥおよびイベントＦそれぞれの映像についての映像重要度の和を採用する、またはより大きい値を持つ映像重要度を採用する等、どのように決定してもよい。これはカメラＢの撮影した映像についても同様である。
【０２７１】
なお、以上の説明では映像は動画としたが、静止画を含んでいてもよい。但し、静止画１枚をその撮影時刻に一瞬表示するだけでは見にくいので、撮影時刻の前後一定時間の間再生することが好ましい。このとき、動画と静止画とを区別するために表示枠の色を変えて明示してもよい。これは、表示規則を変更することにより設定することができる。同様の表示処理は、動画の最初あるいは最後の画像フレームを表示する場合にも有効である。例えば、カメラＡの動画がカメラＢの動画より終了時刻が早い場合、カメラＡの最後の画像フレームが表示された後、その表示枠の色を変化させ、再生終了を知らせる。これにより、オペレータは再生開始または終了時刻の再設定を遅滞なく行うことができる。
【０２７２】
（実施の形態１１）
１１−１．本実施の形態における映像表示装置の動作概要
図３７は、本実施の形態における映像表示装置３７０の構成を示すブロック図である。
【０２７３】
上述した実施の形態９では、イベントが発生したとき、そのイベントの重要性を示すイベント重要度、イベントの発生時刻等に加え、被写体重要度も利用して映像重要度を計算し、映像を表示していた。本実施の形態における映像表示装置２５０は、イベント重要度、イベントの発生時刻等に基づいて映像重要度を計算し、被写体重要度を用いない。映像表示装置２５０は、計算した映像重要度に基づいてイベント発生前後に撮影された映像を表示する。
【０２７４】
１１−２．本実施の形態における映像表示装置の構成および動作
（ａ）本実施の形態における映像表示装置の構成
映像表示装置３７０は、複数のカメラ１１ａ〜１１ｎと、映像情報取得部１２と、映像重要度計算部１５と、表示制御部１６と、表示部１７と、映像情報格納部９３と、地図情報格納部９４と、地図情報読み出し部９５と、操作部９６と、イベント発生検出部２５１とを含む。映像表示装置２５０はさらに、イベント情報管理部３７１と、イベント情報格納部３７２とを含む。
【０２７５】
以下では、映像表示装置３７０の構成要素のうち、実施の形態２における映像表示装置９０（図９）の構成要素と同一の符号が付された構成要素については、映像表示装置９０（図９）の構成要素の構成および機能と同一であるのでその説明は省略する。したがって、以下では、イベント情報管理部３７１と、イベント情報格納部３７２について説明する。他の構成要素について映像表示装置９０（図９）と異なる動作または追加の動作が行われる場合には、その都度説明する。なお映像表示装置９０（図９）の被写体情報管理部１３（図９）および被写体情報格納部１４（図９）は、本実施の形態における映像表示装置３７０には存在しない。被写体情報を利用しないからである。
【０２７６】
イベント情報管理部３７１は、イベント情報を管理する。イベント情報とは、イベントの発生時刻、イベントの種類、イベント重要度、映像との対応関係を含む情報である。イベント情報管理部３７１は、イベント発生検出部２５１によりイベント発生が検出され、イベント情報が送信されてくると、イベント情報を設定しまたは変更する。イベント情報管理部３７１は、設定しまたは変更したイベント情報をイベント情報格納部３７２に格納する。
【０２７７】
イベント情報格納部３７２は、発生したイベントに関するイベント情報が格納される。イベント情報格納部３７２は、イベント情報管理部３７１から、設定されたイベント情報を受け取り、格納する。イベント情報格納部３７２は、映像重要度計算部１５にイベント情報を提供する。イベント情報格納部３７２は、ＲＡＭあるいはＲＯＭに代表される半導体メモリ、ハードディスク等の磁気記憶媒体を用いた記憶装置、光ディスク等、公知のデータ記憶手段であり得る。
【０２７８】
映像重要度計算部１５は、イベント情報および表示規則に基づいて映像重要度を決定する。
【０２７９】
図３８は、本実施の形態におけるイベント情報の例を示す。
【０２８０】
イベント情報は、イベントＩＤ、種類、発生時刻および発生位置、そのイベントを撮影しているカメラＩＤ、イベント重要度等の情報を含む。
【０２８１】
映像重要度計算部１５（図３７）は、イベント情報のこれらの情報を利用して映像重要度を計算する。映像重要度の計算は、実施の形態９で説明した映像重要度の計算と同じであるので、その説明は省略する。
【０２８２】
映像重要度が計算されると、表示制御部１６（図３７）はその映像重要度に基づいて映像の表示を制御する。映像を表示する際には、これまでの実施の形態で説明した表示規則を用いて、位置、時間、品質等の制御を行うことができる。表示規則は適切に変更することができる。
さらに、映像が静止画である場合または動画である場合のいずれの場合でも、これまでの実施の形態で説明した動作と同じ動作を行うことができるが、その説明は省略する。
【０２８３】
以上のように本実施の形態によれば、映像重要度計算部１５（図３７）はイベント情報および表示規則に基づいてイベント発生時刻と撮影時刻との時間関係を用いて映像重要度を計算するので、イベントの発生前後の状態を含むイベントに関する情報のみに基づいたイベントの観察が可能になる。本実施の形態によれば、実施の形態９では必要であった被写体情報は利用しないので、被写体情報の管理が不要になり、映像重要度の計算処理を高速に行うことができる。
【０２８４】
以上、各実施の形態について説明した。
【０２８５】
フローチャートを用いて説明した処理手順を始めとする本発明の映像表示装置の動作は、そのように動作するように記述されたコンピュータプログラムとしても実現され得る。そのようなコンピュータプログラムは、光ディスク、フロッピーディスク等の磁気記憶媒体、フラッシュＲＯＭ等の半導体記憶媒体等、適当な記憶媒体に記録することができる。ここで、「記憶媒体」の概念には、インターネット等の通信回線を介してコンテンツを提供するコンピュータ等の記憶装置も含まれる。したがって、本発明の動作を実現するためのコンピュータプログラムを格納し、そのプログラムを他の端末に送信するためのサーバ等は、コンピュータプログラムを記録した記憶媒体の範疇に含まれる。記憶媒体には、例えば被写体情報は、図５に図示されるように、少なくとも前記被写体の名称と、前記被写体が存在する位置と、前記被写体重要度とをそれぞれを関連づけたデータ構造として記録される。被写体情報にイベント情報が含まれる場合にも、同様に上記実施の形態で説明したイベントに関する情報もあわせて対応付けたデータ構造として、記録媒体に記録される。図示された他の表示レイアウト情報（図８）、映像管理情報（図１２）、カメラ管理情報（図３０）についても同様のデータ構造で記録され得る。
【０２８６】
なお、被写体の位置座標およびカメラの位置座標のいずれも、現実に存在する位置を特定できる情報である。実施の形態１において図５を参照して説明したと同様に、位置座標は緯度および経度によって表されていてもよいし、またはその住所等によってあらわされていてもよい。
【０２８７】
【発明の効果】
以上説明したように、本発明によれば、被写体それぞれの被写体重要度を含む被写体情報に基づいて、映像の映像重要度が計算され、その映像重要度に基づいて映像が表示される。したがって、映像に含まれる被写体の重要性に応じた映像の表示が可能になる。オペレータは、被写体の重要度を決定するだけで被写体の重要性に応じた映像の表示をさせることができるので、表示レイアウトを決定する煩雑な手間が不要になり、かつ客観性のある映像表示を得ることができる。さらに、表示規則を規定することにより、オペレータは重要なグループほど長時間観察でき、画面内でも決められた重要度の配置でみることができるといった、柔軟な映像表示が実現される。
【０２８８】
本発明によれば、重要な被写体あるいはその被写体の近傍を撮影した映像を、地図上で撮影位置を把握しながら見ることができるという視覚的効果を得ることができる。さらに、被写体の追加や削除、被写体の重要度の変化に対しても被写体情報を修正するだけで対応することができ、煩雑な表示レイアウト修正作業を省略することができる。さらに、撮影した映像を一旦蓄積して使用するため、被写体を同じ重要度で、または重要度を変更して、繰り返し複数の映像を表示して観察することができる。
【０２８９】
本発明によれば、映像表示中であっても、被写体情報管理部１３（図１６）は、入力部１６１（図１６）や外部モジュール１６２（図１６）からの要求に従い、被写体情報に含まれる被写体重要度を変更する。表示制御部１６（図１６）は、被写体情報の変更が反映された結果変更された映像重要度に応じて、複数の映像の表示を自動的に変更する。したがって、注目すべき被写体の種類や位置などが状況に応じて変化する場合でも、映像重要度の高い映像が自動的に選択表示され、オペレータは素早く的確な被写体観察ができる。
【０２９０】
本発明によれば、表示規則と映像重要度とに基づいて、表示制御部１６（図１、９、１６）が映像の強調表示の度合いを変化させて表示するようにしたので、重要な映像ほど大きく表示する、または重要な映像を表示するウインドウの枠の色を目立つようにする等、被写体の重要性をより意識して観察することができる。
【０２９１】
本発明によれば、表示制御部は表示規則および映像重要度に基づいてフレームレートや解像度等の映像品質の度合いを変化させて表示するよう映像の表示を制御する。したがって映像重要度の高い映像（すなわち重要な映像）がより見やすくかつ緻密に表示されるので、オペレータは自然と重要な映像を注視することができる。さらに、比較的映像重要度が低い映像（すなわち比較的重要でない映像）は画像フレームが間引かれて表示されるので、映像表示装置にかかる表示処理の負荷は軽くなり、比較的安価な処理回路を用いても映像表示装置を製造することができる。したがって、映像表示装置の製造コストを低く抑えることができる。
【０２９２】
本発明によれば、表示制御部は表示規則および映像重要度に基づいてフレームレートや解像度等の映像品質の度合いを変化させて表示するよう映像の表示を制御する。したがって映像重要度の高い映像（すなわち重要な映像）がより見やすくかつ緻密に表示されるので、オペレータは自然と重要な映像を注視することができる。さらに、比較的映像重要度が低い映像（すなわち比較的重要でない映像）は画像フレームが間引かれて表示されるので、映像表示装置にかかる表示処理の負荷を軽くすることができ、比較的安価な処理回路を用いても映像表示装置を製造することができる。したがって、映像表示装置の製造コストを低く抑えることができる。
【０２９３】
さらに、映像サーバと映像表示端末とがネットワークで接続された構成であっても、ネットワーク上は表示に必要な映像品質の情報しか伝送されないため、ネットワーク資源やサーバおよび端末の計算機資源を有効に利用することができる。それらの資源が有する伝送帯域や処理速度等の性能に応じてその性能を超えないよう映像に映像品質を設定することにより、そのネットワークベースの映像表示装置が有する能力を最大限使った品質のよい複数の映像を表示することができる。
【０２９４】
本発明によれば、動画であっても、１枚の映像の中で注目すべき箇所をより明確に表示することができる。
【０２９５】
本発明によれば、被写体として地点や地域といった撮影対象空間における部分空間も含むようにしたので、建物等の物体だけでなく、町名や地図上での位置などで特定される空間を撮影した映像を優先的に表示することができる。
【０２９６】
本発明によれば、映像管理情報に映像の撮影時刻を記述し、被写体情報にある時刻に発生したイベントに関するイベント情報を記述し、これらに基づいて映像重要度計算部が映像重要度を計算するので、被写体そのものの重要度だけでなく、被写体に発生したセンサ異常等のイベントの重要度に従い、対応する映像を優先的に表示することができる。これにより、素早く的確にイベントを観察することができる。さらにイベント発生時刻の近傍はより強調して表示するなどにより、映像の重要性を時間的に意識して観察することができる。
【０２９７】
本発明によれば、撮影時刻毎に動画をなす画像フレームの映像重要度を計算し、表示制御部が重要度に従って表示サイズを決めて同期して表示するようにしたので、一つのカメラでは撮影できないイベントの周辺状況を見やすく表示することができる。さらに、例えばカメラが移動した場合、または異なる時間に異なる位置で複数のイベントが発生する場合に対しても、全体の状況を広い視点で表示するので時間的・空間的に容易に把握することができる。
【０２９８】
本発明によれば、映像重要度計算部はイベント情報および表示規則に基づいてイベント発生時刻と撮影時刻との時間関係を用いて映像重要度を計算するので、イベントの発生前後の状態を含むイベントに関する情報のみに基づいたイベントの観察が可能になる。さらに被写体情報を利用することなく映像重要度を計算することにより、被写体情報の管理が不要になり、映像重要度の計算処理を高速に行うことができる。
【図面の簡単な説明】
【図１】 実施の形態１における映像表示装置１０の構成を示すブロック図を示す。
【図２】 映像表示装置１０（図１）の処理動作により表示される映像の態様の一例を示す図である。
【図３】 同じ重要度のグループにおける映像の表示位置を示す例を示す図である。
【図４】 映像表示装置１０（図１）の動作手順を表すフローチャートを示す図である。
【図５】 被写体情報の例を示す図である。
【図６】 計算された映像重要度の例を示す図である。
【図７】 表示レイアウトを決定するステップを含む、映像表示装置１０（図１）の動作手順を表すフローチャートを示す図である。
【図８】 表示レイアウト情報の例を示す図である。
【図９】 実施の形態２における映像表示装置９０の構成を示すブロック図を示す図である。
【図１０】 地図とその地図に関連づけられて表示された映像を示す図である。
【図１１】 映像表示装置９０（図９）の動作手順を表すフローチャートを示す図である。
【図１２】 映像管理情報の例を示す図である。
【図１３】 カメラの撮影位置、方向およびズーム画角と被写体の位置関係を示す図である。
【図１４】 被写体がカメラレンズを通して撮像される図である。
【図１５】 撮影範囲と被写体の関係と撮影状態を示す図である。
【図１６】 実施の形態３における映像表示装置１６０の構成を示すブロック図である。
【図１７】 映像表示装置１６０（図１６）の動作手順を表すフローチャートを示す図である。
【図１８】 映像重要度の高い映像ほど表示サイズを大きくする表示画面の例を示す図である。
【図１９】 映像重要度に応じて映像の強調度（表示サイズ）を変更するための表示規則の例を示す図である。
【図２０】 映像重要度の順番にしたがって映像の表示品質を変更するための表示規則の例を示す図である。
【図２１】 実施の形態６における映像表示装置２１０の構成を示すブロック図である。
【図２２】 映像表示装置２１０（図２１）の動作手順を表すフローチャートを示す図である。
【図２３】 ３つの被写体を撮影するカメラと、本実施の形態による処理を施されて表示された被写体の例を示す図である。
【図２４】 空間を被写体として定義した被写体情報の例を示す図である。
【図２５】 実施の形態９における映像表示装置２５０の構成を示すブロック図である。
【図２６】 カメラＡ，ＢおよびＣの位置とイベントＥが発生した位置の位置関係を示す図である。
【図２７】 実施の形態９における、イベントが発生した時に表示される複数の映像の例を示す図である。
【図２８】 イベント情報を含む被写体情報の例を示す図である。
【図２９】 イベント重要度関数Ｗｅ（ｔ）の例を示す図である。
【図３０】 カメラ管理情報の例を示す図である。
【図３１】 イベント重要度関数の例を示す図である。
【図３２】 実施の形態１０における、イベントの発生に応じて表示される複数の映像の例を示す図である。
【図３３】 映像表示装置２５０（図２５）の動作手順を表すフローチャートである。
【図３４】 再生時刻とイベント発生時刻との時間差に応じて配列された画像フレームの集合とイベント重要度関数との対応関係を示す図である。
【図３５】 異なる時刻に発生した２つのイベントとそのイベントを撮影するカメラの位置関係を示す図である。
【図３６】 イベントＥおよびＦが発生したときに用いられるイベント重要度関数の例を示す図である。
【図３７】 実施の形態１１における映像表示装置３７０の構成を示すブロック図である。
【図３８】 実施の形態１１におけるイベント情報の例を示す図である。
【図３９】 従来の映像表示装置３９０の構成を示すブロック図である。
【図４０】 映像情報取得部３９２と表示制御部３９５の具体的な構成を示すブロック図である。
【図４１】 従来の映像表示装置３９０（図３９）において表示レイアウト情報を設定するための操作画面の例を示す図である。
【図４２】 従来の別の映像表示装置システム４２０の構成を示すブロック図である。
【図４３】 優先順位決定部４２４（図４２）が管理する表示優先順位情報の例を示す図である。
【符号の説明】
１０ 映像表示装置、１２ 映像情報取得部、１３ 被写体情報管理部、１４被写体情報格納部、１５ 映像重要度計算部、１６ 表示制御部、１７ 表示部、９２ 映像情報読み出し部、９３ 映像情報格納部、１６１ 入力部、２５１イベント発生検出部、３７１ イベント情報管理部、３７２ イベント情報格納部[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a video display device and a video display method for displaying a plurality of videos, such as security monitoring, plant operation control, facility management, and television broadcasting for sports and festivals.
[0002]
[Prior art]
In security monitoring and plant operation control, video monitoring is performed in which a situation of a site where buildings and facilities exist is photographed by a plurality of monitoring cameras, and a necessary video is switched and displayed at a monitoring center remote from the site. The video to be displayed includes live video and stored video recorded on a VTR (Video Tape Recorder), a hard disk, or the like. Live video is used to check the current status of the monitoring target in real time. The stored video is often used to search the situation before the incident occurs by searching or to display the phenomenon repeatedly and analyze it in detail.
[0003]
The number of cameras installed in the field is increasing year by year as the price of video equipment such as cameras and video storage devices is reduced. Currently, dozens of cameras are installed in large-scale facilities. In such a facility, a contrivance is made so that a small number of operators can efficiently monitor a large number of camera images. For example, multi-screen composite display in which images from a plurality of grouped cameras are reduced and displayed on a monitor screen (see FIG. 2). In this multi-screen composite display, it is possible to switch to a composite screen of another camera group at regular time intervals or according to an operator instruction. Another example is highlighting, in which the size of each video is changed in accordance with the importance of an object or area photographed by the camera, and the monitor is made aware of the importance of the video (for example, FIG. )reference). In another example, a map of the area to be monitored and a layout diagram of the building are displayed as a background so that the shooting position of each video can be easily understood, and the map is displayed near the position corresponding to the shooting position An image is displayed (see, for example, FIG. 18B). “Multi Scope” (NEC System Construction Catalog) is one of conventional video display devices that enable such display.
[0004]
FIG. 39 is a block diagram showing a configuration of a conventional video display device 390. As shown in FIG.
[0005]
The video display device 390 includes a plurality of cameras 391a to 391n, a video information acquisition unit 392, a display layout information correction unit 393, a display layout information storage unit 394, a display control unit 395, and a display unit 396. More specifically, the video information acquisition unit 392 includes a video matrix switcher 401 as shown in FIG. The video matrix switcher 401 outputs N video signals designated from the M input video signals to the display control unit 395 based on the designation from the display control unit 395. The display control unit 395 includes at least a graphics board 402 and a multi-screen overlay board 403. The graphics board 402 and the multi-screen overlay board 403 create video display data to be displayed on the display device based on the received video information and display layout information. Since the graphics board 402 and the multi-screen overlay board 403 are both commercialized, detailed description thereof is omitted.
[0006]
When displaying a video on the video display device 390 configured as described above, the user sets display layout information via the display layout information correction unit 393 (FIG. 39) in advance. As described above, the display control unit 395 controls the display of the video based on the set display layout information and displays the video.
[0007]
FIG. 41 shows an example of an operation screen for setting display layout information in the conventional video display device 390 (FIG. 39).
[0008]
As shown in FIG. 41, the setting of display layout information can be defined interactively. The system designer or operator considers an easy-to-see display layout in his / her head based on the operator's judgment, such as the type, importance, and image quality of the subject captured by each camera. The operator sets the display layout information by inputting the display layout via the display layout information correction unit 393 (FIG. 39).
[0009]
FIG. 42 is a block diagram showing a configuration of another conventional video display device system 420.
[0010]
The video display device system 420 includes a plurality of image transmission terminals 421a to 421n, an image multiplexing device 422, and an image reception terminal 425. Image information photographed by a plurality of image transmission terminals 421a to 421n and subjected to a predetermined process is transmitted to the image multiplexing device 422, further processed, and then transmitted to the image reception terminal 425 for display. The
[0011]
The image transmission terminal 421a includes a camera 421a-1, an image encoding device 421a-2, and a sensor 421a-3. The sensor 421a-3 is provided corresponding to the camera 421a-1. The sensor 421a-3 detects an abnormality that has occurred in the imaging space of the camera 421a-1, and transmits the sensor level signal to the image multiplexing device 422.
[0012]
The image multiplexing device 422 includes an image multiplexing unit 423 and a priority order determination unit 424. The image multiplexing unit 423 multiplexes the received image signal and its display priority information, and transmits the multiplexed image signal to the image receiving terminal 425. The priority order determination unit 424 manages the display priority order information, and when the sensor level changes, the display ordering is performed from the camera with the higher value.
[0013]
FIG. 43 shows an example of display priority information managed by the priority determining unit 424 (FIG. 42). The camera ID shown in FIG. 43 is an ID assigned to each of the cameras 421a-1 to 421n-1.
[0014]
Referring to FIG. 42 again, image receiving terminal 425 includes an image encoding device 426 and a display unit 427. The image receiving terminal 425 decodes the image information received from the image multiplexing device 422, arranges the images in the order in which they should be displayed based on the priority order information, and displays them in a multi-window.
[0015]
[Problems to be solved by the invention]
In the conventional video display device 390 (FIG. 39), a complicated operation is required in which the operator considers the display layout in consideration of the position and importance of the subject in advance and interactively defines the display layout. Furthermore, whenever the position or importance of the subject changes, the operator must repeat the determination of the display layout of the video to be displayed. When more complex display of multiple images including multiple subjects is necessary, the display layout considered by the operator lacks objectivity, and it is unclear whether display according to the importance of the image has been realized. .
[0016]
In the conventional video display system 420 (FIG. 42), a plurality of images can be automatically displayed so that the priorities of each other can be understood based on the measured values of the sensors. For example, in an image monitoring apparatus for factory equipment with a warning level of 500 ° C., images from the camera corresponding to the temperature sensor that detects a value close to 500 ° C. can be displayed on the screen in order.
[0017]
However, since the importance of an image is evaluated only at the sensor level of a sensor provided corresponding to the camera, the importance of the image cannot be determined according to the importance of the subject included in the photographed image. In such a system, when shooting while moving the camera, the importance of the image cannot be determined depending on what is captured in the image shot by the camera, and its use is greatly limited. Furthermore, because each camera has a sensor, it is possible to detect an abnormality of a subject with a single sensor, shoot the abnormality occurrence state of the subject with multiple cameras from different viewpoints, and display them with the same importance. Can not.
[0018]
An object of the present invention is to use a subject information and / or event information to automatically determine a layout or the like according to the importance of a video and display a video, a video display method, and a video. The present invention provides a recording medium on which a program for recording is recorded.
[0019]
[Means for Solving the Problems]
  The video display device according to the present invention includes a subject information management unit that manages subject information including a correspondence relationship between a subject existing in a captured space and a subject importance level indicating the importance of the subject, and the subject information management unit. Based on the subject information managed, MovieCalculated by a video importance calculator that calculates the importance of a video representing the importance of the image, a video information acquisition unit that acquires video information obtained by photographing the space as video information, and at least the video importance calculator A display control unit that controls display of the video represented by the video information acquired by the video information acquisition unit based on the video importance level, and a display unit that displays the video according to control of the display control unit This achieves the above object.
[0020]
The video display device may further include a video information storage unit that stores the video information, and the video information acquisition unit may acquire the video information by reading the video information from the video information storage unit.
[0021]
The video display device further includes an input unit to which a new subject importance is input, and the subject information management unit is responsive to an input from the input unit to input the new subject importance. The video importance calculation unit is responsive to the holding of the new subject importance by the subject information management unit based on the held subject importance. Video importance may be calculated.
[0022]
The subject may be a partial space included in the space.
[0023]
The video information includes shooting time information related to the time when the video was shot, and the subject information includes event information related to an event that occurred in the subject and an event occurrence time that represents the time when the event occurred, The importance calculation unit may further calculate the video importance based on the event information and the shooting time information.
[0024]
  The video display device according to the present invention includes a video information storage unit that stores information about a video of a space and shooting time information related to a time when the video was shot, and a video information read-out that reads the video information from the video information storage unit. And events generated in the spaceGAn event information management unit that manages event information including an occurrence time of the event, an event importance level indicating the importance of the event, and a correspondence relationship with the video, and the event information managed by the event information management unit And the shooting time information stored in the video information storage unit,Represents the importance of videoA video importance calculation unit for calculating video importance, a display control unit for controlling display of a video represented by the video information based on at least the video importance calculated by the video importance calculation unit, and the display A display unit for displaying the video according to the control of the control unit, thereby achieving the above object.
[0025]
The video is a plurality of videos obtained by photographing the space from a plurality of viewpoints, the video importance calculation unit calculates the video importance for each of the plurality of videos, and the display control unit includes at least The display of each of the plurality of videos may be controlled based on the video importance calculated by the video importance calculator.
[0026]
  The display control unit is based on the video importance calculated by the video importance calculation unit and a display rule that defines a rule for displaying the video.The videoThe display may be controlled.
[0027]
The display rule may define a position, time, order, or size at which the video is displayed.
[0028]
The display control unit, based on the video importance calculated by the video importance calculation unit for each of the plurality of videos and a display rule applied according to the video importance, The display may be controlled so that each is highlighted.
[0029]
The display control unit, based on the video importance calculated by the video importance calculation unit for each of the plurality of videos and a display rule applied according to the video importance, May be determined, and the display of each of the plurality of videos may be controlled with the determined quality.
[0030]
The display rule may be changeable.
[0031]
The video is composed of image frames, and the image frame includes two or more areas, the video importance calculator calculates the video importance for each of the two or more areas, and the display controller The display of the image frame may be controlled by determining a mode of displaying each of the two or more areas based on the video importance calculated by the video importance calculator.
[0032]
  The video includes a first video taken at a first time and a second video taken at a second time,The first image and the second image are images obtained by photographing a space from different viewpoints,The video importance calculator calculates a first video importance that is a video importance of the first video and a second video importance that is a video importance of the second video, and the display control A display unit configured to display the video based on a display rule defining a rule for displaying the video and the first video importance and the second video importance calculated by the video importance calculation unit. The display may be controlled so that the first video and the second video are synchronized based on the determined time of shooting.
[0033]
The display control unit may control display of each of the plurality of videos based on a position where each of the plurality of videos is captured and the video importance calculated by the video importance calculation unit.
[0034]
The video may be a still image.
[0035]
The video may be a moving image.
[0036]
The first time and the second time may be the same time.
[0037]
  The video display method of the present invention includes a step of managing subject information including a correspondence relationship between a subject existing in a photographed space and a subject importance level indicating the importance of the subject, and based on the subject information., MovieCalculating the importance of the video representing the importance of the image, obtaining information of the video captured in the space as video information, and at least the video importance represented by the video information based on the video importance The method includes the steps of controlling display and displaying the video according to the display control of the video, thereby achieving the above object.
[0038]
The video display method may further include a step of storing the video information, and the step of acquiring the video information may be a step of acquiring the video information by reading the stored video information. .
[0039]
The video display method further includes a step of inputting a new subject importance level, and the step of managing the subject information is in response to an input by the step of inputting the new subject importance level, Is changed to the new subject importance level and stored, and the step of calculating the video importance level is held in response to the retention of the new subject importance level. It may be a step of calculating video importance based on subject importance.
[0040]
The video information includes shooting time information related to the time when the video was shot, and the subject information includes event information related to an event that occurred in the subject and an event occurrence time that represents the time when the event occurred, The step of calculating importance may be a step of calculating the video importance based on the event information and the shooting time information.
[0041]
  The video display method according to the present invention includes a step of storing information on a video image of a space and shooting time information relating to a time at which the video image was shot, a step of reading out the stored video information, and an event generated in the space Managing event information including an occurrence time of the event, an event importance level indicating the importance of the event, and a correspondence relationship with the video, and the shooting time stored with the managed event information Based on information andIndicates the importance of the videoCalculating a video importance level, controlling display of a video represented by the video information based on at least the calculated video importance level, and displaying the video according to the control, This achieves the above object.
[0042]
The video is composed of image frames, the image frame includes two or more regions, and the step of calculating the video importance is a step of calculating the video importance for each of the two or more regions, The step of controlling the display may be a step of controlling the display of the image frame by determining a mode of displaying each of the two or more regions based on the calculated video importance.
[0043]
The video includes a first video shot at a first time and a second video shot at a second time, and the step of calculating the video importance level includes the step of calculating the video importance level. Calculating a first video importance level that is an importance level and a second video importance level that is a video importance level of the second video, and the step of controlling the display includes a step of displaying the video A mode of displaying the video is determined based on a display rule that defines a rule and the calculated first video importance and second video importance, and the first video and the video based on a shooting time It may be a step of controlling display so that the second video is synchronized.
[0044]
  Of the present inventionComputer readableThe recording medium is configured to manage subject information including a correspondence relationship between a subject existing in a photographed space and a subject importance level indicating the importance of the subject, and based on the subject information., MovieCalculating the importance of the video representing the importance of the image, obtaining information of the video captured in the space as video information, and at least the video importance represented by the video information based on the video importance Controlling the display and displaying the video according to the display control of the video.Let the computer runThis achieves the above object.
[0045]
The video display method may further include a step of storing the video information, and the step of acquiring the video information may be a step of acquiring the video information by reading the stored video information. .
[0046]
The video display method further includes a step of inputting a new subject importance level, and the step of managing the subject information is in response to an input by the step of inputting the new subject importance level, Is changed to the new subject importance level and stored, and the step of calculating the video importance level is held in response to the retention of the new subject importance level. It may be a step of calculating video importance based on subject importance.
[0047]
The video information includes shooting time information related to the time when the video was shot, and the subject information includes event information related to an event that occurred in the subject and an event occurrence time that represents the time when the event occurred, The step of calculating importance may be a step of calculating the video importance based on the event information and the shooting time information.
[0048]
  Of the present inventionComputer readableThe recording medium stores the information of the video imaged in the space and the shooting time information related to the time when the video imaged, the step of reading out the stored video information, the event generated in the space, and the event The event information including the occurrence time of the event, the event importance level indicating the importance of the event and the correspondence relationship with the video, and the managed event information and the stored shooting time information AndIndicates the importance of the videoCalculating the video importance, controlling the display of the video represented by the video information based on at least the calculated video importance, and displaying the video according to the control.Let the computer runThis achieves the above object.
[0049]
The video is composed of image frames, the image frame includes two or more regions, and the step of calculating the video importance is a step of calculating the video importance for each of the two or more regions, The step of controlling the display may be a step of controlling the display of the image frame by determining a mode of displaying each of the two or more regions based on the calculated video importance.
[0050]
The video includes a first video shot at a first time and a second video shot at a second time, and the step of calculating the video importance level includes the step of calculating the video importance level. Calculating a first video importance level that is an importance level and a second video importance level that is a video importance level of the second video, and the step of controlling the display includes a step of displaying the video A mode of displaying the video is determined based on a display rule that defines a rule and the calculated first video importance and second video importance, and the first video and the video based on a shooting time It may be a step of controlling display so that the second video is synchronized.
[0051]
The subject information may have a data structure in which at least the name of the subject, the position where the subject exists, and the subject importance are stored in association with each other.
[0052]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below with reference to the accompanying drawings.
[0053]
In the embodiment of the present invention, the importance of the video is calculated based on the subject information including the correspondence relationship between the subject existing in the captured space and the subject importance indicating the importance of the subject, and the video is displayed. The “Correspondence” is a correspondence between the degree of subject importance of the subject. As a result, if the subject is an important subject, the video importance level is calculated and displayed accordingly, so that any video taken with any camera may be important. It can be displayed as you can see.
[0054]
In the embodiment of the present invention, when a new subject importance is input and held, video importance based on the held subject importance is calculated in response to the input. As a result, when the subject importance level is changed, video display in which the change is immediately reflected is realized.
[0055]
In the embodiment of the present invention, when an event such as an alarm or an action occurs in the subject, the video importance is further calculated based on the event information and the shooting time information. Since the display of the video according to the importance of the event as well as the importance of the subject itself is realized, even if the video is the same subject, the video importance of the video where the event was shot becomes higher, The image can be displayed so that the importance of the image is high. Therefore, it is possible to observe the event quickly and accurately.
[0056]
In the embodiment of the present invention, the video importance level can be calculated also based on event information and shooting time information. This makes it possible to observe an event based only on information about the event including the state before and after the occurrence of the event.
[0057]
(Embodiment 1)
1-1. Outline of operation of video display device in this embodiment
FIG. 1 is a block diagram illustrating a configuration of a video display device 10 according to the present embodiment.
[0058]
The video display apparatus 10 according to the present embodiment divides a live video into groups according to the importance of the video, and performs multi-screen composite display that displays a plurality of videos (FIGS. 2A and 2B). At that time, subject importance is set for the subject included in the captured video, and the video importance of the entire video is determined based on the subject information. The more important the subject, the greater the influence on the video importance. Therefore, if the image includes the subject, the display layout can be automatically determined and displayed so that it can be distinguished from other images by performing highlighting or the like as a highly important image (FIG. 3A and FIG. 3). (B), FIG. 18 (a)).
[0059]
1-2. Configuration and operation of video display device in this embodiment
(A) Configuration of video display device in the present embodiment
The video display device 10 includes a plurality of cameras 11a to 11n, a video information acquisition unit 12, a subject information management unit 13, a subject information storage unit 14, a video importance level calculation unit 15, a display control unit 16, and a display. Part 17.
[0060]
The plurality of cameras 11 a to 11 n capture a predetermined space and transmit the captured video to the video information acquisition unit 12. The plurality of cameras 11a to 11n may be fixed or movable in terms of shooting direction, range, or installation position. In the following description, the video information is described as a plurality of video information respectively transmitted from the plurality of cameras 11a to 11n. However, even a plurality of video information from a single camera may be received from a single camera. It may be single video information.
[0061]
The video information acquisition unit 12 is provided for acquiring video information and supplying the acquired video information to the display control unit 16. The video information acquisition unit 12 receives video information from the plurality of cameras 11a to 11n. The video information acquisition unit 12 further receives a selection instruction command for selecting which video information is to be selected from the display control unit 16. The video information acquisition unit 12 selects necessary video information to be displayed from the received video information based on the selection instruction command, and transmits it to the display control unit 16. A more detailed configuration and operation of the video information acquisition unit 12 will be described later.
[0062]
The subject information management unit 13 manages subject information. Here, the “subject information” is information including a correspondence relationship between the subject existing in the captured space and the subject importance level indicating the importance of the subject (subject information will be described later with reference to FIG. 5). ). “Managing subject information” means that the subject information management unit 13 accepts a change in subject information from a user (operator) and sets new subject information or holds the set subject information. The subject information management unit 13 sends the set subject information to the subject information storage unit 14, and the subject information storage unit 14 holds the subject information. The subject information management unit 13 has a subject information editing function in order to accept changes in subject information from a user (operator). The editing function is provided as means for changing the setting of subject information in terms of software or hardware such as a text editor. When a text editor or the like is used, the operator can interactively change the setting.
[0063]
The subject information storage unit 14 receives and stores the set subject information from the subject information management unit 13. The subject information storage unit 14 provides subject information to the video importance level calculation unit 15. The subject information storage unit 14 may be a known data storage unit such as a semiconductor memory represented by a RAM (Random Access Memory) or a ROM (Read Only Memory), a storage device using a magnetic storage medium such as a hard disk, an optical disk, or the like. .
[0064]
The video importance calculator 15 is provided to calculate the video importance of a video. The “video importance” is a degree that represents the importance of a video obtained by photographing the space including the subject. The video importance level calculation unit 15 receives the subject information from the subject information storage unit 14 and calculates the video importance level based on the received subject information. The video importance calculation unit 15 transmits the calculated video importance to the display control unit 16. The video importance calculator 15 further classifies cameras that shoot videos with high video importance into predetermined groups in order, and assigns a group ID to each group.
[0065]
The display control unit 16 performs data processing necessary for display, for example, determination of a video display position, synthesis processing, and the like. The display control unit 16 receives the video importance calculated from the video importance calculation unit 15. Subsequently, the display control unit 16 sends a selection instruction command for selecting which video information should be selected to the video information acquisition unit 12 based on the received video importance, and receives the video information according to the instruction command. Subsequently, the display control unit 16 controls display of video. “Controlling the display of the video” means determining how to display the video represented by the video information based on the received video importance. The determination of the display of the video includes generating the data of the composite screen by calculating the size and display position of the composite image to be displayed and performing appropriate signal processing. The display control unit 16 transmits the generated composite screen data to the display unit 17.
[0066]
The display unit 17 is a device for displaying a composite screen. The display unit 17 can be a monitor or the like.
[0067]
Next, specific configurations of the video information acquisition unit 12 and the display control unit 16 will be described with reference to FIG.
[0068]
FIG. 40 is a block diagram showing a specific configuration of the video information acquisition unit 12 (FIG. 1) and the display control unit 16 (FIG. 1).
[0069]
The video information acquisition unit 12 (FIG. 1) includes at least a video matrix switcher 401. The video matrix switcher 401 outputs N video signals designated from the M input video signals to the display control unit 16 based on the designation from the display control unit 16 (M ≧ N).
[0070]
The display control unit 16 includes at least a graphics board 402 and a multi-screen overlay board 403. The graphics board 402 receives the composite screen data generated by the multi-screen overlay board 403, and generates video data to be displayed on the display unit 17 (FIG. 1). The multi-screen overlay board 403 receives N video signals from the video information acquisition unit 12 and video importance from the video importance calculation unit (FIG. 1). Based on the received N video signals and video importance, the multi-screen overlay board 403 calculates the size and display position of the video in order to superimpose and display each of the video signals as a video window. The calculated composite screen data is transmitted to the graphics board 402.
[0071]
The video matrix switcher 401, the graphics board 402, and the multi-screen overlay board 403 configured in this manner are all commercialized, and detailed description thereof is omitted.
[0072]
(B) Operation of the video display device in the present embodiment
FIG. 4 is a flowchart showing an operation procedure of the video display device 10 (FIG. 1).
[0073]
First, the video importance calculator 15 (FIG. 1) calculates the video importance of the video represented by the video information based on the subject information stored in the subject information storage 14 (FIG. 1) in advance (step 40). ). As a result, any video from a camera that is shooting a subject with a high subject importance will have a video importance according to the subject importance. Can be displayed. One of the great advantages of using the subject information is when a plurality of subjects are included in one video. When a subject having different importance is included in one video, it is necessary to display the video in consideration of the relationship of subject importance of the subject. If the subject importance is set for each subject, appropriate display can be automatically realized even in such a case. For example, the sum of subject importance levels of a plurality of subjects can be set as the video importance level of the video. Hereinafter, subject information and video importance will be described.
[0074]
FIG. 5 shows an example of subject information. The subject information is information including the correspondence between the subject existing in the space photographed by the camera as described above and the subject importance indicating the importance of the subject. More specifically, the subject information is described in association with the ID for identifying the subject, the name and type of the subject, the position coordinates that exist, the camera ID of the camera that captures the subject, the importance of the subject, and the like. If necessary, a remarks column can be provided to include more specific information. The subject information is stored in the subject information storage unit 14 (FIG. 1) as a text file, for example, and is managed by the subject information management unit 13 (FIG. 1). Any information included in the subject information may be in any form as long as it is information that can specify the characteristics, properties, and the like of the subject. For example, the position coordinates where the subject exists may be expressed by latitude and longitude, or may be expressed by its address or the like.
[0075]
FIG. 6 shows an example of the calculated video importance. The video importance level is calculated based on the subject information. For example, when there are a plurality of subjects described in the subject information in a space photographed by each of the plurality of cameras 11a to 11n (FIG. 1), the importance of each subject is calculated as described above. The sum of the degrees can be set as the video importance of the video shot by the camera. Alternatively, the subject importance of the subject having the greatest subject importance among the subjects in the space to be photographed can be set as the video importance of the video photographed by the camera. Which calculation method is used can be appropriately changed according to the purpose of photographing, the nature of the subject, and the like. The video importance illustrated in FIG. 6 indicates that the importance of the video of the camera specified by the camera ID corresponds to the importance of the subject. The video importance used in the present invention corresponds to the display priority information (FIG. 43) used in the conventional video display device 420 (FIG. 42). However, in the present invention, the video importance is calculated based on the subject importance described in the subject information, whereas in the conventional video display device 420 (FIG. 42), the display priority order of the video taken by the camera is displayed. However, the difference is determined by the measured value of the sensor attached to the camera.
[0076]
Referring to FIG. 4 again, the calculated video importance is stored in a data storage unit (not shown in FIG. 1) inside or outside video importance calculation unit 15 (FIG. 1) (step 41). . The data storage unit (not shown) can be a known semiconductor memory, hard disk, or the like.
[0077]
The video information acquisition unit 12 (FIG. 1) selects and acquires video information of cameras (for example, four cameras) of the camera group that is shooting the video to be displayed among the cameras 11a to 11n (step 42). This can be said to be selecting video information of cameras having the same group ID. As described above, selection of video information is a selection in which the display control unit 16 (FIG. 1) receives video importance and selects which video information should be selected by the video information acquisition unit 12 based on the video importance. This is realized by sending an instruction command.
[0078]
The display control unit 16 (FIG. 1) calculates the display mode of the received video information based on the display rule and the video importance level, and transmits the result to the display unit 17 (FIG. 1) for display (step). 43). Here, the display rule is, for example, a rule that is referred to when displaying a video that describes in what position on the display unit what size and how long it should be displayed according to the priority order. .
[0079]
Subsequently, the subject information management unit 13 (FIG. 1) determines whether or not the subject information has been changed (step 44). This determination is performed by a processing circuit such as a central processing unit not shown in FIG. Alternatively, this determination may be performed by the video importance calculation unit 15 (FIG. 1) regularly or when there is an input from the subject information management unit 13 (FIG. 1).
[0080]
When the subject information management unit 13 (FIG. 1) determines that the subject information has been changed, the display control unit 16 (FIG. 1) stops displaying the video (step 45), and the changed subject information is used as the subject. The information is stored in the information storage unit 14 (FIG. 1) (step 46). Thereafter, the operation from step 40 is repeated, the video importance is calculated from the changed new subject information, and the display reflecting the change is performed again. Here, the subject information is changed when, for example, a certain subject disappears, when a new subject appears, or when the subject importance of the subject already included in the subject information changes. is there.
[0081]
If the subject information management unit 13 (FIG. 1) does not determine that the subject information has been changed, a determination is then made as to whether or not an instruction to stop displaying the video has been issued (step 47). An instruction to stop the display of the video is given from the outside by an operator or the like.
[0082]
If there is no instruction to stop displaying the video, the operation from step 42 is repeated, and video information is acquired to display the video. When this determination is repeated a plurality of times, the acquired video information is the video information of the video having the highest video importance level next to the video importance level of the video selected immediately before. Therefore, video information from a camera group that provides video having the next highest video importance level is acquired. When the display of the video with the lowest video importance is completed, the video with the highest video importance is displayed again.
[0083]
If an instruction is given to cancel the display of the video, the process ends and the display of the video is stopped.
[0084]
As described above, the operation of the video display device 10 (FIG. 1) is performed.
[0085]
The configuration of the video display device 10 (FIG. 1) may further include a display layout unit. The display layout unit (not shown) is provided in, for example, the display control unit 16 (FIG. 1), and displays a video display layout based on the video importance calculated by the video importance calculation unit 15 (FIG. 1). Display layout information for determination is created.
[0086]
FIG. 7 is a flowchart showing the operation procedure of the video display apparatus 10 (FIG. 1) including the step of determining the display layout. In this flowchart, steps 70 and 71 correspond to steps 40 and 41 in FIG. 4, respectively, step 73 corresponds to step 42 in FIG. 4, and steps 75 to 78 correspond to steps 44 to 47 in FIG. Therefore, those explanations are omitted. Therefore, the processing performed in step 72 and step 74 will be described.
[0087]
The display layout unit (not shown) determines the display layout based on the video importance level calculated by the video importance level calculation unit (FIG. 1) and creates display layout information (step 72).
[0088]
FIG. 8 shows an example of display layout information. The display layout information is information used for displaying videos of a plurality of groups, each group including four cameras. The display layout information includes the display time for displaying the video shot by each camera, the camera ID of the camera that shot the video, the display position of the video on the screen, the display size (the number of dots in the vertical and horizontal directions of the display window), etc. Contains information.
[0089]
The video controller 16 (FIG. 1) displays the video represented by the video information based on the display rule and the display layout information (step 74). In step 43 of FIG. 4, the video display device 10 (FIG. 1) displayed the video based on the display rule and the video importance. However, by determining the display layout in advance, a faster video display process is performed. Can be realized.
[0090]
(C) Video display of the video display device in the present embodiment
Below, the display mode of the image | video displayed on the display part 17 (FIG. 1) by the processing operation of the video display apparatus 10 (FIG. 1) is demonstrated.
[0091]
FIGS. 2A and 2B show an example of an aspect of an image displayed by the processing operation of the image display device 10 (FIG. 1). This display is useful when, for example, a tourist guide or the like wants to view a video of a famous building in a predetermined order.
[0092]
First, an image of a public facility-related building of importance level 1 is displayed (FIG. 2A). A high subject importance level is set for each public facility-related building, which is the subject, and the video importance of the image of the public facility is increased (here, the importance level is displayed). From the first). After that, an image of a house-related building of importance nth group is displayed (FIG. 2B). Compared to the subject importance set for the public facility-related building, the subject importance set for the housing-related building is small. For this reason, the video importance of the image of the housing-related building is relatively small (here, the n-th importance), and the housing-related building is displayed later than the public facility-related building.
[0093]
In the present invention, in view of the fact that there is a difference in the video importance of the images taken of the same public facility-related building, a difference is also provided in the position where the video is displayed.
[0094]
FIG. 3 is an example showing video display positions in groups of the same importance.
[0095]
In the same importance group, the video with the highest video importance is displayed on the upper left of the screen, and the video with the lowest video importance is displayed on the lower right of the screen. FIG. 3A shows a display example of video when the screen shown in FIG. 2 is divided into four. FIG. 3B is a display example of video when the screen is divided into nine. Which video importance level is displayed at which position on the screen is appropriately determined and can be defined as a display rule. Furthermore, it is possible to define a display rule in which a predetermined number of videos with high importance are displayed larger and the remaining videos are displayed smaller (see FIG. 18A). It is also possible to select and switch the display of each group interactively according to an operator's instruction.
[0096]
In the display rule, the display time described in the display layout information (FIG. 8) can also be defined. Therefore, it is possible to set a longer display time for a higher importance group. For example, the most important group can be set to 60 seconds, the second is 50 seconds, and the third is 40 seconds.
[0097]
The parameters defined by these display rules are written in a software program executed by a central processing unit (not shown) for operating the video display device 10 (FIG. 1) (hereinafter, “ It is possible to realize a display reflecting the change by separately providing a parameter description file or the like that can be changed and referring to the parameter as needed. Furthermore, the entire display rule is described in a script that is a language that can be easily described by the end user in an expression that is close to natural language, and can be changed as a script file. You can choose from.
[0098]
As described above, according to the video display apparatus 10 (FIG. 1) according to the first embodiment, the video importance level of the video is calculated based on the subject information including the subject importance level of each subject, and the video importance level is calculated. Video is displayed based on the degree. Therefore, it is possible to display a video according to the importance of the subject included in the video. The operator can display the video according to the importance of the subject simply by determining the importance of the subject, so that the troublesome work of determining the display layout is not required and an objective video display is possible. Obtainable. Furthermore, by defining display rules, a flexible video display is realized in which an operator can observe an important group for a long time and can see an arrangement with a determined importance level on the screen.
[0099]
Note that the video represented by the video information may be a moving image, a still image, or a mixture thereof.
[0100]
(Embodiment 2)
2-1. Outline of operation of video display device in this embodiment
FIG. 9 is a block diagram showing the configuration of the video display device 90 in the present embodiment.
[0101]
The video display device 90 according to the present embodiment displays video using video information stored in a storage device instead of directly using video information input from a plurality of cameras.
[0102]
The video display device 90 of the present embodiment further uses a map to visually understand the position where the video was shot, and displays the shot video near the corresponding position on the map (FIG. 10 ( a), (b) and (c)). As described in the first embodiment, the video is further displayed in order from the important video using the video importance calculated based on the subject information. Therefore, all of the advantages obtained with the video display device 10 (FIG. 1) described in the first embodiment can also be obtained.
[0103]
2-2. Configuration and operation of video display device in this embodiment
(A) Configuration of video display device in the present embodiment
The video display device 90 includes a plurality of cameras 11a to 11n, a video information acquisition unit 12, a subject information management unit 13, a subject information storage unit 14, a video importance calculation unit 15, a display control unit 16, and a display. Part 17. The video display device 90 further includes a video information storage unit 93, a map information storage unit 94, a map information reading unit 95, and an operation unit 96.
[0104]
Hereinafter, among the constituent elements of the video display device 90, the constituent elements having the same reference numerals as the constituent elements of the video display device 10 (FIG. 1) are the configurations of the constituent elements of the video display device 10 (FIG. 1). Since it is the same as the function, its description is omitted. Therefore, hereinafter, the video information storage unit 93, the map information storage unit 94, the map information reading unit 95, and the operation unit 96 will be described. When an operation different from the video display device 10 (FIG. 1) or an additional operation is performed, it will be described each time.
[0105]
The video information acquisition unit 12 of the video display device 90 includes a video information input unit 91 and a video information reading unit 92. The video information input unit 91 receives video information from the cameras 11a to 11n. The video information input unit 91 stores the input video information in the video information storage unit 93. The video information reading unit 92 reads the video information stored in the video information storage unit 93 and transmits it to the display control unit 16. Which video information is selected and read is based on a selection instruction command from the display control unit 16 as described in the first embodiment.
[0106]
The video information storage unit 93 is provided for storing video information in advance. The video information storage unit 93 receives video information from the video information input unit 91 and stores the video information. The video information stored in the video information storage unit 93 may be a plurality of pieces of video information captured by a plurality of cameras at a plurality of positions. Similar to the subject information storage unit 14, the video information storage unit 93 may be a known data storage unit such as a semiconductor memory represented by RAM or ROM, a storage device using a magnetic storage medium such as a hard disk, an optical disk, or the like. The video information storage unit 93 reads predetermined video information based on a read signal from the video information read unit 92 and transmits the predetermined video information to the video information read unit 92.
[0107]
As is apparent from the above description, in the video display device 90 according to the present embodiment, the video information from the cameras 11a to 11n is not directly used for display as in the first embodiment, but is temporarily stored in the video information. It is stored in the section 93 and necessary information is used for display as appropriate.
[0108]
The map information storage unit 94 is provided for storing in advance map information that plays a background role in display. The map information storage unit 94 stores map information related to a map for visually displaying a position where a video is taken. The map information storage unit 94 receives map information in advance from an appropriate input means (not shown) and stores the received map information. The map information storage unit 94 provides map information to the map information reading unit.
[0109]
The map information reading unit 95 is provided for reading map information. The map information reading unit 95 receives a map reading command from the display control unit 16 and reads map information from the map information storage unit 94 accordingly. The map information reading unit 95 transmits the read map information to the display control unit 16.
[0110]
As will be described later, the operation unit 96 is used to selectively display a superimposed image in response to an instruction from an operator or to instruct to display another image. The operation unit 96 may be a well-known input unit such as a keyboard, a mouse, or a touch panel.
[0111]
Note that the subject information storage unit 14, the video information storage unit 93, and the map information storage unit 94 may be separate storage devices or regions of different positions in the same storage device (for example, a hard disk).
[0112]
(B) Operation of video display device and video display example in the present embodiment
FIG. 11 is a flowchart showing an operation procedure of the video display device 90 (FIG. 9).
[0113]
First, the video importance level calculation unit 15 (FIG. 9) calculates the video importance level based on the subject information and the video management information (step 110). More specifically, the video importance is calculated based on the position coordinates of the subject included in the video and the position coordinates of the camera that captured the video. The “positional coordinates of the subject included in the video” is included in the subject information (FIG. 5). “Position coordinates of the camera that shot the video” is included in the video management information. The video management information is stored in a data storage unit (not shown in FIG. 9) inside or outside the video importance level calculation unit 15 (FIG. 1).
[0114]
FIG. 12 shows an example of video management information. The video management information is management information that describes the characteristics of each of the plurality of video information. Video management information includes video ID, video type (video or still image, video resolution, encoding method, etc.), camera ID assigned to each camera, video file name, video shooting time, video Includes the shooting location, video importance, etc. The video importance level is stored in step 111 described below.
[0115]
A procedure for calculating video importance will be described. The video importance level calculation unit 15 compares the position of the subject described in the subject information (FIG. 5) with the shooting position described in the video management information (FIG. 12). As a result of the comparison, if the difference is within a certain range (that is, if the subject and the camera are within a certain distance), the video importance calculation unit 15 estimates that the camera is photographing the subject or the vicinity thereof. can do. When there are a plurality of subjects, the video importance calculation unit 15 can estimate which subject the camera is photographing by performing the comparison and estimation described above for each. The video importance calculation unit 15 obtains the video importance by obtaining the sum of the subject importance of the subjects estimated to be taken by the camera or by adopting the maximum value of the subject importance of the subject. Can do.
[0116]
The video importance calculated by the video importance calculator 15 (FIG. 9) is stored in the video management information as a part of the video management information (step 111).
[0117]
Subsequently, the display control unit 16 (FIG. 9) reads the map information stored in the map information storage unit 94 (FIG. 9) via the map information reading unit 95 (FIG. 9), and based on the read map information, The map represented by the map information is displayed on the display unit 17 (FIG. 9) (step 112). At this time, the map display information is stored in a data storage unit (not shown in FIG. 9) inside or outside the display control unit 16 (FIG. 9). The map display information is information representing the characteristics of the displayed map such as the map display range and scale.
[0118]
The display control unit 16 reads the video information based on the video importance and the map display information (step 113). More specifically, the display control unit 16 specifies the shooting position (single or plural) existing in the display range of the map described in the map display information, and the video from the camera at the shooting position is, for example, the video importance level. N are selected in descending order. The display control unit 16 reads the selected video information from the video information storage unit 93 (FIG. 9) via the video information reading unit 92 (FIG. 9). Note that N videos are selected in descending order of video importance, but this can be changed according to display rules.
[0119]
The display control unit 16 displays the read video information in the vicinity of the position on the map corresponding to the actual shooting position based on the display rule and the video importance (step 114).
[0120]
FIG. 10 shows a map and an image displayed in association with the map.
[0121]
FIG. 10A shows an example in which one video is displayed per camera. Since each image and the position where each image was shot are displayed so that they can be visually understood, the situation can be easily grasped. Here, when the shooting positions are the same or when the shooting positions are close to each other on the map, it is possible to read out all of the plurality of videos as one group, and display, for example, videos with old shooting times as representative videos. . At this time, for example, by displaying another window in a superimposed manner under the representative video window, the operator can easily grasp that another video exists.
[0122]
FIG. 10B shows an example in which a plurality of images from a predetermined camera are displayed simultaneously. The example shown in FIG. 10B is basically the same as the example described with reference to FIG. However, in the example of FIG. 10A, when a plurality of images are displayed, one representative image is displayed. However, in this example, the plurality of images are displayed at a predetermined number simultaneously. The designation of the video to be displayed is a video taken by a predetermined camera. The operator can specify the video to be displayed simultaneously with the camera. Note that the display of only the representative video described in the example of FIG. 10A can be shifted to the display of this example by an appropriate operation. For example, when it is desired to view another video belonging to the same group, it is realized by specifying a video on the map, opening a menu, and selecting “other video display” in the menu.
[0123]
Referring to FIG. 11 again, when a map and an image associated with the map are displayed, it is subsequently determined whether there is an instruction from the operator to change the display of the map (step 115). This is a case where the operator performs map operations such as map scrolling and scale change via the operation unit 96 (FIG. 9). This map operation is performed when the scale of the map is too large and it is desired to limit the area and view the video.
[0124]
If there is an instruction to change the display of the map, the display control unit 16 (FIG. 9) changes the display content of the video and the map display information (step 116). That is, the map display information such as the video to be displayed corresponding to the new map and the scale of the map is changed. Thereafter, the process returns to step 112.
[0125]
FIG.10 (c) shows the example which changed and displayed the display setting of the whole country of Japan shown in FIG.10 (a) or (b) to the Kyushu area. Even if the display setting is changed, the video is displayed by the same processing as described with reference to FIG. Therefore, the video with high importance existing in the newly displayed map range is updated and displayed.
[0126]
As described above, according to the present embodiment, it is possible to obtain a visual effect that an image of an important subject or the vicinity of the subject can be viewed while grasping the shooting position on a map. Furthermore, it is possible to cope with the addition or deletion of a subject or a change in the importance level of the subject simply by correcting the subject information, and a complicated display layout correction work can be omitted. Furthermore, since the captured video is temporarily stored and used, it is possible to repeatedly display and observe a plurality of videos with the same importance or changing the importance of the subject.
[0127]
(C) Modification of this embodiment
In the calculation of the video importance in step 110, it is estimated that the camera is shooting the subject or the vicinity thereof only by the distance between the video shooting position and the subject position. However, in this case, there may be a problem that the subject (photographing target) is not within the photographing range of the camera even though the distance is short, and is not photographed. This problem can be solved by describing shooting range information regarding the shooting range of the camera in the video management information (FIG. 12). For example, the position, direction, and zoom angle of view of the camera are described in the shooting range information. Here, the “zoom angle of view” is an angle indicating a range in which the camera can shoot. A procedure for determining whether or not the subject exists in the shooting range of the camera based on the position and direction of the camera and the zoom angle of view will be described.
[0128]
FIG. 13 shows the positional relationship between the shooting position and direction of the camera, the zoom angle of view, and the subject. Whether or not the subject is within the shooting range of the camera can be obtained from geometric calculation. That is, the angles (Φ1 and Φ2) formed by the straight line connecting the camera position and the subject position and the straight line in the camera direction are obtained, and if the absolute value is less than half of the zoom angle θ (θ / 2), the subject is within the shooting range. It can be determined that it exists. For example, subject 1 within the shooting range is | Φ1 | <θ / 2, and subject 2 outside the shooting range is | Φ2 |> θ / 2. Therefore, it can be determined that the subject 1 exists in the shooting range, but the subject 2 does not exist in the shooting range.
[0129]
The zoom angle of view θ can be obtained as follows with reference to FIG.
[0130]
FIG. 14 shows a diagram in which a subject is imaged through a camera lens. As shown in the figure, when the focal length is F and the size of the imaging element is S,
[0131]
[Expression 1]

[0132]
Thus, the value of the zoom angle of view θ can be obtained.
[0133]
Further, by describing circumscribed polygon information representing the size of the subject in the subject information (FIG. 5), it is possible to calculate how large the subject is photographed by the camera. The circumscribed polygon information is a two-dimensional or three-dimensional multi-element circumscribing the subject, which is used to approximately represent the existence area of the subject when the shape of the subject is complicated or includes a curved portion. This refers to square information.
[0134]
FIG. 15 shows the relationship between the shooting range and the subject and the shooting state.
[0135]
FIG. 15A shows a case where a subject (character “A”) exists in the center of the shooting range. The circumscribed polygon information of the character “A” is a rectangle circumscribing the character “A” as illustrated. At this time, as shown in the figure, the subject exists in the center of the video. FIG. 15B shows a case where a subject (character “A”) exists in a part of the shooting range (below the shooting range). At this time, a subject is present below the shot video. Note that the entire subject is not actually displayed, but the entire subject is illustrated in order to clarify the relationship between the imaging surface and the subject.
[0136]
A procedure for quantitatively calculating how much the subject is shown in the cases shown in FIGS. 15A and 15B will be described below. For example, as shown in FIG. 14, first, the imaging surface is set to the (XY) plane. Next, an (xy) plane that exists in the vicinity of the subject and is orthogonal to the shooting direction of the camera is considered, and the circumscribed polygon of the subject is projected onto the (xy) plane. Here, the point at which the (xy) plane and the shooting direction intersect is defined as the origin of the (xy) plane. In this (xy) plane, assuming that one vertex coordinate of the projected circumscribed polygon is (x0, y0), the coordinates (X0, Y0) of the point on the imaging plane (XY plane) corresponding to the vertex )
[0137]
[Expression 2]

[0138]
Given in. Here, L represents the distance from the lens to the plane on which the subject is projected (that is, the xy plane). Since the (xy) plane exists in the vicinity of the subject, the position information of the subject may be used as the distance from the lens to the (xy) plane. According to such a procedure, it is possible to obtain coordinates on the imaging surface corresponding to the vertexes of the circumscribed polygon. When the coordinates are obtained, the ratio of the area of the polygon on the photographing surface surrounded by the coordinates and the area of the photographing surface can be calculated, and the ratio of the subject to the photographing surface can be approximately obtained.
[0139]
By using the shooting range information, as illustrated in FIG. 15B, when only a part of the subject is shot due to the relationship between the shooting range of the camera and the position of the subject, the subject is captured by the camera. If the video importance of the video is the value obtained by multiplying the ratio (area ratio) of the subject and the subject importance of the subject, priority is given to the video that has a high subject importance as the video with high video importance. Can be displayed automatically.
[0140]
When video importance is calculated in this way, the shooting range information should be described for each image frame, not only for cameras with fixed shooting ranges, but also for videos shot with moving cameras and swing cameras. Thus, it is possible to extract and display preferentially only a moving image portion in which an important subject is shown.
[0141]
In all cases described in this embodiment, the above-described video display mode is an example, and a more flexible video display can be realized by appropriately changing the display rule. In the above-described example (2-2. (B)), the display rule stipulates that N images with high importance are displayed, and the display position of the image on the map is taken. It should be in the vicinity of the position, the group of images shot at the same position or a very close position should be displayed from the oldest shooting time, and images other than the representative image among the images belonging to the same group by the operator's operation It is to display selectively. However, for example, a display rule such as displaying all videos having a video importance level equal to or higher than a threshold value, or displaying video images that are equal to or higher than the threshold value and not greater than N, can be used depending on the purpose of the operator.
[0142]
Furthermore, an image that is not displayed with low importance is displayed in the vicinity of the shooting position on the map displayed as, for example, a camera-shaped icon, and the operator selects the camera icon on the map and instructs the image display. In this way, any video can be viewed with a simple operation regardless of the video importance level.
[0143]
(Embodiment 3)
3-1. Outline of operation of video display device in this embodiment
FIG. 16 is a block diagram showing a configuration of video display device 160 in the present embodiment.
[0144]
The video display device 160 in the present embodiment displays a plurality of videos according to the importance of the video. The importance of the video is determined based on the subject importance set for the subject included in the video. The video display device 160 can change the subject importance level even during video display, and reflects the change in real time on the video display. For example, when an alarm is generated from a sensor of a fuel valve in plant monitoring, the importance of a subject existing near the fuel valve or a subject of the fuel system is increased according to an instruction from an operator or an external module that performs operation control. Increasing the importance of video including. Thereby, the subject which is connected in the vicinity or systematically is preferentially displayed.
[0145]
3-2. Configuration and operation of video display device in this embodiment
(A) Configuration of video display device in the present embodiment
The video display device 160 includes a plurality of cameras 11a to 11n, a video information acquisition unit 12, a subject information management unit 13, a subject information storage unit 14, a video importance calculation unit 15, a display control unit 16, and a display. Part 17. The video display device 160 further includes an input unit 161 and / or an external module 162.
[0146]
Hereinafter, among the constituent elements of the video display device 160, the constituent elements having the same reference numerals as those of the constituent elements of the video display device 10 (FIG. 1) are the configurations of the constituent elements of the video display device 10 (FIG. 1). Since it is the same as the function, its description is omitted. Accordingly, the input unit 161 and the external module 162 will be described below. When an operation different from the video display device 10 (FIG. 1) or an additional operation is performed, it will be described each time.
[0147]
The input unit 161 is an input device for an operator to correct, change, and set subject information. The input unit 161 can be, for example, a keyboard. Subject information to be changed or set from the input unit 161 is transmitted to the subject information management unit 13.
[0148]
The external module 162 is a simulator or the like that calculates the plant state, and performs operation control. Subject information changed or set from the external module 162 is transmitted to the subject information management unit 13.
[0149]
The subject information management unit 13 receives an instruction to change the subject information from the input unit 161 and / or the external module 162, and further receives new subject information. The subject information management unit 13 stores the received new subject information in the subject information storage unit 14 in place of the corresponding previous subject information. Note that the subject information change instruction is not received separately from the new subject information, but may be regarded as being given when the subject information management unit 13 receives new subject information.
[0150]
(B) Operation of video display device and video display example in the present embodiment
FIG. 17 is a flowchart showing the operation procedure of the video display device 160 (FIG. 16). Of the steps shown in this flowchart, Steps 170 to 173 for calculating the video importance based on the subject importance and displaying the video represented by the video information are Steps 40 to 40 of the flowchart described with reference to FIG. 43, which are the same as those in FIG.
[0151]
In step 174, it is determined whether or not the subject information has been changed by the subject information management unit 13 (FIG. 16).
[0152]
When it is determined that the subject information has been changed by the subject information management unit 13 (FIG. 16) (that is, the subject that has received an instruction to change the subject importance from the input unit 161 (FIG. 16) or the external module 162 (FIG. 16)) When the information management unit 13 (FIG. 16) changes the predetermined subject information to new subject information received), the subject information management unit 13 (FIG. 16) converts the new subject information into the subject information storage unit 14 (FIG. 16). (Step 175). The video importance level calculation unit 15 (FIG. 16) changes the video importance level in real time and automatically based on the changed new subject information, and changes the video to be displayed. This is realized by executing Steps 170 to 173 successively after Step 175.
[0153]
If it is determined that the subject information has not been changed by the subject information management unit 13 (FIG. 16), the process waits until there is a change instruction, and continues to display the video. It should be noted that video can be displayed one after another by repeating the processing from steps 172 to 174 until there is an instruction to change the subject information. This is the same as repeating the processing of steps 42 to 44 and step 47 shown in FIG.
[0154]
Note that the video represented by the video information may be a moving image, a still image, or a mixture thereof. Although the video is displayed based on the display rule to be displayed as described above, the display rule can be arbitrarily changed.
[0155]
As described above, according to the present embodiment, the subject information management unit 13 (FIG. 16) follows the request from the input unit 161 (FIG. 16) or the external module 162 (FIG. 16) even during video display. The subject importance included in the subject information is changed. The display control unit 16 (FIG. 16) automatically changes the display of a plurality of videos according to the changed video importance as a result of reflecting the change in the subject information. Therefore, even when the type or position of the subject to be noticed changes depending on the situation, a video with a high video importance is automatically selected and displayed, and the operator can observe the subject quickly and accurately. Needless to say, the effect of the video display device 10 (FIG. 1) described in the first embodiment can also be obtained.
[0156]
(Embodiment 4)
4-1. Outline of operation of video display device in this embodiment
In the first to third embodiments, the time is changed in order from the video having the highest video importance, or N videos are selected and displayed from the plurality of videos. In the present embodiment, the display size of the video is changed according to the video importance level, or the degree of emphasis that changes the frame thickness and color of the window for displaying the video is changed.
[0157]
4-2. Video display in this embodiment
The video display device used in the present embodiment may be any of the video display devices described in the first to third embodiments. The video display devices 10, 90, and 160 (FIGS. 1, 9, and 16) display video as follows according to the video importance of the video to be displayed.
[0158]
FIG. 18 shows an example of a display screen in which the display size of the video with higher video importance is increased.
[0159]
FIG. 18A shows an example in which an image is displayed using the image display apparatus 10 (FIG. 1) according to the second embodiment. The display screen is divided into ten, and the video having the highest video importance and the second highest video are displayed in the upper half of the screen, and the other videos are displayed in the lower half of the screen. The display size of the video displayed on the upper half of the screen is, for example, four times the display size of the other video.
[0160]
FIG. 18B shows an example in which a video is displayed using the video display device 90 (FIG. 9) according to the second embodiment. Of the images displayed at the position on the map corresponding to the captured position, two images are enlarged and displayed in descending order of the importance of the image, and the images with the smaller image importance are displayed smaller.
[0161]
FIG. 18C shows an example in which a video is displayed using the video display device 90 (FIG. 9) according to the second embodiment. Of the videos displayed at positions on the map corresponding to the taken positions, the higher the video importance, the thicker the frame of the window for displaying the video and the darker the color.
[0162]
FIG. 19 shows an example of a display rule for changing the enhancement level (display size) of the video according to the video importance level. The display control unit 16 (FIGS. 1, 9, and 16) of the video display devices 10, 90, and 160 (FIGS. 1, 9, and 16) used in FIGS. By using the video importance level, the video display as described above can be realized.
[0163]
FIG. 19A shows a display rule for displaying a video having a high video importance level in a larger size. For example, the size of the video having the first video importance level is 480 × 320 dots, It is defined as 450 × 300 dots.
[0164]
FIG. 19B shows a rule that the frame of a video display window having a high video importance is thick and the color is changed. For example, the first window frame is three-point red and the second is 2 Stipulated as 5 points orange. You may make it use a color in an order from a conspicuous color. It should be noted that the above-described display rule of highlighting is merely an example, and as another example, the operator can display the window frame of an important video blinking, increase the contrast or luminance of the important video, etc. 160 (FIGS. 1, 9, and 16), various display rules can be defined and changed.
[0165]
In these examples, the video represented by the video information may be a moving image, a still image, or a mixture thereof.
[0166]
As described above, according to the present embodiment, the display control unit 16 (FIGS. 1, 9, and 16) changes the degree of video highlight display based on the display rule and the video importance level. Therefore, the importance of the subject can be observed more consciously, such as displaying an important image larger or making the frame color of the window displaying the important image conspicuous.
[0167]
(Embodiment 5)
5-1. Outline of operation of video display device in this embodiment
In the first to fourth embodiments, the video may be either a still image or a moving image, but in the present embodiment, the video is a moving image. In the present embodiment, a video having a higher video importance is displayed with a higher video quality.
[0168]
5-2. Video display in this embodiment
The video display device used in the present embodiment may be any of the video display devices described in the first to fourth embodiments. The video display devices 10, 90, and 160 (FIGS. 1, 9, and 16) display the video described below according to the video importance of the video to be displayed.
[0169]
FIG. 20 shows an example of a display rule for changing the display quality of video according to the order of video importance.
[0170]
The display control unit 16 (FIGS. 1, 9, and 16) displays a video based on the video importance and this display rule. For example, the display quality for displaying the video of the first video importance level is 30 frames / second, the resolution is 480 × 320, the second is 25 frames / second, 450 × 300,. The higher the frame rate, the smoother the motion can be displayed, and the higher the resolution, the more precisely the video can be displayed. As is well known, a moving image is realized when a plurality of temporally continuous image frames are displayed.
[0171]
As described above, according to the present embodiment, the display control unit 16 (FIGS. 1, 9, and 16) displays the video by changing the degree of video quality such as frame rate and resolution based on the display rule and the video importance. Control the display of video. Therefore, since the video having a high video importance (that is, the important video) is displayed more easily and precisely, the operator can naturally watch the important video. Furthermore, since the video with relatively low video importance (that is, video with relatively low importance) is displayed with the image frames thinned out, the display on the video display devices 10, 90, and 160 (FIGS. 1, 9, and 16) is displayed. The processing load is reduced, and the video display devices 10, 90, and 160 (FIGS. 1, 9, and 16) can be manufactured using a relatively inexpensive processing circuit. Therefore, the manufacturing cost of the video display devices 10, 90, and 160 (FIGS. 1, 9, and 16) can be kept low.
[0172]
In addition, when compressed and recorded using an encoding method without inter-frame encoding such as JPEG (Joint Photographic Experts Group), only necessary video information can be efficiently read according to the frame rate. it can. Furthermore, when the video display device 90 (FIG. 9) according to the present invention is used, the video information storage unit 93 (FIG. 9) is divided into the same video at different resolutions or divided into several stages from low frequency components to high frequency components. It is possible to encode (hierarchical encoding), store video information, and read only video information having a resolution necessary for display. Thereby, the load of the reading process can be reduced. Hierarchical coding is a well-known technique and is not the essence of the present invention, so the description is omitted. In view of the fact that JPEG is a coding that is frequently used for still images, the description of this embodiment can be applied not only to moving images but also to still images. When a still image is used, the frame rate condition is invalidated and only the resolution is valid.
[0173]
In the video display devices 10, 90, and 160 (FIGS. 1, 9, and 16) in the present embodiment, the video is displayed so as not to exceed the display capability according to the processing capability such as the video reading speed and display performance that the device has. By setting the display quality to, it is possible to display a plurality of high-quality videos using the maximum capability of the device.
[0174]
(Embodiment 6)
6-1. Outline of operation of video display device in this embodiment
FIG. 21 is a block diagram showing a configuration of video display apparatus 210 in the present embodiment.
[0175]
As described in the fifth embodiment, the video display device 210 according to the present embodiment displays a video with a higher video quality as the video importance is higher. However, the video display apparatus 210 according to the present embodiment receives and stores video as video information, the video server 211 that distributes the video information via the network, and the video display that receives and displays the video information from the network. It is an integrated device including the terminal 212. When requested by the video display terminal 212 to transmit video information of a predetermined video quality, the video server 211 transmits the video information to the video display terminal 212 with the video quality.
[0176]
6-2. Configuration and operation of video display device in this embodiment
(A) Configuration of video display device in the present embodiment
The video display device 210 includes a plurality of cameras 11a to 11n, a video server 211, and a video display terminal 212. The video server 211 plays the role of a video database for storing video information, and provides the video information held in response to a request. The video display terminal 212 requests the video server 211 for necessary video information and displays the received video information.
[0177]
Videos taken by the plurality of cameras 11a to 11n are transmitted to the video server 211 as video information. The video server 211 receives and accumulates video information transmitted from the plurality of cameras 11a to 11n, and transmits the video information to the video display terminal 212 in response to a request from the video display terminal 212. The video display terminal 212 requests the video server 211 to transmit video information, receives the transmitted video information, and then displays the video represented by the video information.
[0178]
The video server 211 includes a video information acquisition unit 12, a video information storage unit 23, and a transmission / reception unit 213.
[0179]
Hereinafter, among the constituent elements of the video display device 210, the constituent elements denoted by the same reference numerals as those of the video display device 10 (FIG. 1) and the video display device 90 (FIG. 9) will be described. Since it is the same as the structure and function of the component of (FIG. 1) and the video display apparatus 90 (FIG. 9), the description is abbreviate | omitted. When an operation different from the operation already described or an additional operation is performed, it will be described each time.
[0180]
The video information acquisition unit 12 receives a transmission request for predetermined video information transmitted from the video display terminal 212 via the transmission / reception unit 213, and reads the video information from the video information storage unit 93. The video information acquisition unit 12 transmits the read video information to the video display terminal 212 via the transmission / reception unit 213. The video information storage unit 93 stores a plurality of pieces of video information representing the same video but having different display qualities.
[0181]
The transmission / reception unit 213 has a function for transmitting / receiving data such as video information. The transmission / reception unit 213 receives the video information transmission request and other information transmitted from the video display terminal 212 and transmits them to the video information acquisition unit 12. The transmission / reception unit 213 receives video information and other information from the video information acquisition unit 12 and transmits the video information to the video display terminal 212. The transmission / reception unit 213 connects the video server 211 to an information communication medium such as a network line.
[0182]
The video display terminal 212 includes a transmission / reception unit 214, a subject information management unit 13, a subject information storage unit 14, a video importance level calculation unit 15, a display control unit 16, and a display unit 17.
[0183]
The display control unit 16 determines the video quality of the video to be displayed based on the display rule and the video importance calculated by the video importance calculation unit 15. The display control unit 16 requests the video server 211 to transmit video information having the video quality via the transmission / reception unit 214. The video display unit 16 controls the display to display the video information transmitted from the video server 211 via the transmission / reception unit 214.
[0184]
The transmission / reception unit 214 has a function for transmitting / receiving data such as video information. The transmission / reception unit 214 receives a video information transmission request and other information from the display control unit 16 and transmits the received video information to the video server 211. The transmission / reception unit 214 receives the video information and other information transmitted from the video server 211 and transmits them to the display control unit 16. The transmission / reception unit 214 connects the video display terminal 212 to an information communication medium such as a network line.
[0185]
(B) Operation of video display device and video display example in the present embodiment
FIG. 22 is a flowchart showing the operation procedure of the video display device 210 (FIG. 21).
[0186]
The calculation of the video importance based on the subject information (step 220) and the storage of the calculated video importance (step 221) are the same as steps 40 and 41 of FIG.
[0187]
In step 222, the display control unit 16 (FIG. 21) determines the display quality of the video to be displayed based on the video importance and the display rule. As described in connection with the fifth embodiment, the display quality represents the frame rate and resolution (FIG. 20) of a moving image determined according to the video importance. When the display quality of the video is determined, the display control unit 16 (FIG. 21) requests the video server 211 (FIG. 21) to transmit the video information of the video. This request is made by transmitting the video ID of the video to be displayed and the video quality of the video. The video information requested to be transmitted is not limited to one, and may be a plurality of video information corresponding to the number of videos that can be displayed.
[0188]
In step 223, the video server 211 (FIG. 21) reads the requested video information and transmits it to the video display terminal 212 (FIG. 21). The display control unit 16 (FIG. 21) of the video display terminal 212 (FIG. 21) receives the transmitted video information. Note that the reception of the video information is completed once for a still image, but may be continuously performed for a moving image while the steps described below are being performed.
[0189]
In step 224, the display control unit 16 (FIG. 21) displays the video represented by the received video information based on the display rule. Since the present embodiment relates to the display quality of video, the same effect as described in the first to fourth embodiments can be obtained by combining with the display mode described in the first to fourth embodiments. Note that the video display devices described in the first to fourth embodiments may include components that are not shown in the video display device 210 (FIG. 21) described in the present embodiment. Even in such a case, the same display elements may be provided in the video display device 210 (FIG. 21).
[0190]
Finally, in step 225, it is determined whether or not the display control unit 16 (FIG. 21) has received an instruction to end the display of the video from an operator or the like. An instruction to stop the display of the video is given from the outside by an operator or the like.
[0191]
If the instruction to end the video display has not been received, the processing from step 222 is repeated again to continue the video display. For example, when the video display device 210 (FIG. 21) according to the present embodiment displays video in the same display pattern as the video display device 10 (FIG. 1) according to the first embodiment, smooth and high resolution is achieved. The video is displayed repeatedly while changing the display time from the group of videos with high video importance displayed in (1) to the group of videos with low video importance thinned out and displayed at a low resolution.
[0192]
When an instruction to end the video display is received, the video display ends.
[0193]
As described above, according to the present embodiment, the display control unit 16 (FIG. 21) displays video so that the video quality such as the frame rate and resolution is changed based on the display rule and the video importance. To control. Therefore, since the video having a high video importance (that is, the important video) is displayed more easily and precisely, the operator can naturally watch the important video. Furthermore, since a video with a relatively low importance level (that is, a relatively unimportant video) is displayed with its image frames thinned out, the display processing load on the video display device 210 (FIG. 21) can be reduced. The video display device 210 (FIG. 21) can be manufactured even by using a relatively inexpensive processing circuit. Therefore, the manufacturing cost of the video display device 210 (FIG. 21) can be kept low. The display rule can be appropriately changed according to the purpose.
[0194]
Furthermore, even if the video server 21 (FIG. 21) and the video display terminal 212 (FIG. 21) are connected via a network, only the video quality information necessary for display is transmitted over the network. Server and terminal computer resources can be used effectively. By setting the video quality to the video so as not to exceed the performance according to the performance such as the transmission band and processing speed of those resources, the capacity of the network-based video display device 210 (FIG. 21) is maximized. It is possible to display multiple videos with good quality.
[0195]
In this embodiment, the video information storage unit 93 stores a plurality of pieces of video information with different display qualities representing the same video. However, it is necessary to store one high-resolution video and convert the resolution appropriately. Display quality video may be used. For this purpose, it is preferable to use a known algorithm that performs geometric transformation such as reducing the resolution of the read encoded data while maintaining the orthogonal transformation region (frequency region obtained by discrete cosine transformation in the case of JPEG). This algorithm is applied to the encoded data that is recorded in the video server 211 (FIG. 21) after the high-resolution video is compressed and encoded using JPEG or the like and then read out. The read encoded data is converted into encoded data having a low resolution by geometric conversion and transmitted. According to this algorithm, the processing time is greatly shortened as compared with the case where the image data is once decoded and the resolution is reduced, and the video having the low resolution is encoded again. Further, it is not necessary to provide an area for storing a plurality of pieces of video information having different display quality representing the same video, and the storage area can be used effectively.
[0196]
(Embodiment 7)
7-1. Outline of operation of video display device in this embodiment
In the present embodiment, attention is paid to each of a plurality of image frames constituting a video. That is, the image of one image frame is divided into, for example, a plurality of regions (partial spaces) each including one subject, and each partial space is highlighted and displayed in a different manner. By applying the same processing to each of the plurality of image frames, each subject included in the video can be highlighted and displayed.
[0197]
7-2. Video display in this embodiment
The video display device used in the present embodiment may be any of the video display devices described in the first to sixth embodiments. The display control unit 16 of the video display apparatuses 10, 90, 160, and 210 (FIGS. 1, 9, 16, and 21) controls video display as follows according to the video importance of the video to be displayed.
[0198]
FIG. 23 illustrates an example of a camera that shoots three subjects and a subject that is displayed after being processed according to the present embodiment. The subject importance levels set for the three subjects “A”, “B”, and “C” are 0.8, 1.0, and 0.6, respectively. The captured video is virtually divided into a plurality of partial areas for each video of the image frame, and the subjects “A”, “B”, and “C” exist in separate partial spaces.
[0199]
FIG. 23A shows an example in which an image is displayed by changing the color of the frame surrounding the displayed subject in accordance with the image importance of the partial area where the subjects “A”, “B”, and “C” exist. Indicates. In FIG. 23A, a central area where the subject “B” is shown is surrounded by a dark frame, and the left and right partial areas where the subject “A” or the subject “C” is shown are lighter colored frames. Surround with. The color of the frame is determined based on the video importance of the partial area. The video importance level of the partial area is calculated by the video importance level calculation unit 15 (FIGS. 1, 9, 16, and 21) for each partial area of the image frame at the moment when the video is displayed. The video importance level is calculated based on the subject importance level of a subject existing in the area. As described above, the image frame is divided into partial areas, and the video importance level is defined for each partial area. Therefore, the video is emphasized for each partial area with a dark frame corresponding to the video importance level for each partial area. It can be displayed.
[0200]
In the present embodiment, the video represented by the video information may be a moving image, a still image, or a mixture thereof.
[0201]
FIG. 23B shows an example in which video is displayed by changing the display quality of the displayed subject in accordance with the video importance of the partial area where the subjects “A”, “B”, and “C” exist. . In the case of a moving image, the display quality represents the frame rate and resolution (FIG. 20) determined according to the video importance as described in connection with the fifth embodiment. In the case of a still image, it represents the resolution of the still image. The procedure for determining the video importance level for each partial area is the same as that described with reference to FIG. In FIG. 23B, the central area where the subject “B” exists has the highest video quality, the area where the subject “A” exists next, the area where the subject “C” exists next, and the video quality. Is lowered. JPEG2000 and the like have already been developed as an encoding method capable of generating encoded video information in which the quality of the video is changed for each partial area, but the description thereof is omitted because it is a known technique. The video may be a real-time video or a video once stored in the video information storage unit 93 (FIGS. 9 and 21).
[0202]
It should be noted that as to which region of an image frame has which subject, as described with reference to FIGS. 14 and 15 in the modification of the second embodiment, shooting range information and subject information for each image frame. And geometric information can be calculated using information such as a circumscribed polygon indicating the position and size of the subject.
[0203]
As described above, according to the present embodiment, it is possible to more clearly display a point to be noted in one image.
[0204]
(Embodiment 8)
8-1. Outline of operation of video display device in this embodiment
In the above-described embodiments, the “subject” mainly describes an object such as a building, a facility, or a person as a photographing target. In the present embodiment, a partial space of the shooting target space designated by a town name or address is also defined as a subject and described in the subject information. For example, in terms of displaying preferentially an image obtained by photographing an important town (a town with high image importance) as an observation target, or displaying the image preferentially by an operator specifying the town name or address, This has the same significance as the video display device 90 (FIG. 9) of the second embodiment.
[0205]
8-2. Operation of video display device in this embodiment
The video display device used in the present embodiment may be any of the video display devices described in the first to seventh embodiments.
[0206]
FIG. 24 shows an example of subject information in which a space is defined as a subject. In the subject information, an ID for identifying a space, a name and district such as a town name, a type of a county, a coordinate indicating a region of the space, a camera ID of a camera that captures the space, a subject importance, and the like are defined. The video importance calculator 15 (FIGS. 1, 9, 16, and 21) calculates the video importance of the video based on the subject information, and the display controller 16 (FIGS. 1, 9, 16, and 21) calculates the video importance. The display is controlled so that an image showing a high space is preferentially displayed. Furthermore, the video display devices 10, 90, 160, and 210 (FIGS. 1, 9, 16, and 21) receive parameters (name, type, area, camera ID, etc.) described in the subject information from the outside by an operator or the like. When received, the image including the subject is displayed with priority over other images. As a result, the operator can observe a video image of an arbitrary subject.
[0207]
As described above, according to the present embodiment, since the subject includes a partial space in the shooting target space such as a point or a region, it is specified not only by an object such as a building but also by a town name or a position on a map. Can be displayed with priority.
[0208]
In the above description, the partial space has been described as a fixed area such as a town or an address, but is not limited thereto. For example, in the video display device 90 linked with the map shown in FIG. 9, a point on the map designated by the operator and the vicinity thereof is defined as a dynamic partial space, and the video importance of this partial space is increased. Thereby, an image showing the vicinity of the designated point is displayed, and an image of a space in which the operator is interested can be displayed based on the map. For each partial space, the display quality can be changed according to the display rule as described in the seventh embodiment. Further, for example, when the mouse is moved along the road on the map, the video in the vicinity thereof is displayed one after another, and video display corresponding to walk-through on the map can also be realized.
[0209]
(Embodiment 9)
9-1. Outline of operation of video display device in this embodiment
FIG. 25 is a block diagram showing a configuration of video display apparatus 250 in the present embodiment.
[0210]
In the video display apparatus 250 according to the present embodiment, an event that has occurred in a subject is detected by the event occurrence detection unit 251, and an event importance level that indicates the importance of the event, an event occurrence time, and the like, and the subject importance level described above. Based on the video importance, the camera that was shooting the position where the event occurred (camera A in FIG. 26) and the camera that was shooting the vicinity of the position where the event occurred (camera ( Camera B) in FIG. 26 displays images taken before and after the event occurrence (FIG. 27). In the present embodiment, the video mainly represents a still image. The case where the video is a moving image will be described in Embodiment 10.
[0211]
9-2. Configuration and operation of video display device in this embodiment
(A) Configuration of video display device in the present embodiment
The video display device 250 includes a plurality of cameras 11a to 11n, a video information acquisition unit 12, a subject information management unit 13, a subject information storage unit 14, a video importance level calculation unit 15, a display control unit 16, and a display. A unit 17, a video information storage unit 93, a map information storage unit 94, a map information reading unit 95, and an operation unit 96 are included. The video display device 250 further includes an event occurrence detection unit 251.
[0212]
Hereinafter, among the constituent elements of the video display device 250, the constituent elements having the same reference numerals as the constituent elements of the video display device 90 (FIG. 9) are the configurations of the constituent elements of the video display device 90 (FIG. 9). Since it is the same as the function, its description is omitted. Therefore, the event occurrence detection unit 251 will be described below. In the case where an operation different from that of the video display device 90 (FIG. 9) or an additional operation is performed with respect to other components, the description will be made each time.
[0213]
In video display device 250 in the present embodiment, the video information stored in video information storage unit 93 includes information representing the actual video and shooting time information representing the time at which the video was shot. . In the procedure for storing video information, the video information input unit 91 receives information representing a video from the cameras 11 a to 11 n and stores the information representing the video in the video information storage unit 93. At the time of storage, a time stamp indicating the time when the image was taken is attached to the information indicating the video. In the present embodiment, the time stamp is attached to each image frame or each set of consecutive image frames. When a time stamp is attached to each set of consecutive image frames, at least the shooting time and the frame rate of the first frame are attached as the time stamp. In the following, it is determined at which time the video information includes the video imaged using this time stamp.
[0214]
The event occurrence detection unit 251 is provided for detecting the occurrence of an event. Examples of events are temperature and pressure abnormalities that occur in plant equipment, and accidents that occur on expressways. In the former case, the event occurrence detection unit 251 corresponds to a sensor such as a temperature sensor or a pressure sensor. In the latter case, the event occurrence detection unit 251 corresponds to a device that detects a notification by a telephone, an emergency button, or the like. When the event occurrence detection unit 251 detects an event, the event occurrence detection unit 251 instructs the subject information management unit 13 to change the subject information, and transmits information related to the generated event to the subject information management unit 13 to be described in the subject information. The event information regarding the event that has occurred includes the ID of the subject in which the event has occurred, the type of event, the time, the importance, and the like.
[0215]
When the subject information management unit 13 receives an instruction to change subject information and event information to be described, the subject information management unit 13 writes the event information in the subject information stored in the subject information storage unit 14.
[0216]
FIG. 28 shows an example of subject information including event information.
[0217]
The subject information includes a subject information table (FIG. 28A) and an event information table (FIG. 28B). In the subject information table (FIG. 28A), the number of events that have occurred in the subject and a pointer for referring to the event information table are described. In the event information table (FIG. 28B), an event ID, type, occurrence time, event importance, and the like are described.
[0218]
(B) Operation of video display device and video display example in the present embodiment
FIG. 26 shows the positional relationship between the positions of the cameras A, B and C and the position where the event E occurs. As shown in FIG. 26, it is understood that the event E occurred at time te and position (xe, ye). The following describes how images are displayed for cameras A, B, and C having such a positional relationship and events.
[0219]
The processing procedure of video display device 250 (FIG. 25) in the present embodiment is the same as the operation procedure of video display device 90 (FIG. 9) shown in FIG. However, only the calculation content of the video importance in “calculating video importance of each video based on subject information and video management information” (FIG. 11, step 110) is different. Hereinafter, the calculation content will be described.
[0220]
The video importance level calculation unit 15 (FIG. 25) calculates that the video closer to the event occurrence time has a higher video importance level if the video is shot by the same camera.
[0221]
When calculating the video importance level, the video importance level calculation unit 15 (FIG. 25) first calculates the time difference t between the video shooting time of the camera that shot the event occurrence position and the event occurrence time. The video shooting time of the camera is described in the video management information (FIG. 12), and the event occurrence time is described in the event information table. The video importance level calculation unit 15 (FIG. 25) uses a camera (camera A in FIG. 26) that was shooting the event occurrence position and a camera (camera B in FIG. 26) that was shooting the vicinity of the event occurrence position. The above-described time difference is calculated with respect to the video. The video for which the time difference is calculated and to be displayed may be a video taken in an appropriate time range, for example, 30 minutes before and after the time when the event occurred. This time range can be adjusted arbitrarily.
[0222]
When the time difference t is calculated, the time difference t is given to the event importance level function We (t), and the video importance level We for the event is calculated.
[0223]
FIG. 29 shows an example of the event importance function We (t). The event importance function is a function that decreases in proportion to the absolute value of the time difference t with the video importance at the event occurrence time as the maximum value (1.0). However, the maximum value of the video importance level is set to 0.5 for the video of the camera that has captured the vicinity of the event occurrence position. Therefore, the camera A is photographing the position where this event has occurred, and the maximum value is 1.0. Since the camera B is photographing the vicinity of the position where the event has occurred, the maximum value is 0.5. On the other hand, since the camera C does not photograph the event occurrence position and its vicinity, the video importance is set to 0 regardless of the time difference t.
[0224]
According to the above procedure, the video importance can be obtained as shown in FIG. That is, it is assumed that the event occurrence time (t = 0) is 13:20, camera A is photographing the event occurrence position, and camera B is photographing the vicinity of the event occurrence position. The shooting time is 13:10 and 13:22 for camera A, 13:15 and 13:25 for camera B, and 13:20 for camera B. The image A1 photographed by the camera A is t = −8 (minutes) and We = 0.3, and the image A2 is t = 2 (minutes) and We = 0.8. The image B1 photographed by the camera B is t = −5 (minutes) and We = 0.3, and the image B2 is t = 5 (minutes) and We = 0.3. An image C1 photographed by the camera C is t = 0 (minutes) and We = 0.0. As the final video importance described above, the value of the event importance function We is used as it is, but the product of the value of the event importance function We and the value of the subject importance Wo described in the subject information is used as the video importance. Also good.
[0225]
A mode in which video is displayed based on the video importance obtained as described above will be described.
[0226]
FIG. 27 shows an example of a plurality of videos displayed when an event occurs in the present embodiment. Along with two images of the camera A that captured the event occurrence position, two images of the camera B that captured the vicinity thereof are displayed.
[0227]
First, regarding the video of camera A that was shooting the event occurrence position, the video importance of the video closer to the event occurrence time (video shot at 13:22) is the highest (We = 0.8). ), The image is displayed in the largest size. The other image is displayed relatively small. As for the video of the camera B that was shooting the vicinity of the event occurrence position, the video importance level is the same for all the videos (We = 0.3), and the video importance level of the video shot by the camera A is Since they are the same, both are displayed in the same size. The video of the camera C that has not photographed the event occurrence position and its vicinity is not displayed because the video importance is 0.
[0228]
The display quality can be changed according to the video importance. However, since the description thereof has already been described in the fifth, sixth and seventh embodiments, the description thereof will be omitted.
[0229]
As described above, according to the present embodiment, video shooting time is described in the video management information (FIG. 12), event information relating to an event that occurred at a time in the subject information is described, and based on these, video important Since the degree calculation unit 15 (FIG. 25) calculates the importance of the video, the corresponding video is preferentially displayed according to the importance of the event such as sensor abnormality occurring in the subject as well as the importance of the subject itself. Can do. Thereby, an event can be observed quickly and accurately. Furthermore, the importance of the video can be observed with time in mind by displaying the vicinity of the event occurrence time in a more emphasized manner.
[0230]
The event importance function described above can be defined as a display rule. Therefore, the video importance can be determined using other event importance functions, not limited to the event importance function shown in FIG.
[0231]
FIG. 31 shows an example of the event importance function.
[0232]
31A to 31C show normal distribution functions. In FIGS. 31B and 31C, there are peaks before and after the time (t = 0) when the event occurs. Such an event importance function is effective when there is an important scene before or after the alarm occurrence time due to the sensitivity of the sensor or the like.
[0233]
FIG. 31 (d) shows a function with a sharp rise and small attenuation. This function is effective when the event occurrence time (t = 0) or later is important.
[0234]
FIG. 31E shows a step function. By using the step function, it is possible to simplify the video importance calculation process.
[0235]
FIG. 31F shows a function having a plurality of peaks. This function is effective when there is an important scene at a certain time before and after the event occurrence time (t = 0).
[0236]
Depending on the type of event and the purpose of observation, the functions shown in FIGS. 31A to 31F and many other functions not illustrated can be used.
[0237]
In the example described above, a pointer to the event information table is described in the subject management information (FIG. 28) to associate the subject with the event, but the present invention is not limited to this. That is, by referring to subject information (FIG. 5) describing the position of the subject and event information (FIG. 38) describing the position information of the event occurrence point, the position of the subject and the event occurrence position are determined. Is within a predetermined distance, it can be estimated that the event is an event that occurred in the subject. In this way, the subject and the event can be associated with each other.
[0238]
Furthermore, it is possible to obtain an image in which a subject and an event are appropriately captured using camera management information that is camera information at the time of shooting.
[0239]
FIG. 30 shows an example of camera management information. The camera management information describes the camera ID, the position of the camera representing the shooting range of the camera, the horizontal and vertical angles, and the zoom degree.
[0240]
In addition to the subject position information described in the subject information (FIG. 6) and the event occurrence position information described in the event information (FIG. 38), the camera position and angle information of the camera management information is used. Not only can the event be estimated to have occurred near the subject, but whether the camera was shooting the subject and event, and if the camera is shooting the subject and event, The proportion of events in the video can also be determined. Details of how the determination is made are described in this specification 2-2. Since it demonstrated in (c), it abbreviate | omits. By using camera management information, when there are multiple cameras that photographed the subject and event, the video of the camera that captures the event and subject most accurately can be used for display. . Therefore, video display can be performed more flexibly.
[0241]
9-1. In this embodiment, it has been described that the video mainly represents a still image, but the present invention is not limited to this. That is, the video may be a moving image. In that case, in the calculation of the time difference from the event occurrence time, any of the shooting start time, the shooting end time, and the time during the shooting time may be set as the above-described video shooting time. Which time is adopted as the video shooting time is not an essential problem of the present invention, and can be selected according to the purpose.
[0242]
(Embodiment 10)
10-1. Outline of operation of video display device in this embodiment
In the ninth embodiment, the case where still images with different shooting times are displayed in relation to the event occurrence time has been mainly described. In the present embodiment, a case will be described in which a plurality of moving images are displayed according to the shooting time in relation to the event occurrence time. The captured video gradually increases in display size when the shooting time approaches the event occurrence time (FIGS. 32A and 32B), and the display size gradually increases as the shooting time elapses from the event occurrence time. It is reduced (FIG. 32 (c)). At this time, each of the plurality of moving images is reproduced and displayed in synchronization based on the shooting time (video displayed in the vicinity of cameras A and B in FIG. 32).
[0243]
In the present embodiment, the term “synchronization” is used when each of a plurality of videos is displayed based on the time when the video was taken. The plurality of videos displayed in synchronism means a plurality of videos shot at the same time at the image frame level and reproduced / displayed based on the shooting time. Note that the “same imaging time” does not necessarily have to be strictly and accurately measured time. When the video is synchronously displayed, it may not be displayed at a time strictly corresponding to the shooting time due to a difference in frame rate of each video, performance of hardware to be displayed, or the like, and a slight deviation may occur.
[0244]
10-2. Operation of video display device and video display example in the present embodiment
The video display device in the present embodiment is the same as the video display device 250 (FIG. 25) described in the ninth embodiment. Therefore, description of the specific configuration of the video display device 250 (FIG. 25) is omitted.
[0245]
The operation of video display device 250 (FIG. 25) in the present embodiment will be described below. Regarding the situation in which the video display device 250 (FIG. 25) operates, the arrangement of the camera, its shooting range, the time of occurrence of the event, and the position of occurrence are as shown in FIG. The video information storage unit 93 (FIG. 25) stores moving images taken by the cameras A, B, and C (FIG. 26) as video information.
[0246]
FIG. 33 is a flowchart showing the operation procedure of the video display device 250 (FIG. 25).
[0247]
The display control unit 16 (FIG. 25) reads the map information stored in the map information storage unit 94 (FIG. 25) via the map information reading unit 95 (FIG. 25), and the map information based on the read map information. Is displayed on the display unit 17 (FIG. 25) (step 330).
[0248]
Subsequently, the display control unit 16 (FIG. 25) receives the video playback start time Ts and playback end time Te input by the operator or the like via the operation unit 96 (FIG. 25) (step 331). The reproduction start time Ts and the reproduction end time Te here are Ts = 13: 10 if, for example, the video is taken from the time 13:00 to the time 13:30, and the time when the reproduction is to be started is the time 13:10. Then, if the time at which playback is to be ended is time 13:25, Te = 13: 25 is set.
[0249]
The display control unit 16 (FIG. 25) sets the reproduction time t to t = Ts (step 332). Similar to the above, the playback time t represents the time when the video being played back was actually shot. Therefore, in the example described above regarding the reproduction start time Ts and the reproduction end time Te, the reproduction time t is a time included between the reproduction start time Ts (time 13:10) and the reproduction end time Te (time 13:25). Say. The “time when the video is actually shot” may include the shooting time of each image frame being played back.
[0250]
The video importance calculator 15 (FIG. 15) calculates the video importance of the video at time t (step 333). In the present embodiment, since the video to be displayed is a moving image, the video importance of the video at time t represents the video importance of the video representing the frame of the video at time t. The shooting time of each image frame constituting the moving image can be obtained from the shooting start time and end time described in the video management information (FIG. 12) and the frame rate. The calculation of the video importance is performed as described in the ninth embodiment. In other words, the video importance level of the image frame at time t can be obtained by calculating the time difference between the reproduction time and the event occurrence time, and using the event importance level function, the video importance value corresponding to the time difference can be obtained.
[0251]
FIG. 34 shows a correspondence relationship between a set of image frames arranged according to the time difference between the reproduction time and the event occurrence time and the event importance function. In this embodiment, since the video is a moving image, the video display is realized by continuous display of image frames. A plane obtained by cutting a rectangular parallelepiped along the time axis in FIG. 34 perpendicularly to the time axis represents an image frame at that time. By calculating the video importance level for each image frame (that is, for each shooting time of the image frame), it can be considered that the video importance level of the moving image is changed in real time. The event importance function is the same as the event importance function shown in the ninth embodiment. That is, the event importance function is a function that becomes smaller in proportion to the absolute value of the time difference t with the video importance at the event occurrence time as the maximum value (1.0). However, the maximum value of the video importance level is set to 0.5 for the video of the camera that has captured the vicinity of the event occurrence position. Therefore, the camera A is photographing the position where this event has occurred, and the maximum value is 1.0. Since the camera B is photographing the vicinity of the position where the event has occurred, the maximum value is 0.5. On the other hand, since the camera C does not photograph the event occurrence position and its vicinity, the video importance is set to 0 regardless of the time difference.
[0252]
Referring to FIG. 33 again, the calculated video importance is stored in the video management information as a part of the video management information (step 334).
[0253]
The display control unit 16 (FIG. 25) reads the video information of the video at time t from the video information storage unit 93 (FIG. 9) via the video information reading unit 92 (FIG. 9) (step 335).
[0254]
The display control unit 16 (FIG. 25) determines the display size based on the display rule and the video importance at the time t, and displays the video at the time t on the display unit 17 (FIG. 25) (step 336).
[0255]
Subsequently, the display control unit 16 determines whether or not the current playback time t has reached the playback end time Te (that is, whether t <Te) (step 336).
[0256]
If the current playback time t has not reached the playback end time Te, the playback time t is advanced by a predetermined time Δt, that is, t is set as t + Δt (step 338). Thereafter, the reproduction proceeds by repeating the processing from step 333 onward.
[0257]
When the current playback time t reaches the playback end time Te (that is, when t ≧ Te), whether or not the playback time of the moving image is subsequently changed, that is, the playback start time Ts or the playback end time Te. It is determined whether or not has been changed (step 339).
[0258]
When the playback time of the moving image is changed, the display control unit 16 (FIG. 25) receives the changed playback time and repeats the processing from step 331 again.
[0259]
If the playback time of the moving image has not been changed, the determination in step 339 is repeated until the operator inputs the next playback time.
[0260]
The processing procedure of the video display apparatus 250 (FIG. 25) in the present embodiment has been described above. Note that the processing described above is performed so that each moving image is reproduced and displayed in synchronization based on the reproduction time. As long as such synchronized playback / display is performed, even if the processing of one step is completed for all the image frames of the plurality of moving images and the process proceeds to the next step in the above-described processing, The processing for each image frame of the moving image may be performed independently and in parallel.
[0261]
FIG. 32 shows an example of a plurality of videos displayed in response to the occurrence of an event in the present embodiment. A moving image of the camera A that captured the vicinity of the moving image of the camera A that captured the event occurrence position is simultaneously displayed based on the respective shooting times. The video taken by the camera C is not displayed because the video importance is 0 as described above.
[0262]
FIG. 32A shows a state in which an image captured at 13:10 is displayed for an event that occurred at 13:20. Here, the video is a moving image. The video importance of each image frame of cameras A and B is relatively small because there is a period until an event occurs. Therefore, the size of the displayed image is relatively small.
[0263]
FIG. 32B shows a state in which a video image taken at the event occurrence time (13:20) is displayed. The video of camera A that has taken the event occurrence position has a larger display size than the video shown in FIG. The display size is smoothly enlarged as the video importance of the image frame increases. That is, the display size of the video of camera A increases as the event occurrence time approaches. The same applies to the video of the camera B that has captured the vicinity of the event occurrence position. Note that the video importance of the video of the camera A at the same time is larger than the video importance of the video of the camera B, so that the video is displayed in a larger size.
[0264]
FIG. 32C shows a state in which an image captured at 13:25 is displayed for an event that occurred at 13:20. As time elapses from the event occurrence time, the video importance of the image frame decreases, and the display size decreases accordingly.
[0265]
In the above example described with reference to FIGS. 32A to 32C, the displayed video is reproduced and displayed in synchronization based on the time actually captured by the respective cameras A and B. Yes.
[0266]
As described above, according to the present embodiment, the video importance of an image frame forming a moving image is calculated at each shooting time, and the display control unit 16 (FIG. 25) determines the display size according to the importance, and the shooting is performed. Since playback and display are performed synchronously based on the time, it is possible to easily display the surrounding situation of events that cannot be taken with one camera. Furthermore, for example, even when the camera moves or when multiple events occur at different positions at different times, the video can be displayed based on the shooting time, and the overall situation can be displayed from a wide viewpoint. It can be easily grasped in time and space. A case where a plurality of events occur at relatively close positions will be briefly described below.
[0267]
FIG. 35 shows the positional relationship between two events that occurred at different times and the camera that captures the event.
[0268]
The two events are events E and F. It is assumed that event F occurs at time te and position (xe, ye), and event F occurs at time tf and position (xf, yf) (te <tf). Further, the camera A captures the position where the event E occurs and also captures the vicinity of the position where the event F occurs. Assume that the camera B is shooting the position where the event F occurs and also shooting the vicinity of the position where the event E occurs.
[0269]
FIG. 36 shows an example of an event importance function used when events E and F occur.
[0270]
FIG. 36A shows an event importance function for calculating the video importance for the video of camera A. FIG. In FIG. 36, the event importance functions used for the camera A when the events E and F occur independently are displayed in an overlapping manner. A different event importance function is used for the time when the position where the event E occurs is imaged and the time when the vicinity of the position where the event F occurs is imaged. This is as described above. The video importance of the camera A is first calculated for each event E and F, and then the final video importance is determined. The final video importance may be determined in any way, such as adopting the sum of video importance for the videos of event E and event F, or adopting a video importance having a larger value. The same applies to the video shot by the camera B.
[0271]
In the above description, the video is a moving image, but may include a still image. However, since it is difficult to see a single still image displayed at the shooting time for a moment, it is preferable to reproduce it for a certain time before and after the shooting time. At this time, in order to distinguish between a moving image and a still image, the color of the display frame may be changed and specified. This can be set by changing the display rule. Similar display processing is also effective when displaying the first or last image frame of a moving image. For example, when the moving image of camera A has an end time earlier than the moving image of camera B, after the last image frame of camera A is displayed, the color of the display frame is changed to notify the end of reproduction. As a result, the operator can reset the reproduction start or end time without delay.
[0272]
(Embodiment 11)
11-1. Outline of operation of video display device in this embodiment
FIG. 37 is a block diagram showing a configuration of video display device 370 in the present embodiment.
[0273]
In the above-described ninth embodiment, when an event occurs, the video importance is calculated using the subject importance in addition to the event importance indicating the importance of the event, the event occurrence time, and the like, and the video is displayed. Was. The video display apparatus 250 in the present embodiment calculates the video importance based on the event importance, the event occurrence time, and the like, and does not use the subject importance. The video display device 250 displays videos taken before and after an event based on the calculated video importance.
[0274]
11-2. Configuration and operation of video display device in this embodiment
(A) Configuration of video display device in the present embodiment
The video display device 370 includes a plurality of cameras 11a to 11n, a video information acquisition unit 12, a video importance calculation unit 15, a display control unit 16, a display unit 17, a video information storage unit 93, and map information storage. A unit 94, a map information reading unit 95, an operation unit 96, and an event occurrence detection unit 251 are included. The video display device 250 further includes an event information management unit 371 and an event information storage unit 372.
[0275]
Hereinafter, among the constituent elements of the video display device 370, the constituent elements denoted by the same reference numerals as the constituent elements of the video display device 90 (FIG. 9) in the second embodiment are referred to as the video display device 90 (FIG. 9). Since it is the same as the configuration and function of the constituent elements, the description thereof is omitted. Therefore, hereinafter, the event information management unit 371 and the event information storage unit 372 will be described. In the case where an operation different from that of the video display device 90 (FIG. 9) or an additional operation is performed with respect to other components, the description will be made each time. Note that the subject information management unit 13 (FIG. 9) and the subject information storage unit 14 (FIG. 9) of the video display device 90 (FIG. 9) do not exist in the video display device 370 in the present embodiment. This is because the subject information is not used.
[0276]
The event information management unit 371 manages event information. The event information is information including an event occurrence time, an event type, an event importance level, and a correspondence relationship with a video. When an event occurrence is detected by the event occurrence detection unit 251 and the event information is transmitted, the event information management unit 371 sets or changes the event information. The event information management unit 371 stores the set or changed event information in the event information storage unit 372.
[0277]
The event information storage unit 372 stores event information related to the event that has occurred. The event information storage unit 372 receives and stores the set event information from the event information management unit 371. The event information storage unit 372 provides event information to the video importance level calculation unit 15. The event information storage unit 372 may be a known data storage unit such as a semiconductor memory represented by RAM or ROM, a storage device using a magnetic storage medium such as a hard disk, an optical disk, or the like.
[0278]
The video importance level calculation unit 15 determines the video importance level based on event information and display rules.
[0279]
FIG. 38 shows an example of event information in the present embodiment.
[0280]
The event information includes information such as an event ID, a type, an occurrence time and an occurrence position, a camera ID that captures the event, and an event importance level.
[0281]
The video importance calculation unit 15 (FIG. 37) calculates the video importance using these pieces of information of the event information. Since the calculation of the video importance is the same as the calculation of the video importance described in the ninth embodiment, the description thereof is omitted.
[0282]
When the video importance is calculated, the display control unit 16 (FIG. 37) controls the display of the video based on the video importance. When displaying an image, the position, time, quality, and the like can be controlled using the display rules described in the above embodiments. The display rules can be changed appropriately.
Furthermore, in either case where the video is a still image or a moving image, the same operation as that described in the above embodiments can be performed, but the description thereof is omitted.
[0283]
As described above, according to the present embodiment, the video importance calculation unit 15 (FIG. 37) calculates the video importance using the time relationship between the event occurrence time and the shooting time based on the event information and the display rule. Therefore, it becomes possible to observe the event based only on the information about the event including the state before and after the occurrence of the event. According to the present embodiment, since the subject information required in the ninth embodiment is not used, management of the subject information becomes unnecessary, and the video importance degree calculation process can be performed at high speed.
[0284]
The embodiments have been described above.
[0285]
The operation of the video display apparatus according to the present invention including the processing procedure described with reference to the flowchart can be realized as a computer program described so as to operate. Such a computer program can be recorded on an appropriate storage medium such as a magnetic storage medium such as an optical disk or a floppy disk, or a semiconductor storage medium such as a flash ROM. Here, the concept of “storage medium” includes a storage device such as a computer that provides content via a communication line such as the Internet. Therefore, a server or the like for storing a computer program for realizing the operation of the present invention and transmitting the program to another terminal is included in the category of a storage medium recording the computer program. For example, the subject information is recorded in the storage medium as a data structure in which at least the name of the subject, the position where the subject exists, and the importance of the subject are associated with each other, as shown in FIG. . Similarly, when event information is included in the subject information, the information related to the event described in the above embodiment is also recorded on the recording medium as a data structure associated with the information. Other display layout information shown in FIG. 8 (FIG. 8), video management information (FIG. 12), and camera management information (FIG. 30) can also be recorded in the same data structure.
[0286]
Note that both the position coordinates of the subject and the position coordinates of the camera are information that can specify a position that actually exists. As described with reference to FIG. 5 in the first embodiment, the position coordinates may be expressed by latitude and longitude, or may be expressed by its address or the like.
[0287]
【The invention's effect】
As described above, according to the present invention, the video importance of a video is calculated based on the subject information including the subject importance of each subject, and the video is displayed based on the video importance. Therefore, it is possible to display a video according to the importance of the subject included in the video. The operator can display the video according to the importance of the subject simply by determining the importance of the subject, so that the troublesome work of determining the display layout is not required and an objective video display is possible. Obtainable. Furthermore, by defining display rules, a flexible video display is realized in which an operator can observe an important group for a long time and can see an arrangement with a determined importance level on the screen.
[0288]
According to the present invention, it is possible to obtain a visual effect that an image obtained by photographing an important subject or the vicinity of the subject can be viewed on the map while grasping the photographing position. Furthermore, it is possible to cope with the addition or deletion of a subject or a change in the importance level of the subject simply by correcting the subject information, and a complicated display layout correction work can be omitted. Furthermore, since the captured video is temporarily stored and used, it is possible to repeatedly display and observe a plurality of videos with the same importance or changing the importance of the subject.
[0289]
According to the present invention, the subject information management unit 13 (FIG. 16) is included in the subject information in accordance with a request from the input unit 161 (FIG. 16) or the external module 162 (FIG. 16) even during video display. Change subject importance. The display control unit 16 (FIG. 16) automatically changes the display of a plurality of videos according to the changed video importance as a result of reflecting the change in the subject information. Therefore, even when the type or position of the subject to be noticed changes depending on the situation, a video with a high video importance is automatically selected and displayed, and the operator can observe the subject quickly and accurately.
[0290]
According to the present invention, since the display control unit 16 (FIGS. 1, 9, and 16) changes the degree of highlighting of the video based on the display rule and the video importance, the important video is displayed. The importance of the subject can be observed more consciously, for example, by displaying a larger size or making the frame color of the window displaying the important video conspicuous.
[0291]
According to the present invention, the display control unit controls the display of the video so that the display is performed by changing the degree of the video quality such as the frame rate and the resolution based on the display rule and the video importance. Therefore, since the video having a high video importance (that is, the important video) is displayed more easily and precisely, the operator can naturally watch the important video. Furthermore, since a video with relatively low video importance (that is, video with relatively low importance) is displayed with its image frames thinned out, the display processing load on the video display device is lightened, and a relatively inexpensive processing circuit. A video display device can also be manufactured by using. Therefore, the manufacturing cost of the video display device can be kept low.
[0292]
According to the present invention, the display control unit controls the display of the video so that the display is performed by changing the degree of the video quality such as the frame rate and the resolution based on the display rule and the video importance. Therefore, since the video having a high video importance (that is, the important video) is displayed more easily and precisely, the operator can naturally watch the important video. Furthermore, since video with relatively low video importance (that is, video with relatively low importance) is displayed with thinned image frames, the display processing load on the video display device can be lightened and relatively inexpensive. A video display device can be manufactured even using a simple processing circuit. Therefore, the manufacturing cost of the video display device can be kept low.
[0293]
Furthermore, even if the video server and video display terminal are connected via a network, only the video quality information necessary for display is transmitted over the network, so network resources and server and terminal computer resources can be used effectively. can do. By setting the video quality to the video so that it does not exceed the performance according to the transmission bandwidth and processing speed of those resources, the quality based on the maximum capability of the network-based video display device is obtained. Multiple images can be displayed.
[0294]
According to the present invention, even in the case of a moving image, it is possible to more clearly display a point to be noted in one image.
[0295]
According to the present invention, since the subject includes a partial space in the shooting target space such as a point or a region, the image is taken not only of an object such as a building but also a space specified by a town name or a position on a map. Can be preferentially displayed.
[0296]
According to the present invention, the shooting time of the video is described in the video management information, the event information related to the event that occurred at the time in the subject information is described, and the video importance calculation unit calculates the video importance based on these information Therefore, according to not only the importance of the subject itself but also the importance of an event such as a sensor abnormality occurring in the subject, the corresponding video can be preferentially displayed. Thereby, an event can be observed quickly and accurately. Furthermore, the importance of the video can be observed with time in mind by displaying the vicinity of the event occurrence time in a more emphasized manner.
[0297]
According to the present invention, the video importance of the image frame forming the moving image is calculated at each shooting time, and the display control unit determines the display size according to the importance and displays it synchronously. It is possible to display the surroundings of events that cannot be performed in an easy-to-see manner. In addition, for example, when the camera moves or when multiple events occur at different positions at different times, the entire situation is displayed from a wide viewpoint, making it easy to grasp in time and space. it can.
[0298]
According to the present invention, since the video importance calculation unit calculates the video importance using the time relationship between the event occurrence time and the shooting time based on the event information and the display rule, the event including the state before and after the occurrence of the event is calculated. It becomes possible to observe events based only on information about Further, by calculating the video importance level without using the subject information, management of the subject information becomes unnecessary, and the video importance level calculation process can be performed at high speed.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of a video display device 10 according to a first embodiment.
FIG. 2 is a diagram showing an example of an aspect of an image displayed by the processing operation of the image display device 10 (FIG. 1).
FIG. 3 is a diagram illustrating an example of video display positions in groups of the same importance.
FIG. 4 is a flowchart showing an operation procedure of the video display device 10 (FIG. 1).
FIG. 5 is a diagram illustrating an example of subject information.
FIG. 6 is a diagram illustrating an example of calculated video importance.
FIG. 7 is a flowchart showing an operation procedure of the video display device 10 (FIG. 1) including a step of determining a display layout.
FIG. 8 is a diagram illustrating an example of display layout information.
9 is a block diagram illustrating a configuration of a video display device 90 according to Embodiment 2. FIG.
FIG. 10 is a diagram showing a map and an image displayed in association with the map.
FIG. 11 is a flowchart illustrating an operation procedure of the video display device 90 (FIG. 9).
FIG. 12 is a diagram illustrating an example of video management information.
FIG. 13 is a diagram illustrating a positional relationship between a shooting position and direction of a camera, a zoom angle of view, and a subject.
FIG. 14 is a diagram in which a subject is imaged through a camera lens.
FIG. 15 is a diagram illustrating a relationship between a shooting range and a subject and a shooting state.
FIG. 16 is a block diagram showing a configuration of a video display device 160 in the third embodiment.
FIG. 17 is a diagram showing a flowchart showing an operation procedure of the video display device 160 (FIG. 16).
FIG. 18 is a diagram illustrating an example of a display screen in which a display size is increased for a video having a higher video importance level.
FIG. 19 is a diagram illustrating an example of a display rule for changing the enhancement level (display size) of a video in accordance with the video importance level.
FIG. 20 is a diagram illustrating an example of display rules for changing the display quality of video according to the order of video importance.
FIG. 21 is a block diagram showing a configuration of a video display apparatus 210 in the sixth embodiment.
FIG. 22 is a flowchart illustrating an operation procedure of the video display device 210 (FIG. 21).
FIG. 23 is a diagram illustrating an example of a camera that shoots three subjects and a subject that is displayed after being processed according to the present embodiment.
FIG. 24 is a diagram illustrating an example of subject information in which a space is defined as a subject.
25 is a block diagram illustrating a configuration of a video display apparatus 250 according to Embodiment 9. FIG.
FIG. 26 is a diagram showing the positional relationship between the positions of cameras A, B and C and the position where an event E occurs.
FIG. 27 is a diagram illustrating an example of a plurality of videos displayed when an event occurs in the ninth embodiment.
FIG. 28 is a diagram illustrating an example of subject information including event information.
FIG. 29 is a diagram illustrating an example of an event importance degree function We (t).
FIG. 30 is a diagram illustrating an example of camera management information.
FIG. 31 is a diagram illustrating an example of an event importance function;
32 is a diagram illustrating an example of a plurality of videos displayed in response to the occurrence of an event in Embodiment 10. FIG.
FIG. 33 is a flowchart showing an operation procedure of the video display device 250 (FIG. 25).
FIG. 34 is a diagram illustrating a correspondence relationship between a set of image frames arranged according to a time difference between a reproduction time and an event occurrence time and an event importance function.
FIG. 35 is a diagram illustrating a positional relationship between two events that occurred at different times and a camera that captures the events.
FIG. 36 is a diagram illustrating an example of an event importance function used when events E and F occur.
FIG. 37 is a block diagram illustrating a configuration of a video display apparatus 370 according to the eleventh embodiment.
38 is a diagram illustrating an example of event information in Embodiment 11. FIG.
FIG. 39 is a block diagram showing a configuration of a conventional video display device 390.
40 is a block diagram showing specific configurations of a video information acquisition unit 392 and a display control unit 395. FIG.
41 is a diagram showing an example of an operation screen for setting display layout information in a conventional video display device 390 (FIG. 39).
42 is a block diagram showing a configuration of another conventional video display device system 420. FIG.
FIG. 43 is a diagram illustrating an example of display priority information managed by the priority determining unit 424 (FIG. 42).
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 10 Video display apparatus, 12 Video information acquisition part, 13 Subject information management part, 14 Subject information storage part, 15 Video importance calculation part, 16 Display control part, 17 Display part, 92 Video information reading part, 93 Video information storage part , 161 input unit, 251 event occurrence detection unit, 371 event information management unit, 372 event information storage unit

Claims (22)

A subject information management unit for managing subject information including a correspondence relationship between a subject existing in a photographed space and a subject importance level indicating the importance of the subject;
Based on the object information managed by the object information management unit, and the video significance calculating unit for calculating a video importance representing the importance of movies image,
A video information acquisition unit that acquires video information of the space as video information;
A display control unit that controls display of the video represented by the video information acquired by the video information acquisition unit based on at least the video importance calculated by the video importance calculation unit;
A display unit configured to display the video according to the control of the display control unit. The video display device
A video information storage unit for storing the video information;
The video display device according to claim 1, wherein the video information acquisition unit acquires the video information by reading the video information from the video information storage unit. The video display device further includes an input unit for inputting a new subject importance level,
The subject information management unit manages by changing the subject importance level to the input new subject importance level in response to an input from the input unit,
The video importance calculation unit calculates the video importance based on the retained subject importance in response to the retention of the new subject importance by the subject information management unit. Video display device.   The video display device according to claim 1, wherein the subject is a partial space included in the space. The video information includes shooting time information related to the time when the video was shot,
The subject information includes event information related to an event that occurred in the subject and an event occurrence time that represents the time when the event occurred,
The video display device according to claim 1, wherein the video importance level calculation unit further calculates the video importance level based on the event information and the shooting time information. A video information storage unit for storing information on video images of the space and shooting time information on the time at which the video was shot;
A video information reading unit for reading the video information from the video information storage unit;
Events generated in the space, and the event information management unit for managing the occurrence time the event occurred, event information including a correspondence relationship between the event severity and the image representing the importance of the event,
A video importance calculation unit that calculates video importance representing video importance based on the event information managed by the event information management unit and the shooting time information stored in the video information storage unit;
A display control unit that controls display of a video represented by the video information based on at least the video importance level calculated by the video importance level calculation unit; and a display unit that displays the video according to control of the display control unit; A video display device comprising: The video is a plurality of videos obtained by shooting the space from a plurality of viewpoints,
The video importance calculator calculates the video importance for each of the plurality of videos;
The video display device according to claim 1, wherein the display control unit controls display of each of the plurality of videos based on at least the video importance calculated by the video importance calculation unit. The display control unit controls display of the video based on the video importance calculated by the video importance calculation unit and a display rule defining a rule for displaying the video. 6. The video display device according to 6 or 7.   The video display device according to claim 8, wherein the display rule defines a position, a time, an order, or a size at which the video is displayed.   The display control unit, based on the video importance calculated by the video importance calculation unit for each of the plurality of videos and a display rule applied according to the video importance, The video display device according to claim 8, wherein the display is controlled to highlight each of them.   The display control unit, based on the video importance calculated by the video importance calculation unit for each of the plurality of videos and a display rule applied according to the video importance, The video display device according to claim 8, wherein the quality of each of the plurality of videos is determined, and display of each of the plurality of videos is controlled with the determined quality.   The video display device according to claim 8, wherein the display rule is changeable. The video is composed of image frames, and the image frames include two or more areas.
The video importance calculation unit calculates the video importance for each of the two or more areas,
The display control unit controls the display of the image frame by determining a mode of displaying each of the two or more regions based on the video importance calculated by the video importance calculation unit. The video display device according to 1, 2, or 6. The video includes a first video shot at a first time and a second video shot at a second time, and the first video and the second video are from different viewpoints. This is a video of the space,
The video importance calculator calculates a first video importance that is a video importance of the first video and a second video importance that is a video importance of the second video;
The display control unit displays the video based on a display rule that defines a rule for displaying the video and the first video importance and the second video importance calculated by the video importance calculation unit. The video display apparatus according to claim 5 or 6, wherein the display is controlled so that the aspect is determined and the first video and the second video are synchronized based on the time of shooting.   The display control unit controls display of each of the plurality of videos based on a position where each of the plurality of videos is captured and the video importance calculated by the video importance calculation unit. The video display device described in 1.   The video display device according to claim 1, wherein the video is a still image.   The video display device according to claim 1, wherein the video is a moving image.   The video display device according to claim 14, wherein the first time and the second time are the same time. Managing subject information including a correspondence relationship between a subject existing in a photographed space and a subject importance level indicating the importance of the subject;
On the basis of the subject information, calculating a video importance representing the importance of movies image,
Obtaining information of a video image of the space as video information;
Controlling the display of the video represented by the video information based on at least the video importance;
Displaying the video according to the display control of the video. Storing the information of the video imaged of the space and the shooting time information related to the time of shooting the video image;
Reading the stored video information;
Managing event information including a correspondence relationship between an event that has occurred in the space, an occurrence time at which the event has occurred, an event importance level indicating the importance of the event, and the video;
Calculating a video importance level representing the importance of the video based on the managed event information and the stored shooting time information;
A video display method, comprising: controlling display of a video represented by the video information based on at least the calculated video importance level; and displaying the video according to the control. Managing subject information including a correspondence relationship between a subject existing in a photographed space and a subject importance level indicating the importance of the subject;
On the basis of the subject information, calculating a video importance representing the importance of movies image,
Obtaining information of a video image of the space as video information;
Controlling the display of the video represented by the video information based on at least the video importance;
Displaying the video according to control of display of the video;
A computer-readable recording medium storing a program for executing a computer. Storing the information of the video imaged of the space and the shooting time information related to the time of shooting the video image;
Reading the stored video information;
Managing event information including a correspondence relationship between an event that has occurred in the space, an occurrence time at which the event has occurred, an event importance level indicating the importance of the event, and the video;
Calculating a video importance level representing the importance of the video based on the managed event information and the stored shooting time information;
Controlling the display of the video represented by the video information based on at least the calculated video importance, and displaying the video according to the control;
A computer-readable recording medium storing a program for executing a computer.

Images