JP2004015516A

JP2004015516A - Automatic tracking photographing method and automatic tracking photographing device for motion picture

Info

Publication number: JP2004015516A
Application number: JP2002167309A
Authority: JP
Inventors: Fumitoki Amano; 天野　史斎; Toshihisa Ishii; 石井　聡寿; Yoshinori Yubikawa; 指川　佳則; Mizuo Tsukidate; 月舘　瑞男
Original assignee: Chuo Electronics Co Ltd
Current assignee: Chuo Electronics Co Ltd
Priority date: 2002-06-07
Filing date: 2002-06-07
Publication date: 2004-01-15
Anticipated expiration: 2022-06-07
Also published as: JP3963261B2

Abstract

<P>PROBLEM TO BE SOLVED: To automatically execute a method which monitors efficiently on a large field of view with a few cameras and photographs an intruder with a clear video image so as to identify the intruder easily. <P>SOLUTION: A pan-tilt camera 1 and a wide angle camera 2 are installed in proximity. A motion picture photographed with the wide angle camera is converted to a digital motion picture data by A/D conversion unit 3. An aggregation S1 of zones is outputted for an inputted motion picture data in a detection unit 4. The aggregation S1 of zones is inputted to estimate a target current position on the picture. The two-dimensional co-ordinates of the estimated position are outputted while a signal is received from a target tracking unit 5, which outputs a target detected signal 10 and the position and the size of a circumscribed rectangle. The pan-tilt camera 1 is circled so that a target may be caught in field of view. A pan-tilt control signal for setting a magnification factor from a camera control unit 6 is inputtted into the pan-tilt camera 1. Video images outputted from the pan-tilt camera 1 are displayed on a video image recording device 7. <P>COPYRIGHT: (C)2004,JPO

Description

【０００１】
【発明の属する技術分野】
本発明は、セキュリティ・システムを構成する監視用カメラを利用して、動く物体を自動的に視野内に捉え続けて撮影する自動追尾撮影方法およびその装置に関する。
【０００２】
【従来の技術】
監視用カメラによって監視対象物を撮影して、監視領域への侵入者を監視する方法としては、次のような方法が利用されていた。
▲１▼監視する領域全体を広角カメラで撮影する方法。
▲２▼監視する領域全体を通常画角のカメラ複数台を使用して、監視領域を分割して分担撮影する方法。
▲３▼パン・チルト雲台つきのカメラを予めプログラムした向きに、順番に向けさせて撮影する方法。
▲４▼パン・チルト雲台つきのカメラを人手で操作して撮影する方法。
【０００３】
【発明が解決しようとする課題】
上述した従来技術による撮影方法では、解決すべき多くの課題があった。
▲１▼広角カメラで撮影すると、カメラの台数は少なくて済むが、撮影した映像の鮮明さが損われる。
例えば、侵入者が小さく写るので特定しにくい。
▲２▼監視領域を分割して分担撮影する方法では、多数のカメラを必要とする。
▲３▼プログラムを内蔵したパン・チルト雲台つきのカメラで撮影する方法では、侵入者を効果的に撮影し難い。
即ち、侵入者の顔など、重要な映像を撮り逃すおそれがある。
▲４▼パン・チルト雲台つきカメラを人手で操作するのでは、常に所要の人員を配置しておく必要が生ずる。
【０００４】
【課題を解決するための手段】
本発明は上述した従来技術による欠点を解消しようとするものであって、
視野の方向を変更可能な撮影手段１で捉えるべきターゲットを、より広画角の撮影手段２の画像上で指定する動画像撮影手段と、
動画像上に映っている動く物体の位置と形状を、該物体が動き続ける条件下で推定し続ける追尾アルゴリズムと、
ターゲット以外の動く物体が多数存在するか、もしくはターゲットを遮る障害物が存在する領域でのターゲット誤推定を自動修正する手段と、
によって構成し、動く物体を自動的に撮影手段１の視野内に捉え続けて追尾撮影する方法である。
【０００５】
さらに、動く物体の中央（重心）より、むしろ上端側を撮影手段１の視野内に捉え続ける（追尾する）撮影方法と、前記撮影手段１の映像を傾かせずに、動く物体を視野内に捉え続ける（追尾する）撮影方法と、
前記撮影手段１の視野の変更に制約を設けて機械系の摩耗を低減させ、かつ安定した撮影（前記撮影手段１の視野内の変更が間欠的にしか発生しない）を実現しつつ、動く物体を視野内に捉え続ける（追尾する）撮影方法と、
前記撮影手段１でズーム撮影するときも、機械系の摩耗を低減し、かつ安定した撮影を実現することと、動く物体を視野内に捉え続けることを同時に満足する撮影方法
とによって自動追尾撮影方法を構成した。
【０００６】
【発明の実施の形態】
以下、本発明の実施例を図面を参照しながら説明する。
図１は、本発明に係る動画像の自動追尾撮影装置の構成を示すブロック図である。
【０００７】
パンチルトカメラ１は、パン機能とチルト機能を備えたカメラであって、侵入者（ターゲット）を鮮明に撮影することができる。
パン回転面が水平になるように設置し、パン回転中心とチルト回転中心が一致するように設置するのがよい。
【０００８】
広角カメラ２は、通常の（遠近投影法に従う）レンズか、魚眼レンズ（魚眼投影法に従う）を備えた動画像撮影手段として使用されるカメラであって、監視対象領域の広い範囲（視野角にして９０°等、少なくともパンチルトカメラ１単体の視野よりは広い）を視野内に収めることができる。
なお、パンチルトカメラ１と広角カメラ２とは近接して設置しておき、視野中心線（レンズの光軸）は斜め下４５°程度に向けておく。
【０００９】
Ａ／Ｄ変換部３は、広角カメラ２で撮影した動画像（アナログ映像信号）をディジタル動画像データに変換する回路である。
【００１０】
動き検出部４は、Ａ／Ｄ変換部３が出力したディジタル画像データを受け取り、動きを検出した区画の集合Ｓ１を出力するものであって、その内部構成は図２に示す通りである。
フレームバッファ１１はディジタル動画像データに１フレーム時間の遅延を与えるＦＩＦＯメモリであって、１フレーム分のディジタル動画像データを一時的に記憶する。
減算器１２は、入力されたディジタル動画像データのＹ成分から、フレームバッファ１１に記憶されている１フレーム時間前の動画像データのＹ成分を画素ごとに引き算し、その結果（濃度差分）を出力する回路である。
２値化回路Ａ１３は、濃度差分の値が閾値Ｔ_１より大きい画素を１、それ以外の画素を０として出力する回路である。
集計回路１４は、２値化回路Ａ１３により値１となった画素の個数を区画ごとに集計する回路である。
２値化回路Ｂ１５は、集計回路１４による集計値が閾値Ｔ_２より大きい区画を１、そうでない区画を０に変換し出力する。
【００１１】
次に、動き検出部４の処理手順を説明する。
▲１▼現在の１フレームと直前（過去）の１フレームのＹ成分の差分を画素ごとに、フレームバッファ１１と減算器１２を使用して取る。
▲２▼２値化回路Ａ１３が、上記差分が定数Ｔ_１より大きい画素を１、それ以外の画素を０に変換する。
▲３▼集計回路１４が値１となった画素の個数を区画ごとに集計する。例えば、区画は長方形とし、横３２０ピクセル、縦２４０ラインの画像を横３２列、縦１５行に区切る。
▲４▼２値化回路Ｂ１５が上記集計値が定数Ｔ_２より大きい区画を１、そうでない区画を０に変換して集合Ｓ１を出力する。
即ち、本実施例では、「区画の集合」を表現するデータ形式として、１フレーム分の画像を構成する区画の中で、該集合に含まれる区画を１、そうでない区画を０に置き換えたデータ形式を採用しており、図３に示す通りである。
【００１２】
ターゲット追尾部５は、動き検出部４が出力した区画の集合Ｓ１を受け取り、画像上でのターゲット現在位置を推定し、その二次元座標を出力する。
但し、ターゲットが存在しない（広角カメラ２の視野内に動いている物体がない）場合もあるため、ターゲットの検知／未検知も別途ターゲット検出信号１０として出力し、後段に与える。
また、パンチルトカメラ１の拡大率制御のため、ターゲットの画像上での大きさも出力する。
【００１３】
ターゲット追尾部５の内部構成は図４に示す通りである。
ＣＰＵ２１は、ターゲット追尾部５を構成する各ブロックの各種処理を実施する中央制御装置であり、記憶手段１・２２は、ターゲットの現在位置・形状・大きさを、区画の集合として保持するメモリである。
連結領域列挙手段２３は、集合Ｓ１に含まれる連結領域のうち、与えられた集合Ｓ２と共通部分をもつものの集合Ｗを求める演算手段である。
また、連結領域選択手段２４は、連結領域列挙手段２３が出力した集合Ｗから、下記関数ｄ（Ｈ）の値が最小となる集合Ｈを選択して出力する手段である。
ｄ（Ｈ）＝（集合Ｈと集合Ｓ１の共通部分に含まれる区画数）÷（集合Ｈに含まれる区画数）
マスク領域メモリ２５は、ターゲット以外の動きがある領域または障害物（ターゲットとカメラの間を遮る）を被う区画の集合Ｍを記憶するメモリであり、内容は装置起動時に書き込み、追尾実行中は基本的に書き換えない。
包含判定手段２６は、ターゲットの位置と形状の推定結果ＨがＭに含まれるか判定するブロックであり、連結領域出力手段１・２７は、ＨがＭに含まれるとき、Ｈと共通部分をもつＭの連結領域Ｍ_Ｃ（Ｈ）を出力するブロックである。
また、動き検出手段２・２８は、ＨがＭに含まれるとき、Ｍ_Ｃ（Ｈ）周縁上で動きを検知したとき、その位置（区画）Ｂを出力する手段であり、連結領域出力手段２・２９は動き検出手段２・２８が出力した区画Ｂを含む連結領域をＳ１から求め、出力する手段である。
セレクタ３０は、包含判定手段２６の結果を受け、記憶手段１・２２に記憶する内容を選択する（Ｈ_２として出力）。
外接長方形算出３１は、画像上でターゲットに外接する長方形の位置と大きさを求める。
演算手段２・３２は、集合Ｈに含まれる区画の座標の相加平均（重心）Ｇ２を求め、ターゲットの代表位置として出力する。
【００１４】
ターゲット追尾部５は、「ターゲット未検知」、「ターゲット追尾中」、「待機中」の３つの状態をもっている。
「ターゲット未検知」状態は、広角カメラ２の視野内にターゲットが現われるのを待っている状態を示し、「ターゲット追尾中」状態は、広角カメラ２の視野内をターゲットが動いており、それを追尾している状態を示し、「待機中」状態は、広角カメラ２の視野内でターゲットが停止した状態を示している。
【００１５】
次に、ターゲット追尾部５における「ターゲット未検知」状態の処理は次の通りである。
▲１▼画面上に予め設定した領域（センスエリア）で動きを検知するのを待つ。
即ち、動き検出部４が出力した集合Ｓ１に、図５に示すセンスエリアを構成する区画の集合との間に共通部分が発生するのを待つ。
▲２▼センスエリアで動きを検知したときは、
ａ．その動いた部分を含む連結領域を集合Ｓ１から切り出し、記憶手段１・２２に初期値として与える。
ｂ．「ターゲット追尾中」状態に遷移する。
【００１６】
「ターゲット追尾中」状態の処理は次の通りである。
▲１▼図４に示すように、記憶手段１・２２の内容（区画の集合Ｓ２）を更新する。
▲２▼下記条件が成立したら、「待機中」状態に遷移する。
ａ．更新前Ｓ２⊂更新後Ｓ２
【００１７】
「待機中」状態の処理は次の通りである。
▲１▼画面内の予め設定した領域（センスエリア）で動きを検知したかを判定する。
即ち、動き検出部４が出力した集合Ｓ１に、センスエリアを構成する区画が含まれるか判定する。
動きを検知したときは、「ターゲット未検知」における▲２▼ａ，ｂと同じ処理を行う。
▲２▼上記条件が成立、かつ下記条件のいずれかが成立したときは、「ターゲット追尾中」状態に遷移する。
ａ．Ｈ_２　⊂集合Ｓ２でない（Ｈ_２がＳ２からはみ出している）
▲３▼上記条件が成立しないままＴ_Ｃ秒経過したときは、「ターゲット未検知」状態に遷移する。
【００１８】
ターゲット検出信号１０は、「ターゲット未検知」状態で０、それ以外の状態において１を出力する。
なお、「待機中」状態を設ける理由は次の通りである。
▲１▼ターゲットが移動を停止すると、動き検出部４が殆ど動きを検出しないため、集合Ｓ１に含まれる区画がターゲットが動いている最中に比べて少なくなる。
▲２▼その状態で記憶手段２に記憶されているＳ２の更新を続けると、Ｓ２が空集合になってしまい、以降の追尾に差し支える。
▲３▼従って、ターゲットが停止してから再び動き出すまでの間は、記憶手段２の更新を停止する必要がある。
【００１９】
カメラ制御部６は、ターゲット追尾部５が出力したターゲットの座標Ｇ、ターゲット検出信号１０および外接長方形の位置と大きさを受け取り、ターゲットを視野内に捉えるようにパンチルトカメラ１を旋回させ、かつ拡大率を設定するものであって、次のステップから成る。
▲１▼座標Ｇより、広角カメラ２から見てターゲットがどの方向に存在するのかを推定する。（ターゲット見込み方向推定）
▲２▼推定された方向より、パンチルトカメラ１に与えるパンおよびチルトの角度を算出する。（カメラの旋回角度算出）
▲３▼算出したパンおよびチルトの角度をパンチルトカメラ１に与え、旋回させる。（旋回指令）
▲４▼外接長方形の横サイズが、パンチルトカメラ１で撮影される画像の横サイズの７０％と一致するように、パンチルトカメラ１にズーム指令を与え拡大させる。（ズーム指令）
但し、ターゲット検出信号１０が０（ターゲット未検知）のときは、上記処理は行わず、待機する。
【００２０】
カメラ制御部６には、広角カメラ２で撮影した画像上でのターゲット位置Ｇが入力されており、これをもとに、広角カメラ２の焦点からターゲットに向かうベクトル成分を求める。
ターゲット見込み方向推定には、広角カメラ２で使用するレンズとして、通常の広角レンズ（遠近投影法に従う）と、魚眼レンズ（魚眼投影法に従う）の２種類があり、以下に記載の原理に従って、カメラ焦点からターゲットに向かうベクトル成分を求める。
広角カメラ２のレンズが通常（遠近投影法に従う）である場合におけるターゲット見込み方向の推定を、図６によって説明する。
図６において広角カメラ２の焦点Ｏから、三次元空間中でのターゲット位置Ｐに向かうベクトルが、求めるべきターゲット見込み方向を与える。
ここで、Ｘ軸は水平方向、Ｚ軸は視野中心線（レンズ光軸）に一致するようにとっている。
ベクトル成分の導出を簡単にするために、ＯＰに平行なベクトルＯＧを求める。ここで、Ｇは直線ＯＰと投影面の交点である。
図６において、ＯＧのＸ軸成分をｘ、Ｙ軸成分をｙ，Ｚ軸成分をｚとすると、次式の通りである。
ｘ＝μ
ｙ＝ν
ｚ＝ｄ＝Ｈ_Ｗ／ｔａｎ（Ｈ_Ａ）……（１）
ここで、
μ：座標ＧのＸ成分
ν：座標ＧのＹ成分
Ｈ_Ｗ：レンズの遠近投影法投影面の全幅の１／２
Ｈ_Ａ：レンズの水平視野角の１／２
ｄ：レンズ焦点から投影面までの距離
ｘ：ベクトルＯＧのＸ成分
ｙ：ベクトルＯＧのＹ成分
ｚ：ベクトルＯＧのＺ成分
【００２１】
広角カメラ２のレンズが魚眼レンズ（魚眼投影法に従う）である場合におけるターゲット見込み方向の推定方法を、図７によって説明する。
図７において、広角カメラ２の焦点Ｏから三次元空間中でのターゲット位置Ｐに向かうベクトルが、求めるべきターゲット見込み方向を与える。
ここで、Ｘ軸は水平方向、Ｚ軸は視野中心線（レンズ光軸）に一致するようにとっている。
ベクトル成分の導出を簡単にするために、ＯＰに平行なベクトルＯＧ´を求める。Ｇ´は平面Ｚ＝γ上の点であり、γは魚眼投影面半径を示す。
図７より、　　Ｄ_２＝γｓ
Ｒ_０＝γＨ_Ａ
従って、
Ｄ_１＝γｔａｎｓ＝Ｒ_０／Ｈ_Ａ・ｔａｎ〔（Ｈ_Ａ／Ｒ_０）・Ｄ_２〕…（２）
ここで、
Ｇ_０：平面ｚ＝γ　と　Ｚ軸（レンズ光軸）との交点
γ　：魚眼投影法投影半径
Ｄ_１：Ｇ_０からＧ´までの距離
Ｄ_２：Ｇ_０からＧまでの円弧長
Ｈ_Ａ：レンズ視野角÷２（一般的な魚眼レンズでは９０°）
Ｓ　：ターゲット見込み方向とＺ軸のなす角
Ｒ_０：画像上での視野角Ｈ_Ａ円の半径
よって、ＯＧ´のＸ成分ｘ、Ｙ成分ｙ、Ｚ成分ｚは次式で与えられる。
【数１】

【００２２】
上述したターゲット見込み方向推定処理において、ターゲット見込み方向ベクトル成分（ｘ，ｙ，ｚ）が得られたので、このベクトルとパンチルトカメラ１の視野中心線（レンズ光軸）が平行になるようなパンおよびチルト角度を図８と図９に従い算出する。
パン角度は、ベクトル（ｘ，ｙ，ｚ）をパン回転面（三次元平面）に射影したベクトルＶ_１を求め、パン回転面上にＸ´軸とＺ´軸からなる正規直交座標系をとり、Ｖ_１のＸ´成分ｘ´と、Ｚ´成分ｚ´を求めると、パン角度Ｐ_Ａは次式によって求められる。
Ｐ_Ａ＝ｔａｎ^−１（ｘ´／ｚ´）
【００２３】
次に、チルト角度は、パン回転面の法線方向にＹ´軸をとり、ベクトル（ｘ，ｙ，ｚ）をベクトルＶ_１とＹ´軸が張る三次元平面に投射したベクトルＶ_２を求め、ベクトルＶ_２のＹ´成分ｙ´とＶ_１成分ｖ´を求めると、次式によってチルト角度Ｐ_Ｔが得られる。
Ｐ_Ｔ＝ｔａｎ^−１（ｙ´／ｖ´）
実施例では、広角カメラ２のレンズ光軸（Ｚ軸）が４５°斜め下を向いているのに対し、パンチルトカメラ１のパン回転面が水平である。
広角カメラ２のレンズ光軸とパンチルトカメラ１のパン回転面とのなす角をθとすると、Ｘ−Ｙ−Ｚ座標系とＸ´−Ｙ´−Ｚ´座標系は図１０に示すようになる。
【００２４】
前記原理により求めたパン角度Ｐ_Ａ、チルト角度Ｐ_Ｔを与えると、パンチルトカメラ１は三次元空間中でのターゲット位置Ｐから広角カメラ２の焦点Ｏに入射光が入射してきた側を向き、かつ該パンチルトカメラ１の視野中心線と直線ＯＰ（入射光の入射方向）は平行となるように向く。さらに、パンチルトカメラ１と広角カメラ２が近接して設置されているため、該視野中心線と直線ＯＰは互いに近接している。すなわちパンチルトカメラ１の視野中心線は三次元空間中でのターゲット位置Ｐに近接しており、よってＰはパンチルトカメラ１の撮影像の中心付近に撮影されることになる。
【００２５】
上記原理から分かるように、パンチルトカメラ１と広角カメラ２を近接して設置すればするほど、三次元空間中でのターゲット位置Ｐが正確にパンチルトカメラ１の視野中心に撮影される。
したがって、パンチルトカメラ１の映像上でＰをどれだけ正確に視野中心に捉えたいかによってパンチルトカメラ１と広角カメラ２の距離（どれだけ近接させねばならないか）を決めればよい。
【００２６】
実施例では、パンチルトカメラ１としてパン回転中心とチルト回転中心が一致するものを使用しており、パンチルトカメラ１と広角カメラ２の距離の決め方は容易になっている。すなわち、三次元空間中でのターゲット位置Ｐに対しそれを狙うパンチルトカメラ１の視野中心線が距離ｄ以上それてはならないなら、二点Ｏ^・Ｏ間の距離がｄ未満になるようパンチルトカメラ１と広角カメラ２を設置せねばならない。
ここで、　　Ｏ　：広角カメラ２の焦点
Ｏ^・：パンチルトカメラ１のパンおよびチルトの回転中心
【００２７】
なお、旋回指令とズーム指令は次の通りである。
▲１▼外接長方形の横サイズがパンチルトカメラ１で撮影される画像の横サイズの７０％と一致するようなパンチルトカメラ１の拡大率を算出する。
▲２▼撮影手段１において、ｚ倍ズームで撮影するときの閾値Ｔ_５を算出し、動き検出部４から入力されたＧが、前回入力時のＧと比べ、距離Ｔ_５以上離れていたときは、パンチルトカメラ１にパン（絶対角度Ｐ_Ａ）、チルト（絶対角度Ｐ_Ｔ）、ズーム（拡大率ｚ）を指令する。
▲３▼現在のＧを記憶し、次のＧの入力を待つ。
【００２８】
ターゲット以外の動く物体が多数存在するか、もしくはターゲットを遮る障害物が存在する領域でのターゲットの位置および形状の誤推定を自動修正する自動修正手段としては、下記の要素を追加している。
▲１▼入力画像を構成する区画のうち、ターゲット以外の動く物体が多数存在するか、もしくはターゲットを遮る障害物が存在する領域を被う区画の集合Ｍを記憶する記憶手段２。
▲２▼追尾中のターゲットが入力画像上に占めると想定される領域を表す区画の集合Ｈが、Ｍに含まれるか否かを判定する包含判定手段２６。
▲３▼ＨがＭに含まれるとき、Ｈと共通部分をもつＭの連結領域Ｍ_Ｃ（Ｈ）を出力する連結領域出力手段１・２７。
▲４▼ＨがＭに含まれるとき、連結領域Ｍ_Ｃ（Ｈ）の周縁上の動きを監視し、動きを検出したときその位置（区画）Ｂを出力する動き検出手段２・２８。
▲５▼動き検出手段２・２８が動きを検知（Ｂを出力）したとき、入力画像を構成する区画からＢを含む連結領域Ｓ_１Ｃ（Ｂ）を出力する連結領域出力手段２・２９。
▲６▼動き検出手段２が動きを検知したとき、集合Ｈの内容を連結領域Ｓ_Ｃ１（Ｂ）の内容で置き換える修正手段セレクタ３０。
【００２９】
上記構成要素により、広角カメラ２の画像上でターゲットが映っていると推定される領域Ｈが領域Ｍ内にある（包含されている）間、Ｈをターゲットとして更新するのと並行して連結領域Ｍ_Ｃ（Ｈ）（Ｈを包含するＭの連結領域）の周縁での動きを監視し、該周縁上の区画Ｂで動きを検知したらＨの更新をやめ、Ｂをターゲットの一部であるとみなして追尾しなおすことにより、領域Ｍの中で永久にターゲット以外のものをターゲットとして誤推定しつづけることを抑止する。
【００３０】
【発明の効果】
以上説明したように、本発明による動画像の自動追尾撮影方法および自動追尾撮影装置は、視野の方向を変更可能な撮影手段１で捉えるべきターゲットをより広角度の撮影手段２の画像上で指定する動画像撮影手段と、同じく撮影手段２の画像上で動く物体の位置と形状を推定し続ける追尾アルゴリズムと、さらに、ターゲット以外の動く物体が多数存在するか、もしくはターゲットを遮る障害物が存在する領域でのターゲット誤推定を自動修正する手段とによって構成しているので、２つのカメラのみによって動く物体を視野内に捉え続けて自動的に追尾撮影することができる極めて高精度のセキュリティ・システムを構築することが可能である。
【図面の簡単な説明】
【図１】本発明による自動追尾撮影装置の構成を示すブロック図。
【図２】動き検出部の構成を示すブロック図。
【図３】区画の集合Ｓ１の例。
【図４】ターゲット追尾部の構成を示すブロック図。
【図５】センスエリアとマスクエリアの例。
【図６】広角カメラのレンズが通常の場合の説明図。
【図７】広角カメラのレンズが魚眼レンズの場合の説明図。
【図８】パン角度算出方法の説明図。
【図９】チルト角度算出方法の説明図。
【図１０】Ｘ−Ｙ−Ｚ座標系とＸ´−Ｙ´−Ｚ´座標系の関係を示す説明図。
【符号の説明】
１　パンチルトカメラ
２　広角カメラ
３　Ａ／Ｄ変換部
４　動き検出部
５　ターゲット追尾部
６　カメラ制御部
７　映像記録装置
１１　フレームバッファ
１２　減算器
１３　２値化回路Ａ
１４　集計回路
１５　２値化回路Ｂ
２０　動き検出手段１
２１　ＣＰＵ
２２　記憶手段１
２３　連結領域列挙手段
２４　連結領域選択手段
２５　マスク領域メモリ
２６　包含判定手段
２７　連結領域出力手段１
２８　動き検出手段２
２９　連結領域出力手段２
３０　セレクタ
３１　外接長方形算出
３２　演算手段２[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an automatic tracking photographing method and apparatus for automatically capturing and photographing a moving object in a field of view by using a surveillance camera constituting a security system.
[0002]
[Prior art]
The following method has been used as a method of monitoring an intruder in a monitoring area by photographing an object to be monitored by a monitoring camera.
(1) A method in which the entire area to be monitored is photographed with a wide-angle camera.
{Circle over (2)} A method in which the monitoring area is divided and the divided area is photographed by using a plurality of cameras having a normal angle of view for the entire area to be monitored.
(3) A method in which a camera with a pan / tilt head is turned in a pre-programmed direction and shot in order.
(4) A method of manually operating a camera with a pan / tilt head and taking a picture.
[0003]
[Problems to be solved by the invention]
The above-described conventional imaging method has many problems to be solved.
{Circle around (1)} When shooting with a wide-angle camera, the number of cameras can be reduced, but the sharpness of the shot image is impaired.
For example, it is difficult to identify an intruder because it is small.
{Circle around (2)} In the method of dividing the monitoring area and performing shared shooting, a large number of cameras are required.
(3) It is difficult to shoot an intruder effectively by using a camera with a pan / tilt head with a built-in program.
That is, there is a possibility that an important image such as the face of an intruder is missed.
(4) When a camera with a pan / tilt pan head is operated manually, it is necessary to always have required personnel.
[0004]
[Means for Solving the Problems]
The present invention seeks to overcome the disadvantages of the prior art described above,
Moving image photographing means for designating a target to be captured by the photographing means 1 capable of changing the direction of the visual field on the image of the photographing means 2 having a wider angle of view;
A tracking algorithm that keeps estimating the position and shape of the moving object reflected on the moving image under conditions where the object continues to move,
Means for automatically correcting an erroneous target estimation in an area where there are many moving objects other than the target or where there are obstacles blocking the target,
In this method, a moving object is automatically captured within the field of view of the image capturing means 1 to perform tracking image capturing.
[0005]
Further, a photographing method in which the upper end rather than the center (center of gravity) of the moving object is kept in the field of view of the photographing means 1 (tracking), and the moving object is placed in the field of view without tilting the image of the photographing means 1 A shooting method that keeps catching (tracking)
Moving object while reducing the wear of the mechanical system by restricting the change of the visual field of the photographing means 1 and realizing stable photographing (the change in the visual field of the photographing means 1 occurs only intermittently) A shooting method that keeps (tracks) in the field of view,
Also when performing zoom photographing with the photographing means 1, an automatic tracking photographing method is realized by a photographing method which simultaneously realizes stable photographing while reducing wear of the mechanical system and simultaneously keeps capturing a moving object in a field of view. Was configured.
[0006]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram showing a configuration of a moving image automatic tracking photographing apparatus according to the present invention.
[0007]
The pan-tilt camera 1 is a camera having a pan function and a tilt function, and can clearly capture an intruder (target).
It is preferable to install the pan rotation surface so that it is horizontal, and to set the pan rotation center and the tilt rotation center to coincide.
[0008]
The wide-angle camera 2 is a camera which is used as a moving image photographing means provided with a normal lens (according to the perspective projection method) or a fish-eye lens (according to the fish-eye projection method). At least 90 °, which is wider than the field of view of the pan-tilt camera 1 alone).
The pan-tilt camera 1 and the wide-angle camera 2 are installed close to each other, and the center line of the visual field (the optical axis of the lens) is directed obliquely downward at about 45 °.
[0009]
The A / D converter 3 is a circuit that converts a moving image (analog video signal) captured by the wide-angle camera 2 into digital moving image data.
[0010]
The motion detector 4 receives the digital image data output from the A / D converter 3 and outputs a set S1 of sections in which motion has been detected. Its internal configuration is as shown in FIG.
The frame buffer 11 is a FIFO memory for delaying the digital moving image data by one frame time, and temporarily stores one frame of digital moving image data.
The subtractor 12 subtracts, for each pixel, the Y component of the moving image data one frame time earlier stored in the frame buffer 11 from the Y component of the input digital moving image data, and calculates the result (density difference). This is the output circuit.
Binarizing circuit A13 is a circuit for outputting a value thresholds T ₁ greater than the pixel density difference 1, the other pixels as 0.
The tallying circuit 14 is a circuit for tallying the number of pixels having the value 1 by the binarizing circuit A13 for each section.
Binarization circuit B15 is the aggregate value by the summing circuit 14 is the threshold value T ₂ is greater than section 1, to convert the partition otherwise zero output.
[0011]
Next, a processing procedure of the motion detection unit 4 will be described.
{Circle around (1)} The difference between the Y component of the current one frame and the immediately preceding (past) one frame is obtained for each pixel by using the frame buffer 11 and the subtractor 12.
{Circle around (2)} The binarizing circuit A13 converts a pixel whose difference is larger than the constant T1 to ₁ and converts other pixels to 0.
(3) The counting circuit 14 counts the number of pixels having the value 1 for each section. For example, the section is rectangular, and an image of 320 pixels horizontally and 240 lines vertically is divided into 32 columns horizontally and 15 rows vertically.
▲ 4 ▼ 2 binarization circuit B15 1 a is the aggregate values constant T ₂ greater compartments, and outputs the set S1 and converts the compartments otherwise zero.
That is, in the present embodiment, as a data format representing a “set of partitions”, data in which partitions included in the set are replaced with 1 and partitions not included in the set are replaced with 0 among partitions constituting an image for one frame. The format is adopted, as shown in FIG.
[0012]
The target tracking unit 5 receives the set S1 of the sections output by the motion detection unit 4, estimates the target current position on the image, and outputs its two-dimensional coordinates.
However, there is a case where the target does not exist (there is no moving object in the field of view of the wide-angle camera 2). Therefore, the detection / non-detection of the target is also output separately as the target detection signal 10 and given to the subsequent stage.
Further, the size of the target on the image is also output for controlling the enlargement ratio of the pan / tilt camera 1.
[0013]
The internal configuration of the target tracking unit 5 is as shown in FIG.
The CPU 21 is a central control unit that performs various processes of each block constituting the target tracking unit 5, and the

storage units

1 and 22 are memories that hold the current position, shape, and size of the target as a set of sections. is there.
The connected area enumeration means 23 is an operation means for calculating a set W of connected areas included in the set S1 that have a common part with the given set S2.
The connected region selecting unit 24 is a unit that selects and outputs a set H that minimizes the value of the following function d (H) from the set W output by the connected region listing unit 23.
d (H) = (the number of sections included in the common part of the set H and the set S1) ÷ (the number of sections included in the set H)
The mask area memory 25 is a memory that stores a set M of areas other than the target where there is movement or a block that obstructs an obstacle (obstructs a gap between the target and the camera). Basically not rewritten.
The inclusion determining means 26 is a block for determining whether or not the estimation result H of the target position and the shape is included in M. When the H is included in M, the connected area output means 1 and 27 have a common part with H. This is a block that outputs a connected area M _C (H) of M.
The motion detecting means 2.28 is a means for outputting a position (section) B when H is included in M and detecting a motion on the periphery of M _C (H). Reference numeral 29 denotes a unit that obtains, from S1, a connected region including the section B output by the motion detection units 2.28, and outputs the connected region.
The selector 30 receives the result of the inclusion determination unit 26, selects the contents to be stored in the storage means 1, 22 (output as _{H 2).}
The circumscribed rectangle calculation 31 calculates the position and size of a rectangle circumscribing the target on the image.
The calculating means 2.32 finds the arithmetic mean (center of gravity) G2 of the coordinates of the sections included in the set H and outputs it as the representative position of the target.
[0014]
The target tracking unit 5 has three states: “target not detected”, “target tracking”, and “standby”.
The “target not detected” state indicates a state in which the target appears in the field of view of the wide-angle camera 2, and the “target tracking” state indicates that the target is moving in the field of view of the wide-angle camera 2 and The “tracking” state indicates that the target is stopped within the field of view of the wide-angle camera 2.
[0015]
Next, the processing of the “target not detected” state in the target tracking unit 5 is as follows.
{Circle around (1)} Wait for detecting a motion in an area (sense area) set in advance on the screen.
That is, the process waits for a common part to be generated between the set S1 output from the motion detection unit 4 and the set of sections constituting the sense area shown in FIG.
(2) When motion is detected in the sense area,
a. A connected area including the moved part is cut out from the set S1 and given to the storage means 1.22 as an initial value.
b. The state transits to the “target tracking” state.
[0016]
The process in the “target tracking” state is as follows.
{Circle around (1)} As shown in FIG. 4, the contents of the storage means 1.22 (set S2 of sections) are updated.
{Circle around (2)} When the following condition is satisfied, the state transits to the “standby” state.
a. Before update S2 後 After update S2
[0017]
The processing in the "waiting" state is as follows.
(1) It is determined whether a motion is detected in a preset area (sense area) in the screen.
That is, it is determined whether or not the set S1 output by the motion detection unit 4 includes a section constituting the sense area.
When a movement is detected, the same processing as in (2) a and b in “Target not detected” is performed.
{Circle around (2)} When the above condition is satisfied and one of the following conditions is satisfied, the state transits to the “target tracking” state.
a. H ₂でない not set S2 (H ₂ protrudes from S2)
{Circle around (3)} When _TC seconds elapse without the above conditions being satisfied, the state transits to the “target not detected” state.
[0018]
The target detection signal 10 outputs 0 in the “target not detected” state and 1 in other states.
The reason for providing the “standby” state is as follows.
{Circle around (1)} When the target stops moving, the motion detecting unit 4 hardly detects the motion, so that the number of sections included in the set S1 is smaller than that during the movement of the target.
{Circle around (2)} If the updating of S2 stored in the storage means 2 is continued in that state, S2 becomes an empty set, which supports subsequent tracking.
{Circle over (3)} Therefore, it is necessary to stop updating the storage means 2 from when the target stops until it starts moving again.
[0019]
The camera control unit 6 receives the coordinates G of the target, the target detection signal 10, and the position and size of the circumscribed rectangle output by the target tracking unit 5, rotates the pan-tilt camera 1 to capture the target in the field of view, and enlarges it. It sets the rate and consists of the following steps:
{Circle around (1)} From the coordinates G, the direction in which the target is viewed from the wide-angle camera 2 is estimated. (Estimated target direction)
(2) Calculate the pan and tilt angles given to the pan-tilt camera 1 from the estimated direction. (Calculation of camera turning angle)
(3) The calculated pan and tilt angles are given to the pan / tilt camera 1 and the camera is turned. (Turn command)
{Circle around (4)} A zoom command is given to the pan-tilt camera 1 to enlarge it so that the horizontal size of the circumscribed rectangle matches 70% of the horizontal size of the image captured by the pan-tilt camera 1. (Zoom command)
However, when the target detection signal 10 is 0 (target not detected), the above process is not performed and the process waits.
[0020]
A target position G on an image captured by the wide-angle camera 2 is input to the camera control unit 6, and a vector component from the focal point of the wide-angle camera 2 toward the target is obtained based on the target position G.
There are two types of lenses used in the wide-angle camera 2 for estimating the target prospective direction: a normal wide-angle lens (according to the perspective projection method) and a fish-eye lens (according to the fish-eye projection method). Find the vector component from the focus to the target.
The estimation of the expected target direction when the lens of the wide-angle camera 2 is normal (according to the perspective projection method) will be described with reference to FIG.
In FIG. 6, a vector from the focal point O of the wide-angle camera 2 to the target position P in the three-dimensional space gives a target prospective direction to be obtained.
Here, the X axis is set to be in the horizontal direction, and the Z axis is set to be coincident with the center line of the visual field (lens optical axis).
To simplify the derivation of the vector components, a vector OG parallel to OP is determined. Here, G is the intersection of the straight line OP and the projection plane.
In FIG. 6, when the X-axis component of the OG is x, the Y-axis component is y, and the Z-axis component is z, the following equation is obtained.
x = μ
y = ν
z = d = H _W / tan (H _A ) (1)
here,
μ: X component of coordinate G ν: Y component of coordinate G H _W : 1/2 of full width of perspective projection plane of lens
_HA : 1/2 of the horizontal viewing angle of the lens
d: distance from lens focal point to projection plane x: X component of vector OG y: Y component of vector OG z: Z component of vector OG
A method of estimating the expected target direction when the lens of the wide-angle camera 2 is a fisheye lens (according to the fisheye projection method) will be described with reference to FIG.
In FIG. 7, a vector from the focal point O of the wide-angle camera 2 to the target position P in the three-dimensional space gives a target prospective direction to be obtained.
Here, the X axis is set to be in the horizontal direction, and the Z axis is set to be coincident with the center line of the visual field (lens optical axis).
In order to simplify the derivation of the vector components, a vector OG 'parallel to OP is obtained. G ′ is a point on the plane Z = γ, and γ indicates a fish-eye projection surface radius.
From FIG. 7, D ₂ = γs
R ₀ = γH _A
Therefore,
D ₁ = γ tans = R ₀ / _HA · tan [( _HA / R ₀ ) · D ₂ ] (2)
here,
G ₀ : Intersection γ between the plane z = γ and the Z axis (lens optical axis) γ: Fish-eye projection radius D ₁ : Distance from G ₀ to G ′ D ₂ : Arc length H _A from G ₀ to G : Lens viewing angle ÷ 2 (90 ° for a general fisheye lens)
S: Angle between the expected direction of the target and the Z axis R ₀ : Viewing angle _HA on the image The X component x, the Y component y, and the Z component z of OG ′ are given by the following equations.
(Equation 1)

[0022]
Since the target prospective direction vector component (x, y, z) has been obtained in the above-described target prospective direction estimation processing, panning and panning such that this vector is parallel to the center line of view (lens optical axis) of the pan-tilt camera 1 are performed. The tilt angle is calculated according to FIGS.
Pan angle, obtains a vector V ₁ obtained by projecting the vector (x, y, z) and pan rotating surface (three-dimensional plane), taking the orthonormal coordinate system consisting Z'-axis and X'-axis on a pan rotation plane and X'component x'of _{V 1,} when obtaining the Z'component z ', the pan angle _{P a} is determined by the following equation.
P _A = tan ⁻¹ (x ′ / z ′)
[0023]
Then, the tilt angle takes the Y'-axis in the normal direction of the panning rotation plane, obtains a vector vector V ₂ which is projected (x, y, z) a three-dimensional plane spanned by Y'-axis vector V ₁ When obtaining the Y'component y'and _{V 1} component v'vector _{V 2,} the tilt angle _{P T} is obtained by the following equation.
_PT = tan ^-1 (y '/ v')
In the embodiment, the lens optical axis (Z axis) of the wide-angle camera 2 faces obliquely downward by 45 °, whereas the pan rotation surface of the pan-tilt camera 1 is horizontal.
Assuming that the angle between the lens optical axis of the wide-angle camera 2 and the pan rotation surface of the pan-tilt camera 1 is θ, the XYZ coordinate system and the X′-Y′-Z ′ coordinate system are as shown in FIG. .
[0024]
Pan angle P _A was determined by the _principle, given a tilt angle P _T, pan tilt camera 1 faces a side of incident light on the focal point O of the wide-angle camera 2 from the target position P has entered in a three-dimensional space, and The center line of view of the pan-tilt camera 1 and the straight line OP (incident light incident direction) are oriented so as to be parallel. Further, since the pan-tilt camera 1 and the wide-angle camera 2 are installed close to each other, the visual field center line and the straight line OP are close to each other. That is, the center line of the visual field of the pan-tilt camera 1 is close to the target position P in the three-dimensional space, and therefore P is photographed near the center of the photographed image of the pan-tilt camera 1.
[0025]
As can be understood from the above principle, the closer the pan / tilt camera 1 and the wide-angle camera 2 are installed, the more accurately the target position P in the three-dimensional space is photographed at the center of the field of view of the pan / tilt camera 1.
Therefore, the distance between the pan-tilt camera 1 and the wide-angle camera 2 (how close it must be) may be determined depending on how accurately P is to be captured in the center of the visual field on the image of the pan-tilt camera 1.
[0026]
In the embodiment, the pan-tilt camera 1 whose pan rotation center and tilt rotation center coincide with each other is used, and it is easy to determine the distance between the pan-tilt camera 1 and the wide-angle camera 2. That is, if the center line of view of the pan-tilt camera 1 that aims at the target position P in the three-dimensional space must not deviate by more than the distance d, the pan-tilt camera 1 is moved so that the distance between the two points O ^and O becomes less than d. And the wide-angle camera 2 must be installed.
Here, O: focus O ^· wide-angle camera 2: pan and tilt rotation center [0027] of the pan-tilt camera 1
The turning command and the zoom command are as follows.
(1) An enlargement ratio of the pan-tilt camera 1 is calculated such that the horizontal size of the circumscribed rectangle coincides with 70% of the horizontal size of the image captured by the pan-tilt camera 1.
▲ 2 ▼ in the photographing unit 1 calculates the threshold T ₅ when shooting in z times zoom, when G input from the motion detector 4, which was away compared to G of the previous input, the distance T ₅ or Commands the pan-tilt camera 1 to perform pan (absolute angle P _A ), tilt (absolute angle P _T ), and zoom (magnification z).
(3) Store the current G and wait for the next G input.
[0028]
The following elements are added as automatic correction means for automatically correcting erroneous estimation of the position and shape of the target in a region where there are many moving objects other than the target or where there are obstacles blocking the target.
{Circle around (1)} A storage unit 2 for storing a set M of sections that cover an area in which there are many moving objects other than the target or an obstacle that blocks the target among the sections constituting the input image.
{Circle around (2)} The inclusion determining means 26 for determining whether or not the set H of the sections indicating the area where the target being tracked occupies on the input image is included in M.
{Circle around (3)} When H is included in M, connected region output means 1 and 27 which output a connected region M _C (H) of M having a common part with H.
{Circle around (4)} When H is included in M, motion detecting means 2.28 which monitors the motion on the periphery of the connection area M _C (H) and outputs the position (section) B when the motion is detected.
{Circle around (5)} When the motion detecting means 2.28 detects a motion (outputs B), a connected area output means 2.29 for outputting a connected area S _1C (B) including B from a section constituting the input image.
{Circle around (6)} When the movement detecting means 2 detects a movement, the correcting means selector 30 replaces the contents of the set H with the contents of the connected area S _C1 (B).
[0029]
With the above-described components, while the area H in which the target is assumed to be projected on the image of the wide-angle camera 2 is within (including) the area M, the connected area is concurrently updated with H as the target. The movement at the periphery of M _C (H) (the connected region of M including H) is monitored, and when the movement is detected in the section B on the periphery, the update of H is stopped, and B is regarded as a part of the target. By re-tracking as it is, it is possible to prevent erroneous estimation of a target other than the target in the area M as a target.
[0030]
【The invention's effect】
As described above, according to the moving image automatic tracking photographing method and the automatic tracking photographing apparatus of the present invention, a target to be captured by the photographing means 1 capable of changing the direction of the visual field is specified on the image of the photographing means 2 having a wider angle. Moving image capturing means, a tracking algorithm for continuously estimating the position and shape of a moving object on the image of the capturing means 2, and a large number of moving objects other than the target or obstacles blocking the target. A means for automatically correcting an erroneous target estimation in a target area, so that an extremely high-precision security system capable of continuously capturing an object moving with only two cameras in a field of view and automatically tracking and photographing. It is possible to construct
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of an automatic tracking photographing apparatus according to the present invention.
FIG. 2 is a block diagram illustrating a configuration of a motion detection unit.
FIG. 3 is an example of a set of partitions S1.
FIG. 4 is a block diagram showing a configuration of a target tracking unit.
FIG. 5 is an example of a sense area and a mask area.
FIG. 6 is an explanatory diagram when a lens of a wide-angle camera is normal.
FIG. 7 is an explanatory diagram when a lens of a wide-angle camera is a fisheye lens.
FIG. 8 is an explanatory diagram of a pan angle calculation method.
FIG. 9 is an explanatory diagram of a tilt angle calculation method.
FIG. 10 is an explanatory diagram showing a relationship between an XYZ coordinate system and an X′-Y′-Z ′ coordinate system.
[Explanation of symbols]
REFERENCE SIGNS LIST 1 Pan-tilt camera 2 Wide-angle camera 3 A / D conversion unit 4 Motion detection unit 5 Target tracking unit 6 Camera control unit 7 Video recording device 11 Frame buffer 12 Subtractor 13 Binarization circuit A
14 Counting circuit 15 Binarization circuit B
20 Motion detection means 1
21 CPU
22 Storage means 1
23 connected area listing means 24 connected area selecting means 25 mask area memory 26 inclusion determining means 27 connected area output means 1
28 Motion detection means 2
29 Connecting area output means 2
30 selector 31 circumscribed rectangle calculation 32 calculation means 2

Claims

A photographing means 1 capable of changing the direction of the visual field;
Shooting means 2 having a wider angle of view;
A moving image photographing means for designating a photographing target (hereinafter referred to as a target) of the photographing means 1 on an image of the photographing means 2;
Tracking means for continuously estimating the position and shape of the target on the moving image captured by the imaging means 2 under conditions in which the target continues to move;
Automatic correction means for automatically correcting erroneous estimation by tracking means in an area where there are many moving objects other than the target, or where there are obstacles blocking the target,
And automatically capturing and tracking a moving object within the field of view of the photographing means 1 and photographing the moving object.

A photographing means 1 capable of changing the direction of the visual field;
A photographing means 2 having a larger horizontal or vertical angle of view than the photographing means 1;
Estimating means for estimating the incident direction of the incident light imaged at coordinates G on the image taken by the photographing means 2;
Control means for changing the direction of the photographing means 1 so that the estimated incident direction of the incident light incident on the coordinates G and the center of the visual field of the photographing means 1 are parallel to each other and the visual field faces the side on which the incident light is incident;
A moving image photographing means for setting the photographing means 1 and the photographing means 2 close to each other and designating a target on an image of the photographing means 2 having a wider angle of view. 2. The method for automatically tracking and capturing a moving image according to 1.

A motion detecting unit 1 that obtains the motion of the input image for each of the sections constituting the image and outputs a set S1 of the sections in which the motion has been detected;
Connected area enumeration means for obtaining a set W of connected areas having a common part with the given set S2 among the connected areas included in the set S1,
Connected area selecting means for selecting and outputting a connected area H in which the value of the following function d (H) is minimum from the set W output by the connected area enumeration means;
d (H) = (the number of sections included in the common part of the set H and the set S1) ÷ (the number of sections included in the set H)
A storage unit 1 for temporarily storing the connected region H output by the connected region selecting unit, and outputting the stored contents as a set S2 in response to a request from the connected region listing unit;
An arithmetic means 2 for calculating an arithmetic mean (center of gravity) G2 of the coordinates of the sections included in the connected area H output by the connected area selecting means and outputting it as a representative position of the target;
3. A method according to claim 1, further comprising tracking means for estimating a position and a shape of a moving object appearing in the moving image.

A storage unit 2 that stores a set M of sections that cover an area in which there are many moving objects other than the target or an obstacle that blocks the target, among the sections configuring the input image;
Inclusion determination means for determining whether or not a set H of sections indicating regions where a target being tracked is occupied on the input image is included in the set M;
When the set H is included in the set M, a connected area output unit 1 that outputs a connected area M _C (H) of the set M having a common part with the set H;
When the set H is included in the set M, the motion detecting means 2 monitors the motion on the periphery of the connected area M _C (H), and outputs the position (section) B when the motion is detected;
A connection area output means 2 for outputting a connection area S _1C (B) including B from a section constituting the input image when the movement detection means 2 detects a movement;
Correction means for replacing the contents of the set H with the contents of the connected area S _C1 (B) when the movement detecting means 2 detects a movement;
And automatic correction means for automatically correcting erroneous estimation of the position and shape of the target in a region where there are many moving objects other than the target or where there are obstacles blocking the target. The method for automatically tracking and photographing a moving image according to claim 1.

A computing unit 4 for obtaining a direction in which the photographing unit 1 is to be directed by the command before the control unit instructs the photographing unit 1 to turn;
A storage unit 3 for storing an output of the arithmetic unit 4 immediately before the instruction when the control unit instructs the photographing unit 1 to actually turn;
The control unit instructs the photographing unit 1 to make a turn because the angle between the direction currently output by the arithmetic unit 4 and the direction stored in the storage unit 3 is a threshold value shown below. only as when T ₄ or more following condition is satisfied, to reduce the wear of the mechanical system imposes constraints on change of the field of view of the imaging unit 1, and while realizing a stable shooting a moving object of the imaging unit 1 5. The method for automatically tracking and capturing a moving image according to claim 1, wherein the tracking is continuously performed while being captured in the field of view.
Threshold value T ₄ = tan ⁻¹ (tan C / Z)
Here, Z: zoom magnification in the imaging unit 1 C: magnification Z = 1 the value of T ₄ T when the (maximum wide-angle state) _4: Minimum targets move to direct the pivot photographing means 1 Angle (threshold) [rad]

When the control means instructs the photographing means 1 to actually turn, a component comprising a storage means 4 for storing the value of the coordinate G is added,
To command the turning from the control means to the imaging unit 1, the distance between the coordinates stored as coordinates G in the storage unit 4, and only when the condition becomes equal to or higher than the threshold T ₅ shown below is satisfied, the photographing means (1) reducing the wear of the mechanical system by restricting the change of the visual field and realizing stable photographing, while continuously tracking a moving object within the visual field of the photographing means 1; 5. The method for automatically tracking and photographing a moving image according to claim 1, wherein:
Threshold value T ₅ = C / Z
Here, T ₅ : minimum target movement amount (threshold) required to instruct the photographing means 1 to turn.
Z: zoom magnification in the imaging unit 1 C: value of _{T 5} when the enlargement ratio Z = 1 (at the maximum wide-angle)

By using a camera that changes the field of view by panning and tilting as the photographing means 1 and setting the pan plane to be horizontal (instead of being parallel to the center of the visual field of the photographing means 2), the image of the photographing means 1 can be obtained. 7. The image capturing apparatus according to claim 1, wherein an image on an upper end side of a center (center of gravity) of the moving object is continuously captured in the field of view of the photographing unit without tilting. Automatic tracking shooting method for the described moving image.

A pan-tilt camera and a wide-angle camera installed close to each other, an A / D conversion unit for converting a moving image captured by the wide-angle camera into digital moving image data, and a set S1 of sections that have input digital image data and have detected movement. The motion detection unit to be output and the set S1 of the sections are input to estimate the current position of the target on the image, to output its two-dimensional coordinates G, and to output the target detection signal 10 and the rectangular shape circumscribing the target on the image. A target tracking unit that outputs a position, receives the coordinates G of the target output by the target tracking unit, the target detection signal 10, and the position and size of the circumscribed rectangle, and rotates the pan-tilt camera to capture the target in the field of view. Camera control section to set the magnification and input to pan / tilt control signal output from camera control section A video recording apparatus for displaying a target image from the pan-tilt camera for photographing a moving image Te,
An automatic tracking and photographing apparatus for moving images characterized by comprising: