JP4284949B2 - Moving shooting system, moving shooting method, and shooting apparatus - Google Patents

Description

【００４５】
この表１に列挙したものは、例示に過ぎないものであるが、カメラを搭載する移動体としては、空中を移動するものに限らず、陸上や水面上、或いは水中などの種々の場所を移動できるものが挙げられる。また、用途についても、行楽用途に限らず、スポーツ記録や映画や自然観察など種々の応用が可能である。場所や具体的なイベントも限定されずに種々のものに適用できる。撮影対象も人だけに限らず、種々の生物であっても良く、車や船などであっても良い。また、撮影対象は単数とは限らず、複数であっても良い。光ビーコン装置３１は、キー入力の操作が加わるために被写体が人間の場合となるが、例えば被写体が動物の場合では操作が不要で光信号を出力するような光ビーコン装置がその被写体としての動物に取り付けられる。
【００４６】
次に図６を参照しながら、光ビーコン装置の操作によって飛行体の飛行予定を変更する例について説明する。撮影の途中であっても、本実施形態の移動撮影システムでは、光ビーコン装置が無人ヘリコプターの如き飛行体と交信でき、都合に応じて飛行予定を変更することができる。
【００４７】
先ず、手順Ｓ１１として、ユーザーが光ビーコン装置を身に付ける。光ビーコン装置は図３に示すような矩形で略平板状の装置とすることも可能であり、所要の光信号を出力できる他の装置であっても良い。
【００４８】
次に、手順Ｓ１２ではユーザーが光ビーコン装置を操作して、例えば無人ヘリコプターの如き飛行体の飛行を開始させる。この時、この場合には、光ビーコン装置に対して飛行開始のための操作を行い、その操作に応じて光ビーコン装置の発光ダイオードを点滅させる。この発光ダイオードの点滅は、飛行体に搭載されるＩＤ認識カメラに捕捉され、飛行体が離陸して飛行が開始される。
【００４９】
飛行体の離陸の後、前述のように、被写体としてのユーザーが離陸した飛行体に自動的に追尾されて撮影が開始する。例えば、飛行体の離陸前の設定では、ユーザーの２０ｍ後ろから４５度の角度で撮影すると言うような設定であった場合において、飛行体の離陸した後で撮影が開始すれば、その離陸前の設定に従って撮影が開始される。このような離陸前の設定は、このフローでは変更可能であり、手順Ｓ１３ではユーザーが光ビーコン装置を操作して、飛行体の位置を変更させる光信号を光ビーコン装置が出力させる。このような飛行体の位置の変更例としては、たとえば飛行体の離陸前の設定でユーザーの２０ｍ後ろから４５度の角度で撮影するという設定内容であったものを、ユーザーの正面２０ｍ手前から４５度の角度で撮影するという設定内容に変更できる。
【００５０】
光ビーコン装置が飛行体の位置を変更させる光信号を出力すると、飛行体のＩＤ認識カメラが光ビーコン装置からの光信号を受信し、その光信号に含まれるＩＤ番号の形式の制御信号に従って飛行中の飛行体がその位置を修正する（手順Ｓ１４）。この修正によって、例えば飛行体は２０ｍ後ろからの追尾状態からユーザーの前側に移動して撮影を続けることになり、このような撮影位置の変更によって、被写体であるユーザーの後ろ姿だけが撮影される状況からユーザーの顔の表情も撮影できるような変更ができることになる。このような撮影位置の変更は、特に離陸前のプログラムとは独立して行うことができ、撮影の状況で撮影位置を移動させる場合に極めて有効である。
【００５１】
次に図７を参照しながら、光ビーコン装置の操作によって撮影方法を変更する例について説明する。本実施形態の移動撮影システムでは、撮影の途中であっても、光ビーコン装置が無人ヘリコプターの如き飛行体と交信でき、都合に応じて撮影方法を変更することができる。
【００５２】
先ず、手順Ｓ２１として、ユーザーが光ビーコン装置を身に付ける。光ビーコン装置は図３に示すような矩形で略平板状の装置とすることも可能であり、所要の光信号を出力できる他の装置であっても良い。
【００５３】
次に、先の図６の手順Ｓ１２と同様に、手順Ｓ２２ではユーザーが光ビーコン装置を操作して、例えば無人ヘリコプターの如き飛行体の飛行を開始させる。この時、この場合には、光ビーコン装置に対して飛行開始のための操作を行い、その操作に応じて光ビーコン装置の発光ダイオードを点滅させる。この発光ダイオードの点滅は、飛行体に搭載されるＩＤ認識カメラに捕捉され、飛行体が離陸して飛行が開始される。
【００５４】
飛行体の離陸の後、前述のように、被写体としてのユーザーが離陸した飛行体に自動的に追尾されて撮影が開始する。例えば、飛行体の離陸前の設定で、被写体の全身を画像のフレーム一杯とするような設定であったとすると、撮影が開始すると、そのようなユーザーの頭から足先にかけての映像がＩＤ認識カメラによって撮影されて行くことになる。このような撮影が進行してうちに、ユーザーの都合や環境や事故などに応じて、ユーザーは身に付けている光ビーコン装置を操作して撮影の方法を変更させる光信号を光ビーコン装置から送出させる（手順Ｓ２３）。このような撮影方法の変更の一例としては、被写体の全身を画像のフレーム一杯とする撮影法から、景色を十分に入れてカメラが引いた状態の撮影をするように変更できる。ある種類の撮影方法の変更は、飛行体の位置の変動を伴わずに可能であり、ズームアップやズームダウンなどのレンズ系の操作や、静止画の場合のシャッター速度の調整、フィルターの付加、時計の表示の制御など各種の変更が可能であり、それらは光ビーコン装置を操作することで可能となる。
【００５５】
光ビーコン装置が撮影の方法を変更させる光信号を出力すると、飛行体のＩＤ認識カメラが光ビーコン装置からの光信号を受信し、その光信号に含まれるＩＤ番号の形式の制御信号に従って撮影の方法が修正される（手順Ｓ２４）。すなわち、その光信号によって、例えば、飛行体の離陸前の設定で、被写体の全身を画像のフレーム一杯とするような設定であった場合、景色を十分に入れてカメラが引いた状態の撮影をするように修正される。このような変更は、カメラ手元にある場合には容易に行うことが可能であるが、カメラが飛行体に搭載されている場合においては、光ビーコン装置からの光信号が有効に変更できる手段として利用できることになる。
【００５６】
次に図８乃至図１３を参照して、自動撮影時の追尾方法の例について説明する。図６、図７のフローチャートの制御は追尾の途中で変更する場合を含んでいるが、以下に説明するように、光ビーコン装置からの光信号を利用して自動的な撮影を追尾しながら行うことも可能である。なお、本明細者においては、"追尾"とは必ずしも後ろから追跡するような場合のみを指すのではなく、任意に側面や正面などに廻り込んで被写体を追うような場合も含むものである。
【００５７】
先ず、自動撮影時の追尾の基本的な流れは、図８に示すように、光ビーコン装置を身に付けたユーザーが該光ビーコン装置を操作して例えば無人ヘリコプターの如き飛行体の飛行を開始させる（手順Ｓ３１）。この開始直後では、飛行体は光ビーコン装置の光信号の捕捉よりも離陸時の姿勢制御が優先することから、光ビーコン装置の光信号を見失うことがあり、そのような場合では、飛行体が旋回して、或いはＩＤ認識カメラが固定でない場合にはＩＤ認識カメラの撮像範囲を変更しながら、ユーザーが身に付けている光ビーコン装置からの光信号を探知する（手順Ｓ３２）。この段階では徐々に光ビーコン装置が光信号によって探知されることになるが、光ビーコン装置の光信号がＩＤ認識カメラに捉えられたところで、光ビーコン装置からの光信号が発見されたことになり、その段階で自動撮影を行う追尾を開始する（手順Ｓ３３）。
【００５８】
この自動撮影による追尾の開始時には、優先するモードが選択される。本システムで使用できるモードは４つあり、その４つのモードはフレーミングモード、距離・角度モード、ズームモード、撮影方向モードである。フレーミングモードは、被写体を画面のどこに配置して撮影するかを設定するモードであり、左側、右側、中央、上側、下側などの設定が可能である。図９はフレーミングモードの一例を示す図であり、図９の（ａ）が被写体を画面の右側に配置する例であり、図９の（ｂ）が被写体を画面の中央に配置する例である。図９において、ユーザー３はフレーム４５の中に捉えられるように写されるが、フレーミングモードの設定によって、ユーザー３の位置が概ね決められることになり、フレーミングモードが右側を被写体とする場合には図９の（ａ）に示す状態とされ、フレーミングモードが中央を被写体とする場合には図９の（ｂ）に示す状態とされて撮影が行われることになる。
【００５９】
図１０は距離・角度モードを説明するための図である。飛行体である無人ヘリコプター１０と光ビーコン装置３１を身に付けたユーザー３との間の距離Ｄと無人ヘリコプター１０の飛行する角度θが設定され、この設定された距離Ｄを角度θを維持するように追尾を行う。このような距離Ｄや角度θについては、無人ヘリコプター１０が常時光ビーコン装置３１からの光信号によって把握できるデータであり、常時設定値とのずれが修正されて撮影が続けられることになる。
【００６０】
図１１はズームモードを説明するための図であって、図１１の（ａ）が被写体をズームダウンして撮影した場合の画面であり、図１１の（ｂ）が被写体をズームアップして撮影した場合の画面である。ズームモードとしては、図１１の（ａ）と（ｂ）の２段階のズームだけではなく、無段階若しくは２以上の多段階とすることもできる。図１１の（ａ）に示すように被写体をズームダウンして撮影した場合では、被写体としてのユーザー３がフレーム４５に対して相対的に小さく撮影されて景色が画面上大きな面積を占めるように写され、図１１の（ｂ）に示すように被写体をズームアップして撮影した場合では、ユーザー３がフレーム４５に対して相対的に大きな面積を占めるように撮影されることになる。
【００６１】
図１２は撮影方向モードを説明するための図であり、被写体をどの方向から撮影するのかが設定できることになる。この撮影方向モードでは、例えば上から、前から、横から、後ろからというような選択が可能であり、前からと言う設定をした場合では、ＩＤ認識カメラは被写体であるユーザー３の正面からの映像を自動的に撮り続けることになる。
【００６２】
また、自動撮影を継続させる際に移動する被写体を追うことや光信号を見失うことなども起こりうる事象であり、本実施形態の移動撮影システムでは、そのような場合についても対処可能であって、特に移動速度を予測して、的確な撮影方向や角度での撮影が可能とされる。図１３は自動撮影時の追跡方法について説明するためのフローチャートである。
【００６３】
先ず、無人ヘリコプターの如き飛行体が飛行を開始したものとすると、その飛行体は周囲を３６０度旋回して光ビーコン装置からの光信号を探索する（手順Ｓ４１）。この光信号の探索には、ＩＤ認識カメラが使用され、具体的には前述の図２の如き回路におけるＩＤカメラ処理部２１によってＩＤ信号の抽出が行われる。この手順Ｓ４２の段階で光ビーコン装置からの光信号が発見されなければ（ＮＯ）、手順Ｓ４３に進み、探索回数ＲＮのカウント数を１つ増加させる。次に手順Ｓ４４で探索回数ＲＮが最大探索数Ｆに達しているか否かが判断され、最大探索数Ｆに達していない場合（ＮＯ）には、もう一度光ビーコン装置からの光信号の探索を繰り返すために、制御は手順Ｓ４１に戻り、飛行体は周囲を３６０度旋回して光ビーコン装置からの光信号を探索する。手順Ｓ４４で探索回数ＲＮが最大探索数Ｆに達している場合（ＹＥＳ）には、飛行体は着陸して（手順Ｓ４５）処理を終了する。
【００６４】
手順Ｓ４２で光ビーコン装置からの光信号を発見した場合（ＹＥＳ）、その光信号に含まれるＩＤ信号が読み取られる（手順Ｓ４６）。この読み取り動作によって、ＩＤ信号がＩＤカメラ処理部で抽出され、それが目標とする被写体のＩＤ番号に対応する信号か否かが判断される（手順Ｓ４７）。すなわち、一台の飛行体で一度に複数の被写体を追尾する場合には、それぞれの被写体が所要の光ビーコン装置を保持し、各光ビーコン装置が異なるＩＤ信号を含むことから、被写体の区別をＩＤ信号で認識できることになる。
【００６５】
読み取った結果のＩＤ信号が目標とする被写体のものではない場合（ＮＯ）、手順Ｓ４１に戻り、再びその飛行体は周囲を３６０度旋回して光ビーコン装置からの光信号を探索して、前述の制御を繰り返す。手順Ｓ４７で読み取った結果のＩＤ信号が目標とする被写体のものである場合（ＹＥＳ）、飛行体は正しい被写体を捉えていることになり、飛行体は追尾を開始する（手順Ｓ４８）。
【００６６】
この自動追尾による撮影時には、光信号が撮影画像の範囲内で移動するのを検知する（手順Ｓ４９）。この光信号の移動は、被写体の移動を意味するものと飛行体側で把握され、光信号の移動により、被写体の次の移動方向や移動速度が予測され、その予測値に対応するように飛行体の向きや位置を変えるものとされる（手順Ｓ５０）。
【００６７】
図１４は光ビーコン装置３１からの信号を捕捉しながら追尾する際の概念図であり、図１４の（ａ）から（ｃ）に亘って順次ユーザー３の移動に従って光ビーコン装置３１が右上から左やや下側に移動していくことになるが、その画面内での移動速度や移動方向がＩＤカメラ処理部で抽出されて、その動きの予測値に応じて飛行体やＩＤ認識カメラの位置が移動して行くことになる。このため例えばスキーのように高度差が大きく移動する距離も長い場合であっても確実にスキーヤーを被写体として捉えることが可能である。
【００６８】
このような撮影を行い、途中で光ビーコン装置３１からの光信号を飛行体が外してしまうこともある。例えば、撮影時に被写体が物陰に隠れたり、光信号が何らかの原因で途絶えた際には、光信号を捕捉しているか否かの判断（Ｓ５１）がＮＯとなり、探索回数ＲＮを一旦リセットした後、手順Ｓ４３に進み、探索回数ＲＮを１つ増加させて、以降、探索回数がＦまで増加した場合には飛行体は着陸する（手順Ｓ４５）。
【００６９】
手順Ｓ５１で、光信号を捕捉しているか否かの判断（Ｓ５１）がＹＥＳの場合、以降、飛行を継続するが、光ビーコン装置からの光信号からなるリモコン信号の受信に応じ、そのリモコン信号に応じて撮影方法を変化させたり、飛行体の飛行を中止したりすることが可能である。これら撮影方法の変更や飛行体の制御については、それぞれ図７や図６を参照しながら説明した前述の例のように制御されるものである。
【００７０】
次に、図１５を参照しながらＩＤ認識カメラの例について更に説明する。本実施形態に移動撮影システムにおいては、光ビーコン装置と共にＩＤ認識カメラが光ビーコン装置からの光信号の受信用に用いられ、種々の制御が可能となる。ここでＩＤ認識カメラとは、十分高速に撮像可能な２次元配列の受光ブロックと各受光器から出力される光電変換によって蓄積された電荷量またはそれを変換した電圧値、電流値、Ａ／Ｄ変換によるデジタル値などを演算することが可能カメラであって、各画素または複数画素単位で、光の点滅パターンからその時系列信号を読み出すことができ、かつ画像も取得することが可能なカメラである。すなわち、このＩＤ認識カメラによれば、画像とＩＤ情報（ＩＤデータ、Ｘ、Ｙ座標）が得られることになる。
【００７１】
ＩＤ認識カメラの構成としては、通常の撮像素子に演算部を結合させることも可能であるが、例えば、文献S.Yoshimura, T.Sujmiyama, K.Yonemoto, K.Ueda, "A 48Kframes/s CMOS Image Sensor for Real-time 3-D Sensing and Motion Detection" ISSCC Digest of Technical Papers, pp.94-95, Feb.2001や特開2001-326857号公報に開示されるような演算機能付き撮像素子を用いても良い。すなわち、演算機能付き撮像素子では、１つの撮像素子でありながら、デジタル画像と演算処理した画像を得ることが可能である。また、CMOS 構造とすることで、CMOS（Complementary Metal-Oxide Semiconductor）の半導体製造技術から従来のCCD素子では困難な高速撮像を実現することも可能である。
【００７２】
図１５にＩＤ認識カメラの一例の要部を示す。ＩＤ認識カメラは、主にイメージセンサ部５１とデコード処理部６１からなる。イメージセンサ部５１は光信号を電気信号に変換するための領域であるが、本カメラにおいては、イメージセンサ部５１は受光エリア５２と演算エリア５３から構成される。なお、イメージセンサ部５１が図２に示す装置の撮像部２０に該当し、デコード処理部６１が図２に示す装置のＩＤカメラ処理部２１に該当する。
【００７３】
受光エリア５２は、カメラのレンズ等を介して受光した光を電気信号に変換するCMOS固体撮像素子の如き受光素子５４と、この受光素子５４からの信号を増幅する増幅部５５とを有している。一方、演算エリア５３は、受光エリア５２からの信号を一時的に蓄積する記憶部５６と、記憶部５６に蓄積された信号のレベルと基準信号レベルを比較するための比較部５７と、比較部５７からの比較結果に応じた出力を行う出力部５８とを有している。記憶部５６には、画素ごとの信号が例えば異なるタイミングで蓄積されて行き、比較部５７には一括して読み出される。
【００７４】
また、前述のデコード処理部６１は、画像信号を処理する画像デコード処理部６２と、これと並列して配置されるＩＤデコード処理部６３とを有している。ＩＤデコード処理部６３が光ビーコン装置から発せられた光信号をデコードしてＩＤ情報を抽出する回路部分であり、また、同時にそのＩＤ情報にかかる座標についても抽出されるように構成されている。
【００７５】
ここで、イメージセンサ部５１で行われる演算とは、
【００７６】
[数１]
F(N)+F(N-1)-F(N-2)-F(N-3)・・・（１）
【００７７】
ここで、Ｎはフレーム番号を表し、F(N)はＮフレーム目の受光部に蓄積された輝度を表すものである。光ビーコン装置からの光信号は点滅する点光源と考えられることから、式（１）の値は、点光源が存在する画素についての４フレーム間の演算となる。この演算により、光の変化を検出することができるため、その演算結果を用いて、後述するＩＤデコード処理を行うことで、点滅パターンを復元し、光ビーコンからの信号を受信することが可能となる。この（１）式によらず、他の演算例えば１次微分や２値化画像の比較などによっても演算は実現可能であるが、本実施形態では、光変化の検出の安定性を考え、（１）式を用いて説明する。
【００７８】
また、本実施形態のＩＤ認識カメラにおいては、外部からの信号、例えば光ビーコン装置からの光信号を用いることによって、上記の画像処理結果とＡ／Ｄ変換したデジタル画像の両方を得ることができる。例えば、前記画像処理結果と前記Ａ／Ｄ変換したデジタル画像を交互に得ることで、ＩＤ認識カメラはＩＤ情報用の画像処理結果とシーンについての画像の両方を取得することが可能である。このＩＤ認識カメラが扱う信号を時分割でモード分けし、例えばここで画像モード、ＩＤモードがそれぞれ30fps（フレーム／秒）や60fpsといったフレームレートにすることも可能である。すなわち、イメージセンサ部５１から出力される信号を画像モードとＩＤモードで交互に送信されるように設定し、それぞれが対応するデコード処理部６２、６３で処理されてデコードされることになる。
【００７９】
図１６は画像モードとＩＤモードが交互に送信される例であり、画像モードには撮影された画像情報が含まれて送信される。ＩＤモードの期間内には、ＩＤデコード処理区間Ｐ２とＩＤ重心区間Ｐ１が含まれており、これらの区間Ｐ１、Ｐ２に含まれるＩＤ情報から上述の如き各種の制御が可能となる。画像モードにおける信号を画像デコード処理部６２で処理することで画像読み出しが実現され、ＩＤモードにおける信号を画像デコード処理部６２で処理することでＩＤ読み出しが実現される。なお、図１６においては、画像モードとＩＤモードが同時間のように示されているが、同時間でも良いし、同時間である必要もない。
【００８０】
次に、前記イメージセンサ部５１における輝度信号レベルについての画像処理について簡単に説明する。図１７に示すように、前記イメージセンサ部５１において、画像モードでは、受光素子（フォトダイオード）５４の各画素で検出した輝度を示す信号レベルが、基準信号レベルＶrefと比較され、前記基準信号レベルＶrefよりも大きくなったときに、アクティブになるような信号が出力される。すなわち、受光素子５４に入射した光は、該受光素子５４で光電変換され、さらに増幅部５５で増幅された信号が、記憶部５６にフレーム毎に蓄積されて行く。図１７はフレーム数に応じた明るい画素（高輝度時）と暗い画素（低輝度時）の輝度信号レベルの変化を示しており、一画面の初めでは、フレーム数はゼロでリセットレベルＶrsに輝度信号レベルがある。しかし、図１７に示すように、明るい画素ほど早く基準信号レベルＶrefに達し（図中TH）、暗い画素ほど基準信号レベルＶrefに達するまでの時間（図中TL）が長くかかることになる。実際のセンサからの出力は、比較部５７において、各画素について各画素の輝度信号が、前記基準信号レベルＶrefに達したかどうかの比較結果（コンパレータ出力）を２値（０，１）で出力することで行われる。図１７では、高輝度時の変化で時間ＴＨのところで、２値の出力が"０"から"１"に変化する例を示している。このイメージセンサ部５１は、高速に撮像して上記演算を行い、１画面分の結果をセンサ外に出力することが可能である。
【００８１】
明るい画素や暗い画素からの信号が基準信号レベルＶrefに達するまでの時間(TH、TL)を計測するためには、このイメージセンサ部５１が１画面単位で結果を出力するので、このフレーム数をカウントしておいて、各画素で何番目のフレームでイメージセンサ部５１の出力信号がアクティブすなわち２値の出力が"０"から"１"に変化するかを記録することで、その画像が得られることになる。画像情報として、正しい明るさに変換するためには、明るい画素ではアクティブに遷移するまでの時間が短く、暗い画素ではアクティブに遷移するまでの時間が長いことから、この基準信号レベルＶrefに到達するまでの時間（フレーム数）の逆数を計算すればよい。この動作により、画像のＡ／Ｄ変換動作が実現できることになる。このイメージセンサ部５１からのセンサ出力が画像処理デコード部６２に送られ、画像処理デコード部６２で画像を得るための処理が行われる。
【００８２】
次にＩＤデコード処理について説明する。まず、イメージセンサ部５１と光ビーコン装置の周波数について説明すると、イメージセンサ部５１のフレームレートfsは、光ビーコン装置の光信号の点滅周波数fpの例えば３倍に設定される。これは「シャノンの標本化定理」より、センサ部のサンプリングレートは、光ビーコン装置からの光信号の点滅周期より２倍以上のサンプリングレートが必要とされるからであり、またＩＤ認識カメラのシステムにおいては、fsとfpの位相の違いが出ることから、この位相差の影響を加味して、本実施形態では３倍とする。例えば、センサ部の周波数が例えば12KHzの場合、光ビーコンの点滅周期は4KHzと設定される。以下説明がわかりやすいように、本実施形態では、センサの周波数が12KHzの場合、光ビーコンの点滅周期は4KHzとして説明する。
【００８３】
次に、光ビーコン装置の構造例について、図１８を参照しながら説明する。先に、光ビーコン装置については、図３、図４を参照して、その一例について説明したが、ここでは図１８を参照して更に他の構成例について説明する。図１８の（ａ）に示す光ビーコン装置の例では、光ビーコン装置７１は、光源７２、点滅制御部７３、送信データ保存用メモリ７４を有する構造とされる。光源７２は点滅制御部７３からの駆動信号に応じて点滅動作するように構成されており、点滅制御部７３は状況や設定に応じた出力データを送信データ用保存用メモリ７４から取り出して動作する。光源７２は高速に点滅することができる光源であれば、如何なる光源でも良く、出力波長もセンサが反応する範囲であれば良い。すなわち、光源７２の波長域としては例えば可視光だけではなく、赤外線域なども使用可能である。応答速度、寿命の点から、ＬＥＤは最適な光源の１つである。また点滅制御部７３は、発信器とデジタル回路（ＩＣ，ＡＳＩＣ，ＦＰＧＡ、ワンチップマイコンなどで実現可能）によって、送信データ保存用メモリ７４の内容に基づいて、上記光源７２を点滅させる。
【００８４】
また、図１８の（ｂ）は、通信路７９を通じて、送信データを可変とすることが可能な光ビーコン装置７５の構成例を示しており、（ａ）と同様に、光ビーコン装置７５は、光源７６、点滅制御部７７、データ送受信部７８を有する構造とされる。光源７６は点滅制御部７７からの駆動信号に応じて点滅動作するように構成されており、点滅制御部７７は通信路７９を介して信号が供給されるデータ送受信部７８からの制御信号に応じて点滅する。ここで通信路７９とは、有線・無線の通信回線、例えば電話回線ISDN, RS-232C, RS-422, Ethernet (登録商標)(10base-T, 100base-T), USB,IEEE1394, IEEE802.11a, IEEE802.11b, BlueTooth (商標名)などの通信手段を指す。またデータ送受信部７８は、この通信手段に対応するデータ通信用ＩＣ及びドライバを指し、通信路７９を介して受信した光ビーコン装置の送信データは、このデータ送受信部７８経由で、点滅制御部７７に渡され、光源７６の点滅パターンとなる。
【００８５】
ここで、光ビーコン装置から送信される光信号の符号化方式について説明する。デジタル通信では、受信側で安定して受信できるように一般に送信データは符号化される。本実施形態では、光ビーコン装置から送信されるデータ（点滅パターン）は、マンチェスタ符号化を用いて符号化される。マンチェスタ符号化は、図１９に示すように、送信データ０に対して発信データを１０、送信データ１に対して発信データを０１とする符号化方式であり、これを用いることで光ビーコン装置が有効なデータを送信している間は、必ず光の輝度変化があることになり、前記イメージセンサ部５１を用いた４フレーム間の輝度変化を調べる演算にて、点滅のパターンを受信するのに都合の良い符号化となることが分かる。図１９の例では、８ビットの"０１００１００１"のデータを送信する場合の符号化について説明しており、それぞれ1ビットが"１"の場合、"０１"に符号化され、１ビットが"０"の場合、"１０"に符号化される。
【００８６】
本実施形態において、固定長８ビットのデータを送る場合を例に挙げて説明する。送信データの１データフレームは、図１９に示す送信データのフォーマットのように、スタートコード、送信データから構成される。前記スタートコードは、送信データの１フレームの先頭であることを受信側に知らせることを目的とするものであるので、例えばマンチェスタ符号化された送信データ部分には現れない"0001"というビットを割り当てる。また前記送信データは8ビットデータをマンチェスタコード符号化した１６ビットデータである。したがって、この１データフレームをＬＥＤの点滅で送信する場合に、送信データの１を点灯、０を消灯と割り当てて、光信号の点滅周期、本実施形態では４KHz(２５０μs毎)で点滅させる。これで８ビットのＩＤデータが送出されることになる。
【００８７】
このような符号化された点滅信号は、前述のＩＤ認識カメラのイメージセンサ部に入射し、イメージセンサ部は、ＩＤモード時には、演算ブロック単位（各画素または複数画素単位）において、光ビーコン装置からの点滅する光信号を受光して、（２）式のような４フレーム間の演算を行い、その演算結果をセンサ外に出力する。
【００８８】
[数２]
F(N)+F(N-1)-F(N-2)-F(N-3)・・・（２）
【００８９】
ただし、ここで実際には、この式のF(N),F(N-1),F(N-2),F(N-3)は本イメージセンサ内ではアナログ量として扱われ、その演算結果を外部から制御可能な閾値によって、２値化される。したがって、本イメージセンサの１つの動作モードとしては、（２）式の結果を、正の値の閾値で、２値化したものを出力することが可能である。すなわち、これは、閾値レベルをTHRとすると、
【００９０】
[数３]
F(N)+F(N-1)-F(N-2)-F(N-3) ≧ THRのとき、１
【００９１】
F(N)+F(N-1)-F(N-2)-F(N-3) < THRのとき、０・・・（３）
【００９２】
を出力する。これは、ＬＥＤの点滅の立ち上がりエッジを検出することになる。図２０に、前項で説明した送信データ(8bitデータ)を送った場合の、センサ出力を示す。このとき、センサのサンプルパルスは12KHzであり、12KHz単位で（３）式の演算結果を出力している。このセンサ出力を、図１５に示すＩＤデコード処理部６３に入力し、ＩＤ（ここでは光ビーコン装置からの8bitの送信データ）と座標を算出する。
【００９３】
前述のように、画像モードとＩＤモードは、画像とＩＤを交互にモードを切り替えて、ＩＤ認識カメラを実現している。例えばこれを30frame/secで切り替えると、このＩＤ認識カメラは、画像を15fps,ＩＤを15fpsで取得可能なカメラとなる。ここで、画像モードの区間では、画像デコード処理を行いつつ、前ＩＤモード区間でデコードした結果（ＩＤ）をＩＤ認識カメラから出力することができる。またＩＤモード区間では、ＩＤデコード処理を行いつつ、前画像デコード区間でデコードした結果（画像）をＩＤ認識カメラから出力することができる。本実施形態では、このＩＤモードは、図１６に示すようにＩＤデコード処理区間Ｐ１とＩＤ重心計算区間Ｐ２の２つに分けている。
【００９４】
ここで図２０を参照しながら、ＩＤデコード方法について説明すると、光ビーコンが固定長8bitのデータ(01001001)を送信する場合、イメージセンサ部は、ＩＤモード時には、前項で説明したように立ち上がりエッジを検出する演算を行い、センサ出力は図２０のようになる。以下、簡単のため、センサの或る１画素（または１演算ユニット）に注目して、ＩＤデコードの手順を示す。
【００９５】
また本実施形態では、固定長8bitで説明しているが、ビット数を拡張することは、本実施形態で開示したものから、当業者は容易に行うことができるものである。また、固定長ではなく、ビット長を可変にしたい場合でも、スタートコードだけではなく、エンドコードを設け、その検出手段（スタートコードと同様の方法）を付加するだけで、実現される。
【００９６】
先ず、スタートコードの検出は、ＩＤデコードの最初の手順として、送信データのスタートコードを検出する。スタートコードは、本実施形態では"0001"と定義したので、センサ出力としては、0が９回（イメージセンサの９フレーム分）以上出た後、初めて1を受信するとそれをスタートコードと見なすことができる。
【００９７】
続いてＩＤデータの受信は次のように進められる。スタートコードを受信を完了した次のフレームから６フレーム目のセンサ出力が、ＩＤデータの０ビット目のデータ（b0）すなわちＬＳＢとなる。そして、このb0から６フレーム毎のセンサ出力がb1, b2, b3, b4, b5, b6, b7となり、８ビットのデータを受け取ることができる。図２０では６フレームは６クロックとして描かれており、６クロック毎にセンサ出力がサンプリングされてＩＤデータの抽出が行われることになる。
【００９８】
本実施形態では、エラーチェック機能が備わっており、光ビーコンの点滅が、蛍光灯照明など光ビーコン以外のものと干渉しないように、光ビーコン以外のものによるセンサへの影響を排除する工夫を施している。すなわち、マンチェスタコード符号化したデータを、光ビーコンの点滅周波数fpの３倍の周波数fsでセンサで受信し、（３）式のような演算を行った場合、次のような法則が成立する。この法則について、図２１を参照しながら説明すると。先ず、b0〜b7の各ビットが"０"である場合、その"０"であるビットの３フレーム前のセンサ出力と６フレーム前のセンサ出力は、必ず反転している。この法則については、図２１では、矢印αで示しており、ｂ１が"０"であることから、その３クロック前のデータと６クロック前のデータが反転していれば正常で、反転していなければエラーとみなされることになる。次の法則はb1〜b7の各ビットが１である場合、３フレーム前のセンサ出力は必ず"０"であるというものであり、この条件を満たさない場合には、それがエラーと見なされる。図２１では、その法則を表す部分を矢印βで示している。ｂ６が"１"であることから、その３クロック前のデータが"０"であれば正常で、逆に"１"であればエラーとして処理されることになる。
【００９９】
次に、本実施形態におけるＩＤ重心計算について説明する。このＩＤ重心計算は前述のような方法で求めた各画素（または各演算モジュール）のＩＤから、そのＩＤの画像上の座標を求める方法であり、その一例をここで説明する。例えば、ＩＤの種類ごと（この場合、ＩＤは８ビットであるから、２５６種類）に、そのＩ，Ｊ座標の和と個数を求め、各Ｉ，Ｊ座標の和を個数で割ると、各ＩＤごとの重心座標を求めることができる。光ビーコン毎にＩＤが異なる場合には、この方法で座標を計算することが可能である。
【０１００】
図２２はＩＤデコード処理部の内部構成図である。センサ出力はこのブロックの入力として、ＩＤデコード回路８１に供給され、このＩＤデコード回路８１は、各画素のＩＤを前述のＩＤデコード手順で示した手順でセンサ出力より復号化する。本実施形態では、ＩＣ，ＡＳＩＣ，ＦＰＧＡなどのデバイスで実現することを念頭において説明をするが、処理能力が十分であれば、マイクロプロセッサやＤＳＰによってソフトウェアで実現することも可能である。
【０１０１】
センサ出力を受けるＩＤデコード回路８１は、フラグレジスタ８２とデータレジスタ８３に接続され、ＩＤデコード回路８１でデコードしたデータが送られる。データレジスタ８３は、デコード途中またはデコードを完了したＩＤを記憶しておくためのレジスタであり、フラグレジスタ８２はＩＤをデコードするために必要なフラグ類を記憶するためのレジスタである。図２３に各レジスタの詳細を示す。フラグレジスタ８２は、最初の１ビットがスタートフラグであり、次の４ビットがゼロカウンタ値である。その次の３ビットが位相カウンタ値であり、次のビットがエンドフラグであり、以下、ＩＤカウンタ値と半位相前ビットが続く。なお、ＩＤカウンタ値は次に受け取るデータが何ビット目のデータかを示す値であり、例えば１であれば、次はb1を受信する順番であることを示すことになる。また、データレジスタ８３は８ビットのデータレジスタであり、ｂ０〜ｂ７までの８ビットのデータが並ぶことになる。
【０１０２】
フラグレジスタ８２のデータとデータレジスタ８３のデータは、フレームメモリ８５に送られて蓄積される。フレームメモリ８５は、センサの各画素または各演算モジュール毎にデコードしたデータ（ＩＤ）とフラグを格納するメモリであって、センサが１画素につき１演算モジュールをもつ場合には、センサの画像サイズであるＭ×Ｎと同じサイズとなり、データ幅は図２３に示すようにデータレジスタ８３とフラグレジスタ８２のビット幅の和、すなわち、本実施形態では、データ幅としては２１ビット以上で構成される。
【０１０３】
図２２のタイミング制御部８４は、ＩＤデコード処理部６３全体のタイミング制御を行うモジュールである。具体的には、ＩＤデコード回路８１とセンサ出力のタイミングの同期をとり、所望のセンサ出力に対して、対応するフレームメモリ８５のデータをデータレジスタ８３、フラグレジスタ８２にロードして、ＩＤデコード処理を進めるためのタイミング信号を生成して供給する。また、このときフレームメモリ８５へのアドレス信号、リードライトなどの制御信号を生成する。また、タイミング制御部８４は、ＩＤ重心計算回路８６、ＩＤレジスタ８７、ＩＤ座標格納メモリ８８のタイミング制御を行う。
【０１０４】
図２４を参照して、ＩＤデコード処理の流れを説明する。画像モードとＩＤモードが交互に送信される信号においてＩＤモードに入ると、まずは手順Ｓ６１においてフレームメモリ８５をすべて初期化して、"０"を書き込む。またタイミング制御部８４に内蔵されているフレームメモリ８５用のアドレスカウンタを初期化する。
【０１０５】
このような初期化の後、手順Ｓ６２でフレームメモリ８５からレジスタへの読み出しが行われる。すなわち、タイミング制御部８４がフレームメモリ８５にアドレス信号、リード信号及びチップセレクト信号を与え、現在ＩＤデコード回路が受け取っているセンサ出力に対応する座標のフラグ情報及びデータ（ＩＤ）情報を読み出し、それぞれフラグ情報をフラグレジスタ８２に、データ（ＩＤ）情報をデータレジスタ８３に書き込む。初期化の直後の手順では、フレームメモリ８５からの出力データは"０"である。
【０１０６】
次に、手順S６３に進み、当該ＩＤデコード処理部としては、ＩＤデコードメイン処理が行われる。ここでＩＤデコードメイン処理の具体的な手順を図２５に示す。先ず、手順Ｓ７１でフラグレジスタ８２内のエンドフラグが"１"であるかどうかが調べられる。もしエンドフラグが"１"である場合には、既に現在のＩＤモード期間中に光ビーコン装置からのデータを受信完了したので、何も処理せずに当該ＩＤデコードメイン処理を抜ける。もしエンドフラグが"０"である場合には、ＩＤデコードメイン処理を続け、手順Ｓ７２に進む。
【０１０７】
手順Ｓ７２では、フラグレジスタ８２内のスタートフラグが"１"か否かが調べられる。もしフラグレジスタ８２内のスタートフラグが"０"の場合には、次に説明するスタートコード検出処理（手順Ｓ７３：図２６参照）を行った後、ＩＤデコードメイン処理を抜ける。もしフラグレジスタ８２内のスタートフラグが"１"の場合には、後述のデータ受信処理（手順Ｓ７４）を行った後、ＩＤデコードメイン処理を抜ける。ＩＤデコードメイン処理を抜けた時点で、図２４の流れの手順Ｓ６４に戻ることになるが、図２４に示す手順Ｓ６４以下については後述する。
【０１０８】
図２６を参照しながら、図２５の手順Ｓ７３の処理であるスタートコード検出処理について説明する。最初に、フラグレジスタ８２内のゼロカウンタ値が"９"であるかどうかを調べる（手順Ｓ８１）。ゼロカウンタは、スタートコード"0001"を検出するための前半の処理に必要であり、イメージセンサの周波数が光ビーコン信号の点滅周波数の３倍であることから、まずは"０"を９回以上受け取っていることが、スタートコードの条件となる。ゼロカウンタ値が"９"以下であれば（ＮＯ）、ゼロカウンタ値を１増加させて（手順Ｓ８２）、スタートコード検出処理を抜ける。またゼロカウンタ値が９であれば（ＹＥＳ）、スタートコード検出処理を続ける。
【０１０９】
続いて手順Ｓ８３で現在センサから来た値が１かどうかを調べる。もし入力が"０"であれば、まだスタートコードが来ていないと判断できるので、何もせずにスタートコード検出処理を抜ける。もし入力が"１"であれば、スタートコードを受信したと判断して、スタートフラグを"１"に設定した後（手順Ｓ８４）、スタートコード検出処理を終える。
【０１１０】
次に、データ受信処理について図２７を参照しながら説明する。データ受信処理は先のスタートフラグが"１"の場合（図２５参照）の手順であり、このデータ受信処理では、初めに、手順Ｓ９１で位相カウンタ値が"４"であるかどうかが調べられる。本実施形態では、前述のように受信データのエラーチェックが行われる。その場合に受信データの各ビットの３フレーム前の受信データが使用され、このエラーチェックのタイミングが位相カウンタの値が"４"の時と同期するように設定されている。手順Ｓ９１では、位相カウンタが"４"であるかどうかを調べ、もし位相カウンタ値が"４"であれば（ＹＥＳ）、半位相前ビットにセンサ出力を書き込む（手順Ｓ９７）。もし位相カウンタ値が"４"でなければ（ＮＯ）、データ受信処理を続け、手順Ｓ９２に進む。
【０１１１】
前述のように、イメージセンサの周波数が光ビーコン信号の点滅周波数の３倍であることから、６クロック毎にデータを受信すれば良い。この位相カウンタは０〜７の値をとり、位相カウンタの値が"７"のときにセンサ出力が、光ビーコン装置からの点滅データから送信データを復元したものになる。そこで手順Ｓ９２では位相カウンタ値が"７"であるかどうかを調べる。もし位相カウンタ値が"７"でなければ（ＮＯ）、手順Ｓ９８に進み位相カウンタを１増加させて、データ受信処理を抜ける。もし位相カウンタ値が"７"であれば（ＹＥＳ）、次のエラーチェック処理に進む。
【０１１２】
ここでエラーチェック処理について図２８を参照しながら説明する。このエラーチェック処理は図２７の手順Ｓ９３の具体的な内容に該当する。初めに、図２８の手順Ｓ１０１で、入力（センサ出力）が"１"であるかどうかを判定する。もし入力が"１"だった場合には、フラグレジスタ８２内の半位相前ビットが"０"であるかどうかを調べ（手順Ｓ１０２）、半位相前ビットが"０"であれば正常（ＯＫ）であり、半位相前ビットが"１"であればエラー発生（ＮＧ）とする。この手順Ｓ１０２の判断は、図２１の矢印βに対応する判断であり、半位相前ビットは３クロック前のデータである。
【０１１３】
もし手順Ｓ１０１で入力が"０"だった場合には、手順Ｓ１０３に進み、半位相前ビットと１ビット前のデータの比較が行われる。１ビット前のデータはデータレジスタ内のデータであり、どのビットかはＩＤカウンタ値からわかる。ただしＩＤカウンタ値が"０"の場合には、１ビット前のデータは"１"とする。手順Ｓ１０３の判断では、同じでなければ即ち反転していれば-正常（ＯＫ）であり、同じであればエラー発生（ＮＧ）とされる。この比較は図２１の矢印αに対応する判断であり、半位相前ビットは３クロック前のデータであり、１前ビットは６クロック前のデータである。
【０１１４】
このようなエラーチェック処理が行われた後、再びデータ受信処理について図２７を参照しながら説明する。エラーチェック処理の結果、もしエラーチェック処理がＮＧであれば、手順Ｓ９６でフラグレジスタ８２、データレジスタ８３に"０"を設定する。逆にエラーチェック処理の結果が、もし正常（ＯＫ）であれば、手順Ｓ９４でＩＤカウンタ値を調べて、入力（センサ出力）をデータレジスタ８３の記録するべき場所に記録する。続いて位相カウンタの値を"１"に設定し、ＩＤカウンタの値を１つ増加させる（手順Ｓ９５）。この処理でデータ受信処理を終えて、データがエラーの無い場合に限り、入力データがデータレジスタ８３に書き込まれることになる。
【０１１５】
このデータ受信処理の終了により、ＩＤデコードメイン処理（図２５）も終了し、図２４のＩＤデコード処理の手順Ｓ６４に戻る。この手順Ｓ６４では、フラグレジスタ８２とデータレジスタ８３の内容がフレームメモリ８５に書き込まれることになる。このときのアドレスは、タイミング制御部８４にあるアドレスカウンタ値である。
【０１１６】
各レジスタ８２、８３からフレームメモリ８５への書き込みが行われた後、デコード完了時間に達しているか否かが判断される。先に図１６に示したように、この本実施例のＩＤ認識カメラでは、画像とＩＤを交互に取得するようになっており、それぞれの区間を画像モード、ＩＤモードと呼んでいる。このＩＤモードの区間も、ＩＤデコード処理区間、ＩＤ重心計算区間に分かれており、それぞれ時間を規定している。ここでデコード完了時間とは、ＩＤデコード処理区間を終える時間のことを指しており、その時間がきたかどうかの判断をこの手順Ｓ６５では行う。
【０１１７】
この時間の管理は、図２２に示すタイミング制御部８４が行う。もしデコード完了時間に達していれば、次のＩＤ重心計算処理（手順Ｓ６９）に進む。もしデコード完了時間になっていなければ、手順Ｓ６６でフレームが終了であるかどうかを調べ、終了であればアドレスカウンタを０に初期化し（手順Ｓ６８）、或いはフレームが終了でなければアドレスカウンタをカウントアップして（手順ＳＳ６７）、それぞれ手順Ｓ６２に戻り、次のフラグ及びデータ（ＩＤ）情報をフレームメモリ８５からレジスタ８２、８３へ転送してＩＤデコード処理を継続する。
【０１１８】
ここでＩＤ重心計算処理について、図２９を参照しながら説明する。先ず、初めにＩＤ座標格納メモリ８８を全てデータ"０"にするように初期化する（手順Ｓ１１１）。次に、手順Ｓ１１２ではフレームメモリ８５からＩＤレジスタ８７へのデータの読み出しが行われる。この時、データ制御部からの制御信号によってフレームメモリ８５からデータ（ＩＤ）情報を読み出し、ＩＤレジスタ８７に書き込みがなされる。
【０１１９】
次に、手順Ｓ１１３で、その読み出したＩＤに相当するデータをＩＤ格納座標メモリ８８から読み出す。ＩＤ格納座標メモリは、図２２に示すように、本実施形態では、送信データを８ビットとしているので２５６種類のＩＤの情報を順番に格納したもので、それぞれのＩＤの情報とは、各ＩＤの重心計算に必要な「Ｉ座標の和」「Ｊ座標の和」「個数（ＩＤが検出された画素数）」から成る。したがって、ここでは、ＩＤレジスタ８７に格納されたＩＤから、それにＩＤに対応する「Ｉ座標の和」「Ｊ座標の和」「個数（ＩＤが検出された画素数）」をＩＤ重心計算部８６に読み出すことになる。
【０１２０】
ＩＤ重心計算部８６はＩ座標の和、Ｊ座標の和、個数＋１を計算で求めるように作動する（手順Ｓ１１４）。すなわち、「Ｉ座標の和」は「Ｉ座標の和」にフレームメモリ８５のＩ座標値を加えた値に更新され、「Ｊ座標の和」は「Ｊ座標の和」＋フレームメモリ８５のJ座標値を加えた値に更新され、「個数（ＩＤが検出された画素数）」は「個数（ＩＤが検出された画素数）」を１つ増加させた値に更新される。その計算結果はＩＤ重心計算部８６からＩＤ座標格納メモリ８８に書き込まれる（手順Ｓ１１５）。タイミング制御部８４がＩＤレジスタ８７の内容を、対応するＩＤ座標格納メモリ８８に書き戻す。
【０１２１】
次に手順Ｓ１１６に進み、タイミング制御部８４のアドレス信号によって、フレームメモリ８５の全座標読み出しを終了したか否かが判断される。もし終了していなかったら、タイミング制御部８４は、そのアドレスカウンタの値を増加させて（手順Ｓ１１７）、フレームメモリ８５から次のアドレスのＩＤデータを、ＩＤレジスタに読み出し、ＩＤ重心計算処理を継続する。手順Ｓ１１６で、もしフレームメモリ８５の全座標読み出しを終了していれば、ＩＤ重心計算処理を終える。
【０１２２】
なお、本実施形態では、これまでにＩＤ認識カメラ内では、「Ｉ座標の和」「Ｊ座標の和」「個数（ＩＤが検出された画素数）」を計算するところを示したが、各ＩＤの重心座標は、これらの値をＩＤ認識カメラから受け取れば次式で簡単に計算できる。
【０１２３】
[数４]
重心座標（Ｉg，Ｊg）＝（Ｉ座標の和、Ｊ座標の和）／個数
【０１２４】
また各ＩＤの位置は、重心以外にも、外接四角形の中心など他の手法も用いることも可能である。
【０１２５】
このような重心座標の計算によって、ＩＤ情報を発しながら大きく移動する被写体についても確実にＩＤ認識カメラの画角内に捉えることができ、被写体を自動的に追尾して確実に撮影できることになる。
【０１２６】
上述の実施形態では、ＩＤデコード処理をセンサの出力順序によって順次行うように説明し、例えばラスタスキャンでセンサ出力がされるならば、ラスタスキャンで読み出し、１つのＩＤデコード回路で復号化をするように構成するが、これに限らずＩＤデコード処理を高速に行いたい場合には、並列化を図り各ライン毎にＩＤデコード回路（Ｎ個）をもつことで、Ｎ倍高速な回路構成が実現できる。また回路規模は大きくなるが、各画素毎にＩＤデコード回路を持っても良い。
【０１２７】
また、上述の実施形態では、信号処理方法としてマンチェスター符号化を用いてエラー検出を行うようにしているが、他の符号化方式を用いても良く、エラー検出だけではなく、エラー訂正などを行うように構成することも可能であり、また、１つの被写体に複数のＩＤを割り当て、確実な追尾を図るような撮影方法も可能である。
【０１２８】
【発明の効果】
上述のように、本発明の移動撮影システム及び移動撮影方法によれば、ＩＤ認識カメラを搭載した飛行体などの移動体によって被写体を撮影することができるため、移動する被写体を自動的に追尾して確実に撮影できる。また、光ビーコン装置を操作して、無人の飛行体などに撮影プログラムや追尾方式の変更などの制御も可能であり、途中で撮影に支障が生じた場合でも柔軟な対応が可能である。また、カメラで複数の被写体を追う場合であっても、それぞれに被写体にＩＤ情報を付与し、且つそれらを位置情報と共に自動的に把握することができるため、移動する複数の被写体であっても確実な撮影が可能である。
【図面の簡単な説明】
【図１】本発明の移動撮影システムの一例の構成を示す模式図である。
【図２】本発明の移動撮影システムの一例の移動体側の構成を示すブロック図である。
【図３】本発明の移動撮影システムにおける光ビーコン装置の一例を示す模式的な外観斜視図である。
【図４】図３の光ビーコン装置の内部回路構成を示すブロック図である。
【図５】本発明の移動撮影システムの一例の一般的な動作を説明するためのフローチャートである。
【図６】本発明の移動撮影システムの一例における飛行体の位置変更動作を説明するためのフローチャートである。
【図７】本発明の移動撮影システムの一例における撮影方法の変更動作を説明するためのフローチャートである。
【図８】本発明の移動撮影システムの一例における自動撮影時の追尾方法を説明するためのフローチャートである。
【図９】本発明の移動撮影システムの一例における自動撮影時のフレーミングモードを説明するための模式図である。
【図１０】本発明の移動撮影システムの一例における自動撮影時の距離・角度モードを説明するための模式図である。
【図１１】本発明の移動撮影システムの一例における自動撮影時のズームモードを説明するための模式図である。
【図１２】本発明の移動撮影システムの一例における自動撮影時の撮影方向モードを説明するための模式図である。
【図１３】本発明の移動撮影システムの一例における自動撮影時の追跡方法について説明するためのフローチャートである。
【図１４】本発明の移動撮影システムの一例における光ビーコン装置からの信号を捕捉しながら追尾する際の概念図である。
【図１５】本発明の移動撮影システムの一例におけるＩＤ認識カメラの構成例を示すブロック図である。
【図１６】本発明の移動撮影システムの一例における画像モードとＩＤモードの関係を示す図である。
【図１７】本発明の移動撮影システムの一例における画像出力のためのＡ／Ｄ変換動作を示す図である。
【図１８】本発明の移動撮影システムの一例における装置の構成例を示すブロック図である。
【図１９】本発明の移動撮影システムの一例における送信データのフォーマットを示す図である。
【図２０】本発明の移動撮影システムの一例におけるＩＤデコード処理を示すタイミングチャートである。
【図２１】本発明の移動撮影システムの一例におけるＩＤデコード処理中のエラーチェック処理を示すタイミングチャートである。
【図２２】本発明の移動撮影システムの一例におけるＩＤデコード処理部の構成を示すブロック図である。
【図２３】本発明の移動撮影システムの一例におけるレジスタの構成を示すブロック図である。
【図２４】本発明の移動撮影システムの一例におけるＩＤモード時の処理手順を示すフローチャートである。
【図２５】本発明の移動撮影システムの一例におけるＩＤデコードメイン処理を示すフローチャートである。
【図２６】本発明の移動撮影システムの一例におけるスタートコード検出処理を示すフローチャートである。
【図２７】本発明の移動撮影システムの一例におけるデータ受信処理を示すフローチャートである。
【図２８】本発明の移動撮影システムの一例におけるエラーチェック処理を示すフローチャートである。
【図２９】本発明の移動撮影システムの一例におけるＩＤ重心計算処理を示すフローチャートである。
【符号の説明】
２ 光ビーコン装置
３ ユーザー
１０ 無人ヘリコプター
１１ ＩＤ認識カメラ
１２ メインローター
１３ テールローター
２０ 撮像部
２１ ＩＤカメラ処理部
２３ メモリ
２４ 飛行体制御部
２５ 飛行動力部
２６ 画像制御部
２７ 表示部
２８ 入力部
２９ 外部出力部
３０ ネットワークインターフェイス
３１ 光ビーコン装置
３２ 保持部
３３ 光ビーコン装置本体
３４ キー入力部
３５、３６ 発光ダイオード
３７ マイコン
３８ 電源
３９ キー入力部
４０ ＬＥＤ駆動部
４１ メモリ
８１ デコード回路
８２ フラグレジスタ
８３ データレジスタ
８４ タイミング制御部
８５ フレームメモリ
８６ ＩＤレジスタ
８７ ＩＤ重心計算回路
８８ ＩＤ座標格納メモリ[0001]
BACKGROUND OF THE INVENTION
INDUSTRIAL APPLICABILITY The present invention is used in a moving shooting system and a moving shooting method for automatically tracking and shooting a moving subject, and the system. Imaging device About.
[0002]
[Prior art]
Conventionally, when going to a scenic place, it is often performed to photograph a person with a still camera or a video camera against a landscape. When a device such as a tripod is not used, the still camera or video camera is configured to be operated by the photographer with his / her hand. I can't see it in the video. In addition, when one of the accompanying family members and friends takes a commemorative photo as a photographer, the person who becomes the photographer needs to sacrifice some time to enjoy the scenery of the scenic spot. Also, when you give a camera to someone else and ask you to take a picture, it takes time to search for a third party who can take the picture, and depending on the model, the camera operation is not easy and you can't expect a photo of the desired quality. There is also.
[0003]
As a proposal for solving such various problems, there is a method in which a camera is fixed by a tripod or the like and a picture is taken by a timer or remote control. In such cases, the photographer needs to set the camera timer on the tripod to start operation each time it is shot, and the subject needs to take into account the timer time. Need to do. As another solution, there is a technique disclosed in, for example, “Guidance shooting system” (see, for example, Patent Document 1). According to this publication, an apparatus called a guidance shooting system at a place where a scenic spot is determined. It is possible for general customers to take pictures with the camera installed in this system.
[0004]
[Patent Document 1]
Japanese Patent Laid-Open No. 9-233465
[0005]
[Problems to be solved by the invention]
However, in the above-described guidance photographing system, since the photographing is performed with the provided camera, it is not possible to arbitrarily select a photographing place. For example, when photographing a person sliding on a ski resort or climbing a mountain. It was impossible to take a picture of a moving person, such as when shooting a person.
[0006]
In addition, there is an imaging system that automatically tracks a subject by automatically controlling the direction of the camera by rotating a pan head on which the camera is installed, as a system for chasing another moving subject. In this case, shooting is impossible when a human subject moves to a position where it cannot be seen from the rotation range of the camera pan head or moves to a location far away from the shooting system. Furthermore, with these conventional technologies, the person who is the subject enters the camera's field of view and is photographed, so it is necessary to be conscious of being photographed, and the person's natural facial expressions and gestures can be photographed. In general, there is no method for reliably recognizing a moving person, and it is not easy to take a picture while automatically tracking the person.
[0007]
SUMMARY An advantage of some aspects of the invention is that it provides a moving shooting system and a moving shooting method that can automatically track a moving subject and reliably shoot.
[0008]
[Means for Solving the Problems]
In view of the technical problems described above, the mobile photographing system of the present invention is mounted on a moving body that is mounted with a photographing device and is movable. Formed by the light flashing pattern A signal generator for generating an optical signal The imaging apparatus restores the blinking pattern from the imaging unit, an image mode in which image data as a scene is acquired from the signal obtained by the imaging unit, and the signal obtained by the imaging unit. And a processing unit that repeats in a time division manner an ID mode that is a mode for receiving the optical signal. It is characterized by that.
[0009]
In the present invention, first, a photographing device for photographing a subject is mounted on a movable body. Therefore, unlike the method of controlling the direction of the camera from a fixed point, even when a large movement is involved, such as a person sliding on a ski resort or a person climbing a mountain, the tracking can be reliably performed. Furthermore, the imaging device mounted on the moving body is An image mode for acquiring image data as a scene; Generated by signal generator Since the blinking pattern forming the optical signal is restored, the ID mode for receiving this optical signal is repeated in a time-sharing manner. It is possible to send a control signal for photographing, an identification signal for recording a photographed image, and a signal for moving the moving body without requiring a photographer on the photographing apparatus side.
[0010]
In addition, the moving shooting method of the present invention includes: Formed by light flashing pattern Attach a signal generator that generates optical signals to the subject, An image mode which is a mode for acquiring image data as a scene from the image obtained from the image pickup unit, the signal obtained by the image pickup unit, and the light signal is received by restoring the blinking pattern from the signal obtained by the image pickup unit. An imaging apparatus having a processing unit that repeats the ID mode, which is a mode to perform, in a time-sharing manner, is mounted on the moving body Move the subject and move forward Transfer Actuate the moving body, During the ID mode The optical signal from the signal generator is detected, the moving body tracks the subject based on the detected optical signal, and the subject is detected by the photographing device. During the image mode It is characterized by shooting.
[0011]
By attaching the signal generation device to the subject, the optical signal from the signal generation device can be transmitted from the subject, for example, even when the photographer who is the subject performs setting and control of shooting. By operating the signal generator, it becomes possible to control the moving body. Since the photographing apparatus is attached to the moving body, tracking can be reliably performed even when a large movement is involved. As a control for tracking this subject, an optical signal from a signal generator can be used. photograph Equipment Image mode for acquiring image data as a scene And receiving optical signals from signal generators Repeat ID mode in time division Thus, the subject can be automatically tracked and reliably photographed.
[0012]
In addition, another mobile shooting method of the present invention is as follows. Formed by light flashing pattern Attach a signal generator that generates optical signals to the subject, An image mode which is a mode for acquiring image data as a scene from the image obtained from the image pickup unit, the signal obtained by the image pickup unit, and the light signal is received by restoring the blinking pattern from the signal obtained by the image pickup unit. A processing unit that repeats the ID mode, which is a mode to perform, in a time-sharing manner After mounting the photographing device on the moving body, the subject is moved and the photograph In the equipment During the ID mode in front Light The mobile body is controlled based on control information included in the optical signal by capturing a signal.
[0013]
For moving objects photograph The device is installed, Image mode for acquiring image data as a scene From the signal generator attached to the subject Light of Receive signal Repeat ID mode in time division . The optical signal can include a control signal. By using the control signal for controlling the moving body, the moving body can be controlled from the subject side.
[0014]
In still another mobile shooting method of the present invention, Formed by light flashing pattern Attach a signal generator that generates optical signals to the subject, An image mode which is a mode for acquiring image data as a scene from the image obtained from the image pickup unit, the signal obtained by the image pickup unit, and the light signal is received by restoring the blinking pattern from the signal obtained by the image pickup unit. A processing unit that repeats the ID mode, which is a mode to perform, in a time-sharing manner After mounting the photographing device on the moving body, the subject is moved and the photograph In the equipment During the ID mode in front Light Based on the control information contained in the optical signal. Select the automatic shooting mode when tracking the subject by the moving object. It is characterized by being.
[0015]
In this moving shooting method, An image mode for acquiring image data as a scene and an ID mode for receiving an optical signal from a signal generator attached to a subject are repeated in a time-division manner by an imaging device attached to the moving body. Automatic shooting mode at the start of subject tracking Is based on the optical signal captured by the camera Choice Even if the subject is a photographer Selecting the automatic shooting mode at the start of tracking Can be performed reliably.
[0016]
In still another mobile shooting method of the present invention, Formed by light flashing pattern Attach a signal generator that generates optical signals to the subject, An image mode which is a mode for acquiring image data as a scene from the image obtained from the image pickup unit, the signal obtained by the image pickup unit, and the light signal is received by restoring the blinking pattern from the signal obtained by the image pickup unit. A processing unit that repeats the ID mode, which is a mode to perform, in a time-sharing manner After mounting the photographing device on the moving body, the subject is moved and the photograph In the equipment During the ID mode in front Light The signal is captured and associated with the recognition information included in the optical signal by the photographing device. Acquired during the image mode The processed image data is photograph It is recorded in a storage unit connected to the apparatus.
[0017]
In this moving shooting method, An image mode for acquiring image data as a scene and an ID mode for receiving an optical signal from a signal generator attached to a subject are repeated in a time-division manner by an imaging device attached to a moving body. Data is Optical signal captured by the camera In association with the recognition information contained in It can be recorded in a predetermined recording unit, which is convenient when, for example, editing is performed on image data while using corresponding recognition information later.
[0018]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, the mobile photographing system and the mobile photographing method of the present invention will be described with reference to the drawings.
[0019]
As shown in FIG. 1, the mobile photographing system according to an embodiment of the present invention uses an unmanned helicopter 10 equipped with an ID recognition camera 11 as a moving body, and a user 3 as a subject is a skier, A tag-shaped optical beacon device 2 is attached near the shoulder. That is, in this embodiment, the user 3 can record in the camera with the unmanned helicopter 10 that tracks his skiing appearance, and can view the captured data later.
[0020]
The user 3 attaches the ski board 4 to both feet via a metal fitting and ski boots (not shown), and a tag-like optical beacon device 2 is attached near the shoulder of ski wear worn by the user 3. As will be described in detail later, the optical beacon device 2 is an optical signal generation device that generates an optical signal. For example, the control of the unmanned helicopter 10 that is a moving body, the control of the ID recognition camera 11, and the recorded image data It is possible to assign an identification number to In FIG. 1, there is one user 3, but in the present embodiment, the number of users 3 is not limited to one, and when there are a plurality of users or an optical beacon device is attached as a person, It is also possible to combine objects, humans and animals, or only animals.
[0021]
The unmanned helicopter 10 is a flying body that functions as a moving body. In particular, in the mobile photographing system of this embodiment, an ID recognition camera 11 as a photographing device is mounted in front of the unmanned helicopter 10. The unmanned helicopter 10 is provided with a main rotor 12 and a tail rotor 13 that rotate in accordance with control from an aircraft control unit described later, and the unmanned helicopter 10 performs a desired flight by the operation of each of these rotors. It can be carried out. The main rotor 12 is a rotary blade that generates upward lift by rotation, and the tail rotor 13 is a rotary blade for canceling the reaction of rotation of the main rotor 12.
[0022]
The ID recognition camera 11 as an imaging device mounted on such an unmanned helicopter 10 functions as an imaging device for acquiring normal image data, and receives an optical signal generated by the optical beacon device 2 and receives the signal. It has a function that makes it possible to analyze the control command, recognition data (ID), etc. Here, the optical signal is formed by a blinking pattern of light emitting diodes, and is processed separately from image data photographed as a scene.
[0023]
FIG. 2 is a diagram showing an internal circuit configuration of the unmanned helicopter 10. The unmanned helicopter 10 is provided with an image pickup unit 20 having a structure for taking in light by, for example, a CMOS solid-state image pickup device, and a signal from the image pickup unit 20 is sent to an ID camera processing unit 21. The ID camera processing unit 21 has a function of separating image data captured using the signal obtained by the imaging unit 20 as a scene and the optical signal transmitted from the optical beacon device 2 and is included in the optical signal. It is possible to extract a signal for controlling the unmanned helicopter 10, a signal of an ID number of data captured by the imaging unit, or a signal regarding a shooting method in the imaging unit 20. The ID recognition camera 11 shown in FIG. 1 is configured by the imaging unit 20 and the ID camera processing unit 21, and a configuration example of the ID recognition camera 11 will be described in detail later with reference to FIG.
[0024]
The ID camera processing unit 21 is electrically connected to the memory 23 and the flying object control unit 24. For example, when the main rotor 12 and the tail rotor 13 of the unmanned helicopter 10 are driven by motors, the flying object control unit 24 sends the control signals sent to the motors of the main rotor 12 and the tail rotor 13 to the next flight power. Send to unit 25. The flight power unit 25 includes motors for the main rotor 12 and the tail rotor 13, actuators for controlling each blade of the rotor for hovering, forward and backward flight, and the like. Operates in response to a control signal from. Therefore, the ID camera processing unit 21 sends the control signal of the unmanned helicopter 10 included in the optical signal to the flight power unit 25 via the flying object control unit 24, and the flight power unit 25 controls the flight of the unmanned helicopter 10. Is done.
[0025]
The ID camera processing unit 21 described above is also connected to a memory 23 composed of a semiconductor memory device such as a DRAM or SRAM. The memory 23 uses image data taken as a scene by the imaging unit 20 as an ID from which data is extracted. It is configured to be stored together with a number. Thus, recording image data together with an ID number is convenient when displaying data while editing or selecting it. The memory 23 is connected to the image control unit 26 via a data bus, and in accordance with instructions from a microcomputer built in the image control unit 26, necessary memory data is output to an external output unit 29, a network interface 30, and an image display unit. 27 can be output. The external output 29 is a connection unit used when outputting image data to other devices after the flight is completed. For example, a connection unit of a pin plug terminal, a DV terminal, a USB port, or Ethernet (registered trademark) for LAN. , RS232C, or a storage device that can use a card-like or stick-like storage medium. The network interface 30 has a function capable of connecting to the Internet or the like wirelessly. For example, short-range wireless communication using spread spectrum communication such as Bluetooth (standard name: trade name) or IEEE (Institute of Electronic) of a wireless LAN. and Electronics Engineers) 802.11 specifications and other communications. By using such a network interface 30, it is also possible to distribute image data via the Internet. For example, the user 3 returns home after taking a picture at a ski resort and skis himself / herself or friends via the Internet. The image data can be viewed where the user is doing, and such image data can be distributed to acquaintances and family members. The image display unit 27 is a device that functions as a small monitor mounted on the unmanned helicopter 10, and for example, a liquid crystal display device, a diode array display, a plasma display, or the like is used. By providing such an image display unit 27, it is possible to easily confirm whether or not editing or re-shooting is necessary at the shooting site. Although not shown, the image display unit 27 is attached to the unmanned helicopter 10 with a structure that can be folded during flight or a shutter that is closed, and the image display unit 27 is viewed from the outside when the image display unit 27 is viewed. Is ready to do.
[0026]
The image control unit 26 is further connected to an input unit 28. The input unit 28 is configured by a switch or a numeric keypad for switching the output of image data between the external output unit 29 and the network interface 30 and controlling the display of the image display unit 27. The input unit 28 may receive a signal from a remote controller for editing after landing of the unmanned helicopter 10, for example.
[0027]
In the unmanned helicopter 10 shown in FIG. 2, the image data of the scene and the optical signal from the optical beacon device 2 are separated from the image captured by the imaging unit 20 from the internal circuit configuration by the ID camera processing unit 21, and each flying object It is possible to assign an ID number to each image data for controlling the flight of the unmanned helicopter 10 and image processing, and to control the photographing method.
[0028]
Next, the optical beacon device 31 will be described with reference to FIG. The optical beacon device 31 used in the present embodiment is an optical signal generation device, and is attached to, for example, a part of the clothes of the user 3 and enables necessary control over the unmanned helicopter 10.
[0029]
First, as shown in FIG. 3, a key input unit 34 is formed on the surface of a rectangular and substantially flat optical beacon device body 33, and the light emitting diode 35 also has a light emission direction on the surface. It is provided on the front side. The optical beacon device main body 33 is provided with a holding part 32 extending from one side surface to the back side like a tag, and is attached to a part of clothes of the user 3 such as a shoulder or a hat using the holding part 32. . The holding part 32 has a belt-like shape and can be attached by passing the holding part 32 through the insertion part when the belt-like part is closed and the insertion part is part of the clothes of the user 3. (Product name) may be detachably attached. In addition, the neck portion of the clothes has a portion that becomes a ring for hanging the tag of the normal product or the clothes, and the holding portion 32 can be passed through the portion so that it does not fall off from the user 3. . In the key input unit 34, for example, a preset input code can be input to perform various controls such as control of the unmanned helicopter 10, imaging method control, and image processing control. Thus, it becomes possible to output a required optical signal.
[0030]
FIG. 4 is a diagram showing an internal circuit configuration of the optical beacon device 31. This internal circuit is configured with the microprocessor 37 as the center. The microprocessor 37 selects a light signal pattern stored in the memory 41 in response to a signal from the key input unit 39, and the LED is driven according to the selection. A signal is sent to the unit 40 to cause the light emitting diode 36 to blink. The optical beacon device 31 is equipped with a power source 38 such as a button battery, a solar cell, or a fuel cell, and the power source 38 is configured to supply power to the microprocessor 37, the memory 41, and the LED driving unit 40. The
[0031]
The optical beacon device described with reference to FIGS. 3 and 4 is merely an example, and various modifications are possible. For example, the outer shape is rectangular and substantially flat and has been described as being attached to the clothes of the user 3. For example, a device in which a signal generating part such as a light emitting diode is attached to the upper surface of a hat or helmet, Use a part of clothing or shoes that incorporates a signal generation part such as a light emitting diode in advance, or provide a part of the signal generation part such as a light emitting diode of a decorative material such as a watch or necklace. It is also possible to provide a signal generating part on a ski, a board, or a part of the stock.
[0032]
The structure of the optical beacon device can also be configured with a simpler structure. For example, in the case where only simple user tracking is executed and the shooting method and image processing method are not changed, or when only a plurality of subjects are distinguished, the key input unit 39 or the like is not provided. Alternatively, a structure in which the light emitting diode 35 is disposed on the tag may be used.
[0033]
Next, the moving image capturing method of the present embodiment will be described according to the procedure with reference to FIG. The mobile imaging method described here is an example in which an optical beacon device is used to control a flying object such as an unmanned helicopter and an image is easily selected after imaging using an ID number.
[0034]
First, in step S1, the user wears the optical beacon device 31. The optical beacon device 31 is a rectangular and substantially flat device as shown in FIG. 3, and the holding portion 32 is inserted into, for example, the shoulder portion of the ski clothes of the user 3 and attached to the user 3.
[0035]
Subsequently, the user 3 operates the optical beacon device 31 to start the flight movement of the unmanned helicopter 10 which is a flying object (step S2). In this case, a key input for starting the flight is performed on the optical beacon device 31, and the light emitting diode 35 of the optical beacon device 31 is blinked in response to the key input. At this time, it is desirable that the position of the user 3 is at a position that can be captured by the ID recognition camera 11 of the unmanned helicopter 10, and the unmanned helicopter 10 takes off by flying the light emitting diode 35 at such a position. Start. Inside the unmanned helicopter 10, when the ID camera processing unit 21 extracts ID information and recognizes that the ID number corresponding to the flight start command has been received, the flight control unit 24 sends a flight start signal. Is sent from the flying vehicle control unit 24 to the flying vehicle power unit 25, and the flight of the unmanned helicopter 10 is started.
[0036]
When the unmanned helicopter 10 takes off, the ID recognition camera 11 always operates to capture a signal from the optical beacon device 31, although it may be accompanied by a flight operation such as rolling to some extent. That is, when the user 3 who is a subject suddenly moves or is hidden behind an obstacle, the ID recognition camera 11 temporarily does not receive a signal from the optical beacon device 31. However, the ID recognition camera 11 always detects the signal from the optical beacon device 31 and performs an operation of expanding the detected area and continuing to search when it is determined that there is no signal from the optical beacon device 31 in a certain direction. . When the subject moves when the ID recognition camera 11 operates, the unmanned helicopter 10 tracks the movement from the position of the optical beacon device 31. Therefore, the unmanned helicopter 10 can automatically and reliably track the user 3 while capturing the signal from the optical beacon device 31 even in a situation where the user 3 as a subject slides down the ski slope.
[0037]
The ID recognition camera 11 always operates so as to detect a signal from the optical beacon device 31, but actually, an image mode for capturing an image as a scene and an optical signal from the optical beacon device 31 are extracted. The ID mode is repeatedly switched. The signal processing in the image mode and the signal processing in the ID mode will be described later. When the ID recognition camera 11 recognizes the optical beacon device 31, the subject is photographed and recorded in the memory 23 together with the ID number (step S3). When the ID recognition camera 11 captures a subject, each of the plurality of subjects has an optical beacon device 31 and may generate an ID number as a signal. In such a case, images and ID numbers for a plurality of different subjects are associated with each other and stored in the memory 23, so that the processing can be quickly performed when the ID number is used for confirmation after shooting or when editing. .
[0038]
Next, in step S4, in the state where the photographing is finished, the optical beacon device 31 is subjected to key input indicating the end of flight of the flying object, and the optical beacon device 31 receives the key input and blinks the light emitting diode 35. The optical signal is sent to the imaging unit 20 of the unmanned helicopter 10. As a result, a signal is sent from the imaging unit 20 to the ID camera processing unit 21, from which a flight stop signal is sent to the flying object control unit 24, and each power unit is stopped from the flying object control unit 24 to the flying power unit 25. A signal is sent to stop the flight of the unmanned helicopter 10. When the unmanned helicopter 10 is at a high position in the air and receives a flight stop signal, the unmanned helicopter 10 is controlled to land by automatic driving.
[0039]
In this state, in the memory 23 of the unmanned helicopter 10, image data for the subject is recorded together with the ID number. In step S5, the user 3 controls the optical beacon device 31 to display an image having its own ID. To control. Specifically, the key input unit 34 of the optical beacon device 31 is operated in the same manner as described above, and a signal instructing image display is output from the optical beacon device 31.
[0040]
In response to the instruction from the user 3, an image related to the ID number of the user 3 is displayed. This display is performed using the image display unit 27 of the unmanned helicopter 10, and the image relating to the user 3 is displayed by a method of continuously feeding frames or arranging them in time series (step S6). While viewing the displayed image, the user 3 selects a necessary image in step S7. The image required here is an image that is stored and edited or used for processing such as printing. The images that can be selected may be still images or moving images. The image selection operation by the user 3 can use the key input unit 34 of the optical beacon device 31 or the input unit 28 of the unmanned helicopter 10.
[0041]
The image selected by the user 3 is sent from the external output unit 29 of the unmanned helicopter 10 to a required audio / video device (step S8). Alternatively, the image selected by the user 3 may be output from an external output unit 29 to which the image selected by the user 3 is sent to a device such as a personal computer, or transmitted to a server or other information terminal connected to the Internet via the network interface 30. You may do it.
[0042]
As described above, in the flow shown in FIG. 5, the operation of the unmanned helicopter 10 such as the start and end of the flight and the recording of the subject together with the ID number are operated remotely and automatically by operating the optical beacon device 31. It can be operated, and can be easily tracked even when the movement of the subject is high speed, and a plurality of subjects are photographed for each ID number by the ID number included in the optical signal from the optical beacon device 31. It is also possible to save it.
[0043]
In the above example, an example in which a flying vehicle equipped with a camera is an unmanned helicopter and its use is taken at a holiday destination such as skis, the mobile shooting system of this embodiment has other forms and uses. Is also applicable. Here, other forms and applications will be described as exemplified in Table 1 below.
[0044]
[Table 1]

[0045]
The ones listed in Table 1 are merely examples, but the moving body on which the camera is mounted is not limited to one that moves in the air, but moves in various places such as land, water, or underwater. What can be mentioned. In addition, the application is not limited to vacation use, and various applications such as sports recording, movies, and natural observation are possible. The place and the specific event are not limited and can be applied to various things. The subject to be photographed is not limited to a person, and may be various creatures, such as a car or a ship. Also, the shooting target is not limited to a single object, but may be a plurality. The optical beacon device 31 is a case where the subject is a human because of an operation of key input. For example, when the subject is an animal, an optical beacon device that outputs an optical signal without requiring an operation is an animal as the subject. Attached to.
[0046]
Next, an example of changing the flight schedule of the flying object by operating the optical beacon device will be described with reference to FIG. Even during shooting, in the mobile shooting system of this embodiment, the optical beacon device can communicate with a flying object such as an unmanned helicopter, and the flight schedule can be changed according to circumstances.
[0047]
First, as procedure S11, a user wears an optical beacon device. The optical beacon device may be a rectangular and substantially flat device as shown in FIG. 3, and may be another device that can output a required optical signal.
[0048]
Next, in step S12, the user operates the optical beacon device to start flying an air vehicle such as an unmanned helicopter. At this time, in this case, an operation for starting the flight is performed on the optical beacon device, and the light emitting diode of the optical beacon device is blinked in accordance with the operation. The blinking of the light emitting diode is captured by an ID recognition camera mounted on the flying object, and the flying object takes off and starts flying.
[0049]
After the flying object takes off, as described above, the user as a subject is automatically tracked by the flying object that has taken off and shooting starts. For example, in the setting before the take-off of the flying object, if the setting is such that the shooting is performed at an angle of 45 degrees from the back of the user 20 m, if shooting starts after the take-off of the flying object, Shooting starts according to the settings. Such a setting before takeoff can be changed in this flow. In step S13, the user operates the optical beacon device, and the optical beacon device outputs an optical signal for changing the position of the flying object. As an example of such a change in the position of the flying object, for example, the setting content of shooting at an angle of 45 degrees from the rear of the user 20 m in the setting before the take-off of the flying object is performed from the front 20 m before the user. You can change the setting to shoot at an angle of degrees.
[0050]
When the optical beacon device outputs an optical signal for changing the position of the flying object, the ID recognition camera of the flying object receives the optical signal from the optical beacon device and flies according to a control signal in the form of an ID number included in the optical signal. The middle flying object corrects its position (step S14). With this correction, for example, the flying object moves from the tracking state from behind 20 m to the front side of the user and continues shooting, and only the rear view of the user as the subject is shot by such a change of the shooting position. Therefore, it will be possible to change the user's facial expression. Such a change in the shooting position can be performed independently of the program before takeoff, and is extremely effective when the shooting position is moved in a shooting situation.
[0051]
Next, an example in which the photographing method is changed by operating the optical beacon device will be described with reference to FIG. In the mobile photographing system of the present embodiment, the optical beacon device can communicate with a flying object such as an unmanned helicopter even during photographing, and the photographing method can be changed according to circumstances.
[0052]
First, in step S21, the user wears an optical beacon device. The optical beacon device may be a rectangular and substantially flat device as shown in FIG. 3, and may be another device that can output a required optical signal.
[0053]
Next, similarly to step S12 of FIG. 6, in step S22, the user operates the optical beacon device to start flying an aircraft such as an unmanned helicopter. At this time, in this case, an operation for starting the flight is performed on the optical beacon device, and the light emitting diode of the optical beacon device is blinked in accordance with the operation. The blinking of the light emitting diode is captured by an ID recognition camera mounted on the flying object, and the flying object takes off and starts flying.
[0054]
After the flying object takes off, as described above, the user as a subject is automatically tracked by the flying object that has taken off and shooting starts. For example, if the setting before the take-off of the flying object is such that the entire body of the subject fills the frame of the image, when shooting starts, such an image from the user's head to the toes is displayed as an ID recognition camera. Will be filmed by. While such shooting is in progress, the optical beacon device sends an optical signal that changes the shooting method by operating the optical beacon device that the user wears according to the convenience of the user, environment, accident, etc. Send out (step S23). As an example of such a change in the photographing method, it is possible to change from a photographing method in which the entire body of the subject is filled with the frame of the image so that the photographing is performed in a state in which the camera is sufficiently drawn with a landscape. Some types of shooting methods can be changed without changing the position of the flying object, such as zoom-in and zoom-out lens system operations, adjusting the shutter speed for still images, adding filters, Various changes such as the control of the display of the clock are possible, and they can be performed by operating the optical beacon device.
[0055]
When the optical beacon device outputs an optical signal that changes the shooting method, the ID recognition camera of the flying object receives the optical signal from the optical beacon device, and performs shooting according to a control signal in the form of an ID number included in the optical signal. The method is modified (procedure S24). In other words, if the light signal is set so that the entire body of the subject fills the frame of the image, for example, the setting before take-off of the flying object, shooting with the camera fully drawn and the camera pulled To be corrected. Such a change can be easily performed when the camera is at hand, but when the camera is mounted on a flying object, the optical signal from the optical beacon device can be effectively changed. Will be available.
[0056]
Next, an example of a tracking method at the time of automatic shooting will be described with reference to FIGS. The control in the flowcharts of FIGS. 6 and 7 includes a case where the control is changed in the middle of tracking, but as described below, it is performed while tracking automatic shooting using an optical signal from the optical beacon device. It is also possible. In the present specification, “tracking” does not necessarily mean only the case of tracking from behind, but also includes the case where the subject is arbitrarily traced to the side or front to follow the subject.
[0057]
First, as shown in FIG. 8, the basic flow of tracking at the time of automatic shooting is as follows. A user wearing an optical beacon device operates the optical beacon device to start flying an aircraft such as an unmanned helicopter. (Step S31). Immediately after this start, since the attitude control at take-off takes priority over the capture of the optical signal of the optical beacon device, the flying object may lose sight of the optical signal of the optical beacon device. When the vehicle turns or the ID recognition camera is not fixed, the optical signal from the optical beacon device worn by the user is detected while changing the imaging range of the ID recognition camera (step S32). At this stage, the optical beacon device is gradually detected by the optical signal, but when the optical signal of the optical beacon device is captured by the ID recognition camera, the optical signal from the optical beacon device is discovered. At this stage, tracking for automatic shooting is started (step S33).
[0058]
At the start of tracking by this automatic photographing, a priority mode is selected. There are four modes that can be used in this system, and the four modes are a framing mode, a distance / angle mode, a zoom mode, and a shooting direction mode. The framing mode is a mode for setting where on the screen a subject is to be photographed, and settings such as left side, right side, center, upper side, and lower side can be set. FIG. 9 is a diagram illustrating an example of the framing mode. FIG. 9A illustrates an example in which the subject is disposed on the right side of the screen, and FIG. 9B illustrates an example in which the subject is disposed in the center of the screen. . In FIG. 9, the user 3 is captured so as to be captured in the frame 45, but the position of the user 3 is almost determined by the setting of the framing mode, and when the framing mode is the subject on the right side, When the state shown in (a) of FIG. 9 is set and the framing mode is the subject at the center, the state shown in (b) of FIG. 9 is set and photographing is performed.
[0059]
FIG. 10 is a diagram for explaining the distance / angle mode. The distance D between the unmanned helicopter 10 as a flying object and the user 3 wearing the optical beacon device 31 and the angle θ at which the unmanned helicopter 10 flies are set, and the set distance D is maintained at the angle θ. Tracking is performed as follows. The distance D and the angle θ are data that the unmanned helicopter 10 can always grasp from the optical signal from the optical beacon device 31. The deviation from the set value is always corrected, and photographing is continued.
[0060]
FIG. 11 is a diagram for explaining the zoom mode. FIG. 11A shows a screen when the subject is zoomed down, and FIG. 11B shows the subject zoomed in. This is the screen when The zoom mode is not limited to the two-stage zoom shown in FIGS. 11A and 11B, but may be a non-stage or two or more stages. When the subject is zoomed in and photographed as shown in FIG. 11A, the user 3 as the subject is photographed relatively small with respect to the frame 45 so that the scenery occupies a large area on the screen. When the subject is zoomed in and photographed as shown in FIG. 11B, the user 3 is photographed so as to occupy a relatively large area with respect to the frame 45.
[0061]
FIG. 12 is a diagram for explaining the shooting direction mode, in which direction from which the subject is shot can be set. In this shooting direction mode, for example, selection from the top, from the front, from the side, from the back is possible. When the setting is set from the front, the ID recognition camera is viewed from the front of the user 3 who is the subject. The video will continue to be taken automatically.
[0062]
In addition, chasing a moving object or losing sight of a light signal when automatic shooting is continued is an event that can occur, and the mobile shooting system of this embodiment can handle such a case, In particular, it is possible to predict the moving speed and perform imaging with an accurate imaging direction and angle. FIG. 13 is a flowchart for explaining a tracking method during automatic photographing.
[0063]
First, assuming that a flying object such as an unmanned helicopter has started flying, the flying object turns 360 degrees around and searches for an optical signal from the optical beacon device (step S41). For this optical signal search, an ID recognition camera is used. Specifically, an ID signal is extracted by the ID camera processing unit 21 in the circuit as shown in FIG. If an optical signal from the optical beacon device is not found in step S42 (NO), the process proceeds to step S43, and the search number RN is incremented by one. Next, in step S44, it is determined whether or not the search count RN has reached the maximum search count F. If the maximum search count F has not been reached (NO), the search for the optical signal from the optical beacon device is repeated once again. Therefore, control returns to step S41, and the flying object turns 360 degrees around and searches for an optical signal from the optical beacon device. If the number of searches RN has reached the maximum number of searches F in step S44 (YES), the flying object will land (step S45) and the process will be terminated.
[0064]
If an optical signal from the optical beacon device is found in step S42 (YES), the ID signal included in the optical signal is read (step S46). By this reading operation, the ID signal is extracted by the ID camera processing unit, and it is determined whether or not it is a signal corresponding to the target ID number of the subject (step S47). That is, when tracking a plurality of subjects at one time with a single flying object, each subject holds a required optical beacon device, and each optical beacon device includes a different ID signal, so that the subject is distinguished. It can be recognized by the ID signal.
[0065]
If the read ID signal is not that of the target subject (NO), the procedure returns to step S41, and the flying object again turns around 360 degrees to search for the optical signal from the optical beacon device, and Repeat the control. If the ID signal obtained as a result of reading in step S47 is that of the target subject (YES), the flying object has captured the correct subject, and the flying object starts tracking (step S48).
[0066]
At the time of photographing by this automatic tracking, it is detected that the optical signal moves within the range of the photographed image (step S49). This movement of the optical signal is grasped on the aircraft side as meaning the movement of the object, and the movement direction of the object and the moving speed of the object are predicted by the movement of the optical signal, and the flying object corresponds to the predicted value. It is assumed that the direction and position of the are changed (step S50).
[0067]
FIG. 14 is a conceptual diagram when tracking while capturing a signal from the optical beacon device 31. The optical beacon device 31 is moved from the upper right to the left according to the movement of the user 3 sequentially from (a) to (c) in FIG. Although it will move slightly downward, the moving speed and moving direction in the screen are extracted by the ID camera processing unit, and the position of the flying object and the ID recognition camera is determined according to the predicted value of the movement. Will move. Therefore, for example, even when the altitude difference is large and the distance traveled is long, such as skiing, it is possible to reliably capture the skier as a subject.
[0068]
Such a photographing is performed, and the flying object may remove the optical signal from the optical beacon device 31 on the way. For example, when the subject is hidden behind the object at the time of shooting or the optical signal is interrupted for some reason, the determination (S51) of whether or not the optical signal is captured becomes NO, and after the search number RN is once reset, Proceeding to step S43, the number of searches RN is increased by one, and thereafter, when the number of searches increases to F, the flying object lands (step S45).
[0069]
If it is determined in step S51 whether or not an optical signal has been captured (S51), the flight continues, but the remote control signal is received in response to reception of the remote control signal composed of the optical signal from the optical beacon device. It is possible to change the shooting method according to the situation, or to stop the flight of the flying object. The change of the photographing method and the control of the flying object are controlled as in the above-described example described with reference to FIGS.
[0070]
Next, an example of an ID recognition camera will be further described with reference to FIG. In the mobile photographing system according to this embodiment, an ID recognition camera is used together with an optical beacon device for receiving an optical signal from the optical beacon device, and various controls are possible. Here, the ID recognition camera refers to a two-dimensional array of light receiving blocks that can be imaged at a sufficiently high speed and the amount of charge accumulated by photoelectric conversion output from each light receiver, or a voltage value, current value, A / D converted from the charge amount. It is a camera that can calculate digital values by conversion, and can read the time-series signal from the blinking pattern of light for each pixel or a plurality of pixels, and can also acquire an image. . That is, according to this ID recognition camera, an image and ID information (ID data, X and Y coordinates) are obtained.
[0071]
As a configuration of the ID recognition camera, a calculation unit can be coupled to a normal image sensor. For example, documents S. Yoshimura, T. Sujmiyama, K. Yonemoto, K. Ueda, "A 48Kframes / s CMOS Image Sensor for Real-time 3-D Sensing and Motion Detection "ISSCC Digest of Technical Papers, pp.94-95, Feb.2001 and JP2001-326857 May be. That is, in the image pickup device with a calculation function, it is possible to obtain a digital image and a calculation processed image even though it is a single image pickup device. In addition, with the CMOS structure, it is possible to realize high-speed imaging that is difficult with a conventional CCD element from CMOS (Complementary Metal-Oxide Semiconductor) semiconductor manufacturing technology.
[0072]
FIG. 15 shows a main part of an example of the ID recognition camera. The ID recognition camera mainly includes an image sensor unit 51 and a decode processing unit 61. The image sensor unit 51 is an area for converting an optical signal into an electrical signal. In the present camera, the image sensor unit 51 includes a light receiving area 52 and a calculation area 53. The image sensor unit 51 corresponds to the imaging unit 20 of the apparatus shown in FIG. 2, and the decode processing unit 61 corresponds to the ID camera processing unit 21 of the apparatus shown in FIG.
[0073]
The light receiving area 52 includes a light receiving element 54 such as a CMOS solid-state imaging device that converts light received through a camera lens or the like into an electrical signal, and an amplification unit 55 that amplifies the signal from the light receiving element 54. Yes. On the other hand, the calculation area 53 includes a storage unit 56 that temporarily accumulates signals from the light receiving area 52, a comparison unit 57 that compares the level of the signal accumulated in the storage unit 56 with a reference signal level, and a comparison unit. And an output unit 58 that outputs in accordance with the comparison result from 57. For example, signals for each pixel are accumulated in the storage unit 56 at different timings, and read out to the comparison unit 57 in a lump.
[0074]
The aforementioned decoding processing unit 61 includes an image decoding processing unit 62 that processes an image signal, and an ID decoding processing unit 63 that is arranged in parallel therewith. The ID decode processing unit 63 is a circuit portion that decodes the optical signal emitted from the optical beacon device and extracts the ID information, and is also configured to extract the coordinates related to the ID information at the same time.
[0075]
Here, the calculation performed in the image sensor unit 51 is:
[0076]
[Equation 1]
F (N) + F (N-1) -F (N-2) -F (N-3) (1)
[0077]
Here, N represents the frame number, and F (N) represents the luminance accumulated in the light receiving unit of the Nth frame. Since the optical signal from the optical beacon device is considered to be a blinking point light source, the value of equation (1) is an arithmetic operation between four frames for a pixel in which the point light source exists. By this calculation, it is possible to detect a change in light. By performing the ID decoding process described later using the calculation result, it is possible to restore the blinking pattern and receive a signal from the optical beacon. Become. Regardless of the equation (1), the calculation can be realized by other calculations such as the comparison of the first-order differentiation and the binarized image. However, in the present embodiment, in consideration of the stability of the detection of the light change, ( 1) It demonstrates using a type | formula.
[0078]
Further, in the ID recognition camera of the present embodiment, both the image processing result and the A / D converted digital image can be obtained by using an external signal, for example, an optical signal from an optical beacon device. . For example, by alternately obtaining the image processing result and the A / D converted digital image, the ID recognition camera can acquire both the image processing result for ID information and the image for the scene. Signals handled by the ID recognition camera can be divided into modes in a time-sharing manner. For example, the image mode and the ID mode can be set to a frame rate of 30 fps (frame / second) or 60 fps, respectively. That is, a signal output from the image sensor unit 51 is set to be alternately transmitted in the image mode and the ID mode, and each is processed and decoded by the corresponding decoding processing units 62 and 63.
[0079]
FIG. 16 shows an example in which the image mode and the ID mode are alternately transmitted, and the image mode includes the captured image information and is transmitted. An ID decoding process section P2 and an ID center-of-gravity section P1 are included in the period of the ID mode, and various controls as described above can be performed from the ID information included in these sections P1 and P2. Image readout is realized by processing the signal in the image mode by the image decoding processing unit 62, and ID readout is realized by processing the signal in the ID mode by the image decoding processing unit 62. In FIG. 16, the image mode and the ID mode are shown as being at the same time, but they may be the same time or need not be the same time.
[0080]
Next, the image processing for the luminance signal level in the image sensor unit 51 will be briefly described. As shown in FIG. 17, in the image sensor unit 51, in the image mode, the signal level indicating the luminance detected by each pixel of the light receiving element (photodiode) 54 is compared with the reference signal level Vref, and the reference signal level A signal that becomes active when Vref is greater than Vref is output. That is, the light incident on the light receiving element 54 is photoelectrically converted by the light receiving element 54, and the signal further amplified by the amplification unit 55 is accumulated in the storage unit 56 for each frame. FIG. 17 shows changes in the luminance signal level of bright pixels (at high luminance) and dark pixels (at low luminance) according to the number of frames. At the beginning of one screen, the number of frames is zero and the luminance reaches the reset level Vrs. There is a signal level. However, as shown in FIG. 17, the brighter pixel reaches the reference signal level Vref earlier (TH in the figure), and the darker pixel takes longer to reach the reference signal level Vref (TL in the figure). The output from the actual sensor is output as a binary (0, 1) comparison result (comparator output) as to whether or not the luminance signal of each pixel has reached the reference signal level Vref in the comparison unit 57. It is done by doing. FIG. 17 shows an example in which the binary output changes from “0” to “1” at time TH due to a change in high luminance. The image sensor unit 51 can perform high-speed imaging, perform the above calculation, and output the result for one screen to the outside of the sensor.
[0081]
In order to measure the time (TH, TL) until the signal from the bright pixel or the dark pixel reaches the reference signal level Vref, the image sensor unit 51 outputs the result in units of one screen. By counting and recording in what number frame each pixel the output signal of the image sensor unit 51 is active, that is, when the binary output changes from “0” to “1”, the image is obtained. Will be. In order to convert to correct brightness as image information, a bright pixel has a short time until active transition and a dark pixel has a long time until active transition, and thus reaches the reference signal level Vref. What is necessary is just to calculate the reciprocal of the time (number of frames) until. With this operation, an A / D conversion operation of the image can be realized. The sensor output from the image sensor unit 51 is sent to the image processing decoding unit 62, and the image processing decoding unit 62 performs processing for obtaining an image.
[0082]
Next, the ID decoding process will be described. First, the frequency of the image sensor unit 51 and the optical beacon device will be described. The frame rate fs of the image sensor unit 51 is set to, for example, three times the blinking frequency fp of the optical signal of the optical beacon device. This is because, according to “Shannon's Sampling Theorem”, the sampling rate of the sensor unit requires a sampling rate more than twice the blinking cycle of the optical signal from the optical beacon device. Since there is a difference in phase between fs and fp, in this embodiment, the phase difference is set to 3 times. For example, when the frequency of the sensor unit is 12 KHz, for example, the blinking cycle of the optical beacon is set to 4 KHz. For ease of explanation below, in this embodiment, when the sensor frequency is 12 kHz, the blinking cycle of the optical beacon is assumed to be 4 kHz.
[0083]
Next, a structural example of the optical beacon device will be described with reference to FIG. First, an example of the optical beacon device has been described with reference to FIGS. 3 and 4, but another configuration example will be described with reference to FIG. 18. In the example of the optical beacon device illustrated in FIG. 18A, the optical beacon device 71 has a structure including a light source 72, a blinking control unit 73, and a transmission data storage memory 74. The light source 72 is configured to perform a blinking operation in accordance with a drive signal from the blinking control unit 73, and the blinking control unit 73 operates by taking out output data corresponding to the situation and setting from the transmission data storage memory 74. . The light source 72 may be any light source as long as it can blink at high speed, and the output wavelength may be in a range in which the sensor responds. That is, as the wavelength region of the light source 72, not only visible light but also an infrared region can be used. From the viewpoint of response speed and life, LED is one of the most suitable light sources. The blinking control unit 73 blinks the light source 72 based on the contents of the transmission data storage memory 74 using a transmitter and a digital circuit (which can be realized by an IC, ASIC, FPGA, one-chip microcomputer, etc.).
[0084]
FIG. 18B shows a configuration example of the optical beacon device 75 that can change the transmission data through the communication path 79. Similarly to FIG. 18A, the optical beacon device 75 The light source 76, the blinking control unit 77, and the data transmission / reception unit 78 are included. The light source 76 is configured to blink in response to a drive signal from the blinking control unit 77, and the blinking control unit 77 responds to a control signal from the data transmission / reception unit 78 to which a signal is supplied via the communication path 79. Blinks. Here, the communication path 79 is a wired / wireless communication line such as a telephone line ISDN, RS-232C, RS-422, Ethernet (registered trademark) (10base-T, 100base-T), USB, IEEE1394, IEEE802.11a. , IEEE802.11b, BlueTooth (trade name) and other communication means. The data transmission / reception unit 78 indicates a data communication IC and driver corresponding to this communication means, and the transmission data of the optical beacon device received via the communication path 79 is transmitted via the data transmission / reception unit 78 to the blinking control unit 77. To the flashing pattern of the light source 76.
[0085]
Here, an encoding method of an optical signal transmitted from the optical beacon device will be described. In digital communication, transmission data is generally encoded so that it can be received stably on the receiving side. In the present embodiment, data (flashing pattern) transmitted from the optical beacon device is encoded using Manchester encoding. As shown in FIG. 19, Manchester encoding is an encoding method in which the transmission data is 10 for transmission data 0 and the transmission data is 01 for transmission data 1. By using this, an optical beacon device is used. While valid data is being transmitted, there is always a change in the brightness of the light, and in order to receive a blinking pattern in an operation for examining the change in brightness between the four frames using the image sensor unit 51. It turns out that it becomes convenient encoding. In the example of FIG. 19, encoding in the case of transmitting 8-bit “01001001” data is described. When 1 bit is “1”, encoding is performed to “01” and 1 bit is “0”. In the case of “10”, it is encoded to “10”.
[0086]
In the present embodiment, a case where fixed-length 8-bit data is transmitted will be described as an example. One data frame of transmission data includes a start code and transmission data as in the transmission data format shown in FIG. The start code is intended to notify the receiving side that it is the head of one frame of transmission data, and therefore, for example, a bit “0001” that does not appear in the transmission data portion encoded by Manchester is assigned. . The transmission data is 16-bit data obtained by Manchester code encoding 8-bit data. Therefore, when this one data frame is transmitted by blinking the LED, 1 of the transmission data is assigned to be turned on and 0 is turned off, and the light signal is blinked at a blinking cycle of 4 kHz (every 250 μs) in this embodiment. As a result, 8-bit ID data is transmitted.
[0087]
Such an encoded blinking signal is incident on the image sensor unit of the above-described ID recognition camera, and the image sensor unit receives an optical beacon device in an arithmetic block unit (each pixel or a plurality of pixels) in the ID mode. The flashing optical signal is received, calculation is performed between four frames as shown in equation (2), and the calculation result is output outside the sensor.
[0088]
[Equation 2]
F (N) + F (N-1) -F (N-2) -F (N-3) (2)
[0089]
However, in reality, F (N), F (N-1), F (N-2), and F (N-3) in this equation are treated as analog quantities in this image sensor, and their calculations are performed. The result is binarized by a threshold that can be controlled from the outside. Therefore, as one operation mode of the image sensor, it is possible to output a result obtained by binarizing the result of the expression (2) with a positive threshold value. That is, if the threshold level is THR,
[0090]
[Equation 3]
F (N) + F (N-1) -F (N-2) -F (N-3) ≥ 1 when THR
[0091]
F (N) + F (N-1) -F (N-2) -F (N-3) <When THR is 0 ... (3)
[0092]
Is output. This will detect the rising edge of the blinking LED. FIG. 20 shows the sensor output when the transmission data (8-bit data) described in the previous section is sent. At this time, the sample pulse of the sensor is 12 KHz, and the calculation result of equation (3) is output in units of 12 KHz. This sensor output is input to the ID decoding processing unit 63 shown in FIG. 15, and the ID (here, 8-bit transmission data from the optical beacon device) and coordinates are calculated.
[0093]
As described above, in the image mode and the ID mode, the ID recognition camera is realized by switching the mode alternately between the image and the ID. For example, when this is switched at 30 frames / sec, this ID recognition camera becomes a camera that can acquire an image at 15 fps and an ID at 15 fps. Here, in the image mode section, the result (ID) decoded in the previous ID mode section can be output from the ID recognition camera while performing the image decoding process. In the ID mode section, the result (image) decoded in the previous image decoding section can be output from the ID recognition camera while performing the ID decoding process. In the present embodiment, this ID mode is divided into two, an ID decoding processing section P1 and an ID centroid calculation section P2, as shown in FIG.
[0094]
Here, the ID decoding method will be described with reference to FIG. 20. When the optical beacon transmits fixed-length 8-bit data (01001001), the image sensor unit has a rising edge in the ID mode as described in the previous section. The detection is performed, and the sensor output is as shown in FIG. Hereinafter, for the sake of simplicity, an ID decoding procedure will be described by paying attention to a certain pixel (or one arithmetic unit) of the sensor.
[0095]
In the present embodiment, the fixed length of 8 bits is described. However, it is easy for those skilled in the art to extend the number of bits from the one disclosed in the present embodiment. Even when it is desired to make the bit length variable instead of the fixed length, it is realized only by providing not only the start code but also an end code and adding its detection means (the same method as the start code).
[0096]
First, detection of a start code detects a start code of transmission data as an initial procedure of ID decoding. Since the start code is defined as “0001” in the present embodiment, the sensor output is regarded as a start code when 0 is received for the first time after 0 is output 9 times (9 frames of the image sensor) or more. Can do.
[0097]
Subsequently, reception of the ID data proceeds as follows. The sensor output of the sixth frame from the next frame after reception of the start code is the 0th bit data (b0) of ID data, that is, LSB. The sensor output for every 6 frames from b0 becomes b1, b2, b3, b4, b5, b6, b7, and 8-bit data can be received. In FIG. 20, 6 frames are drawn as 6 clocks, and sensor data is sampled every 6 clocks to extract ID data.
[0098]
In this embodiment, an error check function is provided, and measures are taken to eliminate the influence on the sensor by things other than the light beacon so that the blinking of the light beacon does not interfere with things other than the light beacon such as fluorescent lighting. ing. That is, when the data encoded by the Manchester code is received by the sensor at a frequency fs that is three times the blinking frequency fp of the optical beacon, and the calculation shown in equation (3) is performed, the following law is established. This law will be described with reference to FIG. First, when each bit of b0 to b7 is “0”, the sensor output three frames before and the sensor output six frames before the bit “0” are always inverted. This law is indicated by an arrow α in FIG. 21 and b1 is “0”. Therefore, if the data before 3 clocks and the data before 6 clocks are inverted, it is normal and inverted. Otherwise it is considered an error. The following rule is that when each bit of b1 to b7 is 1, the sensor output three frames before is always “0”. If this condition is not satisfied, it is regarded as an error. In FIG. 21, the part representing the law is indicated by an arrow β. Since b6 is “1”, if the data three clocks before is “0”, it is normal, and if it is “1”, it is processed as an error.
[0099]
Next, ID center-of-gravity calculation in this embodiment will be described. This ID centroid calculation is a method for obtaining the coordinates on the image of the ID from the ID of each pixel (or each arithmetic module) obtained by the method as described above, and an example thereof will be described here. For example, for each ID type (in this case, since the ID is 8 bits, 256 types), the sum and number of the I and J coordinates are obtained, and the sum of each I and J coordinate is divided by the number. The barycentric coordinates for each can be obtained. When the ID is different for each optical beacon, the coordinates can be calculated by this method.
[0100]
FIG. 22 is an internal block diagram of the ID decoding processing unit. The sensor output is supplied to the ID decoding circuit 81 as an input of this block, and the ID decoding circuit 81 decodes the ID of each pixel from the sensor output in the procedure shown in the above-described ID decoding procedure. In the present embodiment, description will be made with the realization of a device such as an IC, an ASIC, and an FPGA in mind, but if the processing capability is sufficient, it can also be realized by software using a microprocessor or a DSP.
[0101]
The ID decode circuit 81 that receives the sensor output is connected to the flag register 82 and the data register 83, and the data decoded by the ID decode circuit 81 is sent. The data register 83 is a register for storing an ID that is being decoded or has been decoded, and the flag register 82 is a register for storing flags necessary for decoding the ID. FIG. 23 shows details of each register. In the flag register 82, the first 1 bit is a start flag, and the next 4 bits are a zero counter value. The next 3 bits are the phase counter value, the next bit is the end flag, and the ID counter value and the previous half-phase bit follow. The ID counter value is a value indicating the bit number of data received next. For example, if it is 1, it indicates that the next is the order of receiving b1. The data register 83 is an 8-bit data register, and 8-bit data from b0 to b7 are arranged.
[0102]
The data in the flag register 82 and the data in the data register 83 are sent to the frame memory 85 and stored. The frame memory 85 is a memory for storing data (ID) and a flag decoded for each pixel or each arithmetic module of the sensor. When the sensor has one arithmetic module per pixel, the image size of the sensor is used. As shown in FIG. 23, the data width is the sum of the bit widths of the data register 83 and the flag register 82, that is, the data width is 21 bits or more in this embodiment.
[0103]
The timing control unit 84 in FIG. 22 is a module that performs timing control of the entire ID decoding processing unit 63. Specifically, the ID decode circuit 81 and the sensor output timing are synchronized, and the data of the corresponding frame memory 85 is loaded into the data register 83 and the flag register 82 for the desired sensor output, and the ID decode processing is performed. A timing signal for advancing is generated and supplied. At this time, control signals such as an address signal and read / write to the frame memory 85 are generated. In addition, the timing control unit 84 performs timing control of the ID centroid calculation circuit 86, the ID register 87, and the ID coordinate storage memory 88.
[0104]
With reference to FIG. 24, the flow of the ID decoding process will be described. When the ID mode is entered in a signal in which the image mode and the ID mode are transmitted alternately, first, in step S61, all the frame memories 85 are initialized and "0" is written. Also, an address counter for the frame memory 85 built in the timing control unit 84 is initialized.
[0105]
After such initialization, reading from the frame memory 85 to the register is performed in step S62. That is, the timing control unit 84 provides an address signal, a read signal, and a chip select signal to the frame memory 85, reads out flag information and data (ID) information of coordinates corresponding to the sensor output currently received by the ID decoding circuit, The flag information is written in the flag register 82 and the data (ID) information is written in the data register 83. In the procedure immediately after initialization, the output data from the frame memory 85 is “0”.
[0106]
Next, it progresses to procedure S63 and an ID decoding main process is performed as the said ID decoding process part. A specific procedure of the ID decoding main process is shown in FIG. First, in step S71, it is checked whether or not the end flag in the flag register 82 is “1”. If the end flag is “1”, since the reception of data from the optical beacon device has already been completed during the current ID mode period, the ID decoding main process is exited without performing any processing. If the end flag is “0”, the ID decoding main process is continued and the process proceeds to step S72.
[0107]
In step S72, it is checked whether or not the start flag in the flag register 82 is “1”. If the start flag in the flag register 82 is “0”, a start code detection process (step S73: see FIG. 26) described below is performed, and then the ID decoding main process is exited. If the start flag in the flag register 82 is "1", the data reception process (step S74) described later is performed, and the ID decoding main process is exited. When the ID decoding main process is exited, the process returns to step S64 in the flow of FIG. 24. The steps after step S64 shown in FIG. 24 will be described later.
[0108]
The start code detection process that is the process of step S73 of FIG. 25 will be described with reference to FIG. First, it is checked whether or not the zero counter value in the flag register 82 is “9” (step S81). The zero counter is necessary for the first half of the process to detect the start code "0001". Since the frequency of the image sensor is three times the blinking frequency of the optical beacon signal, first receive "0" nine times or more. That is the condition for the start code. If the zero counter value is "9" or less (NO), the zero counter value is incremented by 1 (step S82), and the start code detection process is exited. If the zero counter value is 9 (YES), the start code detection process is continued.
[0109]
Subsequently, in step S83, it is checked whether or not the current value from the sensor is 1. If the input is “0”, it can be determined that the start code has not yet arrived, and the process exits the start code detection process without doing anything. If the input is “1”, it is determined that the start code has been received, the start flag is set to “1” (step S84), and the start code detection process is terminated.
[0110]
Next, the data reception process will be described with reference to FIG. The data reception process is a procedure when the previous start flag is “1” (see FIG. 25). In this data reception process, first, it is checked in step S91 whether the phase counter value is “4”. . In the present embodiment, an error check of received data is performed as described above. In this case, the received data 3 frames before each bit of the received data is used, and the timing of this error check is set to be synchronized with the time when the value of the phase counter is “4”. In step S91, it is checked whether or not the phase counter is "4". If the phase counter value is "4" (YES), the sensor output is written in the bit before half phase (step S97). If the phase counter value is not “4” (NO), the data reception process is continued and the process proceeds to step S92.
[0111]
As described above, since the frequency of the image sensor is three times the blinking frequency of the optical beacon signal, data may be received every 6 clocks. This phase counter takes a value of 0 to 7, and when the value of the phase counter is “7”, the sensor output is obtained by restoring the transmission data from the blinking data from the optical beacon device. In step S92, it is checked whether the phase counter value is “7”. If the phase counter value is not "7" (NO), the process proceeds to step S98, the phase counter is incremented by 1, and the data reception process is exited. If the phase counter value is “7” (YES), the process proceeds to the next error check process.
[0112]
Here, the error check process will be described with reference to FIG. This error check process corresponds to the specific contents of step S93 in FIG. First, in step S101 of FIG. 28, it is determined whether or not the input (sensor output) is “1”. If the input is “1”, it is checked whether or not the half-phase previous bit in the flag register 82 is “0” (step S102). If the half-phase previous bit is “0”, it is normal (OK). ), And if the previous half-phase bit is “1”, an error occurs (NG). The determination in step S102 is a determination corresponding to the arrow β in FIG. 21, and the bit before half phase is the data three clocks before.
[0113]
If the input is “0” in step S101, the process proceeds to step S103, and the half-phase previous bit is compared with the data one bit previous. The data one bit before is the data in the data register, and which bit is known from the ID counter value. However, when the ID counter value is “0”, the data one bit before is “1”. In the determination in step S103, if they are not the same, that is, if they are reversed, -normal (OK), and if they are the same, an error occurs (NG). This comparison is a determination corresponding to the arrow α in FIG. 21, the half-phase previous bit is the data before 3 clocks, and the 1 previous bit is the data before 6 clocks.
[0114]
After such error check processing is performed, the data reception processing will be described again with reference to FIG. If the error check process is NG as a result of the error check process, “0” is set in the flag register 82 and the data register 83 in step S96. On the contrary, if the result of the error check process is normal (OK), the ID counter value is checked in step S94, and the input (sensor output) is recorded in the data register 83 where it should be recorded. Subsequently, the value of the phase counter is set to “1”, and the value of the ID counter is incremented by 1 (step S95). The input data is written into the data register 83 only when the data reception process is completed and there is no error in the data.
[0115]
Upon completion of this data reception process, the ID decoding main process (FIG. 25) is also terminated, and the process returns to step S64 of the ID decoding process in FIG. In this procedure S64, the contents of the flag register 82 and the data register 83 are written into the frame memory 85. The address at this time is an address counter value in the timing control unit 84.
[0116]
After writing from the registers 82 and 83 to the frame memory 85, it is determined whether or not the decoding completion time has been reached. As previously shown in FIG. 16, in the ID recognition camera of this embodiment, images and IDs are obtained alternately, and each section is called an image mode and an ID mode. This ID mode section is also divided into an ID decode processing section and an ID centroid calculation section, each of which defines a time. Here, the decoding completion time refers to the time for finishing the ID decoding processing section, and in this step S65, it is determined whether or not the time has come.
[0117]
This time management is performed by the timing control unit 84 shown in FIG. If the decoding completion time has been reached, the process proceeds to the next ID centroid calculation process (step S69). If the decoding completion time has not been reached, it is checked in step S66 whether or not the frame has ended. If it has ended, the address counter is initialized to 0 (step S68), or if the frame has not ended, the address counter is counted. (Step SS67), the process returns to step S62, and the next flag and data (ID) information is transferred from the frame memory 85 to the registers 82 and 83 to continue the ID decoding process.
[0118]
Here, the ID centroid calculation process will be described with reference to FIG. First, the ID coordinate storage memory 88 is first initialized to all data “0” (step S111). In step S112, data is read from the frame memory 85 to the ID register 87. At this time, data (ID) information is read from the frame memory 85 by a control signal from the data control unit and written to the ID register 87.
[0119]
Next, in step S113, data corresponding to the read ID is read from the ID storage coordinate memory 88. As shown in FIG. 22, the ID storage coordinate memory stores information of 256 types of IDs in order because transmission data is 8 bits in this embodiment. "Sum of I coordinates", "Sum of J coordinates", and "Number (number of pixels in which ID is detected)" required for calculating the center of gravity. Therefore, here, from the ID stored in the ID register 87, the ID centroid calculating unit 86 calculates “sum of I coordinates”, “sum of J coordinates”, and “number (number of pixels in which ID is detected)” corresponding to the ID. Will be read out.
[0120]
The ID center-of-gravity calculation unit 86 operates to calculate the sum of I coordinates, the sum of J coordinates, and the number +1 (step S114). That is, “sum of I coordinates” is updated to a value obtained by adding the I coordinate value of the frame memory 85 to “sum of I coordinates”, and “sum of J coordinates” is “sum of J coordinates” + J of the frame memory 85. The value is updated to a value obtained by adding the coordinate value, and “number (number of pixels in which ID is detected)” is updated to a value obtained by incrementing “number (number of pixels in which ID is detected)” by one. The calculation result is written from the ID centroid calculation unit 86 to the ID coordinate storage memory 88 (step S115). The timing controller 84 writes the contents of the ID register 87 back to the corresponding ID coordinate storage memory 88.
[0121]
Next, the process proceeds to step S116, where it is determined whether or not the reading of all coordinates of the frame memory 85 is completed based on the address signal of the timing control unit 84. If not completed, the timing controller 84 increments the value of the address counter (step S117), reads the ID data of the next address from the frame memory 85 to the ID register, and continues the ID centroid calculation process. To do. In step S116, if all coordinates are read from the frame memory 85, the ID center-of-gravity calculation process is terminated.
[0122]
In the present embodiment, in the ID recognition camera, “Sum of I coordinates”, “Sum of J coordinates”, and “Number (number of pixels in which ID is detected)” are calculated. The center-of-gravity coordinates of the ID can be easily calculated by the following equation if these values are received from the ID recognition camera.
[0123]
[Equation 4]
Center of gravity coordinates (Ig, Jg) = (sum of I coordinates, sum of J coordinates) / number
[0124]
In addition to the center of gravity, other methods such as the center of a circumscribed rectangle can be used for the position of each ID.
[0125]
By calculating the center-of-gravity coordinates, a subject that moves greatly while generating ID information can be reliably captured within the angle of view of the ID recognition camera, and the subject can be automatically tracked and reliably photographed.
[0126]
In the above-described embodiment, the ID decoding process is described so as to be sequentially performed according to the output order of the sensors. For example, if sensor output is performed by raster scanning, reading is performed by raster scanning and decoding is performed by one ID decoding circuit. However, the present invention is not limited to this, and when it is desired to perform ID decoding processing at high speed, a parallel circuit configuration and an ID decoding circuit (N) for each line can realize a circuit configuration that is N times faster. . Further, although the circuit scale increases, an ID decoding circuit may be provided for each pixel.
[0127]
In the above-described embodiment, error detection is performed using Manchester coding as a signal processing method. However, other coding methods may be used, and not only error detection but also error correction is performed. In addition, it is possible to adopt a photographing method in which a plurality of IDs are assigned to one subject and reliable tracking is performed.
[0128]
【The invention's effect】
As described above, according to the mobile shooting system and the mobile shooting method of the present invention, a subject can be shot by a moving body such as a flying body equipped with an ID recognition camera, so that the moving subject is automatically tracked. Can be taken reliably. In addition, by operating the optical beacon device, it is possible to control an imaging program and a tracking method for an unmanned flying object, and it is possible to respond flexibly even if shooting is interrupted. Even when a camera follows a plurality of subjects, ID information can be given to each subject and can be automatically grasped together with position information. Certain shooting is possible.
[Brief description of the drawings]
FIG. 1 is a schematic diagram showing the configuration of an example of a mobile photographing system of the present invention.
FIG. 2 is a block diagram showing a configuration of a mobile object side in an example of a mobile photographing system of the present invention.
FIG. 3 is a schematic external perspective view showing an example of an optical beacon device in the mobile photographing system of the present invention.
4 is a block diagram showing an internal circuit configuration of the optical beacon device of FIG. 3. FIG.
FIG. 5 is a flowchart for explaining a general operation of an example of the mobile photographing system of the present invention.
FIG. 6 is a flowchart for explaining an operation of changing the position of a flying object in an example of the mobile imaging system of the present invention.
FIG. 7 is a flowchart for explaining a photographing method changing operation in an example of the mobile photographing system of the present invention.
FIG. 8 is a flowchart for explaining a tracking method during automatic photographing in an example of the mobile photographing system of the present invention.
FIG. 9 is a schematic diagram for explaining a framing mode during automatic photographing in an example of the mobile photographing system of the present invention.
FIG. 10 is a schematic diagram for explaining a distance / angle mode at the time of automatic photographing in an example of the mobile photographing system of the present invention.
FIG. 11 is a schematic diagram for explaining a zoom mode at the time of automatic photographing in an example of the mobile photographing system of the present invention.
FIG. 12 is a schematic diagram for explaining a photographing direction mode during automatic photographing in an example of the mobile photographing system of the present invention.
FIG. 13 is a flowchart for explaining a tracking method during automatic photographing in an example of the mobile photographing system of the present invention.
FIG. 14 is a conceptual diagram when tracking is performed while capturing a signal from an optical beacon device in an example of the mobile imaging system of the present invention.
FIG. 15 is a block diagram illustrating a configuration example of an ID recognition camera in an example of a mobile shooting system of the present invention.
FIG. 16 is a diagram showing a relationship between an image mode and an ID mode in an example of a mobile shooting system of the present invention.
FIG. 17 is a diagram illustrating an A / D conversion operation for image output in an example of the mobile photographing system of the present invention.
FIG. 18 is a block diagram illustrating a configuration example of an apparatus in an example of a mobile shooting system of the present invention.
FIG. 19 is a diagram showing a format of transmission data in an example of the mobile photographing system of the present invention.
FIG. 20 is a timing chart showing an ID decoding process in an example of the mobile photographing system of the present invention.
FIG. 21 is a timing chart showing error check processing during ID decoding processing in an example of the mobile photographing system of the present invention.
FIG. 22 is a block diagram illustrating a configuration of an ID decoding processing unit in an example of a mobile photographing system of the present invention.
FIG. 23 is a block diagram illustrating a configuration of a register in an example of a mobile photographing system of the present invention.
FIG. 24 is a flowchart showing a processing procedure in an ID mode in an example of the mobile photographing system of the present invention.
FIG. 25 is a flowchart showing an ID decoding main process in an example of the mobile photographing system of the present invention.
FIG. 26 is a flowchart showing start code detection processing in an example of the mobile photographing system of the present invention.
FIG. 27 is a flowchart showing data reception processing in an example of the mobile imaging system of the present invention.
FIG. 28 is a flowchart showing an error check process in an example of the mobile photographing system of the present invention.
FIG. 29 is a flowchart showing ID center-of-gravity calculation processing in an example of the mobile imaging system of the present invention.
[Explanation of symbols]
2 Optical beacon device
3 users
10 Unmanned helicopter
11 ID recognition camera
12 Main rotor
13 Tail rotor
20 Imaging unit
21 ID camera processing unit
23 memory
24 Aircraft Control Unit
25 Flight power section
26 Image controller
27 Display section
28 Input section
29 External output section
30 Network interface
31 Optical beacon device
32 Holding part
33 Optical beacon device body
34 Key input section
35, 36 Light emitting diode
37 Microcomputer
38 Power supply
39 Key input section
40 LED drive unit
41 memory
81 Decode circuit
82 Flag register
83 Data register
84 Timing controller
85 frame memory
86 ID register
87 ID center of gravity calculation circuit
88 ID coordinate storage memory

Claims (17)

A moving body equipped with a photographing device and made movable;
A signal generator that is attached to a subject and generates an optical signal formed by a flashing pattern of light,
The imaging device
An imaging unit;
An image mode that is a mode for acquiring image data as a scene from a signal obtained by the imaging unit, and a mode that receives the optical signal by restoring the blinking pattern from the signal obtained by the imaging unit. A mobile photographing system having a processing unit that repeats the ID mode in a time-sharing manner. The imaging unit of the imaging device, a light receiving unit using a CMOS solid imaging element,
A calculation unit that detects a change in light in units of each pixel or a plurality of pixels from the signal of the light receiving unit during the ID mode;
The mobile imaging system according to claim 1, wherein the processing unit of the imaging apparatus restores the blinking pattern from the detection result of the arithmetic unit and obtains the coordinates of the optical signal on the image in the ID mode. The mobile imaging system according to claim 1, wherein the optical signal from the signal generation device includes recognition information that is different for each of the signal generation devices. The mobile imaging system according to claim 1, wherein the moving body is controlled based on control information included in the optical signal. The mobile imaging system according to claim 4, wherein the moving body is an unmanned control propellable device. A storage unit is connected to the imaging device, and image data acquired in the image mode is associated with recognition information that is different for each of the signal generation devices included in the optical signal received in the ID mode. The moving imaging system according to claim 1 recorded. The mobile imaging system according to claim 6, further comprising an image display unit that processes the image data output from the storage unit to display an image. A signal generator that generates an optical signal formed by a flashing pattern of light is attached to the subject.
An image mode which is a mode for acquiring image data as a scene from the image obtained from the image pickup unit, the signal obtained by the image pickup unit, and the light signal is received by restoring the blinking pattern from the signal obtained by the image pickup unit. An imaging apparatus having a processing unit that repeats the ID mode, which is a mode to perform, in a time-sharing manner, is mounted on the moving body
The object is moved and the moving body is operated, and the optical device detects the optical signal from the signal generating device in the ID mode by the photographing apparatus, and the moving body detects the object based on the detected optical signal. A moving shooting method of tracking and shooting the subject by the shooting device in the image mode. The moving photographing method according to claim 8, wherein the photographing apparatus captures the optical signal during the ID mode, and the moving body is controlled based on control information included in the optical signal. The moving imaging method according to claim 8, wherein the moving body starts a tracking operation when it is confirmed that the moving object is a target subject with a different identification number for each of the signal generation devices included in the optical signal. The moving imaging method according to claim 8, wherein the moving body predicts a moving direction and a moving speed of the subject by moving the optical signal, and the moving body moves so as to match the prediction. The moving imaging method according to claim 8, wherein the moving body performs detection again when the optical signal is lost. A signal generator that generates an optical signal formed by a flashing pattern of light is attached to the subject.
An image mode that is a mode for acquiring image data as a scene from a signal obtained by the imaging unit, a signal obtained by the imaging unit, and the light signal is received by restoring the blinking pattern from the signal obtained by the imaging unit After mounting the imaging device having the processing unit that repeats the ID mode, which is the mode to be performed, in a time-sharing manner on the moving object,
While moving the subject and capturing the optical signal in the ID mode with the photographing device,
A moving imaging method in which the moving body is controlled based on control information included in the optical signal. The moving imaging method according to claim 13, wherein the moving body is an unmanned control propulsible device. A signal generator that generates an optical signal formed by a flashing pattern of light is attached to the subject.
An image mode that is a mode for acquiring image data as a scene from a signal obtained by the imaging unit, a signal obtained by the imaging unit, and the light signal is received by restoring the blinking pattern from the signal obtained by the imaging unit After mounting the imaging device having the processing unit that repeats the ID mode, which is the mode to be performed, in a time-sharing manner on the moving object,
The subject is moved and the optical device captures the optical signal in the ID mode, and an automatic imaging mode at the start of tracking of the subject by the moving body is selected based on control information included in the optical signal. Moving shooting method. As modes for automatic shooting at the start of tracking, a mode in which the position of the subject in the screen is mainly fixed, a mode in which the shooting direction with respect to the subject is mainly fixed, and a distance and an angle between the subject and the moving body are mainly fixed. 16. The moving photographing method according to claim 15, wherein one or a plurality of modes and modes in which the size of the subject in the screen is mainly fixed can be used. A signal generator that generates an optical signal formed by a flashing pattern of light is attached to the subject.
An image mode that is a mode for acquiring image data as a scene from a signal obtained by the imaging unit, a signal obtained by the imaging unit, and the light signal is received by restoring the blinking pattern from the signal obtained by the imaging unit After mounting the imaging device having the processing unit that repeats the ID mode, which is the mode to be performed, in a time-sharing manner on the moving object,
While moving the subject and capturing the optical signal in the ID mode with the photographing device,
The image data acquired in the image mode by the imaging device in association with different recognition information for each of the signal generation devices included in the optical signal is recorded in a storage unit connected to the imaging device. Shooting method.

Images