JP2004112550A - Imaging apparatus, camera, program, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus which controls, traces and reads an accurate object area on a picture in simple constitution and operation and designates and traces the object area without forcing complicated operations on a user, and to provide a program and a recording medium. <P>SOLUTION: The imaging apparatus has a means for designating an object of interest within a field of view of an imaging means, a means for tracing the designated object area, and a means for recording the object area in association with a video signal. Further, the imaging apparatus has a means for calculating similarity between a present video signal and a video signal imaged certain time before, a means for detecting a direction of the imaging means, and the like. Based upon the output of the similarity calculating means or the direction detecting means, a tracing means traces the object area. The imaging apparatus further has a means for displaying the object area outputted by the tracing means, a means for raising an alarm when the tracing means judges that the object area gets out of a picture, and the like. The recording means records text data with the traced result of the object area described therein. <P>COPYRIGHT: (C)2004,JPO

Description

【００９８】
ここで、（１）式における各記号の意味は以下の通りである。
・Ｉ_１（ｕ，ｖ）：テンプレート上の点（ｕ，ｖ）における画素値
・Ｉ_２（ｕ，ｖ）：現画像上の点（ｕ，ｖ）における画素値
以上の計算に基づき、正規化相互相関値Ｓが最大となる現画像における点（ｍ，ｎ）を求める。
【００９９】
次に、図６のステップＳ４５において、上述のテンプレートマッチングの成否を判定する。ここで、テンプレートマッチング成否の判定基準は、例えば、正規化相互相関値Ｓの閾値θを予め決めておき、Ｓの値がθ以上ならば成功、θ以下ならば失敗とするなどの手法をとればよい。
【０１００】
成功と判定された場合には（ステップＳ４５でＹｅｓ）、ステップＳ４３で求められた点（ｍ，ｎ）を現画像における被写体領域として、システム制御部２に出力すると同時に、現画像の点（ｍ，ｎ）を中心とする大きさ（２ｄｕ＋１）　×（２ｄｖ＋１）の領域を、前記テンプレートとして更新する（ステップＳ４６）。また、失敗と判定された場合には（ステップＳ４５でＮｏ）、テンプレートの更新は行わずに、追跡に失敗したことを示す信号をシステム制御部２に出力する（ステップＳ４７）。以上で、処理を終了する。
【０１０１】
次に、本発明による撮像装置、カメラ、プログラム及び記録媒体の一実施形態における被写体領域の記録について説明する。
【０１０２】
静止画の場合、被写体領域に関するデータは、取得した画像におけるヘッダ情報に記録される。例えば、デジタルスチルカメラの静止画のフォーマットとしてＥｘｉｆ（ｅｘｃｈａｎｇｅａｂｌｅ　ｉｍａｇｅ　ｆｉｌｅ　ｆｏｒｍａｔ）を採用する場合、そのヘッダ情報中のＳｕｂｊｅｃｔ　Ａｒｅａと呼ばれる、主要となる被写体の位置を記述することができるフィールドに、当該被写体領域の位置を記録することができる。
【０１０３】
また、動画の場合は、図８のように、左から秒単位の時刻、被写体領域の横方向の座標ｕ、被写体領域の縦方向の座標ｖという順に、テキスト情報として記録される。このテキスト情報は、動画と共に、テキストファイルとして、記録メディアに記録される。
【０１０４】
このとき、動画データとテキストファイルは、関連して記録される。例えば、動画ファイルの名前が”Ｖｉｄｅｏ　０１．ａｖｉ”（ａｖｉ　は、Ａｕｄｉｏ　Ｖｉｄｅｏ　Ｉｎｔｅｒｌｅａｖｉｎｇの略）である場合、当該テキストファイルの名前は”Ｖｉｄｅｏ　０１．ｔｘｔ”と、拡張子以前の名前が揃えられた形で記録される。
【０１０５】
以下、本発明による撮像装置、カメラ、プログラム及び記録媒体の一実施形態たる第２の実施例について説明する。
【０１０６】
前述の焦点領域の検出処理を、撮像部３の向きを検出することにより実行しても構わない。この方法は、静止している被写体を、該被写体までの距離に相対して撮像部３をあまり平行移動させずに撮影した場合に、特に有効である。
【０１０７】
図９は、本実施例に係るデジタルスチルカメラの構成を示す図である。図９におけるデジタルスチルカメラ１は、図１のデジタルスチルカメラに姿勢検出センサ１２が追加されたものである。
なお、図中同一符号は、前述した第１の実施例と同一又は相当部分を示す。
【０１０８】
姿勢検出センサ１２は、撮像部３の主軸（通例光軸）の向きを検出するために設けられた、姿勢検出センサ、該センサからの信号を取得するＡ／Ｄ変換器、及び該Ａ／Ｄ変換器により取得されたデータに演算処理を施すＡＬＵ等からなる、姿勢検出部、を示している。
【０１０９】
姿勢検出センサ１２は、例えば、加速度センサ、磁気センサ、角速度センサ（ジャイロ）又はそれらの組み合わせにより構成される。
【０１１０】
第２の実施例における静止画撮影モード及び動画撮影モードの動作は、第１の実施例と同様であり、それぞれ図３、図４のフローチャートに従って動作する。
【０１１１】
ここで、姿勢検出センサの一例として、加速度センサと磁気センサを使用した場合の、被写体領域の追跡動作を、図１０のフローチャートを用いて説明する。
【０１１２】
まず、システム制御部２は、姿勢検出センサの検出信号より、絶対座標系に対する現在の撮像部３の姿勢を算出し、メモリに記録する（ステップＳ５１）。
【０１１３】
ここで、絶対座標系は図１１のように、東向きをＸ軸、重力の向きをＹ軸、北向きをＺ軸とする、右手系ＸＹＺ座標系であるとする。
【０１１４】
加速度センサは、重力加速度を、また、磁気センサは、地磁気を検出するのに使用され、これらのセンサの出力信号により、絶対座標系に対する撮像部３の向きを容易に計算することができる。
【０１１５】
ここで、被写体領域の追跡処理が前述の初期状態である場合には（ステップＳ５２でＹｅｓ）、以上で処理を終了する。
【０１１６】
一方、初期状態でない場合には（ステップＳ５２でＮｏ）、前時刻で記録された被写体領域及び撮像部３の姿勢を、メモリより読み出し（ステップＳ５３）、これらの情報と現在の撮像部３の姿勢を利用して、現在の被写体領域を算出し、メモリに記録する（ステップＳ５４）。
【０１１７】
ここで、現在の被写体領域を算出する方法について説明する。
【０１１８】
まず、図１２に示される２つの座標系を定義する。
カメラ座標系（ｃａｍｅｒａ　ｃｏｏｒｄｉｎａｔｅ　ｓｙｓｔｅｍ）
・ｘ軸：画面右向きを正
・ｙ軸：画面下向きを正
・ｚ軸：光軸、対象に向かう向きを正
・原点０：カメラの光学中心
・ｆ：焦点距離
画像座標系（ｉｍａｇｅ　ｃｏｏｒｄｉｎａｔｅ　ｓｙｓｔｅｍ　）
・ｕ軸：画面右向きを正
・ｖ軸：画面下向きを正
・（　ｕ０，　ｖ０）　：画面中心、すなわち光軸と画面とが交わる画像座標
【０１１９】
カメラ座標系の光学系は、中心射影モデル（ｐｅｒｓｐｅｃｔｉｖｅ　ｐｒｏｊｅｃｔｉｏｎ　ｍｏｄｅｌ）で記述されると仮定する。また、画像座標系は、画素を基本単位とする。
【０１２０】
また、以下の変数を定義する。
Ｒ_Ｗ，_ｔ−１：前時刻ｔ−１における、絶対座標系に対する撮像部３の姿勢を表す回転行列
Ｒ_Ｗ，_ｔ　：現在の時刻ｔ　の、絶対座標系に対する撮像部３の姿勢を表す回転行列
【０１２１】
上記２つの回転行列より、前時刻から現在にかけての撮像部３の姿勢の変化を示す回転行列Ｒを、次式に基づき計算する。
【０１２２】
【数２】

【０１２３】
ここで、Ｒ_Ｗ，_ｔ−１の右上の添え字Ｔは、行列の転置を表す。
【０１２４】
この回転行列Ｒにより表される姿勢の変化により、前時刻における被写体領域（ｕ_ｔ−１，ｖ_ｔ−１）は、現在の時刻において、次式に示される点（ｕ_ｔ，　ｖ_ｔ）　に移動する。
【０１２５】
【数３】

【０１２６】
であり、これらの行列Ａ_ｔ−１、Ａ_ｔは、それぞれ前時刻、現在における、カメラ座標系と画像座標系との関係を示すもので、カメラ内部行列（ｃａｍｅｒａ　ｉｎｔｒｉｎｓｉｃ　ｍａｔｒｉｘ　）と呼ばれる（参考文献：徐剛，　辻三郎：　『３次元ビジョン』，　共立出版　（１９９８））。
【０１２７】
また、ｋ_ｕ、ｋ_ｖは、それぞれｘ軸、ｙ軸を基準とした時のｕ軸、ｖ軸の単位、すなわち画素密度を表す。
【０１２８】
また、ｆ_ｔ−１、ｆ_ｔは、それぞれ前時刻、現在における焦点距離を表す変数である。また、Ｓは（３）式の左辺の第３成分を１にするための定数である。
【０１２９】
一般に、ｋ_ｕ、ｋ_ｖの値は、ＣＣＤの設計仕様より容易に知ることができる。
【０１３０】
また、焦点距離の値は、撮像部３において、レンズ駆動モータの位置をエンコーダにより読み取るなどの方法により、検出することができる。
以上の計算により、現在の被写体領域の位置（ｕ_ｔ，　ｖ_ｔ）　が算出される。
【０１３１】
続いて、以上の処理で算出された点（ｕ_ｔ　，ｖ_ｔ）　が、画面内にあるか否かを判定し（ステップＳ５５）、画面内にある場合には（Ｙｅｓ）、ステップＳ５３で求められた点（ｕ_ｔ　，ｖ_ｔ　）を、現在の被写体領域として、システム制御部２に出力する（ステップＳ５６）。
【０１３２】
また、画面外にあると判定された場合には（ステップＳ５５でＮｏ）、追跡に失敗したことを示す信号をシステム制御部２に出力する（ステップＳ５７）。以上で、処理を終了する。
【０１３３】
以下、本発明による撮像装置、カメラ、プログラム及び記録媒体の一実施形態たる第３の実施例について説明する。
【０１３４】
本発明に係る撮像装置等は、ＰＣ（Ｐｅｒｓｏｎａｌ　Ｃｏｍｐｕｔｅｒ　）や携帯情報端末により、その機能を実現させることができる。
【０１３５】
近年、各種処理プログラムを読み取ることにより、種々の機能を実現可能なデジタルカメラも登場するようになった。また、向きを回動可能なカメラを具備したノート型ＰＣや、カメラを搭載したＰＤＡ（Ｐｅｒｓｏｎａｌ　Ｄｉｇｉｔａｌ　Ａｓｓｉｓｔａｎｔ）及び携帯電話が普及するようになった。
【０１３６】
このような場合は、これらの機器に第１の実施例または第２の実施例で説明した機能を実現する処理プログラムを実行させることにより、その機能を実現させることができる。
【０１３７】
本実施例は、各種媒体やインターネット等の電気通信回線を介して、上記の機器で読み取り可能なプログラムである。動作の内容は、上述の第１の実施例または第２の実施例と同様である。
【０１３８】
以下、カメラを具備したノート型ＰＣにおいて、該プログラムを実行することにより、第１の実施例または第２の実施例で説明した前述の動作を実現する場合の具体例を説明する。
【０１３９】
ＡＦ等の処理は、該カメラの内部で実行される。
【０１４０】
撮影モードの選択、撮影の開始・終了の制御、被写体領域の追跡動作の開始（前述の実施例における、レリーズＳＷの半押し操作に相当）、画像及び被写体領域の表示は、任意のユーザー・インターフェイスを介して行われる。その一例を、以下に記す。
【０１４１】
撮影モードの選択：カーソルキー
撮影の開始・終了の制御：リターンキー
被写体領域の追跡動作の開始：スペースキー
画像及び被写体領域の表示：ＬＣＤに表示
【０１４２】
以下、本発明による撮像装置、カメラ、プログラム及び記録媒体の一実施形態たる第４の実施例について説明する。
【０１４３】
本発明に係る撮像装置は、第３の実施例で述べた機器に、第１の実施例又は第２の実施例で説明した機能を実現する処理プログラムを格納した記録媒体を読み取らせることにより、その機能を実現させることができる。
【０１４４】
例えば、図１３（ａ）に示されるように、当該プログラムを格納したＣＤ−ＲＯＭ２５を、カメラを具備したノート型ＰＣ３０に装着し、適宜当該プログラムを読み取らせ、実行させることにより、その機能を実現させることができる。
【０１４５】
また、図１３（ｂ）に示されるように、当該プログラムを格納したスマートメディア２６を、それを読み取り可能なデジタルスチルカメラ１に装着し、適宜当該プログラムを実行させることにより、その機能を実現させることができる。
【０１４６】
なお、当該プログラムを格納する記録媒体は上記のものに限られず、例えば、ＣＤ−ＲＷやＤＶＤ−ＲＯＭ等の別の媒体であってもよいことはいうまでもない。
【０１４７】
なお、本実施例における動作は、第３の実施例と同様である。
【０１４８】
以上の各実施例は、本発明のほんの一例を説明したにすぎない。すなわち、本発明の権利範囲を上記実施の形態の通りに不当に限定・縮小すべきではなく、下記のように、本発明の構成要素を別の要素に変更した例も、本発明と均等の発明として権利範囲に含まれる。
【０１４９】
例えば、各実施の形態において、２　ストロークの動作を検出できるレリーズＳＷ、及び静止画・動画撮影機能を具備したデジタルスチルカメラを例に挙げて説明したが、静止画撮影専用のデジタルスチルカメラ、及び動画撮影専用のビデオカメラにおいて、上記実施の形態と同様の機能を有するものであっても、本願の権利範囲に含まれる。ここで、該ビデオカメラが、上述の実施の形態で説明したのと同様の動作を実現する為に、被写体領域の追跡動作を開始するボタンを具備すると、さらに好適である。
【０１５０】
また、被写体領域をＬＣＤ６又はファインダー表示器８に表示する形態は、上記の実施の形態に限定されない。例えば、該表示機能を具備しない撮像装置（請求項６を除く請求の範囲に該当）、被写体領域を追跡している時にのみ該表示を行う撮像装置であっても構わない。
【０１５１】
また、被写体領域の記録については、１　秒毎に被写体領域の位置をテキストファイルに記録すると説明したが、各々のフレーム毎に記録する、またある一定数のフレーム毎に記録するなど、別の形態であっても構わない。また、ファイルフォーマットはテキストファイルではなく、ＭＰＥＧ−７（ＭＰＥＧは，Ｍｏｔｉｏｎ　Ｐｉｃｔｕｒｅ　Ｅｘｐｅｒｔｓ　Ｇｒｏｕｐの略）等の内容記述ファイルとして記録してもよい。このほか、静止画、動画共に、該被写体領域を画像データの中に、ウォーターマーク（電子透かし）として挿入するといった形態であっても構わない。
【０１５２】
また、第２の実施例において、加速度センサと磁気センサを組み合わせて使用することにより、撮像部の姿勢を検出すると説明したが、ジャイロなどの他の素子を使用しても、同様の動作を実現することが可能である。
【０１５３】
【発明の効果】
請求項１から２２記載の発明によれば、簡素な構成・動作で、画面上の正確な被写体領域を制御・追跡・記録することが可能となり、第１の目的が達成される。
【０１５４】
請求項６から１３、１７から２２記載の発明によれば、被写体領域の追跡に要する処理コスト及び消費電力を軽減することができ、第２の目的が達成される。
【０１５５】
請求項７から１３、１８から２２記載の発明によれば、ユーザーに面倒な操作を強いることがなく、被写体領域の指定及び追跡を実行することができ、第３の目的が達成される。
【０１５６】
請求項８から１３、１９から２２記載の発明によれば、簡素な構成で、ユーザーが所望の撮影動作を実行できているかを確認することができ、第４の目的が達成される。
【０１５７】
請求項１０から１３、２１、２２記載の発明によれば、ユーザーが後になって撮影意図・動作を容易に確認することができ、第５の目的が達成される。
【図面の簡単な説明】
【図１】本発明による撮像装置等の一実施形態に係るデジタルスチルカメラの構成図である。
【図２】本発明による撮像装置等の一実施形態に係るデジタルスチルカメラのレリーズＳＷの構成図である。
【図３】本発明による撮像装置等の一実施形態に係る処理の流れを示すフローチャートである。
【図４】本発明による撮像装置等の一実施形態に係る動画撮影の流れを示すフローチャートである。
【図５】本発明による撮像装置等の一実施形態に係る被写体領域又は警告の表示の説明図である。（ａ）は、被写体領域の位置を示すポインタが表示されている場合であり、（ｂ）は被写体領域の追跡に失敗した場合にその旨を警告するインジケータが表示されている場合である。
【図６】本発明による撮像装置等の一実施形態に係る被写体領域の追跡処理の流れを示すフローチャートである。
【図７】本発明による撮像装置等の一実施形態に係る被写体領域の追跡動作の説明図である。（ａ）は、被写体領域の所定の領域を切出したテンプレートあり、（ｂ）は、テンプレート現画像上で移動させた場合である。
【図８】本発明による撮像装置等の一実施形態に係る被写体領域の記録動作を示す図である。
【図９】本発明による撮像装置等の他の実施形態に係るデジタルスチルカメラの構成図である。
【図１０】本発明による撮像装置等の他の実施形態に係る被写体領域の追跡処理の流れを示すフローチャートである。
【図１１】本発明による撮像装置等の他の実施形態に係る絶対座標系を示す図である。
【図１２】本発明による撮像装置等の他の実施形態に係るカメラ座標系及び画像座標系を示す図である。
【図１３】本発明によるプログラム及び記録媒体等の一実施形態の説明図である。（ａ）には、カメラ付きのノート型ＰＣとＣＤ−ＲＯＭが、（ｂ）には、デジタルスチルカメラとスマートメディアが示されている。
【符号の説明】
１　デジタルスチルカメラ
２　システム制御部
３　撮像部
４　画像処理部
５　ＬＣＤ表示制御部
６　ＬＣＤ
７　ファインダー表示制御部
８　ファインダー表示器
９　記録メディアインターフェイス部
１０　記録メディア
１１　ハードキーインターフェイス部
１２　姿勢検出センサ
２１　ポインタ
２２　インジケータ
２３　テンプレート
２４　現画像
２５　ＣＤ−ＲＯＭ
２６　スマートメディア
３０　ノート型ＰＣ[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an imaging device, a camera, a program, and a recording medium, and particularly to an imaging device such as a video camera and a digital still camera, a program, and a recording medium.
[0002]
[Prior art]
For applications such as editing, management, search, and display of video signals, it has become increasingly important to add and record additional information to them.
[0003]
Among them, it is important to record the position of the subject of interest on the screen (hereinafter, referred to as the subject area). This is because there are advantages that the user's intention to shoot can be confirmed based on the subject region later, and that only the subject of interest can be managed and searched.
[0004]
Therefore, conventionally, a camera has been proposed in which a subject area is selected by inputting a line of sight and the subject area information is recorded on a recording medium such as a film (see Patent Documents 1, 2, and 3).
[0005]
Specifically, there is a main subject information recording camera that determines the position of a main subject in a shooting screen and records position information of the main subject on a recording medium (for example, see Patent Document 1).
[0006]
There is also a gaze detection camera that repeatedly detects a main subject based on the detected photographer's eye position information and records information of the detected plurality of main subjects on a storage medium on a film or a film cartridge ( For example, see Patent Document 2).
[0007]
Also, it is determined which of the plurality of subject position areas has been selected by the line of sight detection, and information on the subject position area, which is selected by the line of sight independently of the selection of the focus detection area, such as a film. There is also a camera with a line-of-sight detection function for recording on a recording medium (for example, see Patent Document 3).
[0008]
Further, a digital camera that records image data obtained by performing a shooting operation focused on a predetermined position in a shooting screen and focus position data defining a focus position on the shooting screen has also been proposed (for example, And Patent Document 4).
[0009]
That is, the digital camera records image data obtained by performing a shooting operation focused on a predetermined position in a shooting screen and focus position data defining a focus position on the shooting screen.
[0010]
There are also an imaging apparatus characterized by tracking and recording a focus detection area on an imaging screen, a program for executing the processing, and a recording medium storing the program.
[0011]
[Patent Document 1]
JP-A-5-158132
[Patent Document 2]
JP-A-7-319037
[Patent Document 3]
JP-A-8-328126
[Patent Document 4]
JP 2001-128034 A
[0012]
[Problems to be solved by the invention]
However, the above-mentioned prior arts have a main purpose of notifying a print trader (lab) of optimizing the image quality of a specific part based on the subject area information, and do not aim at the aforementioned application. Did not.
[0013]
Further, in the invention described in Patent Document 4, when the camera is moved after performing a focus lock operation by half-pressing the shutter button, the focus position changes, but in the embodiment described in the document, There is no proposed solution to this. Therefore, there is a problem that an erroneous focus position is recorded due to the movement of the camera during the focus lock operation. In addition, when focusing on a subject that is not the object of interest, there arises a problem that the user's intention to shoot is erroneously interpreted later.
[0014]
The present invention has been made in view of the above-described problems, and has an image pickup apparatus, a camera, and a program that can control, track, and record an accurate subject area on a screen with a simple configuration and operation. And a recording medium.
[0015]
It is a second object of the present invention to provide an imaging device, a camera, a program, and a recording medium that can reduce processing cost and power consumption required for tracking a subject area.
[0016]
A third object is to provide an imaging device, a camera, a program, and a recording medium that can specify and track a subject area without forcing a user to perform a troublesome operation.
[0017]
A fourth object is to provide an imaging device, a camera, a program, and a recording medium that enable a user to confirm whether a desired photographing operation can be performed with a simple configuration.
[0018]
A fifth object is to provide an imaging device, a camera, a program, and a recording medium that enable a user to easily confirm a photographing intention / operation later.
[0019]
[Means for Solving the Problems]
According to the first aspect of the present invention, the imaging apparatus includes: an imaging unit that outputs a subject image input via an optical system as a video signal; and a subject designation unit that designates a subject of interest in a field of view of the imaging unit. Video recording means for recording a video signal, tracking means for tracking a subject area designated by the subject designating means, and additional information recording means for recording the subject area in association with the video signal. Is what you do.
[0020]
According to the second aspect of the present invention, the imaging apparatus further includes a similarity calculating unit configured to calculate a similarity between the current video signal and the video signal captured one time ago. It is characterized by being.
[0021]
According to a third aspect of the present invention, the imaging device is the imaging device according to the second aspect, wherein the tracking means tracks the subject area based on the output of the similarity calculation means.
[0022]
According to a fourth aspect of the present invention, the imaging apparatus is the imaging apparatus according to the first, second, or third aspect, further including a direction detection unit that detects a direction of the imaging unit.
[0023]
According to a fifth aspect of the present invention, the imaging device is the imaging device according to the fourth aspect, wherein the tracking means tracks the subject area based on the output of the direction detecting means.
[0024]
According to the invention described in claim 6, the imaging device is the imaging device according to any one of claims 1 to 5, further including a start unit that starts operation of the tracking unit. is there.
[0025]
According to a seventh aspect of the present invention, the imaging apparatus is the imaging apparatus according to any one of the first to sixth aspects, wherein the subject designating unit stops operating while the tracking unit is operating. Is what you do.
[0026]
According to the invention described in claim 8, the imaging apparatus is the imaging apparatus according to any one of claims 1 to 7, further comprising a display unit that displays a subject area output by the tracking unit. Is what you do.
[0027]
According to the ninth aspect of the present invention, the imaging apparatus further includes a warning unit that issues a warning when the tracking unit determines that the subject area has deviated from the screen. It is characterized by being a device.
[0028]
According to a tenth aspect of the present invention, in the imaging apparatus, the additional information recording unit records text data describing a tracking result of the subject area. It is characterized by the following.
[0029]
According to an eleventh aspect of the present invention, a camera includes the imaging device according to any one of the first to tenth aspects.
[0030]
According to the twelfth aspect of the present invention, the program comprises: a process of outputting a subject image input via an optical system as a video signal; a subject designating process of designating a subject of interest in a field of view of an imaging unit; It is characterized by executing a process of recording a signal, a tracking process of tracking a subject region designated by the subject designation process, and a process of recording the subject region in association with a video signal.
[0031]
According to the thirteenth aspect, the program further executes a similarity calculation process of calculating a similarity between the current video signal and the video signal captured one time ago. It is characterized by having.
[0032]
According to a fourteenth aspect of the present invention, the program is the program according to the thirteenth aspect, further causing a tracking process for tracking a subject area based on an output of the similarity calculation process.
[0033]
According to a fifteenth aspect of the present invention, the program is the program according to the twelfth, thirteenth, or fourteenth aspect, wherein the program further executes a direction detection process for detecting a direction of the imaging unit.
[0034]
According to a sixteenth aspect of the present invention, the program is the program according to the fifteenth aspect, further executing a tracking process of tracking a subject area based on an output of the orientation detection process.
[0035]
According to a seventeenth aspect of the present invention, the program is the program according to any one of the twelfth to sixteenth aspects, wherein the program further executes a process of starting an operation of the tracking process.
[0036]
According to the invention described in claim 18, the program is the program according to any one of claims 12 to 17, wherein the subject designation process further executes a process of stopping the operation during the operation of the tracking process. It is characterized by the following.
[0037]
According to a nineteenth aspect, the program is the program according to any one of the twelfth to eighteenth aspects, further causing a process of displaying a subject area output by the tracking process to be executed. Things.
[0038]
According to the twentieth aspect of the present invention, the program according to any one of the twelfth to nineteenth aspects, further comprising executing a process of issuing a warning when it is determined by the tracking process that the subject area has deviated from the screen. It is characterized by being.
[0039]
According to a twenty-first aspect, the program is the program according to any one of the twelfth to twentieth aspects, wherein the program further executes a process of recording text data describing a tracking result of the subject area. It is assumed that.
[0040]
According to a twenty-second aspect of the present invention, a recording medium is a computer-readable recording medium that records the program according to any one of the twelfth to twenty-first aspects.
[0041]
BEST MODE FOR CARRYING OUT THE INVENTION
Next, embodiments of an imaging device, a camera, a program, and a recording medium according to the present invention will be described in detail with reference to the accompanying drawings.
[0042]
First, taking a digital still camera as an example, a first example as an embodiment of an imaging apparatus, a camera, a program, and a recording medium according to the present invention will be described.
[004 3]
FIG. 1 is a diagram illustrating a configuration of a digital still camera according to an embodiment of an imaging apparatus, a camera, a program, and a recording medium according to the present invention.
[004 4]
In FIG. 1, a digital still camera device 1 includes a system control unit 2, an imaging unit 3, an image processing unit 4, an LCD display control unit 5, an LCD 6, a finder display control unit 7, a finder display 8, , A recording medium interface unit 9, a recording medium 10, and a hard key interface unit 11.
[004 5]
The system control unit 2 includes a CPU (Central Processing Unit), a memory, a timer, and the like, which are provided for controlling the entire digital still camera device 1.
[004 6]
The imaging unit 3 includes an optical system component (a lens and a lens drive motor), a CCD (Charge Coupled Device), a CCD drive circuit, an A / D converter, and the like, which are provided for imaging.
[004 7]
The image processing unit 4 performs various image processing on the image signal obtained by the imaging unit 3 and performs focusing, exposure adjustment, and the like by controlling the CCD drive timing of the imaging unit 3 and the lens drive motor. It comprises an image processing DSP (Digital Signal Processor), a memory, and the like, which are provided to perform compression and decompression of the image data.
[004 8]
The LCD display control unit 5 performs signal processing for displaying the image signal processed by the image processing unit 4 on an LCD (Liquid Crystal Display), generates various graphic images for a user interface, and outputs the image to the LCD. It comprises a D / A converter, an on-screen display controller, etc., provided for displaying.
[0049]
The LCD 6 is provided for displaying an image and for displaying a graphic for a user interface.
[0050]
The finder display control unit 7 performs signal processing for displaying the image signal processed by the image processing unit 4 on a display installed inside the finder, and generates various graphic images for a user interface. It comprises a D / A converter, an on-screen display controller, etc., provided for displaying on the display.
[0051]
The finder display 8 is provided for displaying an image incident on the optical system of the imaging unit 3 and for displaying graphics for a user interface.
[0052]
The recording medium interface unit 9 includes a memory card controller and the like provided for an interface with the recording medium.
[0053]
The recording medium 10 includes a flash memory or the like provided for storing compressed image signals and various information related to images, and is detachable from the digital still camera device 1.
[0054]
The hard key interface unit 11 includes a parallel port and the like provided for detecting a state of hardware for a user interface such as a release switch (not shown), a key (not shown), and a dial (not shown).
[0055]
Here, the configuration of the release SW will be described with reference to FIG. FIG. 2 shows a circuit diagram of the release SW.
[0056]
In FIG. 2, RL1 indicates a contact that is turned on when the release SW is half-pressed. RL2 indicates a contact that is turned on when the release SW is completely pressed. COM indicates a common contact point for both RL1 and RL2.
[0057]
As described above, the release SW is configured to detect a two-stroke operation. The state of the contacts of RL1 and RL2 is determined by the system control unit 2 through the hard key interface unit 11 as described above.
[0058]
Next, the operation according to the first embodiment will be described with reference to the flowchart in FIG.
[0059]
Prior to shooting, the user operates various keys and dials (not shown) to determine a shooting mode (still image shooting mode, moving image shooting mode). The contents of the user's operation are determined by the system control unit 2 through the hard key interface unit 11 (step S1), and in the still image shooting mode (No in step S1), the processes after step S2 are executed as described below. You. In the moving image shooting mode (Yes in step S1), as described later, the operation is performed according to the flowchart shown in FIG. 4 (see step S23).
[0060]
In the case of No in step S1 of FIG. 3, that is, in the case of the still image shooting mode, then, in step S2, the digital still camera device 1 performs the following work for preparing for shooting.
[0061]
That is, the system control unit 2 sets the processing parameters corresponding to the mode in the image processing unit 4. Under the control of the image processing unit 4, the imaging unit 3 starts an imaging operation for displaying a preview image prior to actual shooting. The captured data is continuously sent to the image processing unit 4, which performs processing such as color space conversion, gamma correction, and white balance adjustment, and then sends the image data to the LCD display control unit 5.
[0062]
The LCD display control unit 5 performs signal processing on the image data, displays the signal on the LCD 6, and presents the imaging state to the user.
[0063]
Further, the finder display control section 7 displays various states of the digital still camera 1 on the finder display 8 and presents an imaging state to the user.
[0064]
Further, in this step, the position of the subject area is determined. The subject area is fixed at the center of the screen in advance, and is determined by a known technique such as line-of-sight detection or selection through a user interface such as the hard key interface unit 11.
[0065]
After the preparation for photographing in step S2 is completed, in step S4, when the system control unit 2 determines that the release SW is half-pressed, that is, the RL1 contact is turned on (Yes), focus detection is performed in step S5. (AF), an automatic exposure (AE), and an automatic white balance (AWB) algorithm program are started, and the image processing unit 4 and the imaging unit 3 are controlled based on various evaluation data obtained from the image processing unit 4. Make adjustments.
Here, the determination of the half-pressed state of the release SW is executed by the system control unit 2 through the hard key interface unit 11 in the same manner as the mode setting.
[0066]
When the above process is completed, the digital still camera device 1 starts tracking the subject area in the imaging range of the imaging unit 3 in step S6. That is, the half-press operation of the release SW functions as means for starting the tracking operation of the subject area. The specific contents of this operation will be described later.
[0067]
At the time of the tracking operation of the subject area, which will be described later, the processing cost and the power consumption of the system control unit 2 are not small. As described above, the processing cost and the power consumption can be reduced by executing the tracking operation of the subject area only when the release SW is half-pressed.
[0068]
Further, in the present embodiment, the designation of the subject area and the tracking operation of the subject area are exclusively executed. This simplifies the processing because the designation of the subject area by the user and the tracking operation of the subject area do not interfere with each other. In addition, since it is not necessary to keep the user's gaze on the object of interest during the half-press operation of the release SW or to operate the hard key interface unit 11, there is an advantage that the photographing operation becomes very simple.
[0069]
When the subject area is tracked, a pointer 21 indicating the position of the subject area is displayed on the LCD 6 or the finder display 8 as shown in FIG. When the tracking of the subject area has failed, such as when the subject area goes off the screen, an indicator 22 is displayed as shown in FIG. 5B to warn the user of the failure (step S7).
[0070]
The system control unit 2 continuously monitors the state of the release SW through the hard key interface unit 11 (step S8). If it is determined that the release SW is fully pressed, that is, the RL2 contact is turned on (Yes), the following shooting is performed. The operation starts (step S9).
Here, the determination of the full-press state of the release SW is performed by the system control unit 2 through the hard key interface unit 11 in the same manner as the detection operation of the half-press state of the release SW.
[0071]
The imaging unit 3 captures a predetermined image and sends it to the image processing unit 4 under the control of the image processing unit 4, and the image processing unit 4 performs image processing and compression processing according to the shooting mode.
Then, the system control unit 2 reads out the compressed image data and the tracked subject area, adds header information thereto, and records the data on the recording medium 10 through the recording medium interface unit 9 (step S10).
The recording of the subject area will be described later.
[0072]
When the photographing and recording of the image are completed as described above, the tracking process of the subject area ends (step S12).
[0073]
If it is determined in step S8 that the RL2 contact is off (No), the system control unit 2 detects the state of the RL1 contact (step S11). If it is determined that the RL1 contact is on (Yes), the process returns to step S6, and the processing up to step S11 is repeated. If it is determined that the RL1 contact has been released (No in step S11), the processing of AE, AF, and AWB and the tracking processing of the subject area are terminated (step S12).
[0074]
Finally, when each operation is completed by the power-off detection, the power is turned off and the process ends (Yes in step S13).
[0075]
Here, the operation (step S3) when the moving image shooting mode is selected in step S1 of FIG. 3 will be described based on the flowchart of FIG.
[0076]
First, in step S21 of FIG. 4, in the digital still camera device 1, an operation of preparing for moving image shooting is performed. The operation is similar to that of step S2 in FIG.
[0077]
When it is determined that the release SW is half-pressed after the preparation for shooting is completed, that is, the RL1 contact is turned on (Yes in step S22), as described above, focus detection (AF), automatic exposure (AE), and auto white The balance (AWB) process is started (step S23). This adjustment operation is continuously performed until a half-press of the release SW (RL1 contact ON) is detected, as described later.
[0078]
If it is determined that the RL1 contact is off (No in step S22), the process proceeds to step S35 described later.
[0079]
The system control unit 2 continuously monitors the state of the release SW through the hard key interface unit 11, and when the release SW is pressed as it is (RL2 contact ON, that is, Yes in step S24), a moving image recording operation in step S26 and later described later. Conversely, when the release SW is half-pressed (RL1 contact OFF, that is, No in step S25), the AF, AE, and AWB processes described above are terminated (step S34).
[0080]
When the recording operation is started (step S26), the system control unit 2 monitors the state of the release SW through the hard key interface unit 11 and detects the fully pressed state (RL2 contact ON) of the release SW (Yes in step S27). ), The recording of the moving image is terminated (step S32), and if not, the processing from step S28 is executed. That is, the recording operation is continued until the full press of the release SW during the recording operation is detected.
[0081]
During the recording operation, when the half-press of the release SW is detected (RL1 contact is turned on, that is, Yes in step S28), the operation of tracking the subject area is started.
[0082]
In step S29, the digital still camera device 1 starts tracking the subject area in the imaging range of the imaging unit 3. That is, the half-press operation of the release SW during recording of a moving image functions as a unit for starting a tracking operation of the subject area. The specific contents of this operation will be described later.
[0083]
At the time of the tracking operation of the subject area, which will be described later, the processing cost and the power consumption of the system control unit 2 are not small. As described above, the processing cost and the power consumption described above can be reduced by executing the tracking operation of the subject area only when the release SW is half-pressed.
[0084]
As in the still image mode, the designation of the subject area and the tracking operation of the subject area are exclusively executed. This simplifies the processing because the designation of the subject area by the user and the tracking operation of the subject area do not interfere with each other. In addition, there is no need to keep the line of sight on the subject of interest during shooting or to operate the hard key interface unit 11, and there is an advantage that the shooting operation is very simple.
[0085]
On the other hand, if the release SW is released during the recording operation, the above-described focus detection tracking operation is not performed.
[0086]
When the subject area is tracked, a pointer 21 indicating the position of the subject area is displayed on the LCD 6 or the finder display 8 as shown in FIG.
[0087]
When the tracking of the subject area has failed, such as when the subject area is off the screen, an indicator 22 is displayed to warn the user of the failure, as shown in FIG. 5B (step S30).
[0088]
When the focus detection tracking operation is not performed, the pointer 21 may be displayed at a position of a predetermined subject area (for example, at the center of the screen).
[0089]
The processes of steps S27 to S31 are repeated until the full-press of the release SW (RL2 contact ON) is detected, or the change operation of the shooting mode or the power-off operation is detected (Yes in step S31). Be executed.
[0090]
When the recording of the moving image has been completed (step S32), the system control unit 2 writes the tracking history of the subject area on the recording medium 10 in association with the moving image data (step S33). This operation will be described later.
[0091]
Finally, when a change operation of the shooting mode or a power-off operation is detected, the flow of FIG. 4 is ended (Yes in step S35), otherwise (No in step S35), the processing of steps S22 to S34 is performed. It is executed repeatedly.
[0092]
Here, the tracking operation of the subject area in the embodiment of the imaging device, the camera, the program, and the recording medium according to the present invention will be described based on the flowchart of FIG.
[0093]
First, when the tracking process of the subject area is in the initial state, that is, when the tracking process is executed for the first time after the half-press of the release SW (RL1 contact ON) is detected (step S41), the system control unit 2 performs the imaging process. A predetermined area (hereinafter, referred to as a template) corresponding to the subject area determined in step S2 of FIG. 3 is extracted from the image (hereinafter, referred to as a current image) currently acquired by the unit 3 as described above. Is recorded in the memory (step S42).
This is the end of the process.
[0094]
If the template is not in the initial state (No in step S41), the template recorded in the previous process is read from the memory (step S43), and an area having a high similarity with the template in the current image is detected. .
[0095]
Here, an operation when template matching is used as a means for detecting a portion having a high degree of similarity will be described with reference to FIG.
[0096]
7A, it is assumed that the size of the template is (2du + 1) × (2dv + 1) The template 23 is moved on the current image 24 as shown in FIG. A normalized cross-correlation value S with the point (m, n) in the image 24 is calculated based on the following equation.
[0097]
(Equation 1)

[0098]
Here, the meaning of each symbol in the expression (1) is as follows.
・ I ₁ (U, v): pixel value at point (u, v) on the template
・ I ₂ (U, v): pixel value at point (u, v) on the current image
Based on the above calculation, the point (m, n) in the current image at which the normalized cross-correlation value S is maximum is obtained.
[0099]
Next, in step S45 of FIG. 6, it is determined whether the above-described template matching is successful. Here, as a criterion for determining the success or failure of the template matching, for example, a threshold θ of the normalized cross-correlation value S is determined in advance, and if the value of S is equal to or more than θ, success is determined, and if the value of S is less than θ, failure is determined. Just fine.
[0100]
If it is determined to be successful (Yes in step S45), the point (m, n) obtained in step S43 is output to the system control unit 2 as a subject area in the current image, and at the same time, the point (m , N) is updated as the template with a size of (2du + 1) × (2dv + 1) (step S46). If it is determined that the tracking has failed (No in step S45), a signal indicating that tracking has failed is output to the system control unit 2 without updating the template (step S47). This is the end of the process.
[0101]
Next, recording of a subject area in an embodiment of the imaging device, the camera, the program, and the recording medium according to the present invention will be described.
[0102]
In the case of a still image, data relating to the subject area is recorded in header information in the acquired image. For example, when Exif (exchangeable image file format) is adopted as a format of a still image of a digital still camera, the subject is included in a field called Subject Area in the header information where the position of a main subject can be described. The location of the area can be recorded.
[0103]
Also, in the case of a moving image, as shown in FIG. 8, the time is recorded as text information in the order of the time in seconds from the left, the horizontal coordinate u of the subject area, and the vertical coordinate v of the subject area. This text information is recorded on a recording medium as a text file together with the moving image.
[0104]
At this time, the moving image data and the text file are recorded in association with each other. For example, if the name of the moving image file is "Video 01.avi" (avi is an abbreviation for Audio Video Interleaving), the name of the text file is "Video 01.txt", and the name before the extension is aligned. Recorded in the form.
[0105]
Hereinafter, a second example which is an embodiment of the imaging apparatus, the camera, the program, and the recording medium according to the present invention will be described.
[0106]
The above-described focus area detection processing may be performed by detecting the direction of the imaging unit 3. This method is particularly effective when a still object is photographed without moving the imaging unit 3 in parallel with the distance to the object.
[0107]
FIG. 9 is a diagram illustrating a configuration of the digital still camera according to the present embodiment. The digital still camera 1 in FIG. 9 is obtained by adding a posture detection sensor 12 to the digital still camera in FIG.
The same reference numerals in the drawings denote the same or corresponding parts as in the first embodiment.
[0108]
The attitude detection sensor 12 is provided for detecting the direction of the main axis (usually the optical axis) of the imaging unit 3, an attitude detection sensor, an A / D converter that acquires a signal from the sensor, and the A / D 4 shows an attitude detection unit including an ALU or the like that performs arithmetic processing on data acquired by a converter.
[0109]
The posture detection sensor 12 is configured by, for example, an acceleration sensor, a magnetic sensor, an angular velocity sensor (gyro), or a combination thereof.
[0110]
The operations in the still image shooting mode and the moving image shooting mode in the second embodiment are the same as those in the first embodiment, and operate according to the flowcharts of FIGS.
[0111]
Here, the tracking operation of the subject area when an acceleration sensor and a magnetic sensor are used as an example of the posture detection sensor will be described with reference to the flowchart of FIG.
[0112]
First, the system control unit 2 calculates the current attitude of the imaging unit 3 with respect to the absolute coordinate system from the detection signal of the attitude detection sensor, and records the current attitude in the memory (step S51).
[0113]
Here, it is assumed that the absolute coordinate system is a right-handed XYZ coordinate system in which the eastward direction is the X axis, the gravity direction is the Y axis, and the northward direction is the Z axis, as shown in FIG.
[0114]
The acceleration sensor is used to detect the gravitational acceleration, and the magnetic sensor is used to detect the terrestrial magnetism. The output signals of these sensors make it possible to easily calculate the orientation of the imaging unit 3 with respect to the absolute coordinate system.
[0115]
Here, when the tracking process of the subject area is in the above-described initial state (Yes in step S52), the process ends here.
[0116]
On the other hand, if it is not the initial state (No in step S52), the subject area and the attitude of the imaging unit 3 recorded at the previous time are read out from the memory (step S53), and these information and the current attitude of the imaging unit 3 are read. Is used to calculate the current subject area and record it in the memory (step S54).
[0117]
Here, a method of calculating the current subject area will be described.
[0118]
First, two coordinate systems shown in FIG. 12 are defined.
Camera coordinate system
X-axis: Positive to the right of the screen
・ Y-axis: Positive screen downward
・ Z axis: optical axis, direction toward object is positive
・ Origin 0: Optical center of camera
・ F: focal length
Image coordinate system
・ U-axis: Positive to screen right
・ V-axis: screen downward is positive
• (u0, v0): center of the screen, ie, image coordinates where the optical axis and the screen intersect
[0119]
It is assumed that the optical system of the camera coordinate system is described by a central projection model. The image coordinate system uses pixels as a basic unit.
[0120]
The following variables are defined.
R _W , _t-1 : A rotation matrix representing the attitude of the imaging unit 3 with respect to the absolute coordinate system at the previous time t-1
R _W , _t : Rotation matrix representing the attitude of the imaging unit 3 with respect to the absolute coordinate system at the current time t 2
[0121]
From the two rotation matrices, a rotation matrix R indicating a change in the attitude of the imaging unit 3 from the previous time to the present is calculated based on the following equation.
[0122]
(Equation 2)

[0123]
Where R _W , _t-1 The subscript T at the upper right of the matrix represents the transposition of the matrix.
[0124]
Due to the change in the posture represented by the rotation matrix R, the subject area (u _t-1 , V _t-1 ) At the current time point (u _t , V _t Go to).
[0125]
[Equation 3]

[0126]
And these matrices A _t-1 , A _t Indicates the relationship between the camera coordinate system and the image coordinate system at the previous time and the present, respectively, and is called a camera internal matrix (reference: Xu Go, Saburo Tsuji: “3D Vision”, Kyoritsu) Publishing (1998)).
[0127]
Also, k _u , K _v Represents a unit of the u-axis and the v-axis with reference to the x-axis and the y-axis, respectively, that is, a pixel density.
[0128]
Also, f _t-1 , F _t Are variables representing the focal length at the previous time and the present, respectively. S is a constant for setting the third component on the left side of the equation (3) to 1.
[0129]
In general, k _u , K _v Can be easily known from the CCD design specifications.
[0130]
Further, the value of the focal length can be detected by a method of reading the position of the lens driving motor by the encoder in the imaging unit 3.
By the above calculation, the current position (u _t , V _t ) Is calculated.
[0131]
Subsequently, the points (u _t , V _t ) Is determined to be within the screen (step S55). If it is within the screen (Yes), the point (u) determined at step S53 is determined. _t , V _t ) Is output to the system control unit 2 as the current subject area (step S56).
[0132]
If it is determined that it is outside the screen (No in step S55), a signal indicating that the tracking has failed is output to the system control unit 2 (step S57). This is the end of the process.
[0133]
Hereinafter, a third example which is an embodiment of the imaging apparatus, the camera, the program, and the recording medium according to the present invention will be described.
[0134]
The functions of the imaging device and the like according to the present invention can be realized by a personal computer (PC) or a portable information terminal.
[0135]
In recent years, digital cameras that can realize various functions by reading various processing programs have come to appear. In addition, a notebook PC equipped with a camera whose direction can be rotated, a PDA (Personal Digital Assistant) equipped with the camera, and a mobile phone have become widespread.
[0136]
In such a case, the functions can be realized by causing these devices to execute a processing program that realizes the functions described in the first embodiment or the second embodiment.
[0137]
The present embodiment is a program that can be read by the above-described device via various media or an electric communication line such as the Internet. The contents of the operation are the same as those in the above-described first or second embodiment.
[0138]
Hereinafter, a specific example in which the above-described operation described in the first embodiment or the second embodiment is realized by executing the program in a notebook PC including a camera will be described.
[0139]
Processing such as AF is executed inside the camera.
[0140]
Selection of a shooting mode, control of start / end of shooting, start of tracking operation of a subject area (corresponding to half-pressing operation of the release SW in the above-described embodiment), display of an image and a subject area are performed by an arbitrary user interface. Done through. An example is described below.
[0141]
Select the shooting mode: Cursor key
Shooting start / end control: Return key
Start tracking of subject area: Space key
Display of image and subject area: Display on LCD
[0142]
Hereinafter, a fourth example which is an embodiment of the imaging apparatus, the camera, the program, and the recording medium according to the present invention will be described.
[0143]
The imaging apparatus according to the present invention causes a device described in the third embodiment to read a recording medium storing a processing program for realizing the functions described in the first embodiment or the second embodiment, That function can be realized.
[0144]
For example, as shown in FIG. 13A, the function is realized by mounting the CD-ROM 25 storing the program on a notebook PC 30 equipped with a camera, and reading and executing the program as appropriate. Can be done.
[0145]
Further, as shown in FIG. 13B, the smart media 26 storing the program is mounted on the digital still camera 1 that can read the program, and the program is appropriately executed to realize the function. be able to.
[0146]
It should be noted that the recording medium for storing the program is not limited to the above-described one, but may be another medium such as a CD-RW or a DVD-ROM.
[0147]
The operation in this embodiment is the same as that in the third embodiment.
[0148]
The above embodiments are merely illustrative of the invention. That is, the scope of the rights of the present invention should not be unduly limited or reduced as in the above-described embodiment, and examples in which the constituent elements of the present invention are changed to other elements as described below are equivalent to the present invention. It is included in the scope of right as an invention.
[0149]
For example, in each embodiment, the release SW capable of detecting a two-stroke operation and a digital still camera having a still image / moving image shooting function have been described as examples. Even a video camera dedicated to shooting moving images having the same function as the above embodiment is also included in the scope of the present application. Here, it is more preferable that the video camera includes a button for starting a tracking operation of the subject area in order to realize the same operation as that described in the above embodiment.
[0150]
The manner in which the subject area is displayed on the LCD 6 or the finder display 8 is not limited to the above embodiment. For example, an imaging device that does not have the display function (corresponding to claims other than claim 6) or an imaging device that performs the display only when tracking the subject area may be used.
[0151]
In the description of the recording of the subject area, the position of the subject area is recorded in the text file every second. However, another form such as recording each frame or recording a certain number of frames is used. It does not matter. The file format may be recorded as a content description file such as MPEG-7 (MPEG stands for Motion Picture Experts Group) instead of a text file. In addition, for both still images and moving images, the subject area may be inserted into the image data as a watermark (digital watermark).
[0152]
Further, in the second embodiment, it has been described that the attitude of the imaging unit is detected by using the acceleration sensor and the magnetic sensor in combination. However, the same operation can be realized by using another element such as a gyro. It is possible to
[0153]
【The invention's effect】
According to the invention as set forth in claims 1 to 22, it is possible to control, track, and record an accurate subject area on the screen with a simple configuration and operation, and the first object is achieved.
[0154]
According to the inventions described in claims 6 to 13 and 17 to 22, the processing cost and power consumption required for tracking the subject area can be reduced, and the second object is achieved.
[0155]
According to the seventh to thirteenth and eighteenth to twenty-second aspects, designation and tracking of a subject area can be executed without forcing the user to perform a troublesome operation, and the third object is achieved.
[0156]
According to the inventions described in claims 8 to 13 and 19 to 22, it is possible to confirm with a simple configuration whether the user can execute a desired photographing operation, and the fourth object is achieved.
[0157]
According to the tenth to thirteenth, twenty-first, twenty-second, and twenty-second aspects, the user can easily confirm the photographing intention and operation later, and the fifth object is achieved.
[Brief description of the drawings]
FIG. 1 is a configuration diagram of a digital still camera according to an embodiment of an imaging device and the like according to the present invention.
FIG. 2 is a configuration diagram of a release switch of a digital still camera according to an embodiment of an imaging device or the like according to the present invention.
FIG. 3 is a flowchart illustrating a flow of a process according to an embodiment of an imaging device or the like according to the present invention.
FIG. 4 is a flowchart illustrating a flow of moving image capturing according to an embodiment of an imaging device or the like according to the present invention.
FIG. 5 is an explanatory diagram of a display of a subject area or a warning according to an embodiment of an imaging device or the like according to the present invention. (A) shows a case where a pointer indicating the position of the subject area is displayed, and (b) shows a case where an indicator for warning that the tracking of the subject area has failed is displayed.
FIG. 6 is a flowchart showing a flow of a tracking process of a subject area according to an embodiment of the imaging device or the like according to the present invention.
FIG. 7 is an explanatory diagram of a tracking operation of a subject area according to an embodiment of an imaging device or the like according to the present invention. (A) shows a template obtained by cutting out a predetermined area of the subject area, and (b) shows a case where the template is moved on the template current image.
FIG. 8 is a diagram illustrating a recording operation of a subject area according to an embodiment of an imaging device or the like according to the present invention.
FIG. 9 is a configuration diagram of a digital still camera according to another embodiment such as an imaging device according to the present invention.
FIG. 10 is a flowchart illustrating a flow of a subject area tracking process according to another embodiment of the imaging apparatus and the like according to the present invention.
FIG. 11 is a diagram showing an absolute coordinate system according to another embodiment such as an imaging device according to the present invention.
FIG. 12 is a diagram illustrating a camera coordinate system and an image coordinate system according to another embodiment of the imaging device and the like according to the present invention.
FIG. 13 is an explanatory diagram of an embodiment of a program, a recording medium, and the like according to the present invention. (A) shows a notebook PC with a camera and a CD-ROM, and (b) shows a digital still camera and a smart media.
[Explanation of symbols]
1 Digital still camera
2 System control unit
3 Imaging unit
4 Image processing unit
5 LCD display controller
6 LCD
7 Finder display control unit
8 Viewfinder display
9 Recording media interface
10 Recording media
11 Hard key interface
12 Attitude detection sensor
21 pointer
22 Indicator
23 templates
24 Current image
25 CD-ROM
26 Smart Media
30 Notebook PC

Claims (22)

Imaging means for outputting a subject image input via an optical system as a video signal,
Subject designating means for designating a subject of interest in the field of view of the imaging means;
Video recording means for recording the video signal,
Tracking means for tracking a subject area designated by the subject designating means;
Additional information recording means for recording the subject area in association with the video signal,
An imaging device comprising: A current video signal, and a similarity calculating unit that calculates a similarity between the video signal captured one time ago,
The imaging device according to claim 1, further comprising: The tracking means,
The imaging apparatus according to claim 2, wherein the subject area is tracked based on an output of the similarity calculation unit. Direction detection means for detecting the direction of the imaging means,
The imaging device according to claim 1, 2 or 3, further comprising: The tracking means,
The imaging apparatus according to claim 4, wherein the object area is tracked based on an output of the direction detection unit. Starting means for starting the operation of the tracking means,
The imaging device according to any one of claims 1 to 5, further comprising: The subject designating means includes:
The imaging apparatus according to claim 1, wherein the operation is stopped while the tracking unit is operating. Display means for displaying the subject area output by the tracking means,
The imaging device according to any one of claims 1 to 7, further comprising: Warning means for issuing a warning when the tracking means determines that the subject area has deviated from the screen,
The imaging device according to any one of claims 1 to 8, further comprising: The additional information recording means,
The imaging apparatus according to claim 1, wherein text data describing a tracking result of the subject area is recorded. A camera comprising the imaging device according to claim 1. A process of outputting a subject image input via an optical system as a video signal,
Subject designation processing for designating a subject of interest in the field of view of the imaging means;
Processing for recording the video signal;
A tracking process of tracking a subject area designated by the subject designation process;
Recording the subject area in association with the video signal. 13. The program according to claim 12, further comprising executing a similarity calculation process of calculating a similarity between the current video signal and the video signal captured one time before. 14. The non-transitory computer-readable storage medium according to claim 13, further comprising: executing the tracking process of tracking the subject area based on an output of the similarity calculation process. 15. The program according to claim 12, further comprising executing a direction detection process for detecting a direction of the imaging unit. 16. The program according to claim 15, further comprising: executing the tracking processing for tracking the subject area based on an output of the orientation detection processing. 17. The program according to claim 12, further comprising executing a process of starting the operation of the tracking process. 18. The program according to claim 12, wherein the subject specifying process further executes a process of stopping an operation during the operation of the tracking process. 19. The program according to claim 12, further comprising executing a process of displaying the subject area output by the tracking process. 20. The program according to claim 12, further comprising: executing a process of issuing a warning when it is determined that the subject area has deviated from the screen by the tracking process. 21. The program according to claim 12, further comprising the step of: recording text data describing a tracking result of the subject area. A computer-readable recording medium on which the program according to any one of claims 12 to 21 is recorded.

Images