JP2011130384A5

JP2011130384A5 -

Info

Publication number: JP2011130384A5
Application number: JP2009289653A
Authority: JP
Filing date: 2009-12-21
Publication date: 2012-12-06
Anticipated expiration: 2029-12-21

Description

上述の目的は、画像中の特定の被写体を、時系列的に入力される複数の入力画像に亘って追跡する被写体追跡装置であって、登録手段に登録された複数の基準画像の各々について、入力画像との最大相関度と、当最大相関度を有する入力画像中の領域とを求める算出手段と、算出手段により求められた領域の中から、入力画像における特定の被写体の領域を判定する第１の判定手段とを有し、複数の基準画像が、複数の入力画像のうち第１の入力画像において特定の被写体の領域として指定された領域に基づく第１の基準画像と、複数の入力画像のうち第１の入力画像よりも後に入力される複数の第２の入力画像において、第１の判定手段によって特定の被写体の領域と判定された領域に基づく第２の基準画像であり、第１の判定手段は、第１の基準画像について算出手段が求めた最大相関度が予め定めた閾値以上であれば、第２の基準画像と最大相関度を有する領域によらずに第１の基準画像と最大相関度を有する領域を入力画像における特定の被写体の領域と判定し、第１の基準画像について算出手段が求めた最大相関度が予め定めた閾値未満であれば、第１の基準画像についての最大相関度と第２の基準画像についての最大相関度とに基づいて、第１の基準画像と最大相関度を有する領域と第２の基準画像と最大相関度を有する領域のいずれかを、入力画像における特定の被写体の領域と判定することを特徴とする被写体追跡装置によって達成される。 The object described above is a subject tracking device that tracks a specific subject in an image over a plurality of input images input in time series, and each of a plurality of reference images registered in a registration unit A calculating means for determining a maximum correlation degree with the input image and an area in the input image having the maximum correlation degree, and determining a region of a specific subject in the input image from the areas determined by the calculating means; and a first determination unit, a plurality of reference images, and based Ku first reference image in the specified area as an area of the specific subject in the first input image among the plurality of input images, a plurality of a plurality of second input image input later than the first input image of the input image, based Ku second reference in the area is determined as the region of the specific subject by the first judging means The first determination means is an image. If the threshold value or more to the maximum degree of correlation is computing means obtained for the reference image is determined in advance, the region having a first reference image and the maximum correlation regardless of the region having a second reference image and the maximum correlation If the maximum correlation obtained by the calculation means for the first reference image is less than a predetermined threshold value , the maximum correlation for the first reference image and the second Based on the maximum correlation degree for the reference image, the first reference image, the area having the maximum correlation degree, or the second reference image and the area having the maximum correlation degree are determined based on the specific object in the input image. The object tracking device is characterized in that it is determined as an area.

（第１の実施形態）
図１は、本発明の第１の実施形態に係る被写体追跡装置の一例としての撮像装置１００の機能構成例を示すブロック図である。レンズユニット１０１は結像光学系であり、一般にはフォーカス調整用レンズを含む複数のレンズによって構成される。レンズユニット１０１は被写体の光学像を撮像素子１０２の撮像面上に結像する。撮像素子１０２はＣＣＤイメージセンサやＣＭＯＳイメージセンサなどの光電変換素子であり、被写体光学像を画素単位の電気信号に変換する。この電気信号は、撮像素子１０２で撮像された被写体像を示すアナログの映像信号である。撮像素子１０２は、撮像と出力を連続的に繰り返すことにより、所定のフレームレートで時系列的に映像信号を出力することができる。以下の説明において、「画像」とは、１フレームの映像信号と同義である。 (First embodiment)
FIG. 1 is a block diagram illustrating a functional configuration example of an imaging apparatus 100 as an example of a subject tracking apparatus according to the first embodiment of the present invention. The lens unit 101 is an imaging optical system, and generally includes a plurality of lenses including a focus adjustment lens. The lens unit 101 forms an optical image of the subject on the imaging surface of the image sensor 102. The imaging element 102 is a photoelectric conversion element such as a CCD image sensor or a CMOS image sensor, and converts the subject optical image into an electrical signal in units of pixels. This electrical signal is an analog video signal indicating a subject image captured by the image sensor 102. The image sensor 102 can output video signals in time series at a predetermined frame rate by continuously repeating imaging and output. In the following description, “image” is synonymous with a one-frame video signal.

被写体抽出部１１３では、相関度算出部１１２で求めた各基準画像の相関度に基づいて、１つの参照領域を被写体領域として判定し、抽出する。被写体抽出部１１３は、例えば基準画像登録部１１１による登録時刻が最も早い基準画像に対する最大相関度が所定の閾値以上であれば、当該基準画像と最大相関度を有する参照領域を入力画像中の被写体領域として選択、抽出する。一方、当該基準画像との最大相関度が所定の閾値よりも低ければ、被写体抽出部１１３は、残りの基準画像についての最大相関度のうち最も大きい相関度を有する参照領域を入力画像中の被写体領域として選択、抽出する。 The subject extraction unit 113 determines and extracts one reference region as a subject region based on the correlation degree of each standard image obtained by the correlation degree calculation unit 112. Subject extracting unit 113, for example, the reference image registering unit 111 a maximum correlation degree is a predetermined threshold or more der lever registration time for the earliest reference image by, in the input image a reference area having the reference image and the maximum degree of correlation Select and extract as a subject area. On the other hand, if the maximum correlation with the reference image is lower than the predetermined threshold, the subject extraction unit 113 selects the reference region having the largest correlation among the maximum correlations for the remaining reference images as the subject in the input image. Select and extract as a region.

制御部１０５は、このＡＦ制御やＡＥ制御による撮像条件の決定において、被写体追跡部１１０から供給される被写体領域の抽出結果を用いることができる。具体的には、制御部１０５は、被写体領域として抽出された領域に合焦するように焦点検出条件を制御してＡＦ制御を行ったり、当該領域の露出が適正になるように露出条件を制御してＡＥ制御することができる。また、制御部１０５は、撮像素子１０２の出力タイミングの制御や、読み出す画素の選択制御など、撮像素子１０２の読み出し制御も行う。 Control unit 105 in determining the imaging conditions by the AF control and AE control, it is possible to use an extraction result of the subject region supplied from the subject tracking unit 110. Specifically, the control unit 105 performs AF control by controlling the focus detection condition so that the region extracted as the subject region is in focus, or controls the exposure condition so that the exposure of the region is appropriate. Thus, AE control can be performed. The control unit 105 also performs readout control of the image sensor 102 such as control of output timing of the image sensor 102 and selection control of pixels to be read out.

（被写体追跡部１１０の動作）
次に、被写体追跡部１１０の処理の詳細について、図２及び図３Ａ〜図３Ｄを参照して説明する。なお、以下の説明では、検出及び追跡すべき特定の被写体が人物の顔であるものとする。また、基準画像登録部１１１に登録される基準画像は、次の２つとする。（１）被写体追跡の開始時に被写体指定部１０９によって指定される領域に基づき登録される基準画像（以下、初期基準画像）。（２）相関度算出部１１２が求めた相関度に応じて被写体抽出部１１３が抽出した被写体領域に基づき逐次更新される基準画像（以下、更新基準画像とする）。 (Operation of subject tracking unit 110)
Next, details of the processing of the subject tracking unit 110 will be described with reference to FIGS. 2 and 3A to 3D. In the following description, it is assumed that a specific subject to be detected and tracked is a human face. In addition, the following two reference images are registered in the reference image registration unit 111. (1) A reference image (hereinafter referred to as an initial reference image) registered based on an area designated by the subject designation unit 109 at the start of subject tracking. (2) the reference image in which the correlation calculation unit 112 is subject extracting section 113 in accordance with the degree of correlation calculated is successively updated based on subject region issued extracted (hereinafter referred to as updated reference image).

（被写体追跡部４０１の動作）
次に、被写体追跡部４０１の処理の詳細について、第１の実施形態との差異を中心に、図５、図６Ａ〜図６Ｃを参照して説明する。なお、以下の説明では、第１の実施形態と同様、図５は、第２の実施形態に係る被写体追跡の処理を示すフローチャートである。図６Ａ〜図６Ｃは、被写体追跡における説明図である。なお、以下の説明では、第１の実施形態の説明と同様、検出及び追跡すべき特定の被写体が人物の顔であるものとする。また、基準画像登録部１１１に登録される基準画像も、初期基準画像と更新基準画像との２つであるものとする。 (Operation of subject tracking unit 401)
Next, details of the processing of the subject tracking unit 401 will be described with reference to FIGS. 5 and 6A to 6C with a focus on differences from the first embodiment. In the following description, as in the first embodiment, FIG. 5 is a flowchart showing subject tracking processing according to the second embodiment. 6A to 6C are explanatory diagrams for subject tracking. In the following description, as in the description of the first embodiment, it is assumed that the specific subject to be detected and tracked is the face of a person. Also, it is assumed that the reference images registered in the reference image registration unit 111 are two, that is, an initial reference image and an updated reference image.

Claims

A subject tracking device that tracks a specific subject in an image over a plurality of input images input in time series,
For each of the plurality of reference images registered in the registration means, a calculation means for obtaining a maximum correlation with the input image and a region in the input image having the maximum correlation;
First determining means for determining the area of the specific subject in the input image from the areas obtained by the calculating means;
Wherein the plurality of reference images, and based Ku first reference image in the area specified as the area of the specific subject in the first input image among the plurality of input images, among the plurality of input images a plurality of second input image input later than the first input image, the in the region determined as the region of the specific subject by the first determination unit based at Ku second reference image Yes,
If the maximum correlation obtained by the calculation unit for the first reference image is equal to or greater than a predetermined threshold, the first determination unit does not depend on the region having the maximum correlation with the second reference image. The region having the maximum correlation with the first reference image is determined as the region of the specific subject in the input image, and the maximum correlation obtained by the calculation unit for the first reference image is determined in advance. If less than the threshold , based on the maximum correlation for the first reference image and the maximum correlation for the second reference image, the region having the maximum correlation with the first reference image and the first An object tracking device , wherein any one of the areas having the maximum correlation with the two reference images is determined as the area of the specific object in the input image.

Extracting means for extracting a feature amount of the first reference image;
Second determination means for determining a degree of coincidence of the input image with the feature amount;
Setting means for setting the predetermined threshold according to the determination of the second determination means;
The setting means sets the threshold value to be larger as it is determined that the degree of matching of the input image with the feature amount is higher, and the threshold value is set as it is determined that the degree of matching of the input image with the feature amount is lower. The subject tracking device according to claim 1, wherein the subject tracking device is set to be small.

The extraction means generates a histogram of predetermined component values of pixels included in the first reference image, and extracts a value having a frequency equal to or higher than a predetermined threshold as the feature amount. Item 3. The subject tracking device according to Item 2.

The second determination unit counts pixels that match the feature amount among pixels included in the input image, and calculates a ratio of the number of pixels that matches the feature amount to the number of pixels of the entire input image. The subject tracking device according to claim 3, wherein the subject tracking device determines the degree of coincidence with the feature amount.

A designating unit for designating a region in the input image;
5. The subject tracking apparatus according to claim 1, wherein the user designates the area of the specific subject in the first input image through the designation unit. 6.

A method for controlling a subject tracking device for tracking a specific subject in an image over a plurality of input images input in time series,
A calculating step for calculating a maximum correlation with the input image and a region in the input image having the maximum correlation for each of the plurality of reference images registered in the registration unit;
A determination unit including a first determination step of determining a region of the specific subject in the input image from the regions obtained by the calculation step;
Wherein the plurality of reference images, and based Ku first reference image in the area specified as the area of the specific subject in the first input image among the plurality of input images, among the plurality of input images a plurality of second input image input later than the first input image, the first region is determined as the region of the specific subject by determining steps based at Ku second reference image Yes,
In the first determination step , the determination means determines the maximum correlation with the second reference image if the maximum correlation obtained in the calculation step for the first reference image is equal to or greater than a predetermined threshold. The region having the maximum correlation with the first reference image is determined as the region of the specific subject in the input image regardless of the region having the maximum correlation obtained in the calculation step with respect to the first reference image. is less than degree is the predetermined threshold value, based on the maximum correlation for the maximum correlation and the second reference image for the first reference image, the first reference image and the maximum correlation A method for controlling a subject tracking apparatus , wherein one of a region having a maximum correlation with the second reference image is determined as a region of the specific subject in the input image.

The program for making a computer perform each process of the control method of the to-be-photographed object tracking apparatus of Claim 6.