JP2004362469A - Apparatus and method for detecting moving object using active sensor, and program for detecting animal body - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily distinguish apparent changes of an environment, due to the movement of an observation system other than moving objects from environmental changes due to movement of animal bodies to easily detect moving objects and precisely detect moving objects, by estimating occlusion regions. <P>SOLUTION: A computer estimates the amount of relative movements, which includes the direction and the distance of movement and the amount of rotation, before and after an active sensor moves in step S10. In step S20, the computer estimates a predicted distance image after the movement, to generate it based on all direction distance images acquired before movement and the estimated amount of relative movements. In step S30, the active sensor moves and then the computer estimates occlusion areas that are generated in the predicted distance image. When differential images between the predicted distance image and all direction distance images acquired by the active sensor after movement are generated, the computer removes the occlusion regions and extracts the animal body regions. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

ここで、ｄθは方位角θに位置するエッジの３次元での奥行き（アクティブセンサ１１中心までの距離）である。実際には、多くの場合同じ方位にあるエッジは異なる奥行きをもっているため、式（１）は理想的な場合（同じ方位のエッジは同一の奥行きを持つ場合）だけを表している。
【００５２】
式（１）は、エッジの奥行きに影響されるが、図７（ｃ）に示すように、ｓｉｎで近似することができ、ｓｉｎ曲線と同様に２πの周期を持つ。その総合シフト量δθの符号はπ周期毎に反転する。
【００５３】
なお、図７（ｃ）は全方位エッジヒストグラムの総合シフト量δθを式（１）で計算した場合の曲線とｓｉｎ曲線を表した説明図である。
同図において、Ａはｓｉｎ曲線（正弦波曲線）であり、Ｂは、円筒形の部屋でアクティブセンサ１１が動いた場合の、全方位エッジヒストグラムの総合シフト量δθを式（１）に基づいて演算した曲線である（図７（ａ）参照）。
【００５４】
又、Ｃは正方形の部屋でアクティブセンサ１１が動いた場合の、全方位エッジヒストグラムの総合シフト量δθを式（１）に基づいて演算した曲線である（図７（ｂ）参照）。
【００５５】
本実施形態では、式（１）の符号がπ周期毎に反転する特徴を利用して、全方位エッジヒストグラムの総合シフト量δθからアクティブセンサ１１の移動方向ωと回転角度φを簡単かつロバストに推定するのである。
【００５６】
１．４． Ｓ１４０（動的計画法によるマッチング：ＤＰマッチング）
Ｓ１４０では、アクティブセンサ１１の現在位置（移動後地点）の全方位エッジヒストグラムと移動前地点の全方位エッジヒストグラムをＤＰマッチングし、それぞれの方位角（各移動前地点の方位角）におけるヒストグラムの総合シフト量δθを求める。
【００５７】
移動後地点の全方位エッジヒストグラムは現在位置ヒストグラムに相当し、移動前地点の全方位エッジヒストグラムは移動前地点ヒストグラムに相当する。
この総合シフト量δθを求めることにより、移動前地点に対するアクティブセンサ１１の移動後地点の移動方向ωと回転角度φを算出する。
【００５８】
以下、移動前地点に対する、アクティブセンサ１１の移動後地点の全方位エッジヒストグラムの総合シフト量δθの算出の仕方を詳細に説明する。
（動的計画法マッチング（ＤＰマッチング）について）
ここでは、移動後地点の全方位エッジヒストグラムと、ある移動前地点の全方位エッジヒストグラムのＤＰマッチングについて、すなわち、２つの全方位エッジヒストグラムをＤＰマッチングする方法を説明する。
【００５９】
移動前地点と移動後地点の全方位エッジヒストグラムのそれぞれを
移動前地点：Ｈｐ＝［ｈｐ（ｉ），ｉ＝０，…，…，Ｎ−１］
と
移動後地点：Ｈｃ＝［ｈｃ（ｊ），ｊ＝０，…，…，Ｎ−１｝
とする。
【００６０】
Ｎは３６０の倍数であり、エッジヒストグラムを生成するときの方位角θで決まる。本実施形態では、Ｎ＝７２０である。すなわち、エッジヒストグラムの角度分解能は０．５°である。移動前地点の全方位エッジヒストグラムＨｐの１つのピンｈｐ（ｉ）は、移動と回転によってアクティブセンサ１１の移動後地点の全方位エッジヒストグラムＨｃにおいてシフト量ｓｉが生じたとき、ｈｐ（ｉ）とｈｃ（ｉ＋ｓｉ）と似ていると仮定することができる。
【００６１】
ｈｐ（ｉ）とｈｃ（ｉ＋ｓｉ）との差の二乗をｈｐ（ｉ）とｈｃ（ｉ＋ｓｉ）との間のマッチングコストとすると、次のマッチングコストマトリクスＣ（ｓ，ｉ）が得られる。
【００６２】
【数２】

ここで全方位エッジヒストグラムが２πの周期を持つため、ｉ＋ｓｉ≧Ｎの場合、
ｈｃ（ｉ＋ｓｉ）≡ｈｃ（ｉ＋ｓｉ−Ｎ）とする。
【００６３】
［（ｈｐ（ｉ），ｈｃ（ｉ＋ｓｉ）），ｉ＝０，…，Ｎ−１］が正しいマッチングである場合、それらのマッチングペアはマッチングコストマトリクスＣ（ｓ，ｉ）の中でコストが低く、かつ式（１）の形をした曲線をなす。ここでは、（ｈｐ（ｉ），ｈｃ（ｉ＋ｓｉ））はｈｐ（ｉ）とｈｃ（ｉ＋ｓｉ）とのマッチングペアを表している。
【００６４】
図８は、アクティブセンサ１１の２つの地点における全方位エッジヒストグラムのマッチングコストマトリクスＣ（ｓ，ｉ）を示す。なお、説明の便宜上、マトリクス中のマッチングコストの低いパスをセンタリングし、上下をカットしている。又、図中、「．−＋＊＆％＃＄＠ＡＢＣＤ」の各記号は、コストのレベルをそれぞれ示し、「．−＋＊＆％＃＄＠ＡＢＣＤ」の順番は、左から右に向かってコストの低い順から高い順に並べている。すなわち、マッチングコストは「．」＜「−」＜「＋」＜「＊」＜「＆」＜「％」＜「＃」＜「＄」＜「＠」＜「Ａ」＜「Ｂ」＜「Ｃ」＜「Ｄ」の大小関係となっている。
【００６５】
図中、縦軸はエッジのシフト量ｓｉ、横軸はｉであり、方位角θに相当する。又、図中、Ｃ（ｓ，ｉ）の中のｓｉｎ曲線に似たコストの低い曲線は各方位角θにおけるエッジヒストグラムの総合シフト量δθに対応する。Ｃ（ｓ，ｉ）の中のｓｉｎ曲線に似たコストの低い曲線を探索することで、ＨｐとＨｃの間の全方位ヒストグラムの総合シフト量δθが求められる。
【００６６】
本実施形態では、計算コストの低い動的計画法（ＤＰ）を用いて、Ｃ（ｓ，ｉ）から周期２πを持ち、かつ連続した最小コストのパスを求め、そのパスからロバストに回転角度φと移動方向ωを推定する。
【００６７】
Ｃ（ｓ，ｉ）から周期２πを持ちかつ連続した最小コストのパスは次の条件付き最小化問題に定義することができる。
【００６８】
【数３】

ここで、ｓｉは求めたいエッジヒストグラムの総合シフト量である。
【００６９】
ｈｐ（ｉ）とｈｃ（ｊ）の周期がＮであるため、Ｃ（ｓ，ｉ）のインデックスのｓとｉに関しては、ｓ±Ｎ→ｓとｉ±Ｎ→ｉで計算される。
パスが特定の行ｋから始まる（すなわち、ｓ０＝ｋ）と仮定した場合には、式（２）の最小化は次のように動的計画法で求めることができる。
【００７０】
【数４】

【００７１】
【数５】

【００７２】
【数６】

Ｓ（ｓ，ｉ）を計算するときは、ｍｉｎ［Ｓ（ｓ−１，ｉ−１），Ｓ（ｓ，ｉ−１），Ｓ（ｓ＋１，ｉ−１）］の中のいずれが最小になっていたかを記憶しておき、Ｃｍｉｎ（ｋ）まで来たパスを逆に辿れば、最小コストのパス（以下、最小コストパスという）が得られる。
【００７３】
ｋ＝０，…，Ｎ−１に対して、上記のようにＣｍｉｎ（ｋ）を計算し、そのうち、
最小値Ｃ＾ｍｉｎ（ｋ）＝Ｃｍｉｎ（ｋ＾）
を求める。
【００７４】
Ｃｍｉｎ（ｋ＾）が対応しているパスがＨｐとＨｃの間の最適マッチングとする。
以下、上記パスを最適マッチングパスという。
上記計算により、２πの周期を持ち、かつ連続性のあるエッジヒストグラムのマッチングパスを探索できる。
【００７５】
そして、以下、移動後地点の全方位エッジヒストグラムと他の移動前地点の全方位エッジヒストグラムのマッチングについても同様に処理する。
１．５． Ｓ１５０（姿勢と移動方向の推定）
Ｓ１５０では、Ｓ１４０で得られたアクティブセンサ１１の移動後地点と移動前地点の全方位エッジヒストグラムの総合シフト量から移動前地点に対するアクティブセンサ１１の移動方向ωと回転角度φを推定する。
【００７６】
（回転角度φの推定）
まず、アクティブセンサ１１の回転角度φの推定について説明する。
ＤＰマッチングで得られた最小コストパスであるｓｉ，（ｉ＝０，…，Ｎ−１）は、アクティブセンサ１１の移動後地点と移動前地点のそれぞれの全方位エッジヒストグラムＨｃとＨｐの間の総合シフト量を表している。図９中のｓｉｎ曲線の近傍に示された波形はＤＰマッチングで得られた最小コストパスを示す。
【００７７】
式（１）で示したように、これらの総合シフト量はアクティブセンサ１１の回転角度φで生じたヒストグラム全体の回転量と、移動方向ωにおける平行移動で生じた各方位角での移動量からなる。
【００７８】
式（１）から分かるように、ヒストグラムの総合シフト量は回転角度φに相当する回転量ｓφを引けば、引いた後のシフト量はπ周期毎に反転する。すなわち、回転角度φを中心にπ周期で、上下反転する（図９参照）。
【００７９】
従って、本実施形態のＳ１５０では、回転量ｓφは次の式で演算することにより推定する。すなわち、回転角度φを推定する。
【００８０】
【数７】

【００８１】
【数８】

すなわち、回転量ｓφがエッジヒストグラムのシフト量ｓｉ，（ｉ＝０，…，Ｎ−１）を上下２等分に分けることになる（図９参照）。
【００８２】
（アクティブセンサ１１の移動方向ωの推定）
次に、式（１）がｓｉｎ曲線の周期性を保たれていることを利用して、アクティブセンサ１１の移動方向ωをロバストに推定する。
【００８３】
この推定の根拠は下記の通りである。
上述したように回転量ｓφを引いた後のエッジヒストグラムのシフト量ｓｉ’をｓｉｎ曲線に近似する。しかし、図７に示しているように、これらのシフト量は、エッジの奥行きや空間の形に影響される。
【００８４】
エッジヒストグラムのシフト量ｓｉ’は、エッジの奥行きや空間の形の影響でｓｉｎ曲線からずれるが、図７に示すように（０，π）の区間では、正の値を、（π，２π）の区間では、負の値をもっていると仮定することができる。
【００８５】
従って、シフト量ｓｉ’の符号を用いて、ｓｉｎ曲線をロバストに当てはめることができる。
ここで、回転角度φにより、シフト量ｓｉ’の−１，０，１の３つの値に変換し、−１と１の値に対応するシフト量ｓｉ’の中心がそれぞれなるべくｓｉｎ曲線の負と正のピークに対応するように移動方向ωを決定する。
【００８６】
これは次の最大化問題になる。
【００８７】
【数９】

【００８８】
【数１０】

ここで、回転角度φに相当するｓφとｓφ±１を０にすることにより、エッジヒストグラムのシフト量ｓｉ’は符号へのノイズの影響を軽減することができる。
【００８９】
式（９）の左辺の微分を０とすると、次の方程式が得られる。
【００９０】
【数１１】

上の式（１１）から次のように、移動方向ωを直接求める。
【００９１】
【数１２】

なお、式（１１）は、式（９）の最大化のみならず、最小化も含んでいる。ｔａｎ（θ）の周期がπであるから、移動方向ω又はω＋πのどちらかが式（９）を最大化することが分かる。
【００９２】
上記のようにして、本実施形態のＳ１５０では、式（１２）に基づいて、移動方向ωを算出する。Ｓ１５０では、上記のようにして、移動前地点に対するアクティブセンサ１１の回転角度φや移動方向ωを推定することができる。
【００９３】
１．６． Ｓ１６０（アクティブセンサ１１の移動距離の推定）
Ｓ１６０では、アクティブセンサ１１の移動距離の推定を行う。具体的には、図３（ｃ）のフローチャートに従って移動距離の推定を行う。
【００９４】
Ｓ１６２において、移動前にアクティブセンサ１１で得られた環境Ｋの全方位の距離データに基づくエッジの３次元点群を床平面（ｘ−ｙ平面）に写像し、図１０のようにエッジヒストグラム（床平面投影ヒストグラム）を生成する。この写像により、環境Ｋの壁などの垂直平面（床平面に対する垂直平面）にある多くの３次元点は同じ場所に投影され、垂直平面が存在する場所では、ヒストグラムの値は高くなる。なお、図１０において、ヒストグラムの値が大きな部分については、白く表している。
【００９５】
Ｓ１６４では、前記床平面投影ヒストグラムに対してハフ（Ｈｏｕｇｈ）変換を用いて、主な垂直平面（壁など）の方向を検出する。
（ハフ変換）
ハフ変換について説明する。図１４はハフ変換の説明図である。同図において、Ｘ−Ｙ平面上に、点Ｄ１から点Ｄ６が直線Ｒ上に分布しているものとする。原点Ｏから直線Ｒに下ろした垂線の足をＨ，ＯＨ＝ρ，ＯＨとＸ軸のなす角度をθ，Ｈの座標を（ｘ，ｙ）とすると、直線Ｒは下記の極座標の式で表現できる。
【００９６】
ρ＝ｘｃｏｓθ＋ｙｓｉｎθ
ここで、ρ，θを変数とした場合、ある点（ｘ，ｙ）を通るすべての直線群は、前記極座標の式で表現できる。
【００９７】
図１５はρ−θ平面上で、ある点を通る直線群を示した図である。ρ−θ平面上では、ある点を通る直線群は、唯一１本の曲線として表現でき、ρ，θが一意に決まれば、１本の直線が特定できることになる。
【００９８】
図１６はρ−θ平面上で複数の候補点の各々に対する直線群を示した図である。図１４に示した各候補点について、前記極座標の式に従って、図１５に示すように曲線を求めた一例を図１６に示している。そして、図１６に示すように、複数の曲線が描かれているが、すべての候補点に対し、最も適切な直線は、曲線同士が最も多く交差している点Ｑ（ρ_０ ，θ_０ ）から特定される直線ρ＝ｘｃｏｓθ_０ ＋ｙｓｉｎθ_０ である。
【００９９】
本実施形態においては、例えば図１０の床平面投影ヒストグラムに対してハフ変換を用いると、図１１に示すようにρ−θ平面上に、多数の曲線群が描かれる。なお、図１１では、曲線は白線で表されている。図１１において、曲線同士の公差が多い交点を、主な垂直平面（壁など）に関する直線として検出され、すなわち、垂直平面の方向が検出される。
【０１００】
本実施形態では、図１１のハフ変換のρ−θ平面が、ハフ投票空間とされている。このハフ投票空間から、θが所定角度（例えば６０度）以上離れた交点を検出することによって、図１０に示すように、（Ａ１，Ａ２）と（Ｂ１，Ｂ２）のような主な垂直平面の方向を検出する。
【０１０１】
そして、これらの主な垂直平面の方向のうち、移動方向ωに最も近い１つの垂直平面の方向をドミナント射影方向ｖとして選択し、図１０の床平面投影ヒストグラムを選択した垂直平面の方向（ドミナント射影方向ｖ）に沿って投影し、１次元ヒストグラムを生成する。
【０１０２】
なお、図１２は、Ａ１及びＡ２が互いに平行であって、その平行な方向である垂直平面の方向に沿って投影した方向をドミナント射影方向として、投影したときのヒストグラムを示し、Ａ１，Ａ２で示す部分は、ヒストグラムの値が大きいことを示している。図１３は、Ｂ１及びＢ２が互いに平行であって、その平行な方向である垂直平面の方向に沿って投影した方向をドミナント射影方向として、投影したときのヒストグラムを示し、Ｂ１，Ｂ２で示す部分は、ヒストグラムの値が大きいことを示している。
【０１０３】
Ｓ１６６では、移動後にアクティブセンサ１１にて得られた環境Ｋにおける全方位の距離データに基づくエッジの３次元点群をＺ軸（アクティブセンサ１１が走行した床平面に垂直な軸）を中心に、Ｓ１５０で推定した回転角度φに基づき、「−φ」で回転する。すなわち、
Ｐ’＝Ｒｚ（−φ）Ｐ
ここで、Ｒｚ（−φ）は回転マトリクスである。ＰとＰ’は回転前と回転後の３次元点を表している。
【０１０４】
Ｓ１６８では、Ｓ１６４で得られた主な垂直平面のヒストグラム投影方向をｖ１，ｖ２としたとき（図１０参照）、このヒストグラム投影方向ｖ１，ｖ２のうち、移動方向ωと最も近い方向（例えば、ｖ１）を選び、Ｐ’をその投影方向に投影し、ヒストグラムＨ’（ｖ１）を生成する。
【０１０５】
又、アクティブセンサ１１が走行移動前に、アクティブセンサ１１が得た環境Ｋの全方位の距離データに基づくエッジの３次元点をＰｏとする。そして、Ｓ１６４で得られた主な垂直平面のヒストグラム投影方向をｖ１，ｖ２としたとき（図１０参照）、このヒストグラム投影方向ｖ１，ｖ２のうち、移動方向ωと最も近い方向（例えば、ｖ１）を選び、Ｐｏをその投影方向に投影し、ヒストグラムＨ（ｖ１）を生成する。
【０１０６】
Ｓ１７０では、移動前と移動後で得られたｖ１方向（すなわち、移動方向ωと最も近い方向）のヒストグラムＨ（ｖ１）とＨ’（ｖ１）をマッチングし、２つのヒストグラムのシフト量λを求める。このシフト量λはアクティブセンサ１１が走行してヒストグラム投影方向ｖ１に沿った移動距離と関係する。
【０１０７】
Ｓ１８０では、前記シフト量λに基づいてアクティブセンサ１１が移動方向ωに沿って移動した移動距離Ｌを下記式（１３）を使用して演算する。
【０１０８】
【数１３】

２． Ｓ２０（予測距離画像の算出）
ここではアクティブセンサ１１が移動前に取得した全方位の距離データと、Ｓ１６０で取得した相対移動量（回転角度φ，移動方向ωと移動距離Ｌを含む）とに基づいて、移動後にアクティブセンサ１１の視点で得られる全方位距離画像を推定する。以下では、移動後の予測された全方位距離画像を単に予測距離画像という。
【０１０９】
本実施形態では、アクティブセンサ１１の各ステレオユニット１２により得られた距離画像を各ステレオユニット１２の配置パラメータを用いて、円筒座標系で表現し、統合することにより、全方位距離画像を生成する。
【０１１０】
図２６は、生成した全方位距離画像の例を示しており、横軸は方位角θ、縦軸は仰角γである。図１７は、仰角γの説明図であり、アクティブセンサ１１の視点中心から上を＋とし、下を−としている。なお、図２６は０＜θ＜２π、−π／３＜γ＜π／３の範囲を対象としている。
【０１１１】
ここで、移動前の全方位距離画像の方位角θ、仰角γ方向における距離値ｄ_ｓ（θ，γ）は、移動後の相対移動量（回転角度φ，移動方向ωと移動距離Ｌを含む）により、方位角θ’、仰角γ’方向の位置に、距離値ｄ_ｐ（θ’，γ’）に移動する。ここで、距離値ｄ_ｓ（θ，γ）は、図１８に示すように、アクティブセンサ１１がポイントＳに位置したときの、ポイントＳから距離画像における任意の点Ｗまでの値である。距離値ｄ_ｐ（θ’，γ’）は、アクティブセンサ１１がポイントＳからポイントＰに移動した後のポイントＰから距離画像における点Ｗまでの距離値である。
【０１１２】
方位角θ’、仰角γ’及び距離値ｄ_ｐ（θ’，γ’）は、アクティブセンサ１１の回転角度φ、移動方向ωと、その移動距離Ｌを用いて、下記の式（１４）で求める。
【０１１３】
【数１４】

上記式（１４）により、アクティブセンサ１１が移動後の視点で得られる全方位距離画像、すなわち、予測距離画像が推定される。
【０１１４】
３． Ｓ３０（オクルージョン領域の推定）
Ｓ３０では、アクティブセンサ１１、すなわち、観測系が移動することによって、生じるオクルージョン領域を推定する。オクルージョンは、環境Ｋにおける動物体と、アクティブセンサ１１のカメラの移動量とに深く関係する。すなわち、奥行き（距離値）が急激に変化する部分にオクルージョンが生じる。
【０１１５】
このため、アクティブセンサ１１から得られる距離画像におけるジャンプエッジに注目し、距離画像と移動量に基づいて予測距離画像に生じるオクルージョン領域を推定する。
【０１１６】
ここで、移動前において取得した全方位距離画像におけるジャンプエッジの画素（ピクセル）を、（θ_１，γ_１，ｄ_ｓ（θ_１，γ_１））とする。そして、画素（θ_１，γ_１，ｄ_ｓ（θ_１，γ_１））の所定範囲内としての４−近傍にある距離値ｄ_ｓ（θ_１，γ_１）より最も遠い距離値を有する画素を（θ_２，γ_２，ｄ_ｓ（θ_２，γ_２））とする。
【０１１７】
前記ジャンプエッジの画素は本発明のジャンプエッジ画素に相当し、所定範囲内としての４−近傍にあるｄ_ｓ（θ_１，γ_１）より最も遠い画素は、対比対象画素に相当する。なお、本実施形態の「最も遠い距離値」は、本発明における「所定値」に相当する。 図１９は、オクルージョン領域の説明図である。同図において、ジャンプエッジの画素に対応する部分を（θ_１，γ_１，ｄ_ｓ（θ_１，γ_１））で示しており、その画素から最も遠い画素に相当する部位を（θ_２，γ_２，ｄ_ｓ（θ_２，γ_２））で示している。これらは、前記式（１４）より、アクティブセンサ１１の移動によって、それぞれ下記のように変換する。
【０１１８】
（θ_１，γ_１，ｄ_ｓ（θ_１，γ_１）） → （θ_１’，γ_１’，ｄ_ｐ（θ_１’，γ_１’））
（θ_２，γ_２，ｄ_ｓ（θ_２，γ_２）） → （θ_２’，γ_２’，ｄ_ｐ（θ_２’，γ_２’））
θ_１’はジャンプエッジ画素の推定方位角，及びθ_２’は対比対象画素の推定方位角に相当し、γ_１’ジャンプエッジ画素の推定仰角，及びγ_２’は対比対象画素の推定仰角に相当する。
【０１１９】
この変換により生じた各画素（ピクセル）の位置の差から方位角θと仰角γの方向のオクルージョン区間Ｉθ，Ｉγを推定する。又、オクルージョン区間Ｉθ，Ｉγから、次のようなオクルージョン領域Ｏｃ（θ，γ）を推定する。
【０１２０】
【数１５】

上記式（１５）中、Ｉθ＝（θ_２’，θ_１’）は、θ_１＞θ_２，θ_１’＞θ_２’のとき、移動後においては、方位角θ_２’〜θ_１’の間がオクルージョン区間を意味している。なお、θ_１＞θ_２，θ_１’＞θ_２’の場合は、動物体が、例えば、図１９においては、アクティブセンサ１１が右側に移動する場合に相当する。
【０１２１】
又、Ｉθ＝（θ_１’，θ_２’）は、θ_２＞θ_１，θ_２’＞θ_１’のとき、移動後においては、方位角θ_１’〜θ_２’間がオクルージョン区間を意味している。なお、θ_２＞θ_１，θ_２’＞θ_１’の場合は、例えば、図１９においては、アクティブセンサ１１が左側に移動する場合に相当する。
【０１２２】
そして、上記以外の場合は、方位角においてオクルージョン区間がないこと、すなわち空集合であることを意味する。
上記式中、Ｉγ＝（γ_２’，γ_１’）は、γ_１＞γ_２，γ_１’＞γ_２’のとき、予測距離画像においては、仰角γ_２’〜γ_１’間がオクルージョン区間を意味する。
【０１２３】
又、Ｉγ＝（γ_１’，γ_２’）は、γ_２＞γ_１，γ_２’＞γ_１’のとき、予測距離画像においては、仰角γ_１’〜γ_２’の間がオクルージョン区間を意味している。そして、上記以外の場合は、仰角においてオクルージョン区間がないこと、すなわち空集合であることを意味する。
【０１２４】
さらに、上記式（１５）では、予測距離画像における判定対象画素の仰角γが、推定仰角γ_１’に一致し、予測距離画像における判定対象画素の方位角θがオクルージョン区間Ｉθ内にあるときは、Ｏ（θ，γ）＝１とする。すなわち、判定対象画素はオクルージョン領域内にあると判定する。
【０１２５】
又、予測距離画像における判定対象画素の方位角θが推定方位角θ_１’に一致し、判定対象画素の仰角γがオクルージョン区間Ｉγ内にあるときは、Ｏ（θ，γ）＝１とする。すなわち、判定対象画素はオクルージョン領域内にあると判定する。
【０１２６】
そうでない場合には、Ｏ（θ，γ）＝０とする。
Ｏ（θ，γ）＝１の場合、予測距離画像における、方位角θ，仰角γの画素（ピクセル）はオクルージョン領域の一部であることを意味する。又、Ｏ（θ，γ）＝０の場合、方位角θ，仰角γの画素（ピクセル）はオクルージョン領域ではないことを意味している。
【０１２７】
４． Ｓ４０（差分による動物体領域の抽出）
Ｓ４０では、Ｓ２０で得られた予測距離画像と、アクティブセンサ１１が移動後に実際に得られる全方位距離画像との差分をとり、差分画像を生成し、Ｓ３０にて推定したオクルージョン領域を取り除くことにより、動物体領域を抽出する。具体的には、下記の通り行う。
【０１２８】
ここで、移動後取得する全方位距離画像、前記予測距離画像、及び生成する差分画像におけるθ，γに対する距離値をそれぞれｄ_ｇ（θ，γ），ｄ_ｐ（θ，γ），ｄδ（θ，γ）とする。又、差分画像の３値画像を得るための判定値をｄ_ｄ（θ，γ）とする。なお、Ｓ２０では、予測距離画像の距離値は、ｄ_ｐ（θ’，γ’）で示したが、この欄では、予測距離画像の方位角θ’と、仰角γ’は、移動後に取得した全方位距離画像の方位角θと、仰角γとそれぞれ等しいものとして、説明の便宜上、この欄では、ｄ_ｐ（θ，γ）で示す。
【０１２９】
生成する差分画像の距離値ｄδ（θ，γ）は、差分画像の距離値＝（予測距離画像の距離値）−（移動後の全方位距離画像の距離値）、で得られる。すなわち、生成する差分画像の距離値ｄδ（θ，γ）は、
【０１３０】
【数１６】

で表される。又、この差分画像の距離値ｄδ（θ，γ）に対して閾値Ｔｈ、及び閾値−Ｔｈを用いて、差分画像の３値画像を得るための判定値を得る。なお、閾値Ｔｈ、及び閾値−Ｔｈは、差分画像の距離値ｄδ（θ，γ）に大きな変化があったかどうかを判定するための閾値である。
【０１３１】
【数１７】

式（１７）において、差分画像の距離値ｄδ（θ，γ）が閾値Ｔｈを超える場合であって、Ｏ（θ，γ）＝０、すなわち、画素（ピクセル）がオクルージョン領域にない場合は、その画素について判定値ｄ_ｄ（θ，γ）を「１」とし、正の値とする。この画素（ピクセル）は、前記閾値Ｔｈを超えてアクティブセンサ１１に近い位置へ変化したことを示している。
【０１３２】
又、式（１７）において、差分画像の距離値ｄδ（θ，γ）が閾値−Ｔｈ未満の場合であって、Ｏ（θ，γ）＝０、すなわち、画素がオクルージョン領域にない場合は、判定値ｄ_ｄ（θ，γ）を「−１」とし、負の値とする。この場合、画素（ピクセル）は、前記閾値−Ｔｈを超えてアクティブセンサ１１から離れた位置へ変化したことを示している。
【０１３３】
さらに、上記の条件を満足しない場合は、判定値ｄ_ｄ（θ，γ）を「０」とする。すなわち、この場合、差分画像の距離値ｄδ（θ，γ）は、閾値−Ｔｈ〜閾値Ｔｈの変化であるため、画素（ピクセル）は、大きな変化がないことを示している。
【０１３４】
上記のようにした得られた判定値ｄ_ｄ（θ，γ）に基づいて、前記差分画像を３値化する。
（具体例での説明）
１．具体例１
図２０で示す具体例１を参照して、式（１７）の判定値の設定について説明する。具体例１は、アクティブセンサ１１と、動物体２０とが、同一線上ではない、異なる方向にそれぞれ移動するときの例である。
【０１３５】
図２０は、アクティブセンサ１１の移動前後の位置と、動物体２０の移動前後の位置をそれぞれ示している。なお、説明の便宜上、以下は、アクティブセンサ１１において、仰角γ＝０における画素を前提として説明する。又、閾値Ｔｈを０とする。
【０１３６】
同図中、動物体２０及び、アクティブセンサ１１が移動後に実際に取得した全方位距離画像は、ａ６とａ７間の動物体２０（移動後）の円弧部分，ａ８〜ａ９〜ａ３〜ａ４〜ａ５の環境Ｋの壁を含んでいる。なお、各ａは、方位角θと関連し、以下では、説明の便宜上、例えば、ａ８を指す場合、方位角ａ８、又は、ａ８の方位角という。
【０１３７】
又、Ｓ２０で算出した予測距離画像は、ａ１〜ａ２の動物体２０（移動前）の円弧部分、ａ３〜ａ４〜ａ５〜ａ８〜ａ９の環境Ｋの壁を含んでいる。又、Ｓ３０で推定されたオクルージョン領域は、ａ３〜ａ４の環境Ｋの壁部分である。
【０１３８】
（ａ１〜ａ２の方位角）
さて、ａ１〜ａ２の方位角の範囲では、予測距離画像の距離値ｄ_ｐ（θ，γ）は動物体２０（移動前）の円弧部分のものであり、一方、移動後の全方位距離画像の距離値ｄ_ｇ（θ，γ）は環境Ｋの壁部分のものであって、移動後の全方位距離画像の距離値ｄ_ｇ（θ，γ）の方が、大きくなる。このため、式（１６）を算出すると、生成する差分画像の距離値ｄδ（θ，γ）＜０（＝−Ｔｈ）となる。又、この方位角ａ１〜ａ２は、Ｏ（θ，γ）＝０、すなわち、オクルージョン領域でない。この結果、方位角ａ１〜ａ２の範囲では、式（１７）の算出結果は、「−１」、すなわち負となる。なお、図２０では、式（１７）の算出結果を括弧書で示している。
【０１３９】
（ａ３〜ａ４の方位角）
ａ３〜ａ４の方位角の範囲は、オクルージョン領域であるため、Ｏ（θ，γ）＝１であり、この結果、方位角ａ３〜ａ４の範囲では、式（１７）の算出結果は、「０」となる。
【０１４０】
（ａ４〜ａ５の方位角）
ａ４〜ａ５の方位角の範囲では、予測距離画像の距離値ｄ_ｐ（θ，γ）は環境Ｋの壁部分のものである。又、一方、移動後の全方位距離画像の距離値ｄ_ｇ（θ，γ）も同じ環境Ｋの壁部分のものである。このため、式（１６）を算出すると、生成する差分画像の距離値ｄδ（θ，γ）＝０となる。この結果、方位角ａ４〜ａ５の範囲では、式（１７）の算出結果は、「０」となる。
【０１４１】
（ａ６〜ａ７の方位角）
ａ６〜ａ７の方位角の範囲では、予測距離画像の距離値ｄ_ｐ（θ，γ）は環境Ｋの壁部分のものであり、一方、移動後の全方位距離画像の距離値ｄ_ｇ（θ，γ）は動物体２０（移動後）の円弧部分のものである。このため、式（１６）を算出すると、生成する差分画像の距離値ｄδ（θ，γ）＞０（＝Ｔｈ）となる。又、この方位角ａ６〜ａ７は、Ｏ（θ，γ）＝０、すなわち、オクルージョン領域でない。この結果、方位角ａ６〜ａ７の範囲は、式（１７）の算出結果は、「１」、すなわち正となる。
【０１４２】
（ａ８〜ａ９の方位角）
ａ８〜ａ９の方位角の範囲では、予測距離画像の距離値ｄ_ｐ（θ，γ）は環境Ｋの壁部分のものである。又、一方、移動後の全方位距離画像の距離値ｄ_ｇ（θ，γ）も同じ環境Ｋの壁部分のものである。このため、式（１６）を算出すると、生成する差分画像の距離値ｄδ（θ，γ）＝０となる。この結果、方位角ａ８〜ａ９の範囲では、式（１７）の算出結果は、「０」となる。
【０１４３】
上記のように、差分画像において、アクティブセンサ１１に近い位置へ変化した領域は、「正」の領域となり、一方、遠い位置へ変化した領域は「負」の領域で現れる。どちらの領域においても、移動後に得られる距離画像から、動物体２０を抽出したことになる。すなわち、「正」の領域は、動物体２０の移動後を抽出したことになり、「負」の領域は、動物体２０の移動前を抽出したことになる。
【０１４４】
なお、オクルージョン領域を考慮しない場合、動物体２０が移動後に現れるオクルージョン領域（先の例では、ａ３〜ａ４の方位角の領域）も動物体と検出してしまうことになる。しかし、Ｓ３０において、オクルージョン領域を推定し、推定したオクルージョン領域を式（１７）にて除外して区別しているため、動物体領域のみを検出できる。
【０１４５】
２． 具体例２
次に図２１及び図２２で示す具体例２を参照して、式（１７）の判定値の設定について説明する。具体例２は、アクティブセンサ１１と、動物体２０とが、同一線上において、互いに接近移動するときの例である。
【０１４６】
なお、この例では、Ｓ３０ではオクルージョン領域は、下記のように推定されている。図２２では、ジャンプエッジの画素に対応する部分を（θ_１，γ_１，ｄ_ｓ（θ_１，γ_１））で示しており、その画素から最も遠い画素に相当する部位を（θ_２，γ_２，ｄ_ｓ（θ_２，γ_２））で示す。
【０１４７】
この場合、式（１４）により、アクティブセンサ１１の移動により、
（θ_１，γ_１，ｄ_ｓ（θ_１，γ_１）） → （θ_１’，γ_１’，ｄ_ｐ（θ_１’，γ_１’））
（θ_２，γ_２，ｄ_ｓ（θ_２，γ_２）） → （θ_２’，γ_２’，ｄ_ｐ（θ_２’，γ_２’））
のように変換されている。なお、説明の便宜上、仰角γは０としている。
【０１４８】
そして、この場合、図２２に示すように、θ_１＞θ_２であり、かつ、θ２’＞θ１’であるため、式（１５）により、オクルージョン区間Ｉθは空集合となり、オクルージョン領域Ｏｃ（θ，γ）＝０、すなわち、オクルージョン領域はないとされている。なお、図２２では、ジャンプエッジは、動物体２０の右側の部分としたが、左側にもジャンプエッジが存在する。しかし、前述した同じ理由により、こちらの側もオクルージョン領域はないとされる。
【０１４９】
次に、Ｓ４０における処理を説明する。
図２１は、アクティブセンサ１１の移動前後の位置と、動物体２０の移動前後の位置をそれぞれ示している。なお、説明の便宜上、以下は、アクティブセンサ１１において、仰角γ＝０における画素を前提として説明する。又、閾値Ｔｈを０とする。同図中、動物体２０、及びアクティブセンサ１１が移動後に実際に取得した全方位距離画像は、ａ１〜ａ２〜ａ３〜ａ４間の動物体２０（移動後）の円弧部分，ａ４〜ａ１間の環境Ｋの壁を含んでいる。
【０１５０】
又、Ｓ２０で算出した予測距離画像は、ａ１〜ａ２間の環境Ｋの壁、ａ２〜ａ３間の動物体２０（移動前）の円弧部分、ａ３〜ａ４〜ａ１の環境Ｋの壁を含んでいる。
【０１５１】
（ａ１〜ａ２の方位角）
さて、ａ１〜ａ２の方位角の範囲では、予測距離画像の距離値ｄ_ｐ（θ，γ）は環境Ｋの壁部分のものであり、一方、移動後の全方位距離画像の距離値ｄ_ｇ（θ，γ）は動物体２０（移動後）の円弧部分のものである。従って、予測距離画像の距離値ｄ_ｐ（θ，γ）の方が、移動後の全方位距離画像の距離値ｄ_ｇ（θ，γ）のよりも大きくなる。このため、式（１６）を算出すると、生成する差分画像の距離値ｄδ（θ，γ）＞０（＝Ｔｈ）となる。又、この方位角ａ１〜ａ２は、Ｏ（θ，γ）＝０、すなわち、オクルージョン領域でない。この結果、方位角ａ１〜ａ２の範囲では、式（１７）の算出結果は、「１」、すなわち正となる。なお、図２１では、式（１７）の算出結果を括弧書で示している。
【０１５２】
（ａ２〜ａ３の方位角）
ａ２〜ａ３の方位角の範囲では、予測距離画像の距離値ｄ_ｐ（θ，γ）は移動前の動物体２０の円弧部分のものである。一方、移動後の全方位距離画像の距離値ｄ_ｇ（θ，γ）はアクティブセンサ１１に接近移動後の動物体２０の円弧部分のものである。このため、距離値ｄ_ｐ（θ，γ）＞距離値ｄ_ｇ（θ，γ）となり、式（１６）を算出すると、生成する差分画像の距離値ｄδ（θ，γ）＞０（＝Ｔｈ）となる。又、この方位角ａ２〜ａ３は、Ｏ（θ，γ）＝０、すなわち、オクルージョン領域でない。この結果、方位角ａ２〜ａ３の範囲では、式（１７）の算出結果は、「１」、すなわち正となる。
【０１５３】
（ａ３〜ａ４の方位角）
ａ３〜ａ４の方位角の範囲では、ａ１〜ａ２と同じ理由により、式（１７）の算出結果は、「１」、すなわち正となる。
【０１５４】
（ａ４〜ａ１の方位角）
ａ４〜ａ１の方位角の範囲では、予測距離画像の距離値ｄ_ｐ（θ，γ）及び移動後の全方位距離画像の距離値ｄ_ｇ（θ，γ）は同じ環境Ｋの壁部分のものである。このため、式（１６）を算出すると、生成する差分画像の距離値ｄδ（θ，γ）＝０となる。この結果、方位角ａ４〜ａ１の範囲では、式（１７）の算出結果は、「０」となる。
【０１５５】
このように、差分画像において、アクティブセンサ１１に近い位置へ変化した領域は、「正」の領域となり、移動後に得られる距離画像から、動物体２０を抽出したことになる。すなわち、「正」の領域は、動物体２０の移動後を抽出したことになる。
【０１５６】
３． 具体例３
次に図２３で示す具体例３を参照して、式（１７）の判定値の設定について説明する。具体例３は、アクティブセンサ１１と、動物体２０とが、同一線上において、同方向に移動するときの例である。この場合、具体例２と同じ理由で、Ｓ３０では、オクルージョン領域はないとされる。
【０１５７】
Ｓ４０における処理を説明する。
図２３は、アクティブセンサ１１の移動前後の位置と、動物体２０の移動前後の位置をそれぞれ示している。なお、説明の便宜上、以下は、アクティブセンサ１１において、仰角γ＝０における画素を前提として説明する。又、閾値Ｔｈを０とする。同図中、動物体２０、及びアクティブセンサ１１が移動後に実際に取得した全方位距離画像は、ａ１〜ａ２間の環境Ｋの壁、ａ２〜ａ３間の動物体２０（移動後）の円弧部分，ａ３〜ａ４〜ａ１の環境Ｋの壁を含んでいる。
【０１５８】
又、Ｓ２０で算出した予測距離画像は、ａ１〜ａ２間の動物体２０（移動前）の円弧部分、ａ２〜ａ３間の動物体２０（移動前）の円弧部分、ａ３〜ａ４間の動物体２０（移動前）の円弧部分、ａ４〜ａ１の環境Ｋの壁を含んでいる。
【０１５９】
（ａ１〜ａ２の方位角）
ａ１〜ａ２の方位角の範囲では、予測距離画像の距離値ｄ_ｐ（θ，γ）は動物体２０（移動前）の円弧部分のものであり、一方、移動後の全方位距離画像の距離値ｄ_ｇ（θ，γ）は環境Ｋの壁のものである。従って、予測距離画像の距離値ｄ_ｐ（θ，γ）の方が、移動後の全方位距離画像の距離値ｄ_ｇ（θ，γ）のよりも小さくなる。このため、式（１６）を算出すると、生成する差分画像の距離値ｄδ（θ，γ）＜０（＝−Ｔｈ）となる。又、この方位角ａ１〜ａ２は、Ｏ（θ，γ）＝０、すなわち、オクルージョン領域でない。この結果、方位角ａ１〜ａ２の範囲では、式（１７）の算出結果は、「−１」、すなわち負となる。なお、図２３では、式（１７）の算出結果を括弧書で示している。
【０１６０】
（ａ２〜ａ３の方位角）
ａ２〜ａ３の方位角の範囲では、予測距離画像の距離値ｄ_ｐ（θ，γ）は移動前の動物体２０の円弧部分のものである。一方、移動後の全方位距離画像の距離値ｄ_ｇ（θ，γ）はアクティブセンサ１１から離間移動後の動物体２０の円弧部分のものである。このため、距離値ｄ_ｐ（θ，γ）＜距離値ｄ_ｇ（θ，γ）となり、式（１６）を算出すると、生成する差分画像の距離値ｄδ（θ，γ）＜０（＝−Ｔｈ）となる。又、この方位角ａ２〜ａ３は、Ｏ（θ，γ）＝０、すなわち、オクルージョン領域でない。
【０１６１】
この結果、方位角ａ２〜ａ３の範囲では、式（１７）の算出結果は、「−１」、すなわち負となる。
（ａ３〜ａ４の方位角）
ａ３〜ａ４の方位角の範囲では、ａ１〜ａ２と同じ理由により、式（１７）の算出結果は、「−１」、すなわち負となる。
【０１６２】
（ａ４〜ａ１の方位角）
ａ４〜ａ１の方位角の範囲では、予測距離画像の距離値ｄ_ｐ（θ，γ）及び移動後の全方位距離画像の距離値ｄ_ｇ（θ，γ）は同じ環境Ｋの壁部分のものである。このため、式（１６）を算出すると、生成する差分画像の距離値ｄδ（θ，γ）＝０となる。この結果、方位角ａ４〜ａ１の範囲では、式（１７）の算出結果は、「０」となる。
【０１６３】
このように、差分画像において、アクティブセンサ１１から遠い位置へ変化した領域は、「負」の領域となり、移動後に得られる距離画像から、動物体２０を抽出したことになる。すなわち、「負」の領域は、動物体２０の移動前を抽出したことになる。
【０１６４】
アクティブセンサ１１と動物体２０とが同一直線上において、移動する場合、具体例２及び具体例３以外に、互いに離間する方向に移動する場合や、具体例３とは１８０度反対向きに、アクティブセンサ１１と動物体２０が移動する場合もある。これらの場合も、同様に、動物体２０の領域が「正」又は「負」となり、抽出される。
【０１６５】
さて具体例の説明を終了して、フローチャートの説明に戻る。
Ｓ４０では、前述のように処理した後、３値画像に対して、ノイズ除去処理を行い、Ｓ４０の処理を終了する。なお、ノイズ除去処理は、例えば、画素数（ピクセル数）が所定閾値以下のものを、ノイズとして除去する処理である。
【０１６６】
本実施形態によれば、下記に示す効果を有する。
（１） 本実施形態のアクティブセンサ１１の動物体検出装置では、コンピュータ１６（相対移動量推定手段）は、アクティブセンサ１１の移動前と移動後の、移動方向、移動距離、回転量を含む相対移動量を推定するようにした。又。コンピュータ１６（予測距離画像生成手段）は、移動前に取得した全方位距離画像と、前記推定した相対移動量に基づいて、移動後の予測距離画像を推定して生成するようにした。さらに、コンピュータ１６（オクルージョン領域推定手段）は、アクティブセンサ１１が移動することにより、前記予測距離画像に生ずるオクルージョン領域を推定するようにした。そして、コンピュータ１６（動物体領域抽出手段）は、前記予測距離画像と、移動後にアクティブセンサ１１が取得した全方位距離画像との差分画像を生成する際に、前記オクルージョン領域を取り除き、動物体領域を抽出するようにした。
【０１６７】
又、本実施形態のアクティブセンサの動物体検出プログラムは、コンピュータ１６を上記各手段として機能させるようにした。
又、アクティブセンサ１１の動物体検出方法は、第１ステップとして、アクティブセンサ１１の移動前後の、移動方向、移動距離、回転量を含む相対移動量を推定するようにした。又、第２ステップとして、移動前に取得した全方位距離画像と、前記第１ステップにて推定した相対移動量に基づいて、移動後の予測距離画像を推定して生成するようにした。さらに、第３ステップとして、アクティブセンサ１１が移動することにより、前記予測距離画像に生ずるオクルージョン領域を推定するようにした。又、第４ステップとして、前記予測距離画像と、移動後にアクティブセンサ１１が取得した全方位距離画像との差分画像を生成する際に、前記オクルージョン領域を取り除き、動物体領域を抽出するようにした。
【０１６８】
この結果、観測系が移動することにより生ずる動物体以外の環境の「見かけの変化」と、人物等の動物体による環境変化を容易に区別して動物体の検出を容易にでき、オクルージョン領域を推定して除去することにより、より正確に動物体による環境変化のみを検出することができる効果を奏する。
【０１６９】
（２） 本実施形態では、コンピュータ１６は、移動前の全方位距離画像のジャンプエッジ画素と、同画素の４−近傍（所定範囲内）に位置する画素の中で、アクティブセンサ１１から最も遠い距離値を有する対比対象画素について、相対移動量に基づき、それぞれ移動後の推定方位角、及び推定仰角を求めた。そして、コンピュータ１６は、ジャンプエッジ画素と対比対象画素のそれぞれの推定方位角の区間、及び推定仰角の範囲を、前記予測距離画像における前記オクルージョン領域のオクルージョン区間であると推定するようにした。
【０１７０】
又、本実施形態のアクティブセンサの動物体検出プログラムは、コンピュータ１６を上記手段として機能させるようにした。
又、本実施形態では、第３ステップとして、移動前の全方位距離画像のジャンプエッジ画素と、同画素の４−近傍（所定範囲内）に位置する画素の中で、アクティブセンサ１１から最も遠い距離値を有する対比対象画素について、相対移動量に基づき、それぞれ移動後の推定方位角、及び推定仰角を求めた。そして、第３ステップでは、ジャンプエッジ画素と対比対象画素のそれぞれの推定方位角の区間、及び推定仰角の範囲を、前記予測距離画像における前記オクルージョン領域のオクルージョン区間であると推定するようにした。
【０１７１】
この結果、オクルージョン領域を推定するためのオクルージョン区間を、容易に決定できる。
（３） 本実施形態では、コンピュータ１６は、オクルージョン領域推定手段として、判定対象画素の仰角が、ジャンプエッジ画素の推定仰角に一致し、かつ、判定対象画素の方位角が推定方位角で定めたオクルージョン区間内にあるときは、判定対象画素を、オクルージョン領域内にあると判定するようにした。
【０１７２】
又、本実施形態のアクティブセンサの動物体検出プログラムは、コンピュータ１６を上記手段として機能させるようにした。
又、本実施形態では、第３ステップとして、判定対象画素の仰角が、ジャンプエッジ画素の推定仰角に一致し、かつ、判定対象画素の方位角が推定方位角で定めたオクルージョン区間内にあるとき（条件１）は、判定対象画素は、オクルージョン領域内にあると判定するようにした。
【０１７３】
この結果、条件１が成立したときに、オクルージョン領域の推定を容易にできる。
（４） 又、コンピュータ１６は、オクルージョン領域推定手段として、判定対象画素の方位角が、ジャンプエッジ画素の推定方位角に一致し、かつ、判定対象画素の仰角が推定仰角で定めたオクルージョン区間内にあるとき（条件２）は、判定対象画素をオクルージョン領域内にあると判定するようにした。
【０１７４】
又、本実施形態のアクティブセンサの動物体検出プログラムは、コンピュータ１６を上記手段として機能させるようにした。
又、第３ステップとして、判定対象画素の方位角が、ジャンプエッジ画素の推定方位角に一致し、かつ、判定対象画素の仰角が推定仰角で定めたオクルージョン区間内にあるとき（条件２）は、判定対象画素をオクルージョン領域内にあると判定するようにした。
【０１７５】
この結果、条件２が成立したときに、オクルージョン領域の推定を容易にできる。
（５） 本実施形態では、コンピュータ１６は、動物体領域抽出手段として、差分画像を生成する際に、各画素の距離値の差に基づいて、正及び負の符号判定を行い、この正又は負の符号判定に基づいて、動物体領域抽出を行う。
【０１７６】
又、本実施形態のアクティブセンサの動物体検出プログラムは、コンピュータ１６を上記手段として機能させるようにした。
又、第４ステップとして、差分画像を生成する際に、各画素の距離値の差に基づいて、正及び負の符号判定を行い、この正及び負の符号判定に基づいて、動物体領域抽出を行う。
【０１７７】
この結果、正及び負の符号判定を行うことにより、移動前と移動後の両方の動物体領域抽出を容易に行うことができる。
（実験例）
図２４〜図３１は、実験室において、アクティブセンサ１１は、人物が位置する方向に向かって移動するとともに、動物体としての人物が実験室の壁側に向かって移動する前後の画像を取得し画像処理した結果を示している。なお、各図において、横軸は、方位角θ（０＜θ＜３６０度）であり、縦軸は、仰角γ（−３／３＜γ＜π／３）の範囲を対象として図示している。
【０１７８】
図２４は「移動前」、図２５は「移動後」のもので、両図は、距離画像ではなく、全方位のカラー画像で取得したものを説明の便宜上、白黒の濃淡画像で示している。図２６は図２４に対応した全方位距離画像、図２７は、図２５に対応した距離画像である。
【０１７９】
そして、得られた相対移動量を用いて、図２６から生成した予測距離画像を図２８に、推定したオクルージョン領域を、図２９に示している。図３０は、図２８の予測距離画像と移動後取得した全方位距離画像（図２７）の差分をとり、距離値が「正」であった領域を白、「負」であった領域をグレーで示している。ここでは、グレーで示された領域は、アクティブセンサ１１が移動前存在していたものが、移動後なくなっていることを示している。又、白で示す領域は、アクティブセンサ１１が移動前なかったものが、移動後現れた領域を示している。図２８において、人物像の右側輪郭に沿って現れている黒い領域は、オクルージョン領域推定に相当する。この領域は、図２９で示す、推定したオクルージョン領域にて取り除き、移動後現れた領域のみを図３１に示す。
【０１８０】
このように、オクルージョン領域を考慮することにより、動物体を精度良く検出できている。
なお、本発明の実施形態は上記実施形態に限定されるものではなく、発明の趣旨を逸脱しない範囲で、適宜に変更して次のように実施することもできる。
【０１８１】
（１） 前記実施形態では、アクティブセンサ１１として、「”実環境センシングのための全方位ステレオシステム（ＳＯＳ）”、電気学会論文誌Ｃ．Ｖｏｌ．１２１−Ｃ，Ｎｏ．５，ｐｐ．８７６−８８１．２００１」に記載されているものを使用した。これに限らず、他の全方位カメラから得られた全方位距離画像を入力するようにしてもよい。
【０１８２】
（２） 前記実施形態では、コンピュータ１６は、動物体領域抽出手段として、差分画像を生成する際に、各画素の距離値の差に基づいて、正及び負の符号判定を行い、この正及び負の符号判定に基づいて、動物体領域抽出を行った。
【０１８３】
これに代えて、コンピュータ１６は、動物体領域抽出手段として、差分画像を生成する際に、各画素の距離値の差に基づいて、正又は負のいずれかの符号判定を行い、この正又は負のいずれかの符号判定に基づいて、動物体領域抽出を行うようにしてもよい。
【０１８４】
この場合、移動前、或いは、移動後の動物体領域抽出を行うことができる。
（３） 前記実施形態では、Ｓ３０において、移動前において取得した全方位距離画像におけるジャンプエッジの画素（ピクセル）を、（θ_１，γ_１，ｄ_ｓ（θ_１，γ_１））とする。そして、画素（θ_１，γ_１，ｄ_ｓ（θ_１，γ_１））の所定範囲内としての４−近傍にある距離値ｄ_ｓ（θ_１，γ_１）より最も遠い距離値を有する画素を（θ_２，γ_２，ｄ_ｓ（θ_２，γ_２））とした。そして、「所定値」とは、最も遠い距離値を意味するようにしたが、これに限定するものではない。
【０１８５】
例えば、動物体の移動速度と、アクティブセンサの移動速度が予め分かっており、両者の移動範囲が分かっている場合には、ジャンプエッジと４−近傍にある画素との距離値がとりうる範囲を予め予測できる。この場合には、予測できる距離値の範囲の中で、所定値として予め定数を決定しておき、距離値がこの定数以上の複数の画素を対比対象画素としてもよい。
【０１８６】
（４） 前記実施形態では、ジャンプエッジ画素に対する所定範囲内として４−近傍としたが、８−近傍や１６−近傍等であってもよい。 （５） 前記実施形態では、式（１５）において、予測距離画像における判定対象画素の仰角γが、推定仰角γ_１’に一致し、予測距離画像における判定対象画素の方位角θがオクルージョン区間Ｉθ内にあるときは、Ｏ（θ，γ）＝１とする。すなわち、判定対象画素はオクルージョン領域内にあると判定した。又、予測距離画像における判定対象画素の方位角θが推定方位角θ_１’に一致し、判定対象画素の仰角γがオクルージョン区間Ｉγ内にあるときは、Ｏ（θ，γ）＝１とする。すなわち、判定対象画素はオクルージョン領域内にあると判定した。そうでない場合には、Ｏ（θ，γ）＝０とした。
【０１８７】
これに代えて、上記（３）の場合のように、対比対象画素が複数になった場合、前記複数の対比対象画素で囲まれる範囲をオクルージョン領域として判定してもよい。
【０１８８】
【発明の効果】
以上詳述したように請求項１乃至請求項１０に記載の発明によれば、観測系が移動することにより生ずる動物体以外の環境の「見かけの変化」と、人物等の動物体による環境変化を容易に区別して動物体の検出を容易にできる。そして、オクルージョン領域を推定して除去することにより、より正確に動物体による環境変化のみを検出することができる効果を奏する。
【０１８９】
請求項１１乃至請求項１５の発明によれば、アクティブセンサの動物体検出方法及び動物体検出装置を容易に実現できる動物体検出プログラムを提供できる。
【図面の簡単な説明】
【図１】アクティブセンサ１１の電気的構成のブロック図。
【図２】アクティブセンサ１１の機械的構成の概略図。
【図３】（ａ）〜（ｃ）はコンピュータ１６が実行する位置・姿勢推定プログラムのフローチャート。
【図４】図５に対応するパノラマエッジ画像の説明図。
【図５】エッジヒストグラムの例を示す説明図。
【図６】アクティブセンサ１１の平行移動におけるエッジヒストグラムのシフトを説明するための説明図。
【図７】（ａ）はアクティブセンサ１１を中心にした円筒座標系の説明図、（ｂ）は、アクティブセンサ１１を中心にした四角筒座標系の説明図、（ｃ）は、異なる空間系にエッジを写像したときの、ヒストグラムのシフトとｓｉｎ曲線を表した説明図。
【図８】アクティブセンサ１１の２つの地点における全方位エッジヒストグラムのマッチングコストマトリクスＣ（ｓ，ｉ）の説明図。
【図９】総合シフト量から回転量ｓφと移動方向ωを求める方法の説明図。
【図１０】床平面投影ヒストグラムの説明図。
【図１１】本実施形態において、ρ−θ表面上で複数の候補点の各々に退位する直線群を示した図。
【図１２】ドミナント射影方向のヒストグラム。
【図１３】図１２とは異なるドミナント射影方向のヒストグラム。
【図１４】ハフ変換の説明図。
【図１５】ρ−θ表面上で、ある点を通る直線群を示した図。
【図１６】ρ−θ表面上で複数の候補点の各々に退位する直線群を示した図。
【図１７】仰角γの説明図。
【図１８】距離値の説明図。
【図１９】オクルージョン領域の説明図。
【図２０】具体例１の式（１７）の判定値の設定の説明図。
【図２１】具体例２の式（１７）の判定値の設定の説明図。
【図２２】同じく具体例２のジャンプエッジの画素に関する説明図。
【図２３】具体例３の式（１７）の判定値の設定の説明図。
【図２４】アクティブセンサ１１の「移動前」における白黒の濃淡画像。
【図２５】アクティブセンサ１１の「移動後」における白黒の濃淡画像。
【図２６】図２４に対応した全方位距離画像。
【図２７】図２５に対応した距離画像。
【図２８】予測距離画像。
【図２９】推定して生成されたオクルージョン領域を示す画像。
【図３０】差分画像。
【図３１】動物体移動後の動体検出された画像。
【符号の説明】
１１…アクティブセンサ
１２…ステレオユニット
１６…コンピュータ（相対移動量推定手段、予測距離画像生成手段、オクルージョン領域推定手段、及び動物体領域抽出手段）[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a moving object detecting device and a moving object detecting method of an active sensor, and a moving object detecting program.
[0002]
[Prior art]
In recent years, mobile robots coexisting with humans have been actively developed. In order for the robot to move autonomously, it is necessary to obtain external information and grasp the environment. A visual sensor is a useful sensor for recognizing an environment. Many studies of mobile vision have been for moving in a stationary environment. However, in order to realize a robot that coexists with humans, it is necessary to avoid collisions and move safely even in a dynamic environment where moving objects such as people exist, and to move It is required to detect a moving object while performing the method (Non-Patent Documents 1 to 3).
[0003]
When the observation system moves, the acquired image includes “appearance change” due to movement of the observation system (viewpoint movement) and “change in environment itself due to a moving object such as a person”. To detect a moving object means to extract only "changes in the environment itself", and it is necessary to distinguish these two changes. The “apparent change” due to the movement of the observation system includes a change due to the relative motion of the stationary environment and a change due to the occlusion. The change due to the relative movement of the still environment due to the movement of the viewpoint appears in the image as a change in the position or size of the still environment. The change due to occlusion is an apparent change such that an area that is not visible before the movement is seen due to the movement of the viewpoint, or an area that was seen becomes invisible.
[0004]
This occlusion appears largely as an area, particularly when an object is close to the camera, and it is difficult to distinguish the occlusion from environmental changes caused by moving objects such as people. In order for a robot to move autonomously, it is important to detect information and changes near the robot.
[0005]
Conventionally, as a method of detecting a moving object in consideration of the movement of an observation system, a method of estimating the moving object using an optical flow has been proposed (Non-Patent Documents 4 to 7). In these methods, a region where the optical flow in the region of the stationary object radially distributes around the vanishing point is used as a moving object to detect a region where the characteristic is not satisfied.
[0006]
[Non-patent document 1]
Shigeki Ishikawa and Shunichi Asaka, "A guide method for autonomous mobile robots in a running environment with dynamic changes including moving obstacles", Journal of the Robotics Society of Japan, Vol. 11, No. 6, pp. 856-867, 1993.
[Non-patent document 2]
Akira Inoue, Kenji Inoue, Yoshikuni Okawa, "Online Avoidance Behavior of Autonomous Mobile Robot Based on Prediction of Behavior of Multiple Moving Obstacles", Journal of the Robotics Society of Japan, Vol. 15, No. 2, pp. 249-260, 1997.
[Non-Patent Document 3]
Yukiyuki Matsumura, Yasuyuki Murai, "Fuzzy Robot Obstacle Avoidance Based on Genetic Programming", IEICE Transactions A, Vol. J83-A, no. 12, pp. 1539-1551, 2000.
[Non-patent document 4]
Naoya Ota, "Shape Recovery from Optical Flow with Reliability Information and Its Application to Moving Object Detection", IEICE Transactions on Electronics (D-II), Vol. J76-D-II, No. 8, pp. 1562-1571, 1993.
[Non-Patent Document 5]
Nobuyuki Takeda, Mutsumi Watanabe, Kazunori Onoguchi, "Moving obstacle detection using vanishing point estimation residual method", Proc. 29-34, 1996.
[Non-Patent Document 6]
Takumi Ebine and Nozomu Hamada, "Moving Object Detection Based on Optical Flow Estimation Considering Motion of Observation System", Transactions of the Institute of Electronics, Information and Communication Engineers (D-II), Vol. J83-D-II, No. 6, pp. 1498-1506, 2000.
[Non-Patent Document 7]
Ryuzo Okada, Yoshiaki Shirai, Jun Miura, Yoshinori Kuno “Moving Object Tracking Based on Optical Flow and Distance Information”, IEICE Transactions on Electronics (D-II), Vol. J80-D-II, No. 6, pp. 1530-1538, 1997.
[0007]
[Problems to be solved by the invention]
However, these methods using optical flow have a problem that it is difficult to distinguish the background from the moving object because the flow vector is small when the movement is parallel to the optical axis in principle. In addition, it is assumed that the amount of time movement is small so that a sudden change in appearance does not occur.For example, in the case where the distance between the camera and the moving object is short, the appearance change becomes large even with a slight movement. However, it is difficult to accurately extract an animal body.
[0008]
The present invention relates to a moving object detection method and an animal of an active sensor capable of easily distinguishing an apparent change in an environment other than a moving object due to movement of an observation system from an environmental change due to movement of the moving object and facilitating detection of the moving object. It is intended to provide a body detection device. Another object of the present invention is to provide a moving object detection method and a moving object detection device of an active sensor that can more accurately detect a moving object by estimating an occlusion region.
[0009]
It is another object of the present invention to provide a moving object detection program that can easily realize the moving object detection method and the moving object detection device of the active sensor.
[0010]
[Means for Solving the Problems]
In order to solve the above problem, the invention according to claim 1 is an apparatus for detecting a moving object in an active sensor capable of acquiring an omnidirectional distance image and moving in an environment, before and after movement of the active sensor. The moving direction, the moving distance, the relative moving amount estimating means for estimating the relative moving amount including the amount of rotation, the omnidirectional distance image obtained before the movement, and the relative moving amount estimated by the relative moving amount estimating means. A predicted distance image generating means for estimating and generating an omnidirectional distance image (hereinafter, referred to as a predicted distance image) after the movement, and estimating an occlusion region generated in the predicted distance image when the active sensor moves. An occlusion area estimating means for generating the difference image between the predicted distance image and the omnidirectional distance image acquired by the active sensor after the movement. Remove the John region, it is an gist a moving object detection apparatus of the active sensor, characterized in that a moving object region extraction means for extracting a moving object region.
[0011]
According to a second aspect of the present invention, in the first aspect, the occlusion area estimating means is arranged such that a pixel of a jump edge of the omnidirectional distance image acquired before the movement (hereinafter, referred to as a jump edge pixel) is within a predetermined range of the pixel. Among the pixels located, for a pixel having a distance value equal to or greater than a predetermined value from the active sensor (hereinafter, referred to as a comparison target pixel), based on the relative movement amount, an estimated azimuth after movement and an estimated elevation angle are obtained, The section of the estimated azimuth angle and the range of the estimated elevation angle of each of the jump edge pixel and the comparison target pixel are estimated to be the occlusion section of the occlusion area in the predicted distance image.
[0012]
According to a third aspect of the present invention, in the second aspect, the occlusion area estimating means sets the elevation angle of the determination target pixel when determining whether the determination target pixel in the predicted distance image is in the occlusion area. If the estimated elevation angle of the jump edge pixel matches and the azimuth of the determination target pixel is within the occlusion section defined by the estimated azimuth angle, the determination target pixel is determined to be within the occlusion area. It is characterized by doing.
[0013]
According to a fourth aspect of the present invention, in the second aspect, the occlusion area estimating means determines an azimuth angle of the determination target pixel when determining whether or not the determination target pixel in the predicted distance image is in the occlusion area. Is equal to the estimated azimuth angle of the jump edge pixel, and the elevation angle of the determination target pixel is within the occlusion section defined by the estimated elevation angle, the determination target pixel is determined to be within the occlusion area It is characterized by doing.
[0014]
According to a fifth aspect of the present invention, in the method of any one of the first to fourth aspects, the moving object region extracting means generates the difference image based on a difference between distance values of pixels. At least one of a positive sign and a negative sign is determined, and a moving object region is extracted based on the sign determination.
[0015]
According to a sixth aspect of the present invention, in the moving object detection method of an active sensor capable of acquiring an omnidirectional distance image and moving in an environment, a moving direction, a moving distance, and a rotation before and after the movement of the active sensor are determined. A first step of estimating a relative movement amount including the amount, an omnidirectional distance image acquired before the movement, and an omnidirectional distance image after the movement (hereinafter, referred to as “the omnidirectional distance image”) based on the relative movement amount estimated in the first step. A second step of estimating and generating an estimated distance image), a third step of estimating an occlusion region occurring in the predicted distance image due to the movement of the active sensor, When generating a difference image from the omnidirectional distance image acquired by the sensor, the method may include a fourth step of removing the occlusion region and extracting a moving object region. Moving objects detection method of an active sensor, characterized in it is an gist.
[0016]
According to a seventh aspect of the present invention, in the sixth aspect, the third step includes: determining a position of a jump edge pixel (hereinafter, referred to as a jump edge pixel) of the omnidirectional distance image acquired before the movement within a predetermined range of the pixel. Of the pixels having a distance value greater than or equal to a predetermined value from the active sensor (hereinafter, referred to as comparison target pixels), an estimated azimuth angle and an estimated elevation angle after the movement are determined based on the relative movement amount. Estimating the section of the estimated azimuth angle and the range of the estimated elevation angle of each of the jump edge pixel and the comparison target pixel as the occlusion section of the occlusion area in the predicted distance image.
[0017]
In the invention according to claim 8, in claim 7, in the third step, when determining whether or not the determination target pixel in the predicted distance image is in the occlusion area, the elevation angle of the determination target pixel is: If the estimated elevation angle of the jump edge pixel matches, and the azimuth of the determination target pixel is within the occlusion section defined by the estimated azimuth angle, the determination target pixel is determined to be within the occlusion area It is characterized by the following.
[0018]
In a ninth aspect of the present invention, in the ninth aspect, in the third step, when determining whether or not the determination target pixel in the predicted distance image is in the occlusion area, the azimuth of the determination target pixel is If the estimated azimuth angle of the jump edge pixel matches, and the elevation angle of the determination target pixel is within the occlusion section defined by the estimated elevation angle, the determination target pixel is determined to be within the occlusion area It is characterized by the following.
[0019]
According to a tenth aspect of the present invention, in the method according to any one of the sixth to ninth aspects, the fourth step includes the steps of: generating a difference image based on a difference between distance values of pixels when generating the difference image; At least one of negative sign determination is performed, and a moving object region is extracted based on the code determination.
[0020]
An invention according to claim 11 is a computer, comprising: a computer for estimating a relative movement amount including a movement direction, a movement distance, and a rotation amount before and after movement of an active sensor; A predicted distance image generation unit configured to estimate and generate an omnidirectional distance image (hereinafter, referred to as a predicted distance image) after movement based on the distance image and the relative movement amount estimated by the relative movement amount estimation unit; An occlusion area estimating means for estimating an occlusion area generated in the predicted distance image due to the movement of the active sensor, and generating a difference image between the predicted distance image and the omnidirectional distance image acquired by the active sensor after the movement. An active cell, wherein the occlusion area is removed to function as a moving object area extracting means for extracting a moving object area. The service animal body detecting program of it is an gist.
[0021]
According to a twelfth aspect of the present invention, in addition to the eleventh aspect, a computer (hereinafter, referred to as a jump edge pixel) of a jump edge of an omnidirectional distance image acquired before movement is used as the occlusion area estimating means. Of pixels having a distance value equal to or greater than a predetermined value from the active sensor (hereinafter, referred to as comparison target pixels) among the pixels located within the predetermined range, the estimated azimuth after movement based on the relative movement amount, and Obtaining an estimated elevation angle and causing the section of the estimated azimuth angle of each of the jump edge pixel and the comparison target pixel, and the range of the estimated elevation angle to function as an occlusion section of the occlusion area in the predicted distance image. It is characterized by.
[0022]
According to a thirteenth aspect of the present invention, in the twelfth aspect, when the computer is used as the occlusion area estimating means to determine whether a pixel to be determined in the predicted distance image is in the occlusion area, When the elevation angle of the target pixel matches the estimated elevation angle of the jump edge pixel, and the azimuth of the determination target pixel is within the occlusion section defined by the estimated azimuth angle, the determination target pixel is in the occlusion area. It is made to function to judge that it is within.
[0023]
According to a fourteenth aspect of the present invention, in the twelfth aspect, when the computer is used as the occlusion area estimating means to determine whether or not a determination target pixel in the predicted distance image is in the occlusion area, When the azimuth of the target pixel matches the estimated azimuth of the jump edge pixel and the elevation angle of the determination target pixel is within the occlusion section defined by the estimated elevation angle, the determination target pixel is in the occlusion area. It is made to function to judge that it is within.
[0024]
According to a fifteenth aspect of the present invention, in any one of the eleventh to fourteenth aspects,
Further, when the computer is used as the moving object region extracting means to generate the difference image, at least one of positive and negative signs is determined based on a difference between the distance values of the pixels, and the sign is determined. A moving object region is extracted based on the determination.
[0025]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, an embodiment that embodies the moving object detecting device of the active sensor of the present invention will be described with reference to FIGS.
[0026]
The moving object detecting device of the active sensor according to the present embodiment includes an active sensor 11 and a computer 16.
FIG. 1 is a block diagram showing an electrical configuration of the active sensor. FIG. 2 is a schematic diagram of a mechanical configuration of the active sensor 11.
[0027]
The active sensor 11 includes a vehicle body β having a plurality of wheels, and the wheels are driven by an electric motor (not shown) provided in the vehicle body β to automatically travel toward an arbitrary position in the environment K (linear and linear). Including running along curves). In FIG. 2, the environment K is shown smaller than the vehicle body β for convenience of explanation.
[0028]
The active sensor 11 includes a plurality of three-lens stereo units (hereinafter, simply referred to as a stereo unit 12), a memory unit 15, a synchronization signal generator 17, and the like. The memory unit 15, the computer 16, the synchronization signal generator 17, and the like are stored in the vehicle body β.
[0029]
The plurality of stereo units 12 correspond to an imaging unit. The computer 16 corresponds to a relative movement amount estimating unit, a predicted distance image generating unit, an occlusion region estimating unit, and a moving object region extracting unit.
[0030]
The active sensor 11 is a device capable of acquiring a color image and three-dimensional information (distance image) in all directions (hereinafter, all directions) in a three-dimensional space at the same time in real time. The stereo unit 12 includes three video cameras, and each stereo unit 12 is arranged on each surface of the regular icosahedron. Each of the stereo units 12 has the same characteristics, and the stereo units 12 arranged on each surface can acquire an omnidirectional color image and an omnidirectional black and white image (hereinafter referred to as distance data) at the same time in real time. It is. Thus, a color image and three-dimensional information in all directions in the three-dimensional space can be obtained at the same time.
[0031]
Further, by arranging the stereo units 12 having the same characteristics on each surface of the regular icosahedron, it is possible to equally divide the three-dimensional space and obtain high-resolution information. The active sensor 11 is described in "" Omnidirectional Stereo System (SOS) for Real Environment Sensing ", IEICE Transactions C. Vol. 121-C, No. 5, pp. 876-881, 2001. Has been described.
[0032]
As shown in FIG. 1, the stereo unit 12 includes one reference video camera VCs and a pair of reference video cameras VC. The reference video camera VC is disposed so as to be included in a pair of planes orthogonal to each other with the optical axis of the reference video camera VCs as the line of intersection. These cameras are arranged so as to form two stereo pairs.
[0033]
From each stereo unit 12, a stereo image composed of one color image and two black and white images is acquired, and 20 color images and omnidirectional 40 color images are set as one set at 15 sets / sec. Transfer to the memory unit 15. The memory unit 15 stores the transferred omnidirectional image data.
[0034]
A common external synchronization signal is supplied from the synchronization signal generator 17 to each video camera of each stereo unit 12. As a result, completely synchronized image data can be obtained in the digitized frame.
[0035]
The computer 16 executes a moving object detection program of an active sensor stored in advance in a ROM 16a provided in the computer 16 at predetermined intervals. The computer 16 having the ROM 16a corresponds to a storage unit.
[0036]
Further, the computer 16 accesses the memory unit 15 to acquire an omnidirectional color image and distance data from time to time.
(Action)
Now, the operation of the moving object detecting device of the active sensor configured as described above will be described with reference to FIGS.
[0037]
FIG. 3A is a flowchart of an active sensor moving object detection program executed by the computer 16. The computer 16 executes the moving object detection program at predetermined intervals to detect the moving object of the active sensor 11.
[0038]
Hereinafter, each step will be described.
1. S10 (self position / posture estimation)
In S10, the self-position / posture estimation of the active sensor 11 is performed. Specifically, the processing of S110 to S160 in FIG. 3B is performed.
[0039]
1.1. S110 (edge detection)
In S110, edge detection is performed.
That is, the color image acquired by the reference video camera VCs (center camera) is shaded and passed through a LoG (Lapsian of Gaussian) filter, and the zero cross point is detected as an edge.
[0040]
1.2. S120 (calculation of maximum edge gradient)
Next, the maximum edge gradient is calculated.
That is, the edge gradient (intensity) of each edge pixel is calculated by the Sobel filter, and the maximum edge gradient in the image (that is, the maximum edge intensity) is obtained.
[0041]
This is because the LoG filter is easily affected by noise, and there are many false edges due to the influence of noise and illumination. This is a process for removing this.
By this processing, edge pixels whose intensity is less than 5% of the maximum edge gradient value are deleted from the edge as noise.
[0042]
1.3. S130 (generation of edge histogram at current position)
Next, in S130, the edge point obtained by the reference video camera VCs, which is the center camera of each stereo unit 12, is mapped to a 360 ° panoramic edge image coordinate system, and an edge histogram of vertical projection is generated. The edge histogram corresponds to a statistic.
[0043]
The 360 ° panoramic edge image is also referred to as a 360 ° cylindrical image. That is, the edges in this cylindrical image are projected in the vertical direction (the direction along the axis of the cylinder) to generate an omnidirectional edge histogram.
[0044]
FIG. 5 shows an example of the generated omnidirectional edge histogram. In FIG. 5, the horizontal axis indicates the range of 0 to 360 °, and the vertical axis indicates the histogram. FIG. 4 shows a panoramic edge image corresponding to FIG.
[0045]
(Relationship between position / posture and omnidirectional edge histogram)
Here, the relationship between the position / posture of the active sensor 11 and the histogram will be described.
[0046]
The movement or rotation of the active sensor 11 causes a shift of the edge histogram of the omnidirectional image obtained by the active sensor 11.
The rotation amount of the edge histogram generated by the rotation of the active sensor 11 is constant at all azimuth angles. However, the movement amount of the edge histogram generated by the parallel movement of the active sensor 11 is related to the movement direction and the azimuth of the edge.
[0047]
As shown in FIG. 6, when the active sensor 11 moves in a certain moving direction ω from the pre-movement point α, the moving amount of the edge histogram is small at azimuth angles of ω ± nπ, n = 0,1. n = 0 is the moving direction ω, and n = 1 is the direction opposite to the moving direction ω.
[0048]
The total shift amount is the sum of the rotation amount and the movement amount. The moving direction ω is a direction away from 0 ° by ω ° when the direction of the predetermined direction is 0 ° with respect to the pre-movement point α as shown in FIG.
[0049]
At the azimuth angle of ω ± (2n + 1) π / 2, the movement amount of the edge histogram becomes large.
It is assumed that the active sensor 11 rotates at a rotation angle φ while moving, where L is the moving distance along the moving direction ω.
[0050]
In this case, the total shift amount δθ of the edge pixel located at a certain azimuth angle θ in the cylindrical coordinate system (360 ° panoramic edge image coordinate system) is determined by the following equation (1).
[0051]
(Equation 1)

Here, dθ is a three-dimensional depth (distance to the center of the active sensor 11) of the edge located at the azimuth angle θ. Actually, in many cases, edges in the same direction have different depths, and therefore, Equation (1) represents only an ideal case (edges in the same direction have the same depth).
[0052]
Equation (1) is affected by the depth of the edge, but can be approximated by sin, as shown in FIG. 7C, and has a period of 2π similarly to the sin curve. The sign of the total shift amount δθ is inverted every π period.
[0053]
FIG. 7C is an explanatory diagram showing a curve and a sin curve when the total shift amount δθ of the omnidirectional edge histogram is calculated by the equation (1).
In the figure, A is a sine curve (sinusoidal curve), and B is the total shift amount δθ of the omnidirectional edge histogram based on equation (1) when the active sensor 11 moves in a cylindrical room. This is a calculated curve (see FIG. 7A).
[0054]
C is a curve obtained by calculating the total shift amount δθ of the omnidirectional edge histogram based on Equation (1) when the active sensor 11 moves in a square room (see FIG. 7B).
[0055]
In the present embodiment, the moving direction ω and the rotation angle φ of the active sensor 11 can be simply and robustly calculated from the total shift amount δθ of the omnidirectional edge histogram by using the feature that the sign of the equation (1) is inverted every π period. Estimate.
[0056]
1.4. S140 (Dynamic programming matching: DP matching)
In S140, DP matching is performed between the omnidirectional edge histogram at the current position (post-move point) of the active sensor 11 and the omnidirectional edge histogram at the pre-move point, and the histograms at each azimuth angle (azimuth angle at each pre-move point) are integrated. The shift amount δθ is obtained.
[0057]
The omnidirectional edge histogram at the post-movement point corresponds to the current position histogram, and the omnidirectional edge histogram at the pre-movement point corresponds to the pre-movement point histogram.
By calculating the total shift amount δθ, the moving direction ω and the rotation angle φ of the post-movement point of the active sensor 11 with respect to the pre-movement point are calculated.
[0058]
Hereinafter, a method of calculating the total shift amount δθ of the omnidirectional edge histogram of the post-movement point of the active sensor 11 with respect to the pre-movement point will be described in detail.
(About dynamic programming matching (DP matching))
Here, the DP matching of the omnidirectional edge histogram of the post-movement point and the omnidirectional edge histogram of a certain pre-movement point, that is, a method of DP matching two omnidirectional edge histograms will be described.
[0059]
Each of the omnidirectional edge histograms at the point before and after
Point before movement: Hp = [hp (i), i = 0,..., N−1]
When
Post-move point: Hc = [hc (j), j = 0,..., N−1}
And
[0060]
N is a multiple of 360 and is determined by the azimuth angle θ when the edge histogram is generated. In the present embodiment, N = 720. That is, the angular resolution of the edge histogram is 0.5 °. One pin hp (i) of the omnidirectional edge histogram Hp at the point before the movement is hp (i) when the shift amount si occurs in the omnidirectional edge histogram Hc at the point after the movement of the active sensor 11 due to the movement and rotation. It can be assumed that it is similar to hc (i + si).
[0061]
Assuming that the square of the difference between hp (i) and hc (i + si) is the matching cost between hp (i) and hc (i + si), the following matching cost matrix C (s, i) is obtained.
[0062]
(Equation 2)

Here, since the omnidirectional edge histogram has a period of 2π, when i + si ≧ N,
hc (i + si) ≡hc (i + si−N).
[0063]
If [(hp (i), hc (i + si)), i = 0,..., N−1] is a correct match, those matching pairs have lower costs in the matching cost matrix C (s, i). And a curve in the form of equation (1). Here, (hp (i), hc (i + si)) represents a matching pair of hp (i) and hc (i + si).
[0064]
FIG. 8 shows a matching cost matrix C (s, i) of an omnidirectional edge histogram at two points of the active sensor 11. For convenience of explanation, a path having a low matching cost in the matrix is centered, and upper and lower paths are cut. In the figure, each symbol of “.− + * &% # @ ABCD” indicates a cost level, and the order of “.− + * &% # @ ABCD” is from left to right. Are arranged in ascending order of cost. That is, the matching cost is “.” <“−” <“+” <“*” <“&” <“%” <“#” <“＄” <“＠” <“A” <“B” <“ C><D.
[0065]
In the figure, the vertical axis represents the edge shift amount si and the horizontal axis represents i, which corresponds to the azimuth angle θ. In the drawing, a low-cost curve similar to the sin curve in C (s, i) corresponds to the total shift amount δθ of the edge histogram at each azimuth angle θ. By searching for a low-cost curve similar to the sin curve in C (s, i), the total shift amount δθ of the omnidirectional histogram between Hp and Hc is obtained.
[0066]
In the present embodiment, a path having a period of 2π and a continuous minimum cost is obtained from C (s, i) using a dynamic programming (DP) having a low computation cost, and the rotation angle φ is robustly determined from the path. And the moving direction ω is estimated.
[0067]
A continuous minimum cost path having a period 2π from C (s, i) can be defined in the following conditional minimization problem.
[0068]
[Equation 3]

Here, si is the total shift amount of the edge histogram to be obtained.
[0069]
Since the periods of hp (i) and hc (j) are N, the indexes s and i of C (s, i) are calculated as s ± N → s and i ± N → i.
Assuming that the path starts at a particular row k (ie, s0 = k), minimization of equation (2) can be determined by dynamic programming as follows.
[0070]
(Equation 4)

[0071]
(Equation 5)

[0072]
(Equation 6)

When calculating S (s, i), any one of min [S (s-1, i-1), S (s, i-1), S (s + 1, i-1)] is minimized. If a path that has reached Cmin (k) is stored in reverse, a path with the minimum cost (hereinafter, referred to as a minimum cost path) can be obtained.
[0073]
For k = 0,..., N−1, Cmin (k) is calculated as described above.
Minimum value C ＾ min (k) = Cmin (k ＾)
Ask for.
[0074]
The path to which Cmin (k ＾) corresponds is the optimal matching between Hp and Hc.
Hereinafter, the above path is referred to as an optimal matching path.
By the above calculation, a matching path of an edge histogram having a period of 2π and having continuity can be searched.
[0075]
In the following, matching between the omnidirectional edge histogram of the post-move point and the omnidirectional edge histogram of the other pre-move point is similarly performed.
1.5. S150 (Estimation of posture and moving direction)
In S150, the moving direction ω and the rotation angle φ of the active sensor 11 with respect to the pre-movement point are estimated from the total shift amount of the omnidirectional edge histograms of the post-movement point and the pre-movement point of the active sensor 11 obtained in S140.
[0076]
(Estimation of rotation angle φ)
First, estimation of the rotation angle φ of the active sensor 11 will be described.
The minimum cost path si, (i = 0,..., N−1) obtained by the DP matching is between the omnidirectional edge histograms Hc and Hp of the post-movement point and the pre-movement point of the active sensor 11. This represents the total shift amount. The waveform shown near the sin curve in FIG. 9 shows the minimum cost path obtained by DP matching.
[0077]
As shown in the equation (1), these total shift amounts are obtained from the rotation amount of the entire histogram generated at the rotation angle φ of the active sensor 11 and the movement amount at each azimuth angle caused by the parallel movement in the movement direction ω. Become.
[0078]
As can be seen from Expression (1), the total shift amount of the histogram is obtained by subtracting the rotation amount sφ corresponding to the rotation angle φ, and the shift amount after the subtraction is inverted every π period. That is, it is vertically inverted at a period of π around the rotation angle φ (see FIG. 9).
[0079]
Therefore, in S150 of the present embodiment, the rotation amount sφ is estimated by calculating using the following equation. That is, the rotation angle φ is estimated.
[0080]
(Equation 7)

[0081]
(Equation 8)

That is, the rotation amount sφ divides the shift amount si, (i = 0,..., N−1) of the edge histogram into upper and lower halves (see FIG. 9).
[0082]
(Estimation of the moving direction ω of the active sensor 11)
Next, the moving direction ω of the active sensor 11 is robustly estimated using the fact that the equation (1) maintains the periodicity of the sin curve.
[0083]
The basis for this estimation is as follows.
As described above, the shift amount si ′ of the edge histogram after subtracting the rotation amount sφ is approximated to a sin curve. However, as shown in FIG. 7, these shift amounts are affected by the depth of the edge and the shape of the space.
[0084]
The shift amount si ′ of the edge histogram deviates from the sin curve due to the influence of the depth of the edge and the shape of the space. However, as shown in FIG. Can be assumed to have negative values.
[0085]
Therefore, the sin curve can be robustly applied using the sign of the shift amount si '.
Here, the rotation angle φ is converted into three values of −1, 0, and 1 of the shift amount si ′, and the centers of the shift amounts si ′ corresponding to the values of −1 and 1 are each set to the negative of the sin curve as much as possible. The moving direction ω is determined so as to correspond to the positive peak.
[0086]
This is the next maximization problem.
[0087]
(Equation 9)

[0088]
(Equation 10)

Here, by setting sφ and sφ ± 1 corresponding to the rotation angle φ to 0, the shift amount si ′ of the edge histogram can reduce the influence of noise on the code.
[0089]
Assuming that the differential on the left side of equation (9) is 0, the following equation is obtained.
[0090]
(Equation 11)

The moving direction ω is directly obtained from the above equation (11) as follows.
[0091]
(Equation 12)

Equation (11) includes not only maximization of equation (9) but also minimization. Since the cycle of tan (θ) is π, it can be seen that either the moving direction ω or ω + π maximizes the expression (9).
[0092]
As described above, in S150 of the present embodiment, the movement direction ω is calculated based on Expression (12). In S150, the rotation angle φ and the movement direction ω of the active sensor 11 with respect to the point before the movement can be estimated as described above.
[0093]
1.6. S160 (estimation of moving distance of active sensor 11)
In S160, the movement distance of the active sensor 11 is estimated. Specifically, the travel distance is estimated according to the flowchart in FIG.
[0094]
In S162, a three-dimensional point group of edges based on the omnidirectional distance data of the environment K obtained by the active sensor 11 before the movement is mapped onto a floor plane (xy plane), and an edge histogram ( Floor plane projection histogram). Due to this mapping, many three-dimensional points on a vertical plane such as a wall of the environment K (vertical plane with respect to the floor plane) are projected to the same place, and the value of the histogram becomes high where the vertical plane exists. In FIG. 10, a portion where the value of the histogram is large is shown in white.
[0095]
In S164, the direction of the main vertical plane (such as a wall) is detected using the Hough transform on the floor plane projection histogram.
(Hough transform)
The Hough transform will be described. FIG. 14 is an explanatory diagram of the Hough transform. In the figure, it is assumed that points D1 to D6 are distributed on a straight line R on the XY plane. Assuming that the angle between the origin and the perpendicular from the origin O to the straight line R is H, OH = ρ, OH and the X axis is θ, and the coordinate of H is (x, y), the straight line R is expressed by the following polar coordinate formula. it can.
[0096]
ρ = xcosθ + ysinθ
Here, when ρ and θ are variables, all the straight line groups passing through a certain point (x, y) can be expressed by the polar coordinate expression.
[0097]
FIG. 15 is a diagram showing a group of straight lines passing a certain point on the ρ-θ plane. On the ρ-θ plane, a group of straight lines passing through a certain point can be expressed as a single curve, and if ρ and θ are uniquely determined, one straight line can be specified.
[0098]
FIG. 16 is a diagram illustrating a group of straight lines for each of a plurality of candidate points on the ρ-θ plane. FIG. 16 shows an example in which a curve is obtained as shown in FIG. 15 for each candidate point shown in FIG. 14 according to the polar coordinate equation. Then, as shown in FIG. 16, although a plurality of curves are drawn, the most appropriate straight line for all the candidate points is the point Q (ρ ₀ , Θ ₀ ) Specified by ρ = xcos θ ₀ + Ysinθ ₀ It is.
[0099]
In the present embodiment, for example, when the Hough transform is used for the floor plane projection histogram of FIG. 10, a large number of curves are drawn on the ρ-θ plane as shown in FIG. In FIG. 11, the curve is represented by a white line. In FIG. 11, an intersection having a large tolerance between the curves is detected as a straight line on a main vertical plane (such as a wall), that is, the direction of the vertical plane is detected.
[0100]
In the present embodiment, the ρ-θ plane of the Hough transform in FIG. 11 is the Hough voting space. From this Hough voting space, by detecting an intersection where θ is separated by a predetermined angle (for example, 60 degrees) or more, as shown in FIG. 10, a main vertical plane such as (A1, A2) and (B1, B2) is obtained. The direction of is detected.
[0101]
Then, of these main vertical plane directions, the direction of one vertical plane closest to the moving direction ω is selected as the dominant projection direction v, and the floor plane projection histogram of FIG. The projection is performed along the projection direction v) to generate a one-dimensional histogram.
[0102]
FIG. 12 shows a histogram when A1 and A2 are parallel to each other, and a direction projected along a direction of a vertical plane which is the parallel direction is set as a dominant projection direction, and A1 and A2 show the histogram. The part shown indicates that the value of the histogram is large. FIG. 13 shows a histogram when B1 and B2 are parallel to each other, and the direction projected along the direction of the vertical plane, which is the parallel direction, is set as the dominant projection direction. Indicates that the value of the histogram is large.
[0103]
In S166, the three-dimensional point group of the edge based on the omnidirectional distance data in the environment K obtained by the active sensor 11 after the movement is centered on the Z axis (the axis perpendicular to the floor plane on which the active sensor 11 travels). The rotation is performed at “−φ” based on the rotation angle φ estimated at S150. That is,
P '= Rz (-φ) P
Here, Rz (−φ) is a rotation matrix. P and P ′ represent three-dimensional points before and after rotation.
[0104]
In S168, assuming that the histogram projection directions of the main vertical plane obtained in S164 are v1 and v2 (see FIG. 10), of these histogram projection directions v1 and v2, the direction closest to the moving direction ω (for example, v1 ), P ′ is projected in the projection direction, and a histogram H ′ (v1) is generated.
[0105]
Further, before the active sensor 11 travels, a three-dimensional point of an edge based on distance data in all directions of the environment K obtained by the active sensor 11 is defined as Po. Then, assuming that the histogram projection directions of the main vertical plane obtained in S164 are v1 and v2 (see FIG. 10), of these histogram projection directions v1 and v2, the direction closest to the moving direction ω (for example, v1) Is selected, and Po is projected in the projection direction to generate a histogram H (v1).
[0106]
In S170, the histograms H (v1) and H ′ (v1) in the v1 direction (ie, the direction closest to the moving direction ω) obtained before and after the movement are matched, and a shift amount λ between the two histograms is obtained. . The shift amount λ is related to the moving distance along which the active sensor 11 travels along the histogram projection direction v1.
[0107]
In S180, the moving distance L of the active sensor 11 moved along the moving direction ω is calculated based on the shift amount λ using the following equation (13).
[0108]
(Equation 13)

2. S20 (Calculation of predicted distance image)
Here, based on the omnidirectional distance data acquired by the active sensor 11 before the movement and the relative movement amount (including the rotation angle φ, the movement direction ω, and the movement distance L) acquired in S160, the active sensor 11 is moved after the movement. The omnidirectional range image obtained from the viewpoint is estimated. Hereinafter, the predicted omnidirectional distance image after the movement is simply referred to as a predicted distance image.
[0109]
In the present embodiment, the omnidirectional distance image is generated by expressing the distance image obtained by each stereo unit 12 of the active sensor 11 in a cylindrical coordinate system using the arrangement parameters of each stereo unit 12 and integrating them. .
[0110]
FIG. 26 shows an example of the generated omnidirectional distance image, in which the horizontal axis is the azimuth angle θ and the vertical axis is the elevation angle γ. FIG. 17 is an explanatory diagram of the elevation angle γ, in which the upper part from the viewpoint center of the active sensor 11 is +, and the lower part is −. FIG. 26 covers a range of 0 <θ <2π and a range of −π / 3 <γ <π / 3.
[0111]
Here, the distance value d in the azimuth angle θ and the elevation angle γ direction of the omnidirectional distance image before the movement _s (Θ, γ) is a distance value d at a position in the azimuth angle θ ′ and the elevation angle γ ′ direction according to the relative movement amount after movement (including the rotation angle φ, the movement direction ω, and the movement distance L). _p (Θ ′, γ ′). Here, the distance value d _s (Θ, γ) is a value from the point S to an arbitrary point W in the distance image when the active sensor 11 is located at the point S, as shown in FIG. Distance value d _p (Θ ′, γ ′) is a distance value from the point P to the point W in the distance image after the active sensor 11 has moved from the point S to the point P.
[0112]
Azimuth angle θ ', elevation angle γ' and distance value d _p (Θ ′, γ ′) is obtained by the following equation (14) using the rotation angle φ, the movement direction ω, and the movement distance L of the active sensor 11.
[0113]
[Equation 14]

From the above equation (14), an omnidirectional distance image obtained from the viewpoint after the movement of the active sensor 11, that is, a predicted distance image is estimated.
[0114]
3. S30 (estimation of occlusion area)
In S30, the active sensor 11, that is, the occlusion region generated by the movement of the observation system is estimated. Occlusion is deeply related to the moving object in the environment K and the amount of movement of the camera of the active sensor 11. That is, occlusion occurs in a portion where the depth (distance value) changes rapidly.
[0115]
For this reason, attention is paid to a jump edge in the distance image obtained from the active sensor 11, and an occlusion region occurring in the predicted distance image is estimated based on the distance image and the movement amount.
[0116]
Here, the pixel (pixel) of the jump edge in the omnidirectional distance image acquired before the movement is (θ ₁ , Γ ₁ , D _s (Θ ₁ , Γ ₁ )). Then, the pixel (θ ₁ , Γ ₁ , D _s (Θ ₁ , Γ ₁ )) The distance value d in the vicinity of 4 within the predetermined range _s (Θ ₁ , Γ ₁ The pixel having the farthest distance value than () ₂ , Γ ₂ , D _s (Θ ₂ , Γ ₂ )).
[0117]
The pixel of the jump edge corresponds to the jump edge pixel of the present invention, and d near the 4-neighborhood as a predetermined range. _s (Θ ₁ , Γ ₁ The pixel farthest from ()) corresponds to the comparison target pixel. The “farthest distance value” in the present embodiment corresponds to the “predetermined value” in the present invention. FIG. 19 is an explanatory diagram of the occlusion area. In the figure, the part corresponding to the pixel of the jump edge is represented by (θ ₁ , Γ ₁ , D _s (Θ ₁ , Γ ₁ )), And the part corresponding to the pixel farthest from that pixel is (θ ₂ , Γ ₂ , D _s (Θ ₂ , Γ ₂ )). These are converted as follows by the movement of the active sensor 11 according to the above equation (14).
[0118]
(Θ ₁ , Γ ₁ , D _s (Θ ₁ , Γ ₁ )) → (θ ₁ ', Γ ₁ ', D _p (Θ ₁ ', Γ ₁ '))
(Θ ₂ , Γ ₂ , D _s (Θ ₂ , Γ ₂ )) → (θ ₂ ', Γ ₂ ', D _p (Θ ₂ ', Γ ₂ '))
θ ₁ 'Is the estimated azimuth of the jump edge pixel, and θ ₂ 'Corresponds to the estimated azimuth of the pixel to be compared, and γ ₁ 'Estimated elevation angle of jump edge pixel and γ ₂ 'Corresponds to the estimated elevation angle of the comparison target pixel.
[0119]
The occlusion sections Iθ and Iγ in the directions of the azimuth angle θ and the elevation angle γ are estimated from the difference between the positions of the pixels generated by this conversion. Further, the following occlusion area Oc (θ, γ) is estimated from the occlusion sections Iθ, Iγ.
[0120]
(Equation 15)

In the above equation (15), Iθ = (θ ₂ ', Θ ₁ ') Is θ ₁ > Θ ₂ , Θ ₁ '> Θ ₂ ', After moving, the azimuth θ ₂ '~ Θ ₁ 'Means an occlusion section. Note that θ ₁ > Θ ₂ , Θ ₁ '> Θ ₂ The case of 'corresponds to the case where the moving object moves to the right in FIG. 19, for example.
[0121]
Also, Iθ = (θ ₁ ', Θ ₂ ') Is θ ₂ > Θ ₁ , Θ ₂ '> Θ ₁ ', After moving, the azimuth θ ₁ '~ Θ ₂ 'Between means an occlusion section. Note that θ ₂ > Θ ₁ , Θ ₂ '> Θ ₁ 19 corresponds to, for example, the case where the active sensor 11 moves to the left in FIG.
[0122]
In other cases, it means that there is no occlusion section in the azimuth, that is, it is an empty set.
In the above formula, Iγ = (γ ₂ ', Γ ₁ ') Is γ ₁ > Γ ₂ , Γ ₁ '> Γ ₂ ′, The elevation angle γ ₂ '~ Γ ₁ 'Means an occlusion section.
[0123]
Also, Iγ = (γ ₁ ', Γ ₂ ') Is γ ₂ > Γ ₁ , Γ ₂ '> Γ ₁ ′, The elevation angle γ ₁ '~ Γ ₂ 'Means an occlusion section. In other cases, it means that there is no occlusion section at the elevation angle, that is, it is an empty set.
[0124]
Further, in the above equation (15), the elevation angle γ of the determination target pixel in the predicted distance image is determined by the estimated elevation angle γ ₁ 'And O (θ, γ) = 1 when the azimuth θ of the determination target pixel in the predicted distance image is within the occlusion section Iθ. That is, it is determined that the determination target pixel is within the occlusion area.
[0125]
Further, the azimuth θ of the pixel to be determined in the predicted distance image is the estimated azimuth θ ₁ And the angle of elevation γ of the pixel to be determined is within the occlusion section Iγ, O (θ, γ) = 1. That is, it is determined that the determination target pixel is within the occlusion area.
[0126]
Otherwise, O (θ, γ) = 0.
When O (θ, γ) = 1, it means that the pixel at the azimuth angle θ and the elevation angle γ in the predicted distance image is a part of the occlusion area. When O (θ, γ) = 0, it means that the pixel having the azimuth angle θ and the elevation angle γ is not an occlusion area.
[0127]
4. S40 (extraction of moving object region by difference)
In S40, the difference between the predicted distance image obtained in S20 and the omnidirectional distance image actually obtained after the movement of the active sensor 11 is obtained, a difference image is generated, and the occlusion region estimated in S30 is removed. , To extract a moving body region. Specifically, it is performed as follows.
[0128]
Here, the distance values for θ and γ in the omnidirectional distance image acquired after the movement, the predicted distance image, and the generated difference image are denoted by d, respectively. _g (Θ, γ), d _p (Θ, γ) and dδ (θ, γ). Also, the determination value for obtaining the ternary image of the difference image is d. _d (Θ, γ). In S20, the distance value of the predicted distance image is d _p (Θ ′, γ ′), in this column, the azimuth θ ′ and the elevation γ ′ of the predicted distance image are respectively equal to the azimuth θ and the elevation γ of the omnidirectional distance image acquired after the movement. For convenience of explanation, in this column, d _p (Θ, γ).
[0129]
The distance value dδ (θ, γ) of the generated difference image is obtained by the following equation: distance value of difference image = (distance value of predicted distance image) − (distance value of omnidirectional distance image after moving). That is, the distance value dδ (θ, γ) of the generated difference image is
[0130]
(Equation 16)

Is represented by Further, the threshold value Th and the threshold value −Th are used for the distance value dδ (θ, γ) of the difference image to obtain a determination value for obtaining a ternary image of the difference image. The threshold value Th and the threshold value −Th are threshold values for determining whether or not the distance value dδ (θ, γ) of the difference image has changed significantly.
[0131]
[Equation 17]

In the equation (17), when the distance value dδ (θ, γ) of the difference image exceeds the threshold Th and O (θ, γ) = 0, that is, when the pixel is not in the occlusion area, Determination value d for that pixel _d (Θ, γ) is “1” and is a positive value. This pixel (pixel) indicates that it has changed to a position close to the active sensor 11 beyond the threshold value Th.
[0132]
In Expression (17), when the distance value dδ (θ, γ) of the difference image is less than the threshold value −Th and O (θ, γ) = 0, that is, when the pixel is not in the occlusion area, Judgment value d _d (Θ, γ) is “−1” and is a negative value. In this case, it is indicated that the pixel has changed to a position farther from the active sensor 11 than the threshold value -Th.
[0133]
Further, when the above condition is not satisfied, the judgment value d _d (Θ, γ) is set to “0”. That is, in this case, since the distance value dδ (θ, γ) of the difference image is a change from the threshold value −Th to the threshold value Th, the pixel (pixel) does not show a large change.
[0134]
Determination value d obtained as described above _d The difference image is binarized based on (θ, γ).
(Explanation in specific example)
1. Example 1
The setting of the determination value of Expression (17) will be described with reference to a specific example 1 shown in FIG. The specific example 1 is an example in which the active sensor 11 and the moving object 20 move in different directions that are not on the same line.
[0135]
FIG. 20 shows the positions of the active sensor 11 before and after the movement and the positions of the moving object 20 before and after the movement, respectively. For convenience of explanation, the following description will be made on the assumption that the active sensor 11 has a pixel at an elevation angle γ = 0. Also, the threshold Th is set to 0.
[0136]
In the figure, the omnidirectional distance image actually acquired after the moving object 20 and the active sensor 11 are moved is an arc portion of the moving object 20 (after the movement) between a6 and a7, a8 to a9 to a3 to a4 to a5. Environment K wall. Note that each a is related to the azimuth angle θ, and hereinafter, for convenience of description, for example, when pointing to a8, it is referred to as the azimuth angle a8 or the azimuth angle of a8.
[0137]
The predicted distance image calculated in S20 includes the arc portion of the moving object 20 (before movement) of a1 and a2 and the wall of the environment K of a3 to a4 to a5 to a8 to a9. The occlusion region estimated in S30 is a wall portion of the environment K of a3 to a4.
[0138]
(Azimuth angles of a1 and a2)
Now, in the range of azimuth angles a1 to a2, the distance value d of the predicted distance image _p (Θ, γ) is for the arc portion of the moving object 20 (before moving), while the distance value d of the omnidirectional distance image after moving is _g (Θ, γ) is for the wall portion of the environment K, and is the distance value d of the omnidirectional distance image after moving. _g (Θ, γ) is larger. Therefore, when Expression (16) is calculated, the distance value dδ (θ, γ) <0 (= −Th) of the generated difference image is obtained. Also, the azimuths a1 and a2 are not in the occlusion region, that is, O (θ, γ) = 0. As a result, in the range of the azimuths a1 to a2, the calculation result of Expression (17) is “−1”, that is, negative. In FIG. 20, the calculation result of Expression (17) is shown in parentheses.
[0139]
(Azimuth angle of a3 to a4)
Since the range of the azimuth angles a3 to a4 is the occlusion region, O (θ, γ) = 1. As a result, in the range of the azimuth angles a3 to a4, the calculation result of Expression (17) is “0 ".
[0140]
(Azimuth angle of a4 to a5)
In the range of azimuth angles a4 to a5, the distance value d of the predicted distance image _p (Θ, γ) is for the wall of the environment K. On the other hand, the distance value d of the omnidirectional distance image after the movement _g (Θ, γ) also belongs to the wall portion of the same environment K. Therefore, when Expression (16) is calculated, the distance value dδ (θ, γ) of the generated difference image is equal to zero. As a result, in the range of the azimuth angles a4 to a5, the calculation result of Expression (17) is “0”.
[0141]
(Azimuth angle of a6 to a7)
In the range of azimuth angles a6 to a7, the distance value d of the predicted distance image _p (Θ, γ) is for the wall portion of the environment K, while the distance value d of the omnidirectional distance image after the movement is _g (Θ, γ) is for the arc portion of the moving object 20 (after movement). Therefore, when Expression (16) is calculated, the distance value dδ (θ, γ)> 0 (= Th) of the generated difference image is obtained. Further, the azimuth angles a6 to a7 are not in the occlusion region, that is, O (θ, γ) = 0. As a result, in the range of the azimuth angles a6 to a7, the calculation result of Expression (17) is “1”, that is, positive.
[0142]
(Azimuth angle of a8 to a9)
In the range of azimuth angles a8 to a9, the distance value d of the predicted distance image _p (Θ, γ) is for the wall of the environment K. On the other hand, the distance value d of the omnidirectional distance image after the movement _g (Θ, γ) also belongs to the wall portion of the same environment K. Therefore, when Expression (16) is calculated, the distance value dδ (θ, γ) of the generated difference image is equal to zero. As a result, in the range of the azimuths a8 to a9, the calculation result of Expression (17) is “0”.
[0143]
As described above, in the difference image, an area that has changed to a position closer to the active sensor 11 is a “positive” area, while an area that has changed to a far position appears as a “negative” area. In both areas, the moving object 20 is extracted from the distance image obtained after the movement. That is, the “positive” region is extracted after the moving object 20 is moved, and the “negative” region is extracted before the moving object 20 is moved.
[0144]
If the occlusion area is not considered, the occlusion area (the azimuth angle of a3 to a4 in the above example) that appears after the moving object 20 is moved is also detected as the moving object. However, in S30, the occlusion region is estimated, and the estimated occlusion region is excluded and distinguished by Expression (17), so that only the moving object region can be detected.
[0145]
2. Example 2
Next, the setting of the determination value of Expression (17) will be described with reference to a specific example 2 shown in FIGS. The specific example 2 is an example in which the active sensor 11 and the moving object 20 move closer to each other on the same line.
[0146]
In this example, in S30, the occlusion area is estimated as follows. In FIG. 22, the portion corresponding to the pixel of the jump edge is (θ ₁ , Γ ₁ , D _s (Θ ₁ , Γ ₁ )), And the part corresponding to the pixel farthest from that pixel is (θ ₂ , Γ ₂ , D _s (Θ ₂ , Γ ₂ )).
[0147]
In this case, by the movement of the active sensor 11 according to Expression (14),
(Θ ₁ , Γ ₁ , D _s (Θ ₁ , Γ ₁ )) → (θ ₁ ', Γ ₁ ', D _p (Θ ₁ ', Γ ₁ '))
(Θ ₂ , Γ ₂ , D _s (Θ ₂ , Γ ₂ )) → (θ ₂ ', Γ ₂ ', D _p (Θ ₂ ', Γ ₂ '))
Has been converted as follows. Note that the elevation angle γ is set to 0 for convenience of explanation.
[0148]
Then, in this case, as shown in FIG. ₁ > Θ ₂ And θ2 ′> θ1 ′, the expression (15) indicates that the occlusion section Iθ is an empty set, and that the occlusion area Oc (θ, γ) = 0, that is, there is no occlusion area. In FIG. 22, the jump edge is the right part of the moving object 20, but the jump edge also exists on the left side. However, for the same reason described above, this side is also assumed to have no occlusion area.
[0149]
Next, the processing in S40 will be described.
FIG. 21 shows the positions of the active sensor 11 before and after the movement and the positions of the moving object 20 before and after the movement, respectively. For convenience of explanation, the following description will be made on the assumption that the active sensor 11 has a pixel at an elevation angle γ = 0. Also, the threshold Th is set to 0. In the figure, the omnidirectional distance image actually acquired after the moving object 20 and the active sensor 11 have moved is an arc portion of the moving object 20 (after movement) between a1 to a2 and a3 to a4, and between the a4 and a1. Includes environment K wall.
[0150]
The predicted distance image calculated in S20 includes the wall of the environment K between a1 and a2, the arc portion of the moving object 20 (before moving) between a2 and a3, and the wall of the environment K of a3 to a4 to a1. I have.
[0151]
(Azimuth angles of a1 and a2)
Now, in the range of azimuth angles a1 to a2, the distance value d of the predicted distance image _p (Θ, γ) is for the wall portion of the environment K, while the distance value d of the omnidirectional distance image after the movement is _g (Θ, γ) is for the arc portion of the moving object 20 (after movement). Therefore, the distance value d of the predicted distance image _p (Θ, γ) is the distance value d of the omnidirectional distance image after the movement. _g (Θ, γ). Therefore, when Expression (16) is calculated, the distance value dδ (θ, γ)> 0 (= Th) of the generated difference image is obtained. Also, the azimuths a1 and a2 are not in the occlusion region, that is, O (θ, γ) = 0. As a result, in the range of the azimuths a1 to a2, the calculation result of Expression (17) is “1”, that is, positive. In FIG. 21, the calculation result of Expression (17) is shown in parentheses.
[0152]
(Azimuth angle of a2 to a3)
In the range of azimuth angles a2 to a3, the distance value d of the predicted distance image _p (Θ, γ) is for the arc portion of the moving object 20 before moving. On the other hand, the distance value d of the omnidirectional distance image after the movement _g (Θ, γ) is for the arc portion of the moving object 20 after moving close to the active sensor 11. Therefore, the distance value d _p (Θ, γ)> distance value d _g (Θ, γ), and when Expression (16) is calculated, the distance value dδ (θ, γ)> 0 (= Th) of the generated difference image is obtained. The azimuths a2 to a3 are not O (θ, γ) = 0, that is, not in the occlusion region. As a result, in the range of the azimuths a2 to a3, the calculation result of Expression (17) is “1”, that is, positive.
[0153]
(Azimuth angle of a3 to a4)
In the range of azimuth angles a3 to a4, the calculation result of Expression (17) is "1", that is, positive, for the same reason as for a1 to a2.
[0154]
(Azimuth angle of a4 to a1)
In the range of azimuth angles a4 to a1, the distance value d of the predicted distance image _p (Θ, γ) and the distance value d of the omnidirectional distance image after moving _g (Θ, γ) are for the wall portion of the same environment K. Therefore, when Expression (16) is calculated, the distance value dδ (θ, γ) of the generated difference image is equal to zero. As a result, in the range of the azimuth angles a4 to a1, the calculation result of Expression (17) is “0”.
[0155]
As described above, in the difference image, the area that has changed to a position close to the active sensor 11 is the “positive” area, and the moving object 20 has been extracted from the distance image obtained after the movement. In other words, the “positive” region is extracted after the movement of the moving object 20.
[0156]
3. Example 3
Next, the setting of the determination value of Expression (17) will be described with reference to a specific example 3 shown in FIG. Example 3 is an example in which the active sensor 11 and the moving object 20 move in the same direction on the same line. In this case, for the same reason as in the specific example 2, it is determined that there is no occlusion area in S30.
[0157]
The process in S40 will be described.
FIG. 23 shows the positions of the active sensor 11 before and after the movement, and the positions of the moving object 20 before and after the movement, respectively. For convenience of explanation, the following description will be made on the assumption that the active sensor 11 has a pixel at an elevation angle γ = 0. Also, the threshold Th is set to 0. In the figure, the omnidirectional distance image actually acquired after the moving object 20 and the active sensor 11 have moved is the wall of the environment K between a1 and a2, and the arc portion of the moving object 20 (after moving) between a2 and a3. , A3 to a4 to a1.
[0158]
The predicted distance image calculated in S20 includes an arc portion of the moving object 20 (before moving) between a1 and a2, an arc portion of the moving object 20 (before moving) between a2 and a3, and a moving object between a3 and a4. 20 (before movement), including the wall of the environment K of a4 to a1.
[0159]
(Azimuth angles of a1 and a2)
In the range of azimuth angles a1 to a2, the distance value d of the predicted distance image _p (Θ, γ) is for the arc portion of the moving object 20 (before moving), while the distance value d of the omnidirectional distance image after moving is _g (Θ, γ) is for the wall of environment K. Therefore, the distance value d of the predicted distance image _p (Θ, γ) is the distance value d of the omnidirectional distance image after the movement. _g (Θ, γ). Therefore, when Expression (16) is calculated, the distance value dδ (θ, γ) <0 (= −Th) of the generated difference image is obtained. Also, the azimuths a1 and a2 are not in the occlusion region, that is, O (θ, γ) = 0. As a result, in the range of the azimuths a1 to a2, the calculation result of Expression (17) is “−1”, that is, negative. In FIG. 23, the calculation result of Expression (17) is shown in parentheses.
[0160]
(Azimuth angle of a2 to a3)
In the range of azimuth angles a2 to a3, the distance value d of the predicted distance image _p (Θ, γ) is for the arc portion of the moving object 20 before moving. On the other hand, the distance value d of the omnidirectional distance image after the movement _g (Θ, γ) is for the arc portion of the moving object 20 after moving away from the active sensor 11. Therefore, the distance value d _p (Θ, γ) <distance value d _g (Θ, γ), and when Expression (16) is calculated, the distance value dδ (θ, γ) <0 (= −Th) of the generated difference image is obtained. The azimuths a2 to a3 are not O (θ, γ) = 0, that is, not in the occlusion region.
[0161]
As a result, in the range of the azimuths a2 to a3, the calculation result of Expression (17) is “−1”, that is, negative.
(Azimuth angle of a3 to a4)
In the range of azimuth angles a3 to a4, the calculation result of Expression (17) is “−1”, that is, negative for the same reason as for a1 to a2.
[0162]
(Azimuth angle of a4 to a1)
In the range of azimuth angles a4 to a1, the distance value d of the predicted distance image _p (Θ, γ) and the distance value d of the omnidirectional distance image after moving _g (Θ, γ) are for the wall portion of the same environment K. Therefore, when Expression (16) is calculated, the distance value dδ (θ, γ) of the generated difference image is equal to zero. As a result, in the range of the azimuth angles a4 to a1, the calculation result of Expression (17) is “0”.
[0163]
As described above, in the difference image, the area that has changed to a position far from the active sensor 11 is a “negative” area, and the moving object 20 has been extracted from the distance image obtained after the movement. That is, the “negative” region is extracted before the moving object 20 moves.
[0164]
When the active sensor 11 and the moving object 20 move on the same straight line, in addition to the specific examples 2 and 3, the active sensor 11 and the moving body 20 move in directions away from each other, or in an opposite direction to the specific example 180 by 180 degrees. The sensor 11 and the moving object 20 may move. In these cases, similarly, the region of the moving object 20 becomes “positive” or “negative” and is extracted.
[0165]
Now, the description of the specific example ends, and the description returns to the flowchart.
In S40, after performing the processing as described above, noise removal processing is performed on the ternary image, and the processing in S40 ends. The noise removal process is a process of removing, for example, a pixel whose number of pixels (the number of pixels) is equal to or less than a predetermined threshold as noise.
[0166]
According to the present embodiment, the following effects are obtained.
(1) In the moving object detecting device of the active sensor 11 of the present embodiment, the computer 16 (relative moving amount estimating means) includes a relative direction including a moving direction, a moving distance, and a rotating amount before and after the movement of the active sensor 11. The amount of movement has been estimated. or. The computer 16 (predicted distance image generating means) estimates and generates a predicted distance image after the movement based on the omnidirectional distance image acquired before the movement and the estimated relative movement amount. Further, the computer 16 (occlusion area estimating means) estimates the occlusion area generated in the predicted distance image when the active sensor 11 moves. Then, the computer 16 (moving object region extracting means) removes the occlusion region when generating a difference image between the predicted distance image and the omnidirectional distance image acquired by the active sensor 11 after moving, and removes the moving object region. Was extracted.
[0167]
Further, the moving object detection program of the active sensor according to the present embodiment causes the computer 16 to function as each of the above-described units.
In the moving object detection method of the active sensor 11, as a first step, a relative movement amount including a movement direction, a movement distance, and a rotation amount before and after the movement of the active sensor 11 is estimated. As a second step, a predicted distance image after the movement is estimated and generated based on the omnidirectional distance image acquired before the movement and the relative movement amount estimated in the first step. Further, as a third step, an occlusion area generated in the predicted distance image due to the movement of the active sensor 11 is estimated. As a fourth step, when generating a difference image between the predicted distance image and the omnidirectional distance image acquired by the active sensor 11 after moving, the occlusion region is removed and a moving object region is extracted. .
[0168]
As a result, the "appearance change" of the environment other than the moving object caused by the movement of the observation system can be easily distinguished from the environmental change caused by the moving object such as a person, and the detection of the moving object can be easily performed, and the occlusion area is estimated. By doing so, there is an effect that it is possible to more accurately detect only environmental changes due to the moving body.
[0169]
(2) In the present embodiment, the computer 16 is the farthest from the active sensor 11 among the jump edge pixels of the omnidirectional distance image before the movement and the pixels located in the 4-neighborhood (within a predetermined range) of the pixels. For the comparison target pixel having the distance value, the estimated azimuth angle and the estimated elevation angle after the movement were obtained based on the relative movement amount. Then, the computer 16 estimates the section of the estimated azimuth angle and the range of the estimated elevation angle of each of the jump edge pixel and the comparison target pixel as the occlusion section of the occlusion area in the predicted distance image.
[0170]
Further, the moving object detection program of the active sensor according to the present embodiment causes the computer 16 to function as the above means.
In the present embodiment, as a third step, among the jump edge pixels of the omnidirectional distance image before the movement and the pixels located in the vicinity of 4- (within a predetermined range) of the pixel, the farthest from the active sensor 11. For the comparison target pixel having the distance value, the estimated azimuth angle and the estimated elevation angle after the movement were obtained based on the relative movement amount. Then, in the third step, the section of the estimated azimuth angle and the range of the estimated elevation angle of each of the jump edge pixel and the comparison target pixel are estimated to be the occlusion section of the occlusion area in the predicted distance image.
[0171]
As a result, an occlusion section for estimating the occlusion area can be easily determined.
(3) In the present embodiment, as the occlusion area estimating means, the computer 16 determines that the elevation angle of the determination target pixel matches the estimated elevation angle of the jump edge pixel, and that the azimuth angle of the determination target pixel is determined by the estimated azimuth angle. When it is within the occlusion section, the determination target pixel is determined to be within the occlusion area.
[0172]
Further, the moving object detection program of the active sensor according to the present embodiment causes the computer 16 to function as the above means.
In the present embodiment, as the third step, when the elevation angle of the determination target pixel matches the estimated elevation angle of the jump edge pixel, and the azimuth angle of the determination target pixel is within the occlusion section defined by the estimated azimuth angle. (Condition 1) determines that the determination target pixel is in the occlusion area.
[0173]
As a result, when the condition 1 is satisfied, the occlusion region can be easily estimated.
(4) In addition, the computer 16 determines that the azimuth of the pixel to be determined matches the estimated azimuth of the jump edge pixel and that the elevation of the pixel to be determined is within the occlusion section defined by the estimated elevation. (Condition 2), the determination target pixel is determined to be in the occlusion area.
[0174]
Further, the moving object detection program of the active sensor according to the present embodiment causes the computer 16 to function as the above means.
As a third step, when the azimuth of the determination target pixel matches the estimated azimuth of the jump edge pixel and the elevation angle of the determination target pixel is within the occlusion section defined by the estimated elevation angle (condition 2), In addition, the determination target pixel is determined to be within the occlusion area.
[0175]
As a result, when the condition 2 is satisfied, the occlusion area can be easily estimated.
(5) In the present embodiment, when generating the difference image, the computer 16 performs positive and negative sign determination based on the difference between the distance values of the pixels when generating the difference image, and The moving object region is extracted based on the negative sign determination.
[0176]
Further, the moving object detection program of the active sensor according to the present embodiment causes the computer 16 to function as the above means.
As a fourth step, when generating a difference image, positive and negative sign determination is performed based on the difference between the distance values of the pixels, and moving object region extraction is performed based on the positive and negative sign determination. I do.
[0177]
As a result, by performing the positive and negative sign determination, it is possible to easily perform the moving object region extraction both before and after the movement.
(Experimental example)
FIGS. 24 to 31 show that in the laboratory, the active sensor 11 acquires images before and after the person as a moving object moves toward the wall of the laboratory while moving in the direction in which the person is located. The result of image processing is shown. In each drawing, the horizontal axis represents the azimuth angle θ (0 <θ <360 degrees), and the vertical axis represents the elevation angle γ (−3/3 <γ <π / 3). I have.
[0178]
FIG. 24 shows “before moving” and FIG. 25 shows “after moving”. In both figures, not a distance image but an omnidirectional color image is shown as a monochrome grayscale image for convenience of explanation. . 26 is an omnidirectional distance image corresponding to FIG. 24, and FIG. 27 is a distance image corresponding to FIG.
[0179]
FIG. 28 shows the predicted distance image generated from FIG. 26 using the obtained relative movement amount, and FIG. 29 shows the estimated occlusion region. FIG. 30 shows the difference between the predicted distance image of FIG. 28 and the omnidirectional distance image (FIG. 27) obtained after the movement, and the area where the distance value is “positive” is white, and the area where the distance value is “negative” is gray. Indicated by. Here, the area shown in gray indicates that the active sensor 11 existing before the movement has disappeared after the movement. The area shown in white indicates an area in which the active sensor 11 did not move before, but appeared after the movement. In FIG. 28, a black area appearing along the right outline of the human image corresponds to occlusion area estimation. This region is removed from the estimated occlusion region shown in FIG. 29, and only the region that appears after the movement is shown in FIG.
[0180]
As described above, by considering the occlusion region, the moving object can be detected with high accuracy.
The embodiments of the present invention are not limited to the above-described embodiments, and can be implemented as follows with appropriate modifications without departing from the spirit of the invention.
[0181]
(1) In the above embodiment, as the active sensor 11, "" omnidirectional stereo system (SOS) for real environment sensing ", IEEJ Transactions on Information C. Vol. 121-C, No. 5, pp. 876- 881.2001 ". The present invention is not limited to this, and an omnidirectional distance image obtained from another omnidirectional camera may be input.
[0182]
(2) In the above embodiment, the computer 16 performs the positive and negative sign determination based on the difference between the distance values of the pixels when generating the difference image as the moving object region extracting means. A moving object region was extracted based on the negative sign determination.
[0183]
Instead, when generating the difference image, the computer 16 performs either positive or negative sign determination based on the difference between the distance values of the pixels as the moving object region extracting means, and determines whether the sign is positive or negative. The moving object region extraction may be performed based on one of the negative sign determinations.
[0184]
In this case, the moving object region can be extracted before or after the movement.
(3) In the embodiment, in S30, the pixel (pixel) of the jump edge in the omnidirectional distance image acquired before the movement is set to (θ ₁ , Γ ₁ , D _s (Θ ₁ , Γ ₁ )). Then, the pixel (θ ₁ , Γ ₁ , D _s (Θ ₁ , Γ ₁ )) The distance value d in the vicinity of 4 within the predetermined range _s (Θ ₁ , Γ ₁ The pixel having the farthest distance value than () ₂ , Γ ₂ , D _s (Θ ₂ , Γ ₂ )). The “predetermined value” means the farthest distance value, but is not limited to this.
[0185]
For example, when the moving speed of the moving object and the moving speed of the active sensor are known in advance, and the moving ranges of both are known, the range in which the distance value between the jump edge and the pixel in the vicinity of 4- can be taken. Can be predicted in advance. In this case, a constant may be determined in advance as a predetermined value within the range of the predictable distance value, and a plurality of pixels whose distance value is equal to or greater than the constant may be set as the comparison target pixels.
[0186]
(4) In the embodiment, the predetermined range for the jump edge pixel is 4-neighborhood, but may be 8-neighborhood or 16-neighborhood. (5) In the above embodiment, in Expression (15), the elevation angle γ of the determination target pixel in the predicted distance image is the estimated elevation angle γ ₁ 'And O (θ, γ) = 1 when the azimuth θ of the determination target pixel in the predicted distance image is within the occlusion section Iθ. That is, it was determined that the determination target pixel was within the occlusion area. Further, the azimuth θ of the pixel to be determined in the predicted distance image is the estimated azimuth θ ₁ And the angle of elevation γ of the pixel to be determined is within the occlusion section Iγ, O (θ, γ) = 1. That is, it was determined that the determination target pixel was within the occlusion area. Otherwise, O (θ, γ) = 0.
[0187]
Alternatively, as in the case of the above (3), when there are a plurality of comparison target pixels, a range surrounded by the plurality of comparison target pixels may be determined as the occlusion region.
[0188]
【The invention's effect】
As described in detail above, according to the first to tenth aspects of the present invention, the “apparent change” of the environment other than the moving object caused by the movement of the observation system and the environmental change caused by the moving object such as a person. Can be easily distinguished to facilitate detection of a moving object. Then, by estimating and removing the occlusion region, there is an effect that it is possible to more accurately detect only the environmental change caused by the moving object.
[0189]
According to the eleventh to fifteenth aspects, it is possible to provide a moving object detection program that can easily realize the moving object detection method and the moving object detection device of the active sensor.
[Brief description of the drawings]
FIG. 1 is a block diagram of an electrical configuration of an active sensor 11.
FIG. 2 is a schematic diagram of a mechanical configuration of an active sensor 11;
FIGS. 3A to 3C are flowcharts of a position / posture estimation program executed by a computer 16;
FIG. 4 is an explanatory diagram of a panoramic edge image corresponding to FIG. 5;
FIG. 5 is an explanatory diagram showing an example of an edge histogram.
FIG. 6 is an explanatory diagram for explaining a shift of an edge histogram in a parallel movement of an active sensor.
7A is an explanatory diagram of a cylindrical coordinate system centered on the active sensor 11, FIG. 7B is an explanatory diagram of a rectangular tube coordinate system centered on the active sensor 11, and FIG. FIG. 4 is an explanatory diagram showing a histogram shift and a sin curve when an edge is mapped to FIG.
FIG. 8 is an explanatory diagram of a matching cost matrix C (s, i) of an omnidirectional edge histogram at two points of the active sensor 11;
FIG. 9 is an explanatory diagram of a method of obtaining a rotation amount sφ and a movement direction ω from a total shift amount.
FIG. 10 is an explanatory diagram of a floor plane projection histogram.
FIG. 11 is a diagram illustrating a group of straight lines that retreat to each of a plurality of candidate points on a ρ-θ surface in the embodiment.
FIG. 12 is a histogram of a dominant projection direction.
FIG. 13 is a histogram of a dominant projection direction different from FIG.
FIG. 14 is an explanatory diagram of Hough transform.
FIG. 15 is a diagram showing a group of straight lines passing a certain point on the ρ-θ surface.
FIG. 16 is a diagram showing a group of straight lines that retreat at each of a plurality of candidate points on the ρ-θ surface.
FIG. 17 is an explanatory diagram of an elevation angle γ.
FIG. 18 is an explanatory diagram of a distance value.
FIG. 19 is an explanatory diagram of an occlusion area.
FIG. 20 is an explanatory diagram of setting of a determination value of Expression (17) in Specific Example 1.
FIG. 21 is an explanatory diagram of setting of a determination value of Expression (17) in Specific Example 2.
FIG. 22 is an explanatory diagram of a pixel at a jump edge according to the second embodiment.
FIG. 23 is an explanatory diagram of setting of a determination value of Expression (17) in Specific Example 3.
FIG. 24 is a black-and-white grayscale image “before moving” of the active sensor 11;
FIG. 25 is a black-and-white grayscale image of “after movement” of the active sensor 11;
FIG. 26 is an omnidirectional distance image corresponding to FIG. 24;
FIG. 27 is a distance image corresponding to FIG. 25;
FIG. 28 is a predicted distance image.
FIG. 29 is an image showing an occlusion region generated by estimation.
FIG. 30 is a difference image.
FIG. 31 is an image in which a moving object is detected after moving a moving object.
[Explanation of symbols]
11 Active sensor
12… Stereo unit
16. Computer (relative movement amount estimating means, predicted distance image generating means, occlusion area estimating means, and moving object area extracting means)

Claims (15)

An omnidirectional distance image can be acquired, and in an active sensor moving object detection device that can move in the environment,
Before and after the movement of the active sensor, a moving direction, a moving distance, a relative moving amount estimating means for estimating a relative moving amount including a rotation amount,
Prediction generated by estimating an omnidirectional distance image after moving (hereinafter referred to as a predicted distance image) based on an omnidirectional distance image acquired before moving and a relative moving amount estimated by the relative moving amount estimating means. Distance image generating means;
Occlusion area estimating means for estimating an occlusion area generated in the predicted distance image by the movement of the active sensor,
When generating a difference image between the predicted distance image and the omnidirectional distance image acquired by the active sensor after the movement, removing the occlusion region and providing a moving object region extracting means for extracting a moving object region. A moving object detection device for an active sensor. The occlusion area estimating means includes:
A pixel of a jump edge of an omnidirectional distance image acquired before moving (hereinafter, referred to as a jump edge pixel), and a pixel having a distance value equal to or more than a predetermined value from an active sensor among pixels located within a predetermined range of the pixel. (Hereinafter referred to as a comparison target pixel), an estimated azimuth angle and an estimated elevation angle after the movement are obtained based on the relative movement amount, respectively.
The section of the estimated azimuth angle and the range of the estimated elevation angle of each of the jump edge pixel and the comparison target pixel are estimated to be an occlusion section of the occlusion area in the predicted distance image. Moving object detection device of active sensor. The occlusion area estimating means includes:
When determining whether or not the determination target pixel in the predicted distance image is in the occlusion area, the elevation angle of the determination target pixel matches the estimated elevation angle of the jump edge pixel, and the determination target pixel The moving object detecting apparatus for an active sensor according to claim 2, wherein when the azimuth is within an occlusion section defined by the estimated azimuth, the determination target pixel is determined to be within an occlusion area. . The occlusion area estimating means includes:
When determining whether or not the determination target pixel in the predicted distance image is in the occlusion area, the azimuth of the determination target pixel matches the estimated azimuth of the jump edge pixel, and the determination target The moving object detecting device for an active sensor according to claim 2, wherein when the elevation angle of the pixel is within the occlusion section determined by the estimated elevation angle, the determination target pixel is determined to be within the occlusion area. . The moving object region extracting means,
The active sensor according to any one of claims 1 to 4, wherein when the difference image is generated, a moving object area is extracted based on a difference between distance values of the pixels. Moving object detection device. An omnidirectional distance image can be acquired, and a moving object detection method of an active sensor that can move in the environment,
A first step of estimating a relative movement amount including a movement direction, a movement distance, and a rotation amount before and after the movement of the active sensor;
A second step of estimating and generating an omnidirectional distance image after movement (hereinafter referred to as a predicted distance image) based on the omnidirectional distance image acquired before the movement and the relative movement amount estimated in the first step; When,
A third step of estimating an occlusion area generated in the predicted distance image by moving the active sensor;
When generating a difference image between the predicted distance image and the omnidirectional distance image acquired by the active sensor after the movement, a fourth step of removing the occlusion region and extracting a moving object region is included. A method for detecting a moving object by a sensor. The third step is
A pixel of a jump edge of an omnidirectional distance image acquired before moving (hereinafter, referred to as a jump edge pixel), and a pixel having a distance value equal to or more than a predetermined value from an active sensor among pixels located within a predetermined range of the pixel. (Hereinafter referred to as a comparison target pixel), an estimated azimuth angle and an estimated elevation angle after the movement are obtained based on the relative movement amount, respectively.
The method according to claim 1, further comprising: estimating a section of the estimated azimuth angle of the jump edge pixel and the comparison target pixel and a range of the estimated elevation angle to be an occlusion section of the occlusion area in the predicted distance image. 7. The moving object detecting method of the active sensor according to 6. The third step is
When determining whether or not the determination target pixel in the predicted distance image is in the occlusion area, the elevation angle of the determination target pixel matches the estimated elevation angle of the jump edge pixel, and the determination target pixel The method according to claim 7, wherein when the azimuth is within the occlusion section defined by the estimated azimuth, the determination target pixel is determined to be within an occlusion area. . The third step is
When determining whether or not the determination target pixel in the predicted distance image is in the occlusion region, the azimuth of the determination target pixel matches the estimated azimuth of the jump edge pixel, and the determination target The method according to claim 7, wherein when the elevation angle of the pixel is within the occlusion section defined by the estimated elevation angle, the determination target pixel is determined to be within the occlusion area. . The fourth step is
The active sensor according to any one of claims 6 to 9, wherein when the difference image is generated, a moving object region is extracted based on a difference between distance values of each pixel. Animal detection method. Computer
A relative movement amount estimating means for estimating a relative movement amount including a movement direction, a movement distance, and a rotation amount before and after movement of the active sensor,
Prediction generated by estimating an omnidirectional distance image after moving (hereinafter referred to as a predicted distance image) based on an omnidirectional distance image acquired before moving and a relative moving amount estimated by the relative moving amount estimating means. Distance image generating means;
Occlusion area estimating means for estimating an occlusion area generated in the predicted distance image by the movement of the active sensor,
When generating a difference image between the predicted distance image and the omnidirectional distance image obtained by the active sensor after the movement, the occlusion region is removed, and the function as a moving object region extracting means for extracting a moving object region is performed. A moving object detection program for active sensors. In claim 11,
In addition, the computer
As the occlusion area estimating means,
A pixel of a jump edge of an omnidirectional distance image acquired before moving (hereinafter, referred to as a jump edge pixel), and a pixel having a distance value equal to or more than a predetermined value from an active sensor among pixels located within a predetermined range of the pixel. (Hereinafter referred to as a comparison target pixel), an estimated azimuth angle and an estimated elevation angle after the movement are obtained based on the relative movement amount, respectively.
An active section characterized in that the section of the estimated azimuth angle and the range of the estimated elevation angle of each of the jump edge pixel and the comparison target pixel are estimated to be an occlusion section of the occlusion area in the predicted distance image. A moving object detection program for the sensor. In claim 12,
In addition, the computer
As the occlusion area estimating means,
When determining whether or not the determination target pixel in the predicted distance image is in the occlusion area, the elevation angle of the determination target pixel matches the estimated elevation angle of the jump edge pixel, and the determination target pixel When the azimuth is within the occlusion section defined by the estimated azimuth, the determination target pixel functions to determine that the pixel is within the occlusion area. In claim 12,
In addition, the computer
As the occlusion area estimating means,
When determining whether or not the determination target pixel in the predicted distance image is in the occlusion region, the azimuth of the determination target pixel matches the estimated azimuth of the jump edge pixel, and the determination target A moving object detection program for an active sensor, wherein the determination target pixel functions to determine that the pixel is within an occlusion area when an elevation angle of the pixel is within an occlusion section determined by the estimated elevation angle. In any one of claims 11 to 14,
Further, a computer as the moving object region extracting means,
A moving object detection program for an active sensor, wherein a moving object region is extracted based on a difference between distance values of pixels when generating the difference image.

Images