JP4588914B2 - Object position measuring device - Google Patents

Description

【００４７】
を物体外形の見え方モデルＭＬⁱ _j （１≦ｊ≦Ｎｍ；Ｎｍは用意する物体形状モデルの種別数）とする。この物体外形の見え方モデルＭＬⁱ _j の大きさで距離画像内を走査していき、以下に述べる判断条件（１）を満たす場合には、物体外形全体が求まったと見なす。このとき、ｋ（ｉ）番目の物体候補Ｏ^ij _k(i)を生成して、Ｏ^ij _k(i)に対してその走査位置を記録するとともに、全体外形物体検出処理部５へ移行するフラグを立てる。このｋ（ｉ）とは、ｉ番目の物体断片Ｐ_i に対して検出された物体候補に付けられる通し番号で、候補毎に異なる番号が付され、複数の候補が検出されても良いことを意味する。
【００４８】
また、後述する判断条件（２）を満たす場合には、物体外形の一部が求まったと見なし、ｋ（ｉ）番目の物体候補Ｏ^ij _k(i)を生成して、その走査位置を記録するとともに、部分外形物体検出処理部６へ移行するフラグを立てる。
【００４９】
条件（１）も（２）も満足しない場合は、物体は存在しないと見なし、走査作業を続行する。走査範囲は、図５に示すように、物体断片Ｐ_i と物体の見え方モデルＭＬⁱ _j が重なる最左端から最右端までの範囲、すなわち、物体断片Ｐ_i の右側と見え方モデルＭＬⁱ _j の右側が一致する位置から、物体断片Ｐ_i の左側と見え方モデルＭＬⁱ _j の左側が一致する位置までの範囲とする。
【００５０】
判断条件（１）：（外形全体が得られているかの検査）
ここでは、ステレオカメラが並んだ方向を水平軸と考えたとき（実際に水平である必要は無い）、左側に位置するカメラ（画像）を左、右側に位置するカメラ（画像）を右とする。このとき、ステレオの性質上、物体の縦成分しか距離が求まらない。従って、物体外形として重要なのは、左右端の縦の外形である。そこで、この左右端の外形の有無を外形全体が得られているか否かの判断基準とする。
【００５１】
物体外形の見え方モデルＭＬⁱ _j を距離画像内に設置し、図６に示すような調査領域（Ｒ１〜Ｒ３）を見え方モデルＭＬⁱ _j の位置を基準に設定する。ここで、Ｒ１は外形の左端側の存在を検査するための領域、Ｒ２は右端側の外形の存在を検査する領域、Ｒ３は物体の内部か外部かを判定する領域である。領域の大きさは次のように定める。Ｒ１、Ｒ２については、高さは見え方モデルＭＬⁱ _j の高さＭＨⁱ _j と同じで、幅Ｒ_w は見え方モデルＭＬⁱ _j の幅ＭＷⁱ _j を基準にある関数Ｆ（ＭＷⁱ _j ）で与えるものとし、例えばＦ（Ｘ）＝Ｘ／４などである。Ｒ１とＲ２を設定する位置は図６に示したように、Ｒ１やＲ２の幅の１／２が見え方モデルと重なるようにする。また、Ｒ３は見え方モデル内でＲ１、Ｒ２でない部分とする。従って、領域Ｒ３の幅はＭＷⁱ _j −Ｆ（ＭＷⁱ _j ）で、高さはＭＨⁱ _j である。
【００５２】
外形の検出領域Ｒ１、Ｒ２について、以下に示す条件の両者を満足する時に、物体の外形が全て見つかったと判断する。
【００５３】
（左端側の検査領域Ｒ１内にある距離を有すエッジ点の数）≧閾値Ｔｈ₁
（右端側の検査領域Ｒ２内にある距離を有すエッジ点の数）≧閾値Ｔｈ₁
ここで、閾値Ｔｈ₁ は、領域Ｒ１（またはＲ２）に存在すべき距離を有すエッジ点の数の最低値を定めたノイズによる影響を抑制するための閾値である。これ以上の数のエッジ点がある時に安定して距離を有すエッジ点の塊が存在する、すなわち距離を有す外形が存在すると見なす。
【００５４】
判断条件（２）：（外形の一部分が得られているか、すなわち一部が欠けているかの検査）
判断条件（１）と同様に、物体外形の見え方モデルＭＬⁱ _j をステレオ画像のうちの一方の距離画像内に設置し、図６で示した調査領域（Ｒ１、Ｒ２）について、以下のいずれかの条件を満足するか調べる。
【００５５】
【数２】

【００５６】
ここで、「距離が求まっていないエッジ点」とは、物体と背景との間の明るさの変化が少ないなどの理由によりエッジ点の位置がずれてしまい、ステレオ処理で左右エッジ画像間で同一箇所を定められず、距離が求まっていないエッジ点を指す。このいずれかの条件を満たすならば、一方の端では３次元位置を定められる距離情報を持つ物体外形があり、他方端では、距離は分からないが、塊として物体外形を形成し得る特徴（痕跡）が存在することを表す。ます、距離が求まっていないエッジ点の数に対する閾値Ｔｈ₂ は、閾値Ｔｈ₁ に対して
【００５７】
【数３】

【００５８】
なる関係を持つ値とする。ここで式中のＴは、全エッジ点に対する平均対応率であり、あらかじめ、例えば実験により求めておくものであり、全エッジ数のうち、平均的に距離が求まるエッジ点の数の割合を表す。通常は、全てのエッジ点に対して距離が求まらないので、１以下の数値となる。この関数Ｔを用いることで、距離が求まっているエッジ点の数と、距離が求まっていないエッジ点の数と影響がほぼ等しくなるように調整している。
【００５９】
もし、式２の条件のいずれかを満足するならば、物体の外形の一部が見つかった（一部が欠けている）可能性があると判断して、下記の次処理で説明する、他方画像で同様に外形の有無を調査する。この調査の目的は、互いに異なる物体の一部分ずつを組合せて、一部の外形が欠けた物体であるとする誤認識を防ぐためであり、ステレオ画像間で矛盾無く「外形が欠けている」と判断できるかを確認している。
【００６０】
もし、式２のどちらも満足しないならば、判断条件（２）を満足しないとして返る。
【００６１】
次処理である他方画像での物体外形の有無の判定は、次のように行う。まず、元の画像における外形検出領域Ｒ１またはＲ２のうち、「距離を有すエッジ点を多く含む領域」側の平均距離に基づいて、他方画像における物体の見えるべき位置を算出する（後述）。次に、他方画像での見えるべき位置で図６で示した調査領域を設定し、元の画像で「距離が求まっていないエッジ点を多く含む領域」側と同一側の領域において、以下の条件を満足するか調べる。
【００６２】
（距離が求まっているエッジ点の数）≦閾値Ｔｈ₃
（距離が求まっていないエッジ点の数）≦閾値Ｔｈ₂
この条件は、ステレオ画像間で、物体の左右端の同一側で距離を有す外形が欠けていることを判断するためのものである。上記の２つの条件を満足するならば、矛盾無く外形の一部が欠けているので、判断条件（２）を満足するとして返る。もし満足しないならば、判断条件（２）は満足しないとする。
【００６３】
他方画像における物体の見えるべき位置の算出は、次のように行う。一方画像のエッジ点位置Ｅ₁ （ｘ^e _1,ｙ^e ₁ ）と他方画像でのエッジ点位置Ｅ₂ （ｘ^e _2,ｙ^e ₂ ）の関係は、エッジ点の距離Ｄ_E 、カメラの焦点距離ｆ、カメラ間距離ｂによって以下のように関連付けられる（水平に設置されたステレオの場合）。
【００６４】
【数４】

【００６５】
従って、物体外形のうち距離が求まっている側の平均距離Ｄmeanを用いて、物体外形の見え方モデルＭＬⁱ _j の矩形領域をｘ方向に計算値分だけずらすことで、他方画像での物体の見えるべき位置を決定できる。
【００６６】
もし、ｉ番目の物体断片Ｐ_i について、全ての物体モデルを用いて走査を終了したならば、他の物体断片Ｐ_k （ｋ＝_i+1 ）について以上の操作を繰り返す。もし全ての物体断片について以上の処理を施したならば、物体検出処理選択部４の処理を終え、それぞれの物体候補Ｏ^ij _k(i)毎に関連付けられた物体検出処理（全体外形物体検出処理部５、又は、部分外形物体検出処理部６）へ処理を移す。
【００６７】
次に、全体外形物体検出処理部５について説明する。全体外形物体検出処理部５では、物体断片Ｐ_i に関して走査により得られた距離を有する外形が全体から得られている物体候補Ｏ^ij _k について、物体の有無を判定し、物体があると判定されたならば、その３次元位置を決定する。処理の詳細について説明する。まず、ｉ番目の物体断片に対して、ｊ番目の物体モデル（見え方モデルＭＬⁱ _j ）を走査して求まったｋ番目の物体候補Ｏ^ij _k （０≦ｋ≦ＮＯ^j _i ；ＮＯ^j _i はｉ番目の物体断片に対して、ｊ番目のモデルを走査したときに得られる物体候補の数）について、見え方モデルＭＬⁱ _j を当てはめた時の当てはめ度合の良し悪しを評価するためのモデル一致度Ｅ１^ij _k を計算し、物体断片Ｐ_i 毎に、用いた見え方モデル全てから得られた物体候補の中でモデル一致度が最大となる値Ｅ１ⁱ _max と、それを与えるモデル位置（ｊ，ｋ）を次のように求める。
【００６８】
【数５】

【００６９】
もし、最大値Ｅ１ⁱ _max が閾値以上なら、（ｊ，ｋ）で与えられる位置（物体見え方モデルＭＬⁱ _j をｋ番目の物体候補を与える位置に置いた位置）に物体が存在すると決定する。もし最大値Ｅ１ⁱ _max が閾値以下なら、物体断片Ｐ_i に対する物体は存在しないと決定する。
【００７０】
ここで、見え方モデルとの一致度Ｅ１^ij _k は、与えられたモデルと画像から得られる物体の特徴量がどれだけ符合するかを表す量であれば、その実現方法については特に言及しないが、例えば次のようなものである。図６に示した外形検出領域Ｒ１、Ｒ２で、距離を有すエッジ点の個数の合計をＥ１ⁱⁿ ₁ とし、物体内部検出領域Ｒ３で距離を有すエッジ点の内、外形検出領域Ｒ１、Ｒ２の平均距離との差が閾値以内の点数をＥ１ⁱⁿ ₂ 、閾値以上の点数をＥ１^out とする時、以下の式でモデル一致度を与える。
【００７１】
Ｅ１^ij _k ＝Ｅ１ⁱⁿ ₁ ＋Ｅ１ⁱⁿ ₂ −Ｅ１^out
これは、物体の外形部や内部から物体と同一距離を持つエッジ点が得られるほど、また、物体の内部からは検査中の物体以外の証拠である距離の異なるエッジ点が得られていないほど高い得点になり、物体らしいとするものである。
【００７２】
続いて、部分外形物体検出処理部６について説明する。本処理では、物体断片Ｐ_i に関して走査により得られた距離を有す外形が一部（左右端のいずれか）からしか得られない場合に、物体の有無を判定し、物体があると判定されたならば、その３次元位置を決定する処理である。処理の詳細は、見え方モデルとの一致度の計算方法が異なることを除いては、先の全体外形物体検出処理部５と同じである。
【００７３】
部分外形での物体検出処理での見え方モデル一致度Ｅ２^ij _k は、全体外形での物体検出処理の時と同様に、与えられたモデルと画像から得られる物体の特徴量がどれだけ符合するかを表す量であれば、その実現方法については特に言及しないが、例えば次のようなものである。
【００７４】
図６に示した外形検出領域Ｒ１、Ｒ２のいずれかで距離を有すエッジ点を含む側での距離を有すエッジ点の個数をＥ２ⁱⁿ ₁ 、他方の距離が求まらないエッジ点を含む側の外形検出領域内にある距離が求まらないエッジ点の個数から求まる評価値をＥ２ⁱⁿ ₂ とし、物体内部検出領域Ｒ３における距離を有すエッジ点の内、外形検出領域Ｒ１、Ｒ２のいずれかで距離を有す側から求まる平均距離との差が閾値以内の点数をＥ２ⁱⁿ ₃ 、閾値以上の点数をＥ２^out とする時、以下の式でモデル一致度を与える。
【００７５】
Ｅ２^ij _k ＝Ｅ２ⁱⁿ ₁ ＋Ｅ２ⁱⁿ ₂ ＋Ｅ２ⁱⁿ ₃ −Ｅ２^out
ただし、Ｅ２ⁱⁿ ₂ ＝（外形検出領域内のエッジ点の数）×Ｔ（Ｔは式２と同じ平均対応率）
この式の意味は、先の全体外形物体検出処理部５と同じであり、物体の外形部や内部から物体と同一距離を持つエッジ点が得られるほど、また、物体の内部からは検査中の物体外である証拠である距離の異なるエッジ点が得られていないほど高い得点になり、物体らしいとするものである。ただし、距離が求まるエッジ点数と距離が求まらないエッジ点数の影響を同じにするために、Ｅ２ⁱⁿ ₂ の算出で計数Ｔをかけてある。
【００７６】
本発明の第２の実施の形態を図７により他図を参考にして説明する。図７において、他図と同記号は同一部分を示し、１０は物体検出処理部、１１は全体外形物体検出処理部、１２は部分外形物体検出処理部、１３は物体位置決定処理部である。
【００７７】
物体検出処理部１０は、前記図３に示す物体検出処理部３と同様に、ステレオ処理部２から距離情報とエッジ情報を受け取り、物体位置を出力するものではあるが、図３と異なる点は、物体検出処理部１０を構成する全体外形物体検出処理部１１と、部分外形物体検出処理部１２は、前記図３の全体外形物体検出処理部５と、部分外形物体検出処理部６のように、各物体断片Ｐ_i に対して唯一の物体位置を決定するものではなく、複数のＫ個（Ｋ≧２）の物体位置候補Ｕⁱ _k （ｋは物体位置候補の番号を表し、０≦ｋ≦Ｋ）を選定することと、この複数の物体位置候補Ｕⁱ _k を新たに設けた物体位置決定処理部１３により、過去の物体の情報を用いて、複数の物体位置候補Ｕⁱ _k から、それぞれの物体断片Ｐ_i に対応する物体位置を決定することである。
【００７８】
この構成により、画像から得られる情報のみで物体位置を決定せずに、過去の物体の、例えば移動情報等の、情報にもっとも符合する位置を決定することができ、常に安定して正しく物体位置を決定することができる。
【００７９】
以下に、前記図３の構成と異なる部分について詳述する。
【００８０】
まず、全体外形物体検出処理部１１について述べる。前記図３に示す全体外形物体検出処理部５と比較して違う点は、ｉ番目の物体断片Ｐ_i に対してｊ番目の見え方モデルＭＬⁱ _j を走査して得られたｋ番目の物体候補Ｏ^ij _k について、見え方モデルＭＬⁱ _j を当てはめた時の当てはめ度合いの良し悪しを評価するモデル一致度Ｅ１^ij _k を用いて、物体断片Ｐ_i 毎に、用いた見え方モデル全てを通して得られた物体候補Ｏ^ij _k （０≦ｉ≦Ｎ，１≦ｊ≦Ｎｍ）の中から、モデル一致度が大きいものから上位ｋ番目までを物体位置候補Ｕⁱ _k （０≦ｋ≦Ｋ）として出力する点である。
【００８１】
次に、部分外形物体検出処理部１２について述べる。前記図３に示す部分外形物体検出処理部６と比較して違う点は、先ほどと同様に、物体候補Ｏ^ij _k について、見え方モデルＭＬⁱ _j を当てはめた時の当てはめ度合の良し悪しを評価するモデル一致度Ｅ２^ij _k を用いて、物体断片Ｐ_i 毎に、用いた見え方モデル全てを通して得られた物体候補Ｏ^ij _k （０≦ｉ≦Ｎ，１≦ｊ≦Ｎｍ）の中から、モデル一致度が大きいものから上位ｋ番目までを物体位置候補Ｕⁱ _k （０≦ｋ≦Ｋ）として出力する点である。
【００８２】
続いて、物体位置決定処理部１３について述べる。今、既にＮ₀ 個の物体Ｗ_q （_q は物体の番号を表し、０≦ｑ≦Ｎ₀ である）が検出されているとする。それぞれの物体Ｗ_q 毎に運動モデルを持ち、前回までの物体位置の履歴から、運動モデルを用いて今回存在し得る物体位置Ｗ^pre _q を予測する。運動モデルの実際については特段規定せず、例えば一時刻前の位置からの線形予測モデルや、カルマンフィルタを用いた予測モデルで良い。ある予測物体位置Ｗ^pre _q に対して、Ｎ個の物体断片Ｐ_i のそれぞれについて求めた物体位置候補Ｕⁱ _k （０≦ｋ≦Ｋ）とを比較して、３次元での距離差が閾値以内なら、予測物体位置Ｗ^pre _q と物体位置候補Ｕⁱ _k から物体位置Ｗ^pre _q ＝Ｆ_p （Ｗ^pre _q ，Ｕⁱ _k ）として、物体位置Ｗ^pre _q を更新する。そしてＷ^pre _q と対応しなかった残りの物体位置候補Ｕⁱ _k は、同時には存在し得ないので、物体位置候補から削除する。
【００８３】
この操作により、物体位置Ｗ^pre _q と対応が求まらなかった物体位置候補Ｕⁱ _k （０≦ｋ≦Ｋ，０≦ｉ≦Ｎ）については各物体断片Ｐ_i から検出された物体位置候補Ｕⁱ _k （０≦ｋ≦Ｋ）毎に、評価値が最も高い候補位置に新たに物体が検出されたとして、物体位置Ｗ^pre _q+1 として登録する。この結果、検出された物体数をＮ₀ ←Ｎ₀ ＋１とする。
【００８４】
また、物体位置候補Ｕⁱ _k （０≦ｋ≦Ｋ，０≦ｉ≦Ｎ）のいずれとも対応が付かなかった物体位置Ｗ^pre _q があるならば、その物体位置Ｗ^pre _q を削除する。
この結果、検出された物体数をＮ₀ ←Ｎ₀ −１とする。
【００８５】
このようにして予測した位置のものともっとも近いものを抽出することができるので、背景のエッジなど物体ではない部分のエッジを用いて物体外形と誤認するようなことはなく、過去の物体位置の情報つまり運動の履歴情報等から動きの滑らかさなどを用いて、もっとも物体である可能性の高いものを最終的な物体位置として正確に出力できる。
【００８６】
本発明は、距離情報から物体を検出する方式において、従来からの物体外形の全体で距離情報が求まっている場合だけでなく、物体外形の一部の距離情報が欠けている場合には、予め用意した物体の外形の見え方のモデルを用いて、外形のどこが欠けているか識別し、欠けた部分では再度外形の距離を算出する処理を追加し、この２つの方式を適宜切り替えることで、物体外形の一部が欠けても安定に物体を検出できるようにしたことにより、照明状態の変化などによって映像全体の明るさが増減して、結果として物体の一部の距離情報が欠落する場合においても、安定に物体を検出できる。
【００８７】
【発明の効果】
本発明によれば下記の効果を奏することができる。
【００８８】
（１）物体外形の全体で距離が得られている場合に物体位置を決定する処理部に加えて、物体外形の一部の距離情報が欠けている場合でも、予め用意した物体の外形の見え方のモデルを用いて外形の欠けた部分に対しても距離を算出する処理部を設けたので、物体外形全体で距離が得られている場合は勿論のこと、物体外形の一部が欠けている場合でも、正確に物体を検出することができる。
【００８９】
（２）ステレオ処理で距離が求められない、一部の欠けた外形について、エッジ点などのように距離とは異なる特徴量の塊を使用して外形を形成するので、一部が欠けた外形に対しても外形部分を正確に得ることができる。
【００９０】
（３）過去に得られた物体位置の情報を用いて複数の物体位置候補から正しい物体位置を時系列的に得ることができるので、背景にもエッジ点があるような場合でもそれを誤認することなく正確に物体を検出することができる。
【図面の簡単な説明】
【図１】本発明の概略構成図である。
【図２】本発明における外形の一部が欠けている場合の処理説明図である。
【図３】本発明の一実施の形態である。
【図４】距離画像説明図である。
【図５】走査範囲説明図である。
【図６】物体外形の見え方モデルと外形検出領域の関係説明図である。
【図７】本発明の第２の実施の形態である。
【図８】ステレオ画像による位置ずれ検出説明図である。
【符号の説明】
１ ステレオ画像撮影部
２ ステレオ処理部
３、１０ 物体検出処理部
４ 物体検出処理選択部
５、１１ 全体外形物体検出処理部
６、１２ 部分外形物体検出処理部[0001]
BACKGROUND OF THE INVENTION
The present invention is an apparatus for detecting an object shown in a video based on distance information to each object in the video obtained by a stereo distance measurement technique. The present invention relates to an apparatus that stably detects an object and measures the three-dimensional position of the object even when the distance information of some of the object is lost as a result.
[0002]
A device that detects an arbitrary object in a video and identifies its three-dimensional position is, for example, a traffic flow measurement system that measures the position of a vehicle from a road image and calculates the presence / absence of passing of the vehicle, The present invention can be applied to general object monitoring systems such as a person monitoring system that measures the positions and the number of people passing from a moving image.
[0003]
In addition, in a logistics system that determines the type of article carried on the belt conveyor, it can be applied to article identification for identifying the article being carried from the size and image characteristics of the article. It can be applied as an object detection means for identifying an obstacle on a travel route in a transport system, or can be applied to a recognition process for measuring the position and size of a surrounding object in an autonomous robot system.
[0004]
[Prior art]
The conventional measurement method for measuring the position of an object using distance information around the sensor first measures the distance to each position of the object around the sensor (this is called distance information) by the stereo distance measurement method. Then, a part having a specific size or shape is extracted from the distance information obtained and identified as a corresponding object. Since the identified object has distance information, the three-dimensional position can be measured by a stereo image viewed from the sensor based on the distance information.
[0005]
As representative examples of this method, Keiji Miyoshi, Shinji Tsuji, Noriyuki Togawa, Kazuma Arai, etc., “Advancing situation recognition system for driving assistance using stereo images”, IEICE Technical Report PRMU 97-25-36pp. The method 39-46 will be described as an example.
[0006]
According to this, as shown in FIG. 8, a scene including the object 21 to be measured is photographed using the two television cameras 22 and 23 to obtain a stereo image. Here, since a television camera mounted horizontally is used, a pair of left and right stereo images 24 and 25 can be acquired.
[0007]
Next, edge points, which are point groups whose brightness changes suddenly, are extracted from the stereo images 24 and 25 by differentiating the stereo image to obtain edge images 26 and 27 for the left and right images, respectively. Subsequently, as shown in the image displacement detection screen 28 for each edge point by a stereo distance measurement process described later, a displacement amount d is detected. Based on the displacement amount d and the distance between the television cameras 22 and 23, this edge amount is detected. The distance to the object is measured for the edge point.
[0008]
In the stereo distance measurement process, a local region of N × N pixels is provided at each position in one edge image, for example, the edge image 26. For each local region, a local region of N × N pixels is provided in the other edge image 27, and the same edge as the local region of the edge image 26 is scanned by scanning this local region in the other edge image 27. A local region in the edge image 27 in which points are distributed is detected. That is, a process of searching for a portion indicating the same object from the left edge image 26 and the right edge image 27 is performed. This is the stereo distance measurement process.
[0009]
Since the position of the object in the left and right images is different depending on the distance, that is, the position to be projected differs, by detecting the amount of positional deviation by searching the left and right images for the projection position of the same object image, The distance can be calculated.
[0010]
Since the distance information obtained at each edge point of the image by this stereo processing is mapped to one point in the three-dimensional space, mapping is performed for each edge point for which the distance is obtained, and the position distribution in the three-dimensional space is performed. Get. In this three-dimensional space, for example, a vehicle that is a measurement target can be identified by detecting a three-dimensional lump that is above a predetermined road surface position and that matches the width, height, and depth of the vehicle. it can.
[0011]
[Problems to be solved by the invention]
In the sequential method of acquiring the distance information to each position of the target scene and measuring the object position based only on the distance information, if the distance information to the object is obtained, the object is The three-dimensional position can be measured.
[0012]
However, for example, when operating outdoors, because the illuminance increases or decreases due to changes in sunlight or weather, the entire image becomes too dark or too bright, and the brightness of the target object and the background becomes similar. As a result, the detection position of the edge point indicating the outer shape of the target object may shift.
[0013]
In addition, since the position shifts in the left, right, up, and down directions are independent for each edge, even in the stereo processing for each edge point, even if the edge point distribution shape for each local region is compared, Even if a matching portion cannot be found and the distance of the object outer shape cannot be measured, or even if a matching portion is found, the position of the edge includes a shift, and thus information on an incorrect distance is presented. As a result, there is a problem that distance information indicating the outer shape of the object is lost and the object cannot be detected, or that the object is erroneously recognized as being in the wrong position using the incorrect distance information.
[0014]
In stereo processing using two images, the distance cannot be obtained in principle for the edge in the direction parallel to the two installed cameras. For example, when a camera is installed horizontally on the left and right, the distance of the horizontal line edge cannot be calculated, and the distance can be obtained only in the vertical outline of the object. Observing how the distance information of the object outline is missing when illuminance changes actually occur, it is often the case that one of the outlines at the left and right ends of the object is missing, compared to when the entire object outline cannot be obtained. . If either of the left and right ends of the object outline is missing, the three-dimensional width and height change, and therefore cannot be detected as an object.
[0015]
Considering such a situation, it is necessary to be able to accurately detect the position of an object even when distance information cannot be obtained from a part of the outer shape of the object. Accordingly, an object of the present invention is to provide an object position measuring apparatus capable of accurately detecting the position of an object even when distance information cannot be obtained from a part of the outer shape of the object.
[0016]
[Means for Solving the Problems]
The principle configuration of the present invention is shown in FIG. In FIG. 1, 1 is a stereo image photographing unit, 2 is a stereo processing unit, 3 is an object detection processing unit, 4 is an object detection processing selection unit, 5 is an overall external object detection processing unit, and 6 is a partial external object detection processing unit. . The object of the present invention can be achieved by the following (1) to (3).
[0017]
  (1) In an object position measurement device that detects an object from stereo measurement distance information, an object detection processing selection unit 4 that determines whether distance information is obtained for the entire outer shape of the object to be detected;
  When distance information is obtained for the entire outer shape, the entire outer shape object detection processing means 5 for determining the object position;
  When the distance information of a part of the outer shape of the object is missing, the missing part of the outer shape is identified using a model of the appearance of the outer shape of the object prepared in advance.An outline is formed on the basis of a feature block such as an edge point at a corresponding position, and stereo matching is performed based on the block.An object position measuring apparatus comprising a partial outline object detection processing means 6 for calculating an outline distance.
[0018]
  (2)The partial outline object detection processing means 6,SaidA block of features such as edge pointsBased on the average position ofPerform stereo mappingTheThe object position measuring apparatus according to (1), characterized in that it is characterized in that
[0019]
(3) The overall outline object detection processing means 5 and the partial outline object detection processing means 6 calculate a plurality of object position candidates and send them to the object position determination processing means. The object position measuring apparatus according to (1) or (2), wherein a correct object position can be determined from a plurality of object position candidates using information on an object position obtained in the past.
[0020]
As a result, the following effects can be obtained.
[0021]
(1) In addition to the processing unit for determining the object position when the distance is obtained for the entire object outline, even if the distance information of a part of the object outline is missing, the appearance of the prepared object outline is visible A processing unit is provided that calculates the distance even for parts with missing outlines using the other model, so if the distance is obtained for the entire object outline, part of the object outline is missing. Even if it is, the object can be detected accurately.
[0022]
(2) For some missing outlines for which the distance cannot be obtained by stereo processing, the outline is formed using a lump of feature amount different from the distance, such as an edge point. The outer shape portion can be accurately obtained.
[0023]
(3) Since the correct object position can be obtained in time series from a plurality of object position candidates using the object position information obtained in the past, it is mistaken even when there is an edge point in the background. An object can be detected accurately without any problems.
[0024]
DETAILED DESCRIPTION OF THE INVENTION
The outline of the present invention will be described with reference to FIGS. FIG. 1 is a schematic configuration diagram of the present invention, and FIG. 2 is an explanatory diagram of a processing state when a part of the outer shape is missing.
[0025]
First, a stereo image obtained by the stereo image photographing unit 1 having two cameras is sent to the stereo processing unit 2 to acquire distance information. This distance information is obtained by the same method as in the conventional method, and the edge portion obtained by the differential processing of the stereo image is detected and then the edge portion obtained from the left and right images is the same. A part is detected, and a distance is obtained by obtaining a deviation of the same part. Conventionally, when the object detection processing, and mapping the distance information in three-dimensional space, a model (three-dimensional model of the width, height and depth of the object from the 3-dimensional point information, what size detection object in advance The entire outline object detection processing unit 5 performs only the process of detecting an object using a plurality of fixed cases or a variable size expressed by an expression). It was.
[0026]
On the other hand, in the present invention, in the process of detecting an object from distance information, not only when the distance is obtained for the entire outer shape of the object but also when the distance information of a part of the outer shape of the object is missing Is used to identify where the outline is missing using a model of the appearance of the outline of the object prepared in advance, and the outline distance is detected again in the missing part, and the partial outline object detection processing unit 6 shown in FIG. The object detection processing selection unit 4 performs switching control between the processing in the case where the distance of the entire outer shape is obtained and the processing in the case where the distance information of a part of the object outer shape is missing. The object can be stably detected even when the whole is obtained or when a part of the object outline is missing.
[0027]
The processing when the part of the object outline is missing is performed by the partial outline object detection processing unit 6 in FIG. 1 and is an important component of the present invention. The process of calculating the outer distance again will be described with reference to FIG. In FIG. 2, it is assumed that the camera is attached in a horizontal state and the shape of the object is a rectangle (a rectangular parallelepiped), and distance information is obtained only for the outer shape of the left end.
[0028]
First, the appearance model M of the outer shape of the object is applied to the distance image (left distance image) in which the distance is determined for each edge position of one image (here, the left image taken by the left camera), and the outer shape is missing. It is checked whether it is present (process {circle around (1)}). Thereby, it is detected that the outer right edge is missing.
[0029]
Next, the edge image is directly examined for the portion determined to be missing (in this case, the right end), and the presence / absence of an edge point whose distance corresponding to the outer shape of the object is not found is examined (processing (2)).
[0030]
If there is an edge point for which the distance is not obtained, the edge point group is extracted as a lump (processing (3)). Then, this edge lump is regarded as the missing right edge of the outer shape (processing (4)).
[0031]
This operation is performed in the same manner for the other image (here, the right distance image) to detect the edge block forming the outer right edge (processing (1) ′, (2) ′, (3) ′, (4) ′). .
[0032]
Next, the distance information outer shape obtained from the left distance image is set as the outer left edge (processing (5)), and the outer right edge obtained from the processing (4) is integrated to create a left integrated screen. Similarly, the distance information outer shape obtained from the right distance image is set as the outer left end (process (5) '), and the right outer edge obtained from the process (4)' is integrated to create a right integrated screen. Then, the right end chunks of the left integrated image and the right integrated image are associated with each other (for example, the average position is calculated), and the distance of the right end outline is calculated (processing (6)).
[0033]
In this way, the edge point itself exists as in the case where the brightness difference between the object and the background is reduced, but the accuracy of the detected position is low, and the correspondence between the edge points between the left and right images, that is, the edge point This means that even if coincidence detection is impossible, the edge blocks are associated with each other.
[0034]
However, when the object outline is determined only by using the edge points, there is a case where the object outline is mistakenly used by using an edge of an incorrect part such as a background edge that is not an object. Therefore, in the present invention, as described in a second embodiment to be described later, the partial outline object detection processing unit 6 determines the model coincidence (evaluation value of object-likeness) and searches for the object position. A plurality of top K-th candidate objects (K is 1 or more) having a high model matching degree are output. Then, the object position determination process is performed by determining the smoothness of the movement from the information of the past object position such as the history information of the object movement, and the object having the highest possibility of being the object is finally determined from a plurality of candidates. Output as an accurate object position. Thus, even if a part of the object outline is missing, the correct object position can be determined without being affected by the background or other objects.
[0035]
An embodiment of the present invention will be described with reference to FIGS. 3 is an embodiment of the present invention, FIG. 4 is an explanatory diagram of a distance image, FIG. 5 is an explanatory diagram of a scanning range, and FIG. 6 is an explanatory diagram of a relationship between an appearance model of an object outer shape and an outer shape detection region.
[0036]
In the figure, the same symbols as those in the other figures indicate the same parts, 1 is a stereo image photographing unit, 2 is a stereo processing unit, 3 is an object detection processing unit, 4 is an object detection processing selection unit, and 5 is an overall outline object detection processing unit. , 6 are partial outline object detection processing units.
[0037]
The stereo image photographing unit 1 is composed of two or more cameras having different installation positions, for example, two cameras, and obtains, for example, two stereo images by this stereo camera device.
[0038]
The stereo processing unit 2 uses the stereo image obtained by the stereo image capturing unit 1 as an input, first performs a collection process for each input image, extracts a point where the brightness changes suddenly, that is, an edge point, and extracts the edge image. create. Next, stereo association processing for obtaining the same portion is performed by using the edge image of the stereo image as an input, and the distance is obtained from the shift amount of the image for the same portion. Then, a distance image having depth information for each edge point is created only for those for which a distance is required, and is used as the output of the stereo processing unit 2. At the same time, the edge image group obtained first is also output as the output of the stereo processing unit 2 for subsequent processing.
[0039]
The object detection processing unit 3 receives the distance image and the edge image group, which are the outputs of the stereo processing unit 2, and performs processing to detect an object and determine a three-dimensional position. The object detection processing selection unit 4, An overall external object detection processing unit 5, a partial external object detection processing unit 6 and the like are provided.
[0040]
The object detection processing selection unit 4 uses the input distance information (distance image) to determine the part P that seems to be an object._i(_i≧ 1) (referred to as an object fragment) is extracted, and an object shape model M representing a three-dimensional shape of an object prepared in advance_j(_jAppearance model ML with an image determined from ≧ 1)ⁱ _j(_j≧ 1) is the object fragment P_iTo determine whether the object outline is missing from the distance information. The appearance model is an object that is shown depending on the position where the object is viewed.The shape of the object changes depending on the position where the object is seen.For example, in the case of a rectangular parallelepiped, the appearance model is rectangular when viewed from the side. Thus, when viewed from an oblique lateral direction, the visible model has a shape in which the side surface can be seen in addition to the front portion. The example of FIG. 2 shows the appearance model M when the rectangular parallelepiped is viewed from the side.
[0041]
If there is no chip, the object detection processing selection unit 4 moves the process to the entire outer shape object detection processing unit 5 to determine the presence / absence of the object using the distance information of the entire outer shape of the object, and determines the position if there is an object. To do. If the object detection processing selection unit 4 determines that the object outline is missing, the process is transferred to the partial outline object detection processing unit 6 and the object information is obtained using the partial outline distance information and the edge point information in the missing part. If there is an object, its position is determined.
[0042]
The detailed contents of each part will be described below.
[0043]
First, the object detection process selection unit 4 will be described. In the object detection processing selection unit 4, first, the object fragment P is_i(0 ≦_i≦ N; N is the number of object fragments). P_iAs shown in FIG. 4, first, the distance image Dimg has a width and height of G as shown in FIG._w× G_hSmall region G_kDivide into (grids), find the average value and variance value of edge point distance distribution included in each small area, and the number of edge points, and the area where the variance value of distance is greater than the threshold or the number of edge points is less than the threshold The area is deleted as an unstable area whose distance is not constant. For the remaining small regions, the small region G on the distance image (two-dimensional)_kDistance difference between₁Difference in depth (depth difference) L₂Are grouped together into a small area group each within the threshold, and the object fragment P_iExtract as This means that a block located three-dimensionally close is extracted as an object fragment.
[0044]
Next, each obtained object fragment P_iOn the other hand, the three-dimensional shape model M of the object_j(1 ≦ j ≦ Nm; Nm is the number of types of object shape models to be prepared)_i"Object outline appearance model ML that represents how the object outline looks when placed at the position (distance)ⁱ _j(Where i is the number of the object fragment, j is the number of the model being used, 1 ≦ j ≦ Nm; Nm is the number of types of object shape models to be prepared). Any 3D shape model of the object may be used, but here, as one of the realization examples, the object width MW_j, Height MH_j, Depth MD_jA rectangular parallelepiped model is used.
[0045]
Each object fragment P_iCenter position (Cx_i,Cy_i,Cz_i) So that the front of the rectangular parallelepiped model is aligned, each vertex of the rectangular parallelepiped is (Cx_i± MW_j/ 2, Cy_i± MH_j/ 2, Cz_i) X (Cx_i± MW_j/ 2, Cy_i± MH_j/ 2, Cz_i+ MD_j) (The code combination is arbitrary). Each vertex position is set to (X, Y, Z), and the projection position (x, y) on the image is expressed by the following formula (X) using the focal length f and the distance between two cameras b, which are internal parameters of the camera. The minimum rectangular area obtained by 1) and surrounding the projection point
[0046]
[Expression 1]

[0047]
A model ML for the appearance of an objectⁱ _j(1 ≦ j ≦ Nm; Nm is the number of types of object shape models to be prepared). This object external appearance model MLⁱ _jWhen the range image is scanned within the range image and the determination condition (1) described below is satisfied, it is considered that the entire object outline has been obtained. At this time, the k (i) th object candidate O^ij _{k (i)}To generate O^ij _{k (i)}In addition, the scanning position is recorded, and a flag for shifting to the entire outline object detection processing unit 5 is set. This k (i) is the i-th object fragment P_iIs a serial number assigned to the detected object candidate, and a different number is assigned to each candidate, which means that a plurality of candidates may be detected.
[0048]
In addition, when a determination condition (2) described later is satisfied, it is considered that a part of the object outline has been obtained, and the k (i) th object candidate O^ij _{k (i)}Is generated, the scanning position is recorded, and a flag for shifting to the partial outline object detection processing unit 6 is set.
[0049]
If neither condition (1) nor (2) is satisfied, it is assumed that no object exists and the scanning operation is continued. The scanning range is as shown in FIG._iAnd object appearance model MLⁱ _jIs the range from the leftmost edge to the rightmost edge where_iRight side and view model MLⁱ _jFrom the position where the right side of the object coincides,_iLeft side and view model MLⁱ _jThe range to the position where the left side of the line matches.
[0050]
Judgment condition (1): (Inspection of whether the entire outer shape is obtained)
Here, when the direction in which the stereo cameras are arranged is considered as the horizontal axis (it does not have to be horizontal), the left camera (image) is the left and the right camera (image) is the right. . At this time, due to the nature of stereo, only the vertical component of the object can be obtained. Therefore, what is important as the outer shape of the object is the vertical outer shape at the left and right ends. Therefore, the presence / absence of the outer shape at the left and right ends is used as a criterion for determining whether or not the entire outer shape is obtained.
[0051]
Object appearance model MLⁱ _jIs placed in the distance image, and the investigation area (R1 to R3) as shown in FIG.ⁱ _jThe position of is set as a reference. Here, R1 is a region for inspecting the presence of the outer shape on the left end side, R2 is a region for inspecting the presence of the outer shape on the right end side, and R3 is a region for determining whether the object is inside or outside. The size of the area is determined as follows. For R1 and R2, the height is the appearance model MLⁱ _jHeight MHⁱ _jSame as Width R_wIs a visible model MLⁱ _jWidth MWⁱ _jFunction F (MWⁱ _j), For example, F (X) = X / 4. As shown in FIG. 6, the positions where R1 and R2 are set are set so that half of the widths of R1 and R2 overlap with the appearance model. R3 is a portion that is not R1 or R2 in the appearance model. Therefore, the width of the region R3 is MWⁱ _j-F (MWⁱ _j), The height is MHⁱ _jIt is.
[0052]
Regarding the outer shape detection regions R1 and R2, when both of the following conditions are satisfied, it is determined that all the outer shapes of the object have been found.
[0053]
(Number of edge points having a distance in the inspection region R1 on the left end side) ≧ threshold Th₁
(The number of edge points having a distance in the inspection region R2 on the right end side) ≧ threshold Th₁
Here, the threshold Th₁Is a threshold value for suppressing the influence of noise that defines the minimum value of the number of edge points having a distance that should exist in the region R1 (or R2). When there are more edge points than this, it is considered that there is a lump of edge points stably having a distance, that is, there is an outline having a distance.
[0054]
Judgment condition (2): (Inspection of whether a part of the outer shape is obtained, that is, a part is missing)
Similar to the determination condition (1), the object external appearance appearance model MLⁱ _jIs installed in one of the stereo images, and the investigation region (R1, R2) shown in FIG. 6 is examined to satisfy any of the following conditions.
[0055]
[Expression 2]

[0056]
Here, the “edge point whose distance is not determined” is the same between the left and right edge images in stereo processing because the position of the edge point is shifted due to a small change in brightness between the object and the background. An edge point whose location cannot be determined and whose distance is not determined. If one of these conditions is met, there is an object outline having distance information that can determine a three-dimensional position at one end, and the distance (at the other end does not know the distance, but a feature that can form the object outline as a lump (trace) ) Is present. First, the threshold Th for the number of edge points for which the distance is not determined₂Is the threshold Th₁Against
[0057]
[Equation 3]

[0058]
A value having the relationship Here, T in the equation is an average correspondence rate with respect to all edge points, and is obtained in advance by experiment, for example, and represents a ratio of the number of edge points whose distance is obtained on average among the total number of edges. . Usually, since the distance is not obtained for all edge points, the numerical value is 1 or less. By using this function T, the number of edge points for which the distance is determined is adjusted to be substantially equal to the number of edge points for which the distance is not determined.
[0059]
If any of the conditions of Equation 2 is satisfied, it is determined that a part of the outer shape of the object may have been found (partially missing), and will be described in the following processing. In the same way, examine the image for the presence of an outline. The purpose of this survey is to prevent misrecognition that an object lacks a part of its outline by combining parts of different objects. Check whether it can be judged.
[0060]
If neither of the expressions 2 is satisfied, it returns that the judgment condition (2) is not satisfied.
[0061]
The determination of the presence or absence of the object outline in the other image, which is the next process, is performed as follows. First, a position where an object should be visible in the other image is calculated based on the average distance on the “region including many edge points having a distance” side in the outer shape detection region R1 or R2 in the original image (described later). Next, the investigation area shown in FIG. 6 is set at a position to be seen in the other image, and the following condition is satisfied in the area on the same side as the “area including many edge points whose distance is not determined” in the original image. Find out if you are satisfied.
[0062]
(Number of edge points for which distance is determined) ≦ threshold Th_Three
(Number of edge points for which distance is not determined) ≦ threshold Th₂
This condition is for determining that a stereo image having a distance on the same side of the left and right ends of an object is missing between stereo images. If the above two conditions are satisfied, a part of the outer shape is missing without contradiction, and it is returned that the determination condition (2) is satisfied. If not satisfied, it is assumed that the judgment condition (2) is not satisfied.
[0063]
On the other hand, the calculation of the position where the object should be visible in the image is performed as follows. On the other hand, the edge point position E of the image₁(X^e _1,y^e ₁) And edge point position E in the other image₂(X^e _2,y^e ₂) Is related to the distance D of the edge points._E, And the camera focal length f and the inter-camera distance b are related as follows (in the case of a stereo installed horizontally).
[0064]
[Expression 4]

[0065]
Therefore, using the average distance Dmean on the side where the distance is found in the object outline, the object outline appearance model ML is used.ⁱ _jThe position where the object should be visible in the other image can be determined by shifting the rectangular area by the calculated value in the x direction.
[0066]
If i-th object fragment P_iIf the scanning is completed using all the object models, another object fragment P_k(K =_{i + 1}) Repeat the above operation. If all the object fragments have been subjected to the above processing, the processing of the object detection processing selection unit 4 is finished, and each object candidate O^ij _{k (i)}The process is transferred to the object detection process (the entire outer object detection processing unit 5 or the partial outer object detection processing unit 6) associated with each.
[0067]
Next, the overall outer shape object detection processing unit 5 will be described. In the overall outline object detection processing unit 5, the object fragment P_iObject candidate O from which the outline having the distance obtained by scanning is obtained from the whole^ij _kWith respect to, the presence or absence of an object is determined, and if it is determined that there is an object, its three-dimensional position is determined. Details of the processing will be described. First, for the i-th object fragment, the j-th object model (view model MLⁱ _j) To find the kth object candidate O^ij _k(0 ≦ k ≦ NO^j _iNO^j _iIs the appearance model ML for the number of object candidates obtained when the jth model is scanned with respect to the ith object fragment.ⁱ _jModel agreement E1 for evaluating the degree of fit when fitting^ij _kTo calculate the object fragment P_iEvery time, the value E1 that maximizes the model coincidence among the object candidates obtained from all the appearance models used.ⁱ _maxAnd the model position (j, k) that gives it is determined as follows.
[0068]
[Equation 5]

[0069]
If the maximum value E1ⁱ _maxIs equal to or greater than the threshold value, the position given by (j, k) (object appearance model MLⁱ _jIs determined to be present at a position where the kth object candidate is to be provided. If the maximum value E1ⁱ _maxIs less than the threshold, the object fragment P_iIt is determined that there is no object for.
[0070]
Here, the degree of coincidence E1 with the appearance model^ij _kIs a quantity that represents how much the feature quantity of an object obtained from a given model and an image matches, but the implementation method is not particularly mentioned, but for example, is as follows. In the outer shape detection regions R1 and R2 shown in FIG. 6, the total number of edge points having a distance is E1.ⁱⁿ ₁Of the edge points having a distance in the object internal detection region R3 and the difference between the average distances of the outer shape detection regions R1 and R2 within a threshold is E1.ⁱⁿ ₂, E1 points above the threshold^out, The model agreement is given by the following formula.
[0071]
E1^ij _k= E1ⁱⁿ ₁+ E1ⁱⁿ ₂-E1^out
This is because the edge point having the same distance as the object is obtained from the outer part of the object or the inside of the object, and the edge point having a different distance that is evidence other than the object being examined is not obtained from the inside of the object. It is a high score and it seems to be an object.
[0072]
Next, the partial outline object detection processing unit 6 will be described. In this process, the object fragment P_iIf the outer shape having a distance obtained by scanning can be obtained only from a part (one of the left and right ends), the presence or absence of the object is determined, and if it is determined that there is an object, its three-dimensional position It is a process to determine. The details of the process are the same as those of the overall outline object detection processing unit 5 except that the method of calculating the degree of coincidence with the appearance model is different.
[0073]
Appearance model coincidence E2 in object detection processing with partial outline^ij _kAs with the object detection processing for the entire outline, there is no particular mention as to how to realize it as long as it represents how much the feature quantity of the object obtained from a given model and image matches. For example, it is as follows.
[0074]
The number of edge points having a distance on the side including the edge point having a distance in either one of the outer shape detection regions R1 and R2 shown in FIG.ⁱⁿ ₁E2 is an evaluation value obtained from the number of edge points whose distance is not found in the contour detection region on the side including the edge point whose other distance is not found.ⁱⁿ ₂E2 is the number of points within the threshold that the difference from the average distance obtained from the side having the distance in either of the outer shape detection areas R1 and R2 among the edge points having the distance in the object internal detection area R3 is E2.ⁱⁿ _Three, E2 points above the threshold^out, The model agreement is given by the following formula.
[0075]
E2^ij _k= E2ⁱⁿ ₁+ E2ⁱⁿ ₂+ E2ⁱⁿ _Three-E2^out
However, E2ⁱⁿ ₂= (Number of edge points in the contour detection region) × T (T is the same average correspondence rate as in Equation 2)
The meaning of this expression is the same as that of the entire outline object detection processing unit 5 described above, and the edge point having the same distance as the object is obtained from the outline part or the inside of the object. The score is so high that edge points with different distances, which are evidence of being out of the object, are not obtained, and the object seems to be an object. However, in order to make the influence of the number of edge points from which the distance can be obtained and the number of edge points from which the distance cannot be obtained the same, E2ⁱⁿ ₂The count T is multiplied by the calculation of.
[0076]
A second embodiment of the present invention will be described with reference to FIG. In FIG. 7, the same symbols as those in the other figures indicate the same parts, 10 is an object detection processing unit, 11 is an overall external object detection processing unit, 12 is a partial external object detection processing unit, and 13 is an object position determination processing unit.
[0077]
Similar to the object detection processing unit 3 shown in FIG. 3, the object detection processing unit 10 receives distance information and edge information from the stereo processing unit 2 and outputs an object position, but differs from FIG. The entire outer object detection processing unit 11 and the partial outer object detection processing unit 12 constituting the object detection processing unit 10 are the same as the entire outer object detection processing unit 5 and the partial outer object detection processing unit 6 of FIG. , Each object fragment P_iAre not determined for a single object position, but a plurality of K (K ≧ 2) object position candidates Uⁱ _k(K represents the number of the object position candidate, 0 ≦ k ≦ K), and the plurality of object position candidates Uⁱ _kThe object position determination processing unit 13 newly provided with a plurality of object position candidates U using the past object information.ⁱ _kTo each object fragment P_iThe object position corresponding to is determined.
[0078]
With this configuration, it is possible to determine the position of the past object that best matches the information, such as movement information, without determining the object position based only on the information obtained from the image, and always stably and correctly position the object. Can be determined.
[0079]
In the following, portions different from the configuration of FIG. 3 will be described in detail.
[0080]
First, the overall outer shape object detection processing unit 11 will be described. The difference from the overall outline object detection processing unit 5 shown in FIG. 3 is that the i-th object fragment P_iThe jth appearance model MLⁱ _jKth object candidate O obtained by scanning^ij _kAbout the appearance model MLⁱ _jModel agreement E1 that evaluates whether the degree of fitting is good or bad when fitting^ij _kUsing the object fragment P_iFor each candidate object O obtained through all the appearance models used^ij _kFrom (0 ≦ i ≦ N, 1 ≦ j ≦ Nm), the object position candidate U from the one with the highest model matching degree to the top k-thⁱ _k(0 ≦ k ≦ K).
[0081]
Next, the partial outline object detection processing unit 12 will be described. The difference from the partial outline object detection processing unit 6 shown in FIG. 3 is that the object candidate O is similar to the above.^ij _kAbout the appearance model MLⁱ _jModel agreement E2 that evaluates whether the degree of fitting is good or bad when fitting^ij _kUsing the object fragment P_iFor each candidate object O obtained through all the appearance models used^ij _kFrom (0 ≦ i ≦ N, 1 ≦ j ≦ Nm), the object position candidates U from the one with the highest model matching degree to the top kthⁱ _k(0 ≦ k ≦ K).
[0082]
Next, the object position determination processing unit 13 will be described. N already₀Object W_q(_qRepresents the number of the object, 0 ≦ q ≦ N₀) Is detected. Each object W_qEach object has a motion model, and the object position W that can exist this time using the motion model from the history of the object position until the previous time^pre _qPredict. The actual motion model is not particularly specified, and for example, a linear prediction model from a position one hour before or a prediction model using a Kalman filter may be used. Predicted object position W^pre _qN object fragments P_iObject position candidate U obtained for eachⁱ _k(0 ≦ k ≦ K), and if the three-dimensional distance difference is within the threshold, the predicted object position W^pre _qAnd object position candidate Uⁱ _kTo object position W^pre _q= F_p(W^pre _q, Uⁱ _k) As the object position W^pre _qUpdate. And W^pre _qThe remaining object position candidate U that did not correspond toⁱ _kSince they cannot exist simultaneously, they are deleted from the object position candidates.
[0083]
By this operation, the object position W^pre _qObject position candidate U for which correspondence was not foundⁱ _k(0 ≦ k ≦ K, 0 ≦ i ≦ N), each object fragment P_iObject position candidate U detected fromⁱ _kFor each (0 ≦ k ≦ K), it is assumed that a new object is detected at the candidate position with the highest evaluation value.^pre _{q + 1}Register as As a result, the number of detected objects is expressed as N₀← N₀+1.
[0084]
Also, the object position candidate Uⁱ _kObject position W that does not correspond to any of (0 ≦ k ≦ K, 0 ≦ i ≦ N)^pre _qIf there is an object position W^pre _qIs deleted.
As a result, the number of detected objects is expressed as N₀← N₀-1.
[0085]
Since the object closest to the predicted position can be extracted in this way, the edge of the part that is not the object such as the background edge is not mistaken for the object outline, and the past object position Using information such as motion history information and the like, smoothness of movement can be used to accurately output the most likely object as the final object position.
[0086]
In the method of detecting an object from distance information, the present invention is not limited to the case where the distance information is obtained for the entire object outline in the past, but when the distance information of a part of the object outline is missing. Using the prepared model of the appearance of the object, identify where the outline is missing, add processing to calculate the distance of the outline again at the missing part, and switch between these two methods as appropriate. When it is possible to detect an object stably even if a part of the outline is missing, the brightness of the entire image increases or decreases due to changes in lighting conditions, etc. However, the object can be detected stably.
[0087]
【The invention's effect】
According to the present invention, the following effects can be obtained.
[0088]
(1) In addition to the processing unit for determining the object position when the distance is obtained for the entire object outline, even if the distance information of a part of the object outline is missing, the appearance of the prepared object outline is visible A processing unit is provided that calculates the distance even for parts with missing outlines using the other model, so if the distance is obtained for the entire object outline, part of the object outline is missing. Even if it is, the object can be detected accurately.
[0089]
(2) For some missing outlines for which the distance cannot be obtained by stereo processing, the outline is formed using a lump of feature amount different from the distance, such as an edge point. The outer shape portion can be accurately obtained.
[0090]
(3) Since the correct object position can be obtained in time series from a plurality of object position candidates using the object position information obtained in the past, it is mistaken even when there is an edge point in the background. An object can be detected accurately without any problems.
[Brief description of the drawings]
FIG. 1 is a schematic configuration diagram of the present invention.
FIG. 2 is an explanatory diagram of processing when a part of the outer shape is missing in the present invention.
FIG. 3 is an embodiment of the present invention.
FIG. 4 is an explanatory diagram of a distance image.
FIG. 5 is an explanatory diagram of a scanning range.
FIG. 6 is an explanatory diagram of a relationship between an object appearance appearance model and an outline detection region.
FIG. 7 is a second embodiment of the present invention.
FIG. 8 is an explanatory diagram of misregistration detection using stereo images.
[Explanation of symbols]
1 Stereo image capture unit
2 Stereo processing unit
3, 10 Object detection processing unit
4 Object detection processing selection section
5, 11 Overall outline object detection processing unit
6, 12 Partial outline object detection processing unit

Claims (3)

In an object position measuring device that detects an object from stereo measurement distance information,
Object detection processing selection means for determining whether or not distance information excluding a horizontal component and a stereo camera whose distance is not obtained by stereo measurement in the outer shape of the object to be detected;
When distance information is obtained for the entire outer shape, the entire outer shape object detection processing means for determining the object position;
When the distance information of a part of the outer shape of the object is missing, the missing part of the outer shape is identified using a model of the appearance of the outer shape of the object prepared in advance, and the edge at the position corresponding to the missing part A partial external object detection processing unit is provided that calculates an external distance of the missing portion by forming an external shape based on a mass of feature quantities such as points and performing stereo matching based on the mass. An object position measuring device. Said partial outline object detection processing means, an object position measuring device according to claim 1, characterized in that the stereo correspondence based on the average position of the mass of such characteristic amounts of the edge points. In the whole outer shape object detection processing means and the partial outer shape object detection processing means, a plurality of object position candidates are calculated and sent to the object position determination processing means. 3. The object position measuring apparatus according to claim 1, wherein a correct object position can be determined from a plurality of object position candidates using the object position information.

Images