JP2004301796A - Robot, and method and system for localizing landmark, and the landmark - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately estimate the position of a flat landmark having suitable form pattern and color pattern, according to its installation attitude. <P>SOLUTION: The relative distance from a robot to a landmark is roughly measured with a general range finding technology, however, in order to measure more accurately, calculation methods vary, depending on the installation attitude of the landmark. The landmark in a photographed image is estimated as to whether it is installed horizontally or vertically. Based on the estimation result, an appropriate localization algorithm for the landmark is selected. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

【００７５】
表１からも判るように、内円半径を小さくし過ぎると、ランドマークの画像に占める内円の比率が低下するため認識率は低下する。逆に、内円半径を大きくしすぎると外円半径が細くなってしまうためやはり認識率が低下してしまう。この結果、上述したように、内円と外円の半径の比率が３：４＜ｄ_ｉ：ｄ_ｏ＜１：３の範囲内において、良好な認識率を確保することができる。
【００７６】
また、上述したように、内円と外円とは互いに異なる色には塗布され、ＵＶ色空間上で分離し易い配色が施されていることが好ましい。
【００７７】
明暗変化に対する色抽出のし易さを考えた場合（照度変化に対してゲインを調整）、ゲイン調整さえ合っていれば、どの色もＵＶ空間上で放射線状に分布する。この観点で、どの色であっても内円、外円の配色に利用することができる。
【００７８】
また、ゲインが適切でない場合の色抽出のし易さ（照度固定で、ゲイン変更）王道色と特性色とでＵＶ空間上の分布は相違する。
【００７９】
また、色温度変化に対する色抽出のし易さを考えた場合（照明色温度変化に対してホワイト・バランス調整（２７００Ｋ，４５００Ｋ，５０００Ｋ，６５００Ｋ：：９００１ｘ））、桃色系は狭い範囲で収まってくる。また、青や緑は周りが広い。
【００８０】
ホワイト・バランスが適切でない場合の色抽出のし易さは（照明色温度変化に対してホワイト・バランス調整（２７００Ｋ，４５００Ｋ，５０００Ｋ，６５００Ｋ：：９００１ｘ））、黄色系は橙色に変化し、橙色はオーバーラップしてしまう。この観点からは、黄色は使用せず、コンパクトのオレンジを採用することが好ましい。
【００８１】
Ｃ．ランドマークの認識及び姿勢推定アルゴリズム
ロボットからランドマークまでの相対距離は、例えばカメラを用いた一般的な測距技術によりある程度測定することができる。しかしながら、より正確に距離を測定するためには、ランドマークの画像認識結果に基づいてより厳密な計算処理を行なう必要がある。このとき、ランドマークが床面上に水平に設置されているか、あるいは壁に垂直に貼設されているか、すなわちランドマークの姿勢に応じて計算方法が相違する。
【００８２】
この項では、前項Ｂにおいて説明した色が異なる同心円状の内円と外円からなる平面ランドマークを使用した場合についての、ランドマークの認識及び姿勢推定アルゴリズムについて詳解する。
【００８３】
本実施形態では、以下の手順に従ってランドマークの認識及び姿勢推定を行なう。
【００８４】
（１）カメラによって画像を取得
（２）色抽出
（３）連結領域分割
（４）連結領域からランドマークを認識
（５）すべてのランドマークに対して、
５−ａ）水平（床面）に設置されたランドマークであるとの仮定に基づき、３次元位置を推定
５−ｂ）垂直（壁）に設置されたランドマークであるとの仮定に基づき、３次元位置を推定
５−ｃ）ランドマークが水平又は垂直のいずれの姿勢で設置されているかを判断
【００８５】
カメラでランドマークを捕捉した画像をピクセル単位で色検出することにより、色抽出を行なう。さらに、同じ色からなる領域毎に取り出すことにより、色連結領域を取り出すことができる。
【００８６】
ランドマークの認識過程では、画像を色連結領域に分割することにより、以下の（矩形）領域情報を取得することができる。
【００８７】
ａ）各領域の色
ｂ）領域の左上座標（ｘ’_１，ｙ’_１）と右下座標（ｘ’_２，ｙ’_２）
【００８８】
次いで、すべての領域の組み合わせに対して以下のチェックを行なう。
【００８９】
ａ）一方の領域が他方の領域を内包していること
ｂ）領域の幅の比ｗ_ｌ：ｗ_ｓ（ｗ_ｌ：大きい領域の幅、ｗ_ｓ：小さい領域の幅）が閾値（例えば３：１）を越えないこと
ｃ）領域の高さの比ｗ_ｌ：ｗ_ｓ（ｗ_ｌ：大きい領域の幅、ｗ_ｓ：小さい領域の幅）が閾値（例えば３：１）を越えないこと
【００９０】
そして、上述した条件ａ〜ｃをすべて満たす領域の組み合わせをランドマークとして認識する。色の組み合わせによってランドマークを一意に特定することができる。大きい領域をランドマークのサイズとする。
【００９１】
ここで、ランドマークの姿勢推定処理について説明する前に、本実施形態において使用されるカメラ・モデルについて述べる。
【００９２】
本実施形態では、中心投影を行なうピンホール・カメラを使用するものとする。中心投影とは、投影中心と３次元物体表面の点とを結ぶ直線（「視線」とも言う）とカメラの投影スクリーンとの交点に物体表面の点の色濃度値を配置していくことで、投影画像を形成する。中心投影では、同じ大きさの物体であっても、カメラの投影中心に近づくにつれて大きな像として投影され、逆に、投影中心Ｃから遠ざかるにつれて小さく投影される性質を持つ。
【００９３】
ここで、カメラ中心を原点とし、焦点距離をｆ、光軸中心を（ｐ_ｘ，ｐ_ｙ）、画素サイズを（ｕ_ｘ，ｕ_ｙ）、ランドマーク位置の３次元座標を（ｘ，ｙ，ｚ）、この３次元位置に対応する画素座標を（ｘ’，ｙ’）とすると、幾何光学的に以下の式が見出される。
【００９４】
【数１】

【００９５】
但し、ｆ_ｘ＝ｆ／ｕ_ｘ、ｆ_ｙ＝ｆ／ｕ_ｙと定義する。
【００９６】
また、本実施形態では、カメラは、例えば、２腕２足の直立歩行型ロボットの頭部に搭載されている。そして、カメラの姿勢は図６に示すように、ロール、ピッチ、ヨーの各軸回りの回転位置で表現される。
【００９７】
以下の説明では、カメラがピッチ軸回りに回転して、前方のランドマークを捕捉するものとする。図７に示すように、カメラがピッチ軸回りに所定の回転位置を以って撮影しているとする。このとき、カメラの撮影画像（ｃａｍｅｒａ ｉｍａｇｅ）中に同じ大きさでランドマークを検出したとき（ｐｉｃｔｕｒｅ ｏｆ ｌａｎｄｍａｒｋ）、同図中で参照番号１〜４で示した場合を含め、ランドマークがとりうる姿勢は複数想定される。以下の説明では、ランドマークは、壁又は床面のいずれかにのみ貼設されている、すなわちランドマークは垂直な姿勢１又は水平な姿勢２のいずれか一方であることを前提とする。
【００９８】
カメラによって捕捉されたランドマークの位置（又は、カメラとランドマークの相対距離）は、カメラの姿勢（本実施形態ではピッチ軸周りの回転位置ｐｉｔｃｈ）と、カメラレンズの焦点距離ｆ並びに光学中心（ｐ_ｘ，ｐ_ｙ）などの与えられたパラメータ値と、撮影画像中のランドマークの高さや幅を用いて算出することができる。但し、ランドマークが床面上に水平に設置されているか、あるいは壁に垂直に貼設されているか、すなわちランドマークの姿勢に応じて計算方法が相違する。
【００９９】
図８には、ランドマークが水平に設置されていると仮定した場合に、撮影画像中のランドマークの高さを用いてランドマークの位置を推定する方法を図解している。
【０１００】
図示の例では、カメラはピッチ軸回りに所定の回転位置ｐｉｔｃｈを以ってランドマークを撮影している。３次元空間上でランドマークは、点（ｙ_１，ｚ_１）と点（ｙ_２，ｙ_２）の間に存在し、その中心位置は（ｙ_ｖ，ｚ_ｖ）である。そして、ランドマークは撮影画像上では、高さ方向に点（ｙ_２’，ｆ_ｙ）と点（ｙ_１’，ｆ_ｙ）の間に存在し、投影平面上に射影された像の高さは点（ｙ_２，ｚ_２）と点（ｙ_３，ｚ_２）を結ぶ線分に等しい。この場合、幾何光学的に以下の式が成り立つ。
【０１０１】
【数２】

【０１０２】
そして、ランドマーク中心の３次元位置は下式のように表される。
【０１０３】
【数３】

【０１０４】
また、図９には、ランドマークが垂直に設置されていると仮定した場合に、撮影画像中のランドマークの高さを用いてランドマークの位置を推定する方法を図解している。
【０１０５】
図示の例では、カメラはピッチ軸回りに所定の回転位置ｐｉｔｃｈを以ってランドマークを撮影している。３次元空間上でランドマークは点（ｙ_２，ｚ_２）と点（ｙ_１，ｚ_１）の間に存在し、その中心位置は（ｙ_ｈ，ｚ_ｈ）である。そして、ランドマークは撮影画像上では高さ方向に点（ｙ_２’，ｆ_ｙ）と点（ｙ_１’，ｆ_ｙ）の間に存在し、投影平面上に射影された像の高さは点（ｙ_３，ｚ_１）と点（ｙ_１，ｚ_１）を結ぶ線分に等しい。この場合、幾何光学的に以下の式が成り立つ。
【０１０６】
【数４】

【０１０７】
そして、ランドマーク中心の３次元位置は下式のように表される。
【０１０８】
【数５】

【０１０９】
また、図１０には、ランドマークが水平に設置されていると仮定した場合に、撮影画像中のランドマークの幅を用いてランドマークの位置を推定する方法を図解している。
【０１１０】
図示の例では、カメラは、中心位置（ｘ，ｚ）で幅方向に点（ｘ_１，ｚ）と点（ｘ_２，ｚ）の間に存在するランドマークを撮影している。ランドマークは撮影画像上では幅方向に点（ｘ_１’，ｚ）と点（ｘ_２’，ｚ）の間に存在し、投影平面上に射影された像の幅は、点（ｘ１，ｚ）と点（ｘ２，ｚ）の間を結ぶ線分に等しい。この場合、幾何光学的に以下の式が成り立つ。
【０１１１】
【数６】

【０１１２】
そして、ランドマーク中心の３次元位置は下式のように表される。
【０１１３】
【数７】

【０１１４】
オクルージョンなどがなく水平に設置されたランドマークが完全に見えている場合、撮影画像中のランドマークの高さにより推定されたランドマークまでの相対距離ｚｈと、撮影画像中のランドマークの幅により推定されたランドマークまでの相対距離ｚｗは等しくなる（ｚ_ｗ＝ｚ_ｈ）。
【０１１５】
また、図１１には、ランドマークが垂直に設置されていると仮定した場合に、撮影画像中のランドマークの幅を用いてランドマークの位置を推定する方法を図解している。
【０１１６】
この場合、光軸に対しランドマークが垂直であるときを相対距離が最大として、光軸に対してさまざまな回転位置で設置されているランドマークが、撮影画像中で同じ幅方向の位置すなわち点（ｘ_１，ｆ_ｘ）から点（ｘ_２，ｆ_ｘ）の間に観測される。言い換えれば、光軸に対するランドマークの回転がわからないと、その３次元位置を推定することはできない。
【０１１７】
光軸に対しランドマークが垂直であるとき、ランドマーク中心の３次元位置は上式［数７］と同様となる。そして、オクルージョンなどがなく垂直に設置されたランドマークが完全に見えている場合、撮影画像中のランドマークの幅により推定されたランドマークまでの相対距離ｚ_ｗは、撮影画像中のランドマークの高さにより推定されたランドマークまでの相対距離ｚｖ以上となる（ｚ_ｗ≧ｚ_ｖ）。
【０１１８】
図１０並びに図１１に示した原理に従えば、撮影画像中のランドマークの幅により推定されたランドマークまでの相対距離ｚ_ｗと、撮影画像中のランドマークの高さにより推定されたランドマークまでの相対距離ｚ_ｖとを大小比較することにより、撮影画像中のランドマークが水平又は垂直いずれの姿勢で設置されているかを推定することができる。そして、この推定結果に基づいて、適切なランドマークの位置推定アルゴリズムを選択することができる（後述）。
【０１１９】
ランドマークが水平又は垂直いずれの姿勢で設置されているか応じて計算方法が相違するが、これとは別に、カメラが遠くを見ているときにランドマークを認識した場合と、カメラが近くを見ているときにランドマークを認識した場合とでは、最適な位置計算方法は変わってくる。
【０１２０】
図１２には、カメラが遠くのランドマークを眺めている様子を示している。図示の例では、カメラはピッチ軸回りに所定の回転位置ｐｉｔｃｈを以ってランドマークを撮影している。
【０１２１】
この場合、撮影画像中で同じ高さを持つランドマークが検出されたとすると、水平に設置されたランドマークよりも垂直に設置されたランドマークの方がより遠くに存在することになる。すなわち、カメラが遠くを眺めている場合には、ランドマークが水平に設置されているとの仮定により撮影画像中のランドマークの高さを用いて推定されたランドマークまでの相対距離ｚ_ｈよりも、ランドマークが垂直に設置されているとの仮定により撮影画像中のランドマークの高さを用いて推定されたランドマークまでの相対距離ｚ_ｖの方が大きい値となる（ｚ_ｈ＜ｚ_ｖ）。
【０１２２】
また、図１３には、カメラが近くのランドマークを眺めている様子を示している。図示の例では、カメラはピッチ軸回りに所定の回転位置ｐｉｔｃｈを以ってランドマークを撮影している。
【０１２３】
この場合、撮影画像中で同じ高さを持つランドマークが検出されたとすると、水平に設置されたランドマークよりも垂直に設置されたランドマークの方がより近くに存在することになる。すなわち、カメラが近くを眺めている場合には、ランドマークが水平に設置されているとの仮定により撮影画像中のランドマークの高さを用いて推定されたランドマークまでの相対距離ｚ_ｈよりも、ランドマークが水平に設置されているとの仮定により撮影画像中のランドマークの高さを用いて推定されたランドマークまでの相対距離ｚ_ｖの方が小さい値となる（ｚ_ｈ＞ｚ_ｖ）。
【０１２４】
図１２及び図１３に示した原理に従えば、ランドマークが水平に設置されているとの仮定により撮影画像中のランドマークの高さを用いて推定されたランドマークまでの相対距離ｚ_ｈと、ランドマークが水平に設置されているとの仮定により撮影画像中のランドマークの高さを用いて推定されたランドマークまでの相対距離ｚ_ｖとを大小比較することにより、カメラが遠く又は近くのいずれを眺めているのかを推定することができる。そして、この推定結果に基づいて、適切なランドマークの位置推定アルゴリズムを選択することができる（後述）。
【０１２５】
図１４には、下を見下ろしているカメラが水平並びに垂直に設置されているランドマークを撮影する様子を示している。ここでは、ランドマークの内円の中心位置を（ｘ_ｉｃ，ｙ_ｉｃ）とし、ランドマークの外円の中心位置を（ｘ_ｏｃ，ｙ_ｏｃ）とおく。
【０１２６】
中心投影では、同じ大きさの物体であっても、カメラの投影中心に近づくにつれて大きな像として投影され、逆に、投影中心から遠ざかるにつれて小さく投影される性質を持つ（前述）。
【０１２７】
したがって、図１４からも判るように、ランドマークの内円の中心の高さｙ_ｉｃよりも外円の中心の高さｙ_ｏｃの方が小さい場合には（ｙ_ｉｃ＞ｙ_ｏｃ）、ランドマークは垂直に設置されていると推定される。逆に、ランドマークの内円の中心の高さｙ_ｉｃよりも外円の中心の高さｙ_ｏｃの方が大きい場合には（ｙ_ｉｃ＜ｙ_ｏｃ）、ランドマークは水平に設置されていると推定される。そして、この推定結果に基づいて、適切なランドマークの位置推定アルゴリズムを選択することができる（後述）。
【０１２８】
図１５には、ランドマークの位置推定アルゴリズムの一例をフローチャートの形式で示している。また、この位置推定アルゴリズムの擬似プログラム・コードを以下に示している。但し、ここではランドマーク全体がオクルージョンなしに見えていると仮定する。
【０１２９】
【数８】

【０１３０】
まず、ステップＳ１において、ランドマークが水平に設置されていると仮定して、撮影画像中のランドマークの高さを用いてランドマークの位置を推定する（図８を参照のこと）。
【０１３１】
次いで、ステップＳ２において、ランドマークが垂直に設置されていると仮定して、撮影画像中のランドマークの高さを用いてランドマークの位置を推定する（図９を参照のこと）。
【０１３２】
さらに、ステップＳ３において、ランドマークが水平に設置されていると仮定して、撮影画像中のランドマークの幅を用いてランドマークの位置を推定する（図１０を参照のこと）。
【０１３３】
ここで、ランドマークが水平に設置されているとの仮定により撮影画像中のランドマークの高さを用いて推定されたランドマークまでの相対距離ｚ_ｈと、ランドマークが垂直に設置されているとの仮定により撮影画像中のランドマークの高さを用いて推定されたランドマークまでの相対距離ｚ_ｖとを大小比較することにより、カメラが遠く又は近くのいずれを眺めているのかを推定する（ステップＳ４）（図１２及び図１３を参照のこと）。
【０１３４】
図１５に示す位置推定アルゴリズムはカメラが遠くのランドマークを眺めているときには適しているが、近くのランドマークを眺めているときに配置を正確に推定することができない。このため、ランドマークが水平に設置されているとの仮定により撮影画像中のランドマークの高さを用いて推定されたランドマークまでの相対距離ｚ_ｈよりも、ランドマークが垂直に設置されているとの仮定により撮影画像中のランドマークの高さを用いて推定されたランドマークまでの相対距離ｚ_ｖの方が小さい値となる場合には（ｚ_ｈ＞ｚ_ｖ）、カメラが近くを眺めているので、答えが出ない（Ｎｏ Ｒｅｓｕｌｔ）として本処理ルーチン全体を終了する（ステップＳ５）。
【０１３５】
一方、ランドマークが水平に設置されているとの仮定により撮影画像中のランドマークの高さを用いて推定されたランドマークまでの相対距離ｚ_ｈよりも、ランドマークが水平に設置されているとの仮定により撮影画像中のランドマークの高さを用いて推定されたランドマークまでの相対距離ｚ_ｖの方が大きい値となる場合には（ｚ_ｈ＜ｚ_ｖ）、カメラが遠くを眺めていると推定される。この場合、さらに、撮影画像中のランドマークの幅により推定されたランドマークまでの相対距離ｚ_ｗと、ランドマークが垂直に設置されていると仮定して撮影画像中のランドマークの高さにより推定されたランドマークまでの相対距離ｚ_ｖとを大小比較することにより、撮影画像中のランドマークが水平又は垂直いずれの姿勢で設置されているかを推定する（ステップＳ６）（図１１を参照のこと）。
【０１３６】
そして、ｚ_ｗ＞ｚ_ｖとなる場合には、ランドマークが垂直に設置されていると推定するとともに、ステップＳ２においてランドマークが垂直に設置されていると仮定して、撮影画像中のランドマークの高さを用いて推定されたランドマークの位置（ｘ_ｖ，ｙ_ｖ，ｚ_ｖ）を推定結果として出力する（ステップＳ７）。
【０１３７】
一方、ｚ_ｗ≦ｚ_ｖとなる場合には、ランドマークが水平に設置されている可能性とカメラに対して回転していない垂直な姿勢で設置されている可能性が残されている。そこで、ランドマークが水平に設置されていると仮定して撮影画像中のランドマークの高さにより推定されたランドマークまでの相対距離ｚ_ｈと撮影画像中のランドマークの幅により推定されたランドマークまでの相対距離ｚ_ｗとの差分｜ｚ_ｈ−ｚ_ｗ｜と、ランドマークが垂直に設置されていると仮定して撮影画像中のランドマークの高さにより推定されたランドマークまでの相対距離ｚ_ｖと撮影画像中のランドマークの幅により推定されたランドマークまでの相対距離ｚ_ｗとの差分｜ｚ_ｖ−ｚ_ｗ｜とを比較して（ステップＳ８）、ランドマークが垂直又は水平いずれに設置されているかどうかを判別する。
【０１３８】
｜ｚ_ｖ−ｚ_ｗ｜の方が小さい場合には、ランドマークはカメラに対して回転していない垂直な姿勢で設置されていると推定される。この場合、ステップＳ２においてランドマークが垂直に設置されていると仮定して、撮影画像中のランドマークの高さを用いて推定されたランドマークの位置（ｘ_ｖ，ｙ_ｖ，ｚ_ｖ）を推定結果として出力する（ステップＳ９）。
【０１３９】
一方、｜ｚ_ｈ−ｚ_ｗ｜の方が小さい場合には、ランドマークは水平に設置されていると判断する。そして、ステップＳ１においてランドマークが水平に設置されていると仮定して、撮影画像中のランドマークの高さを用いて推定されたランドマークの位置（ｘ_ｈ，ｙ_ｈ，ｚ_ｈ）を推定結果として出力する（ステップＳ１０）。
【０１４０】
図１５に示したランドマークの位置推定アルゴリズムは、カメラが遠くを眺めている場合（図１２を参照のこと）に適している。なお、ランドマークの一部が隠れている場合（オクルージョン）、結果が異なることがある。また、ランドマークが垂直又は水平いずれにも設置されていない場合にも、結果が異なることがある。また、カメラの回転（ロール軸回り）に関しても、式が多少変わるが、同様の結果を得ることができる。
【０１４１】
また、図１６には、ランドマークの位置推定アルゴリズムについての他の例をフローチャートの形式で示している。また、この位置推定アルゴリズムの擬似プログラム・コードを以下に示している。但し、ここではランドマーク全体がオクルージョンなしに見えていると仮定する。
【０１４２】
【数９】

【０１４３】
まず、ステップＳ１１において、ランドマークが水平に設置されていると仮定して、撮影画像中のランドマークの高さを用いてランドマークの位置を推定する（図８を参照のこと）。
【０１４４】
次いで、ステップＳ１２において、ランドマークが垂直に設置されていると仮定して、撮影画像中のランドマークの高さを用いてランドマークの位置を推定する（図９を参照のこと）。
【０１４５】
さらに、ステップＳ１３において、ランドマークが水平に設置されていると仮定して、撮影画像中のランドマークの幅を用いてランドマークの位置を推定する（図１０を参照のこと）。
【０１４６】
カメラが水平並びに垂直に設置されているランドマークを見下ろして撮影した場合、ランドマークの内円の中心の高さｙ_ｉｃよりも外円の中心の高さｙ_ｏｃの方が小さいときには（ｙ_ｉｃ＞ｙ_ｏｃ）、ランドマークは垂直に設置されていると推定され、ランドマークの内円の中心の高さｙ_ｉｃよりも外円の中心の高さｙ_ｏｃの方が大きいときには（ｙ_ｉｃ＜ｙ_ｏｃ）、ランドマークは垂直に設置されていると推定される（図１４を参照のこと）。
【０１４７】
そこで、ステップＳ１４では、撮影画像中のランドマークの内円の高さｙ_ｉｃと外円の高さｙ_ｉｃを大小比較して、ランドマークが水平又は垂直いずれの姿勢で設置されているかを判別する。
【０１４８】
ここで、ランドマークの内円の中心の高さｙ_ｉｃよりも外円の中心の高さｙ_ｏｃの方が大きい場合には（ｙ_ｉｃ＜ｙ_ｏｃ）、ランドマークは水平に設置されていると推定される。
【０１４９】
この場合、さらに、ランドマークが水平に設置されているとの仮定により撮影画像中のランドマークの高さを用いて推定されたランドマークまでの相対距離ｚ_ｈと、撮影画像中のランドマークの幅を用いて推定されたランドマークまでの相対距離ｚ_ｗとを大小比較する（ステップＳ１５）。
【０１５０】
そして、ｚ_ｈがｚ_ｗよりも大きくない場合には、ステップＳ１１においてランドマークが水平に設置されているとの仮定により撮影画像中のランドマークの高さを用いて推定されたランドマークの位置（ｘ_ｈ，ｙ_ｈ，ｚ_ｈ）を、ランドマークが水平に設置されているという推定結果とともに出力する（ステップＳ１６）。
【０１５１】
一方、ｚ_ｈがｚ_ｗよりも大きい場合には、オクルージョンが大きく、誤認識の可能性が高いと推定されることから、より近くに見える推定方法である撮影画像中のランドマークの幅を用いて推定されたランドマークの位置（ｘ_ｗ，ｙ_ｗ，ｚ_ｗ）を、ランドマークが水平に設置されているという推定結果とともに出力する（ステップＳ１７）。
【０１５２】
また、ランドマークの内円の中心の高さｙ_ｉｃよりも外円の中心の高さｙ_ｏｃの方が小さい場合には（ｙ_ｉｃ＞ｙ_ｏｃ）、ランドマークは垂直に設置されていると推定される。
【０１５３】
この場合、さらに、ランドマークが垂直に設置されているとの仮定により撮影画像中のランドマークの高さを用いて推定されたランドマークまでの相対距離ｚ_ｖと、撮影画像中のランドマークの幅を用いて推定されたランドマークまでの相対距離ｚ_ｗとを大小比較する（ステップＳ１８）。
【０１５４】
そして、ｚ_ｖがｚ_ｗよりも大きくない場合には、ステップＳ１２においてランドマークが垂直に設置されていると仮定して、撮影画像中のランドマークの高さを用いて推定されたランドマークの位置（ｘ_ｖ，ｙ_ｖ，ｚ_ｖ）を、ランドマークが垂直に設置されているという推定結果とともに出力する（ステップＳ１９）。
【０１５５】
一方、ｚ_ｖがｚ_ｗよりも大きい場合には、オクルージョンや視線に対するランドマークの傾きが大きく、誤認識の可能性が高いと推定されることから、より近くに見える推定方法である撮影画像中のランドマークの幅を用いて推定されたランドマークの位置（ｘ_ｗ，ｙ_ｗ，ｚ_ｗ）を、ランドマークが垂直に設置されているという推定結果とともに出力する（ステップＳ２０）。
【０１５６】
図１６に示したランドマークの位置推定アルゴリズムは、カメラが遠くを眺めている場合（図１３を参照のこと）に適している。なお、ランドマークの一部が隠れている場合、領域情報の不整合により、回転方向の推定が異なることがある。また、ランドマークが垂直又は水平いずれにも設置されていない場合にも、結果が異なることがある。また、カメラの回転（ロール軸回り）に関しても、式が多少変わるが、同様の結果を得ることができる。図１６に示したアルゴリズムは、画像に占めるランドマークの割合が大きいときに特に有効であるが、割合が小さい場合（ランドマークが著しく遠い場合や、ランドマークとカメラの支線のなす角が小さい場合）には結果が異なることがある。
【０１５７】
図１５に示したランドマークの位置推定アルゴリズムはカメラが遠くのランドマークを認識した場合に適しており、さらにランドマークが水平又は垂直いずれに設置されているかに応じて最適なランドマークの位置推定方法を選択するようになっている。同様に、図１６に示したランドマークの位置推定アルゴリズムはカメラが近くのランドマークを認識した場合に適しており、さらにランドマークが水平又は垂直いずれに設置されているかに応じて最適なランドマークの位置推定方法を選択するようになっている。
【０１５８】
図１７には、さらにカメラが遠く又は知覚いずれのランドマークを認識しているかに応じて、図１５に示したアルゴリズム又は図１６に示したアルゴリズムの一方を自動的に選択することができるランドマークの位置推定アルゴリズムをフローチャートの形式で示している。また、この位置推定アルゴリズムの擬似プログラム・コードを以下に示している。
【０１５９】
【数１０】

【０１６０】
まず、ステップＳ２１において、ランドマークが水平に設置されていると仮定して、撮影画像中のランドマークの高さを用いてランドマークの位置を推定する（図８を参照のこと）。
【０１６１】
次いで、ステップＳ２２において、ランドマークが垂直に設置されていると仮定して、撮影画像中のランドマークの高さを用いてランドマークの位置を推定する（図９を参照のこと）。
【０１６２】
さらに、ステップＳ２３において、ランドマークが水平に設置されていると仮定して、撮影画像中のランドマークの幅を用いてランドマークの位置を推定する（図１０を参照のこと）。
【０１６３】
ここで、ランドマークが水平に設置されているとの仮定により撮影画像中のランドマークの高さを用いて推定されたランドマークまでの相対距離ｚ_ｈと、ランドマークが垂直に設置されているとの仮定により撮影画像中のランドマークの高さを用いて推定されたランドマークまでの相対距離ｚ_ｖとを大小比較することにより、カメラが遠く又は近くのいずれを眺めているのかを推定する（ステップＳ２４）（図１２及び図１３を参照のこと）。
【０１６４】
ランドマークが水平に設置されているとの仮定により撮影画像中のランドマークの高さを用いて推定されたランドマークまでの相対距離ｚ_ｈよりも、ランドマークが水平に設置されているとの仮定により撮影画像中のランドマークの高さを用いて推定されたランドマークまでの相対距離ｚ_ｖの方が大きい値となる場合には（ｚ_ｈ＜ｚ_ｖ）、カメラが遠くを眺めていると推定される。この場合、図１５に示したランドマークの位置推定アルゴリズム（但し、ステップＳ４以降）を実行する（ステップＳ２５）。
【０１６５】
一方、ランドマークが水平に設置されているとの仮定により撮影画像中のランドマークの高さを用いて推定されたランドマークまでの相対距離ｚ_ｈよりも、ランドマークが垂直に設置されているとの仮定により撮影画像中のランドマークの高さを用いて推定されたランドマークまでの相対距離ｚ_ｖの方が小さい値となる場合には（ｚ_ｈ＞ｚ_ｖ）、カメラが近くを眺めていると推定される。この場合、図１６に示したランドマークの位置推定アルゴリズム（但し、ステップＳ１４以降）を実行する（ステップＳ２６）。
【０１６６】
なお、ランドマークの一部が隠れている場合（オクルージョン）、結果が異なることがある。また、ランドマークが垂直又は水平いずれにも設置されていない場合にも、結果が異なることがある。また、カメラの回転（ロール軸回り）に関しても、式が多少変わるが、同様の結果を得ることができる。
【０１６７】
［追補］
以上、特定の実施例を参照しながら、本発明について詳解してきた。しかしながら、本発明の要旨を逸脱しない範囲で当業者が該実施例の修正や代用を成し得ることは自明である。
【０１６８】
本発明の要旨は、必ずしも「ロボット」と称される製品には限定されない。すなわち、電気的若しくは磁気的な作用を用いて人間の動作に似せた運動を行なう機械装置あるいはその他一般的な移動体装置、あるいはこれら装置の動作を記述したデータを演算処理するデータ処理システムであるならば、例えば玩具などのような他の産業分野に属する製品であっても、同様に本発明を適用することができる。
【０１６９】
要するに、例示という形態で本発明を開示してきたのであり、本明細書の記載内容を限定的に解釈するべきではない。本発明の要旨を判断するためには、冒頭に記載した特許請求の範囲の欄を参酌すべきである。
【０１７０】
【発明の効果】
以上詳記したように、本発明によれば、人工的なランドマークを含む環境において、ランドマークの探索結果を手掛かりに自己位置を同定することができる、優れたロボットを提供することができる。
【０１７１】
また、本発明によれば、床や壁に設置されている平面的なランドマークを設置された姿勢に応じて位置を好適に推定することができる、優れたロボット、並びにランドマークの姿勢推定システム及び推定方法を提供することができる。
【０１７２】
また、本発明によれば、好適な形状パターン並びに色パターンからなる平面的なランドマークを設置された姿勢に応じて位置を正確に推定することができる、優れたロボット、並びにランドマークの姿勢推定システム及び推定方法を提供することができる。
【図面の簡単な説明】
【図１】本発明に実施に供される移動ロボット１の機能構成を模式的に示した図である。
【図２】制御ユニット２０の構成をさらに詳細に示した図である。
【図３】本発明の実施に供される平面ランドマークの構成を模式的に示した図である。
【図４】平面ランドマークの認識率を測定する実験環境を模式的に示した図である。
【図５】本発明の実施形態において使用されるカメラ・モデルを説明するための図である。
【図６】カメラの姿勢の表現方法を説明するための図である。
【図７】カメラがピッチ軸回りに所定の回転位置を以って撮影している様子を示した図である。
【図８】ランドマークが水平に設置されていると仮定した場合に、撮影画像中のランドマークの高さを用いてランドマークの位置を推定する方法を説明するための図である。
【図９】ランドマークが垂直に設置されていると仮定した場合に、撮影画像中のランドマークの高さを用いてランドマークの位置を推定する方法を説明するための図である。
【図１０】ランドマークが水平に設置されていると仮定した場合に、撮影画像中のランドマークの幅を用いてランドマークの位置を推定する方法を説明するための図である。
【図１１】ランドマークが垂直に設置されていると仮定した場合に、撮影画像中のランドマークの幅を用いてランドマークの位置を推定する方法を説明するための図である。
【図１２】カメラが遠くのランドマークを眺めている様子を示した図である。
【図１３】カメラが近くのランドマークを眺めている様子を示した図である。
【図１４】下を見下ろしているカメラが水平並びに垂直に設置されているランドマークを撮影する様子を示した図である。
【図１５】ランドマークの位置推定アルゴリズムの一例を示したフローチャートである。
【図１６】ランドマークの位置推定アルゴリズムについての他の例を示したフローチャートである。
【図１７】ランドマークの位置推定アルゴリズムについての他の例を示したフローチャートである。
【符号の説明】
１…移動ロボット
１５…ＣＣＤカメラ
１６…マイクロフォン
１７…スピーカ
１８…タッチ・センサ
１９…ＬＥＤインジケータ
２０…制御部
２１…ＣＰＵ
２２…ＲＡＭ
２３…ＲＯＭ
２４…不揮発メモリ
２５…インターフェース
２６…無線通信インターフェース
２７…ネットワーク・インターフェース・カード
２８…バス
２９…キーボード
４０…入出力部
５０…駆動部
５１…モータ
５２…エンコーダ
５３…ドライバ[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a robot that identifies its own position based on sensor information of the robot and operation information of the robot itself. In particular, in an environment including artificial landmarks, the self-position is identified based on a search result of a landmark. About a robot to do.
[0002]
More specifically, the present invention relates to a robot for estimating a position of a planar landmark installed on a floor or a wall in accordance with an installed attitude, and a position estimation system and an estimation method of a landmark, and has a suitable shape. The present invention relates to a robot for accurately estimating a position according to a posture in which a planar landmark including a pattern and a color pattern is installed, and a position estimation system and a position estimation method for a landmark.
[0003]
[Prior art]
A mechanical device that performs a motion resembling a human motion using an electric or magnetic action is called a “robot”. It is said that the robot is derived from the Slavic word "ROBOTA (slave machine)". In Japan, robots began to spread from the late 1960's, but most of them were industrial robots (industrial robots) such as manipulators and transfer robots for the purpose of automation and unmanned production work in factories. Met.
[0004]
Recently, various types of legged locomotion such as "humanoid" or "humanoid" robots designed based on the body mechanisms and movements of animals such as humans who walk upright on two legs have been designed. Research and development on robots has progressed, and expectations for practical use have been increasing.
[0005]
Most of the working space and living space of human beings are formed according to the human body mechanism and behavior style of bipedal upright walking, and the current mechanical system using wheels and other driving devices as moving means will move. There are many barriers. Therefore, in order for the mechanical system, that is, the robot, to perform various human tasks and penetrate deep into the human living space, it is preferable that the movable range of the robot is almost the same as that of a human. This is the reason why practical use of the legged mobile robot is greatly expected.
[0006]
Such a humanoid robot can be applied to various types of operations in industrial activities and production activities. For example, sites where humans cannot easily step into, such as maintenance work in nuclear power plants, thermal power plants, petrochemical plants, transport and assembly of parts in manufacturing factories, cleaning in high-rise buildings, rescue in fire sites, etc. Have a humanoid robot take over dangerous and difficult tasks at the site.
[0007]
Further, other uses of the humanoid robot include what is called "symbiosis" for making the living space the same as a human, and what is called "entertainment" when youth. In this type of application, the robot is installed and used in the same working environment as a human such as a home environment. In this case, the character of close contact with life is more prominent than the life support such as work substitution.
[0008]
Intelligent mobile robots can autonomously search for a route in a given work space and take free actions. At this time, the identification of the position of the robot is an extremely important technique for the robot to perform services such as delivery. Because if you make a mistake in getting to the target point, it is inefficient, even if you can recover, and may even lead to dangers such as colliding with obstacles and entering dangerous areas. Because.
[0009]
One of the typical methods for a robot to perform self-identification is to use a “landmark”. Usually, visually recognizable information is formed on the surface of the landmark. The robot can geographically search for its own position based on the relative position information from the visually recognized landmark.
[0010]
A landmark that has been used as a conventional landmark is one that has a visible pattern written on a flat plate, a tape for visual recognition attached to a wall, floor, ceiling, etc. A tape is stretched around the building premises along a desired path), or a three-dimensional object is installed in the work environment.
[0011]
Such a planar landmark may be attached to a floor or a wall, for example. For example, when considering the use of landmarks in an environment where installation space is limited, such as in a general home, specify the location of the landmark on either the floor or the wall, or be sure to use a unique posture for the landmark. It is impossible to install.
[0012]
The relative distance from the robot to the landmark can be measured to some extent by, for example, a general ranging technique using a camera. However, in order to measure the distance more accurately, it is necessary to perform more strict calculation processing based on the image recognition result of the landmark. At this time, the method of calculating the position of the landmark differs depending on whether the landmark is installed horizontally on the floor or vertically attached to the wall, that is, depending on the posture of the landmark. Therefore, in order for the mobile robot to more accurately identify its own position in accordance with the landmark, it is necessary to first estimate the attitude of the found landmark.
[0013]
For example, a technology for identifying landmarks from the surrounding environment (see, for example, Patent Document 1), a robot device for providing an identifier of a different color pattern on an object, and detecting a position so as to detect the color pattern by image processing Some proposals have been made (see, for example, Patent Document 2), but they all disclose that the position is accurately estimated according to the attitude at which the landmark is installed. Absent.
[0014]
[Patent Document 1]
U.S. Pat. No. 5,957,984
[Patent Document 2]
U.S. Pat. No. 6,088,469
[0015]
[Problems to be solved by the invention]
An object of the present invention is to provide an excellent robot which can identify a self-position in an environment including an artificial landmark based on a landmark search result.
[0016]
A further object of the present invention is to provide an excellent robot, and a landmark posture estimation system and estimation that can appropriately estimate a position according to a posture in which a planar landmark installed on a floor or a wall is installed. It is to provide a method.
[0017]
A further object of the present invention is to provide an excellent robot capable of accurately estimating a position in accordance with a posture in which a planar landmark including a suitable shape pattern and a color pattern is installed, and a landmark posture estimation system. And an estimation method.
[0018]
Means and Action for Solving the Problems
The present invention has been made in view of the above problems, a landmark position estimation system for estimating the position according to the orientation of the landmark installed,
Image holding means for holding a captured image of the landmark;
First position calculating means for calculating the position of the landmark based on the height component of the landmark in the captured image, assuming that the landmark is installed horizontally;
Second position calculating means for calculating the position of the landmark based on the height component of the landmark in the captured image, assuming that the landmark is installed vertically;
Third position calculating means for calculating the position of the landmark based on the width component of the landmark in the captured image;
Attitude estimating means for estimating whether the landmark is installed in a horizontal or vertical orientation,
A selection output unit that selects one of the calculation results by the first to third position calculation units in accordance with the estimated posture of the landmark;
It is a landmark position estimation system characterized by comprising:
[0019]
However, the term “system” as used herein refers to a logical collection of a plurality of devices (or functional modules that realize specific functions), and each device or functional module is in a single housing. It does not matter in particular.
[0020]
The relative distance from the robot to the landmark can be measured to some extent by, for example, a general ranging technique using a camera. However, in order to measure the distance more accurately, it is necessary to perform more strict calculation processing based on the image recognition result of the landmark. At this time, the calculation method differs depending on whether the landmark is placed horizontally on the floor or vertically attached to the wall, that is, depending on the posture of the landmark.
[0021]
According to the present invention, it is possible to estimate whether a landmark in a captured image is installed in a horizontal or vertical orientation, and to select an appropriate landmark position estimation algorithm based on the estimation result.
[0022]
Here, the relative distance z to the landmark estimated using the height of the landmark in the photographed image by the first position calculating means on the assumption that the landmark is installed horizontally is assumed._hAnd the relative distance z to the landmark estimated using the height of the landmark in the captured image by the second position calculating means on the assumption that the landmark is installed vertically._vAnd z, z_h> Z_vWhen it is determined that the captured image has been captured by the imaging means looking close,_h<Z_vIn the case of, it can be determined that the captured image has been captured by a captured image looking far away.
[0023]
In such a case, the selection output means may select any one of the calculation results by the first to third position calculation means depending on whether the captured image is viewed near or far. Good.
[0024]
Further, the attitude estimating means calculates a relative distance z to the landmark estimated based on the width of the landmark in the captured image._wAnd a relative distance z to the landmark estimated based on the height of the landmark in the captured image, assuming that the landmark is installed vertically._vAnd z, z_w> Z_vIn this case, it can be estimated that the landmark is installed vertically.
[0025]
The selection output means can output a position calculation result by the second position calculation means in response to the fact that the landmark is estimated to be installed vertically.
[0026]
Further, the attitude estimating means is z_w≦ z_vIn this case, there is a possibility that the landmark is installed horizontally and that the landmark is installed in a vertical posture that is not rotated with respect to the camera.
[0027]
Therefore, the attitude estimating means assumes that the landmark is installed horizontally, and determines the relative distance z to the landmark estimated based on the height of the landmark in the captured image._hAnd the relative distance z to the landmark estimated from the width of the landmark in the captured image_w| Z_h-Z_w| And the relative distance z to the landmark estimated from the height of the landmark in the captured image, assuming that the landmark is installed vertically_vAnd the relative distance z to the landmark estimated from the width of the landmark in the captured image_w| Z_v-Z_wIs compared with | to determine whether the landmark is installed vertically or horizontally.
[0028]
And | z_v-Z_wWhen | is smaller, it can be estimated that the landmark is installed in a vertical posture that is not rotated with respect to the camera. The selection output means may output the calculation result of the second position calculation means in response to this.
[0029]
Or | z_h-Z_wWhen | is smaller, it can be estimated that the landmark is installed horizontally. The selection output means may output the calculation result of the first position calculation means in response to this.
[0030]
The landmark has, for example, a radius d_iAnd the inner circle has a radius d substantially concentrically arranged._oAnd an outer circle.
[0031]
In such a case, the posture estimating means calculates the height y of the inner circle of the landmark in the captured image._icAnd the height y of the outer circle_ocAre compared, and the height y of the center of the inner circle of the landmark is calculated._icHeight of the center of the outer circle than_ocIs smaller than (y_ic> Y_oc) It is assumed that the landmark is installed vertically, and the height y of the center of the inner circle of the landmark is assumed._icHeight of the center of the outer circle than_ocIs larger (y_ic<Y_oc) It can be assumed that the landmark is installed vertically.
[0032]
The attitude estimating means calculates the height y of the center of the inner circle of the landmark._icHeight of the center of the outer circle than_ocIs smaller than (y_ic> Y_oc), The relative distance z to the landmark estimated using the height of the landmark in the captured image on the assumption that the landmark is installed vertically_vAnd the relative distance z to the landmark estimated using the width of the landmark in the captured image_wMay be further compared in size.
[0033]
And, the selection output means, z_vIs z_wIf it is not larger, the calculation result by the second position calculation means is output and z_vIs z_wIf it is larger than this, it is estimated that the inclination of the landmark with respect to the occlusion and the line of sight is large and the possibility of erroneous recognition is high. .
[0034]
Further, the attitude estimating means calculates the height y of the center of the inner circle of the landmark._icHeight of the center of the outer circle than_ocIs larger (y_ic<Y_oc), The relative distance z to the landmark estimated using the height of the landmark in the captured image on the assumption that the landmark is installed horizontally._hAnd the relative distance z to the landmark estimated using the width of the landmark in the captured image_wMay be further compared in size.
[0035]
And, the selection output means, z_hIs z_wIf it is not larger, the calculation result by the first position calculation means is output and z_hIs z_wIf it is larger, it is presumed that the occlusion is large and the possibility of erroneous recognition is high, so the calculation result by the third position calculation means may be output.
[0036]
A second aspect of the present invention is a landmark for a robot to perform environment recognition based on visual recognition,
Radius d arranged on a plane_iAnd the inner circle of
The inner circle is a radius d substantially concentrically arranged._oAnd the outer circle (where d_i<D_o),
It is a landmark characterized by having.
[0037]
The planar landmark according to the second aspect of the present invention has a radius d_iAnd an inner circle having a radius d substantially concentrically arranged._o(Except for d_i<D_o). The inner circle and the outer circle are applied to colors different from each other, and are more preferably provided with a color arrangement that is easily separated in a UV color space.
[0038]
In order to further enhance the discrimination between the inner circle and the outer circle, the width w_iAre provided. In addition, in order to ensure the discrimination and separation between the flat landmark and the floor or wall on which the flat landmark is affixed, the width w_oAre provided. Both the inner and outer boundaries may be black.
[0039]
The following parameters can be used to improve the performance of recognizing the inner circle and the outer circle, and the outer circle and the landmark installation surface.
[0040]
0mm <w_i<10mm
0mm <w_o<50mm
[0041]
Further, in order to improve the recognition performance of the outer circle and the inner circle, it is more preferable to set the ratio between the outer circle radius and the inner circle radius in the following range.
[0042]
3: 4 <d_i: D_o<1: 3
[0043]
For example, d_i= 60mm, d_o= 120mm, w_i= 2mm, w_o= 20, then d_i: D_o= 1: 2, which satisfies the above condition.
[0044]
Further objects, features, and advantages of the present invention will become apparent from more detailed descriptions based on embodiments of the present invention described below and the accompanying drawings.
[0045]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
[0046]
A. Configuration of mobile robot
FIG. 1 schematically shows a functional configuration of a mobile robot 1 used in the present invention. As shown in FIG. 1, the mobile robot 1 includes a control unit 20 that performs overall control of the entire operation and other data processing, an input / output unit 40, a driving unit 50, and a power supply unit 60. . Hereinafter, each unit will be described.
[0047]
The input / output unit 40 includes a CCD camera 15 corresponding to an eye of the mobile robot 1 as an input unit, a microphone 16 corresponding to an ear, and a touch sensor disposed at a part such as a head or a back to detect a user's contact. 18, including a pawl switch disposed on each sole or other various sensors corresponding to the five senses. The output unit is provided with a speaker 17 corresponding to a mouth, an LED indicator (eye lamp) 19 that forms a facial expression by a combination of blinking and lighting timing. These output units can express the user feedback from the mobile robot 1 in a form other than a mechanical movement pattern by a leg or the like.
[0048]
The CCD camera 15 can function as, for example, external observation means for observing landmarks arranged at various parts in the work space. Of course, in addition to (or in addition to) the CCD camera 15, a range finder or the like may be provided as external observation means.
[0049]
The drive unit 50 is a functional block that implements the body operation of the robot device according to a predetermined movement pattern commanded by the control unit 20, and is a control target by behavior control. The drive unit 50 is a functional module for realizing a degree of freedom at each joint of the robot device 00, and is configured by a plurality of drive units provided for each joint axis such as roll, pitch, and yaw at each joint. Each drive unit performs a rotation operation around a predetermined axis, an encoder 52 that detects a rotation position of the motor 51, and adaptively controls a rotation position and a rotation speed of the motor 51 based on an output of the encoder 52. It is composed of a combination of drivers 53.
[0050]
The hardware configuration is determined by the combination of the drive units. For example, the robot device may be configured as a legged mobile robot such as bipedal walking or quadrupedal walking, or may be configured as a tired robot.
[0051]
The attitude of the camera 15 can be specified by kinematics based on the output from the encoder 52 provided for each drive unit.
[0052]
The power supply unit 60 is a functional module that supplies power to each electric circuit and the like in the mobile robot 1, as the name implies. The mobile robot 1 according to the present embodiment is of an autonomous driving type using a battery, and a power supply unit 60 includes a charge battery 61 and a charge / discharge control unit 62 that manages a charge / discharge state of the charge battery 61. .
[0053]
The rechargeable battery 61 is configured, for example, in the form of a “battery pack” in which a plurality of nickel-cadmium battery cells are packaged in a cartridge type.
[0054]
Further, the charge / discharge control unit 62 grasps the remaining capacity of the battery 61 by measuring a terminal voltage and a charge / discharge current amount of the battery 61, an ambient temperature of the battery 61, and determines a start time and an end time of charging. The decision is made. The start and end timings of charging determined by the charge / discharge control unit 62 are notified to the control unit 20, and serve as triggers for the mobile robot 1 to start and end charging operations.
[0055]
The control unit 20 corresponds to a “brain”, and is mounted on, for example, the head or the body of the mobile robot 1.
[0056]
FIG. 2 illustrates the configuration of the control unit 20 in more detail. As shown in FIG. 1, the control unit 20 has a configuration in which a CPU (Central Processing Unit) 21 as a main controller is bus-connected to a memory and other circuit components and peripheral devices. The bus 27 is a common signal transmission path including a data bus, an address bus, a control bus, and the like. Each device on the bus 27 is assigned a unique address (memory address or I / O address). The CPU 21 can communicate with a specific device on the bus 28 by specifying an address.
[0057]
A RAM (Random Access Memory) 22 is a writable memory including a volatile memory such as a DRAM (Dynamic RAM), and loads a program code executed by the CPU 21 or temporarily stores work data by the execution program. Used to save and so on.
[0058]
The ROM (Read Only Memory) 23 is a read-only memory that permanently stores programs and data. The program code stored in the ROM 23 includes a self-diagnosis test program executed when the power of the mobile robot 1 is turned on, an operation control program for defining the operation of the mobile robot 1, and the like.
[0059]
The control program of the robot 1 includes a “sensor input processing program” for processing sensor inputs of the camera 15 and the microphone 16 and the like, and generates an action, that is, a motion pattern of the mobile robot 1 based on the sensor inputs and a predetermined operation model. An action command program, a "drive control program" for controlling the driving of each motor and the sound output of the speaker 17 according to the generated motion pattern, and a geographical location of one or more landmarks arranged in the work space. "Self-location identification program" for searching for a suitable location. The self-position identification program according to the present embodiment includes an algorithm for estimating a posture of a landmark discovered and recognized based on an image captured by the camera 15, a position calculation algorithm corresponding to the posture of the landmark, and the like. ing.
[0060]
The non-volatile memory 24 is composed of a memory element that can be electrically erased and rewritten, such as an EEPROM (Electrically Erasable and Programmable ROM), and is used to hold data to be sequentially updated in a non-volatile manner. The data to be sequentially updated includes security information such as a serial number and an encryption key.
[0061]
The interface 25 is a device for interconnecting with devices outside the control unit 20 and enabling data exchange. The interface 25 performs data input / output with the camera 15, the microphone 16, and the speaker 17, for example. Also, the interface 25 inputs and outputs data and commands to and from each of the drivers 53-1 in the drive unit 50.
[0062]
The interface 25 includes a serial interface such as a RS (Recommended Standard) -232C, a parallel interface such as an IEEE (Institute of Electrical and Electronics Engineers) 1284, a USB (Universal Serial I / E) serial interface, and a USB (Universal Serial I / E) serial bus. , A general-purpose interface for connecting peripheral devices of a computer, such as a SCSI (Small Computer System Interface) interface, a memory card interface (card slot) for receiving a memory stick, and the like. Professional among The movement of the gram or data may be performed.
[0063]
As another example of the interface 25, an infrared communication (IrDA) interface may be provided to perform wireless communication with an external device.
[0064]
Further, the control unit 20 includes a wireless communication interface 26, a network interface card (NIC) 27, and the like, and performs near field wireless data communication such as Bluetooth, a wireless network such as IEEE 802.11b, or a wide area such as the Internet. Data communication can be performed with various external host computers via the network.
[0065]
By such data communication between the mobile robot 1 and the host computer, complicated operation control of the mobile robot 1 can be calculated or remotely controlled using remote computer resources.
[0066]
B. Composition of plane landmark
FIG. 3 schematically shows a configuration of a plane landmark used in the embodiment of the present invention.
[0067]
As shown in the figure, the planar landmark according to the present embodiment has a radius d_iAnd an inner circle having a radius d substantially concentrically arranged._o(Except for d_i<D_o). The inner circle and the outer circle are applied to colors different from each other, and are more preferably provided with a color arrangement that is easily separated in a UV color space. In order to further enhance the discrimination between the inner circle and the outer circle, the width w_iAre provided. In addition, in order to ensure the discrimination and separation between the flat landmark and the floor or wall on which the flat landmark is affixed, the width w_oAre provided. Both the inner and outer boundaries may be black.
[0068]
In the present embodiment, the following parameters are used in order to improve the recognition performance of the inner circle and the outer circle, and the outer circle and the landmark installation surface.
[0069]
0mm <w_i<10mm
0mm <w_o<50mm
[0070]
Further, in order to improve the recognition performance of the outer circle and the inner circle, the ratio between the outer circle radius and the inner circle radius is set within the following range.
[0071]
3: 4 <d_i: D_o<1: 3
[0072]
For example, d_i= 60mm, d_o= 120mm, w_i= 2mm, w_o= 20, then d_i: D_o= 1: 2, which satisfies the above condition. It should be noted that only the ratio is important for the relationship between the outer circle radius and the inner circle radius, and the actual dimensions of each are not. This is because, as a basic property of the center projection, as the image approaches the projection center C of the camera, the image is projected as a large image, and conversely, as the distance from the projection center C decreases, the image is projected smaller. This is because only the cutout is handled (or the landmark is zoomed), so that the distance and ten-dimensional components are omitted, and only the dimensional ratio remains.
[0073]
Where the outer circle radius d_oIs fixed to 120 mm and the inner circle radius d_iWas changed in steps of 10 mm from 40 mm to 80 mm (see FIG. 4), and the landmark recognition rate in each case was measured. Table 1 below shows the measurement results.
[0074]
[Table 1]

[0075]
As can be seen from Table 1, when the inner circle radius is too small, the ratio of the inner circle to the landmark image decreases, and the recognition rate decreases. Conversely, if the inner circle radius is too large, the outer circle radius becomes smaller, so that the recognition rate also decreases. As a result, as described above, the ratio of the radius of the inner circle to the outer circle is 3: 4 <d_i: D_oWithin the range of <1: 3, a good recognition rate can be secured.
[0076]
Further, as described above, it is preferable that the inner circle and the outer circle are applied to colors different from each other, and are provided with a color arrangement that is easily separated in the UV color space.
[0077]
Considering the ease of color extraction for changes in brightness (adjusting the gain for changes in illuminance), any color is distributed radially in the UV space as long as the gain adjustment is correct. From this viewpoint, any color can be used for coloring the inner circle and the outer circle.
[0078]
Further, the distribution in the UV space is different between the royal road color and the characteristic color when the gain is not appropriate (the illuminance is fixed and the gain is changed).
[0079]
Also, in consideration of the ease of color extraction with respect to color temperature change (white balance adjustment with respect to illumination color temperature change (2700K, 4500K, 5000K, 6500K :: 9001x)), the pink system falls within a narrow range. come. Blue and green are wide.
[0080]
When the white balance is not appropriate, the ease of color extraction is as follows (white balance adjustment (2700K, 4500K, 5000K, 6500K :: 9001x) with respect to a change in illumination color temperature). Will overlap. From this viewpoint, it is preferable to employ a compact orange without using yellow.
[0081]
C. Landmark recognition and pose estimation algorithm
The relative distance from the robot to the landmark can be measured to some extent by, for example, a general ranging technique using a camera. However, in order to measure the distance more accurately, it is necessary to perform more strict calculation processing based on the image recognition result of the landmark. At this time, the calculation method differs depending on whether the landmark is placed horizontally on the floor or vertically attached to the wall, that is, depending on the posture of the landmark.
[0082]
In this section, a landmark recognition and posture estimation algorithm in the case of using a planar landmark composed of concentric inner and outer circles of different colors described in the preceding section B will be described in detail.
[0083]
In the present embodiment, landmark recognition and posture estimation are performed according to the following procedure.
[0084]
(1) Acquire images by camera
(2) Color extraction
(3) Connecting area division
(4) Recognize landmarks from connected areas
(5) For all landmarks,
5-a) Estimating a three-dimensional position based on the assumption that the landmark is a horizontal (floor) landmark
5-b) Estimate the three-dimensional position based on the assumption that the landmark is placed on a vertical (wall)
5-c) Determine whether the landmark is installed horizontally or vertically
[0085]
The color is extracted by detecting the color of the image in which the landmark is captured by the camera in pixel units. Furthermore, by taking out each region having the same color, a color connection region can be taken out.
[0086]
In the landmark recognition process, the following (rectangular) area information can be obtained by dividing an image into color connection areas.
[0087]
a) Color of each area
b) Upper left coordinates (x 'of the area)₁, Y '₁) And lower right coordinates (x '₂, Y '₂)
[0088]
Next, the following check is performed for all combinations of regions.
[0089]
a) One region contains the other region
b) ratio w of the width of the region_l: W_s(W_l: Width of large area, w_s: Width of small area) does not exceed a threshold value (for example, 3: 1)
c) ratio of the height of the region w_l: W_s(W_l: Width of large area, w_s: Width of small area) does not exceed a threshold value (for example, 3: 1)
[0090]
Then, a combination of areas satisfying all of the above conditions a to c is recognized as a landmark. A landmark can be uniquely specified by a combination of colors. Let the large area be the size of the landmark.
[0091]
Here, before describing the landmark posture estimation processing, a camera model used in the present embodiment will be described.
[0092]
In this embodiment, a pinhole camera that performs central projection is used. Center projection is a method of arranging the color density value of a point on the object surface at an intersection of a straight line (also referred to as a “line of sight”) connecting the projection center and a point on the three-dimensional object surface with the projection screen of the camera. Form a projection image. In the central projection, even objects of the same size are projected as a large image as approaching the projection center of the camera, and conversely, as the distance from the projection center C decreases, they are projected.
[0093]
Here, the camera center is the origin, the focal length is f, and the optical axis center is (p_x, P_y), Pixel size (u_x, U_y), The three-dimensional coordinates of the landmark position are (x, y, z), and the pixel coordinates corresponding to the three-dimensional position are (x ′, y ′).
[0094]
(Equation 1)

[0095]
Where f_x= F / u_x, F_y= F / u_yIs defined.
[0096]
In the present embodiment, the camera is mounted on the head of an upright walking robot having two arms and two legs, for example. Then, the posture of the camera is represented by rotational positions around the roll, pitch, and yaw axes as shown in FIG.
[0097]
In the following description, it is assumed that the camera rotates around the pitch axis to capture a landmark in front. As shown in FIG. 7, it is assumed that the camera is taking an image at a predetermined rotational position around the pitch axis. At this time, when landmarks having the same size are detected in a camera-captured image (camera image) (picture of landmark), landmarks can be taken including those indicated by reference numerals 1 to 4 in FIG. A plurality of postures are assumed. In the following description, it is assumed that the landmark is attached to only the wall or the floor surface, that is, the landmark is in one of the vertical posture 1 and the horizontal posture 2.
[0098]
The position of the landmark captured by the camera (or the relative distance between the camera and the landmark) depends on the attitude of the camera (rotational position pitch around the pitch axis in this embodiment), the focal length f of the camera lens, and the optical center ( p_x, P_y), And the height and width of a landmark in a captured image. However, the calculation method differs depending on whether the landmark is placed horizontally on the floor or vertically attached to the wall, that is, depending on the posture of the landmark.
[0099]
FIG. 8 illustrates a method of estimating the position of a landmark using the height of the landmark in a captured image, assuming that the landmark is installed horizontally.
[0100]
In the illustrated example, the camera captures a landmark at a predetermined rotation position pitch around the pitch axis. In the three-dimensional space, the landmark is a point (y₁, Z₁) And point (y₂, Y₂), And its center position is (y_v, Z_v). Then, the landmark is a point (y₂’, F_y) And point (y₁’, F_y), And the height of the image projected on the projection plane is the point (y₂, Z₂) And point (y₃, Z₂). In this case, the following equation holds geometrically.
[0101]
(Equation 2)

[0102]
Then, the three-dimensional position of the landmark center is represented by the following equation.
[0103]
(Equation 3)

[0104]
FIG. 9 illustrates a method of estimating the position of a landmark by using the height of the landmark in a captured image, assuming that the landmark is installed vertically.
[0105]
In the illustrated example, the camera captures a landmark at a predetermined rotation position pitch around the pitch axis. A landmark is a point (y₂, Z₂) And point (y₁, Z₁), And its center position is (y_h, Z_h). The landmark is a point (y₂’, F_y) And point (y₁’, F_y), And the height of the image projected on the projection plane is the point (y₃, Z₁) And point (y₁, Z₁). In this case, the following equation holds geometrically.
[0106]
(Equation 4)

[0107]
Then, the three-dimensional position of the landmark center is represented by the following equation.
[0108]
(Equation 5)

[0109]
FIG. 10 illustrates a method of estimating the position of the landmark using the width of the landmark in the captured image, assuming that the landmark is installed horizontally.
[0110]
In the illustrated example, the camera moves in the width direction at the center position (x, z) to a point (x₁, Z) and point (x₂, Z). A landmark is a point (x₁, Z) and point (x₂′, Z), and the width of the image projected on the projection plane is equal to the line segment connecting the points (x1, z) and (x2, z). In this case, the following equation holds geometrically.
[0111]
(Equation 6)

[0112]
Then, the three-dimensional position of the landmark center is represented by the following equation.
[0113]
(Equation 7)

[0114]
When a landmark placed horizontally without occlusion is completely visible, the relative distance zh to the landmark estimated by the height of the landmark in the captured image and the width of the landmark in the captured image The relative distance zw to the estimated landmark becomes equal (z_w= Z_h).
[0115]
FIG. 11 illustrates a method of estimating the position of a landmark using the width of the landmark in a captured image, assuming that the landmark is installed vertically.
[0116]
In this case, the relative distance is the maximum when the landmark is perpendicular to the optical axis, and the landmarks installed at various rotational positions with respect to the optical axis are positioned in the same width direction in the captured image, that is, a point. (X₁, F_x) To the point (x₂, F_xObserved during). In other words, unless the rotation of the landmark with respect to the optical axis is known, its three-dimensional position cannot be estimated.
[0117]
When the landmark is perpendicular to the optical axis, the three-dimensional position of the center of the landmark is the same as the above equation [Equation 7]. When the vertically installed landmark is completely visible without occlusion or the like, the relative distance z to the landmark estimated from the width of the landmark in the captured image is obtained._wIs greater than or equal to the relative distance zv to the landmark estimated from the height of the landmark in the captured image (z_w≧ z_v).
[0118]
According to the principle shown in FIGS. 10 and 11, the relative distance z to the landmark estimated from the width of the landmark in the captured image_wAnd the relative distance z to the landmark estimated from the height of the landmark in the captured image_vBy comparing the sizes of the landmarks, it is possible to estimate whether the landmark in the captured image is installed in a horizontal or vertical posture. Then, an appropriate landmark position estimation algorithm can be selected based on the estimation result (described later).
[0119]
The calculation method differs depending on whether the landmark is installed in a horizontal or vertical position.However, apart from this, when the camera recognizes the landmark when looking far away, and when the camera looks closer. The optimal position calculation method differs depending on whether or not a landmark is recognized.
[0120]
FIG. 12 shows that the camera is looking at a distant landmark. In the illustrated example, the camera captures a landmark at a predetermined rotation position pitch around the pitch axis.
[0121]
In this case, assuming that landmarks having the same height are detected in the captured image, landmarks installed vertically are farther than landmarks installed horizontally. That is, when the camera is looking far away, the relative distance z to the landmark estimated using the height of the landmark in the captured image on the assumption that the landmark is installed horizontally is assumed._hRather than the relative distance z to the landmark estimated using the height of the landmark in the captured image on the assumption that the landmark is installed vertically._vHas a larger value (z_h<Z_v).
[0122]
FIG. 13 shows a state in which the camera looks at a nearby landmark. In the illustrated example, the camera captures a landmark at a predetermined rotation position pitch around the pitch axis.
[0123]
In this case, assuming that landmarks having the same height are detected in the captured image, landmarks installed vertically are closer to landmarks installed horizontally. That is, when the camera is looking at the vicinity, the relative distance z to the landmark estimated using the height of the landmark in the captured image on the assumption that the landmark is installed horizontally is assumed._hRather than the relative distance z to the landmark estimated using the height of the landmark in the captured image on the assumption that the landmark is installed horizontally._vIs smaller (z_h> Z_v).
[0124]
According to the principle shown in FIGS. 12 and 13, the relative distance z to the landmark estimated using the height of the landmark in the captured image on the assumption that the landmark is installed horizontally is assumed._hAnd the relative distance z to the landmark estimated using the height of the landmark in the captured image on the assumption that the landmark is installed horizontally._vBy comparing the magnitudes of the two, it is possible to estimate whether the camera is looking far or near. Then, an appropriate landmark position estimation algorithm can be selected based on the estimation result (described later).
[0125]
FIG. 14 shows a state in which a camera looking down captures landmarks installed horizontally and vertically. Here, the center position of the inner circle of the landmark is (x_ic, Y_ic), And the center position of the outer circle of the landmark is (x_oc, Y_oc)far.
[0126]
In central projection, even objects of the same size are projected as a large image as they approach the projection center of the camera, and conversely, become smaller as they move away from the projection center (described above).
[0127]
Therefore, as can be seen from FIG. 14, the height y of the center of the inner circle of the landmark is obtained._icHeight of the center of the outer circle than_ocIs smaller than (y_ic> Y_oc), The landmark is presumed to be installed vertically. Conversely, the height y of the center of the inner circle of the landmark_icHeight of the center of the outer circle than_ocIs larger than (y_ic<Y_oc), The landmark is presumed to be installed horizontally. Then, an appropriate landmark position estimation algorithm can be selected based on the estimation result (described later).
[0128]
FIG. 15 shows an example of a landmark position estimation algorithm in the form of a flowchart. The pseudo program code of the position estimation algorithm is shown below. However, it is assumed here that the entire landmark is visible without occlusion.
[0129]
(Equation 8)

[0130]
First, in step S1, assuming that the landmark is set horizontally, the position of the landmark is estimated using the height of the landmark in the captured image (see FIG. 8).
[0131]
Next, in step S2, assuming that the landmark is installed vertically, the position of the landmark is estimated using the height of the landmark in the captured image (see FIG. 9).
[0132]
Further, in step S3, assuming that the landmark is set horizontally, the position of the landmark is estimated using the width of the landmark in the captured image (see FIG. 10).
[0133]
Here, the relative distance z to the landmark estimated using the height of the landmark in the captured image on the assumption that the landmark is installed horizontally is assumed._hAnd the relative distance z to the landmark estimated using the height of the landmark in the captured image on the assumption that the landmark is installed vertically_vThen, it is estimated whether the camera is looking far or near (step S4) (see FIGS. 12 and 13).
[0134]
The position estimation algorithm shown in FIG. 15 is suitable when the camera is looking at a distant landmark, but cannot accurately estimate the arrangement when looking at a nearby landmark. For this reason, the relative distance z to the landmark estimated using the height of the landmark in the captured image on the assumption that the landmark is installed horizontally is assumed._hRather than the relative distance z to the landmark estimated using the height of the landmark in the captured image on the assumption that the landmark is installed vertically._vIs smaller than (z_h> Z_vSince the camera is looking at the vicinity, no answer is given (No Result), and the entire processing routine ends (step S5).
[0135]
On the other hand, the relative distance z to the landmark estimated using the height of the landmark in the captured image on the assumption that the landmark is installed horizontally is assumed._hRather than the relative distance z to the landmark estimated using the height of the landmark in the captured image on the assumption that the landmark is installed horizontally._vIs larger than (z_h<Z_v), It is estimated that the camera is looking far away. In this case, furthermore, the relative distance z to the landmark estimated from the width of the landmark in the captured image_wAnd a relative distance z to the landmark estimated based on the height of the landmark in the captured image, assuming that the landmark is installed vertically._vBy comparing the sizes of the landmarks, it is estimated whether the landmarks in the captured image are set in the horizontal or vertical posture (step S6) (see FIG. 11).
[0136]
And z_w> Z_vIn this case, it is estimated that the landmark is installed vertically, and it is assumed in step S2 that the landmark is installed vertically, and is estimated using the height of the landmark in the captured image. (X_v, Y_v, Z_v) Is output as the estimation result (step S7).
[0137]
On the other hand, z_w≦ z_vIn this case, there is a possibility that the landmark is installed horizontally and that the landmark is installed in a vertical posture that is not rotated with respect to the camera. Therefore, assuming that the landmark is installed horizontally, the relative distance z to the landmark estimated by the height of the landmark in the captured image_hAnd the relative distance z to the landmark estimated from the width of the landmark in the captured image_w| Z_h-Z_w| And the relative distance z to the landmark estimated from the height of the landmark in the captured image, assuming that the landmark is installed vertically_vAnd the relative distance z to the landmark estimated from the width of the landmark in the captured image_w| Z_v-Z_w(Step S8) to determine whether the landmark is installed vertically or horizontally.
[0138]
| Z_v-Z_wWhen | is smaller, it is estimated that the landmark is installed in a vertical posture that is not rotated with respect to the camera. In this case, assuming that the landmark is installed vertically in step S2, the position of the landmark estimated using the height of the landmark in the captured image (x_v, Y_v, Z_v) Is output as the estimation result (step S9).
[0139]
On the other hand, | z_h-Z_wIf | is smaller, it is determined that the landmark is installed horizontally. Then, in step S1, assuming that the landmark is set horizontally, the position of the landmark estimated using the height of the landmark in the captured image (x_h, Y_h, Z_h) Is output as the estimation result (step S10).
[0140]
The landmark position estimation algorithm shown in FIG. 15 is suitable for a case where the camera is looking far away (see FIG. 12). When a part of the landmark is hidden (occlusion), the result may be different. Also, the results may differ if the landmarks are not placed vertically or horizontally. The same result can be obtained with respect to the rotation of the camera (around the roll axis), although the formula slightly changes.
[0141]
FIG. 16 is a flowchart illustrating another example of the landmark position estimation algorithm. The pseudo program code of the position estimation algorithm is shown below. However, it is assumed here that the entire landmark is visible without occlusion.
[0142]
(Equation 9)

[0143]
First, in step S11, assuming that the landmark is installed horizontally, the position of the landmark is estimated using the height of the landmark in the captured image (see FIG. 8).
[0144]
Next, in step S12, assuming that the landmark is installed vertically, the position of the landmark is estimated using the height of the landmark in the captured image (see FIG. 9).
[0145]
Further, in step S13, assuming that the landmark is set horizontally, the position of the landmark is estimated using the width of the landmark in the captured image (see FIG. 10).
[0146]
When the camera looks down at landmarks that are horizontally and vertically installed, the height y of the center of the inner circle of the landmarks_icHeight of the center of the outer circle than_ocIs smaller than (y_ic> Y_oc), The landmark is estimated to be installed vertically, and the height y of the center of the inner circle of the landmark_icHeight of the center of the outer circle than_ocIs larger (y_ic<Y_oc), The landmarks are assumed to be installed vertically (see FIG. 14).
[0147]
Therefore, in step S14, the height y of the inner circle of the landmark in the captured image_icAnd the height y of the outer circle_icAre compared to determine whether the landmark is installed in a horizontal or vertical posture.
[0148]
Here, the height y of the center of the inner circle of the landmark_icHeight of the center of the outer circle than_ocIs larger than (y_ic<Y_oc), The landmark is presumed to be installed horizontally.
[0149]
In this case, furthermore, the relative distance z to the landmark estimated using the height of the landmark in the photographed image on the assumption that the landmark is set horizontally is assumed._hAnd the relative distance z to the landmark estimated using the width of the landmark in the captured image_wAre compared with each other (step S15).
[0150]
And z_hIs z_wIf not larger than the landmark position (x) estimated using the height of the landmark in the captured image on the assumption that the landmark is installed horizontally in step S11._h, Y_h, Z_h) Is output together with the estimation result that the landmark is installed horizontally (step S16).
[0151]
On the other hand, z_hIs z_wIf it is larger, it is estimated that the occlusion is large and the possibility of erroneous recognition is high. Position (x_w, Y_w, Z_w) Is output together with the estimation result that the landmark is installed horizontally (step S17).
[0152]
The height y of the center of the inner circle of the landmark_icHeight of the center of the outer circle than_ocIs smaller than (y_ic> Y_oc), The landmark is presumed to be installed vertically.
[0153]
In this case, the relative distance z to the landmark estimated using the height of the landmark in the captured image on the assumption that the landmark is installed vertically is further determined._vAnd the relative distance z to the landmark estimated using the width of the landmark in the captured image_wAre compared with each other (step S18).
[0154]
And z_vIs z_wIf not larger, the position of the landmark (x) is estimated using the height of the landmark in the captured image, assuming that the landmark is installed vertically in step S12._v, Y_v, Z_v) Is output together with the estimation result that the landmark is installed vertically (step S19).
[0155]
On the other hand, z_vIs z_wIf it is larger, the inclination of the landmark with respect to the occlusion and the line of sight is large, and it is estimated that there is a high possibility of misrecognition. (X_w, Y_w, Z_w) Is output together with the estimation result that the landmark is installed vertically (step S20).
[0156]
The landmark position estimation algorithm shown in FIG. 16 is suitable for a case where the camera is looking far away (see FIG. 13). When a part of the landmark is hidden, the estimation of the rotation direction may be different due to inconsistency of the area information. Also, the results may differ if the landmarks are not placed vertically or horizontally. The same result can be obtained with respect to the rotation of the camera (around the roll axis), although the formula slightly changes. The algorithm shown in FIG. 16 is particularly effective when the proportion of the landmark in the image is large. ) May have different results.
[0157]
The landmark position estimation algorithm shown in FIG. 15 is suitable when the camera recognizes a distant landmark, and furthermore, the optimum landmark position estimation according to whether the landmark is set horizontally or vertically. You have to choose a method. Similarly, the landmark location estimation algorithm shown in FIG. 16 is suitable when the camera recognizes a nearby landmark, and furthermore, the optimal landmark is determined depending on whether the landmark is set horizontally or vertically. Is selected.
[0158]
FIG. 17 also shows landmarks that can automatically select either the algorithm shown in FIG. 15 or the algorithm shown in FIG. 16 depending on whether the camera recognizes landmarks that are far or perceptible. Is shown in the form of a flowchart. The pseudo program code of the position estimation algorithm is shown below.
[0159]
(Equation 10)

[0160]
First, in step S21, assuming that the landmark is set horizontally, the position of the landmark is estimated using the height of the landmark in the captured image (see FIG. 8).
[0161]
Next, in step S22, assuming that the landmark is installed vertically, the position of the landmark is estimated using the height of the landmark in the captured image (see FIG. 9).
[0162]
Further, in step S23, assuming that the landmark is set horizontally, the position of the landmark is estimated using the width of the landmark in the captured image (see FIG. 10).
[0163]
Here, the relative distance z to the landmark estimated using the height of the landmark in the captured image on the assumption that the landmark is installed horizontally is assumed._hAnd the relative distance z to the landmark estimated using the height of the landmark in the captured image on the assumption that the landmark is installed vertically_vThen, it is estimated whether the camera is looking far or near (Step S24) (see FIGS. 12 and 13).
[0164]
The relative distance z to the landmark estimated using the height of the landmark in the captured image on the assumption that the landmark is installed horizontally._hRather than the relative distance z to the landmark estimated using the height of the landmark in the captured image on the assumption that the landmark is installed horizontally._vIs larger than (z_h<Z_v), It is estimated that the camera is looking far away. In this case, the landmark position estimation algorithm shown in FIG. 15 (however, after step S4) is executed (step S25).
[0165]
On the other hand, the relative distance z to the landmark estimated using the height of the landmark in the captured image on the assumption that the landmark is installed horizontally is assumed._hRather than the relative distance z to the landmark estimated using the height of the landmark in the captured image on the assumption that the landmark is installed vertically._vIs smaller than (z_h> Z_v), It is estimated that the camera is looking close. In this case, the landmark position estimation algorithm shown in FIG. 16 (however, after step S14) is executed (step S26).
[0166]
When a part of the landmark is hidden (occlusion), the result may be different. Also, the results may differ if the landmarks are not placed vertically or horizontally. The same result can be obtained with respect to the rotation of the camera (around the roll axis), although the formula slightly changes.
[0167]
[Supplement]
The present invention has been described in detail with reference to the specific embodiments. However, it is obvious that those skilled in the art can modify or substitute the embodiment without departing from the spirit of the present invention.
[0168]
The gist of the present invention is not necessarily limited to products called “robots”. That is, it is a mechanical device or other general mobile device that performs a motion similar to a human motion using an electric or magnetic action, or a data processing system that performs arithmetic processing on data describing the operation of these devices. Then, the present invention can be similarly applied to products belonging to other industrial fields such as toys.
[0169]
In short, the present invention has been disclosed by way of example, and the contents described in this specification should not be interpreted in a limited manner. In order to determine the gist of the present invention, the claims described at the beginning should be considered.
[0170]
【The invention's effect】
As described above in detail, according to the present invention, it is possible to provide an excellent robot that can identify a self-position in an environment including an artificial landmark based on a landmark search result.
[0171]
Further, according to the present invention, it is possible to appropriately estimate a position according to a posture in which a planar landmark installed on a floor or a wall is installed, an excellent robot, and a landmark posture estimation system and An estimation method can be provided.
[0172]
Further, according to the present invention, an excellent robot capable of accurately estimating a position in accordance with an attitude in which a planar landmark including a suitable shape pattern and a color pattern is installed, and an attitude estimation of a landmark A system and estimation method can be provided.
[Brief description of the drawings]
FIG. 1 is a diagram schematically showing a functional configuration of a mobile robot 1 used in the present invention.
FIG. 2 is a diagram showing the configuration of a control unit 20 in further detail.
FIG. 3 is a diagram schematically showing a configuration of a plane landmark provided for implementing the present invention.
FIG. 4 is a diagram schematically illustrating an experimental environment for measuring a recognition rate of a plane landmark.
FIG. 5 is a diagram for explaining a camera model used in the embodiment of the present invention.
FIG. 6 is a diagram for explaining a method of expressing the attitude of the camera.
FIG. 7 is a diagram illustrating a state in which the camera is capturing an image at a predetermined rotation position about a pitch axis.
FIG. 8 is a diagram for explaining a method of estimating the position of a landmark using the height of the landmark in a captured image when it is assumed that the landmark is installed horizontally.
FIG. 9 is a diagram for explaining a method of estimating the position of a landmark using the height of the landmark in a captured image when it is assumed that the landmark is installed vertically.
FIG. 10 is a diagram for explaining a method of estimating the position of a landmark using the width of the landmark in a captured image when it is assumed that the landmark is installed horizontally.
FIG. 11 is a diagram for explaining a method of estimating the position of a landmark using the width of the landmark in a captured image when it is assumed that the landmark is installed vertically.
FIG. 12 is a diagram illustrating a state in which a camera looks at a distant landmark.
FIG. 13 is a diagram showing a state in which a camera looks at a nearby landmark.
FIG. 14 is a diagram showing a state in which a camera looking down captures landmarks installed horizontally and vertically.
FIG. 15 is a flowchart illustrating an example of a landmark position estimation algorithm.
FIG. 16 is a flowchart illustrating another example of a landmark position estimation algorithm.
FIG. 17 is a flowchart illustrating another example of a landmark position estimation algorithm.
[Explanation of symbols]
1. Mobile robot
15 ... CCD camera
16 ... Microphone
17… Speaker
18 ... Touch sensor
19 ... LED indicator
20 ... Control unit
21 ... CPU
22 ... RAM
23… ROM
24: Non-volatile memory
25 ... Interface
26 ... Wireless communication interface
27 ... Network interface card
28 ... Bus
29 ... Keyboard
40 ... input / output unit
50 ... Drive unit
51 ... motor
52 ... Encoder
53 ... Driver

Claims (22)

A robot for estimating a position according to a posture in which a landmark is installed,
Shooting means for shooting landmarks,
First position calculating means for calculating the position of the landmark based on the height component of the landmark in the captured image, assuming that the landmark is installed horizontally;
Second position calculating means for calculating the position of the landmark based on the height component of the landmark in the captured image, assuming that the landmark is installed vertically;
Third position calculating means for calculating the position of the landmark based on the width component of the landmark in the captured image;
Attitude estimating means for estimating whether the landmark is installed in a horizontal or vertical orientation,
A selection output unit that selects one of the calculation results by the first to third position calculation units in accordance with the estimated posture of the landmark;
A robot comprising: A landmark position estimation system for estimating a position according to a posture in which a landmark is installed,
Image holding means for holding a captured image of the landmark;
First position calculating means for calculating the position of the landmark based on the height component of the landmark in the captured image, assuming that the landmark is installed horizontally;
Second position calculating means for calculating the position of the landmark based on the height component of the landmark in the captured image, assuming that the landmark is installed vertically;
Third position calculating means for calculating the position of the landmark based on the width component of the landmark in the captured image;
Attitude estimating means for estimating whether the landmark is installed in a horizontal or vertical orientation,
A selection output unit that selects one of the calculation results by the first to third position calculation units in accordance with the estimated posture of the landmark;
A landmark position estimation system, comprising: And relative distance z _h to landmarks landmark is estimated using the height of the landmark in the photographed image by the first position calculating means assumption that is installed horizontally, the landmark vertical the relative distance z _v to landmarks which are estimated using the height of the landmark in the photographed image by the second position calculating means and compares with assumption that are installed in, z _h> z _v captured image if made was determined to have been captured by the imaging means looking at near, z h _{<z v} become captured image if the imaging localization to determine that captured by the imaging image looking at distant Further comprising means,
The selection output unit selects one of the calculation results obtained by the first to third position calculation units depending on whether the captured image is viewed from near or far.
3. The landmark position estimation system according to claim 2, wherein: The posture estimating means, and the relative distance z _w to landmarks estimated by the width of the landmark in the photographed image, the height of the landmark in the photographed image on the assumption that the landmark is in the upright position a relative distance zv to landmarks which are estimated magnitude comparison, if the z _w> z _v is estimated that landmark is installed vertically, the
The selection output unit outputs a position calculation result by the second position calculation unit in response to the estimation that the landmark is installed vertically.
3. The landmark position estimation system according to claim 2, wherein: The posture estimating means further compares | z _h −z _w | and | z _v −z _w | when z _w ≦ z _v ,
_| Z v _-z w | when the person is small, estimated to landmark is installed in a vertical position which is not rotated relative to the camera, in response thereto, the selection output means wherein the Outputting the calculation result of the position calculation means of No. 2;
When | z _h −z _w | is smaller, it is estimated that the landmark is installed horizontally, and in response to this, the selection output unit outputs the calculation result of the first position calculation unit. Do
The landmark position estimation system according to claim 4, wherein: The landmark is a plane shape formed by the outer circle of radius d _o is an inner circle and the inner circle arranged in a substantially concentric radius d _i (where, _d i _{<d o),}
The posture estimating means compares the height y _ic of the inner circle of the landmark in the captured image with the height y _oc of the outer circle, and compares the height y _ic of the outer circle with the height y _ic of the center of the inner circle of the landmark. When the height y _oc of the center is smaller (y _ic > y _oc ), it is estimated that the landmark is installed vertically, and the height of the center of the outer circle is larger than the height y _ic of the center of the inner circle of the landmark. When the height y _oc is larger (y _ic <y _oc ), it is estimated that the landmark is set vertically.
3. The landmark position estimation system according to claim 2, wherein: When the height y _oc of the center of the outer circle is smaller than the height y _ic of the center of the inner circle of the landmark (y _ic > y _oc ), the posture estimating means determines that the landmark is installed vertically. and relative distance z _v to landmarks which are estimated using the height of the landmark in the photographed image by assumption that there, the relative distances to the landmark, which is estimated using the width of the landmark in the captured image z _w is compared with the size,
Said selective output means, z _v is z if not greater than _w and outputs the calculation result by the second position calculating means, z _v the third position calculating means when larger than z _w Output the calculation result by
7. The landmark position estimation system according to claim 6, wherein: When the height y _oc at the center of the outer circle is larger than the height y _ic at the center of the inner circle of the landmark (y _ic <y _oc ), the posture estimating means determines that the landmark is horizontally installed. and relative distance z _h to landmarks which are estimated using the height of the landmark in the photographed image by assumption that there, the relative distances to the landmark, which is estimated using the width of the landmark in the captured image z _w is compared with the size,
It said selective output means, z _h is z if not greater than _w and outputs the calculation result of the first position-calculating means, when z _h is greater than z _w is the third position calculating means Output the calculation result by
7. The landmark position estimation system according to claim 6, wherein: The inner circle and the outer circle of the landmark are arranged in different colors that are easily separated from each other on a UV color space.
7. The landmark position estimation system according to claim 6, wherein: An inner boundary line having a width w _i and / or an outer boundary line having a width w _o provided outside the outer circle are provided between the inner circle and the outer circle to further enhance discrimination.
7. The landmark position estimation system according to claim 6, wherein: In order to improve the recognition performance of the outer circle and the inner circle, the ratio of the radius of the inner circle and the radius of the outer circle is set in the following range,
3: 4 <d _i : d _o <1: 3
7. The landmark position estimation system according to claim 6, wherein: A landmark position estimation method for estimating a position according to a posture in which a landmark is installed,
Obtaining a captured image of the landmark;
A first position calculating step of calculating a position of the landmark based on a height component of the landmark in the captured image, assuming that the landmark is installed horizontally;
A second position calculating step of calculating the position of the landmark based on the height component of the landmark in the captured image, assuming that the landmark is installed vertically;
A third position calculating step of calculating the position of the landmark based on the width component of the landmark in the captured image;
A posture estimation step of estimating whether the landmark is installed in a horizontal or vertical posture,
A selection output step of selecting one of the calculation results of the first to third position calculation steps according to the estimated posture of the landmark;
A method for estimating a position of a landmark, comprising: And relative distance z _h to landmarks landmark is estimated using the height of the landmark in the photographed image by the first position-calculating step in assumption that is installed horizontally, the landmark vertical the relative distance z _v to landmarks which are estimated using the height of the landmark in the photographed image by the second position calculating step and compares with assumption that are installed in, z _h> z _v captured image if made was determined to have been captured by the imaging means looking at near, z h _{<z v} become captured image if the imaging localization to determine that captured by the imaging image looking at distant Further comprising steps,
In the selection output step, one of the calculation results in the first to third position calculation steps is selected according to whether the captured image is viewed near or far.
The method according to claim 12, wherein the position of the landmark is estimated. Wherein in the position estimation step, the relative distance z _w to landmarks estimated by the width of the landmark in the photographed image, the height of the landmark in the photographed image on the assumption that the landmark is in the upright position the relative distance z _v to landmarks which are estimated magnitude comparison, if the z _w> z _v is estimated that landmark is installed vertically, the
Outputting the position calculation result of the second position calculation step in response to the landmark being estimated to be vertically set in the selection output step;
The method according to claim 12, wherein the position of the landmark is estimated. When said a _{z w} ≦ _{z v} in pose estimation step _| z h -z _{w |} a _| z v -z _{w |} and further compares large and small,
_| Z v _-z w | when the person is small, estimated to landmark is installed in a vertical position which is not rotated relative to the camera, in response thereto, the first in the selection output step Outputting the calculation result of the position calculation step of 2,
When | z _h −z _w | is smaller, it is estimated that the landmark is installed horizontally, and in response to this, the selection output step outputs the calculation result of the first position calculation step. Do
The landmark position estimation method according to claim 14, wherein: The landmark is a plane shape formed by the outer circle of radius d _o is an inner circle and the inner circle arranged in a substantially concentric radius d _i (where, _d i _{<d o),}
In the posture estimation step, the height y _ic of the inner circle of the landmark in the captured image is compared with the height y _oc of the outer circle, and the height of the outer circle is larger than the height y _ic of the center of the inner circle of the landmark. When the height y _oc of the center is smaller (y _ic > y _oc ), it is estimated that the landmark is installed vertically, and the height of the center of the outer circle is larger than the height y _ic of the center of the inner circle of the landmark. When the height y _oc is larger (y _ic <y _oc ), it is estimated that the landmark is set vertically.
The method according to claim 12, wherein the position of the landmark is estimated. In the posture estimating step, when the height y _oc at the center of the outer circle is smaller than the height y _ic at the center of the inner circle of the landmark (y _ic > y _oc ), the landmark is installed vertically. and relative distance z _v to landmarks which are estimated using the height of the landmark in the photographed image by assumption that there, the relative distances to the landmark, which is estimated using the width of the landmark in the captured image z _w is compared with the size,
Wherein the selection outputting step, z _v and outputs the calculated result of said second position calculation step if not greater than z _w, z _v the third position calculating step of, if is greater than z _w Output the calculation result by
17. The method according to claim 16, wherein the position of the landmark is estimated. In the posture estimating step, when the height y _oc at the center of the outer circle is larger than the height y _ic at the center of the inner circle of the landmark (y _ic <y _oc ), the landmark is placed horizontally. and relative distance z _h to landmarks which are estimated using the height of the landmark in the photographed image by assumption that there, the relative distances to the landmark, which is estimated using the width of the landmark in the captured image z _w is compared with the size,
In the selection output step, when z _h is not greater than z _{w, the} calculation result of the first position calculation step is output. When z _h is greater than z _w , the third position calculation step is performed. Output the calculation result by
17. The method according to claim 16, wherein the position of the landmark is estimated. A landmark for the robot to perform environment recognition based on visual recognition,
An inner circle of radius d _i arranged on a plane,
And the outer circle of radius d _o which is disposed substantially concentrically and within the circle (where, _d i _{<d o),}
A landmark comprising: The inner circle and the outer circle are arranged in different colors that are easily separated from each other on the UV color space.
20. The landmark according to claim 19, wherein: An inner boundary line having a width w _i and / or an outer boundary line having a width w _o provided outside the outer circle are provided between the inner circle and the outer circle to further enhance discrimination.
20. The landmark according to claim 19, wherein: In order to improve the recognition performance of the outer circle and the inner circle, the ratio of the radius of the inner circle and the radius of the outer circle is set in the following range,
3: 4 <d _i : d _o <1: 3
20. The landmark according to claim 19, wherein:

Images