JP2004152032A - Solid with three-dimensional attitude input condition designating function, three-dimensional attitude input method and device and its program and recording medium with its program recorded - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To carry out the detection/input of not only the three-dimensional attitude of a solid for three-dimensional attitude detection but also the condition of the solid by changing the color of the graphic of the surface according to a predetermined operation. <P>SOLUTION: A sphere 1 for attitude input with an indirect illumination type click function being a solid with a condition designating function for three-dimensional attitude input is formed with a hollow, and three holes 1a, 1b, and 1c are punched on the external surface, and a blue LED (470 nm) 3a and a green LED (630 nm) 3b to be able to blink are arranged near those holes 1a, 1b, and 1c. The external surface of the sphere 1 is painted in red. When a switch 2a of a power source 2 is depressed, the blue LED 3a or the green LED 3b is turned on. The sphere 1 is structured in the same way as an integrated sphere so that when the LED is turned on, the blue light or green light can be emitted so as to be spread through the holes 1a, 1b, and 1c to the outside. Thus, the sphere 1 is able to express three kinds of conditions. <P>COPYRIGHT: (C)2004,JPO

Description

【００４０】
なお、これらの関係は、Ｄ_ＣＰ、すなわち実空間におけるＣと対象点Ｐとの間の距離さえ既知であれば、対象点Ｐが楕円体の極以外の任意の点であっても、対象点Ｐと直線ＣＰの姿勢との関係として成立する。
【００４１】
図２４は撮像画像から楕円体の部分を切り出した部分画像を示している。図２４（ａ）は濃淡画像であり、図２４（ｂ）は同画像を２値化した画像である。説明のため、図２４（ｂ）では図形をその輪郭のみで示している。図２４（ｂ）において、Ｐ_Ｓは極Ｐの撮像画像上での位置であり、これは撮像画像上での図形の重心に一致している。
【００４２】
図２５は立体２１の姿勢および状態を入力する手順を示すフローチャートである。
１．撮像画像を２値化した画像（図２４（ｂ））から図形の重心を求め、その位置をＰ_Ｓとする（ステップ４１）。
２．式（１）を用いて、軸Ｌの姿勢すなわち角度Ｂ_１およびＢ_２を求める（ステップ４２）。
３．２で得られた角度Ｂ_２に最も近い角度を図２２に示したテーブルの角度θ_１欄から選択し、マッチング処理に用いるテンプレート画像群の列を決定する（ステップ４３）。
４．撮像画像（図２４（ａ））をＣ_Ｓの周りに角度−Ｂ_１だけ回転させた濃淡画像を得る（ステップ４４）。
５．３で決定された列に含まれる全てのテンプレート画像（図２２の例では３６０個）と４で得られた濃淡画像との正規化相関手法等によるマッチング処理を点Ｐ_Ｓの回転後の点の近傍において逐次行う（ステップ４５）。
６．５で得られたマッチングの度合い（スコア）が最大となるテンプレート画像を見つけ、そのテンプレート画像に対応する角度θ_２を求め、これを立体２１の軸Ｌの周りの回転方向の姿勢を表わす角度Ｂ_３とする（ステップ４６）。
７．以上により得られた角度Ｂ_１、Ｂ_２、およびＢ_３を立体２１の姿勢とする（ステップ４７）。
８．最後に、撮像画像の孔の位置における色情報および輝度情報の少なくとも一方に基づいて立体２１の指定された状態を検出する（ステップ４８）。
【００４３】
図２６に上記と同様の手順で姿勢および状態を入力することができる立体２１の第１３の例として、１個の非回転対称図形が表面に描かれた正四角台形錐体を示す。図２６（ａ）は立体２１の例、図２６（ｂ）は立体２１の表面上の図形の配置を説明するための図である。図２６（ｂ）において、Ｌは正四角台形錐体の軸であり、Ｐは正四角台形錐体の表面と軸Ｌとが交わる点である。非回転対称図形（この例では文字「Ａ」）は、図２６（ａ）のように、その重心ＧがＰの位置に一致するように正四角台形錐体の表面に描かれている。
【００４４】
図２７に上記と同様の手順で姿勢および状態を入力することができる立体２１の第１４の例として、１個の非回転対称図形が表面に描かれた正四角台形錐の殻状体を示す。図２７（ａ）は立体２１の例、図２７（ｂ）は立体２１の表面上の図形の配置を説明するための図である。図２７（ｂ）において、Ｌは正四角台形錐殻状体の軸であり、Ｐは正四角台形錐殻状体の表面と軸Ｌとが交わる点である。非回転対称図形（この例では文字「Ａ」）は、図２７（ａ）のように、その重心ＧがＰの位置に一致するように正四角台形錐殻状体の表面に描かれている。
【００４５】
図２８に上記と同様の手順で姿勢を検出することができる立体２１の第１５の例として、１個の非回転対称図形が表面に描かれた準正多面体を示す。非回転対称図形（この例では文字「Ａ」）は、図２８のように、その重心Ｇが準正多面体の１つの面の重心の位置に一致するように準正多面体の表面に描かれている。
【００４６】
図２９に上記と同様の手順で姿勢を検出することができる立体２１の第１６の例として、１個の非回転対称図形が表面に描かれた正多面体を示す。非回転対称図形（この例では文字「Ａ」）は、図２９のように、その重心Ｇが正多面体の１つの面の重心の位置に一致するように正多面体の表面に描かれている。
【００４７】
［第２の実施形態］
本実施形態では、３次元姿勢検出用立体２１の第１７の例として、２種類の大きさの４個の回転対称図形と３個の非回転対称図形が表面に描かれた既知の楕円体を用い、その撮像画像から立体２１の姿勢を求める方法を説明する。
【００４８】
図３０は立体２１の表面上の図形の配置を説明するための図である。わかり易くするため、図３０では楕円体面上の図形を平面的に表現している。図３０（ａ）は上面図であり、図３０（ｂ）は側面図である。楕円体の極Ｐの位置に大きいドットが、極Ｐを中心とした回転対称の位置Ｑ_１、Ｑ_２、Ｑ_３に小さいドットがそれぞれ描かれている。また、Ｑ_１、Ｑ_２、Ｑ_３に描かれた各ドットからＰに描かれたドットに向かう方向を姿勢の基準として、Ｐに描かれたドットとＱ_１、Ｑ_２、Ｑ_３に描かれた３個のドットとの各中点にそれぞれ異なる３個の図形が描かれている。
【００４９】
図３１は立体２１に描かれた各ドットの３次元的な位置関係を説明するための図である。図３１（ａ）において、Ｐは楕円体の極であり、大きいドットの位置である。Ｑ_ｉ（ｉ＝１，２，３）は各小さいドットの位置、Ｃは楕円体の重心、Ｄ_ＰＱは実空間上でのＰとＱ_ｉ間の距離、Ｄ_ＱＱは実空間上でのＱ_ｉとＱ_ｊ間の距離、Ｄ_ＣＱは実空間上でのＣとＱ_ｉ間の距離である。図３３（ｂ）において、Ｍ_ｉは点Ｐ、Ｑ_ｉおよびＣを含む平面上の点であり、線分ＰＱ_ｉの垂直二等分線が楕円体の表面と交わる点である。
【００５０】
図３２（ａ）および（ｂ）は、それぞれ大きいドットおよび小さいドットを検出するためのテンプレート画像であり、実際にはガウシアンフィルタ等によるぼかし処理を施して作成されている。
【００５１】
図３３は互いに異なる３種類の図形を検出するためのテンプレート画像群が配置されたテーブルである。各テンプレート画像は第１の実施形態で説明した方法で作成してある。
【００５２】
図３４は撮像画像から楕円体の部分を切り出した部分画像を示している。図３４（ａ）は濃淡画像であり、図３４（ｂ）は楕円体の姿勢を求める手順を説明するための図である。説明のため、図３４（ｂ）では各ドットをその輪郭のみで示している。図３４（ｂ）において、Ｐ_Ｓは極Ｐの撮像画像上での位置であり、これは撮像画像上での大きいドットの位置に一致している。また、Ｑ_Ｓｉ（ｉ＝１，２，３）はそれぞれＱ_ｉの撮像画像上での位置、Ｍ_ＳｉはＭ_ｉの撮像画像上での位置である。
【００５３】
図３５は立体２１の姿勢を求める手順を示すフローチャートである。
１．大きいドットを検出するためのテンプレート画像（図３２（ａ））と濃淡画像（図３２（ａ））との正規化相関手法等によるマッチング処理を行い、マッチングスコアが最大となる撮像画像上での大きいドットの位置を求め、その位置をＰ_Ｓとする（ステップ５１）。
２．小さいドットを検出するためのテンプレート画像（図３２（ｂ））と濃淡画像（図３４（ａ））との正規化相関手法等によるマッチング処理を行い、マッチングスコアが最大となる撮像画像上での小さいドットの位置Ｑ_Ｓｉを求める（ステップ５２）。
３．１で求められたＰ_Ｓから、式（１）、式（２）を用いて、軸Ｌの姿勢すなわち角度Ｂ_１、Ｂ_２、および点Ｐの実空間上での位置を求める（ステップ５３）。
４．２で求められたＱ_Ｓｉから、式（１）、式（２）を用いて、直線ＣＱ_ｉの姿勢すなわち角度Ｂ_Ｑｉ１、Ｂ_Ｑｉ２、および点Ｑ_ｉの実空間上での位置を求める（ステップ５４）。ただし、式（１）、式（２）において、Ｄ_ＣＰの代わりにＤ_ＣＱを用いることとする。
５．点Ｐおよび点Ｑ_ｉの実空間上での位置から、図３１（ｂ）の関係を用いて、点Ｍ_ｉの位置と直線ＣＭ_ｉの姿勢すなわち角度Ｂ_Ｍｉ１およびＢ_Ｍｉ２を求める（ステップ５５）。
６．式（２）を用いて、点Ｍ_ｉの撮像画像上での位置Ｍ_Ｓｉを求める（ステップ５６）。ただし、式（２）において、Ｄ_ＣＰの代わりに、ＣとＭ_ｉ間の距離を用いることとする。
７．５で得られた角度Ｂ_Ｍｉ２に最も近い角度を図３３に示したテーブルの角度θ_１欄から選択し、マッチング処理に用いる３種類のテンプレート画像群の列を決定する（ステップ５７）。
８．撮像画像（図３２（ａ））をＣ_Ｓの周りで角度−Ｂ_Ｍｉ１だけ回転させた濃淡画像を得る（ステップ５８）。
９．７で決定された列に含まれる３種類の全てのテンプレート画像（図３１の例では１０８０個）と８で得られた濃淡画像との正規化相関手法等によるマッチング処理を点Ｍ_Ｓｉの回転後の点の近傍において逐次行う（ステップ５９）。
１０．９で得られたマッチングの度合い（スコア）が最大となるテンプレート画像を見つけ、対象としている図形が３種類の図形のうちの何れであるかを特定することにより、対象とした小さいドットＱ_ｉがＱ_１、Ｑ_２、Ｑ_３の何れであるかを特定する（ステップ６０）。
１１．特定されたＱ_ｉに基づき、立体２１の軸Ｌの周りの回転方向の姿勢を表わす角度Ｂ_３を算出する。すなわち、特定されたＱ_ｉを軸Ｚの周りで角度−Ｂ_１だけ回転させ、さらに軸Ｘの周りで角度−Ｂ_２だけ回転させた点の軸Ｚの周りの偏角を求め、これをＢ_３とする（ステップ６１）。
１２．以上により得られた角度Ｂ_１、Ｂ_２、およびＢ_３を立体２１の姿勢とする（ステップ６２）。
１３．最後に、撮像画像の孔の位置における色情報および輝度情報の少なくとも一方に基づき立体２１の指定された状態を検出する（ステップ６３）。
【００５４】
［第３の実施形態］
本実施形態では、３次元姿勢検出用立体２１の第１８の例として、同一の大きさの４個の回転対称図形と３個の非回転対称図形が表面に描かれた既知の楕円体を用い、その撮像画像から立体２１の姿勢を求める方法を説明する。第２の実施形態とは、楕円体の極Ｐの位置に描かれたドットとその周囲に描かれたドットとの判別方法のみが異なるので、この判別方法のみを説明する。
【００５５】
図３６は立体２１の表面上の図形の配置を説明するための図である。わかり易くするため、図３６では楕円体面上の図形を平面的に表現している。図３６（ａ）は上面図であり、図３６（ｂ）は側面図である。楕円体の極Ｐの位置および極Ｐを中心とした回転対称の位置Ｑ_１、Ｑ_２、Ｑ_３に全て同一の大きさのドットが描かれている。また、Ｑ_１、Ｑ_２、Ｑ_３に描かれた各ドットからＰに描かれたドットに向かう方向を姿勢の基準として、Ｐの描かれたドットとＱ_１、Ｑ_２、Ｑ_３に描かれた３個のドットとの各中点にそれぞれ異なる３個の図形が描かれている。さらに、立体２１に描かれた各ドットの３次元的な位置関係は図３１と同様であるとする。ただし、本実施形態では、Ｄ_ＰＱとＤ_ＱＱとが十分異なるように各ドットの位置を設定しておく。
【００５６】
図３７はドットを検出するためのテンプレート画像であり、実際にはガウシアンフィルタ等によるぼかし処理を施して作成されている。
【００５７】
図３８は撮像画像から楕円体の部分を切り出した部分画像を示している。図４０（ａ）は濃淡画像であり、図３８（ｂ）は楕円体の姿勢を求める手順を説明するための図である。説明のため、図３８（ｂ）では各ドットをその輪郭のみで示している。図３８（ｂ）において、Ｐ_Ｓは極Ｐの撮像画像上での位置、Ｑ_Ｓｉ（ｉ＝１，２，３）はそれぞれＱ_ｉの撮像画像上での位置である。
【００５８】
図３９は立体２１のＰに描かれたドットとＱ_１、Ｑ_２、Ｑ_３に描かれたドットのうちの１つを判別する手順を示すフローチャートである。
１．ドットを検出するためのテンプレート画像（図３７）と濃淡画像（図３８（ａ））との正規化相関手法等によるマッチング処理を行い、各ドットについて、マッチングスコアが最大となる撮像画像上での位置を全て求める（ステップ７１）。
２．求められた各点について、式（１）、式（２）を用いて、実空間上での位置を算出する（ステップ７２）。ただし、式（１）、式（２）において、Ｄ_ＣＰの代わりに全てＤ_ＣＱを用いることとする。本実施形態では、Ｄ_ＰＱとＤ_ＱＱとが十分異なるため、このような近似を用いても各ドットを正しく判別することができる。
３．求められた点が２個以下の場合は処理を中止する（ステップ７３）。
４．求められた点が３個以上の場合、これらの中から２個を選ぶ全ての組合わせについて、選んだ２個の点間の実空間における距離を求め、この距離と立体２１における既知のＤ_ＱＱとの差の絶対値が所定の閾値より大きくなる１組を選定する（ステップ７４）。
５．選定された組の２個の点をＰ_１、Ｐ_２、それ以外の任意の点をＱ_ｉとし、Ｐ_１とＱ_ｉ間の距離を求める（ステップ７５）。その際、Ｐ_１、Ｐ_２の実空間上での位置として、Ｄ_ＣＰにＣとＰ間の距離を代入した式（１）、式（２）により算出したものを用いる。
６．５で求められたＰ_１とＱ_ｉ間の距離と立体２１における既知のＤ_ＰＱとの差の絶対値が、所定の閾値より小さければＰ_１をＰに描かれたドットとし、そうでなければＰ_２をＰに描かれたドットとする（ステップ７６）。
【００５９】
以下、第２の実施形態における手順３以降を実行すればよい。
【００６０】
図４０に上記と同様の手順で姿勢を検出することができる立体２１の第１９の例として、第１４の例の立体におけるドットに代えて円が表面に描かれた楕円体を示す。わかり易くするため、図４０では楕円体面上の図形を平面的に表現している。図４０（ａ）は上面図であり、図４０（ｂ）は側面図である。楕円体の極Ｐの位置および極Ｐを中心とした回転対称の位置Ｑ_１、Ｑ_２、Ｑ_３に全て同一の大きさの円が描かれている。また、Ｑ_１、Ｑ_２、Ｑ_３に描かれた各円からＰに描かれた円に向かう方向を姿勢の基準として、Ｐに描かれた円とＱ_１、Ｑ_２、Ｑ_３に描かれた３個の円との各中点にそれぞれ異なる３個の図形が描かれている。さらに、立体２１に描かれた各円の３次元的な位置関係は図３１と同様であるとする。ただし、Ｄ_ＰＱとＤ_ＱＱとが十分異なるように各円の位置を設定しておく。
【００６１】
図４１は円を検出するためのテンプレート画像であり、実際にはガウシアンフィルタ等によるぼかし処理を施して作成されている。
【００６２】
図４２に上記と同様の手順で姿勢を検出することができる立体２１の第２０の例として、同一の大きさの４個の回転対称図形と３個の非回転対称図形が表面に描かれた正六角台形錐体を示す。わかり易くするため、図４２では正六角台形錐体面上の図形を平面的に表現している。図４２（ａ）は上面図であり、図４２（ｂ）は側面図である。Ｌは正六角台形錐体の軸であり、Ｐは軸Ｌと上底面とが交わる点である。Ｐの位置および１個おきの側面の左右の中央で、かつＰから等距離の位置Ｑ_１、Ｑ_２、Ｑ_３に全て同一の大きさのドットが描かれている。また、Ｑ_１、Ｑ_２、Ｑ_３に描かれた各ドットからＰに描かれたドットに向かう方向を姿勢の基準として、Ｐに描かれたドットとＱ_１、Ｑ_２、Ｑ_３に描かれた３個のドットとの各中点にそれぞれ異なる３個の図形が描かれている。ただし、ＰとＱ_ｉ間の距離Ｄ_ＰＱとＱ_ｉとＱ_ｊ間の距離Ｄ_ＱＱとが十分異なるように各ドットの位置を設定しておく。
【００６３】
図４３に上記と同様の手順で姿勢を検出することができる立体２１の第１０の例として、同一の大きさの４個の回転対称図形と３個の非回転対称図形が表面に描かれた放物面体を示す。わかり易くするため、図４３では放物面体の表面上の図形を平面的に表現している。図４３（ａ）は上面図であり、図４３（ｂ）は側面図である。Ｌは放物面体の軸であり、Ｐは軸Ｌと放物面体の表面とが交わる点、すなわち、放物面体の極である。放物面体の極Ｐの位置および極Ｐを中心とした回転対称の位置Ｑ_１、Ｑ_２、Ｑ_３に全て同一の大きさのドットが描かれている。また、Ｑ_１、Ｑ_２、Ｑ_３に描かれた各ドットからＰに描かれたドットに向かう方向を姿勢の基準として、Ｐに描かれたドットとＱ_１、Ｑ_２、Ｑ_３に描かれた３個のドットとの各中点にそれぞれ異なる３個の図形が描かれている。ただし、ＰとＱ_ｉ間の距離Ｄ_ＰＱとＱ_ｉとＱ_ｊ間の距離Ｄ_ＱＱとが十分異なるように各ドットの位置を設定しておく。
【００６４】
［第４の実施形態］
本実施形態では、３次元姿勢検出用立体２１の第２２の例として、同一の大きさでかつ互いに異なる色の４個の回転対称図形が表面に描かれた既知の楕円体を用い、その撮像画像から立体２１の姿勢を求める方法を説明する。
【００６５】
図４４は立体２１の表面上の図形の配置を説明するための図である。わかり易くするため、図４４では楕円体面上の図形を平面的に表現している。図４６（ａ）は上面図であり、図４４（ｂ）は側面図である。楕円体の極Ｐの位置に黒のドットが、極Ｐを中心とした回転対称の位置Ｑ_１、Ｑ_２、Ｑ_３にそれぞれ赤、緑、青のドットが描かれている。また、立体２１に描かれた各ドットの３次元的な位置関係は図３１と同様であるとする。さらに、ドットを検出するためのテンプレート画像は図３７と同様である。
【００６６】
図４５は撮像画像から楕円体の部分を切り出した部分画像を示している。図４５（ａ）は濃淡画像であり、図４５（ｂ）はカラー画像である。Ｐ_Ｓは極Ｐの撮像画像上での位置、Ｑ_Ｓｉ（ｉ＝１，２，３）はそれぞれＱ_ｉの撮像画像上での位置である。
【００６７】
図４６は立体２１の姿勢および状態を求める手順を示すフローチャートである。
１．ドットを検出するためのテンプレート画像（図３７）と濃淡画像（図４５（ａ））との正規化相関手法等によるマッチング処理を行い、各ドットについて、マッチングスコアが最大となる撮像画像上での位置を全て求める（ステップ８１）。
２．求められた各点におけるカラー画像（図４５（ｂ））の色情報から各ドットを判別する（ステップ８２）。
３．２で判別されたＰ_Ｓから、式（１）を用いて、軸Ｌの姿勢すなわち角度Ｂ_１およびＢ_２を求める（ステップ８３）。
４．Ｑ_ｉ（Ｑ_１、Ｑ_２、Ｑ_３のいずれか）に描かれた１個のドットについて、式（１）、式（２）を用いて、実空間上での位置を求める（ステップ８４）。ただし、式（１）、式（２）において、Ｄ_ＣＰの代わりにＤ_ＣＱを用いることとする。
５．４で求められたＱ_ｉに基づき、立体２１の軸Ｌの周りの回転方向の姿勢を表わす角度Ｂ_３を算出する（ステップ８５）。すなわち、求められたＱ_ｉを軸Ｚの周りで角度−Ｂ_１だけ回転させ、さらに軸Ｘの周りで角度−Ｂ_２だけ回転させた点の軸Ｚの周りの偏角を求め、これをＢ_３とする。
６．以上により得られた角度Ｂ_１、Ｂ_２、およびＢ_３を立体２１の姿勢とする（ステップ８６）。
７．最後に、撮像画像の孔の位置における色情報および輝度情報の少なくとも一方に基づき立体２１の指定された状態を検出する（ステップ８７）。
【００６８】
以上の実施形態で説明した、立体の姿勢を求める方法では、各々のマッチング処理において、立体表面に描かれた図形の一部分である比較的小さなサイズの図形を用いるので、立体全体ではなくその図形の領域にのみ対応したテンプレート画像を用いればよく、テンプレート画像の記憶容量およびマッチング処理の計算コストを小さく抑えることができる。
【００６９】
［第５の実施形態］
図４７は本発明の第５の実施形態の３次元姿勢入力装置を示した図である。
【００７０】
３次元姿勢入力装置１０３は位置算出部１３１と図形検出部１３２と姿勢算出部３３と状態検出部３４から構成され、３次元姿勢入力用立体（以下、立体と称す）１０１をカメラ１０２で撮像して得られた撮像画像から、立体１０１の３次元位置、姿勢、および状態を検出する。
【００７１】
図４８は立体１０１の第２３の例を示す図、図４９はその各部の位置関係を示す図である。
【００７２】
立体１０１は、表面に縁２１１を有する薄い円板（半径ｒ）からなる主要部１０１と、これに細い構造体を介して支持されている同一サイズで、同一色の２個のドット状の第１、第２の回転対称図形２２１、２３１からなっている。第１の回転対称図形２２１は中心が主要部２０１の回転対称軸上にあり、かつ主要部２０１の表面を含む平面と平行な、該表面から距離ｈ離れた平面上にある。第２の回転対称図形２３１は主要部２０１を含む平面上で、主要部２０１の領域外で、主要部２０１の回転対称軸から距離ｇ離れた位置にある。回転対称図形２２１および２３１は、サイズを小さくするほど、高精度にその位置検出ができ、また、立体１０１全体とのサイズ差が大きくなるため、高精度に立体１０１の３次元姿勢を算出できる。したがって、撮像画像上での識別可能な最小のサイズとすることが望ましい。
【００７３】
本実施形態では、立体１０１の位置は、撮像画像上において、主要部２０１の表面の縁２１１が楕円に近似できる程度にカメラ１０２から離れているとする。また、本実施形態では、立体１０１の傾きは、撮像画像上において、回転対称図形２２１が主要部２０１の表面の縁２１１の外部に出ない範囲内であるとする。
【００７４】
３次元位置・姿勢検出装置１０３は、まず、位置算出部１３１によって、撮像画像上における立体１０１の主要部２０１の輪郭から、立体１０１の位置を算出し、次に、図形検出部１３２によって、撮像画像上における回転対称図形２２１、２３１の位置を検出し、さらに、姿勢算出部１３３によって、立体１０１の主要部２０１の回転対称軸の向きおよび該回転対称軸周りの回転角度、すなわち立体１０１の姿勢を算出する。最後に、状態検出部１３４によって、撮像画像の孔の位置における色情報および輝度情報の少なくとも一方に基づいて立体１０１の指定された状態を検出する。
【００７５】
以下、本実施形態で、この立体１０１を用い、その撮像画像から立体１０１の３次元位置および姿勢を求める方法を説明する。
【００７６】
図５０はカメラ１０２のレンズの光学中心Ｃを原点とし、レンズの光軸をＺ軸とする、カメラ１０２を基準とした実空間での座標系において、位置（ｘ，ｙ，ｚ）に立体１０１が置かれている様子を示した図である。Ｘは実空間での水平方向の軸、Ｙは実空間での垂直方向の軸である。立体１０１は図４８に示したものと同一のデザインである。カメラ１０２については、撮像画像の光学的歪みを補正することにより、レンズ歪等のない理想的なカメラとして扱うことができる。
【００７７】
図５１はドット状の図形２２１、２３１を検出するためのテンプレート画像であり、実際にはガウシアンフィルタ等によるぼかし処理を施して作成されている。
【００７８】
図５２（ａ）は立体１０１の撮像画像、図５２（ｂ）は前記撮像画像における立体１の主要部２０１の輪郭２１１およびドット状の各回転対称図形２２１、２３１の検出結果を示す図である。ｕは画像の水平方向の軸、ｖは画像の垂直方向の軸である。図５３（ｂ）において、２１１ｓは検出した立体１の主要部２０１の縁２１１にフィットさせて得られた輪郭を表わす楕円、２２１ｓは検出されたドット状の円板２２１の位置、２３１ｓは検出されたドット状の円板２３１の位置である。ここで、撮像画像上の点を座標（ｕ，ｖ）で表すと、楕円２１１ｓは次式によって表すことができる。
【００７９】
【数２】

【００８０】
図５３は撮像画像上でのドット状の図形の位置２２１ｓ、２３１ｓと実空間での座標系ＸＹＺにおけるドット状の回転対称図形の位置との幾何的関係を示す図である。１０４は式（１）によって表される楕円２１１ｓに対応する円の中心、すなわち立体１０１の主要部２０１の表面の中心の実空間における位置である。１０５は前記円を含む平面、すなわち立体１０１の主要部２０１の表面およびその縁２１１を含む平面である。１０６は１０４を中心とする半径ｈの球である。１０７は２２１ｓに対応する点、すなわち回転対称図形２２１の実空間における位置、１０８は２３１ｓに対応する点、すなわち回転対称図形２３１の実空間における位置である。Ｃはカメラ１０２のレンズの光学中心である。位置１０７は、位置２２１ｓとレンズ中心Ｃとを結ぶ直線と球１０６との交点である。また、位置１０８は、位置２３１ｓとレンズ中心Ｃとを結ぶ直線と平面１０５との交点である。
【００８１】
図５３で示した立体１０１の主要部２０１の表面の中心の実空間における位置１０４および主要部２０１の表面およびその縁２１１を含む平面１０５は、式（１）の各係数α、β、γ、δ、ε、およびζを用いて、以下の方法によって求めることができる（例えば、文献 Ｋｅｎｉｃｈｉ Ｋａｎａｔａｎｉ， ”Ｇｅｏｍｅｔｒｉｃ Ｃｏｍｐｕｔａｔｉｏｎ ｆｏｒ Ｍａｃｈｉｎｅ Ｖｉｓｉｏｎ”， Ｐ．２２８ を参照）。
【００８２】
まず、次式で表されるマトリックスＱを作り、Ｑの固有値λ_１、λ_２、λ_３、および、これら固有値に対応する単位固有ベクトルｎ_１、ｎ_２、ｎ_３を計算する。ここで、固有値はλ_３＜０＜λ_１≦λ_２となるように選ぶ。
【００８３】
【数３】

【００８４】
ただし、ｆは図５３におけるｕｖ平面とレンズの光学中心Ｃとの距離であり、また、κはｄｅｔＱ＝−１となるように選ぶ。
【００８５】
次に、上記で得られたＱの固有値λ_１、λ_２、λ_３、および単位固有ベクトルｎ_１、ｎ_２、ｎ_３を用いて、次式によって、平面１０５の法線ベクトルｎ、および平面５とレンズの光学中心Ｃとの距離ｄを計算する。
【００８６】
【数４】

【００８７】
ただし、ｒは主要部２０１の面の縁２１１の半径である。平面１０５は、この法線ベクトルｎおよび距離ｄにより求められる。
【００８８】
さらに、上記で得られたＱの逆行列Ｑ^−１、および法線ベクトルｎを用いて、次式によって、レンズの光学中心Ｃを始点とする位置ベクトルｍを計算する。
【００８９】
【数５】

【００９０】
位置１０４は、この位置ベクトルｍと平面１０５との交点として求められる。
【００９１】
図５４は立体１０１の３次元位置および姿勢を求める手順を示すフローチャートである。
１．撮像画像（図５２（ａ））において、ハフ変換等を用いて、立体１０１の主要部１０１の輪郭１１１にフィットさせた楕円１１１ｓを検出し、式（１）における各係数α、β、γ、δ、ε、およびζを求める（ステップ３０１）。
２．１で求められたα、β、γ、δ、ε、およびζから、式（１）〜式（５）で示した方法により、立体１０１の主要部２０１の表面およびその縁２１１を含む平面１０５の法線ベクトルおよび主要部１０１の表面の中心の位置１０４を求め、さらに平面１０５を求める（ステップ３０２）。
３．ドット状の図形を検出するためのテンプレート画像（図５１）と撮像画像（図５２（ａ））との正規化相関手法等によるマッチング処理を行い、マッチングスコアが極大となる撮像画像上でのドット状の図形の位置を検出する（ステップ３０３）。
４．３で検出された位置の各々と楕円２１１ｓとの位置関係から各位置が回転対称図形２２１および回転対称図形２３１のいずれに対応しているかを判別する（ステップ３０４）。すなわち、検出された位置のうち楕円２１１ｓの内側のものを２２１ｓ、外側のものを２３１ｓとする。
５．図５３に示した幾何的関係から、位置１０７および位置１０８を求める（ステップ３０５）。
６．位置１０４および位置１０７から、立体１０１の主要部２０１の回転対称軸の向きを求め、位置１０４および位置１０８から、立体１０１の該回転対称軸周りの回転角度を求める（ステップ３０６）。
７．２で求められた位置１０４を立体１０１の３次元位置とし、６で求められた立体１０１の主要部２０１の回転対称軸の向きおよび該回転対称軸周りの回転角度を立体１の３次元姿勢とする（ステップ３０７）。
８．最後に、撮像画像の孔の位置における色情報および輝度情報の少なくとも一方に基づいて立体１０１の指定された状態を検出する（ステップ３０８）。
【００９２】
［第６の実施形態］
第５の実施形態では、立体１０１に設けた第１および第２の回転対象図形２２１、２３１が同一の色であり、立体１０１の傾きは、撮像画像上において、図形１２１が主要部２０１の縁２１１の外部に出ない範囲内であるとした。
【００９３】
これに対し、本実施形態では、立体１０１に設けた第１および第２の回転対象図形を互いに異なる色とし、立体１０１の傾きの制限をなくすものである。
【００９４】
図５５は立体１０１の一例である。この立体１０１は、表面に縁２１２を有する薄いリングである主要部２０２と、これに細い構造体を介して支持されている同一サイズで互いに異なる色の２個のドット状の図形２２２および２３２が描かれた円板とから構成されている。ここでは、図形２２２が赤、図形２３２が青でそれぞれ描かれているとする。図形２２２および２３２のサイズについては、第５の実施形態と同様、撮像画像上で識別可能な最小のサイズとすることが望ましい。
【００９５】
本実施形態における３次元位置および姿勢の検出方法は、検出されたドット状の図形２２２、２３２の位置の各々が２個のドット状の図形２２２、２３２のいずれに対応しているかを判別する手順のみが、第５の実施形態と異なっている。そのため、本実施形態の方法も、第５の実施形態と同様、図５０ないし図５３を用いて説明する。ただし、本実施形態においては、図５２に相当する撮像画像が色情報を有するカラー画像であるとする。
【００９６】
図５６は立体１０１の３次元位置および姿勢を求める手順を示すフローチャートである。
１．撮像画像（図５２（ａ））において、ハフ変換等を用いて、立体１０１の主要部１０２の輪郭にフィットさせた楕円２１２ｓを検出し、式（１）における各係数α、β、γ、δ、ε、およびζを求める（ステップ４０１）。ただし、ここでは撮像画像の濃淡情報のみ使用する。
２．１で求められたα、β、γ、δ、ε、およびζから、式（３）〜式（７）で示した方法により、立体１０１の主要部２０２の表面およびその縁２１２を含む平面１０５の法線ベクトルおよび主要部２０２の表面の中心の位置１０４を求め、さらに平面１０５を求める（ステップ４０２）。
３．ドット状の図形を検出するためのテンプレート画像（図５１）と撮像画像（図５２（ａ））との正規化相関手法等によるマッチング処理を行い、マッチングスコアが極大となる撮像画像上でのドット状の図形の位置を検出する（ステップ４０３）。ただし、ここでは撮像画像の濃淡情報のみ使用する。
４．３で検出された位置の各々における撮像画像（図５２（ａ））の色情報から各位置が図形２２２および図形２３２のいずれに対応しているかを判別する（ステップ４０４）。すなわち、検出された位置の画素が赤の色情報を有していればその位置を２２２ｓ、青の色情報を有していればその位置を２３２ｓとする。
５．図５３に示した幾何的関係から位置１０７と位置１０８を求める（ステップ４０５）。
６．位置１０４および位置１０７から立体１０１の主要部１０２の回転対称軸の向きを求め、位置１０４および位置１０８から、立体１０１の該回転対称軸周りの回転角度を求める（ステップ４０６）。
７．３で求められた位置１０４を立体１０１の３次元位置とし、５で求められた立体１０１の主要部２０２の回転対称軸の向き、および該回転対称軸周りの回転角度を立体１０１の３次元姿勢とする（ステップ４０７）。
８．撮像画像の孔の位置における色情報および輝度情報の少なくとも一方に基づき立体１０１の指定された情報を検出する（ステップ４０８）。
【００９７】
図５７ないし図５９は第５または第６の実施形態で示した手順で３次元位置および姿勢を検出することができる立体１０１のその他の例を示している。ただし、立体１０１に設けた第１および第２の回転対象図形が同一の色の場合には、第５の実施形態で示した手順が適用でき、立体１０１に設けた第１および第２の回転対象図形が互いに異なる色の場合には、第５および第６の実施形態で示した手順が適用できる。
【００９８】
図５７に示す立体は、リングである主要部２０３と、これに細い構造体を介して支持されている同一サイズの２個の小球２２３、２３３とから構成されている。
【００９９】
図５８に示す立体は、円板である主要部２０４と、これに支持されており、かつ表面の一部にそれぞれ同一サイズのドット状の図形２２４、２３４が描かれた棒状および板状の構造体とから構成されている。
【０１００】
図５９に示す立体は、リング状の主要部２０５と、これに支持されており、かつ表面の一部にそれぞれ同一サイズのドット状の図形２２５、２３５が描かれた筒状および板状の構造体とから構成されている。
【０１０１】
以上説明したように、本実施形態の立体の３次元位置、姿勢、および状態を求める方法では、立体の主要部を円板もしくはリングにしたので、位置の算出と姿勢の算出とを分離でき、高精度を確保しつつ、計算コストを抑えることができる。また、立体の一部に描かれた比較的小さなサイズの図形を用いるので、立体全体ではなくその図形の領域にのみ対応したテンプレート画像を用いればよく、マッチング処理におけるテンプレート画像の記憶容量および計算コストを小さく抑えることができる。その結果、高精度を確保しつつ、全体の計算コストを大幅に抑えることができる。
【０１０２】
なお、以上の実施形態では、立体１０１の２個の回転対称図形として、同一サイズ、同一形状の図形である例を示した。しかし、これらの図形を検出する際に必要となるテンプレート画像を変えることにより、２個の回転対称図形は、互いに異なるサイズ、異なる形状（例えば、ドット状の図形と円状の図形など）であっても、本発明は同様の効果を奏することができる。
【０１０３】
〔第７の実施形態〕
図６０は本発明の３次元姿勢入力用状態指定機能付立体の第２７の例と３次元姿勢入力装置を示す図、図６１は立体５１１の画像を示す図である。
【０１０４】
立体５１１は、表面の少なくとも一部分が球面であり、立体５１１の球面には「Ａ」の文字が描かれている。
【０１０５】
３次元位置・姿勢検出装置５１３は位置算出部５１３ａと姿勢算出部５１３ｂと状態検出部５１３ｃからなる。位置算出部５１３ａは、立体５１１をカメラ５１２で撮影して得られた画像（図６１（ａ））に二値化処理を施すことで立体５１１の輪郭画像（図６１（ｂ））を得、その輪郭画像上での立体の輪郭の位置と大きさとから立体５１１の実空間での位置ｘ，ｙ，ｚを算出する。姿勢算出部５１３ｂは、輪郭画像（図６１（ｂ））をもとに立体５１１（図６１（ａ））上での立体部位を切り出し、その画像（図６１（ｃ））と図形の種々の姿勢に対応する図形の全体または部分のテンプレート画像群とを比較することにより立体５１１の姿勢θ，φ，ψを算出する。状態検出部５１３ｃは撮像画像の孔の位置における色情報および輝度が少なくとも一方に基づき立体５１１の指定された位置を検出する。
【０１０６】
図６２は立体５１１の例を示している。この立体５１１では球面に内接する正五角形と正六角形とから構成される準正多面体の３個の頂点が該球面と接する点の位置に全て同じサイズのドットが、また該準多面体の中心と該準多面体の面の重心とを結ぶ直線が該球面と交わる点の位置に１個の文字が、それぞれ設けられている。これらのドットは、サイズを小さくするほど、高精度にその位置検出ができ、また、立体５１１とのサイズ差が大きくなるため、高精度に立体の３次元姿勢を算出できる。したがって、撮像画像上で識別可能となる最小のサイズとすることが望ましい。
【０１０７】
図６３は、レンズの光学中心Ｃを原点とし、レンズの光軸をＺ軸とするカメラ５１２を基準とする実空間での座標系において、位置（ｘ，ｙ，ｚ）に立体５１１が置かれている様子を示した図である。立体５１１は、図６２に示したものと同一のデザインである。カメラ５１２については、撮像画像の光学的歪みを補正することにより、レンズ歪等のない理想的なカメラとして扱うことができる。
【０１０８】
図６４は立体５１１を撮像した画像を示している。ｕは画像の水平方向の軸、ｖは画像の垂直方向の軸である。図において、５１４は、検出した立体のエッジに楕円をフィットさせて得られた輪郭である。ξは、輪郭の重心と画像原点とを結ぶ直線がｕ軸となす角である。ｄは輪郭の重心と画像原点との距離である。ｂは輪郭楕円の長径である。
【０１０９】
図６５は、撮像画像上でのドットの位置Ｐ_ｓと実空間でのドットの位置Ｐとの幾何的関係を示す図である。図において、点Ｐは、撮像画像上でのドットの位置Ｐ_ｓとレンズの光学中心Ｃとを結ぶ直線が立体５１１の球面と交わる点である。
【０１１０】
図６６は、撮像画像上で検出された３個のドットの位置（図６６（ａ））と、これらドットの位置を基にして計算により作成された３枚の文字用テンプレートを示す図（図６６（ｂ））である。ここで、ドットが図の位置となるような球の姿勢は３通りに限られる。それら３通りの姿勢の全てについて、例えば以下のような方法でテンプレートを作成する。
【０１１１】
図６７および図６８は図形のテンプレートの計算機による作成方法を説明するための図である。図６７において、Ｃ_Ｖは無限遠に配置してある仮想カメラである。Ｏは計算機内の仮想空間の原点、Ｘは水平方向の軸、Ｙは垂直方向の軸、Ｚは仮想カメラＣ_Ｖの光軸である。Ｒは軸Ｚ上の点であり、距離ＯＲは仮想球の半径に等しい。線分ＯＧは線分ＯＲを軸Ｚから角度Ｑ_１だけ回転させた線分である。Ｓ_ＶはＧを通り線分ＯＧに垂直な仮想平面である。Ｇを通る直線Ｔは、軸Ｙと軸Ｚとを含む平面上にあり、線分ＯＧに垂直である。角度Ｑ_２は仮想平面Ｓ_Ｖが線分ＯＧを軸として回転した際の、直線Ｔからの偏角である。なお、図形はその重心がＧに一致するように仮想平面Ｓ_Ｖ上に配置してある。説明のために、図形については輪郭のみを示してある。
【０１１２】
図６８は仮想平面Ｓ_Ｖ上に配置した図形をＸＹ平面に射影して得られた仮想カメラＣ_Ｖの画像である。Ｇ_ＴはＧの画像上での位置である。テンプレートは、この仮想カメラＣ_Ｖの画像を図形を含みかつ位置Ｇ_Ｔを中心とする矩形領域で切り出した部分画像にガウシアンフィルタ等によるぼかし処理を施して作成する。
【０１１３】
図６９は、図６２の立体５１１を撮像した画像から、図６０の装置によって、実空間での立体５１１の位置・姿勢を算出、状態を検出する方法を示すフローチャートである。なお、立体５１１の球面部の半径は既知であるとする。
１．立体５１１を撮像した画像の光学的歪みを補正し、補正画像を得る（ステップ５３１）。
２．補正画像において、ハフ変換等を用いて球の輪郭である楕円を検出し、図６４で示した楕円のパラメータ（ξ，ｄ，ｂ）を算出する（ステップ５３２）。
３．図６３で示した立体５１１の位置（ｘ，ｙ，ｚ）と図６４で示した楕円のパラメータ（ξ，ｄ，ｂ）との間には、球の半径が既知であるため、固定した１対１の対応関係がある。この対応関係を表わす方程式をＮｅｗｔｏｎ−Ｒａｐｈｓｏｎ法等を用いて解くことにより、（ξ，ｄ，ｂ）から（ｘ，ｙ，ｚ）を算出する（ステップ５３３）。
４．補正画像において、１個のドット図形用のテンプレートを用いたパターンマッチング処理により比較し、３個のドットの画像上での位置を各々独立に検出し、図６５に示した幾何的関係を用いて３個のドットの実空間での位置を算出する。算出された３個のドットの位置と立体５１１の位置（ｘ，ｙ，ｚ）とを基に、立体５１１の姿勢を算出する（ステップ５３４）。ただし、３個のドットの位置を基にした場合、得られる立体５１１の姿勢は３通りあるため、それら３通りの姿勢の全てを算出し、これらを立体５１１の姿勢の候補とする。立体５１１の姿勢の算出方法は次の通りである。立体５１１のモデルとして、表面に３個のドットを有する球を考え、図６５に示す実空間での各ドットの位置をそれぞれＰ１、Ｐ２、Ｐ３とする。各ドットは球に固定されているので、球の姿勢が変化すると、各位置Ｐ１、Ｐ２、Ｐ３も変化する。算出された３個のドットの位置をＱ１、Ｑ２、Ｑ３として、
【０１１４】
【数６】

を最小にする球の姿勢を求め、これを立体５１１の姿勢とする。ただし、
【０１１５】
【外１】

はそれぞれＰ１とＱ１、Ｐ２とＱ２、Ｐ３とＱ３の距離である。
５．算出された３通りの姿勢の全てについて、図６７および図６８で説明した方法等により、文字の部分を切り出したテンプレートを３個作成する。ドットと文字との位置関係は既知であり、この関係から補正画像上での文字の位置（重心位置）を求めることができるため、この位置を基準として、これら３個のテンプレートを用いたパターンマッチング処理を逐次行う。得られた各テンプレートと補正画像とのマッチングの程度を比較し、最も評価の高いテンプレートに対応する姿勢を選択し、これを立体５１１の正確な姿勢とする（ステップ５３５）。
【０１１６】
〔第８の実施形態〕
図７０は、立体５１１の球面上に設けられる図形の例である。球面に内接する正二十面体の２個の頂点が該球面と接する点の位置に互いに異なる図形が、該正二十面体の中心と該正二十面体の面の重心とを結ぶ直線が該球面と交わる点の位置に４個のドットが、それぞれ設けられている。これらのドットは、第２の実施形態と同様、撮像画像上で識別可能な最小のサイズとすることが望ましい。図７８（ａ）において図形は「Ｏ」と「ハ」に似た図形であり、図７０（ｂ）において図形は、「コ」と「ハ」に似た図形である。なお、図では平面的に示してある。
【０１１７】
本実施形態の立体においても、第２の実施形態と同様の装置により３次元位置・姿勢を検出することができる。本実施形態によれば、文字の代わりに単純な図形を用いているため、さらなる計算コストの低減、処理の高速化が図れる。
【０１１８】
なお、第７および第８の実施形態においても、立体５１１として、内部に赤外線光源を備えた半透明の立体を用い、カメラ５１２として、赤外線通過フィルタを備える等の方法により赤外線像を撮像可能なカメラを用いてもよい。こうすることにより、撮像画像上で立体５１１の部位を切り出す際に、環境光の影響を少なくし、立体５１１の輪郭を鮮明にすることができる。
【０１１９】
また、第７および第８の実施形態では、立体２１１の球面に内接する正多面体もしくは準正多面体の頂点の位置に３ないし４個の同じサイズのドットを設け、該多面体の中心と該多面体の面の重心とを結ぶ直線が球面と交わる点の位置に１ないし２個の互いに異なる図形を設けた例、および、ドットと互いに異なる図形との位置関係をこれとは逆にした例を示した。しかし、これらのドットの代わりに円など他の回転対称図形を用いてもよく、また、これら回転対称図形のサイズは必ずしも全て同じでなくてもよく、個数は５個以上であっても、画像上に撮像されていれば２個だけであってもよい。さらに、回転対称図形と互いに異なる図形との位置関係は、既知であれば、必ずしも上記実施形態で示した関係でなくてもよい。
【０１２０】
なお、以上説明した３次元位置、姿勢、状態入力方法は専用のハードウェアにより実現されるもの以外に、以上説明した方法を実行するためのプログラムを、コンピュータ読み取り可能な記録媒体に記録して、この記録媒体に記録されたプログラムをコンピュータシステムに読み込ませ、実行するものであってもよい。コンピュータ読み取り可能な記録媒体とは、フロッピーディスク、光磁気ディスク、ＣＤ−ＲＯＭ等の記録媒体、コンピュータシステムに内蔵されるハードディスク装置等の記憶装置を指す。さらに、コンピュータ読み取り可能な記録媒体は、インターネットを介してプログラムを送信する場合のように、短時間の間、動的にプログラムを保持するもの（伝送媒体もしくは伝送波）、その場合のサーバとなるコンピュータシステム内部の揮発性メモリのように、一定時間プログラムを保持しているものも含む。
【０１２１】
【発明の効果】
以上説明したように、本発明によれば、下記の効果がある。
１．立体の３次元姿勢および状態を少ない計算コストで検出・入力することができ、３次元マウスとして応用できる。
２．立体が球または円板の場合には、立体の３次元位置をも少ない計算コストで検出・入力することができる。
【図面の簡単な説明】
【図１】本発明の３次元装置入力用状態指定機能付立体の第１の例を示す図である。
【図２】本発明の３次元装置入力用状態指定機能付立体の第２の例を示す図である。
【図３】本発明の３次元装置入力用状態指定機能付立体の第３の例を示す図である。
【図４】本発明の３次元装置入力用状態指定機能付立体の第４の例を示す図である。
【図５】本発明の３次元装置入力用状態指定機能付立体の第５の例を示す図である。
【図６】本発明の３次元装置入力用状態指定機能付立体の第６の例を示す図である。
【図７】本発明の３次元装置入力用状態指定機能付立体の第７の例を示す図である。
【図８】本発明の３次元装置入力用状態指定機能付立体の第８の例を示す図である。
【図９】本発明の３次元装置入力用状態指定機能付立体の第９の例を示す図である。
【図１０】球外面の反射率のスペクトル分布と内部光源および塗装面のスペクトル分布を示す図である。
【図１１】本発明の３次元装置入力用状態指定機能付立体の第１０の例を示す図である。
【図１２】本発明の３次元装置入力用状態指定機能付立体の第１１の例を示す図である。
【図１３】システムの概要を示す図である。
【図１４】球の画像と姿勢を表すクオータニオンとを関連づける姿勢検出用テンプレートを示す図である。
【図１５】球の画像の孔付近における画像信号パターンを説明するための図である。
【図１６】発光輝度の連続的な制御により連続値の指定を可能にする方法を説明するための図である。
【図１７】時系列的な発光パターンと多値の指定を示す図である。
【図１８】本発明の３次元姿勢検出用立体の第１２の例と第１の実施形態の３次元姿勢入力装置のブロック図である。
【図１９】立体２１の例と、立体２１の表面上の図形の配置を説明するための図である。
【図２０】図形のテンプレート画像の計算機による作成方法を説明するための図である。
【図２１】仮想平面Ｓ_Ｖ上の図形をＸＹ平面に投影して得られた仮想カメラＣ_Ｖの画像を示す図である。
【図２２】作成されたテンプレート画像群が配置されたテーブルを示す図である。
【図２３】楕円体の軸Ｌと実空間における姿勢と極Ｐの撮像画像上における位置の関係を説明するための図である。
【図２４】撮像画像から楕円体の部分を切り出した部分画像を示す図である。
【図２５】第１の実施形態における、立体２１の姿勢を求める手順を示すフローチャートである。
【図２６】本発明の３次元装置入力用状態指定機能付立体の第１３の例を示す図である。
【図２７】本発明の３次元装置入力用状態指定機能付立体の第１４の例を示す図である。
【図２８】本発明の３次元装置入力用状態指定機能付立体の第１５の例を示す図である。
【図２９】本発明の３次元装置入力用状態指定機能付立体の第１６の例を示す図である。
【図３０】本発明の３次元装置入力用状態指定機能付立体の第１７の例の表面上の図形の配置を説明するための図である。
【図３１】立体２１に描かれた各ドットの３次元的な位置関係を説明するための図である。
【図３２】大きいドット、小さいドットを検出するためのテンプレート画像を示す図である。
【図３３】互いに異なる３種類の図形を検出するためのテンプレート画像群が配置されたテーブルを示す図である。
【図３４】撮像画像から楕円体の部分を切り出した部分画像を示す図である。
【図３５】第２の実施形態の３次元姿勢入力装置における立体２１の姿勢を求める手順を示すフローチャートである。
【図３６】立体２１の第１８の例の表面上の図形の配置を説明するための図である。
【図３７】ドットを検出するためのテンプレート画像を示す図である。
【図３８】撮像画像から楕円体の部分を切り出した部分画像を示す図である。
【図３９】第３の実施形態の３次元姿勢入力装置における立体２１の姿勢を求める手順を示すフローチャートである。
【図４０】本発明の３次元装置入力用状態指定機能付立体の第１９の例を示す図である。
【図４１】円を検出するためのテンプレート画像を示す図である。
【図４２】本発明の３次元装置入力用状態指定機能付立体の第２０の例を示す図である。
【図４３】本発明の３次元装置入力用状態指定機能付立体の第２１の例を示す図である。
【図４４】本発明の３次元装置入力用状態指定機能付立体の第２２の例の表面上の図形の配置を説明するための図である。
【図４５】撮像画像から楕円体の部分を切り出した部分画像を示す図である。
【図４６】第４の実施形態の３次元姿勢入力装置における立体２１の姿勢を求める手順を示すフローチャートである。
【図４７】本発明の第５の実施形態の３次元姿勢入力装置の構成を示す図である。
【図４８】本発明の３次元装置入力用状態指定機能付立体１０１の第２３の例を示す図である。
【図４９】図４８に示す立体１０１における主要部１０１と回転対称図形１２１、１３１との位置関係を示す図である。
【図５０】カメラ１０２のレンズの光学中心Ｃを原点とし、レンズの光軸をＺ軸とする、カメラ１０２を基準とした実空間での座標系において、立体１０１が置かれている様子を示す図である。
【図５１】ドット状の図形を検出するためのテンプレート画像を示す図である。
【図５２】立体１０１の撮像画像（同図（ａ））と、撮像画像における立体１０１の主要部２０１の輪郭および各ドット状の図形の検出結果を示す図である。
【図５３】撮像画像上でのドット上の図形の位置と実空間での座標系におけるドット状の図形２２１、２３１との幾何的関係を示す図である。
【図５４】図４８に示す立体１０１の３次元位置および姿勢を求める手順を示すフローチャートである。
【図５５】図４８に示す立体１０１の第２の例を示す図である。
【図５６】図５５に示す立体１０１の３次元位置および姿勢を求める手順を示すフローチャートである。
【図５７】立体１０１の第３の例を示す図である。
【図５８】立体１の第４の例を示す図である。
【図５９】立体１０１の第５の例を示す図である。
【図６０】本発明の第７の実施形態の３次元姿勢入力装置の構成図である。
【図６１】図６０中のカメラ２１２で得られた画像（同図（ａ））、３次元姿勢入力装置で得られる画像（同図（ｂ）（ｃ））を示す図である。
【図６２】立体２１１の例を示す図である。
【図６３】カメラ２１２を基準とする実空間での座標系において立体２１１が置かれている様子を示す図である。
【図６４】立体２１１を撮像した画像を示す図である。
【図６５】撮像画像上でのドットの位置Ｐと実空間でのドット位置ｐとの幾何学的関係を示す図である。
【図６６】撮像画像上で検出された３個のドットの位置と、これらドットの位置により作成された３枚の文字用テンプレートを示す図である。
【図６７】計算機を用いた、図形テンプレートの作成方法を説明するための図である。
【図６８】計算機で算出された、仮想カメラの画像を示す図である。
【図６９】第７の実施形態における、立体２１１の位置・姿勢を算出する処理を示すフローチャートである。
【図７０】立体２１１の球面上に設けられる図形の例を示す図である。
【符号の説明】
１ 球
１ａ，１ｂ，１ｃ 孔
２ 電源
２ａ スイッチ
２ｂ 液晶コントローラ
３ａ 青色ＬＥＤ
３ｂ 緑色ＬＥＤ
３ｃ 拡散板
４ 光源ユニット
４ａ ボディ
４ｂ 拡散面
４ｃ 青色ＬＥＤ
５ 信号ユニット
５ａ ハウジング
５ｂ ピストン
５ｃ スプリング
５ｄ 端面
６ ワイヤー
７ スライダー
８ カラー液晶
９ 光ファイバー
９ａ 拡散面
１０ 支持体
１０ａ 貫通穴
１１ 支持棒
１１ａ レバー
１１ｂ 笠状パレット
１２ 円板
１２ａ 円板部
１２ｂ 筒状体
１２ｃ 拡散板
１３ カラーカメラ
２１ ３次元姿勢入力用立体
２２ カメラ
２３ ３次元姿勢入力装置
３１ 軸姿勢算出部
３２ 立体姿勢算出部
３３ 状態検出部
４１〜４８，５１〜６３，７１〜７６，８１〜８７ ステップ
１０１ ３次元位置・姿勢入力用立体
１０２ カメラ
１０３ ３次元位置・姿勢検出装置
１０４ 楕円２１１ｓに対応する円の中心
１０５ 円を含む平面
１０６ 中心１０４の球
１０７ 図形２２１の実空間の位置
１０８ 位置１３１ｓとレンズ中心Ｃとを結ぶ直線と平面１０５の交点
１３１ 位置算出部
１３２ 図形検出部
１３３ 姿勢算出部
１３４ 状態検出部
２０１，２０２，２０３ 主要部
２１１ 縁
２２１，２２２，２２３ 第１の回転対称図形
２３１，２３２，２３３ 第２の回転対称図形
２１１ｓ 縁１１１にフィットされて得られた輪郭を表わす円
２２１ｓ 検出されたドット状の円板２２１の位置
２３１ｓ 検出されたドット状の円板２３１の位置
３０１〜３０８，４０１〜４０８ ステップ
５１１ 立体
５１２ カメラ
５１３ ３次元位置・姿勢検出装置
５１３ａ 位置算出部
５１３ｂ 姿勢算出部
５１３ｃ 状態検出部
５１４ 輪郭
５３１〜６３６ ステップ[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a three-dimensional posture input solid and a three-dimensional posture input method and apparatus.
[0002]
[Prior art]
As a technique that can be applied as a three-dimensional mouse, Patent Document 1 discloses a three-dimensional position / posture measuring apparatus.
[0003]
[Patent Document 1]
Japanese Patent No. 3299441
[0004]
[Problems to be solved by the invention]
However, in the technique of Patent Document 1, the use of CRT is indispensable and cannot be applied when a liquid crystal display or the like is used as a display.
[0005]
An object of the present invention is to provide a solid with a three-dimensional posture input state designation function capable of inputting a state together with the three-dimensional posture of the solid.
[0006]
Another object of the present invention is to provide a method, an apparatus, a program, and a recording medium for inputting a three-dimensional posture and state of an object using the three-dimensional posture input state specifying function-added solid.
[0007]
[Means for Solving the Problems]
The present invention is a system for photographing a solid whose figure is drawn on the surface and whose posture changes, and measuring the posture in the space of the solid from the outline of the solid on the image and the appearance pattern of the figure. After providing the light emission part of the spectral distribution S1 on the three-dimensional image, the image of the filter F2 of the spectral distribution S2 different from the filters F1 and S1 of the spectral distribution S1 is taken, the posture is measured using the image of the filter F2, On the other hand, the state (mode) designation is input by examining the light emission or color state of the light emission or color change part based on the luminance distribution in the vicinity of the light emission or color change part in the image by the filter F1.
[0008]
DETAILED DESCRIPTION OF THE INVENTION
Next, embodiments of the present invention will be described with reference to the drawings.
[0009]
FIGS. 1-12 image | photograph the ball | bowl with the position and attitude | position which the dot was provided in the surface, and measure the position and attitude | position in the space of the ball | bowl from the outline and the appearance pattern of a dot on an image. It is a figure which shows the solid with a state designation | designated function for 3D attitude | position input which the click function which is a kind of state (mode) designation | designated function implement | achieves in a system.
[0010]
FIG. 1 shows a first example of a solid with a state designating function for three-dimensional posture input according to the present invention, and shows a posture input ball with an indirect illumination click function. The sphere 1 is hollow, and three holes 1a, 1b, 1c are formed on the outer surface, and a blue LED (470 nm) 3a and a green LED (630 nm) 3b that can blink in the vicinity of the holes 1a, 1b, 1c. Is arranged. The outer surface of the sphere 1 is painted red (matte). When the switch 2a of the power supply 2 is pressed, the blue LED 3a or the green LED 3b is turned on. Since this sphere 1 has a structure equivalent to that of an integrating sphere, when lit, blue light or green light is diffusely irradiated to the outside through the holes 1a, 1b, and 1c. Thereby, this sphere 1 can represent three kinds of states (modes).
[0011]
FIG. 2 shows a second example of a solid with a state designation function for 3D posture input according to the present invention, and shows a posture input ball with an indirect illumination click function. The sphere 1 has a structure similar to that of the sphere 1 in FIG. 1 and the outer surface is painted red (matte), but a diffusion plate 3c is provided in front of the blue LED 3a in order to prevent direct light irradiation.
[0012]
FIG. 3 shows a third example of a solid with a state designation function for three-dimensional posture input according to the present invention, and shows a ball for a position / posture input device with a direct illumination click function. A light source unit 4 having a transparent diffusing surface 4b with a transparent diffusing surface 4b and a built-in blue LED 4c is disposed in the hole 1a, 1b, 1c of the sphere 1 with a transparent acrylic body 4a. Has been. When the switch 2a of the power supply 2 is pressed, the blue LED 4c is turned on, the transparent acrylic body 4a serves as a light guide, and blue light is diffusely irradiated from the transparent diffusion surface 4b through the holes 1a, 1b, 1c. The outer surface of the sphere 1 is painted red (matte).
[0013]
FIG. 4 shows a fourth example of a three-dimensional posture input state specifying function solid according to the present invention, and shows a posture input ball with a mechanical click function. The ball 1 has a signal unit 5 disposed in the holes 1a, 1b, and 1c. The outer surface is red painted (matte) like the spheres of FIGS. 1 to 3, but the inner surface is black painted (matte). It has become. The signal unit 5 includes a housing 5a, a piston 5b disposed in the housing 5a, and a spring 5c that constantly urges the piston 5b to the left in the figure. The end surface 5d of the piston 5b is painted blue (matte), and the piston 5b The piston 7 provided outside the sphere 1 is moved against the spring force of the spring 5 c via the wire 6. Usually, since the blue painted surface of the end surface 5d of the piston 5b of the signal unit 5 is exposed on the surface of the sphere 1 by the spring 5c, blue is confirmed in the holes 1a, 1b and 1c. When the piston 7 at the gripping portion is operated, the blue painted surface of the end surface 5d of the piston 5b of the signal unit 5 is buried in the sphere 1, so that black is confirmed at the portions of the holes 1a, 1b, 1c.
[0014]
FIG. 5 shows a fifth example of the three-dimensional posture input state specifying function solid according to the present invention, and shows a color liquid crystal type click input posture sphere with a click function. A color liquid crystal 8 is disposed in the holes 1 a, 1 b, and 1 c of the sphere 1, and a liquid crystal controller 2 c is disposed in the power source 2. Similar to the sphere 1 of FIG. 4, the sphere 1 has a black paint (matte) on the inner surface. The outer surface is red painted (matte) like the sphere 1 of FIGS. The color and display pattern of the color liquid crystal 8 are changed by operating the switch 2a.
[0015]
FIG. 6 shows a sixth example of a solid with a state designation function for three-dimensional posture input according to the present invention, and shows a posture input ball with a mechanical click function. End portions of the optical fiber 9 having a transparent diffusion surface 9a are inserted into the holes 1a, 1b, and 1c of the solid 1. A cylindrical support 10 is attached to the outside of the sphere 1, a pin 11a protruding from the through hole 10a of the support 10 is fixed to the side surface, and a part of the support bar 11 that supports the cap-shaped pallet 11b at the end. Is inserted into the support 10. The end face of the shaded pallet 11b has a circular blue paint (matte) part, and the other parts are black paint (matte). In the state where the lever 11a of the support bar 11 is in the position directly above, since there is a black coating portion of the shaded pallet 11b on the end face of the optical fiber 9, black is confirmed in the holes 1a, 1b, and 1c portions. In a state where the lever 11a is in an oblique position, since the blue coating portion of the shaded pallet 11b is present on the end surface 9a of the optical fiber 9, blue is confirmed in the holes 1a, 1b and 1c.
[0016]
FIG. 7 shows a seventh example of a solid with a state designation function for three-dimensional posture input according to the present invention, and shows a posture input disc with a direct illumination type click function. A blue LED 3a is disposed in a cylindrical body 12b provided at the center of the disk portion 12a of the circular disk 12, and a diffusion plate 12c is disposed at the open end of the cylindrical body 12b. In this example, illumination is turned on by the same principle as the sphere 1 shown in FIG. A diffusion plate 12c is provided so that the lighting state can be confirmed even from an oblique direction. The disk 12 has the same external appearance as a solid 103 shown in FIG. 48 described later, and has a first rotational symmetry 12d and a second rotational symmetry figure 12e.
[0017]
FIG. 8 shows an eighth example of a solid with a state designation function for three-dimensional posture input, and shows a position / posture input device disc with a mechanical click function. This example operates on the same principle as the sphere 1 shown in FIG.
[0018]
FIG. 9 shows a ninth example of a solid with a state designation function for three-dimensional posture input, and shows a disk for an input device with a color liquid crystal type click function. This example operates on the same principle as the sphere 1 shown in FIG.
[0019]
FIG. 10 shows the spectral distribution of the reflectance of the outer surface of the sphere and the spectral distribution of the internal light source and the painted surface. The light source is a blue light emitting diode, the painted surface is blue, and the outer surface of the sphere shows the luminance and reflectance when painted in red.
[0020]
FIG. 11 shows a tenth example of the three-dimensional posture input state specifying function-added solid, and shows an example of the click function-added position / posture input device solid. The sphere 1 is a sphere having four holes.
[0021]
FIG. 12 shows an eleventh example of the three-dimensional posture input state specifying function-added solid, and shows another example of the position / posture input solid with click function. The example of FIG. 12A is a cube and has four holes. The example of FIG. 12 (b) is a sphere, where the slanted bar portion of the letter A is painted, and the horizontal bar portion of the letter A is a hole.
[0022]
FIG. 13 is an overview of the system. Both the center position of the solid 1 and the position of the color camera 13 photographing the solid 1 are fixed in space, and only the posture of the solid 1 fluctuates. When a sphere or a disk is used as the solid 1, the position can be changed and detected.
[0023]
FIG. 14 shows a posture detection template that associates an image of the sphere 1 photographed while changing its posture at a sufficiently small interval and a quaternion (posture) representing the posture.
[0024]
FIG. 15 is a diagram for explaining an image signal pattern in the vicinity of a hole in a sphere image. A blue channel image and a red channel image separated from the camera image signal are displayed for each of when the internal light source is turned on and when it is turned off. In the blue channel image, the hole area has a higher luminance than the spherical surface area when the internal light source is turned on, and is lower when the internal light source is turned off. In the red channel image, the brightness of the hole region is lower than that of the spherical surface region regardless of whether the internal light source is turned on or off. Therefore, if the position of the hole is detected in the red channel image and the on / off state of the internal light source is detected by comparing the brightness of the hole area and the spherical surface area in the blue channel image, the click function is realized. can do.
[0025]
FIG. 16 is a diagram for explaining a method of enabling specification of a continuous value by continuous control of light emission luminance. 15 differs from the case of FIG. 15 in that the luminance of the hole region is measured as an absolute value in the blue channel image, and a continuous value is designated by this absolute value.
[0026]
FIG. 17 is a diagram for explaining a method of enabling multi-value designation by controlling the light emission pattern in time series. With the 256 types of light emission patterns, the luminance pattern of the hole area in the blue channel image is associated with a value of 0-255.
[0027]
Next, a device for inputting the three-dimensional posture and state of the three-dimensional posture input state-added three-dimensional object 1 will be described. Note that the solid described below has the structure of the solid 1 or 12 described above, and the state of the solid can also be input.
[0028]
[First Embodiment]
FIG. 18 is a block diagram of a twelfth example of a three-dimensional posture input state specifying function-equipped solid of the present invention and the three-dimensional posture input device of the first embodiment.
[0029]
The three-dimensional attitude detection device 23 includes an axis attitude calculation unit 31, a three-dimensional attitude calculation unit 32, and a state detection unit 33. A three-dimensional attitude input function specifying solid 21 (hereinafter referred to as a three-dimensional 21) is a camera 22. The posture and state of the solid 21 are calculated from the captured image.
[0030]
In the present embodiment, it is assumed that the position of the center of gravity of the solid 21 on the captured image is known by fixing the center of gravity C of the solid 21 at a predetermined position on the optical axis of the camera 22.
[0031]
The three-dimensional posture detection device 23 first detects the surface and the solid 21 from the image obtained by capturing the solid 21 with the camera 22 based on the position of the figure drawn on the surface of the solid 21 by the shaft posture calculation unit 31. The position of the point (that is, the pole) where the axis intersects is detected, the attitude of the axis of the solid 21 is calculated, and then the figure and the template image group prepared in advance are calculated by the solid attitude calculation unit 32. Based on the comparison or the color information of the figure, the attitude of the solid 21 is calculated. Then, the state detection unit 33 detects the designated state of the solid 21 based on at least one of the color information and the luminance information at the hole position of the captured image.
[0032]
Hereinafter, in the present embodiment, as a twelfth example of the three-dimensional posture input state specifying function solid 21, a known ellipsoid having one non-rotationally symmetric figure drawn on the surface is used. A method of inputting 21 postures and states will be described.
[0033]
FIG. 19A is an example of the solid 21, and FIG. 19B is a diagram for explaining the arrangement of figures on the surface of the solid 21. In FIG. 19B, L is the axis of the ellipsoid, and P is the point where the surface of the ellipsoid and the axis L intersect, that is, the ellipsoid pole. The non-rotationally symmetric figure (in this example, the letter “A”) is drawn on the surface of the ellipsoid so that the center of gravity G coincides with the position of the pole P as shown in FIG.
[0034]
20 and 21 are diagrams for explaining a method of creating a graphic template image by a computer.
[0035]
In FIG. 20, C _V Is a virtual camera located at infinity, O is the origin of the virtual space in the computer, X is a horizontal axis, Y is a vertical axis, and Z is a virtual camera C _V Optical axis of S _V Is a virtual plane on which a figure (the letter “A” in this example) is placed, and G is a virtual plane S _V It is the upper point and the center of gravity of the figure. Virtual plane S _V Is perpendicular to the line segment OG, and the distance OG is equal to the major radius of the virtual ellipsoid. The point G is on the YZ plane, and the angle between the line segment OG and the axis Z is θ ₁ And Here, the virtual plane S _V If the intersecting line between Y and the YZ plane is T, the straight line T passes through the point G and is perpendicular to the line segment OG. Therefore, the virtual plane S _V Is expressed as an angle with reference to the straight line T when θ is rotated about the line segment OG. ₂ And For the sake of explanation, FIG. 20 shows only the outline of the figure.
[0036]
FIG. 21 shows a virtual plane S _V A virtual camera C obtained by projecting the above figure (in this example, the letter “A”) onto the XY plane _V It is an image. G _T Is the position of the center of gravity G on the image. The template image is the virtual camera C _V Image, including graphics, and G _T A partial image cut out in a rectangular area centered at is subjected to blurring processing using a Gaussian filter or the like.
[0037]
FIG. 22 is a table in which created template image groups are arranged. Virtual space S shown in FIG. _V At an angle θ ₁ For example, in steps of 1 degree from 0 degree to 90 degrees, the angle θ ₂ For example, in steps of 1 degree from 0 degree to 359 degrees, these angles θ ₁ , Θ ₂ Virtual camera C for all combinations of _V Are calculated, template images corresponding to the images are created, and arranged in a table.
[0038]
FIG. 23 is a diagram for explaining the relationship between the posture of the ellipsoidal axis L in the real space and the position of the pole P on the captured image. FIG. 23A is a perspective view of an ellipsoid, and FIG. 23B is an image obtained by photographing the ellipsoid with the camera 22. In FIG. 23A, C is the center of the ellipsoid, X is the horizontal axis, Y is the vertical axis, Z is the optical axis of the camera 22, and B ₁ Is the angle between the plane containing the pole P and the axis Z and the axis Y, B ₂ Is the angle between the axis L and the axis Z, and D is the distance between the pole P and the axis Z. In FIG. 23B, C _S Is the position on the captured image of the center C of the ellipsoid, P _S Is the position of the pole P on the captured image, B ₁ Is straight line C _S P _S The angle that D makes clockwise from the vertical direction, D _S Is C _S And P _S Is the distance between. P _S Let (u, v) be the coordinates on the captured image and (x, y, z) be the coordinates of P in real space. Here, the center of gravity C of the ellipsoid on the captured image _S Is known, so this is the origin on the captured image, and the imaging magnification of the camera is α,
u = αx, v = αy, D _S = ΑD
The relationship is established. Where the distance between points C and P in real space is D _CP Then, the following relationship holds.
[0039]
[Expression 1]

[0040]
These relationships are expressed as D _CP That is, if the distance between C and the target point P in the real space is known, the relationship between the target point P and the posture of the straight line CP even if the target point P is an arbitrary point other than the pole of the ellipsoid. It is established as.
[0041]
FIG. 24 shows a partial image obtained by cutting out an ellipsoidal part from the captured image. FIG. 24A is a grayscale image, and FIG. 24B is an image obtained by binarizing the image. For the sake of explanation, FIG. 24B shows the figure only by its outline. In FIG. 24B, P _S Is the position of the pole P on the captured image, which coincides with the center of gravity of the figure on the captured image.
[0042]
FIG. 25 is a flowchart showing a procedure for inputting the posture and state of the solid 21.
1. The center of gravity of the figure is obtained from the binarized image (FIG. 24B), and the position is set to P _S (Step 41).
2. Using equation (1), the attitude of the axis L, that is, the angle B ₁ And B ₂ Is obtained (step 42).
Angle B obtained in 3.2 ₂ Is the angle θ of the table shown in FIG. ₁ A column of template images to be used for matching processing is selected from the column (step 43).
4). The captured image (FIG. 24A) is represented by C _S Around the angle -B ₁ A grayscale image that has been rotated only by this amount is obtained (step 44).
Matching processing by a normalized correlation method or the like between all template images (360 in the example of FIG. 22) included in the column determined in 5.3 and the grayscale image obtained in 4 is performed at point P. _S Are sequentially performed in the vicinity of the point after the rotation (step 45).
A template image having the maximum matching degree (score) obtained in 6.5 is found, and an angle θ corresponding to the template image is found. ₂ , And this is an angle B representing the orientation of the solid body 21 around the axis L in the rotational direction. ₃ (Step 46).
7). The angle B obtained by the above ₁ , B ₂ And B ₃ Is the posture of the solid 21 (step 47).
8). Finally, the designated state of the solid 21 is detected based on at least one of color information and luminance information at the position of the hole in the captured image (step 48).
[0043]
FIG. 26 shows a regular quadrangular trapezoidal cone with one non-rotationally symmetric figure drawn on the surface as a thirteenth example of the solid 21 in which the posture and state can be input in the same procedure as described above. FIG. 26A is an example of the solid 21 and FIG. 26B is a diagram for explaining the arrangement of figures on the surface of the solid 21. In FIG. 26B, L is the axis of the regular square trapezoidal cone, and P is the point where the surface of the regular square trapezoidal cone and the axis L intersect. The non-rotationally symmetric figure (in this example, the letter “A”) is drawn on the surface of the regular quadrangular trapezoidal cone so that the center of gravity G coincides with the position P as shown in FIG.
[0044]
FIG. 27 shows a shell of a regular quadrangular trapezoidal cone with one non-rotationally symmetric figure drawn on the surface as a fourteenth example of the solid 21 in which the posture and state can be input in the same procedure as described above. . FIG. 27A is an example of the solid 21, and FIG. 27B is a diagram for explaining the arrangement of figures on the surface of the solid 21. In FIG. 27 (b), L is the axis of the regular quadrangular trapezoidal shell, and P is the point where the surface of the regular quadrangular trapezoidal shell and the axis L intersect. The non-rotationally symmetric figure (in this example, the letter “A”) is drawn on the surface of the regular quadrangular trapezoidal cone-shaped body so that the center of gravity G coincides with the position P as shown in FIG. .
[0045]
FIG. 28 shows a quasi-regular polyhedron in which one non-rotationally symmetric figure is drawn on the surface as a fifteenth example of the solid 21 whose posture can be detected by the same procedure as described above. As shown in FIG. 28, the non-rotationally symmetric figure (the letter “A” in this example) is drawn on the surface of the quasi-regular polyhedron so that the center of gravity G coincides with the position of the centroid of one surface of the quasi-regular polyhedron. Yes.
[0046]
FIG. 29 shows a regular polyhedron in which one non-rotationally symmetric figure is drawn on the surface as a sixteenth example of the solid 21 whose posture can be detected by the same procedure as described above. As shown in FIG. 29, the non-rotationally symmetric figure (character “A” in this example) is drawn on the surface of the regular polyhedron such that the center of gravity G coincides with the position of the center of gravity of one surface of the regular polyhedron.
[0047]
[Second Embodiment]
In the present embodiment, as a seventeenth example of the three-dimensional posture detecting solid 21, a known ellipsoid in which four rotationally symmetric figures of two different sizes and three non-rotary symmetric figures are drawn on the surface is used. A method for obtaining the posture of the solid 21 from the captured image will be described.
[0048]
FIG. 30 is a diagram for explaining the arrangement of figures on the surface of the solid 21. In order to make it easy to understand, in FIG. 30, the figure on the ellipsoidal surface is represented in a plane. FIG. 30A is a top view, and FIG. 30B is a side view. A large dot at the position of the ellipsoid pole P is a rotationally symmetric position Q about the pole P. ₁ , Q ₂ , Q ₃ Small dots are drawn on each. Q ₁ , Q ₂ , Q ₃ The dots drawn in P and Q, with the direction from each dot drawn in P to the dot drawn in P as a reference for posture ₁ , Q ₂ , Q ₃ Three different figures are drawn at the midpoints of the three dots drawn in FIG.
[0049]
FIG. 31 is a diagram for explaining the three-dimensional positional relationship between the dots drawn on the solid 21. In FIG. 31A, P is an ellipsoidal pole and is the position of a large dot. Q _i (I = 1, 2, 3) is the position of each small dot, C is the center of gravity of the ellipsoid, D _PQ Is P and Q in real space _i Distance between, D _QQ Q in real space _i And Q _j Distance between, D _CQ Is C and Q in real space _i Is the distance between. In FIG. 33 (b), M _i Are points P and Q _i And a point on the plane including C, and the line segment PQ _i Is the point where the perpendicular bisector intersects the surface of the ellipsoid.
[0050]
FIGS. 32A and 32B are template images for detecting large dots and small dots, respectively, and are actually created by performing blurring processing using a Gaussian filter or the like.
[0051]
FIG. 33 is a table in which template image groups for detecting three different types of graphics are arranged. Each template image is created by the method described in the first embodiment.
[0052]
FIG. 34 shows a partial image obtained by cutting out an ellipsoidal part from the captured image. FIG. 34 (a) is a grayscale image, and FIG. 34 (b) is a diagram for explaining a procedure for obtaining an ellipsoidal posture. For the sake of explanation, in FIG. 34 (b), each dot is shown only by its outline. In FIG. 34 (b), P _S Is the position of the pole P on the captured image, which corresponds to the position of the large dot on the captured image. Q _Si (I = 1, 2, 3) is Q _i Position on the captured image, M _Si Is M _i The position on the captured image.
[0053]
FIG. 35 is a flowchart showing a procedure for obtaining the posture of the solid 21.
1. A matching process using a normalized correlation method or the like between the template image (FIG. 32A) and the grayscale image (FIG. 32A) for detecting a large dot is performed, and the captured image with the maximum matching score is displayed. Find the position of the large dot and set the position to P _S (Step 51).
2. Matching processing by a normalized correlation method or the like between the template image (FIG. 32 (b)) and the grayscale image (FIG. 34 (a)) for detecting small dots is performed on the captured image that maximizes the matching score. Small dot position Q _Si Is obtained (step 52).
P found in 3.1 _S From Equation (1) and Equation (2), the attitude of the axis L, that is, the angle B ₁ , B ₂ And the position of the point P in the real space is obtained (step 53).
Q found in 4.2 _Si From equation (1) and equation (2), straight line CQ _i Posture or angle B _Qi1 , B _Qi2 And point Q _i Is obtained in real space (step 54). However, in Formula (1) and Formula (2), D _CP Instead of D _CQ Will be used.
5). Point P and point Q _i From the position in the real space, using the relationship of FIG. _i Position and straight line CM _i Posture or angle B _Mi1 And B _Mi2 Is obtained (step 55).
6). Using equation (2), the point M _i Position M on the captured image _Si Is obtained (step 56). However, in Formula (2), D _CP Instead of C and M _i The distance between them will be used.
Angle B obtained at 7.5 _Mi2 Is the angle θ of the table shown in FIG. ₁ A column of three types of template images to be selected from the column and used for matching processing is determined (step 57).
8). The captured image (FIG. 32 (a)) is represented by C. _S Angle -B around _Mi1 A grayscale image that has been rotated only by this amount is obtained (step 58).
Matching processing using a normalized correlation method or the like between all three types of template images (1080 in the example of FIG. 31) included in the column determined in 9.7 and the grayscale image obtained in 8 is performed at point M. _Si Are sequentially performed in the vicinity of the point after rotation (step 59).
Find the template image that maximizes the degree of matching (score) obtained in 10.9, and specify which of the three types of the target graphic is the target small dot Q _i Is Q ₁ , Q ₂ , Q ₃ (Step 60).
11. Q identified _i Based on the angle B representing the orientation in the rotational direction around the axis L of the solid 21 ₃ Is calculated. That is, the identified Q _i The angle around axis Z -B ₁ Only an angle -B around axis X ₂ The declination around the axis Z of the point rotated only by ₃ (Step 61).
12 The angle B obtained by the above ₁ , B ₂ And B ₃ Is the posture of the solid 21 (step 62).
13. Finally, the designated state of the solid 21 is detected based on at least one of the color information and the luminance information at the position of the hole in the captured image (step 63).
[0054]
[Third Embodiment]
In the present embodiment, as an eighteenth example of the three-dimensional posture detection solid 21, a known ellipsoid having four rotationally symmetric figures and three non-rotationally symmetric figures of the same size drawn on the surface is used. A method for obtaining the posture of the solid 21 from the captured image will be described. The second embodiment is different from the second embodiment only in the method of discriminating between the dot drawn at the position of the ellipsoid pole P and the dots drawn around it, and only this discrimination method will be described.
[0055]
FIG. 36 is a diagram for explaining the arrangement of figures on the surface of the solid 21. For easy understanding, in FIG. 36, the figure on the ellipsoidal surface is represented in a plane. FIG. 36A is a top view, and FIG. 36B is a side view. Position of ellipsoid pole P and rotationally symmetric position Q about pole P ₁ , Q ₂ , Q ₃ Are all drawn with the same size dot. Q ₁ , Q ₂ , Q ₃ Using the direction from each dot drawn in P to the dot drawn in P as a reference for posture, the dot drawn in P and Q ₁ , Q ₂ , Q ₃ Three different figures are drawn at the midpoints of the three dots drawn in FIG. Furthermore, it is assumed that the three-dimensional positional relationship between the dots drawn on the solid 21 is the same as that in FIG. However, in this embodiment, D _PQ And D _QQ The position of each dot is set so that is sufficiently different.
[0056]
FIG. 37 shows a template image for detecting dots, which is actually created by performing a blurring process using a Gaussian filter or the like.
[0057]
FIG. 38 shows a partial image obtained by cutting out an ellipsoidal part from the captured image. FIG. 40A is a grayscale image, and FIG. 38B is a diagram for explaining the procedure for obtaining the ellipsoidal posture. For the sake of explanation, in FIG. 38 (b), each dot is shown only by its outline. In FIG. 38 (b), P _S Is the position of the pole P on the captured image, Q _Si (I = 1, 2, 3) is Q _i The position on the captured image.
[0058]
FIG. 39 shows dots and Q drawn on P of the solid 21. ₁ , Q ₂ , Q ₃ 5 is a flowchart illustrating a procedure for determining one of the dots drawn in (1).
1. Matching processing using a normalized correlation method or the like between the template image for detecting dots (FIG. 37) and the grayscale image (FIG. 38 (a)) is performed. All positions are obtained (step 71).
2. For each obtained point, the position in the real space is calculated using Expression (1) and Expression (2) (Step 72). However, in Formula (1) and Formula (2), D _CP D instead of all _CQ Will be used. In this embodiment, D _PQ And D _QQ Are sufficiently different from each other, each dot can be correctly identified even using such an approximation.
3. If the number of obtained points is two or less, the processing is stopped (step 73).
4). When the number of obtained points is three or more, the distance in the real space between the two selected points is obtained for all combinations for selecting two of these points, and this distance and the known D in the solid 21 are obtained. _QQ A set in which the absolute value of the difference between the two is larger than a predetermined threshold value is selected (step 74).
5). The two points of the selected set are P ₁ , P ₂ , Q any other point _i And P ₁ And Q _i The distance between them is obtained (step 75). At that time, P ₁ , P ₂ As the position in real space, D _CP The values calculated by the equations (1) and (2) in which the distance between C and P is substituted into are used.
P found in 6.5 ₁ And Q _i Distance between and known D in solid 21 _PQ If the absolute value of the difference is smaller than a predetermined threshold, P ₁ Is the dot drawn on P, otherwise P ₂ Is a dot drawn on P (step 76).
[0059]
Hereinafter, step 3 and subsequent steps in the second embodiment may be executed.
[0060]
FIG. 40 shows an ellipsoid in which a circle is drawn on the surface in place of the dots in the solid of the fourteenth example as a nineteenth example of the solid 21 whose posture can be detected in the same procedure as described above. For easy understanding, in FIG. 40, the figure on the ellipsoidal surface is represented in a plane. 40A is a top view, and FIG. 40B is a side view. Position of ellipsoid pole P and rotationally symmetric position Q about pole P ₁ , Q ₂ , Q ₃ Are all drawn the same size circle. Q ₁ , Q ₂ , Q ₃ The direction drawn from each circle drawn in P to the circle drawn in P is the posture reference, and the circle drawn in P and Q ₁ , Q ₂ , Q ₃ Three different figures are drawn at the respective midpoints of the three circles drawn in (1). Furthermore, it is assumed that the three-dimensional positional relationship between the circles drawn on the solid 21 is the same as in FIG. However, D _PQ And D _QQ The position of each circle is set so that and are sufficiently different.
[0061]
FIG. 41 shows a template image for detecting a circle, which is actually created by performing a blurring process using a Gaussian filter or the like.
[0062]
As a twentieth example of the solid 21 whose posture can be detected in the same procedure as described above in FIG. 42, four rotationally symmetric figures and three non-rotary symmetric figures of the same size are drawn on the surface. A regular hexagonal trapezoidal cone is shown. In order to make it easy to understand, in FIG. 42, the figure on the regular hexagonal trapezoidal cone surface is represented in a plane. FIG. 42 (a) is a top view and FIG. 42 (b) is a side view. L is an axis of a regular hexagonal trapezoidal cone, and P is a point where the axis L intersects with the upper bottom surface. Position Q at the center of the left and right sides of every other side and at the same distance from P ₁ , Q ₂ , Q ₃ Are all drawn with the same size dot. Q ₁ , Q ₂ , Q ₃ The dots drawn in P and Q, with the direction from each dot drawn in P to the dot drawn in P as a reference for posture ₁ , Q ₂ , Q ₃ Three different figures are drawn at the midpoints of the three dots drawn in FIG. However, P and Q _i Distance D between _PQ And Q _i And Q _j Distance D between _QQ The position of each dot is set so that is sufficiently different.
[0063]
In FIG. 43, four rotationally symmetric figures and three non-rotationally symmetric figures of the same size are drawn on the surface as a tenth example of the solid 21 whose posture can be detected by the same procedure as described above. Indicates a paraboloid. In order to make it easy to understand, in FIG. 43, the figure on the surface of the paraboloid is expressed planarly. 43A is a top view, and FIG. 43B is a side view. L is the axis of the paraboloid, and P is the point where the axis L and the surface of the paraboloid intersect, that is, the pole of the paraboloid. The position of the paraboloid pole P and the rotationally symmetric position Q about the pole P ₁ , Q ₂ , Q ₃ Are all drawn with the same size dot. Q ₁ , Q ₂ , Q ₃ The dots drawn in P and Q, with the direction from each dot drawn in P to the dot drawn in P as a reference for posture ₁ , Q ₂ , Q ₃ Three different figures are drawn at the midpoints of the three dots drawn in FIG. However, P and Q _i Distance D between _PQ And Q _i And Q _j Distance D between _QQ The position of each dot is set so that is sufficiently different.
[0064]
[Fourth Embodiment]
In the present embodiment, as a twenty-second example of the three-dimensional posture detection solid 21, a known ellipsoid having four rotationally symmetric figures of the same size and different colors drawn on the surface is used. A method for obtaining the posture of the solid 21 from the image will be described.
[0065]
FIG. 44 is a diagram for explaining the arrangement of figures on the surface of the solid 21. For easy understanding, the figure on the ellipsoidal surface is represented in a planar manner in FIG. FIG. 46A is a top view and FIG. 44B is a side view. A black dot at the position of the pole P of the ellipsoid is a rotationally symmetric position Q about the pole P. ₁ , Q ₂ , Q ₃ Each has red, green and blue dots. Further, the three-dimensional positional relationship between the dots drawn on the solid 21 is the same as that shown in FIG. Further, a template image for detecting dots is the same as that shown in FIG.
[0066]
FIG. 45 shows a partial image obtained by cutting out an ellipsoidal part from the captured image. FIG. 45A is a gray image, and FIG. 45B is a color image. P _S Is the position of the pole P on the captured image, Q _Si (I = 1, 2, 3) is Q _i The position on the captured image.
[0067]
FIG. 46 is a flowchart showing a procedure for obtaining the posture and state of the solid 21.
1. Matching processing using a normalized correlation method or the like between the template image for detecting dots (FIG. 37) and the grayscale image (FIG. 45 (a)) is performed. All positions are obtained (step 81).
2. Each dot is discriminated from the color information of the obtained color image (FIG. 45B) at each point (step 82).
P determined in 3.2 _S From equation (1), the attitude of the axis L, that is, the angle B ₁ And B ₂ Is obtained (step 83).
4). Q _i (Q ₁ , Q ₂ , Q ₃ With respect to one dot drawn in any one of (1), the position in the real space is obtained using the equations (1) and (2) (step 84). However, in Formula (1) and Formula (2), D _CP Instead of D _CQ Will be used.
Q found in 5.4 _i Based on the angle B representing the orientation in the rotational direction around the axis L of the solid 21 ₃ Is calculated (step 85). That is, the calculated Q _i The angle around axis Z -B ₁ Only an angle -B around axis X ₂ The declination around the axis Z of the point rotated only by ₃ And
6). The angle B obtained by the above ₁ , B ₂ And B ₃ Is the posture of the solid 21 (step 86).
7). Finally, the designated state of the solid 21 is detected based on at least one of the color information and the luminance information at the position of the hole in the captured image (step 87).
[0068]
In the method for obtaining the posture of a solid described in the above embodiment, each matching process uses a relatively small size figure that is a part of the figure drawn on the surface of the solid. A template image corresponding only to the region may be used, and the storage capacity of the template image and the calculation cost of the matching process can be reduced.
[0069]
[Fifth Embodiment]
FIG. 47 is a diagram showing a three-dimensional posture input apparatus according to the fifth embodiment of the present invention.
[0070]
The three-dimensional posture input device 103 includes a position calculation unit 131, a figure detection unit 132, a posture calculation unit 33, and a state detection unit 34, and images a three-dimensional posture input solid (hereinafter referred to as a solid) 101 with the camera 102. The three-dimensional position, posture, and state of the solid 101 are detected from the captured image obtained in this way.
[0071]
FIG. 48 is a diagram showing a twenty-third example of the solid 101, and FIG. 49 is a diagram showing the positional relationship of each part.
[0072]
The three-dimensional object 101 is composed of a main part 101 made of a thin disk (radius r) having an edge 211 on the surface, and two dots of the same color and the same color supported by a thin structure on the main part 101. 1 and second rotationally symmetric figures 221, 231. The first rotationally symmetric figure 221 has a center on the rotational symmetry axis of the main part 201 and a plane parallel to the plane including the surface of the main part 201 and a distance h away from the surface. The second rotationally symmetric figure 231 is on the plane including the main part 201 and is located outside the region of the main part 201 at a distance g from the rotational symmetry axis of the main part 201. As the size of the rotationally symmetric figures 221 and 231 is reduced, the position of the rotationally symmetric figures 221 and 231 can be detected with high accuracy, and the size difference from the whole of the solid 101 becomes large. Therefore, it is desirable that the minimum size is identifiable on the captured image.
[0073]
In the present embodiment, it is assumed that the position of the solid 101 is far from the camera 102 on the captured image so that the edge 211 of the surface of the main part 201 can be approximated to an ellipse. In the present embodiment, it is assumed that the inclination of the solid 101 is within a range where the rotationally symmetric figure 221 does not come out of the edge 211 of the surface of the main part 201 on the captured image.
[0074]
In the three-dimensional position / posture detection apparatus 103, first, the position calculation unit 131 calculates the position of the solid 101 from the outline of the main part 201 of the solid 101 on the captured image, and then the graphic detection unit 132 captures the image. The positions of the rotationally symmetric figures 221 and 231 on the image are detected, and the orientation calculation unit 133 further detects the direction of the rotationally symmetric axis of the main part 201 of the solid 101 and the rotation angle around the rotationally symmetric axis, that is, the posture of the solid 101 Is calculated. Finally, the state detection unit 134 detects the designated state of the three-dimensional object 101 based on at least one of color information and luminance information at the position of the hole in the captured image.
[0075]
Hereinafter, in the present embodiment, a method of obtaining the three-dimensional position and orientation of the solid 101 from the captured image using the solid 101 will be described.
[0076]
FIG. 50 illustrates a solid 101 at a position (x, y, z) in a coordinate system in real space with the optical center C of the lens of the camera 102 as the origin and the optical axis of the lens as the Z axis, with the camera 102 as a reference. It is the figure which showed a mode that was placed. X is a horizontal axis in real space, and Y is a vertical axis in real space. The solid 101 has the same design as that shown in FIG. The camera 102 can be handled as an ideal camera without lens distortion or the like by correcting the optical distortion of the captured image.
[0077]
FIG. 51 is a template image for detecting the dot-like figures 221 and 231 and is actually created by performing a blurring process using a Gaussian filter or the like.
[0078]
FIG. 52A is a captured image of the solid 101, and FIG. 52B is a diagram illustrating detection results of the contour 211 of the main part 201 of the solid 1 and the dot-shaped rotationally symmetric figures 221, 231 in the captured image. . u is the horizontal axis of the image, and v is the vertical axis of the image. In FIG. 53 (b), 211s is an ellipse representing an outline obtained by fitting the edge 211 of the main part 201 of the detected solid 1, 221s is the position of the detected dot-shaped disk 221 and 231s is detected. This is the position of the dot-shaped disc 231. Here, when a point on the captured image is represented by coordinates (u, v), the ellipse 211s can be represented by the following equation.
[0079]
[Expression 2]

[0080]
FIG. 53 is a diagram showing a geometric relationship between positions 221s and 231s of dot-shaped figures on the captured image and positions of dot-like rotationally symmetric figures in the coordinate system XYZ in real space. Reference numeral 104 denotes the position in the real space of the center of the circle corresponding to the ellipse 211 s represented by the expression (1), that is, the center of the surface of the main part 201 of the solid 101. Reference numeral 105 denotes a plane including the circle, that is, a plane including the surface of the main part 201 of the solid 101 and its edge 211. Reference numeral 106 denotes a sphere having a radius h centered at 104. Reference numeral 107 denotes a point corresponding to 221s, that is, the position of the rotationally symmetric figure 221 in the real space, and reference numeral 108 denotes a point corresponding to 231s, that is, the position of the rotationally symmetric figure 231 in the real space. C is the optical center of the lens of the camera 102. A position 107 is an intersection of a straight line connecting the position 221 s and the lens center C and the sphere 106. A position 108 is an intersection of a straight line connecting the position 231 s and the lens center C and the plane 105.
[0081]
The plane 104 including the position 104 in the real space at the center of the surface of the main part 201 of the solid 101 shown in FIG. 53 and the surface of the main part 201 and its edge 211 is represented by the coefficients α, β, γ, Using δ, ε, and ζ, it can be determined by the following method (see, for example, the literature Kenichi Kanatani, “Geometric Computation for Machine Vision”, p. 228).
[0082]
First, a matrix Q represented by the following equation is created, and an eigenvalue λ of Q ₁ , Λ ₂ , Λ ₃ , And unit eigenvectors n corresponding to these eigenvalues ₁ , N ₂ , N ₃ Calculate Where the eigenvalue is λ ₃ <0 <λ ₁ ≦ λ ₂ Choose to be.
[0083]
[Equation 3]

[0084]
Here, f is the distance between the uv plane in FIG. 53 and the optical center C of the lens, and κ is selected so that detQ = −1.
[0085]
Next, the eigenvalue λ of Q obtained above ₁ , Λ ₂ , Λ ₃ , And unit eigenvector n ₁ , N ₂ , N ₃ Is used to calculate the normal vector n of the plane 105 and the distance d between the plane 5 and the optical center C of the lens.
[0086]
[Expression 4]

[0087]
However, r is the radius of the edge 211 of the surface of the main part 201. The plane 105 is obtained from the normal vector n and the distance d.
[0088]
Furthermore, the inverse matrix Q of Q obtained above. ^-1 , And a normal vector n, a position vector m starting from the optical center C of the lens is calculated by the following equation.
[0089]
[Equation 5]

[0090]
The position 104 is obtained as an intersection between the position vector m and the plane 105.
[0091]
FIG. 54 is a flowchart showing a procedure for obtaining the three-dimensional position and orientation of the solid 101.
1. In the captured image (FIG. 52A), an ellipse 111s fitted to the contour 111 of the main part 101 of the solid 101 is detected using Hough transform or the like, and the coefficients α, β, γ, δ, ε, and ζ are obtained (step 301).
From the α, β, γ, δ, ε, and ζ determined in 2.1, the surface of the main part 201 of the solid body 101 and its edge 211 are included by the method shown in the formulas (1) to (5). The normal vector of the plane 105 and the center position 104 of the surface of the main part 101 are obtained, and the plane 105 is further obtained (step 302).
3. Matching is performed by a normalized correlation method or the like between a template image (FIG. 51) and a captured image (FIG. 52 (a)) for detecting a dot-shaped figure, and dots on the captured image where the matching score is maximized The position of the figure is detected (step 303).
Whether each position corresponds to the rotationally symmetric figure 221 or the rotationally symmetric figure 231 is determined from the positional relationship between each of the positions detected in 4.3 and the ellipse 211s (step 304). That is, of the detected positions, the inside of the ellipse 211s is 221s, and the outside is 231s.
5). The position 107 and the position 108 are obtained from the geometric relationship shown in FIG. 53 (step 305).
6). From the position 104 and the position 107, the direction of the rotational symmetry axis of the main part 201 of the solid 101 is obtained, and from the position 104 and the position 108, the rotation angle around the rotational symmetry axis of the solid 101 is obtained (step 306).
The position 104 obtained in 7.2 is set as the three-dimensional position of the solid 101, and the direction of the rotational symmetry axis of the main part 201 of the solid 101 obtained in 6 and the rotation angle around the rotational symmetry axis are determined as the three-dimensional of the solid 1. The posture is set (step 307).
8). Finally, the designated state of the solid 101 is detected based on at least one of the color information and the luminance information at the position of the hole in the captured image (step 308).
[0092]
[Sixth Embodiment]
In the fifth embodiment, the first and second rotation target figures 221 and 231 provided on the solid 101 are the same color, and the inclination of the solid 101 is the edge of the main part 201 on the captured image. It is assumed that it is within a range not coming out of 211.
[0093]
On the other hand, in the present embodiment, the first and second rotation target figures provided on the solid 101 are different from each other, and the restriction on the inclination of the solid 101 is eliminated.
[0094]
FIG. 55 shows an example of the solid 101. This solid 101 has a main part 202 which is a thin ring having an edge 212 on the surface, and two dot-shaped figures 222 and 232 of the same size and different colors supported by a thin structure on the main part 202. It consists of a drawn disc. Here, it is assumed that the figure 222 is drawn in red and the figure 232 is drawn in blue. About the size of the figures 222 and 232, it is desirable to make it the minimum size which can be identified on a captured image similarly to 5th Embodiment.
[0095]
The three-dimensional position and orientation detection method in this embodiment is a procedure for determining which of the two dot-shaped figures 222 and 232 corresponds to each of the detected positions of the dot-shaped figures 222 and 232. Only the fifth embodiment is different. Therefore, the method according to this embodiment will be described with reference to FIGS. 50 to 53 as in the fifth embodiment. However, in the present embodiment, it is assumed that the captured image corresponding to FIG. 52 is a color image having color information.
[0096]
FIG. 56 is a flowchart showing a procedure for obtaining the three-dimensional position and orientation of the solid 101.
1. In the captured image (FIG. 52A), an ellipse 212s fitted to the outline of the main part 102 of the solid 101 is detected using Hough transform or the like, and the coefficients α, β, γ, δ in the equation (1) are detected. , Ε, and ζ are obtained (step 401). However, only the shading information of the captured image is used here.
From the α, β, γ, δ, ε, and ζ obtained in 2.1, the surface of the main part 202 of the solid body 101 and its edge 212 are included by the method shown in the equations (3) to (7). The normal vector of the plane 105 and the center position 104 of the surface of the main part 202 are obtained, and the plane 105 is further obtained (step 402).
3. Matching is performed by a normalized correlation method or the like between a template image (FIG. 51) and a captured image (FIG. 52 (a)) for detecting a dot-shaped figure, and dots on the captured image where the matching score is maximized The position of the figure is detected (step 403). However, only the shading information of the captured image is used here.
It is determined from the color information of the captured image (FIG. 52A) at each of the positions detected in 4.3 whether each position corresponds to the graphic 222 or the graphic 232 (step 404). That is, if the pixel at the detected position has red color information, the position is 222 s, and if the pixel has blue color information, the position is 232 s.
5). The position 107 and the position 108 are obtained from the geometric relationship shown in FIG. 53 (step 405).
6). The direction of the rotational symmetry axis of the main part 102 of the solid 101 is obtained from the position 104 and the position 107, and the rotation angle around the rotational symmetry axis of the solid 101 is obtained from the position 104 and the position 108 (step 406).
The position 104 obtained in 7.3 is set as the three-dimensional position of the solid 101, and the direction of the rotational symmetry axis of the main part 202 of the solid 101 obtained in 5 and the rotation angle around the rotational symmetry axis are represented by 3 of the solid 101. A dimension posture is set (step 407).
8). The designated information of the solid 101 is detected based on at least one of color information and luminance information at the position of the hole in the captured image (step 408).
[0097]
57 to 59 show other examples of the three-dimensional object 101 that can detect the three-dimensional position and orientation by the procedure shown in the fifth or sixth embodiment. However, when the first and second rotation target figures provided on the solid 101 have the same color, the procedure shown in the fifth embodiment can be applied, and the first and second rotations provided on the solid 101 can be applied. When the target graphic has a different color, the procedures shown in the fifth and sixth embodiments can be applied.
[0098]
The solid body shown in FIG. 57 is composed of a main portion 203 that is a ring and two small spheres 223 and 233 of the same size supported by a thin structure.
[0099]
The solid shown in FIG. 58 is a rod-like and plate-like structure in which a main part 204 that is a disk and a dot-like figure 224, 234 of the same size are drawn on a part of the surface thereof. It is composed of the body.
[0100]
The solid body shown in FIG. 59 has a cylindrical and plate-like structure in which a ring-shaped main portion 205 is supported, and dot-shaped figures 225 and 235 of the same size are drawn on a part of the surface. It is composed of the body.
[0101]
As described above, in the method for obtaining the three-dimensional position, posture, and state of the solid according to the present embodiment, since the main part of the solid is a disk or a ring, the calculation of the position and the calculation of the posture can be separated. Calculation costs can be reduced while ensuring high accuracy. Further, since a relatively small size graphic drawn on a part of a solid is used, it is sufficient to use a template image corresponding only to the area of the graphic, not the entire solid, and the storage capacity and calculation cost of the template image in the matching process Can be kept small. As a result, it is possible to greatly reduce the overall calculation cost while ensuring high accuracy.
[0102]
In the above embodiment, an example in which the two rotationally symmetric figures of the solid 101 are figures having the same size and the same shape has been described. However, by changing the template image required to detect these figures, the two rotationally symmetric figures have different sizes and different shapes (for example, dot-like figures and circular figures). However, the present invention can achieve the same effect.
[0103]
[Seventh Embodiment]
FIG. 60 is a diagram showing a three-dimensional posture input apparatus according to a twenty-seventh example of a three-dimensional posture input state designation function three-dimensional posture input device of the present invention, and FIG. 61 is a diagram showing a three-dimensional 511 image.
[0104]
The solid 511 has at least a part of its surface spherical, and the letter “A” is drawn on the spherical surface of the solid 511.
[0105]
The three-dimensional position / attitude detection device 513 includes a position calculation unit 513a, an attitude calculation unit 513b, and a state detection unit 513c. The position calculation unit 513a obtains a contour image (FIG. 61 (b)) of the solid 511 by performing binarization processing on the image (FIG. 61 (a)) obtained by photographing the solid 511 with the camera 512. The position x, y, z in the real space of the solid 511 is calculated from the position and size of the solid contour on the contour image. The posture calculation unit 513b cuts out a three-dimensional part on the solid 511 (FIG. 61 (a)) based on the contour image (FIG. 61 (b)), and performs various images and various shapes on the image (FIG. 61 (c)). The postures θ, φ, and ψ of the solid 511 are calculated by comparing the whole or part of the template image group corresponding to the posture. The state detection unit 513c detects the designated position of the solid 511 based on at least one of color information and luminance at the hole position of the captured image.
[0106]
FIG. 62 shows an example of a solid 511. In this solid 511, dots of the same size are located at the points where the three vertices of a quasi-polyhedron composed of a regular pentagon and a regular hexagon inscribed in a sphere contact the sphere, and the center of the quasi-polyhedron and One character is provided at each point where a straight line connecting the center of gravity of the surface of the quasi-polyhedron intersects the spherical surface. The position of these dots can be detected with higher accuracy as the size is reduced, and the size difference from the solid 511 is increased, so that the three-dimensional posture of the solid can be calculated with high accuracy. Therefore, it is desirable that the minimum size is identifiable on the captured image.
[0107]
In FIG. 63, a solid 511 is placed at a position (x, y, z) in a coordinate system in a real space based on the camera 512 with the optical center C of the lens as the origin and the optical axis of the lens as the Z axis. FIG. The solid 511 has the same design as that shown in FIG. The camera 512 can be handled as an ideal camera without lens distortion or the like by correcting the optical distortion of the captured image.
[0108]
FIG. 64 shows an image obtained by capturing the solid 511. u is the horizontal axis of the image, and v is the vertical axis of the image. In the figure, reference numeral 514 denotes a contour obtained by fitting an ellipse to the detected solid edge. ξ is an angle formed by a straight line connecting the center of gravity of the contour and the image origin to the u axis. d is the distance between the center of gravity of the contour and the image origin. b is the major axis of the contour ellipse.
[0109]
FIG. 65 shows the dot position P on the captured image. _s It is a figure which shows the geometrical relationship with the position P of a dot in real space. In the figure, the point P is the dot position P on the captured image. _s And a straight line connecting the optical center C of the lens and the spherical surface of the solid 511.
[0110]
FIG. 66 is a diagram showing the positions of three dots detected on the captured image (FIG. 66 (a)) and three character templates created by calculation based on the positions of these dots (FIG. 66). 66 (b)). Here, the posture of the sphere such that the dot is positioned in the figure is limited to three. For all of these three postures, templates are created by the following method, for example.
[0111]
67 and 68 are diagrams for explaining a method of creating a graphic template by a computer. In FIG. 67, C _V Is a virtual camera located at infinity. O is the origin of the virtual space in the computer, X is the horizontal axis, Y is the vertical axis, and Z is the virtual camera C _V Is the optical axis. R is a point on the axis Z, and the distance OR is equal to the radius of the phantom sphere. Line segment OG is an angle Q from line Z to line segment OR ₁ Is a line segment rotated only by S _V Is a virtual plane passing through G and perpendicular to the line segment OG. A straight line T passing through G is on a plane including the axis Y and the axis Z, and is perpendicular to the line segment OG. Angle Q ₂ Is the virtual plane S _V Is the declination from the straight line T when rotated about the line segment OG. It should be noted that the figure has a virtual plane S so that its center of gravity matches G. _V Located above. For illustration purposes, only the outline is shown for the graphic.
[0112]
FIG. 68 shows a virtual plane S. _V Virtual camera C obtained by projecting the graphic placed above onto the XY plane _V It is an image. G _T Is the position of G on the image. The template is this virtual camera C _V The image contains a figure and the position G _T A partial image cut out in a rectangular area centered at is subjected to blurring processing using a Gaussian filter or the like.
[0113]
FIG. 69 is a flowchart illustrating a method for detecting the state by calculating the position / posture of the solid 511 in the real space from the image obtained by capturing the solid 511 of FIG. 62 using the apparatus of FIG. It is assumed that the radius of the spherical portion of the solid 511 is known.
1. The optical distortion of the image obtained by imaging the solid 511 is corrected to obtain a corrected image (step 531).
2. In the corrected image, an ellipse that is the outline of the sphere is detected using Hough transform or the like, and the ellipse parameters (ξ, d, b) shown in FIG. 64 are calculated (step 532).
3. Since the radius of the sphere is known between the position (x, y, z) of the solid 511 shown in FIG. 63 and the parameters (ξ, d, b) of the ellipse shown in FIG. There is a one-to-one correspondence. (X, y, z) is calculated from (ξ, d, b) by solving an equation representing this correspondence using the Newton-Raphson method or the like (step 533).
4). In the corrected image, comparison is performed by pattern matching processing using a template for one dot figure, and the positions of the three dots on the image are detected independently, and the geometric relationship shown in FIG. 65 is used. The position of the three dots in the real space is calculated. Based on the calculated positions of the three dots and the position (x, y, z) of the solid 511, the attitude of the solid 511 is calculated (step 534). However, when the positions of the three dots are used, there are three postures of the obtained solid 511. Therefore, all of these three postures are calculated and set as candidates for the posture of the solid 511. The calculation method of the posture of the solid 511 is as follows. As a model of the solid 511, a sphere having three dots on the surface is considered, and the positions of the dots in the real space shown in FIG. 65 are P1, P2, and P3, respectively. Since each dot is fixed to the sphere, when the attitude of the sphere changes, the positions P1, P2, and P3 also change. The calculated positions of the three dots are Q1, Q2, and Q3.
[0114]
[Formula 6]

Is determined as a posture of the solid 511. However,
[0115]
[Outside 1]

Are distances between P1 and Q1, P2 and Q2, and P3 and Q3, respectively.
5). For all three calculated postures, three templates are created by cutting out the character portion by the method described with reference to FIGS. 67 and 68. Since the positional relationship between the dot and the character is known, the position of the character (center of gravity position) on the corrected image can be obtained from this relationship. Therefore, pattern matching using these three templates with this position as a reference. Processing is performed sequentially. The degree of matching between the obtained templates and the corrected image is compared, and the posture corresponding to the template having the highest evaluation is selected, and this is set as the accurate posture of the solid 511 (step 535).
[0116]
[Eighth Embodiment]
FIG. 70 is an example of a figure provided on the spherical surface of the solid 511. Different figures at the positions of the two vertices of the regular icosahedron inscribed in the spherical surface touch the spherical surface, and a straight line connecting the center of the regular icosahedron and the center of gravity of the surface of the regular icosahedron Four dots are respectively provided at the positions of the points that intersect the spherical surface. As in the second embodiment, it is desirable that these dots have a minimum size that can be identified on the captured image. In FIG. 78A, the graphic is a graphic similar to “O” and “C”, and in FIG. 70B, the graphic is a graphic similar to “C” and “C”. In the figure, it is shown in a plan view.
[0117]
Also in the solid according to the present embodiment, the three-dimensional position / orientation can be detected by the same device as in the second embodiment. According to this embodiment, since simple figures are used instead of characters, the calculation cost can be further reduced and the processing speed can be increased.
[0118]
In the seventh and eighth embodiments as well, it is possible to capture an infrared image by a method such as using a translucent solid body provided with an infrared light source as the solid body 511 and using an infrared transmission filter as the camera 512. A camera may be used. By doing so, when the part of the solid 511 is cut out on the captured image, the influence of ambient light can be reduced and the outline of the solid 511 can be made clear.
[0119]
In the seventh and eighth embodiments, 3 to 4 dots of the same size are provided at the vertex positions of a regular polyhedron or a quasi-regular polyhedron inscribed in the spherical surface of the solid 211, and the center of the polyhedron and the polyhedron of the polyhedron An example in which one or two different graphics are provided at the position where the straight line connecting the center of gravity of the surface intersects the spherical surface, and an example in which the positional relationship between the dots and the different graphics is reversed are shown. . However, other rotationally symmetric figures such as circles may be used in place of these dots, and the sizes of these rotationally symmetric figures are not necessarily the same. If two images are captured above, only two may be used. Furthermore, the positional relationship between the rotationally symmetric graphic and the graphic different from each other is not necessarily the relationship shown in the above embodiment as long as it is known.
[0120]
In addition to the above-described three-dimensional position, posture, and state input method implemented by dedicated hardware, a program for executing the above-described method is recorded on a computer-readable recording medium. The program recorded on the recording medium may be read by a computer system and executed. The computer-readable recording medium refers to a recording medium such as a floppy disk, a magneto-optical disk, a CD-ROM, or a storage device such as a hard disk device built in the computer system. Furthermore, a computer-readable recording medium is a server that dynamically holds a program (transmission medium or transmission wave) for a short period of time, as in the case of transmitting a program via the Internet, and a server in that case. Some of them hold programs for a certain period of time, such as volatile memory inside computer systems.
[0121]
【The invention's effect】
As described above, the present invention has the following effects.
1. A three-dimensional posture and state of a solid can be detected and input at a low calculation cost, and can be applied as a three-dimensional mouse.
2. When the solid is a sphere or a disk, the three-dimensional position of the solid can be detected and input with a small calculation cost.
[Brief description of the drawings]
FIG. 1 is a diagram showing a first example of a solid with a state designation function for 3D device input according to the present invention.
FIG. 2 is a diagram showing a second example of a solid with a state designation function for 3D device input according to the present invention.
FIG. 3 is a diagram illustrating a third example of a solid with a state designation function for inputting a three-dimensional device according to the present invention.
FIG. 4 is a diagram showing a fourth example of a solid with a state designation function for inputting a three-dimensional device according to the present invention.
FIG. 5 is a diagram showing a fifth example of a solid with a state designation function for inputting a three-dimensional device according to the present invention;
FIG. 6 is a diagram illustrating a sixth example of a solid with a state designation function for inputting a three-dimensional device according to the present invention;
FIG. 7 is a diagram showing a seventh example of a solid with a state designation function for inputting a three-dimensional device according to the present invention.
FIG. 8 is a diagram illustrating an eighth example of a solid with a state designation function for inputting a three-dimensional device according to the present invention;
FIG. 9 is a diagram showing a ninth example of a solid with a state designation function for inputting a three-dimensional device according to the present invention;
FIG. 10 is a diagram showing the spectral distribution of the reflectance of the outer surface of the sphere and the spectral distribution of the internal light source and the painted surface.
FIG. 11 is a diagram illustrating a tenth example of the three-dimensional device input state specifying function-added solid according to the present invention;
FIG. 12 is a diagram showing an eleventh example of the three-dimensional device input state specifying function-added solid according to the present invention;
FIG. 13 is a diagram showing an outline of a system.
FIG. 14 is a diagram showing a posture detection template for associating a sphere image with a quaternion representing a posture.
FIG. 15 is a diagram for explaining an image signal pattern in the vicinity of a hole in a sphere image;
FIG. 16 is a diagram for explaining a method of enabling specification of a continuous value by continuous control of light emission luminance.
FIG. 17 is a diagram showing a time-series light emission pattern and multi-value designation.
FIG. 18 is a block diagram of a twelfth example of a three-dimensional posture detection solid according to the present invention and a three-dimensional posture input apparatus according to the first embodiment;
FIG. 19 is a diagram for explaining an example of a solid 21 and an arrangement of figures on the surface of the solid 21;
FIG. 20 is a diagram for explaining a method of creating a graphic template image by a computer.
FIG. 21 is a virtual plane S _V Virtual camera C obtained by projecting the above figure onto the XY plane _V FIG.
FIG. 22 is a diagram showing a table in which created template image groups are arranged.
FIG. 23 is a diagram for explaining the relationship between an ellipsoid axis L, a posture in real space, and a position of a pole P on a captured image;
FIG. 24 is a diagram illustrating a partial image obtained by cutting out an ellipsoidal part from a captured image.
FIG. 25 is a flowchart showing a procedure for obtaining the posture of the solid in the first embodiment.
FIG. 26 is a diagram illustrating a thirteenth example of a three-dimensional device with a state designation function for inputting a three-dimensional device according to the present invention.
FIG. 27 is a diagram illustrating a fourteenth example of a three-dimensional device with a state designation function for inputting a three-dimensional device according to the present invention.
FIG. 28 is a diagram illustrating a fifteenth example of the three-dimensional device input state specifying function-added three-dimensional object according to the present invention.
FIG. 29 is a diagram illustrating a sixteenth example of a three-dimensional device with a state designation function for inputting a three-dimensional device according to the present invention.
FIG. 30 is a diagram for explaining the arrangement of figures on the surface of a seventeenth example of a three-dimensional device with state designation function for three-dimensional device input according to the present invention;
FIG. 31 is a diagram for explaining a three-dimensional positional relationship between dots drawn on a solid 21;
FIG. 32 is a diagram showing a template image for detecting large dots and small dots.
FIG. 33 is a diagram showing a table in which template image groups for detecting three different types of graphics are arranged.
FIG. 34 is a diagram illustrating a partial image obtained by cutting out an ellipsoidal part from a captured image.
FIG. 35 is a flowchart illustrating a procedure for obtaining a posture of a solid 21 in the three-dimensional posture input apparatus according to the second embodiment.
FIG. 36 is a diagram for explaining the arrangement of figures on the surface of the 18th example of the solid 21;
FIG. 37 is a diagram showing a template image for detecting dots.
FIG. 38 is a diagram illustrating a partial image obtained by cutting out an ellipsoidal part from a captured image.
FIG. 39 is a flowchart illustrating a procedure for obtaining a posture of a solid 21 in the three-dimensional posture input apparatus according to the third embodiment.
FIG. 40 is a diagram illustrating a nineteenth example of a three-dimensional device with a state specification function for inputting a three-dimensional device according to the present invention.
FIG. 41 is a diagram showing a template image for detecting a circle.
FIG. 42 is a diagram illustrating a twentieth example of a solid with a state specifying function for inputting a three-dimensional device according to the present invention.
FIG. 43 is a diagram illustrating a twenty-first example of a three-dimensional device with a state designation function for inputting a three-dimensional device according to the present invention.
FIG. 44 is a diagram for explaining the arrangement of figures on the surface of a twenty-second example of a three-dimensional apparatus with state designation function for three-dimensional device input according to the present invention;
FIG. 45 is a diagram illustrating a partial image obtained by cutting out an ellipsoidal part from a captured image.
FIG. 46 is a flowchart illustrating a procedure for obtaining the posture of the solid 21 in the three-dimensional posture input apparatus according to the fourth embodiment.
FIG. 47 is a diagram showing a configuration of a three-dimensional posture input apparatus according to a fifth embodiment of the present invention.
FIG. 48 is a diagram illustrating a twenty-third example of the three-dimensional device with state specification function-added solid 101 according to the present invention.
49 is a diagram showing a positional relationship between a main part 101 and rotationally symmetric figures 121 and 131 in the solid 101 shown in FIG. 48. FIG.
50 shows how a solid 101 is placed in a coordinate system in real space with the camera 102 as a reference, with the optical center C of the lens of the camera 102 as the origin and the optical axis of the lens as the Z axis. FIG.
FIG. 51 is a diagram showing a template image for detecting a dot-like figure.
52 is a diagram illustrating a captured image of the solid 101 (FIG. 52A) and the detection result of the outline of the main part 201 of the solid 101 and each dot-shaped figure in the captured image. FIG.
FIG. 53 is a diagram showing a geometric relationship between the position of a figure on a dot on a captured image and the dot-like figures 221 and 231 in the coordinate system in the real space.
54 is a flowchart showing a procedure for obtaining a three-dimensional position and posture of the solid 101 shown in FIG. 48. FIG.
55 is a diagram showing a second example of the solid 101 shown in FIG. 48. FIG.
FIG. 56 is a flowchart showing a procedure for obtaining a three-dimensional position and orientation of the solid 101 shown in FIG. 55;
FIG. 57 is a diagram showing a third example of the solid 101. FIG.
58 is a diagram illustrating a fourth example of the solid 1. FIG.
FIG. 59 is a diagram illustrating a fifth example of the solid 101. FIG.
FIG. 60 is a configuration diagram of a three-dimensional posture input device according to a seventh embodiment of the present invention.
61 is a diagram showing an image obtained by the camera 212 in FIG. 60 (FIG. 61A) and an image obtained by the three-dimensional posture input device (FIG. 61B, FIG. 60C).
62 is a diagram illustrating an example of a solid 211. FIG.
FIG. 63 is a diagram illustrating a state in which a solid 211 is placed in a coordinate system in real space with the camera 212 as a reference.
FIG. 64 is a diagram illustrating an image obtained by capturing a solid 211;
FIG. 65 is a diagram showing a geometric relationship between a dot position P on a captured image and a dot position p in real space;
FIG. 66 is a diagram showing the positions of three dots detected on a captured image and three character templates created based on the positions of these dots.
FIG. 67 is a diagram for explaining a graphic template creation method using a computer;
FIG. 68 is a diagram illustrating an image of a virtual camera calculated by a computer.
FIG. 69 is a flowchart showing processing for calculating the position and orientation of a solid 211 in the seventh embodiment.
FIG. 70 is a diagram showing an example of a figure provided on the spherical surface of the solid 211;
[Explanation of symbols]
1 ball
1a, 1b, 1c hole
2 Power supply
2a switch
2b LCD controller
3a Blue LED
3b Green LED
3c diffusion plate
4 Light source unit
4a body
4b Diffusion surface
4c Blue LED
5 Signal unit
5a housing
5b Piston
5c spring
5d end face
6 wires
7 Slider
8 Color LCD
9 Optical fiber
9a Diffusion surface
10 Support
10a Through hole
11 Support rod
11a lever
11b Shade-shaped pallet
12 disc
12a Disc part
12b Tubular body
12c Diffusion plate
13 Color camera
21 Solid for 3D posture input
22 Camera
23 3D posture input device
31 axis posture calculation unit
32 Solid posture calculation unit
33 State detector
41-48, 51-63, 71-76, 81-87 steps
101 Solid for 3D position / posture input
102 camera
103 3D Position / Attitude Detection Device
104 Circle center corresponding to ellipse 211s
105 plane including circle
106 Center 104 sphere
107 Real space position of figure 221
108 The intersection of the plane 105 and the straight line connecting the position 131s and the lens center C
131 Position calculator
132 Figure detector
133 Attitude calculation unit
134 State detection unit
201, 202, 203 Main part
211
221, 222, 223 First rotationally symmetric figure
231, 232, 233 Second rotationally symmetric figure
211s Circle representing the contour obtained by fitting to the edge 111
221s The position of the detected dot-shaped disk 221
231s The position of the detected dot-shaped disk 231
301-308, 401-408 steps
511 solid
512 camera
513 3D position / attitude detection device
513a Position calculation unit
513b Posture calculation unit
513c state detection unit
514 contour
531 to 636 steps

Claims (35)

By inputting a captured image to a three-dimensional posture input device, the three-dimensional posture and state can be input,
Inside, one flashable light source that emits light with a predetermined spectral distribution or two or more flashable light sources that emit light with different spectral distributions,
The outer surface has one or more holes through which light emitted from the light source passes,
The inner surface has a spectral distribution reflection characteristic that includes the spectral distribution of the light source, and the outer surface has a spectral distribution reflection characteristic that is different from any of the spectral distributions of the inner light source.
Solid with state designation function for 3D posture input. By inputting a captured image to a three-dimensional posture input device, the three-dimensional posture and state can be input,
Inside, a movable reflecting surface having reflection characteristics of a predetermined spectral distribution or two or more movable reflecting surfaces having reflection characteristics of different spectral distributions are provided,
The outer surface has one or more holes through which the signal reflected by the reflecting surface passes,
The outer surface has a reflection characteristic of a spectral distribution different from any of the spectral distributions of the reflective surface;
Solid with state designation function for 3D posture input. By inputting a captured image to a three-dimensional posture input device, the three-dimensional posture and state can be input,
Inside, it has one or more reflective surfaces whose reflection characteristics can be controlled to a predetermined spectral distribution,
The outer surface has one or more holes through which the light reflected by the reflecting surface passes,
The outer surface has a reflection characteristic of a spectral distribution different from any of the spectral distributions of the reflective surface;
Solid with state designation function for 3D posture input. An outer surface of the rotationally symmetric body including at least a part that is a rotationally symmetric body and including a figure disposed at a position where the outer surface of the rotationally symmetric body and the axis of the rotationally symmetric body intersect. Have
The solid with a state designation function for inputting a three-dimensional posture according to any one of claims 1 to 3. The figure arranged at the position where the outer surface of the rotationally symmetric body and the axis of the rotationally symmetric body intersect is a non-rotational symmetric figure.
The solid with a state designation function for the three-dimensional posture input device according to claim 4. The figure that looks different depending on the posture is arranged at a rotationally symmetric figure arranged at a position where the surface of the rotationally symmetric body and the axis of the rotationally symmetric body intersect, and a point-symmetrical position around the rotationally symmetric figure. Including two or more rotationally symmetric figures,
The solid with a state designating function for three-dimensional posture input according to claim 4. The figure different in appearance depending on the posture further includes a mutually different figure paired with the rotationally symmetric figure arranged at the point-symmetrical position, respectively.
The solid with a state designation function for inputting a three-dimensional posture according to claim 6. The rotationally symmetric figures arranged at the point-symmetrical positions are different colors from each other.
The solid with a state designation function for inputting a three-dimensional posture according to claim 6. The rotationally symmetric body is one of an ellipsoid, a cuboid, a rotationally symmetric trapezoidal cone, a regular polyhedron, a quasi-regular polyhedron, and a shell-like body thereof.
A three-dimensional body with a state designation function for inputting a three-dimensional posture according to any one of claims 4 to 8. It has a disc-shaped or ring-shaped main part,
A first rotationally symmetric figure arranged on a plane parallel to the main part, the center of which is on the rotational symmetry axis of the main part and does not include the main part;
A member having a second rotationally symmetric figure disposed on a plane including the main part and outside a region surrounded by an edge of the main part.
The solid with a state designation function for inputting a three-dimensional posture according to any one of claims 1 to 3. The first and second rotationally symmetric figures have the same size and the same shape,
The solid with a state designation function for inputting a three-dimensional posture according to claim 10. The first and second rotationally symmetric figures are different colors;
The solid with a state designation function for inputting a three-dimensional posture according to claim 10. At least a portion of the outer surface is a spherical surface, and has a figure on the spherical surface that looks different depending on the posture,
The solid with a state designation function for inputting a three-dimensional posture according to any one of claims 1 to 3. The figure includes two or more rotationally symmetric figures and one or more different figures different from the rotationally symmetric figures.
The solid with a state designation function for inputting a three-dimensional posture according to claim 13. The two or more rotationally symmetric figures are all the same.
The solid with a state designation function for inputting a three-dimensional posture according to claim 14. Having the rotationally symmetric figure at the position of the vertex of a regular polyhedron or a quasi-regular polyhedron inscribed in the spherical surface;
The straight line connecting the different figure to the center of the polyhedron and the center of gravity of the surface of the polyhedron is located at a point where the spherical surface intersects.
The solid with a state designating function for three-dimensional posture input according to claim 14 or 15. Having a figure different from each other at a vertex of a regular polyhedron or a quasi-regular polyhedron inscribed in the spherical surface;
The rotationally symmetric figure has a straight line connecting the center of the polyhedron and the center of gravity of the surface of the polyhedron at the point where the spherical surface intersects.
The solid with a state designating function for three-dimensional posture input according to claim 14 or 15. A method for inputting a three-dimensional posture and a state of a three-dimensional posture from a three-dimensional posture-input state specifying function-added three-dimensional captured image according to any one of claims 4 to 9,
An axis orientation calculation step of detecting a position where an outer surface of the rotationally symmetric body and an axis of the rotationally symmetric body intersect from the captured image, and calculating an orientation of the axis;
A three-dimensional posture calculating step of calculating the three-dimensional posture by comparing the captured image and a template image group of the figure corresponding to various postures of the three-dimensional shape;
State detection means for detecting a specified state of the solid based on at least one of color information and luminance information at the position of the hole of the captured image;
3D posture input method. The solid is a rotationally symmetric figure arranged at a position where an outer surface of the rotationally symmetric body and an axis of the rotationally symmetric body intersect, and two or more arranged at point-symmetrical positions around the rotationally symmetric figure A rotationally symmetrical figure and a rotationally symmetrical figure arranged at the point-symmetrical position and a pair of mutually different figures,
The solid posture calculation step includes:
Calculating the three-dimensional posture candidate by comparing the captured image and a template image group of rotationally symmetric figures arranged at the point-symmetrical positions corresponding to the various postures of the solid;
Selecting the exact posture of the solid from the candidates by comparing the captured image and the template images of different figures corresponding to the various postures of the solid,
The three-dimensional posture input method according to claim 18. The solid is a rotationally symmetric figure arranged at a position where an outer surface of the rotationally symmetric body and an axis of the rotationally symmetric body intersect, and two or more arranged at point-symmetrical positions around the rotationally symmetric figure And rotationally symmetric figures of different colors,
The solid posture calculation step includes:
Calculating the three-dimensional posture candidate by comparing the captured image and a template image group of rotationally symmetric figures arranged at the point-symmetrical positions corresponding to the various postures of the solid;
Selecting an accurate posture of the solid from the candidates based on color information at any position of a rotationally symmetric figure arranged at the point-symmetric position of the captured image,
The three-dimensional posture input method according to claim 18. A method for inputting a three-dimensional posture and a state of a three-dimensional posture from a captured image of a three-dimensional posture input state designation function according to any one of claims 10 to 12,
A position calculating step of detecting a contour of the main part in the captured image and obtaining a three-dimensional position of the solid from the contour;
A graphic detection stage for detecting and determining the positions of the first rotationally symmetric graphic and the second rotationally symmetric graphic in the captured image;
The three-dimensional posture of the solid is calculated based on the three-dimensional position of the solid obtained in the position calculating step and the positions of the first rotationally symmetric graphic and the second rotationally symmetric graphic detected in the graphic detecting step. Posture calculation stage to perform,
A state detection step of detecting a designated state of the cube based on at least one of color information and luminance information at the position of the hole of the captured image.
3D posture input method. The figure detection step includes:
A first stage of detecting a position of the rotationally symmetric figure on the captured image by comparing the captured image and a template image of the rotationally symmetric figure;
Determining whether each position corresponds to the first rotationally symmetric graphic or the second rotationally symmetric graphic from the positional relationship between each of the positions detected in the first stage and the contour; And having a stage of
The three-dimensional posture input method according to claim 21. The first and second rotationally symmetric figures of the solid are different colors;
The figure detection step includes:
A first stage of detecting a position of the rotationally symmetric figure on the captured image by comparing the captured image and a template image of the rotationally symmetric figure;
Determining whether each position corresponds to the first rotationally symmetric graphic or the second rotationally symmetric graphic from the color information of the captured image at each of the positions detected in the first stage; Having a stage,
The three-dimensional posture input method according to claim 21. A method for inputting a three-dimensional posture and a state of a solid from a three-dimensional captured image with a state designation function for inputting a three-dimensional posture according to any one of claims 13 to 17,
Obtaining a three-dimensional contour image from the captured image, and calculating a three-dimensional position of the solid from the contour image;
A posture calculation step of calculating the posture of the solid by comparing the captured image and a template image group of the whole or a part of the figure corresponding to various postures of the solid;
State detection means for detecting a specified state of the solid based on at least one of color information and luminance information at the position of the hole of the captured image;
3D posture input method. The solid has two or more rotationally symmetric figures on the spherical surface and one or more different figures different from the rotationally symmetric figures,
The posture calculating step includes:
Calculating the three-dimensional posture candidates by comparing the captured image and the two or more rotationally symmetric template images corresponding to the various postures of the solid;
Selecting the exact posture of the solid from the candidates by comparing the captured image and the template images of different figures corresponding to the various postures of the solid,
The three-dimensional posture input method according to claim 24. A device for inputting a three-dimensional posture and a state of a three-dimensional posture by inputting a three-dimensional captured image with a state designation function for three-dimensional posture input according to any one of claims 4 to 9,
An axis attitude calculating means for detecting a position where the surface of the rotationally symmetric body and the axis of the rotationally symmetric body intersect from the captured image, and calculating an attitude of the axis;
A three-dimensional posture calculating means for calculating the three-dimensional posture by comparing the captured image and a template image group of the figure corresponding to various postures of the three-dimensional object;
State detection means for detecting a specified state of the solid based on at least one of color information and luminance information at the position of the hole of the captured image;
3D posture input device. The solid is a rotationally symmetric figure arranged at a position where an outer surface of the rotationally symmetric body and an axis of the rotationally symmetric body intersect, and two or more arranged at point-symmetrical positions around the rotationally symmetric figure A rotationally symmetrical figure and a rotationally symmetrical figure arranged at the point-symmetrical position and a pair of mutually different figures,
The solid posture calculating means includes
Means for calculating candidates for the three-dimensional posture by comparing the captured image with a template image group of rotationally symmetric figures arranged at the point-symmetrical positions corresponding to the various postures of the three-dimensional object;
Means for selecting an accurate posture of the solid from the candidates by comparing the captured image and the template images of different figures corresponding to the various postures of the solid;
The three-dimensional posture input device according to claim 26. The solid is a rotationally symmetric figure arranged at a position where an outer surface of the rotationally symmetric body and an axis of the rotationally symmetric body intersect, and two or more arranged at point-symmetrical positions around the rotationally symmetric figure And rotationally symmetric figures of different colors,
The solid posture calculating means includes
Means for calculating candidates for the three-dimensional posture by comparing the captured image with a template image group of rotationally symmetric figures arranged at the point-symmetrical positions corresponding to the various postures of the three-dimensional object;
Means for selecting an accurate posture of the solid from the candidates based on color information at any position of a rotationally symmetric figure arranged at the point-symmetric position of the captured image;
The three-dimensional posture input device according to claim 26. A device for inputting a three-dimensional posture and a state of a three-dimensional posture by inputting a three-dimensional captured image with a state designation function for three-dimensional posture input according to any one of claims 10 to 12,
Position calculating means for detecting the outline of the main part in the captured image and obtaining the three-dimensional position of the solid from the outline;
Graphic detection means for detecting and determining the positions of the first rotationally symmetric graphic and the second rotationally symmetric graphic in the captured image;
The three-dimensional posture of the solid is calculated based on the three-dimensional position of the solid obtained by the position calculating means and the positions of the first rotationally symmetric graphic and the second rotationally symmetric graphic detected by the graphic detecting means. Attitude calculating means for performing,
State detection means for detecting a specified state of the solid based on at least one of color information and luminance information at the position of the hole of the captured image;
3D posture input device. The graphic detection means includes
A first means for detecting a position of the rotationally symmetric figure on the captured image by comparing the captured image with a template image of the rotationally symmetric figure;
Determining whether each position corresponds to the first rotationally symmetric figure or the second rotationally symmetric figure from the positional relationship between each of the positions detected by the first means and the contour; Having the means of
The three-dimensional posture input device according to claim 29. The first and second rotationally symmetric figures of the solid are different colors;
The graphic detection means includes
A first means for detecting a position of the rotationally symmetric figure on the captured image by comparing the captured image with a template image of the rotationally symmetric figure;
A second determining unit determines whether each position corresponds to the first rotationally symmetric graphic or the second rotationally symmetric graphic from the color information of the captured image at each of the positions detected by the first means. Having means,
The three-dimensional posture input device according to claim 29. An apparatus for inputting a three-dimensional posture and state of a solid by inputting a three-dimensional captured image with a state designation function for three-dimensional posture input according to any one of claims 13 to 17,
Position calculation means for obtaining the three-dimensional position of the solid from the contour image, obtaining the three-dimensional outline image from the captured image;
Posture calculation means for calculating the posture of the solid by comparing the captured image with a template image group of the whole or a part of the figure corresponding to various postures of the solid;
State detection means for detecting a specified state of the solid based on at least one of color information and luminance information at the position of the hole of the captured image;
3D posture input device. The solid has two or more rotationally symmetric figures on the spherical surface and one or more different figures different from the rotationally symmetric figures,
The posture calculating means includes
Means for calculating a candidate for the three-dimensional posture by comparing the captured image with a template image of the two or more rotationally symmetric figures corresponding to various postures of the three-dimensional object;
Means for selecting an accurate posture of the solid from the candidates by comparing the captured image and the template images of different figures corresponding to the various postures of the solid;
The three-dimensional posture input device according to claim 32. A three-dimensional posture input program for causing a computer to execute the three-dimensional posture input method according to any one of claims 18 to 25. A three-dimensional posture input program for causing a computer to execute the three-dimensional posture input method according to any one of claims 18 to 25 is recorded.
A computer-readable recording medium.

Images