JP3612347B2

JP3612347B2 - 3D force tactile display

Info

Publication number: JP3612347B2
Application number: JP10218794A
Authority: JP
Inventors: 亮吉平田; ▲すすむ▼ 舘; 太郎前田; 洋星野
Original assignee: Yaskawa Electric Corp
Current assignee: Yaskawa Electric Corp
Priority date: 1994-04-15
Filing date: 1994-04-15
Publication date: 2005-01-19
Anticipated expiration: 2020-01-19
Also published as: JPH08257947A

Description

【０００１】
【産業上の利用分野】
本発明は、仮想環境もしくは遠隔環境からの力覚及び触覚を、３次元空間において位置と姿勢を制御できる位置姿勢制御装置に搭載され、平面、曲面、辺、及び頂点を備え、隣り合う面と辺と面とで凸面もしくは凹面を構成した環境提示ツールの３次元空間内における位置と姿勢を、操作者の指先の位置と姿勢を基に制御することにより、操作者の指先に環境提示ツールを接触させることで、操作者に形状を提示する３次元力触覚ディスプレイ装置に関する。また、環境提示ツールに可動部を設け、操作者に親指と人差指で把持させることで、仮想環境ないしは遠隔環境において操作者が保持した物体の厚さ及び特性（慣性、粘性、及び剛性）を操作者に提示する３次元力触覚ディスプレイ装置に関する。
【０００２】
【従来の技術】
この種の装置は、例えば小宇宙（galaxy）での仮想環境とか、原子炉での遠隔環境とかにおける実際に触手できない環境の物体に対する３次元の力覚・触覚を操作者に提示することである。ところで、従来例としての力覚ディスプレイ装置においては、ジョイストリング型マスター・システム（joy string type master system）のように操作者の指がワイヤや糸で空間的に吊られており。このワイヤの他端にはワイヤの長さと張力を測定するセンサを配し、また適切な張力を発生するためのモータが設けられている。操作者の手先の位置と姿勢の６自由度の情報がワイヤの長さのデータからコンピュータで推定される。それと同時に操作者に仮想環境ないしは遠隔環境からの反力をワイヤの張力の差としてフィードバックする。
【０００３】
【発明が解決しようとする課題】
しかしながら、従来の力覚ディスプレイ装置では、ジョイストリング型マスター・システムのように操作者の指がワイヤや糸で空間的に吊られているために、操作者の指は常に拘束されており、操作者が速い動作をしたときに動作を制限されるという問題点があった。
また、従来の力覚ディスプレイ装置では、仮想環境もしくは遠隔環境において操作者が接触する物体の、頂点、辺（凹凸）、操作者が面上を触る感覚及び操作者の指が壁に突き当たる感覚は提示できないという問題点があった。
【０００４】
本発明は、仮想環境もしくは遠隔環境において操作者が接触した場合と同様な物体の触覚や力覚を操作者に提示することを目的としている。
また、仮想環境もしくは遠隔環境において操作者が保持した場合と同様な物体の厚さを操作者に提示することを目的としている。
さらに、操作者が仮想環境において接触したと同様な物体の計算機内でのモデルの特性（慣性、粘性、及び剛性）、ないしは操作者が遠隔環境において接触したと同様な物体の特性（慣性、粘性、及び剛性）を操作者に提示することを目的としている。
さらにまた、仮想環境もしくは遠隔環境において、操作者が非接触状態にあるときは操作者の動作が拘束されないようにすることを目的としている。
【０００５】
【課題を解決するための手段】
上記の目的を達成するために、本発明の３次元力触覚ディスプレイ装置においては、平面、曲面、辺、及び頂点を備え、隣り合う面と面とで凸面および凹面を構成した環境提示ツールを備え、この環境提示ツールを３次元空間において位置と姿勢を制御できる位置姿勢制御装置の先端に搭載し、操作者の指先の位置と姿勢を基に制御することにより、操作者の指先に環境提示ツールを接触させて、操作者に仮想環境もしくは遠隔環境における物体の形状、力覚及び触覚を提示するようにしており、環境提示ツールの材料には、操作者の指が環境提示ツールに接触、及び操作者が環境提示ツールを把持したとき変形しにくい軽量な材料が好ましい。
さらに、予め対象物体の位置情報に基づく接触面情報を記憶していて、操作者の指先の位置と姿勢を検出するセンサからの入力に基づいて接触面情報を送出する接触面情報コンピュータと、
上記接触面情報を逐次入力して、操作者の指先の位置と姿勢を検出するセンサと、操作者の指先の接近を検出する近接センサと、操作者の指先から加えられた力を検出する力センサからの入力により、３次元空間において位置と姿勢を制御できる位置姿勢制御装置の先端部に搭載された環境提示ツールの位置と姿勢のメカニカルインピーダンス制御の処理を実施するコンピュータと、
操作者の指先の位置と姿勢に基づいた上記コンピュータの演算出力により３次元空間における位置と姿勢の駆動指令を位置姿勢制御装置へ送出するサーボコントローラと、
前記サーボコントローラからの駆動指令に従い３次元空間において前記環境提示ツールの位置と姿勢を制御する前記位置姿勢制御装置とを具備している。
【０００６】
【作用】
上記のように、３次元空間において位置と姿勢を制御できる位置姿勢制御装置の先端に搭載され、平面、曲面、辺、及び頂点を備え、隣り合う面と面とで凸面もしくは凹面を構成した環境提示ツールの３次元空間における位置と姿勢を制御して、操作者に環境提示ツールを接触させることで仮想環境もしくは遠隔環境における物体の形状や触覚を提示する。
【０００７】
【実施例】
以下に、本発明の一実施例について図面を参照して説明する。
図１ないし図４は、環境提示ツール１０の例を示しており、図１は正面図、図２は平面図、図３は斜視図、図４（ａ）は側面図、図４（ｂ）は図４（ａ）のＡ−Ａ線に沿った矢視方向の断面図、図４（ｃ）は図４（ａ）のＢ−Ｂ線に沿った矢視方向の断面図をそれぞれ示している。この環境提示ツールは、平面、曲面、辺、及び頂点を備え、隣り合う面と面とで凸面および凹面を構成した構造になっている。
この環境提示ツール１０を搭載する位置姿勢制御装置１１の正面図および側面図を図５、図６に示す。環境提示ツール１０を位置姿勢制御装置１１の回転軸中心部分（図６の先端部１２）に搭載しており、３次元空間において環境提示ツール１０の位置と姿勢を制御することができる。
【０００８】
操作者が仮想環境もしくは遠隔環境におけるマニピュレータ１５で、平面物体に接触したときは、操作者の指先の位置と姿勢に基づいて、３次元空間において環境提示ツール１０の位置と姿勢を制御して操作者に図３の６の部分の中心に接触させることで、マニピュレータ１５の触覚と同じ触覚を操作者に提示する。
また、操作者が仮想環境もしくは遠隔環境において、凸面物体に接触したときは、操作者の指先の位置と姿勢に基づいて３次元空間において環境提示ツールの位置と姿勢を制御して、操作者に図３の１の部分を接触させることで、凸面物体との触覚を操作者に提示する。
また、操作者が仮想環境もしくは遠隔環境において、凹面物体に接触したときは、操作者の指先の位置と姿勢に基づいて３次元空間において環境提示ツールの位置と姿勢を制御して操作者に図３の２の部分を接触させることで、凹面物体の触覚を操作者に提示する。
同様に、凸曲面物体に接触したときは、環境提示ツール(図３)の３の部分に接触させ、凹曲面物体に接触したときは、環境提示ツール(図３)の４の部分に接触させる。
【０００９】
なお、操作者が仮想環境もしくは遠隔環境において物体を保持したとき、図３の開閉部分５の幅を制御して操作者に把持させることにより物体の厚さを提示する。すなわち、開閉部分５は上部５ｂ（固定部）と下部５ａ（可動部）から成り、中央部５ｃが５ｄを枢軸として下方に自在に開き、その開いた距離が物体の厚さを表す。
【００１０】
例えば、図７の斜視図で示す物体に触ったときの力覚及び触覚を操作者に提示するためには、操作者が仮想環境もしくは遠隔環境で物体に接触しているのは物体全体の一部分、すなわち図７のウインドウで示す部分（１本の指先を取り巻く矩形状の枠内）である。そして、図８の斜視図で示すように、操作者が仮想環境もしくは遠隔環境で物体と点接触している部分、たとえば、図８のウインドウ部（１本の指先を取り巻く矩形状の枠内）の×の部分を、位置姿勢制御装置１１の先端に搭載された環境提示ツールの３次元空間における位置と姿勢を制御して操作者に接触させることで、操作者に物体全体の力覚及び触覚を提示する。
これは、操作者の指先の位置と姿勢にあわせて、接触物体の接触近傍における接平面、接線となるように環境提示ツールの３次元空間における位置と姿勢を制御して接触近傍空間を実現することで、物体全体の触覚を操作者に提示することができるという考えに基づいている。また、位置姿勢制御装置１１に搭載された力センサにより操作者の加えた力を検出して、位置姿勢制御装置の３次元空間における位置と姿勢をインピーダンス制御して、操作者に接触させることで、物体の特性（慣性、粘性、及び剛性）を提示する。
【００１１】
例えば、図９（ａ）、（ｂ）、（ｃ）の斜視図で示す物体の一部分を１本の指先が接触する場合は、位置姿勢制御装置１１の先端に搭載された環境提示ツールの３次元空間における位置と姿勢を図１０（ａ）、（ｂ）、（ｃ）の斜視図で示すように制御して、操作者に接触させる。
つまり、図９（ａ）の物体の力覚及び触覚を操作者に提示するためには、図１０（ａ）に示すように、位置姿勢制御装置１１の先端に搭載された環境提示ツール１０の３次元空間における位置と姿勢を制御して、環境提示ツールの凸曲面を操作者に接触させることにより、図９（ａ）の物体の力覚及び触覚を操作者に提示する。また、位置姿勢制御装置１１に搭載された力センサにより操作者の加えた力を検出して、位置姿勢制御装置の３次元空間における位置と姿勢をインピーダンス制御して、操作者に接触させることで、物体の特性（慣性、粘性、及び剛性）を提示させる。図９（ｂ）、（ｃ）の場合、それぞれ図１０（ｂ）、（ｃ）に示すようにしてそれぞれの物体の力覚及び触覚を操作者に提示する。
【００１２】
すなわち、位置姿勢制御装置１１の先端に搭載された環境提示ツール１０の３次元空間における位置と姿勢を制御して、環境提示ツール１０の凸面、凹面、凸曲面、及び凹曲面を組み合わせて操作者に接触させることで、操作者が連続的に任意の形状の物体に接触した場合でも、任意の物体の力覚及び触覚を操作者に提示させる。
なお、平面、凸曲面及び凹曲面は、ある面の中央を常に接触させるように位置姿勢制御装置１０先端の３次元空間における位置と姿勢を制御して（接触物体の接触点における接平面になるように制御する）操作者に力覚及び触覚を提示することもできる。
【００１３】
次に制御系統図を説明する。
図１１はこの一実施例における制御構成を示すブロック図である。
また、図１２（ａ）はＣＣＤカメラからの遠隔環境に適用した全体システム図、図１２（ｂ）はコンピュータからの仮想環境に適用した全体システム図、図１３（ａ）はＨＭＤ周辺に設けた制御要素の斜視説明図、図１３（ｂ）、（ｃ）は物体の辺に接触するときの説明図である。
【００１４】
図１１において、図１３（ａ）、（ｂ）、（ｃ）に示すように、１本の指先の位置と姿勢を検出するパッシブ・マスター・アーム１１１からの位置と姿勢情報１１１ｐは、接触面情報の第１のコンピュータ１１２と、頭上搭載ディスプレイ（ＨＭＤ）に写る画像を変化させる仮想環境処理の第２のコンピュータ１１３と、環境提示ツール１０の３次元空間における位置と姿勢をインピーダンス制御する第３のコンピュータとに与えられる。
【００１５】
一方で操作者は、ＨＭＤ１１４とポヒマス・センサ（磁場検出原理に基づく位置姿勢センサpolhemus sensor）１１５を頭上に装着して、仮想環境処理の第２のコンピュータ１１３に写る仮想環境１４、もしくはＣＣＤカメラ１５に写る遠隔環境(Ｘ物体１６、Ｙ物体１７・・・など)を見る。その視界に入るそれぞれの環境は、ポヒマス・センサ１１５からの操作者の頭の位置と姿勢の検出により異なる環境が映し出される。つまり、仮想環境においては、コンピュータ内の環境の位置と姿勢が変更され、遠隔環境においては、ロボットコントローラ１３(ＣＣＤカメラ制御手段)を介してＣＣＤカメラ１５、マイクロホン１５ａ、マニピュレータ１５ｂを具備するロボットが、その３次元空間における位置と姿勢を制御される。
【００１６】
ここで、遠隔環境（ＣＣＤカメラ１５、マイクロホン１５ａ、マニピュレータ１５ｂ）について説明する。
遠隔環境のロボットには操作者と同じように、頭と腕が搭載されていて、操作者の動きに追従してロボットが動く。ロボットの眼の位置にＣＣＤカメラ１５と、耳の位置にマイクロホン１５ａ（ここで集音された遠隔環境の音声情報はＨＭＤ１１４の内部に備えるスピーカーを介して操作者に提示される）および右腕の位置にマニピュレータ１５ｂを備えており、接触面情報の第１のコンピュータ１１２は、予めポヒマス・センサ１１５の出力により駆動されるロボットコントローラ１３により制御されるＣＣＤカメラ１５に撮像される遠隔環境のＸ物体１６、Ｙ物体１７・・・について、時々刻々に変化する位置と姿勢情報１１１ｐに対応して格納されている接触面情報を環境提示ツール１０の位置と姿勢をインピーダンス制御する第３のコンピュータ１１６へ逐次送出している。遠隔環境では操作者の腕に遠隔環境でのロボット・アームが追従して（このロボット・アームは人間の腕と同じ自由度配置の機構を持っており、操作者の腕に追従して動作する）ロボットの力触覚を操作者に提示する。
【００１７】
次に、仮想環境について説明する。仮想環境１３での接触面情報の第１のコンピュータ１１２は仮想環境のモデルのワールド座標系に対する位置を記憶しており、時々刻々に変化する位置と姿勢情報１１１ｐに対応して格納されている接触面情報を、環境提示ツール１０の３次元空間における位置と姿勢をインピーダンス制御する第３のコンピュータ１１６へ逐次送出している。
【００１８】
そして、位置姿勢制御装置１１の先端に搭載された環境提示ツール１０の３次元空間における位置と姿勢をインピーダンス制御する第３のコンピュータ１１６では、操作者の指先の位置と姿勢を検出するパッシブ・マスター・アーム１１１からの位置と姿勢情報１１１ｐと、それに対応する第３のコンピュータ１１６からの接触面情報により環境提示ツール１０の３次元空間における位置と姿勢を制御する。さらに、環境提示ツール１０に近接する１本の指先の近接情報を近接センサ１１９ｂ（操作者の指先が環境提示ツールの面のどの位置にあるかを検出する）で検出するとともに、環境提示ツール１０が接触する１本の指先へ加える力を位置姿勢制御装置１１８に備えられた力センサ１１９ａで検出し、それらの検出した近接情報と接触の力情報を基に位置と姿勢情報１１１ｐのサーボ演算を実施する。
【００１９】
第３のコンピュータ１１６のディジタル演算出力（例えば電圧（指令））を受けて、サーボコントローラ１１７では、次段の位置姿勢制御装置１１８の駆動に適したアナログ駆動指令を作成して送出し、その駆動指令を受けた位置姿勢制御装置１１８はその先端部に搭載された環境提示ツール１０の３次元空間における位置と姿勢を制御して、操作者に接触させることで、接触面情報の第１のコンピュータ１１２における仮想環境もしくは遠隔環境に対応した力触覚を操作者へ提示する。ここで、逐次、サーボコントローラ１１７から位置姿勢制御装置の位置と姿勢情報が第３のコンピュータ１１６へフィードバックされている。
【００２０】
例えば、現在の指先から１０ｃｍ先に物体が存在しているときは、指先が１０ｃｍ移動したときにその物体の力覚及び触覚を操作者へ提示するように、位置姿勢制御装置の先端に搭載された環境提示ツールの３次元空間における位置と姿勢をを制御することで、仮想環境もしくは遠隔環境における物体に１本の指先が接触するのと全く同じ力覚と触覚を操作者へ提示する。また、位置姿勢制御装置に搭載された力センサにより検出された力情報を基に、環境提示ツールの３次元空間内における位置と姿勢をインピーダンス制御すれば、仮想環境もしくは遠隔環境における物体に１本の指先が接触するのと全く同じ物体の特性（慣性、粘性、剛性）を操作者へ提示する。さらに、環境提示ツール１０の開閉部５の幅を制御して操作者に親指と人差指で把持させることにより、物体の厚さを提示する。
【００２１】
次に、操作者が仮想環境もしくは遠隔環境において接触する物体（例えばＸ物体１６）の大きさと環境提示ツール１０の移動距離について説明する。
操作者が遠隔環境もしくは仮想環境において接触する物体と、環境提示ツール１０に設けられている物体との大きさの比率により、補間しながら環境提示ツール１０の位置３自由度と姿勢３自由度を変化させる必要がある。
【００２２】
例えば、仮想環境もしくは遠隔環境において、図１３（ｂ）に示す物体Ａ―Ｂの辺をなぞるように接触する場合、最初は図１３（ｃ）に示す環境提示ツールのａ点を触っていて、操作者の動きに追従して環境提示ツール１０の位置を変更しながら、最後はｂ点を触っているように位置姿勢制御装置１１を制御する。
遠隔環境もしくは仮想環境で頂点に接触した場合には、操作者に環境提示ツール１０の頂点を触って貰う。つまり、できるだけ環境提示ツール１０の形状を用いて、触覚を操作者へ提示する。なお、１本の指先が物体表面を滑る感覚は、環境提示ツール１０に実際に面が存在しているので、位置姿勢制御装置１１を固定して、操作者が面をなぞれば、指が物体表面を滑る感覚も提示できる。
【００２３】
本発明は、予め対象物体の位置情報に基づく接触面情報を記憶していて、操作者の指先の位置と姿勢を検出するセンサの入力に基づく接触面情報の送出処理を実施するコンピュータと、
上記送出された接触面情報を逐次入力していて、操作者の指先の位置と姿勢を検出するセンサと、操作者の指先の接近を検出する近接センサと、操作者の指先から加えられた力を検出する力センサの入力により、３次元空間における位置と姿勢を制御できる位置姿勢制御装置の先端部に搭載された、平面、曲面、辺、及び頂点を備え、隣り合う面と面とで凸面もしくは凹面を構成した環境提示ツールの位置と姿勢のメカニカルインピーダンス制御の処理を実施するコンピュータと、
操作者の指先の位置と姿勢に基づいた上記コンピュータの演算出力により３次元空間における位置と姿勢の駆動指令を位置姿勢制御装置へ送出するサーボコントローラと、
前記サーボコントローラからの駆動指令に従い３次元空間において前記環境提示ツールの３次元空間における位置と姿勢を制御する前記位置姿勢制御装置と、
を具備することを特徴とする３次元力触覚ディスプレイ装置である。
【００２４】
また、本発明は、予め対象物体の位置情報に基づく接触面情報を記憶して、操作者の指先の位置と姿勢を検出するセンサの入力に基づく接触面情報の送出処理
を実施するコンピュータと、
上記送出された接触面情報を逐次入力していて、操作者の指先の位置と姿勢を検出するセンサと、操作者の指先の接近を検出する近接センサと、操作者の指先から加えられた力を検出する力センサの入力により、３次元空間において位置と姿勢を制御できる位置姿勢制御装置の先端部に搭載された、平面、曲面、辺、および頂点を備え、隣り合う面と辺と面とで凸面もしくは凹面を構成した環境提示ツールの３次元空間における位置と姿勢のメカニカルインピーダンス制御の処理を行うコンピュータと、
予め対象物体と、操作者の身体モデルを記憶していて、操作者の頭部及び腕部の位置と姿勢を検出するセンサの入力に基づく視聴覚情報の送出処理を実施するコンピュータと、
操作者の指先の位置と姿勢に基づいた上記コンピュータの演算出力により３次元空間における位置と姿勢の駆動指令を位置姿勢制御装置へ送出するサーボコントローラと、
前記サーボコントローラからの駆動指令に従い３次元空間において前記環境提示ツールの位置と姿勢を制御する前記位置姿勢制御装置と、
を具備することを特徴とする３次元力触覚ディスプレイ装置である。
【００２５】
さらに、本発明は、予め対象物体の位置情報に基づく接触面情報を記憶していて、操作者の指先の位置と姿勢を検出するセンサの入力に基づく接触面情報の送出処理を実施するコンピュータと、
上記送出された接触面情報を逐次入力していて、操作者の指先の位置と姿勢を検出するセンサと、操作者の指先の接近を検出する近接センサと、操作者の指先から加えられた力を検出する力センサの入力により、３次元空間において位置と姿勢を制御できる位置姿勢制御装置の先端部に搭載された、平面、曲面、辺、及び頂点を備え、隣り合う面と面とで凸面もしくは凹面を構成した環境提示ツールの位置と姿勢のメカニカルインピーダンス制御の処理を実施するコンピュータと、
操作者の頭部の位置と姿勢を検出するセンサの入力により、操作者の頭部の位置と姿勢に追従するように制御される遠隔環境のロボットに搭載されたＣＣＤカメラと、
遠隔環境のロボットに搭載されたマイクロホンと、
操作者の腕部の位置と姿勢を検出するセンサの入力により、操作者の腕部の位置と姿勢に追従するように制御される遠隔環境のマニピュレータと、
操作者の指先の位置と姿勢に基づいた上記コンピュータの演算出力により３次元空間における位置と姿勢の駆動指令を位置姿勢制御装置へ送出するサーボコントローラと、
前記サーボコントローラからの駆動指令に従い３次元空間において前記環境提示ツールの３次元空間における位置と姿勢を制御する前記位置姿勢制御装置と、操作者に視聴覚情報を提示する視聴覚情報提示装置と、
を具備することを特徴とする３次元力触覚ディスプレイ装置である。
【００２６】
【発明の効果】
本発明は、以上説明したように構成されるので、以下に記載されるような多くの効果を奏する。
位置姿勢制御装置の先端に搭載された環境提示ツールに平面、凸面、凹面、凸曲面、凹曲面を設けて３次元空間における位置と姿勢を制御して操作者に環境提示ツールの平面部を接触させることで、操作者が仮想環境もしくは遠隔環境で接触した平面物体、凸面物体、凹面物体、凸曲面物体、凹曲面物体の触覚を操作者に提示することができ、操作者は平面物体の触覚を感じながら効率よく作業を実施できる。
【００２７】
しかも、操作者が仮想環境もしくは遠隔環境で任意の形状の物体に連続的に接触した場合、操作者が仮想環境もしくは遠隔環境に接触している任意の接触面の接平面、接線となるように位置姿勢制御装置に搭載された環境提示ツールの３次元空間における位置と姿勢を制御して、操作者に接触させることで、任意の物体全体の触覚を操作者に提示することができ、操作者は任意の物体の触覚を感じながら効率よく作業を実施できる。
なお、環境提示ツールに可動部を設けて可動部の幅を制御して操作者に可動部を開閉させることで、操作者が仮想環境もしくは遠隔環境で保持した物体の厚さを操作者に提示することができ、操作者は物体の厚さを感じながら効率よく作業を実施できる。
【００２８】
さらに、位置姿勢制御装置に搭載された環境提示ツールの位置と姿勢及び環境提示ツールの可動部をインピーダンス制御することにより、操作者が仮想環境において接触した物体の特性（慣性、粘性及び剛性）あるいは操作者が遠隔環境において接触した物体の特性（慣性、粘性及び剛性）を操作者に提示することができ、操作者は物体の厚さを感じながら効率よく作業を実施できる。
しかもなお、操作者の指先の位置と姿勢を検出するパッシブ・マスター・アームの作業点と位置姿勢制御装置に搭載された環境提示ツールを追従させて常に非接触状態を保つことにより、操作者が仮想環境もしくは遠隔環境で非接触状態にあるとき操作者の動作を拘束することがない。
【図面の簡単な説明】
【図１】本発明の一実施例における３次元力触覚ディスプレイ装置に適用される環境提示ツールの正面図である。
【図２】本発明の一実施例における環境提示ツールの平面図である。
【図３】本発明の一実施例における環境提示ツールの斜視図である。
【図４】本発明の一実施例における環境提示ツールの側面図と側断面図である。
【図５】本発明の一実施例における環境提示ツールを搭載する位置制御装置の正面図である。
【図６】本発明の一実施例における位置姿勢制御装置の側面図である。
【図７】本発明の一実施例における環境提示ツールにより操作者に力覚及び触覚を提示することの可能な物体例を示す斜視図である。
【図８】本発明の一実施例における環境提示ツールにより、図７に記載された物体の力覚及び触覚を操作者に提示する接触物体の接触点近傍を示す斜視図である。
【図９】本発明の一実施例における環境提示ツールにより、操作者に力覚及び触覚を提示することの可能な幾つかの物体例を示す斜視図である。
【図１０】本発明の一実施例における環境提示ツールにより、図９に記載された物体の力覚及び触覚を操作者に提示する状態を示す斜視図である。
【図１１】本発明の一実施例における制御構成を示すブロック図である。
【図１２】本発明の一実施例における全体システム図
【図１３】頭上搭載ディスプレイの周辺の制御要素を説明する斜視図及び一部の動作説明図である。
【符号の説明】
１凸面部
２凹面部
３凸曲面部
４凹曲面部
５可動部
６平面部
１０環境提示ツール
１１位置姿勢制御装置
１２位置姿勢制御装置における環境提示ツールの搭載位置
１３ロボットコントローラ
１４仮想環境
１５ＣＣＤカメラ（電荷結合デバイス）
１５ａマイクロホン
１５ｂマニピュレータ
１６Ｘ物体
１７Ｙ物体
１１１パッシブ・マスター・アーム
１１１ｐ１本の指先の位置と姿勢情報
１１２接触面情報コンピュータ
１１３仮想環境の映像を送出するコンピュータ
１１４ＨＭＤ
１１５ポヒマス・センサ
１１６位置・姿勢インピーダンスコンピュータ
１１７サーボコントローラ
１１８位置姿勢制御装置
１１９ａ力センサ
１１９ｂ近接センサ[0001]
[Industrial application fields]
The present invention is mounted on a position and orientation control device capable of controlling the position and orientation in a three-dimensional space for force and touch from a virtual environment or a remote environment , and includes a plane, a curved surface, a side, and a vertex, By controlling the position and posture in the 3D space of the environment presentation tool that is convex or concave with sides and surfaces based on the position and posture of the operator's fingertip, the environment presentation tool can be applied to the operator's fingertip. The present invention relates to a three-dimensional force / tactile display device that presents a shape to an operator by bringing it into contact. In addition, a movable part is provided in the environment presentation tool, and the thickness and characteristics (inertia, viscosity, and rigidity) of the object held by the operator in a virtual environment or a remote environment can be operated by allowing the operator to grasp with the thumb and index finger. The present invention relates to a three-dimensional force / tactile display device presented to a person.
[0002]
[Prior art]
This type of device presents the operator with a three-dimensional force / tactile sensation on an object that cannot be actually touched in a virtual environment such as a galaxy or a remote environment in a nuclear reactor. By the way, in the force display device as a conventional example, an operator's finger is spatially hung with a wire or a thread like a joy string type master system. A sensor for measuring the length and tension of the wire is disposed at the other end of the wire, and a motor for generating an appropriate tension is provided. Information on the six degrees of freedom of the position and posture of the operator's hand is estimated by the computer from the wire length data. At the same time, the reaction force from the virtual environment or the remote environment is fed back to the operator as a difference in wire tension.
[0003]
[Problems to be solved by the invention]
However, in the conventional force display device, the operator's finger is always restrained because the operator's finger is suspended spatially with a wire or thread as in the joy string type master system. There is a problem that the operation is restricted when the person performs a fast operation.
Further, in the conventional force display device, in the virtual environment or the remote environment, the vertices and sides (unevenness) of the object that the operator contacts, the sense that the operator touches the surface, and the sense that the operator's finger hits the wall are There was a problem that it could not be presented.
[0004]
An object of the present invention is to present an operator with a tactile sensation or a force sensation similar to that when an operator touches in a virtual environment or a remote environment.
Another object of the present invention is to present to the operator the same object thickness as that held by the operator in a virtual environment or a remote environment.
In addition, the model characteristics (inertia, viscosity, and stiffness) of the same object in the computer as the operator touched in the virtual environment, or the characteristics of the object (inertia, viscosity) similar to the operator touched in the remote environment. And rigidity) to the operator.
It is another object of the present invention to prevent the operation of the operator from being restricted when the operator is in a non-contact state in a virtual environment or a remote environment.
[0005]
[Means for Solving the Problems]
In order to achieve the above object, the three-dimensional force / tactile display device of the present invention includes an environment presentation tool including a plane, a curved surface, a side, and a vertex, and a convex surface and a concave surface formed by adjacent surfaces. The environment presentation tool is mounted on the tip of a position and orientation control device capable of controlling the position and orientation in a three-dimensional space, and is controlled based on the position and orientation of the operator's fingertip, thereby providing the environment presentation tool to the operator's fingertip. To present the shape, force and tactile sensation of an object in a virtual environment or a remote environment to the operator, and the material of the environment presentation tool includes the operator's finger touching the environment presentation tool, and A lightweight material that is difficult to deform when the operator holds the environment presentation tool is preferable.
Further, contact surface information based on position information of the target object is stored in advance, and a contact surface information computer that sends out the contact surface information based on an input from a sensor that detects the position and posture of the fingertip of the operator;
A sensor that sequentially inputs the contact surface information to detect the position and posture of the operator's fingertip, a proximity sensor that detects the approach of the operator's fingertip, and a force that detects the force applied from the operator's fingertip A computer for performing mechanical impedance control processing of the position and orientation of the environment presentation tool mounted on the tip of the position and orientation control device capable of controlling the position and orientation in a three-dimensional space by input from the sensor;
A servo controller for sending a position and orientation driving command in a three-dimensional space to the position and orientation control device by the computation output of the computer based on the position and orientation of the fingertip of the operator;
The position and orientation control device that controls the position and orientation of the environment presentation tool in a three-dimensional space in accordance with a drive command from the servo controller.
[0006]
[Action]
As described above, an environment that is mounted at the tip of a position and orientation control device that can control the position and orientation in a three-dimensional space, includes a plane, a curved surface, a side, and a vertex, and a convex surface or a concave surface is configured by adjacent surfaces. By controlling the position and orientation of the presentation tool in the three-dimensional space and bringing the environment presentation tool into contact with the operator, the shape and tactile sensation of the object in the virtual environment or the remote environment are presented.
[0007]
【Example】
An embodiment of the present invention will be described below with reference to the drawings.
1 to 4 show an example of the environment presentation tool 10. FIG. 1 is a front view, FIG. 2 is a plan view, FIG. 3 is a perspective view, FIG. 4 (a) is a side view, and FIG. Is a cross-sectional view taken along the line AA in FIG. 4A, and FIG. 4C is a cross-sectional view taken along the line BB in FIG. 4A. Yes. This environment presentation tool includes a plane, a curved surface, a side, and a vertex, and has a structure in which a convex surface and a concave surface are formed by adjacent surfaces.
A front view and a side view of the position / orientation control device 11 on which the environment presentation tool 10 is mounted are shown in FIGS . The environmental presentation tools 10 are mounted on the rotation axis center portion of the position and orientation control unit 11 (tip 12 of FIG. 6), it is possible to control the position and orientation of the environment presented tool 10 in a three-dimensional space.
[0008]
When the operator touches a planar object with the manipulator 15 in a virtual environment or a remote environment, the position and orientation of the environment presentation tool 10 are controlled in the three-dimensional space based on the position and orientation of the fingertip of the operator. By bringing the person into contact with the center of the portion 6 in FIG. 3, the same tactile sense as that of the manipulator 15 is presented to the operator.
Further, when the operator touches a convex object in a virtual environment or a remote environment, the position and posture of the environment presentation tool are controlled in the three-dimensional space based on the position and posture of the operator's fingertip, By bringing the portion 1 in FIG. 3 into contact, a tactile sensation with the convex object is presented to the operator.
When the operator touches a concave object in a virtual environment or a remote environment, the position and orientation of the environment presentation tool are controlled in a three-dimensional space based on the position and orientation of the fingertip of the operator, and the operator is prompted. By bringing the second part of 3 into contact with each other, the tactile sensation of the concave object is presented to the operator.
Similarly, when contacting a convex curved surface object, it contacts 3 part of the environment presentation tool (FIG. 3), and when contacting a concave curved surface object, it contacts 4 part of the environment presentation tool (FIG. 3). .
[0009]
When the operator holds the object in a virtual environment or a remote environment, the thickness of the object is presented by controlling the width of the opening / closing portion 5 in FIG. That is, the opening / closing part 5 includes an upper part 5b (fixed part) and a lower part 5a (movable part). The central part 5c freely opens downward about 5d as a pivot, and the opened distance represents the thickness of the object .
[0010]
For example, in order to present the haptic sense and tactile sensation when the object shown in the perspective view of FIG. 7 is touched to the operator, the operator is in contact with the object in a virtual environment or a remote environment. That is, it is a portion shown in the window of FIG. 7 (within a rectangular frame surrounding one fingertip). As shown in the perspective view of FIG. 8, a portion where the operator is in point contact with an object in a virtual environment or a remote environment, for example, the window portion of FIG. 8 (in a rectangular frame surrounding one fingertip). By controlling the position and orientation in the three-dimensional space of the environment presentation tool mounted at the tip of the position / orientation control device 11 with the X part of the Present.
This realizes a near-contact space by controlling the position and orientation of the environment presentation tool in the three-dimensional space so that it becomes a tangent plane and tangent in the vicinity of the contact of the contact object in accordance with the position and posture of the fingertip of the operator. This is based on the idea that the tactile sensation of the entire object can be presented to the operator. Further, the force applied by the operator is detected by a force sensor mounted on the position / orientation control device 11, the position and orientation of the position / orientation control device in the three-dimensional space are impedance-controlled, and the operator is contacted. , Present the properties of the object (inertia, viscosity, and stiffness).
[0011]
For example, when one fingertip contacts a part of the object shown in the perspective views of FIGS. 9A, 9B, and 9C, the environment presentation tool 3 mounted on the tip of the position and orientation control device 11 is used. The position and orientation in the dimensional space are controlled as shown in the perspective views of FIGS. 10 (a), 10 (b), and 10 (c) to contact the operator.
That is, in order to present the haptic and tactile sensations of the object of FIG. 9A to the operator, as shown in FIG. 10A, the environment presentation tool 10 mounted at the tip of the position and orientation control device 11 is used. By controlling the position and orientation in the three-dimensional space and bringing the convex curved surface of the environment presentation tool into contact with the operator, the haptic and tactile sensations of the object in FIG. 9A are presented to the operator. Further, the force applied by the operator is detected by a force sensor mounted on the position / orientation control device 11, the position and orientation of the position / orientation control device in the three-dimensional space are impedance-controlled, and the operator is contacted. To present the properties (inertia, viscosity, and stiffness) of the object. In the case of FIGS. 9B and 9C, the sense of force and touch of each object are presented to the operator as shown in FIGS. 10B and 10C, respectively.
[0012]
That is, the position and orientation of the environment presentation tool 10 mounted at the tip of the position and orientation control device 11 is controlled in a three-dimensional space , and the convex surface, the concave surface, the convex curved surface, and the concave curved surface of the environment presentation tool 10 are combined. By making the contact, the operator can present the force sense and tactile sense of the arbitrary object even when the operator continuously contacts the object of arbitrary shape.
The plane, the convex curved surface, and the concave curved surface are controlled by controlling the position and posture in the three-dimensional space of the tip of the position / orientation control device 10 so as to always contact the center of a certain surface (the tangent plane at the contact point of the contact object) It is also possible to present haptic and tactile sensations to the operator.
[0013]
Next, a control system diagram will be described.
FIG. 11 is a block diagram showing a control configuration in this embodiment.
12A is an overall system diagram applied to a remote environment from a CCD camera, FIG. 12B is an overall system diagram applied to a virtual environment from a computer, and FIG. 13A is provided around the HMD. FIG. 13B and FIG. 13C are explanatory diagrams when contacting the sides of the object.
[0014]
In FIG. 11, as shown in FIGS. 13 (a), (b), and (c), the position and posture information 111p from the passive master arm 111 that detects the position and posture of one fingertip is obtained from the contact surface. A first computer 112 for information, a second computer 113 for virtual environment processing for changing an image captured on an overhead display (HMD), and a third for impedance control of the position and orientation of the environment presentation tool 10 in a three-dimensional space. Given to computers.
[0015]
On the other hand, the operator wears the HMD 114 and the Pohimass sensor (position and orientation sensor polhemus sensor based on the magnetic field detection principle) 115 above the head, and the virtual environment 14 or the CCD camera 15 reflected in the second computer 113 for virtual environment processing. The remote environment (X object 16, Y object 17. Each environment that enters the field of view is displayed by detecting the position and posture of the operator's head from the Pohimas sensor 115. That is, in the virtual environment, the position and orientation of the environment in the computer are changed, and in the remote environment, the robot including the CCD camera 15, the microphone 15a, and the manipulator 15b is connected via the robot controller 13 (CCD camera control means). The position and orientation in the three-dimensional space are controlled.
[0016]
Here, the remote environment (CCD camera 15, microphone 15a, manipulator 15b) will be described.
A robot in a remote environment is equipped with a head and arms like an operator, and the robot moves following the movement of the operator. A CCD camera 15 at the position of the robot's eye, a microphone 15a at the position of the ear (the voice information of the remote environment collected here is presented to the operator via a speaker provided in the HMD 114), and the position of the right arm The first computer 112 for contact surface information is an X object 16 in a remote environment that is imaged by the CCD camera 15 controlled in advance by the robot controller 13 driven by the output of the Pohimass sensor 115. , Y object 17..., And the contact surface information stored in correspondence with the position and orientation information 111p that changes from moment to moment are sequentially transmitted to the third computer 116 that controls impedance of the position and orientation of the environment presentation tool 10. Sending out. In a remote environment, the robot arm in the remote environment follows the operator's arm (this robot arm has the same degree of freedom as the human arm and operates following the operator's arm) ) Present the robot's haptic sensation to the operator.
[0017]
Next, the virtual environment will be described. The first computer 112 of the contact surface information in the virtual environment 13 stores the position of the model of the virtual environment with respect to the world coordinate system, and the contact stored corresponding to the position and posture information 111p that changes from moment to moment. the surface information, and sequentially sends the position and orientation in three-dimensional space environment presentation tool 10 to a third computer 116 for impedance control.
[0018]
In the third computer 116 that impedance-controls the position and posture in the three-dimensional space of the environment presentation tool 10 mounted at the tip of the position and orientation control device 11, the passive master detects the position and posture of the operator's fingertip. The position and orientation of the environment presentation tool 10 in the three-dimensional space is controlled by the position and orientation information 111p from the arm 111 and the corresponding contact surface information from the third computer 116. Further, the proximity information of one fingertip in proximity to the environment presentation tool 10 is detected by the proximity sensor 119b (detects the position of the operator's fingertip on the surface of the environment presentation tool), and the environment presentation tool 10 A force sensor 119a provided in the position / orientation control device 118 detects a force applied to one fingertip in contact with the finger, and performs servo calculation of the position and orientation information 111p based on the detected proximity information and contact force information. carry out.
[0019]
In response to the digital operation output (for example, voltage (command)) of the third computer 116, the servo controller 117 creates and sends an analog drive command suitable for driving the next-stage position / orientation control device 118, and drives it. Upon receiving the command, the position / orientation control device 118 controls the position and orientation in the three-dimensional space of the environment presentation tool 10 mounted on the tip thereof, and makes the operator contact the first computer of the contact surface information. The tactile sensation corresponding to the virtual environment or the remote environment in 112 is presented to the operator. Here, the position and orientation information of the position and orientation control device is sequentially fed back from the servo controller 117 to the third computer 116.
[0020]
For example, when an object exists 10 cm away from the current fingertip, it is mounted at the tip of the position / orientation control device so as to present the haptic and tactile sensations of the object to the operator when the fingertip moves 10 cm. By controlling the position and orientation of the environment presentation tool in the three-dimensional space, the same force and tactile sensation as when one fingertip contacts an object in a virtual environment or a remote environment is presented to the operator. Moreover, if impedance control is performed on the position and orientation of the environment presentation tool in the three-dimensional space based on the force information detected by the force sensor mounted on the position and orientation control device, one object is provided in the virtual environment or the remote environment. The same characteristics (inertia, viscosity, rigidity) of the object as the fingertip touches are presented to the operator. Further, the thickness of the object is presented by controlling the width of the opening / closing part 5 of the environment presentation tool 10 and causing the operator to hold it with the thumb and index finger.
[0021]
Next, the size of an object (for example, X object 16) that the operator contacts in a virtual environment or a remote environment and the moving distance of the environment presentation tool 10 will be described.
Depending on the ratio of the size of the object that the operator contacts in the remote environment or the virtual environment and the size of the object provided in the environment presentation tool 10, the degree of freedom of position 3 and the degree of freedom of posture 3 of the environment presentation tool 10 are interpolated. Need to change.
[0022]
For example, in a virtual environment or a remote environment, when touching to trace the side of the object AB shown in FIG. 13B, the user first touches the point a of the environment presentation tool shown in FIG. While changing the position of the environment presentation tool 10 following the movement of the operator, the position and orientation control device 11 is controlled so as to touch the point b at the end.
When the apex is touched in a remote environment or a virtual environment, the operator touches the apex of the environment presentation tool 10. That is, the tactile sensation is presented to the operator using the shape of the environment presentation tool 10 as much as possible. Note that the feeling that one fingertip slides on the object surface is that the surface is actually present in the environment presentation tool 10, so if the position and orientation control device 11 is fixed and the operator traces the surface, A sense of sliding on the object surface can also be presented.
[0023]
The present invention stores in advance contact surface information based on position information of a target object, and performs a process for sending contact surface information based on an input of a sensor that detects the position and posture of an operator's fingertip;
A sensor that detects the position and posture of the operator's fingertips, a proximity sensor that detects the approach of the operator's fingertips, and the force applied from the operator's fingertips. It is equipped with a plane, curved surface, side, and vertex mounted on the tip of a position and orientation control device that can control the position and orientation in a three-dimensional space by the input of a force sensor that detects the surface, and is convex between adjacent surfaces Alternatively, a computer that performs mechanical impedance control processing of the position and posture of the environment presentation tool that constitutes the concave surface;
A servo controller for sending a position and orientation driving command in a three-dimensional space to the position and orientation control device by the computation output of the computer based on the position and orientation of the fingertip of the operator;
The position and orientation control device that controls the position and orientation of the environment presentation tool in a three-dimensional space in a three-dimensional space in accordance with a drive command from the servo controller;
A three-dimensional force-tactile display device.
[0024]
In addition, the present invention stores a contact surface information based on the position information of the target object in advance, and performs a transmission process of contact surface information based on an input of a sensor that detects the position and posture of the operator's fingertips;
A sensor that detects the position and posture of the operator's fingertips, a proximity sensor that detects the approach of the operator's fingertips, and the force applied from the operator's fingertips. A plane, a curved surface, a side, and a vertex, which are mounted at the tip of a position and orientation control device capable of controlling the position and orientation in a three-dimensional space by the input of a force sensor that detects the A computer that performs mechanical impedance control processing of the position and posture in a three-dimensional space of an environment presentation tool configured with a convex surface or a concave surface with
A computer that stores a target object and an operator's body model in advance, and performs audiovisual information transmission processing based on an input of a sensor that detects the position and posture of the operator's head and arms;
A servo controller for sending a position and orientation driving command in a three-dimensional space to the position and orientation control device by the computation output of the computer based on the position and orientation of the fingertip of the operator;
The position and orientation control apparatus for controlling the position and orientation of the environment presentation tool in a three-dimensional space in accordance with a drive command from the servo controller;
A three-dimensional force-tactile display device.
[0025]
Furthermore, the present invention stores in advance contact surface information based on the position information of the target object, and performs a sending process of the contact surface information based on the input of a sensor that detects the position and posture of the operator's fingertips. ,
A sensor that detects the position and posture of the operator's fingertips, a proximity sensor that detects the approach of the operator's fingertips, and the force applied from the operator's fingertips. It is equipped with a plane, curved surface, side, and vertex mounted on the tip of a position and orientation control device that can control the position and orientation in a three-dimensional space by the input of a force sensor that detects the surface, and is convex between adjacent surfaces Alternatively, a computer that performs mechanical impedance control processing of the position and posture of the environment presentation tool that constitutes the concave surface;
A CCD camera mounted on a robot in a remote environment controlled to follow the position and posture of the operator's head by the input of a sensor that detects the position and posture of the operator's head;
A microphone mounted on a robot in a remote environment,
A remote environment manipulator that is controlled to follow the position and posture of the operator's arm by the input of a sensor that detects the position and posture of the operator's arm;
A servo controller for sending a position and orientation driving command in a three-dimensional space to the position and orientation control device by the computation output of the computer based on the position and orientation of the fingertip of the operator;
The position and orientation control device for controlling the position and orientation of the environment presentation tool in a three-dimensional space in a three-dimensional space in accordance with a drive command from the servo controller; an audiovisual information presentation device for presenting audiovisual information to an operator;
A three-dimensional force-tactile display device.
[0026]
【The invention's effect】
Since the present invention is configured as described above, it has many effects as described below.
The environment presentation tool mounted at the tip of the position and orientation control device is provided with a plane, convex surface, concave surface, convex curved surface, and concave surface to control the position and orientation in the three-dimensional space, and the operator touches the plane portion of the environment presentation tool. By doing so, the tactile sensation of the planar object, convex object, concave object, convex curved object, and concave curved object that the operator touched in a virtual environment or a remote environment can be presented to the operator. You can work efficiently while feeling.
[0027]
In addition, when the operator continuously touches an object of any shape in a virtual environment or a remote environment, the operator becomes a tangent plane or tangent of any contact surface in contact with the virtual environment or the remote environment. By controlling the position and orientation of the environment presentation tool mounted on the position and orientation control device in the three-dimensional space and bringing it into contact with the operator, it is possible to present the tactile sensation of an arbitrary object to the operator. Can efficiently work while feeling the touch of any object.
In addition, by providing a movable part in the environment presentation tool and controlling the width of the movable part to allow the operator to open and close the movable part, the operator presents the thickness of the object held in the virtual or remote environment to the operator The operator can perform work efficiently while feeling the thickness of the object.
[0028]
Furthermore, by controlling the impedance and position of the environment presentation tool mounted on the position and orientation control device and the movable part of the environment presentation tool, the characteristics (inertia, viscosity and rigidity) of the object that the operator touches in the virtual environment or The characteristics (inertia, viscosity, and rigidity) of the object that the operator contacts in the remote environment can be presented to the operator, and the operator can efficiently perform the work while feeling the thickness of the object.
In addition, by keeping the contact point of the passive master arm that detects the position and posture of the operator's fingertips and the environment presentation tool mounted on the position / orientation control device, the operator can maintain a non-contact state at all times. The operator's movement is not restrained when in a non-contact state in a virtual environment or a remote environment.
[Brief description of the drawings]
FIG. 1 is a front view of an environment presentation tool applied to a three-dimensional force / tactile display device according to an embodiment of the present invention.
FIG. 2 is a plan view of an environment presentation tool according to an embodiment of the present invention.
FIG. 3 is a perspective view of an environment presentation tool according to an embodiment of the present invention.
FIGS. 4A and 4B are a side view and a side sectional view of an environment presentation tool according to an embodiment of the present invention. FIGS.
FIG. 5 is a front view of a position control device equipped with an environment presentation tool according to an embodiment of the present invention.
FIG. 6 is a side view of the position / orientation control apparatus according to the embodiment of the present invention.
FIG. 7 is a perspective view showing an example of an object capable of presenting a haptic sense and a tactile sensation to an operator using an environment presentation tool according to an embodiment of the present invention.
8 is a perspective view showing the vicinity of a contact point of a contact object that presents the haptic and tactile sensation of the object shown in FIG. 7 to an operator using the environment presentation tool according to the embodiment of the present invention.
FIG. 9 is a perspective view showing several examples of objects that can present a force sense and a tactile sensation to an operator using an environment presentation tool according to an embodiment of the present invention.
10 is a perspective view showing a state in which the haptic and tactile sensations of the object shown in FIG. 9 are presented to the operator by the environment presentation tool in one embodiment of the present invention.
FIG. 11 is a block diagram showing a control configuration in an embodiment of the present invention.
FIG. 12 is an overall system diagram according to an embodiment of the present invention.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Convex part 2 Concave part 3 Convex part 4 Concave part 5 Movable part 6 Plane part 10 Environment presentation tool 11 Position and orientation control device 12 Position of environment presentation tool in position and orientation control device 13 Robot controller 14 Virtual environment 15 CCD camera (Charge coupled device)
15a Microphone 15b Manipulator 16 X object 17 Y object 111 Passive master arm 111p Position and posture information of one fingertip 112 Contact surface information computer 113 Computer 114 HMD for sending video of virtual environment
115 Posimus Sensor 116 Position / Attitude Impedance Computer 117 Servo Controller 118 Position / Attitude Control Device 119a Force Sensor 119b Proximity Sensor

Claims

A computer that stores in advance contact surface information based on the position information of the target object, and performs a process of sending contact surface information based on an input of a sensor that detects the position and posture of the operator's fingertip;
The contact surface information to be sent out is sequentially input, a sensor for detecting the position and posture of the operator's fingertip, a proximity sensor for detecting the approach of the operator's fingertip, and the force applied from the operator's fingertip It is equipped with a plane, curved surface, side, and vertex mounted on the tip of a position and orientation control device that can control the position and orientation in a three-dimensional space by the input of a force sensor that detects the surface, and is convex between adjacent surfaces Alternatively, a computer that performs mechanical impedance control processing of the position and posture of the environment presentation tool that constitutes the concave surface;
A servo controller for sending a position and orientation driving command in a three-dimensional space to the position and orientation control device by the computation output of the computer based on the position and orientation of the fingertip of the operator;
The position and orientation control apparatus for controlling the position and orientation of the environment presentation tool in a three-dimensional space in accordance with a drive command from the servo controller;
A three-dimensional force-tactile display device comprising:

A computer that stores in advance contact surface information based on the position information of the target object, and performs a process of sending contact surface information based on an input of a sensor that detects the position and posture of the operator's fingertip;
The contact surface information to be sent out is sequentially input, a sensor for detecting the position and posture of the operator's fingertip, a proximity sensor for detecting the approach of the operator's fingertip, and the force applied from the operator's fingertip It is equipped with a plane, curved surface, side, and vertex mounted on the tip of a position and orientation control device that can control the position and orientation in a three-dimensional space by the input of a force sensor that detects the surface, and is convex between adjacent surfaces Alternatively, a computer that performs mechanical impedance control processing of the position and posture of the environment presentation tool that constitutes the concave surface;
A computer that stores a target object and an operator's body model in advance, and performs audiovisual information transmission processing based on an input of a sensor that detects the position and posture of the operator's head and arms;
A servo controller for sending a position and orientation driving command in a three-dimensional space to the position and orientation control device by the computation output of the computer based on the position and orientation of the fingertip of the operator;
The position and orientation control apparatus for controlling the position and orientation of the environment presentation tool in a three-dimensional space in accordance with a drive command from the servo controller;
A three-dimensional force-tactile display device comprising:

A computer that stores in advance contact surface information based on the position information of the target object, and performs a process of sending contact surface information based on an input of a sensor that detects the position and posture of the operator's fingertip;
The touch surface information to be sent out is sequentially input and added from a sensor for detecting the position and posture of the operator's fingertip, a proximity sensor for detecting the approach of the operator's fingertip, and the operator's fingertip. It is equipped with a plane, curved surface, side, and vertex mounted on the tip of a position and orientation control device that can control the position and orientation in a three-dimensional space by the input of a force sensor that detects force. A computer that performs processing of mechanical impedance control of the position and posture of the environment presentation tool that constitutes the convex surface or the concave surface;
A CCD camera mounted on a robot in a remote environment controlled to follow the position and posture of the operator's head by the input of a sensor that detects the position and posture of the operator's head;
A microphone mounted on a robot in a remote environment,
A remote environment manipulator that is controlled to follow the position and posture of the operator's arm by the input of a sensor that detects the position and posture of the operator's arm;
A servo controller for sending a position and orientation driving command in a three-dimensional space to the position and orientation control device by the computation output of the computer based on the position and orientation of the fingertip of the operator;
The position and orientation control device that controls the position and orientation of the environment presentation tool in a three-dimensional space in accordance with a drive command from the servo controller; and an audiovisual information presentation device that presents audiovisual information to an operator;
A three-dimensional force-tactile display device comprising:

The position and posture in the three-dimensional space of the plane of the environment presentation tool mounted on the tip of the position and orientation control device is controlled so as to follow the position and posture of the operator's fingertip, and the fingertip of the operator The three-dimensional force / tactile display device according to claim 1, wherein an operator presents a tactile sensation of a plane touched in a virtual environment or a remote environment by contacting the plane.

The position and orientation in three-dimensional space of the convex surface of the environment presented tool mounted on the position and orientation control unit controls so as to follow the position and orientation of the fingertip of the operator, the contact convex on the operator's fingertip The three-dimensional force haptic display device according to claim 1, wherein an operator presents a haptic sensation of a convex object touched in a virtual environment or a remote environment.

The position and orientation in three-dimensional space of the concave of the environment presented tool mounted on the position and orientation control unit controls so as to follow the position and orientation of the operator's fingertip, contacts the concave surface to the operator's fingertip The three-dimensional force haptic display device according to claim 1, wherein an operator presents a haptic sensation of a concave object touched in a virtual environment or a remote environment.

A convex curved surface is formed on the operator's fingertip by controlling the position and posture of the convex curved surface of the environment presentation tool mounted on the position and orientation control device in the three-dimensional space so as to follow the position and posture of the operator's fingertip. The three-dimensional force / tactile display device according to claim 1, wherein an operator presents a tactile sensation of a convexly curved object touched in a virtual environment or a remote environment.

The position and orientation of the concave surface of the environment presentation tool mounted on the position and orientation control device in the three-dimensional space are controlled so as to follow the position and posture of the operator's fingertip, and the concave surface is placed on the operator's fingertip. The three-dimensional force / tactile display device according to claim 1, wherein an operator presents a tactile sensation of a concave curved surface object touched in a virtual environment or a remote environment.

The position and orientation in the three-dimensional space of the plane center of the environment presentation tool mounted on the position and orientation control device is controlled to be a tangential plane at the contact point of the contact object based on the position and orientation of the fingertip of the operator. The three-dimensional force according to claim 1, wherein the operator presents a tactile sensation of a plane, convex curved surface, or concave curved surface object in contact with a virtual environment or a remote environment by bringing the center of the plane into contact with the fingertip of the operator. Tactile display device.

The position and orientation of the environment presentation tool mounted on the position and orientation control device is controlled so that the position and orientation of the plane, curved surface, edge, and vertex in the three-dimensional space follow the position and orientation of the fingertip of the operator. 2. The tactile sensation according to claim 1, wherein the operator presents a tactile sensation in continuous contact with an object having an arbitrary shape in contact with the virtual environment or the remote environment by bringing a plane, a curved surface, a side, and a vertex into contact with the fingertip of the person. 3D force tactile display device.

An object held by the operator in a virtual environment or a remote environment by providing a movable part in the environment presentation tool, changing the width of the movable part according to the thickness of the object, and holding it with the thumb and index finger of the operator The three-dimensional force / tactile display device according to claim 1, wherein the three-dimensional force / tactile display device presents the thickness of the display.

The position and orientation of the environment presentation tool mounted on the position and orientation control device in a three-dimensional space , and the movable part of the environment presentation tool are mechanically impedance controlled to contact an operator with the environment presentation tool, and the environment By grasping the movable part of the presentation tool, the characteristics of the model in the computer (inertia, viscosity, and rigidity) of the object that the operator touches in the virtual environment, or the characteristics of the object that the operator touches in the remote environment ( The three-dimensional force / tactile display device according to claim 1, which presents inertia, viscosity, and rigidity).

When the operator is in a non-contact state in a virtual environment or a remote environment, the work point of the sensor that detects the position and orientation of the fingertip of the operator and the environment presentation tool mounted on the position and orientation control device follow, The three-dimensional force / tactile display device according to claim 1, wherein the operation of the operator is not restricted by maintaining a non-contact state between the fingertip of the operator and the environment presentation tool.