JPH08257947A - Three-dimensional display device for inner force sense and touch sense - Google Patents

Abstract

PURPOSE: To enable the touch of a planar object touched in a virtual or remote environment to be fed back to the operator by providing an environment presenting tool mounted in a position/attitude control device with a plurality of planes, and making the operator touch any of the planar parts after controlling the position and attitude of that part. CONSTITUTION: In an environment presenting tool 10 mounted in the center of the rotating shaft of a position/attitude control device, when the operator has touched a planar object in a virtual environment, the operator is made to touch a planar part 6 so that the touch of the planar object is fed back to the operator. When the operator has touched a convex object, the operator is made to touch a convex part 1, and a concave part 2 in the case of a concave part, so that the touch of the object is fed back to the operator. The position and attitude of such an environment presenting tool 10 are controlled in a three-dimensional space following drive commands from a servo controller, where the inner force sense and touch sense any object can be fed back to the operator.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention controls the position and orientation of an environment presenting tool mounted on a position and orientation control device, so that haptic and tactile sensations from a virtual environment or a remote environment can be obtained by tangential plane approximation. 3D force tactile display device for feedback to a user. In addition, a three-dimensional force tactile sensation that feeds back to the operator the thickness and characteristics (inertia, viscosity, and rigidity) of an object held by the operator in a virtual environment or a remote environment with a finger that is in point contact with the presentation surface and another finger. Display device

[0002]

2. Description of the Related Art Devices of this type are used, for example, in the microcosm (galax).
It is a device that displays three-dimensional force / tactile sensation to an object in an environment that cannot be actually touched, such as a virtual environment in y) or a remote environment in a nuclear reactor. By the way, in the conventional force display device, a joy string type master system (joy string type master system) is used.
s-tem), the operator's fingers are spatially suspended by wires or threads. A sensor for measuring the length and tension of the wire is arranged at the other end of the wire, and a motor for generating an appropriate tension is provided. Information on six degrees of freedom of the position and orientation of the operator's hand is estimated by a computer from the wire length data. At the same time, the reaction force from the virtual environment or the remote environment is fed back to the operator as the difference in the wire tension.

[0003]

However, in the conventional haptic display device, since the operator's finger is spatially suspended by the wire or thread as in the joystring type master system. There has been a problem that when the operator makes a fast motion, he is restricted by a wire or thread.
Further, in the conventional haptic display device, in the virtual environment or the remote environment for the operator, the vertices, edges (concavities and convexities) of the object with which the operator is in contact, the sensation and operation of the operator touching the surface There was a problem that the feeling of the person's finger hitting the wall could not be fed back.

An object of the present invention is to feed back the tactile sensation of an object touched by an operator in a virtual environment or a remote environment to the operator. It is also intended to feed back the thickness of the object held by the operator in the virtual environment or the remote environment to the operator. In addition, the characteristics of the model (inertia, viscosity and rigidity) of the object that the operator touches in the virtual environment in the computer or the characteristics of the object (inertia, viscosity and rigidity) that the operator touches in the remote environment are fed back to the operator. The purpose is to do. Furthermore, it is an object of the present invention to enable the operator to perform a fast motion without being restrained when the operator is in a non-contact state in a virtual environment or a remote environment.

[0005]

In order to achieve the above object, in a three-dimensional force / tactile display device of the present invention, an environment presenting tool is mounted on a position / orientation control device. The position where the environment presentation tool is mounted can be set at an appropriate position of the position and orientation control device, but it is particularly effective to mount the environment presentation tool at the center of rotation of the posture axis of the position and posture control device for the reasons described below. Is. Further, the environment presentation tool is preferably a lightweight material that is not easily deformed when the operator's finger touches the environment presentation tool and the operator holds the environment presentation tool. Then, the present invention (a) stores contact surface information based on position information of the target object in advance, and outputs contact surface information based on input of a sensor that detects the position and orientation of the operator's fingertip (b) ) The contact surface information is sequentially input, and the mechanical position / orientation of the environment presentation tool installed in the position / orientation control device is input by the sensor for detecting the position and orientation of the operator's fingertip, the proximity sensor, and the force sensor. A computer that performs the above-described processing of impedance control, a servo controller that sends a drive command by calculation output, and an environment presentation tool freely controls the position and orientation in a three-dimensional space according to the drive command from the servo controller. A three-dimensional force / tactile display device including a position / orientation control device.

[0006]

By controlling the position and orientation of the environment presentation tool mounted on the position and orientation control device configured as described above, the shape of the object contacted by the operator in the virtual environment or the remote environment is fed back to the operator. Work like. In addition, the operator works to feed back the thickness of the object held in the virtual environment or the remote environment to the operator. Then, the characteristics of the model in the computer (inertia, viscosity and rigidity) of the object that the operator touched in the virtual environment or the characteristics (inertia, viscosity and rigidity) of the object that the operator touched in the remote environment were fed back to the operator. Work to do.
Further, when the operator is in a non-contact state in the virtual environment or the remote environment, the operator is not restricted in the operation, and thus the operator can perform a fast operation.

[0007]

An embodiment of the present invention will be described below with reference to the drawings. First, FIGS. 5 and 6 are front and side views of the position and orientation control device that grips the environment presentation tool, and the environment presentation tool 10 is located at the center of the rotation axis of the position and orientation control device 11 (the mounting position 12 in FIG. 6). To be installed on. In all the drawings, the same symbols represent the same or corresponding members. Then, the form of the environment presentation tool adopted in this embodiment is
Fig. 1 is a front view, Fig. 2 is a plan view, Fig. 3 is a perspective view, and Fig. 4 is a side view of (a). Fig. 4 is a sectional view taken along the line AA of (a) of Fig. Sections (C) and (a) are cross-sectional views taken along the line BB of FIG.

In the environment presentation tool having such a shape, in FIG. 3, when the operator touches a surface object in a virtual environment or a remote environment, the operator touches the portion 6 to make a flat object Feed back the sense of touch to the operator. Further, in FIG. 3, when the operator touches the convex object in the virtual environment or the remote environment, the tactile sensation of the convex object is fed back to the operator by touching the part 1 with the operator. Further, in FIG. 3, when the operator touches the concave object in the virtual environment or the remote environment, the tactile sensation of the planar object is fed back to the operator by touching the part 2 with the operator. Further, in FIG. 3, when the operator touches a convex curved surface object in the virtual environment or the remote environment,
The tactile sensation of the convex curved surface object is fed back to the operator by touching the part of. Further, in FIG. 3, when the operator touches the concave curved surface object in a virtual environment or a remote environment, the tactile sensation of the concave curved surface object is fed back to the operator by touching the portion 4 with the operator.
Also, in FIG. 3, when the operator touches a flat surface, a convex curved surface, or a concave curved surface object in a virtual environment or a remote environment, the central portion of the operator is always kept in contact with the object (to make it a tangent plane of the model). ), The tactile sensation of a flat surface, a convex curved surface, and a concave curved surface object is fed back to the operator.

In FIG. 3, when the operator holds an object in a virtual environment or a remote environment, the operator opens and closes the portion 5 to feed back the thickness of the object to the operator. That is, the opening / closing part 5 is the upper part
5b (fixed part) and lower part 5a (movable part), the central part 5c
It freely opens downward with 5d as the axis of rotation, and the distance of the open tip represents the thickness of the object.

For example, in order to feed back the force sense and tactile sense of the object shown in the perspective view of FIG. 7 to the operator, when the operator is in contact with the object in the virtual environment or the remote environment, the contacted part Is a part of the entire object [the window part in FIG. 7 (in the rectangular frame surrounding one fingertip)]. In this embodiment, as shown in the perspective view of FIG. 8, the portion where the operator is in point contact with the object in the virtual environment or the remote environment [the window portion (the rectangular shape surrounding one fingertip is shown. [X] in the frame) is expressed by an environment presentation tool and brought into contact with the operator, and the force and tactile sensations of the entire object are fed back to the operator.

To explain the above concept as a whole, for example, a portion of an object touched by one fingertip shown in the perspective views of FIGS. 9A, 9B, and 9C is shown in FIG. ), (b), and (c) are perspective views, and can be expressed by an environment presentation tool that can be touched by one fingertip and brought into contact with the operator. That is, in order to feed back the force sense and tactile sense of the object shown in FIG. 9 (a) to the operator, as shown in FIG. 10 (a), the position and posture of the environment presentation tool installed in the position / orientation control device are determined. By changing and bringing the convex curved surface of the environment presentation tool into contact with the operator, the operator is fed back to the operator with the force sense and tactile sense of the object shown in FIG. 9 (a). In the case of Fig. 9 (b) and (c), Fig. 10 (b) and
As shown in (c), the force sense and tactile sense of each object are fed back to the operator.

That is, by changing the position and orientation of the environment presentation tool 10 mounted on the position and orientation control device and combining the convex surface, the concave surface, the convex curved surface, and the concave curved surface of the environment presentation tool with the operator, the operation is performed. Even when a person continuously contacts an object of any shape, the haptic and tactile sensations of any object are fed back to the operator. An operator who controls the position and orientation of the position / orientation control device (controls to be a tangent plane at the contact point) so that the center of a plane is always in contact with the plane, the convex curved surface, and the concave curved surface. It is also possible to feed back force and touch.

Since the whole of the environment presentation tool applied to this embodiment has been clarified as described above, the control system diagram of how to control this environment presentation tool will be explained. FIG. 11 is a block diagram showing the control configuration in this embodiment. Fig. 12 (a) is an overall system diagram for applying a remote environment from a CCD camera, Fig. 12 (b) is an overall system diagram for applying a virtual environment from a computer, and Fig. 13 (a) is an HM.
FIGS. 13B and 13C are perspective explanatory views of control elements provided around D, and FIGS. 13B and 13C are explanatory views when measuring the thickness of the object and the like.

In FIG. 11, FIG. 12 (a), 12 (b) and FIG.
Position and posture information from the passive master arm 111 that detects the position and posture of one fingertip, as shown in (a).
111p is a first computer 112 for contact surface information, a second computer 113 for virtual environment processing that changes an image displayed on an overhead display (HMD), and a first computer for impedance control of the position / orientation of the environment presentation tool 10. Given to 3 computers.

On the other hand, the operator wears the HMD 114 and the Pohimas sensor [polhe-mus sensor (position and orientation sensor based on the magnetic field detection principle)] 115 above the head, and the virtual image is displayed on the second computer 113 for virtual environment processing. Environment 14 or remote environment reflected by CCD camera 15 [X object 16, Y object 17 ...
Etc.] The respective environments in the field of view are projected through the robot controller 14 such that the CCD camera 15, the microphone 15a, and the robot controller 14 can project different environments based on the detection of the position and posture of the operator's head from the Pohimas sensor 115.
A robot equipped with the manipulator 15b is controlled in its position and orientation [CCD camera control means].

On the other hand, the environment presentation tool 10 in the remote environment [CCD camera 15, microphone 15a, manipulator 15b] operates as follows. That is, the robot in the remote environment has the same head and arms as the operator, and the robot moves following the movement of the operator. The CCD camera 15 is located at the eye position of the robot and the microphone 15a is located at the ear position.
[The voice information of the remote environment collected here is fed back to the operator through the speaker provided inside the HMD 114], the manipulator 15b is provided at the position of the right arm, and the first computer of the contact surface information is provided. 112 is a position and attitude information that changes from moment to moment with respect to the X object 16, Y object 17 ... In the remote environment, which is imaged by the CCD camera 15 controlled by the robot controller 13 which is driven in advance by the output of the Pohymas sensor 115. The contact surface information stored in correspondence with 111p is sequentially given to the third computer 116 that controls the impedance of the position / orientation of the environment presentation tool 10. In a remote environment, a robot arm in a remote environment follows the operator's arm [This robot arm has a mechanism with the same degree of freedom arrangement as a human arm, and moves in accordance with the operator's arm. Do the same motion], and feed back the tactile sensation of the robot to the operator.

Alternatively, the first computer 112 of the contact surface information in the virtual environment 13 stores the position of the model of the virtual environment with respect to the world coordinate system, and corresponds to the position and attitude information 111p that changes momentarily. The stored contact surface information is sequentially given to the third computer 116 that performs impedance control of the position / orientation of the environment presentation tool 10.

Then, in the third computer 116 for impedance control of the position / orientation of the environment presentation tool 10,
Position and posture information 111p from the passive master arm 111 that detects the position and posture of the fingertip and the corresponding third
The position / orientation of the environment presentation tool 10 is controlled by the contact surface information from the computer 116 of the computer. At that time, the proximity sensor 119 b [information of the proximity of one fingertip near the environment presentation tool 10 is used]. The position sensor detects the position of the hand of the environment presenting tool on the surface of the environment presenting tool]. The detected proximity information and contact force information are fed back to perform servo calculation of position and orientation information 111p.

The digital operation output [eg voltage (command)] of the third computer 116 is received to
The controller 117 creates an analog drive command suitable for driving the next position / orientation control device 118, and sends this command.
The position / orientation control device 118 that has received the drive command drives and controls the position / orientation of the environment presentation tool 10 attached to the tip end of the position and attitude control device 118, and the position corresponding to the virtual environment or the remote environment in the first computer 112 of the contact surface information.・ Show your posture. Here, the position / orientation is fed back from the servo controller 117 to the third computer 116.

In this way, the operator uses the overhead display (HMD) 114 and the Poimas sensor 115 to
For example, when an object exists 10 cm away from the current fingertip, if position and orientation information 111p is positioned with one fingertip so that the force and tactile sensations of the object are fed back when the object moves 10 cm. Following the above, the above-mentioned first to third computers etc. perform calculations, and the position / orientation of the environment presenting tool 10 is automatically changed and controlled, and the force and tactile sensation of the object up to the previously positioned fingertip. And servo. This is exactly the same force and tactile as one fingertip touches an object in a virtual environment or a remote environment, as well as inertia, rigidity, and viscosity of the characteristics of the object, and also the opening / closing part of the environment presentation tool 10 depending on the thickness of the object. 5, the contact surface information stored in advance is extracted from the position / orientation information 111p, and the position / orientation of the environment presentation tool 10 and the surface state represented by the information, 119 force sensor 119a, proximity sensor 119b. It is shown that a complete three-dimensional force tactile display device can be configured that freely controls the position and orientation in a three-dimensional space according to the drive command from the servo controller from the servo calculation by the feedback of the force information detected by. ing.

It should be noted that the size of an object (for example, X object 16) with which the operator contacts in the virtual environment or the remote environment and the moving distance of the environment presentation tool 10 will be referred to. It is necessary to change the position and orientation of the environment presentation tool 10 while performing interpolation depending on the size ratio of the object that the operator contacts in the remote environment or the virtual environment and the object provided in the environment presentation tool 10.

For example, when touching the edge of the object AB shown in FIG. 13B in a virtual environment or a remote environment,
The point a at the beginning of the environment presentation tool 10 shown in 13 (c) is touched, the position of the environment presentation tool 10 is changed following the movement of the operator, and the point b is touched at the end. When the vertex is touched in the remote environment or the virtual environment, the operator is asked to touch the vertex of the environment presentation tool 10. That is, the tactile sensation is fed back to the operator by using the shape of the environment presentation tool 10 as much as possible. In addition,
The feeling that one fingertip slides on the surface of an object is an environment presentation tool.
Since the surface actually exists in 10, the position and orientation control device 11
If you fix the position and the operator traces the surface, you can feel the finger slide on the surface of the object.

The main points of the present invention are as follows. That is, (a) contact surface information based on position information of the target object is stored in advance, and contact surface information transmission processing based on input of a sensor that detects the position and orientation of the fingertip of the operator (b) the contact surface Processing of mechanical impedance control of the position / orientation of the environment presentation tool installed in the position / orientation control device by inputting information sequentially, the sensor that detects the position and orientation of the operator's fingertip, the proximity sensor, and the force sensor Of the computer, a servo controller that sends a drive command by calculation output, and a position and attitude control that freely controls the position and attitude of the environment presentation tool in a three-dimensional space according to the drive command from the servo controller. A three-dimensional force tactile display device including a device, and (a) storing contact surface information based on position information of a target object in advance. The contact surface information is sent out based on the input of the sensor that detects the position and orientation of the operator's fingertip. (B) The sensor that sequentially inputs the contact surface information and detects the position and orientation of the operator's fingertip. And processing of mechanical impedance control of the position and orientation of the environment presentation tool installed in the position and orientation control device by inputting the proximity sensor and force sensor (c) The target object and the body model of the operator are stored in advance, and the operation is performed. A computer that performs the above-described processing of audiovisual information transmission processing based on the input of a sensor that detects the position and posture of the person's head and arm, a servo controller that transmits a drive command by calculation output, and this servo controller Position and orientation control device for freely controlling the position and orientation of the environment presentation tool in a three-dimensional space in accordance with the driving command of A three-dimensional force / tactile display device including a haptic information presenting device, and (a) a sensor that stores contact surface information based on position information of a target object in advance and detects the position and orientation of the operator's fingertip. (B) The above-mentioned contact surface information is sequentially input, and the position / orientation control device is input by a sensor that detects the position and attitude of the operator's fingertip, a proximity sensor, and a force sensor. The position of the operator's head is input by the computer that performs the above-mentioned processing of mechanical impedance control of the position / orientation of the installed environment presentation tool and the sensor that detects the position and orientation of the operator's head. The CCD camera mounted on the robot in the remote environment, which is controlled so as to follow the position and the posture, the microphone mounted on the robot in the remote environment, and the position and posture of the operator's arm are detected. The manipulator in the remote environment is controlled to follow the position and posture of the operator's arm by the input of the sensor, the servo controller that sends the drive command by the calculation output, and the environment according to the drive command from this servo controller. It can be said that the presentation tool is a three-dimensional force / tactile display device including a position / orientation control device that freely controls the position and orientation of the presentation tool in a three-dimensional space, and an audiovisual information presentation device that presents audiovisual information to the operator.

[0024]

Since the present invention is configured as described above, it has many special effects as described below. By providing a flat surface on the environment presentation tool installed in the position and orientation control device to control the position and orientation to bring the operator into contact with the flat surface portion of the environment presentation tool, the plane that the operator touches in a virtual environment or a remote environment The tactile sense of the object can be fed back to the operator. By providing a convex surface on the environment presentation tool installed in the position and orientation control device to control the position and orientation and bringing the convex surface of the environment presentation tool into contact with the operator, a convex object that the operator has contacted in a virtual environment or a remote environment The tactile sense of can be fed back to the operator. In addition, by providing a concave surface on the environment presentation tool installed in the position and orientation control device and controlling the position and posture to bring the operator into contact with the concave surface of the environment presentation tool, the operator has contacted in a virtual environment or a remote environment. The tactile sense of the concave object can be fed back to the operator. Furthermore, by providing a convex curved surface on the side surface of the environment presentation tool installed in the position and orientation control device to control the position and posture to bring the operator into contact with the convex curved surface of the environment presentation tool, the operator can operate in a virtual environment or remotely. It is possible to feed back the tactile sensation of a convex curved surface object that comes into contact with the environment to the operator. Furthermore, by providing a concave curved surface on the side surface of the environment presentation tool installed in the position and orientation control device to control the position and posture to bring the concave curved surface of the environment presentation tool into contact with the operator, the operator can create a virtual environment or It is possible to feed back the tactile sensation of a concave curved surface object contacted in a remote environment to the operator. Moreover,
When an operator continuously contacts an object of any shape in a virtual environment or a remote environment, the operator controls the position and orientation of the environment presentation tool installed in the position and orientation control device to allow the operator to operate in the virtual or remote environment. By implementing a part of an arbitrary object that is in contact with the environment presentation tool and bringing it into contact with the operator, the tactile sense of the entire arbitrary object can be fed back to the operator. In addition, by providing a movable part in the environment presentation tool and controlling the movable part to allow the operator to open and close the movable part, the operator can feed back the thickness of the object held in the virtual environment or the remote environment to the operator. You can Furthermore, by controlling the position and orientation of the environment presentation tool installed in the position and orientation control device and impedance control of the movable part of the environment presentation tool, the characteristics of the model in the computer of the object that the operator touches in the virtual environment ( The inertia (viscosity and rigidity) or the characteristics (inertia, viscosity and rigidity) of an object with which the operator contacts in a remote environment can be fed back to the operator. Moreover, by keeping the working point of the passive master arm that detects the position and posture of the operator's fingertip and the environment presentation tool mounted on the position and posture control device, and keeping the non-contact state at all times,
When the operator is in a non-contact state in the virtual environment or the remote environment, the operator is not restricted in the operation, and thus the operator can perform a fast operation.

[Brief description of drawings]

FIG. 1 is a front view of an environment presentation tool applied to a three-dimensional force / tactile display device according to an embodiment of the present invention.

FIG. 2 is a plan view of an environment presentation tool according to an embodiment of the present invention.

FIG. 3 is a perspective view of an environment presentation tool according to an embodiment of the present invention.

FIG. 4 is a side view and a side sectional view of the environment presentation tool according to the embodiment of the present invention.

FIG. 5 is a front view of a position / orientation control device for mounting an environment presentation tool according to an embodiment of the present invention.

FIG. 6 is a side view of a position / orientation control device according to an embodiment of the present invention.

FIG. 7 is a perspective view showing an example of an object capable of feeding back a force sense and a tactile sense to an operator by the environment presentation tool according to the embodiment of the present invention.

FIG. 8 is a perspective view showing the vicinity of a contact point for feeding back the force sense and tactile sense of the object described in FIG. 7 to the operator by the environment presentation tool according to the embodiment of the present invention.

FIG. 9 is a perspective view showing some examples of objects capable of feeding back force and tactile sensations to an operator by the environment presentation tool according to the embodiment of the present invention.

FIG. 10 is a perspective view showing a state in which the force sense and tactile sense of the object described in FIG. 9 are fed back to the operator by the environment presentation tool according to the embodiment of the present invention.

FIG. 11 is a block diagram showing a control configuration in an embodiment of the present invention.

FIG. 12 is an overall system diagram in an embodiment of the present invention.

FIG. 13 is a perspective view and a part of operation explanatory view for explaining control elements around the overhead display.

[Explanation of symbols]

1 Convex part 2 Concave part 3 Convex curved part 4 Concave curved part 5 Movable part 6 Plane part 10 Environment presentation tool 11 Position and posture control device 12 Position of environment presentation tool in position and posture control device 13 Robot controller 14 Virtual environment 15 CCD Camera (charge coupled device) 15a Microphone 15b Manipulator 16 X object 17 Y object 111 Passive master arm 111p Position and posture information of one fingertip 112 Contact surface information computer 113 Computer sending virtual environment image 114 HMD (overhead) On-board display) 115 Poimas sensor (head position and posture detection means) 116 Computer for mechanical impedance control of the position and orientation of the environment presentation tool mounted on the position and orientation control device 117 Servo controller 118 Position and orientation control device 119a Sensor 119b Proximity sensor (operator's fingertip shows environment Means for detecting whether at any position Le of the surface)

 ─────────────────────────────────────────────────── ─── Continuation of the front page (71) Applicant 594081179 Hiroshi Hoshino 1-15-10 Kita-Sentsu, Ota-ku, Tokyo (72) Inventor Ryokichi Hirata 2-1, Kurosaki-joseki, Hachiman-nishi-ku, Kitakyushu, Fukuoka Yaskawa Electric Co., Ltd. (72) Inventor Tate ▲ Susumu ▼ 2-31-14 Umezono, Tsukuba-shi, Ibaraki (72) Inventor Taro Maeda 1-13-32 Sengen 1-chome, Tsukuba, Ibaraki Prefecture Room 101 (72) Inventor Hiroshi Hoshino 1-15-10 Kita-senzuka, Ota-ku, Tokyo

Claims (13)

[Claims] 1. (a) Sending processing of contact surface information based on input of a sensor which stores contact surface information based on position information of a target object in advance and which detects a position and orientation of a fingertip of an operator (b) The mechanical impedance of the position / orientation of the environment presentation tool installed in the position / orientation control device by inputting the contact surface information sequentially and inputting the sensor for detecting the position and attitude of the operator's fingertip, the proximity sensor, and the force sensor. A computer that performs the above-described processing of control, a servo controller that sends a drive command by a calculation output, and an environment presentation tool that freely controls the position and orientation in a three-dimensional space according to the drive command from this servo controller. A three-dimensional force / tactile display device, comprising: a position / orientation control device. 2. (a) Sending processing of contact surface information based on input of a sensor that stores contact surface information based on position information of a target object in advance and that detects a position and orientation of an operator's fingertip (b) The mechanical impedance of the position / orientation of the environment presentation tool installed in the position / orientation control device by inputting the contact surface information sequentially and inputting the sensor for detecting the position and attitude of the operator's fingertip, the proximity sensor, and the force sensor. Control processing (c) The target object and the body model of the operator are stored in advance, and the above-described processing of transmitting audiovisual information based on the input of the sensor that detects the position and posture of the operator's head and arm is described above. A computer that performs the processing shown, a servo controller that sends a drive command by calculation output, and an environment presentation tool that freely moves its position in a three-dimensional space according to the drive command from the servo controller. Position and orientation control unit for controlling the attitude, the operator three-dimensional force tactile display apparatus characterized by comprising the audiovisual information presentation device for presenting audiovisual information to. 3. (a) Sending processing of contact surface information based on input of a sensor which stores contact surface information based on position information of a target object in advance and which detects a position and orientation of a fingertip of an operator (b) The mechanical impedance of the position / orientation of the environment presentation tool installed in the position / orientation control device by inputting the contact surface information sequentially and inputting the sensor for detecting the position and attitude of the operator's fingertip, the proximity sensor, and the force sensor. The remote environment is controlled so as to follow the position and posture of the operator's head by the input of the computer that performs the above-mentioned processing of control and the sensor that detects the position and posture of the operator's head. The CCD camera mounted on the robot, the microphone mounted on the robot in the remote environment, and the sensor that detects the position and posture of the operator's arm are used to track the position and posture of the operator's arm. The manipulator in the remote environment that is controlled to operate, the servo controller that sends the drive command by the operation output, and the position that freely controls the position and orientation of the environment presentation tool in the three-dimensional space according to the drive command from this servo controller. A three-dimensional force / tactile display device comprising an attitude control device and an audiovisual information presenting device for presenting audiovisual information to an operator. 4. The method according to claim 1, wherein the environment presenting tool mounted on the position / orientation control device is provided with a flat surface, and a tactile sense of the flat surface contacted by the operator in a virtual environment or a remote environment is fed back to the operator. Dimensional force tactile display device. 5. The environment presentation tool mounted on the position / orientation control device is provided with a convex surface, and the tactile sensation of a convex object contacted by the operator in a virtual environment or a remote environment is fed back to the operator. 3D force tactile display device. 6. The environment presentation tool mounted on the position / orientation control device is provided with a concave surface, and the tactile sensation of a concave object contacted by the operator in a virtual environment or a remote environment is fed back to the operator. 3D force tactile display device. 7. The environment presentation tool mounted on the position / orientation control device is provided with a convex curved surface, and the tactile sensation of a convex curved object contacted by the operator in a virtual environment or a remote environment is fed back to the operator. The three-dimensional force tactile display device described. 8. The method according to claim 1, wherein the environment presentation tool mounted on the position / orientation control device is provided with a concave curved surface, and the tactile sensation of the concave curved object that the operator contacts in a virtual environment or a remote environment is fed back to the operator. The three-dimensional force tactile display device described. 9. A flat surface, a convex curved surface, which an operator contacts in a virtual environment or a remote environment, by providing a flat surface on the environment presentation tool mounted on the position and orientation control device and keeping the center of the flat surface in contact with the operator. The three-dimensional force tactile display device according to claim 1, wherein the tactile sense of the concave curved surface object is fed back to the operator. 10. The position / orientation control device for the planar object, the convex object, the concave object, the convex curved object, and the concave curved object when an operator continuously contacts an object of an arbitrary shape in a virtual environment or a remote environment. The three-dimensional force tactile display device according to claim 1, wherein a tactile sense of an entire arbitrary object is fed back to an operator by combining the environment presenting tools installed in the. 11. The three-dimensional force / tactile display device according to claim 1, wherein the environment presenting tool is provided with a movable portion to feed back the thickness of an object held by the operator in a virtual environment or a remote environment to the operator. 12. The mechanical impedance control of the position and orientation of the environment presentation tool and the movable part of the environment presentation tool mounted on the position and orientation control device,
The feedback of the inertial, viscous and rigid characteristics of the model of the object contacted by the operator in the virtual environment in the computer or the inertial, viscous and rigid characteristics of the object contacted by the operator in the remote environment to the operator. The three-dimensional force / tactile display device according to item 1. 13. A working point of a sensor for detecting the position and orientation of the fingertip of the operator when the operator is in a non-contact state in a virtual environment or a remote environment, and the environment presentation mounted on the position and orientation control device. The operation is performed without being constrained by the operator by keeping the working point of the sensor that constantly detects the position and posture of the fingertip and the environment presentation tool in a non-contact state by following the tool. 3D force tactile display device.