JPH05232859A - Artificial reality providing device - Google Patents

Abstract

PURPOSE:To provide the artificial reality providing device in simple constitution which enables forth feedback corresponding to the motion of a human hand, a finger, and a body. CONSTITUTION:A touch sense part 1 in double-cylinder structure is provided covering a finger and the space between an outside skin 2 and an inside skin 3 is divided into plural sections 5 by plural partition walls. One of a liquid injection pipe 7 is coupled with each of the sections 5. Further, the touch sense part 1 is provided with a curvature quantity detection sensor along the internal surface of the cylinder body and liquid is injected into the specific section from the liquid injection pipe 7 according to a curvature quantity of finger detected by this sensor. The injection of the liquid makes the sense part 1 unflexible to limit the bending of the finger, and the operator feel a touch in the finger.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a device for realizing artificial reality, and provides force feedback to a human hand or finger so that an object does not actually exist therein. The present invention relates to a device for perceiving an object as if it were present.

[0002]

2. Description of the Related Art Recently, an environment in which an action that is difficult to actually experience is realized in a virtual world calculated by a computer, and humans are acting in the virtual world. The research on artificial reality that gives rise to sensation has become popular (see, for example, US Pat. No. 4,988,981). Conventionally, the field of computer graphics that appeals to human vision has been researched as a device that realizes artificial reality, but when the application field expands, a device that appeals to not only human vision but also tactile sense is needed. Came. Consider, for example, artificially creating a virtual ball that does not actually exist by moving a real human hand to grab the ball.
In this case, a means for detecting the amount of bending of the finger when the actual hand is moved is required.

As a means for detecting the actual bending amount of the finger, for example, US Pat. No. 4,937,44 is used.
As described in Japanese Patent No. 4 (Japanese Patent No. 4), it is known that a bending amount detection sensor is arranged along the finger portion of the glove. A glove for data input equipped with such a bending amount detection sensor and other position sensors is called a data glove. A virtual hand is displayed on the display based on various signals from the data glove.

When the data glove is put on the hand and the real hand is moved, the bending amount detection sensor provided along the finger of the hand detects the bending amount of the finger, and the display amount is displayed based on the bending amount of the finger. The virtual hand that appeared in moves.
Then, it is possible to display an image in which the virtual ball existing on the display is grasped by the virtual hand moving in accordance with the movement of the real hand.

In this case, however, it can only be confirmed visually on the display that the ball touches the finger and the finger cannot be bent further, and the presence of the ball is fed back to the actual hand as a force. You cannot do it. That is, it cannot be confirmed as a tactile sensation.

Therefore, in an artificial reality environment, it has been proposed to feed back a force corresponding to the movement of the finger of the hand to the finger, that is, to realize a tactile sensation by force feedback (for example, Iwata: "Force Display"). , Measurement and Control, Vol. 30, No. 6 (19
June 1991), p. 472-477).

As a means for realizing the force feedback, the above-mentioned document "Force Display" is used.
As described in (1) using a master arm, (2) using a joystick,
(3) There are various types such as those using yarn.

In the master arm of (1), a driving source such as a motor is incorporated in a master manipulator used for remote operation to obtain an operation reaction force. Further, (2) realizes force feedback by applying a reaction force to the joystick used as a pointing device with a motor or the like. Further, (3) is, for example, Sato et al .: “Interface device for virtual workspace-SPIDA
R- ", IEICE Technical Report, PRU, 89
As disclosed in -88, 51/58, (1989), the force feedback is generated by the yarn. In this form, a thread for limiting the movement of the finger is put out from the fingertip of the actual hand, and the tip is wound with a reel. The thread is reeled out freely until the finger touches the virtual ball, but by stopping the reel rotation at the point where the finger seems to have contacted the virtual ball and limiting the movement of the finger, it is as if the ball were there. It gives a sense of being present in.

[0009]

[Problems to be Solved by the Invention] However, (1)
It was difficult to give different force feedback to each joint of the fingers of the hand as the master arm of the device became bulky. Moreover, in the device using the joystick of (2), the direction of the reaction force is limited, and for example, the reaction force as when an object is gripped with a finger cannot be obtained. Further, in the device using the thread of (3), the movement of the finger is always limited only to the force in the direction from the fulcrum of the thread, and the force feedback when touching an object facing various directions that can actually occur. Is difficult to give.

It is therefore an object of the present invention to provide an artificial reality realizing device having a simple structure and capable of giving force feedback corresponding to the movement of a finger of a human hand or a joint of a body.

[0011]

In order to achieve the above-mentioned object, the artificial reality realization device of the present invention has a flexible container to be arranged along a part whose posture changes, and a fluid inside the container. Fluid injection means for injecting, and a bend amount detection means arranged along the container for detecting the bend amount of the container,
And a fluid amount control means for controlling the amount of fluid injected from the fluid injection means into the container based on the amount of bend detected by the bend amount detection means.

[0012]

The container is arranged along the finger, for example. When the amount of the fluid injected into the container is small, the container can be deformed by the flexibility of the container itself, so that there is no resistance even if the finger is bent. When it is detected from the bending amount of the container that the finger touches the virtual object, the fluid is injected into the container and expanded. Therefore, the container is less likely to be deformed, and a drag force is applied to the finger, which becomes a feedback force.

[0013]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The features of the present invention will be specifically described below with reference to the accompanying drawings.

FIG. 1 shows in principle the haptic part of the artificial reality realizing device according to the present invention. It should be noted that here, as an example, an apparatus having a configuration for covering one finger will be described. The actual device is constructed by combining these and expanding them to the hands and arms.

The tactile portion 1 is a cylindrical tubular body as a whole, and its inner diameter is approximately equal to the thickness of a human finger A. As shown in FIG. It is fitted like a wrap. The tactile part 1 has a double cylindrical structure, and the space between the outer skin 2 and the inner skin 3 is divided into a plurality of compartments 5 in the axial direction by a plurality of axial partition walls 4, and further, in FIG. As shown, a plurality of circumferential partition walls 6 are also divided into a plurality of sections in the circumferential direction. Each compartment 5 is filled with a suitable fluid B such as a liquid such as silicone oil or a gas such as an inert gas. This compartment 5 functions as a flexible container for filling with the fluid B. One end of a flexible fluid injection pipe 7 made of synthetic resin or the like for taking in and out the fluid B is connected to each section 5. The haptic unit 1 functions as a force feedback mechanism as described later.

Further, the tactile portion 1 is provided with a bending amount detecting sensor 8 along the inner surface of the cylindrical body. As shown in FIG. 2, this bending amount detection sensor 8 is, for example,
It has a structure in which a light emitting element 10 such as a light emitting diode is attached to one end of a flexible sensor pipe 9 formed of rubber or a black synthetic resin having flexibility, and one end of an optical fiber 11 is connected to the other end. An aluminum film is vapor-deposited on the inner surface of the sensor pipe 9 to form a light reflecting surface 12. A light receiving element 13 such as a phototransistor is attached to the other end of the optical fiber 11, as described later.

The light from the light emitting element 10 is reflected directly and by the light reflecting surface 12 and is applied to one end of the optical fiber 11. The light incident on one end of the optical fiber 11 passes through the optical fiber 11 and is supplied to the light receiving element 13 to be converted into an electric signal. When the bend sensor pipe 9 bends, the amount of light attenuation from the light emitting element 10 to one end of the optical fiber 11 increases, so that the amount of bending of the finger can be detected by detecting the output of the light receiving element 13.

The other end of each fluid injection pipe 7 and the other end of the optical fiber 11 led out from the tactile unit 1 are connected to the control unit 14 of the artificial reality realizing device shown in FIG.

The control unit 14 converts the light from the optical fiber 11 into an electric signal from the light receiving element 13 (see FIG. 2) to detect a bending amount of the finger, and a bending amount detecting circuit 15 and the bending amount detecting circuit 15. Calculating the strength of the feedback force to be given to the finger based on the amount of bending of the finger detected by the calculator 16, and driving according to the strength of the feedback force calculated by the calculation of the calculator 16. An electromagnetic actuator 18 driven via a circuit 17 and a fluid amount control device 19 axially driving a piston 19b in a cylinder 19a by the electromagnetic actuator 18 are provided. Drive circuit 1
7, electromagnetic actuator 18, and fluid amount control device 19
Constitute a force feedback mechanism drive device 20.

One space sandwiching the piston 19b of the fluid amount control device 19 communicates with the atmosphere through a through hole 19c, and the other space is filled with a fluid and led out from the tactile unit 1 to the fluid injection pipe 7. Is connected to the other end. Drive circuit 17, electromagnetic actuator 18, and fluid amount control device 19
May be provided corresponding to each section 5 of the haptic unit 1 and driven independently, but may be configured to drive several sections in the vicinity in common.

Next, the operation of the above-mentioned artificial reality realizing device will be described with reference to the flowchart of FIG.

As an example, consider a case where a finger bends and hits the ball, and the finger cannot be bent even if the finger is bent further. The bending amount of the finger is always detected by the bending amount detection sensor 8 (step 101). Initially, the fluid B in each compartment 5 of the haptic part 1 is not sufficiently filled in a state in which the finger freely moves. Therefore, the tactile portion 1 maintains the softness (flexibility) of the material itself of the outer skin 2 and the inner skin 3, and the finger can be freely moved. When the finger is bent, the signal from the bending amount detection circuit 15 informs the calculator 16 of the bending amount of the finger, and when the finger is bent to the position where the virtual ball exists (step 102), the driving circuit 17 is operated. The electromagnetic actuator 18 is urged toward the protruding side, the fluid B in the fluid amount control device 19 is pushed out, and the fluid is injected into each compartment 5 inside the finger (lower side in FIG. 1A) via the fluid injection pipe 7. Injected (step 10
3). When the fluid is fed into the compartment 5 of the inner part of the finger of the haptic part 1, the compartment expands and tries to expand to the maximum volume of the compartment, and as a result, the material of the outer skin 2 and the inner skin 3 is soft. Loses. Therefore, a force acts on the part inside the finger in the direction of extending the finger to give force feedback, and as a result, the finger feels as if there is an object there. That is, the tactile unit 1 acts as a force feedback mechanism. At this time, if independent force feedback is performed for each finger joint, it is possible to give a difference in the degree of force application due to the difference in each part of each finger joint.

Here, if the finger is extended, the computer 1 is operated based on the signal from the bending amount detection circuit 15 (step 104).
6 determines that the amount of bending of the finger does not reach the ball (step 105), urges the electromagnetic actuator 18 to the backward side through the drive circuit 17, draws the fluid B in the fluid amount control device 19, and draws the finger. The amount of fluid in the inner compartment 5 is reduced (step 106), and the state where the movement of the finger is not restricted is restored.

As described above, the fluidity is fed into and returned from the compartment 5 of the tactile section 1 to control the bendability of the tactile section 1, and the expanded compartment 5 does not try to change its shape. By using, you can give a feeling as if an object is there by applying force to your finger. That is, artificial reality can be realized.

FIG. 5 shows an embodiment in which the haptic part is incorporated in the data glove. However, the drawing is a cross-sectional view of the right-hand data glove cut along the palm surface and viewed from the palm side.

The data glove 21 is provided with a tactile unit 1 that individually wraps five fingers.
Are connected by a flexible member made of synthetic resin, synthetic leather or the like covering the palm and the back of the hand.

FIG. 6 is a sectional view showing a detailed structure of the tactile unit 1 provided on the fingertip portion of the data glove 21. Two
The outer skin 2 and the inner skin 3 of the heavy-cylindrical tactile unit 1 are made of vinyl,
It is made of a flexible polymer material such as polyethylene. The material of the outer skin 2 and the inner skin 3 is processed by a method such as high frequency welding to divide the space between the cylindrical outer skin 2 and the inner skin 3 into several independent sections 5. One end of one fluid injection pipe 7 is connected to one section, and the other end of this fluid injection pipe 7 is connected to the fluid amount control device 19 of the force feedback mechanism drive device 20 as shown in FIG.
(See FIG. 3).

Inside the tactile part 1, the bending amount detecting sensor 8 (FIG. 5) having the structure shown in FIG.
(Refer to FIG. 3) is arranged, and the optical fiber 11 is led out from each bending amount detection sensor 8 and coupled to the bending amount detection circuit 15.

In each section 5 of the tactile part 1, the outer skin 2
A beam member 5a having a large number of communication holes is provided between and the inner skin 3. The beam member 5a is made of a flexible material that is difficult to stretch, for example, a synthetic resin film in which Kevlar (registered trademark) fibers or carbon fibers are woven, and connects the outer skin 2 and the inner skin 3 to each other. .. This beam member 5a
Is bent and deformed when a force in the compression direction is applied from the outside of the compartment 5, and when the fluid is injected into the interior of the compartment 5, the distance between the outer skin 2 and the inner skin 3 is kept substantially constant and the compartment 5 is To prevent spherical expansion. Further, Kevlar (registered trademark) fibers or carbon fibers may be woven into the outer skin 2 and the inner skin 3 themselves.

When the data glove 21 is fitted in the hand and the finger is bent, the haptic part 1 bends. The bending amount of the haptic part 1 is detected by the bending amount detecting sensor 8 and supplied to the bending amount detecting circuit 15, which bends. The strength of the feedback force to be applied to the finger is calculated by the computer 16 according to the amount of bending of the finger detected by the quantity detection circuit 15, and the force feedback mechanism drive device 20 is calculated according to the strength of the feedback force. To control. That is, the fluid amount control device 19 (see FIG. 3) included in the force feedback mechanism drive device 20 is driven to control the amount of fluid injected into each section. A liquid or gas can be used as the fluid, silicone oil is used as the liquid, and an inert gas is used as the gas.

An image generation circuit 22 is connected to the computer 16 described above, and this image generation device 22 displays an image C showing an operator's hand on the display screen of the display 23 as shown in FIG. An image signal for displaying an image D of the ball to be grabbed is generated. The posture of the image C showing the hand is changed according to the bending amount of the finger detected by the bending amount detecting circuit 15. That is, as the amount of bending increases, the amount of bending of the displayed finger also increases.

Here, a case of performing an operation of gripping the ball with a hand will be described as an example.

The data glove 21 is fitted in the hand,
In the initial state in which the fingers are not bent, the compartment 5 is not filled with fluid and is in a substantially straight state along the posture of the fingers (not shown) as shown in FIG. 9A. The amount of bending of the finger at this time is detected by the amount of bending detection sensor 8 as described above, and the display 23 displays the image C with the hand open as shown in FIG.

Next, when the finger is bent, at this point, the section 5
Since no fluid pressure is applied to the haptic part 1, the haptic part 1 has flexibility, and the compartment 5 bends as shown by an arrow P in FIG. 9B according to the bending of the finger without receiving resistance. That is, no feedback force is given at this stage.

Next, when the finger touches the virtual ball, the calculator 16 causes the force feedback mechanism driving device 20 to move.
An injection command is sent to the compartment 5, and the fluid is injected into the compartment 5,
The interior is filled with fluid and the compartment 5 expands, making it impossible for the compartment 5 to bend further. Therefore, the feedback force is applied to the operator's finger in the direction indicated by the arrow Q in FIG. 9C, and the operator feels that the finger has touched something.
At this time, an image in which the finger of the hand touches the ball is displayed on the display screen of the display 23, so that the operator can recognize that the ball is grabbed by the hand.

Next, when the bending amount of the finger is reduced when the finger is opened, it is detected by the bending amount detection sensor 8, and when the finger is separated from the virtual ball, a discharge command is issued from the computer 16 to the force feedback mechanism driving device 20. Once delivered, the fluid in compartment 5 is drained and compartment 5 is allowed to bend freely. Therefore, when the finger is extended, the compartment 5 returns to the original state as shown in FIG.

In the above description, in order to facilitate understanding of the present invention, only the amount of bending of the finger of the hand is detected, but in order to detect the three-dimensional position of the hand, For example, as disclosed in the above-mentioned US Pat. No. 4,988,981, three ultrasonic wave receivers are provided along the outer periphery of the display screen of the display 23, and the data globe 21 is periodically arranged. An ultrasonic transducer that transmits ultrasonic waves is attached, the position of the hand is detected from the transmission time of the ultrasonic wave from the ultrasonic transducer to each ultrasonic receiver, and the display 23 is displayed according to the position of the hand.
It is also possible to change the position of the image of the hand displayed in, and calculate the distance between the ball and the finger.

By the way, in order to effectively apply the feedback force to the finger, it is necessary to prevent unnecessary expansion, extension or deformation of the compartment 5 when the fluid is injected into the compartment 5. Hereinafter, a structural example for preventing unnecessary deformation of each section will be described.

In FIG. 10, a sponge-like breathable porous elastic body 24 is provided in the compartment 5, and the peripheral surface of the porous elastic body 24 is adhered to the outer skin 2 and the inner skin 3 with an adhesive 25.
In the case of the example shown in FIG. 10, when the fluid is injected from the fluid injection pipe 7, the fluid permeates into the porous elastic body 24 and pressure is applied to the compartment 5, but the outer skin 2 and the inner skin 3 of the compartment 5 are porous elastic. Since they are connected by the body 24, the compartment 5 does not bulge into a spherical shape, and unnecessary deformation can be prevented.

Further, as shown in FIG. 11, the outer skin 2 and the inner skin 3 are connected by a reinforced fiber 26 such as carbon fiber or Kevlar (registered trademark) fiber, which is flexible but has a small amount of elongation, and each partition is similarly formed. It is possible to prevent unnecessary deformation.

By thus preventing unnecessary deformation of the compartments, the flexibility of each compartment can be controlled more accurately.

In the above embodiment, the case where the force feedback is performed according to the amount of bending of the finger has been described, but the present invention is not limited to this, and the posture of the human body can be changed. Each position, for example,
Force feedback can be performed by detecting the amount of bending at the joint position.

Further, the means for injecting the fluid into the compartment is not limited to the above-mentioned embodiment, and any means can be adopted as long as the injection amount can be controlled according to the control signal. Good.

[0044]

As described above, according to the present invention,
Since the force feedback is given to the hand or finger by the fluid pressure, the force feedback can be given in any direction at any position.

[Brief description of drawings]

FIG. 1 is an explanatory view showing in principle a haptic part of an artificial reality realization device according to the present invention.

FIG. 2 is a cross-sectional view showing a structure of a bending amount detection sensor.

FIG. 3 is an explanatory diagram showing in principle the control unit of the artificial reality realization device according to the present invention.

FIG. 4 is a flowchart for explaining the operation of the artificial reality implementation device.

FIG. 5 is a cross-sectional view showing an embodiment in which a haptic part is incorporated in a data glove.

FIG. 6 is a cross-sectional view showing a detailed structure of a haptic part.

FIG. 7 is a block diagram showing the overall configuration of the artificial reality realization device.

FIG. 8 is a schematic diagram showing a relationship between an image showing artificial reality displayed on a display and a data globe.

FIG. 9 is an explanatory diagram of an operation of a section.

FIG. 10 is a partial schematic cross-sectional view showing a configuration example for preventing unnecessary deformation of a section.

FIG. 11 is a partial schematic cross-sectional view showing another configuration example for preventing unnecessary deformation of a section.

[Explanation of symbols]

1 tactile part, 2 outer skin, 3 inner skin, 4 axial partition wall, 5
Sections, 5a Beam members, 6 Circumferential bulkheads, 7 Fluid injection pipes, 8 Bending amount sensors, 9 Sensor pipes, 10 Light emitting elements, 11 Optical fibers, 12 Light reflecting surfaces, 13 Light receiving elements, 14 Control section, 15 Bending amount detection Circuit, 16 Calculator, 17 Drive Circuit, 18 Electromagnetic Actuator, 19 Fluid Control Device, 19a Cylinder, 19b Piston, 19c Through Hole, 20 Force Feedback Mechanism Drive Device, 21 Data Globe, 22 Image Generation Circuit, 23 Display, 24
Porous elastic body, 25 adhesive, 26 reinforcing fiber, A
Image of finger, B fluid, C hand, D ball

Claims (1)

[Claims] 1. A flexible container to be arranged along a portion whose posture changes, a fluid injection means for injecting a fluid into the container, and a bending amount of the container arranged along the container. A bend amount detecting means for detecting, and a fluid amount control means for controlling the amount of the fluid injected from the fluid injecting means into the container based on the bend amount detected by the bend amount detecting means. An artificial reality realization device characterized by.