JP2002304246A - Tactile presenting device, and imaginary space system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To permit the tactile presentation to an operator under the environment of an imaginary space by fitting a lightweight and wearable device to a wrist or the like of the operator. SOLUTION: A tactile presenting means having a holding part, a supporting point part and a variable length connection part under the environment for expressing the imaginary space is fitted to the operator, and the tactility is given to the operator according to an imaginary tactile reception object in the imaginary space and/or the state of the imaginary object.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention comprises an image display terminal device for expressing a two-dimensional or three-dimensional virtual space in which an operator can freely operate (operate) on his / her own will, and arbitrarily displays images in the virtual space. Force sense presentation device and virtual space for presenting a force sense provided by virtual contact between a virtual object and a virtual force sense reception object displayed as an alter ego of the operator or a grasped object of the operator to the operator It is about the system.

[0002]

2. Description of the Related Art As a device for giving a force sense to an operator in a virtual environment, (i) a motor is attached to each joint axis of a manipulator, and the motor is driven to control a reaction force of each joint axis to present a force sense. Manipulator type (PHANTOM:
http://www.sensable.com/), (ii) Fit a ring with a thread on the operator's finger, attach the other end of the thread to a motor installed on the ceiling, floor, etc., and control the thread tension to control the force. SP that presents senses
IDAR (Sato, et al .: "Spatial Interface Device SP
Proposal of IDER ”, IEICE, 74, D-2, 7, 887/897 (199
1)) HapticGear (Hirose, et al .: Using Wire Tension), which allows the operator to hold a threaded pen, attach the other end of the thread to the motor on the back, and control the thread tension to present a force sensation Development of Wearable Force Display ",
Transactions of the Virtual Reality Society of Japan, Vol.3pp1-4, (19
98.8)).

[0003]

In the conventional apparatus, there are problems such as mounting load due to the weight of the equipment, and operational restrictions such as being unable to move freely because the movable range for presenting a force sense to the operator is narrow. .

According to the present invention, a lightweight, integrated wearable device including a driving part is attached to the operator's wrist or the like, so that the operator can freely move around and freely move his / her arm. The purpose is to enable the presentation of force sense to the operator with few restrictions.

[0005]

According to a first aspect of the present invention, a virtual force sense receiving object and a virtual object are expressed in a virtual space, and the virtual force sense receiving object and the virtual object are expressed in a virtual space. It is used in an environment that represents a virtual space in which the virtual space is controlled in accordance with the state of, and a gripper that is gripped by an operator and presents a sense of force, and a plurality of grippers attached to the operator from the gripper. It is possible to change the position of the grip portion with respect to the fulcrum by changing the length of the fulcrum composed of a plurality of fulcrums linearly stretched by wires and the length of the wire stretched from the grip to the fulcrum. Measuring means for measuring a length of the variable-length wire stretched from the gripping portion to the fulcrum portion, and A grip position calculating unit that calculates at least a position and an orientation of the grip with respect to the fulcrum according to the length of the wire measured by the unit; and detecting at least a position and an orientation of the force sense presentation unit. According to at least a position and an orientation calculated by the force sense presentation unit detection means and the grip position calculation unit or the force sense presentation unit detection unit, the grip unit of the force sense presentation unit is placed in the virtual space. A haptic receiving unit projecting means for projecting a virtual haptic receiving object and / or an operator or a part of the body of the operator holding the grip as the virtual haptic receiving object in the virtual space; Virtual contact calculation means for calculating a virtual contact state between the virtual force sense receiving object and the virtual object in accordance with a state of the virtual force reception object and the virtual object; According to the calculation result, the connection length calculation means for calculating the wire length of the variable length connection portion of the force sense presentation means, and from the wire length calculated by the connection portion calculation means, or to control the wire length. And a connecting portion control means that can be provided.

In a second aspect based on the first aspect, the force sense presenting means controls a thickness or a shape of the grip portion in accordance with a state of the virtual force sense reception object or the virtual object. It was configured to include a shape control unit.

In a third aspect based on the first or second aspect, the virtual object is switched to the virtual force sense receiving object or the virtual force sensed object is switched according to the state of the virtual force sense receiving object or the virtual object. The virtual haptic reception object switching means capable of switching the haptic reception object to the virtual object is provided.

[0008] A fourth invention is directed to a virtual space system corresponding to the first invention. The fifth invention is directed to a system in which virtual space systems are linked via a network.

In a sixth aspect, the force sense presentation means includes a grip shape control unit that controls a thickness or a shape of the grip portion according to a state of the virtual force sense reception object or the virtual object. Is used.

According to a seventh aspect of the present invention, the force sense presenting means includes:
A virtual haptic reception object switching unit that can switch the virtual object to the virtual haptic reception object or switch the virtual haptic reception object to a virtual object according to the state of the virtual haptic reception object or the virtual object. The one provided with

[0011]

Embodiments of the present invention will be described below with reference to the drawings.

[First Embodiment] FIG. 1 is a conceptual diagram of force sense presentation means of the present invention, and FIG. 2 is a conceptual diagram of a first embodiment of the present invention.

In the figure, reference numeral 1 denotes a gripping portion, 2 denotes a variable length connecting portion, 3 denotes a fulcrum portion, 10 denotes a force sense presentation means, 20 denotes an image display terminal, 30 denotes a virtual space, 40 denotes a virtual object, and 50 denotes a virtual object.
Represents a virtual force sense receiving object.

The haptic device according to the first embodiment is a virtual haptic receiving object as an alter ego of the operator or possessed by the operator in the three-dimensional virtual space 30 displayed on the image display terminal 20. 50 and virtual object 4
A force sense is presented by the force sense means 10 of the present invention in response to a virtual contact with zero.

FIG. 3 is a block diagram of the first embodiment of the present invention. In the figure, reference numeral 10 denotes a force sense presentation unit, 60 denotes a measurement unit, 61 denotes a grip part position calculation unit, 62 denotes a force sense presentation unit detection unit, 63 denotes a force sense reception unit projection unit, 64 denotes a virtual contact calculation unit, 65 Is a connection part calculation means, 66 is a connection part control means,
67 is a virtual space system, 68 is an image display terminal device, 6
Reference numeral 9 denotes virtual space control means.

The virtual space system 67 has an image display terminal device 68 for expressing a three-dimensional virtual space in which the operator can freely operate (operate) on his / her own will, and also has a virtual object 40 and a virtual force sense receiving object. A virtual space control means 69 for controlling (a video of) the virtual space in accordance with the state of 50.

The virtual object 40 is an object constituting a virtual space, and one or more virtual objects 40 can be generated in the virtual space 30. As the image display terminal device 68, a standard-sized or life-size display, an image projected on an HMD or the like can be used, and a monocular or multi-view (stereoscopic) display can also be used.

As shown in FIG. 1, the force sensation presentation means 10 is provided with a grip portion 1 which is gripped by an operator and presents a force sensation, and is attached at a position near the wrist of the operator's arm, and is provided with four points from both ends of the grip portion 1. A fulcrum 3 composed of four fulcrums linearly stretched by a wire or the like, and a variable-length connecting part 2 capable of changing the length of the wire stretched from the grip 1 to the fulcrum 3
And By controlling the wire length and changing the position of the grip 1 with respect to the fulcrum 3 and changing the position of the arm to which it is attached and the position of the wrist being gripped, the force against bending of the wrist is presented. .

As the variable length connecting portion 2, four wires, each of which is linearly stretched from the grip portion 1 gripped by hand to the fulcrum portion 3 attached to the arm, are attached to the fulcrum portion 3 and the like. It is conceivable to change the wire length by pulling with a motor.

FIG. 4 shows an embodiment of the haptic device shown in FIG. Reference numeral 4 in the drawing denotes a wire,
5 is an encoder, 6 is a pulley, and 7 is a motor.

If the length of the wire changes, the length from the arm to which the fulcrum 3 is attached to the wrist and the length of the hand or wrist that is being gripped do not change. When the wire is pulled, a force is applied which causes the hand to fall left or right with respect to the palm (see FIG. 5), and when the two wires on the palm side are pulled, the hand falls forward with respect to the arm. Force will be applied (see FIG. 6).

The grip 1 may be held in parallel with the wrist. The grip 1 may be tilted with respect to the wrist, the thumb may be raised, and the finger may be moved toward the little finger. (See FIG. 7).

The measuring means 60 measures the length of the variable-length wire 4 stretched from the grip 1 to the fulcrum 3. When the wire 4 is made to have a variable length by being pulled by the motor 7 attached to the fulcrum 3, the wire 4 is wound around the pulley 6 attached to the motor 7, and the rotation angle of the pulley 6 is counted by the encoder 5. A method of calculating the length of the changed wire can be considered.

The grip position calculating means 61 calculates the position and orientation of the grip 1 with respect to the fulcrum 3 in accordance with the wire length measured by the measuring means 60.

An equation for calculating the position of the gripper 1 based on the wire length will be described with reference to FIG. As shown in FIG. 8, when holding the grip 1 with the palm facing forward and the wrist extended, the basic posture is x. ), The direction from the front to the back is y, and the direction from the bottom to the top is z, and the four coordinates that are the fulcrum where the fulcrum 3 and the variable-length connecting part 2 are connected are Pa = (xa, ya, za). Pb = (xb, yb, zb) Pc = (xc, yc, zc) Pd = (xd, yd, zd), and the length of the grip portion is set to s. At this time, positions P = (x, y, z) and PP = (xx, yy, zz) at both ends of the gripper 1 of the force sense presentation means 10, and the length la of the variable length connecting part 2 connected to the gripper 1. , L
The following equation (1) holds for b, lc, and ld.

[0026]

(Equation 1)

Also, since the hand rotates around the center of the wrist (see FIG. 9), if the gripper 1 is not re-held by hand, both ends of the gripper 1 will have their rotation center points P ₀ = (x ₀ , In y ₀ , z ₀ ), the rotation is performed with radii r _p and r _pp , and the following expression (2) holds.

[0028]

(Equation 2)

By solving the equations (1) and (2), the positions P = (x, y, z) and PP = (xx, yy, zz) at both ends of the gripper 1 are determined by the length of the variable-length connector 2. It can be represented and calculated by constants of la, lb, lc, ld and the fulcrum coordinates.

For example, if the four fulcrums are Pa = (0,0,0) Pb = (0, m, 0) Pc = (n, 0,0) Pd = (n, m, 0) (1) (2) from the _{equation, P i = (x i,} y i, z i) PP i = (xx i, yy i, zz i) _{is, la i, lb i, lc} i, following from ld _i Can be expressed as

[0031]

(Equation 3)

The radius of gyration r used in the above equation_p, R
_pp, Rotation center point P₀= (X₀, Y₀, Z₀) Is how many
Calibration in the form of a hand
It can be calculated by calculating the position. First,
When attaching the force sense presentation means 10, a fulcrum on the x-axis
The wrist rotation center point x is located at the center of Pa and Pc. _o
Attach it to your wrist so that it matches. X_oIs
It is represented as below. Note that the mounting position is known.
If it is, it does not have to be the center.

X _o = n / 2 (4) Next, since the rotation center point P ₀ is located at the center of the wrist, FIG.
The wrist is stretched as shown at _0, and the hand is shaped so that the y coordinate of P at one end of the gripper 1 coincides with the rotation center point y0 of the wrist. This is referred to as calibration # 1, and coordinates of both ends of the gripper 1 at this time are represented by P ₁ = (x ₁ , y ₁ , z ₁ ), PP
₁ = (xx ₁ , yy ₁ , zz ₁ ). The wire length la ₁ of the variable length connection part 2 of the calibration # 1
From lb ₁ , lc ₁ , and ld ₁ , y ₁ , yy can be obtained from equation (1).
_{Since 1} can be calculated and y ₀ = y ₁ , y ₀ is expressed as follows.

[0034]

(Equation 4)

[0035] Next, the x coordinate from the state of calibration # 1 to defeat the 90-degree wrist remains of x ₁ (x ₂ =
x _1), the grip portion 1 at one end z coordinate wrist rotation center point z ₀ of P
Take a hand shape that matches. This is referred to as calibration # 2, and is represented by coordinates P ₂ = (x ₂ , y ₂ , z ₂ ). In this case z ₀ = z _2, and becomes the following relation.

Z₁-Z_Two= Y₁-Y_Two (6) Wire of variable length connection part 2 in calibration # 2
Long la_Two, Lb_TwoFrom equation (3), y_TwoCan be calculated
You. Also P₁, P_TwoIn equation (1)₁, Y _TwoFor
From the relation of equation (6)₁, Z_TwoCan be calculated. This
More than z₀= Z_Two, Z₀Is represented by
You.

[0037]

(Equation 5)

From the above, the wrist rotation center point P ₀ = (x ₀ ,
y ₀ , z ₀ ) are known, and x ₁ , x ₀ is calculated using equation (1).
Rotation radius r _p of the grip portion 1 at one end P from equation (2) is represented by the following since ₂ can be calculated.

[0039]

(Equation 6)

X ₁ , y ₁ , z ₁ , yy obtained earlier
_Since xx ₁ and zz ₁ can be calculated by substituting ₁ into Expression (1), the rotation radius r _pp of one end PP of the gripper ₁ is similarly expressed by Expression (2) from Expression (2).

[0041]

(Equation 7)

As another calibration method, similarly, the coordinates of the gripper 1 are set to P ₁ = (x ₁ , y ₁ , z ₁ ) and PP ₁ = (xx ₁ , yy) using the hand shape as a calibration.
₁ , zz ₁ ), and the wrist is tilted with x ₁ , xx ₁ from the basic form with respect to the x coordinate of the gripper 1, that is, coordinates P ₂ = (x ₂ , y ₂ , z ₂ ), PP ₂ = ( xx ₂ , yy
₍₂ , zz ₂ ) can be used as the calibration # 2 to calculate (y ₀ , z ₀ ) and the like using the equations (1) and (2).

Further, in order to obtain the rotation origin x ₀ on the x-axis by calibration, the coordinates of the gripper 1 are set as P ₃ = (x ₃ , y ₃ , z ₃ ) and PP ₃ = (xx ₃ , yy
₃ , zz ₃ ), a point rotating in the x-axis direction is measured so that x ₃ ≠ x ₁ .

From equations (1) and (2), x_Three ^Two+ Y_Three ^Two+ Z_Three ^Two= La_Three ^Two x_Three ^Two+ (Y_Three-M)^Two+ Z_Three ^Two= Lb_Three ^Two (Xx_Three-N)^Two+ Yy_Three ^Two+ Zz_Three ^Two= Lc_Three ^Two (Xx_Three-N)^Two+ (Yy_Three-M)^Two+ Zz_Three ^Two= Ld_Three ^Two (X_Three-Xx_Three)^Two+ (Yy_Three-Y_Three)^Two+ (Z_Three-Zz_Three)^Two= S^Two (X_Three-X₀)^Two+ (Y_Three-Y₀)^Two+ (Z_Three-Z₀)^Two= (X₁-X₀)^Two + (Y₁-Y₀)^Two+ (Z₁-Z₀)^Two (Xx_Three-X₀)^Two+ (Yy_Three-Y₀)^Two+ (Zz_Three-Z₀)^Two= (Xx₁−x₀)^Two+ (Yy₁-Y₀)^Two+ (Zz₁-Z₀)^Two By taking the above seven equations, the variable x_Three, Y_Three, Z_Three.
xx_Three, Yy_Three, Zz _Three, X₀Are obtained, and
This gives x₀Is calculated.

The force sense presentation part detection means 62 detects the position and orientation of the gripping part 1 or the fulcrum part 3 of the force sense presentation means 10 with respect to the entire space. Examples of the force sense presentation unit detection unit 62 include a detection unit using a magnetic sensor and infrared rays, and a detection unit using video analysis by marking. Further, by using the position information and the direction information detected by the force sense presentation autodetection means 62, the moving speed, the movement direction, the acceleration, and the like can be calculated and used as the state value of the force sense presentation means 10.

The force sense receiving unit projection means 63 is a grip position calculation means 61 for calculating the position and orientation of the grip 1 with respect to the fulcrum 3 of the force sense presenting means 10, or a force sense presenting means for the entire space. The force sense presentation unit detection means 62 for detecting the position and orientation of the grip unit 1 or the fulcrum unit 3
The gripper 1 of the force sense presentation unit 10 for the entire space is calculated from the position and orientation calculated by
The gripper 1 of the force sense presentation means 10 or the gripper 1
The operator or a part of the body of the operator holding the object is projected on the virtual space 30 as the virtual force sense receiving object 50. When the fulcrum 3 of the force sense presentation means 10 is attached near the wrist of the arm and the grip 1 is gripped by hand, the grip 1 or the operator or the hand gripped by the operator is in the virtual space 30. As a virtual force sense receiving object 50.

By calculating the virtual contact between the virtual object 40 in the virtual space 30 and the virtual force sense receiving object 50, a contact with the operator or the operator's hand in the virtual space 30 can be presented. Also, by projecting the gripper 1 as a virtual force sense receiving object 50, when the operator's hand is gripping the virtual object even in the virtual space 30, such as when the operator has a pointing stick or a racket, By calculating virtual contact with a grasped object and presenting a force sense, a force sense may be indirectly presented to the operator.

The virtual contact calculation means 64 calculates a virtual contact state between the virtual force sense receiving object 50 and the virtual object 40 in the virtual space 30 according to the state of the virtual force sense receiving object 50 and the virtual object 40 in the virtual space 30. That is, a state such as whether a contact has occurred within the virtual contact, whether there is a possibility of contact from each position or movement trajectory, how the contact is made, and the like are calculated.

The connecting part calculating means 65 calculates the wire length of the variable length connecting part 2 of the force sense presenting means 10 according to the calculation result of the virtual contact calculating means 64. For example, force sense presentation means 1
When the fulcrum 3 is attached near the wrist of the arm and the grip 1 is gripped by hand, the two wires on the back of the hand can be pulled to present contact with the palm or the little finger. By pulling two wires, a contact on the thumb side can be presented. At this time, it is necessary to draw other wires at least so as not to sag. The length or tension of each wire for that purpose is calculated.

The connecting part control means 66 is provided with the connecting part calculating means 6.
From the wire length calculated by step 5, the force sense presentation means 10
The wire length or wire tension of the wire, which is the variable-length connecting portion 2 of, is controlled. That is, the wire is drawn so as to have the calculated wire length and tension. Further, at this time, a pulling force from the grasping hand is generated, so that the presented wire length and tension are sequentially compared with the current state in order to approach or maintain the length and tension value of the wire to be presented, and feedback is performed. Take control.

Next, the processing flow of the haptic device according to the present embodiment will be described in detail. FIG. 11 is a diagram for explaining the flow of the processing.

Step (S101): First, the fulcrum of the force sense presentation means is attached near the wrist of the operator's arm, the grip is gripped by the hand on the same side, and the wire can be freely moved but not loosened. To the extent, always keep it in tension at low tension.

Next, the wire length in the form of several hands is detected in order to hold the grip and perform calibration. On the basis of the wire length at this time, an initial value necessary for the subsequent calculation of the gripping portion position is obtained.

Step (S102): Also, the wire length is always measured by the measuring means, and the position of the gripping part with respect to the fulcrum is calculated by the gripping part position calculating means from the initial value obtained first.

Step (S103): Next, the position and orientation of the grip portion or the fulcrum portion of the force sense presentation unit with respect to the entire space are detected by the force sense presentation unit detection unit. Further, the grip part position calculation means for calculating the position and orientation of the grip part with respect to the fulcrum part of the force sense presentation means, or the position and orientation of the grip part or the fulcrum part of the force sense presentation means with respect to the entire space are detected. Depending on the position and orientation calculated by the force sense presentation unit detection means, the grip of the force sense presentation means, or the operator holding the grip or a part of the body of the operator, in the virtual space. The projection is performed as the virtual force sense receiving object.

Step (S104): Next, a virtual object is generated by the virtual space system and expressed in the virtual space.

Step (S105): Next, the virtual contact calculation means calculates a virtual contact state between the virtual force sense receiving object and the virtual object in the virtual space according to the state of the virtual force sense receiving object and the virtual object. In other words, depending on the position or the trajectory of the virtual object in the virtual space and the position or orientation of the gripping portion or the hand gripped by the operator, the virtual contact calculation means It calculates whether virtual contact with the virtual force sense receiving object has occurred.

Step (S106): It is checked whether or not there is a virtual contact. Step (S107): As a result of this calculation, when a virtual contact has occurred, the connecting part calculating means calculates the wire length or tension of the variable length connecting part of the force sense presenting means according to the calculation result of the virtual contact calculating means. Is calculated.

Step (S108): The connecting part control means outputs the wire length and the tension which are the variable length connecting parts of the force sense presentation means.

Step (S109): In order to approach or maintain the length and tension of the wire to be presented,
Feedback control is performed by sequentially comparing the presented wire length and tension with the current status.

Step (S110): Further, the virtual space is controlled by the virtual space control means according to the state of the virtual object or the projected virtual force sense receiving object when the wire is controlled by the connection control means. .

Step (S111): Further, if the virtual space system causes the operator to generate a contact event with the virtual force sense receiving object again according to the represented virtual space, these are repeated again.

As a specific method of using the haptic device described above, the hand of the operator holding the grip 1 is projected as a virtual haptic reception object 50 as shown in FIG. It is conceivable that the touch event is received by the operator's hand in the virtual space (or similar one). In addition, by presenting inertia to the movement of an object, such as moving a stationary object, stopping a moving object, or resisting when moving, it can present a force sense when operating an object, or Presentation of a force sense such as a collision can be considered, for example, movement or operation of an object, opening and closing of a shoji (see FIG. 13), and the like.

As shown in FIG. 14, it is assumed that the user is holding the object, and the user's body is not projected as a virtual force sense receiving object, but is held by the operator. It is conceivable to project 1 as a virtual force sense receiving object, and to receive a contact event by the virtual object with an object held by the operator in the virtual space. For example, a force sense such as a collision with an object is presented through a gripping rod such as tennis or golf (see FIG. 15), a pointing stick, a cooking utensil, etc. Mixing, cutting, scooping, etc. are conceivable.

The specific processing flow is as follows. First, the fulcrum 3 of the force sense presentation means 10 is attached near the wrist of the operator's arm, and the grip 1 is gripped by the hand on the same side.
Always keep it under tension. The wire length is always measured by the measuring means 60, and the position of the grip 1 with respect to the fulcrum 3 is calculated by the grip position calculating means 61 from the initial values obtained from some hand shapes first.

Further, the fulcrum presentation unit detecting means 62 detects the fulcrum 3 of the haptic presentation means 10 in which a magnetic sensor or the like is attached to the arm near the wrist of the operator, or the operator's arm from which the fulcrum 3 can be calculated. The position in the virtual space 30 is detected by attaching to the gripper 1 of the force sense presentation means 10 held by the operator. Since the positions of the grip 1 and the fulcrum 3 of the haptic presentation unit 10 are detected from the value obtained by the grip unit position calculation unit 61 and the value obtained by the haptic presentation unit detection unit 62, The position of the arm near the wrist of the user and the position of the hand of the gripping operator are detected. Note that the position of this hand is equivalent to the position of the grip 1, but the size of the hand taken in advance and the grip 1
It is also possible to calculate the position of the hand from the way of holding and use it as the position of the hand.

Based on the calculated value of the gripper 1, the gripper 1 of the force sense presentation means 10, the operator holding the gripper 1, or a part of the operator's body is moved to the virtual space 30. To the virtual force sense receiving object 50. In the case of opening / closing a shoji by an operator's hand in a virtual space, the force sense receiving unit projecting means 63 projects the operator's hand as a virtual hand as the virtual force sense receiving object 50, or operates with a gripping object such as tennis. In the case of, the grip 1 held by the operator as the virtual force sense receiving object 50 is
Project as a virtual tennis racket.

In the virtual space 30, a virtual object 40 such as a shoji or a ball is generated. Virtual contact calculation means 64
Calculates the state of the virtual force sense receiving object 50 and the virtual object 40. For example, a state is calculated in which a virtual hand or a virtual racket, which is the virtual force sense receiving object 50, contacts a shoji or a ball in the virtual space, which is the virtual object 40.

When the hand of the operator in the virtual space comes into contact with the virtual shoji or moves in a direction in which the operator enters the virtual shoji, the hand is affected by inertia force when moving the object or friction between the ground and the object. Virtual contact calculation means 6 to present force
Reference numeral 4 denotes a virtual haptic reception object 5 as a contact state between the virtual haptic reception object 50 and the virtual object 40.
It calculates the biting width of the zero virtual object 40 and its speed.

From this, the connection unit calculating means 65 prevents the virtual force sense receiving object 50 from digging into the virtual object 40 or presents the impact force at that time.
The pulling length or the pulling force of the variable length connecting portion 2 of the force sense presenting means 10 is calculated, and the connecting portion control means 66 sets the wire length and the tension of the variable length connecting portion 2 of the force sense presenting means 10 to be equal. Draw the wire. For example, in the case of contact on the palm side, two wires on the back side of the hand are pulled. Since the two wires on the palm side are stretched with a weak tension that does not loosen and can be moved freely,
The wire on the palm side is stretched by being pulled on the back side.

Further, if it is necessary to present force sense again according to the states of the virtual force sense receiving object 50 and the virtual object 40, feedback is performed. In other words, when the wire length to be presented does not become the desired length due to the operator's own force at the time of presentation, or when the wire length is too long, or when the force is maintained, the presentation is necessary again. This ends the repetition when the difference between the value to be presented and the actual value falls within a certain range, or when the specified number of repetitions or when the contact is stopped.

Further, in the virtual space 30, the virtual force control object 69 causes the virtual force sense receiving object 50 such as a virtual hand or a virtual racket and the virtual object 40 such as a shoji or a ball in the virtual space to be displayed. Is expressed according to the state of

Further, if a contact event occurs again, for example, while a virtual shoji is being opened with a virtual hand,
Execute the process again.

[Second Embodiment] FIG. 16 shows a second embodiment of the present invention.
It is a lineblock diagram of an embodiment. The reference numerals in FIG.
And a remote virtual space reflection unit 70 and a communication network 71 are added.

Here, two force sense presentation devices shown in FIG. 3 are provided, and the virtual space system of each force sense presentation device is connected via a communication network. Each haptic device is
A force sensation is presented to both operators so as to virtually exchange a reaction force bidirectionally between the operators via the communication network 71. In this case as well, each haptic device can display the alter ego's alter ego, and the operator can operate his alter ego to express a three-dimensional virtual space in which he can freely move (operate) with his own will. I do. In this virtual space,
The alter ego of the operator who operates another force sense presentation device is also expressed.

The virtual space control means 69 of the virtual space system
Controls the virtual space in accordance with the state of the virtual force sense receiving object projected by the force sense receiving unit projecting means 63 and the generated virtual object, as described with reference to FIG. The remote virtual space reflecting unit 70 shown in FIG. 16 converts the position information and orientation information of the virtual object or the virtual force reception object expressed by the virtual space control unit 69 into the position information and orientation of the virtual object of another virtual system. The information is transmitted to another virtual space system via the communication network 71 as information, receives the position information and orientation information of the virtual object sent from the other virtual system, and expresses the virtual object in its own virtual space.

As a specific method of use, exchange of reaction force on the network is conceivable, and tennis (see FIG. 17) via the network, fighting with kendo (see FIG. 18), and the like are conceivable.

Regarding both operators of the virtual space system via the network 71, the grip of the force sense presentation means,
By projecting the hand of the operator holding the grip as a virtual force sense receiving object and sending the position information, orientation information, and the like of the virtual object to another virtual system via a network, or By sending and receiving the receiving object as a virtual object, the operator receives a contact event from the virtual object by the virtual force sense receiving object and exchanges a reaction force.

For example, in the case of tennis, a grip portion held by an operator is projected as a virtual force sense receiving object onto a virtual tennis racket held by an alter ego of the operator in a virtual space, and the virtual object is virtualized. It is conceivable that the reaction is exchanged by sending the virtual ball position information, the direction information, and the like, and hitting the virtual ball with the alter ego of the operator of another virtual space system.

In the case of a game such as kendo, the grip portion held by the operator is projected as a virtual force sense receiving object onto a virtual sword held by the alter ego of the operator in the virtual space. The sword's position information, orientation information, etc. are sent to another virtual space system as a virtual object, and the alter ego of the operator in another virtual space sent as a virtual object from another virtual space system is holding it. It is conceivable that a virtual virtual sword is expressed and a contact event occurs.

FIG. 19 shows a specific processing flow. Steps S101 ′ to S111 ′ in FIG. 19 correspond to FIG. 16, and steps S112 and S113 are newly added.

First, the fulcrum of the force sense presentation means is attached near the wrist of the operator's arm, and the grip is gripped by the hand on the same side.
Always keep it under tension. The wire length is always measured by the measuring means, and an initial value is first obtained from some hand shapes (S101 '). Then, the position of the gripper relative to the fulcrum is calculated by the gripper position calculator from the initial value (S102 '). In addition, by the force sense presentation unit detection means, a magnetic sensor or the like is attached to the fulcrum of the force sense presentation means attached to the arm near the wrist of the operator, or at the position of the arm or the like of the operator whose fulcrum can be calculated, The position in the virtual space is detected by being attached to the grip portion of the force sense presentation means gripped by the operator. Since the positions of the gripper and the fulcrum of the force sense presentation unit are detected from the value obtained by the grip position calculation unit and the value obtained by the force sense presentation unit detection unit, the arm near the wrist of the operator is detected. And the position of the gripping operator's hand are detected. Although the position of this hand is equivalent to the position of the grip, the position of the hand can be calculated from the size of the hand taken in advance and the way of holding the grip, and used as the position of the hand.

Based on the calculated value of the gripping portion, the gripping portion of the haptic presentation means, the operator holding the gripping portion, or a part of the operator's body is placed in the virtual space in the virtual force sensation. Project as a receiving object (S103 ').
In the case of tennis, a meeting, or the like, the grip portion held by the operator as a virtual force sense receiving object is projected as a virtual racket or a virtual sword. In the virtual space, a virtual object such as a ball is generated. In the case of a meeting, a virtual sword that is a virtual force sense receiving object in another virtual space operated by another person is generated as a virtual object (S104 ′).

The virtual contact calculation means calculates the state of the virtual force sense receiving object and the virtual object. For example, a state is calculated in which the virtual racket, which is the virtual force sense receiving object, contacts the ball in the virtual space, which is the virtual object (S105 '). Then, it is checked whether a virtual function has occurred (S106 ').

Thus, the connecting portion calculating means determines whether the virtual force sense receiving object does not bite into the virtual object or presents the impact force at that time, so that the length of the variable length connecting portion of the force sense presenting means is reduced, or The pulling force or the like is calculated (S107 '), and the connecting part control means draws the wire so that the wire length and the tension of the variable length connecting part of the force sense presenting means become the same (S108').

Further, if it is necessary to present force sense again according to the state of the virtual force sense receiving object and the virtual object, feedback is performed. In other words, when the wire length to be presented does not become the desired length due to the operator's own force at the time of presentation, or when the wire length is too long, or when the force is maintained, the presentation is necessary again. This ends the repetition when the difference between the value to be presented and the actual value is within a certain range, or when the repetition is performed a specified number of times, or when there is no more contact (S109 ').

Further, in the virtual space, a virtual force sense receiving object such as a virtual racket and a virtual sword and a virtual object such as a ball and a sword operated by another person in the virtual space are controlled by the virtual space control means. It is represented according to those states (S110 ').

The position information and orientation information of a virtual object such as a virtual ball are transmitted to another virtual space system via a network by remote virtual space reflecting means. Alternatively, the position information and the orientation information of the virtual force sense receiving object such as the virtual sword are transmitted to another virtual space system as a virtual object (S112).

The position and orientation of a virtual object such as a virtual ball or a virtual sword transmitted from a remote place are reflected in the virtual space by the remote virtual space reflecting means (S113).
Further, if there is a contact event again, the process is executed again (S111 ′).

[Third Embodiment] FIG. 20 shows a third embodiment of the present invention.
It is an example of a block diagram of embodiment. The reference numerals in FIG. 20 correspond to those in FIG. 3, and a grip shape control unit 72 is added.

The grip shape control section 72 adjusts the thickness or shape of the grip section of the grip section held by the operator of the haptic presentation means according to the state of the virtual force sense reception object or the virtual object in the virtual space. Control.

As a specific method of changing the thickness, for example, as shown in FIG. 21, the cylinder of the gripping portion made of a material such as shape memory is expanded so that the gripping portion becomes thicker, or as shown in FIG. By controlling the air pressure, such as by attaching a spring so that the size of the tube changes according to the size of the tube, and the tube that expands by air pressure in the tube, or by applying mechanical force such as a motor, the inside of the tube It is conceivable to change the thickness and shape by spreading out from the outside. In the case of FIG. 23, the outer peripheral surface of the grip portion is divided into, for example, three, and a bulge is provided corresponding to three disassembly boundaries. The inflatable body is controlled so as to expand and contract so as to be able to expand and contract by, for example, a motor means (not shown) so as to divide the disassembly boundary as indicated by an arrow in the figure. This changes the thickness of the grip.

As a usage method, for example, a haptic reception object that projects the grip portion of the haptic presentation means in a virtual space, or a virtual object with which the haptic reception object that projects an operator's hand touches, is used. It is conceivable to change the thickness and shape according to the size and shape.

[Fourth Embodiment] FIG. 24 shows a fourth embodiment of the present invention.
It is an example of a block diagram of embodiment. Reference numerals in FIG. 24 correspond to those in FIG. 3, and a virtual force sense receiving object switching means 73 is added.

Virtual force sense receiving object switching means 73
Can switch the virtual object to the virtual haptic receiving object or the virtual haptic receiving object to the virtual object according to the state of the virtual haptic receiving object or the virtual object. By being able to switch between the virtual force sense receiving object and the virtual object, the projection of the virtual hand of the operator onto the virtual force sense receiving object can be changed to a grip portion held by the operator with a virtual pointing rod. When switching to projection or when projecting the gripping part held by the operator, various virtual objects may be sequentially switched to virtual force sense receiving objects.

As a specific usage method, by changing what the operator is holding, it is projected on various cooking utensils as shown in FIG. 25, for example, and a virtual kitchen knife, virtual whisk, virtual Virtual cooking that presents a sense of force by operating each object such as a spoon is conceivable.

[0097]

As described above, according to the present invention, the operator can present a sense of force to the operator while restricting his or her movement and freely moving around and freely moving his arm.

[Brief description of the drawings]

FIG. 1 is a conceptual diagram of force sense presentation means of the present invention.

FIG. 2 is a conceptual diagram of the first embodiment of the present invention.

FIG. 3 is a configuration diagram of a first embodiment of the present invention.

FIG. 4 is an embodiment of a haptic device.

FIG. 5 is an operation example 1 of the haptic device.

FIG. 6 is an operation example 2 of the haptic device.

FIG. 7 is an example of gripping the force sense device.

FIG. 8 is a diagram showing grip position coordinates.

FIG. 9 is a diagram illustrating a rotation center point of the gripping unit.

FIG. 10 is a diagram illustrating a state of calibration.

FIG. 11 is a flowchart of a process according to the first embodiment of the present invention.

FIG. 12 is a specific usage method 1 of the first embodiment.

FIG. 13 is a usage example 1 of the first embodiment.

FIG. 14 is a usage method 2 of the first embodiment.

FIG. 15 is a usage example 2 of the first embodiment.

FIG. 16 is a configuration diagram of a second embodiment of the present invention.

FIG. 17 is a usage example 1 of the second embodiment.

FIG. 18 is a usage example 2 of the second embodiment.

FIG. 19 is a flowchart of a process according to the second embodiment of the present invention.

FIG. 20 is a configuration diagram of a third embodiment of the present invention.

FIG. 21 is a method example 1 of the third embodiment of the present invention.

FIG. 22 is a second example of the method according to the third embodiment of the present invention.

FIG. 23 is a third method example of the third embodiment of the present invention.

FIG. 24 is a configuration diagram of a fourth embodiment of the present invention.

FIG. 25 shows a method of using the fourth embodiment.

[Explanation of symbols]

 1: grip part 2: variable length connecting part 3: fulcrum part 10: force sense presentation means 20: image display terminal 30: virtual space 40: virtual object 50: virtual force sense reception object 60: measuring means 61: grip part position calculation Means 62: Force sense presentation part detection means 63: Force sense reception part projection means 64: Virtual contact calculation means 65: Connection part calculation means 66: Connection part control means 67: Virtual space system 68: Image display terminal device 69: Virtual space Control means 70: Remote virtual space reflection means 71: Communication network 72: Holding shape control unit 73: Virtual force sense reception object switching means

 ────────────────────────────────────────────────── ─── Continuing on the front page (72) Inventor Satoshi Ishibashi 2-3-1, Otemachi, Chiyoda-ku, Tokyo F-term in Nippon Telegraph and Telephone Corporation (reference) 2C001 BA00 BC00 BC10 BD07 CA00 CA09 5E501 AC15 AC16 BA03 CA02 CA10 CC14 FA15 FA27 FA36

Claims (7)

[Claims] 1. A virtual space in which a virtual force sense receiving object and a virtual object are expressed in a virtual space, and the virtual space is controlled according to a state of the virtual force sense receiving object and the virtual object. And a fulcrum portion that is held by the operator and presents a sense of force, and a fulcrum portion that is attached to the operator and includes a plurality of fulcrums linearly stretched by a plurality of wires from the gripping portion. A force sense presentation unit having a variable length connection unit that can change the length of the wire stretched from the grip unit to the fulcrum unit and change the position of the grip unit with respect to the fulcrum unit; Measuring means for measuring the length of the variable-length wire stretched from the grip portion to the fulcrum, and according to the length of the wire measured by the measuring means,
Grip part position calculating means for calculating at least the position and direction of the grip part with respect to the fulcrum part; force sense presenting part detecting means for detecting at least the position and direction of the force sense presenting means; and grip part position calculation The gripping portion of the haptic presentation unit in the virtual space as the virtual haptic reception object, and / or the gripping unit in accordance with at least the position and the orientation calculated by the haptic presentation unit detecting unit. A force sense receiving unit projecting means for projecting the operator or a part of the body of the operator holding the object as the virtual force sense receiving object in the virtual space, and the virtual force sense receiving object and the virtual object Virtual contact calculating means for calculating a virtual contact state between the two in accordance with the state; and A connecting portion calculating means for calculating the wire length of the variable length connecting portion, and from the wire length calculated by the connecting portion calculating means,
Or a connection control means capable of controlling a wire length. 2. The force sense presentation means according to claim 1,
A haptic device, comprising: a grip shape control unit configured to control a thickness or a shape of the grip unit according to a state of the virtual haptic reception object or the virtual object. 3. The virtual haptic reception object according to claim 1, wherein the virtual haptic reception object is switched to the virtual haptic reception object or the virtual haptic reception object is switched to a virtual object according to a state of the virtual haptic reception object or the virtual object. A haptic device, comprising: a virtual haptic reception object switching unit capable of switching to a haptic device. 4. An image display terminal device for representing a virtual force sense receiving object and a virtual object in a virtual space, and a virtual device controlling the virtual space according to a state of the virtual force sense receiving object and the virtual object. In a virtual space system having space control means, a fulcrum comprising a gripper gripped by an operator and presenting a sense of force, and a plurality of fulcrums attached to the operator and linearly stretched by a plurality of wires from the gripper. A force sense presentation unit having a unit and a variable length connection unit that can change the length of the wire stretched from the grip unit to the fulcrum unit and change the position of the grip unit with respect to the fulcrum unit; Measuring means for measuring the length of the variable-length wire stretched from the grip portion to the fulcrum, and according to the length of the wire measured by the measuring means,
Grip part position calculating means for calculating at least the position and direction of the grip part with respect to the fulcrum part; force sense presenting part detecting means for detecting at least the position and direction of the force sense presenting means; and grip part position calculation The gripping part of the haptic presentation unit in the virtual space as the virtual haptic reception object, and / or the gripping unit, in accordance with at least the position and the orientation calculated by the haptic presentation unit detecting means. The operator or a part of the operator's body
As the virtual force receiving object in the virtual space,
Haptic receiving unit projecting means for projecting, Virtual contact calculating means for calculating a virtual contact state between the virtual haptic receiving object and the virtual object according to a state of the virtual object, and calculation by the virtual contact calculating means According to the result, a connection part calculation means for calculating the wire length of the variable length connection part of the force sense presentation means, From the wire length calculated by the connection part calculation means,
Or a connection part control means capable of controlling a wire length, a virtual space system comprising: 5. The virtual object according to claim 4, wherein said virtual space system further comprises remote virtual space reflection means, wherein said remote virtual space reflection means is transmitted from another virtual space system via a communication network. In the virtual space, and transmitting the virtual haptic reception object and / or the virtual object to the other virtual space system as a virtual object via the communication network. 6. The haptic presentation unit includes a grip shape control unit that controls a thickness or a shape of the grip unit according to a state of the virtual haptic reception object or the virtual object. The virtual space system according to claim 4, wherein 7. The haptic presentation means switches the virtual object to the virtual haptic reception object or switches the virtual haptic reception object to a virtual object according to the state of the virtual haptic reception object or the virtual object. 7. The virtual space system according to claim 4, further comprising a virtual force sense receiving object switching unit capable of switching to a virtual force sense receiving object.