JP7120568B2 - Information processing equipment - Google Patents

Description

The present invention relates generally to virtual reality environment generators and controller devices that utilize illusion and sensory properties.

More specifically, the present invention provides a man-machine interface mounted on devices used in the field of VR (Virtual Reality), devices used in the field of games, mobile phones, PDAs (Personal Digital Assistants), and the like. The present invention relates to an illusionary tactile force sense interface device, an illusionary tactile force sense information presentation method, and a virtual reality environment generation device for.

As a conventional haptic interface device in VR, a haptic device in contact with a human sensory organ and a haptic interface device main unit are connected by a wire or an arm in the haptic presentation of tension or reaction force. There is (Non-Patent Document 1). In addition, as a non-grounded, non-base type haptic interface device that does not have a base in the body, by independently controlling the rotation of three flywheels arranged in a three-axis orthogonal coordinate system, it can be used in any direction or in any direction. A non-base type haptic interface device that can present torque in magnitude has been proposed (Non-Patent Document 2). In addition, in a non-based man-machine interface that gives people the presence of virtual objects and reaction force, sensations that cannot be presented only by the physical characteristics of the haptic interface device, such as haptic sensations such as torque and force. A device and method for continuously perceiving sensations in the same direction have been proposed (Patent Document 1). This tactile force sense interface device allows a person to experience a force that cannot physically exist by appropriately controlling physical quantities using human sensory characteristics.

In addition, by using a "twin eccentric rotor system" consisting of two eccentric rotors instead of the torque generating flywheel, a 3-DOF hybrid type haptics that can simultaneously present not only rotational force sensations but also translational force sensations. An interface device (Non-Patent Document 3) has been developed. This is a haptic interface device with a hybrid function that can continuously present the sensation of both translational force and rotational force in an arbitrary direction within a plane with a single interface. By skillfully utilizing the non-linear human sensory characteristics, the Gyro Cube Sensus in hand becomes heavier, lighter, and finally floats, creating the illusion of force.

Norio Nakamura, “Experience the Illusion Sense of Non-Grounded Force Sense Interface 'Push/Pull/Lift'”, Inspection Techniques, Nippon Kogyo Publishing, Vol.11, No.2, pp.6-11(2006/02) Yokichi Tanaka, Katsutaka Sakai, Yuka Kono, Yukio Fukui, Juri Yamashita, Norio Nakamura, “Mobile Torque Display and Haptic Characteristics of Human Palm”, INTERNATIONAL CONFERENCE ON ARTIFICIAL REALITY AND TELEXISTENCE, pp.115-120(2001/12) Nakamura, N., Fukui, Y. : “An Innovative Non-grounding Haptic Interface ‘GyroCubeSensuous’ displaying Illusion Sensation of Push, Pull, and Lift“, Proceedings. of ACM Siggraph2005, 2005.

Japanese Patent Application Laid-Open No. 2005-190465

When wires and arms are used, their presence constrains the movement of humans, and since they can only be used in the effective space where the haptic presentation system and the haptic presentation part are connected by wires and arms, the usable spatial extent is increased. There is a limit. The method of generating torque by controlling the angular momentum composite vector generated by the three gyro motors is free from restrictions by wires and arms, has a relatively simple structure, and is easy to control. However, there are also problems in that it is not possible to continuously present tactile force sensations and to present force sensations other than torque.

Furthermore, conventional haptic interface devices have problems such as poor response of the interface to user's movements and insufficient interaction capable of expressing the shape and texture of virtual objects. In addition, reduction of heat generation and energy consumption is a major issue in the practical use and commercialization of the conventional acceleration/deceleration mechanism with an eccentric rotor that uses a motor. It is also essential to improve operability and ease of use.

In view of the above points, a first object of the present invention is to utilize illusionary tactile force sensations to provide a full tactile experience of touching virtual objects and game characters in a non-based interface. A virtual reality environment that can express the presence, shape, and texture of virtual objects, such as friction and roughness, in addition to 3D images and 3D sound images, by controlling the drag caused by the illusionary tactile force sense according to movement. It is to provide a generating apparatus and method.

The second object of the present invention is to realize an audiovisual and tactile virtual reality environment that integrates virtual and real spaces that can be used in daily life. and to provide a device and method that suppresses energy consumption and facilitates miniaturization and mobility. In addition, the present invention provides an apparatus and method with excellent operability and responsiveness while freely designing an interface that matches the characteristics and uses of individual users, in response to large individual differences in hand size, taste, and sensation of users. That is.

In order to achieve the above object, a first aspect of the present invention provides an illusionary tactile force sense interface apparatus including an illusionary tactile force sense device, and an illusionary tactile force sense device drive control for driving and controlling the illusionary tactile force sense device. and a virtual reality environment generation device.

A second embodiment of the present invention provides an illusionary tactile force sense inducing device that generates an illusionary tactile force sense induction function that matches content using illusionary tactile force sense data, and an illusionary tactile force sense device. A virtual reality environment generating apparatus comprising an interface device and an illusionary tactile force sense device drive control device that drives and controls the illusionary tactile force sense device.

A third embodiment of the present invention is a content creation device that creates content based on information from various sensors and content data, and an illusionary tactile force sense induction function that matches the content using the illusionary tactile force sense data. A virtual reality environment generation device comprising an illusionary tactile force sense induction device for generating, an illusionary tactile force sense interface device having an illusionary tactile force sense device, and an illusionary tactile force sense device drive control device for driving and controlling the illusionary tactile force sense device. is.

A fourth embodiment according to the present invention is provided with a content creation device that creates content based on information from various sensors and content data, and a learner and/or corrector, and induces an illusionary tactile force sense that matches the content. An illusionary tactile force sense induction device that generates functions using illusionary tactile force sense data, an illusionary tactile force sense interface device that includes an illusionary tactile force sense device, and an illusionary tactile force sense device drive that drives and controls the illusionary tactile force sense device. and a controller.

A fifth embodiment according to the present invention is provided with a content creation device for creating content based on information from various sensors and content data, a learner and/or a corrector, and an illusionary tactile force sense matching the content. An illusionary tactile force sense induction device that generates functions using illusionary tactile force sense data, an illusionary tactile force sense interface device that includes an illusionary tactile force sense device, and an illusionary tactile force sense device drive that drives and controls the illusionary tactile force sense device. and a control device, wherein the illusionary tactile force sense inducing device generates an illusionary tactile force sense induction function for learning after the instruction for learning, and the user's reaction/force sense information presented according to this function. It is a virtual reality environment generation device that senses actions, estimates the user's illusionary tactile force sense sensory characteristics as an illusionary tactile force sense quantity, and calculates data for correcting individual differences related to the illusionary tactile force sense induction function and control.

A sixth embodiment according to the present invention is provided with a content creation device that creates content based on information from various sensors and content data, and a learning device and/or a corrector, and induces an illusionary tactile force sense that matches the content. An illusionary tactile force sense induction device that generates functions using illusionary tactile force sense data, an illusionary tactile force sense interface device that includes an illusionary tactile force sense device, and an illusionary tactile force sense device drive that drives and controls the illusionary tactile force sense device. and a control device, wherein the illusionary tactile force sense inducing device senses the user's reaction/behavior to the illusionary tactile force sense information in each content, and estimates the user's illusionary tactile force sense characteristics with respect to the feature amount in the content. , an illusionary tactile force sense induction function and a virtual reality environment generation device that calculates and uses data for correcting individual differences related to control.

According to the seventh aspect of the present invention, the illusionary tactile force sense device includes an acceleration/deceleration mechanism.

According to the eighth aspect of the present invention, the illusionary tactile force sense device drive control device controls the speed of the acceleration/deceleration mechanism via the oscillation circuit.

According to the ninth aspect of the present invention, the illusionary tactile force sense device drive control device operates the motor of the illusionary tactile force sense device according to the illusionary tactile force sense induction function generated by the illusionary tactile force sense induction device. Control the phase, direction, speed of rotation, or the phase, direction, speed of an actuator.

According to a tenth aspect of the present invention, the virtual reality environment generating device includes a sensor, the sensor includes a position sensor for detecting and measuring the movement of the part to which the illusionary tactile force sense interface device is attached, the shape of the real object, and the It is at least one of a shape sensor that measures surface shape, a pressure sensor that detects and measures contact/grip force between a real object and a user, a biosignal sensor, and an acceleration sensor.

According to an eleventh aspect of the present invention, an illusionary tactile force sense interface device has a mounting section, and a member having nonlinear stress characteristics between the illusionary tactile force sense device and the mounting section.

According to a twelfth aspect of the present invention, the illusionary tactile force sense interface device includes a vibration-resistant member between the illusionary tactile force sense device and the acceleration sensor.

According to a thirteenth aspect of the present invention, an illusionary tactile force sense interface device includes an acceleration sensor, and a finger attachment unit between the illusionary tactile force sense device and the acceleration sensor.

According to the fourteenth aspect of the present invention, an illusionary tactile force sense interface device includes at least one of a CPU, a memory, and a communication device.

According to the fifteenth aspect of the present invention, the content creation device performs physics simulation calculation, generation and update of virtual reality space, creation and display of computer graphics, and illusionary tactile force information based on information from sensors. process.

According to a sixteenth aspect of the present invention, an illusionary tactile force sense interface device comprises two sets or a plurality of sets of illusionary tactile force sense devices driven at different frequencies and/or different acceleration/deceleration.

According to a seventeenth aspect of the present invention, an illusionary tactile force sense interface device has an attachment section for attachment to a finger or a body.

An eighteenth form according to the present invention is a controller device comprising a base having deformable means and an illusionary tactile force sense interface device comprising an illusionary tactile force sense device device.

A nineteenth form according to the present invention is a virtual controller comprising an illusionary tactile and force sense interface device that creates virtual actions to provide virtual existence, tactile sensations, and button operation sensations, and an audiovisual display that presents virtual objects. It is a device.

By implementing the virtual reality environment generating apparatus and method according to the present invention, the following special effects can be obtained.

(1) Conventional non-based haptic interfaces can only perceive a sense of vibration from the repetition of periodic motion such as vibration, and there is sufficient interaction by force feedback to understand the shape and texture of a virtual object. I didn't get it. On the other hand, in the present invention, by using an illusionary tactile force sense, force and motion components that do not exist physically are perceived, and force continuously works in a certain direction psychophysically. Able to perceive sensations. Furthermore, this illusion actually causes a physical phenomenon in which the arm holding the interface is lifted against gravity, despite the fact that it is a non-base type interface that is used in a state of being held in the air without any force support. .

(2) By controlling the drag caused by the illusionary tactile force sense according to the movement of the finger and body wearing the illusionary tactile force sense interface device, the presence, shape, and texture of the virtual object, such as the friction and roughness sensations. can be expressed. In particular, by presenting a negative drag force (acceleration) against movement, a smooth feeling of gliding on ice is realized. In addition, by controlling the illusionary tactile force sensation while monitoring the grip pressure with the real object, it is possible to edit the feel of the real object and replace it with the feel of the virtual object.

(3) By deforming the shape of the illusionary tactile force sense interface device in synchronization with the illusionary tactile force sense, the sense of force induced by the illusionary tactile force sense is emphasized and reality is improved.

(4) The sensory characteristics of the illusionary tactile force sense differ from user to user, and there are large individual differences in perceived intensity and texture. It can be handled in the same way as a sensory interface device. In addition, since individual differences can be corrected in real time by measuring myoelectric responses, it is possible to improve the learning of illusionary tactile force sensations and optimize control for each user.

(5) Heat generation and energy consumption are major problems in the conventional acceleration/deceleration mechanism using an eccentric rotor that utilizes a motor. On the other hand, by controlling the speed of the acceleration/deceleration mechanism with an oscillation circuit or by using multiple sets of illusionary tactile force devices driven at different frequencies, it is possible to achieve the same illusionary tactile sensation as the acceleration/deceleration mechanism while rotating at a constant speed. Achieving haptics reduces heat generation and energy consumption, facilitating miniaturization and mobility.

(6) In the conventional arm-type haptic interface device, the position/orientation is measured by the angle of the arm attached to the fingertip, and the contact/interference with the virtual object is determined and the stress to be presented is determined in response to minute movements of the fingertip. There was a problem that a response delay occurred because the recalculation of was repeated. On the other hand, in the present invention, the CPU and memory are installed in the illusionary tactile force sense interface device, which is the erasure unit, instead of the content generation device, which is the central unit, and real-time control is performed, thereby enabling virtual button presses, etc. responsiveness is improved, and the reality and operability are improved.

(7) In conventional driving simulators, it was not possible to experience a feeling of continuous acceleration by any method other than the use of gravity. There is a sense of incongruity in the feeling of acceleration in an environment where In contrast, in the present invention, even an arcade-type game machine that repeats periodic movements in a narrow space on a pedestal can experience a sense of continuous acceleration. You can feel the power.

(8) A conventional game controller is a "pseudo-physical" game in which the user moves his or her own body, and sufficient interaction cannot be obtained with force feedback based on vibration. On the other hand, in the present invention, by using an illusionary tactile force sense interface device, a "full bodily sensation type controller" that can tactilely touch virtual objects and game characters is realized.

(9) Game controllers of various shapes, sizes, and button arrangements are on the market, but it is often difficult to find an easy-to-use controller that matches the user's hand size and preferences. In contrast, the present invention eliminates the need to purchase a dedicated controller suitable for the content of the game, and uses a virtual controller technology that freely designs the shape and button arrangement of the controller to match the palm of the individual. You can freely transform and change the controller according to the scenes and stories within.

(10) In contrast to conventional audio-visual virtual reality, a non-based technology for making practical virtual objects tactile is provided, and an audio-visual sensation that integrates virtual and real spaces that can be used in everyday life. provides a virtual reality environment.

FIG. 4 is an explanatory diagram showing basic units of the virtual reality environment generating device; FIG. 4 is an explanatory diagram showing a processing flowchart of the virtual reality environment generating device; FIG. 4 is an explanatory diagram showing a processing flowchart of calibration; FIG. 4 is an explanatory diagram showing a processing flowchart of sensing; FIG. 4 is an explanatory diagram showing a processing flowchart of the content creation device; FIG. 10 is an explanatory diagram showing a processing flowchart for presentation; FIG. 10 is an explanatory diagram showing a processing flowchart of learning; FIG. 10 is an explanatory diagram showing an example of a control method for a device that induces an illusionary tactile force sense; It is a figure which shows the shape of an eccentric weight. FIG. 9 is an explanatory diagram schematically showing the phenomenon of FIG. 8 and its effect; FIG. 10 is an explanatory diagram relating to individual differences in illusionary tactile force sense; FIG. 10 is an explanatory diagram showing texture representation of a virtual flat plate; FIG. 10 is an explanatory diagram showing the direction of the illusionary tactile force sense based on the initial phase of the phase pattern; FIG. 4 is an explanatory diagram showing an example of an illusionary tactile force sense interface device; FIG. 4 is an explanatory diagram showing an example of an illusionary tactile force sense interface device; FIG. 4 is an explanatory diagram showing an example of an illusionary tactile force sense interface device; FIG. 10 is an explanatory diagram showing an example of implementation of the illusionary tactile force sense interface device; FIG. 10 is an explanatory diagram showing an example of implementation of the illusionary tactile force sense interface device; 2 is an explanatory diagram showing an example of a control system of the illusionary tactile force sense interface device 101. FIG. FIG. 10 is an explanatory diagram showing a processing flowchart of the illusionary tactile force sense device; It is an explanatory view showing a motor control device using a pulse train. FIG. 10 is an explanatory diagram showing the effects of the illusionary tactile force sense interface device; FIG. 4 is an explanatory diagram showing a control algorithm for initial phase lag; FIG. 4 is an explanatory diagram showing nonlinear characteristics used in the illusionary tactile force sense interface device; FIG. 11 is an explanatory diagram showing an alternative/illusory tactile force sense device; FIG. 4 is an explanatory diagram showing a control algorithm using different viscoelastic materials; FIG. 4 is an explanatory diagram showing the effect of using different viscoelastic materials; FIG. 4 is an explanatory diagram showing a control algorithm using hysteresis material; FIG. 4 is an explanatory diagram showing a control algorithm using a viscoelastic material whose properties change with applied voltage; FIG. 4 is an explanatory diagram showing a control algorithm using an oscillator circuit; FIG. 10 is an explanatory diagram showing an arrangement example and an application example of the illusionary tactile force sense device; FIG. 10 is an explanatory diagram showing an arrangement example and an application example of the illusionary tactile force sense device; FIG. 10 is an explanatory diagram showing an arrangement example and an application example of the illusionary tactile force sense device; FIG. 4 is an explanatory diagram showing a deformable illusionary tactile force sense interface device; FIG. 4 is an explanatory diagram showing a method of implementing a virtual controller; FIG. 10 is an explanatory diagram showing an illusionary tactile force sense device using a set of units and a control method; FIG. 10 is an explanatory diagram showing an illusionary tactile force sense device using a set of units and a control method; FIG. 10 is an explanatory diagram showing an illusionary tactile force sense device using a set of units and a control method; FIG. 10 is an explanatory diagram showing an illusionary tactile force sense device using a plurality of sets of units and a control method; FIG. 10 is an explanatory diagram showing an illusionary tactile force sense device using a plurality of sets of units and a control method; FIG. 10 is an explanatory diagram showing an illusionary tactile force sense device using a plurality of sets of units and a control method; FIG. 10 is an explanatory diagram showing an illusionary tactile force sense device using a plurality of sets of units and a control method; FIG. 10 is an explanatory diagram showing an illusionary tactile force sense device using a plurality of sets of units and a control method; FIG. 10 is an explanatory diagram showing an illusionary tactile force sense device using multiple units having eccentric weights of different weights and a control method; It is an explanatory view showing a wearing part. FIG. 10 is an explanatory diagram showing an embodiment using a virtual reality environment generation device;

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment according to the present invention will be described below with reference to the drawings.

FIG. 1 shows a hardware block diagram of an illusionary tactile force sense interface device 101 used in a virtual reality environment generation device (VR environment generation device) 100. As shown in FIG. Here, the case where the illusionary tactile force sense interface device 101 is attached to the fingertip 533 will be described as an example, but the place where the illusionary tactile force sense interface device 101 is attached is not limited to the fingertip. In the figure, the acceleration sensor 108, the pressure sensor 109, and the myoelectric sensor 110 are integrated with the illusionary tactile force sense device 107 and arranged in the illusionary tactile force sense interface device 101 attached to the fingertip 533. Although an example is shown, these sensors may be worn at different body positions from the illusionary tactile force sense device 107 . In this specification, even when the illusionary tactile force sense device 107 and the sensor are separately attached to different parts of the body, they are collectively referred to as the illusionary tactile force sense interface device 101 .

Contents are created based on the information from various sensors and the content data 104, and an illusionary tactile force sense induction function 1713 matching this content is generated by an illusionary tactile force sense induction function generator 115 using the illusionary tactile force sense data 106. The illusionary tactile force sense device 107 is controlled by the illusionary tactile force sense device drive controller 112 .

The phase, direction, and rotational speed of the eccentric motor 815 of the illusionary tactile force sense device 107 are controlled according to the illusionary tactile force sense inducing function generated by the illusionary tactile force sense inducing function generator 115 . In the illusionary tactile force sense device 107, the change in momentum (acceleration/deceleration pattern) generated by the rotation of the eccentric weight 814 by the eccentric motor induces an illusion related to the tactile force sense (illusory tactile force sense). By using this illusionary tactile force sense evoking function, it is possible to perceive a sensation different from the force (physical information) generated by the change in the amount of exercise presented by utilizing the illusion, which is a non-linear sensory characteristic. That is, it is possible to perceive force and motion components that do not physically exist. For example, physically, the periodically repeated vibration does not have force information only in one direction because the direction of the force changes periodically, but according to this illusionary tactile force sense induction function, the momentum acceleration/deceleration pattern Psychophysically, it is possible to perceive a continuous force in only one direction by controlling . The illusionary tactile force sense induction device 103 has a learner 116 and a corrector 117, and is optimized according to the user's individual characteristics.

The movement of the fingertip 533 wearing the illusionary tactile force sense interface is sensed by the position sensor 111 and the acceleration sensor 108, and based on the position/velocity/acceleration information obtained by the position sensor and the acceleration sensor, the physics simulator 113 in the content creation apparatus 102 is used. The contact determination between the virtual object generated by and the finger 533 and the force acting on the virtual object are calculated. Further, the contact/grip force between the real object and the user is detected by the pressure sensor 109 and the myoelectric sensor 110 . The contents are converted into video and audio images by the computer graphics 114 and the sound source simulator 119 and displayed on the audiovisual display 105 . As a result, a practical non-based haptic interface is provided for virtual reality, which has conventionally been biased toward audiovisual, and an audiovisual and tactile virtual reality environment that can be used in daily life is provided.

The illusionary tactile force sense induction device 103 can be activated by using simulation data of a physical simulator of another device (for example, a conventional game machine) without using a content creation device, or by manually setting physical quantities by a user. It can also be controlled and used.

In addition, although control is generally performed via the content generation device, which is the central part, the CPU installed in the illusionary tactile force sense interface device, which is the deletion part, does not use the content generation device and the illusionary tactile force sense induction device 103. And by performing real-time control using memory, responsiveness, reality, and operability such as pressing a virtual button are improved. It can also be used by connecting it to a conventional device.

By using the illusionary tactile force sense interface device 101, it is also possible to present information related to the conventional haptic sense.

Information transfer and/or connection between devices such as devices, peripherals, databases, and sensors may be wired or wireless.

FIG. 2A shows a processing flowchart of the VR environment generating device 100. FIG. The VR environment generation device 100 is calibrated, and based on sensing information from the connected peripheral device 118 and sensors, VR environment content is generated by modeling that forms a virtual space and virtual objects, and content information is generated. Content is generated and updated based on the sensed movement of the user and information from the peripheral device 118, and information based on the content is presented to the user. The user perceives and recognizes this presented information, and the results of reactions and actions are monitored by sensing.

Calibration is performed on the illusionary tactile force sense interface device and the illusionary tactile force sense induction device. Sensing is performed by an illusionary tactile force sense interface device (acceleration sensor, pressure sensor, myoelectric sensor) and a position sensor. Content generation is performed on a content generation device. The presentation is performed by an illusionary tactile force sense interface device and an illusionary tactile force sense induction device.

Information is also exchanged with the real space through the peripheral device 118 to generate and control a virtual reality environment in which the virtual space and the real space are combined.

In FIG. 2B, the VR environment generating device 100 has a communication device, and each VR environment generating device 100 owned by a plurality of users communicates with the VR environment generating device 100 in a separate space. It shows the generation of two large VR environments. By sharing content and sensing information through communication, a plurality of users in remote locations can coexist and share information in the same VR environment, and can share the operation and feel of the same virtual object. Also, a wearable VR environment is formed by cooperatively operating a plurality of illusionary tactile force sense interface devices 101 worn by the same user.

FIG. 3 shows a flowchart of calibration processing for various sensors, the haptic interface device, and the illusionary tactile force sense interface device 101 .

In each calibration flow, a signal for calibration is generated, various sensors, the haptic interface, and the illusionary tactile force interface device 101 are controlled according to this signal, and the calibration is performed by sensing the control result. will be

FIG. 4 shows a flowchart of sensing processing. In sensing, the position, posture, acceleration, pressure between the interface and the skin, and myoelectric potential of the illusionary tactile force sense interface device 101 are measured. These information are used in calibration, learning, content creation and presentation. Instead of this myoelectric sensor, a biosignal sensor that measures biosignals such as electroencephalograms, heartbeats, respiration, blood pressure, blood flow, blood gas, and skin resistance may be used. 101 control and biofeedback control facilitate calibration, learning and effective illusion induction. The biosignal sensor and biofeedback control may use existing measurement sensors and control methods used in medical care and the like. The biosignal sensor may be worn on the body at a position separate from the illusionary tactile force sense device. For example, an electroencephalogram sensor as a biosignal sensor that constitutes part of the illusionary tactile force sense interface device is attached to the head, and at the same time, an illusionary tactile force sense device that constitutes a part of the illusionary tactile force sense interface device is attached to the fingertip. You may take the form which mounts|wears.

FIG. 5(a) shows a processing flowchart of content generation. In content creation, based on the read content data 104 and sensing information, based on physical simulation calculations and other model calculations, VR space is generated and updated, CG is created and displayed, and illusionary tactile force sensations are generated. and haptic information is processed.

FIG. 5B shows a physics simulation 520 modeled by a spring/damper physics model 528 using a wireframe representation of a freely deforming hollow sphere as an example of content.

When the grid point p1 is connected to the adjacent grid points p2 to p4, the force vector f12 that the grid point p1 receives from the grid point p2 is
f12 =－k×(∥p2－p1∥－L ₀ )×(p2－p1)／∥p2－p1∥－c×(v2－v1) (1)
is represented. however,
pi: Position vector of grid point pi
vi: velocity vector of lattice point pi
k: modulus of elasticity of the spring,
c: damper viscosity coefficient,
L ₀ : Length of spring in equilibrium If the resultant force of the forces received by grid point p1 with mass m1 from surrounding grid points p2 to grid points p4 is f1, then the equation of motion for grid point p1 is
m1×d2p1/ ^{dt2 =} f1=f12+f13+f14 (2)
is represented.

When the fingertip 533 on which the illusionary tactile force sense interface device 101 is worn touches the lattice point p1 of the virtual object/physical model 520, the lattice point p1 changes to the position p'1 of the fingertip, and the reaction force acts on the fingertip. (-f) is
－f=（f12+f13+f14）－m1×d2p'1/ ^dt2 ( ³ )
is represented. A movement of the fingertip 533 for determining contact is sensed by the position sensor 111 and the acceleration sensor 108 .

In an actual numerical simulation, the position p'1, velocity v'1, and force f'1 of grid point p1 at time t' are obtained from variables p1, v1, and f1 one time before t.

in short,
Velocity vector: v'1=v1+(f1/m1)×Δt (4)
Position vector: p'1=p1+v1×Δt (5)
Similarly, the position and velocity of p2 of mass m2 are calculated.

Velocity vector: v'2=v2+(f2/m2)×Δt (6)
Position vector: p'2=p2+v2×Δt (7)
Finally, the force vector acting between grid point p1 and grid point p2
f'12=－k×(∥p'2－p'1∥－L ₀ )×(p'2－p'1)／∥p'2－p'1∥－c×(v'2－v '1) (8)
is calculated.

The position, velocity, and force of each grid point are calculated for each calculation and stored in memory. This stored value is used to calculate position, velocity and force for the next time. As a result, a reaction force to the fingertip 533 is presented, and a 3D sound image and a tactile virtual object made into a 3D image are realized on the audiovisual display.

Similar to the physics simulation for the virtual object 531 described above, the VR environment is based on the real object in the real space sensed by the peripheral device and the user's motion information sensed by the position sensor 111 and the acceleration sensor 108. are modeled in the same VR environment, the force of contact and grip with the content is calculated, and a VR space in which the virtual space and the real space are fused is generated.

FIG. 5C shows deformation of the virtual object 531 when the illusionary tactile force sense interface device 101 attached to the fingertip 533 is moved. The movement of the illusionary tactile force sense interface device 101 is monitored by a sensor, contact with the virtual object 531 is detected, and the physical simulation of the virtual object/physics model 520 calculates the deformation of the model, the deformation force, and the reaction force transmitted to the fingertip. The tactile sensation is presented through the illusionary tactile force sense interface device 101 . Based on the calculation results of the physics simulator 113, the illusionary tactile force sensation from the virtual object 531 that deforms according to the movement of the finger 533 to the fingertip 533 is controlled. It is possible to transform and move the virtual object 531 while feeling the stickiness of stretching the slime.

6(a) and 6(b) show a flowchart of presentation processing.

The content data 104 related to the illusionary tactile force sense and the haptic force sense in the VR space created by the content creation device 102 is read, the illusionary tactile force sense induction function and the haptic force sense function are generated, and the information obtained by sensing and each Correction is performed by the corrector 117 according to the characteristics of the user. According to this function, the illusionary tactile force sense device 107 is feedback-controlled.

Since the illusionary tactile force sense interface device uses an illusion, there are large individual differences in sensitivity to the illusionary tactile force sense and improvement in sensitivity through learning. Therefore, even if the same stimulus is presented, the intensity of perception differs depending on the user. Therefore, in order to perceive stimuli of the same intensity without depending on the user, it is necessary to learn and correct the stimuli.

7(a) and 7(b) show processing flowcharts of the learner 116, including active learning and unconscious learning. In the active learning method, the following illusory tactile sensation evoking function for learning is generated after the instruction for learning. The intensity of the illusionary tactile force sense obtained by sensing the user's reaction/behavior to the illusionary tactile force sense information presented according to this function indicates the user's illusionary tactile force sense characteristics. An illusionary tactile force sense evoking function is created based on the data of the illusionary tactile force sense characteristics measured in advance for a large number of subjects. This illusionary tactile force sense induction function and the illusionary tactile force sense sensory characteristics of each user are compared to calculate correction data 1714 indicating individual differences and stored in a memory or a user characteristics database.

Specifically, in the active learning, after wearing the illusionary tactile force sense interface device 101, according to the instructions, a constant force by the illusionary tactile force sense is sequentially applied in the directions of 0°, 180°, 90°, and 270°. is presented. Unlike tactile force sense, illusionary tactile force sense is presented while changing the presentation direction discretely. The direction of force becomes clear. After learning for one minute, the intensity of the illusionary tactile force sense is gradually weakened, and the intensity at which the subject cannot perceive is estimated as the sensory threshold of the illusionary tactile force sense. This sensation threshold differs for each user and direction of presentation, and this sensation threshold is stored in a memory or database as correction data for correcting individual static characteristics. As the learning progresses, the threshold converges to a certain value, which is the illusionary tactile force sense characteristic, and the degree of learning is determined by the convergence time constant, which is the convergence rate. Equality level curves are then determined by psychophysical pairwise comparison.

Similarly, in the unconscious learning method, the user's reaction/behavior to the illusionary tactile force information in each content is sensed, and the feature amount (illusory tactile force sense intensity and time pattern) related to the illusionary tactile force sense information in the content is evaluated. The user's illusionary tactile force sensory characteristics are measured, and response characteristics (dynamic characteristics) are estimated by estimating transfer functions. Response characteristics for each user are stored in a memory or database as data for correcting individual differences as an illusionary tactile force sense induction function.

Thus, the illusionary tactile force sense is a characteristic that varies greatly among individuals, but by using learning and correction, even if the illusionary tactile force sense interface device is used, the stimulus intensity is the same as that of the conventional haptic force sense interface device. It can be treated as a presentation device.

The features of the illusionary tactile force sense are shown below.
In conventional tactile force sense interface devices, forces and motions that physically reproduce physical phenomena related to tactile force senses are presented and perceived on the fingertips and palms.・A phenomenon in which a force or motion that is different from motion or that does not exist is perceived and recognized. For example, even though the interface does not actually (physically) float, the sensation of being floated is perceived.

In conventional haptic interface devices, a base that supports the reaction force when a force is presented to the fingertip is essential in order to make the user feel as if a force is acting from the outside. On the other hand, a tactile interface device using a non-base type vibration motor, which has no base, only vibrates around the center of gravity, which is the vibration equilibrium point, and does not feel any external force. could not. On the other hand, the illusionary tactile force sense interface device 101 of the present invention is a device that presents a tactile force sense sensation using an illusion that is not a base type but can present the sensation of being pushed from the outside. There is (Non-Patent Document 3).

Illusory tactile force sense is not limited to illusory sensory perception, but also causes a physical phenomenon in which the arm holding the interface is actually lifted. This is because the user himself/herself unconsciously moves his or her arm due to the feeling deceived by the illusion, or the muscle of the arm moves due to reflex. In this respect, unlike conventional haptic interfaces that have been invented and developed to reproduce the physical force acting between an object and the human body, the present invention induces the illusion of haptic sensation. The present invention relates to a device that effectively induces an illusionary tactile force sensation.

In addition, the illusionary tactile force sense interface device 101 of the present invention also has functions and effects as a conventional haptic force sense interface device, and a synergistic effect of both presentation sensations can be achieved.

FIGS. 8-1(a) to 8-1(d) show an example of a control method for a device that induces an illusionary tactile force sensation.

FIG. 8-1(a) shows an acceleration/deceleration mechanism, which is composed of two eccentric rotors A and B. FIG. FIG. 8-1(b) schematically illustrates the case where these two eccentric rotors are synchronously rotated in opposite directions. As a result of this synchronous rotation in opposite directions, linear acceleration and deceleration forces can be synthesized in any direction in the plane. FIG. 8-1(c) schematically illustrates the case where sensory characteristics such as vibration, force, and torque are logarithmic characteristics. Considering the case where a positive force is generated at the operating point A and a negative force in the opposite direction is generated at the operating point B, the force sensation is expressed as shown in FIG. 8-1(d). . The magnitude of the resultant momentum of the two eccentric rotors is the sum of the angular momentums of the eccentric rotors A and B, and the force is proportional to the time derivative of the magnitude of the resultant momentum of the two eccentric rotors.

Figures 8-2(a) to 8-2(c) show the shape of the eccentric weight, which can be streamlined as shown in Figure 8-2(b) or specific gravity as shown in Figure 8-(c). By unevenly arranging the materials with different values, resistance due to rotation is reduced, and large rotation acceleration/deceleration can be obtained.

FIG. 9 schematically shows the phenomenon of FIG. 8 and its effect. Vibration that periodically accelerates and decelerates around the equilibrium point by controlling the rotation pattern of the eccentric motor 815 and changing the combined momentum of the two eccentric rotors in consideration of the sensory characteristics related to the illusionary tactile force sensation. From 904, an illusion 905 can be induced in which a force continuously acting in a certain direction is perceived. In other words, although there is no physical component like a force acting in a certain direction, an illusion is induced in which a force is perceived as acting in a certain direction.

When alternately accelerating and decelerating at the operating point A and the operating point B for each phase of 180°, a force sensation 905 in a fixed direction is continuously perceived. The force physically returns to the initial state in one cycle, and the integral of the momentum and force is zero. In other words, it stays around the equilibrium point and the acceleration/deceleration mechanism never moves to the left. However, the sensory integral value of force sensation, which is a sensory quantity, does not become zero. At this time, the perception of the positive force integral 908 is reduced and only the negative force integral 909 is perceived.

Here, the time derivative of the angular momentum is the torque, and the time derivative of the momentum is the force. Therefore, the method of periodically rotating a rotating body or the like is not suitable for continuously presenting a haptic sensation in a fixed direction. In particular, it is physically impossible to present a continuous force in one direction in non-based interfaces used in mobile devices and the like.

However, humans have non-linear sensory characteristics, and by using the method of the present invention, it is possible to create forces and force patterns that are different from physical characteristics by using perceptual sensitivity related to illusionary tactile force sense characteristics and controlling the acceleration/deceleration pattern of momentum. can be perceived as an illusion. For example, sensitivity is the ratio of the perceived stimulus magnitude to the applied stimulus intensity, but human sensory characteristics differ in sensitivity to the applied stimulus intensity, being more sensitive to weaker stimuli, Insensitive to strong stimuli. Therefore, by controlling the acceleration/deceleration phase of the motor rotation and repeating the acceleration/deceleration periodically, we succeeded in presenting a continuous haptic in the direction in which the weak stimulus was presented. Also, by selecting the appropriate action points A and B for the sensory characteristics, it is possible to present a continuous haptic sensation even in the direction in which the strong stimulus is presented.

A driving simulator is associated with a similar device, but in a driving simulator, after giving a desired force (acceleration feeling), the acceleration feeling of the car is reproduced by slowly returning to the original position with a small acceleration that is not noticed. is presented. Therefore, the presentation of force becomes intermittent, and in such a biased acceleration system, it is not possible to continuously present a sense of force or acceleration in a certain direction. The same is true for conventional haptic interface devices. However, in the present invention, in spite of the drive method 904 that continuously repeats acceleration and deceleration in the forward and reverse directions above the sensory threshold at a short period of, for example, 50 Hz, it is possible to continuously drive in a certain direction by using an illusion. A translational force sensation 905 is presented. In particular, the feature of the illusionary tactile force sense interface device 101 using illusion is that a continuous force is perceived in a direction opposite to the direction of the intermittent force presented in the above-described driving simulator using a physical method. is.

In other words, by using the non-linear human sensory characteristics that sensitivity varies depending on this intensity, although the integral of the force generated by periodic acceleration/deceleration and vibration is physically zero, sensory Not only is the force 908 in the positive direction not canceled, but the force 908 in the positive direction is not perceived, and the sense of translational force 905 and the sense of torque can be presented continuously in the negative direction 909, which is the target direction. (Refer to FIG. 20(c) for the method of generating a continuous torque sensation.) These phenomena can be applied to the physical quantity 832 whose sensory characteristic 831 is a stimulus, even if the physical quantity 832 is non-logarithmic or non-linear. The same effect can be obtained. This effect is obtained not only in the non-base type but also in the base type.

In FIG. 9, by bringing the rotation continuation time Ta at the operating point A closer to zero, the momentum in each section of the rotation continuation time Ta and the rotation continuation time Tb is equal. Although the momentum increases and the force also increases, the force sensation changes logarithmically and the sensitivity decreases, so the integral of the sensation value in the interval of the rotation duration Ta approaches zero. Therefore, the force sensation in the section of the rotation continuation time Tb becomes relatively large, and the continuity of the force sensation 905 in one direction is improved. As a result, the operating point A and the operating point B are appropriately selected, the operating point A duration and the operating point B duration are appropriately set, and the synchronous phases of the two eccentric rotors A and B are adjusted. By doing so, it is possible to continue presenting force sensations freely in any direction.

Like the sensory characteristics shown in FIGS. 10(a) to 10(c), different users have different sensory characteristics. For this reason, some people can clearly perceive the illusionary tactile force sense, some people have difficulty perceiving it, and some people can easily perceive it through learning. The present invention has a device for correcting this individual difference. In addition, when the same stimulus is continuously presented, the sense of the stimulus may be dulled. Therefore, it is effective to prevent habituation by fluctuating the intensity, cycle, and direction of stimulation.

FIG. 10(d) shows an example of a method of presenting force in a certain direction using an illusionary tactile force sense. In the method of rotating two eccentric oscillators in opposite rotation directions to synthesize vibration components, the high speed rotation speed ω1 (high frequency f1) 1002a at the operating point A and the low speed rotation speed ω2 (low frequency f2) at the operating point B 1002b is presented alternately every 180° phase, the illusionary tactile force intensity (II) is proportional to the logarithm of the acceleration/deceleration ratio Δf/f of the frequency, which is the rotation speed of the eccentric rotor (Fig. 10(e) ). However, (f=(f1+f2)/2, .DELTA.f=f1-f2). The slope n when plotting the logarithmic value of the illusionary tactile force sense intensity and Δf/f shows individual differences.

In addition, the vibration sense intensity (VI) indicates the intensity of the vibration component that is perceived at the same time as the sense of force in a certain direction due to the illusion. is increased, the vibration sensation intensity (VI) is relatively decreased (FIG. 10(f)). By controlling the intensity of this vibration component, the feel of the force when the illusionary tactile force is presented changes. The slope m when plotted in logarithm indicates individual differences. Note that n and m, which indicate individual differences, change as learning progresses, and converge to constant values when learning is saturated.

FIGS. 11(a) to 11(c) show a texture representation method of the virtual flat plate 1100. FIG. The movement (position/orientation angle, velocity, acceleration) of the illusionary tactile force sense interface device 101 monitored by sensing represents the movement 1101 of the virtual object. In addition, by controlling the direction and intensity of the drag force 1102 and the texture parameter (contained vibration component) of the illusionary tactile force sense, the friction sensation 1109 and the roughness sensation 1111, which are the textures of the virtual flat plate, and the shape are controlled.

FIG. 11A shows a drag force 1103 acting from the virtual flat plate to the virtual object and a drag force 1102 against the movement when the virtual object (illusory tactile force sense interface device 101) is moved on the virtual flat plate 1100. FIG.

FIG. 11(b) shows that the frictional force 1104 acting between the illusionary tactile force sense interface device 101 and the virtual flat plate 1100 vibratingly repeats dynamic friction and static friction when the two objects come into contact with each other. In addition, the presence and shape of the virtual flat plate are perceived by feedback-controlling the pushing-back force 1106 so that the illusionary tactile force sense interface device 101 stays within the error thickness 1107 of the virtual flat plate. When the illusionary tactile force sense interface device 101 does not exist in the virtual flat plate 1100, it does not present the pushing-back drag force, and only when it exists, the presence of the wall is perceived.

FIG. 11(c) shows a method of expressing surface roughness. By presenting a resistance in a direction opposite to the direction 1101 in which the illusionary tactile force sense interface device 101 is moved in accordance with the movement speed/acceleration, a sense of resistance and a sense of viscosity 1108 are perceived. By presenting a negative drag force (acceleration force 1113) in the same direction as the moving direction, it is possible to emphasize the smoothness 1110 of the virtual flat plate as if it were sliding on ice. This sense of acceleration/smoothness 1110 is difficult to present with a conventional non-based tactile-force sense interface device using a vibrator. It is an effect. Further, by vibratingly changing the drag (oscillating drag 1112), the surface roughness sensation 1111 of the virtual flat plate is perceived.

FIG. 12(a) shows the direction of the illusionary tactile force sense induced and perceived by the initial phase (θi) of the phase pattern.

The illusionary tactile force sense device 107 changes the initial phase (θi) of rotation start shown in FIG. It can be controlled in the direction of the initial phase (θi). For example, by changing the initial phase (θi) as shown in FIG. 12(c), it can be induced in any direction of 360° within the plane.

At this time, if the illusionary tactile force sense interface device 101 itself is heavy, the upward force sensation 1202 due to the illusionary tactile force and the downward force sensation 1204 due to gravity cancel each other out, and it is difficult to obtain the buoyancy sensation 1202 to float up. Sometimes you can feel it. At that time, by slightly shifting the upward direction of the illusionary tactile force sense from the direction opposite to the direction of gravity to induce the illusionary tactile force sense 1203, it is possible to suppress the reduction or inhibition of the levitation sensation due to gravity.

If it is desired to present in the direction opposite to the direction of gravity, there is also a method of inducing an illusionary tactile force sensation alternately in the direction of gravity and in directions slightly deviated from the vertical such as 180°+α° and 180°−α°.

FIGS. 13-1(a) to 13-2(g) show implementation examples of the illusionary tactile force sense interface device 101. FIG.

As shown in FIGS. 13-1(a) and 13-1(b), it is attached to a fingertip 533 using an adhesive tape 1301 or a finger insertion portion 1303 of a housing 1302. FIG. Alternatively, it may be worn between the fingers 533 (FIGS. 13-1(c) and 13-1(e)), or may be held between the fingers 533 (FIG. 13-1(d)). The housing 1302 may be made of a hard material that is less deformable, a material that is easily deformed, or a viscoelastic slime. FIGS. 13-2(a) to 13-2(g) are also conceivable as modifications of these mounting methods. In FIGS. 13-2(e) to 13-2(g), by controlling the phase of the two basic units of the illusionary tactile force sense device by flexible adhesion and housing, in addition to left, right, up, and down force sensations, , sensation of expansion, and sensation of compression/pressure can also be expressed. As described above, an adhesive tape, a housing having a finger-insertion part, or the like, for attaching the illusionary tactile force sense interface device 101 to the body or the like is called an attachment part. The attachment part may be of any form, such as a sheet type, a belt type, or tights, as long as it can be attached to an object or the body, in addition to the housing having the adhesive tape and the finger insertion part. In a similar manner, they are worn on the fingertips, palms, arms, thighs, and other parts of the body.

It should be noted that the terms viscoelastic material and viscoelastic properties used herein refer to those having viscous and/or elastic properties.

FIG. 14 shows another implementation example of the illusionary tactile force sense interface device 101 .

In FIG. 14( a ), the illusionary tactile force sense device 107 that generates vibration is detected as noise vibration by the acceleration sensor 108 . 108 is reduced. In addition, by canceling the noise vibration detected by the acceleration sensor 108 based on the control signal of the illusionary tactile force sense device 107, noise mixture is reduced.

In FIGS. 14(c) to 14(e), an anti-vibration material 1405 is interposed between the illusionary tactile force sense device 107 and the acceleration sensor 108 to suppress noise and vibration.

FIG. 14(d) shows an illusionary tactile force sense interface device 101 that perceives an illusionary tactile force sensation while touching a real object. An illusionary tactile force sensation is added to the tactile sensation of a real object. In the conventional data glove, a force sense is presented by attaching a wire to the finger and pulling the finger to present the tactile force sense. When a data glove is used to touch a real object while presenting a tactile force sensation, it is difficult to combine the sensations of the real object and the virtual object, such as the finger moving away from the real object and the grip being hindered. The illusionary tactile force sense interface device 101 does not have such a problem, and realizes a mixed reality (mixed reality) in which a real object is firmly grasped and touched while a virtual touch is also added.

In FIG. 14( e ), by adding an illusionary tactile force sensation according to the contact and grip pressure with the real object measured by the pressure sensor 109 , the grasp/contact feel of the real object can be edited, and the virtual object 531 replaced by the feeling of In FIG. 14(f), instead of the pressure sensor in FIG. 14(e), a shape sensor (for example, a photosensor) that measures surface shape and shape deformation is used to measure the shape and surface shape of the gripped object related to tactile sensation. , and measurement of gripping force, strain shear elasticity, and contact due to deformation. These provide a tactile magnifier that emphasizes measured stresses, shear forces and surface topography. The fine surface shape can be visually confirmed on the display like a microscope, and the shape can also be confirmed tactilely. In addition, if a photo sensor is used as the shape sensor, the shape can be measured without contact, so the shape of the object can be felt by holding the hand over the distant object.

In addition, in the case of variable touch buttons, where the command on the touch panel changes depending on the usage situation and context, especially when the button is hidden by the finger when pressing it, such as on a mobile phone, the command of the variable button is hidden and unreadable. Similarly, in the case of variable buttons in the virtual space of VR content, the menu notation and commands change depending on the context, so when pressing a button, the content of the button to be pressed is unclear. Therefore, as shown in FIG. 14(e), by displaying it on the display 1406 of the illusionary tactile force sense interface device 101, it is possible to press the illusionary tactile force sense button while confirming the command contents of the button.

In order for the virtual object 531 and the virtual button pressing information and pressing reaction force on the virtual controller to be felt and operated without a sense of discomfort in the same way as the real object, the time delay between pressing and presentation of the pressing reaction force is a problem. Become. For example, in the case of an arm-type ground-type haptic interface, the position of the gripping finger is measured by the angle of the arm, etc., and after contact/interference judgment with the digital model is performed, the stress to be presented is calculated, and the motor's Since the rotation is controlled and the motion/stress of the arm is presented, a response delay may occur. In particular, since button operations during a game are reflexively performed at high speed, it may not be possible to monitor and control on the content side in time. Therefore, the illusionary tactile force sense interface device side 101 is also equipped with a CPU and a memory that monitor the sensors (108, 109, 110) and control the illusionary tactile force sense device 107 and the viscoelastic material 1404, thereby performing real-time control. By doing so, the responsiveness of pressing a virtual button is improved, and the reality and operability are improved.

It also has a communication device 205 and communicates with another illusionary tactile force sense interface device 101 . For example, when the illusionary tactile force sense interface device 101 is attached to five fingers, the illusionary tactile force sense interface device is deformed by the shape deforming material (1403 in FIG. 14B) in conjunction with the movement of each finger. Real time and operability are improved by changing the shape and feel of the virtual controller and performing virtual button operations in real time.

In FIG. 14(a), in order to effectively utilize the hysteresis characteristics of sensation and muscle, the myoelectric response is measured by the myoelectric sensor 110, and an illusionary tactile force sense is generated so that the time and strength of contraction of the muscle increases. The induction function is feedback-corrected. One of the factors that affect the induction of the illusionary tactile force sense is how the illusionary tactile force sense interface device 101 is attached to the finger or palm (how it is pinched/strength of pinching), and how the force from the illusionary tactile force sense interface device 101 is received. There is a way to apply force to the arm by the user. There are individual differences in the sensitivity of the illusionary tactile force sense. Some people feel the illusionary tactile force sense more sensitively with a light grip, while others feel more sensitive with a strong grip. Similarly, the sensitivity also changes depending on how it is tightened when worn. In order to absorb this individual difference, the pressure sensor 109 and myoelectric sensor 110 monitor the state of the grip, measure the individual difference, and correct the illusionary tactile force sense induction function in real time. By getting used to and learning the physics simulation in the content, the person will learn how to grip properly, and this correction has the effect of promoting this.

In FIGS. 14(a) to 14(e), the illusionary tactile force sense interface device 101 is thick in order to show the component configuration, but each component can also be thin in sheet form.

FIG. 15 shows an example of mounting on the fingertips 533 of five fingers.

The feature of this implementation example is that the tactile force sense interface device installed in the controller of a conventional game machine simply changes the strength and frequency of vibration, but in this implementation method, it is possible to present an illusionary tactile force sense. The technique makes it possible to continuously perceive a force in a certain direction. Using this, the direction and magnitude of the force of the illusionary tactile sensation are feedback-controlled according to the movement of the finger 533 and palm by the method shown in FIG. present a feeling. In addition, by detecting the movement of the finger 533 with the acceleration sensor 108, the position sensor 111, etc., and feedback-controlling the illusionary tactile force sense, it is possible to continuously present the sense of gravity, sense of mass, and force in a fixed direction. Although it is a base-type interface, it is possible to present the presence, shape, and tactile sensation of the virtual object 531 .

FIG. 16 shows an implementation example different from that in FIG. The respective illusionary tactile force sense interface devices 101 can communicate with each other at high speed and cooperate to present information on the illusionary tactile force sense.

When used as a device for inputting selection/intention by gestures, interactive gesture input with the virtual object 531 enables intuitive gesture input and operation.

In addition to the finger 533 and people, it can be attached to all objects such as writing utensils such as pencils and brushes, daily necessities such as toothbrushes, and toys such as stuffed animals and toys. For example, by attaching or embedding it in the hand of a stuffed animal, it is possible to present the feeling of being pulled or pushed when the hand of the stuffed animal is grasped. It can also be used for training how to use and move pencils and brushes.

Each of them is also a controller, and the aggregate also becomes one large controller, so various forms of controllers can be realized.

FIG. 17A shows an example of a control system for the illusionary tactile force sense interface device 101. FIG.

Based on the information accumulated in the illusionary tactile force sense database 1710, an illusionary tactile force sense induction function is generated according to the tactile sensation to be presented by the content information. The generated function is corrected by the corrector 1702 based on the user characteristics and the position/acceleration/pressure information of the illusionary tactile force sense interface device 101. Then, the controller of the illusionary tactile force sense device 107 A motor controller 1703 converts it into a control signal to drive a motor 1704 connected to an eccentric weight. The encoder 1705 monitors the rotational phase, and the motor controller 1703 feedback-controls the motor rotation so that it rotates appropriately. This rotation/phase pattern induces an illusionary tactile force sensation.

Further, in order to improve the induction effect of the illusionary tactile force sense, as an alternative to the illusionary tactile force sense device 107 that generates the acceleration/deceleration pattern, the viscoelastic property controller 1706 converts it into a control signal, 1407 properties are controlled. By changing the viscoelastic characteristics of the viscoelastic material 1407 over time, even an eccentric rotor that rotates at a constant speed can induce the same effect as the rotation/phase pattern described above due to the motion characteristics via the viscoelastic material 1407 .

The materials and methods are not limited to the above two methods, and any materials and methods can be used as long as they can change the vibration and momentum with a control pattern that induces an illusionary tactile force sensation.

FIG. 17(b) shows iso-level curves relating to the illusionary tactile force sense recorded in the database 1710 of the illusionary tactile force sense. Using the illusionary tactile force sense level curve stored in the illusionary tactile force sense database, it is equivalent to the sensed amount for the reaction force (-f) obtained in the physical simulation of the content, for example, 30 dB. A physical intensity of 15 dB (1725) that induces an illusionary tactile force sense level is calculated, and an illusionary tactile force sense induction function F is generated.

The illusionary tactile force sense induction function F1713 generated by the illusionary tactile force sense induction function generator consists of the direction vector u(x, y, z) to which the force should be presented, the illusionary tactile force sense intensity II, the vibration sensation intensity VI, the response A phase pattern .theta.(t)=F(u, II, VI, R) for controlling the rotational acceleration/deceleration of the eccentric rotor is calculated from the characteristic R (P, I, D). However, P, I, and D represent the proportional gain, integral gain, and differential gain of PID control (specific calculation methods are shown in the example of FIG. 35).
Correction data 1714 obtained from the above-mentioned illusionary tactile force sense/equal sensation level curve and the user's personal illusionary tactile force sense/equal sensation level curve are stored in the user characteristic database 1711, and are read out by the corrector 1702 using this data. individual differences are corrected. The illusionary tactile force sense is determined by the contact pressure CP between the fingertip and the illusionary tactile force sense interface device 101, the gravitational effect PG due to the posture, and the inertia force FI due to the acceleration when the device is moved. S(CP, PG, FI)) are different. This sensitivity S is obtained by a subject experiment in advance and stored in the illusionary tactile force data 1710, and is corrected by adding the sensitivity S and correction data 1714 as a threshold increase of the illusionary tactile force/equal sensation level curve. is performed, and as a result, a corrected illusionary tactile force sense induction function is obtained.

FIG. 18 shows a processing flowchart of the illusionary tactile force sense device and the haptic force sense device.

In the illusionary tactile force sense device 107, the motor 1704 that presents the illusionary tactile force sense information is feedback-controlled based on the illusionary tactile force sense induction function and the illusionary tactile force sense function data 1710, thereby presenting the desired sensation.

The illusionary tactile force sense device 107 also has a function of presenting a tactile force sense (haptic force sense device). Simultaneously presenting the illusionary tactile force sense and the tactile force sense provides a synergistic effect of improving the texture.

FIG. 19 shows an example of control of the illusionary tactile force sense interface device 101 .

In this device, the control of the motor 1704 is divided into a motor feedback (FB) characteristic controller that controls the feedback characteristic of the motor 1704 and a control signal generator that converts the illusionary tactile force sense induction pattern into a motor control signal. In the present invention, it is essential to control the synchronization of the motor rotation phase pattern θ(t)=F(u, II, VI, R), and it is necessary to perform synchronization control with high temporal accuracy. Therefore, position control using a pulse train for controlling a servomotor is shown here as an example of a method. When a stepping motor is used for position control, sudden acceleration and deceleration often result in easy step-out and loss of control. Therefore, the pulse position control by the servomotor will be explained here. By separating the control of the motor feedback (FB) control characteristics and the motor control by the pulse position control method, in the present invention in which a large number of the illusionary tactile force sense interface devices 101 are synchronously controlled and used, the motor Consistency of control signals, speeding up of illusionary tactile force sense induced pattern generation, and scalability that can easily cope with an increase in the number of control motors to be synchronously controlled are ensured. In addition, it becomes easy to correct individual differences.

In the illusionary tactile force sense induction function generator 1701, the pulse signal train gi is separated into control signals for controlling the motor FB characteristics controller and the motor control signal generator, and the motor control signal generator controls the phase position of the motor. (t)=gi(f(t)) is generated to control the phase pattern θ(t) of the motor.

In this method, the rotation phase of the motor is feedback-controlled by the number of pulses. For example, one pulse rotates the motor by 1.8°. As for the direction of rotation, forward rotation or reverse rotation is selected by a direction control signal. By using this pulse control technique, arbitrary acceleration/deceleration patterns (rotational speed, rotational acceleration) are controlled at arbitrary phase timings while maintaining the phase relationship of two or more motors.

FIGS. 20(a) to 20(f) show an example of control of an illusionary tactile force device (tactile force device) that presents a basic tactile force sensation and an illusionary tactile force sensation.

FIG. 20(a) schematically shows a method for generating a rotational force in the illusionary tactile force sense device 107, and FIG. 20(d) schematically shows a method for generating a translational force. is. The rotations of the two eccentric weights 814 in FIG. 20(a) are rotated in the same direction with a phase delay of 180°. On the other hand, in FIG. 20(d), they are rotating in opposite directions.

(1) As shown in Fig. 20(b), when two eccentric rotors are synchronously rotated in the same direction with a phase delay of 180 degrees, the two eccentric rotors become point symmetric and the center of gravity coincides with the center of the rotation axis. As a result, rotation of equal torque without eccentricity is synthesized. This makes it possible to present a sense of rotational force. However, the time derivative of angular momentum is torque, and in order to continuously present torque in a certain direction, it is necessary to continuously accelerate the rotation speed of the motor. It is difficult to

(2) As shown in FIG. 20(c), by synchronously controlling the angular velocities .omega.1 and .omega.2, an illusionary tactile force sensation (continuous torque sensation) of continuous rotational force in a fixed direction is induced.

(3) As shown in FIG. 20(e), when synchronously rotating in the opposite direction at a constant angular velocity, by controlling the initial phase θi 1201, it is possible to synthesize a linearly oscillating force (single motion) in an arbitrary direction.

(4) As shown in FIG. 20(f), when synchronously rotating in opposite directions at angular velocities ω1 and ω2 according to the sensory characteristics related to the illusionary tactile force sensation, the illusionary tactile force sensation of a continuous translational force in a fixed direction. (continuous force sensation) is induced.

In the illusionary tactile force sense interface device 101, as shown in FIGS. Since the combination of (ω1, ω2) alone can induce an illusionary tactile force, the control circuit can be simplified.

FIG. 21 shows changes in sensory intensity related to the illusionary tactile force sense when the initial phase delay of the eccentric weight 814 of the illusionary tactile force sense device 107 is changed. FIGS. 21(a) and 21(b) show cases where there is no initial phase delay, and FIGS. 21(d) and 21(e) show cases where there is an initial phase delay. (d) schematically shows the phase relationship between the two eccentric weights 814. FIG. FIG. 21(c) is sensory characteristics showing the relationship between the vibration amplitude generated by the illusionary tactile force device and the sensory intensity of the illusionary tactile force sense induced by the illusionary tactile force device.

21(b) and 21(e) are cases where the initial phase delays during acceleration/deceleration of each eccentric rotor are 0° and -90°, and the synthesized acceleration/deceleration patterns are different. Since (e) can generate a greater intensity change (physical quantity (amplitude)) of acceleration/deceleration, a greater sensory intensity of the illusionary tactile force sense is presented. As shown in FIG. 21(f), by controlling the initial phase delay, the sensory intensity of the illusionary tactile force sense can be controlled.

FIG. 22 shows the nonlinear characteristics used in the illusionary tactile force sense interface device. )), which shows the hysteresis characteristics of the viscoelastic material (Fig. 22(d)).

FIG. 22(b) is a schematic diagram showing human sensory characteristics having a threshold value 2206 for physical quantities such as vibration and force, as in FIG. By controlling , it is shown that a sensation that does not exist physically is induced as an illusionary tactile force sensation.

As shown in Fig. 22(c), a material having a physical property showing non-linear stress characteristics with respect to an applied force is sandwiched between a device that generates a driving force such as vibration, torque, and force, and the human skin/sensory organs. Occasionally, a similar illusionary tactile force sensation is induced.

Also, as shown in FIG. 22(d), the sensory characteristics are often not isotropic and show hysteresis sensory characteristics when the displacement increases and decreases, such as when the muscles are stretched and contracted. When a muscle is pulled, it immediately contracts strongly. Generating such a strong hysteresis characteristic promotes induction of a similar illusionary tactile force sensation.

FIG. 23 shows an alternative device for the illusionary tactile haptic device 107 .

Instead of the eccentric weight 814 of the eccentric rotor in FIG. 23(a) and the eccentric motor 815 that drives it, weights 2302 and expansion members 2303 are used in FIGS. For example, FIGS. 23(b) and 23(d) show a plan view, a front view, and a side view of eight and four elastic members 2303 supporting the weight 2302, respectively. In each figure, the weight can be moved in an arbitrary direction by contracting/expanding a pair of elastic members 2303 . As a result, translational and rotational vibrations can be generated. Any structure can be used as a substitute as long as it has an acceleration/deceleration mechanism capable of generating and controlling translational movement of the center of gravity and rotational torque.

FIG. 24 shows control algorithms using different viscoelastic materials.

As shown in FIG. 24(g), materials (2403, 2404) having physical properties showing non-linear stress characteristics (FIG. 24(c)) with respect to applied force are used as devices for generating driving forces such as vibration, torque, and force. A similar illusionary tactile force sensation 905 is generated when sandwiched between human skin and sensory organs.

For example, as shown in FIG. 24(a), by attaching materials (2403, 2404) with different stress-deformation characteristics to the -90 to 90° phase region and the 90 to 270° phase region of the illusionary tactile force sense device surface, Even though the eccentric rotor rotates at a constant angular velocity (Fig. 24(b)), it can transmit the force transmitted through the viscoelastic deformable material non-linearly (Fig. 24(d)). As a result, in the same way as when the eccentric rotor is accelerated or decelerated, different forces (physical quantities) are presented in the -90 to 90° phase region and the 90 to 270° phase region, and the biased center of gravity position x (2402) changes. The force 905 of the illusionary tactile force sensation generated in one direction can be felt (FIG. 24g) with the addition of the non-linearity of the sensory characteristics (FIG. 24(f)). The direction of the force caused by the illusionary tactile force sense is determined by the positions where the different viscoelastic deformation materials are applied. As a result, energy consumption can be suppressed as compared with the method of accelerating and decelerating the rotation speed. In addition, when the number of revolutions is not constant but accelerated or decelerated, the viscoelastic material (2403, 2404) can increase the effect of illusionary tactile force. In addition, FIG.24(d) and FIG.24(e) are the same figures.

FIG. 25 shows the directions of the two different viscoelastic deformable materials in FIG.

FIGS. 25(a), (b), (c), and (d) show materials A and B having viscoelastic properties acting at operating points A and B in FIG. ) when material A is used in the phase 180-360° region and material B is used in the 0-180° region, (2) when material A is used in the phase 90-270° region and material B is used in the -90-90° region , (3) when material A is used in the phase 0 to 180 ° region and material B is used in the 180 to 360 ° region, (4) material A is used in the phase −90 to 90 ° region and material B is used in the 90 to 270 ° region Compatible when used. In FIG. 25(a), an upward illusionary tactile force works, and the user can get the feeling that the interface is floating. In FIG. 25(b), a leftward illusionary tactile force acts, and the user can feel that the interface is being pulled to the left. In FIG. 25(c), a downward illusionary tactile force works, and the user can feel that the weight of the interface has increased. In FIG. 25(d), a rightward illusionary tactile force acts, and the user can feel that the interface is being pulled to the right.

FIG. 26 shows a control algorithm using hysteresis material.

If the force-displacement hysteresis stress characteristics are different at the operating point B where the force increases and the operating point A where the force decreases, as in FIG. Varies according to stress characteristics. As a result, when the eccentric rotor is accelerated or decelerated as shown in FIG. 26(b), the hysteresis stress characteristic materials 2601 and 2602 in FIG. Acceleration/deceleration motion is generated by indicating the corresponding displacement, and the user having the sensory characteristics shown in FIG. As a result, each nonlinear effect enhances the nonlinear effect of the system as a whole, and a large illusionary tactile force is obtained. In this way, by inserting a material having hysteresis characteristics between a device that generates a driving force such as vibration, torque, or force and the human skin or sensory organs, the effect of acceleration and deceleration is enhanced, resulting in an illusionary tactile force sensation. increases the inductive effect of The case of having hysteresis stress characteristics as shown in FIG. 26(e) is similar to the case of FIG. 26(c). In addition, as in the case where the hysteresis stress characteristic material is attached to the surface of the illusionary tactile force sense device as shown in FIG. 26(a), as in FIG. good too.

FIG. 27 shows a control algorithm using a viscoelastic material whose properties change with applied voltage.

In the method using a viscoelastic material in FIG. 24, materials (2403, 2404) with different stress-deformation characteristics were attached, but as shown in FIG. may By controlling the applied voltage, the viscoelastic coefficient is changed (Fig. 27(b)), and the transmission rate of the periodically changing momentum generated by the eccentric rotor to the palm is determined from the rotation phase of the eccentric rotor. By synchronizing and changing, even if the eccentric rotor rotates at a constant rotational speed (constant speed rotation) as shown in FIG. Since the amount of momentum transmitted to the palm and fingertips can be controlled by changing the characteristic values at the operating point B and the operating point A over time, the same effect as accelerating or decelerating the rotation speed of the eccentric rotor can be obtained. In addition, this method has the same effect as artificially changing the physical characteristics of the skin, and has the effect of artificially changing the sensory characteristic curve (Fig. 27(e)). It can also be used for control to increase the induction efficiency of the illusionary tactile force sense. The viscoelastic material may be attached to the fingertip or body as in f), where the viscoelastic material can control the stress-strain characteristics non-linearly by applying voltage. In addition, the control method is not limited to control by applied voltage as long as nonlinear control can be performed.

As shown in FIG. 26(b), repeated acceleration and deceleration of the rotation of the motor causes a large loss of energy and heat generation. Since /f2 is a value close to 1 and the characteristics are changed by the applied voltage, the energy consumption of this method can be kept smaller than the energy consumption due to acceleration/deceleration of the motor.

FIG. 28 shows a control algorithm using an oscillator circuit.

FIG. 28(a) shows an example of an energy-efficient illusionary tactile force sense interface device using an oscillation circuit. In general, when the motor repeats acceleration and deceleration such as repeating high speed rotation 1002a and low speed rotation 1002b, a large energy loss and heat generation occur. Energy loss and heat generation are major obstacles for mobile and wireless applications. Therefore, energy consumption is suppressed by controlling the rotation speed of the eccentric rotation motor so as to generate an illusionary tactile force sensation (Fig. 28(b)) through an oscillation circuit that combines a coil, a capacitor, and a resistor. becomes possible. In particular, oscillation with nonlinear characteristics and hysteresis is desirable. The oscillator circuit shown in FIG. 28(a) is an example, and a combination of parallel circuits or the like, or an oscillator circuit using a semiconductor element for power control may be used.

Figures 29-1 to 29-3 show an apparatus using a plurality of basic units of an illusionary tactile force sense device according to the intended use of an application or controller.

FIG. 29-1(a) shows a basic unit of an illusionary tactile force sense device arranged facing each other.

FIG. 29-1(b) shows a basic unit of an illusionary tactile force sense device arranged facing each other.

FIG. 29-1(c) shows basic units of an illusionary tactile force sense device arranged in parallel.

FIG. 29-1(d) shows the basic units of an illusionary tactile force sense device arranged in parallel and facing each other.

FIG. 29-1(e) shows basic units of an illusionary tactile force sense device that are arranged facing each other and in parallel.
FIG. 29-1(f) shows basic units of an illusionary tactile force sense device arranged in parallel.

FIG. 29-1(g) shows basic units of an illusionary tactile force sense device arranged in parallel.

FIG. 29-1(h) shows basic units of an illusionary tactile force sense device arranged three-dimensionally at the vertices of a regular tetrahedron.

FIG. 29-1(i) shows basic units of an illusionary tactile force sense device that are arranged facing each other and in parallel.

FIG. 29-1(j) shows the basic units of an illusionary tactile force sense device arranged in parallel and facing each other.

FIG. 29-1(k) shows basic units of an illusionary tactile force sense device arranged in parallel.

FIG. 29-2(a) shows a basic unit of an illusionary tactile force sense device in which opposing types are two-dimensionally arranged.

FIG. 29-2(b) shows a basic unit of an illusionary tactile force sense device in which opposed type and parallel type are two-dimensionally arranged.

FIG. 29-2(c) shows a basic unit of an illusionary tactile force sense device in which opposed type and parallel type are two-dimensionally arranged.

FIG. 29-2(d) shows a basic unit of an illusionary tactile force sense device in which opposing types are arranged three-dimensionally.

FIG. 29-2(e) shows a basic unit of an illusionary tactile force sense device in which facing type and parallel type are arranged three-dimensionally.

FIG. 29-2(f) shows a basic unit of an illusionary tactile force sense device in which opposing types and parallel types are arranged three-dimensionally.

Figures 29-3(a) and 29-3(b) show the basic unit of an illusionary haptic device placed in a cylindrical game controller.

FIGS. 29-3(c) and 29-3(d) show the basic units of an illusionary tactile force sense device arranged three-dimensionally at twisted positions.

FIG. 29-3(e) shows the basic unit of the illusionary haptic device placed inside the game controller.

FIG. 30(a) shows that in addition to the illusionary tactile force sensation induced by the illusionary tactile force sense device, the shape 3001 of the illusionary tactile force sense interface device is deformed by the shape deformation motor 3002 in synchronization with the illusionary tactile force. shows a device that emphasizes the illusionary tactile force sensation 905 induced by .

For example, as shown in FIG. 30(b), when applied to a fishing game, the interface shape 3001 is warped according to the pulling of the fishing rod by the fish, thereby further emphasizing the tension sensation of the fishing line induced by the illusionary tactile force sensation 905. be done. At this time, if you just transform the interface without the illusionary tactile force sense, you will not be able to experience such a realistic fish pulling. Also, by spatially arranging the basic units of the illusionary tactile force sense device as shown in FIG.

The shape deformation is not limited to the shape deformation motor 3002, and any mechanism that can change the shape, such as a driving device using a shape memory alloy or a piezoelectric element, may be used.

FIG. 31 shows a virtual controller 3101 using the illusionary tactile force sense interface device 101 .

The virtual controller 3101 generated in the content creation device 102 is visualized in the palm of the hand using the audiovisual display 105 such as a hologram, autostereoscopic display, or head-mounted display, and can be touched. In terms of haptics, the illusionary tactile force sense interface device 101 is used to create a virtual controller 3101, and the presence, tactile sensation, and button operation sensation of the virtual controller are presented. Conventional methods using vibration could not express the shape of a virtual object tactilely. The returned reaction force is expressed.

Conventional game controllers allow the user to enjoy a sensory game by moving the user's own body, and were "pseudo-sensory" with no feedback based on haptic information except for vibrations. On the other hand, if the illusionary tactile force sense interface device 101 is used, it is possible to realize a "full bodily sensation type controller" that can tactilely touch the virtual object 531 or the game character.

The effect of the virtual controller 3101 using the illusionary tactile force sense interface device 101 is that the controller shape and button layout can be freely designed according to the content of the game. In particular, since the palms and fingers have different lengths depending on the age and gender, the virtual controller 3101 can be designed and transformed to match the palms of individuals. In addition, it is possible to form a shape that matches the content, or change the shape in accordance with the development of the story. For example, in conventional game controllers, game controllers adapted to game content have been released. On the other hand, when a single game controller is used to operate a wide variety of content, the controller may not be optimal for the content, making it impossible to operate intuitively, or the content creation may be restricted to match the game controller. I had a problem. On the other hand, in this implementation example, it is possible to virtually create a controller that matches the content, so there is no need to repurchase a dedicated controller, and the controller can be freely transformed according to the scenes and stories in the content.・Can be changed.

In particular, when new game software is released, virtual controller information can be included in the software, so a virtual controller optimized for the content of the game can be used. Since the virtual controller can be distributed as an item via the network, version upgrades and sales can be done at low cost and easily.

In the case of a real game controller, it is difficult to press the housing with the ring finger and the little finger while rapidly pressing multiple buttons in succession. Also, since there is no inertial force due to the weight of the game controller, the controller can be moved quickly. Conversely, the virtual controller 3101 based on the illusionary tactile force sense can generate the weight and inertial force of the controller as needed.

In conventional game controllers, all inputs are done with buttons on the controller. Therefore, when operating switches, door knobs, etc. in the VR space, they were selected and operated with the button of the controller. Therefore, it takes time for a user who is not accustomed to games to learn the functions and operation methods assigned to the buttons of the game controller, and to learn the operation methods for each game. However, with the virtual controller 3101, the functions of the game controller can be placed on the virtual buttons 3102 in the original VR space, so the buttons in the VR space can be directly operated by the user's familiar operation method. This eliminates the need for learning time and enables intuitive operation.

FIGS. 32-1 to 32-8 and 33 show an illusionary tactile force sense device and control method using one set of units or multiple sets of units. Figures 32-1 to 32-3 are for when one set of units is used, Figures 32-4, 32-5, 32-7, 32-8 and 33 are for when two sets of units are used is shown.

FIG. 32-1(a) schematically shows the phase relationship of the eccentric weights. FIG. 32-1(b) shows the phase pattern of rotation of the eccentric weight. FIG. 32-1(c) shows temporal changes in displacement of the center of gravity of the illusionary tactile force sense device synthesized with the phase pattern of FIG. 32-1(b). As shown in FIG. 32-1(c), by changing the phase delay θd indicating the timing of accelerating the rotation speed, the fundamental period of vibration (duration of operating point A + duration of operating point B) can be changed to The ratio of acceleration/deceleration on the plus side and minus side of the displacement of the center of gravity is controlled while being constant, as shown in FIG. 32-1(c). However, when θd is negative, it means phase lag, and when it is positive, it means phase lead. As a result, as shown in FIG. 32-1(d), it is possible to change the sensory intensity and direction of the illusionary tactile force sensation even if the period of vibration remains constant. When the phase delay θd=0 and π, the force direction of the illusionary tactile force sense is not felt, and is perceived as a mere vibration.

Further, as shown in FIG. 32-1(e), by changing the ratio of the duration of the operating point A and the duration of the operating point B (duration of the operating point A/duration of the operating point B), , changes the temporal transition of the displacement of the center of gravity as shown in FIG. 32-1(f). In other words, by changing the angular velocity ratio (duration of operating point B/duration of operating point A), even if the fundamental period of vibration and the maximum amplitude of displacement of the center of gravity remain constant, The sensory intensity of the illusionary tactile force sense can be changed as shown. As described above, it is possible to change the sensory intensity and texture of the illusionary tactile force sense while independently controlling the period, eccentric amplitude, and acceleration/deceleration.

32-2(a) to (h) show the phase relationship (phase θ: 0 to 7π/4) of the eccentric weight when the 0° and 180° directions in FIG. 32-1 are the vibration directions, It is a linear vibration that does not include rotational vibration. On the other hand, FIGS. 32-3(a) to 32-3(h) show the phase relationship (phase θ: π/2 to 9π/4) of the eccentric weight when the 90° and 270° directions are the vibration directions. , are linear vibrations including rotary vibrations. This rotational vibration dulls the sense of direction in the induction of the illusionary tactile force sense.

This rotational vibration can then be reduced by using two sets of units as shown in FIG. 32-4. FIG. 32-4(a) shows the phase relationship when the 0° direction is the direction of the illusionary tactile force sense. FIG. 32-4(b) shows the phase relationship when the 90° direction is the direction of the illusionary tactile force sense. FIG. 32-4(c) shows the phase relationship when the 180° direction is the direction of the illusionary tactile force sense. FIG. 32-4(d) shows the phase relationship when the 270° direction is the direction of the illusionary tactile force sense. Similarly, by increasing the number of units, rotational vibration can be reduced.

Here, when the phases θ1 and θ2 between the two sets of units are changed, as shown in FIGS. 32-5(c) and 32-5(d), by adjusting the phase difference θ2−θ1 , the sensory intensity of the illusionary tactile force can be changed.

As shown in Figure 32-6, by adjusting the phase relationship of multiple units, translational illusionary tactile force sensations (Figures 32-6(a) and 32-6(b)), rotational illusionary Haptic force sensations (FIGS. 32-6(c) and 32-6(d)) can be presented.

An energy efficient illusionary tactile force control device is also possible by using multiple sets of units, an example of which is shown in Figures 32-7 to 32-8.

Two pairs of illusionary tactile force sense devices 107a and 107b each composed of two eccentric rotors are prepared as shown in FIG. is rotated by ω ₀ and 2ω ₀ , the displacement of the center of gravity is synthesized as shown in FIG. 32-7(c). Especially when the phase is shifted by 90° (3203), the difference between the maximum value and the minimum value becomes maximum. As a result, even if each motor continues to rotate at a constant speed without using an oscillation circuit such as that shown in FIG. .

Here, such a synthesizing method is not limited to the case where the rotational speeds of the two basic units are ω ₀ and 2ω ₀ , but may be any relationship having a natural number ratio such as mω ₀ and nω ₀ (where m and n are natural numbers). Just do it.

On the other hand, as shown in FIGS. 32-7(d) to 32-7(f), by temporally changing the rotational speeds ω and 2ω of the motor, the same effect as in FIG. Obtainable. Also, as in FIG. 32-4, the direction of the illusionary tactile force sense can be selected as shown in FIG. 32-8.

FIG. 33 shows an illusionary tactile force sense device and control method using multiple units having eccentric weights of different weights. Although a plurality of the same eccentric weights are used in FIG. 32-7(d), the weight and shape of the eccentric weights may be different between the two sets as in FIG. 33(a). Furthermore, in the above-described method as well, by using this method using two pairs of illusionary tactile force sense devices, energy-efficient illusionary tactile force sense control becomes possible.

As shown in FIG. 34, the illusionary tactile force sense interface device 101 can be worn anywhere on the body 3400 by a mounting portion such as a housing having an adhesive tape or a finger insertion portion.

FIG. 35 shows, as an embodiment using a virtual reality environment generating device, a case where a plurality of users cooperate and perform virtual pottery between remote locations.

After all devices of the VR environment generating device A and the VR environment generating device B are calibrated, communication between the VR environment generating devices is ensured. Users exist in different spaces corresponding to respective VR environment generating devices, and information on each other's VR environment is shared via a communication device.

Sensing by the sensor will be described below with reference to FIG.

As data content data, the initial information of the model (the position Po of the model vertex) regarding the virtual clay mass is read from the content data 104 .

Next, information vector group Mu' (position Xu', posture Pu', velocity Vu', angular velocity Ru', acceleration Au', angular acceleration Tu') relating to each part of the user's body is detected by a plurality of position sensors 111 and acceleration sensors . is measured. Here, a position sensor that can also measure orientation information is used. Velocity, angular velocity, acceleration, and angular acceleration are obtained by differentiation and second-order differentiation of position information, and information from the acceleration sensor is used for fast movements. In the physics simulator 113, an information vector group Mo (position Xo, velocity Vo, acceleration Ao, force Fo acting between vertices) on the vertices of the virtual clay physical model, a virtual force vector group Fuo acting on the vertices from the user, Sound source data, information vectors Mu (position Xu, posture Pu, velocity Vu, angular velocity Ru, acceleration Au, angular acceleration Tu) related to the user model (virtual user), and virtual A memory space for storing the force vector group Fou is secured in the content creation device 102 . Based on the information vector group in the memory space that is updated from moment to moment, the physical simulation of the content virtual clay and the virtual user is repeated, and the information in the memory space is updated.

The physical simulation will be described below using the model in FIG.

In the physical simulator, the virtual clay is represented by the spring-damper model shown in FIG. 5(b), and the information vector groups Mu and Mo are calculated and updated. From the posture Pu1 of the virtual user's first measurement point p1 (for example, fingertip) and the virtual force vector Fou1 acting on this finger, the direction vector u1 of the force to be presented by the illusionary tactile force sense interface is
u1=Fou1/∥Fou1∥−Pu1
is asked. Other measurement points pi are similarly calculated.

As shown in FIG. 12, the initial phase .theta.i is obtained using the relationship u=(cos .theta.i, sin .theta.i, 0) between the initial phase .theta.i and the direction vector u to which the force should be presented. The initial phase lag θd is set to -90° which gives maximum perceived intensity. It should be noted that the initial phase delay θd may be adjusted according to the dynamic range of sensory intensity to be provided.

The above was the case of presenting a force in an arbitrary direction within a sliced cross-section of a fingertip using one set of illusionary tactile force sense devices. It can be extended to methods that present forces in any omnidirectional direction.

As for the physical intensity to be presented, the physical intensity corresponding to the illusionary tactile force sense intensity II to be presented is referenced using the numerical table representing the illusionary tactile force sense/equal sensation level curve in FIG. 17(b). The physical quantity Δf/f can be obtained from the characteristic graph of the intensity of the illusionary tactile force sense shown in FIG. 10(e). As the texture, the vibration sensation intensity VI representing the roughness sensation 1111 in FIG. 11(c) is obtained as a physical quantity f from the characteristic graph of the vibration sensation intensity in FIG. 10(f). Angular velocities ω1 and ω2 are obtained from these physical quantity Δf/f and physical quantity f. An interpolating function such as a spline function is used to obtain a value from the above characteristic curve. Angular velocities ω1 and ω2 are obtained as follows.
ω1= 2π/f1, ω2= 2π/f2 However, f1= f+Δf/2, f2= f－Δf/2
The phase pattern .theta.(t) is expressed using the initial phase .theta.i and the angular velocities .omega.1 and .omega.2 as shown in FIG. 12(b).

The P, I, and D parameters are selected for the response characteristic R of the motor, which do not cause vibration due to overshoot and have a good convergence response. The control method using P, I, D parameters is a servo motor control method generally used by the same industry, and the P, I, D parameters are selected according to the selection method provided by the motor manufacturer. When it is desired to emphasize the vibration sensation intensity VI representing the roughness sensation 1111, the parameters are fed back in the motor FB characteristics controller while monitoring with the acceleration sensor so that the P and D parameters increase so that vibration occurs. set.

As described above, the phase pattern θ(t) is obtained from the illusionary tactile force sense induction function F as f(t)=F(u, II, VI, R).

When the resolution of the motor control is 1.8°, the phase pattern θ(t) is used to divide the phase 360° of the vertical axis into 200 points in 1.8° increments. Find the time on the corresponding horizontal axis. This time is the timing for generating the control pulse train. As described above, the control pulse train g(t) is obtained from the phase pattern θ(t).

The difference between the deformable damper model and the spring damper model in FIG. 5(b) is that while the model in FIG. It is a point.

Another difference is that the length _L0 of the spring in the equilibrium state in FIG. It is to be updated as it is. However, if this process is repeated many times and the spring is folded and deformed like clay, the length of the spring will grow infinitely. Therefore, the lattice point division at the time of modeling is performed again so that the length of the spring becomes uniform each time the deformation occurs.

When grid point 1 is connected to adjacent grid points 2 to 4, the force vector f12 received by grid point 1 from grid point 2 is
f12 =－k×(∥p2－p1∥- L12)×(p2－p1)／∥p2－p1∥－c×(v2－v1) (9)
is represented. however,
pi: Position vector of grid point pi
vi: velocity vector of lattice point pi
k: modulus of elasticity of the spring,
c: damper viscosity coefficient,
Lij: Natural length of spring between grid point i and grid point j If grid point 1 with mass m1 receives force f1 from surrounding grid points 2 to 4, the equation of motion of grid point 1 is
m1×d2p1/ ^{dt2 =} f1=f12+f13+f14 (10)
is represented.

When a fingertip with an illusionary tactile force sense interface device touches grid point 1 (p1) of this virtual object/physical model, grid point 1 (p1) changes to position 1 (p'1) of the fingertip. , the reaction force (-f) acting on the fingertip is
－f=（f12+f13+f14）－m1×d2p'1/ ^{dt2 (} 11)
is represented. A fingertip movement for determining contact is sensed by a position sensor and an acceleration sensor.

In an actual numerical simulation, the position p'1, velocity v'1 and force f'1 of grid point 1 at time t' can be obtained from variables p,1 v1 and f1 one time before t. in short,
Velocity vector: v'1=v1+(f1/m1)×Δt (12)
Position vector: p'1=p1+v1×Δt (13)

Similarly, the position and velocity of grid point 2 with mass m2 are calculated.
Velocity vector: v'2=v2+(f2/m2)×Δt (14)
Position vector: p'2=p2+v2×Δt (15)
Finally, the force acting between grid point 1 and grid point 2 is different from that in Fig. 5(b).
f'12=0 (16)
is calculated as

By the physical simulation described above, the force acting on the virtual clay from the fingertip of the virtual user is calculated, and the virtual clay is deformed. Also, the stress acting on the virtual user's fingertip from the virtual clay is calculated. Based on the stress calculation results, the illusionary tactile force sense interface device is controlled by the illusionary tactile force sense induction device and the illusionary tactile force sense device drive control device in the presentation, so that the user (entity) can see on the audiovisual display. The virtual clay is deformed while confirming the shape of the virtual object by the feel, and the virtual vase is completed. At this time, if the virtual object A and the virtual object B are the same in the VR space, the virtual vase will be completed through collaborative work.

Note that the virtual object A and the virtual object B may be real objects, and the image and shape of the real object are measured by a peripheral device, and the result is transmitted to the VR environment generation device A and the VR environment generation device B via a communication device. Data shared. When the virtual object A is a real object, the user B shares the pottery experience of the user A.

The position, velocity, and force of each grid point are calculated for each calculation and stored in memory. This stored value is used to calculate position, velocity and force for the next time. With these, the reaction force to the fingertip is presented, and the virtual object is made haptic.

Similar to the physics simulation for virtual objects described above, the VR environment is based on real objects in real space sensed by peripheral devices and user movement information sensed by position sensors and acceleration sensors. It is modeled in the VR environment, the contact/grip force of the content is calculated, and the VR space in which the virtual space and the real space are fused is generated.

The virtual controller shown in FIG. 31 can also be implemented in the same manner as the virtual pottery described above.

This device can be applied to various fields other than the field of virtual reality.

In terms of information presentation and expression using virtual reality technology, there are many people who do not get motion sickness in real vehicles but get motion sickness in simulators, and there are many people who cannot feel the three-dimensional effect in stereoscopic virtual reality. However, the perception of reality differs greatly from person to person. In addition to this, physical differences such as palm size and muscle strength, differences in the weight and shape of the interface, and user proficiency in how to handle the interface can affect the gullibility of virtual reality technology. Feelings vary greatly between men and women of all ages. Therefore, the effects of learning and correction differ depending on the application.

If it is applied to information terminals such as mobile phones and PDAs, the information volume, comprehensibility and operability of tactile force sense information can be improved by matching individual characteristics.

For example, by using this device instead of a vibrator for silent mode, it will be possible to effectively display the direction of travel in navigation and alerts that are easy to overlook with tactile force, in contrast to conventional vibrations that do not have direction information. .

When an information terminal incorporates an illusionary tactile force device and a haptic force device, the relative relationship between the weight and shape of the terminal and the size and muscle strength of the palm will affect the strength and sensation of the haptic force. come. In addition, in the case of a non-base type that is used by holding it in the palm and shaking it, even if the same tactile information is presented, it feels different due to the inertial force due to the mass and moment of inertia. Therefore, the correction function of this device is effective for appropriately presenting the haptic information.

If it is used in a mobile phone game using a motion sensor, it is possible to effectively obtain force-based output in response to force-based input, thereby improving interactivity, reality, and intuitive operability. When used in touch pens (styluses) and tablet PCs, it improves the click feeling when icons are clicked with a finger or touch pen, and by changing the frictional resistance of each window in the display, it becomes possible to identify overlapping windows, resulting in visual impairment. Usability for users is improved.

Also, if it is applied to various training devices such as surgical simulators, it can be adjusted according to individual characteristics and learning levels, and information such as characteristic points to be learned and points that are easily overlooked can be haptically emphasized and expressed. This improves operability, comprehensibility, and learning effect.

Since the enhancement by illusion is included, it is necessary to correct the enhancement based on the human sensory characteristics instead of simply increasing the physical quantity or enhancing the contrast as in haptic information presentation. In addition, in order to express the variety of surgical instruments and the tools that are divided into beginner and expert tools, the sense of reality changes according to the frequency of use and proficiency, and the ease of deceiving with virtual reality technology. conduct.

101 illusionary tactile force sense interface device 102 content creation device 103 illusionary tactile force sense induction device 104 content data 105 audiovisual display 106 tactile force sense data and illusionary tactile force sense data 107 illusionary tactile force sense device 107a illusionary tactile force sense device 107b illusionary tactile sense Haptic device 108 Acceleration sensor 109 Pressure sensor 110 Myoelectric sensor 111 Position sensor 112 Controller 113 Physical simulator 114 Computer graphics 115 Illusory tactile force sense induction function generator 116 Learning device 117 Corrector 118 Peripheral device 119 Sound source simulator 205 Communication device 520 Virtual Objects (Physical Models)
528 Spring/damper physical model 531 Virtual object 533 Finger 535 Stress acting from virtual object 814 Eccentric weight 815 Eccentric motor 901 Physical phenomenon 902 Nonlinear sensory characteristics 903 Psychological phenomenon 904 Left-right vibration 905 Illusory tactile force sensation 908 Duration time Ta of operating point A Integral amount 909 of force sensation at Integral amount 1002a of force sensation at duration Tb of operating point B High-speed rotation speed ω1
1002b Low speed ω2
1100 Virtual plate 1101 Movement of virtual object 1102 Drag against movement 1103 Drag from virtual plate 1104 Frictional force 1105 Viscous drag 1106 Drag pushing back virtual plate into surface 1107 Error thickness of virtual plate 1108 Sense of resistance/viscosity 1109 Sense of friction 1110 Smooth Sensation/acceleration sensation 1111 Roughness sensation 1112 Vibrational drag 1113 Acceleration force (negative drag)
1201 initial phase θi
1202 Levitation sensation 1203 due to illusionary tactile force sense Levitation sensation 1204 due to illusionary tactile force sense Gravity sensation 1301 due to illusionary tactile force sense Adhesive tape 1302 Housing 1303 Finger insertion part 1403 Shape deformation material 1404 Viscoelastic material 1405 Anti-seismic material 1406 Display 1702 Corrector 1703 Motor controller 1704 Motor 1705 Encoder 1706 Viscoelastic property controller 1707 Viscoelastic material 1713 Illusory tactile force sense induction function 1714 Correction data 1725 Physical strength 15 dB
1901 Motor FB characteristic controller 1902 Control signal generator 2206 Threshold value 2302 Weight 2303 Elastic material 2400 Constant speed rotation 2403 Viscoelastic material A
2404 Viscoelastic Material B
2601 hysteresis material A
2602 hysteresis material B
3001 Illusory tactile force sense interface device shape 3002 Shape deformation motor 3003 Flexible deformation material 3101 Virtual controller 3102 Virtual button 3202 Center of gravity displacement (phase difference 0°)
3203 center of gravity displacement (phase difference 180°)

Claims (40)

An illusionary tactile force sense interface device comprising an illusionary tactile force sense device and a CPU ,
(1) Data comprising at least one of a physical quantity including momentum presented by the illusionary tactile force sense device, a characteristic indicating the relationship between a stimulus and human senses, and acquired content information. and/or (2) data comprising operation information for the illusionary tactile force sense device;
controlling the CPU according to to induce an illusionary tactile force sense having a nonlinear sensation;
Here, by changing at least one of the shape, direction, phase, position, velocity, angle, angular velocity, angular acceleration, arrangement, and arrangement of at least some elements of the illusionary tactile force sense interface device, the non-linear sensation is generated. The illusionary tactile force sense provided is induced,
The illusionary tactile force sense interface device presents a sense of force in a direction opposite to the direction of intermittent force included in the induced illusionary tactile force sense,
The illusionary tactile force sense interface device, wherein presenting the sense of force in the opposite direction includes presenting a shape of a virtual object, a sense of friction, and a sense of roughness. An illusionary tactile force sense interface device comprising a CPU ,
(1) data comprising information on at least one of a real object, a virtual object, and a displayed object; and/or (2) data comprising operation information for at least one of a real object, a virtual object, and a displayed object. controlling the CPU according to to induce an illusionary tactile force sense having a nonlinear sensation;
Here, by changing at least one of the shape, direction, phase, position, velocity, angle, angular velocity, angular acceleration, arrangement, and arrangement of at least some elements of the illusionary tactile force sense interface device, the non-linear sensation is generated. The illusionary tactile force sense provided is induced,
The illusionary tactile force sense interface device presents a sense of force in a direction opposite to the direction of intermittent force included in the induced illusionary tactile force sense,
The illusionary tactile force sense interface device, wherein presenting the sense of force in the opposite direction includes presenting the shape of the virtual object, the sense of friction, and the sense of roughness. 3. The illusionary tactile force sense interface device according to claim 1, wherein the induction is generated by driving an illusionary tactile force sense induction function generation device that generates an illusionary tactile force sense induction function. 3. The induction is generated by driving an illusionary tactile force sense induction function generator that generates an illusionary tactile force sense induction function and/or a corrected illusionary tactile force sense induction function. The illusionary tactile haptic interface device described. The induction is obtained by driving the illusionary tactile force sense device, and/or is obtained by changing the momentum and/or the acceleration pattern according to the non-linear sensory characteristics, and/or is obtained by changing the momentum based on the data. 2. The illusionary tactile force sense interface device according to claim 1, wherein the illusionary tactile force sense interface device is obtained by changing the change and/or acceleration pattern. The illusionary tactile force sense interface device
1. Provided in any one of a driving device, an audiovisual display, a simulator , a game machine, a panel , a button, a pen, a training device, a sensory device, a writing utensil, daily necessities, and a toy. 3. The illusionary tactile force sense interface device according to 2. 5. The illusionary tactile force sense interface device according to any one of claims 1 to 4, wherein the device presents a feeling that an arm holding the interface is lifted. 5. The illusionary tactile force sense interface device according to any one of claims 1 to 4, wherein the device is a device for presenting the existence of a virtual object. 5. The illusionary tactile force sense interface device according to any one of claims 1 to 4, wherein the device is a device for presenting pressing of a virtual object. 5. The illusionary tactile force sense interface device according to any one of claims 1 to 4, wherein the device is a device that presents bodily sensations and/or acceleration sensations. 5. The illusionary tactile force sense interface device according to any one of claims 1 to 4, wherein the illusionary tactile force sense interface device presents a sensation of force acting. 5. The illusionary tactile force sense interface device according to any one of claims 1 to 4, characterized in that it is a device that presents sensations by freely designing the shape of the controller and the arrangement of the buttons according to the palm of the individual . . 5. The illusionary tactile force sense interface device according to any one of claims 1 to 4, wherein the device presents sensations by freely transforming and changing contents . 5. The illusionary tactile force sense interface device according to any one of claims 1 to 4, wherein the device presents a virtual reality environment in which a virtual space and a real space are fused. A claim characterized by being a device that coexists and shares information with a plurality of users in the same VR environment by sharing content and sensing information through communication , and presents the operation and/or feel of a virtual object in a shared manner. 5. The illusionary tactile force sense interface device according to any one of items 1 to 4. 5. The illusionary tactile force sense interface device according to any one of claims 1 to 4, wherein the device presents a sensation of being pressed from the outside . 5. The illusionary tactile force sense interface device according to any one of claims 1 to 4, wherein the device is a device that presents a continuous translational force sensation. Although the integral of the force generated by acceleration/deceleration and vibration is physically zero, not only is it not sensuously canceled, but the force in the positive direction is not perceived, and the force in the negative direction, which is the target direction, is not perceived. 5. The illusionary tactile force sense interface device according to claim 1, wherein the illusionary tactile force sense interface device is a device that presents a sense of translational force or torque. 5. The illusionary tactile force sense interface device according to any one of claims 1 to 4, wherein the device presents a sensation of fluctuation in the intensity and/or cycle and direction of stimulation . The illusionary tactile force sense interface device according to any one of claims 1 to 4, wherein the illusionary tactile force sense interface device is a device that presents the illusionary tactile force sense by changing the texture of the force . sensory interface device. 5. The illusionary tactile force sense interface device according to any one of claims 1 to 4, wherein the device presents a sense of friction and a sense of roughness, which are the textures of a virtual flat plate. 5. The illusionary tactile force sense interface device according to any one of claims 1 to 4, wherein the device suppresses reduction and/or inhibition of floating sensation and presents it. 5. The illusionary tactile force sense interface device according to any one of claims 1 to 4, wherein the device presents an illusionary tactile force sense to the tactile sensation of a real object. 5. The illusionary tactile force sense interface device according to any one of claims 1 to 4, wherein the device presents complex sensations to which a virtual touch is added while holding and/or touching a real object. 5. The illusionary tactile force sense interface device according to any one of claims 1 to 4, wherein the device edits and presents gripping and/or contact feeling of a real object. 5. The illusionary tactile force sense interface device according to any one of claims 1 to 4, wherein the device presents the shape of a remote object by holding a hand over the object. 5. The illusionary tactile force sense interface device according to any one of claims 1 to 4, wherein the device is a device that continuously presents force in a fixed direction to give a sense of gravity and/or a sense of mass. 5. The illusionary tactile force sense interface device according to any one of claims 1 to 4, wherein the device presents the presence and feel of a virtual object in a fingertip and/or palm. 5. The illusionary tactile force sense interface device according to any one of claims 1 to 4, wherein the device presents the presence, shape and tactile sensation of a virtual object. 3. The illusionary tactile force sense interface device according to claim 1, wherein the illusionary tactile force sense interface device is driven by gesture input or operation. 3. The illusionary tactile force sense interface device according to claim 1 or 2, wherein a VR space in which a real space and a virtual space are fused is generated. 5. The device according to any one of claims 1 to 4, wherein the device presents sensations based on measurement of the position, posture, acceleration, pressure between the interface and the skin, and/or myoelectric potential of the illusionary tactile force sense interface device . 2. The illusionary tactile force sense interface device according to 1 or 2. 5. The illusionary tactile force sense interface device according to any one of claims 1 to 4, wherein the audio- visual display presents the sensation of a virtual object that has been transformed into a stereophonic image and/or a stereoscopic image. 5. The illusionary tactile force sense interface device according to any one of claims 1 to 4, wherein the device replaces the feel of a real object with the feel of a virtual object and presents it. 5. The illusionary tactile force sense interface device according to any one of claims 1 to 4, wherein the device presents a sense of measured stress and/or shear force and surface shape. 5. The illusionary tactile force sense interface device according to claim 1, wherein the illusionary tactile force sense interface device presents a sensation of the surface shape displayed on the display . 5. The illusionary tactile force sense interface device according to any one of claims 1 to 4, wherein sensations are presented based on created CG. 5. The illusionary tactile force sense interface device according to any one of claims 1 to 4, wherein the device presents a sensation of being touched by a virtual object or a game character. 5. The illusionary tactile force according to any one of claims 1 to 4, wherein the illusionary tactile force sense interface device is a device that displays buttons on a display and/or presents command contents of the buttons. sensory interface device. 5. The illusionary tactile force sense interface device according to any one of claims 1 to 4, wherein the contents are visualized and/or sound-imaged by computer graphics and a sound source simulator and displayed on an audiovisual display.

Images