JP2023148856A - Control device, control method, haptic feedback system, and computer program - Google Patents

Abstract

To provide a control device, a control method, a haptic feedback system, and a computer program, configured to allow an operator to recognize completion of presentation of a haptic sensation.SOLUTION: A control device is for a haptic feedback device having a displacement unit that is provided to be displaced in response to an operation by an operator. The haptic feedback device is configured to generate a force sense for operation on the displacement unit according to a displacement amount of the displacement unit and present a haptic sensation of a displayed object. The control device preliminarily stores, for each object, control data for generating a reaction force and a completion displacement amount to complete presentation of the haptic sensation presented by the displacement unit, in association with the displacement amount of the displacement unit, outputs, to the haptic feedback device, the control data and the completion displacement amount associated with the displacement amount of the displacement unit, and completes presentation of the haptic sensation of the haptic feedback device when the displacement amount of the displacement unit reaches the completion displacement amount.SELECTED DRAWING: Figure 7

Description

The present invention relates to a control device, a control method, a tactile presentation system, and a computer program that allow an operator to recognize the end of tactile presentation by a tactile presentation device.

2. Description of the Related Art Among the five human senses, image and audio transmission technologies related to visual and auditory senses are becoming more precise, and in recent years, various technologies for presenting tactile sensations (haptics) have been proposed.

Tactile presentation technology uses mainly three types of vibrations (ERM: Eccentric Rotating Mass, LRA: Linear resonant actuator, piezo element) to reproduce the vibrations that occur when touching an image of an object displayed on a display with a built-in touch panel. Proposed. However, when presenting tactile sensations on a touch panel, it is difficult to reproduce deeper tactile sensations, that is, not only the skin of fingers but also the force sensations generated in muscles and tendons.

Based on knowledge of magnetorheological fluids, the inventors have proposed a tactile presentation device that can be operated by an operator and reproduces different sensations depending on the object (Patent Document 1, etc.).

Patent No. 6906275

The tactile presentation device includes a displacement section that is a curved plate or the like that can be displaced relative to the base by an operation of an operator. The displacement part can be displaced by the operator without resistance when the tactile presentation device is not controlled, but the tactile presentation device under control can move the displacement part according to the amount of displacement (position/angle) of the displacement part. The tactile sensation is presented by changing the magnitude of the force sensation generated, the output method of the force sensation, etc.

A tactile presentation device is used while visually or audibly outputting what a tactile object is. Some objects have an irreversible structure that does not return to its original shape and breaks after the displacement part is displaced. In this case, if the operator is allowed to continue the operation after being presented with the feeling that the object will break with one operation, the operator will feel uncomfortable.

The present invention has been made in view of the above circumstances, and aims to provide a control device, a control method, a tactile presentation system, and a computer program that allow an operator to recognize the end of tactile presentation. .

A control device according to an embodiment of the present disclosure includes a displacement section that is displaceable in response to an operation by an operator, and generates a force sensation in response to an operation on the displacement section according to the amount of displacement of the displacement section. A control device for a tactile presentation device that presents a tactile sensation of an object to be displayed, the control device comprising control data for generating the force sensation in association with a displacement amount of the displacement portion, and presentation of the tactile sensation by the displacement portion. The end displacement amount at which the displacement portion ends is stored for each object, and the control data and the end displacement amount associated with the displacement amount of the displacement portion are outputted to the tactile sense presentation device, and the displacement amount of the displacement portion is outputted to the tactile sense presentation device. When the amount reaches the end displacement amount, the tactile presentation in the tactile presentation device is ended.

A control method according to an embodiment of the present disclosure includes a displacement section that is displaceable in response to an operation by an operator, and generates a force sensation in response to an operation on the displacement section according to a displacement amount of the displacement section. A control method for a tactile presentation device that presents a tactile sensation of a displayed object by causing a computer connected to the tactile presentation device to generate the haptic sensation in association with a displacement amount of the displacement portion. Control data and an end displacement amount for ending the presentation of a tactile sensation by the displacement section are stored for each object, and the control data and end displacement amount associated with the displacement amount of the displacement section are stored in the tactile sense. When the amount of displacement of the displacement portion reaches the end displacement amount, the tactile presentation in the tactile presentation device is ended.

A computer program according to an embodiment of the present disclosure includes a displacement section that is displaceable in response to an operation by an operator, and generates a force sensation in response to an operation on the displacement section according to a displacement amount of the displacement section. a computer connected to a tactile presentation device that presents a tactile sensation of an object to be displayed, control data for causing the force sensation in association with the amount of displacement of the displacement portion, and presentation of the tactile sensation by the displacement portion; The end displacement amount at which the displacement portion ends is stored for each object, and the control data and the end displacement amount associated with the displacement amount of the displacement portion are outputted to the tactile sense presentation device, and the displacement amount of the displacement portion is outputted to the tactile sense presentation device. When the amount reaches the end displacement amount, a process of ending the tactile presentation in the tactile presentation device is executed.

In the control device, control method, and computer program of the present disclosure, when the displacement amount of the displacement portion reaches the end displacement amount, the tactile presentation ends. The operator can recognize the end of the tactile presentation by executing a process in which the force sensation generated in the displacement part becomes a specific torque or zero, or otherwise indicates the end.

A control device according to an embodiment of the present disclosure includes control data for causing a predetermined force sensation to be generated in the displacement portion or a force sensation generated in the displacement portion after the displacement portion reaches an end displacement amount for each target object. Control data for setting the force sensation to zero is stored, and when the displacement amount of the displacement portion reaches the end displacement amount, the predetermined force sensation is generated or the force sensation is generated in the displacement portion. Makes the sense of force zero.

In the control device of the present disclosure, when the displacement amount of the displacement section reaches the end displacement amount, the tactile presentation ends. The force sensation generated at the displacement part becomes a specific torque or becomes zero. This allows the operator to recognize the end of the tactile presentation.

The control device according to an embodiment of the present disclosure causes the display unit to display an image indicating the end, or the audio output unit to make a sound indicating the end, when the displacement amount of the displacement unit reaches the end displacement amount. output.

In the control device of the present disclosure, when the tactile presentation ends when the end displacement amount is reached in the tactile presentation device, the operator is notified of the end by an image displayed together with the tactile presentation or an audio outputted together. This allows the operator to recognize the end of the tactile presentation.

A tactile presentation system according to an embodiment of the present disclosure includes a displacement section that is displaceable in response to an operation by an operator, and generates a force sensation in response to an operation on the displacement section according to the amount of displacement of the displacement section. a tactile presentation device that generates and presents a tactile sensation of a displayed object; and an information processing device that is communicatively connected to the tactile presentation device and includes a display section, and the information processing device is configured to detect a displacement amount of the displacement section. control data that causes the force sensation and an end displacement amount that ends presentation of the tactile sensation by the displacement section are stored for each object in association with the displacement amount of the displacement section. outputting the control data and the end displacement amount to the tactile sense presentation device, and when the displacement amount of the displacement section reaches the end displacement amount, the tactile sense presentation in the tactile sense presentation device is ended, and the display section shows: Display an image indicating completion.

In the tactile presentation system of the present disclosure, when the displacement amount of the displacement section reaches the end displacement amount, the tactile presentation ends and the end is notified at least on the display section, so the operator recognizes the end of the tactile presentation. It becomes possible.

According to the present disclosure, it becomes possible for the operator to recognize the end of the tactile presentation by the tactile presentation device, and it is possible to improve the degree of tactile perception.

FIG. 1 is a schematic diagram showing a tactile presentation system. FIG. 1 is a block diagram showing the configuration of an information processing device. It is an explanatory view showing an example of contents of a sensory DB. FIG. 2 is a block diagram showing the configuration of a tactile presentation device. It is a flowchart which shows an example of the processing procedure of the tactile sense presentation in a tactile sense presentation system. It is a flowchart which shows an example of the processing procedure of the tactile sense presentation in a tactile sense presentation system. FIG. 2 is a schematic diagram showing an example of tactile presentation in the tactile presentation system. FIG. 7 is a schematic diagram showing another example of tactile presentation in the tactile presentation system. FIG. 2 is a schematic diagram of a tactile presentation system in a second embodiment. It is a block diagram showing the composition of HMD of a 2nd embodiment.

The present disclosure will be specifically described with reference to drawings showing embodiments thereof. In the following embodiments, implementation of a setting method in a tactile presentation system will be described.

(First embodiment)
FIG. 1 is a schematic diagram showing a tactile presentation system 100. The tactile presentation system 100 includes an information processing device 1 and a tactile presentation device 2. The information processing device 1 and the tactile presentation device 2 are communicably connected by short-range wireless communication and exchange data with each other.

The information processing device 1 uses a smartphone as shown in FIG. The information processing device 1 may be a tablet terminal or a laptop-type PC (Personal Computer) instead of a smartphone.

The tactile presentation device 2 is a device that an operator can operate by moving his or her finger while holding the finger along the displacement portion 202 . The tactile presentation device 2 reads the position of the displacement section 202 that is displaced by the operator's finger movement, controls the built-in MRF (Magneto-Rheological Fluid) device 24 according to the position, and moves the displacement section 202 of the operator. This is a device that produces a sense of force using a reaction force (rotational resistance) to the operation of the machine, and presents a tactile sensation. The form of the displacement part 202 of the tactile presentation device 2 is not limited to that shown in FIG. 1, but may be stick-like or cushion-like covered with a cover. The tactile presentation device 2 may employ a motor, a piezo element, or the like instead of the MRF device 24, and may generate a force sensation by rotational force or vibration in response to an operation by the operator. It may be combined with something that provides a warm or cold sensation. The structure may be such that it is installed on the ground or a wall and is operated by the operator's palm or foot.

In the tactile presentation system 100, the tactile presentation device 2 cooperates with the information processing device 1 and displays the tactile sensation of the object while outputting images and sounds of the object using the information processing device 1 or other display devices, speakers, etc. is outputted by the tactile presentation device 2. In the example shown in FIG. 1, when a soft object is displayed on the display unit 13 of the information processing device 1 and the operator pushes the displacement part 202 of the tactile presentation device 2 with a finger, the tactile presentation device 2 becomes squishy or fluffy. While outputting a tactile sensation, the display unit 13 displays an image that changes as if the object is being pushed in, and the audio output unit 14 outputs an audible “squeak” sound that corresponds to the object being pushed in. Output audio.

FIG. 2 is a block diagram showing the configuration of the information processing device 1. As shown in FIG. The information processing device 1 includes a processing section 10, a storage section 11, a communication section 12, a display section 13, an audio output section 14, and an operation section 15. The processing unit 10 is a processor using a CPU (Central Processing Unit) and/or a GPU (Graphics Processing Unit). The processing unit 10 executes a process described below based on a control program P1 for tactile presentation stored in the storage unit 11.

The storage unit 11 uses nonvolatile memory such as flash memory or SSD (Solid State Drive). The storage unit 11 stores data referenced by the processing unit 10. The control program P1 (program product) is downloaded from the information processing device 1 or another program server device via the communication unit 12 and stored in an executable manner. The control program P1 stored in the storage unit 11 may be a computer program P8 stored in the computer-readable storage medium 8, which is read out and stored by the processing unit 10.

The storage unit 11 stores a sensory database (DB) 110 including tactile data, visual data, and auditory data of the object to be output. The sensory DB 110 stores tactile data, visual data (images, videos), and auditory data to be output at each displacement for each displacement data of a displacement part in the tactile presentation device 2, in association with an object ID that identifies the object. (audio) (see Figure 3). The storage unit 11 further stores setting data regarding the tilt of the tactile presentation device 2, which will be described later.

The communication unit 12 is a communication module for short-range wireless communication, for example, Bluetooth (registered trademark). The processing unit 10 can transmit and receive data to and from the tactile presentation device 2 through the communication unit 12 .

The display unit 13 is a display such as a liquid crystal display or an organic EL (Electro Luminescence) display. The display unit 13 is, for example, a display with a built-in touch panel. The processing unit 10 displays, on the display unit 13, an operation screen for causing the tactile sensation presentation device 2 to output a tactile sensation and an image of the target object, based on the control program P1.

The audio output unit 14 includes a speaker and the like. The processing unit 10 causes the audio output unit 14 to output the sound of the object, music, etc. based on the control program P1.

The operation unit 15 is a user interface that allows input and output to and from the processing unit 10, and is a touch panel built into the display unit 13. The operation unit 15 may be a physical button. The operation unit 15 may be a voice input unit.

FIG. 3 is an explanatory diagram showing an example of the contents of the sensory DB 110. As shown in FIG. 3, the sensory DB 110 stores the current value of the current to the MRF device of the tactile presentation device 2 for each displacement amount (angle) of the displacement part 202 as tactile data in association with the object ID. There is. It is not limited to a current value, but may be a voltage value or the like. The tactile data includes, for each object, a displacement amount (hereinafter referred to as an end displacement amount) that is a condition for ending the tactile presentation of the object, and data on a control current value after reaching the end displacement amount.

Note that there are multiple objects in the sensory DB 110, such as real objects such as still life objects such as balls or balloons, foods such as gummies, vegetables, or fruits, and animals such as dogs, cats, or fish. It may also include characters such as slimes and monsters. Therefore, depending on the object, it is possible to reversibly present a sensation by having an elastic force, so that the end displacement amount and the control current value after reaching the end displacement amount do not need to be associated with each other.

The relationship between the amount of displacement of the displacement section 202 of the sensory DB 110 and the current value is set according to the actual size of the movable range of the displacement section 202 and the size of the object. For example, if the movable range of the displacement unit 202 is 50 mm, and the object is a sphere that bursts like a balloon, rotational resistance that gives a feeling of bursting when a certain displacement amount, such as 30 mm, is reached. A current value is set that causes In addition, if the target object is a large object such as the seat surface of a sofa, and the size greatly exceeds the movable range of the displacement unit 202, the tactile data may be such as to present the sensation up to 50 mm from the surface of the sofa seat surface. The current value is set. If the target object is a virtual character such as a monster, the current value is set based on the size according to the setting of the character.

Similarly, the sensory DB 110 stores images (frame images) for each displacement amount (angle) of the displacement portion 202 as visual data in association with the object ID. Here, the frame image is one still image that is recognized as an animation image when displayed continuously. Similarly, audio for each displacement amount (angle) of the displacement unit 202 is stored as auditory data in association with the object ID. The auditory data may be waveform data that differs depending on each angle. It may also be a timestamp of the audio corresponding to each angle.

Regarding visual data and auditory data, for objects that irreversibly present a tactile sensation, the image and sound that should be displayed at the time when the end displacement amount is reached and after the end of the tactile sensation correspond to the object ID, respectively. It is attached and memorized.

FIG. 4 is a block diagram showing the configuration of the tactile presentation device 2. As shown in FIG. As shown in FIG. 1, the tactile presentation device 2 is configured by providing a flat bottomed cylindrical grip 200 with a band-shaped flat plate displacement portion 202 having a curved portion partially along the circumferential direction. Although the displacement part 202 is made of a flexible material, it may also be made of a highly rigid material and rotatably supported via the grip body 200 and a support shaft. A binding tool 203 in the form of a cloth tape is provided on the outer surface of the distal end of the displacement portion 202 . A link mechanism 204 is provided on the inner surface of the distal end of the displacement portion 202 to connect to the rotating shaft of the rotor of the MRF device 24 housed inside the gripper 200 .

As shown in FIG. 1, the operator uses the holding body 200 by, for example, holding the holding body 200 with the thumb and middle finger, and inserting the finger such as the index finger into the binding tool 203 along the displacement part 202. The operator can move the displacement part 202 by pushing in the index finger, or can move the displacement part 202 away from the grip body 200 by extending the index finger.

The tactile presentation device 2 includes a gripping body 200 as shown in FIG. The grip body 200 has an MRF device 24 built therein. The control section 20, the storage section 21, the communication section 22, and the power supply section 23 may be provided integrally with the grip body 200, or may be provided as separate bodies connected to the grip body 200 wirelessly or by wire.

The control unit 20 includes a processor such as a CPU and an MPU (Micro-Processing Unit), and a memory such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The control unit 20 is, for example, a microcontroller. The control unit 20 controls each component based on a control program P2 stored in the built-in ROM to realize tactile presentation.

The storage unit 21 is an auxiliary storage memory for the control unit 20, and stores control data (tactile data) of the MRF device 24 in a rewritable manner.

The communication unit 22 is a communication module for short-range wireless communication, for example, Bluetooth (registered trademark). The control unit 20 can transmit and receive data to and from the information processing device 1 through the communication unit 22 .

The control unit 20 is connected to a power supply unit 23, an MRF device 24, and a sensor 25 via I/O, and exchanges signals with each other.

Power supply unit 23 includes a rechargeable battery. The power supply unit 23 supplies power to each component and the MRF device 24 when turned on.

The MRF device 24 has a yoke that sandwiches a disk-shaped rotor with a gap between them, and generates a magnetic field by passing a control current through a coil provided in the yoke, thereby generating a magnetic field sealed in the gap. The viscosity (shear stress) of the viscous fluid is controlled to provide rotational resistance to the rotor. When the control unit 20 controls the magnitude of the control current to the MRF device 24, the rotational resistance is immediately changed.

The sensor 25 measures the amount of displacement (angle) of the displacement section 202 and outputs it to the control section 20 . The sensor 25 measures and outputs the displacement of the displacement portion 202 as an angle. The sensor 25 may include a plurality of sensors such as a gyro sensor and an acceleration sensor.

In the tactile presentation device 2 configured as described above, when the displacement section 202 is operated by the operator, the displacement of the displacement section 202 is caused to move in the direction of rotation of the MRF device 24 toward the rotation axis of the rotor via the link mechanism 204. communicated. Since the rotating shaft rotates freely when the MRF device 24 is not operating, that is, while the control current is zero, the displacement portion 202 fluctuates without resistance. On the other hand, when the MRF device 24 operates and the control current is not zero, the viscosity (shear stress) of the magnetorheological fluid inside the MRF device 24 is changed depending on the magnitude of the current flowing to the MRF device 24. By continuously changing the magnitude of the current to the MRF device 24 or by vibrating the current value at a predetermined frequency, the control unit 20 can change the resistance force applied to the displacement unit 202 and the method of its appearance. .

In this way, the tactile sensation presentation device 2 can vary the resistance (current value) according to the pushing amount (displacement amount) of the displacement section 202 to present a smooth tactile sensation, or increase the resistance as the pushing amount increases. It is possible to present a tactile sensation that is hard and hard, or to present a tactile sensation that is crunchy by repeatedly increasing and decreasing the resistance.

The tactile presentation device 2 is controlled so that when the displacement amount of the displacement unit 202 reaches the end displacement amount, the presentation of the tactile sensation ends and is recognized as having ended. As described with reference to FIG. 3, the end displacement amount is stored in the sensation DB 110 for each object.

5 and 6 are flowcharts illustrating an example of a tactile presentation processing procedure in the tactile presentation system 100. When the operator starts the control program P1 and turns on the power of the tactile presentation device 2, the processing unit 10 of the information processing device 1 starts the following process in cooperation with the tactile presentation device 2.

The processing unit 10 displays on the display unit an operation screen including a list of object candidates for which tactile sensation is to be displayed (step S101), and accepts selection of the object (step S102). The processing unit 10 reads the data of the tactile data, visual data, auditory data, end displacement amount, and control current value after the end corresponding to the object ID of the selected object from the sensory DB 110 of the storage unit 11 and temporarily stores the data. It is stored (step S103).

The processing unit 10 establishes communication between the tactile presentation device 2 and the communication unit 12 (step S104), and transmits the temporarily stored tactile data to the tactile presentation device 2 (step S105). The processing unit 10 transmits data of the end displacement amount read in step S103 and the control current value after the end to the tactile presentation device 2 (step S106).

The processing unit 10 causes the display unit 13 to display that tactile presentation is to be started (step S107).

The control unit 20 of the tactile presentation device 2 receives the tactile data and stores it in the storage unit 21 (step S201), and receives the data of the end displacement amount and the control current value after the end and stores it in the storage unit 21. (Step S202).

The control unit 20 samples a signal corresponding to the displacement amount (angle) of the displacement unit 202 output from the sensor 25 (step S203). The control unit 20 determines whether the displacement amount obtained by sampling has reached the end displacement amount stored in step S202 (step S204). If it is determined that the end displacement amount has been reached (S204: YES), the control unit 20 notifies the information processing device 1 that the end displacement amount has been reached (step S205).

The control unit 20 outputs the current of the control current value after the end to the MRF device 24 (step S206), returns the process to step S206, and continues to flow the current of the control current value. The process in step S206 continues until a termination operation is performed on the information processing device 1 side. As a result, even if the displacement section 202 returns to its original position (displacement amount: 0), only rotational resistance is generated based on the control current value after the end, and no tactile sensation is presented. The control current value after completion is zero or one that makes the rotational resistance feel maximum. The displacement unit 202 does not need to return until the end operation is executed, or may return to the initial position (angle 0°).

If it is determined that the end displacement amount has not been reached (S204: NO), the control unit 20 transmits the displacement amount and slope obtained by sampling to the information processing device 1 (step S207), and A current value corresponding to the amount of displacement is referenced from the tactile data stored in the storage unit 21 (step S208), a current having the referenced current value is output to the MRF device 24 (step S209), and the process returns to step S203. Until the displacement amount reaches the end displacement amount, the control unit 20 continues the processing of steps S203 to S206 until a termination operation is performed on the information processing device 1 side.

The information processing device 1 determines whether or not a notification of reaching the end displacement amount has been received from the tactile presentation device 2 (step S108). If it is determined that it has been received (S108: YES), the processing unit 10 causes the display unit 13 to display an image when the end displacement amount is reached (step S109), and determines whether or not the end operation has been performed. (Step S110). If it is determined that the end operation has not been performed (S110: NO), the processing unit 10 returns the process to step S109. In step S109, the processing unit 10 may output the audio when the end displacement amount is reached from the audio output unit 14.

If it is determined that the termination operation has been performed (S110: NO), the display is terminated, communication with the tactile presentation device 2 is disconnected (Step S111), and the process is terminated.

If it is determined that the amount of displacement has not been received (S108: NO), the processing unit 10 receives the displacement amount from the tactile presentation device 2 (step S112), and stores the image and audio corresponding to the received displacement amount in the storage unit 11. Reference is made from visual data and auditory data (step S113). The processing unit 10 outputs the image and audio from the display unit 13 and the audio output unit 14 (step S114), and advances the process to step S115. Each time the amount of displacement is transmitted from the tactile presentation device 2, the information processing device 1 outputs an image and audio corresponding to the amount of displacement.

The processing unit 10 determines whether a termination operation has been performed (step S115). If it is determined that the end operation has not been performed (S115: NO), the process returns to step S108, and the processes of steps S108-S115 are repeated.

If it is determined that the end operation has been performed (S115: YES), the processing unit 10 advances the process to step S111.

FIG. 7 is a schematic diagram showing an example of tactile presentation in the tactile presentation system 100. FIG. 7 shows changes in images and sounds displayed on the display unit 13 in response to the amount of displacement in the tactile data sent to the tactile presentation device 2. In the example of FIG. 7, the target object is a collapsible object like a balloon, and the tactile presentation device 2 presents a fluffy tactile sensation as a result of the displacement of the displacement unit 202, and shows the collapse of the target object as visual data. show. As shown in FIG. 7, the image of the object changes depending on the pushing amount (displacement amount) of the displacement section 202, and the way the object is concave on the image changes. As shown in FIG. 7, when displaying an image of the operator's finger in the image, contact, that is, contact with the target object in the image from the initial position (displacement amount is zero) output to the MRF device 24. An image of a finger may also be displayed.

As shown at the bottom of FIG. 7, when the displacement amount of the displacement part 202 reaches 48 degrees, which corresponds to the end displacement amount, an image indicating the end of tactile presentation and the words "Ruptured (END)" are displayed. be done. In the example shown in FIG. 7, an image and text of the object bursting are displayed. Rupture is irreversible. This allows the operator to recognize that the presentation of the tactile sensation has ended. After the displacement amount of the displacement section 202 reaches the end displacement amount, a large rotational resistance is applied to the displacement section 202 in the tactile sense presentation device 2, and the movement of the displacement section 202 is restricted. This allows the operator to recognize that the presentation of the tactile sensation has ended.

FIG. 8 is a schematic diagram showing another example of tactile presentation in the tactile presentation system 100. FIG. 8 shows changes in images and sounds displayed on the display unit 13 in response to the amount of displacement in the tactile data transmitted to the tactile presentation device 2. In the example shown in FIG. 8, an image of a mountain of sand is displayed, and the tactile presentation device 2 presents the smooth and crunchy feel of the sand, while the visual data shows that the mountain of sand is irreversibly collapsing. shows. As shown in FIG. 8, the image of the object changes depending on the pushing amount (displacement amount) of the displacement section 202, and the mountain of sand gradually collapses on the image.

As shown at the bottom of FIG. 8, when the amount of displacement of the displacement unit 202 reaches an angle corresponding to the end displacement amount, an image indicating the end of the tactile presentation and the word "end" are displayed. In the example shown in FIG. 8, an image and text of a collapsed pile of sand are displayed. The collapse of the sand pile is irreversible. This allows the operator to recognize that the presentation of the balloon's tactile sensation has ended.

As shown in FIGS. 7 and 8, the image and tactile presentation device outputs a predetermined rotational resistance, so that the operator can recognize that the tactile presentation of the breakable object in the tactile presentation device 2 has ended; The operator can use the tactile presentation device 2 appropriately. After the amount of displacement of the displacement section 202 reaches the end displacement amount, the rotational resistance to the displacement section 202 in the tactile presentation device 2 becomes zero. As a result, the object disappears and the operator can recognize that the presentation of the tactile sensation has ended.

(Second embodiment)
In the second embodiment, an HMD (Head Mounted Display) is used to present visual information regarding a target object as a three-dimensional image. FIG. 9 is a schematic diagram of the tactile presentation system 100 in the second embodiment, and FIG. 10 is a block diagram showing the configuration of the HMD 3. The configuration of the tactile presentation system 100 in the second embodiment is the same as that of the tactile presentation system 100 in the first embodiment, except that the HMD 3 is used and the details of the processing by using the HMD 3 are different. . Therefore, among the configurations of the tactile presentation system 100 of the second embodiment, the same components as those of the tactile presentation system 100 of the first embodiment are given the same reference numerals, and detailed description thereof will be omitted.

The HMD 3 includes a display section 31, a motion detection section 32, a space detection section 33, and a connection section 34. The HMD 3 may be provided with a display section 31, a motion detection section 32, a space detection section 33, and a connection section 34 in the main body, or some may be provided separately and mutually exchange control signals via a communication medium. It may be possible to do so. The HMD 3 may have the functions of the information processing device 1 and be configured integrally.

The display unit 31 includes, for example, a small liquid crystal display, an optical lens, and an optical system mechanism, and is capable of displaying a three-dimensional image at a viewing angle of 110° or more. The display unit 31 is a transparent or semi-transparent glass-like display, and receives an image signal (including a video signal) output from the information processing device 1 and displays it in a superimposed manner in the operator's actual field of view. The display section 31 is not limited to a glass-like one, and may display an image based on an image signal output from the information processing device 1 superimposed on an image taken of real space by a camera provided facing forward. It's okay.

The motion detection unit 32 includes a plurality of triaxial acceleration sensors and gyro sensors installed in various directions on the main body (the cover of the display unit 31 and the attachment belt), and aggregates signals from a group of these sensors. and a control circuit that outputs the output. The movement detection unit 32 detects the movement of the wearer's head.

The space detection unit 33 includes two or more infrared cameras arranged outwardly in parallel on the outer surface of the main body, and an infrared LED disposed at an intermediate position between the infrared cameras so as to similarly emit infrared rays outward. use The space detection unit 33 functions as a depth sensor that measures the distance to an object that exists outside the main body of the HMD 3. The space detection unit 33 can measure and output the distance from the HMD 3 main body to a still object such as a wall or floor in the location of the operator wearing the HMD 3. Similarly, the space detection unit 33 can measure and output the distance from the HMD 3 main body to the operator's arm, hand, and finger.

The connection unit 34 is an interface for connecting to the information processing device 1. The HMD 3 outputs signals corresponding to the results measured by the motion detection unit 32 and the space detection unit 33 to the information processing device 1, and also acquires the video signal output from the information processing device 1 and displays it on the display unit. 31.

The HMD 3 may be provided with a speaker, and the audio output from the audio output unit 14 of the information processing device 1 may be output from the speaker.

In the haptic presentation system 100 of the third embodiment configured as described above, the information processing device 1 uses the HMD 3 and the distance to the still object in the real space measured by the space detection unit 33 to determine the tertiary shape of the virtual object. The original image is superimposed and displayed in accordance with the operator's movement detected by the movement detection unit 32 to realize AR display. To select an object, the space detection section 33 uses the tactile presentation device 2 to select the coordinates in the three-dimensional image of the virtual object displayed on the display section 31 of the HMD 3 (coordinates that match the real space). When it is detected that the gripped fingers are overlapped, the processing unit 10 of the information processing device 1 detects this as being selected.

Also in the third embodiment, the information processing device 1 sends and receives displacement and tilt data to and from the tactile presentation device 2 to control the image displayed in AR and the tactile sensation output by the tactile presentation device 2. do. The contents of the process are similar to the process shown in the first embodiment.

By combining the AR display using the HMD 3 and the tactile presentation device 2, the operator can visually observe an image in which the object is virtually placed in real space, and feel the touch of the object using the tactile presentation device. It is possible to memorize it in 2. At this time, similar to what was explained in the first embodiment, the tactile presentation start position in the tactile presentation device 2 can be corrected according to each operator, thereby further improving the degree of perception. .

The embodiments disclosed above are illustrative in all respects and are not restrictive. The scope of the present invention is indicated by the claims, and includes all changes within the meaning and range equivalent to the claims.

1 Information processing device 10 Processing section 11 Storage section 13 Display section P1 Control program 2 Tactile presentation device 20 Control section 21 Storage section 202 Displacement section 24 MRF device

Claims (6)

It has a displacement part that is displaceable in response to an operation by an operator, and generates a force sensation in response to the operation on the displacement part according to the amount of displacement of the displacement part, and presents a tactile sensation of the displayed object. A control device for a tactile presentation device, comprising:
storing control data for generating the force sensation and an end displacement amount for terminating the presentation of the tactile sensation by the displacement unit for each object in association with the displacement amount of the displacement unit;
outputting control data and an end displacement amount associated with the displacement amount of the displacement part to the tactile presentation device;
A control device that terminates tactile presentation in the tactile presentation device when the displacement amount of the displacement portion reaches the end displacement amount. For each of the objects, control data for causing a predetermined sense of force to be generated in the displacement part after the displacement part reaches a final displacement amount, or control data for zeroing out the sense of force to be generated in the displacement part. Remember,
The control device according to claim 1, wherein when the displacement amount of the displacement section reaches the end displacement amount, the predetermined force sensation is generated, or the force sensation generated in the displacement section is made to be zero. 3. When the displacement amount of the displacement section reaches the end displacement amount, the display section displays an image indicating the end, or the audio output section outputs a sound indicating the end. Control device. It has a displacement part that is displaceable in response to an operation by an operator, and generates a force sensation in response to the operation on the displacement part according to the amount of displacement of the displacement part, and presents a tactile sensation of the displayed object. A method for controlling a tactile presentation device, comprising:
A computer connected to the tactile presentation device,
storing control data for generating the force sensation and an end displacement amount for terminating the presentation of the force sensation by the displacement unit for each object in association with the displacement amount of the displacement unit;
outputting control data and an end displacement amount associated with the displacement amount of the displacement part to the tactile presentation device;
A control method including a processing section that ends tactile presentation in the tactile presentation device when the displacement amount of the displacement section reaches the end displacement amount. It has a displacement part that is displaceable in response to an operation by an operator, and generates a force sensation in response to the operation on the displacement part according to the amount of displacement of the displacement part, and presents a tactile sensation of the displayed object. a tactile presentation device that
an information processing device that is communicatively connected to the tactile presentation device and includes a display unit;
The information processing device includes:
storing control data for generating the force sensation and an end displacement amount for terminating the presentation of the tactile sensation by the displacement unit for each object in association with the displacement amount of the displacement unit;
outputting control data and an end displacement amount associated with the displacement amount of the displacement part to the tactile presentation device;
When the displacement amount of the displacement portion reaches the end displacement amount, terminating the tactile presentation in the tactile presentation device;
A tactile presentation system that causes the display unit to display an image indicating termination. It has a displacement part that is displaceable in response to an operation by an operator, and generates a force sensation in response to the operation on the displacement part according to the amount of displacement of the displacement part, and presents a tactile sensation of the displayed object. a computer connected to a tactile presentation device that
storing control data for generating the force sensation and an end displacement amount for terminating the presentation of the tactile sensation by the displacement unit for each object in association with the displacement amount of the displacement unit;
outputting control data and an end displacement amount associated with the displacement amount of the displacement part to the tactile presentation device;
A computer program that causes the tactile presentation device to terminate tactile presentation when the displacement amount of the displacement portion reaches the end displacement amount.

Images