JP2023148854A - Control device, control method, haptic feedback system, and computer program - Google Patents

Abstract

To provide a control device, a control method, a haptic feedback system, and a computer program, for improving perceptibility of haptic sensations using a control depending on attitude.SOLUTION: A control device is for a haptic feedback device having a displacement unit that is provided to be displaced in response to an operation by an operator. The haptic feedback device is configured to generate a force sense for operation on the displacement unit according to a displacement amount of the displacement unit and present a haptic sensation of a displayed object. The control device preliminarily stores, for each object, control data for generating the force sense and data on attitude of the haptic feedback device for modifying the content of the haptic sensation to be presented by the control data, in association with the displacement amount of the displacement unit, and modifies the control data for generating the force sense according to the attitude of the haptic feedback device.SELECTED DRAWING: Figure 1

Description

The present invention relates to a tactile presentation device, and more particularly to a control device, a control method, a tactile presentation system, and a computer program for a tactile presentation device that improves tactile perception.

2. Description of the Related Art Among the five human senses, image and audio transmission technologies related to visual and auditory senses are becoming more precise, and in recent years, various technologies for presenting tactile sensations (haptics) have been proposed.

Tactile presentation technology uses mainly three types of vibrations (ERM: Eccentric Rotating Mass, LRA: Linear resonant actuator, piezo element) to reproduce the vibrations that occur when touching an image of an object displayed on a display with a built-in touch panel. Proposed. However, when presenting tactile sensations on a touch panel, it is difficult to reproduce deeper tactile sensations, that is, not only the skin of fingers but also the force sensations generated in muscles and tendons.

Based on knowledge of magnetorheological fluids, the inventors have proposed a tactile presentation device that can be operated by an operator and reproduces different sensations depending on the object (Patent Document 1, etc.).

Patent No. 6906275

The tactile presentation device includes a displacement section that is a curved plate or the like that can be displaced relative to the base by an operation of an operator. The displacement part can be displaced by the operator without resistance when the tactile presentation device is not controlled, but the tactile presentation device under control can move the displacement part according to the amount of displacement (position/angle) of the displacement part. The tactile sensation is presented by changing the magnitude of the force sensation generated, the output method of the force sensation, etc.

The tactile presentation device is used by being held by an operator's hand. The tactile presentation device is used in various postures, especially when it is realized in a small enough type to be held and lifted in the hand.

The present invention has been made in view of the above circumstances, and aims to provide a control device, a control method, a tactile presentation system, and a computer program for causing complaints about tactile perception through control according to posture. .

A control device according to an embodiment of the present disclosure includes a displacement section that is displaceable in response to an operation by an operator, and generates a force sensation in response to an operation on the displacement section according to the amount of displacement of the displacement section. A control device for a tactile sense presentation device that presents a tactile sense of a displayed object by controlling the force, the control data for generating the force sense in association with the amount of displacement of the displacement part, and the presentation content of the tactile sense based on the control data. data on the posture of the tactile presentation device that changes the tactile sensation presentation device is stored for each object, and control data that causes the force sensation to be generated is changed in accordance with the posture of the tactile presentation device.

A control method according to an embodiment of the present disclosure includes a displacement section that is displaceable in response to an operation by an operator, and generates a force sensation in response to an operation on the displacement section according to a displacement amount of the displacement section. A control method for a tactile presentation device that presents a tactile sensation of a displayed object by causing a computer connected to the tactile presentation device to generate the haptic sensation in association with a displacement amount of the displacement portion. Control data and data on the posture of the tactile presentation device that changes the content of the tactile presentation based on the control data are stored for each object, and the haptic sensation is generated according to the posture of the tactile presentation device. This includes processing to change the control data for

A computer program according to an embodiment of the present disclosure includes a displacement section that is displaceable in response to an operation by an operator, and generates a force sensation in response to an operation on the displacement section according to a displacement amount of the displacement section. a computer connected to a tactile presentation device that presents a tactile sensation of the displayed object, control data that causes the force sensation to be generated in association with the amount of displacement of the displacement portion, and content of the tactile sensation presentation based on the control data; Data on the posture of the tactile presentation device that changes the tactile sensation presentation device are stored for each object, and processing for changing control data that causes the force sensation is executed in accordance with the posture of the tactile presentation device.

In the control device, control method, and computer program of the present disclosure, control data is changed with reference to information on the posture of the tactile presentation device in addition to the displacement amount of the displacement portion of the tactile presentation device. Depending on the posture, it is possible to interrupt the process or switch to control content that is appropriate for the posture.

A control device according to an embodiment of the present disclosure interrupts presentation of a tactile sensation by the tactile sensation presentation device when the posture of the tactile sensation presentation device is a specific posture.

In the control device of the present disclosure, when the posture is a specific posture, the tactile sensation presentation in the tactile sensation presentation device is interrupted. Tactile presentation is possible only in appropriate postures.

The control device according to an embodiment of the present disclosure stores control data for causing the force sensation in association with the displacement amount of the displacement portion for each posture of the tactile sensation presentation device for each of the objects, and Control data for each posture is output to the tactile presentation device.

With the control device of the present disclosure, it is also possible to perform control to provide different tactile sensations depending on the posture. It is also possible to present a tactile sensation depending on the direction of contact with the object.

A tactile presentation system according to an embodiment of the present disclosure includes a displacement section that is displaceable in response to an operation by an operator, and generates a force sensation in response to an operation on the displacement section according to the amount of displacement of the displacement section. The tactile presentation device includes a tactile presentation device that generates and presents a tactile sensation of a displayed object, and an information processing device that is communicatively connected to the tactile presentation device and includes a display unit, and the tactile presentation device includes a sensor that detects a posture. Furthermore, the information processing device includes control data for generating the force sensation and data on a posture of the tactile sensation presentation device that changes the presentation content of the tactile sensation based on the control data, in association with the amount of displacement of the displacement section. , the control data that is stored for each object and that causes the force sensation is changed according to the orientation detected by the sensor of the tactile presentation device, and the orientation obtained from the sensor and the displacement portion are changed. An image is output to the display section according to the amount of displacement.

In the tactile presentation system of the present disclosure, in addition to the amount of displacement of the displacement portion of the tactile presentation device, control data and the display on the display unit are changed with reference to information on the posture of the tactile presentation device. Depending on the posture, it is possible to interrupt the process or switch to control content that is appropriate for the posture.

According to the present disclosure, it is possible to perform control according to the posture of the tactile presentation device itself, and it is possible to improve the perceptibility of the tactile sense.

FIG. 1 is a schematic diagram showing a tactile presentation system. FIG. 1 is a block diagram showing the configuration of an information processing device. It is an explanatory view showing an example of contents of a sensory DB. FIG. 2 is a block diagram showing the configuration of a tactile presentation device. It is a flowchart which shows an example of the processing procedure of the tactile sense presentation in a tactile sense presentation system. It is a flowchart which shows an example of the processing procedure of the tactile sense presentation in a tactile sense presentation system. FIG. 2 is a schematic diagram showing an example of tactile presentation in the tactile presentation system. An example of the content of tactile data in the second embodiment is shown. 12 is a flowchart illustrating an example of a tactile presentation processing procedure in the second embodiment. 12 is a flowchart illustrating an example of a tactile presentation processing procedure in the second embodiment. FIG. 7 is a schematic diagram showing an example of tactile presentation in a tactile presentation system according to a second embodiment. FIG. 3 is a schematic diagram of a tactile presentation system in a third embodiment. It is a block diagram showing the composition of HMD of a 3rd embodiment.

The present disclosure will be specifically described with reference to drawings showing embodiments thereof. In the following embodiments, implementation of a setting method in a tactile presentation system will be described.

(First embodiment)
FIG. 1 is a schematic diagram showing a tactile presentation system 100. The tactile presentation system 100 includes an information processing device 1 and a tactile presentation device 2. The information processing device 1 and the tactile presentation device 2 are communicably connected by short-range wireless communication and exchange data with each other.

The information processing device 1 uses a smartphone as shown in FIG. The information processing device 1 may be a tablet terminal or a laptop-type PC (Personal Computer) instead of a smartphone.

The tactile presentation device 2 is a device that an operator can operate by moving his or her finger while holding the finger along the displacement portion 202 . The tactile presentation device 2 reads the position of the displacement section 202 that is displaced by the operator's finger movement, controls the built-in MRF (Magneto-Rheological Fluid) device 24 according to the position, and moves the displacement section 202 of the operator. This is a device that produces a sense of force using a reaction force (rotational resistance) to the operation of the machine, and presents a tactile sensation. The form of the displacement part 202 of the tactile presentation device 2 is not limited to that shown in FIG. 1, but may be stick-like or cushion-like covered with a cover. The tactile presentation device 2 may employ a motor, a piezo element, or the like instead of the MRF device 24, and may generate a force sensation by rotational force or vibration in response to an operation by the operator. It may be combined with something that provides a warm or cold sensation. It may also be a ring-shaped structure that is worn on the operator's finger and operated with another finger.

In the tactile presentation system 100, the tactile presentation device 2 cooperates with the information processing device 1 and displays the tactile sensation of the object while outputting images and sounds of the object using the information processing device 1 or other display devices, speakers, etc. is outputted by the tactile presentation device 2. In the example shown in FIG. 1, when a soft object is displayed on the display unit 13 of the information processing device 1 and the operator pushes the displacement part 202 of the tactile presentation device 2 with a finger, the tactile presentation device 2 becomes squishy or fluffy. While outputting a tactile sensation, the display unit 13 displays an image that changes as if the object is being pushed in, and the audio output unit 14 outputs an audible “squeak” sound that corresponds to the object being pushed in. Output audio.

FIG. 2 is a block diagram showing the configuration of the information processing device 1. As shown in FIG. The information processing device 1 includes a processing section 10, a storage section 11, a communication section 12, a display section 13, an audio output section 14, and an operation section 15. The processing unit 10 is a processor using a CPU (Central Processing Unit) and/or a GPU (Graphics Processing Unit). The processing unit 10 executes a process described below based on a control program P1 for tactile presentation stored in the storage unit 11.

The storage unit 11 uses nonvolatile memory such as flash memory or SSD (Solid State Drive). The storage unit 11 stores data referenced by the processing unit 10. The control program P1 (program product) is downloaded from the information processing device 1 or another program server device via the communication unit 12 and stored in an executable manner. The control program P1 stored in the storage unit 11 may be a computer program P8 stored in the computer-readable storage medium 8, which is read out and stored by the processing unit 10.

The storage unit 11 stores a sensory database (DB) 110 including tactile data, visual data, and auditory data of the object to be output. The sensory DB 110 stores tactile data, visual data (images, videos), and auditory data to be output at each displacement for each displacement data of a displacement part in the tactile presentation device 2, in association with an object ID that identifies the object. (audio) (see Figure 3). The storage unit 11 further stores setting data regarding the tilt of the tactile presentation device 2, which will be described later.

The communication unit 12 is a communication module for short-range wireless communication, for example, Bluetooth (registered trademark). The processing unit 10 can transmit and receive data to and from the tactile presentation device 2 through the communication unit 12 .

The display unit 13 is a display such as a liquid crystal display or an organic EL (Electro Luminescence) display. The display unit 13 is, for example, a display with a built-in touch panel. The processing unit 10 displays, on the display unit 13, an operation screen for causing the tactile sensation presentation device 2 to output a tactile sensation and an image of the target object, based on the control program P1.

The audio output unit 14 includes a speaker and the like. The processing unit 10 causes the audio output unit 14 to output the sound of the object, music, etc. based on the control program P1.

The operation unit 15 is a user interface that allows input and output to and from the processing unit 10, and is a touch panel built into the display unit 13. The operation unit 15 may be a physical button. The operation unit 15 may be a voice input unit.

FIG. 3 is an explanatory diagram showing an example of the contents of the sensory DB 110. As shown in FIG. 3, the sensory DB 110 stores the current value of the current to the MRF device of the tactile presentation device 2 for each displacement amount (angle) of the displacement part 202 as tactile data in association with the object ID. There is. It is not limited to a current value, but may be a voltage value or the like. The objects are plural, and may include real objects that present a tactile sensation, such as still life objects such as balls or balloons, foods such as gummies, vegetables, or fruits, and animals such as dogs, cats, or fish. Characters such as slimes and monsters may also be included.

The relationship between the amount of displacement of the displacement section 202 of the sensory DB 110 and the current value is set according to the actual size of the movable range of the displacement section 202 and the size of the object. For example, if the movable range of the displacement unit 202 is 50 mm, and the object is a sphere that will not burst, such as a small gummy bear, and the size is set to 15 mm, the tactile sensation will be reproduced within the first 15 mm, In the remaining 35 mm range, a current value is set that produces a rotational resistance that makes it difficult to push in any further. Alternatively, a large current value that makes it impossible to push in the range of 12 mm to 15 mm may be set, and the current values thereafter may be set to zero. In addition, if the target object is a large object such as the seat surface of a sofa, and the size greatly exceeds the movable range of the displacement unit 202, the tactile data may be such as to present the sensation up to 50 mm from the surface of the sofa seat surface. The current value is set. If the target object is a virtual character such as a monster, the current value is set based on the size according to the setting of the character.

Similarly, the sensory DB 110 stores images (frame images) for each displacement amount (angle) of the displacement portion 202 as visual data in association with the object ID. Here, the frame image is one still image that is recognized as an animation image when displayed continuously. Similarly, audio for each displacement amount (angle) of the displacement unit 202 is stored as auditory data in association with the object ID. The auditory data may be waveform data that differs depending on each angle. It may also be a timestamp of the audio corresponding to each angle.

FIG. 4 is a block diagram showing the configuration of the tactile presentation device 2. As shown in FIG. As shown in FIG. 1, the tactile presentation device 2 is configured by providing a flat bottomed cylindrical grip 200 with a band-shaped flat plate displacement portion 202 having a curved portion partially along the circumferential direction. Although the displacement part 202 is made of a flexible material, it may also be made of a highly rigid material and rotatably supported via the grip body 200 and a support shaft. A binding tool 203 in the form of a cloth tape is provided on the outer surface of the distal end of the displacement portion 202 . A link mechanism 204 is provided on the inner surface of the distal end of the displacement portion 202 to connect to the rotating shaft of the rotor of the MRF device 24 housed inside the gripper 200 .

As shown in FIG. 1, the operator uses the holding body 200 by, for example, holding the holding body 200 with the thumb and middle finger, and inserting the finger such as the index finger into the binding tool 203 along the displacement part 202. The operator can move the displacement part 202 by pushing in the index finger, or can move the displacement part 202 away from the grip body 200 by extending the index finger.

The tactile presentation device 2 includes a gripping body 200 as shown in FIG. The grip body 200 has an MRF device 24 built therein. The control section 20, the storage section 21, the communication section 22, and the power supply section 23 may be provided integrally with the grip body 200, or may be provided as separate bodies connected to the grip body 200 wirelessly or by wire.

The control unit 20 includes a processor such as a CPU and an MPU (Micro-Processing Unit), and a memory such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The control unit 20 is, for example, a microcontroller. The control unit 20 controls each component based on a control program P2 stored in the built-in ROM to realize tactile presentation.

The storage unit 21 is an auxiliary storage memory for the control unit 20, and stores control data (tactile data) of the MRF device 24 in a rewritable manner.

The communication unit 22 is a communication module for short-range wireless communication, for example, Bluetooth (registered trademark). The control unit 20 can transmit and receive data to and from the information processing device 1 through the communication unit 22 .

The control unit 20 is connected to a power supply unit 23, an MRF device 24, and a sensor 25 via I/O, and exchanges signals with each other.

Power supply unit 23 includes a rechargeable battery. The power supply unit 23 supplies power to each component and the MRF device 24 when turned on.

The MRF device 24 has a yoke that sandwiches a disk-shaped rotor with a gap between them, and generates a magnetic field by passing a control current through a coil provided in the yoke, thereby generating a magnetic field sealed in the gap. The viscosity (shear stress) of the viscous fluid is controlled to provide rotational resistance to the rotor. When the control unit 20 controls the magnitude of the control current to the MRF device 24, the rotational resistance is immediately changed.

The sensor 25 measures the amount of displacement (angle) of the displacement section 202 and outputs it to the control section 20 . Sensor 25 includes an angle sensor 251 and a gyro sensor 252. The angle sensor 251 measures and outputs the displacement of the displacement portion 202 as an angle. The gyro sensor 252 detects the inclination in the three axes directions, and sets the tactile presentation device 2 in an initial posture in which a line connecting the center of the gripping body 200 and the tip of the displacement portion 202 of the tactile presentation device 2 points in the vertical direction. Output the slope. The sensor 25 may also include an acceleration sensor.

In the tactile presentation device 2 configured as described above, when the displacement section 202 is operated by the operator, the displacement of the displacement section 202 is caused to move in the direction of rotation of the MRF device 24 toward the rotation axis of the rotor via the link mechanism 204. communicated. Since the rotating shaft rotates freely when the MRF device 24 is not operating, that is, while the control current is zero, the displacement portion 202 fluctuates without resistance. On the other hand, when the MRF device 24 operates and the control current is not zero, the viscosity (shear stress) of the magnetorheological fluid inside the MRF device 24 is changed depending on the magnitude of the current flowing to the MRF device 24. By continuously changing the magnitude of the current to the MRF device 24 or by vibrating the current value at a predetermined frequency, the control unit 20 can change the resistance force applied to the displacement unit 202 and the method of its appearance. .

In this way, the tactile sensation presentation device 2 can vary the resistance (current value) according to the pushing amount (displacement amount) of the displacement section 202 to present a smooth tactile sensation, or increase the resistance as the pushing amount increases. It is possible to present a tactile sensation that is hard and hard, or to present a tactile sensation that is crunchy by repeatedly increasing and decreasing the resistance.

Further, the tactile presentation device 2 in the first embodiment interrupts the tactile presentation process when detecting that the user is in a specific posture. Information on the inclination corresponding to a specific posture is stored in the storage unit 11 of the information processing device 1 as setting data, and is referred to and used by the information processing device 1 and the tactile presentation device 2. The specific posture is, for example, a posture in which the displacement portion 202 is tilted 90° or more in the width direction (left-right direction). In addition, the specific posture may be a posture tilted by 90 degrees or more in the length direction of the displacement section 202. In addition to a specific posture, the method of displacement of the displacement unit 202 or a combination of vibrations applied to the tactile presentation device 2 may be used.

5 and 6 are flowcharts illustrating an example of a tactile presentation processing procedure in the tactile presentation system 100. When the operator starts the control program P1 and turns on the power of the tactile presentation device 2, the processing unit 10 of the information processing device 1 starts the following process in cooperation with the tactile presentation device 2.

The processing unit 10 displays on the display unit an operation screen including a list of object candidates for which tactile sensation is to be displayed (step S101), and accepts selection of the object (step S102). The processing unit 10 reads the tactile data, visual data, and auditory data corresponding to the object ID of the selected object from the sensory DB 110 of the storage unit 11 and temporarily stores them (step S103).

The processing unit 10 establishes communication between the tactile presentation device 2 and the communication unit 12 (step S104), and transmits the temporarily stored corrected tactile data to the tactile presentation device 2 (step S105). The processing unit 10 transmits data on a specific posture range of the tactile presentation device 2 included in the setting data to the tactile presentation device 2 (step S106).

The processing unit 10 causes the display unit 13 to display that tactile presentation is to be started (step S107).

The control unit 20 of the tactile presentation device 2 receives tactile data and stores it in the storage unit 21 (step S201), and receives data on a specific posture range and stores it in the storage unit 21 (step S202).

The control unit 20 samples a signal corresponding to the displacement amount (angle) of the displacement unit 202 output from the angle sensor 251 (step S203). The control unit 20 samples a signal indicating the tilt of the tactile presentation device 2 output from the gyro sensor 252 (step S204).

The control unit 20 determines whether the inclination obtained by sampling is included in the specific posture range stored in step S202 (step S205). If it is determined that it is within the specific range (S205: YES), the control unit 20 notifies the information processing device 1 that the presentation of the tactile sensation will be interrupted (step S206), and returns the process to step S203. .

If it is determined that it is not included in the specific range (S205: NO), the control unit 20 transmits the displacement amount and slope obtained by sampling to the information processing device 1 (step S207), and the obtained displacement The current value corresponding to the amount is referenced from the tactile data stored in the storage unit 21 (step S208), the referenced current is output to the MRF device 24 (step S209), and the process returns to step S203. The processing in steps S203 to S209 continues until an operation to end the processing is performed on the information processing device 1 side.

In step S209, the control unit 20 may change the current to be output according to the slope detected in step S204. For example, if the posture of the displacement unit 202 is such that it rolls forward in the length direction, the output current may be doubled.

The information processing device 1 determines whether a notification of interruption of tactile presentation has been received from the tactile presentation device 2 (step S108). If it is determined that it has been received (S108: YES), the processing unit 10 displays on the display unit 13 that the tactile presentation is to be interrupted (step S109), and returns the process to step S108. In step S109, the processing unit 10 may cause the display unit 13 to go to sleep (turn off the display output).

If it is determined that the displacement amount and the inclination have not been received (S108: NO), the processing unit 10 receives the displacement amount and the inclination (step S110), and stores the image and audio corresponding to the received displacement amount as the visual data and the storage unit 11. Reference is made from auditory data (step S111). The processing unit 10 outputs the visual data and the auditory data from the display unit 13 and the audio output unit 14 (step S112), and returns the process to step S108. Each time the amount of displacement is transmitted from the tactile presentation device 2, the information processing device 1 outputs an image and audio corresponding to the amount of displacement.

In step S112, the processing unit 10 may change and output images and sounds according to the tilt received in step S110.

The processing unit 10 determines whether a termination operation has been performed (step S113). If it is determined that the end operation has not been performed (S113: NO), the process returns to step S108, and the processes of steps S108-S113 are repeated.

If it is determined that the end operation has been performed (S113: YES), the processing unit 10 ends the display, disconnects communication with the tactile presentation device 2 (step S114), and ends the process.

FIG. 7 is a schematic diagram showing an example of tactile presentation in the tactile presentation system 100. FIG. 7 shows changes in images and sounds displayed on the display unit 13 in response to the amount of displacement in the tactile data sent to the tactile presentation device 2. As shown in FIG. 7, the image changes depending on the amount of depression of the displacement part 202, and the way the dent corresponds to the tactile sensation changes. As shown in FIG. 7, when displaying an image of the operator's finger in the image, contact, that is, contact with the target object in the image from the initial position (displacement amount is zero) output to the MRF device 24. An image of a finger may also be displayed.

In the first embodiment, as shown at the bottom of FIG. 7, when the tactile presentation device 2 is tilted over 90 degrees in the width direction of the displacement section 202, the presentation of the tactile sensation in the tactile presentation device 2 is interrupted. At the same time, image and audio output is also interrupted. The operator recognizes that the tactile presentation device 2 is controlled so as not to present the tactile sensation in that posture, and uses the tactile presentation device 2 in a posture that is easy for the operator to operate within the range in which the tactile sensation is presented.

In this way, by interrupting the tactile sensation output from the tactile presentation device 2 depending on the posture of the tactile presentation device 2, the operator can use the tactile presentation device 2 appropriately, and the perception level by the operator can be improved. will improve.

(Second embodiment)
In the second embodiment, the gyro sensor 252 installed in the tactile presentation device 2 is used to change the tactile sensation presented by the tactile presentation device 2 depending on not only the amount of displacement of the displacement section 202 of the tactile presentation device 2 but also the posture. do.

FIG. 8 shows an example of the content of tactile data in the second embodiment. As shown in FIG. 8, in the second embodiment, the tactile data in the tactile DB 110 is sent to the MRF device of the tactile presentation device 2 for each displacement amount (angle) of the displacement unit 202 in association with the object ID. The current value of the current is stored as tactile data. Similarly, visual data and auditory data may also be stored for each posture.

Control using tactile data for each posture as shown in FIG. 8 will be described. 9 and 10 are flowcharts illustrating an example of a tactile presentation processing procedure in the second embodiment. Among the processing procedures shown in the flowcharts of FIGS. 9 and 10, the same steps as those shown in the flowcharts of FIGS. 5 and 6 in the first embodiment are given the same step numbers and detailed explanations are omitted. do.

The processing unit 10 reads the posture-specific tactile data, visual data, and auditory data corresponding to the object ID of the object received in step S102 from the sensory DB 110 of the storage unit 11 and temporarily stores them (step S123).

When the communication connection is established, the processing unit 10 transmits tactile data for each posture to the tactile presentation device 2 (step S125), and also transmits data for a specific posture range (S106).

In the tactile presentation device 2, the control unit 20 receives and stores the tactile data for each posture (step S221), and uses it thereafter. The control unit 20 samples the displacement amount and the inclination (S203, S204), and if it is determined that the inclination is not included in the specific range (S205: NO), the control unit 20 samples the detected displacement amount from the tactile data corresponding to the posture. The corresponding current value is referred to (step S228). The control unit 20 outputs the current having the referenced current value to the MRF device 24 (S209), and returns the process to step S203.

When the processing unit 10 in the information processing device 1 receives the displacement amount and inclination (S101), it refers to the visual data and auditory data of the posture and displacement amount corresponding to the received inclination (step S131), and calculates the referenced visual data and Images and audio are output based on the auditory data (S112).

FIG. 11 is a schematic diagram showing an example of tactile presentation in the tactile presentation system 100 according to the second embodiment. Similar to the first embodiment, FIG. 11 shows changes in images and sounds displayed on the display unit 13 in response to the amount of displacement in the tactile data transmitted to the tactile presentation device 2. As shown in FIG. 11, the image changes depending on the amount of depression of the displacement portion 202, and the way the depression corresponds to the tactile sensation changes. When displaying an image of the operator's finger in the image, the image of the finger touching the object in the image is displayed from the initial position (displacement amount is zero) of contact, that is, output to the MRF device 24. You can do it like this.

In the second embodiment, as shown in FIG. 11, the feel and the displayed image are changed depending on whether the posture of the tactile presentation device 2 is tilted by 60 degrees or more. In the example shown in FIG. 11, the feeling of being crushed from above and the feeling of being touched from the side are changed. The angle of the finger image has also been changed. In this way, by further combining the displacement amount of the displacement unit 202 of the tactile presentation device 2 with the attitude of the tactile presentation device 2 itself, the perceptibility of the tactile sense can be further improved.

(Third embodiment)
In the third embodiment, an HMD (Head Mounted Display) is used to present visual information regarding an object as a three-dimensional image. FIG. 12 is a schematic diagram of the tactile presentation system 100 in the third embodiment, and FIG. 13 is a block diagram showing the configuration of the HMD 3. The configuration of the tactile presentation system 100 in the third embodiment is the same as that of the tactile presentation system 100 in the first embodiment, except that the HMD 3 is used and the details of the processing by using the HMD 3 are different. . Therefore, among the configurations of the tactile presentation system 100 of the third embodiment, the same components as those of the tactile presentation system 100 of the first embodiment are given the same reference numerals, and detailed description thereof will be omitted.

The HMD 3 includes a display section 31, a motion detection section 32, a space detection section 33, and a connection section 34. The HMD 3 may be provided with a display section 31, a motion detection section 32, a space detection section 33, and a connection section 34 in the main body, or some may be provided separately and mutually exchange control signals via a communication medium. It may be possible to do so. The HMD 3 may have the functions of the information processing device 1 and be configured integrally.

The display unit 31 includes, for example, a small liquid crystal display, an optical lens, and an optical system mechanism, and is capable of displaying a three-dimensional image at a viewing angle of 110° or more. The display unit 31 is a transparent or semi-transparent glass-like display, and receives an image signal (including a video signal) output from the information processing device 1 and displays it in a superimposed manner in the operator's actual field of view. The display section 31 is not limited to a glass-like one, and may display an image based on an image signal output from the information processing device 1 superimposed on an image taken of real space by a camera provided facing forward. It's okay.

The motion detection unit 32 includes a plurality of triaxial acceleration sensors and gyro sensors installed in various directions on the main body (the cover of the display unit 31 and the attachment belt), and aggregates signals from a group of these sensors. and a control circuit that outputs the output. The movement detection unit 32 detects the movement of the wearer's head.

The space detection unit 33 includes two or more infrared cameras arranged outwardly in parallel on the outer surface of the main body, and an infrared LED disposed at an intermediate position between the infrared cameras so as to similarly emit infrared rays outward. use The space detection unit 33 functions as a depth sensor that measures the distance to an object that exists outside the main body of the HMD 3. The space detection unit 33 can measure and output the distance from the HMD 3 main body to a still object such as a wall or floor in the location of the operator wearing the HMD 3. Similarly, the space detection unit 33 can measure and output the distance from the HMD 3 main body to the operator's arm, hand, and finger.

The connection unit 34 is an interface for connecting to the information processing device 1. The HMD 3 outputs signals corresponding to the results measured by the motion detection unit 32 and the space detection unit 33 to the information processing device 1, and also acquires the video signal output from the information processing device 1 and displays it on the display unit. 31.

The HMD 3 may be provided with a speaker, and the audio output from the audio output unit 14 of the information processing device 1 may be output from the speaker.

In the haptic presentation system 100 of the third embodiment configured as described above, the information processing device 1 uses the HMD 3 and the distance to the still object in the real space measured by the space detection unit 33 to determine the three-dimensional shape of the virtual object. The original image is superimposed and displayed in accordance with the operator's movement detected by the movement detection unit 32 to realize AR display. To select an object, the space detection section 33 uses the tactile presentation device 2 to select the coordinates in the three-dimensional image of the virtual object displayed on the display section 31 of the HMD 3 (coordinates that match the real space). When it is detected that the gripped fingers are overlapped, the processing unit 10 of the information processing device 1 detects this as being selected.

In the third embodiment as well, the information processing device 1 sends and receives displacement and tilt data to and from the tactile presentation device 2 to control the image displayed in AR and the tactile sensation output by the tactile presentation device 2. do. The content of the process may be any of the processes shown in the first embodiment and the second embodiment.

By combining AR display using the HMD 3 and the tactile presentation device 2, the operator can visually observe an image in which the object is virtually placed in real space, and feel the touch of the object using the tactile presentation device. It is possible to memorize it in 2. At this time, similar to what was explained in the first embodiment, the tactile presentation start position in the tactile presentation device 2 can be corrected according to each operator, thereby further improving the degree of perception. .

The embodiments disclosed above are illustrative in all respects and are not restrictive. The scope of the present invention is indicated by the claims, and includes all changes within the meaning and range equivalent to the claims.

1 Information processing device 10 Processing section 11 Storage section 13 Display section 130 Setting screen P1 Control program 2 Tactile presentation device 20 Control section 21 Storage section 202 Displacement section 24 MRF device 25 Sensor 251 Angle sensor 252 Gyro sensor

Claims (6)

It has a displacement part that is displaceable in response to an operation by an operator, and generates a force sensation in response to the operation on the displacement part according to the amount of displacement of the displacement part, and presents a tactile sensation of the displayed object. A control device for a tactile presentation device, comprising:
Control data for generating the force sensation and posture data of the tactile sensation presentation device for changing the presentation content of the tactile sensation based on the control data are stored for each of the objects in association with the displacement amount of the displacement portion. Ok,
A control device that changes control data for generating the force sensation according to a posture of the tactile sensation presentation device. The control device according to claim 1, wherein when the posture of the tactile sensation presentation device is a specific posture, presentation of the tactile sensation by the tactile sensation presentation device is interrupted. For each object, for each posture of the tactile sensation presentation device, control data for causing the force sensation is stored in association with a displacement amount of the displacement portion,
The control device according to claim 2, wherein control data for each posture is output to the tactile presentation device. It has a displacement part that is displaceable in response to an operation by an operator, and generates a force sensation in response to the operation on the displacement part according to the amount of displacement of the displacement part, and presents a tactile sensation of the displayed object. A method for controlling a tactile presentation device, comprising:
A computer connected to the tactile presentation device,
Control data for generating the force sensation and posture data of the tactile sensation presentation device for changing the presentation content of the tactile sensation based on the control data are stored for each of the objects in association with the displacement amount of the displacement portion. Ok,
A control method including a process of changing control data for generating the force sensation according to a posture of the tactile sensation presentation device. It has a displacement part that is displaceable in response to an operation by an operator, and generates a force sensation in response to the operation on the displacement part according to the amount of displacement of the displacement part, and presents a tactile sensation of the displayed object. a tactile presentation device that
an information processing device that is communicatively connected to the tactile presentation device and includes a display unit;
The tactile presentation device further includes a sensor that detects posture,
The information processing device includes:
Control data for generating the force sensation and posture data of the tactile sensation presentation device for changing the presentation content of the tactile sensation based on the control data are stored for each of the objects in association with the displacement amount of the displacement portion. Ok,
changing control data that causes the force sensation according to the posture detected by the sensor of the tactile presentation device;
A tactile presentation system that outputs an image to the display unit according to a posture obtained from the sensor and a displacement amount of the displacement unit. It has a displacement part that is displaceable in response to an operation by an operator, and generates a force sensation in response to the operation on the displacement part according to the amount of displacement of the displacement part, and presents a tactile sensation of the displayed object. a computer connected to a tactile presentation device that
Control data for generating the force sensation and posture data of the tactile sensation presentation device for changing the presentation content of the tactile sensation based on the control data are stored for each of the objects in association with the displacement amount of the displacement portion. Ok,
A computer program that causes a computer program to execute a process of changing control data that causes the force sensation according to a posture of the tactile sensation presentation device.

Images