JP2023148853A - Control device, control method, haptic feedback system, and computer program - Google Patents

Abstract

To provide a control device, a control method, a haptic feedback system, and a computer program, for reducing processing load in a haptic feedback device.SOLUTION: A control device is for a haptic feedback device having a displacement unit that is provided to be displaced in response to an operation by an operator. The haptic feedback device is configured to generate a force sense for operation on the displacement unit according to a displacement amount of the displacement unit and present a haptic sensation of a displayed object. A start displacement amount for starting presentation of the force sense in the displacement unit and control data for generating the force sense associated with a relative displacement amount from the start displacement amount are stored in advance for each object. On the basis of a start displacement amount corresponding to a selected object and control data for each relative displacement amount, the stored control data is converted into control data associated with an absolute displacement amount of the displacement unit, to output the converted control data to the haptic feedback device.SELECTED DRAWING: Figure 3

Description

The present invention relates to a control device, a control method, a tactile presentation system, and a computer program that improve the perceptibility of tactile sensations presented by a tactile presentation device.

2. Description of the Related Art Among the five human senses, image and audio transmission technologies related to visual and auditory senses are becoming more precise, and in recent years, various technologies for presenting tactile sensations (haptics) have been proposed.

Tactile presentation technology uses mainly three types of vibrations (ERM: Eccentric Rotating Mass, LRA: Linear resonant actuator, piezo element) to reproduce the vibrations that occur when touching an image of an object displayed on a display with a built-in touch panel. Proposed. However, when presenting tactile sensations on a touch panel, it is difficult to reproduce deeper tactile sensations, that is, not only the skin of fingers but also the force sensations generated in muscles and tendons.

Based on knowledge of magnetorheological fluids, the inventors have proposed a tactile presentation device that can be operated by an operator and reproduces different sensations depending on the object (Patent Document 1, etc.).

Patent No. 6906275

It is desirable that the tactile presentation device has a compact configuration that can be worn on an operator's hand, and that the configuration is simple. Furthermore, since it is a tactile control, it is desirable that the response speed of the control is high.

The present invention has been made in view of the above circumstances, and aims to provide a control device, a control method, a tactile presentation system, and a computer program that reduce the processing load on a tactile presentation device.

A control device according to an embodiment of the present disclosure includes a displacement section that is displaceable in response to an operation by an operator, and generates a force sensation in response to an operation on the displacement section according to the amount of displacement of the displacement section. A control device for a tactile presentation device that presents a tactile sensation of an object to be displayed, the control device comprising: a starting displacement amount at which the displacement unit starts presenting the haptic sensation; and a relative displacement amount from the starting displacement amount. Control data for producing the associated force sensation is stored for each object, and based on the starting displacement amount corresponding to the selected object and the control data for each relative displacement amount. , converting the stored control data into control data associated with the absolute displacement amount of the displacement section, and outputting the converted control data to the tactile presentation device.

A control method according to an embodiment of the present disclosure includes a displacement section that is displaceable in response to an operation by an operator, and generates a force sensation in response to an operation on the displacement section according to a displacement amount of the displacement section. A method for controlling a tactile sense presentation device that presents a tactile sense of a displayed object by controlling the tactile sense presentation device, wherein a computer connected to the tactile sense presentation device determines a starting displacement amount at which the displacement portion starts presenting the haptic sense. , control data for generating the force sensation associated with a relative displacement amount from the starting displacement amount are stored for each object, and a starting displacement amount corresponding to the selected object; Converting the stored control data into control data associated with the absolute displacement amount of the displacement portion based on the control data for each relative displacement amount, and transmitting the converted control data to the tactile presentation device. Output.

A computer program according to an embodiment of the present disclosure includes a displacement section that is displaceable in response to an operation by an operator, and generates a force sensation in response to an operation on the displacement section according to a displacement amount of the displacement section. and a computer connected to a tactile presentation device that presents a tactile sensation of the object to be displayed, a starting displacement amount at which the displacement unit starts presenting the haptic sensation, and a relative displacement amount from the starting displacement amount. Control data for producing the associated force sensation is stored for each object, and based on the starting displacement amount corresponding to the selected object and the control data for each relative displacement amount. , converting the stored control data into control data associated with the absolute displacement amount of the displacement section, and outputting the converted control data to the tactile presentation device.

In the control device, control method, and computer program of the present disclosure, when the displacement portion is displaced to a predetermined displacement amount, the predetermined displacement amount at which a force sensation starts to be presented is stored as the start displacement amount. Further, in the control device of the present disclosure, control data for presenting the feel of the object is stored in association with the relative displacement amount from the starting displacement amount. This allows the amount of data to be stored to be reduced.

In the control device according to an embodiment of the present disclosure, the control data does not include data on a displacement amount corresponding to a region outside the tactile sense presentation range of the object.

In the control device of the present disclosure, the control data includes the data of the displacement amount in a range that does not cause a force sensation even in the relative displacement amount from the starting displacement amount, that is, outside the range where the touch of the object is emitted, and the data corresponding to the displacement amount. Control data is not included. This compacts the data.

A control device according to an embodiment of the present disclosure receives adjustment of at least one of control data stored in association with the relative displacement amount and the start displacement amount.

A tactile presentation system according to an embodiment of the present disclosure includes a displacement section that is displaceable in response to an operation by an operator, and generates a force sensation in response to an operation on the displacement section according to the amount of displacement of the displacement section. a tactile presentation device that generates and presents a tactile sensation of an object to be displayed, and an information processing device that is communicatively connected to the tactile presentation device and includes a display section, the information processing device that A starting displacement amount for starting presentation of a force sensation and control data for producing the force sensation, which is associated with a relative displacement amount from the starting displacement amount, are stored for each of the objects, and when the force sensation is selected, converting the stored control data into control data associated with the absolute displacement amount of the displacement portion, based on the starting displacement amount corresponding to the object and the control data for each relative displacement amount; The subsequent control data is output to the tactile presentation device, and an image is output to the display unit according to the amount of displacement of the displacement unit based on the visual data.

In the tactile presentation system of the present disclosure, when the displacement portion is displaced to a predetermined displacement amount, the predetermined displacement amount that starts to generate a force sensation as a touch is stored as the start displacement amount. On the other hand, it is preferable that the image outputted in synchronization with the tactile data be displayed even for a displacement amount smaller than the starting displacement amount. Furthermore, when outputting auditory sensations using a speaker, data from the same timing as the starting displacement amount is preferably stored and used.

According to the present disclosure, data used for controlling tactile presentation in a tactile presentation device can be made compact, and the processing load can be reduced.

FIG. 1 is a schematic diagram showing a tactile presentation system. FIG. 1 is a block diagram showing the configuration of an information processing device. It is an explanatory view showing an example of contents of a sensory DB. FIG. 2 is a block diagram showing the configuration of a tactile presentation device. 2 is a flowchart illustrating an example of a basic processing procedure for tactile presentation in the tactile presentation system. 2 is a flowchart illustrating an example of a basic processing procedure for tactile presentation in the tactile presentation system. FIG. 2 is a schematic diagram showing an example of tactile presentation in the tactile presentation system. FIG. 6 is an explanatory diagram of a method for adjusting tactile data or a starting displacement amount. It is a flowchart which shows an example of the basic processing procedure of tactile sense presentation in a tactile sense presentation system of a 2nd embodiment. It is a flowchart which shows an example of the basic processing procedure of tactile sense presentation in a tactile sense presentation system of a 2nd embodiment. FIG. 3 is a schematic diagram of a tactile presentation system in a third embodiment. It is a block diagram showing the composition of HMD of a 3rd embodiment.

The present disclosure will be specifically described with reference to drawings showing embodiments thereof. In the following embodiments, implementation of a setting method in a tactile presentation system will be described.

(First embodiment)
FIG. 1 is a schematic diagram showing a tactile presentation system 100. The tactile presentation system 100 includes an information processing device 1 and a tactile presentation device 2. The information processing device 1 and the tactile presentation device 2 are communicably connected by short-range wireless communication and exchange data with each other.

The information processing device 1 uses a smartphone as shown in FIG. The information processing device 1 may be a tablet terminal or a laptop-type PC (Personal Computer) instead of a smartphone.

The tactile presentation device 2 is a device that an operator can operate by moving his or her finger while holding the finger along the displacement portion 202 . The tactile presentation device 2 reads the position of the displacement section 202 that is displaced by the operator's finger movement, controls the built-in MRF (Magneto-Rheological Fluid) device 24 according to the position, and moves the displacement section 202 of the operator. This is a device that produces a sense of force using a reaction force (rotational resistance) to the operation of the machine, and presents a tactile sensation. The form of the displacement part 202 of the tactile presentation device 2 is not limited to that shown in FIG. 1, but may be stick-like or cushion-like covered with a cover. The tactile presentation device 2 may employ a motor, a piezo element, or the like instead of the MRF device 24, and may generate a force sensation by rotational force or vibration in response to an operation by the operator. It may be combined with something that provides a warm or cold sensation. The structure may be such that it is installed on the ground or a wall and is operated by the operator's palm or foot.

In the tactile presentation system 100, the tactile presentation device 2 cooperates with the information processing device 1 and displays the tactile sensation of the object while outputting images and sounds of the object using the information processing device 1 or other display devices, speakers, etc. is outputted by the tactile presentation device 2. In the example shown in FIG. 1, when a soft object is displayed on the display unit 13 of the information processing device 1 and the operator pushes the displacement part 202 of the tactile presentation device 2 with a finger, the tactile presentation device 2 becomes squishy or fluffy. While outputting a tactile sensation, the display unit 13 displays an image that changes as if the object is being pushed in, and the audio output unit 14 outputs an audible “squeak” sound that corresponds to the object being pushed in. Output audio.

FIG. 2 is a block diagram showing the configuration of the information processing device 1. As shown in FIG. The information processing device 1 includes a processing section 10, a storage section 11, a communication section 12, a display section 13, an audio output section 14, and an operation section 15. The processing unit 10 is a processor using a CPU (Central Processing Unit) and/or a GPU (Graphics Processing Unit). The processing unit 10 executes a process described below based on a control program P1 for tactile presentation stored in the storage unit 11.

The storage unit 11 uses nonvolatile memory such as flash memory or SSD (Solid State Drive). The storage unit 11 stores data referenced by the processing unit 10. The control program P1 (program product) is downloaded from the information processing device 1 or another program server device via the communication unit 12 and stored in an executable manner. The control program P1 stored in the storage unit 11 may be a computer program P8 stored in the computer-readable storage medium 8, which is read out and stored by the processing unit 10.

The storage unit 11 stores a sensory database (DB) 110 including tactile data, visual data, and auditory data of the object to be output. The sensory DB 110 stores, for each relative displacement amount of the displacement unit 202 in the tactile presentation device 2, tactile data and auditory data (sound) to be output at that relative displacement amount in association with the object ID that identifies the object. , visual data (images) for each displacement amount of the displacement unit 202 are stored (see FIG. 3). The sensory DB 1100 stores the displacement amount of the displacement unit 202 (hereinafter referred to as the start displacement amount) corresponding to the tactile or auditory presentation start position in association with the object ID.

The communication unit 12 is a communication module for short-range wireless communication, for example, Bluetooth (registered trademark). The processing unit 10 can transmit and receive data to and from the tactile presentation device 2 through the communication unit 12 .

The display unit 13 is a display such as a liquid crystal display or an organic EL (Electro Luminescence) display. The display unit 13 is, for example, a display with a built-in touch panel. The processing unit 10 displays, on the display unit 13, an operation screen for causing the tactile sensation presentation device 2 to output a tactile sensation and an image of the target object, based on the control program P1.

The audio output unit 14 includes a speaker and the like. The processing unit 10 causes the audio output unit 14 to output the sound of the object, music, etc. based on the control program P1.

The operation unit 15 is a user interface that allows input and output to and from the processing unit 10, and is a touch panel built into the display unit 13. The operation unit 15 may be a physical button. The operation unit 15 may be a voice input unit.

FIG. 3 is an explanatory diagram showing an example of the contents of the sensory DB 110. As shown in FIG. 3, the sensory DB 110 stores, as tactile data, the current value of the current to the MRF device of the tactile presentation device 2 for each relative displacement amount (angle) of the displacement unit 202 in association with the object ID. ing. It is not limited to a current value, but may be a voltage value or the like. The sensory DB 110 also stores the starting displacement amount in association with the object ID. Here, the relative displacement amount corresponds to the difference from the starting displacement amount. The objects are plural, and may include real objects that present a tactile sensation, such as still life objects such as balls or balloons, foods such as gummies, vegetables, or fruits, and animals such as dogs, cats, or fish. Characters such as slimes and monsters may also be included.

The relationship between the relative displacement amount of the displacement part 202 of the sensory DB 110 and the current value is set according to the actual size of the movable range of the displacement part 202 and the size of the object. For example, if the movable range of the displacement unit 202 is 50 mm, and the target object is a sphere that will not burst, such as a small gummy candy, and the size is set to 15 mm, the relative displacement amount in the tactile data is as shown in FIG. As shown in FIG. In addition, if the target object is a large object such as the seat surface of a sofa, and the size greatly exceeds the movable range of the displacement unit 202, the tactile data may be such as to present the sensation up to 50 mm from the surface of the sofa seat surface. The current value is set. If the target object is a virtual character such as a monster, the current value is set based on the size according to the setting of the character.

Similarly, the sensory DB 110 stores sounds for each relative displacement amount (angle) of the displacement unit 202 as auditory data in association with the object ID. Auditory data is audio data that reproduces the sound emitted when an object is touched. The auditory data may be waveform data that differs depending on each angle. It may also be a timestamp of the audio corresponding to each angle.

The sensory DB 110 stores images (frame images) for each displacement amount (angle) of the displacement unit 202 as visual data in association with the object ID. Unlike tactile data and auditory data, visual data can be visually captured before contact, so image data is are associated. Here, the frame image is one still image that is recognized as an animation image when displayed continuously.

FIG. 4 is a block diagram showing the configuration of the tactile presentation device 2. As shown in FIG. As shown in FIG. 1, the tactile presentation device 2 is configured by providing a flat bottomed cylindrical grip 200 with a band-shaped flat plate displacement portion 202 having a curved portion partially along the circumferential direction. Although the displacement part 202 is made of a flexible material, it may also be made of a highly rigid material and rotatably supported via the grip body 200 and a support shaft. A binding tool 203 in the form of a cloth tape is provided on the outer surface of the distal end of the displacement portion 202 . A link mechanism 204 is provided on the inner surface of the distal end of the displacement portion 202 to connect to the rotating shaft of the rotor of the MRF device 24 housed inside the gripper 200 .

As shown in FIG. 1, the operator uses the holding body 200 by, for example, holding the holding body 200 with the thumb and middle finger, and inserting the finger such as the index finger into the binding tool 203 along the displacement part 202. The operator can move the displacement part 202 by pushing in the index finger, or can move the displacement part 202 away from the grip body 200 by extending the index finger.

The tactile presentation device 2 includes a gripping body 200 as shown in FIG. The grip body 200 has an MRF device 24 built therein. The control section 20, the storage section 21, the communication section 22, and the power supply section 23 may be provided integrally with the grip body 200, or may be provided as separate bodies connected to the grip body 200 wirelessly or by wire.

The control unit 20 includes a processor such as a CPU and an MPU (Micro-Processing Unit), and a memory such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The control unit 20 is, for example, a microcontroller. The control unit 20 controls each component based on a control program P2 stored in the built-in ROM to realize tactile presentation.

The storage unit 21 is an auxiliary storage memory for the control unit 20, and stores control data (tactile data) of the MRF device 24 in a rewritable manner.

The communication unit 22 is a communication module for short-range wireless communication, for example, Bluetooth (registered trademark). The control unit 20 can transmit and receive data to and from the information processing device 1 through the communication unit 22 .

The control unit 20 is connected to a power supply unit 23, an MRF device 24, and a sensor 25 via I/O, and exchanges signals with each other.

Power supply unit 23 includes a rechargeable battery. The power supply unit 23 supplies power to each component and the MRF device 24 when turned on.

The MRF device 24 has a yoke that sandwiches a disk-shaped rotor with a gap between them, and generates a magnetic field by passing a control current through a coil provided in the yoke, thereby generating a magnetic field sealed in the gap. The viscosity (shear stress) of the viscous fluid is controlled to provide rotational resistance to the rotor. When the control unit 20 controls the magnitude of the control current to the MRF device 24, the rotational resistance is immediately changed.

The sensor 25 measures the amount of displacement (angle) of the displacement section 202 and outputs it to the control section 20 . The sensor 25 measures and outputs the displacement of the displacement portion 202 as an angle. The sensor 25 may include a plurality of sensors such as a gyro sensor and an acceleration sensor.

In the tactile presentation device 2 configured as described above, when the displacement section 202 is operated by the operator, the displacement of the displacement section 202 is caused to move in the direction of rotation of the MRF device 24 toward the rotation axis of the rotor via the link mechanism 204. communicated. Since the rotating shaft rotates freely when the MRF device 24 is not operating, that is, while the control current is zero, the displacement portion 202 fluctuates without resistance. On the other hand, when the MRF device 24 operates and the control current is not zero, the viscosity (shear stress) of the magnetorheological fluid inside the MRF device 24 is changed depending on the magnitude of the current flowing to the MRF device 24. By continuously changing the magnitude of the current to the MRF device 24 or by vibrating the current value at a predetermined frequency, the control unit 20 can change the resistance force applied to the displacement unit 202 and the method of its appearance. .

In this way, the tactile sensation presentation device 2 can vary the resistance (current value) according to the pushing amount (displacement amount) of the displacement section 202 to present a smooth tactile sensation, or increase the resistance as the pushing amount increases. It is possible to present a tactile sensation that is hard and hard, or to present a tactile sensation that is crunchy by repeatedly increasing and decreasing the resistance.

5 and 6 are flowcharts showing an example of a basic processing procedure for tactile presentation in the tactile presentation system 100. When the operator starts the control program P1 and turns on the power of the tactile presentation device 2, the processing unit 10 of the information processing device 1 starts the following process in cooperation with the tactile presentation device 2.

The processing unit 10 displays on the display unit an operation screen including a list of object candidates for which tactile sensation is to be displayed (step S101), and accepts selection of the object (step S102). The processing unit 10 reads out the tactile data, auditory data (sound), starting displacement amount, and visual data (image) corresponding to the object ID of the selected object from the sensory DB 110 of the storage unit 11 (step S103).

The processing unit 10 converts the read tactile data into tactile data that is an absolute displacement amount obtained by adding the relative displacement amount to the starting displacement amount, and temporarily stores it (step S104). The processing unit 10 converts the read auditory data into auditory data that is a starting displacement amount and an absolute displacement amount obtained by adding a relative displacement amount to the starting displacement amount, and temporarily stores the auditory data (step S105).

The processing unit 10 temporarily stores the read visual data (step S106).

The processing unit 10 establishes communication between the tactile presentation device 2 and the communication unit 12 (step S107), and transmits the temporarily stored converted tactile data to the tactile presentation device 2 (step S108). The processing unit 10 causes the display unit 13 to display that tactile presentation is to be started (step S109).

Upon receiving the tactile data (step S201), the control unit 20 of the tactile presentation device 2 stores it in the storage unit 21 (step S202).

The control unit 20 samples a signal corresponding to the displacement amount (angle) of the displacement unit 202 output from the sensor 25 (step S203). The control unit 20 transmits the displacement amount obtained by sampling to the information processing device 1 (step S204), and refers to the current value corresponding to the obtained displacement amount from the tactile data stored in the storage unit 21 in step S202. (step S205), outputs the referenced current to the MRF device 24 (step S206), and returns the process to step S203. In step S205, if the displacement amount corresponding to the obtained displacement amount cannot be referenced (is not included) in the tactile data, the control unit 20 skips the process of step S206. The processing in steps S203 to S206 continues until an operation to end the processing is performed on the information processing device 1 side.

The information processing device 1 receives the amount of displacement from the tactile presentation device 2 (step S110), and the processing unit 10 temporarily stores the image and audio corresponding to the received amount of displacement, visual data and converted auditory data. (Step S111). The processing unit 10 outputs the referenced visual data and auditory data from the display unit 13 and the audio output unit 14 (step S112), and returns the process to step S110. Each time the amount of displacement is transmitted from the tactile presentation device 2, the information processing device 1 outputs an image and audio corresponding to the amount of displacement.

The processing unit 10 determines whether a termination operation has been performed (step S113). If it is determined that the end operation has not been performed (S113: NO), the process returns to step S110, and the processes of steps S110 to S113 are repeated.

If it is determined that the end operation has been performed (S113: YES), the processing unit 10 ends the display, disconnects communication with the tactile presentation device 2 (step S114), and ends the process.

FIG. 7 is a schematic diagram showing an example of tactile presentation in the tactile presentation system 100. FIG. 7 shows changes in images and sounds displayed on the display unit 13 in response to the amount of displacement in the tactile data sent to the tactile presentation device 2. As shown in FIG. 7, the image changes depending on the amount of depression (displacement amount) of the displacement portion 202, and the way the dent corresponds to the tactile sensation changes. Specifically, from the starting displacement amount of 10°, the tactile sensation is presented on the tactile sensation presentation device 2 at the displacement amount at which the finger starts to touch, and the image changes. As shown in FIG. 7, when displaying an image of an operator's finger in an image, the finger image in the image is The object may be displayed so as to touch the object.

In the tactile presentation system 100 that presents a tactile sensation in this manner, tactile data is stored in the sensation DB 110 as data of a relative displacement amount from a starting displacement amount. This can be expected to reduce the size of haptic data and reduce the processing load.

(Second embodiment)
By storing the starting displacement amount and the tactile data as separate data, it becomes easy to adjust only the starting displacement amount for each operator, or to adjust only the tactile data by expanding or contracting it according to the scale. .

The configuration of the tactile presentation system 100 in the second embodiment is the same as that of the tactile presentation system 100 in the first embodiment except for the adjustment method. Therefore, among the configurations of the tactile presentation system 100 of the second embodiment, the same components as those of the tactile presentation system 100 of the first embodiment are given the same reference numerals, and detailed description thereof will be omitted.

FIG. 8 is an explanatory diagram of a method for adjusting tactile data or a starting displacement amount. As shown in FIG. 8, when using tactile data in which current values to the tactile presentation device 2 are associated with relative displacement amounts from 0° to 48°, the processing unit 10 sets the scale to half and uses 0. It is possible to adjust to tactile data with a relative displacement amount of ° to 24 °. Furthermore, when the sensory DB 110 stores the starting displacement amount as 5° for the same object, the processing unit 10 can make an adjustment to add 5 to the starting displacement amount. In this case, the converted tactile data sent to the tactile presentation device 2 is such that, as shown in FIG. This is the data. Although FIG. 8 shows an example in which both the starting displacement amount and the tactile data are adjusted, the present invention is not limited to this, and only one of the starting displacement amount and the tactile data may be adjusted. Note that changing the scale and adjusting the starting displacement amount are also applied to auditory data and visual data.

9 and 10 are flowcharts illustrating an example of the basic processing procedure for tactile presentation in the tactile presentation system 100 of the second embodiment. Among the processing procedures shown in the flowcharts of FIGS. 9 and 10, the same steps as those shown in the flowcharts of FIGS. 5 and 6 shown in the first embodiment are given the same step numbers and detailed The explanation will be omitted.

When the scale is set in the second embodiment, the processing unit 10 selects the tactile data, the auditory data, the start displacement amount, and the start The scale of one of the displacement amounts is adjusted (step S121). Step S121 corresponds to the adjustment at the upper right in FIG.

Based on the starting displacement amount and tactile data after adjusting the scale, the tactile data is converted into an absolute displacement amount obtained by adding the relative displacement amount to the starting displacement amount, and is temporarily stored (step S122).

Similarly, the processing unit 10 converts the starting displacement amount and auditory data after the scale adjustment into auditory data that is an absolute displacement amount obtained by adding a relative displacement amount to the starting displacement amount and the starting displacement amount, and temporarily stores the data. (Step S123).

The processing unit 10 temporarily stores the scale-adjusted visual data (step S124). Thereafter, the processing unit 10 executes the processing of steps S107-S114.

In this way, by separately storing control data for the starting displacement amount and the relative displacement amount in the sensation DB 110, scale adjustment and correction processing becomes easy.

(Third embodiment)
In the third embodiment, an HMD (Head Mounted Display) is used to present visual information regarding an object as a three-dimensional image. FIG. 11 is a schematic diagram of a tactile presentation system 100 in the third embodiment, and FIG. 12 is a block diagram showing the configuration of the HMD 3. The configuration of the tactile presentation system 100 in the third embodiment is the same as that of the tactile presentation system 100 in the first embodiment, except that the HMD 3 is used and the details of the processing by using the HMD 3 are different. . Therefore, among the configurations of the tactile presentation system 100 of the third embodiment, the same components as those of the tactile presentation system 100 of the first embodiment are given the same reference numerals, and detailed description thereof will be omitted.

The HMD 3 includes a display section 31, a motion detection section 32, a space detection section 33, and a connection section 34. The HMD 3 may be provided with a display section 31, a motion detection section 32, a space detection section 33, and a connection section 34 in the main body, or some may be provided separately and mutually exchange control signals via a communication medium. It may be possible to do so. The HMD 3 may have the functions of the information processing device 1 and be configured integrally.

The display unit 31 includes, for example, a small liquid crystal display, an optical lens, and an optical system mechanism, and is capable of displaying a three-dimensional image at a viewing angle of 110° or more. The display unit 31 is a transparent or semi-transparent glass-like display, and receives an image signal (including a video signal) output from the information processing device 1 and displays it in a superimposed manner in the operator's actual field of view. The display section 31 is not limited to a glass-like one, and may display an image based on an image signal output from the information processing device 1 superimposed on an image taken of real space by a camera provided facing forward. It's okay.

The motion detection unit 32 includes a plurality of triaxial acceleration sensors and gyro sensors installed in various directions on the main body (the cover of the display unit 31 and the attachment belt), and aggregates signals from a group of these sensors. and a control circuit that outputs the output. The movement detection unit 32 detects the movement of the wearer's head.

The space detection unit 33 includes two or more infrared cameras arranged outwardly in parallel on the outer surface of the main body, and an infrared LED disposed at an intermediate position between the infrared cameras so as to similarly emit infrared rays outward. use The space detection unit 33 functions as a depth sensor that measures the distance to an object that exists outside the main body of the HMD 3. The space detection unit 33 can measure and output the distance from the HMD 3 main body to a still object such as a wall or floor in the location of the operator wearing the HMD 3. Similarly, the space detection unit 33 can measure and output the distance from the HMD 3 main body to the operator's arm, hand, and finger.

The connection unit 34 is an interface for connecting to the information processing device 1. The HMD 3 outputs signals corresponding to the results measured by the motion detection unit 32 and the space detection unit 33 to the information processing device 1, and also acquires the video signal output from the information processing device 1 and displays it on the display unit. 31.

The HMD 3 may be provided with a speaker, and the audio output from the audio output unit 14 of the information processing device 1 may be output from the speaker.

In the tactile presentation system 100 of the second embodiment configured as described above, the information processing device 1 uses the HMD 3 and the distance to the still object in the real space measured by the space detection unit 33 to determine the three-dimensional shape of the virtual object. The original image is superimposed and displayed in accordance with the operator's movement detected by the movement detection unit 32 to realize AR display. To select an object, the space detection section 33 uses the tactile presentation device 2 to select the coordinates in the three-dimensional image of the virtual object displayed on the display section 31 of the HMD 3 (coordinates that match the real space). When it is detected that the gripped fingers are overlapped, the processing unit 10 of the information processing device 1 detects this as being selected.

Also in the third embodiment, the information processing device 1 sends and receives displacement and tilt data to and from the tactile presentation device 2 to control the image displayed in AR and the tactile sensation output by the tactile presentation device 2. do. The content of the process may be any of the processes shown in the first embodiment and the second embodiment.

By combining the AR display using the HMD 3 and the tactile presentation device 2, the operator can visually observe an image in which the object is virtually placed in real space, and feel the touch of the object using the tactile presentation device. It is possible to memorize it in 2. At this time, also in the second embodiment, the tactile data can be made compact, and the processing load can be reduced, similar to what was explained in the first embodiment. This can be expected to provide an immediate tactile response.

The embodiments disclosed above are illustrative in all respects and are not restrictive. The scope of the present invention is indicated by the claims, and includes all changes within the meaning and range equivalent to the claims.

1 Information processing device 10 Processing unit 11 Storage unit 13 Display unit 130 Setting screen P1 Control program 2 Tactile presentation device 20 Control unit 21 Storage unit 202 Displacement unit 24 MRF device

Claims (6)

It has a displacement part that is displaceable in response to an operation by an operator, and generates a force sensation in response to the operation on the displacement part according to the amount of displacement of the displacement part, and presents a tactile sensation of the displayed object. A control device for a tactile presentation device, comprising:
A starting displacement amount for starting presentation of the force sensation in the displacement unit and control data for producing the force sensation, which is associated with a relative displacement amount from the starting displacement amount, are stored for each of the objects. Keep it
Converting the stored control data into control data associated with the absolute displacement amount of the displacement portion based on the starting displacement amount corresponding to the selected object and the control data for each relative displacement amount. ,
A control device that outputs the converted control data to the tactile presentation device. The control device according to claim 1, wherein the control data does not include data on a displacement amount corresponding to an area outside the tactile sense presentation range of the target object. The control device according to claim 2, wherein the control device receives adjustment of at least one of control data stored in association with the relative displacement amount and the start displacement amount. It has a displacement part that is displaceable in response to an operation by an operator, and generates a force sensation in response to the operation on the displacement part according to the amount of displacement of the displacement part, and presents a tactile sensation of the displayed object. A method for controlling a tactile presentation device, comprising:
A computer connected to the tactile presentation device,
A starting displacement amount for starting presentation of the force sensation in the displacement unit and control data for producing the force sensation, which is associated with a relative displacement amount from the starting displacement amount, are stored for each of the objects. Keep it
Converting the stored control data into control data associated with the absolute displacement amount of the displacement portion based on the starting displacement amount corresponding to the selected object and the control data for each relative displacement amount. ,
A control method including a processing unit that outputs converted control data to the tactile presentation device. It has a displacement part that is displaceable in response to an operation by an operator, and generates a force sensation in response to the operation on the displacement part according to the amount of displacement of the displacement part, and presents a tactile sensation of the displayed object. a tactile presentation device that
an information processing device that is communicatively connected to the tactile presentation device and includes a display unit;
The information processing device includes:
A starting displacement amount for starting presentation of the force sensation in the displacement unit and control data for producing the force sensation, which is associated with a relative displacement amount from the starting displacement amount, are stored for each of the objects. Keep it
Converting the stored control data into control data associated with the absolute displacement amount of the displacement portion based on the starting displacement amount corresponding to the selected object and the control data for each relative displacement amount. ,
outputting the converted control data to the tactile presentation device;
A tactile presentation system that outputs an image to the display unit according to the amount of displacement of the displacement unit based on visual data. It has a displacement part that is displaceable in response to an operation by an operator, and generates a force sensation in response to the operation on the displacement part according to the amount of displacement of the displacement part, and presents a tactile sensation of the displayed object. a computer connected to a tactile presentation device that
A starting displacement amount for starting presentation of the force sensation in the displacement unit and control data for producing the force sensation, which is associated with a relative displacement amount from the starting displacement amount, are stored for each of the objects. Keep it
Converting the stored control data into control data associated with the absolute displacement amount of the displacement portion based on the starting displacement amount corresponding to the selected object and the control data for each relative displacement amount. ,
A computer program that executes a process of outputting the converted control data to the tactile presentation device.

Images