JP7308624B2 - Haptic-auditory presentation device and control method for tactile-auditory presentation device - Google Patents

Description

The present disclosure relates to a haptic/auditory presentation device and a control method for the haptic/auditory presentation device.

Patent Documents 1 to 3 disclose devices for presenting auditory stimulation and tactile stimulation.

JP-A-2002-359888 JP 2018-97706 A Japanese Utility Model Laid-Open No. 2-28197

However, with the devices described in Patent Documents 1 to 3, when a human recognizes auditory stimulation and tactile stimulation, he or she may feel uncomfortable.

An object of the present disclosure is to provide a tactile-auditory presentation device and a control method for the tactile-auditory presentation device, which allow the user to perceive an auditory stimulus and a tactile stimulus in a natural way.

According to the present disclosure, there is provided a movable section, a vibration imparting section that imparts vibration to the movable section, an audio output section that outputs audio, and a control section that controls the vibration imparting section and the audio output section. The control unit causes the sound output unit to output the sound when a preset time elapses after causing the vibration applying unit to apply the vibration, and the sound output unit outputs the sound for the preset time. is the time until the tactile stimulation received by the user's body part due to the vibration of the vibration applying unit is transmitted to the brain through the nerves in the body, and the sound output from the sound output unit reaches the user's ear. A tactile-auditory presentation device is provided that is set based on the difference between the time it takes to be transmitted to the brain .

According to the present disclosure, auditory stimulation and tactile stimulation can be perceived in a natural way.

It is a perspective view which shows the structure of the operating device which concerns on embodiment. It is a top view which shows the structure of the operating device which concerns on embodiment. It is a sectional view showing the composition of the operating device concerning an embodiment. It is a schematic diagram which shows the change of the operating device accompanying pressing operation. It is a figure which shows the structure of a control apparatus. 4 is a flow chart showing a first example of contents of processing by the control device; FIG. 10 is a diagram showing the relationship between the change in height and the timing of sound generation in the first example; 9 is a flowchart showing a second example of contents of processing by the control device; FIG. 10 is a diagram showing the relationship between the change in height and the timing of sound generation in the second example; 4 is a flow chart showing details of processing by the control device of the tactile and auditory presentation device triggered by a signal input from the outside.

Embodiments of the present disclosure will be specifically described below with reference to the accompanying drawings. In the present specification and drawings, constituent elements having substantially the same functional configuration may be denoted by the same reference numerals, thereby omitting redundant description.

1 is a perspective view showing the configuration of the operating device according to the embodiment, FIG. 2 is a top view showing the configuration of the operating device according to the embodiment, and FIG. 3 is the configuration of the operating device according to the embodiment. It is a cross-sectional view showing the. FIG. 3 corresponds to a cross-sectional view taken along line II in FIG. The operating device is an example of a tactile and auditory presentation device.

As shown in FIGS. 1 to 3, the operating device 100 according to the embodiment has a fixed base 110 and a decorative panel 141 inside the fixed base 110. As shown in FIGS. An electrostatic sensor 142 is provided on the side of the decorative panel 141 on which a flat plate portion 111 (to be described later) of the fixed base 110 is positioned. A movable base 130 is provided on the flat plate portion 111 side of the touch pad 140 . The movable base 130 has a flat plate portion 131 wider than the touch pad 140 in plan view, and a wall portion 132 extending from the edge of the flat plate portion 131 toward the flat plate portion 111 . The fixed base 110 has a flat plate portion 111 wider than the flat plate portion 131 in plan view, and a wall portion 112 extending upward from the edge of the flat plate portion 111 outside the wall portion 132 .

An actuator 160 is provided on the flat plate portion 111 . Actuator 160 contacts flat plate portion 111 and flat plate portion 131 . The actuator 160 is positioned substantially at the center of the flat plate portion 111 and the flat plate portion 131 in plan view. In addition, a plurality of pretension springs 150 are provided around the actuator 160 between the flat plate portions 111 and 131 to bias the flat plate portions 111 and 131 toward each other in the vertical direction. The touch pad 140 is an example of an operation section, and the movable base 130 and the touch pad 140 are included in the movable section. Also, the actuator 160 is an example of a vibration applying unit.

A panel guide 190 is provided between the wall portion 112 and the wall portion 132 to contact the wall portion 112 and the wall portion 132 . For example, the panel guide 190 has elasticity and guides the movable base 130 inside the fixed base 110 .

A plurality of reflective photointerrupters 170 are provided on the flat plate portion 111 of the fixed base 110 . The photointerrupter 170 irradiates light onto the flat plate portion 131 of the movable base 130 above it, receives the light reflected by the flat plate portion 131, and detects the distance to the portion of the flat plate portion 131 irradiated with the light. can do. For example, the photo interrupters 170 are arranged inside the four corners of the touch pad 140 in plan view. Photointerrupter 170 is an example of a detector.

A speaker 200 is attached to the outer surface of the wall portion 112 of the fixed base 110 . Speaker 200 is an example of an audio output unit.

Furthermore, a control device 180 is provided on the fixed base 110 . The control device 180 controls the actuator 160 and the speaker 200 in accordance with the operation of the touch pad 140 to present a stimulus to the user's tactile sense and a stimulus to the auditory sense of the user. The control device 180 is, for example, a semiconductor chip. Although the control device 180 is provided on the flat plate portion 111 in this embodiment, the location where the control device 180 is provided is not limited, and may be provided between the touch pad 140 and the movable base 130, for example. .

Actuator 160 may be a magnetic actuator or a piezoelectric actuator. Actuator 160 may be a magnetostrictive actuator. The direction of vibration of the actuator 160 is not particularly limited, and may be a direction perpendicular to the surface of the movable base 130, a direction parallel to the surface of the movable base 130, or any direction that can be synthesized by combining these directions. may

A movable part including the movable base 130 and the touch pad 140 is pressed by the user's fingertip 10, for example. 4A and 4B are schematic diagrams showing changes in the operating device 100 accompanying the pressing operation.

Assume that the height h of the upper surface of the touch pad 140 with respect to the upper surface of the flat plate portion 111 is _h0 in a state where no pressing operation is performed, as shown in FIG. 4A. When the movable portion is pressed, the pretension spring 150 is vertically compressed and the height h is reduced to _hx . Photointerrupter 170 can detect this change in height h through change in the distance to the lower surface of flat plate portion 131 . This change in height h is input to the controller 180 . Then, the control device 180 converts the change in the height h into a pressure load applied to the movable portion. The amount of change in height h (displacement amount) and the pressing load are examples of physical quantities.

Next, control of actuator 160 and speaker 200 by control device 180 will be described. FIG. 5 is a diagram showing the configuration of the control device 180. As shown in FIG.

The control device 180 includes a CPU (Central Processing Unit) 181 , a ROM (Read Only Memory) 182 , a RAM (Random Access Memory) 183 and an auxiliary storage section 184 . The CPU 181, ROM 182, RAM 183 and auxiliary storage unit 184 constitute a so-called computer. Each part of the controller 180 is interconnected via a bus 185 .

The CPU 181 executes various programs (for example, a pressing load determination program) stored in the auxiliary storage unit 184 .

ROM 182 is a non-volatile main memory device. The ROM 182 stores various programs, data, etc. necessary for the CPU 181 to execute various programs stored in the auxiliary storage unit 184 . Specifically, the ROM 182 stores boot programs such as BIOS (Basic Input/Output System) and EFI (Extensible Firmware Interface).

The RAM 183 is a volatile main storage device such as DRAM (Dynamic Random Access Memory) or SRAM (Static Random Access Memory). The RAM 183 functions as a work area that is developed when various programs stored in the auxiliary storage unit 184 are executed by the CPU 181 .

The auxiliary storage unit 184 is an auxiliary storage device that stores various programs executed by the CPU 181 and various data generated by the execution of the various programs by the CPU 181 .

The control device 180 has the hardware configuration as described above, and performs the following processing.

First, a first example of the contents of processing by the control device 180 will be described. FIG. 6 is a flow chart showing a first example of the contents of processing by the control device 180. As shown in FIG. FIG. 7 is a diagram showing the relationship between the change in height h and the sound generation timing in the first example.

The control device 180 monitors whether or not the pressing load on the movable portion has reached a preset press threshold (step S101), and when the pressing load reaches the press threshold (YES in step S101), the actuator 160 is driven to apply vibration to the movable portion (step S102). FIG. 7 shows changes in the height h before conversion to the pressing load, and the pressing load reaches the pressing threshold when the height h reaches the threshold Pth. A press threshold is an example of a threshold.

After that, when the preset time Δt has passed (YES in step S103), the control device 180 causes the speaker 200 to output sound (step S104). This sound is, for example, a sound imitating the sound produced when the click switch is pressed, and is an example of a pseudo click sound. For example, for a user having an average physique, the time Δt is the time t1 for a tactile sensation received by the fingertip 10 to be transmitted to the brain through the nerves in the body, and the time t1 for the sound output from the speaker 200 to reach the ear. , the difference from the time t2 until it is transmitted to the brain. For example, if the length of the nerve from the fingertip 10 to the brain is 85 cm, the time Δt can be 5 ms to 30 ms.

After that, the control device 180 monitors whether or not the pressing load on the movable part has reached a preset release threshold (step S105). Then (YES in step S105), the actuator 160 is driven to apply vibration to the movable portion (step S106). In the example shown in FIG. 7, the pressure load reaches the release threshold when the height h reaches the threshold Rth.

After that, when the preset time Δt has passed (YES in step S107), the control device 180 causes the speaker 200 to output sound (step S108). This sound is, for example, a sound imitating the sound produced when the click switch is pushed and then released, and is an example of a pseudo-click sound.

According to the operation device 100 including the control device 180 that performs the control of the first example, the tactile sensation imparted to the fingertip 10 in step S102 and the auditory stimulation generated in step S104 are combined. reaches the brain at approximately the same time. Further, the tactile stimulus applied to the fingertip 10 in step S106 and the auditory stimulus generated in step S108 reach the brain almost simultaneously. Therefore, the user can perceive auditory stimuli and tactile stimuli in a natural way.

When the pressing load reaches the pressing threshold value, not only is the actuator 160 applied with vibration to the movable part, but also the speaker 200 is caused to output sound. quickly perceived by the brain. Therefore, the user is likely to feel uncomfortable. According to the present embodiment, such discomfort can be reduced.

Next, a second example of the contents of processing by the control device 180 will be described. FIG. 8 is a flow chart showing a second example of the contents of processing by the control device 180. As shown in FIG. FIG. 9 is a diagram showing the relationship between the change in height h and the sound generation timing in the second example.

The control device 180 monitors whether or not the pressing load on the movable part has reached a preset first pressing threshold (step S201), and when the pressing load reaches the first pressing threshold (step YES in S201), the actuator 160 is driven to apply vibration to the movable portion (step S202). FIG. 9 shows changes in the height h before conversion to the pressing load, and the pressing load reaches the first pressing threshold when the height h reaches the threshold Pth1. The first press threshold is an example of the first threshold.

After that, the control device 180 monitors whether or not the pressing load on the movable part has reached a preset second pressing threshold (step S203). (YES in step S203), the speaker 200 is caused to output sound (step S204). This sound is, for example, a sound imitating the sound produced when the click switch is pressed, and is an example of a pseudo click sound. The second press threshold is, for example, when an average pressing operation is performed, the time Δt in the first example, such as 5 msec to 30 msec, has elapsed since the pressing load reached the first press threshold. It is the load that is reached in time. While the user increases the pressing load, the pressing load tends to increase, and the second pressing threshold is greater than the first pressing threshold. The second press threshold can be set according to the spring constant of the pretension spring 150, for example. In the example shown in FIG. 9, the pressure load reaches the second press threshold when the height h reaches the threshold Pth2. The second press threshold is an example of a second threshold.

After that, the control device 180 monitors whether or not the pressing load on the movable portion has reached a preset first release threshold (step S205), and the user weakens the pressing load so that the pressing load increases to the first release threshold. When the release threshold value of 1 is reached (YES in step S205), the actuator 160 is driven to apply vibration to the movable portion (step S206). In the example shown in FIG. 9, the pressing load reaches the first release threshold when the height h reaches the threshold Rth1. The first release threshold is another example of the first threshold.

After that, the control device 180 monitors whether or not the pressing load on the movable portion has reached a preset second release threshold (step S207), and when the pressing load reaches the second release threshold, (YES in step S207), the speaker 200 is caused to output sound (step S208). This sound is, for example, a sound imitating the sound produced when the click switch is pushed and then released, and is an example of a pseudo-click sound. The second release threshold is, for example, when an average pressing operation is performed, the time Δt in the first example, such as 5 ms to 30 seconds, has elapsed since the pressing load reached the first release threshold. It is the load that is reached in time. While the user weakens the pressing load, the pressing load tends to decrease, and the second release threshold is smaller than the first pressing threshold. The second release threshold can be set according to the spring constant of the pretension spring 150, for example. In the example shown in FIG. 9, the pressing load reaches the second release threshold when the height h reaches the threshold Rth2. The second release threshold is another example of the second threshold.

According to the operation device 100 including the control device 180 that performs the control of the second example, the tactile stimulus given to the fingertip 10 in step S202 and the auditory stimulus generated in step S204 are combined. reaches the brain at approximately the same time. Also, the tactile stimulus applied to the fingertip 10 in step S206 and the auditory stimulus generated in step S208 reach the brain almost simultaneously. Therefore, the user can perceive auditory stimuli and tactile stimuli in a natural way.

In the second example, after the pressing load reaches the first pressing threshold, the pressing load is weakened or maintained near the first pressing threshold before the pressing load reaches the second pressing threshold. If so, the audio is not presented in step S204. Therefore, the amplitude of the vibration presented in step S202 is adjusted, and even if the user does not increase the pressing load after reaching the first press threshold, the height h becomes the threshold after reaching the first press threshold. Pth2 may be reached. By adjusting the amplitude of the vibration in this way, it is possible to reliably present the sound in step S204. The time from reaching the first press threshold until the height h reaches the threshold Pth2 is preferably about the same as the time Δt in the first example.

Similarly, in the second example, after the pressure load reaches the first release threshold, the pressure load is conversely strengthened before the pressure load reaches the second release threshold, or the pressure load is increased near the first release threshold. , the audio is not presented in step S208. Therefore, the amplitude of the vibration presented in step S206 is adjusted, and even if the user does not reduce the pressure load after reaching the first release threshold, the height h is the threshold after reaching the first release threshold. You may make it reach|attain Rth2. By adjusting the amplitude of the vibration in this way, it is possible to reliably present the sound in step S208. The time from reaching the first press threshold until the height h reaches the threshold Rth2 is preferably about the same as the time Δt in the first example.

The operating device 100 is particularly suitable as an operating device operated by a driver of an automobile. The driver can receive vibration feedback in response to the pressing operation performed by the driver on the operating device without taking his or her line of sight away from the traveling direction of the vehicle. Moreover, in this tactile feedback, not only the tactile sensation but also the auditory stimulation similar to the sound produced when a mechanical switch is operated can be received. Furthermore, since the brain can perceive the stimulus to the sense of touch and the stimulus to the sense of hearing at almost the same time, the sense of incongruity that occurs when vibration and sound are generated at the same time is alleviated, and the stimulus to the sense of hearing and the sense of touch can be stimulated with a natural sensation. can be perceived.

Note that the operation unit is not limited to one that is pressed by the user's fingertips. For example, the operation unit may be an accelerator pedal, brake pedal, or clutch pedal of an automobile that is pressed with the sole of the foot. The length of the nerve from the brain to the sole of the foot is about twice the length of the nerve to the fingertips of the hand. Therefore, it is preferable that the time Δt of the operating device pressed with the sole is about twice the time Δt of the operating device pressed with the fingertip, eg, about 10 ms to 60 ms.

Further, the tactile and auditory presentation device is not limited to an operation device having an operation unit, and does not have an operation unit, and presents tactile and auditory stimuli triggered by signals input from the outside. It may be a device. For example, the tactile-auditory presentation device may be attached to the steering wheel of an automobile, or may be attached to the driver's seat.

FIG. 10 is a flow chart showing the details of processing by the control device of the tactile and auditory presentation device triggered by a signal input from the outside. This tactile and auditory presentation device has a movable part, an actuator, a speaker, and a control device, and the control device performs control according to the flowchart shown in FIG.

The control device monitors whether or not there is an input of a specific trigger signal from the outside (step S301), and when the trigger signal is input (YES in step S301), the actuator is driven to move the actuator. Vibration is applied to the part (step S302). For example, if the tactile-auditory presentation device is attached to the steering wheel of an automobile, the triggering signal is a signal indicating that a lane departure has been detected. Further, when the tactile and auditory presentation device is attached to the driver's seat of a car, the trigger signal is a signal indicating the approach of the rear vehicle. That is, in this tactile-auditory presentation device, vibrations and sounds are presented regardless of the intention of the user to whom they are presented.

After that, when the preset time Δt has passed (YES in step S303), the control device causes the speaker to output sound (step S304). This sound is, for example, a warning sound for the driver. The time Δt is, for example, the time t1 for a user with an average physique to transmit the tactile sensation to the brain through the nerves in the body, and the time t1 for the sound output from the speaker to enter the ear and be transmitted to the brain. It is about the difference from the time t2. For example, when the tactile and auditory presentation device is provided in the driver's seat at the driver's back, the time Δt is set to 3 msec to 20 msec, which is shorter than the time difference in the first example of the control method of the operation device 100. be able to.

The control device of this tactile and auditory presentation device operates as described above. As in the second example of the control method of the operating device 100, the control device of the tactile and auditory presentation device does not use the time Δt, but uses two levels of press thresholds and two levels of release thresholds. may be performed.

A tactile and auditory presentation device attached to an accelerator pedal or brake pedal of an automobile may perform control as shown in FIG. In this case, for example, when activating automatic braking for avoiding a frontal collision, a signal indicating activation is used as a trigger to present vibration by the actuator, and then to present voice by the speaker.

In addition, a tactile and auditory presentation device is attached to the door handle of a car, and when the door is about to be opened, the approach of the vehicle from behind triggers the presentation of vibration by the actuator, followed by the sound by the speaker. can be presented.

In addition, the tactile and auditory presentation device can also be used for smartphones, microwave ovens, refrigerators, game controllers equipped with virtual reality (VR) functions, gloves, body suits, controllers for remote control of heavy machinery, and the like.

100 operating device (tactile and auditory presentation device)
110 fixed base 130 movable base (movable part)
140 touch pad (operation unit)
141 decorative panel 142 electrostatic sensor 150 pretension spring 160 actuator (vibration applying unit)
170 Photointerrupter (Detector)
180 control device (control unit)
200 speaker (audio output unit)

Claims (6)

a movable part;
a vibration imparting portion that imparts vibration to the movable portion;
an audio output unit that outputs audio;
a control unit that controls the vibration imparting unit and the audio output unit;
has
The control unit causes the sound output unit to output the sound when a preset time elapses after causing the vibration applying unit to apply the vibration,
The preset time includes the time until the tactile stimulation received by the user's body part due to the vibration of the vibration applying unit is transmitted to the brain through the nerves in the body, and the voice output from the audio output unit. is set based on the difference from the time from entering the ear of the user to being transmitted to the brain. The movable part has an operation part that is pressed and operated,
Further comprising a detection unit that detects a pressing operation of the operation unit,
2. The tactile and auditory presentation apparatus according to claim 1, wherein the control section causes the vibration applying section to apply the vibration using detection of the pressing operation by the detecting section as a trigger. The operation unit is an operation unit that a user presses with the sole of the foot,
The preset time is between 10 milliseconds and 60 milliseconds.
3. The tactile and auditory presentation device according to claim 2, characterized in that: a movable portion having an operation portion to be pressed;
a vibration imparting portion that imparts vibration to the movable portion;
an audio output unit that outputs audio;
a control unit that controls the vibration imparting unit and the audio output unit;
a detection unit that detects a displacement amount associated with the pressing operation of the operation unit;
has
The control unit causes the vibration applying unit to apply the vibration when the displacement amount detected by the detection unit reaches a preset first threshold value, and the displacement amount detected by the detection unit. causes the audio output unit to output the audio when reaches a preset second threshold;
The tactile sense presentation device, wherein the displacement amount detected by the detection section of the operating section reaches the second threshold value due to the amplitude of the vibration applied by the vibration applying section. 5. The tactile and auditory presentation device according to claim 1 , wherein the sound output unit outputs a pseudo-click sound as the sound. a movable part;
a vibration imparting portion that imparts vibration to the movable portion;
an audio output unit that outputs audio;
A control method for a tactile and auditory presentation device comprising:
causing the sound output unit to output the sound when a preset time elapses after the vibration is applied to the vibration applying unit;
The preset time includes the time until the tactile stimulation received by the user's body part due to the vibration of the vibration applying unit is transmitted to the brain through the nerves in the body, and the voice output from the audio output unit. is set based on a difference from the time from entering the ear of the user to being transmitted to the brain.

Images