JP7192813B2 - User interface device - Google Patents

Description

The technology disclosed in this specification relates to a user interface device for providing stereophonic sound.

Patent Literature 1 discloses a technique of selecting tactile information from a library according to content and presenting a user with a vibration sense corresponding to the selected tactile sense. A user interface device that presents stereophonic sound and is equipped with a plurality of speakers is known. A related technique is disclosed in Patent Document 1.

Japanese Patent No. 6530496

In a user interface that presents stereophonic sound and is equipped with a plurality of speakers, sound source localization is not clear at positions between speakers, resulting in discontinuity in stereophonic sound.

The user interface device disclosed herein comprises a plurality of speakers arranged around a reference point where a user is located. A user interface device includes a plurality of transducers arranged in a plurality of directions when viewed from a reference point. A plurality of transducers are arranged in such a manner that they can come into contact with the user's body surface located at the reference point. A user interface device includes a control unit that controls sound waves output from a plurality of speakers and controls vibrations of a plurality of transducers. The control unit identifies the sound source direction of sound waves output from the plurality of speakers as seen from a reference point. The control unit selects a specific transducer corresponding to the direction of the sound source from among the plurality of transducers. The control unit causes the specific vibrator to output vibration based on the output sound wave when the sound wave is output.

The user interface device of the present specification specifies the direction of the sound source as seen from the reference point when outputting a certain sound. A specific transducer corresponding to the sound source direction is selected from among a plurality of transducers. Then, when sound is output from the speaker, the specific vibrator is vibrated based on the output sound. As a result, the user can be made to recognize that the sound source is localized in the direction in which the specific vibrator exists. By presenting the vibrotactile sensation to the user, the position of the sound source can be clarified, so the spatial resolution of the stereophonic sound can be improved.

The control unit may temporally control the direction of the sound source. The control unit may switch the selection of the specific transducer according to the movement of the sound source direction. The control unit may vibrate the switched specific vibrator. Details of the effect will be described in Examples.

The user interface device may further include a biometric information acquisition unit that acquires biometric information of the user. The control unit may adjust the intensity of vibrations output from the plurality of vibrators based on the acquired biological information. Details of the effect will be described in Examples.

The biometric information may be the weight of the user. The control unit may adjust the intensity of the vibration output from the plurality of vibrators to be higher as the body weight increases. Details of the effect will be described in Examples.

The biometric information acquisition unit may include a camera capable of monitoring the blinking state of the user. The biometric information may be the user's arousal level calculated based on the blinking state. The control unit may adjust the intensity of vibration output from the plurality of vibrators to be higher as the arousal level becomes lower. Details of the effect will be described in Examples.

The user interface device may include a propagation function acquisition unit that gives the user's body surface a sense of vibration with a varied frequency and acquires a propagation function that indicates frequency characteristics when the vibration propagates on the user's body surface. The control unit may adjust the intensity of vibrations output from the plurality of vibrators using a filter of an inverse function of the propagation function. Details of the effect will be described in Examples.

The user interface device may further comprise a sheet positioned at the reference point. A plurality of vibrators may be arranged on the seating surface and the back surface of the seat.

A plurality of vibrators may be arranged on clothing that can be worn by the user.

1 is a diagram illustrating a schematic configuration of a stereophonic sound system 1; FIG. 1 is a layout diagram of the stereophonic sound system 1 viewed from above; FIG. It is a figure explaining a stereophonic sound. 6 is a flowchart for explaining filter setting processing; It is an example of a propagating function and a backpropagating function.

(Schematic configuration of stereophonic sound system 1)
A schematic configuration of the stereophonic sound system 1 will be described with reference to FIG. FIG. 2 shows a layout of the stereophonic sound system 1 viewed from above. In FIG. 2, the user 2 in a seated state is indicated by a dotted line. 1 and 2, the front is the y direction, the left and right directions are the x direction, and the vertically upward direction is the +z direction.

The stereophonic sound system 1 is a system for presenting stereophonic sound in space. The stereophonic sound system 1 includes speakers 11 to 14, a seat 20, a control section 40, and a monitoring camera 50. FIG. The speakers 11 to 14 are arranged around the user's 2 reference point RP. The reference point RP is the position of the center of gravity of the human body (the position of the navel) in this embodiment.

The seat 20 has a seat surface 20s and a back surface 20b. Vibrators 21 to 24 and an acceleration sensor 32 are embedded in the rear surface 20b. Vibrators 25 to 28, a load sensor 31, and an acceleration sensor 32 are embedded in the seat surface 20s. As shown in FIG. 2, the transducers 21-28 are arranged in a plurality of directions when viewed from the reference point RP. The vibrators 21 to 28 are embedded in the seat 20 and arranged in such a manner that they can come into contact with the body surface of the seated user 2 . The load sensor 31 is a part that acquires the weight, which is the biological information of the user 2 . The acceleration sensor 32 is a part that acquires a propagation function that indicates frequency characteristics when vibration propagates on the user's body surface.

The control unit 40 has a CPU 41 and a memory 42 . The CPU 41 executes various processes according to programs stored in the memory 42 . The memory 42 is composed of a volatile memory, a nonvolatile memory, or the like. In this embodiment, the control unit 40 controls the sound waves output from the speakers 11-14 and controls the vibrations of the vibrators 21-28.

The monitoring camera 50 is a camera that monitors the state of the user. Specific examples of monitoring contents include the number of times the user 2 blinks, the length of time the eyelids are closed, the position of the head, and the like.

(Increased resolution of sound source position)
Stereophonic sound in this embodiment will be described with reference to FIG. Similar to FIG. 2, FIG. 3 is a layout diagram of the stereophonic sound system 1 viewed from above. In the stereophonic system 1, speakers 11 to 14 are arranged so as to surround the reference point RP. This makes it possible to perform sound source localization at any position within 360° around the reference point RP. Further, the stereophonic system 1 identifies the direction viewed from the reference point RP when performing certain sound source localization. Then, one or more of the transducers 21 to 28 are selected as specific transducers corresponding to the direction of the sound source. Sound is output from the selected speaker, and at the same time, a specific vibrator is vibrated with a waveform similar to that of the output sound. This allows the user to recognize that the sound source exists in the direction in which the specific vibrator exists. Since the sound source position can be clarified, the spatial resolution of stereophonic sound can be improved.

This is based on a psychophysical phenomenon called phantom sensation. Phantom sensation is a phenomenon in tactile sensation in which, when vibration stimuli are applied to two points on the skin, the stimuli are fused and the vibration position is perceived. Also, it is known that the perceived position is biased toward the stronger intensity depending on the intensity ratio of the two points of vibration stimulation. Therefore, even when a small number of vibration stimulation points are used, the user can be made to recognize various perceptual positions by changing the intensity ratio. Further, in the stereophonic sound system 1 of this embodiment, not only the sound source but also the vibration is used to interpolate the direction, thereby suppressing the sense of discontinuity of the stereophonic sound. As described above, it is possible for the user to recognize that the sound source is located in the direction in which the specific vibrator exists.

A specific example will be used for explanation. When the sound of the sound source position SL1 is output from the speaker, the sound source direction D1, which is the direction of the sound source position SL1 viewed from the reference point RP, is specified. The vibrators 27 and 28 corresponding to the sound source direction D1 are selected, and vibrations of the same intensity are output to the vibrators 27 and 28 . This allows the user to recognize that the sound source position SL1 is located in the sound source direction D1 connecting the reference point RP and the midpoints of the transducers 27 and 28 .

Similarly, when outputting the sound of the sound source position SL2 from the speaker, the vibrator 26 corresponding to the sound source direction D2 is selected and vibration is output. This allows the user to recognize that the sound source position SL2 is positioned in the sound source direction D2 that connects the reference point RP and the transducer 26 .

When the sound of the sound source position SL3 is to be output from the speaker, the transducers 24 and 26 corresponding to the sound source direction D3 are selected, and vibrations of the same intensity are output to the transducers 24 and 26. FIG. This allows the user to recognize that the sound source position SL3 is positioned in the sound source direction D3 connecting the reference point RP and the midpoints of the transducers 24 and 26 .

That is, in the stereophonic sound system 1 of the present embodiment, by realizing stereophonic sound using the transducers 21 to 28, speakers 61 to 68 are virtually arranged in addition to the speakers 11 to 14 that are actually arranged. I did. Since the distance between the speakers can be increased, the spatial resolution of stereophonic sound can be increased.

Note that the virtual speakers 61 to 68 in FIG. 3 are an example. According to the principle of the phantom sensation described above, the perceived position can be freely set by the intensity ratio of the vibration stimulation at two points. Therefore, it is possible to completely surround the reference point RP with virtual speakers.

(Processing for moving sound source position)
Further, the control unit 40 temporally controls the direction of the sound source. The selection of the specific vibrator is switched according to the movement of the sound source direction, and the switched specific vibrator is vibrated. As a result, the virtual sound source position can be moved smoothly.

A specific example will be used for explanation. Consider the case where the virtual sound source position moves from sound source position SL2 to SL4 (arrows Y1 and Y2). At the sound source position SL2, the vibrator 26 is selected and vibrates. When moving from sound source position SL2 to SL3 (arrow Y1), both transducers 26 and 24 are selected. As the sound source position approaches the sound source position SL3, the vibration of the vibrator 26 weakens and the vibration of the vibrator 24 increases. Then, when the sound source position SL3 is reached, the vibration intensities of the vibrators 26 and 24 become the same.

When the sound source position moves from the sound source position SL3 to SL4 (arrow Y2), the vibration of the vibrator 24 becomes stronger and the vibration of the vibrator 26 becomes weaker as it approaches the sound source position SL4. Then, when the sound source position SL4 is reached, only the vibrator 24 is selected and vibrates, and the vibration of the vibrator 26 stops. As a result, the sense of vibration moves from the vibrator 26 to the vibrator 24 (arrow Y0) in accordance with the movement of the sound source position from SL2 to SL4 (arrows Y1 and Y2). The phantom sensation described above enables the user to recognize that the sound source position is moving.

It should be noted that the vector in the traveling direction of the stereophonic sound (arrows Y1 and Y2) and the vector in the traveling direction of the vibrosense (arrow Y0) need to be close (in phase). In other words, it is desirable that the inner product of both vectors is large. This makes it possible for the user to recognize that the virtual sound source position is moving smoothly.

(filter setting process)
Stereophonic sound using vibratory sense causes individual differences in user recognition. The stereophonic system 1 of this embodiment has filter settings for adjusting the vibration intensity of the vibrator and the acoustic output of the speaker in order to compensate for this individual difference. Filter setting processing will be described with reference to the flowchart of FIG. This processing is executed by the CPU 41 of the control unit 40 according to the program in the memory 42 . The flow of FIG. 4 is started by turning on the stereophonic sound system 1 .

In S10, filter setting of the transducer using sweep measurement is performed. A specific description will be given. As a first step, a sweep vibration sensation with a varying frequency (eg, sweeping the frequency of sinusoidal vibration stimulus) is given to the body surface of the user 2, and the acceleration sensor 32 measures the vibration propagating on the body surface. The vibratory sensation can be given using the vibrators 21-28. Thereby, it is possible to obtain a propagation function that indicates frequency characteristics when vibration propagates on the user's body surface. In other words, the transmission of vibrations on the body surface is graphed when the vibrator is vibrated from a low frequency range to a high frequency range. FIG. 5 shows an example of the propagation function PF. The horizontal axis is the vibration frequency, and the vertical axis is the propagation intensity of the vibration. The frequency variation range may be in the audible range (approximately 20-20000 Hz).

As a second step, the inverse propagation function BF, which is the inverse function of the propagation function PF, is obtained (dotted line in FIG. 5). The backpropagation function BF is the graph of the inverse shape of the propagation function PF. As a third step, vibration is given from the vibrator according to the backpropagation function BF. As a result, it is possible to give the user a sense of vibration with a constant output in the entire frequency band of the audible range. Also, by setting the back propagation function BF for each of the vibrators 21 to 28, it is possible to present a constant output vibratory sensation to various parts of the body.

In S20, speaker filter setting using biological information (height) is performed. Different heights result in different relative positions between the ears of a seated user and the speaker. Therefore, the sound output from the speaker is adjusted according to the relative position between them. For example, since the distance between the speaker and the ear increases as the sitting height increases, the filter may be set so that the acoustic output increases as the sitting height increases. The sitting height may be calculated based on the position of the head detected by the surveillance camera 50 .

In S30, filter setting of the vibrator using the biological information (weight) is performed. A specific description will be given. Vibration propagation changes with body weight. Therefore, the vibration intensity of the vibrator is adjusted according to the weight. For example, the heavier the person, the more difficult it is for the vibration to propagate, so the filter may be set so that the heavier the person, the higher the vibration intensity. The weight of user 2 can be measured by load sensor 31 .

At S40, filter settings for transducers and speakers are performed using biofeedback. When the user 2 is in a state of low arousal (a state of feeling drowsy), hearing and tactile sensation are lowered. Therefore, filter settings are made to adjust the acoustic output and vibration intensity according to the degree of arousal.

A specific description will be given. A monitoring camera 50 is used to constantly monitor the blinking state of the user. For example, the percentage of time the eyes are closed in a predetermined period of time (eye closure rate), the number of blinks, and the like are monitored. Then, the user's arousal level is calculated based on the monitoring result. For example, it is calculated that the higher the eye closure rate is, the lower the wakefulness is. Filter settings are made so that the lower the arousal level, the higher the vibration intensity of the vibrator and the louder the sound output of the speaker.

At S60, based on the filter settings set at S10-S40, vibrations are output from the vibrators 21-28, and sound is output from the speakers 11-14.

In S70, it is determined whether or not the arousal level being monitored by the monitoring camera 50 has changed. If the determination is affirmative (S70: YES), the process proceeds to S80, and the filter setting is readjusted based on the changed wakefulness level. Then, the process returns to S60. On the other hand, if the determination is negative (S70: NO), the process proceeds to S90.

At S90, it is determined whether or not the stereophonic system 1 has been switched off. If the determination is negative (S90: NO), the process returns to S60, and if the determination is positive (S90: YES), the flow ends.

(effect)
In a stereophonic sound system, when the distance between the speakers increases, the virtual sound source position may not move smoothly. This is defined as stereophonic discontinuity. In the stereophonic sound system 1 of this embodiment, the sense of vibration given to the user can be moved so as to follow the movement of the sound source position (see arrows Y0, Y1, and Y2 in FIG. 3). The principle of phantom sensation described above allows the user to smoothly recognize the movement of the sound source position. It is possible to suppress the discontinuity of stereophonic sound.

In a stereophonic system, if an object exists between the speaker and the user, the object impedes sound propagation, thus reducing the effect of stereophonic sound (occlusion problem). For example, in the space shown in FIG. 3, consider the case where an object OB indicated by a dotted line exists on the left side (-x direction side) of the reference point RP. The object OB is, for example, a passenger sitting in the front passenger seat if the space is the interior of the vehicle. In this case, sound propagation from speakers 12 and 13 to reference point RP is blocked by object OB. Then, due to the occlusion problem, for example, when the virtual sound source position moves back and forth or left and right, the user may perceive the movement of the sound source position discontinuously. However, according to the technology of this specification, it is possible to vibrate the transducers 21, 23, 25, and 27 in the direction in which the object OB exists with the same waveform and timing as the sound waves output from the speakers 12 and 13. FIG. That is, it is possible to present the moving direction of the virtual sound source to the user by vibrotactile sensation. As a result, the user can recognize the movement of the virtual sound source position without being obstructed by the object OB, so that the sense of discontinuity in the movement of the sound source position can be alleviated.

Although specific examples of the present invention have been described in detail above, these are merely examples and do not limit the scope of the claims. The technology described in the claims includes various modifications and changes of the specific examples illustrated above. The technical elements described in this specification or in the drawings exhibit technical usefulness alone or in various combinations, and are not limited to the combinations described in the claims as of the filing. In addition, the techniques exemplified in this specification or drawings can simultaneously achieve a plurality of purposes, and achieving one of them has technical utility in itself.

(Modification)
In this embodiment, the form in which the vibrator is arranged on the seat 20 has been described, but the form is not limited to this. The vibrator may be arranged in any manner as long as it is arranged in such a manner that it can contact the user's body surface. For example, even in a mode in which the vibrator is arranged on clothing that can be worn by the user, it is possible to exhibit the effects described in this embodiment.

The numbers and positions of arrangement of the speakers and transducers in this embodiment are examples.

In this embodiment, the sound source position in the horizontal direction (xy plane direction in FIGS. 1 and 2) has been described, but the present invention is not limited to this form. By arranging a plurality of speakers and vibrators in the height direction (z direction), it is possible to achieve the effects described in this embodiment also regarding the sound source position in the height direction.

Biological information such as height, sitting height, weight, etc. is not limited to being acquired by a sensor. The biometric information may be acquired by receiving the biometric information input by the user, or may be acquired through communication from a server (not shown) or the like.

The measurement target for biofeedback (S40) is not limited to the blinking state. For example, various biological information such as heart rate, body temperature, blood pressure, etc. may be measured.

The space to which the stereophonic sound system 1 of this embodiment is applied is not limited to vehicles, and can be applied to various spaces.

1: Stereophonic system 11-14: Speaker 20: Seat 21-28: Vibrator 31: Load sensor 32: Acceleration sensor 40: Control unit 41: CPU 42: Memory 50: Surveillance camera

Claims (6)

a plurality of speakers positioned around a reference point where the user is located;
a plurality of vibrators arranged in a plurality of directions when viewed from the reference point, the plurality of vibrators being arranged in a manner capable of contacting the user's body surface positioned at the reference point;
a control unit that controls sound waves output from the plurality of speakers and controls vibrations of the plurality of transducers;
a biometric information acquisition unit that acquires biometric information of a user, the biometric information acquisition unit including a camera capable of monitoring the blinking state of the user;
with
The biological information is a user's arousal level calculated based on the blinking state,
The control unit
Identifying a sound source direction of sound waves output from the plurality of speakers viewed from the reference point,
selecting a specific transducer corresponding to the direction of the sound source from among the plurality of transducers;
causing the specific vibrator to output vibration based on the sound wave to be output when the sound wave is output ;
Adjusting the intensity of the vibration output from the plurality of oscillators to a higher level according to the lowering of the arousal level;
User interface device. a plurality of speakers positioned around a reference point where the user is located;
a plurality of vibrators arranged in a plurality of directions when viewed from the reference point, the plurality of vibrators being arranged in a manner capable of contacting the user's body surface positioned at the reference point;
a control unit that controls sound waves output from the plurality of speakers and controls vibrations of the plurality of transducers;
a propagation function acquisition unit that gives a user's body surface a sense of vibration with a varied frequency and acquires a propagation function that indicates frequency characteristics when the vibration propagates on the user's body surface;
with
The control unit
identifying a sound source direction of sound waves output from the plurality of speakers as seen from the reference point;
selecting a specific transducer corresponding to the direction of the sound source from among the plurality of transducers;
causing the specific transducer to output vibration based on the sound wave to be output when the sound wave is output ;
Adjusting the intensity of the vibration output from the plurality of oscillators using a filter of the inverse function of the propagation function;
User interface device. The control unit
temporally controlling the direction of the sound source;
switching the selection of the specific transducer according to the movement of the sound source direction;
3. The user interface device according to claim 1, wherein the switched specific vibrator is vibrated. the biometric information is the weight of the user;
2. The user interface device according to claim 1 , wherein said control unit adjusts the intensity of vibrations output from said plurality of vibrators to be higher as said body weight increases. further comprising a sheet positioned at the reference point;
The user interface device according to any one of claims 1 to 4 , wherein the plurality of vibrators are arranged on the seating surface and the back surface of the seat. The user interface device according to any one of claims 1 to 4 , wherein the plurality of vibrators are arranged on clothing wearable by the user.

Images