JP6086453B2 - Acoustic system and method for setting virtual sound source thereof - Google Patents

Description

  The present invention relates to an acoustic system that can be suitably used mainly for a mixed reality system and a method for setting a virtual sound source thereof.

  In recent years, mixed reality (MR) that amplifies and expands real-world information by superimposing and displaying virtual information such as CG (computer graphics) images and characters on real-world (real space) images taken by a camera. : Mixed Reality) Research and development are actively conducted.

  In addition, the direction and distance of the sound source is such that sound is generated from virtual information superimposed on the real environment by combining auditory elements with such visual mixed reality technology. Attempts have been made to obtain a higher sense of reality by localizing the sound (sound image localization). For example, Patent Document 1 discloses a technique in which a plurality of sound generation sources are provided inside a headphone worn by a listener, and the listener is localized with a sound image according to the position of the sound generation source that generates the sound. .

JP-A-5-336599

  However, in the technique of Patent Document 1, since it is assumed that the headphone is worn by the listener, only a single person wearing the headphone can experience the virtual environment, and the head-related transfer function is personal. There is also a problem that the sound image localization accuracy is lowered due to the difference. In addition, the listener is bothered by wearing headphones.

  The present invention has been made in view of the above circumstances, and sets a virtual sound source in a space between a reflective surface such as a wall surface of a room and a listener, and accurate sound image localization without wearing headphones. It is an object of the present invention to provide an acoustic system and its construction method.

The acoustic system of the present invention includes a signal source that generates a signal wave in an audible band, a plurality of ultrasonic speakers that generate a carrier wave in an ultrasonic band, and emit a modulated wave obtained by modulating the carrier wave with the signal wave; In order to set a virtual sound source of the signal wave at a predetermined position of a transmission path of the modulated wave in the air between a reflecting surface that reflects the modulated wave radiated from the ultrasonic speaker and the listener, A control unit that controls sound pressure, and the control unit is set at a plurality of positions at different distances from the listener in the transmission path that linearly connects the reflection position on the reflecting surface and the listener. setting one of the control point on in order to set the virtual sound source, based on the sound pressure of the control points to be the virtual sound source, the same sound pressure and virtual sound source more at a long distance control points from the listener , More in control point distance is closer from the listener, characterized in that a filter to be set to a smaller sound pressure than the virtual sound source.

The method of setting a virtual sound source in the acoustic system of the present invention radiates a modulated wave obtained by modulating a carrier wave in an ultrasonic band with a signal wave in an audible band by an ultrasonic speaker, and reflects the emitted modulated wave on a reflecting surface. A plurality of control points having different distances from the listener are set in the transmission path of the modulated wave that linearly connects the reflection position on the reflecting surface and the listener, and the sound pressure at any of the control points is used as a reference. The sound pressure at the control point farther away from the listener is set to the same sound pressure as any of the control points, and the sound pressure at the control point closer to the listener is smaller than any of the control points. by setting, and sets the one of the control points to the virtual sound source of the signal wave.

  In the acoustic system and the virtual sound source setting method of the present invention, a modulated wave is generated and radiated by modulating a carrier wave in an ultrasonic band with a signal wave, and the modulated wave is reflected on a reflecting surface to be generated from the modulated wave. Listening sound (signal wave) is transmitted to the listener. In addition, an audible virtual sound source is set between the reflecting surface and the listener. For this reason, it is possible to give the listener a sense that sound is generated from an aerial position and direction different from the position of the actual sound source, and to perform accurate sound image localization without wearing headphones or the like Is possible.

Thus, by setting a plurality of control points in the transmission path of the modulated wave and setting the relative relationship of the sound pressures at these control points to a predetermined value, the virtual sound source can be set appropriately. For example, if the sound pressure at one control point is set higher than the sound pressure at the other control point, the listener can obtain a sense that sound is generated from the one control point. One control point can be recognized as a virtual sound source .

It is the schematic which shows the usage type of the acoustic system which concerns on embodiment of this invention. It is a block diagram of an acoustic system. It is explanatory drawing which shows the acoustic transmission system model for setting the characteristic of a filter. It is explanatory drawing which shows the basic model of an acoustic transmission system. It is a figure which shows the relationship between input / output, a transfer function, and a filter. It is a graph of a sound pressure level-frequency characteristic showing an output result when a filter is not used. It is a graph of a sound pressure level-frequency characteristic showing an output result when a filter is set according to condition 1. It is a graph of a sound pressure level-frequency characteristic showing an output result when a filter is set according to condition 2. 6 is a graph of sound pressure level-frequency characteristics showing an output result when a filter is set according to condition 3. It is a graph of the sound pressure level-frequency characteristic of the original signal input to the acoustic system. (A) is a graph which shows the evaluation result of the sound image localization of the subjective virtual sound source by the listener, and (b) is a table showing the evaluation result. (A) is a schematic plan view which shows the usage type of an acoustic system at the time of providing the reflective surface (reflective member) which concerns on other embodiment in the wall surface of a room, (b) is a perspective view of a reflective member. It is a schematic plan view which shows the usage type of an acoustic system at the time of providing the reflective surface (reflective member) which concerns on other embodiment on the wall surface of a room. It is a perspective view which shows the ultrasonic speaker which concerns on other embodiment. It is a schematic plan view which shows the usage pattern of the ultrasonic speaker shown by FIG.

<< Configuration of acoustic system >>
FIG. 1 is a schematic view showing a usage pattern of an acoustic system according to an embodiment of the present invention.
The acoustic system 10 of the present embodiment transmits the sound radiated from the speaker SP in the room to the listener H in a specific place in the room R, and the sound source is at the position of the actual speaker SP. Instead, the sound image is localized as if it is emitted from the virtual sound source SS set in the indoor space. Therefore, the acoustic system 10 is used in combination with the mixed reality (MR) technology that superimposes and displays the virtual space on the real space, for example, a virtual image in which sound emitted from the speaker SP is superimposed and displayed in the air. It is possible to give the listener H a sensation as if it originated from

In the present embodiment, an ultrasonic speaker is used as the speaker SP. An ultrasonic speaker uses ultrasonic waves that cannot be perceived as sound by humans at a high frequency of 20 kHz or higher as a carrier wave. It radiates. Since the modulated wave is distorted by the nonlinearity of air in the process of propagating in the air, the audible signal wave self-demodulates due to this distortion, and a highly directional sound field is formed. .
In this embodiment, the virtual sound source SS is set in the transmission path of the modulated wave reflected by the wall surface (reflection surface) W of the room R.

FIG. 2 is a block diagram of the acoustic system.
The acoustic system 10 of this embodiment includes a signal source 21, a filter processing unit 22, a carrier wave generation unit 23, a modulation unit 24, an amplification unit 25, and an oscillator 26. Among these, the carrier wave generation unit 23, the modulation unit 24, the amplification unit 25, and the oscillator 26 are components of the ultrasonic speaker (ultrasonic speaker system) SP of the present embodiment.
The signal source 21 generates an audible signal wave such as an audio signal or an audio signal, and outputs the signal wave to the filter processing unit 22. The filter processing unit 22 includes a filter for setting the virtual sound source SS in the indoor space, and gives a predetermined characteristic to the signal wave by this filter and outputs the signal wave to the modulation unit 24. Therefore, in this embodiment, the filter processing unit 22 constitutes the control unit of the present invention.

The carrier wave generation unit 23 generates a carrier wave composed of ultrasonic waves having a predetermined frequency and outputs the carrier wave to the modulation unit 24.
The modulation unit 24 widens and modulates the carrier wave input from the carrier wave generation unit 23 with the signal wave input from the filter processing unit 22 to generate a modulated wave. The modulated wave is radiated from the oscillator 26 in a state amplified by the amplifying unit 25.

  The signal source 21, the filter processing unit 22, the carrier wave generation unit 23, and the modulation unit 24 of the present embodiment are obtained from a personal computer including an arithmetic unit such as a CPU, a memory, a storage unit such as an HDD, and an input / output interface. It is configured. The personal computer functions as the signal source 21, the filter processing unit 22, the carrier wave generation unit 23, and the modulation unit 24 by executing software installed in the personal computer.

《Filter setting method》
Next, a method for setting a filter provided in the filter processing unit 22 will be described. Prior to that, a method for transmitting sound between a sound source and a sound receiving point will be described using a basic model of an acoustic transmission system as an example. .
(Basic model of acoustic transmission system)
FIG. 4 is an explanatory diagram showing a basic model of an acoustic transmission system. In this basic model, one sound source (speaker) SP and one sound receiving point (microphone) MIC are installed in a room R. A signal x (t) is input to the speaker SP, and the microphone MIC collects a signal radiated from the speaker SP and outputs a signal y (t).

  In FIG. 4, G (z) is a transfer function representing how sound is transmitted between the speaker SP and the microphone MIC. This transfer function G (z) is obtained by z-transforming a discrete number sequence g (0), g (1), g (2),... Obtained by sampling the impulse response observed by the microphone MIC. . Then, assuming that the signals x (t) and y (t), respectively z-transformed, are X (z) and Y (z), respectively, there is a relationship shown in Expression (1).

  On the other hand, when an input of X (z) = 1 is given to such an acoustic transmission system, the following equation (2) can be obtained.

  Therefore, the transfer function G (z) can be obtained by obtaining the output Y (z) obtained by the input X (z) = 1. In this embodiment, the filter characteristics are set using the relationship between the inputs / outputs X (z), Y (z), and the transfer function G (z), as will be described below.

(Acoustic transmission system model of embodiment)
FIG. 3 is an explanatory diagram of an acoustic transmission system model for setting filter characteristics. In order to set the virtual sound source SS on the transmission path of the modulated wave radiated from the ultrasonic speaker SP and reflected by the reflecting surface W, a plurality of control points are set on the transmission path, and the sound pressure at the control point is set. Control by filter. In the present embodiment, three control points are set on the transmission path of the modulated wave, and microphones MIC ₁ to MIC ₃ are provided at each control point.

It has been established by “MINT (Multiple Input / Output Inverse Theory) theory” that the sound pressures of N control points can be controlled by using (N + 1) or more speakers. Therefore, also in this embodiment, based on this “MINT theory”, four ultrasonic speakers SP _{1 to} SP ₄ are used for the _three control points MIC _{1 to} MIC ₃ .

Then, enter the original signal X (z) into four ultrasonic speaker _SP 1 to SP _4, a modulated wave radiated from the ultrasonic speaker _SP 1 to SP ₄ is reflected by the reflecting surface W, the modulated wave Are collected by a plurality of microphones MIC ₁ , MIC ₂ , and MIC ₃ arranged at control points (sound receiving points). The outputs of the microphones MIC ₁ , MIC ₂ , and MIC ₃ are Y ₁ (z), Y ₂ (z), and Y ₃ (z), respectively. The input X (z) is filtered by the filters H ₁ (z), H ₂ (so that the desired outputs Y ₁ (z) -Y ₃ (z) can be obtained at the respective control points MIC ₁ -MIC ₃ . z), H ₃ (z), and H ₄ (z) are processed into predetermined characteristics.

FIG. 5 is a diagram illustrating the relationship between input / output signals, transfer functions, and filters.
When the input X (z) is radiated from the ultrasonic speakers SP _{1 to} SP ₄ after being processed by the _four filters H ₁ (z) to H ₄ (z) and picked up by one microphone MIC ₁ , Transfer functions representing how sound is transmitted between the ultrasonic speakers SP _{1 to} SP ₄ and the microphone MIC ₁ are G _1,1 (z), G _1,2 (z), G _1,3 (z), It can be represented by G _1,4 (z). The transfer functions G _1,1 (z) to G _1,4 (z) can be obtained by a method similar to the method described above with reference to the basic model (FIG. 4). The output Y ₁ (z) of the microphone MIC ₁ can be expressed as shown in Expression (3) using the transfer functions G _1,1 (z) to G _1,4 (z).
Similarly, for the output Y ₂ (z) and the output Y ₃ (z), using the transfer functions G _{2,1 to} G _2,4 , G _{3,1 to} G _3,4 , the expressions (4) and (4) It can be expressed as shown in (5).

  Here, when the output Y (z), the transfer function G (z), and the filter H (z) are expressed as matrices as shown in equations (6) to (8), the above equations (3) to (5) are expressed as follows. , Can be expressed as in equation (9).

When designing the filter, the input X (z) = 1 is input to obtain Expression (10). Then, by using the inverse matrix G ⁻¹ (z) of the transfer function G (z), the filter H (z) can be obtained as in Expression (11).

In designing the filter using Equation (11), it is necessary to set the values of the outputs Y ₁ (z) to Y ₃ (z) of the microphones MIC _{1 to} MIC ₃ and their relative relationship. Here, when the position of the microphone MIC ₁ that is farthest from the listener H is to be set as the virtual sound source SS (condition 1), the microphone MIC ₁ reproduces the signal wave that is the original sound. Determine the value of Y ₁ (z). Then, the relative relationship between the sound pressures of the microphones MIC _{1 to} MIC ₃ is set as shown in the following equation (12).

(Condition 1) Y ₁ (z)> Y ₂ (z)> Y ₃ (z) (12)

That is, the sound pressure at the position of the microphone MIC ₁ farthest from the listener H is set to be the highest, and then the sound pressure is set to decrease in the order of the microphone MIC ₂ and the microphone MIC ₃ . By setting the relationship between the outputs Y ₁ (z) to Y ₃ (z) in this way, the listener H can feel the audible sound generated at the position of the microphone MIC ₁ most greatly, and from this position It is possible to localize the sound image as if the listening sound is generated.

When the position of the microphone MIC ₂ is set in the virtual sound source SS (condition 2), the value of the output Y ₂ (z) is determined so that the microphone MIC ₂ reproduces the signal wave that is the original sound. Then, the relationship between the output Y ₂ (z) and the other outputs Y ₁ (z), Y ₃ (z) is set as shown in Expression (13).
(Condition 2) Y ₁ (z) = Y ₂ (z)> Y ₃ (z) (13)

Here, Y ₁ (z) = Y ₂ (z) is set so that the output Y ₁ (z) at the position of the microphone MIC ₁ is the same as the output Y ₂ (z) at the position of the microphone MIC ₂ . However, since the microphone MIC ₁ is away from the listener H, the audible sound at the position is attenuated before being transmitted to the listener H, and the sound pressure is reduced. Therefore, by setting the outputs Y ₁ (z) to Y ₃ (z) of the microphones MIC ₁ to MIC ₃ as described above, the listener H feels the audible sound generated at the position of the microphone MIC ₂ most greatly. The sound image can be localized so that an audible sound is generated from this position.

Similarly, when the position of the microphone MIC ₃ is set to the virtual sound source SS (condition 3), the value of the output Y ₃ (z) is determined so that the microphone MIC ₃ reproduces the signal wave that is the original sound. At the same time, the relationship between this output Y ₃ (z) and the other outputs Y ₁ (z), Y ₂ (z) is set as shown in Expression (14).
(Condition 3) Y ₁ (z) = Y ₂ (z) = Y ₃ (z) (14)

In this case, similarly to the above, even if the outputs Y ₁ (z) and Y ₂ (z) of the microphones MIC ₁ and MIC ₂ are the same as the output Y ₃ (z) of the microphone MIC ₃ , the microphone MIC ₁ , MIC ₂ is away from the listener H, the audible sound at that position is attenuated before being transmitted to the listener H, and the sound pressure is reduced. Therefore, by setting the outputs Y ₁ (z) to Y ₃ (z) of the microphones MIC ₁ to MIC ₃ as described above, the listener H feels the audible sound generated at the position of the microphone MIC ₃ most greatly. The sound image can be localized so that an audible sound is generated from this position.

(Output measurement result)
The output results of the microphones MIC _{1 to} MIC ₃ obtained by setting the filter by the method as described above are shown in FIGS.
7 to 9 are graphs of sound pressure level-frequency characteristics showing outputs when a filter is used. FIG. 6 is a comparative example, and is a graph of sound pressure level-frequency characteristics showing an output when no filter is used. The input was white noise having a sound pressure level-frequency characteristic shown in FIG.

FIG. 7 shows an output result when a filter is set according to the above (Condition 1). In this case, the position of MIC ₁ is the virtual sound source SS. In this case, the output of MIC ₁ is high, the output of MIC ₂ is suppressed by about 1.8 dB on average compared to MIC _1, and the output of MIC ₃ is suppressed by about 4.6 dB on average.

FIG. 8 shows an output result when a filter is set according to the above (Condition 2). In this case, the position of MIC ₂ is the virtual sound source SS. In this case, the outputs of MIC ₁ and MIC ₂ are high, and the output of MIC ₃ is suppressed by about 4.0 dB on average compared to MIC ₁ and MIC ₂ .

FIG. 9 shows an output result when the filter is set according to the above (Condition 3). In this case, the position of the MIC ₃ is the virtual sound source SS. In this case, it can be seen that there is almost no difference in the sound pressure levels of MIC _{1 to} MIC ₃ .

Therefore, it can be seen that the sound pressures at the positions of the microphones MIC _{1 to} MIC ₃ can be appropriately controlled by using the filters H ₁ (z) to H ₄ (z).

(Sound image localization result)
FIG. 11A is a graph showing the evaluation results of the subjective sound source localization of the virtual sound source by the listener, and FIG. 11B is a table showing the evaluation results. This evaluation uses the filters H ₁ (z) to H ₄ (z) set as described above to set a virtual sound source SS in the indoor space as shown in FIG. Evaluates whether or not the virtual sound source SS can be accurately localized. The virtual sound source SS was set at a position away from the wall surface W by distances (presentation distances) L1 to L3, and the listener performed evaluation at a position away from the wall surface W by a distance L4. The position of the virtual sound source SS was 60 cm (= L1), 100 cm (= L2), and 140 cm (= L3) from the wall surface W, and the position of the listener H was 150 cm (= L4) from the wall surface W. Then, the distance from the position of the virtual sound source SS localized by each listener H to the wall surface W was measured, and the average answer distance was obtained.

  From the results shown in FIG. 11, the position of the virtual sound source SS set using the filter and the position of the virtual sound source localized by the listener H do not exactly match, but from the wall surface W to the virtual sound source SS. As the distance becomes longer, the distance from the wall surface W where the listener has localized to the virtual sound source also becomes longer. Therefore, it was confirmed that by using the filter, the virtual sound source SS can be set to such an extent that the distance from the wall surface W can be distinguished. Further, it was found that when the filter is not used, it is localized that the virtual sound source SS is at a position closest to the wall surface W.

<< Other Embodiments of Reflecting Surface >>
FIG. 12A is a schematic plan view showing a state in which a reflection surface (reflection member) according to another embodiment is provided on the wall surface of the room, and FIG. 12B is a perspective view of the reflection member. In the acoustic system 10 described above, an example in which the modulated wave is reflected with the indoor wall surface W as a reflection surface is shown. However, as shown in FIG. 12B, the reflection member 31 having a plurality of reflection surfaces is provided in the room R. You may provide with respect to the wall surface. The reflecting member 31 is formed in a regular quadrangular pyramid having a square bottom surface and a regular triangular side surface. Then, by using such a reflecting member 31, it is possible to set the reflection direction of the modulated wave as desired. For example, as shown in FIG. 12 (a), a plurality of ultrasonic speakers SP directed in different directions are installed together in one place, and self-demodulated waves (signal waves) radiated from the speakers SP are transmitted to the listener H. It is possible to concentrate and transmit to.

  FIG. 13: is a schematic plan view which shows the usage type of an acoustic system at the time of providing the reflective surface (reflective member) which concerns on other embodiment on the wall surface of a room. The reflecting member 31 is formed in a hemispherical shape. By using such a reflecting member 31, it becomes possible to diffuse the modulated wave after reflection to some extent (expand the directivity range). Therefore, the audible area of the self-demodulated wave can be expanded and the movement of the listener H can be dealt with.

<< Other Embodiments of Ultrasonic Speaker >>
FIG. 14 is a perspective view showing an ultrasonic speaker according to another embodiment. This ultrasonic speaker SP is provided with a plurality of oscillators 26 in a casing 41 made of a polyhedron. In the illustrated example, the oscillator 26 is provided for a plurality of surfaces included in the regular icosahedron. Each oscillator 26 is attached to the housing 41 so that the angle can be adjusted so that the radiation direction of the modulated wave can be adjusted.

By using such an ultrasonic speaker SP, a modulated wave can be radiated in any direction from one place in the room R, and a virtual sound source can be set in any place in the room R.
Further, as shown in FIG. 15, by using the ultrasonic speaker SP and the reflecting member 31 in combination, it is possible to concentrate and transmit the modulated wave at a specific location in the room R. Further, in the acoustic system 10 shown in FIG. 3, the plurality of ultrasonic speakers SP _{1 to} SP ₄ can be configured by the ultrasonic speakers SP shown in FIG. 14.

  Note that the casing 41 constituting the ultrasonic speaker SP is not limited to a regular icosahedron, but may be a polyhedron different from the icosahedron. May be provided.

The present invention is not limited to the above embodiment, and can be appropriately changed within the scope described in the claims.
For example, the acoustic system of the present invention is not limited to use with a mixed reality system, but can also be used as an acoustic system installed in an exhibition hall, a concert hall, or the like.

10 sound system 21 signal source 22 filter processing section H the listener MIC ₁ ~MIC ₃ microphone (control point)
SP ₁ ~SP ₄ ultrasonic speaker SS virtual sound source

Claims (2)

A signal source for generating an audible signal wave;
A plurality of ultrasonic speakers for generating a carrier wave in an ultrasonic band and emitting a modulated wave obtained by modulating the carrier wave with the signal wave;
In order to set a virtual sound source of the signal wave at a predetermined position of a transmission path of the modulated wave in the air between a reflecting surface that reflects the modulated wave radiated from the ultrasonic speaker and the listener, A control unit for controlling the sound pressure;
With
The control unit sets, as a virtual sound source, one of control points set at a plurality of positions at different distances from the listener in the transmission path that linearly connects the reflection position on the reflecting surface and the listener. Therefore, based on the sound pressure of the control point serving as the virtual sound source, the control point is set to the same sound pressure as the virtual sound source at a control point farther away from the listener, and at a control point closer to the listener. An acoustic system equipped with a filter that sets a sound pressure lower than that of a virtual sound source . A modulated wave obtained by modulating a carrier wave in the ultrasonic band with a signal wave in the audible band is radiated by an ultrasonic speaker, and the radiated modulated wave is reflected by a reflecting surface, and the reflection position on the reflecting surface and the listener are linear. A plurality of control points having different distances from the listener in the transmission path of the modulated wave to be connected to the control point, and at a control point farther away from the listener, the sound pressure at any of the control points is used as a reference. set to one of the control points the same sound pressure, by setting a smaller sound pressure than the one of the control points more in control point distance is closer from the listener, the said one of the control points A method of setting a virtual sound source in an acoustic system, wherein the setting is a virtual sound source of a signal wave.