JP7487060B2 - Audio device and audio control method - Google Patents

Description

The present invention relates to an audio device and an audio control method.

Conventionally, for example, there is a technology that generates pseudo-reverberant sound signals from a stereo sound source and outputs the reverberant sound together with the sound source to reproduce surround sound on multiple channels (see, for example, Patent Document 1).

Japanese Patent No. 5372142

However, conventional technology leaves room for improvement when it comes to providing higher quality surround sound.

The present invention has been made in consideration of the above, and aims to provide an audio device and an audio control method that can perform high-quality surround sound reproduction.

In order to solve the above-mentioned problems and achieve the object, the acoustic device of the present invention detects , based on a sound source signal, feature information that is characteristic of the acoustic components contained in the sound source signal, determines, based on the feature information, a gain of a filter for generating artificial reverberation sound from the sound source signal, and sets the determined gain as the gain of the filter .

The present invention allows for high quality surround sound reproduction.

FIG. 1A is a diagram showing an overview of an acoustic control method according to an embodiment. FIG. 1B is a diagram showing an overview of the acoustic control method according to the embodiment. FIG. 2 is a block diagram illustrating an example of the configuration of the audio device according to the embodiment. FIG. 3 is a diagram illustrating a determination process performed by the determination unit. FIG. 4 is a flowchart showing the processing procedure of the processing executed by the audio device according to the embodiment.

Embodiments of the audio device and audio control method disclosed in this application will be described in detail below with reference to the attached drawings. Note that the present invention is not limited to the embodiments described below.

First, an overview of the acoustic control method according to the embodiment will be described with reference to Figs. 1A and 1B. Figs. 1A and 1B are diagrams showing an overview of the acoustic control method according to the embodiment. The acoustic control method according to the embodiment is executed, for example, by the acoustic device 1 shown in Fig. 1A.

Figure 1A shows a case where surround sound is reproduced in a vehicle interior by outputting direct sound, which is a sound source signal, and actual reverberant sound from two speakers FR, FL located on the left and right sides of the front, and outputting pseudo reverberant sound (hereinafter, pseudo reverberant sound) from two speakers RL, RR located on the left and right sides of the rear.

The sound source signal here is a signal from a sound source that has a spatial spread (sound image width) by outputting different sounds from each of the two speakers FR and FL. In other words, the sound source signal is a stereo signal that is reproduced in stereo on two channels (speakers FR and FL).

The sound source signal may be a signal of a mixture of sounds from multiple musical instruments and voices (vocals), such as classical music or opera, i.e., a signal of a mixture of sounds from multiple sound sources, or a signal of the sound of a single instrument (sound source), such as only a piano or only a violin.

In the acoustic control method according to the embodiment, the gain of the filter that generates the artificial reverberation sound is changed for each sound source (musical instrument or voice), so that a more natural artificial reverberation sound can be output from the sound source signal of a single piece of music. Therefore, a signal in which the sound source changes over time is preferable. The filter is, for example, an impulse response filter such as an FIR (Finite Impulse Response) filter or an IIR (Infinite Impulse Response) filter.

Specifically, in the acoustic control method according to the embodiment, the acoustic device 1 generates pseudo-reverberation sound using a filter whose gain is optimized for each sound source based on the characteristics of each sound source contained in the sound source signal, thereby achieving high-quality surround sound reproduction.

As shown in FIG. 1A, it is known that a listener L who hears sound from a sound source S in a space can perceive two types of spatial impressions. One spatial impression is the sound image width ASW, which is defined as the "width of the apparent sound source" that is perceived as blending with the direct sound in both time and space, and the other spatial impression is the sense of envelopment LEV, which is defined as the "feeling that the listener is surrounded by sound sources other than the apparent sound source." In other words, the sound image width ASW is a sound image derived from the direct sound and early reflected sound components, which are the early components of the sound source signal, and the sense of envelopment LEV is a sound image derived from the reverberation sound components, which are the later components of the sound source signal.

When designing and evaluating the sound image width ASW and the sense of envelopment LEV, an index using the so-called "law of the first wave front" may be used. In this index, as shown in Figure 1B, two regions R1 and R2 (threshold ranges) are defined, which are separated by two thresholds TH1 and TH2.

Region R1 is a region that contains, among the components contained in the sound source signal, components (initial components) that mainly contain direct sound. For example, if the initial components of region R1 are large, the sound image width ASW becomes large, and the listener L may perceive the sound as being diffused, and depending on the sound source, the sound quality may be evaluated as poor (unclear). Note that direct sound is, for example, sound recorded directly from voice (vocals) or musical instruments, and does not include sound reflected by walls, etc.

Region R2 is a region that contains, among the components contained in the sound source signal, components that mainly contain reverberation sounds (late components). For example, if the reverberation sound components in region R2 are large, the sense of envelopment LEV increases, and the listener L will perceive the sound as being diffused, which will result in a rich sense of envelopment. Note that reverberation sounds are, for example, recorded sounds of voices, musical instruments, etc. that are reflected by walls, etc., and are sounds that are delayed in time from the direct sound.

In other words, in the first wave front law, if the early components contained in region R1 and the later components contained in region R2 can be clearly separated, it is possible to achieve both a clear sound image and a rich sense of envelopment.

Here, in the past, signal processing was sometimes used to add reverberation by extracting uncorrelated components from the left and right channels of the sound source. In such cases, only the vocals, which are correlated components of the left and right channels, could be heard clearly, while other instruments, etc., could be heard unnaturally. As such, in the past, there was room for improvement in terms of achieving high-quality surround playback.

In the acoustic control method according to the embodiment, the threshold range in the first wave front law is changed according to the characteristics of the sound source, and the gain of the filter is determined so that the pseudo-reverberation sound generated by the filter falls within region R2 of the threshold range.

Specifically, in the acoustic control method according to the embodiment, first, feature information that is characteristic of the acoustic components (direct sound components and reverberation sound components) contained in the sound source signal is detected based on the sound source signal. Details of the feature information will be described later.

Next, in the acoustic control method according to the embodiment, the gain of a filter for generating artificial reverberation sound is determined based on the feature information. Then, in the acoustic control method according to the embodiment, the artificial reverberation sound is generated using a filter to which the determined gain is set.

As a result, even if the characteristics of the reverberation sound of the sound source are different, such as for voice, percussion instruments, classical music, etc., it is possible to generate an optimal pseudo-reverberation sound for each sound source, and optimal surround reproduction according to the characteristics of the sound source is possible. In other words, the acoustic control method according to the embodiment makes it possible to perform high-quality surround reproduction.

Next, a configuration example of the acoustic device 1 according to the embodiment will be described with reference to FIG. 2. FIG. 2 is a block diagram showing a configuration example of the acoustic device 1 according to the embodiment. The acoustic device 1 shown in FIG. 2 is connected to a sound source device 100 and a plurality of speakers FL, FR, RL, and RR. As shown in FIG. 2, the acoustic device 1 also includes a control unit 2 and a storage unit 3.

The sound source device 100 outputs a sound source signal to the audio device 1. The sound source signal is, for example, a stereo signal, and a sound source S with spatial expansion is generated by outputting different signals from two speakers FL and FR, which are two channels.

The multiple speakers FL, FR, RL, and RR output the signal output by the audio device 1 as sound. Specifically, the speakers FL and FR output direct sound, which is a sound source signal, and the speakers RL and RR output pseudo-reverberation sound generated from the sound source signal.

Here, the audio device 1 includes, for example, a computer having a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), flash memory, input/output ports, and various other circuits.

The computer's CPU functions as the acquisition unit 21, detection unit 22, determination unit 23, generation unit 24 and output unit 25 of the control unit 2, for example, by reading and executing a program stored in the ROM.

In addition, at least one or all of the acquisition unit 21, detection unit 22, decision unit 23, generation unit 24 and output unit 25 of the control unit 2 can be configured with hardware such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

The storage unit 3 corresponds to a RAM or a flash memory. The RAM or the flash memory can store information about various programs. The audio device 1 may also acquire the above-mentioned programs and various information via other computers or portable recording media connected to a wired or wireless network.

Next, each function of the control unit 2 (acquisition unit 21, detection unit 22, determination unit 23, generation unit 24, and output unit 25) will be described in detail.

The acquisition unit 21 acquires various information and signals. For example, the acquisition unit 21 acquires a sound source signal from the sound source device 100. For example, the acquisition unit 21 acquires a sound source signal that is a stereo signal. Specifically, the acquisition unit 21 acquires sound source signals output from two speakers FL and FR, which are two channels.

The acquisition unit 21 also acquires sound source information related to the sound source signal. The sound source information acquired includes, for example, the type (genre) of the sound source signal and information related to the recording environment. The type of sound source signal includes, for example, voice, musical instruments (percussion instruments, wind instruments, etc.), classical music, etc. The recording environment includes, for example, information on the location where the sound was recorded, such as a recording studio or concert hall.

The acquisition unit 21 acquires the sound source information, for example, through an input by a user such as a listener L, or acquires the sound source information from a server or the like via the Internet. The acquisition unit 21 may also acquire the sound source information by analyzing the acquired sound source signal.

The detection unit 22 detects feature information, which is a feature of the acoustic components contained in the sound source signal, based on the sound source signal.

For example, the detection unit 22 detects channel relationship information related to the relationship between the sound source signals reproduced on each of the two channels as feature information related to the features of the acoustic components. Specifically, the channel relationship information includes information on the LR channel difference and the LR channel correlation.

The LR channel difference is information about the difference between two sound source signals reproduced in stereo on two channels. Specifically, the LR channel difference is the inter-channel level difference (ICLD) or the inter-channel time difference (ICLD).

The LR channel correlation is information about the correlation components of two sound source signals reproduced in stereo on two channels. Specifically, the LR channel correlation is the inter-channel cross correlation (ICC) between the channels.

In this way, the detection unit 22 can detect acoustic components with high accuracy by detecting channel relationship information as feature information, and therefore can generate optimal pseudo-reverberation sounds through filtering in the downstream generation unit 24.

The detection unit 22 then continuously detects the channel relationship information at regular intervals, arranges the information in a time series, and calculates a moving average of the detected time series channel relationship information, thereby detecting the moving average as feature information. This makes it possible to smooth out any sudden changes in the channel relationship information, and therefore to smooth out any sudden changes in the gain determined by the determination unit 23 in the subsequent stage. As a result, it is possible to smooth out the changes in the pseudo reverberation sound generated by the generation unit 24, thereby achieving more natural surround playback.

The detection unit 22 maximizes the channel relationship information for sections of the time-series channel relationship information where the sound pressure level (amplitude) of the sound source signal is less than a predetermined value, and then calculates a moving average. In other words, the detection unit 22 calculates a moving average so as to control the time-series channel relationship information so that surround sound is not output for sections where the sound pressure level of the sound source signal is less than a predetermined value, and surround sound is not output around the sections where the sound pressure level is less than the predetermined value.

This lowers the upper limit at which the law of the first wave front applies when there is silence between beats, such as in a solo percussion performance, so the filter gain for the percussion instruments is lowered to control the surround level to an appropriate level.

The detection unit 22 also detects information related to the duration and frequency of the acoustic components as feature information.

The duration is the time that the sound source signal continues. Specifically, the detection unit 22 calculates the envelope of the sound source signal, differentiates the calculated envelope, and then calculates the envelope again, and detects the duration from the slope of the resulting envelope.

The frequency is information about the frequency components of the sound source signal. Specifically, the detection unit 22 detects the center of gravity of the frequency of the sound source signal expressed in frequency by performing a Fourier transform on the sound source signal expressed in time.

In this way, the detection unit 22 can detect acoustic components with high accuracy by detecting duration and frequency information as feature information, and therefore can generate optimal pseudo-reverberation sounds through filtering in the downstream generation unit 24.

The determination unit 23 determines the gain of a filter used by the generation unit 24, which will be described later. Specifically, the determination unit 23 determines the gain of the filter based on at least one of the feature information detected by the detection unit 22 and the sound source information acquired by the acquisition unit 21.

Specifically, the determination unit 23 corrects the threshold range determined from the first wave front law based on at least one of the feature information and the sound source information, and determines the gain based on the corrected threshold range. Here, the process of determining the gain based on the first wave front law will be described with reference to FIG. 3.

Figure 3 is a diagram showing the gain determination process performed by the determination unit 23. As shown in Figure 3, the determination unit 23 first corrects the threshold range obtained from the first wave front law based on the feature information (channel relationship information, duration, and frequency) and sound source information.

For example, the threshold ranges shown in the graph in the middle of FIG. 3, regions R1 and R2, are set as the reference ranges. In this case, the determination unit 23 corrects regions R1 and R2 by correcting thresholds TH1 and TH2 on the two axes of time and amplitude based on the feature information and sound source information.

For example, the larger the channel relationship information, the more the determination unit 23 corrects the thresholds TH1 and TH2 so that the time and amplitude become larger. In other words, the larger the channel relationship information, the more the determination unit 23 corrects the regions R1 and R2 so that they become larger.

On the other hand, the smaller the channel relationship information, the more the determination unit 23 corrects the thresholds TH1 and TH2 in the direction of decreasing the time and amplitude. In other words, the smaller the channel relationship information, the more the determination unit 23 corrects the regions R1 and R2 to become smaller.

The channel relationship information becomes larger the greater the inter-channel level difference, or the greater the inter-channel time difference, or the lower the cross-correlation (the more uncorrelated components there are).

In addition, the determination unit 23 corrects the thresholds TH1 and TH2 in the direction of increasing the time and amplitude as the duration of the sound component becomes longer. In other words, the determination unit 23 corrects the regions R1 and R2 so that they become larger as the duration of the sound component becomes longer.

On the other hand, the determination unit 23 corrects the thresholds TH1 and TH2 in a direction that reduces the time and amplitude as the duration of the sound component becomes shorter. In other words, the determination unit 23 corrects the regions R1 and R2 so that they become smaller as the duration of the sound component becomes shorter.

The determination unit 23 also corrects the thresholds TH1 and TH2 in a direction in which the time and amplitude increase as the frequency (center of gravity) of the sound component decreases. In other words, the determination unit 23 corrects the regions R1 and R2 to become larger as the frequency (center of gravity) of the sound component decreases.

On the other hand, the determination unit 23 corrects the thresholds TH1 and TH2 in a direction that reduces the time and amplitude as the frequency (center of gravity) of the sound component increases. In other words, the determination unit 23 corrects the regions R1 and R2 to be smaller as the frequency (center of gravity) of the sound component increases.

Furthermore, when the sound source signal is determined to be classical based on the sound source information, the determination unit 23 corrects the thresholds TH1 and TH2 in the direction of increasing the time and amplitude. In other words, when the sound source signal is classical, the determination unit 23 corrects the regions R1 and R2 to become larger.

On the other hand, when the sound source information indicates that the sound source signal is voice, the determination unit 23 corrects the thresholds TH1 and TH2 in a direction that reduces the time and amplitude. In other words, when the sound source signal is voice, the determination unit 23 corrects the regions R1 and R2 to be smaller.

Then, the determination unit 23 determines the gain of the filter to be used in the subsequent generation unit 24 based on the corrected threshold range. Specifically, the determination unit 23 determines the gain so that the pseudo reverberation sound generated by the filter falls within the corrected threshold range.

In other words, when regions R1 and R2 are corrected to be larger, the determination unit 23 determines the gain so that the time and amplitude of the artificial reverberation sound are larger. On the other hand, when regions R1 and R2 are corrected to be smaller, the determination unit 23 determines the gain so that the time and amplitude of the artificial reverberation sound are smaller.

In this way, the determination unit 23 can determine the gain for generating the optimal pseudo-reverberation sound by determining the gain from the threshold range of the first wave front law corrected based on the feature information and sound source information.

The generator 24 generates a reverberation signal representing a pseudo-reverberation sound from the sound source signal using a filter having a gain determined by the determiner 23. The filter may be, for example, an impulse response filter such as an FIR (Finite Impulse Response) filter or an IIR (Infinite Impulse Response) filter.

The output unit 25 performs predetermined processing on the sound source signal input from the acquisition unit 21 and the reverberation signal input from the generation unit 24, and outputs the results from the speakers FL, FR, RL, and RR.

Specifically, the output unit 25 performs D/A conversion on the sound source signal, amplifies the D/A converted sound source signal, and outputs it from the speakers FL and FR. The output unit 25 also performs D/A conversion on the reverberation signal, amplifies the D/A converted reverberation signal, and outputs it from the speakers RL and RR.

Next, the procedure of the process executed by the audio device 1 according to the embodiment will be described with reference to FIG. 4. FIG. 4 is a flowchart showing the procedure of the process executed by the audio device 1 according to the embodiment.

As shown in FIG. 4, first, the acquisition unit 21 acquires a sound source signal from the sound source device 100 (step S101). Next, the acquisition unit 21 acquires sound source information related to the sound source signal (step S102).

Next, the detection unit 22 detects feature information that is a feature of the acoustic components contained in the sound source signal based on the sound source signal and sound source information acquired by the acquisition unit 21 (step S103).

Next, the determination unit 23 determines the threshold range in the first wave front law based on the characteristic information detected by the detection unit 22 (step S104).

Next, the determination unit 23 determines the gain of the filter (generation unit 24) based on the determined threshold range so that the pseudo reverberation sound falls within the threshold range (step S105).

Next, the generation unit 24 generates a reverberation signal that indicates a simulated reverberation sound using a filter to which the determined gain is set (step S106).

Then, the output unit 25 performs surround playback by outputting the reverberation signal generated by the generation unit 24 and the sound source signal acquired by the acquisition unit 21 through the multiple speakers FL, FR, RL, and RR (step S107), and ends the process.

As described above, the acoustic device 1 according to the embodiment includes a detection unit 22, a determination unit 23, and a generation unit 24. The detection unit 22 detects feature information that is a feature of the acoustic components contained in the sound source signal based on the sound source signal. The determination unit 23 determines the gain of a filter for generating pseudo reverberation sound from the sound source signal based on the feature information. The generation unit 24 generates a reverberation signal indicating reverberation sound from the sound source signal using a filter to which the gain determined by the determination unit 23 is set. This enables high-quality surround reproduction.

Further advantages and modifications may readily occur to those skilled in the art. Therefore, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and equivalents thereof.

Reference Signs List 1 Acoustic device 2 Control unit 3 Storage unit 21 Acquisition unit 22 Detection unit 23 Determination unit 24 Generation unit 25 Output unit 100 Sound source device FL, FR, RL, RR Speaker L Listener S Sound source

Claims (7)

Detecting feature information that is a feature of an acoustic component included in a sound source signal based on the sound source signal;
correcting a threshold range obtained from the first wavefront law based on the characteristic information;
determining a gain of a filter for generating a pseudo reverberation sound from the sound source signal based on the corrected threshold range ;
The acoustic device sets the determined gain as a gain of the filter. The sound source signal is
A stereo signal consisting of two channel playback signals,
The audio device according to claim 1 , wherein channel relationship information relating to a relationship between the sound source signals reproduced on each of the two channels is detected as the feature information. The audio device according to claim 2 , wherein the channel relationship information is continuously detected at regular intervals, and a moving average calculated for the detected time-series channel relationship information is detected as the feature information. The audio device according to claim 3 , wherein the moving average is performed by maximizing the channel relation information in a section in which the sound pressure level of the sound source signal is less than a predetermined value, among the time-series channel relation information. 5. The audio device according to claim 1, wherein information regarding a duration and a frequency of the sound component is detected as the feature information. obtaining sound source information relating to the sound source signal;
The acoustic device according to claim 1 , further comprising: a gain of the filter determined based on the characteristic information and the sound source information. a detection step of detecting feature information that is a feature of an acoustic component included in a sound source signal based on the sound source signal;
a correction step of correcting a threshold range obtained from the first wavefront law based on the characteristic information;
determining a gain of a filter for generating a pseudo reverberation sound from the sound source signal based on the corrected threshold range ;
a setting step of setting the gain determined by the determining step as the gain of the filter;
and generating a reverberation signal indicating the reverberation sound from the sound source signal using the filter whose gain has been set in the setting step.

Images