JP2018133660A - Audio-signal compensation device, audio-signal compensation method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide audio-signal compensation device, audio-signal compensation method and program, which allow for natural hearing even when hearing reproduced sound signals in a state added at a hearing point, when reproducing a sound signal composed of multiple channels with multiple speakers.SOLUTION: Mutual correlation coefficients of multi-channel sound signal are calculated, a high correlation channel composite signal is generated by adding channel sound signals of high correlation coefficients between channel sound signals, levels of the high correlation channel composite signal and other channel sound signals, and noise are calculated, compensation amount of each channel sound signal is calculated based on respective levels, and then the multi-channel sound signal is compensated based on the calculated compensation amount. The correlation coefficients and the compensation amount can be calculated for each frequency range.SELECTED DRAWING: Figure 6

Description

  The present invention relates to an audio signal compensation device, an audio signal compensation method, and a program, and more particularly, when a multi-channel audio signal (acoustic signal) is reproduced by a plurality of speakers in a noisy environment, it is naturally natural to hear. The present invention relates to an audio signal compensation device, an audio signal compensation method, and a program that enable listening.

  When playing program sounds in public viewing or at exhibition halls, the sound signals that are played may be difficult to hear or the sound quality may deteriorate due to background noise or noise generated by people coming and going. . Therefore, conventionally, automatic volume control has been proposed that automatically adjusts the signal level according to the level of ambient noise and maintains an easy-to-hear situation (for example, Patent Documents 1 to 3). The inventions described in these patent documents perform adaptive sound quality volume control that compensates masking for a reproduced audio signal in a noisy environment.

  A MAEQ (Masking Adaptation EQualizer) method has attracted attention as a method for compensating an audio signal for ambient noise while maintaining sound quality (Non-patent Document 1). The MAEQ method compensates for the masking amount due to noise for each frequency range, and is an algorithm for determining the compensation amount so as to coincide with the loudness when there is no noise. FIG. 13 shows an outline of the MAEQ method. The diagram on the left side of FIG. 13 shows an example of the relationship between the audio signal and noise (the horizontal axis is frequency, the vertical axis is signal level), the bar graph is the level for each audio signal band, and the horizontal line is for each noise band. Level. Then, a level difference between the voice signal and the noise is obtained for each band, and a compensation amount for each band is obtained based on the level difference. For example, the compensation amount is obtained by the following equation (1).

M = 0.0073 (S-N) 2 - 0.53 (S-N) + 7.31 ----- (1)
M: Compensation amount to sound signal for each band [dB]
S: Band level of audio signal [dB]
N: Noise band level [dB]

  The diagram on the right side of FIG. 13 shows a state in which the audio signal compensation amount (upward arrow) for each band is added to the audio signal (bar graph), and the broken horizontal line indicates the frequency of each audio signal after compensation. Indicates the level. With this compensation, the sound signal level exceeds the noise level in each band, and the reproduced sound can be heard in the same manner as when there is no noise regardless of the noise characteristics.

JP-A-9-116361 JP-A-9-116362 JP 2002-232247 A

Murano, T., et al., “Sound quality control method considering masking due to ambient noise”, Acoustic Meeting of the Acoustical Society of Japan, March 1997, p. 523-524

  In the above-described prior art, since it is assumed that there is one audio signal to be compensated, when reproducing the audio signal with multiple channels (multiple speakers), the audio signal of each channel is processed. If simple compensation such as compensation for noise masking or masking compensation for a single audio signal with all channels added together, the audio signal reproduced at the listening point is added. Depending on the audio signal, the compensation may be excessive or insufficient.

  FIG. 14 shows a conceptual diagram of the prior art in which compensation is performed for a plurality of audio signals. Conventionally, for example, the audio signals a and b of the two channels are compensated independently, the compensation amount α is added to the audio signal a so as to exceed the noise level, the signal level is a + α, and the audio signal b is compensated. The signal level is set to b + β by adding β. The audio signal a + α is reproduced from the A channel speaker Ach, and the audio signal b + β is reproduced from the B channel speaker Bch. At this time, when the A channel sound and the B channel sound are independent from each other, there is no problem with such a compensation method. However, when both channels are the same sound, the sound of both channels is heard at the listening point. It is added and heard as one voice.

  Therefore, if the same component is contained in multiple channels at a low level, if compensation for noise masking is performed for each channel, the amount of compensation will be larger than necessary when added at the listening point. Or the compensation is concentrated on a specific band and the sound quality deteriorates. On the other hand, even if different acoustic events are included in different channels, the amount of masking for each acoustic event is not considered when the compensation amount by masking is determined for one signal that is added together. There is a problem of lack of compensation.

  Accordingly, an object of the present invention made in view of the above-described problems is that when a multi-channel audio signal is reproduced by a multi-speaker, the reproduced audio signal is added at the listening point. An object of the present invention is to provide an audio signal compensation device, an audio signal compensation method, and a program that can be heard naturally even after listening.

  In order to solve the above problems, an audio signal compensator according to the present invention includes an inter-channel signal correlation calculation unit that calculates a correlation coefficient between multi-channel audio signals, and the correlation coefficient between channel audio signals is a predetermined value. A high correlation channel synthesis unit that generates a highly correlated channel synthesized signal by adding together the frequency components of all channel audio signals that are higher than the threshold, and was not used for the generation of the highly correlated channel synthesized signal and the highly correlated channel synthesized signal. An audio signal level calculation unit that calculates the level of each channel audio signal, an additional audio signal level calculation unit that calculates the level of an additional audio signal other than the multi-channel audio signal, the high correlation channel synthesis signal, and the high correlation channel synthesis The level of each channel signal that was not used for signal generation and the level of the additional audio signal. A multi-channel audio signal compensation amount calculating unit that calculates a compensation amount of each channel audio signal, and a multi-channel audio signal compensation unit that compensates the multi-channel audio signal based on the compensation amount. And

  The audio signal compensator calculates a correlation coefficient between the multi-channel audio signals, generates the highly correlated channel synthesized signal, calculates a level of the audio signal, and calculates a compensation amount of the multi-channel audio signal. Is preferably performed for each frequency range defined in advance to compensate for the frequency characteristics of the multi-channel audio signal.

  In the audio signal compensator, it is preferable that the correlation calculation unit between channel signals calculates a correlation coefficient for a combination of two different channels for all input channel audio signals.

  In the audio signal compensator, the inter-channel signal correlation calculation unit selects a channel that is higher than a predetermined level or a channel that is within a predetermined range from a maximum signal level among the input channel audio signals. It is desirable to calculate a correlation coefficient for a combination of two different channels for all the selected channels.

  In the audio signal compensator, the inter-channel signal correlation calculation unit selects a channel that is higher than a predetermined level or a channel that is within a predetermined range from a maximum signal level among the input channel audio signals. Determining a preset channel set that is assumed to have a high predefined correlation for each of the selected channels, and then combining the channels in the preset channel set corresponding to the selected channel It is desirable to calculate a correlation coefficient for.

  In the audio signal compensation device, the correlation calculation unit between channel signals skips correlation coefficient calculation and adds the correlation coefficient calculation unit to the high correlation channel synthesis unit when a combination of channels having a high correlation is specified in advance. It is desirable to output channel combination information.

  In the audio signal compensator, the inter-channel signal correlation calculating unit obtains a cross-correlation function between the channel audio signals, and sets the largest function value within a predetermined time difference of the cross-correlation function as the channel audio signal. It is desirable to use it as a correlation coefficient.

  Further, the audio signal compensator is configured to generate a multi-channel audio signal based on a level of each channel audio signal not used for generating the high correlation channel composite signal and the high correlation channel composite signal and a level of the additional audio signal. It is desirable to perform compensation selectively in the frequency range.

  In addition, the audio signal compensator is configured such that a level difference between each channel signal that is not used for generating the high correlation channel combined signal and the high correlation channel combined signal and the additional audio signal for each predetermined frequency range is set in advance. Compensate only when it is within the specified range. Compensate in order from the highest priority band or channel according to the upper limit of the overall compensation amount given in advance, and when the compensation amount exceeds the upper limit. Omit compensation processing for uncompensated bands or channels, or select a channel audio signal that is higher than a pre-defined level, and calculate the average level for each pre-defined frequency range of the entire selected channel. Determining the compensation range using the level difference between the calculated average level and the level of each additional frequency range of the additional signal. It is desirable to select the range to be compensated by either.

  Also, the audio signal compensation method according to the present invention calculates the mutual correlation coefficient of multi-channel audio signals, and the frequency components of all channel audio signals in which the correlation coefficient between the channel audio signals is higher than a predetermined threshold. A high correlation channel composite signal is generated by adding together, and each channel audio signal not used for generating the high correlation channel composite signal and the high correlation channel composite signal, and additional audio signals other than the multi-channel audio signal A level is calculated, and a compensation amount of each channel audio signal is calculated from the level of each channel signal not used for the generation of the highly correlated channel synthesized signal and the highly correlated channel synthesized signal and the level of the additional audio signal, The multi-channel audio signal is compensated based on the compensation amount.

  A program according to the present invention causes a computer to function as the audio signal compensation device.

  According to the audio signal compensation device, the audio signal compensation method, and the program of the present invention, the same audio signal component included in a plurality of channels can be regarded as one signal, while different audio signals for each channel are separated. By determining the compensation amount, it is possible to prevent the compensation amount from being excessive or insufficient. Therefore, even when the same component is included in a plurality of channels among the channels of the multi-channel audio signal, it can be heard naturally in terms of hearing.

It is a figure which shows notionally the method of compensating the audio | voice signal with the high correlation between channels of this invention. It is a figure which shows notionally the method (first half) which performs the compensation for every frequency component of this invention. It is a figure which shows notionally the method (latter half) which performs the compensation for every frequency component of this invention. It is a figure which shows the example of the environment which performs multichannel sound reproduction. It is a block diagram of the audio | voice signal compensation apparatus of this invention. It is an example of the block diagram of the signal compensation amount determination part of Example 1 of the audio | voice signal compensation apparatus of this invention. 6 is a flowchart of processing of a signal compensation amount determination unit according to the first embodiment. It is an example of the block diagram of the signal compensation amount determination part of Example 2 of the audio | voice signal compensation apparatus of this invention. 10 is a flowchart of processing of a signal compensation amount determination unit according to the second embodiment. It is a figure which shows the channel arrangement | positioning of 22.2 multichannel sound. It is an example of the block diagram of the signal compensation amount determination part of Example 3 of the audio | voice signal compensation apparatus of this invention. This is an example in which compensation is selectively performed based on a level difference between a sound signal and noise for each frequency range. It is a figure which shows the outline | summary of MAEQ method. It is a conceptual diagram which shows the prior art which compensates with respect to several audio | voice signal.

  First, the principle of the multi-channel audio signal compensation method of the present invention will be briefly described with reference to FIGS.

  FIG. 1 is a diagram conceptually showing a method for compensating an audio signal having a high correlation between channels. As shown in FIG. 1 (1), an audio signal having a high correlation between channels (for example, the same audio signal) is reproduced by a speaker (Ach, Bch) of each channel, and then synthesized at a listening point. It will be heard as a voice.

  Therefore, such an audio signal having a high correlation between channels is not separately compensated, but as shown in FIG. 1 (2), the audio signals a and b of the respective channels are added and added together ( A compensation amount γ by masking is determined for a + b). At the listening point, if there is a signal level of (a + b) + γ, the reproduced sound is sufficiently unaffected by noise. Then, the compensation amount γ is assigned to two channels and reproduced from the speakers Ach and Bch, respectively. As a result, the reproduction sound of Ach and Bch becomes an appropriate compensation amount when added in space. Note that audio signals with low correlation between channels, that is, audio signals that are independent of each other, may be compensated separately as in the past.

  Therefore, the mutual correlation coefficient of the multi-channel audio signal is calculated, and the channel audio signals having a high correlation coefficient between the channel audio signals are added to generate a highly correlated channel synthesized signal. The level of each channel audio signal and the noise level are calculated, the compensation amount of each channel audio signal is calculated based on each level, and the multi-channel audio signal is compensated based on the calculated compensation amount. it can.

  2 and 3 are diagrams conceptually showing a method of performing compensation for each frequency component, and show an example in which the MAEQ method is applied to a multi-channel audio signal. In each figure, the horizontal axis represents the frequency, and the vertical axis represents the signal level, showing the audio signal in each processing step.

  In FIG. 2, first, as the stage (a), the Ach audio signal and the Bch audio signal are each subjected to spectrum analysis and divided into bands.

  Next, as a step (b), the correlation between channels is calculated for each same band. In the figure, a band drawn with right-down hatching is a band with high signal correlation, and a band drawn with left-down hatching is a band with low signal correlation.

  In the stage (c), the Ach audio signal and the Bch audio signal are added for a highly correlated band. In the subsequent processing, the signal in the high correlation band is handled with respect to the added audio signal. On the other hand, the audio signals in the band with low correlation handle Ach and Bch as independent audio signals.

  Next, in FIG. 3, in the stage (d), the audio signal is converted to a constant level for each band, and the signal level in each band used in the subsequent processing is obtained. For the high correlation band, level conversion is performed on the audio signal in which Ach and Bch are added. In addition, the signal level of environmental noise is obtained separately from the audio signal. Regarding the environmental noise, after obtaining the spectrum, the same band division as that of the audio signal is performed, the noise is converted into a constant level for each band, and the noise level in each band used in the subsequent processing is obtained.

  In the stage (e), the level difference between voice and noise (signal-noise ratio, (S−N) dB) is calculated for each band. In the low correlation band, the voice level and the noise level are compared independently for each of Ach and Bch, and in the high correlation band, the voice level obtained by adding Ach and Bch is compared with the noise level.

  In step (f), a compensation amount based on the level difference between voice and noise is calculated for each band. As the compensation amount calculation formula, for example, Formula (1) that is a compensation amount calculation formula of the MAEQ method can be used. The obtained compensation amount is handled as a compensation amount for each channel in the low correlation band, and is treated as a compensation amount common to both channels (Ach, Bch) in the high correlation band. In subsequent audio reproduction, the obtained compensation amount for each band is reproduced by a speaker in addition to the audio signal of each channel.

  In this way, appropriate compensation / reproduction in multi-channel becomes possible.

  Hereinafter, specific embodiments of the present invention will be described.

(Embodiment)
An example of performing sound reproduction using the audio signal compensator of the present invention in an environment where noise is uniformly distributed over the entire listening area in a viewing space where a plurality of speakers are installed will be described with reference to the drawings. .

  FIG. 4 shows an example of an environment in which multi-channel sound reproduction is performed. Ten speakers are arranged so as to surround the viewing area, and each speaker is sp1, sp2,. ., Called sp10. In FIG. 4, five speakers are arranged on the front of the listener, three speakers are arranged on the back, and one speaker is arranged on each side. One microphone M is installed in the viewing area, and in this embodiment, the noise in the listening area recorded by the microphone M (sometimes referred to as “additional voice”) and reproduction by each speaker 10. The audio signal of the channel (ch) is input, the audio signal is adjusted by the audio signal compensation device 100, and is output to each speaker. It is assumed that 10 channels of audio signals are assigned in order from 1ch to sp1, sp2, sp3,. Here, the noise picked up by the microphone M may be a signal obtained by performing a process of extracting a noise component by a sound source separation process on the signal picked up at the time of reproducing the sound signal. A signal that has been collected in advance may be used. In this embodiment, noise is picked up by a single microphone. However, actually, a plurality of microphones may be installed, and a process of extracting a noise component using a signal picked up by each microphone may be performed.

  FIG. 5 shows a block diagram of the audio signal compensation apparatus 100 of the present invention. Note that the audio signal compensation apparatus 100 may constitute an audio compensation signal processing unit of the entire audio (sound) reproduction apparatus. The audio signal compensation apparatus 100 includes a signal compensation amount determination unit 110 and an audio signal compensation unit 130.

  The signal compensation amount determination unit 110 receives the multi-channel audio signal and noise from the microphone M (additional audio signal other than the multi-channel audio signal) as inputs, and outputs the determined compensation amount to the audio signal compensation unit 130. The signal compensation amount determination unit 110 can previously input or register the characteristics of the playback speaker, the transfer function up to the listening point in the playback environment, and the like. By using an audio signal that takes into account the frequency characteristics and transfer characteristics of a reproduction speaker for analysis, a compensation amount suitable for the reproduction environment can be determined. For example, when the frequency characteristics of the playback speaker are not flat, a signal similar to the playback signal when actually listening at the listening position can be obtained by adding the characteristics to the audio signal in advance. The compensation amount can be determined in accordance with. The audio signal in the present invention includes an audio signal that takes into account the frequency characteristics and transfer characteristics of such a reproduction speaker.

  The audio signal compensation unit 130 compensates the multi-channel audio signal based on the compensation amount obtained by the signal compensation amount determination unit 110. That is, the frequency characteristic of each channel audio signal is adjusted. The audio signal compensation unit 130 may be adjusted as an equalizer using an IIR (Infinite impulse response) filter, or may be realized by convolution using an FIR (Finite Impulse Response) filter. Good. The adjustment (compensation) processing of the audio signal can be realized using known digital processing or the like.

  FIG. 6 shows an example of a block diagram of the signal compensation amount determination unit 110 of the first embodiment in the audio signal compensation apparatus of the present invention. FIG. 7 shows a flowchart of processing in this block diagram. Here, an example in which the correlation between all the channels is examined and the compensation amount is determined for each frequency range (for example, for each frequency bin) will be described by taking as an example the case of performing the processing for obtaining the correlation in the frequency domain.

  The signal compensation amount determination unit 110 according to the first embodiment includes a channel spectrum correlation calculation unit 111 having a frequency spectrum calculation unit 112 and a frequency-specific channel correlation calculation unit 113, a highly correlated channel synthesis unit 114, and an audio signal level calculation unit. 115, an additional audio signal level calculation unit 116, and a compensation amount calculation unit 117.

  The correlation calculation part 111 between channel signals calculates the correlation according to frequency range between the combinations of all the channels with respect to the input acoustic signal. In the example of FIG. 4, since there are 10 channels, the correlation is examined for 45 combinations in each frequency range.

  First, a multi-channel (here, 10 channels) audio signal is input to the frequency spectrum calculation unit 112 (step 11 [ST11] in the flowchart of FIG. 7). Then, the frequency spectrum calculation unit 112 calculates a frequency spectrum for each audio signal (step 12 [ST12]) and outputs the frequency spectrum to the inter-frequency channel correlation calculation unit 113.

  For each frequency range, the inter-frequency correlation calculation unit 113 calculates correlations for two different channel combinations (here, 45) for the audio signals of all channels (step 13 [ST13]).

  The configuration of the correlation calculation unit 111 between channel signals in FIG. 6 is an example in the case of performing correlation calculation in the frequency domain, and a calculation method such as obtaining a frequency spectrum and then obtaining a correlation coefficient between channels by cross spectrum calculation or the like Is adopted. Note that the correlation calculation in the frequency domain is one method, and a configuration (not shown) for performing the correlation calculation in the time domain may be employed. When calculating the correlation in the time domain, a correlation value that does not allow a deviation on the time axis, such as obtaining a correlation coefficient by directly calculating two channel audio signals in the time domain, may be obtained. A cross-correlation function between channel audio signals allowing the above deviation (time difference τ) is obtained, and the maximum function value within a predetermined time difference of the cross-correlation function is used as a correlation coefficient between the channel audio signals. Also good. This calculation considering the time difference τ corresponds to a multi-channel audio reproduction method in which the same audio signal is output with the time shifted based on the speaker arrangement or the like. Even when calculating correlation in the time domain, there is a method such as calculating the correlation coefficient and cross-correlation function in the time domain after dividing each frequency range with a digital filter or the like. The method can be adopted.

  Then, it is determined whether the calculated correlation coefficient is higher or lower than a threshold value (step 14 [ST14]). When the correlation coefficient calculated for each frequency range is higher (or higher) than a predetermined threshold, for example, the correlation value normalized from 0 to 1 is 0.8 or higher (this threshold can be set as appropriate). Are output as a highly correlated channel set to the highly correlated channel synthesizing unit 114, and other channels and frequency ranges are directly output as low correlated channels to the audio signal level calculating unit 115. To do.

  The highly correlated channel synthesis unit 114 adds the frequency components of the channel set input as the highly correlated channel set (step 15 [ST15]) and outputs the result to the audio signal level calculation unit 115. For example, in a certain frequency range, when signals from 1ch to 10ch have energy of a, b, c,..., J, respectively, and it is determined that only the channel set of 1ch and 2ch has a correlation higher than the threshold, Only in the frequency range of 2ch, the energy of both is added to obtain a + b. That is, the frequency ranges of 1ch and 2ch are common components, and the compensation amount of 1ch and 2ch is determined using a value of a + b obtained by adding both. Similarly, when the correlation is high in three combinations of 1ch to 2ch, 1ch and 2ch, 2ch and 3ch, 1ch and 3ch, the compensation amount of 1, 2 and 3ch is determined using the energy of a + b + c. It shall be. In addition, when the correlation between two combinations of 1ch to 2ch, 2ch and 3ch is high among 1ch to 10ch, the compensation amount of 1, 2 and 3ch may be determined using the energy of a + b + c. The compensation amount may be determined using a + b + c for a + b and 2ch, and b + c for 3ch. In the above description, signals are added by energy addition, but may be added by amplitude values in a time waveform. Alternatively, energy addition and amplitude addition may be combined, and for example, the band for performing energy addition and the band for performing amplitude addition may be used separately according to the distance between the speakers of the channels to be added. The same applies to the following description.

  The audio signal level calculation unit 115 calculates the audio signal level for each frequency range (step 16 [ST16]). That is, by the above processing, of the input audio signals of the channels, channels having a highly correlated frequency range are added together only in the frequency range, and a level (power spectrum) is calculated. For other channels and frequency ranges, the level (power spectrum) is calculated for each channel. The calculated signal level is output to the compensation amount calculation unit 117.

  On the other hand, for the noise signal, the level for each frequency range is simply calculated (step 17 [ST17]). That is, the additional audio signal level calculation unit 116 receives the additional audio signal from the microphone M, calculates the power spectrum of the additional audio signal, and outputs the result to the compensation amount calculation unit 117. In addition, as an additional audio | voice signal, although noise and noise are generally assumed, it is not restricted to this, Arbitrary acoustic signals other than a multichannel audio | voice signal may be contained.

  Thereafter, the compensation amount calculation unit 117 determines the compensation amount using the audio signal level for each frequency range of each channel and the noise level for each frequency range (step 18 [ST18]). For the channels added in the above processing (ST15), the compensation amount is determined using the added value, and the determined compensation amount is applied to the original channel.

  The method for determining the compensation amount is not limited to a specific method. For example, only the frequency range in which the level of each channel is buried in noise may be determined, and the compensation amount may be determined for only the frequency range according to the level difference from the noise, or in the entire frequency band of each channel. The compensation amount may be determined so that the ratio between the signal level and the noise level is constant. Further, the compensation amount may be determined for each frequency range based on the above-described MAEQ method.

  As described above, as a compensation method and compensation device for noise of multi-channel audio signals, correlation between all channels can be calculated and a compensation amount for each frequency range can be determined.

(Modification 1 of Example 1)
In the first embodiment, the inter-frequency correlation calculation unit calculates the correlation coefficient for all the combinations of the input channel audio signals, but the combinations for calculating the correlation coefficient may be limited. That is, after calculating the frequency spectrum for each audio signal, the signal level for each channel of the input multi-channel audio signal is calculated, and the level is within a predetermined range from a channel higher than a predetermined level or the maximum signal level. A channel is selected, and a correlation coefficient for a combination of two different channels is calculated for the selected channel with a large level. The processing after obtaining the highly correlated channel set is the same as in the first embodiment. As a result, since the process of obtaining the correlation coefficient is performed only between channels having a large level that is dominant during listening, the process can be performed efficiently.

(Modification 2 of Example 1)
In the first embodiment, the correlation between channels (correlation coefficient) is calculated by the inter-frequency correlation calculation unit, and a set of channels whose correlation coefficient is higher than a predetermined threshold is defined as a highly correlated channel set. However, when a combination of channels having a high correlation between channels is specified in advance, for example, when an audio signal to which a combination of channels is added as acoustic metadata is input, the correlation coefficient calculation is skipped, This combination information is output to highly correlated channel combining section 114. Alternatively, an audio signal (or its frequency spectrum) to be added to the highly correlated channel synthesizer 114 may be output based on the combination information of the channels. As a result, the process of calculating the correlation coefficient of the channel can be omitted, and the processing time can be shortened.

  If the correlations of all the channel pairs are calculated, the number of channel combinations is large, so that the processing time becomes long and a lot of calculation resources are required. Therefore, it is desirable to automatically select a channel set for calculating the correlation when there is a restriction in calculation time and amount such as real-time processing.

  FIG. 8 is an example of a block diagram of the signal compensation amount determination unit 110 according to the second embodiment in the audio signal compensation apparatus of the present invention, and the signal compensation amount determination unit 110 in the case of calculating only the correlation of some channels. Indicates. FIG. 9 shows a flowchart of processing in this block diagram. 8 and 9, the case where the processing for obtaining the correlation in the frequency domain is described as an example. However, it is possible to adopt a configuration (not shown) for performing the correlation calculation in the time domain. 1 can be used for calculating the correlation coefficient.

  The signal compensation amount determination unit 110 according to the second embodiment includes a high-level channel set selection unit 118, a frequency spectrum calculation unit 112, and an inter-channel correlation calculation unit 111 having a frequency-specific channel correlation calculation unit 113, and a high correlation channel synthesis unit. 114, an audio signal level calculation unit 115, an additional audio signal level calculation unit 116, and a compensation amount calculation unit 117.

  In the second embodiment, the inter-channel signal correlation calculation unit 111 calculates a correlation for each frequency range between a combination of a channel having a high level of the input acoustic signal and a channel that is assumed to have a high correlation with the channel. .

  First, multi-channel (here, 10 channels) audio signals are input to the frequency spectrum calculation unit 112 and the high-level channel set selection unit 118 (step 21 [ST21] in the flowchart of FIG. 9). Then, the frequency spectrum calculation unit 112 calculates a frequency spectrum for each audio signal (step 22 [ST22]).

  On the other hand, the high level channel set selection unit 118 calculates the signal level for each channel of the input multi-channel audio signal (step 23 [ST23]). Next, the maximum signal level among the calculated signal levels is obtained, and all channels that fall within a predetermined difference α (for example, −3 dB) from the maximum signal level are selected (step 24 [ST24]). In other words, only the top few channels with a high level are selected from the inputs, and this is set as a high level channel set. As a means for selecting a channel audio signal having a high level, a predetermined level may be set in advance, and a channel higher than the specified level may be selected. Then, a corresponding preset channel set is selected from a group of preset channel sets defined in advance for the selected high-level channel.

  The preset channel set is a combination of channels that are defined in advance and are assumed to have a high correlation with a certain channel. For example, in the environment of FIG. 4, it is assumed that the audio signal (2ch) assigned to the speaker sp2 is likely to have many components common to the audio signals assigned to the nearby speakers, and the speakers [sp1, sp3] are set to 1. The channel [1ch, 3ch] assigned to each group is a channel set, and this indicates a preset channel set defined in advance for 2ch. For each high level channel selected in step 24, a corresponding predefined channel set (linked channel set) is selected. That is, when 2ch is selected as the high-level channel, two channels [1ch, 3ch] are additionally selected as the preset channel set. In this way, the high level channel and the preset channel set associated with it are selected. The high level channel set selection unit 118 outputs a high level channel set (and a corresponding preset channel set).

  Based on the output of the high-level channel set selection unit 118, the frequency spectrum is input to the inter-frequency channel correlation calculation unit 113 for the high-level channel and the preset channel set associated with the high-level channel. Note that the frequency spectrum of the channels not selected as the high level channel set is directly output from the frequency spectrum calculation unit 112 to the audio signal level calculation unit 115.

  The inter-frequency correlation calculation unit 113 calculates the correlation for each frequency range using the combination of the selected high-level channel and its preset channel set (step 25 [ST25]). For example, when only 2ch is selected as a high-level channel and [1ch, 3ch] is selected as the preset channel set, only a combination of 2ch and 1ch, 2ch and 3ch audio signals is provided for each frequency range. Calculate the correlation coefficient.

  In this way, a correlation coefficient between a channel having a high signal level and a preset channel associated with the channel in advance is calculated for each frequency. Then, it is determined whether the correlation coefficient calculated for each frequency is higher (or higher) than a predetermined threshold (step 26 [ST26]), and a set of channels having a high correlation value is set as a highly correlated channel set, It outputs to the high correlation channel synthetic | combination part 114, and it outputs directly to the audio | voice signal level calculation part 115 as a low correlation channel about other channels and frequency ranges.

  In the subsequent processing, the compensation amount may be determined in the same manner as in the first embodiment. That is, the highly correlated channel combining unit 114 adds the frequency components of the channel set input as the highly correlated channel set (step 27 [ST27]) and outputs the result to the audio signal level calculating unit 115.

  For example, in FIG. 4, 1ch and 5ch are selected as high-level channels, and a preset channel set that is assumed to have a high predefined correlation for 1ch is [10ch, 2ch, 5ch], and a predefined correlation for 5ch. It is assumed that the preset channel set that is assumed to be high is [1ch, 4ch, 6ch]. Here, 1ch and 5ch are selected for each preset channel set. In this case, it may be determined that the correlation between 1ch and 5ch, 5ch and 6ch is high, but in that case, all three channels may be added in the same way as in the first embodiment. 5ch is the signal level for compensation, 5ch is the signal level for compensation, 5ch is the signal level for compensation, 6ch is the signal level for compensation, and so on. These may be processed separately.

  The audio signal level calculation unit 115 calculates the audio signal level for each frequency range (step 28 [ST28]). The calculated signal level (power spectrum) is output to the compensation amount calculation unit 117.

  The additional audio signal level calculation unit 116 receives the additional audio signal (noise, noise), calculates a signal level (noise power spectrum) for each frequency range of the additional audio signal (step 29 [ST29]), The result is output to the compensation amount calculation unit 117.

  Thereafter, the compensation amount calculation unit 117 determines the compensation amount using the audio signal level for each frequency range of each channel and the noise level for each frequency range (step 30 [ST30]). For the channels added in the above processing (ST27), the compensation amount is determined using the added value, and the determined compensation amount is applied to the original channel.

  In the description of the second embodiment, the preset preset channel set has been described by taking as an example the environment in which ten speakers of FIG. 4 are installed, but the preset channel set can also be set in 22.2 multi-channel sound. .

  FIG. 10 shows the channel arrangement of 22.2 multi-channel sound. The channels (speakers) are arranged in three layers, an upper layer, a middle layer, and a lower layer, and the front surface, the intermediate surface, and the rear surface may be grouped.

  The preset channel sets that are assumed to be highly correlated in 22.2 multi-channel sound are summarized in Tables 1-7. One preset channel set is set for each target channel. Therefore, in 22.2 multi-channel sound, 22 preset channel sets are set.

  Table 1 shows an example of a basic channel set defined for each of the 22 channels.

  In addition, a preset channel set can be defined by a combination of the front surface, the rear surface, and the intermediate surface. Table 2 is a channel using a front speaker, Table 3 is a channel using an intermediate speaker, and Table 4 is a channel using a rear speaker, and a preset channel set may be defined for each group. . For example, in Table 2, the set defined in advance for the FL channel is a preset channel set for each channel of FR, FC, FLc, FRc, TpFL, TpFR, TpFC, BtFL, BtFC, and BtFR other than the FL channel. That is, in each table, a channel set other than its own channel is set as a preset channel set for the channel.

  Further, the channel sets shown in Tables 2 to 4 may be added to the respective channels in the preset channel set shown in Table 1.

  In addition, the preset channel set can be defined by a combination of each of the upper layer, the middle layer, and the lower layer. Table 5 shows channels using middle-layer speakers, Table 6 shows channels using upper-layer speakers, and Table 7 shows channels using lower-layer speakers. Each group may define a preset channel set. Similarly to Tables 2 to 4, in each table, a channel other than its own channel is set as a preset channel set of the channel.

  Further, the channel sets in Tables 5 to 7 may be added to the respective channels in the preset channel set in Table 1.

  The “frequency range” used in the description of the first and second embodiments is a spectrum frequency bin as shown in FIGS. 6 and 8, but a frequency band in which compensation processing is predetermined as shown in FIG. (For example, it is set every 100 Hz, every 1/3 octave band, etc.).

  FIG. 11 shows an example of a block diagram of the signal compensation amount determination unit 110 of the third embodiment in the audio signal compensation apparatus of the present invention. The third embodiment is a signal compensation amount determination unit 110 in a case where a compensation process is performed by calculating only correlations of some channels in a predetermined frequency band.

  The signal compensation amount determination unit 110 of the third embodiment includes a high-level channel set selection unit 118, a band division unit 119, and an inter-channel correlation calculation unit 111 including a band-specific channel correlation calculation unit 120, and a high correlation channel synthesis unit 114. And an audio signal level calculation unit 115, an additional audio signal level calculation unit 116, and a compensation amount calculation unit 117.

  In the third embodiment, the inter-channel signal correlation calculation unit 111 divides each channel in a predetermined frequency band, and the channel in which the level of the input acoustic signal is high and the correlation with the channel is assumed to be high in advance. The correlation for each band between the combinations of set channels is calculated.

  First, a multi-channel audio signal is input to the band dividing unit 119 and the high level channel set selecting unit 118. Then, as in the second embodiment, the high level channel set selection unit 118 calculates the signal level for each channel of the input multi-channel audio signal, selects the higher number channel having the higher signal level, and selects the selected high level channel. A corresponding preset channel set is selected from a group of preset channel sets defined in advance for the level channel. As a result, the high level channel and the preset channel set associated therewith are selected. Further, the band dividing unit 119 divides each channel in a predetermined frequency band.

  Based on the output of the high-level channel set selection unit 118, the band-specific channel correlation calculation unit 120 receives a high-level channel and a band-specific audio signal for the preset channel set associated with the high-level channel. Is done. Note that the audio signal of the channel not selected as the high level channel set is directly output from the band dividing unit 119 to the audio signal level calculating unit 115.

  The inter-band correlation calculation unit 120 calculates a correlation for each frequency band using a combination of the high-level channel and the selected preset channel set. Then, it is determined whether or not the correlation coefficient calculated for each band is higher (or higher) than a predetermined threshold value. For a set of channels having a high correlation value, a highly correlated channel set 114 The other channels and frequency bands are directly output to the audio signal level calculation unit 115 as low correlation channels.

  The highly correlated channel combining unit 114 adds the band components of the channel set input as the highly correlated channel set, and outputs the sum to the audio signal level calculating unit 115.

  The audio signal level calculation unit 115 calculates the level of the audio signal for each band, and the calculated signal level is output to the compensation amount calculation unit 117.

  The additional audio signal level calculation unit 116 receives an additional audio signal (noise, noise), calculates a signal level (noise level) for each band of the additional audio signal, and outputs the result to the compensation amount calculation unit 117. To do.

  Thereafter, the compensation amount calculation unit 117 determines the compensation amount using the audio signal level for each band of each channel (the signal level added for the added channels) and the noise level for each band. The determined compensation amount is applied to each band of each original channel.

  In the third embodiment, the high-level channel set is selected and processed in the same manner as in the second embodiment. However, as in the first embodiment, the frequency band is set for all channels without selecting the high-level channel set. A correlation may be obtained separately, and a technique as shown in the modification may be used.

  In the compensation amount calculation unit 117 according to the first to third embodiments, the compensation amount for each frequency range is determined in the entire frequency range using the level for each frequency range of each channel audio signal and the level for each frequency range of noise. .

  In the fourth embodiment, when determining the compensation amount, the multi-channel audio signal is selectively compensated in the frequency range based on the level of each channel audio signal (or synthesized signal) and the level of the additional audio signal (noise). To do. For example, by setting a level difference range for compensation from the level difference of each channel audio signal and noise for each frequency range, no compensation is performed when the original signal is small relative to noise. By this selective compensation, a sufficiently small signal with respect to noise is excessively compensated, and it is possible to prevent the timbre from abruptly changing unnaturally or disturbing the listening of the sound of other channels.

  FIG. 12 shows an example in which selective compensation is performed based on the level difference between the audio signal and noise for each frequency range. The horizontal axis of FIG. 12 is frequency [Hz], the vertical axis is level [dB], the rough broken line is the original audio signal (displayed as “no compensation”), and the thin solid line is the “noise” level, which is thick. The solid line is the audio signal after performing “selection compensation”. In addition, a signal when a conventional compensation method (sometimes referred to as “collective adjustment”) for compensating the entire frequency region at a constant rate is indicated by a fine broken line. Note that the selection compensation and the compensation for the batch adjustment are set so that the total energy of the entire multi-channels in all times coincides.

  According to the present embodiment, in FIG. 12, in the region I of about 2800 Hz or less where the level difference between the audio signal and the noise is small, the audio signal is compensated so as not to be buried in the noise. On the other hand, in the region II of about 2900 Hz to 3600 Hz where the audio signal is much smaller than the noise, the level difference between the audio signal and the noise exceeds the range of the level difference to be compensated, and therefore no compensation is performed.

  As a method of setting the compensation range, for example, it may be designated by a value such as up to a signal 9 dB lower than the level for each noise band, or the entire compensation amount may be designated, and a high priority band. Alternatively, compensation may be performed in order from a channel (for example, a band or channel with a large level), and compensation processing for an uncompensated band or channel may be omitted when the compensation amount exceeds the upper limit. Alternatively, a higher number of higher-level channels may be selected from the audio signal, an average level for each band of the selected channel may be calculated, and a compensation range may be determined using a level difference from the level for each noise band. . This is because, for example, the average level difference of the entire band below the noise level among the average levels of the high-level several channels with high levels is used as the compensation range, so that at least the signals of the high-level several channels with high levels are within the compensation range. A compensation range that is included can be set.

  In the selection compensation of the fourth embodiment, for example, the compensation amount calculation unit 117 obtains a channel and frequency range for which compensation is performed (or a channel and frequency range for which compensation is not performed), and outputs this to the audio signal compensation unit 130. This is executed by the audio signal compensation unit 130. This selective compensation can be combined with the processing of any of the multi-channel signal correlation calculation units in the first to third embodiments.

  As described above, when a multi-channel audio signal is reproduced in a noisy environment by the audio signal compensator of the present invention, even when a common signal is included in multiple channels, the multiple channels are compensated collectively according to the correlation. By doing so, you can hear it naturally.

  Note that a computer can be suitably used to cause the above-described audio signal compensation apparatus to function, and such a computer stores a program describing processing contents for realizing each function of the audio signal compensation apparatus. This program can be realized by reading out and executing this program by the CPU of the computer. This program can be recorded on a computer-readable recording medium.

(Verification of the effect of the invention)
For a plurality of contents, the case where the multi-channel audio compensation of the present invention (including the selective compensation of the fourth embodiment) was actually performed and the compensation by the collective adjustment of the conventional technology were compared. Six evaluators compared the uncompensated voice, the compensated voice according to the present invention, and the compensated voice by the conventional batch adjustment for six contents, and evaluated which voice is easy to hear. .

  As a result, both the compensation method according to the present invention and the collective adjustment improved the listening in all contents compared to before compensation, but there were a total of 34 responses that the compensation method according to the present invention is easier to hear. A total of two respondents said that compensation by collective adjustment was easy to hear. Thereby, it was verified that the effect of the compensation method of the present invention is high.

  Although the above embodiment has been described as a representative example, it will be apparent to those skilled in the art that many changes and substitutions can be made within the spirit and scope of the invention. Therefore, the present invention should not be construed as being limited by the above-described embodiments, and various modifications and changes can be made without departing from the scope of the claims. For example, a plurality of constituent blocks described in the embodiments can be combined into one, or one constituent block can be divided.

DESCRIPTION OF SYMBOLS 100 Audio signal compensation apparatus 110 Signal compensation amount determination part 111 Channel signal correlation calculation part 112 Frequency spectrum calculation part 113 Frequency-specific channel correlation calculation part 114 High correlation channel synthesis part 115 Audio signal level calculation part 116 Additional audio signal level calculation part 117 Compensation amount calculation unit 118 High-level channel set selection unit 119 Band division unit 120 Inter-channel correlation calculation unit 130 Audio signal compensation unit

Claims (11)

A correlation calculation unit between channel signals for calculating a correlation coefficient between multi-channel audio signals;
A highly correlated channel combining unit that generates a highly correlated channel combined signal by adding together the frequency components of all the channel sound signals whose correlation coefficient between channel sound signals is higher than a predetermined threshold;
An audio signal level calculation unit that calculates the level of each channel audio signal that was not used for generating the highly correlated channel synthesized signal and the highly correlated channel synthesized signal;
An additional audio signal level calculation unit for calculating the level of an additional audio signal other than the multi-channel audio signal;
A multi-channel audio signal compensation amount that calculates a compensation amount of each channel audio signal from the level of each channel signal that was not used to generate the highly correlated channel synthesized signal and the highly correlated channel synthesized signal and the level of the additional audio signal A calculation unit;
A multi-channel audio signal compensator for compensating a multi-channel audio signal based on the compensation amount;
An audio signal compensator comprising:   2. The audio signal compensator according to claim 1, wherein calculation of the correlation coefficient of the multi-channel audio signal, generation of the highly correlated channel composite signal, calculation of the level of the audio signal, and compensation of the multi-channel audio signal are performed. An audio signal compensator, wherein the quantity is calculated for each predetermined frequency range, and the frequency characteristics of the multi-channel audio signal are compensated.   3. The audio signal compensator according to claim 1, wherein the inter-channel signal correlation calculation unit calculates a correlation coefficient for a combination of two different channels for all input channel audio signals. An audio signal compensator.   3. The audio signal compensator according to claim 1 or 2, wherein the inter-channel signal correlation calculation unit changes a channel larger than a predetermined level or a maximum signal level within a predetermined range from input channel audio signals. An audio signal compensation apparatus, wherein a certain channel is selected, and a correlation coefficient for a combination of two different channels is calculated for all the selected channels.   3. The audio signal compensator according to claim 1 or 2, wherein the inter-channel signal correlation calculation unit changes a channel larger than a predetermined level or a maximum signal level within a predetermined range from input channel audio signals. Select a channel, determine a preset channel set that is assumed to have a high predefined correlation for each of the selected channels, and then the preset channel set corresponding to the selected channel An audio signal compensator for calculating a correlation coefficient for a combination of channels.   3. The audio signal compensator according to claim 1, wherein when the channel signal correlation calculation unit preliminarily specifies a combination of channels having high correlation, the correlation coefficient calculation is skipped and the high correlation channel is calculated. An audio signal compensator, which outputs channel combination information to be added to a synthesizer.   7. The audio signal compensation apparatus according to claim 1, wherein the inter-channel signal correlation calculation unit obtains a cross-correlation function between the channel audio signals, and within a predetermined time difference of the cross-correlation function. An audio signal compensator using the largest function value as a correlation coefficient between the channel audio signals.   3. The audio signal compensator according to claim 2, wherein the multi-channel is based on the level of each channel audio signal not used for generating the high correlation channel composite signal and the high correlation channel composite signal and the level of the additional audio signal. An audio signal compensator characterized by selectively performing audio signal compensation in a frequency range. The audio signal compensator according to claim 8,
Only when the level difference for each predefined frequency range between each channel signal that was not used to generate the highly correlated channel synthesized signal and the highly correlated channel synthesized signal and the additional audio signal is within a predefined range, Providing compensation,
Compensating in order from the highest priority band or channel in accordance with the upper limit of the overall compensation amount given in advance, and omitting compensation processing to the uncompensated band or channel when the compensation amount exceeds the upper limit Or select a channel audio signal that is higher than a predetermined level, calculate an average level for each of the selected frequency ranges of the entire selected channel, and calculate the calculated average level for each frequency range of the additional signal. Determining the compensation range using the level difference from the level of
An audio signal compensation device, wherein a compensation range is selected by any of the above. Calculate the correlation coefficient of multi-channel audio signals,
Generating a highly correlated channel composite signal by adding together the frequency components of all channel audio signals whose correlation coefficient between channel audio signals is higher than a predetermined threshold;
Calculating the level of each channel audio signal that was not used to generate the high correlation channel synthesis signal and the high correlation channel synthesis signal, and the additional audio signal other than the multi-channel audio signal;
The amount of compensation of each channel audio signal is calculated from the level of each channel signal that was not used for generating the high correlation channel synthesis signal and the high correlation channel synthesis signal and the level of the additional audio signal,
Compensating a multi-channel audio signal based on the compensation amount, a method of compensating an audio signal. The program for functioning a computer as an audio | voice signal compensation apparatus as described in any one of Claims 1 thru | or 9.

Images