JP6846822B2 - Audio signal processor, audio signal processing method, and audio signal processing program - Google Patents

Description

One aspect of the present invention relates to an audio signal processing device, an audio signal processing method, and an audio signal processing program.

A method of changing the number of channels of an audio signal has been conventionally known. Specifically, a method called upmix that converts an M channel audio signal into an N channel (however, N> M) audio signal and a method called a downmix that converts an N channel audio signal into an M channel audio signal. Exists. For example, converting a 2-channel (left channel and right channel) audio signal to a 5.1 channel audio signal is an example of an upmix. Also, the conversion of a 5.1 channel audio signal to a 2 channel audio signal is an example of downmixing.

For example, Patent Document 1 below describes a surround playback device that makes a stereo broadcast of a live sports program of television or radio an announcement that has a powerful presence and is easy to hear. This device has front left / right channel signal creation means, front center channel signal creation means, and rear left / right surround channel signal creation means. The front left / right channel signal creation means selectively adds a reverberant sound to each front left / right channel audio signal obtained by performing matrix processing on a 2-channel audio signal input, and adjusts the front volume. And output as each audio signal for the front left / right channel. The front center channel signal creation means adjusts the center volume as an audio signal for the front center channel and outputs it as an audio signal for the front center channel without adding reverberation to the audio signal obtained by extracting in-phase components from the 2-channel audio signal input. .. The rear left / right surround channel signal creation means adds reverberation to each front left / right channel audio signal obtained by matrix processing and adjusts the rear volume to adjust the rear left / right channel audio. Output as a signal.

The following non-patent documents 1 and 2 are documents that describe an upmix method. Non-Patent Document 1 describes a method of dividing a stereo signal into bands, dividing the stereo signal into a main signal and an ambience signal for each band, and reproducing the ambience signal from the rear channel of 5.1 channels. Non-Patent Document 2 describes a method in which a stereo signal is band-divided, the stereo signal is divided into a direct sound component and a reverberation sound component, and the reverberation sound component is reproduced from the side.

Each of the following non-patent documents 3 and 4 discloses a method of generating an audio signal of 3 channels or more by dividing a multi-channel audio signal into a pair of audio signals of 2 channels.

JP-A-2007-28065

C. Avendano and J-M Jot, "A Frequency-Domain Approach to Multichannel Upmix," J. Audio Eng. Soc., Vol. 52, No. 7/8, pp. 740-749, 2004 C. Faller, "Multiple-Loudspeaker Playback of Stereo Signals," J. Audio Eng. Soc., Vol. 54, No. 11, pp. 1051-1064, 2006 J. Thompson, B. Smith, A. Warmer and J-M Jot, “Direct-Diffuse Decomposition of Multichannel Signals Using a System of Pairwise Correlations,” Proc. Audio Eng. Soc. 133rd Convention, Paper no. 8807, 2012 C. Faller, L. Altmann, J. Levinson and M. Schmidt, “Multichannel Ring Upmix,” Proc. Audio Eng. Soc. 134th Convention, Paper no. 8908, 2013

Since the surround reproduction device described in Patent Document 1 adds a reverberant sound to the original sound, the atmosphere (for example, timbre) of the reproduced sound is changed or impaired from the original sound. On the other hand, the methods described in Non-Patent Documents 1 and 2 do not add reverberation, but in principle, they can be applied only to two-channel audio signals (that is, stereo signals).

In the methods described in Non-Patent Documents 3 and 4, a component having a high correlation between two channels of audio signals is extracted as a coherent component, so that sound information located near the middle of the two speakers is acquired. .. Therefore, in an audio system with three or more channels, only sound information near the middle of any two speakers can be extracted as a coherent component, and the sound located in the central part of the area surrounded by all speakers can be extracted. Information cannot be extracted.

Therefore, regardless of the number of channels of the original sound, a method of maintaining the atmosphere of the original sound as much as possible when changing the number of channels of the audio signal is desired.

The audio signal processing device according to one aspect of the present invention is a reception unit that receives audio signals of a plurality of channels and a division unit that executes division processing for dividing the audio signal into a coherent component and a field component for each channel. When the division process uses one channel to be divided as the target channel, the estimated signal calculated by using at least the audio signal of the channel other than the target channel is the audio signal of the target channel. The step including extracting the estimated signal having the highest correlation as the coherent component of the target channel and extracting the difference between the audio signal of the target channel and the coherent component of the target channel as the field component of the target channel. It includes a division unit and an output unit that outputs a coherent component and a field component of each channel extracted by the division unit.

The audio signal processing method according to one aspect of the present invention includes a reception step in which the audio signal processing device receives audio signals of a plurality of channels, and a division in which the audio signal processing device divides the audio signal into a coherent component and a field component. This is a division step in which processing is executed for each channel, and when the division processing uses one channel that is the target of the division processing as the target channel, it is calculated using at least the audio signals of channels other than the target channel. The step of extracting the estimated signal having the highest correlation with the audio signal of the target channel among the estimated signals as the coherent component of the target channel, and the difference between the audio signal of the target channel and the coherent component of the target channel are the target channel. The division step includes a step of extracting as a field component of the above, and an output step in which the audio signal processing device outputs a coherent component and a field component of each channel extracted in the division step.

The audio signal processing program according to one aspect of the present invention is a reception step for receiving audio signals of a plurality of channels and a division step for dividing the audio signal into a coherent component and a field component for each channel. When the division process uses one channel to be divided as the target channel, the estimated signal calculated by using at least the audio signal of the channel other than the target channel is the audio signal of the target channel. The step including extracting the estimated signal having the highest correlation as the coherent component of the target channel and extracting the difference between the audio signal of the target channel and the coherent component of the target channel as the field component of the target channel. The computer is made to execute the division step and the output step of outputting the coherent component and the field component of each channel extracted in the division step.

In such an aspect, a signal that is estimated using audio signals of channels other than the target channel and has the highest correlation for each actual audio signal of the target channel is extracted as a coherent component of the target channel. Further, the difference between the actual audio signal of the target channel and its coherent component is extracted as the field component of the target channel. This coherent and field component is obtained for each channel. In this way, by obtaining the coherent component and the field component of each channel using only the original audio signal without adding sound, the atmosphere of the original sound can be maintained as much as possible. In addition, since the coherent component and the field component can be obtained for the original number of channels, this method can be applied regardless of the number of channels of the original sound.

According to one aspect of the present invention, the atmosphere of the original sound can be maintained as much as possible when changing the number of channels of the audio signal regardless of the number of channels of the original sound.

It is a figure which shows the example of the audio signal processing which concerns on embodiment. It is a figure which shows the hardware configuration of the computer which functions as the audio signal processing apparatus which concerns on embodiment. It is a figure which shows the functional structure of the audio signal processing apparatus which concerns on embodiment. It is a figure which shows the block which is the unit which processes an audio signal. It is a figure which shows the processing in a certain channel. It is a flowchart which shows the operation of the audio signal processing apparatus which concerns on embodiment. It is a flowchart which shows the detail of the extraction of the coherent component shown in FIG. It is a figure which shows the structure of the audio signal processing program which concerns on embodiment. It is a figure which shows the example of the extraction of the coherent component in the conventional method. It is a figure which shows the example of the extraction of the coherent component in an embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same or equivalent elements are designated by the same reference numerals, and duplicate description will be omitted.

The functions and configurations of the audio signal processing device 10 according to the embodiment will be described with reference to FIGS. 1 to 5. The audio signal processing device 10 is a computer that divides each of the audio signals of a plurality of channels into a coherent component and a field component. The audio signal is a digital signal including sounds in a frequency band (generally about 20 Hz to 20000 Hz) that can be heard by humans, and is converted into an analog signal as needed. Examples of sounds represented by audio signals include, but are not limited to, voice, music, video sounds, natural sounds, or any combination thereof.

FIG. 1 shows an example of processing an audio signal by the audio signal processing device 10, and more specifically, shows processing of two channels (L channel and R channel), that is, a stereo audio signal. The audio signal processing device 10 divides the signal of each channel into a coherent component and a field component.

The coherent component of one channel is a component that has a high correlation with the audio signal of the other channel. The field component of a channel is the difference between the audio signal of that channel (ie, the original signal) and the coherent component of that channel. More specifically, the field component is a component obtained by subtracting the coherent component from the audio signal. The coherent component is a sound with a clear direction, while the field component is a diffusive, ambient sound. In the following, the sound corresponding to the field component is also referred to as “field sound”.

In FIG. 1, the audio signal processing device 10 divides the L-channel audio signal into the L-channel coherent component Lγ and the field component Lφ, and divides the R-channel audio signal into the R-channel coherent component Rγ and the field component Rφ. Is shown. The coherent component Lγ is a component having a high correlation with the audio signal of the R channel, and the coherent component Rγ is a component having a high correlation with the audio signal of the L channel.

Although FIG. 1 shows the processing of audio signals of two channels, the audio signal processing device 10 may process an arbitrary number of audio signals. The audio signal processing device 10 may process audio signals of 3 or more channels, and may process, for example, 22.2 channels of audio signals for 8K Super Hi-Vision.

In order to realize a stereophonic effect that can reproduce the direction, distance, and spread of sound in three-dimensional space, multi-channel audio signals are recorded by multiple microphones distributed in three-dimensional space. .. A multi-channel audio signal is recorded in a form in which a plurality of object sounds are mixed with each other or the target sound is mixed with a field sound. In general, the distance from the sound source is different for each microphone, so that the time when a specific sound arrives is different for each microphone, and as a result, the coherence of the recorded audio signal is low. If the coherent component can be extracted from the audio signal of each channel, the clarity of sound and the apparent width of the sound source (ASW) can be improved. In addition, by extracting the field component and using it in the upmix, it is possible to produce a good ambience effect (feeling that the sound surrounds the listener). In general, the coherent component corresponds to the target sound emitted from the main sound source (for example, singing voice, musical instrument sound, sound emitted from the speaker, etc.), and the field component corresponds to the sound whose direction is not clear (for example, echo, growl). Etc.).

Assuming that the audio signal of the lth channel among the N channels is x _l (n), this audio signal x _l (n) is the M target sound q _lm (n) (m = 1, ..., M). And the field sound v _l (n). That is, the audio signal x _l (n) is represented by the equation (1).

As shown by this equation (1), the target sound and the field sound can be regarded as statistically independent of each other. The coherent component γ _l (n) of the audio signal x _l (n) is represented by the equation (2).

The field component φ _l (n) of the audio signal x _l (n) is represented by the equation (3).

The specific implementation method of the audio signal processing device 10 is not limited. For example, the audio signal processing device 10 may be realized by installing a predetermined program (for example, an audio signal processing program P1 described later) on a computer such as a personal computer, a server, or a mobile terminal. Alternatively, an audio device such as an amplifier may function as the audio signal processing device 10.

FIG. 2 shows a general hardware configuration of a computer 100 that functions as an audio signal processing device 10. The computer 100 includes a processor (for example, a CPU) 101 that executes an operating system, an application program, and the like, a main storage unit 102 that is composed of ROM and RAM, and an auxiliary storage unit 103 that is composed of a hard disk, flash memory, and the like. It includes a communication control unit 104 composed of a network card or a wireless communication module, an input device 105 such as a keyboard and a mouse, and an output device 106 such as a monitor.

Each functional element of the audio signal processing device 10 is realized by loading predetermined software (for example, an audio signal processing program P1 described later) on the processor 101 or the main storage unit 102 and executing the software. The processor 101 operates the communication control unit 104, the input device 105, or the output device 106 according to the software, and reads and writes data in the main storage unit 102 or the auxiliary storage unit 103. The data or database required for processing is stored in the main storage unit 102 or the auxiliary storage unit 103.

The audio signal processing device 10 may be composed of one computer or a plurality of computers. When a plurality of computers are used, one audio signal processing device 10 is logically constructed by connecting these computers via a communication network such as the Internet or an intranet.

FIG. 3 shows the functional configuration of the audio signal processing device 10. As shown in FIG. 3, the audio signal processing device 10 includes a reception unit 11, a division unit 12, and an output unit 13 as functional components.

The reception unit 11 is a functional element that receives audio signals of a plurality of channels. “Receiving an audio signal” means that the audio signal processing device 10 acquires an audio signal by an arbitrary method. In other words, "accepting an audio signal" means that the audio signal is input to the audio signal processing device 10. The specific method of accepting the audio signal of each channel is not limited. For example, the reception unit 11 may receive the audio signal by accessing the database or other device and reading the data file of the audio signal. Alternatively, the reception unit 11 may receive an audio signal sent from another device via the communication network. Alternatively, the reception unit 11 may acquire the audio signal input by the audio signal processing device 10. In any case, the reception unit 11 outputs the audio signal of each received channel to the division unit 12.

The dividing unit 12 is a functional element that divides the audio signal of each channel into a coherent component and a field component. The following description is based on the premise that the dividing unit 12 processes the N-channel audio signal { _xl (n) | l = 1, ..., N} represented by the equation (4).

First, the division unit 12 divides the audio signal of each channel into signals in a plurality of time intervals. Specifically, the dividing unit 12 uses a window function (for example, a Kaiser-Vessel window) to divide an audio signal into signals having a short time interval (this is referred to as a “frame”). For example, if 1024 frequency points are used in the modified discrete cosine transform (MDCT) described later, the dividing unit 12 uses a Kaiser-Vessel window corresponding to the length of 2048 points to divide the audio signal into a plurality of frames. To divide. Normally, the number of samples in one frame is determined so as to obtain appropriate frequency resolution, but the number of samples is not sufficient for estimating the coherent component. Therefore, the dividing unit 12 sets a plurality of consecutive frames (for example, 24 frames) as a signal for one time interval (this is referred to as a "block"). FIG. 4 shows the concept of generating such a block, and more specifically, shows a process of dividing each of the audio signals of two channels (L channel and R channel) into a plurality of blocks.

When the audio signal of each channel is divided into a plurality of blocks, the division unit 12 executes the following processing for each block of each channel. In the present specification, a channel that is a target for dividing an audio signal into a coherent component and a field component (that is, a target for division processing) is referred to as a “target channel”. Here, the processing in one target channel will be described.

The dividing unit 12 extracts the coherent component of the target channel, and then extracts the field component of the target channel. FIG. 5 shows the concept of extraction of coherent components, which corresponds to the first half of the series of treatments. _{The division unit 12 uses a filter bank to convert the audio signal x l} (n) of the l-th channel, which is the target channel, into a signal of K frequency bands (sub-bands) (this is referred to as a “sub-band signal”). Divide into. _{Then, the dividing unit 12 extracts the coherent component γ l} ^(k) (n) (k = 1, ..., K) in each subband using the audio signals of channels other than the target channel. The dividing unit 12 uses the least squares method for this extraction. _{Then, the dividing unit 12 extracts the coherent component γ l} (n) of the target channel by adding the coherent components of all the subbands. After that, the dividing unit 12 extracts the field component φ _l (n) by subtracting the coherent component γ _{l (n) from the original audio signal x l (n).}

The dividing unit 12 executes the following processing for each block of the audio signal of the target channel.

The dividing unit 12 divides the audio signal x _l (n) of each channel into K subband signals x _l ^(k) (n) using a filter bank. This division is represented by the equation (5).

_{The subband signals x l} ^(k) (n) represented by the equation (5) are signals in the time domain, and are therefore time domain subband signals. Unlike the methods of Non-Patent Documents 1 to 4 that use signals in the frequency domain, the audio signal processing device 10 uses time domain subband signals, so that signals of any number of consecutive frames are processed as one block signal. By doing so, the estimated section length can be extended. As a result, the audio signal of each channel can be processed without impairing the sound quality of the obtained coherent component.

Subsequently, the dividing unit 12 _{applies the} ^{subband signals x l (k)} _{(n) to the subband signals {x m} ^(k) (n) of the same band (same subband) of N-1 channels other than the target channel. n) Estimated from the linear combination of | m = 1, ..., l-1, l + 1, ..., N}. This linear combination corresponding to a certain block is represented by the equation (6).

は、他チャネル（対象チャネル以外のＮ−１個のチャネル）の同帯域の信号との相関が高い成分と考えることができる。対象チャネルのサブバンド信号とこの推定信号との推定誤差ｅ_ｌ ^（ｋ）（ｎ）は式（７）で示される。

Estimated signal

Can be considered as a component having a high correlation with signals of the same band of other channels (N-1 channels other than the target channel). _{The estimation error el} ^(k) (n) between the subband signal of the target channel and this estimated signal is represented by the equation (7).

Dividing unit 12, the coefficient estimation errors to minimize ^{_{{a m (k) | m}} = 1, ..., l-1, l + 1, ..., N} seek the least squares method. The error function to be minimized is shown by Eq. (8).

とすると、最適な係数群

は式（９）を満たす。

here,

Then, the optimum coefficient group

Satisfies equation (9).

ここで、

である。When this equation (9) is combined with m = 1, ..., L-1, l + 1, ..., N, the equation (10) is obtained.

here,

Is.

_{The coefficient vector a ^ l} ^(k) of the target channel in the k-th subband is obtained by Eq. (11).

_{The coherent components γ l} ^(k) (n) of the target channel in the k-th subband are obtained by the formula (12). The coherent components γ _l ^(k) (n) correspond to the estimated signals having the highest correlation with the audio signals of the target channel among the estimated signals calculated using the audio signals of channels other than the target channel.

The dividing unit 12 obtains coherent components for all subbands. Then, the dividing unit 12 obtains the coherent component of the target channel by adding the coherent components of all the subbands. This process is represented by the equation (13).

Further, the dividing unit 12 obtains the field component of the target channel by subtracting the coherent component from the original audio signal of the target channel. This process is represented by the above equation (3).

対象チャネルのフィールド成分φ_ｌ（ｎ）は式（１５）により得られる。

The division unit 12 may obtain the field component by subtracting the coherent component from the audio signal in each subband, and may obtain the field component of the target channel by adding the field components of all the subbands. _{Specifically, the field components φ l} ^(k) (n) of the target channel in the k-th subband are obtained by the equation (14).

The field component φ _l (n) of the target channel is obtained by the equation (15).

The dividing unit 12 executes the above processing for each block of the audio signal of the target channel. Then, the dividing unit 12 extracts the coherent component of the target channel by connecting the coherent components of all the blocks. Further, the dividing unit 12 generates the field component of the target channel by connecting the field components of all the blocks.

The dividing unit 12 sets each of the plurality of channels as a target channel and executes the above processing to generate a coherent component and a field component for all the channels. Then, the division unit 12 outputs the coherent component and the field component of all channels to the output unit 13.

In this way, the dividing unit 12 divides the audio signal of each channel into a coherent component and a field component without adding another signal to the audio signal of each channel (that is, without adding another sound to the original sound). To divide.

The output unit 13 is a functional element that outputs the coherent component and the field component of each channel generated by the dividing unit 12 as a processing result. It can be said that this processing result realizes an upmix from N channel to 2N channel. The output method of the processing result is not limited in any way. For example, the output unit 13 may store the processing result in a storage device such as a memory or a database, or may transmit the processing result to another device via the communication network. Alternatively, the output unit 13 may output the coherent component and the field component of each channel to the corresponding speaker. In any case, the processing result of the audio signal processing device 10 can be used to use the existing audio material for producing content having a larger number of channels, or to reproduce the existing audio material in an audio system having a larger number of channels. It becomes possible to do.

The audio signal processor 10 may upmix an N-channel audio signal into a number of channels greater than 2N. Specifically, the audio signal processing device 10 generates signals having different correlations between channels by uncorrelated the extracted plurality of field components by the method described in the following reference. As a result, a larger number of field components than N can be obtained. For example, stereo audio material can be converted to 5.1-channel audio material, or reproduced with a higher sense of presence using a 5.1-channel audio system. Alternatively, 5.1-channel audio material can be converted to 22.2-channel audio material, or reproduced with a higher sense of presence using a 22.2-channel audio system.
(Reference) J. Breebaart and C. Fallar, “Spatial Audio Processing --MPEG Surround and Other Applications,” Wiley, 2007.

The audio signal processing device 10 may upmix an N-channel audio signal into an audio signal of J audio signals (where J> N) smaller than 2N. Specifically, the audio signal processing device 10 realizes an upmix from N channel to J channel by mixing N field components.

The processing result by the audio signal processing device 10 can be used not only for upmixing but also for downmixing.

Next, the operation of the audio signal processing device 10 will be described with reference to FIGS. 6 and 7, and the audio signal processing method according to the present embodiment will be described. In the audio signal processing device 10, first, the reception unit 11 receives audio signals of a plurality of channels (reception step). Subsequently, the division unit 12 executes a division process for dividing the audio signal into a coherent component and a field component for each channel (division step). Then, the output unit 13 outputs the coherent component and the field component of each channel (output step). In the following, a particularly important process (division step) of the division unit 12 will be described in detail.

FIG. 6 shows a process of generating a coherent component and a field component of one target channel.

First, the division unit 12 divides the audio signal of each channel into a plurality of blocks (step S11). By storing the audio signals of each channel and each block divided in step S11, step S11 can be omitted when processing the second and subsequent target channels.

Subsequently, the division unit 12 sets one of the plurality of blocks of the target channel as the processing target (step S12). Subsequently, the dividing unit 12 extracts the estimated signal having the highest correlation with the audio signal of the target channel among the estimated signals calculated using the audio signals of the channels other than the target channel as the coherent component of the target channel. (Step S13). Subsequently, the division unit 12 extracts the difference between the audio signal of the target channel and the coherent component thereof as a field component of the target channel (step S14). By such processing, the dividing unit 12 obtains one block of coherent component and field component of the target channel.

When the dividing unit 12 processes one block, it moves to the processing of the next block (see step S15). That is, the dividing unit 12 sets the next block as a processing target (step S12), and generates a coherent component and a field component of the block (steps S13 and S14). The dividing unit 12 executes the processes of steps S12 to S14 for all the blocks to generate the coherent component and the field component of all the blocks (YES in step S15). Then, the dividing unit 12 obtains the final coherent component of the target channel by connecting the coherent components of all the blocks, and obtains the final field component of the target channel by connecting the field components of all the blocks.

FIG. 7 shows the details of the process of step S13 in FIG. 6, that is, the details of the process of generating the coherent component of the target channel. The process shown in FIG. 7 is executed for each block of the audio signal of the target channel.

First, the division unit 12 generates a plurality of subband signals by dividing the block signal into a plurality of subbands for each channel (target channel and all other channels) (step S131). Subsequently, the dividing unit 12 sets one of the plurality of subbands as a processing target (step S132). Subsequently, the dividing unit 12 processes the estimated signal having the highest correlation with the subband signal of the target channel among the estimated signals calculated using the subband signals of channels other than the target channel. It is extracted as a coherent component of the target channel in (step S133). The dividing unit 12 executes the processes of steps S132 and S133 for all the subbands (see step S134). When the coherent components of all subbands are generated for the target channel (YES in step S134), the dividing unit 12 adds the coherent components of the target channel to generate the coherent components of the target channel (more specifically, one block of coherent components). ) Is generated (step S135).

Next, the audio signal processing program P1 for making the computer function as the audio signal processing device 10 will be described with reference to FIG.

The audio signal processing program P1 includes a main module P10, a reception module P11, a division module P12, and an output module P13. The main module P10 is a portion that comprehensively executes processing of audio signals. The functions realized by executing the reception module P11, the division module P12, and the output module P13 are the same as the functions of the reception unit 11, the division unit 12, and the output unit 13, respectively.

The audio signal processing program P1 may be provided after being fixedly recorded on a tangible recording medium such as a CD-ROM, a DVD-ROM, or a semiconductor memory. Alternatively, the audio signal processing program P1 may be provided via a communication network as a data signal superimposed on a carrier wave.

As described above, the audio signal processing device according to one aspect of the present invention executes a reception unit that receives audio signals of a plurality of channels and a division process that divides the audio signal into a coherent component and a field component for each channel. Of the estimated signals calculated by using at least the audio signals of channels other than the target channel when the division process uses one channel to be the target of the division process as the target channel. The step of extracting the estimated signal having the highest correlation with the audio signal of the channel as the coherent component of the target channel and the difference between the audio signal of the target channel and the coherent component of the target channel are extracted as the field component of the target channel. The division unit including the step and an output unit for outputting the coherent component and the field component of each channel extracted by the division unit are provided.

The audio signal processing method according to one aspect of the present invention includes a reception step in which the audio signal processing device receives audio signals of a plurality of channels, and a division in which the audio signal processing device divides the audio signal into a coherent component and a field component. This is a division step in which processing is executed for each channel, and when the division processing uses one channel that is the target of the division processing as the target channel, it is calculated using at least the audio signals of channels other than the target channel. The step of extracting the estimated signal having the highest correlation with the audio signal of the target channel among the estimated signals as the coherent component of the target channel, and the difference between the audio signal of the target channel and the coherent component of the target channel are the target channel. The division step includes a step of extracting as a field component of the above, and an output step in which the audio signal processing device outputs a coherent component and a field component of each channel extracted in the division step.

The audio signal processing program according to one aspect of the present invention is a reception step for receiving audio signals of a plurality of channels and a division step for dividing the audio signal into a coherent component and a field component for each channel. When the division process uses one channel to be divided as the target channel, the estimated signal calculated by using at least the audio signal of the channel other than the target channel is the audio signal of the target channel. The step including extracting the estimated signal having the highest correlation as the coherent component of the target channel and extracting the difference between the audio signal of the target channel and the coherent component of the target channel as the field component of the target channel. The computer is made to execute the division step and the output step of outputting the coherent component and the field component of each channel extracted in the division step.

In such an aspect, a signal that is estimated using audio signals of channels other than the target channel and has the highest correlation for each actual audio signal of the target channel is extracted as a coherent component of the target channel. Further, the difference between the actual audio signal of the target channel and its coherent component is extracted as the field component of the target channel. This coherent and field component is obtained for each channel. In this way, by obtaining the coherent component and field component of each channel using only the original audio signal without adding sound, the atmosphere of the original sound (for example, the original timbre) is maintained as much as possible or completely. Can be done. In addition, since the coherent component and the field component can be obtained for the original number of channels, this method can be applied regardless of the number of channels of the original sound. For example, one aspect of the present invention can be applied to an audio signal of any number of channels such as 2 channels, 3 channels, 5.1 channels, 22.2 channels and the like.

The superiority of the above aspect will be described with reference to FIGS. 9 and 10. FIG. 9 is a diagram showing an example of extraction of a coherent component in a conventional method, and FIG. 10 is a diagram showing an example of extraction of a coherent component in the above aspect. Both FIGS. 9 and 10 show an example in which an audio signal is output from three speakers 90 arranged in a triangular shape, and therefore this example shows a three-channel audio system.

As shown in FIG. 9, in the method described in Non-Patent Documents 3 and 4 described above, a component having a high correlation between two channels of audio signals is extracted as a coherent component 91 (note that the broken line 92 indicates a field component). ). Therefore, in such a conventional method, only sound information located in the intermediate portion 93 of the two speakers (channels) 90 can be acquired, and the central portion of the region surrounded by the three speakers (channels) 90. The sound information located at 94 cannot be extracted.

On the other hand, in the above aspect, the coherent component of one speaker (channel) 90 is estimated from the signal of another speaker (channel) 90. Therefore, as shown in FIG. 10, sound information located in the central portion 95 of the region surrounded by the three speakers (channels) 90 can be extracted. The central portion 95 may correspond to the sum of the portions 93 and 94 in FIG.

In the audio signal processing apparatus according to the other aspect, the division process combines a step of performing a process of dividing the audio signal into a plurality of frames for each channel by using a window function and at least two consecutive frames into one block. It may include a step of executing a process of generating a plurality of blocks for each channel by executing the process for the entire plurality of frames, and a step of extracting a coherent component of the target channel in each of the blocks.

By adopting a block composed of a plurality of frames, the number of samples for estimating the coherent component is increased, so that the coherent component can be extracted more accurately.

In the audio signal processing device according to the other aspect, the dividing unit divides the audio signal of each channel into a plurality of subbands to generate a plurality of subband signals for each channel, and a step of generating a plurality of subband signals and a plurality of subbands. In each case, a step of extracting the coherent component of the target channel and a step of extracting the coherent component of the target channel by adding the coherent components in a plurality of subbands may be included.

In general, some frequencies are often more important than others in speech processing. By processing each subband, the coherent component can be extracted according to the accuracy required for each frequency band, and the coherent component and the field component can be extracted with high accuracy.

Hereinafter, the present invention will be specifically described based on examples, but the present invention is not limited thereto.

Seven stereo audio materials (that is, two-channel audio signals) shown in Table 1 were prepared. All audio materials were obtained from commercially available CDs, and the sampling frequency was 44.1 kHz. The name column in Table 1 indicates the title of the song or the type of the song, and the explanation column indicates the form of the performance. In the mixing column, "Artifical" indicates that the material has been mixed, and "Natural" indicates that the material has not been mixed. The length column indicates the playback time.

In order to construct a filter bank that can completely reconstruct the audio signal, a superposition addition method using a modified discrete cosine transform (MDCT) was adopted. The Kaiser-Vessel window was used as a window function for dividing the audio signal into multiple frames. The frame length is 2048 points, which means that 1024 frequency points can be obtained in M DCT. The frequency points are grouped into 23 subbands as shown in Table 2. These sub-bands are a collection of 69 sub-bands in a 48 kHz long FFT (Fast Fourier Transform), one for each of three consecutive sub-bands, with reference to the MPEG-2 AAC standard. Twenty-four frames were regarded as one block. If the sampling frequency was 44.1 kHz, the block length was equivalent to 0.58 seconds.

The experimental results were evaluated by the intercorrelation coefficient between channels. Table 3 shows the intercorrelation coefficients of the original sound, the coherent component, and the field component. The coherent component showed a higher correlation than the original sound. Such a coherent component provides an atmosphere of a sound field narrower than the original sound. On the other hand, the field components showed a negative cross-correlation except for one material (“Quiet Night”). A good ambience effect can be obtained by reproducing the field component showing a negative cross-correlation with a speaker installed on the side or the rear. As a result, it is possible to reproduce a highly realistic sound.

The present invention has been described in detail above based on the embodiment. However, the present invention is not limited to the above embodiment. The present invention can be modified in various ways without departing from the gist thereof.

In the above embodiment, the division unit 12 estimates the coherent component of one target channel using audio signals of channels other than the target channel. As an example of this modification, the dividing unit estimates the coherent component of the target channel using the audio signal of the other channel, the past audio signal of the target channel, and at least one of the past audio signals of the other channel. You may. Here, the "past audio signal" is an audio signal of a block that is time ahead of the block to be processed. By estimating the audio signal of the target channel in the block to be processed by using the past audio signals of one or both of the target channel and the other channel, it can be expected that the coherent component can be extracted more accurately.

The procedure of the audio signal processing method executed by at least one processor is not limited to the example in the above embodiment. For example, the audio signal processing device may omit a part of the above-mentioned steps (processing), or may execute each step in a different order. Further, any two or more steps among the above-mentioned steps may be combined, or a part of the steps may be modified or deleted. Alternatively, the audio signal processor may perform other steps in addition to each of the above steps.

The audio signal processor may use either of the two criteria "greater than or equal to" and "greater than" when comparing the magnitude relations of the two numbers, and the two criteria "less than or equal to" and "less than". Either of these may be used. The selection of such criteria does not change the technical significance of the process of comparing the magnitude relations of two numbers.

10 ... Audio signal processing device, 11 ... Reception unit, 12 ... Division unit, 13 ... Output unit, el ... Estimation error, P1 ... Audio signal processing program, P10 ... Main module, P11 ... Reception module, P12 ... Division module, P13 … Output module.

Claims (5)

A reception unit that accepts audio signals from multiple channels,
A division unit that executes a division process for dividing the audio signal into a coherent component and a field component for each channel.
When one of the channels to be divided is used as the target channel, the correlation with the audio signal of the target channel among the estimated signals calculated by using at least the audio signals of channels other than the target channel. Is the step of extracting the highest estimated signal as the coherent component of the target channel, and
A division portion comprising a step of extracting a difference between the audio signal of the target channel and the coherent component of the target channel as the field component of the target channel.
An audio signal processing device including an output unit that outputs the coherent component and the field component of each channel extracted by the division unit. The division process
The step of executing the process of dividing the audio signal into multiple frames using the window function for each channel, and
A step of executing a process of generating a plurality of the blocks by executing a process of combining at least two consecutive frames into one block for the entire plurality of frames, and a step of executing the process of generating a plurality of the blocks for each channel.
Each of the blocks comprises a step of extracting the coherent component of the target channel.
The audio signal processing device according to claim 1. The divided part
A step of generating multiple subband signals for each channel by dividing the audio signal of each channel into multiple subbands,
A step of extracting the coherent component of the target channel in each of the plurality of subbands,
A step of extracting the coherent component of the target channel by adding the coherent components in the plurality of subbands is included.
The audio signal processing device according to claim 1 or 2. A reception step in which the audio signal processor accepts audio signals from multiple channels,
The audio signal processing device is a division step of executing a division process for dividing the audio signal into a coherent component and a field component for each channel, and the division process is a division process.
When one of the channels to be divided is used as the target channel, the correlation with the audio signal of the target channel among the estimated signals calculated by using at least the audio signals of channels other than the target channel. Is the step of extracting the highest estimated signal as the coherent component of the target channel, and
The division step including a step of extracting the difference between the audio signal of the target channel and the coherent component of the target channel as the field component of the target channel.
An audio signal processing method in which the audio signal processing apparatus includes an output step of outputting the coherent component and the field component of each channel extracted in the dividing step. A reception step that accepts audio signals from multiple channels,
A division step of executing a division process for dividing the audio signal into a coherent component and a field component for each channel, wherein the division process is performed.
When one of the channels to be divided is used as the target channel, the correlation with the audio signal of the target channel among the estimated signals calculated by using at least the audio signals of channels other than the target channel. Is the step of extracting the highest estimated signal as the coherent component of the target channel, and
The division step including a step of extracting the difference between the audio signal of the target channel and the coherent component of the target channel as the field component of the target channel.
An audio signal processing program that causes a computer to execute the coherent component of each channel extracted in the division step and an output step that outputs the field component.

Images