JP4840423B2 - Audio signal processing apparatus and audio signal processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a sound signal processor capable of excellently separating sound signals of a specified sound source selected by a user from sound signals of two systems including sound signals of a plurality of sound sources. <P>SOLUTION: The sound signals of the two systems are divided into a plurality of frequency bands, and a level ratio or a level difference between the sound signals of the two systems in each of the plurality of frequency bands is calculated. A plurality of multiplication coefficient generating sections are provided each for generating a multiplication coefficient corresponding to the calculated level ratio or level difference and corresponding to the sound source to be separated. In accordance with a control signal generated in response to the user's selection of the sound source to be separated, a switch means is controlled to select two of the plurality of multiplication coefficient generating sections. Multiplication coefficients from the selected two multiplication coefficient generating sections are used to output frequency divided spectrums obtained from the sound signals of the two systems, respectively, while controlling the level thereof. <P>COPYRIGHT: (C)2009,JPO&amp;INPIT

Description

  The present invention relates to an audio signal processing apparatus and method for separating audio signals of a specific sound source from two systems of input audio time series signals composed of audio signals from a plurality of sound sources.

  Many audio signals of each channel of stereo music signals of two left and right channels recorded on a record, a compact disc, or the like are composed of audio signals from a plurality of sound sources. In such a stereo audio signal, when reproduced by two speakers, a level difference may be added and recorded in each channel so that each of the plurality of sound sources is localized as a sound image between the speakers. Many.

For example, when recording the signals of five sound sources 1 to 5 as S1 to S5 and recording them as the left and right channel audio signals SL and SR,
SL = S1 + 0.9S2 + 0.7S3 + 0.4S4
SR = S5 + 0.4S2 + 0.7S3 + 0.9S4
As described above, the signals S1 to S5 of the sound sources 1 to 5 are added and mixed with a level difference between the left and right channels to form audio signals of the respective channels.

  In this way, a stereo audio signal recorded with the level difference added and the signals of the sound sources 1 to 5 distributed to the left and right channel audio signals is recorded by two speakers 1L and 1R as shown in FIG. 19, for example. Suppose that it was played. Then, the listener 2 can perceive sound images A, B, C, D, and E corresponding to the sound sources 1, 2, 3, 4, and 5, respectively. Further, it is known that the sound images A, B, C, D, and E are localized between the speaker 1L and the speaker 1R.

  Further, as shown in FIG. 20, the listener 2 wears the headphone device 3, and the left and right two-channel stereo audio signals are reproduced by the left speaker unit 3L and the right speaker unit 3R of the headphone device 3. Think about the case. In this case, as shown in the figure, the listener 2 can perceive sound images A, B, C, D, and E corresponding to the sound sources 1, 2, 3, 4, and 5 in the head. .

  If only the sound signal of a specific sound source can be separated and output from the general 2-channel stereo sound signal as described above, only the sound of a vocal or only the sound of a specific sound source such as a violin is extracted. Can be used for various purposes.

  As described above, a method shown in FIG. 21 is known as an example of a method for separating and outputting a sound signal of a specific sound source from a two-channel stereo sound signal.

  In the example of FIG. 21, a band-pass filter for extracting a portion having a large frequency energy constituting the sound signal of the sound source to be separated is prepared according to the sound source to be separated. Then, the sound signal of the sound source to be separated is separated from the two-channel stereo sound signal by this band pass filter.

  In the example of FIG. 21, the sound signal Sa of the sound source a and the sound signal Sb of the sound source b are separated from the sound signal SL of the left channel, and the sound signal Sc of the sound source c and the sound source d are separated from the sound signal SR of the right channel. This is a case of separating the audio signal Sd. The sound source separation processing circuit 7 includes four band pass filters 3 to 6 corresponding to the sound sources a to d, respectively.

  That is, as shown in FIG. 21, the audio signal SL of the left channel has a frequency that constitutes the audio signal Sb of the bandpass filter 3 and the audio source b that extract the portion of the frequency energy constituting the audio signal Sa of the audio source a. The band signals are supplied to a band pass filter 4 that extracts a portion with large energy, and audio signals Sa and Sb are obtained from the band pass filters 3 and 4 respectively.

  The right channel audio signal SR is a bandpass filter 5 that extracts a portion of the frequency signal constituting the sound signal Sc of the sound source c and a band that extracts a portion of the frequency signal that constitutes the sound signal Sd of the sound source d. The audio signals Sc and Sd are obtained from the band-pass filters 5 and 6, respectively.

Referenced patent documents are as follows.
Special table 2003-515771 gazette

  In the method of FIG. 21 described above, for example, when the center frequency constituting the sound source is in a different band, such as a bass guitar and a cymbal, it can be separated to some extent. However, in the case of sound sources with many frequency bands shared with each other, there is a problem that good separation cannot be performed including the overlapping of the frequency bands and the harmonics of each sound source outside the selected region of the band pass filter. was there.

  An object of the present invention is to provide an audio signal processing apparatus and method capable of satisfactorily separating an audio signal of a specific sound source from two types of audio signals including audio signals of a plurality of sound sources. .

Oite the present invention to solve the aforementioned problem, the audio signal processing apparatus, a time-series audio signals of two systems, the first and second converting means for converting the frequency domain signals, respectively, first Level calculation means for calculating the level ratio or level difference between the corresponding frequency division spectra using the frequency domain signals from the conversion means and the second conversion means, and multiplication coefficients corresponding to the sound sources to be separated are preset. A plurality of multiplication coefficient generators for generating a multiplication coefficient in response to an input of a level ratio or a level difference from the level calculation means , and a separation for generating a control signal in accordance with a selection operation of a sound source to be separated by a user a sound source selection signal generating means, based on the control signal from the separation sound source selection signal generating means, from among the multiplier coefficient generating unit of the multiple, is an output of the first and second converting means, separated pairs Switch means for selecting two multiplier coefficient generating unit that occur a multiplication coefficient corresponding to the sound source, using the multiplication factor from the two multiplier coefficient generating unit selected by the switch means, first converting means and the frequency domain signal obtained from each of the second conversion means, and an output control means for outputting the control the level of the frequency spectral, frequency domain signal from the output control unit, the time series signal Inverse conversion means for conversion is provided .

In the present invention, the level ratio or level difference of the audio signal of a predetermined sound sources, makes use of the fact that are mixed in time series audio signals of two systems.

Therefore, in the present invention , the sound signal of a specific sound source is improved by using the level ratio or level difference from two systems of time series sound signals in which the sound signals of the sound sources are mixed at a predetermined level ratio or level difference. The sound source that the user wants to separate can be dynamically changed.

According to the present invention, the audio signal processing device includes the first and second conversion units, the first conversion unit, and the second conversion unit that convert the two time series audio signals into frequency domain signals, respectively. Level calculation means for calculating the level ratio or level difference between the corresponding frequency division spectra using the frequency domain signal from, and the multiplication coefficient corresponding to the sound source to be separated from each other, the level ratio from the level calculation means Alternatively, a plurality of multiplication coefficient generators for generating a multiplication coefficient in response to an input of a level difference, separated sound source selection signal generating means for generating a control signal in accordance with a selection operation of a sound source to be separated by a user, and a separated sound source Based on the control signal from the selection signal generating means, it is used for the output of the first and second conversion means from among a plurality of multiplication coefficient generating sections, and generates a multiplication coefficient corresponding to the sound source to be separated. Each of the first conversion means and the second conversion means using the switching means for selecting the two multiplication coefficient generation sections to be used, and the multiplication coefficients from the two multiplication coefficient generation sections selected by the switching means. Output control means for controlling the frequency-divided spectrum level and outputting the frequency domain signal obtained from the above, and inverse conversion means for converting the frequency domain signal from the output control means into a time-series signal. Accordingly, from the time series audio signals of two systems of audio signals of each sound source are mixed at a predetermined level ratio or level difference, to better separate the audio signal of a specific sound source by using the level ratio or level difference In addition, the sound source that the user wants to separate can be dynamically changed.

  Embodiments of an audio signal processing apparatus and method according to the present invention will be described below with reference to the drawings.

  In the following description, a description will be given of a case where sound source separation is performed from the stereo audio signal composed of the left channel audio signal SL and the right channel audio signal SR described above.

  For example, the level difference between the sound signals S1 to S5 of the sound sources 1 to 5 and the left channel sound signal SL and the right channel sound signal SR is as shown in the following (Expression 1) and (Expression 2). It shall be attached, distributed and mixed.

SL = S1 + 0.9S2 + 0.7S3 + 0.4S4 (Formula 1)
SR = S5 + 0.4S2 + 0.7S3 + 0.9S4 (Formula 2)

  Comparing (Equation 1) and (Equation 2), the audio signals S1 to S5 of the sound sources 1 to 5 have a level difference as described above, and are divided into the left channel audio signal SL and the right channel audio signal SR. Distributed. Therefore, if the sound source can be sorted again from the left channel audio signal SL and / or the right channel audio signal SR by this distribution ratio, the original sound source can be separated.

  In the following embodiments, by utilizing the fact that each sound source generally has a different spectrum component, each of the left and right two-channel stereo audio signals is converted into the frequency domain by FFT processing having sufficient resolution. Divide into a number of frequency division spectral components. And the level ratio or level difference of each corresponding frequency division spectrum about the audio | voice signal of each channel is calculated | required. Then, in (Equation 1) and (Equation 2), a frequency division spectrum having a level ratio or level difference corresponding to the distribution ratio for the sound signal of the sound source to be separated is detected, and the detected frequency division spectrum component is detected. Isolate. This enables sound source separation with little influence from other sound sources.

  Before describing an embodiment of an audio signal processing apparatus according to the present invention, examples to be referred to in order to understand an embodiment of an audio signal processing apparatus according to an embodiment of the present invention are described as first to fourth embodiments. Will be described.

[Configuration of Audio Signal Processing Device of First Embodiment]
FIG. 1 is a block diagram illustrating an audio signal processing apparatus according to the first embodiment.

  The left channel audio signal SL of the two-channel stereo signal is supplied to an FFT (Fast Fourier Transform) unit 11 as an example of orthogonal transform means. In the FFT unit 11, when the signal SL is an analog signal, the signal SL is converted into a digital signal, and then subjected to FFT processing (fast Fourier transform) to convert the time-series audio signal into frequency domain data. Needless to say, when the signal SL is a digital signal, the analog-digital conversion in the FFT unit 11 is unnecessary.

  On the other hand, the right channel audio signal SR of the two-channel stereo signal is supplied to the FFT unit 12 as an example of orthogonal transform means. The FFT unit 12 converts the signal SR into a digital signal when the signal SR is an analog signal, and then performs FFT processing (fast Fourier transform) to convert the time-series audio signal into frequency domain data. Needless to say, when the signal SR is a digital signal, the analog-digital conversion in the FFT unit 12 is not necessary.

  The FFT units 11 and 12 of this example have the same configuration, and divide each time series signal SL, SR into frequency division spectrum components of a plurality of different frequencies. Here, the number of frequency divisions obtained as the frequency division spectrum is a large number according to the accuracy of the separation degree of the sound source, for example, 500 or more, preferably 4000 or more. This number of frequency divisions corresponds to the number of points in the FFT section.

  The frequency division spectrum outputs F1 and F2 from the FFT units 11 and 12 are supplied to the frequency division spectrum comparison processing unit 13 and the frequency division spectrum control processing unit 14, respectively.

  The frequency division spectrum comparison processing unit 13 calculates the level ratio between the same frequencies of the frequency division spectrum components F1 and F2 from the FFT unit 11 and the FFT unit 12, and supplies the calculated level ratio to the frequency division spectrum control processing unit 14. Output.

  The frequency division spectrum control processing unit 14 receives the information of the level ratio from the frequency division spectrum comparison processing unit 13, and converts only the frequency division spectrum components having the predetermined level ratio into the FFT unit 11 and the FFT unit. Extract from at least one of the twelve outputs. Then, the frequency division spectrum control processing unit 14 outputs the extraction result output Fex to the inverse FFT unit 15. In this example, the frequency division spectrum control processing unit 14 extracts the frequency division spectrum component having a predetermined level ratio from both the outputs of the FFT unit 11 and the FFT unit 12, and outputs the extraction result output Fex. Is output to the inverse FFT unit 15.

  In the frequency division spectrum control processing unit 14, the level ratio of the frequency division spectrum component to be extracted is set in advance by the user according to the sound source to be separated. Therefore, the frequency division spectrum control processing unit 14 extracts only the frequency division spectrum components of the sound signal of the sound source that is distributed to the left and right channels at the level ratio that is set to be separated by the user.

  The inverse FFT unit 15 converts the frequency division spectrum component of the extraction result output Fex from the frequency division spectrum control processing unit 14 into the original time-series signal, and the converted output signal of the sound source set by the user as desired to be separated Output as an audio signal SO. When the output audio signal is an analog signal, a D / A converter is provided on the output side of the inverse FFT unit 15 to convert it to an analog audio signal. The same applies to the following embodiments.

[Configuration of Frequency Division Spectrum Comparison Processing Unit 13]
In this example, the frequency division spectrum comparison processing unit 13 is functionally configured as shown in FIG. That is, the frequency division spectrum comparison processing unit 13 includes level detection units 21 and 22, level ratio calculation units 23 and 24, and a selector 25.

  The level detection unit 21 detects the level of each frequency component of the frequency division spectrum component F1 from the FFT unit 11, and outputs the detection output D1. In addition, the level detection unit 22 detects the level of each frequency component of the frequency division spectrum component F2 from the FFT unit 12, and outputs the detection output D2. In this example, the level of each frequency division spectrum detects an amplitude spectrum. A power spectrum may be detected as the level of each frequency division spectrum.

  Then, the level ratio calculation unit 23 calculates D1 / D2. Further, the level ratio calculation unit 24 calculates D2 / D1 of the reciprocal thereof. The level ratio calculated by the level ratio calculation unit 23 and the level ratio calculation unit 24 is supplied to the selector 25, and one of the level ratios is taken out from the selector 25 as the output level ratio r.

  The selector 25 selects and controls which one of the output of the level ratio calculation unit 23 and the output of the level ratio calculation unit 24 should be selected according to the sound source set by the user to be separated and its level ratio. A selection control signal SEL is supplied. The output level ratio r obtained from the selector 25 is supplied to the frequency division spectrum control processing unit 14.

  In this example, the value used as the level ratio of the sound source to be separated in the frequency division spectrum control processing unit 14 is always level ratio r ≦ 1. That is, the level ratio r input to the frequency division spectrum control processing unit 14 is obtained by dividing the level of the frequency division spectrum having the smaller level by the level of the frequency division spectrum having the larger level.

  Therefore, when the frequency division spectrum control processing unit 14 separates the sound source signal distributed so as to be included more in the left channel audio signal SL, the level from the level ratio calculation unit 23 The ratio calculation output is used. On the other hand, when separating the sound source signal distributed so as to be included more in the right channel audio signal SR, the level ratio calculation output from the level ratio calculation unit 24 is used.

  For example, it is determined that the user sets and inputs values PL and PR (PL and PR are values of 1 or less) of the left channel and right channel signals as the level ratio of the sound source to be separated. Think of when you are. In this case, when the set distribution ratio values PL and PR are PR / PL ≦ 1, the selection control signal SEL outputs the output (D2 / D1) of the level ratio calculation unit 23 from the selector 25 to the output level ratio. The selection control signal is selected as r. When the set distribution ratio values PL and PR satisfy PR / PL> 1, the selection control signal SEL outputs the output (D1 / D2) of the level ratio calculation unit 24 from the selector 25 to the output level ratio r. As a selection control signal.

  When the distribution ratio values PL and PR set by the user are equal (level ratio r = 1), the selector 25 selects either the output of the level ratio calculation unit 23 or the output of the level ratio calculation unit 24. You may choose.

[Configuration of Frequency Division Spectrum Control Processing Unit 14]
In this example, the frequency division spectrum control processing unit 14 is functionally configured as shown in FIG. That is, the frequency division spectrum control processing unit 14 includes a multiplication coefficient generation unit 31 and a sound source separation unit 32. The sound source separation unit 32 includes multiplication units 33 and 34 and an addition unit 35.

  The multiplication unit 33 is supplied with the frequency division spectrum component from the FFT unit 11 and the multiplication coefficient w from the multiplication coefficient generation unit 31, and the multiplication result of both is supplied from the multiplication unit 33 to the addition unit 35. Supplied. In addition, the frequency division spectrum component from the FFT unit 12 is supplied to the multiplication unit 34, and the multiplication coefficient w from the multiplication coefficient generation unit 31 is supplied. 35. The output of the adding unit 35 is the output Fex of the frequency division spectrum control processing unit 14.

  The multiplication coefficient generator 31 receives the output of the output level ratio r from the selector 25 of the frequency division spectrum comparison processor 13 and generates a multiplication coefficient w according to the level ratio r. The multiplication coefficient generation unit 31 is configured by a function generation circuit related to the multiplication coefficient w with the level ratio r as a variable, for example. Which function is selected as the function used for the multiplication coefficient generator 31 depends on the distribution ratio values PL and PR set by the user in accordance with the sound source to be separated.

  Since the level ratio r supplied to the multiplication coefficient generation unit 31 changes in units of each frequency component of the frequency division spectrum, the multiplication coefficient w from the multiplication coefficient generation unit 31 is also set in each frequency component unit of the frequency division spectrum. Will change.

  Therefore, in the multiplication unit 33, the level of each frequency division spectrum from the FFT unit 11 is controlled by the multiplication coefficient w, and in the multiplication unit 34, the level of each frequency division spectrum from the FFT unit 12 is changed to the multiplication coefficient w. Controlled by

  FIG. 4 shows an example of a function used in a function generation circuit as the multiplication coefficient generation unit 31. For example, when separating the sound signal S3 of the sound source localized in the center between the sound images of the left and right channels from the sound signals SL and SR of the left and right channels shown in the above (Expression 1) and (Expression 2), multiplication is performed. As the coefficient generating unit 31, a function generating circuit having characteristics as shown in FIG.

  The characteristic of the function of FIG. 4 (a) is that when the level ratio r of the left and right channels is 1 or close to 1, that is, in the frequency division spectrum component where the left and right channels are the same level or close to the same level, 1 vicinity. In this case, the multiplication coefficient w is 0 in a region where the level ratio r of the left and right channels is about 0.6 or less.

  Therefore, the multiplication coefficient w for the frequency division spectrum component for which the level ratio r from the selector 25 is 1 or close to 1 is 1 or a value close to 1. Therefore, the frequency division spectrum components are output from the multipliers 33 and 34 at almost the same level. On the other hand, since the multiplication coefficient w for the frequency division spectrum component for which the level ratio r from the selector 25 is about 0.6 or less is 0, the multiplication units 33 and 34 indicate that the frequency division spectrum component is The output level is set to 0 and no output is made.

  That is, from the multiple frequency division spectrum components, the left and right same level and the frequency division spectrum components in the vicinity thereof are output from the multiplication units 33 and 34 at almost the same level. Then, the frequency division spectrum component having a large level difference between the left and right channels is set to an output level of 0 and is not output. As a result, only the frequency division spectrum component of the sound signal S3 of the sound source distributed at the same level to the left and right channel sound signals SL and SR is obtained from the adder 35.

  Also, for example, when the sound signal S1 or S5 of the sound source localized only on one side of the left and right channels is separated from the left and right channel audio signals SL and SR shown in the (Expression 1) and (Expression 2). As the multiplication coefficient generation unit 31, a function generation circuit having characteristics as shown in FIG. 4B is used.

  In this case, in this embodiment, when the audio signal S1 is separated, the user sets and inputs the left / right distribution ratio PL: PR = 1: 0 for the sound source to be separated. Alternatively, settings are input such that PL = 1 and PR = 0. When the user sets in this way, the selector 25 is given a selection control signal SEL for controlling to select the level ratio from the level ratio calculator 23.

  On the other hand, when the audio signal S5 is separated, the user inputs the setting of the left / right distribution ratio PL: PR = 0: 1 for the sound source to be separated. Alternatively, settings are input such that PL = 0 and PR = 1. When the user sets in this way, the selector 25 is given a selection control signal SEL for controlling to select the level ratio from the level ratio calculation unit 24.

  The characteristic of the function of FIG. 4B is that the frequency ratio spectrum component in which the level ratio r of the left and right channels is 0 or near 0, the multiplication coefficient w is 1 or a value in the vicinity of 1, and the level ratio r of the left and right channels is about In the region of 0.4 or more, the multiplication coefficient w is 0.

  Therefore, the multiplication coefficient w for the frequency division spectrum component in which the level ratio r from the selector 25 is 0 or close to 0 is 1 or a value close to 1. Therefore, the frequency division spectrum components are output from the multipliers 33 and 34 at almost the same level. On the other hand, since the multiplication coefficient w for the frequency division spectrum component in which the level ratio r from the selector 25 is about 0.4 or more is 0, the multiplication units 33 and 34 indicate that the frequency division spectrum component is The output level is set to 0 and no output is made.

  That is, from the multiple frequency division spectral components, the frequency division spectral components in which one of the left and right channels is at a very large level compared to the other of the many frequency division spectral components are output at almost the same level. The Then, the frequency division spectrum component having a small level difference between the left and right channels is set to an output level of 0 and is not output. As a result, only the frequency division spectrum component of the sound signal S1 or S5 of the sound source that is distributed to only one of the left and right two-channel sound signals SL and SR is obtained from the adder 35.

  Further, for example, the sound signal S2 or S4 of the sound source distributed with a predetermined level difference to the left and right channels from the sound signals SL and SR of the left and right channels shown in the (Expression 1) and (Expression 2). 4 is used as the multiplication coefficient generator 31 as a function generating circuit having characteristics as shown in FIG.

  That is, the audio signal S2 is distributed to the left and right channels at a level ratio of D2 / D1 (= SR / SL) = 0.4 / 0.9 = 0.44. The audio signal S4 is distributed to the left and right channels at a level ratio of D1 / D2 (= SL / SR) = 0.4 / 0.9 = 0.44.

  In this case, in this embodiment, when the audio signal S2 is separated, the user sets and inputs the left / right distribution ratio PL: PR = 0.9: 0.4 for the sound source to be separated. Alternatively, settings are input such that PL = 0.9 and PR = 0.4. When the user sets in this way, since PR / PL <1, the selector 25 is supplied with a selection control signal SEL for controlling to select the level ratio from the level ratio calculator 23.

  On the other hand, when the audio signal S4 is separated, the user inputs the setting of the left / right distribution ratio PL: PR = 0.4: 0.9 for the sound source to be separated. Alternatively, settings are input such that PL = 0.4 and PR = 0.9. Since the PR / PL> 1 is set by the user in this way, the selector 25 is given a selection control signal SEL for controlling to select the level ratio from the level ratio calculation unit 24.

  The characteristic of the function of FIG. 4C is that the level ratio r of the left and right channels is 1 when D2 / D1 (= PR / PL) = 0.4 / 0.9 = 0.44, or the level ratio r is 0. For frequency division spectral components close to 44, the multiplication coefficient w is 1 or close to 1. The multiplication coefficient w is 0 in a region where the level ratio r of the left and right channels is not in the vicinity of about 0.44.

  Therefore, the multiplication coefficient w for the frequency division spectrum component in which the level ratio r from the selector 25 is 0.44 or in the vicinity of 0.44 is 1 or a value close to 1. Therefore, the frequency division spectrum components are output from the multiplication units 33 and 34 at almost the same level.

  On the other hand, the multiplication coefficient w for the frequency division spectrum component in which the level ratio r from the selector 25 is a value below about 0.44 and a value above about 0.44 is zero. Therefore, the frequency division spectrum components are not output from the multipliers 33 and 34 because the output level is 0.

  That is, from the multiple frequency division spectrum components, the frequency division spectrum components whose level ratio of the left and right channels is 0.44 or in the vicinity thereof are output from the multiplication units 33 and 34 at almost the same level. . The frequency division spectrum component in which the level ratio r of the left and right channels is a value below about 0.44 and a value above about 0.44 is set to an output level of 0 and is not output.

  As a result, only the frequency division spectrum component of the sound signal S2 or S4 of the sound source distributed at the level ratio of 0.44 to the left and right two-channel sound signals SL and SR is obtained from the adder 35.

  As described above, according to this embodiment, the sound signal of the sound source distributed to the left and right channels at a predetermined distribution ratio can be separated from the sound signal of the two channels based on the distribution ratio. it can.

  In this case, in the above-described embodiment, the sound signal of the sound source to be separated is extracted from both of the two-channel sound signals. However, it is not always necessary to separate and extract from both channels, but only from one channel. You may make it extract.

  In the above-described embodiment, the sound source signal is separated from the two audio signals based on the level ratio of the sound source signals distributed to the two audio signals. However, the sound source signal may be separated and extracted from at least one of the two systems of sound signals based on the level difference between the sound source signals and the two systems of sound signals.

  In the above description, the left and right two-channel stereo signals in which each sound source is distributed to the left and right channels according to (Equation 1) and (Equation 2) have been described as an example. However, even in a normal stereo music signal that is not intentionally distributed, the corresponding sound source can be separated according to the selection characteristic of the function shown in FIG.

  In another example, as shown in FIGS. 4D, 4E, etc., by changing the function, the level ratio range to be separated is changed, widened, narrowed, etc., so as to have different sound source selectivity. You can also

  With regard to the spectrum configuration of the sound source, many stereo music signals are composed of sound sources having different spectra, but these sound sources can also be separated in the same manner as described above.

  In addition, even for sound sources having many spectrum overlapping portions, the quality of sound source separation can be further improved by increasing the frequency resolution in the FFT units 11 and 12, for example, by using an FFT circuit of 4000 points or more.

[Configuration of Audio Signal Processing Device of Second Embodiment]
In the first embodiment described above, the sound signal of one sound source distributed with a predetermined level ratio or level difference from the two audio signals, in the above example, the left and right two-channel stereo signals SL and SR. In addition, it is separated and extracted from at least one of the two audio signals.

  In the second embodiment described below, instead of separating and extracting only the sound signal of one sound source from the two sound signals, the sound signals are distributed to the two sound signals with a predetermined level ratio or level difference. The audio signals of a plurality of sound sources are separated and extracted at the same time.

  FIG. 5 shows an example of the configuration of the audio signal processing apparatus according to the second embodiment. Parts corresponding to those of the first embodiment shown in FIG. However, in this example, the configurations of the frequency division spectrum comparison processing unit 13 and the frequency division spectrum control processing unit 14 are configured to separate audio signals of a plurality of sound sources, and the first implementation shown in FIG. Different from form. In addition, as many inverse FFT units as the number of outputs to be separated and extracted are provided.

  FIG. 6 shows an internal configuration example of the frequency division spectrum comparison processing unit 13 and the frequency division spectrum control processing unit 14 in the second embodiment.

  The frequency division spectrum comparison processing unit 13 in the second embodiment includes level detection units 21 and 22 and level ratio calculation units 23 and 24 as in the first embodiment described above. Then, the level ratio calculation units 23 and 24 detect the level ratios D2 / D1 and D1 / D2 of each frequency division spectrum component from the FFT units 11 and 12. In this example, the level ratio detection outputs from the level ratio calculation units 23 and 24 are respectively supplied to a plurality of selectors 251, 252,... 25n equal to the number of sound sources to be separated.

  Each of the plurality of selectors 251, 252,... 25n receives level ratio detection outputs from the level ratio calculation units 23 and 24 according to the distribution ratio of the sound signal of the sound source to be separated to the left and right channels. Selection control signals SEL1, SEL2,..., SELn for selecting one are supplied. That is, as described above, the selection control signals SEL1, SEL2,..., SELn have the level ratios with selectors 251, 252,. ... 25n is selected.

  The frequency division spectrum control processing unit 14 includes a plurality of multiplication coefficient generation units 311, 312,..., 31 n equal to the number of sound sources to be separated and sound source separation units 321, 322,. The level ratios r1, r2,..., Rn from the plurality of selectors 251, 252,... 25n of the frequency division spectrum comparison processing unit 13 are the multiplication coefficient generation units 311, 312,. , 31n respectively.

  Each of the multiplication coefficient generators 311, 312,..., 31n has a level corresponding to the distribution ratio of the sound signal of the sound source to be separated to the left and right two-channel sound signals, as in the first embodiment. A function of a multiplication coefficient with respect to the ratio (see the function example in FIG. 4 described above) is set.

  Therefore, the multiplication factor generators 311, 312,..., 31n correspond to the level ratios r1, r2,..., Rn from the selectors 251, 252,. Multiplication coefficients w1, w2,..., Wn corresponding to the sound signal of the sound source to be separated are generated. The multiplication coefficients w1, w2,..., Wn from the multiplication coefficient generation units 311, 312,..., 31n are supplied to the sound source separation units 321, 322,.

  Each of the sound source separation units 321, 322,..., 32 n is similar to the sound source separation unit 32 described above, and each of the multiplication units 33 and 34 that multiplies each of the outputs F 1 and F 2 by a multiplication coefficient, and both multiplication units. And an adder 35 that adds the outputs of 33 and 34.

  From the multipliers 33 and 34 of the sound source separation units 321, 322,..., 32n, the distribution ratio of the sound signal of the sound source to be separated to the left and right two-channel sound signals, or the level ratio in the vicinity thereof is obtained. The existing frequency division spectrum component is output at almost the same level. The other frequency division spectrum components from the multipliers 33 and 34 are set to a small level or level 0. As a result, extraction outputs Fex1, Fex2,..., Fexn of the frequency division spectrum components of the sound sources desired to be separated are obtained from the sound source separation units 321, 322,.

  Then, the extracted outputs Fex1, Fex2,..., Fexn from the sound source separation units 321, 322,..., 32n are supplied to the corresponding inverse FFT units 151, 152,. In the inverse FFT units 151, 152,..., 15n, the extracted outputs Fex1, Fex2,..., Fexn are returned to the original audio signals as time series signals, and the audio signals of the separated sound sources are separated. Output as SO1, SO2,..., SOn.

[Configuration of Audio Signal Processing Device According to Third Embodiment]
The third embodiment is an example in the case where the audio signals of the same sound source or the audio signals of different sound sources are separated and extracted from the audio signals SL and SR of the respective left and right channel audio signals.

  FIG. 7 is a block diagram showing a configuration example of the audio signal processing device according to the third embodiment. Also in this example, the outputs F1 and F2 comprising the frequency division spectrum components from the FFT units 11 and 12 are supplied to the frequency division spectrum comparison processing unit 13 and the frequency division spectrum control processing unit 14.

  Then, as described later, the frequency division spectrum control processing unit 14 extracts the frequency division spectrum component output FexL of the audio signal of the predetermined sound source extracted from the audio signal SL of the left channel and the audio signal SR of the right channel. The frequency division spectrum component output FexR of the sound signal of the predetermined sound source is obtained, supplied to the inverse FFT units 15L and 15R, respectively, and returned to the original time-series audio signal, and the inverse FFT units 15L and 15R Are derived as output sound signals SOL and SOR of a predetermined sound source.

  The frequency division spectrum comparison processing unit 13 in the third embodiment includes level detection units 21 and 22 and level ratio calculation units 23 and 24 as in the first embodiment described above. Then, the frequency division spectrum comparison processing unit 13 detects the level ratios D2 / D1 and D1 / D2 of the frequency division spectrum components from the FFT units 11 and 12. In this example, the level ratio detection outputs from the level ratio calculation units 23 and 24 are supplied to the left channel selector 25L and the right channel selector 25R, respectively.

  Each of the selectors 25L and 25R receives one of the level ratio detection outputs from the level ratio calculation units 23 and 24 in accordance with the distribution ratio of the sound signal of the sound source to be separated from the left and right channels to the left and right channels. Selection control signals SELL and SELR for selection are supplied. That is, as described above, the selection control signals SELL and SELR are such that each of the selectors 25L and 25R selects a level ratio in which the level on the channel side where more audio signals of the sound source to be separated are distributed becomes the denominator. Signal.

Frequency spectral control unit 14 is configured with a multiplier coefficient generating unit 31R for the multiplier coefficient generating unit 31L and the right channel for the left channel, the multiplying unit 32L and the multiplying unit 32R for the right channel for the left channel The The level ratio rL from the selector 25L of the frequency division spectrum comparison processing unit 13 is supplied to the multiplication coefficient generation unit 31L, and the level ratio rR from the selector 25R is supplied to the multiplication coefficient generation unit 31R.

  Each of the multiplication coefficient generators 31L and 31R has a function of a multiplication coefficient for the level ratio corresponding to the distribution ratio of the sound signal of the sound source to be separated to the left and right two-channel sound signals, as in the first embodiment. (See the function example in FIG. 4 described above).

  Therefore, each of the multiplication coefficient generators 31L and 31R is a multiplication coefficient corresponding to each of the level ratios rL and rR from each of the selectors 25L and 25R, and is a multiplication coefficient corresponding to the sound signal of the sound source to be separated. wL and wR are supplied to the multipliers 32L and 32R, respectively.

  Thereby, from each of the multipliers 32L and 32R, the distribution ratio of the sound signal of the sound source to be separated to the left and right two-channel sound signals, or the frequency division spectrum component which is the level ratio in the vicinity thereof is almost unchanged. The other frequency division spectral components are output at a level, and are set to a small level or level 0. As a result, the frequency division spectrum component extraction outputs FexL and FexR of the sound source desired to be separated are obtained from the multipliers 32L and 32R, respectively.

  Then, the extracted outputs FexL and FexR from the multipliers 32L and 32R are respectively supplied to the corresponding inverse FFT units 15L and 15R to be returned to the original audio signal as the time series signal, and the sound of the separated sound source Output as signal outputs SOL and SOR.

  In the third embodiment, the functions set in the multiplication coefficient generators 31L and 31R are not limited to those corresponding to different sound sources to be separated from the left and right two channels, but are also predetermined for the two left and right channels. The sound signal of the same sound source distributed with the level ratio or level difference can be used as a function for separating.

  In the latter case, the selectors 25L and 25R may select and output the same level ratio of the level ratio calculation units 23 and 24, and the multiplication coefficient generation units 31L and 31R may use the same function. . Thereby, for example, the signals S2 and S4 in (Equation 1) and (Equation 2) for the stereo signals SL and SR of the left and right channels described above are separated and extracted from the audio signals SL and SR of the left and right channels, respectively. It can be derived as outputs SOL and SOR.

  In this case, the characteristic of the function of the level ratio versus the multiplication coefficient set in the multiplication coefficient generators 31L and 31R is not the same characteristic when the same sound source is separated. For example, FIG. , (B), the characteristic curve of the function can be similar, and the magnitude of the multiplication coefficient w with respect to the level ratio r can be varied.

  In this way, for example, the sound signal of the sound source distributed with a level difference between the left and right channels is output at the same level as the sound signals SOL and SOR separated from the left and right channel sound signals SL and SR. Will be able to.

[Configuration of Audio Signal Processing Device of Fourth Embodiment; Automatic Music Recording Device]
FIG. 9 shows a fourth embodiment as a modification of the embodiment of FIG. In the fourth embodiment, the audio signal processing apparatus is configured as an automatic music recording apparatus.

  That is, in the embodiment of FIG. 9, frequency division spectrum maximum level detection units 16L and 16R are provided on the output side of the frequency division spectrum control processing unit 14 instead of the inverse FFT units 15L and 15R in FIG.

  In this embodiment, it is determined from the spectrum configuration of the separated sound source that the frequency division spectral component having the maximum amplitude level is the fundamental sound of the sound source. Based on this, the frequency division spectrum maximum level detection units 16L and 16R determine the frequency of the frequency division spectrum component having the maximum amplitude level from the outputs FexL and FexR from the frequency division spectrum control processing unit 14. To detect. The detected frequencies f1 and f2 and levels V1 and V2 are output as data.

  And although illustration was abbreviate | omitted, the frequency f1, f2 and level V1, V2 from the frequency division spectrum maximum level detection parts 16L and 16R are supplied to a pitch detection apparatus, for example, and a pitch is detected. Then, the detected pitch can be recorded on a recording medium, or can be entered on a score using a device (scoring device) for writing on the score.

  As described above, according to the fourth embodiment, the sound source is first separated from the stereo sound signal, and then the spectrum of the separated sound source is analyzed to detect the pitch of the sound source. Since automatic music recording can be performed based on the detected pitch, a system that enables automatic music recording from a stereo sound source in which a plurality of sound sources are mixed can be realized.

  In the example of FIG. 9, automatic music transcription is performed by separating the sound source from each of the left channel and the right channel. However, as in the second embodiment described with reference to FIGS. 5 and 6, the automatic musical score device is also used in the example of extracting the frequency division spectrum components of a plurality of sound sources from each of the two-channel audio signals. Can be realized. That is, in FIG. 5, all of the inverse FFT units 151, 152,..., 15 n are replaced with the frequency division spectrum maximum level detection unit, and the frequency and level of the maximum level frequency division spectrum are obtained as outputs thereof. To do. At the same time, these frequency and level outputs are supplied to the music transcription device via the pitch detection device.

  Further, the automatic music transcription device of the fourth embodiment can also be applied to the case of the first embodiment. Needless to say, the present invention is also applicable to an embodiment of an audio signal processing apparatus that performs sound source separation, which will be described later.

[Configuration of Audio Signal Processing Device in Fifth Embodiment]
The fifth embodiment is an embodiment of an audio signal processing device according to the present invention. In the fifth embodiment, the sound source that the user wants to separate from the two-channel audio signal can be dynamically changed.

  This fifth embodiment is applied to the third embodiment. That is, when the sound signals of different sound sources (or the same sound source) are separated from each of the two-channel sound signals SL and SR, the user can dynamically select and change the sound sources to be separated. This is the case.

  In the fifth embodiment shown in FIG. 10, the frequency division spectrum control processing unit 14 is provided with a plurality of multiplication coefficient generation units 31L1, 31L2,..., 31Ln as multiplication coefficient generation units for the left channel. . Further, the frequency division spectrum control processing unit 14 selects a multiplication coefficient from any one of the plurality of multiplication coefficient generation units 31L1, 31L2,..., 31Ln, and multiplies the selected multiplication coefficient. A switch circuit 36L that supplies the coefficient wL to the multiplier 32L is provided.

  Similarly, the frequency division spectrum control processing unit 14 includes a plurality of multiplication coefficient generation units 31R1, 31R2,..., 31Rn as right channel multiplication coefficient generation units. Then, the frequency division spectrum control processing unit 14 selects a multiplication coefficient from any one of the plurality of multiplication coefficient generation units 31R1, 31R2,..., 31Rn, and multiplies the selected multiplication coefficient. A switch circuit 36R that supplies the multiplier unit 32R as a coefficient wR is provided.

  Each of the plurality of multiplication coefficient generators 31L1, 31L2,..., 31Ln and 31R1, 31R2,. A function of level ratio to multiplication factor to be used is set.

  Further, the frequency division spectrum comparison processing unit 13 receives the level ratio calculation outputs of the level ratio calculation units 23 and 24, and outputs one of the level ratio calculation outputs to the multiplication coefficient generation units 31L1, 31L2,. 31Ln, 31R1, 31R2,..., 31Rn are provided with a selective distribution circuit 250 for supplying them.

  In the fifth embodiment, a separated sound source selection signal generation unit 17 is provided. The separated sound source selection signal generation unit 17 receives a signal Ma according to a selection operation of a sound source to be separated by a user through a selection operation unit as will be described later, and receives a selection signal SELT to be supplied to the selection distribution circuit 250. appear. Further, the separated sound source selection signal generation unit 17 generates a signal SWL that controls the switch circuit 36L and a signal SWR that controls the switch circuit 36R.

  Although not shown, the audio signal processing apparatus of this example accepts a selection operation of a sound source to be separated from a user through a graphical user interface through a display unit such as a selection operation knob or button or an LCD with a touch panel, for example. Like that. At this time, the selection operation target is a plurality of sound sources that can be separated by the functions set in the multiplication coefficient generators 31L1, 31L2,..., 31Ln, 31R1, 31R2,.

  For example, the plurality of separable sound sources may be configured such that the sound image localization position is gradually changed between the sound image localization position of the left channel and the sound image localization position of the right channel.

  In this case, the user can designate sound sources to be separated independently for each of the left channel and the right channel.

  For example, when a sound source that can be separated from the left channel audio signal SL by the multiplication coefficient from the left channel multiplication coefficient generator 31L1 is selected by the user through the selection operation knob, button, or graphical user interface. The separated sound source selection signal generator 17 that receives the signal Ma according to the selection operation generates the switch control signal SWL and the selection signal SELT corresponding to the signal Ma.

  At this time, the switch circuit 36L is switched to the state of selecting the multiplication coefficient generator 31L1 by the switch control signal SWL from the separated sound source selection signal generator 17. The selection / distribution circuit 250 selects one of the level ratio calculation units 23 and 24 (the one in which the level ratio is 1 or less) based on the selection signal SELT and supplies the selected one to the multiplication coefficient generation unit 31L1.

  Thus, the frequency division spectrum component FexL of the sound source as selected and designated is obtained from the multiplication unit 32L, and is returned to the original time-series audio signal by the inverse FFT unit 15L and output as the output SOL.

  Similarly, in the right channel, the sound signal of the sound source to be separated and set by the user is extracted.

  Note that the fifth embodiment of FIG. 10 is a case where the audio signal of a predetermined sound source is separately extracted from each of the audio signals of two channels (when applied to the third embodiment). This embodiment can also be applied to the first embodiment and the second embodiment.

  That is, for example, when applied to the first embodiment, a plurality of multiplication coefficient generators are provided in place of the multiplication coefficient generator 31 in FIG. In addition, a switch circuit is provided between the plurality of multiplication coefficient generation units and the sound source separation unit 32 so as to supply the multiplication coefficient from one of the plurality of multiplication coefficient generation units to the sound source separation unit 32. Provide. Further, a signal for receiving the user's selection operation signal Ma, switching the switch circuit, and controlling the multiplication coefficient generator to supply the appropriate level of the outputs of the level ratio calculators 23 and 24. Is provided with a separated sound source selection signal generator.

  For example, when applied to the second embodiment, a plurality of multiplication coefficient generators are provided in place of each of the multiplication coefficient generators 311, 312,..., 31 n in FIG. Further, a multiplication coefficient from one of the plurality of multiplication coefficient generators is set between the plurality of multiplication coefficient generators and each of the sound source separators 321, 322,. A plurality of switch circuits are provided so as to be supplied to 321, 322,. Further, it accepts a user's selection operation signal Ma and generates a switch control signal for controlling each switch circuit. Then, a separate sound source selection signal generation unit that generates a signal to be controlled so as to supply an appropriate level of the outputs of the level ratio calculation units 23 and 24 to each of the multiplication coefficient generation units is provided.

[Configuration of Audio Signal Processing Device of Sixth Embodiment]
In the above embodiment, the phase when the sound signal of each sound source is distributed to the sound signal of 2 channels is the same phase of 2 channels, but the sound signal of the sound source may be distributed in the opposite phase. . As an example, consider stereo audio signals SL and SR in which audio signals S1 to S6 from six sound sources MS1 to MS6 are distributed to two left and right channels as in the following (Equation 3) and (Equation 4).

SL = S1 + 0.9S2 + 0.7S3 + 0.4S4 + 0.7S6 (Formula 3)
SR = S5 + 0.4S2 + 0.7S3 + 0.9S4-0.7S6 (Formula 4)

  That is, the sound signal S3 of the sound source MS3 and the sound signal S6 of the sound source MS6 are distributed to the left and right channels at the same level. However, the audio signal S3 of the sound source MS3 is distributed in the same phase to the left and right channels, whereas the audio signal S6 of the MS6 is distributed in the opposite phase to the left and right channels.

  For this reason, as in the above-described embodiment, it is possible to separate and extract either the sound signal S3 of the sound source MS3 or the sound signal S6 of the sound source MS6 using only the level ratio or the level difference without considering the phase. , Either one cannot be separated and extracted. This is because the audio signals S3 and S6 are distributed to the left and right channels at the same level.

  Therefore, in the sixth embodiment, after the audio component is separated using the level ratio or the level difference in the same manner as described above, and further separated using the phase difference, the audio signal S3 of the sound source MS3 is separated from the audio signal S3. The sound signal S6 of the sound source MS6 can also be separated and output.

  FIG. 11 is a block diagram showing a configuration example of the audio signal processing apparatus according to the sixth embodiment. The frequency division spectrum comparison processing unit 103 in the audio signal processing device according to the sixth embodiment includes a level comparison processing unit 1031 and a phase comparison processing unit 1032.

Also, the frequency division spectral control processing unit 104 in the sixth embodiment, the first frequency spectral control unit 1041, the second round wavenumber spectrum control processor for executing sound source separation processing based on the phase difference Unit 1042.

  FIG. 12 is a block diagram illustrating a detailed configuration example of portions of the frequency division spectrum comparison processing unit 103 and the frequency division spectrum control processing unit 104 according to the sixth embodiment. That is, the level comparison processing unit 1031 of the frequency division spectrum comparison processing unit 103 has the same configuration as the frequency division spectrum comparison processing unit 13 of the first embodiment described above, and the level detection units 21 and 22 and the level ratio calculation. Units 23 and 24 and a selector 25.

  The first frequency division spectrum control processing unit 1041 of the frequency division spectrum control processing unit 104 also has substantially the same configuration as the frequency division spectrum control processing unit 14 of the first embodiment described above. That is, the frequency division spectrum control processing unit 1041 has a configuration of a sound source separation unit 32 including a multiplication coefficient generation unit 31 and multiplication units 33 and 34. However, the frequency division spectrum control processing unit 1041 does not include the adding unit 35.

  As shown in FIGS. 11 and 12, the level ratio output r from the level comparison processing unit 1031 is the same as that in the first embodiment, and the multiplication coefficient generation unit of the first frequency division spectrum control processing unit 1041 is used. 31. A multiplication coefficient wr corresponding to the function set in the multiplication coefficient generation unit 31 is generated from the multiplication coefficient generation unit 31 and supplied to the multiplication units 33 and 34.

  The multiplication unit 33 is supplied with the frequency division spectrum component from the FFT unit 11, and the multiplication result of the frequency division spectrum component and the multiplication coefficient wr is obtained from the multiplication unit 33. Further, the frequency division spectrum component from the FFT unit 12 is supplied to the multiplication unit 34, and a multiplication result of the frequency division spectrum component and the multiplication coefficient wr is obtained from the multiplication unit 34.

  That is, the multipliers 33 and 34 provide outputs in a state in which the frequency division spectrum components from the FFT units 11 and 12 are level-controlled in accordance with the multiplication coefficient wr from the multiplication coefficient generator 31.

  As described above, the multiplication coefficient generation unit 31 includes a function generation circuit related to the multiplication coefficient wr with the level ratio r as a variable. Which function is selected as the function used for the multiplication coefficient generator 31 depends on the distribution ratio of the sound source to be separated to the left and right channel audio signals.

  For example, a function relating to the level ratio of the multiplication coefficient wr having characteristics as shown in FIG. For example, when the sound signal of a sound source distributed to the left and right channels at the same level is separated and extracted, the specific function shown in FIG. 4A is set in the multiplication coefficient generator 31 as described above. The

  In the sixth embodiment, the outputs of the multiplying units 33 and 34 are respectively supplied to the phase comparison processing unit 1032 of the frequency division spectrum comparison processing unit 103 and the second frequency division spectrum of the frequency division spectrum control processing unit 104. It is supplied to the control processing unit 1042.

  As shown in FIG. 12, the phase comparison processing unit 1032 includes a phase difference detection unit 26 that detects the phase difference φ of the outputs of the multiplication units 33 and 34, and the information of the phase difference φ is subjected to a second frequency division spectrum control process. Supplied to the unit 1042.

  The second frequency division spectrum control processing unit 1042 includes two multiplication coefficient generation units 301 and 305, multiplication units 302 and 303, multiplication units 306 and 307, and addition units 304 and 308.

  The multiplication unit 302 is supplied with the output of the multiplication unit 33 of the first frequency division spectrum control processing unit 1041 and also supplied with the multiplication coefficient wp1 from the multiplication coefficient generation unit 301. The data is supplied from the multiplier 302 to the adder 304. Further, the multiplication unit 303 is supplied with the output of the multiplication unit 34 of the first frequency division spectrum control processing unit 1041 and the multiplication coefficient wp1 from the multiplication coefficient generation unit 301. The data is supplied from the multiplier 303 to the adder 304. The output of the adding unit 304 is the first output Fex1 of the frequency division spectrum control processing unit 104.

  Further, the multiplication unit 306 is supplied with the output of the multiplication unit 33 of the first frequency division spectrum control processing unit 1041 and is also supplied with the multiplication coefficient wp2 from the multiplication coefficient generation unit 305. The data is supplied from the multiplier 306 to the adder 308. The multiplication unit 307 is supplied with the output of the multiplication unit 34 of the first frequency division spectrum control processing unit 1041 and the multiplication coefficient wp2 from the multiplication coefficient generation unit 305. The data is supplied from the multiplier 307 to the adder 308. The output of the adding unit 308 is the second output Fex2 of the frequency division spectrum control processing unit 104.

  Multiplication coefficient generators 301 and 305 receive information on phase difference φ from phase difference detection unit 26 and generate multiplication coefficients wp1 and wp2 corresponding to the phase difference φ. Multiplication coefficient generators 301 and 305 are configured by a function generation circuit related to multiplication coefficient wp using phase difference φ as a variable. Which function is selected as the function used for the multiplication coefficient generators 301 and 305 is set by the user according to the phase difference of the sound source to be separated with respect to the two channels.

  Since the phase difference φ supplied to the multiplication coefficient generators 301 and 305 changes for each frequency component of the frequency division spectrum, the multiplication coefficients wp1 and wp2 from the multiplication coefficient generators 301 and 305 are also frequency division. Varies with each frequency component of the spectrum.

  Therefore, in multiplier 302 and multiplier 306, the level of each frequency division spectrum from multiplier 33 is controlled by multiplication coefficients wp1 and wp2. In addition, in the multiplication unit 303 and the multiplication unit 307, the level of each frequency division spectrum from the multiplication unit 34 is controlled by the multiplication coefficients wp1 and wp2.

  FIG. 13 shows an example of a function used in the function generation circuit as the multiplication coefficient generation units 301 and 305.

  The characteristic of the function of FIG. 13A is that the multiplication coefficient wp is 1 or near 1 when the phase difference φ between the left and right channels is 0 or close to 0, that is, in the frequency division spectrum component where the left and right channels are in phase or close to in phase It becomes. The multiplication coefficient wp is 0 in the region where the phase difference φ between the left and right channels is about π / 4 or more.

  For example, when the function of the characteristic shown in FIG. 13A is set in the multiplication coefficient generator 301, the frequency division spectrum component in which the phase difference φ from the phase difference detector 26 is 0 or close to 0. The multiplication coefficient wp for is 1 or a value close to 1. Therefore, the frequency division spectrum components are output from the multipliers 302 and 303 at almost the same level. On the other hand, since the multiplication coefficient wp for the frequency division spectrum component in which the phase difference φ from the phase difference detection unit 26 has a value of about π / 4 or more is 0, the multiplication units 302 and 303 receive the frequency division. Spectral components are not output at an output level of 0.

  That is, from the multiple frequency division spectrum components, the frequency division spectrum components having a phase difference between the left and right in-phase and the vicinity thereof are output from the multiplication units 302 and 303 at almost the same level. A frequency division spectrum component having a large phase difference between the left and right channels is set to an output level of 0 and is not output. As a result, only the frequency division spectrum component of the sound signal of the sound source distributed in phase with the left and right two-channel sound signals SL and SR is obtained from the adder 304.

  That is, the characteristic function shown in FIG. 13A is used to extract a sound source signal distributed in phase to the left and right channels.

  In addition, the characteristic of the function of FIG. 13B is that when the phase difference φ between the left and right channels is π or close to π, that is, in the frequency division spectrum component where the left and right channels are close to or out of phase, the multiplication coefficient wp 1 or 1 vicinity. The multiplication coefficient wp is 0 in the region where the phase difference φ between the left and right channels is about 3π / 4 or less.

  For example, when the function of the characteristic shown in FIG. 13B is set in the multiplication coefficient generator 301, the frequency division spectrum component in which the phase difference φ from the phase difference detector 26 is π or in the vicinity of π. The multiplication coefficient wp for is 1 or a value close to 1. Therefore, the frequency division spectrum components are output from the multipliers 302 and 303 at almost the same level. On the other hand, since the multiplication coefficient wp for the frequency division spectrum component in which the phase difference φ from the phase difference detection unit 26 is about 3π / 4 or less is 0, the multiplication units 302 and 303 receive the frequency division. Spectral components are not output at an output level of 0.

  That is, from the multiple frequency division spectrum components, the frequency division spectrum components having a phase difference between the left and right phases and the vicinity thereof are output from the multiplication units 302 and 303 at almost the same level. Then, the frequency division spectrum component having a small phase difference between the left and right channels is set to an output level of 0 and is not output. As a result, only the frequency division spectrum component of the sound signal of the sound source distributed in opposite phases to the left and right two-channel sound signals SL and SR is obtained from the adding unit 304.

  That is, the characteristic function shown in FIG. 13B is used to extract a sound source signal distributed in opposite phases to the left and right channels.

  Similarly, the characteristic function of FIG. 13C shows that the multiplication coefficient wp is 1 or near 1 in the frequency division spectrum component when the phase difference φ between the left and right channels is about π / 2 or close to about π / 2. Thus, the multiplication coefficient wp is 0 in other regions of the phase difference φ. Therefore, the function of the characteristic shown in FIG. 13C is used when extracting the sound source signal distributed to the left and right channels with phases different from each other by about π / 2.

  In addition, in the multiplication coefficient generators 301 and 305, a function of characteristics as shown in FIGS. 13D and 13E is set according to the phase difference when the sound signal of the sound source to be separated is distributed to two channels. You can also

  As described above, the first output Fex1 and the second output Fex2 obtained from the frequency division spectrum control processing unit 104 are supplied to the inverse FFT units 1501 and 1502, respectively, and returned to the original time-series audio signal. It is. This time-series audio signal is derived as the first and second output signals SO10 and SO20. When these first and second output signals SO10 and SO20 are derived as analog signals, D / A converters are provided at the output stages of inverse FFT units 1501 and 1502.

  In the sixth embodiment, for example, sound sources distributed at the same level but distributed to the left and right channels from the left and right two-channel audio signals SL and SR shown in (Expression 3) and (Expression 4). When separating the audio signal S3 of MS3 and the audio signal S6 of the sound source MS6 distributed to the left and right channels in opposite phases as outputs Fex1 and Fex2, the multiplication coefficient generator 31 has the configuration shown in FIG. A specific function as shown in FIG. 13 is set, a function having characteristics as shown in FIG. 13A is set in the multiplication coefficient generation unit 301, and a function shown in FIG. A function having characteristics as shown in b) is set.

  Then, as shown in FIG. 11 and FIG. 12, the multiplication unit 33 of the first frequency division spectrum control processing unit 1041 of the frequency division spectrum control processing unit 104 receives a signal (frequency division spectrum) obtained by performing FFT on the audio signal SL of the left channel. ) Of the frequency division spectrum component of (S3 + S6) is obtained, and the multiplication unit 34 obtains (S3-S6) of the signal (frequency division spectrum) obtained by FFT of the audio signal SR of the right channel. A frequency division spectral component is obtained. That is, since the signals S3 and S6 are distributed to the left and right channels at the same level, the first frequency division spectrum control processing unit 1041 outputs them without being separated.

  However, in the sixth embodiment, the signals S3 and S6 are separated as follows using the fact that the signals S3 and S6 are distributed to the left and right channels in opposite phases.

  That is, the outputs of the multipliers 33 and 34 are supplied to the phase difference detection unit 26 constituting the phase comparison processing unit 1032 of the frequency division spectrum comparison processing unit 103, and the phase difference φ between both outputs is detected. Information on the phase difference φ detected by the phase difference detection unit 26 is supplied to the multiplication coefficient generation unit 301 and also to the multiplication coefficient generation unit 305.

  In the multiplication coefficient generator 301, the function of the characteristic as shown in FIG. 13A is set. Therefore, the multipliers 302 and 303 extract the sound signal of the sound source distributed in phase to the left and right channels. . That is, only the frequency division spectral components of the audio signal S3 of the sound source MS3 in the in-phase relationship among the frequency division spectral components (S3 + S6) and the frequency division spectral components (S3-S6) are obtained from the multipliers 302 and 303, respectively. And supplied to the adding unit 304.

  Therefore, the frequency division spectrum component of the audio signal S3 of the sound source MS3 is derived as the output signal Fex1 from the adding unit 304 and supplied to the inverse FFT unit 1501. The separated audio signal S3 is returned to the time series signal by the inverse FFT unit 1501 and output as the output signal SO10.

  On the other hand, in the multiplication coefficient generation unit 305, the function of the characteristic as shown in FIG. 13B is set. Therefore, in the multiplication units 306 and 307, the sound signal of the sound source distributed in opposite phases to the left and right channels. To extract. That is, only the frequency division spectrum component of the audio signal S6 of the sound source MS6 in the opposite phase among the frequency division spectrum component (S3 + S6) and the frequency division spectrum component (S3-S6) is obtained from each of the multipliers 306 and 307. Obtained and supplied to the adder 308.

  Therefore, the frequency division spectrum component of the audio signal S6 of the sound source MS6 is derived from the addition unit 308 as the output signal Fex2, and is supplied to the inverse FFT unit 1502. The separated audio signal S6 is returned to the time series signal by the inverse FFT unit 1502 and output as the output signal SO20.

  In the embodiment shown in FIG. 11 and FIG. 12, the second frequency division spectrum control processing unit 1042 has two signals that cannot be separated using the level ratio in the first frequency division spectrum control processing unit 1041, the above-described example. Then, the in-phase signal S3 and the anti-phase signal S6 are separated using the multiplication coefficient and the multiplication unit, respectively. However, one of the two signals that cannot be separated using the level ratio is Once separated using the phase difference φ and the multiplication coefficient, the separated signal is summed with the signal from the first frequency division spectrum control processing unit 1041 (the signal obtained by adding the output of the multiplier 33 and the output of the multiplier 34). By subtracting from the other signal, the other signal of the two signals can be separated.

  In the embodiment shown in FIGS. 11 and 12, two separated sound source signals are obtained. However, one separated sound source signal may be output. Needless to say, the sixth embodiment can also be applied to the case where a plurality of sound sources are simultaneously separated as in the second embodiment.

  11 and 12 extract the sound source component distributed at the same level in the two audio signals on the basis of the level ratio of the two frequency division spectrums. The desired sound source separation is performed based on the phase difference of the frequency division spectrum of the system. For example, when the input audio signal is a two-system audio signal such as (S3 + S6) and (S3-S6) Needless to say, sound source separation can be performed based only on the phase difference.

  The sixth embodiment can also be applied to the automatic musical score device described as the fourth embodiment.

[Audio Signal Processing Device of Seventh Embodiment]
FIG. 14 is a block diagram illustrating a configuration example of the audio signal processing device according to the seventh embodiment. In the example of FIG. 14, one of the left and right channel audio signals SL and SR, and in the example of the figure, the left channel audio signal SL is applied to the left and right channels with a predetermined level ratio or level difference using a digital filter. The sound signal of the distributed sound source is separated.

  That is, the audio signal SL (digital signal in this example) SL of the left channel is supplied to the digital filter 42 through the delay unit 41 for timing adjustment. As will be described later, the digital filter 42 is supplied with filter coefficients formed based on the level ratio of the sound signal of the sound source to be separated to the left and right channels, and the sound signal of the sound source to be separated is supplied to the digital filter 42. It is extracted from the filter 42.

  The filter coefficient is formed as follows. First, the left and right channel audio signals SL and SR (digital signals) are supplied to the FFT unit 43 and the FFT unit 44, respectively, and subjected to FFT processing to convert the time series audio signal into frequency domain data. From each of the FFT units 44, a large number of frequency division spectrum components having different frequencies are output.

  Each of the frequency division spectrum components from each of the FFT units 43 and 44 is supplied to the level detection units 45 and 46, and the amplitude spectrum or power spectrum thereof is detected, whereby the level is detected. The level values D1 and D2 detected by each of the level detection units 45 and 46 are supplied to the level ratio calculation unit 47, and one of the level ratios D1 / D2 or D2 / D1 is calculated.

  The level ratio value calculated by the level ratio calculation unit 47 is supplied to the weighting coefficient generation unit 48. The weighting coefficient generation unit 48 corresponds to the multiplication coefficient generation unit of the above-described embodiment, and is large in the mixing level ratio of the sound signal of the sound source to be separated to the sound signal of the left and right channels and the level ratio in the vicinity thereof. A value weighting coefficient is output, and a small weighting coefficient is output for other level ratios. This weighting coefficient is obtained for each frequency of the frequency division spectrum component that is the output of the FFT units 43 and 44.

  The frequency domain weighting coefficient from the weighting coefficient generating section 48 is supplied to the filter coefficient generating section 49 and converted into a time axis domain filter coefficient. The filter coefficient generation unit 49 obtains a filter coefficient to be supplied to the digital filter 42 by performing inverse FFT on the weighting coefficient in the frequency domain.

  Then, the filter coefficient from the filter coefficient generation unit 49 is supplied to the digital filter 42, and the sound signal component of the sound source corresponding to the function set in the weighting coefficient generation unit 48 is separated and extracted from the digital filter 42. , Output SO. The delay unit 41 is for adjusting the processing delay time until the filter coefficient supplied to the digital filter 42 is generated.

  Although the example of FIG. 14 considers only the level ratio, it may be configured to consider only the phase difference or the level ratio and the phase difference together. That is, for example, when considering the level ratio and the phase difference together, although not shown, the outputs of the FFT units 43 and 44 are also supplied to the phase difference detection unit, and the detected phase difference is also weighted. Supply to the coefficient generator. In this example, the weighting coefficient generator has a function generating circuit that generates a weighting coefficient using not only the level difference of the sound signal to be separated from the left and right channel audio signals but also the phase difference as a variable.

  In other words, the weighting coefficient generator in this case has the level ratio of the sound signal of the sound source to be separated in the left and right channels and the level ratio in the vicinity thereof. In the case of the phase difference between the left and right two channels and the phase difference in the vicinity thereof, the function is set so as to generate a large weighting coefficient and otherwise generate a small coefficient.

  Then, the weighting coefficient from the weighting coefficient generation unit is subjected to inverse FFT to be a filter coefficient of the digital filter 42.

  In FIG. 14, the sound signal of the desired sound source is separated from only the left channel. However, for the sound signal of the right channel, a system for generating a filter coefficient is provided in the same manner in a similar manner. An audio signal of a predetermined sound source can be separated.

[Audio signal processing apparatus of other embodiment]
In the above-described embodiment, when FFT is performed on an input audio signal, it is difficult to perform FFT processing on a long time-series signal as it is in a musical sound. Therefore, it is divided into predetermined analysis sections, and division data for each analysis section is obtained. To perform the FFT processing.

  However, when time series data is simply taken out to a certain length, and after performing sound source separation processing and combined by inverse FFT transformation, a waveform discontinuity is generated at that connection point and heard as sound There is a problem of generating noise.

  Therefore, in the eighth embodiment, as shown in FIG. 15, in order to extract the segment data, the lengths of section 1, section 2, section 3, section 4,. However, in the adjacent sections, each section is set so that, for example, a section of ½ of the length of the unit section overlaps, and the segment data of each section is extracted. In FIG. 15, x0, x1, x2, x3,..., Xn indicate sample data of the digital audio signal.

  When processed in this manner, time series data (y0, y1, y2, y3,..., Yn) subjected to sound source separation processing and inverse FFT transformed as in the above-described embodiment is also output as shown in FIG. Like the segment data 1 and 2, there will be overlapping sections.

  In the eighth embodiment, as shown in FIG. 16, the triangular window as shown in FIG. 16 is used for the output section data adjacent to each other with overlapping sections, for example, the overlapping sections of the output section data 1 and 2. The window functions 1 and 2 having the above characteristics are processed, and the same time data in the overlapping sections of the output segment data 1 and 2 are added to obtain output composite data as shown in FIG. As a result, a separated output audio signal having no waveform discontinuity, that is, no noise is obtained.

  Furthermore, in the ninth embodiment, as shown in FIG. 17, the segment data is extracted as a certain segment of adjacent segment data, such as segment 1, segment 2, segment 3, and segment 4, as shown in FIG. At the same time, before the FFT processing is performed on the division data of each section, the window functions of the triangular window functions 1, 2, 3, and 4 as shown in FIG. 17 are performed.

  Then, after the window function process as shown in FIG. 17 is performed, the FFT conversion process is performed. Then, when the signal subjected to appropriate sound source separation processing is subjected to inverse FFT conversion, output segment data 1 and 2 as shown in FIG. 18 are obtained. Since this output segment data has already been subjected to window function processing in the overlapped portion, the output unit can be separated without any discontinuous points in the waveform by simply adding each overlapping segment data portion. It is possible to obtain a sound signal.

  In addition to the triangular window, a Hanning window, a Hamming window, a Blackman window, or the like can be used as the above window function.

  In the above-described embodiment, the time discrete signal is orthogonally transformed to be converted into a frequency domain signal, and the frequency division spectrum between the stereo channels is compared. It may be configured such that the same processing is performed for each frequency band by subdividing by a number of band-pass filters. However, as in the above-described embodiment, the FFT processing is easier to increase the frequency resolution and can improve the separation degree of the sound source to be separated, and thus has great practicality.

  In the above-described embodiment, the two-channel stereo signal has been described as the two audio signals to which the present invention is applied. However, in the present invention, the sound signal of the sound source is distributed with a predetermined level ratio or level difference. Any two audio signals can be applied as long as they are two audio signals. The same applies to the phase difference.

  Further, in the above-described embodiment, the level ratio of the frequency division spectrum for the two audio signals is obtained, and the multiplication coefficient generator uses the function of the level ratio versus the multiplication coefficient. However, for the two audio signals The level difference of the frequency division spectrum may be obtained, and the multiplication coefficient generation unit may use a function of the level difference versus the multiplication coefficient.

  Further, the orthogonal transform means for converting the time series signal into the frequency domain signal is not limited to the FFT processing means, and any means can be used as long as the level and phase of the frequency division spectrum can be compared. It may be a thing.

1 is a block diagram showing a configuration example of a first embodiment for explaining an audio signal processing device according to the present invention; FIG. It is a block diagram which shows the structural example of the frequency division spectrum comparison process part which is a part of FIG. It is a block diagram which shows the structural example of the frequency division spectrum control process part which is a part of FIG. It is a figure which shows some examples of the function set to the multiplication coefficient generation part 31 of a frequency division spectrum control process part. It is a block diagram which shows the structural example of 2nd Embodiment for demonstrating the audio | voice signal processing apparatus by this invention. It is a block diagram which shows the structural example of the one part frequency division spectrum comparison process part of FIG. 5, and a frequency division spectrum control process part. It is a block diagram which shows the structural example of 3rd Embodiment for demonstrating the audio | voice signal processing apparatus by this invention. It is a figure which shows some examples of the function set to the multiplication coefficient generation | occurrence | production parts 31L and 31R in the example of FIG. It is a block diagram which shows the structural example of 4th Embodiment for demonstrating the audio | voice signal processing apparatus by this invention. It is a block diagram which shows the structural example of 5th Embodiment which is embodiment of the audio | voice signal processing apparatus by this invention. It is a block diagram which shows the structural example of 6th Embodiment of the audio | voice signal processing apparatus by this invention. It is a block diagram which shows the structural example of the one part frequency division spectrum comparison process part of FIG. 11, and a frequency division spectrum control process part. It is a figure which shows some examples of the function set to the multiplication coefficient generation part 301,302 of FIG. It is a block diagram which shows the structural example of 7th Embodiment of the audio | voice signal processing apparatus by this invention. It is a figure for demonstrating the structural example of 8th Embodiment of the audio | voice signal processing apparatus by this invention. It is a figure for demonstrating the structural example of 8th Embodiment of the audio | voice signal processing apparatus by this invention. It is a figure for demonstrating the structural example of 9th Embodiment of the audio | voice signal processing apparatus by this invention. It is a figure for demonstrating the structural example of 9th Embodiment of the audio | voice signal processing apparatus by this invention. It is a figure for demonstrating the sound image localization by the signal of 2 channels consisting of a several sound source. It is a figure for demonstrating the sound image localization by the signal of 2 channels consisting of a several sound source. It is a block diagram for demonstrating the conventional separation apparatus of the audio | voice signal of a specific sound source.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Audio | voice signal processing apparatus, 11, 12 ... FFT part, 13 ... Frequency division spectrum comparison process part, 14 ... Frequency division spectrum control processing part, 15 ... Inverse FFT part, 21,22 ... Level detection part, 23, 24 ... Level ratio calculation unit, 25 ... selector, 31 ... multiplication coefficient generation unit, 32 ... sound source separation unit, 33, 34 ... multiplication unit, 35 ... addition unit, 16L, 16R ... frequency division spectrum maximum level detection unit, 1032 ... phase comparison Processing part

Claims (11)

First and second conversion means for converting two systems of time-series audio signals into frequency domain signals,
Using the frequency domain signal from said first converting means and the second conversion means, and the level calculating means for calculating a level ratio or level difference between corresponding frequency division spectrums,
A plurality of multiplication coefficient generators configured to generate multiplication coefficients according to the input of the level ratio or the level difference from the level calculation unit, in which multiplication coefficients corresponding to the sound sources to be separated from each other are preset .
Depending on the sound source of the selection operation to be separated by the user, and separating the sound source selection signal generating means for generating a control signal,
Based on the control signal from the separated sound source selection signal generating means, it is used for output of the first and second conversion means from among the plurality of multiplication coefficient generating sections, and depends on the sound source to be separated. switch means for selecting two multiplier coefficient generating unit that occur the multiplication coefficient,
Using the multiplication factor from the two multiplier coefficient generating unit selected by said switch means, said frequency-domain signal obtained from each of said first converting means and the second conversion means, wherein Output control means for controlling and outputting the level of the frequency division spectrum; and
And inverse conversion means for converting the frequency domain signal from the output control unit, the time series signal,
An audio signal processing apparatus comprising: The switch means includes
Based on the control signal from the separated sound source selection signal generating means, two multiplication coefficient generations for generating the multiplication coefficients corresponding to the different sound sources to be separated from the plurality of multiplication coefficient generation units, respectively. Select
The output control means includes
Using the multiplication coefficients from the two multiplication coefficient generators selected by the switch means, the frequency domain signal obtained from each of the first conversion means and the second conversion means, The level of the frequency division spectrum is controlled and output as two frequency domain signals each having the frequency division spectrum corresponding to the sound source different from the separation target.
The audio signal processing apparatus according to claim 1. The switch means includes
Based on the control signal from the separated sound source selection signal generating means, two multiplication coefficients for generating the multiplication coefficient corresponding to the same sound source to be separated from the plurality of multiplication coefficient generation units, respectively. Select the generator,
The output control means includes
Using the multiplication coefficients from the two multiplication coefficient generators selected by the switch means, the frequency domain signal obtained from each of the first conversion means and the second conversion means, The level of the frequency division spectrum is controlled and output as two frequency domain signals each having the frequency division spectrum corresponding to the same sound source to be separated.
The audio signal processing apparatus according to claim 1. The switch means includes
Based on the control signal from the separated sound source selection signal generating means, two multiplication coefficient generations for generating the multiplication coefficients corresponding to the different sound sources to be separated from the plurality of multiplication coefficient generation units, respectively. Select
The output control means includes
Obtained from each of the first conversion means and the second conversion means using the multiplication coefficient from one of the two multiplication coefficient generation sections selected by the switch means. The frequency domain signal to be output is output as two frequency domain signals having the frequency division spectrum corresponding to one of the sound sources different from each other by controlling the level of the frequency division spectrum, Of the two multiplication coefficient generators selected by the switch means, the multiplication coefficient from the other multiplication coefficient generator is used to obtain each of the first conversion means and the second conversion means. The frequency domain signal is controlled by controlling the level of the frequency division spectrum, and corresponds to the other sound source among the different sound sources to be separated. And outputs it as the frequency domain signals of two systems having the frequency spectrum
The audio signal processing apparatus according to claim 1. The output control unit adds the two frequency domain signals having the frequency division spectrum corresponding to the one sound source, outputs an obtained frequency domain signal, and corresponds to the other sound source. Adding means for adding the two frequency domain signals having a frequency division spectrum and outputting the resulting frequency domain signal
With
The inverse conversion means includes
The two types of added frequency domain signals from the adding means are converted into the time series signals, respectively.
The audio signal processing device according to claim 4. The plurality of multiplication coefficient generators are
Each of the multiplication factors as a function of the level ratio or level difference from the level calculation means is preset,
The output control means includes
The multiplication coefficient from said switch said selected by means of two multipliers coefficient number rounds producing unit multiplies the frequency domain signal obtained from each of said first converting means and the second converting means Te, that determine the output level of the frequency division spectrum
The audio signal processing apparatus according to claim 1 . The plurality of multiplication coefficient generators are
As a function of each of the level ratio or level difference from the level calculating means, the multiplication factor which the level ratio or level difference is zero the level of the frequency spectral components other than the frequency spectral is a predetermined range set in advance The
The audio signal processing apparatus according to claim 6 . Detecting a frequency spectral maximum level of the frequency spectral contained in the frequency domain signal from the previous SL output control unit, and outputs the detected frequency spectral as output data detection means
The audio signal processing apparatus according to claim 1 . The two time-series audio signals are divided into predetermined sections to be divided data , and some sections of adjacent divided data are overlapped, and the divided data is sent to the first and second conversion means. A segmentation means to supply;
Wherein from the inverse conversion means, the time-series signals corresponding to each partitioned data processing window function, the audio signal according to claim 1 and an output means for adding and outputting the time-series signal to each other at the same time Processing equipment. The time-series audio signals of the two systems, as well as the divided to partitioned data in a predetermined period, to overlap a portion of the section of the partitioned data adjacent processes window function, said partitioned data the first and Segmentation means for supplying to the second conversion means;
The time-series signal from the inverse conversion means, for adding and outputting the time series signals with each other at the same time
The audio signal processing apparatus according to claim 1 . A conversion step in which the first and second conversion means convert the two systems of time-series audio signals into frequency domain signals, respectively, to obtain two systems of frequency division spectrum;
A level calculation step in which a level calculation means calculates a level ratio or a level difference between corresponding frequency division spectra of the two frequency division spectra using the frequency domain signal obtained in the conversion step;
Multiplication coefficient multiplication coefficient corresponding to the sound source to be separated, each plurality of multiplication coefficient generating unit that is set in advance, the multiplication coefficient occurs in response to an input of the level ratio or level difference calculated by the level calculation step Generation process,
Separating sound source selection signal generating means, in response to the selection operation of the sound source to be separated by the user, and separating the sound source selection signal generating step of generating a control signal,
Switching means, based on the control signal from the separating sound source selection signal generating means, from among the plurality of multiplication coefficient generating unit is an output of the first and second converting means, the separation subject a switch step of selecting two multiplier coefficient generating unit that occur the multiplication coefficient corresponding to the sound source,
Output control means, said selected in switch step by using the multiplication factor from the two multiplier coefficient generating unit, the frequency-domain signal obtained by the conversion process, the frequency division spectrum of the two systems an output control step of outputting the level of their respective control to,
An inverse conversion step of converting the frequency domain signal obtained in the output control step into a time-series signal;
An audio signal processing method comprising:

Images