JP2007264431A - Sound source separation system, encoder and decoder - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a sound source separation system with wide application. <P>SOLUTION: The sound source separation system comprises: an encoder in which, while a mixed signal is generated by adding a sound signal to each other with a predetermined sound volume ratio, the sound signal being output from a plurality of sound sources, an envelope signal indicating an envelope of each sound signal in each of a plurality of frequency bands different from each other is generated, and the envelope signal and the mixed signal are output; and a decoder in which the mixed signal and the envelope signal are received from the encoder, and each of frequency band components of the mixed signal is made a carrier signal in the frequency band, and an amplitude of each carrier signal is adjusted according to the signal value of the envelope signal corresponding to a sound source which is designated to be separated and its frequency band, and thereafter, the carrier signal is added to each other and output. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a sound source separation technique for separating the sound of each sound source from an acoustic signal in which sounds output from a plurality of sound sources are mixed.

  When extracting only the voice by automatically removing the noise and reverberation from the audio signal mixed with noise and reverberation as pre-processing for voice recognition and voice authentication, Sound source separation technology can be cited as a core technology for practical use of karaoke, MMO, and automatic minutes creation.

  In general, the sound source separation technique does not require any information about the sound source to be separated (for example, the position of the sound source and the type of the sound source, hereinafter referred to as sound source information), and a method for performing sound source separation with reference to the sound source information. And classified.

As an example of the former, extraction of the fundamental frequency of the target speech by harmonic structure analysis (step 1) → spectrogram extraction by short-time Fourier transform by applying a window function to the target speech (step 2) → integer multiple of the fundamental frequency There is a technique of performing sound source separation by a procedure of emphasizing (or extracting) only harmonic components (step 3) → recombining sounds from emphasized (or extracted) harmonic components (step 4).
On the other hand, in the technologies disclosed in Non-Patent Document 1 and Patent Document 1, sound source separation is performed without performing the processing of step 3 by referring to sound source information and event information indicating temporal changes in the driving state of the sound source. Technology is disclosed.
Masataka Goto, Yoichi Muraoka, “Sound source separation system for percussion instruments”, [online] Information Technology Division, National Institute of Advanced Industrial Science and Technology <URL: http: //www.staff_aist.go.jp/am-goto/ PROJ / sss-j.html JP 2004-258442 A

However, the sound source separation technique using the harmonic structure analysis has a problem in that the application range is limited for a complicated and large-scale process. This problem is caused by the fact that separation is difficult when the sound source to be separated has substantially the same spectrum. On the other hand, the techniques disclosed in Non-Patent Document 1 and Patent Document 1 also require the sound source information and event information, so that there is still a problem that the applicable range is limited due to lack of universality.
The present invention has been made in view of the above problems, and an object thereof is to provide a sound source separation technique with a wide application range.

  In order to solve the above-described problems, the present invention provides an encoder that outputs a mixed signal obtained by mixing acoustic signals output from each of a plurality of different sound sources at a predetermined volume ratio, and the plurality of sound sources. And a decoder that separates and outputs an acoustic signal corresponding to each of the mixed signals and outputs the plurality of different frequencies for each of the acoustic signals output from the plurality of sound sources. Envelope signal generating means for specifying an envelope in each of the bands and generating an envelope signal indicating the envelope; and first output means for outputting the envelope signal, wherein the decoder is output from the encoder Each subband signal in the plurality of frequency bands of the mixed signal is conveyed in the frequency band Corresponding amplitude adjustment according to the signal value in each frequency band of the envelope signal corresponding to the sound source corresponding to each of the plurality of sound sources corresponding to the carrier signal generating means for generating as a signal And a second output unit that generates and outputs the carrier signals that have been subjected to the amplitude adjustment after being applied to the carrier signal.

In a more preferred aspect, the first output means performs modulation to shift the frequency of the envelope signal generated by the envelope signal generation means to a frequency outside the audible band, and then adds and outputs the mixed signal. On the other hand, the decoder includes a first filter whose upper limit of the pass band matches the upper limit of the audible band, a second filter whose upper limit of the stop band matches the upper limit of the audible band, and a signal output from the encoder Separating the mixed signal and the envelope signal from the signal output from the encoder by passing one of them through the first filter and the other through the second filter. It is characterized by having means.
Instead of the modulation process for shifting to a frequency outside the audible band, a shift to a frequency band that is not audibly important may be used.

  In another preferred aspect, the envelope signal generation means includes a first filter group including a plurality of bandpass filters each having a passband that matches any of the plurality of frequency bands, While calculating the effective value of the signal obtained by passing each of the acoustic signals output from a plurality of sound sources through each of the plurality of bandpass filters, the envelope of each acoustic signal in the frequency band is specified The carrier signal generation means has a second filter group having the same configuration as the first filter group, and each band pass that constitutes the second filter group by using the mixed signal output from the encoder. A carrier signal in each frequency band is generated by passing through a filter.

  In another preferred aspect, the envelope signal generating means performs wavelet transform on each of the acoustic signals output from the plurality of sound sources, thereby enclosing the envelope of each of the acoustic signals in each of the plurality of frequency bands. It is characterized by specifying.

  In another preferred aspect, the carrier signal generation means includes a transmitter that generates a signal indicating band noise belonging to each of the plurality of frequency bands or a sine wave of a frequency belonging to the frequency band, For each of a plurality of sound sources, it is determined for each frequency band whether the signal value of the envelope signal corresponding to the sound source is smaller than a predetermined threshold, and the frequency band determined to be smaller than the threshold For the frequency band, the signal generated by the transmitter is used as a carrier signal in the frequency band, and for the frequency band that is equal to or higher than the threshold, the frequency band component of the mixed signal is used as the carrier signal in the frequency band. It is characterized by.

In order to solve the above problem, the present invention provides a mixed signal generating means for generating a mixed signal by adding each of acoustic signals output from a plurality of sound sources at a predetermined volume ratio, and the plurality of sound sources. For each of the acoustic signals output from the, an envelope in each of a plurality of different frequency bands is specified, envelope signal generation means for generating an envelope signal indicating the envelope, and the mixed signal and the envelope signal are output And an output means.
In another aspect of the present invention, the computer apparatus includes a mixed signal generating unit that generates a mixed signal by adding each of the acoustic signals output from the plurality of sound sources at a predetermined volume ratio, and the plurality of the plurality of sound signals. For each acoustic signal output from the sound source, an envelope in each of a plurality of different frequency bands is specified, envelope signal generating means for generating an envelope signal indicating the envelope, and the mixed signal and the envelope signal are output. It is also possible to provide a program characterized by functioning as output means.

In order to solve the above problem, the present invention receives a mixed signal obtained by mixing acoustic signals output from each of a plurality of different sound sources at a predetermined volume ratio, and receives a plurality of different sound signals. Carrier signal generating means for generating a sub-band signal of the mixed signal in each of the frequency bands as a carrier signal in the frequency band; and receiving envelope signals indicating envelopes in the plurality of frequency bands of each of the acoustic signals; After the acoustic signal corresponding to each sound source is subjected to amplitude adjustment according to the signal value in each frequency band of the envelope signal corresponding to the sound source to the carrier signal corresponding to the frequency band, the amplitude adjustment is performed. Output means for adding the generated carrier signals to each other and generating the output. To provide.
In another aspect of the present invention, the computer apparatus receives a mixed signal obtained by mixing acoustic signals output from each of a plurality of different sound sources at a predetermined volume ratio, and has a plurality of different ones. Receiving a carrier signal generating means for generating a sub-band signal of the mixed signal in each of the frequency bands as a carrier signal in the frequency band; and receiving envelope signals indicating envelopes in the plurality of frequency bands of the respective acoustic signals, The amplitude adjustment is performed after the acoustic signal corresponding to each sound source is subjected to amplitude adjustment according to the signal value in each frequency band of the envelope signal corresponding to the sound source to the carrier signal corresponding to the frequency band. To function as an output means to generate and output by adding each applied carrier signal It may provide a program characterized.

  According to the present invention, it is possible to provide a sound source separation technique with a wide application range.

According to the present invention, “human hearing is more influenced by articulation than the waveform of the sound source signal itself.” Therefore, “if only event information and changes in the frequency spectrum are faithfully reproduced, the signal can be effectively restored. It is based on the idea that "is possible."
The best mode for carrying out the present invention will be described below with reference to the drawings.
(A: Configuration)
(A-1: Configuration of the sound source separation system 10)
FIG. 1 is a block diagram illustrating a configuration example of a sound source separation system 10 according to an embodiment of the present invention. As shown in FIG. 1, the sound source separation system 10 includes an encoder 100 connected to a communication network 300 such as the Internet, and a decoder 200 that is also connected to the communication network 300. The encoder 100 and the decoder 200 are configured to be able to communicate via the communication network 300. In the present embodiment, the case where the communication network 300 is the Internet will be described. However, any communication network may be used as long as the communication between the encoder 100 and the decoder 200 can be mediated. .

  In the sound source separation system 10 shown in FIG. 1, the encoder 100 is supplied with acoustic signals from two different sound sources A and B. In the following description, it is assumed that the sound signal A (t) is supplied from the sound source A and the sound signal B (t) is supplied from the sound source B. The encoder 100 in FIG. 1 is configured to be able to generate a mixed signal X (t) by mixing both at a predetermined volume ratio. Therefore, for example, when the acoustic signal A (t) is an accompaniment sound of a certain music and the acoustic signal B (t) is a singing sound of the music, the mixed signal X output from the encoder 100 is used. (T) represents the singing sound accompanied by the music. In addition, the encoder 100 generates a control signal characteristic of the present invention in addition to the mixed signal X (t), and adds the control signal to the mixed signal X (t) to obtain an integrated signal Y (t ) To the decoder 200 via the communication network 300.

On the other hand, the decoder 200 of FIG. 1 receives the integrated signal Y (t) sent from the encoder 100 via the communication network 300, and from this integrated signal Y (t), the mixed signal X (t) and the above control signal. And using the control signal, the acoustic signals corresponding to the two types of sound sources are separated from the mixed signal X (t) and output.
Hereinafter, the configuration of the encoder 100 and the decoder 200 will be mainly described.

(A-2: Configuration of encoder 100)
FIG. 2 is a block diagram illustrating a configuration example of the encoder 100.
As shown in FIG. 2, encoder 100 includes volumes 110A and 110B, envelope signal generation means 120A and 120B, compression / conversion circuits 130A and 130B, adders 140 and 150, and modulation processing circuit 160. It is out.
The acoustic signal A (t) is input to the volume 110A in FIG. 2, and the acoustic signal B (t) is input to the volume 110B. The volumes 110A and 110B are configured such that the acoustic signal A (t) and the volume ratio specified by the user with respect to the acoustic signals A (t) and B (t) are satisfied by operating an operation unit (not shown). The volume of B (t) is adjusted and output. In the present embodiment, when the mixed signals X (t) are generated by mixing the acoustic signals A (t) and B (t) at a volume ratio specified by the user, the volume levels of the two are set by the volumes 110A and 110B. Although the case where it adjusts suitably is demonstrated, it cannot be overemphasized that it is not necessary to provide these volumes 110A and 110B in the aspect which produces | generates mixed signal X (t), without adjusting the volume which concerns.

  As shown in FIG. 2, the acoustic signal A (t) output from the volume 110A is divided into two, one of which is input to the envelope signal generating means 120A and the other is input to the adder 140. Similarly, the acoustic signal B (t) output from the volume 110B is also divided into two, one of which is input to the envelope signal generation means 120B and the other is input to the adder 140. The acoustic signals A (t) and B (t) input to the adder 140 are added by the adder 140, and a mixed signal X (t) that is the sum signal of the two is output from the adder 140. The

  Envelope signal generation means 120A and 120B have the same configuration as shown in FIG. Hereinafter, when it is not necessary to distinguish between the two, they are referred to as “envelope signal generation means 120”. As shown in FIG. 2, the envelope signal generating means 120 has J (J is a natural number of 2 or more) band-pass filters 121-j (j = 1 to J: the same applies hereinafter) each having a different pass band. An effective value detection circuit (denoted as “rms” in FIG. 2) 122-j is connected to the output side of the bandpass filter 121-j. Regarding the frequency band division number J, with reference to the critical bandwidth of human hearing, the bandwidth of each pass band is about one-several octave (for example, 1/5 octave to 1/2 octave). Any value is acceptable.

  As shown in FIG. 2, the acoustic signal input to the envelope signal generation means 120 is divided by the number of band-pass filters 121-j and input to each of the band-pass filters 121-j. As shown in FIG. 2, the subband signal (signal indicating the passband component of the acoustic signal) output from each of the bandpass filters 121-j is input to the effective value detection circuit 122-j. The effective value detection circuit 122-j has the circuit configuration shown in FIG. 3, and calculates and outputs the effective value (envelope) of the input signal. That is, the envelope signal generation unit 120 outputs an envelope signal indicating an envelope in each of the plurality of passbands of the acoustic signal input to the envelope signal generation unit 120. As for the time constant of the effective value detection circuit 122-j shown in FIG. 3, a suitable value may be appropriately determined by experiment, but about several tens of milliseconds along the detection limit of the temporal resolution of human hearing. It is desirable that the value of The reason is that when the time constant of the effective value detection circuit 122-j is determined to be about several tens of milliseconds, the frequency of the envelope signal output from the effective value detection circuit 122-j becomes very low and is almost close to direct current. . As described above, when the envelope signal is substantially close to direct current, the amount of information can be greatly reduced as compared with the original acoustic signal.

In the present embodiment, as shown in FIG. 2, the envelope signal generation means 120A passes the acoustic signal A (t) through each of the plurality of bandpass filters 121-j, thereby subband signal A _j. (T) (j = 1 to J) are obtained, and by inputting these subband signals A _j (t) to the effective value detection circuit 122, envelope signals (a1, a2... For the acoustic signal A (t) are obtained. a _J ) is output. Similarly, an envelope signal (b1, b2,... B _J ) is output from the envelope signal generating means 120B for the acoustic signal B (t). Here, the relationship shown in the following Equation 1 is established between A _j (t) and a _j and between B _j (t) and b _j .

As shown in FIG. 2, the envelope signals (a1, a2,... A _J ) output from the envelope signal generation means 120A are input to the compression / conversion circuit 130A and output from the envelope signal generation means 120B. b2... b _J ) are input to the compression / conversion circuit 130B. The compression / conversion circuits 130 </ b> A and 130 </ b> B perform compression processing according to a predetermined compression algorithm such as a logarithmic compression algorithm on the input signal and output the result to the modulation processing circuit 160. The reason why the compression processing is applied to the envelope signals is to suppress noise in a transmission system (not shown) that transmits the envelope signals via the communication network 300 and to reduce the amount of information of the envelope signals. . However, the compression processing as described above is not necessarily essential, and it is of course possible to avoid such compression processing. In such a case, it goes without saying that the compression / conversion circuits 130A and 130B can be omitted.

  Next, the modulation processing circuit 160 in FIG. 2 will be described in detail with reference to FIG. As shown in FIG. 4, the modulation processing circuit 160 has as many transmitters 161-n (n) as the number obtained by multiplying the number of sound sources to be separated (in this embodiment, 2) by the frequency band division number J. = 1 to 2J), a voltage controlled amplifier (indicated as “VCA” in FIG. 4) 162-n connected to each transmitter 161-n, and each voltage controlled amplifier 162-n And an adder 163 connected thereto.

  The transmitter 161-n generates a sine wave signal having a predetermined frequency fn higher than the audible range and outputs it to the voltage controlled amplifier 162-n. In the present embodiment, the case where the sine wave signal is output by the transmitter 161-n will be described. However, a narrowband noise signal whose frequency fn is the center frequency may be output.

A voltage-controlled amplifier 162-n receives a sine wave signal having a frequency of fn and a control voltage corresponding to a signal value of a predetermined component of the envelope signal described above. The sine wave signal is amplified at an amplification factor corresponding to the voltage and output to the adder 163. More specifically, as shown in FIG. 4, a control voltage corresponding to the value of a _j of the envelope signal is applied to each of the voltage controlled amplifiers 162-j (j = 1 to J). The On the other hand, a control voltage corresponding to the value of b _j of the envelope signal is applied to the voltage control type amplifier 162-J + j (j = 1 to J).

  The adder 163 adds the signals output from the voltage controlled amplifiers 162-n (n = 1 to 2J) to generate a control signal F (t), and the control signal F (t) is added to the adder 163. To 150.

Returning to FIG. 2, the adder 150 adds the mixed signal X (t) supplied from the adder 140 and the control signal F (t) supplied from the modulation processing circuit 160 to obtain an integrated signal Y (t). And the integrated signal Y (t) is transmitted to the decoder 200 via the communication network 300 by a transmission system (not shown).
The above is the configuration of the encoder 100.

(A-3: Configuration of decoder 200)
Next, the configuration of the decoder 200 will be described with reference to FIGS. 5 and 6.
As shown in FIG. 5, the decoder 200 includes separation means 210, carrier signal generation means 220A and 220B, and restoration signal output means 230A and 230B.

  Separation means 210 separates and outputs mixed signal X (t) and each envelope signal from integrated signal Y (t), and has the configuration shown in FIG. As shown in FIG. 6, the separation unit 210 divides the integrated signal Y (t) into two, inputs one into the low-pass filter 211, and inputs the other into the high-pass filter 212.

  Here, in the present embodiment, the upper limit of the pass band of the low-pass filter 211 matches the upper limit frequency of the audible range. For this reason, from the low-pass filter 211, only the audible frequency component of the frequency component of the integrated signal Y (t) is output. As described above, since the audible frequency component included in the integrated signal Y (t) is only the frequency component of the mixed signal X (t), the integrated signal Y (t) is passed through the low-pass filter 211. As a result, only the mixed signal X (t) is output. On the other hand, in the present embodiment, the upper limit of the stop band of the high pass filter 212 matches the upper limit frequency of the audible range. For this reason, the high-pass filter 212 outputs only the frequency components in the frequency band higher than the audible range (that is, the super audible range) among the frequency components of the integrated signal Y (t). For this reason, only the control signal F (t) is output by passing the integrated signal Y (t) through the high-pass filter 212. The control signal F (t) output from the high pass filter 212 is divided into two, one of which is input to the envelope signal demodulator 213A and the other is input to the envelope signal demodulator 213B.

The envelope signal demodulator 213A in FIG. 6 demodulates the envelope signal for the acoustic signal A (t) from the control signal F (t) and supplies it to the restored signal output means 230A. The envelope signal demodulator 213B The envelope signal for the acoustic signal B (t) is demodulated from the control signal F (t) and supplied to the restoration signal output means 230B.
As shown in FIG. 6, each of envelope signal demodulation sections 213A and 213B includes J narrowband bandpass filters, an effective value detection circuit connected to each of these J filters, and a decompression circuit.・ Consists of a conversion circuit. The decompression / conversion circuit is a circuit that performs an inverse operation on the arithmetic processing performed by the compression / conversion circuit described above.

Here, it should be noted that the passband is different between the narrowband filter group of the envelope signal demodulator 213A and the narrowband filter group of the envelope signal demodulator 213B. Specifically, the center frequency of the pass band of the J narrowband filter of the envelope signal demodulator 213A in order from the lower frequency, frequencies f1, f2 ultra audible range, to be a ... f _J, The center frequencies of the passbands of the J narrowband filters of the envelope signal demodulator 213B are the super audible frequencies f _{J + 1} , f _{J + 2} ,. As described above, the control signal F (t) is the frequency f1, f2 ultra audible range, ... the amplitude of the sine wave of f _J, respectively, the envelope signal number a1, a2, ... a _J, in accordance with the value while adjusting the frequency f _{J + 1} super audible range, f J _{+ 2,} ... the amplitude of the sine wave of F2j, respectively, the envelope signal number b1, b2, ... b _J, after adjusting in response to the value of, Since these sine waves are added, the envelope signal demodulator 213A extracts only the envelope signal numbers a1, a2,... _AJ , and the envelope signal demodulator 213B b1, b2, and only ... b _J is extracted.

Returning to FIG. 5, the mixed signal X (t) output from the separating means 210 is divided into two. One is input to the carrier signal generating means 220A, and the other is input to the carrier signal generating means 220B. As shown in FIG. 5, since the carrier signal generating means 220A and 220B have the same configuration, they are referred to as “carrier signal generating means 220” when it is not necessary to distinguish between them. As shown in FIG. 5, the carrier signal generation unit 220 includes a filter group (bandpass filter 221-j (j = 1 to J)) having the same configuration as the filter group included in the envelope signal generation unit 120. Yes. As shown in FIG. 5, the mixed signal X (t) input from the separation unit 210 to the carrier signal generation unit 220 is divided by the number of bandpass filters 221-j within the carrier signal generation unit 220, and the band Input to each of the pass filters 221-j. As a result, the sub-band signal X _j (t) is output from the band-pass filter 221-j. Then, the carrier signal generation unit 220A inputs the subband signal X _j (t) output from the bandpass filter 221-j to the restoration signal output unit 230A as a carrier signal in the pass band of the bandpass filter 221-j. The carrier signal generation means 220B inputs the subband signal X _j (t) output from the bandpass filter 221-j to the restoration signal output means 230B as a carrier signal in the pass band of the bandpass filter 221-j.

As is apparent from FIG. 5, the restoration signal output means 230A and 230B have the same configuration. Therefore, when there is no need to distinguish between the two, the restoration signal output means 230A and 230B are expressed as “restoration signal output means 230”. To do. As shown in FIG. 5, the restoration signal output unit 230 includes J voltage-controlled amplifiers 231-j (j = 1 to J) and an adder 232.
More specifically, the voltage-controlled amplifier 231-j of the restoration signal output unit 230A is connected to the bandpass filter 221-j of the carrier signal generation unit 220A. Further, a control voltage corresponding to the value of the envelope signal a _j output from the separation unit 210 is applied to the voltage control type amplifier 231-j of the restoration signal output unit 230 </ b _> A. For this reason, the amplitude of the carrier signal X _j (t) input to the voltage-controlled amplifier 231-j of the restoration signal output means 230A is amplified according to the value of the envelope signal a _j . As a result, the voltage control type amplifier 231-j of the restoration signal output means 230 A outputs a signal A _j ^′ (t) that conforms to the spectrum in the j th frequency band of the original acoustic signal A (t). become. In this way, the signal A _j ^′ (t) output from the voltage control type amplifier 231-j is added by the adder 232, and the adder 232 receives the restored signal A in accordance with the original acoustic signal A (t). ^' (T) is output.

On the other hand, in the restoration signal output means 230B, as in the restoration signal output means 230A, the carrier signal X _j (t) is input to the voltage control type amplifier 231-j and the envelope output from the separation means 210 is output. A control voltage corresponding to the value of the signal b _j is applied, and the voltage controlled amplifier 231-j outputs a signal B _j ^′ (t) that follows the spectrum in the j-th frequency band of the original acoustic signal B (t). Is output. These B _j ^′ (t) are added by the adder 232 of the restoration signal output means 230B, and a restoration signal B ^′ (t) that conforms to the spectrum of the original acoustic signal B (t) is output.
The above is the configuration of the decoder 200.

As described above, according to the decoder 200, the restored signal A ^′ (t) that follows the original acoustic signal A (t) from the mixed signal X (t) and the original acoustic signal B (t). A restoration signal B ^′ (t) that follows the spectrum is output, but the point to be considered here is, for example, the restoration obtained as described above when performing sound source separation for the sound source A. The acoustic signal B (t) is considered to be superposed as masking noise in each frequency band of the signal A ^′ (t).
However,
(1) When a _j is sufficiently larger than b _{j In} this case, since the acoustic signal A (t) to be separated is dominant, the acoustic signal B (t) does not become masking noise.
(2) When a _j and b _j are substantially equal In this case, the acoustic signal A (t) and the acoustic signal B (t) remain mixed, but both are signals of the same narrowband and of approximately the same spectrum. Therefore, one is unlikely to be a large noise source or masker for the other. Moreover, both of them change with time, and the time probability that the amplitudes are completely equal is generally not large.
(3) When a _j is sufficiently smaller than b _{j In} this case, the effect of the present invention is most noticeable. That is, if sound source separation according to the present invention is not performed, for example, even if the amplitude of the acoustic signal A (t) is substantially zero at a certain time t (that is, when the sound of the sound source A is not sounding), masking is performed. Although the sound of the sound source B is heard as a sound and the separation feeling is greatly hindered, according to the present invention, the amplitude of the restoration signal A ^′ (t) is lowered to a predetermined level (in this case, silence), The original state of “nothing or no sound when not ringing” is secured.

By the way, the restoration signal output from the decoder 200 and the original acoustic signal corresponding to the restoration signal are not necessarily the same. However, the pass bandwidth of the bandpass filter 121-j is appropriately set according to the critical bandwidth of human hearing, and the time constant of the effective value detection circuit 122-j is set in accordance with the detection limit of human hearing. If set appropriately, within the range of human hearing, in the above restoration signal, the temporal change in the spectrum of the original signal and the component for each frequency band is almost reliably reproduced, and it is audibly close to the original signal. It becomes.
In addition, the sound source separation in the sound source separation system 10 according to the present embodiment is sound source separation that does not require sound source information or event information, and does not require observation signals at a plurality of sound receiving points, and can be separated. There is no limit on the number of sound sources.
As described above, according to the present embodiment, by performing pre-processing such as generating envelope signals indicating envelopes in different frequency bands in advance for each of a plurality of acoustic signals to be mixed, it is possible to perform the conventional processing. It is possible to provide a sound source separation technique with a wide application range.

(B: Deformation)
Although one embodiment of the present invention has been described above, it is needless to say that the embodiment may be modified as described below.

(1) In the above-described embodiment, the integrated signal Y () obtained by adding the mixed signal X (t) obtained by mixing the two acoustic signals at a predetermined volume ratio and the control signal F (t). The case where t) is transmitted from the encoder 100 to the decoder 200 via the communication network 300 has been described. However, the integration signal Y (t) is written by causing the encoder to execute the process of writing the integration signal Y (t) on a recording medium such as a CD or a DVD, and causing the decoder to read the integration signal Y (t) written on the recording medium. Of course, t) may be transmitted from the encoder to the decoder.

(2) In the above-described embodiment, the case of mixing and separating two types of acoustic signals has been described. However, it can be said that the present invention can be applied to mixing and separation of three or more types of acoustic signals. Not too long. Specifically, the envelope signal generation means may be provided on the encoder side for the number of acoustic signals. Further, in the above-described embodiment, the case where the encoder is provided with the envelope signal generation unit corresponding to each of the acoustic signals input to the encoder has been described. However, it is of course possible to generate envelope signals corresponding to each acoustic signal by sequentially using one envelope signal generating means.

(3) In the above-described embodiment, the case where the mixed signal X (t) is divided into subbands by a plurality of bandpass filters each having a different passband, and each subband signal is used as a carrier signal in each passband has been described. . However, as shown in FIG. 7, the carrier signal generating means 220 is provided with a changeover switch SW _j on the path from the bandpass filter 221-j to the voltage controlled amplifier 231-j, and the bandpass filter 221-j A transmitter G _j (j = 1 to J) for generating a signal indicating a band noise (for example, pink noise) belonging to the pass band or a sine wave having a frequency belonging to the frequency band is provided, and the changeover switch SW _j is switched (ie, , Switching between whether the band-pass filter 221-j is connected to the voltage-controlled amplifier 231-j or whether the transmitter G _j is connected to the frequency band (that is, the pass band of the band-pass filter 221-j) It may be performed according to the signal value of the envelope signal corresponding to.
Specifically, it is determined for each frequency band whether the signal value is smaller than a predetermined threshold, and for the frequency band determined to be smaller than the threshold, the connection is switched to the transmitter G _j , For the frequency band that is equal to or greater than the threshold, the connection may be switched to the bandpass filter 221-j. However, when band noise or a sine wave is used as the carrier signal, it is desirable that the passband width of the filter used when generating the envelope signal be sufficiently narrow. When a sine wave is used as the carrier signal, the sine wave may be appropriately subjected to frequency modulation in order to make the carrier signals in each frequency band uncorrelated with each other.

(4) In the above-described embodiment, the case where the integrated signal Y (t) obtained by adding the control signal F (t) and the mixed signal X (t) is transmitted from the encoder 100 to the decoder 200 has been described. However, the control signal F (t) and the mixed signal X (t) may be individually transmitted from the encoder to the decoder, and if the decoder side already has the mixed signal X (t), Only the control signal F (t) may be transmitted from the encoder to the decoder.
In addition, when the aspect in which only the control signal (or envelope signal) described in this modification is transmitted and the aspect in which band noise or sine wave is used as the carrier signal described in the modification (3) are combined, an acoustic signal It is expected that it is possible to significantly reduce the amount of information transmitted during transmission of noise and to effectively eliminate noise. Specifically, on the transmission side of the acoustic signal, envelopes in a plurality of different frequency bands are specified for the acoustic signal, an envelope signal indicating the envelope is transmitted to the reception side of the acoustic signal, and the envelope signal is transmitted. On the receiving side, the band noise or sine wave described in the modification (3) is used as the carrier signal in each frequency band, and the amplitude of each carrier signal is an envelope signal corresponding to the frequency band of the carrier signal. After the adjustment according to the value, the restored signals corresponding to the acoustic signals may be generated by adding the carrier signals.

(5) In the above-described embodiment, the case where the envelope signal generation unit is configured by a plurality of bandpass filters and an effective value calculation circuit connected to each of the bandpass filters has been described. However, it goes without saying that the envelope signal generation means may be configured by a circuit that performs wavelet transform on the input signal. If it does in this way, according to the characteristic of each acoustic signal which is the object of mixing and separation, there will be an effect that it becomes possible to rationally set the frequency bandwidth and time resolution when extracting the envelope signal.

(6) In the above-described embodiment, the signal format of the acoustic signal mixed and separated in the sound source separation system 10 is not particularly mentioned, but a musical sound encoding method such as MPEG1 / LayerIII (hereinafter “MP3”) It is also possible to perform sound source separation in combination. Here, MP3 is a method of musical sound encoding, and is realized by a procedure such as filter bank analysis of sound signal → orthogonal transformation (MDCT: Modified Discrete Cosine Transform is often applied in MP3). (For example, refer to the configuration example of the encoder and decoder shown in FIG. 8). When the sound source separation according to the present invention is applied to an acoustic signal encoded according to MP3, the control signal described above is embedded in the DCT coefficient (MDCT coefficient) obtained by the orthogonal transformation. good. Note that most of the musical tone coding schemes can dynamically allocate the bits for the nonlinear quantization process of FIG. 3, so that the minimum necessary bits for decoding the control signal are allocated to the encoder. If the control signal is embedded in the bits allocated in this way, the control signal can be encoded more efficiently. Note that what band should be embedded in the control signal and how many bits should be assigned to the control signal may be appropriately determined by experiment.
However, when the decoder side does not support the embedding of the control signal (for example, in the case of a decoder having the configuration shown in FIG. 8), the reproduced sound may be distorted according to the embedded control signal. is there. However, this type of distortion is similar to that which occurs when the bit rate is low and is relatively familiar to users of MP3 terminals. For this reason, even if the reproduction sound is distorted as described above, the user does not feel extreme discomfort.

(7) In the above-described embodiment, the case where the encoder 100 and the decoder 200 according to the present invention are configured by combining hardware modules each having a unique function has been described. However, by installing a program for causing a CPU (Central Processing Unit) to realize the same function as each hardware module constituting the encoder 100 into a general computer device, and operating the CPU of the computer device according to the program Of course, the same function as that of the encoder according to the present invention may be given to the computer apparatus. Similarly, a program for causing a CPU to realize the same function as each hardware module constituting the decoder 200 is installed in a general computer apparatus, and the CPU of the computer apparatus is operated in accordance with the program, whereby the computer apparatus Of course, the same function as that of the decoder according to the present invention may be added to the above. Further, when distributing each program, the program may be written on a CD-ROM (Compact Disk-Read Only Memory), or may be distributed by downloading via a telecommunication line such as the Internet. You may make it do.

1 is a diagram illustrating a configuration example of a sound source separation system 10 according to an embodiment of the present invention. 2 is a diagram illustrating a configuration example of an encoder 100. FIG. 3 is a diagram illustrating a configuration example of an effective value detection circuit 122. FIG. 3 is a diagram illustrating a configuration example of a modulation processing circuit 160. FIG. 3 is a diagram illustrating a configuration example of a decoder 200. FIG. 3 is a diagram illustrating a configuration example of a separation unit 210. FIG. It is a figure which shows the structural example of the decoder which concerns on a modification (3). It is a figure which shows the structural example of the encoder and decoder in MP3 regarding a modification (6).

Explanation of symbols

  DESCRIPTION OF SYMBOLS 100 ... Encoder, 200 ... Decoder, 300 ... Communication network.

Claims (7)

An encoder for outputting a mixed signal obtained by mixing acoustic signals output from each of a plurality of different sound sources at a predetermined volume ratio, and an acoustic signal corresponding to each of the plurality of sound sources from the mixed signal A sound source separation system including a decoder for separating and outputting,
The encoder is
Envelope signal generating means for identifying envelopes in each of a plurality of different frequency bands for each of the acoustic signals output from the plurality of sound sources, and generating an envelope signal indicating the envelope;
First output means for outputting the envelope signal,
The decoder
Carrier signal generating means for generating each subband signal in the plurality of frequency bands of the mixed signal output from the encoder as a carrier signal in the frequency band;
The amplitude of the acoustic signal corresponding to each of the plurality of sound sources after the amplitude adjustment according to the signal value in each frequency band of the envelope signal corresponding to the sound source is performed on the carrier signal corresponding to the frequency band And a second output unit that generates and outputs the adjusted carrier signals, and outputs the sound source separation system. The first output means includes
After performing modulation that shifts the frequency of the envelope signal generated by the envelope signal generating means to a frequency outside the audible band, and then adding and outputting the mixed signal,
The decoder
A first filter whose upper limit of the passband matches the upper limit of the audible band;
A second filter whose upper limit of the stopband matches the upper limit of the audible band;
The mixed signal and the envelope signal are output from the encoder by dividing the signal output from the encoder into two, one of which passes through the first filter and the other through the second filter. The sound source separation system according to claim 1, further comprising: a separation unit that separates the signal from the generated signal. The envelope signal generating means includes
A first filter group comprising a plurality of band-pass filters each having a pass band that matches any of the plurality of frequency bands;
By calculating the effective value of the signal obtained by passing each of the acoustic signals output from the plurality of sound sources through each of the plurality of bandpass filters, the envelope in each frequency band of the acoustic signal is specified. on the other hand,
The carrier signal generation means includes
A second filter group having the same configuration as the first filter group;
2. The sound source according to claim 1, wherein a carrier signal of each frequency band is generated by allowing the mixed signal output from the encoder to pass through each bandpass filter constituting the second filter group. Separation system. The envelope signal generating means includes
The sound source separation according to claim 1, wherein an envelope of each acoustic signal in each of the plurality of frequency bands is specified by performing wavelet transform on each of the acoustic signals output from the plurality of sound sources. system. The carrier signal generation means includes
A transmitter for generating a signal indicating band noise belonging to each of the plurality of frequency bands or a sine wave of a frequency belonging to the frequency band;
For each of the plurality of sound sources, it is determined for each frequency band whether the signal value of the envelope signal corresponding to the sound source is smaller than a predetermined threshold, and the frequency determined to be smaller than the threshold For a band, the signal generated by the transmitter is a carrier signal in that frequency band, while for a frequency band that is greater than or equal to the threshold, the frequency band component of the mixed signal is the carrier signal in that frequency band. The sound source separation system according to claim 1. Mixed signal generating means for generating a mixed signal by adding each of the acoustic signals output from a plurality of sound sources at a predetermined volume ratio;
Envelope signal generating means for identifying envelopes in each of a plurality of different frequency bands for each of the acoustic signals output from the plurality of sound sources, and generating an envelope signal indicating the envelope;
An encoder comprising: output means for outputting the mixed signal and the envelope signal. A mixed signal obtained by mixing acoustic signals output from each of a plurality of different sound sources at a predetermined volume ratio is received, and a subband signal of the mixed signal in each of a plurality of different frequency bands is received. Carrier signal generating means for generating a carrier signal in a frequency band;
An envelope signal indicating an envelope of each of the sound signals in the plurality of frequency bands is received, and an acoustic signal corresponding to each of the plurality of sound sources is determined according to a signal value in each frequency band of the envelope signal corresponding to the sound source. Output means for adding and generating each of the carrier signals subjected to the amplitude adjustment after performing the amplitude adjustment on the carrier signal corresponding to the frequency band, and outputting the added signal.

Images