JP2015070291A - Sound collection/emission device, sound source separation unit and sound source separation program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a sound collection/emission device which excellently extracts a target sound from an intended sound source regardless to the sound kinds of an emitted sound while the emitted sound exists.SOLUTION: A sound collection/emission device emits a sound from one or a plurality of speakers and collects ambient sound from two microphones. The sound collection/emission device comprises: sound emission non-target sound removal means which removes a sound emission non-target sound emitted from the speaker and collected from respective microphones, and is diverted from an acoustic echo canceller structure; and sound source separation means which extracts a target sound from the output thereof. The sound source separation means generates a parameter for the sound source separation from the input sound signal from which the sound emission non-target sound is removed, and corrects the parameter on the basis of a pseudo sound emission non-target sound signal generated in the sound emission non-target sound removal means to use the parameter for the sound source separation.

Description

  The present invention relates to a sound collecting / sound emitting device, a sound source separation unit, and a sound source separation program. For example, only sound coming from a sound source in a predetermined direction (hereinafter referred to as a target sound) from captured sound by a microphone, captured sound, etc. The present invention can be applied to communication terminals, audio devices, etc. that want to be separated.

  For example, when inputting call voice to a smartphone or inputting voice commands to an audio device or smartphone, the device to which the sound is input is only the sound from the front where the user's mouth seems to exist. Is preferably distinguished from voice, music, noise, etc. from other directions.

  A system that captures sound input to two microphones, suppresses ambient noise based on the phase difference between the input sounds (electrical signals), and extracts a target sound that arrives from a predetermined direction (for example, the front) of the microphone ( (Sound source separation method) is described in Patent Document 1.

  The target sound extraction method described in Patent Document 1 as the third embodiment uses a suppression coefficient corresponding to the correlation between two signals obtained by forming two directivities having blind spots on the left and right sides of a microphone. This is a technique for suppressing noise components (non-target sounds) coming from the left and right by multiplying an input sound signal for each component. The target sound extraction method described as the fourth embodiment in Patent Document 1 forms a directivity having a blind spot in front of a microphone, and inputs a signal obtained as a noise component coming from the left and right. This is a technique for suppressing noise components (non-target sounds) coming from the left and right by subtracting from the sound signal.

JP 2013-061421 A

Kitawaki Nobuhiko, “Digital Voice / Audio Technology (Future Netto Technology Series)”, published by Telecommunications Association, p218-p243, 1999

  Incidentally, in recent years, as shown in FIG. 8, a pair of speakers 3L and 3R are arranged and connected on both sides of a sound collecting device 2 having a communication function such as a portable terminal (for example, a smartphone or a tablet terminal). The sound collecting / sound emitting device 1 for making a call with a remote place with such a configuration has come to be used. Also, with the same configuration, sound (music) from music files recorded in the sound collecting device 2 or music files acquired from music distribution sites on the Internet is emitted from the speakers 3L and 3R on both sides. In this state, a method in which the user receives a command by a voice emitted from the front of the microphone of the sound collecting device 2 is also being studied.

  In the state where music is emitted from the speakers 3L and 3R on both sides, the target sound coming from the front is extracted and the utterance content is communicated to the other party, or the voice command is recognized through voice recognition processing. When the processing corresponding to the voice command is executed, the sound emitted from the speakers 3L and 3R becomes noise, which greatly reduces the call sound quality and the voice recognition rate.

  Therefore, it is necessary to apply a sound source separation method such as the technology described in Patent Document 1 described above, suppress noise components coming from the speakers 3L and 3R on both sides, and extract the target sound from the front. In the case of applying the sound source separation method described in Patent Document 1, it is necessary to mount or externally attach two microphones 4L and 4R to the sound collecting device 1, as shown in FIG.

  However, when the user enjoys the music from the sound collecting / sound emitting device 1, the volume is large and the loud music is captured by the microphones 4 </ b> L and 4 </ b> R as noise components (non-target sounds). Even if the target sound is extracted by applying the sound source separation method, many noise components remain in the extracted target sound signal.

  In order to avoid this, after the user stops outputting the music (sound emission), the user may pronounce the input voice such as a call voice or voice command. However, if the key operation for stopping the output is performed as described above, the merit of the voice command is reduced, and it is easier to input the command by the key operation. Further, in the case of a call from an incoming call, the voice output stop operation cannot be performed, or the incoming call is delayed due to the execution of the output stop operation.

  Therefore, there is a demand for a sound collecting / sound emitting device, a sound source separation unit, and a sound source separation program capable of extracting a target sound from an intended sound source with a good S / N ratio even in a situation where there is a sound emission. .

  According to a first aspect of the present invention, there is provided a sound collection / sound emission device having a sound collection unit in which at least two microphones capture ambient sounds and a sound emission unit that emits sound from one or more speakers. Sound source separation means for extracting a target sound from a sound source in a predetermined direction based on an input sound signal in which two microphones capture ambient sound; and (2) a sound signal emitted by the sound emitting unit is input; By generating a pseudo-sounding non-target sound signal that simulates a non-target sound that is emitted from the speaker and captured by each microphone, and subtracting it from the input sound signal from each microphone, The sound source non-target sound removing means provided up to the path to the sound source separation means for removing the sound non-target sound captured by each microphone is provided. (1) The sound source separation means is (1- 1) The above non-target sound removal hand A first separation parameter generating unit that generates a first parameter for sound source separation from the input sound signal from which the sound non-target sound output from is removed, and (1-2) the sound non-purpose sound output A second separation parameter generation unit that generates a second parameter for sound source separation based on the pseudo sound emission non-target sound signal generated in the sound removal means; (1-3) the first A parameter correcting unit that corrects the parameter using the second parameter to obtain a final parameter used for sound source separation; and (1-4) a sound source separating unit that performs sound source separation using the corrected parameter. And (3) removing the non-target sound by the sound non-target sound removing means and extracting the target sound by removing the other non-purpose sound by the sound source separating means. To do.

  The second aspect of the present invention is a sound source separation applied to a sound collection / sound emission device having a sound collection unit in which at least two microphones capture ambient sounds and a sound emission unit that emits sound from one or more speakers. (1) sound source separation means for extracting a target sound from a sound source in a predetermined direction based on an input sound signal obtained by capturing the ambient sound by the two microphones; and (2) the sound emitting unit. A sound signal to be emitted is input, emitted from the speaker, and generates a pseudo-non-target sound signal that simulates a non-target sound accompanying the sound output captured by each microphone, and is input from each microphone. A sound non-target sound removing means provided up to a path to the sound source separating means for removing the sound non-purpose sound captured by each of the microphones by subtracting from the sound signal; (1) The sound source separation means is (1 1) a first separation parameter generating unit that generates a first parameter for sound source separation from an input sound signal from which the sound non-target sound output from the sound non-target sound removing means is removed; (1-2) a second separation parameter generating unit that generates a second parameter for sound source separation based on the pseudo-non-target sound signal generated in the sound non-target sound removing means; (1-3) a parameter correction unit that corrects the first parameter using the second parameter to obtain a final parameter to be used for sound source separation; and (1-4) the corrected parameter. And (3) removing the non-target sound by the sound non-target sound removing means and removing the other non-target sound by the sound source separating means. And extracting a target sound.

  According to a third aspect of the present invention, there is provided a computer mounted on a sound collection / sound emission device having a sound collection unit in which at least two microphones capture ambient sounds and a sound emission unit that emits sound from one or more speakers. A sound source separation program to be executed, the computer comprising: (1) sound source separation means for extracting a target sound from a sound source in a predetermined direction based on an input sound signal obtained by capturing the ambient sound by the two microphones; (2) A sound signal emitted by the sound emitting unit is input, sound is emitted from the speaker, and a pseudo sound emitting non-purpose sound signal simulating a non-purpose sound accompanying sound emission captured by each microphone is generated. Then, by subtracting from the input sound signal from each of the microphones, the emitted non-target sound captured by each of the microphones is removed, and the emitted non-target sound removal provided up to the path to the sound source separation means (1) The sound source separation means to be functioned is (1-1) sound source separation from the input sound signal from which the sound non-target sound output from the sound non-purpose sound removal means is removed. A first separation parameter generation unit for generating a first parameter for the sound source, and (1-2) a sound source based on the pseudo sound emission non-purpose sound signal generated in the sound emission non-purpose sound removal means A second parameter generation unit for separation that generates a second parameter for separation; and (1-3) a final that is used for sound source separation after the first parameter is modified using the second parameter. And (1-4) a sound source separation unit that performs sound source separation by applying the modified parameters, and (3) the non-target sound is a non-target sound. Remove with the removal means and other non-target sounds Removing a source separating means and extracting a target sound.

  According to the present invention, a sound collection / sound emission device and a sound source separation unit that can extract a target sound from an intended sound source with a good SN ratio regardless of the sound type of the sound emission even in a situation where the sound emission is present And a sound source separation program.

It is a block diagram which shows the structure of the sound collection and sound emission apparatus of 1st Embodiment. It is a block diagram which shows the detailed structure of the sound source separation process part in the sound collection / sound emission apparatus of 1st Embodiment. It is a block diagram which shows the detailed structure of the suppression coefficient calculation part in the sound source separation process part in the sound collection / sound emitting apparatus of 1st Embodiment. It is a block diagram which shows the detailed structure of the sound source separation process part in the sound collection / sound emission apparatus of 2nd Embodiment. It is a block diagram which shows the detailed structure of the sound emission non-purpose sound type determination part in the sound source separation process part in the sound collection and sound emission apparatus of 2nd Embodiment. It is explanatory drawing which shows the relationship between the kind of sound source data, and the behavior of coherence in connection with 2nd Embodiment. It is a block diagram which shows the other detailed structural example applicable as a sound emission non-target sound canceller process part of each embodiment. It is explanatory drawing which shows the mode of the connection of the speaker in the conventional sound collection and sound emission apparatus. It is explanatory drawing which shows the mode of mounting of the microphone in the case of applying a sound source separation system to the conventional sound collecting / sound emitting device.

(A) First Embodiment Hereinafter, a first embodiment of a sound collecting / sound emitting device, a sound source separation unit, and a sound source separation program according to the present invention will be described with reference to the drawings.

(A-1) Configuration of the First Embodiment The sound collection / sound emission device of the first embodiment is equipped with a pair of microphones or externally attached, and a pair of speakers. Or it is an external one. For example, in the case of a sound collecting / sound emitting device using a sound collecting device such as a smartphone or a tablet terminal, a pair of microphones is mounted and a pair of speakers are externally configured. Further, for example, if a speaker integrated audio device is a corresponding sound collecting / sound emitting device, a pair of microphones and a pair of speakers are mounted. As described above, the connection forms of the pair of microphones and the pair of speakers are various, but any connection form may be applied.

  In the following, the sound collection / sound emission device of the first embodiment will be described on the assumption that a pair of microphones are mounted and a pair of speakers are externally attached as shown in FIG. 9 described above. In addition, for the components described in FIG. 9, the symbols used in FIG. 9 are used as they are for the components in the sound collection / sound emission device of the first embodiment.

  FIG. 1 is a block diagram illustrating a configuration of a sound collection / sound emission device 10 according to the first embodiment.

  The sound collection / sound emission device 10 of the first embodiment may be constructed by connecting various hardware components, and some components (for example, a speaker, a microphone, The functions of the analog / digital conversion unit (A / D conversion unit) and digital / analog conversion unit (except for the D / A conversion unit) are realized by applying program execution configurations such as CPU, ROM, and RAM. It may be constructed to do so. Regardless of which construction method is applied, the functional detailed configuration of the sound collection / sound emission device 10 is the configuration shown in FIG. When applying the program, the program may be written in the memory of the sound collecting / sound emitting device 10 from the time of shipment of the device, or may be installed by downloading. good. For example, in the latter case, a program is prepared as an application for a smartphone, and a user who needs it can download and install it via the Internet.

  In FIG. 1, the sound collection / sound emission device 10 of the first embodiment includes a sound emission unit 20 and a sound collection unit 30.

  The sound emitting unit 20 has the same configuration as the existing sound emitting unit. The sound emitting unit 20 includes sound source data storage units 21L and 21R for L channel and R channel, D / A conversion units 22L and 22R, and speakers 3L and 3R.

  On the other hand, the sound collection unit 30 includes L-channel and R-channel microphones 4L and 4R, A / D conversion units 31L and 31R, a sound emission non-target sound canceller processing unit 32, and a sound source whose detailed configuration is shown in FIG. And a separation processing unit 33. Here, the entire sound collection unit 30 having an input terminal for sound source data, which will be described later, may be constructed as a sound source separation unit and provided on the market. Further, the part composed of the A / D conversion units 31L and 31R, the sound emission non-target sound canceller processing unit 32, and the sound source separation processing unit 33 has a sound source data input terminal, which will be described later, and is constructed as a sound source separation unit. You may use for a commercially available thing. That is, in the sound collection / sound emission device 10, in particular, the sound collection unit 30 may be constructed using a sound source separation unit.

  The sound source data storage units 21L and 21R store the sound source data (digital signals) sigL and sigR for the L channel and the R channel, respectively, and read and output the sound source data sigL and sigR under the control of a sound emission control unit (not shown). Is. The sound source data sigL and sigR may be, for example, music data or electronic data such as an electronic book for reading out. Each of the sound source data storage units 21L and 21R may be a recording medium access device loaded with a recording medium such as a CD-ROM, and stores sound source data acquired by communication from an external device such as a site on the Internet. It may be configured by a storage unit of the apparatus. The sound source data storage units 21L and 21R may correspond to, for example, external devices connected by USB connector connection. Furthermore, each sound source data storage unit 21L, 21R is named “storage unit”, but the concept of each sound source data storage unit 21L, 21R includes received sound source data such as a digital audio broadcast receiver. A configuration for outputting in real time is also included.

  The D / A converters 22L and 22R convert the sound source data sigL and sigR output from the corresponding sound source data storage units 21L and 21R into analog signals and give them to the corresponding speakers 3L and 3R.

  The speakers 3L and 3R output sound sources (sound generation output) from the sound source signals supplied from the corresponding D / A converters 22L and 22R, respectively. Here, the sound or sound output from the speakers 3L and 3R is not intended to be captured by the microphones 4R and 4L, but becomes non-target sound when viewed from the capturing function of the microphones 4R and 4L. It has become.

  In the above, the music and sound emitted from the speakers 3L and 3R are shown as digital signals (sound source data). However, the configuration corresponding to the sound source data storage units 21L and 21R is a record player. An audio cassette tape recorder, an AM or FM radio receiver, and the like may output an audio signal or an audio signal that is an analog signal. In this case, the D / A converters 22L and 22R are omitted, and an A / D converter for the L channel and the R channel is provided separately to convert an analog acoustic signal or audio signal into a digital signal. The sound is output to the non-target sound canceller processing unit 32.

  Each of the microphones 4R and 4L captures ambient sound and converts it into an electrical signal (analog signal). A stereo signal is obtained by the pair of microphones 4R and 4L. Each of the microphones 4R and 4L has directivity that mainly captures sound coming from the front of the sound collection / sound emission device 10, but emits sound from the speakers 3L and 3R arranged on both sides. It also captures the generated sound. The speakers 3L and 3R are preferably arranged on both sides of the pair of microphones 4R and 4L, but are not limited to this arrangement.

  The microphones 4R and 4L are attached to, for example, a cylinder provided in the housing of the sound collecting / sound emitting device 10. Here, a sound insulating member made of a synthetic resin is provided on the inner surface of the cylindrical body so that when the microphones 4R and 4L are attached, there is no path through which the sound passes inside and outside the housing. This prevents as much as possible the microphones 4R and 4L from capturing the noise generated inside the housing or the noise entering the housing from the outside and going out of the housing by reflection. Can do.

  The A / D conversion units 31L and 31R convert the input sound signals captured by the corresponding microphones 4R and 4L into digital signals inputL and inputR, respectively, and give them to the sound emission non-target sound canceller processing unit 32. Each A / D conversion unit 31L, 31R converts, for example, a digital signal having the same sampling rate as the sampling rate of the sound source data sigL, sigR.

  The sound emission non-target sound canceller processing unit 32 is also supplied with sound source data sigL and sigR output from the sound source data storage units 21L and 21R. Here, it is necessary that the sampling rates of the four digital signals input to the sound emission non-target sound canceller processing unit 32 are the same. For example, the sampling rates of the sound source data sigL and sigR downloaded from the Internet site and stored in the sound source data storage units 21L and 21R are different from the sampling rates of the digital signals inputL and inputR from the A / D conversion units 31L and 31R. In this case, the downloaded sound source data sigL and sigR are directly supplied to the D / A conversion units 22L and 22R, and the sound source data obtained by converting the sampling rate of the sound source data sigL and sigR is supplied to the sound emission non-target sound canceller processing unit 32. You should give it.

  The sound emission non-target sound canceller processing unit 32 is based on the sound source data sigL and sigR output from the sound source data storage units 21L and 21R, and includes the speakers 3L and 3R included in the input sound signals (digital signals) inputL and inputR. Removes (or reduces) the non-target sound component (hereinafter referred to as the sound non-target sound as appropriate) that is emitted from the sound, and provides the sound source separation processing unit 33 with the input sound signals ECoutL and ECoutR after the removal processing. Is.

  Here, the sound that is emitted from the speakers 3L and 3R and captured by the microphones 4R and 4L and is unnecessary from the target sound (non-target sound) is the same as the acoustic echo that is a problem in telephone communication. Can be considered. Therefore, in the first embodiment, the sound emission non-target sound canceller processing unit 32 is configured using the acoustic echo canceller technique. For example, Non-Patent Document 1 describes “stereo echo canceller”. In the first embodiment, it is assumed that the sound output non-target sound canceller processing unit 32 described in FIG. 3.71 or FIG. 3.75 of Non-Patent Document 1 is applied.

  In the sound emission non-target sound canceller processing unit 32 having a stereo echo canceller configuration, in order to remove the sound emission target sound from the input sound signals inputL and inputR, a pseudo sound emission target sound signal (hereinafter referred to as “pseudo sound emission target sound signal”) is internally generated. PSechoL and PSechoR (referred to as sound target sound signals) are generated. In the case of the first embodiment, the pseudo sounding target sound signals PSechoL and PSechoR are also supplied to the sound source separation processing unit 33.

  The sound source separation processing unit 33 has the detailed configuration shown in FIG. 2, and based on the input sound signals ECoutL and ECoutR from which the sound emission non-target sound has been removed and the pseudo sound emission target sound signals PSechoL and PSechoR (for example, Only the target sound from the sound source located in the front) is extracted. The sound source separation method applied by the sound source separation processing unit 33 is a coherence filter method to which a coherence coefficient whose characteristics change depending on the direction of the sound source is applied.

  2, the sound source separation processing unit 33 includes an FFT (Fast Fourier Transform) unit 41, a first coherence coefficient calculation unit 42, a second coherence coefficient calculation unit 43, a suppression coefficient calculation unit 44, a suppression coefficient multiplication unit 45, and An IFFT (Inverse Fast Fourier Transform) unit 46 is included.

  The FFT unit 41 is an input sound signal ECoutL (n), ECoutR (n) from which a sound non-target sound, which is a signal in the time domain, is removed, and a pseudo sound output target sound signal PSechoL (n), PSechoR (n). Are converted into frequency domain signals XL (f, K), XR (f, K), YL (f, K), and YR (f, K), respectively. In the above, “n” is a parameter representing time, “f” is a parameter representing frequency, and “K” is a parameter representing the order of frames defining a block of a predetermined number of input samples to be subjected to conversion. It is.

  The first coherence coefficient calculation unit 42 uses the frequency domain signals XL (f, K) and XR (f, K) obtained from the input sound signals ECoutL (n) and ECoutR (n) from which the emitted non-target sound has been removed. ) To calculate the coherence coefficient Xcoef (f, K).

  The second coherence coefficient calculator 43 calculates the coherence coefficient based on the frequency domain signals YL (f, K) and YR (f, K) obtained from the pseudo sound emission target sound signals PSechoL (n) and PSechoR (n). Ycoef (f, K) is calculated.

  As the calculation formulas of the coherence coefficients Xcoef (f, K) and Ycoef (f, K), those described in Patent Document 1 can be applied (see Expressions (1), (2), and (4) of Patent Document 1). ).

  The suppression coefficient calculation unit 44 calculates a suppression coefficient NRcoef (f, K) for suppressing the non-target sound from the two coherence coefficients Xcoef (f, K) and Ycoef (f, K), and supplies the suppression coefficient multiplication unit 45 with the suppression coefficient NRcoef (f, K). To give. As shown in FIG. 3, the suppression coefficient calculation unit 44 is a coefficient reception unit that receives the coherence coefficients Xcoef (f, K) and Ycoef (f, K) from the first and second coherence coefficient calculation units 42 and 43. 51, a suppression coefficient calculation unit 52 that calculates the suppression coefficient NRcoef (f, K) according to the equation (1), and a suppression coefficient transmission unit 53 that supplies the obtained suppression coefficient NRcoef (f, K) to the suppression coefficient multiplication unit 45 And become.

NRcoef (f, K)
= Xcoef (f, K)-[alpha] * Ycoef (f, K)
However, α is a value in the range of 0.0 <α ≦ 1.0 (1)
The suppression coefficient multiplying unit 45 applies the suppression coefficient NRcoef () to the one frequency domain signal XL (f, K) obtained from the input sound signal from which the emitted non-target sound has been removed, as shown in Equation (2). The frequency domain signal (in other words, the frequency domain signal of the target sound) Z (f, K) from which the non-target sound is removed by multiplying by f, K) is obtained.

Z (f, K) = XL (f, K) × NRcoef (f, K) (2)
The IFFT unit 46 converts the non-target sound suppression signal Z (f, K), which is a frequency domain signal, into a time domain signal z (n). If the latter circuit is configured to process the frequency domain signal Z (f, K) as it is, the IFFT unit 46 can be omitted.

  Similarly to the sound source separation processing unit 33, the sound emission non-target sound canceller processing unit 32 also has a non-target sound removal function. The reason why the sound non-target sound canceller processing unit 32 is provided in addition to the sound source separation processing unit 33 is as follows. That is, rather than capturing non-target sounds at once, the non-target sound and the background non-target sound are distinguished, and a removal method suitable for each is considered, and the non-target sound is output as a non-target sound canceller. The processing unit 32 removes the background non-target sound and the sound source separation processing unit 33 removes it. That is, a sound emission non-target sound canceller processing unit 32 is provided as a pre-processing unit of the sound source separation processing unit 33, and a non-target sound component having a strong correlation between the L channel and the R channel, which is not good at the sound source separation processing unit 33, is not emitted. By suppressing in advance by the target sound canceller processing unit 32, the function of the sound source separation processing unit 33 is fully exhibited, and at the same time, the non-target sound component that could not be suppressed by the emitted non-target sound canceller processing unit 32 is obtained. Suppression is performed by the sound source separation processing unit 33, and a much higher performance of suppressing non-target sound is obtained than when the sound source separation processing unit 33 is applied alone.

  If the coherence filter method is simply applied as the sound source separation method of the sound source separation processing unit 33, the non-target sound is suppressed from the input sound signals ECoutL (n) and ECoutR (n) from which the emitted non-target sound is removed. What is necessary is just to obtain the suppression coefficient to be used. In the first embodiment, not only the input sound signals ECoutL (n) and ECoutR (n) from which the sound non-target sound is removed but also the pseudo sound output target sound signals PSechoL (n) and PSechoR (n). As a result, a suppression coefficient NRcoef (f, K) used for suppressing non-target sounds is obtained. The reason for this is as follows.

  The sound emitted from the speakers 3L and 3R is, for example, music, and includes impact sounds (for example, drum sounds in rock) that suddenly have components at all frequencies, such as percussion instrument sounds. In this case, the follow-up in the emitted non-target sound canceller processing unit 32 (the adaptive filter) is not in time, and the emitted non-target sound cannot be sufficiently suppressed. Moreover, since the impact sound has components at all frequencies, even if the direction of arrival is not the front, the sounds emitted from the left and right speakers 3L and 3R have a strong correlation with each other so that the kite also comes from the front. It has special characteristics. Therefore, when the suppression coefficient used for suppressing the non-target sound is obtained only from the input sound signals ECoutL (n) and ECoutR (n) from which the non-target sound is removed, the non-target sound is the impact sound. Sometimes the removal of the non-target sound is insufficient.

  In order to avoid such an inconvenience, the pseudo sound emission target sound signals PSechoL and PSechoR are also used to form the suppression coefficient NRcoef used for suppressing the non-target sound.

  The pseudo sound emission target sound signals PSechoL and PSechoR calculated by the sound emission non-target sound canceller processing unit 32 are signals obtained by convolving the sound source data sigL and sigR with transfer characteristics from the speakers 3L and 3R to the microphones 4L and 4R. Therefore, it can be said that the microphones 4L and 4R have characteristics close to those of the disturbing sound components included in the input sound signals inputL and inputR captured by the microphones 4L and 4R. Therefore, it can be expected that the suppression performance to the impact sound can be improved by referring to the characteristic amount obtained from the pseudo sound emission target sound signals PSechoL and PSechoR, or the pseudo sound emission target sound signals PSEchoL and PSechoR.

  Therefore, in the first embodiment, the pseudo sound emission target sound signals PSechoL and PSechoR are also used for forming the suppression coefficient NRcoef used for suppressing the non-target sound.

  Next, it will be described in more detail that the pseudo sound emission target sound signals PSechoL and PSechoR can be used to form the suppression coefficient NRcoef used for suppressing the non-target sound.

  Considering the device configuration assumed by the first embodiment (see FIGS. 8 and 9 described above), the disturbing sound cannot come from the front. When this behavior is associated with the behavior of the feature quantity directly linked to the arrival direction such as the coherence described in Patent Document 1, it can be said that the interference sound does not take a coherence value equal to or higher than the target sound coming from the front. it can. However, as described above, when the impact sound includes the impact sound, the correlation between the disturbing sounds emitted from the left and right speakers 3L and 3R is remarkably increased, and it comes from the front even though it is a disturbing sound. Behaves as if That is, the coherence value of the disturbing sound when the impact sound is included is equal to or greater than the target sound. Therefore, the coherence filter method that sets the noise suppression gain according to the direction of arrival of the interference sound cannot sufficiently suppress the interference sound. By the way, the pseudo sound emission target sound signals PSEchoL and PSechoR are sounds obtained by convolution of the transmission characteristics from the speakers 3L and 3R to the microphones 4L and 4R into the sound source data sigL and sigR that become sound non-target sounds when emitted. Therefore, the target sound component is not included, and the signal is derived only from the disturbing sound component coming from the speakers 3L and 3R on both sides. Therefore, the range of the coherence value obtained from the two pseudo sound emission target sound signals PSechoL and PSechoR is smaller than the range of the target sound, and if the disturbing sound source data sigL and sigR include an impact sound, The coherence of the target sound signals PSechoL and PSechoR increases. In other words, it is possible to detect the occurrence of an impact sound by a sudden increase in the coherence of the pseudo sound emission target sound signals PSechoL and PSechoR. By referring to the coherence filter coefficient Ycoef obtained from the pseudo sound emission target sound signals PSechoL and PSechoR, the component of the impact sound can be obtained for each frequency. The coherence filter coefficient Xcoef obtained from the input sound signals ECoutL and ECoutR from which the sound non-target sound output from the sound non-target sound canceller processing unit 32 has been removed is obtained from the pseudo sound output target sound signals PSechoL and PSechoR. By adjusting the coherence filter coefficient Ycoef as shown in the equation (2), the component derived from the impact sound can be removed from the coherence filter coefficient, and a more accurate suppression coefficient Zcoef can be calculated.

(A-2) Operation of the First Embodiment Next, the operation of the sound collection / sound emission device 10 of the first embodiment will be described. In the following description, it is assumed that the sound source data is music data and the target sound is a sound produced by a user located in front of the sound collecting / sound emitting device 10.

  The sound source data (music data) read from the sound source data storage units 21L and 21R are converted into analog signals by the corresponding D / A conversion units 22L and 22R, and then emitted from the speakers 3L and 3R. The When such music is flowing from the sound collecting / sound emitting device 10, the sound produced by the user toward the sound collecting / sound emitting device 10 is captured by both microphones 4 </ b> L and 4 </ b> R. At this time, since music from the speakers 3L and 3R is also flowing, music from the speaker 3L is also captured by both microphones 4L and 4R, and music from the speaker 3R is also captured by both microphones 4L and 4R. Furthermore, ambient background noise (such as driving sound of an air conditioner, traveling sound from a vehicle traveling nearby) is also captured by both microphones 4L and 4R.

  In other words, the input sound signals obtained by the microphones 4L and 4R include non-purpose sounds such as music emitted by the device itself and non-purpose sounds such as background noise, in addition to the target sound of the user's voice. Sound (hereinafter referred to as background non-purpose sound as appropriate) is included.

  Input sound signals obtained by the microphones 4L and 4R are converted into digital signals inputL and inputR by the corresponding A / D converters 31L and 31R, respectively, and are supplied to the sound emission non-target sound canceller processing unit 32. The sound emission non-target sound canceller processing unit 32 is also provided with sound source data sigL and sigR.

  In the sound non-target sound canceller processing unit 32, the sound non-target sound is removed by subtracting the internally generated pseudo sound target sound signal PSchoL from the input sound signal (digital signal) inputL related to the L channel. Similarly, by subtracting the internally generated pseudo sound emission target sound signal PSEchoR from the input sound signal (digital signal) inputR relating to the R channel, the sound emission non-purpose sound is obtained. The removed input sound signal ECoutR is obtained. The pair of input sound signals ECoutL and ECoutR obtained by removing the non-target sound output thus obtained are supplied to the sound source separation processing unit 33 together with the pair of internally generated pseudo sound output target sound signals PSechoL and PSechoR. .

  In the sound source separation processing unit 33, the input sound signals ECoutL (n) and ECoutR (n) from which the sound non-target sound, which is a time domain signal, is removed by the FFT unit 41, and the pseudo sound output target sound signal PSchoL ( n) and PSechoR (n) are converted into frequency domain signals XL (f, K), XR (f, K), YL (f, K), YR (f, K), respectively.

  Then, the first coherence coefficient calculation unit 42 performs frequency domain signals XL (f, K) and XR (f) obtained from the input sound signals ECoutL (n) and ECoutR (n) from which the emitted non-target sound has been removed. , K), the coherence coefficient Xcoef (f, K) is calculated, and the second coherence coefficient calculation unit 43 obtains the frequency region obtained from the pseudo sound emission target sound signals PSechoL (n) and PSechoR (n). A coherence coefficient Ycoef (f, K) is calculated based on the signals YL (f, K) and YR (f, K).

  Thereafter, the suppression coefficient calculation unit 44 calculates a suppression coefficient NRcoef (f, K) for suppressing the non-target sound from the two coherence coefficients Xcoef (f, K) and Ycoef (f, K) according to the equation (1). Is supplied to the suppression coefficient multiplication unit 45, and the suppression coefficient multiplication unit 45 converts the suppression coefficient NRcoef () into one frequency domain signal XL (f, K) obtained from the input sound signal from which the emitted non-target sound has been removed. f, K) is multiplied for each frequency component to obtain a frequency domain signal Z (f, K) from which the non-target sound has been removed. By converting the non-target sound suppression signal Z (f, K), which is a frequency domain signal, into the time domain signal z (n) by the IFFT unit 46, an output signal output (= z (n)) including only the target sound component. ) Is obtained.

(A-3) Effects of the first embodiment According to the first embodiment, the non-target sounds are not collectively detected, but are classified into the emitted non-target sounds and the background non-target sounds, which are suitable for each. Since the target sound is extracted by applying the removal process, the target sound extraction accuracy can be made extremely high.

  In addition, according to the first embodiment, since the characteristics of the coherence filter coefficient calculated from the simulated sound emission target sound signal are reflected in the noise suppression coefficient, even if the sound source data includes an impact sound, the The target sound can be sufficiently suppressed.

  As a result, for example, when the voice that is the extracted target sound component is used for a call, the call sound quality can be improved, and when the voice that is the extracted target sound component is used for voice recognition, the recognition rate is increased. Can do.

(B) Second Embodiment Next, a second embodiment of the sound collecting / sound emitting device, sound source separation unit, and sound source separation program according to the present invention will be described with reference to the drawings.

  The second embodiment is different from the first embodiment in the internal configuration of a sound source separation processing unit (hereinafter, reference numeral 33A is used).

  FIG. 4 is a block diagram showing a detailed configuration of the sound source separation processing unit 33A of the second embodiment, and the same reference numerals are given to the same and corresponding parts as in FIG. 2 according to the first embodiment. It shows.

  4, the sound source separation processing unit 33A of the second embodiment includes an FFT unit 41, a first coherence coefficient calculation unit 42, a second coherence coefficient calculation unit 43, a suppression coefficient calculation unit 44A, and a suppression coefficient multiplication unit 45. In addition to the IFFT unit 46, a coherence calculation unit 47 and a sound emission non-target sound type determination unit 48 are provided. The suppression coefficient calculation unit 44A is also changed from that of the first embodiment.

  The coherence calculator 47 calculates the coherence COH (K) from the second coherence coefficient Ycoef (f, K) obtained by the second coherence coefficient calculator 43. Coherence COH (K) is calculated as an average value of coherence coefficients Ycoef (f, K) for all M frequency components, as shown in equation (5) of Patent Document 1.

  The sound emission non-purpose sound type determination unit 48 determines the sound type of the sound source data sigL and sigR that are the sound non-purpose sound based on the coherence COH (K) obtained by the coherence calculation unit 47. For example, it is determined whether the sound source data sigL and sigR includes impact sound, or the sound source data sigL and sigR that hardly includes impact sound.

  The sound emission non-purpose sound type determination unit 48 is realized by a program, for example, and functionally, as shown in FIG. 5, a coherence reception unit 61, a long-term average calculation unit 62, a variance calculation unit 63, a determination unit 64 and a determination result output unit 65.

  The coherence receiving unit 61 takes in the coherence COH (K) obtained by the coherence calculating unit 47.

  The long-term average calculation unit 62 calculates the long-term average value avecoh (K) of the coherence COH (K), for example, according to the equation (3), and the variance calculation unit 63 performs the coherence according to a general equation for variance. The variance var of COH (K) is calculated.

avecoh (k)
= Β × COH (K) + (1−β) × COH (K−1)
However, β is a value in the range of 0.0 <β <1.0 (3)
The determination unit 64 determines the sound types of the sound source data sigL and sigR that are the non-target sound emission from the long-term average value avecoh (K) of the coherence COH (K) and the variance var. For example, when the long-term average value avecoh (K) exceeds a preset threshold value and the variance var exceeds a preset threshold value, the determination unit 64 determines that the sound source data sigL and sigR are impact sounds. If the combination of the long-term average value avecoh (K) and the variance var is other than the above, it is determined that the sound source data sigL and sigR do not include the impact sound.

  The determination result output unit 65 gives the obtained sound type determination result to the suppression coefficient calculation unit 44A.

  FIG. 6 shows temporal changes in coherence COH (K) obtained when the music of the sound source data sigL and sigR is a classical music with a gentle change and a rock with a strong change including an impact sound. In the case of classic, the long-term average value of coherence COH (K) is small and the variance is also small. In the case of rock, the shock wave part raises the long-term average and increases the dispersion. Therefore, based on the long-term average value and variance of the coherence COH (K), it can be determined whether or not the sound source data sigL and sigR includes an impact sound.

  The reason why the coherence is applied is as follows. The coherence is an average of the coherence coefficients for each frequency component normalized by the signal level, and thus can be calculated without being affected by the volume of the emitted non-target sound. Therefore, even between songs such as rock and classical music with greatly different volumes, the characteristics can be compared without depending on the volume, and it is possible to eliminate as much as possible that the classical music with a large volume is erroneously determined to be rock. it can.

  The suppression coefficient calculation unit 44A of the second embodiment switches the calculation method of the suppression coefficient NRcoef (f, K) according to the determination result of the sound emission non-target sound type determination unit 48.

  For example, when the determination result of the sound emission non-target sound type determination unit 48 is a result that the sound source data sigL and sigR include an impact sound, the suppression coefficient calculation unit 44A uses the expression (1) as in the first embodiment. On the other hand, the suppression coefficient NRcoef (f, K) is calculated according to the above, and when the determination result of the sound emission non-target sound type determination unit 48 is that the sound source data sigL and sigR do not include the impact sound, the first coherence coefficient The coherence coefficient Xcoef (f, K) obtained by the calculation unit 42 is directly used as the suppression coefficient NRcoef (f, K). Other measures may be taken for the presence or absence of an impact sound. For example, the suppression coefficient calculation unit 44A may switch α in the expression (1) according to the presence or absence of an impact sound (note that α is increased when the impact sound is included).

  The first embodiment has a contrivance to prevent a decrease in accuracy of sound source separation when the sound source data sigL and sigR include impact sound, but when the sound source data sigL and sigR do not include impact sound, Ingenuity may affect accuracy.

  According to the second embodiment, since the sound source data sigL and sigR that are sound emission non-target sounds include the impact sound, the calculation method of the suppression coefficient of the non-target sound is switched. Regardless of whether the sound source data sigL and sigR include impact sound, the sound source separation accuracy can be improved.

(C) Other Embodiments In the description of each of the above-described embodiments, various modified embodiments have been referred to. However, modified embodiments as exemplified below can be given.

  In each of the above-described embodiments, the sound emission non-target sound canceller processing unit 32 that uses the stereo echo canceller technique is shown. However, as the sound emission non-target sound canceller processing unit 32, 4 as shown in FIG. The configuration of two monaural echo cancellers 71LL, 71RL, 71LR, 71RR may be used. 7 can also be regarded as belonging to the category of stereo echo canceller (see FIG. 3.73 of Non-Patent Document 1).

  When a monaural echo canceller is used, there are two speakers 3L and 3R and two microphones 4L and 4R, so that the acoustic path is congested, and the acoustic path characteristic cannot be accurately estimated, and a sufficient suppression effect may not be obtained.

Therefore, prior to reproduction of the sound source data sigL and sigR, white noise is emitted only from the speaker 3L, and the acoustic path characteristic H _LL from the speaker 3L to the microphone 4L and the acoustic path characteristic H _LR from the speaker 3L to the microphone 4R are obtained. , mono echo canceller 71LL and the adaptive filter estimates of 71LR, then sound white noise only from the speaker 3R, acoustic path from the acoustic path characteristics _{H RL} and the speaker 3R of the speaker 3R to the microphone 4L to the microphone 4R The characteristic H _RR is estimated by the monaural echo cancellers 71RL and 71RR adaptive filters, and is initially set. Thereafter, it is possible to suppress the sound non-target sound by subtracting the pseudo sound non-target sound signal obtained by convolving the four sound path characteristics and the corresponding sound source data from the input sound signal captured by the microphone. it can. As described above, the adaptive filters of the four monaural echo cancellers 71LL, 71RL, 71LR, and 71RR learn the four acoustic path characteristics in advance before sound source data is emitted, thereby reducing the congestion of the acoustic path. It is possible to prevent and suppress the non-target sound emission.

  After the white noise period is over, the update of the coefficients of the adaptive filters of the four monaural echo cancellers 71LL, 71RL, 71LR, 71RR is stopped, and the sound that has been learned with white noise is always used to remove the emitted non-target sound. You may make it do.

  In the first embodiment, the suppression coefficient calculation unit 44 calculates the suppression coefficient using the expression (1). However, the flooring process is performed after the calculation of the expression (1) so that the suppression coefficient does not become too small. You may make it give. In this way, it is possible to prevent deterioration in sound quality due to excessive suppression.

  In the first embodiment, the case where the coefficient α in the expression (1) calculated by the suppression coefficient calculation unit 44 is fixed is shown, but a variable coefficient may be applied as the coefficient α. For example, the coefficient α may be controlled according to the degree of inclusion of the emitted non-target sound (or the entire non-target sound). For example, the coefficient α may be varied with the power of sound source data that is a non-sound emission sound as the noise content. This makes it possible to control the suppression performance according to the noise content.

  In the second embodiment, the determination of the sound type is a determination of whether or not the sound source data includes an impact sound. However, there are three or more determinations such as a strong, weak, and no impact sound. In this case, the coefficient α may be switched depending on how the impact sound is included.

  In each of the above embodiments, the arithmetic expression for correcting the first coherence coefficient by using the second coherence coefficient is the subtraction shown in the expression (1), but other arithmetic expressions (functions) are The first coherence coefficient may be corrected by applying the second coherence coefficient. For example, the suppression coefficient may be calculated by dividing the first coherence coefficient by a value obtained by multiplying the second coherence coefficient by a coefficient.

  In the second embodiment, the characteristic amount used for the determination of the sound emission non-target sound (interfering sound) is the coherence variance and the long-term average value. However, the characteristic as shown in FIG. 6 can be distinguished. If so, other statistics may be used. For example, a value obtained by dividing the maximum coherence value by the average value or a coefficient of variation (= standard deviation / average value) may be used as the feature amount.

  Further, the feature amount may be calculated using not the coherence but the coherence coefficient of one or a plurality of frequency components that are not all. Furthermore, the presence / absence of an impact sound and the stage where the impact sound is mixed may be determined based on the power change of the pseudo sound emission non-target sound signal without calculating the coherence coefficient or coherence. Furthermore, the feature value used for the determination is not limited to the feature value obtained from the pseudo sound emission non-target sound signal. For example, instead of the feature amount obtained from the pseudo-non-target sound signal, or in addition to the feature amount obtained from the pseudo-non-target sound signal, the sound non-sound output from the sound non-target sound canceller processing unit is output. You may make it use the feature-value obtained from the input sound signal from which the target sound was removed for determination of the kind of sound of a non-target sound to be emitted.

  In the second embodiment, the sound type determination result is reflected in the suppression coefficient calculation method. However, instead of or in addition to this, an adaptive filter in the sound emission non-target sound canceller processing unit is used. It may be used to change the step size. For example, when an impact sound is included, the step size is increased to speed up the follow-up performance.

  In each of the embodiments described above, the sound source separation processing unit has shown the target sound and the background non-target sound separated according to the coherence filter method, but the separation method is not limited to this. For example, a combination of the coherence filter method and the frequency subtraction method (spectral subtraction method) may be applied, or a combination of the coherence filter method and the Wiener filter method may be applied. The coherence filter method and the frequency A combination of the subtraction method and the Wiener filter method may be applied.

  When applying the frequency subtraction method, the ratio of subtracting the spectrum of the noise component from the spectrum of the input speech signal may be changed according to the determination result of the sound type determined in the second embodiment. Further, when applying the Wiener filter method, the Wiener filter coefficient to be multiplied with respect to the spectrum of the input audio signal may be changed according to the determination result of the sound type determined in the second embodiment.

  In each of the above-described embodiments, the case where there are two speakers is shown, but there may be one speaker or three or more speakers. Also, the number of microphones is not limited to two and may be three or more. The internal configuration of the sound emission non-target sound canceller processing unit 32 may be designed in consideration of the number of sound emission sound paths determined according to the number of speakers and microphones.

  In each of the above-described embodiments, the sound collection / sound emission device alone performs all processing. However, the non-target sound removal processing may be performed by an external server. For example, when the sound collection / sound emission device is a smartphone, the system may be configured by a so-called cloud and updated so that the user does not know the presence of the external server. The claim of “sound collecting / sound emitting device” in the claims includes a case where an external server that is invisible to the user performs processing.

10 ... Sound collecting / sound emitting device,
20 ... Sound emission part, 21L, 21R ... Sound source data storage part, 22L, 22R ... D / A conversion part, 3L, 3R ... Speaker
30 ... Sound collection unit, 4L, 4R ... Microphone, 31L, 31R ... A / D conversion unit, 32 ... Sound emission non-target sound canceller processing unit, 33, 33A ... Sound source separation processing unit, 41 ... FFT unit, 42th 1 coherence coefficient calculation unit 43 43 second coherence coefficient calculation unit 44 44A suppression coefficient calculation unit 45 suppression coefficient multiplication unit 46 IFFT unit 47 coherence calculation unit 48 sound non-purpose Sound type determination unit.

Claims (7)

In a sound collection / sound emission device having a sound collection unit in which at least two microphones capture ambient sound and a sound emission unit that emits sound from one or more speakers,
Sound source separation means for extracting a target sound from a sound source in a predetermined direction based on an input sound signal obtained by capturing the ambient sound by the two microphones;
A sound signal emitted by the sound emitting unit is input, sound is emitted from the speaker, and a pseudo sound emitting non-purpose sound signal that simulates a non-purpose sound accompanying sound emission captured by each microphone is generated, A sound emission non-target sound removing means provided up to a route to the sound source separation means for removing the sound non-purpose sound captured by each microphone by subtracting from the input sound signal from each microphone; Prepared,
The sound source separation means is
A first separation parameter generating unit that generates a first parameter for sound source separation from the input sound signal from which the sound non-target sound output from the sound non-target sound removing means is removed;
A second separation parameter generating unit that generates a second parameter for sound source separation based on the pseudo-sound non-target sound signal generated in the sound non-target sound removing means;
A parameter correction unit for correcting the first parameter using the second parameter to obtain a final parameter used for sound source separation;
A sound source separation unit that performs sound source separation by applying the corrected parameters;
A sound collecting / sound emitting device characterized in that the non-target sound is removed by the sound non-target sound removing means and the non-target sound is removed by the sound source separation means to extract the target sound.   2. The sound collection / sound emission device according to claim 1, wherein the sound source separation means performs sound source separation according to a coherence filter method, and the parameter is a suppression coefficient.   The parameter correction unit multiplies the second coherence filter coefficient, which is the second parameter, by a positive correction value of 1.0 or less, and then subtracts it from the first coherence filter coefficient, which is the first parameter. The sound collection / sound emission device according to claim 2, wherein a suppression coefficient that is a final parameter is calculated.   The sound collection / sound emission device according to claim 3, wherein the parameter correction unit performs flooring processing on the calculated suppression coefficient.   The sound source separation unit further includes a sound type determination unit that determines a sound type of the sound emission non-target sound based on the pseudo sound emission non-purpose sound signal generated in the sound emission non-purpose sound removal unit, The sound collecting / sound emitting device according to any one of claims 1 to 4, wherein the parameter correcting unit selects whether or not the sound type of the determined sound emission non-target sound is corrected or a correction method. A sound source separation unit applied to a sound collection / sound emission device having a sound collection unit in which at least two microphones capture ambient sound and a sound emission unit that emits sound from one or more speakers,
Sound source separation means for extracting a target sound from a sound source in a predetermined direction based on an input sound signal obtained by capturing the ambient sound by the two microphones;
A sound signal emitted by the sound emitting unit is input, sound is emitted from the speaker, and a pseudo sound emitting non-purpose sound signal that simulates a non-purpose sound accompanying sound emission captured by each microphone is generated, A sound emission non-target sound removing means provided up to a route to the sound source separation means for removing the sound non-purpose sound captured by each microphone by subtracting from the input sound signal from each microphone; Prepared,
The sound source separation means is
A first separation parameter generating unit that generates a first parameter for sound source separation from the input sound signal from which the sound non-target sound output from the sound non-target sound removing means is removed;
A second separation parameter generating unit that generates a second parameter for sound source separation based on the pseudo-sound non-target sound signal generated in the sound non-target sound removing means;
A parameter correction unit for correcting the first parameter using the second parameter to obtain a final parameter used for sound source separation;
A sound source separation unit that performs sound source separation by applying the corrected parameters;
A sound source separation unit characterized in that a non-target sound is removed by the sound non-target sound removing means, and a target sound is extracted by removing other non-purpose sounds by the sound source separation means. A sound source separation program executed by a computer mounted on a sound collecting / sound emitting device having a sound collecting unit in which at least two microphones capture ambient sound and a sound emitting unit emitting sound from one or more speakers. And
The above computer
Sound source separation means for extracting a target sound from a sound source in a predetermined direction based on an input sound signal obtained by capturing the ambient sound by the two microphones;
A sound signal emitted by the sound emitting unit is input, sound is emitted from the speaker, and a pseudo sound emitting non-purpose sound signal that simulates a non-purpose sound accompanying sound emission captured by each microphone is generated, Functions as sound emission non-target sound removal means provided up to the path to the sound source separation means, which removes sound emission non-purpose sound captured by each microphone by subtracting from the input sound signal from each microphone Let
The sound source separation means to be operated is
A first separation parameter generating unit that generates a first parameter for sound source separation from the input sound signal from which the sound non-target sound output from the sound non-target sound removing means is removed;
A second separation parameter generating unit that generates a second parameter for sound source separation based on the pseudo-sound non-target sound signal generated in the sound non-target sound removing means;
A parameter correction unit for correcting the first parameter using the second parameter to obtain a final parameter used for sound source separation;
A sound source separation unit that performs sound source separation by applying the corrected parameters;
A sound source separation program for removing a non-target sound by the sound non-target sound removing means and extracting a target sound by removing other non-target sounds by the sound source separation means.

Images