JP2015070292A - Sound collection/emission device and sound collection/emission program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a sound collection/emission device which can improve accuracy of an expected processing regardless to the kinds of an interference sound.SOLUTION: A sound collection/emission device emits a sound from a speaker and collects ambient sound from two microphones. The sound collection/emission device comprises: interference sound removal means with the emitted sound into which the emitted sound signal is inputted to generate a pseudo interference sound signal simulating an interference sound with the emitted sound intruded from the speaker into respective microphones, subtract the pseudo interference sound signal from the input signal from the respective microphones so that the interference sound with the emitted sound is removed; sound kind determination means which determines a sound kind of the interference sound with the emitted sound on the basis of the pseudo interference sound signal; and one or a plurality of sound kind reflection processing means which switch own processing according to the determination result. The sound kind reflection processing means is sound source separation means, for example.

Description

  The present invention relates to a sound collection / sound emission device and a sound collection / sound emission program, for example, in communication terminals, audio devices, and the like that want to remove components emitted by a speaker from sound captured by a microphone, captured sound, and the like. It can be applied.

  For example, when inputting call voice to a smartphone or inputting voice commands to an audio device or smartphone, the device to which the sound is input is only the sound from the front where the user's mouth seems to exist. Is preferably distinguished from voice, music, noise, etc. from other directions.

  Sounds input to two microphones are captured, ambient noise is suppressed based on the phase difference between the input sounds (electrical signals), and sound arriving at the microphone from a sound source in a predetermined direction (for example, the front) Patent Document 1 discloses a method (sound source separation method) that extracts only separation (referred to as sound).

  The target sound extraction method described in Patent Document 1 as the third embodiment uses a suppression coefficient corresponding to the correlation between two signals obtained by forming two directivities having blind spots on the left and right sides of a microphone. This is a technique for suppressing noise components (non-target sounds) coming from the left and right by multiplying an input sound signal for each component. The target sound extraction method described as the fourth embodiment in Patent Document 1 forms a directivity having a blind spot in front of a microphone, and inputs a signal obtained as a noise component coming from the left and right. This is a technique for suppressing noise components (non-target sounds) coming from the left and right by subtracting from the sound signal.

  Incidentally, in recent years, as shown in FIG. 6, a pair of speakers 3L and 3R are arranged and connected on both sides of a sound collecting device 2 having a communication function such as a portable terminal (for example, a smartphone or a tablet terminal). The sound collecting / sound emitting device 1 for making a call with a remote place with such a configuration has come to be used. Also, with the same configuration, sound (music) from music files recorded in the sound collecting device 2 or music files acquired from music distribution sites on the Internet is emitted from the speakers 3L and 3R on both sides. In this state, a method in which the user receives a command by a voice emitted from the front of the microphone of the sound collecting device 2 is also being studied.

  In the state where music is emitted from the speakers 3L and 3R on both sides, the target sound coming from the front is extracted and the utterance content is communicated to the other party, or the voice command is recognized through voice recognition processing. When the processing corresponding to the voice command is executed, the sound emitted from the speakers 3L and 3R becomes noise, which greatly reduces the call sound quality and the voice recognition rate.

  Therefore, it is necessary to apply a sound source separation method such as the technology described in Patent Document 1 described above, suppress noise components coming from the speakers 3L and 3R on both sides, and extract the target sound from the front. When applying the sound source separation method described in Patent Document 1, it is necessary to mount or externally attach two microphones 4L and 4R to the sound collecting device 1, as shown in FIG.

JP 2013-061421 A

Kitawaki Nobuhiko, “Digital Voice / Audio Technology (Future Netto Technology Series)”, published by Telecommunications Association, p218-p243, 1999

  However, when the user enjoys the music from the sound collecting / sound emitting device 1, the volume is large and the loud music is captured by the microphones 4 </ b> L and 4 </ b> R as noise components (non-target sounds). Even if the target sound is extracted by applying the sound source separation method, many noise components remain in the extracted target sound signal.

  In order to avoid this, after the user stops outputting the music (sound emission), the user may pronounce the input voice such as a call voice or voice command. However, if the key operation for stopping the output is performed as described above, the merit of the voice command is reduced, and it is easier to input the command by the key operation. Further, in the case of a call from an incoming call, the voice output stop operation cannot be performed, or the incoming call is delayed due to the execution of the output stop operation.

  Here, there are various sounds and sounds emitted from the sound collection / sound emission device 1, which are captured by the speakers 3L and 3R, and various disturbing sounds (non-target sounds) with respect to the target sound. That is, there are various types of disturbing sounds such as music and voice phrases, and the acoustic characteristics vary greatly depending on the types of the disturbing sounds. For example, music (songs) includes music of various genres from classical music to rock. If the interference sound is rock, the SN ratio becomes poor because the volume is high, and a lot of sudden impact sounds are generated by percussion instruments such as drums, resulting in unsteady characteristics. On the other hand, in the case of classical music, since the volume is relatively low, the S / N ratio with the target sound is good, and sudden impact sound is rarely generated, so that it is a steady characteristic. Therefore, when the disturbing sound is a voice phrase or classical music, a sufficient suppression effect can be obtained without performing powerful suppression processing on the disturbing sound (non-target sound). And powerful suppression processing that can cope with sudden fluctuations must be done.

  As described above, if the interference noise suppression process is uniformly performed ignoring the difference in acoustic characteristics depending on the type of the interference sound (non-target sound), the suppression performance is insufficient or the interference sound is excessively suppressed. This causes problems such as poor sound quality.

  Therefore, there is a demand for a sound collection / sound emission device and a sound collection / sound emission program that can improve the accuracy of desired processing regardless of the type of interference sound.

  According to a first aspect of the present invention, there is provided a sound collection / sound emission device having a sound collection unit in which at least two microphones capture ambient sounds and a sound emission unit that emits sound from one or more speakers. A sound signal emitted by the sound emitting unit is input, emitted from the speaker, generates a pseudo-interference sound signal that simulates the interference sound accompanying the sound output captured by each microphone, and is input from each microphone. A sound emission disturbing sound removing means using an acoustic echo canceller configuration for removing the sound emission disturbing sound captured by each of the microphones by subtracting from the sound signal; and (2) the sound emission disturbing sound removing means. Sound type determination means for determining the sound type of the sound emission disturbing sound based on the pseudo sound emission disturbance sound signal generated in the sound source, and (3) self-processing according to the determination result of the sound type determination means. One or more sound type reflection processes to be switched Characterized in that it comprises a stage.

  According to a second aspect of the present invention, there is provided a computer mounted on a sound collection / sound emission device having a sound collection unit in which at least two microphones capture ambient sounds and a sound emission unit that emits sound from one or more speakers. A sound collection / sound emission program to be executed, wherein the computer is (1) a sound signal emitted by the sound emission unit is input, sound is emitted from the speaker, and is emitted by the microphones. A sound echo canceller configuration that diverts the sound emission interference sound captured by each microphone by generating a pseudo interference sound signal that simulates the accompanying interference sound and subtracting it from the input sound signal from each microphone is diverted. And (2) sound type determination means for determining the sound type of the sound emission disturbing sound based on the pseudo sound emission disturbance sound signal generated in the sound emission interference sound removing means. (3) The above sound types According to the determination result of the constant means and be made to function as one or more note type reflection processing unit switches its own processing.

  According to the present invention, it is possible to realize a sound collection / sound emission device and a sound collection / sound emission program that can improve the accuracy of desired processing regardless of the type of interference sound.

It is a block diagram which shows the structure of the sound collection and sound emission apparatus of 1st Embodiment. It is a block diagram which shows the detailed structure of the sound source separation process part in the sound collection / sound emission apparatus of 1st Embodiment. It is a block diagram which shows the detailed structure of the sound emission non-purpose sound type determination part in the sound source separation process part in the sound collection / sound emission apparatus of 1st Embodiment. It is explanatory drawing which shows the relationship between the kind of sound source data and the behavior of coherence in relation to 1st Embodiment. It is a block diagram which shows the detailed structure of the sound emission non-target sound canceller process part in the sound collection and sound emission apparatus of 2nd Embodiment. It is explanatory drawing which shows the mode of the connection of the speaker in the conventional sound collection and sound emission apparatus. It is explanatory drawing which shows the mode of mounting of the microphone in the case of applying a sound source separation system to the conventional sound collecting / sound emitting device.

(A) First Embodiment Hereinafter, a first embodiment of a sound collecting / sound emitting device and a sound collecting / sound emitting program according to the present invention will be described with reference to the drawings.

(A-1) Configuration of the First Embodiment The sound collection / sound emission device of the first embodiment is equipped with a pair of microphones or externally attached, and a pair of speakers. Or it is an external one. For example, in the case of a sound collecting / sound emitting device using a sound collecting device such as a smartphone or a tablet terminal, a pair of microphones is mounted and a pair of speakers are externally configured. Further, for example, if a speaker integrated audio device is a corresponding sound collecting / sound emitting device, a pair of microphones and a pair of speakers are mounted. As described above, the connection forms of the pair of microphones and the pair of speakers are various, but any connection form may be applied.

  In the following, the sound collection / sound emission device of the first embodiment will be described on the assumption that a pair of microphones are mounted and a pair of speakers are externally attached as shown in FIG. 7 described above. In addition, for the components described in FIG. 7, the symbols used in FIG. 7 are used as they are for the components in the sound collection / sound emission device of the first embodiment.

  FIG. 1 is a block diagram illustrating a configuration of a sound collection / sound emission device 10 according to the first embodiment.

  The sound collection / sound emission device 10 of the first embodiment may be constructed by connecting various hardware components, and some components (for example, a speaker, a microphone, The functions of the analog / digital conversion unit (A / D conversion unit) and digital / analog conversion unit (except for the D / A conversion unit) are realized by applying program execution configurations such as CPU, ROM, and RAM. It may be constructed to do so. Regardless of which construction method is applied, the functional detailed configuration of the sound collection / sound emission device 10 is the configuration shown in FIG. When applying the program, the program may be written in the memory of the sound collecting / sound emitting device 10 from the time of shipment of the device, or may be installed by downloading. good. For example, in the latter case, a program is prepared as an application for a smartphone, and a user who needs it can download and install it via the Internet.

  In FIG. 1, the sound collection / sound emission device 10 of the first embodiment includes a sound emission unit 20 and a sound collection unit 30.

  The sound emitting unit 20 has the same configuration as the existing sound emitting unit. The sound emitting unit 20 includes sound source data storage units 21L and 21R for L channel and R channel, D / A conversion units 22L and 22R, and speakers 3L and 3R.

  On the other hand, the sound collection unit 30 includes L-channel and R-channel microphones 4L and 4R, A / D conversion units 31L and 31R, a sound emission non-target sound canceller processing unit 32, and a sound source whose detailed configuration is shown in FIG. And a separation processing unit 33. Here, the entire sound collection unit 30 having an input terminal for sound source data, which will be described later, may be constructed as a sound source separation unit and provided on the market. Further, the part composed of the A / D conversion units 31L and 31R, the sound emission non-target sound canceller processing unit 32, and the sound source separation processing unit 33 has a sound source data input terminal, which will be described later, and is constructed as a sound source separation unit. You may use for a commercially available thing.

  The sound source data storage units 21L and 21R store the sound source data (digital signals) sigL and sigR for the L channel and the R channel, respectively, and read and output the sound source data sigL and sigR under the control of a sound emission control unit (not shown). Is. The sound source data sigL and sigR may be, for example, music data or electronic data such as an electronic book for reading out. Each of the sound source data storage units 21L and 21R may be a recording medium access device loaded with a recording medium such as a CD-ROM, and stores sound source data acquired by communication from an external device such as a site on the Internet. It may be configured by a storage unit of the apparatus. The sound source data storage units 21L and 21R may correspond to, for example, external devices connected by USB connector connection. Furthermore, each sound source data storage unit 21L, 21R is named “storage unit”, but the concept of each sound source data storage unit 21L, 21R includes received sound source data such as a digital audio broadcast receiver. A configuration for outputting in real time is also included.

  The D / A converters 22L and 22R convert the sound source data sigL and sigR output from the corresponding sound source data storage units 21L and 21R into analog signals and give them to the corresponding speakers 3L and 3R.

  The speakers 3L and 3R output sound sources (sound generation output) from the sound source signals supplied from the corresponding D / A converters 22L and 22R, respectively. Here, the sound or sound output from the speakers 3L and 3R is not intended to be captured by the microphones 4R and 4L, but becomes non-target sound when viewed from the capturing function of the microphones 4R and 4L. It has become.

  In the above, the music and sound emitted from the speakers 3L and 3R are shown as digital signals (sound source data). However, the configuration corresponding to the sound source data storage units 21L and 21R is a record player. An audio cassette tape recorder, an AM or FM radio receiver, and the like may output an audio signal or an audio signal that is an analog signal. In this case, the D / A converters 22L and 22R are omitted, and an A / D converter for the L channel and the R channel is provided separately to convert an analog acoustic signal or audio signal into a digital signal. The sound is output to the non-target sound canceller processing unit 32.

  Each of the microphones 4R and 4L captures ambient sound and converts it into an electrical signal (analog signal). A stereo signal is obtained by the pair of microphones 4R and 4L. Each of the microphones 4R and 4L has directivity that mainly captures sound coming from the front of the sound collection / sound emission device 10, but emits sound from the speakers 3L and 3R arranged on both sides. It also captures the generated sound. The speakers 3L and 3R are preferably arranged on both sides of the pair of microphones 4R and 4L, but are not limited to this arrangement.

  The microphones 4R and 4L are attached to, for example, a cylinder provided in the housing of the sound collecting / sound emitting device 10. Here, a sound insulating member made of a synthetic resin is provided on the inner surface of the cylindrical body so that when the microphones 4R and 4L are attached, there is no path through which the sound passes inside and outside the housing. This prevents as much as possible the microphones 4R and 4L from capturing the noise generated inside the housing or the noise entering the housing from the outside and going out of the housing by reflection. Can do.

  The A / D conversion units 31L and 31R convert the input sound signals captured by the corresponding microphones 4R and 4L into digital signals inputL and inputR, respectively, and give them to the sound emission non-target sound canceller processing unit 32. Each A / D conversion unit 31L, 31R converts, for example, a digital signal having the same sampling rate as the sampling rate of the sound source data sigL, sigR.

  The sound emission non-target sound canceller processing unit 32 is also supplied with sound source data sigL and sigR output from the sound source data storage units 21L and 21R. Here, it is necessary that the sampling rates of the four digital signals input to the sound emission non-target sound canceller processing unit 32 are the same. For example, the sampling rates of the sound source data sigL and sigR downloaded from the Internet site and stored in the sound source data storage units 21L and 21R are different from the sampling rates of the digital signals inputL and inputR from the A / D conversion units 31L and 31R. In this case, the downloaded sound source data sigL and sigR are directly supplied to the D / A conversion units 22L and 22R, and the sound source data obtained by converting the sampling rate of the sound source data sigL and sigR is supplied to the sound emission non-target sound canceller processing unit 32. You should give it.

  The sound emission non-target sound canceller processing unit 32 is based on the sound source data sigL and sigR output from the sound source data storage units 21L and 21R, and includes the speakers 3L and 3R included in the input sound signals (digital signals) inputL and inputR. Removes (or reduces) the non-target sound component (hereinafter referred to as the sound non-target sound as appropriate) that is emitted from the sound, and provides the sound source separation processing unit 33 with the input sound signals ECoutL and ECoutR after the removal processing. Is.

  Here, the sound that is emitted from the speakers 3L and 3R and captured by the microphones 4R and 4L and is unnecessary from the target sound (non-target sound) is the same as the acoustic echo that is a problem in telephone communication. Can be considered. Therefore, in the first embodiment, the sound emission non-target sound canceller processing unit 32 is configured using the acoustic echo canceller technique. For example, Non-Patent Document 1 describes “stereo echo canceller”. In the first embodiment, it is assumed that the sound output non-target sound canceller processing unit 32 described in FIG. 3.71 or FIG. 3.75 of Non-Patent Document 1 is applied.

  In the sound emission non-target sound canceller processing unit 32 having a stereo echo canceller configuration, in order to remove the sound emission target sound from the input sound signals inputL and inputR, a pseudo sound emission target sound signal (hereinafter referred to as “pseudo sound emission target sound signal”) is internally generated. PSechoL and PSechoR (referred to as sound target sound signals) are generated. In the case of the first embodiment, the pseudo sounding target sound signals PSechoL and PSechoR are also supplied to the sound source separation processing unit 33.

  The sound source separation processing unit 33 has the detailed configuration shown in FIG. 2, and based on the input sound signals ECoutL and ECoutR from which the sound emission non-target sound has been removed and the pseudo sound emission target sound signals PSechoL and PSechoR (for example, Only the target sound from the sound source located in the front) is extracted. The sound source separation method applied by the sound source separation processing unit 33 is a coherence filter method to which a coherence coefficient whose characteristics change depending on the direction of the sound source is applied.

  2, the sound source separation processing unit 33 includes an FFT (Fast Fourier Transform) unit 41, a first coherence coefficient calculation unit 42, a second coherence coefficient calculation unit 43, a suppression coefficient calculation unit 44, a suppression coefficient multiplication unit 45, An IFFT (Inverse Fast Fourier Transform) unit 46, a coherence calculation unit 47, and a sound emission non-target sound type determination unit 48 are included.

  The FFT unit 41 is an input sound signal ECoutL (n), ECoutR (n) from which a sound non-target sound, which is a signal in the time domain, is removed, and a pseudo sound output target sound signal PSechoL (n), PSechoR (n). Are converted into frequency domain signals XL (f, K), XR (f, K), YL (f, K), and YR (f, K), respectively. In the above, “n” is a parameter representing time, “f” is a parameter representing frequency, and “K” is a parameter representing the order of frames defining a block of a predetermined number of input samples to be subjected to conversion. It is described when it is necessary to clarify the explanation. That is, even if not described, it is an omission of description, and these parameters are referred to in the processing.

  The first coherence coefficient calculation unit 42 uses the frequency domain signals XL (f, K) and XR (f, K) obtained from the input sound signals ECoutL (n) and ECoutR (n) from which the emitted non-target sound has been removed. ) To calculate the coherence coefficient Xcoef (f, K).

  The second coherence coefficient calculator 43 calculates the coherence coefficient based on the frequency domain signals YL (f, K) and YR (f, K) obtained from the pseudo sound emission target sound signals PSechoL (n) and PSechoR (n). Ycoef (f, K) is calculated.

  As the calculation formulas of the coherence coefficients Xcoef (f, K) and Ycoef (f, K), those described in Patent Document 1 can be applied (see Expressions (1), (2), and (4) of Patent Document 1). ).

  The coherence calculator 47 calculates the coherence COH (K) from the second coherence coefficient Ycoef (f, K) obtained by the second coherence coefficient calculator 43. Coherence COH (K) is calculated as an average value of coherence coefficients Ycoef (f, K) for all M frequency components, as shown in equation (5) of Patent Document 1.

  The sound emission non-purpose sound type determination unit 48 determines the sound type of the sound source data sigL and sigR that are the sound non-purpose sound based on the coherence COH (K) obtained by the coherence calculation unit 47. For example, it is determined whether the sound source data sigL and sigR includes impact sound, or the sound source data sigL and sigR that hardly includes impact sound.

  The sound emission non-purpose sound type determination unit 48 is realized by a program, for example, and functionally, as shown in FIG. 3, a coherence reception unit 51, a long-term average calculation unit 52, a variance calculation unit 53, a determination unit 54 and a determination result output unit 55.

  The coherence receiving unit 51 takes in the coherence COH (K) obtained by the coherence calculating unit 47.

  The long-term average calculation unit 52 calculates the long-term average value avecoh (K) of the coherence COH (K), for example, according to the equation (1), and the variance calculation unit 53 performs the coherence according to a general equation for variance. The variance var of COH (K) is calculated.

avecoh (k)
= Β × COH (K) + (1−β) × COH (K−1)
However, β is a value in the range of 0.0 <β <1.0 (1)
The determination unit 54 determines the sound types of the sound source data sigL and sigR that are the non-target sound emission from the long-term average value avecoh (K) of the coherence COH (K) and the variance var. For example, when the long-term average value avecoh (K) exceeds a preset threshold value and the variance var exceeds a preset threshold value, the determination unit 54 determines that the sound source data sigL and sigR are impact sounds. If the combination of the long-term average value avecoh (K) and the variance var is other than the above, it is determined that the sound source data sigL and sigR do not include the impact sound.

  The determination result output unit 55 gives the obtained sound type determination result to the suppression coefficient calculation unit 44.

  FIG. 4 shows temporal changes in coherence COH (K) obtained when the music of the sound source data sigL and sigR is a classical music with a gentle change and a rock with a strong change including an impact sound. In the case of classic, the long-term average value of coherence COH (K) is small and the variance is also small. In the case of rock, the impact sound part raises the long-term average value and increases the dispersion. Therefore, based on the long-term average value and variance of the coherence COH (K), it can be determined whether or not the sound source data sigL and sigR includes an impact sound.

  The suppression coefficient calculation unit 44 calculates a suppression coefficient NRcoef (f, K) for suppressing the non-target sound from the two coherence coefficients Xcoef (f, K) and Ycoef (f, K), and supplies the suppression coefficient multiplication unit 45 with the suppression coefficient NRcoef (f, K). To give. The suppression coefficient calculation unit 44 switches the calculation method of the suppression coefficient NRcoef (f, K) according to the determination result of the sound emission non-target sound type determination unit 48.

  For example, when the determination result of the sound emission non-target sound type determination unit 48 is that the sound source data sigL and sigR include impact sound, the suppression coefficient calculation unit 44 suppresses the suppression coefficient NRcoef (f, K) according to the equation (2). On the other hand, when the determination result of the sound emission non-target sound type determination unit 48 is that the sound source data sigL and sigR do not include an impact sound, the coherence coefficient obtained by the first coherence coefficient calculation unit 42 Xcoef (f, K) is directly used as the suppression coefficient NRcoef (f, K). Other measures may be taken for the presence or absence of an impact sound. For example, the suppression coefficient calculation unit 44 may switch α in Expression (2) according to the presence or absence of an impact sound (note that α is increased when the impact sound is included).

NRcoef (f, K)
= Xcoef (f, K)-[alpha] * Ycoef (f, K)
However, α is a value in the range of 0.0 <α ≦ 1.0 (2)
The suppression coefficient multiplication unit 45 applies the suppression coefficient NRcoef () to the one frequency domain signal XL (f, K) obtained from the input sound signal from which the emitted non-target sound has been removed, as shown in Equation (3). The frequency domain signal (in other words, the frequency domain signal of the target sound) Z (f, K) from which the non-target sound is removed by multiplying by f, K) is obtained.

Z (f, K) = XL (f, K) × NRcoef (f, K) (3)
The IFFT unit 46 converts the non-target sound suppression signal Z (f, K), which is a frequency domain signal, into a time domain signal z (n). If the latter circuit is configured to process the frequency domain signal Z (f, K) as it is, the IFFT unit 46 can be omitted.

  Similarly to the sound source separation processing unit 33, the sound emission non-target sound canceller processing unit 32 also has a non-target sound removal function. The reason why the sound non-target sound canceller processing unit 32 is provided in addition to the sound source separation processing unit 33 is as follows. That is, rather than capturing non-target sounds at once, the non-target sound and the background non-target sound are distinguished, and a removal method suitable for each is considered, and the non-target sound is output as a non-target sound canceller. The processing unit 32 removes the background non-target sound and the sound source separation processing unit 33 removes it. That is, a sound emission non-target sound canceller processing unit 32 is provided as a pre-processing unit of the sound source separation processing unit 33, and a non-target sound component having a strong correlation between the L channel and the R channel, which is not good at the sound source separation processing unit 33, is not emitted. By suppressing in advance by the target sound canceller processing unit 32, the function of the sound source separation processing unit 33 is fully exhibited, and at the same time, the non-target sound component that could not be suppressed by the emitted non-target sound canceller processing unit 32 is obtained. Suppression is performed by the sound source separation processing unit 33, and a much higher performance of suppressing non-target sound is obtained than when the sound source separation processing unit 33 is applied alone.

  If the coherence filter method is simply applied as the sound source separation method of the sound source separation processing unit 33, the non-target sound is suppressed from the input sound signals ECoutL (n) and ECoutR (n) from which the emitted non-target sound is removed. What is necessary is just to obtain the suppression coefficient to be used. In this embodiment, not only the input sound signals ECoutL (n) and ECoutR (n) from which the sound non-target sound has been removed, but also the pseudo sound emission target sound signals PSechoL (n) and PSechoR (n) are applied. The suppression coefficient NRcoef (f, K) used for suppressing the non-target sound is obtained. The reason for this is as follows.

  The sound emitted from the speakers 3L and 3R is, for example, music, and includes impact sounds (for example, drum sounds in rock) that suddenly have components at all frequencies, such as percussion instrument sounds. In this case, the follow-up in the emitted non-target sound canceller processing unit 32 (the adaptive filter) is not in time, and the emitted non-target sound cannot be sufficiently suppressed. Moreover, since the impact sound has components at all frequencies, even if the direction of arrival is not the front, the sounds emitted from the left and right speakers 3L and 3R have a strong correlation with each other so that the kite also comes from the front. It has special characteristics. Therefore, when the suppression coefficient used for suppressing the non-target sound is obtained only from the input sound signals ECoutL (n) and ECoutR (n) from which the non-target sound is removed, the non-target sound is the impact sound. Sometimes the removal of the non-target sound is insufficient.

  In order to avoid such an inconvenience, the pseudo sound emission target sound signals PSechoL and PSechoR are also used to form the suppression coefficient NRcoef used for suppressing the non-target sound.

  The pseudo sound emission target sound signals PSechoL and PSechoR calculated by the sound emission non-target sound canceller processing unit 32 are signals obtained by convolving the sound source data sigL and sigR with transfer characteristics from the speakers 3L and 3R to the microphones 4L and 4R. Therefore, it can be said that the microphones 4L and 4R have characteristics close to those of the disturbing sound components included in the input sound signals inputL and inputR captured by the microphones 4L and 4R. Therefore, it can be expected that the suppression performance to the impact sound can be improved by referring to the characteristic amount obtained from the pseudo sound emission target sound signals PSechoL and PSechoR, or the pseudo sound emission target sound signals PSEchoL and PSechoR.

  Therefore, in the first embodiment, the pseudo sound emission target sound signals PSechoL and PSechoR are also used to form the suppression coefficient NRcoef used for suppressing the non-target sound.

  Next, it will be described in more detail that the pseudo sound emission target sound signals PSechoL and PSechoR can be used to form the suppression coefficient NRcoef used for suppressing the non-target sound.

  Considering the device configuration assumed by the first embodiment (see FIGS. 6 and 7 described above), no disturbing sound can come from the front. When this behavior is associated with the behavior of the feature quantity directly linked to the arrival direction such as the coherence described in Patent Document 1, it can be said that the interference sound does not take a coherence value equal to or higher than the target sound coming from the front. it can. However, as described above, when the impact sound includes the impact sound, the correlation between the disturbing sounds emitted from the left and right speakers 3L and 3R is remarkably increased, and it comes from the front even though it is a disturbing sound. Behaves as if That is, the coherence value of the disturbing sound when the impact sound is included is equal to or greater than the target sound. Therefore, the coherence filter method that sets the noise suppression gain according to the direction of arrival of the interference sound cannot sufficiently suppress the interference sound. By the way, the pseudo sound emission target sound signals PSEchoL and PSechoR are sounds obtained by convolution of the transmission characteristics from the speakers 3L and 3R to the microphones 4L and 4R into the sound source data sigL and sigR that become sound non-target sounds when emitted. Therefore, the target sound component is not included, and the signal is derived only from the disturbing sound component coming from the speakers 3L and 3R on both sides. Therefore, the range of the coherence value obtained from the two pseudo sound emission target sound signals PSechoL and PSechoR is smaller than the range of the target sound, and if the disturbing sound source data sigL and sigR include an impact sound, The coherence of the target sound signals PSechoL and PSechoR increases. In other words, it is possible to detect the occurrence of an impact sound by a sudden increase in the coherence of the pseudo sound emission target sound signals PSechoL and PSechoR. By referring to the coherence filter coefficient Ycoef obtained from the pseudo sound emission target sound signals PSechoL and PSechoR, the component of the impact sound can be obtained for each frequency. The coherence filter coefficient Xcoef obtained from the input sound signals ECoutL and ECoutR from which the sound non-target sound output from the sound non-target sound canceller processing unit 32 has been removed is obtained from the pseudo sound output target sound signals PSechoL and PSechoR. By adjusting the coherence filter coefficient Ycoef as shown in the equation (2), the component derived from the impact sound can be removed from the coherence filter coefficient, and a more accurate suppression coefficient Zcoef can be calculated.

  Further, since the coherence is an average of the coherence coefficients for each frequency component normalized by the signal level, it can be calculated without being influenced by the volume of the emitted non-target sound. Therefore, even between songs such as rock and classical music with greatly different volumes, the characteristics can be compared without depending on the volume, and it is possible to eliminate as much as possible that the classical music with a large volume is erroneously determined to be rock. it can.

  The expression (2) has a contrivance to prevent a reduction in accuracy of sound source separation when the sound source data sigL and sigR includes impact sound. However, when the sound source data sigL and sigR do not include impact sound, the contrivance is obtained. However, the accuracy may be affected.

  Therefore, in this embodiment, the calculation method of the suppression coefficient for the non-target sound (equation (2)) is applied depending on whether the sound source data sigL and sigR that are the sound non-target sound include the impact sound. The sound source separation accuracy is improved regardless of whether the sound source data sigL and sigR include impact sound.

(A-2) Operation of the First Embodiment Next, the operation of the sound collection / sound emission device 10 of the first embodiment will be described. In the following description, it is assumed that the sound source data is music data and the target sound is a sound produced by a user located in front of the sound collecting / sound emitting device 10.

  The sound source data (music data) read from the sound source data storage units 21L and 21R are converted into analog signals by the corresponding D / A conversion units 22L and 22R, and then emitted from the speakers 3L and 3R. The When such music is flowing from the sound collecting / sound emitting device 10, the sound produced by the user toward the sound collecting / sound emitting device 10 is captured by both microphones 4 </ b> L and 4 </ b> R. At this time, since music from the speakers 3L and 3R is also flowing, music from the speaker 3L is also captured by both microphones 4L and 4R, and music from the speaker 3R is also captured by both microphones 4L and 4R. Furthermore, ambient background noise (such as driving sound of an air conditioner, traveling sound from a vehicle traveling nearby) is also captured by both microphones 4L and 4R.

  In other words, the input sound signals obtained by the microphones 4L and 4R include non-purpose sounds such as music emitted by the device itself and non-purpose sounds such as background noise, in addition to the target sound of the user's voice. Sound (background non-target sound) is included.

  Input sound signals obtained by the microphones 4L and 4R are converted into digital signals inputL and inputR by the corresponding A / D converters 31L and 31R, respectively, and are supplied to the sound emission non-target sound canceller processing unit 32. The sound emission non-target sound canceller processing unit 32 is also provided with sound source data sigL and sigR.

  In the sound non-target sound canceller processing unit 32, the sound non-target sound is removed by subtracting the internally generated pseudo sound target sound signal PSchoL from the input sound signal (digital signal) inputL related to the L channel. Similarly, by subtracting the internally generated pseudo sound emission target sound signal PSEchoR from the input sound signal (digital signal) inputR relating to the R channel, the sound emission non-purpose sound is obtained. The removed input sound signal ECoutR is obtained. The pair of input sound signals ECoutL and ECoutR obtained by removing the non-target sound output thus obtained are supplied to the sound source separation processing unit 33 together with the pair of internally generated pseudo sound output target sound signals PSechoL and PSechoR. .

  In the sound source separation processing unit 33, the input sound signals ECoutL (n) and ECoutR (n) from which the sound non-target sound, which is a time domain signal, is removed by the FFT unit 41, and the pseudo sound output target sound signal PSchoL ( n) and PSechoR (n) are converted into frequency domain signals XL (f, K), XR (f, K), YL (f, K), YR (f, K), respectively.

  Then, the first coherence coefficient calculation unit 42 performs frequency domain signals XL (f, K) and XR (f) obtained from the input sound signals ECoutL (n) and ECoutR (n) from which the emitted non-target sound has been removed. , K), the coherence coefficient Xcoef (f, K) is calculated, and the second coherence coefficient calculation unit 43 obtains the frequency region obtained from the pseudo sound emission target sound signals PSechoL (n) and PSechoR (n). A coherence coefficient Ycoef (f, K) is calculated based on the signals YL (f, K) and YR (f, K).

  Thereafter, the coherence calculation unit 47 calculates the coherence COH (K) from the second coherence coefficient Ycoef (f, K) obtained by the second coherence coefficient calculation unit 43, and the sound emission non-target sound type determination unit 48. Thus, based on the behavior of the coherence COH (K), it is determined whether or not the sound source data sigL and sigR, which are sound emission non-target sounds, include an impact sound, and the determination result is given to the suppression coefficient calculation unit 44. .

  When the suppression coefficient calculation unit 44 gives a determination result that the sound source data includes an impact sound, the suppression coefficient NRcoef (f, K) is calculated according to the above-described equation (2), while the sound source data includes the impact sound. If the determination result is given, the coherence coefficient Xcoef (f, K) obtained by the first coherence coefficient calculator 42 is directly used as the suppression coefficient NRcoef (f, K). Then, the suppression coefficient NRcoef (f, K) is added to each frequency component in one frequency domain signal XL (f, K) obtained from the input sound signal from which the emitted non-target sound is removed by the suppression coefficient multiplier 45. A frequency domain signal Z (f, K) from which the non-target sound has been removed by multiplication is obtained. By converting the non-target sound suppression signal Z (f, K), which is a frequency domain signal, into the time domain signal z (n) by the IFFT unit 46, an output signal output (= z (n)) including only the target sound component. ) Is obtained.

(A-3) Effects of the First Embodiment According to the first embodiment, the non-target sounds are not collectively captured, but are classified into the sound non-target sounds and the background non-target sounds, and the sound is further emitted. Since the target sound is extracted by determining the sound type of the non-target sound and applying the removal process suitable for each non-target sound, the target sound extraction accuracy can be made extremely high.

  As a result, for example, when the voice that is the extracted target sound component is used for a call, the call sound quality can be improved, and when the voice that is the extracted target sound component is used for voice recognition, the recognition rate is increased. Can do.

(B) Second Embodiment Next, a second embodiment of the sound collecting / sound emitting apparatus and sound collecting / sound emitting program according to the present invention will be described focusing on differences from the first embodiment.

  FIG. 5 is a block diagram illustrating a detailed configuration of a sound emission non-target sound canceller processing unit (hereinafter, reference numeral 32A is used) according to the second embodiment.

  In FIG. 5, the sound emission non-purpose sound canceller processing unit 32A includes four pseudo sound emission non-purpose sound generation units 61LL to 61RR and four subtraction units 62LL to 62RR.

  As described above, the non-target sound emitted from the speakers 3L and 3R and captured by the microphones 4R and 4L can be regarded in the same manner as the acoustic echo that is a problem in telephone communication. In the second embodiment, the sound emission non-target sound canceller processing unit 32A is configured by applying four monaural echo canceller techniques. 4 can also be regarded as belonging to the category of stereo echo canceller (see FIG. 3.73 of Non-Patent Document 1).

  The simulated sound emission non-purpose sound generation unit 61LL simulates the sound emission non-purpose sound that is included in the L channel input sound signal inputL and is emitted from the speaker 3L and captured by the microphone 4L. The sound is generated based on the sound source data sigL, and the subtraction unit 62LL subtracts the pseudo sound emission non-purpose sound generated by the pseudo sound emission non-purpose sound generation unit 61LL from the L channel input sound signal inputL, From the input sound signal inputL, the component of the non-target sound emitted from the speaker 3L and captured by the microphone 4L is removed.

  The simulated sound emission non-purpose sound generation unit 61RL simulates the sound emission non-purpose sound that is included in the L channel input sound signal inputL and is emitted from the speaker 3R and captured by the microphone 4L. The sound is generated based on the sound source data sigR, and the subtracting unit 62RL generates the pseudo sound emitting non-purpose sound generated by the pseudo sound emitting non-purpose sound generating unit 61RL from the output sound signal of the pseudo sound emitting non-purpose sound generating unit 61LL. Subtraction is performed to remove the component of the sound non-target sound emitted from the speaker 3R and captured by the microphone 4L from the output sound signal of the pseudo sound non-purpose sound generation unit 61LL.

  As a result, the input sound signal ECoutL output from the simulated sound emission non-purpose sound generation unit 61RL is emitted from the input sound signal inputL and emitted from the speaker 3L and captured by the microphone 4L. The sound non-target sound component emitted from the speaker 3R and captured by the microphone 4L is excluded.

  The pseudo sound emission non-purpose sound generation unit 61LR simulates the sound non-purpose sound that is emitted from the speaker 3L and captured by the microphone 4R and included in the R channel input sound signal inputR. The sound is generated based on the sound source data sigL, and the subtraction unit 62LR subtracts the pseudo sound emission non-purpose sound generated by the pseudo sound emission non-purpose sound generation unit 61LR from the input sound signal inputR of the R channel, From the input sound signal inputR, the component of the non-target sound emitted from the speaker 3L and captured by the microphone 4R is removed.

  The simulated sound emission non-purpose sound generation unit 61RR simulates the sound emission non-purpose sound that is included in the R channel input sound signal inputL and is emitted from the speaker 3R and captured by the microphone 4R. The sound is generated based on the sound source data sigR, and the subtraction unit 62RR generates the pseudo sound emission non-purpose sound generated by the pseudo sound emission non-purpose sound generation unit 61RR from the output sound signal of the pseudo sound emission non-purpose sound generation unit 61LR. Subtraction is performed to remove the component of the sound non-target sound emitted from the speaker 3R and captured by the microphone 4R from the output sound signal of the pseudo sound non-purpose sound generation unit 61LR.

  As a result, the input sound signal ECoutR output from the simulated sound emission non-purpose sound generation unit 61RR is emitted from the input sound signal inputR and emitted from the speaker 3L and captured by the microphone 4R. The sound non-target sound component emitted from the speaker 3R and captured by the microphone 4R is excluded.

  Each of the simulated sound emission non-target sound generation units 61LL to 61RR is configured by an adaptive filter used in an acoustic echo canceller. Although the adaptive algorithm which these adaptive filters apply is not limited, for example, a learning identification algorithm can be applied.

  When a monaural echo canceller is used, there are two speakers 3L and 3R and two microphones 4L and 4R, so that the acoustic path is congested, and the acoustic path characteristic cannot be accurately estimated, and a sufficient suppression effect may not be obtained.

Therefore, prior to reproduction of the sound source data sigL and sigR, white noise is emitted only from the speaker 3L, and the acoustic path characteristic H _LL from the speaker 3L to the microphone 4L and the acoustic path characteristic H _LR from the speaker 3L to the microphone 4R are obtained. estimates the pseudo sound non-target sound generator 61LL and 61LR (adaptive filter) is, then sound white noise only from the speaker 3R, acoustic path characteristics _{H RL} and the speaker 3R of the speaker 3R to the microphone 4L the acoustic path characteristics _{H RR} to the microphone 4R estimated pseudo sound non-target sound generator 61RL and 61RR (adaptive filter) is from previously initialized.

  Thereafter, the pseudo sound emission non-target sound signal obtained by convolving the four sound path characteristics and the corresponding sound source data sigL and sigR is subtracted from the input sound signal captured by the microphones 4L and 4R, thereby preventing sound emission. The target sound can be suppressed.

  At this time, the sound type determination results of the sound source data sigL and sigR by the sound emission non-purpose sound type determination unit 48 (see FIG. 2) are given to the pseudo sound emission non-purpose sound generation units 61LL to 61RR and used. Each of the simulated sound emission non-purpose sound generation units 61LL to 61RR changes the step size of the adaptive filter according to the sound type determination result. Each of the pseudo sound emission non-target sound generation units 61LL to 61RR has a smaller step size and quicker follow-up in the case of sound source data including impact sound than in the case of sound source data not including impact sound. Like that.

  Also according to the second embodiment, the non-target sounds are not collectively captured, but are classified into the emitted non-purpose sounds and the background non-purpose sounds, and the sound types are determined for the emitted non-purpose sounds. Since the target sound is extracted by applying a suitable removal process, the target sound can be extracted with very high accuracy.

  As a result, for example, when the voice that is the extracted target sound component is used for a call, the call sound quality can be improved, and when the voice that is the extracted target sound component is used for voice recognition, the recognition rate is increased. Can do.

(C) Other Embodiments In the description of each of the above-described embodiments, various modified embodiments have been referred to. However, modified embodiments as exemplified below can be given.

  In each of the above embodiments, the suppression coefficient calculation unit 44 calculates the suppression coefficient according to the expression (2) as necessary. However, the calculation of the expression (2) is performed so that the suppression coefficient does not become too small. You may make it perform a flooring process later. In this way, it is possible to prevent deterioration in sound quality due to excessive suppression.

  In each of the above embodiments, the determination of the sound type is a determination of whether or not the sound source data includes an impact sound, but there are three or more types of determinations such as whether the impact sound is strongly included, weakly included, or not included. In this case, the coefficient α in the equation (2) may be switched depending on how the impact sound is included. The three types of determination methods include two thresholds to be compared with the long-term average and two thresholds to be compared with the variance, and if the long-term average and the variance are larger, the impact sound is It is determined that the sound is strongly included, and when the long-term average and the variance are less than the smaller threshold value, it is determined that the sound is not included, and except for the two cases described above, the method is determined that the sound is weakly included. it can.

  In each of the above embodiments, the arithmetic expression for correcting the first coherence coefficient using the second coherence coefficient is the subtraction shown in the expression (2), but other arithmetic expressions (functions) are The first coherence coefficient may be corrected by applying the second coherence coefficient. For example, the suppression coefficient may be calculated by dividing the first coherence coefficient by a value obtained by multiplying the second coherence coefficient by a coefficient.

  In each of the above-described embodiments, the characteristic amount used for the determination of the sound emission non-target sound (interfering sound) is the coherence variance and the long-term average value. However, if the behavior as shown in FIG. 3 can be distinguished. For example, other statistics may be used. For example, a value obtained by dividing the maximum coherence value by the average value or a coefficient of variation (= standard deviation / average value) may be used as the feature amount.

  Further, the feature amount may be calculated using not the coherence but the coherence coefficient of one or a plurality of frequency components that are not all. Furthermore, the presence / absence of an impact sound and the stage where the impact sound is mixed may be determined based on the power change of the pseudo sound emission non-target sound signal without calculating the coherence coefficient or coherence. Furthermore, the feature value used for the determination is not limited to the feature value obtained from the pseudo sound emission non-target sound signal. For example, instead of the feature amount obtained from the pseudo-non-target sound signal, or in addition to the feature amount obtained from the pseudo-non-target sound signal, the sound non-sound output from the sound non-target sound canceller processing unit is output. You may make it use the feature-value obtained from the input sound signal from which the target sound was removed for determination of the kind of sound of a non-target sound to be emitted.

  In the first embodiment, the sound type determination result is reflected in the suppression coefficient calculation method. In the second embodiment, the sound type determination result is used as the suppression coefficient calculation method and the step size of the adaptive filter. However, the method of using the determination result of the sound type is not limited to these. For example, only the step size of the adaptive filter may be reflected. When another sound source separation method (described later) is applied as the sound source separation method, it is reflected in switching of parameters used in the processing. May be.

  In each of the embodiments described above, the sound source separation processing unit has shown the target sound and the background non-target sound separated according to the coherence filter method, but the separation method is not limited to this. For example, a combination of the coherence filter method and the frequency subtraction method (spectral subtraction method) may be applied, or a combination of the coherence filter method and the Wiener filter method may be applied. The coherence filter method and the frequency A combination of the subtraction method and the Wiener filter method may be applied. In the case of applying the frequency subtraction method, the ratio of subtracting the noise component spectrum from the spectrum of the input voice signal may be changed according to the sound type determination result. Further, when the Wiener filter method is applied, the Wiener filter coefficient to be multiplied with respect to the spectrum of the input audio signal may be changed according to the determination result of the sound type.

  The technical idea of the present invention is also applicable to a sound collecting / sound emitting device that includes only the sound non-target sound canceller processing unit 32 and does not include the sound source separation processing unit 33. For example, a mode in which the sound type of the emitted non-target sound is reflected in the step sizes of the emitted non-purpose sound canceller processing units 32 and 32A can be exemplified.

  In each of the above-described embodiments, the case where there are two speakers is shown, but there may be one speaker or three or more speakers. Also, the number of microphones is not limited to two and may be three or more. The internal configuration of the sound emission non-target sound canceller processing unit 32 may be designed in consideration of the number of sound emission sound paths determined according to the number of speakers and microphones.

  In each of the above-described embodiments, the sound collection / sound emission device alone performs all processing. However, the non-target sound removal processing may be performed by an external server. For example, when the sound collection / sound emission device is a smartphone, the system may be configured by a so-called cloud and updated so that the user does not know the presence of the external server. The claim of “sound collecting / sound emitting device” in the claims includes a case where an external server that is invisible to the user performs processing.

10 ... Sound collecting / sound emitting device,
20 ... Sound emission part, 21L, 21R ... Sound source data storage part, 22L, 22R ... D / A conversion part, 3L, 3R ... Speaker
30 ... Sound collection unit, 4L, 4R ... Microphone, 31L, 31R ... A / D conversion unit, 32, 32A ... Sound emission non-target sound canceller processing unit, 33 ... Sound source separation processing unit, 41 ... FFT unit, 42th 1 coherence coefficient calculation unit, 43 ... second coherence coefficient calculation unit, 44 ... suppression coefficient calculation unit, 45 ... suppression coefficient multiplication unit, 46 ... IFFT unit, 47 ... coherence calculation unit, 48 ... sound emission non-target sound type Determination unit, 61LL to 61RR... Pseudo sound emission non-purpose sound generation unit, 62LL to 62RR.

Claims (6)

In a sound collection / sound emission device having a sound collection unit in which at least two microphones capture ambient sound and a sound emission unit that emits sound from one or more speakers,
A sound signal emitted by the sound emitting unit is input, emitted from the speaker, and generates a pseudo-interfering sound signal that simulates an interfering sound accompanying the emitted sound captured by each of the microphones. A sound emission disturbing sound removing means that diverts an acoustic echo canceller configuration that removes the sound emission disturbing sound captured by each microphone by subtracting from the input sound signal;
Sound type determination means for determining the sound type of the sound emission disturbing sound based on the pseudo sound emission disturbance sound signal generated in the sound emission interference sound removing means;
A sound collecting / sound emitting device comprising: one or a plurality of sound type reflection processing means for switching own processing according to a determination result of the sound type determination means.   The sound type determining means calculates a coherence whose value changes depending on the direction of arrival from the pseudo sound emission disturbing sound signal, and determines the sound type of the sound emission disturbing sound according to the behavior of the coherence. Item 2. The sound collecting and sound emitting device according to Item 1.   The sound collection / sound emission device according to claim 2, wherein the sound type determination means distinguishes the behavior of coherence by a long-term average and variance of coherence.   One of the sound type reflection processing means is sound source separation means for extracting a target sound from a sound source in a predetermined direction from the input sound signal after removal of the sound emission disturbing sound output from the sound emission interference sound removing means. The sound collection / sound emission device according to claim 1, wherein the parameter for separation is switched according to the determination result of the sound type.   The sound emission disturbing sound removing means is one of the sound type reflection processing means, and the sound emission disturbing sound removing means switches the step size of the internal adaptive filter in accordance with the sound type determination result. The sound collecting / sound emitting device according to any one of claims 1 to 4. Sound collection / sound emission executed by a computer mounted in a sound collection / sound emission device having a sound collection unit in which at least two microphones capture ambient sound and a sound emission unit emitting sound from one or more speakers A program,
The above computer
A sound signal emitted by the sound emitting unit is input, emitted from the speaker, and generates a pseudo-interfering sound signal that simulates an interfering sound accompanying the emitted sound captured by each of the microphones. A sound emission disturbing sound removing means that diverts an acoustic echo canceller configuration that removes the sound emission disturbing sound captured by each microphone by subtracting from the input sound signal;
Sound type determination means for determining the sound type of the sound emission disturbing sound based on the pseudo sound emission disturbance sound signal generated in the sound emission interference sound removing means;
A sound collection / sound emission program which functions as one or a plurality of sound type reflection processing means for switching its own processing according to the determination result of the sound type determination means.

Images