JP2017187687A - Sound source separation device, sound source separation method, program and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a sound source separation device that suppresses unwanted sound coming around from other sound sources and separates sound of a main sound source.SOLUTION: The sound source separation device includes: a gain calculation unit 130 for calculating a gain G(ω) from a power spectrum P(ω) of a frequency domain signal X(ω) and a power spectrum P(ω) of a frequency domain signal X(ω) (k being an integer of 1 or more and N or less, except m); and a gain multiplication unit 140 for generating a post-correction frequency domain signal Y(ω) as X(ω) G(ω) from the gain G(ω) and the frequency domain signal X(ω).SELECTED DRAWING: Figure 1

Description

  The present invention relates to a sound source separation technique, and more particularly, to a technique for separating voice collected by a main speaker microphone by suppressing the voice of another speaker that has entered the main speaker microphone.

  In a conference in which many people participate, a plurality of microphones may be prepared and the conference may proceed. For example, as shown in FIG. 13, a plurality of microphones are arranged on the table, and when one participant utters, the utterance is picked up by the plurality of microphones. Further, when a plurality of speakers speak at the same time, a sound signal obtained by mixing these sounds is picked up by each microphone.

  As a technique for separating the main speaker's voice from the voice signal collected by the main speaker's microphone that can be used in such a situation, there is a technique disclosed in Patent Document 1 (see FIG. 14). . In the technique of Patent Document 1, utterance detection and speech recognition are performed independently for each speaker (channel).

Japanese Patent Laying-Open No. 2015-155982

  As described above, in the technique disclosed in Patent Document 1, speech detection processing and speech recognition processing are performed independently for each microphone. In an environment such as a conference room with multiple microphones, a microphone far from the speaker (a microphone other than the microphone for the main speaker who spoke) will contain a lot of noise and reverberation. There was a problem that would occur.

  Therefore, an object of the present invention is to provide a sound source separation device that suppresses unnecessary sounds that have circulated from other sound sources and separates the sound of the main sound source.

In one embodiment of the present invention, N is an integer of 2 or more, m is an integer of 1 to N, and X _n (ω) (n = 1,..., N, ω is a frequency) is collected by a microphone n A sound source separation that generates a corrected frequency domain signal Y _m (ω) from the frequency domain signal X _m (ω) using a frequency domain signal obtained by frequency domain conversion of the signal and a microphone m as a microphone that collects the voice of the main speaker. an apparatus, the power spectrum _{P k (ω) (k} power spectra _{P m} (omega) and the frequency domain signal _{X k} (omega) of the frequency domain signals _{X m} (omega) is 1 or more N or less except for m A gain calculation unit for calculating a gain G _m (ω) from the integer G), and the corrected frequency domain signal Y _m (ω) from the gain G _m (ω) and the frequency domain signal X _m (ω) to X _m (omega) · including a gain multiplication unit that generates a G m _(ω) .

  According to the present invention, it is possible to separate the sounds of the main sound sources by mutually using signals collected by a plurality of microphones and suppressing unnecessary sounds that have circulated from other sound sources. .

FIG. 2 is a block diagram showing a configuration of a sound source separation device 100. 5 is a flowchart showing the operation of the sound source separation device 100. The block diagram which shows the structure of the gain calculation part 130-m. The flowchart which shows operation | movement of the gain calculation part 130-m. The figure which shows an example of a subtraction coefficient table. The block diagram which shows the structure of the gain calculation part 230-m. The block diagram which shows the structure of the stationary noise estimation part 231. FIG. 5 is a flowchart showing the operation of a stationary noise estimation unit 231. FIG. 3 is a block diagram showing a configuration of a sound source separation device 300. The block diagram which shows the structure of the subtraction coefficient update part 330. FIG. 5 is a flowchart showing the operation of a subtraction coefficient update unit 330. FIG. 3 is a block diagram showing a configuration of a sound source separation device 400. The figure which shows the mode of the speech detection which is an example of the utilization scene of this invention. The figure which shows the mode of the process by the technique of patent document 1. FIG.

  Hereinafter, embodiments of the present invention will be described in detail. In addition, the same number is attached | subjected to the structure part which has the same function, and duplication description is abbreviate | omitted.

<Embodiment 1>
Hereinafter, the sound source separation apparatus 100 according to the first embodiment will be described with reference to FIGS. FIG. 1 is a block diagram showing a configuration of the sound source separation device 100. FIG. 2 is a flowchart showing the operation of the sound source separation device 100. As shown in FIG. 1, the sound source separation device 100 includes a frequency domain conversion unit 110-1,..., 110 -N, a power spectrum calculation unit 120-1, 120 -N, and a gain calculation unit 130-1. , 130-N, gain multipliers 140-1, ..., 140-N, time domain converters 150-1, ..., 150-N, and a subtraction coefficient recording unit 190 (where N is 2 or more) An integer). The sound source separation device 100 is connected to microphones 910-1,..., 910 -N in order to collect the voice of the speaker as a sound collection signal.

The microphones 910-1,..., 910 -N collect sound from a plurality of sound sources, for example, utterances of conference participants, and generate a sound collection signal (S 910). The frequency domain transforming units 110-1,..., 110-N perform frequency domain transformation on the collected sound signals collected by the microphones 910-1, ..., 910-N, respectively, to generate frequency domain signals (S110). Hereinafter, the frequency domain signal is represented as X _m (ω) (m = 1,..., N, ω is a frequency). Therefore, X _m (ω) is a complex number representing a frequency domain transformed signal.

The power spectrum calculation unit 120-1, ..., 120-N calculates the power spectrum of the frequency domain signal X _m (ω) that is the output of the frequency domain conversion unit 110-1, ..., 110-N (S120). . Hereinafter, the power spectrum is represented as P _m (ω). For example, it may be calculated as P _m (ω) = X _m (ω) ² . Alternatively, P _m (ω) = | X _m (ω) | may be calculated.

The gain calculators 130-1,..., 130-N are connected to the power spectrum P _m (ω) of the main channel m (m is an integer from 1 to N) corresponding to the voice from the main speaker and the other channels k. The gain G _m (ω) is calculated from the power spectrum P _k (ω) (k is an integer from 1 to N other than _m ) (S130).

  Hereinafter, the gain calculation unit 130-m will be described with reference to FIGS. FIG. 3 is a block diagram illustrating a configuration of a gain calculation unit 130-m that calculates the gain of the main channel m. FIG. 4 is a flowchart showing the operation of the gain calculator 130-m. As shown in FIG. 3, the gain calculation unit 130-m includes N-1 multiplication units 131-1,..., 131- (m−1), 131- (m + 1),. 132, a subtractor 133, and a power spectrum ratio calculator 134.

First, the multiplication unit 131-k (k is an integer from 1 to N other than m) reads the subtraction coefficient α _mk from the subtraction coefficient recording unit 190 and multiplies the power spectrum P _k (ω) by the subtraction coefficient α _mk ( S131). The adding unit 132 calculates the sum Σα _mk · P _k (ω) of α _mk · P _k (ω) (S132). The subtracting unit 133 subtracts the sum Σα _mk · P _k (ω) calculated by the adding unit 132 from the power spectrum P _m (ω) of the main channel m, so that the voice from the main speaker corresponding to the main channel m is obtained. The power spectrum S _m (ω) is calculated (S133). Finally, the power spectrum ratio calculation unit 134 is a ratio S _m (ω) / P _m (ω) between the power spectrum S _m (ω) of the voice from the main speaker and the power spectrum P _m (ω) of the main channel m. Is calculated (S134). This ratio is the gain G _m (ω).

Here, the subtraction coefficient α _mk is a value determined according to the positions of the microphone m and the microphone k. The subtraction coefficient α _mk is obtained in advance and recorded in the subtraction coefficient recording unit 190. An example of the subtraction coefficient table in which the subtraction coefficients are tabulated is shown in FIG. Α _mm is 1.

Subtraction factor alpha _mk, for example, can be calculated as the ratio of the distance r _k from the distance r _m and the utterance location from the utterance location when assuming that the speaking in front of the microphone m to the microphone m to the microphone k .

The gain multipliers 140-1,..., 140-N use the gains G ₁ (ω),..., G _N (ω) calculated by the gain calculators 130-1,. 1,..., 110-N outputs X ₁ (ω),..., X _N (ω) are multiplied and corrected frequency domain signal Y ₁ (ω) = X ₁ (ω) · G ₁ (ω) ,..., Y _N (ω) = X _N (ω) · G _N (ω) is generated (S140). Time domain transforming section 150-1, ..., 150-N, the gain multiplication unit, which is the output of the corrected frequency domain signal _{Y 1 (ω), ...,} Y N (ω) and the time domain conversion, a time-domain signal Generate (S150).

According to the invention of the present embodiment, the power channel P _m (ω) of the main channel is used near the microphone corresponding to the main channel using the sum of values obtained by multiplying the power spectrum other than the main channel by the subtraction coefficient obtained in advance. The power spectrum S _m (ω) of the voice of the main speaker uttered at is extracted, and the frequency domain signal is corrected using the gain G _m (ω) that is the ratio S _m (ω) / P _m (ω). As a result, unnecessary sound contained in the microphone of the main channel can be suppressed, and as a result, the sound of the main sound source (the sound of the main speaker) can be separated.

<Embodiment 2>
In the first embodiment, by subtracting Σα _mk · P _k (ω) calculated using a power spectrum other than the main channel m from the power spectrum P _m (ω) of the main channel m, the main channel m corresponding to the main channel m is subtracted. The power spectrum S _m (ω) of the voice from the speaker was obtained. However, S _m (ω) includes stationary noise such as air-conditioning sound and personal computer fan sound. Therefore, S _m (ω) from which stationary noise has been removed is estimated, and the gain is calculated using this S _m (ω). This makes it possible to output a clear sound excluding the air conditioning and fan sounds.

  The sound source separation device 200 of the second embodiment is different from the sound source separation device 100 only in the configuration of the gain calculation unit. Therefore, the configuration of the gain calculation unit will be described below. The gain calculation unit 230-m will be described with reference to FIG. FIG. 6 is a block diagram illustrating a configuration of a gain calculation unit 230-m that calculates the gain of the main channel m. As shown in FIG. 6, the gain calculation unit 230-m includes N−1 multiplication units 131-1,..., 131- (m−1), 131- (m + 1),. 132, a subtraction unit 133, a power spectrum ratio calculation unit 134, and a stationary noise estimation unit 231. The gain calculation unit 230-m is different from the gain calculation unit 130-m in that it includes a stationary noise estimation unit 231.

The processing in the multiplication unit 131-k (k is an integer from 1 to N other than m), the subtraction unit 133, and the power spectrum ratio calculation unit 134 is exactly the same as that in the first embodiment. The processing in the adding unit 132 is exactly the same except that the stationary noise component R _m (ω) that is the output of the stationary noise estimating unit 231 is also added. Therefore, the power spectrum S _m (ω) of the voice from the main speaker corresponding to the main channel m, which is the output of the subtracting unit 133, and the gain G _m (ω), which is the output of the power spectrum ratio calculating unit 134, are as follows: It is expressed by the following formula.

  Therefore, hereinafter, the stationary noise estimation unit 231 will be described with reference to FIGS. FIG. 7 is a block diagram illustrating a configuration of the stationary noise estimation unit 231. FIG. 8 is a flowchart showing the operation of the stationary noise estimation unit 231. As shown in FIG. 7, the stationary noise estimation unit 231 includes a time average power calculation unit 235, a dip hold processing unit 236, and a weight recording unit 239.

The stationary noise estimation unit 231 estimates the stationary noise component R _m (ω) included in the power spectrum P _m (ω). First, the time average power calculation unit 235, the power spectrum _{P m} (omega) average from the time average for a period of time power spectrum P ^- calculating the _{m (ω)} (S235). The dip hold processing unit 236 performs dip hold processing for holding the minimum value of P _m (ω), reads the weights β and γ from the weight recording unit 239, and obtains the dip hold power D _m (ω) by the following equation. Finally, the stationary noise component R _m (ω) is estimated (S236).

  However, both β and γ are real numbers of 0 or more and 1 or less and satisfy β> γ.

The closer the β or γ is to 1, the longer the smoothed power is calculated. Time-averaged power spectrum P ^- _m a coefficient β to be applied is larger than _(omega) dip hold power D _{m (ω),} the time-averaged power spectrum P ^- _{m (ω)} dip hold power D _{m (omega)} By making it larger than the coefficient γ applied in the following cases, a gradual change occurs when the power increases, and a rapid power change occurs when the power decreases. Thus dip hold power after calculation D _{m (omega)} is the time-average power spectrum P ^- can be the minimum power in the vicinity of _{m (omega),} to estimate the power near stationary noise.

The stationary noise component R _m (ω) can be calculated by multiplying the dip hold power D _m (ω) by a fixed coefficient λ set in advance.

According to the invention of the present embodiment, it is possible to suppress the stationary noise by calculating the gain using the power spectrum S _m (ω) of the voice from the main speaker from which the stationary noise has been removed. A clear voice can be separated even in an environment where stationary noise exists.

<Embodiment 3>
In the first embodiment, it is assumed that the subtraction coefficient is recorded in the subtraction coefficient recording unit 190 in advance. However, if the subtraction coefficient is fixed to a preset value in this way, the power spectrum S _m (ω) of the voice from the main speaker is changed to P _m (when the relative positional relationship of the microphone changes, such as a layout change. ω) cannot be calculated correctly. Therefore, the ratio of the average of the power spectrum of the channel where the speech is detected and the average of the power spectra of all the channels including the channel is appropriately calculated, and the subtraction coefficient table is updated using this ratio as a subtraction coefficient. This makes it possible to correctly calculate the power spectrum S _m (ω) of the voice from the main speaker even when the relative positional relationship between the microphones varies.

FIG. 9 shows the configuration of the sound source separation device 300 according to the third embodiment. The sound source separation device 300 of the third embodiment is different from the sound source separation device 100 in that it includes a subtraction coefficient updating unit 330. The subtraction coefficient updating unit 330 calculates the subtraction coefficient α _mn from the power spectrum P _n (ω) (n = 1,..., N) and the channel number m of the channel in which the speech is detected, and records it in the subtraction coefficient recording unit 190. The mth row of the subtraction coefficient table is updated (see FIG. 5). The channel number in which the utterance is detected may be specified by detecting an event in which the speaker switches on the microphone.

  Hereinafter, the subtraction coefficient updating unit 330 will be described with reference to FIGS. FIG. 10 is a block diagram illustrating a configuration of the subtraction coefficient updating unit 330. FIG. 11 is a flowchart showing the operation of the subtraction coefficient updating unit 330. As shown in FIG. 10, the subtraction coefficient updating unit 330 includes an average power calculation unit 331 and a power ratio calculation unit 332.

The average power calculation unit 331 calculates an average power spectrum Q _n that is averaged in time frequency from the power spectrum P _n (ω) (n = 1,..., N) (S331). The power ratio calculation unit 332 calculates the average power spectrum ratio Q _n / Q _m using the average power spectrum Q _m of the input channel number m as the denominator and the average power spectrum Q _n of the channel number n including _m as the numerator. (S332). The subtraction coefficient recording unit 190 is updated with this as the subtraction coefficient α _mn (S190).

It is assumed that the initial value of the subtraction coefficient α _mn is set in advance as in the first embodiment.

According to the invention of the present embodiment, the subtraction coefficient α _mn that needs to be determined according to the positions of the microphone m and the microphone n is the value of the microphone signal including the average power spectrum Q _m and m of the microphone signal that has spoken. it is possible to appropriately updated using the average power spectrum Q _n, and any modifications to the arrangement of the microphone, it is possible to correct subtraction coefficient according to the relative positional relationship is set.

<Application example>
By combining the sound source separation apparatus described in the first to third embodiments with an utterance detection technique, it is possible to output only a time domain signal that is an output of a channel in which an utterance is detected.

  FIG. 12 shows a sound source separation device 400 provided with an utterance detection unit. The sound source separation device 400 includes an utterance detection unit 410 and an output channel selection unit 420 in addition to the sound source separation devices 100/200/300 of the first to third embodiments.

  The utterance detection unit 410 identifies the channel number m of the channel that detected the utterance from the collected sound signal. Further, as described above, the channel number m may be specified by using an event signal for the speaker to switch on the microphone as an input instead of the sound pickup signal. Output channel selection section 420 selects a time domain signal to be finally output using channel number m, and outputs a time domain signal of channel number m.

  In this way, by combining the speech detection technology for identifying the speaker and the sound source separation technology, it is possible to output only the voice of the main speaker of the channel in which the speech is detected. Accordingly, the output of unnecessary channels can be made zero, and for example, when applied to speech recognition in a conference system having a plurality of microphones, speech recognition can be performed more accurately. Also, when applied to recording, useless recording can be avoided and the amount of recorded data can be reduced.

<Supplementary note>
The apparatus of the present invention includes, for example, a single hardware entity as an input unit to which a keyboard or the like can be connected, an output unit to which a liquid crystal display or the like can be connected, and a communication device (for example, a communication cable) capable of communicating outside the hardware entity. Can be connected to a communication unit, a CPU (Central Processing Unit, may include a cache memory or a register), a RAM or ROM that is a memory, an external storage device that is a hard disk, and an input unit, an output unit, or a communication unit thereof , A CPU, a RAM, a ROM, and a bus connected so that data can be exchanged between the external storage devices. If necessary, the hardware entity may be provided with a device (drive) that can read and write a recording medium such as a CD-ROM. A physical entity having such hardware resources includes a general-purpose computer.

  The external storage device of the hardware entity stores a program necessary for realizing the above functions and data necessary for processing the program (not limited to the external storage device, for example, reading a program) It may be stored in a ROM that is a dedicated storage device). Data obtained by the processing of these programs is appropriately stored in a RAM or an external storage device.

  In the hardware entity, each program stored in an external storage device (or ROM or the like) and data necessary for processing each program are read into a memory as necessary, and are interpreted and executed by a CPU as appropriate. . As a result, the CPU realizes a predetermined function (respective component requirements expressed as the above-described unit, unit, etc.).

  The present invention is not limited to the above-described embodiment, and can be appropriately changed without departing from the spirit of the present invention. In addition, the processing described in the above embodiment may be executed not only in time series according to the order of description but also in parallel or individually as required by the processing capability of the apparatus that executes the processing. .

  As described above, when the processing functions in the hardware entity (the apparatus of the present invention) described in the above embodiments are realized by a computer, the processing contents of the functions that the hardware entity should have are described by a program. Then, by executing this program on a computer, the processing functions in the hardware entity are realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only). Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto-Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, the computer reads a program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to the computer. Each time, the processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In this embodiment, a hardware entity is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

100 sound source separation device 110 frequency domain conversion unit 120 power spectrum calculation unit 130 gain calculation unit 131 multiplication unit 132 addition unit 133 subtraction unit 134 power spectrum ratio calculation unit 140 gain multiplication unit 150 time domain conversion unit 190 subtraction coefficient recording unit 200 sound source separation Device 230 Gain calculation unit 231 Stationary noise estimation unit 235 Time average power calculation unit 236 Dip hold processing unit 239 Weight recording unit 300 Sound source separation device 330 Subtraction coefficient update unit 331 Average power calculation unit 332 Power ratio calculation unit 400 Sound source separation device 410 Detection unit 420 Output channel selection unit 910 Microphone

Claims (7)

N is an integer greater than or equal to 2, m is an integer greater than or equal to 1 and less than or equal to N, and X _n (ω) (n = 1,... The area signal, microphone m is the microphone that picks up the voice of the main speaker,
A sound source separation device that generates a corrected frequency domain signal Y _m (ω) from the frequency domain signal X _m (ω),
Gain from the power spectrum P _m (ω) of the frequency domain signal X _m (ω) and the power spectrum P _k (ω) of the frequency domain signal X _k (ω) (k is an integer from 1 to N excluding m) A gain calculation unit for calculating G _m (ω);
A gain multiplier that generates the corrected frequency domain signal Y _m (ω) as X _m (ω) · G _m (ω) from the gain G _m (ω) and the frequency domain signal X _m (ω). Sound source separation device. The sound source separation device according to claim 1,
α _mk is a subtraction coefficient determined according to the relative position of the microphone m and the microphone k,
The gain calculator is

A sound source separation device that calculates the gain G _m (ω) by: The sound source separation device according to claim 1,
α _mk is a subtraction coefficient determined according to the relative position of the microphone m and the microphone k, and R _m (ω) is a stationary noise component collected by the microphone m.
The gain calculator is

A sound source separation device that calculates the gain G _m (ω) by: The sound source separation device according to claim 2 or 3,
further,
A sound source separation device including a subtraction coefficient updating unit that updates a ratio Q _n / Q _m calculated from an average power spectrum Q _n which is a time frequency average of the power spectrum P _n (ω) as the subtraction coefficient α _mn . N is an integer greater than or equal to 2, m is an integer greater than or equal to 1 and less than or equal to N, and X _n (ω) (n = 1,... The area signal, microphone m is the microphone that picks up the voice of the main speaker,
A sound source separation apparatus including a gain calculation unit and a gain multiplication unit, which generates a corrected frequency domain signal Y _m (ω) from the frequency domain signal X _m (ω),
The gain calculation section, the power spectrum _{P k (ω) (k} power spectra _{P m} (omega) and the frequency domain signal _{X k} (omega) of the frequency domain signals _{X m} (omega) is 1 or more, excluding the m N A gain calculating step for calculating the gain G _m (ω) from the following integer):
The gain multiplier generates the corrected frequency domain signal Y _m (ω) as X _m (ω) · G _m (ω) from the gain G _m (ω) and the frequency domain signal X _m (ω). A sound source separation method including a gain multiplication step.   The program for functioning a computer as a sound source separation apparatus of any one of Claims 1 thru | or 4.   A computer-readable recording medium on which any one of the programs according to claim 6 is recorded.

Images