JP4964204B2 - Multiple signal section estimation device, multiple signal section estimation method, program thereof, and recording medium - Google Patents

Description

  The present invention belongs to the technical field of signal processing. In particular, the present invention relates to a multi-signal section estimation apparatus, a multi-signal section estimation method, a program thereof, and a recording medium for estimating a section in which each person's speech signal is emitted for acoustic data in which a plurality of persons' speech signals are mixed.

  Voice segment detection technology that uses multiple microphones to record conversations by multiple people and estimates "when and who spoke", for example, automatically assigning a speaker to each utterance, This is useful when speaker information is added to the recorded data to make it easier to search or find the recorded data.

  Examples of conventional speech segment detection techniques include the methods disclosed in Patent Document 1, Non-Patent Document 1, and the like. FIG. 11 shows an example of a functional configuration of the conventional multiple signal section estimation apparatus 100, and FIG. The multiple signal section estimation apparatus 100 includes a frequency domain transform unit 110, a speech section estimation unit 120, an arrival direction estimation unit 130, and an arrival direction classification unit 140.

The frequency domain transform unit 110 cuts out (cuts out) the time domain observation signals x _j (t) (j = 1,..., M) recorded by M microphones, for example, every 32 ms. One section is hereinafter referred to as “frame”), and for each cut out frame (index is τ), the frequency domain observation signal x _j (f, τ) (f = 1,..., L (S1).

ここで、λ_N(f)は周波数ｆにおけるノイズの平均パワー（音声が明らかに存在しない録音ファイルの冒頭区間などで求める）、ｘ(f,τ)はＭ本のマイクにおける周波数領域の観測信号ｘ₁(f,τ)〜ｘ_M(f,τ)の中から任意に選んだいずれか１本についての周波数領域の観測信号である。なお、ｘ(f,τ)はすべてのマイクの振幅の平均値として次のように求めても構わない。

音声区間推定部１２０は、式(1)により求めた音声存在確率ｐ_V(τ)をそのまま出力してもよいし、ｐ_V(τ)がある閾値より大きければそのフレームは音声区間Ｐ_Sであると判定し、小さければ非音声（ノイズ）区間Ｐ_Nと判定して結果を出力してもよい。 The speech segment estimation unit 120 estimates whether or not speech exists in each frame of the observation signal converted into the frequency domain by the frequency domain transform unit 110 by calculating a speech presence probability (S2). In calculating the speech existence probability, for example, methods described in Non-Patent Document 2 and Non-Patent Document 3 can be used. In the former case, the speech existence probability p _V (τ) in the corresponding frame is obtained by the following equation.

Here, λ _N (f) is the average noise power at frequency f (obtained at the beginning of a recording file in which no sound is clearly present), and x (f, τ) is the frequency domain observation signal of M microphones. This is an observation signal in the frequency domain for any one arbitrarily selected from x ₁ (f, τ) to x _M (f, τ). Note that x (f, τ) may be obtained as follows as an average value of amplitudes of all microphones.

The speech segment estimation unit 120 may output the speech existence probability p _V (τ) obtained by the equation (1) as it is, or if p _V (τ) is larger than a certain threshold, the frame is the speech segment P _S. If it is determined that it is present, and if it is smaller, it may be determined as a non-speech (noise) section _PN and the result may be output.

The arrival direction estimation unit 130 estimates the arrival direction of the observation signal converted into the frequency domain by the frequency domain conversion unit 110 for each frame or for each frequency component of each frame (S3). Specifically, the arrival time difference q ′ _{jj ′} between the microphones j and j ′ of the observation signal is obtained for all microphone pairs, and the speech arrival direction vector is estimated from the vertical vector in which they are arranged and the coordinate system of the microphone. .

これをすべてのマイクペアｊｊ´について求めて、それらを並べた縦ベクトルをvq´(τ)とする。なお、すべてのマイクペアを用いる代わりに、ある基準マイクを決め、基準マイクとその他のマイクに関するすべてのペアを用いてもよい。音声到来方向ベクトルvq(τ)は、vq´(τ)と音速ｃとマイクの座標系VDとから次式により推定する。
vq(τ)＝ｃ・VD⁺・vq´(τ) (4)
ここで、^＋はMoore-Penroseの疑似逆行列を表し、vd_jがマイクｊの座標を[x,y,z]と並べたベクトルであるとき、VD＝[vd₁−vd_j,・・・，vd_M−vd_j]^Tである。このように求めた音声到来方向ベクトルvq(τ)は、到来方向の水平角がθ、仰角がφとすると、次式のように表すことができる。
vq(τ)＝[cosθ・cosφ，sinθ・cosφ，sinφ]^T (5) As a technique for calculating the arrival time difference q ′ _{jj ′} for each frame, there is a technique called GCC-PHAT disclosed in Non-Patent Document 4. In this method, the arrival time difference q ′ _{jj ′} (τ) is calculated according to the following equation.

This is obtained for all microphone pairs jj ′, and the vertical vector in which they are arranged is defined as vq ′ (τ). Instead of using all microphone pairs, a certain reference microphone may be determined, and all pairs related to the reference microphone and other microphones may be used. The voice arrival direction vector vq (τ) is estimated by the following equation from vq ′ (τ), the sound velocity c, and the microphone coordinate system VD.
vq (τ) ＝ c ・ VD ⁺・ vq´ (τ) (4)
Here, ⁺ represents the Moore-Penrose pseudo-inverse matrix, and when vd _j is a vector in which the coordinates of microphone j are aligned with [x, y, z], VD = [vd ₁ −vd _j ,. , Vd _M −vd _j ] ^T. The voice arrival direction vector vq (τ) obtained in this way can be expressed by the following equation, where the horizontal angle of the arrival direction is θ and the elevation angle is φ.
vq (τ) = [cosθ ・ cosφ, sinθ ・ cosφ, sinφ] ^T (5)

これをすべてのマイクペアｊｊ´について求めて（又は上記のように基準マイクに対して求めて）、それらを並べた縦ベクトルをvq´(f,τ)とし、式(4)と同様にして音声到来方向ベクトルvq (f,τ)を推定する。 When the arrival time difference q ′ _{jj ′} is calculated for each frequency component of each frame, the arrival time difference q ′ _{jj ′} (f, τ) between the microphone j and the microphone j _′ is calculated according to the following equation.

This is obtained for all microphone pairs jj ′ (or obtained with respect to the reference microphone as described above), and the vertical vector in which they are arranged is set as vq ′ (f, τ), and the voice is obtained in the same manner as in equation (4). An arrival direction vector vq (f, τ) is estimated.

  Note that the process of the speech segment estimation unit 120 and the process of the arrival direction estimation unit 130 may be performed in parallel, or the speech segment is estimated by the process of the speech segment estimation unit 120 and corresponds to the speech segment. The process of the arrival direction estimation unit 130 may be performed by focusing on the frame.

The direction-of-arrival classification unit 140 determines that each frame corresponding to the speech section P _S has a similar speech arrival direction (vector vq (τ) or vq (f, τ)) for each speaker section P _k (k = 1). ,..., N), and for each cluster, a set of the cluster index k and the indexes τ of all frames belonging to the cluster is output (S4).

As a clustering method, a known k-means method or hierarchical clustering may be used, or online clustering may be used (see Non-Patent Document 5). The speaker C in which the cluster C _k classified by the clustering process is in the angular direction indicated by the centroid obtained from the cluster members (vector vq (τ) or vq (f, τ)) that form the cluster. Each frame τ corresponding to the cluster member constitutes a speaker section P _k by the speaker _k .

In the above description, the arrival direction estimation unit 130 estimates the arrival time difference vector vq ′ (τ) or vq ′ (f, τ) between the microphones, and further, the voice arrival direction vector vq (τ) or vq ( f, τ) is estimated, but it is also possible to simply estimate the arrival time difference vector. Therefore, in this case, as shown in FIG. 13, the arrival direction estimation unit 130 is configured as an arrival time difference estimation unit 131, and the arrival direction classification unit 140 serves as an arrival time difference classification unit 141 as vq (τ) or vq (f, τ). Instead, vq ′ (τ) or vq ′ (f, τ) may be classified.
Special Table 2000-512108 S.Araki, M.Fujimoto, K.Ishizuka, H.Sawada and S.Makino, "Speaker indexing and speech enhancement in real meetings / conversations," IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP-2008), 2008, p.93-96 J.Sohn, NSKim and W.Sung, "A Statistical Model-Based Voice Activity Detection," IEEE Signal Processing letters, 1999, vol.6, no.1, p.1-3 Fujimoto, Ishizuka, Nakatani, “Examination and Consideration of Adaptive Integration of Multiple Voice Interval Detection Methods”, IEICE Technical Committee on Speech, 2007, SP2007-97, p.7-12 CHKnapp and GCCarter, "The generalized correlation method for estimation of time delay," IEEE Trans. Acoust. Speech and Signal Processing, 1976, vol.24, no.4, p.320-327 RODuda, PEHart and DGStork, "Pattern Classification," 2nd edition, Wiley Interscience, 2000

In the prior art, speaker identification was performed based only on voice direction-of-arrival information, so if a speaker who was in one location moved to another location, a new story was spoken even though it was the same speaker. There is a problem of being identified as a speaker, or being misidentified as another speaker who was previously in that position despite being a new speaker.
An object of the present invention is to provide a multi-signal section estimation device, a multi-signal, and a multi-signal section estimation device capable of assigning the same index to the same speaker before and after the movement even if the movement of the speaker position occurs during audio recording. An object is to provide a section estimation method, a program thereof, and a recording medium.

  The multi-signal section estimation device of the present invention estimates each speaker's utterance section from observation signals that are recorded by a plurality of microphones and includes speech uttered by a plurality of speakers, and is a frequency domain transform. A speech section estimation unit, an arrival direction estimation unit, an arrival direction classification unit, and a speaker identification unit.

The frequency domain transform unit sequentially cuts the observation signal into frames of a predetermined length, and transforms the frames into the frequency domain for each frame.
The speech segment estimation unit estimates whether each frame corresponds to a speech segment based on the observation signal converted into the frequency domain.
The arrival direction estimation unit estimates the arrival direction of the observation signal for each frame based on the observation signal converted into the frequency domain.

The arrival direction classification unit classifies each frame estimated to correspond to the speech section into a cluster for each speaker based on the similarity of the arrival directions.
Then, the speaker identification unit creates a model of the speaker related to the cluster for each cluster based on the observation signal converted into the frequency domain of each frame classified into the same cluster by a predetermined time. The speaker of the observation signal after the time is estimated based on the model of each speaker.

  According to the multi-signal section estimation apparatus of the present invention, it is possible to determine speaker identity in addition to estimation and classification of voice arrival directions when estimating speaker sections for a plurality of speakers. Therefore, even if the speaker position moves during recording of the voice, the same index can be assigned to the same speaker before and after the movement.

[First Embodiment]
FIG. 1 (solid line part) shows an example of the functional configuration of the multiple signal section estimation apparatus 200 of the present invention, and FIG. 2 (solid line part) shows an example of the processing flow. The multiple signal section estimation device 200 includes the frequency domain conversion section 110, the speech section estimation section 120, the arrival direction estimation section 130, the arrival direction classification section 140, and the speaker identification section 250 described in the background art. . Moreover, the process of the speaker identification part 250 is performed following S4 of the flow shown in FIG. Therefore, the explanation of the contents explained as the background art is assumed to be a minimum here, and the explanation will be given with emphasis on the processing in the speaker identification unit 250.
FIG. 3 (solid line portion) shows a functional configuration example of the speaker identification unit 250. The speaker identification unit 250 includes feature extraction means 251, model learning means 252, and likelihood calculation means 253.

In the processing of the speaker identification unit 250, it is assumed that there is no movement of the speaker position from the start of recording of the observation signal to a predetermined time t _train , and the model M _{k of} each speaker is created from the cluster created during that time. Let's create. Then, the time t _train after assuming obtained has moved the position of the speaker, for all the speech segments of time t _train after (consecutive frames classified into the same cluster), to the speaker the time t _train before Which speaker has spoken is determined based on the model of each speaker created in the initial part of the observation signal (from the start of recording until time t _train ). Thus, by creating each speaker's model at the initial part of the observation signal, the speaker can be identified without preparing the speaker's model in advance after time _ttrain . Note that t _train is set to a time after the point when all speakers to be identified speak at least once.

The feature extraction unit 251 selects any one observation signal x (f, τ) arbitrarily selected from the frequency domain observation signals x ₁ (f, τ) to x _M (f, τ) in the _M microphones. ) Voice feature vector vf (τ) is calculated for each frame (S5). As the speech feature vector vf (τ), for example, a 12-dimensional MFCC (Mel-Frequency Cepstrum Coefficient) can be used. Further, the fundamental frequency F0 (τ) estimated by the autocorrelation method or the like may be used together and included as a component of the speech feature vector vf (τ).

The model learning means 252 is predetermined from the start of recording the observation signal among the frames classified into the same cluster C _k (when the number of speakers is N, k = 1,..., N) by the arrival direction classification unit 140. Using the speech feature vector vf (τ) related to each frame up to time t _train, a model of speaker k, that is, model parameter φ _k is created and output, and each frame up to a predetermined time t _train is output. A pair of the index τ and the index k of the speaker related to the cluster to which each index belongs is output (S6). In addition, when frames of the same speaker are continuous, the start time and end time of the continuous frames may be output instead of outputting the index of each frame.

ここで、ｐ_ｋ,ｍ(vf(τ))は話者ｋのｍ番目の多次元（次元数ｄは音声特徴量ベクトルの次元と同じ）ガウシアンを表している。Ｍ_ｇは例えば１０とする。モデルパラメータφ_ｋは、ＥＭアルゴリズムなどを用いて、所定の時刻ｔ_trainまでのクラスタＣ_ｋに属する全てのフレームに基づき、次式によって求められる対数尤度Ｌが最大となるφ_ｋの値として計算することができる。

ここで、ＥＭアルゴリズムは、「汪他、”計算統計Ｉ〜確率計算の新しい手法〜”、岩波書店、2003、p158-162」等にて公知の技術である。 As a speaker model, a mixed normal distribution (GMM: Gaussian Mixture Model) is used here, but other speaker identification and speaker recognition methods (such as hidden Markov models and vector quantization) are used. May be. When the number of Gaussians in the GMM is M _g, and the model parameters of the model M _k are φ _k = (average μ _{k, m} , covariance matrix Σ _{k, m} , Gaussian weight w _{k, m} ), the GMM is It can be expressed as:

Here, p _{k, m} (vf (τ)) represents the m-th multidimensional of the speaker k (the dimension number d is the same as the dimension of the speech feature vector) Gaussian. For example, _Mg is 10. The model parameter φ _k is calculated as a value of φ _k that maximizes the log likelihood L obtained by the following equation based on all the frames belonging to the cluster C _k up to a predetermined time t _train using an EM algorithm or the like. can do.

Here, the EM algorithm is a well-known technique such as “Tatsumi et al.,“ Calculation Statistics I—A New Method for Probability Calculation ”, Iwanami Shoten, 2003, p158-162”.

In the model learning unit, in improving the estimation accuracy of the model parameter phi _k, it is desirable that each frame τ are connected to each other. Therefore, an example of a processing method when not connected will be described. FIG. 4A shows an example of a time series of the arrival direction of the observation signal. In this example, the arrival direction changes in the order of θ ₁ → θ ₂ → θ ₃ → θ ₂ → θ ₁ from the start of recording to time t _train , that is, speaker 1 → speaker 2 → speaker 3. This is the case where the speaker 2 speaks in the order of the speaker 2 → the speaker 1. Among them, the speaker 3 speaks a total of three times with a short gap. In such a case where there is a short gap (for example, 300 ms or less), it is desirable to learn the model on the assumption that speech sections are continuous as shown in FIG. For speaker 1 and speaker 2, the interval between the first utterance and the second utterance is wide. In such a case, as shown in FIG. 4B, the model is learned on the assumption that the first utterance and the second utterance are integrated. It should be noted that the index τ output from the model learning unit 252 is τ before connection.

Likelihood calculation means 253 uses all the speakers created in model learning means 252 for speech features of mutually connected frames (hereinafter referred to as “segments”) classified into the same cluster after a predetermined time t _train. The likelihood of the model is calculated, and the index k of the speaker related to the model having the maximum likelihood and the indices τ of all frames included in the segment are output (S7). In addition, when frames of the same speaker are continuous, the start time and end time of the continuous frames may be output instead of outputting the index of each frame.

  When GMM is used as the speaker model, the log likelihood is calculated by Equation (10) by substituting the speech feature vector vf (τ) of all frames τ included in the segment into each speaker model. Then, the index k of the model having the largest log likelihood is assigned as the speaker index of the segment. Note that speaker identification is not necessarily performed for each segment, and may be performed for each frame. In this case, the log likelihood is calculated by an expression obtained by removing Σ in Expression (10).

  As described above, in the present invention, speaker estimation is performed in addition to estimation and classification of voice arrival directions when estimating speaker sections for a plurality of speakers. Therefore, even if the speaker position moves during recording of the voice, the same index can be assigned to the same speaker before and after the movement.

[Second Embodiment]
In the first embodiment, the frequency domain observation signal x (f, τ) output from the frequency domain conversion unit 110 is used as it is in the processing in the feature extraction unit 251. However, in an actual conference, a plurality of speakers often speak at the same time, but each frame needs to be identified as one speaker's speech, and the other speaker's speech becomes a noise component. Therefore, if the observed signal x (f, τ) in the simultaneously-spoken frame τ is used as it is, the feature model may not be appropriately extracted due to the small SN ratio, and the speaker model estimation accuracy may deteriorate. Therefore, in the second embodiment, a functional configuration / processing method for improving the SN ratio will be described.

  The difference in the functional configuration from the first embodiment is that the configuration of the dotted line portion in FIG. 1 is further added, that is, the voice enhancement unit 260 is added. The difference in the processing flow is that the processing of the dotted line portion is further added in FIG. In the point.

In the speech enhancement unit 260, the speech signal component of each speaker k is enhanced. Here, a known beamforming method using observation signals from a plurality of microphones (for example, see Reference 1) may be used, or a method for processing observation signals from one microphone (for example, A noise-removal method using Wiener Filter may be used.
[Reference 1] S. Araki, H. Sawada and S. Makino, "Blind Speech Separation in a Meeting Situation with Maximum SNR beamformers," proc. Of ICASSP2007, 2007, vol.I, p.41-45

In the case of the S / N maximization type beamformer of Reference Document 1, the observation signal vector vx (f, τ) = [x ₁ (f, τ),..., x _M (f, τ)] ^T and the information on the frame τ belonging to each cluster C _k from the arrival direction classifying unit 140, and the speaker related to the cluster C _k to which each frame τ belongs. A frequency domain signal y _k (f, τ) in which the utterance signal component of k is emphasized is generated (S8), and this is used for processing in the feature extraction means 251 instead of x (f, τ).

  Thus, by adding the processing by the speech enhancement unit 260 to the configuration of the first embodiment, the SN ratio of the utterance signal component of each speaker k input to the feature extraction unit 251 can be improved. The estimation accuracy can be increased.

[Third Embodiment]
In the above embodiment, determined by the observation signal of the model parameters phi _k until time t _train, fixedly applied it to the time t _train after the speaker identification process. However, the acoustic environment in which the conversation is recorded usually changes over time, and the obtained model parameter φ _k may become unsuitable for the environment over time.

The third embodiment is a configuration for avoiding such a situation, and an example of a processing flow is shown in FIG. After assigning a speaker index k to a segment after time t _{train in} S7, the likelihood calculation of the speech feature vector vf (τ) of each frame belonging to the segment is calculated as shown by the one-dot chain line in FIG. Feedback is made from the means 254 to the model learning means 253, and φ _k is calculated again by Equation (10) using these speech feature vector vf (τ) to update the model parameters (S9). Updates may be performed sequentially or at predetermined update intervals.

  With this configuration, even if the acoustic environment in which the conversation is recorded changes over time, speaker identification processing can be performed using appropriate model parameters.

[Fourth Embodiment]
In each of the embodiments described above, speaker identification in the likelihood calculation means 253 is performed by substituting the speech feature vector vf (τ) of all frames τ included in the identification target segment into each speaker model M _k. The log likelihood is calculated, and this is performed under the rule that the index k of the model having the maximum log likelihood is the speaker index of the segment. However, under such rules, even if there is an utterance by a newly joined speaker, one of the models of the speaker who participated from the beginning will have the maximum log likelihood, It will be identified as the speaker of the model.

The fourth embodiment is a configuration for avoiding such a situation. An example of the processing flow is shown in FIG. The likelihood calculating means 253 calculates the log likelihood by substituting the speech feature vector into each speaker model for each segment after the predetermined time t _train (S7-1), and the maximum log likelihood is predetermined. If it is larger than the threshold, the speaker's index k related to the model having the maximum likelihood and the indexes τ of all the frames included in the segment are output (S7-). 2) If it is smaller than the threshold, it is determined that a new speaker has joined, and a new speaker index is assigned to the segment, and the speech feature vector vf (τ) of each frame belonging to the segment is assigned. , is fed back to the model learning unit 253 from the likelihood calculating unit 254 as shown in dashed line in FIG. 3, with these audio feature vector vf (tau), calculate phi _k by equation (10) Te be added as model parameters for a new speaker (S10).

  With this configuration, even when a new speaker joins, by detecting it and generating a model of the speaker, the identification process can be performed for the speaker thereafter.

[Fifth Embodiment]
Each of the above embodiments is obtained by observing the signal of the model parameters until time t _train, is configured to perform time t _train subsequent speaker identification process using it. However, if a plurality of speaker voices that are supposed to be uttered can be obtained in advance, a model of each speaker is prepared in advance based on that and speaker identification processing is performed using the model prepared in advance. It is possible.
The fifth embodiment is configured in such a case, and can be realized by configuring the speaker identification unit 250 as shown in FIG. 7, for example. A difference in functional configuration from the above embodiments is that the model learning means 252 in FIG. 3 is replaced with a speaker model DB 264 in which model parameters of a speaker prepared in advance are stored.

With this configuration, it is not necessary to obtain model parameters by learning, so that the speaker calculation can be performed by the likelihood calculation means 253 from the beginning of voice recording. Further, by storing the speaker name information in the DB in association with the speaker model parameter, the speaker index k can have the speaker name information in addition to the direction information.
When the configuration of the multiple signal section estimation device of each of the above embodiments is realized by a computer, the processing contents of the functions that each device should have are described by a program. The processing functions are realized on the computer by executing the program on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. The computer-readable recording medium may be any medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, or a semiconductor memory. Specifically, for example, the magnetic recording device may be a hard disk device or a flexible Discs, magnetic tapes, etc. as optical disks, DVD (Digital Versatile Disc), DVD-RAM (Random Access Memory), CD-ROM (Compact Disc Read Only Memory), CD-R (Recordable) / RW (ReWritable), etc. As the magneto-optical recording medium, MO (Magneto-Optical disc) or the like can be used, and as the semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory) or the like can be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  As an execution form different from the above-described embodiment, the computer may read the program directly from the portable recording medium and execute processing according to the program. Each time is transferred, the processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

In this embodiment, the present apparatus is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.
Further, the various processes described above are not only executed in time series according to the description, but may be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes. In addition, it can change suitably in the range which does not deviate from the meaning of this invention.

[Confirmation of effect]
In order to confirm the effect of the invention, a speaker section estimation experiment was performed on conference data for 5 minutes with four participants in a measurement environment using three microphones as shown in FIG. At the conference, first two male and female speakers are seated in the positions of male 1, female 1, male 2, and female 2, respectively, and then introduce themselves, then each speaker moves to position PP in turn. And made a remark. Assume that self-introduction was performed between 120 seconds from the start of recording, t _train was set to 120 seconds, observation signals from the start of recording to 120 seconds were used to create a speaker identification model, and speaker identification was performed after 120 seconds. Went. The short-time Fourier transform has a frame length of 64 ms and a frame shift length of 32 ms.

ここで、ＤＥＲは誤棄却（missed speaker time: ＭＳＴ、誰かが話しているにもかかわらず話していないと判定した時間長）、誤受理（false alarm speaker time:ＦＡＴ、誰も話していないにもかかわらず誰かが話していると判定した時間長）、話者誤り（speaker error time: ＳＥＴ、話者を誤って判定した時間長）の３つの誤検出を含む指標となっている。つまりこの指標においては、ＤＥＲ値が小さい方が話者区間推定の精度が高いことを示しており、特に本発明においては話者を正しく判定できているかが問題となるため、効果の程度はＳＥＴに顕著に現れるはずである。 As an evaluation index, dialization error rate (DER) was used.

Here, DER is falsely rejected (missed speaker time: MST, the length of time that someone has spoken but not determined to speak), false acceptance (false alarm speaker time: FAT, no one is speaking) Regardless of the length of time that someone has determined to be speaking) and speaker error time (SET), this is an index that includes three false detections. In other words, this index indicates that the smaller the DER value, the higher the accuracy of the speaker section estimation. In particular, in the present invention, the problem is whether the speaker is correctly determined. Should appear prominently.

  The confirmation result is shown in FIG. FIG. 10 illustrates the results. (A) shows the correct answer, (b) shows the estimation result by the conventional method, and (c) shows the estimation result by the method of the present invention. The arrival directions of male 1, female 1, male 2, and female 2 are 100 °, 50 °, −50 °, and −100 °, respectively, position PP is in the arrival direction of −160 °, and male 1 Corresponds to speaker 1, woman 1 corresponds to speaker 2, man 2 corresponds to speaker 3, and woman 2 corresponds to speaker 4. As can be seen from FIG. 10 (b), in the conventional method, the speaker at the position PP is estimated as another speaker 5 other than the speakers 1 to 4, and the SET becomes large as shown in FIG. 9 (a). ing. On the other hand, in the method of the present invention, the speaker can be distinguished in the −160 ° direction in almost all the time intervals in the same manner as in FIG. 10 (a), and SET is improved as shown in FIG. 9 (a). It can be seen that the DER value, which is the overall performance, is also improved.

  Moreover, the result of having performed the conference simulation in 10 speaker combinations is shown in FIG. This uses conference simulation data created using an audio signal and an impulse response measured in the measurement environment of FIG. In FIG. 9 (b), simulation 1 is a case where there is no overlap between the voices of the speakers, and simulation 2 is a result when there is an overlap between the voices of the speakers. It can be seen that the method of the present invention shows better results than the conventional method for SET.

  INDUSTRIAL APPLICABILITY The present invention can be used for a system or apparatus that needs to estimate each speaker's voice section from acoustic data in which voice signals of a plurality of speakers are mixed, and in particular, speaker position during voice recording. This is effective when the movement of.

The figure which shows the function structural example of the multiple signal area estimation apparatus of 1st, 2 embodiment. The figure which shows the example of a processing flow of the multiple signal area estimation apparatus of 1st, 2 embodiment. The figure which shows the function structural example of the speaker identification part of the multiple signal area estimation apparatus of 1st-4th embodiment. Diagram explaining how to connect and process when the frame is not connected The figure which shows the example of a processing flow of the multiple signal area estimation apparatus of 3rd Embodiment. The figure which shows the example of a processing flow of the multiple signal area estimation apparatus of 4th Embodiment. The figure which shows the function structural example of the multiple signal area estimation apparatus of 5th Embodiment. Diagram showing the measurement environment used to confirm the effect Table showing effect confirmation results The figure which shows the basis data of the confirmation result of the effect The figure which shows the function structural example of the multiple signal area estimation apparatus of a prior art The figure which shows the example of a processing flow of the multiple signal area estimation apparatus of a prior art The figure which shows another functional structural example of the multiple signal area estimation apparatus of a prior art

Claims (12)

A multi-signal section estimation device for estimating each speaker's utterance section from an observation signal recorded by a plurality of microphones and containing speech uttered by a plurality of speakers,
A frequency domain converter that sequentially cuts out the observation signal into frames of a predetermined length and converts the frames into a frequency domain for each frame;
A speech interval estimation unit that estimates whether each frame corresponds to a speech interval based on the observation signal converted into the frequency domain (hereinafter referred to as a “frequency domain observation signal”);
Based on the frequency domain observation signal, an arrival direction estimation unit that estimates the arrival direction of the frequency domain observation signal for each frame;
A direction-of-arrival classification unit that classifies each frame estimated to correspond to the speech section into a cluster for each speaker based on the similarity of the direction of arrival;
Based on the frequency domain observation signal of each frame classified into the same cluster by a predetermined time, a model of the speaker related to the cluster is created for each cluster, and the speaker of the observation signal after the predetermined time A speaker identification unit for estimating
A multi-signal section estimation apparatus comprising: In the multiple signal area estimation device according to claim 1,
The above speaker identification unit
Feature extraction means for calculating speech feature values of each frame of the frequency domain observation signal;
Using the speech feature value of each frame classified into the same cluster by the predetermined time, the speaker model related to the cluster is created and output for each cluster, and each of the models up to the predetermined time is output. Model learning means for outputting a set of a frame index and a speaker index associated with a cluster to which each frame belongs;
Frames connected to each other that are classified into the same cluster after the predetermined time (hereinafter,
The likelihood of the speaker's model is calculated for each model, and the speaker's index related to the model having the maximum likelihood is assigned to the segment. A likelihood calculating means for outputting together with an index of each included frame;
A multi-signal section estimation apparatus comprising: A multi-signal section estimation device for estimating each speaker's utterance section from an observation signal recorded by a plurality of microphones and containing speech uttered by a plurality of speakers,
A frequency domain converter that sequentially cuts out the observation signal into frames of a predetermined length and converts the frames into a frequency domain for each frame;
A speech interval estimation unit that estimates whether each frame corresponds to a speech interval based on the observation signal converted into the frequency domain (hereinafter referred to as a “frequency domain observation signal”);
Based on the frequency domain observation signal, an arrival direction estimation unit that estimates the arrival direction of the frequency domain observation signal for each frame;
A direction-of-arrival classification unit that classifies each frame estimated to correspond to the speech section into a cluster for each speaker based on the similarity of the direction of arrival;
A speech enhancement unit that generates a signal (hereinafter referred to as an “emphasis signal”) that emphasizes the speech signal component for each speaker related to the cluster, based on the frequency domain observation signal;
Speaker identification that creates a speaker model for each speaker based on the emphasis signal up to a predetermined time, and estimates the speaker of the observed signal after the predetermined time based on the model of each speaker And
A multi-signal section estimation apparatus comprising: In the multiple signal section estimation device according to claim 3,
The above speaker identification unit
Feature extraction means for calculating a speech feature amount of each frame of the enhancement signal;
Using the speech feature value of each frame of the enhancement signal up to the predetermined time, the speaker model is created and output for each speaker, and the index of each frame up to the predetermined time and the frame Model learning means for outputting a pair with a speaker index related to a cluster to which each frame belongs,
The likelihood of the speaker's model is calculated for each model for speech features of mutually connected frames (hereinafter referred to as “segments”) classified into the same cluster after the predetermined time, and the maximum likelihood is calculated. A likelihood calculating means for assigning an index of a speaker related to the model to be output to the segment together with an index of each frame included in the segment;
A multi-signal section estimation apparatus comprising: A multi-signal section estimation method for estimating the utterance section of each speaker from an observation signal recorded by a plurality of microphones and including speech uttered by a plurality of speakers,
A frequency domain conversion step of sequentially cutting out the observation signal into frames of a predetermined length and converting each frame into the frequency domain;
A speech interval estimation step for estimating whether or not each frame corresponds to a speech interval based on the observation signal converted into the frequency domain (hereinafter referred to as “frequency domain observation signal”);
Based on the frequency domain observation signal, the direction of arrival estimation step for estimating the direction of arrival of the frequency domain observation signal for each frame;
A direction-of-arrival classification step of classifying each frame estimated to fall within the speech section into clusters for each speaker based on the similarity of the directions of arrival;
Based on the frequency domain observation signal of each frame classified into the same cluster by a predetermined time, a model of the speaker related to the cluster is created for each cluster, and the speaker of the observation signal after the predetermined time Speaker identification step for estimating each speaker based on the model of each speaker;
The multiple signal section estimation method characterized by performing. In the multiple signal section estimation device according to claim 5,
The speaker identification step is
A feature extraction substep for calculating a speech feature amount of each frame of the frequency domain observation signal;
Using the speech feature value of each frame classified into the same cluster by the predetermined time, the speaker model related to the cluster is created and output for each cluster, and each of the models up to the predetermined time is output. A model learning sub-step for outputting a set of a frame index and a speaker index associated with a cluster to which each frame belongs;
The likelihood of the speaker's model is calculated for each model for speech features of mutually connected frames (hereinafter referred to as “segments”) classified into the same cluster after the predetermined time, and the maximum likelihood is calculated. A likelihood calculation sub-step of assigning to the segment an index of a speaker related to the model taking the following and outputting together with an index of each frame included in the segment;
The multiple signal section estimation method characterized by performing. A multi-signal section estimation method for estimating the utterance section of each speaker from an observation signal recorded by a plurality of microphones and including speech uttered by a plurality of speakers,
A frequency domain conversion step of sequentially cutting out the observation signal into frames of a predetermined length and converting each frame into the frequency domain;
A speech interval estimation step for estimating whether or not each frame corresponds to a speech interval based on the observation signal converted into the frequency domain (hereinafter referred to as “frequency domain observation signal”);
Based on the frequency domain observation signal, the direction of arrival estimation step for estimating the direction of arrival of the frequency domain observation signal for each frame;
A direction-of-arrival classification step of classifying each frame estimated to fall within the speech section into clusters for each speaker based on the similarity of the directions of arrival;
Based on the frequency domain observation signal, a speech enhancement step of generating a signal (hereinafter referred to as “emphasis signal”) that emphasizes the speech signal for each speaker related to the cluster ;
Speaker identification that creates a speaker model for each speaker based on the emphasis signal up to a predetermined time, and estimates the speaker of the observed signal after the predetermined time based on the model of each speaker Steps,
The multiple signal section estimation method characterized by performing. The multiple signal section estimation method according to claim 7,
The speaker identification step is
A feature extraction sub-step for calculating a speech feature amount of each frame of the enhancement signal;
Using the speech feature value of each frame of the enhancement signal up to the predetermined time, the speaker model is created and output for each speaker, and the index of each frame up to the predetermined time A model learning sub-step for outputting a pair with a speaker index associated with the cluster to which the frame belongs;
The likelihood of the speaker's model is calculated for each model for speech features of mutually connected frames (hereinafter referred to as “segments”) classified into the same cluster after the predetermined time, and the maximum likelihood is calculated. A likelihood calculation sub-step of assigning to the segment an index of a speaker related to the model taking the following and outputting together with an index of each frame included in the segment;
The multiple signal section estimation method characterized by performing. The multiple signal section estimation method according to claim 6 or 8,
Furthermore, after assigning a speaker index to the segment in the likelihood calculation sub-step, a model of the speaker is newly created based on the speech feature amount of each frame belonging to the segment, and the speaker model A method for estimating a plurality of signal sections, comprising executing a model update step for updating the signal. In the multiple signal area estimation method according to any one of claims 6, 8, and 9,
Further, when the calculated maximum likelihood is smaller than a predetermined threshold, it is determined that a new speaker has joined, and an index of the new speaker is assigned to the segment, and each frame belonging to the segment is assigned. A multi-signal section estimation method, comprising: executing a model addition step of creating a new speaker model based on a speech feature amount.   The program for functioning a computer as an apparatus in any one of Claims 1-4.   A computer-readable recording medium on which the program according to claim 11 is recorded.

Images