KR100925256B1 - A method for discriminating speech and music on real-time - Google Patents

Abstract

The present invention relates to a method for classifying voice and music in real time, wherein the voice / music classification method comprises: (1) detecting a voice signal with respect to an input signal, and (2) detecting the voice signal. In the case of detecting a voice at, the feature vectors (hereinafter, referred to as 'SMV based feature vectors') having statistically excellent voice / music classification characteristics are extracted from the feature vectors obtained in the SMV encoding process for the input signal. And (3) classifying voice / music in real time using the extracted SMV based feature vectors as feature vectors of a Gaussian Mixture Model (GMM). .

According to the speech / music classification method of the present invention, by extracting feature vectors having excellent music / voice classification characteristics statistically from the feature vectors obtained in the SMV encoding process and using them as feature vectors of a Gaussian Mixture Model (GMM) It can classify music and voice in real time with excellent accuracy. In particular, since the results obtained in the existing SMV encoding process are used in large part, system resources can be effectively used.

Speech / Music Classification, Selectable Mode Vocoder (SMV), Gaussian Mixture Model (GMM), Feature Vector, Real-Time Classification, Speech Signal Detection (VAD), Linear Predictive Coding

Description

How to classify voice and music in real time {A METHOD FOR DISCRIMINATING SPEECH AND MUSIC ON REAL-TIME}

1 is a diagram illustrating a process of processing an input signal by a Selectable Mode Vocoder (SMV);

2 is a block diagram of a conventional SMV speech / music classification algorithm.

FIG. 3 compares statistical classification characteristics of Mean Energy, Spectral Difference, Normalized Pitch Correlation, and Music Continuity Counter with excellent statistical classification characteristics. Showing drawings.

4 is a block diagram of a speech / music classification algorithm using GMM-based SMV coding parameters in accordance with an embodiment of the present invention.

5 shows a file used for testing.

6 is a diagram illustrating a ROC curve for a conventional SMV method and a voice / music classification method according to the present invention.

FIG. 7 is a diagram illustrating comparison of waveforms of speech / music and classification results of two systems on a time axis in order to evaluate real-time speech / music classification performance of the existing SMV method and the method proposed in the present invention.

<Explanation of symbols for main parts of the drawings>

405: input signal

410: step of LPC analysis

420: VAD stage

430: music detection step

440: Feature Extraction Step

450: Block to check the VAD value

455: judge by silence

460: GMM Model

465: judge by voice

466: judging by music

The present invention relates to a method for classifying voice and music in real time. In particular, a Gaussian mixture model is extracted by extracting feature vectors having excellent music / voice classification characteristics among feature vectors obtained in a Selectable Mode Vocoder (SMV) encoding process. By using them as feature vectors of a Mixture Model (GMM), a new real-time music / voice classification method of real-time classification of music and voice with excellent accuracy is provided.

Recently, with the development of mobile communication, multimedia services using wireless communication devices have become commonplace, and researches for effectively transmitting voice in a limited frequency band have been continuously conducted. Currently, various voice codecs with variable bit rates have been developed to efficiently use limited communication networks. In fact, the technique of allocating different bits according to the type of input speech signal is treated as a key technique for accurate signal classification because it greatly affects the sound quality of the final speech. In particular, the importance of voice / music classification for effectively transmitting music signals through mobile communication networks has been actively studied.

Selected Mode Vocoder (SMV), a 3rd Generation Partnership Project2 (3GPP2) standard low-rate speech codec from the European Telecommunications Standards Institute (ETSI), which employs a variable rate algorithm, is also based on real-time speech / music classification techniques. Doing. The SMV voice encoder has four bit rates: Rate 1 (8.55 kbps), Rate 1/2 (4.0 kbps), Rate 1/4 (2.0 kbps), Rate 1/8 (0.8 kbps), and Mode 0 (premium), Mode 1 It has four modes of operation (standard), Mode 2 (economy), and Mode 3 (supereconomy), and thus has various average bit rates (ABR), so that performance is selectively selected in relation between performance and sound quality of CDMA system. Has the advantage of being adjustable. However, SMV voice incubator has a problem that voice / music classification performance is not so good.

The present invention has been proposed to solve the above problems, and extracts feature vectors with excellent music / voice classification characteristics among feature vectors obtained in the SMV encoding process to obtain a Gaussian Mixture Model (GMM). By using it as feature vectors, it aims to propose a new real time music / voice classification method that can classify music and voice in real time with excellent accuracy.

According to a feature of the present invention for achieving the above object, a real-time music / voice classification method,

(1) detecting a speech signal with respect to the input signal;

(2) When it is detected that the voice is detected in the voice signal detection step, among the feature vectors obtained in the SMV (Selectable Mode Vocoder) encoding process for the input signal, a feature vector having excellent voice / music classification characteristics statistically (hereinafter referred to as' SMV-based) Extracting feature vectors'; And

(3) classifying voice / music in real time using the extracted SMV based feature vectors as feature vectors of a Gaussian Mixture Model (GMM);

It characterized by including the.

Preferably, feature vectors (hereinafter, referred to as 'non-SMV based feature vectors') having excellent speech / music classification characteristics that are not normalized in the SMV encoding process may be applied as feature vectors of the GMM together with the SMV based feature vectors. Can be.

In particular, it is more preferred that the non-SMV based feature vectors include a pitch feature vector.

In addition, preferably, the SMV-based feature vectors may include a mean energy feature vector, a spectral difference (SD) feature vector, a normalized pitch correlation feature vector, and a music continuous counter (music). continuity counter) feature vector may be included.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

1 is a diagram illustrating a process in which an SMV processes an input signal. As shown in FIG. 1, in the SMV, the mode and the transmission bit rate of the frame are determined every 160 samples (20 ms) for the input signal sampled at 8 kHz. Specifically, the input signal removes background noise through preprocessing such as silence increase, high band filtering, noise suppression, and adaptive tilt compensation. The signal from which the background noise is removed during the preprocessing is subjected to frame processing such as LPC analysis, open-loop pitch search, signal change and classification. Linear predictive analysis and open-loop pitch surveys are used to extract parameters for speech / music classification. Signal classification classifies a frame into one of noise, silence, unvoiced sound, start sound, abnormal voiced sound, and normal voiced sound. In addition, the bit rate of the frame is determined by a Rate Determination Algorithm (RDA) based on a mode determined according to the type of the current frame and the communication state. If the frame is determined to be music, the bit rate of the frame is determined to be Rate 1 and the highest transmission rate is assigned. Otherwise, the transmission rate is determined and encoded according to a predetermined threshold.

SMV evaluates the input signal sampled at 8 kHz every 20 ms. The voice signal is classified into a silent and a voice by a voice activity detector (VAD), and then a non-silent part (voice) is classified into a voice and a music. 2 is a block diagram of a conventional SMV speech / music classification algorithm. As shown in FIG. 2, the existing SMV speech / music classification algorithm inputs feature vectors extracted from a voice detector (VAD), an open loop pitch, and a linear prediction coding (LPC). After receiving, the running mean value of each feature vector is used as feature vectors of the speech / music classification algorithm. These feature vectors are compared to a fixed threshold (threshold is empirically set) for speech / music classification. Feature vectors input to the existing SMV speech / music classification algorithm will be described in more detail below.

1. Feature Vectors Extracted from LPC Analysis

(1) Reflection Coefficient

It can be obtained as in Equation 1 below, and the weight is given to the last quarter of the encoding frame.

는 자기 상관 함수를,

는 j번째 LPC 계수를, E는 에너지값을 각각 나타낸다.In Equation 1,

Is an autocorrelation function,

Denotes the j th LPC coefficient and E denotes the energy value, respectively.

(2) Line Spectral Frequency (LSF)

The 10 th order LPC value obtained by giving weight to the last quarter of the encoding frame may be changed as shown in Equation 2 below.

와

로 나누어진다.Here, A (z) is a prediction error filter transfer function and a _i is an LPC value. A (z) is two transfer functions which can be represented by the following equations (3) and (4).

Wow

Divided into.

.here,

.

에서 구해진 5개의 해가 lsf(i) [i=1,3,5,7,9]의 값이 되고,

에서 구해진 5개의 해가 lsf(i) [i=2,4,6,7,10]의 값이 된다.The functions represented by the equations (3) and (4) can be expressed in the frequency domain as shown in the following equations (5) and (6), and the solution of the two transfer functions between w = 0 and w = 1 becomes the LSF value. here,

The five solutions found at are lsf (i) [i = 1,3,5,7,9],

The five solutions found at are lsf (i) [i = 2,4,6,7,10].

.here,

.

2. Feature Vectors Extracted from VAD

(1) Energy

As an energy value for the frame, it has a value of at least 10 as can be seen in Equation 7 below.

값은 LPC 분석에서 구해지는 자기 상관 함수의 값이고, L_LPC 값은 LPC 윈도우의 길이로 240을 갖는다.here,

The value is the value of the autocorrelation function obtained from the LPC analysis, and L _LPC The value has 240 as the length of the LPC window.

(2) Residual Energy

As the energy value for the error component in the LPC analysis, it may be expressed as Equation 8 below.

(3) VAD Decision Flag

A comparison between the parameters extracted from the VAD and a fixed threshold value determines whether the current frame is silent or voice. When the current frame is not voice, that is, silent, it has a value of 1 and is passed to the music classification algorithm.

3. Feature Vectors of Music Classification Algorithm

(1) Mean of Line Spectral Frequency

A moving average value of lsf ₁ (i) input through the LPC analysis may be expressed as in Equation 9 below.

(2) Mean Energy

As a moving average of frame energy, it can be calculated | required as following Formula (10).

(3) Spectral Difference

The reflection coefficient in the noise section and the reflection coefficient value input from the LPC may be obtained as in Equation 11 below.

(4) Partial Residual Energy

It is incremented when VAD classifies a frame as noise, and is initialized to 0 when the next frame is judged as voice. It can be obtained as shown in Equation 12 below.

(5) Mean Reflection Coefficients of Noise / Silent

This is calculated when VAD classifies a frame as noise and is used to find the spectral difference (SD) of the next frame. It can be obtained as in Equation 13.

(6) Normalized Pitch Correlation

Using normalized pitch correlation stored in five buffers, the following equation (14) can be obtained.

After the operation, it is updated as in Equations 15 and 16 for the next frame.

(7) Mean of the Periodicity Counter

이면 프레임을 음악으로 분류한다.The following equation (17) can be obtained and updated every 32 frames.

If it is, the frame is classified as music.

Here, the weight value used has a value such as the following Equation 18,

The value of c _pr is increased by the following conditional expression.

Here, std is a normalized deviation value of the pitch lag.

(8) Mean of Normalized Pitch Correlation of Noise

값이 13보다 크고 VAD에서 잡음으로 분류됐을 경우 다음 수학식 19와 같이 계산된다.Of the current frame

If the value is greater than 13 and classified as noise in the VAD, the following equation is calculated.

(9) Music Continuity Counter

값이 200보다 크면 SMV 음성/음악 분류 알고리즘은 프레임을 음악으로 판단한다. 다음 수학식 20과 같이 구할 수 있다.The eight feature vectors obtained so far are increased or decreased compared with a fixed threshold, empirically learned.

If the value is greater than 200, the SMV speech / music classification algorithm judges the frame as music. It can be obtained as shown in Equation 20 below.

Next, the speech / music classification method based on Gaussian mixture model (GMM) will be described.

GMM is a pattern recognizer that shows excellent performance in speaker recognition and music recognition. A Gaussian-type mixed model based probability density function is given by Equation 21 below.

,

.here,

,

.

는 혼합된 가우시안 밀도의 가중치를,

는 D개의 특징 벡터들을,

는 평균 벡터들을,

는 공분산 행렬을,

는 다음 수학식 22와 같은 GMM의 모델 파라미터를 나타낸다.Actually parameter

Is the weight of the mixed Gaussian density,

Denotes D feature vectors,

Is the mean vectors,

Is the covariance matrix,

Denotes a model parameter of the GMM as shown in Equation 22 below.

For pattern recognition using GMM, GMM model for voice / music is created through training based on Expectation-Maximization (EM) algorithm using the set feature vector. In the test process, the likelihood ratio is calculated by calculating the likelihood ratio obtained by inputting the real data into the GMM model for the already trained voice and music, and classifying the voice / music by comparing with the threshold value. A real-time classification method suitable for speech coding using a smoothed likelihood ratio (SLR) in consideration of the correlation of the error processing and the correlation is proposed as shown in Equation 23 below.

는 k번째 프레임의 우도를 나타내고,

는 이전 프레임의 우도를 나타내며, β는 실험적으로 최적화된 가중치로 0.5로 설정한다.here,

Represents the likelihood of the kth frame,

Denotes the likelihood of the previous frame, and β is set to 0.5 as an experimentally optimized weight.

Now, the GMM-based speech / music classification method using the SMV coding feature vectors proposed in the present invention will be described in earnest.

In the present invention, the parameters generated automatically in the encoding process of the SMV are used as feature vectors without a separate calculation process, and voice / music is classified in real time by selectively using the SMV coding parameters to construct an effective GMM. In the GMM which uses very precise probability density function, the better the distinction of the feature vectors is, the better the performance is. Therefore, the statistical classification characteristics of the feature vectors used in the music classification of SMV are investigated. 3 compares statistical classification characteristics of Mean Energy, Spectral Difference, Normalized Pitch Correlation, and Music Continuity Counter with excellent statistical classification characteristics. It is a diagram showing. In addition, in an embodiment of the present invention, since pitch exhibits excellent performance in speech / music classification, a total of five feature vectors are proposed as GMM feature vectors by extracting a pitch feature vector that is not normalized in SMV. .

4 is a block diagram of a speech / music classification algorithm using GMM-based SMV coding parameters according to an embodiment of the present invention. As shown in FIG. 4, when the VAD 420 included in the voice / music classification algorithm of the SMV determines that there is no silence, that is, there is a signal (the value is '1' at 450), The GMM 460 generates a likelihood by extracting feature vectors generated in the LPC analysis 410, the VAD 420, and the music detection 430 included in the SMV, and applying the same to the GMM model 460. Voice 465 / music 466 are classified.

In order to examine the performance of the GMM-based real-time speech / music classification method applied to the speech / music classification algorithm proposed by the present invention, a comparison experiment with the existing algorithm of SMV was performed. The speech database used for this experiment was clean speech sampled at 8 kHz, composed of two sentences with different meanings, and music was used by down-sampling CD at 8 kHz.

To create a model of the proposed speech / music classification algorithm, we trained 50 speech / music files and used 16 mixes. 5 is a diagram illustrating a file used for a test. As shown in FIG. 5, the test file was made using 10 voice / music files, and was composed of a voice section 5 seconds, a music section 10 seconds, and a silent section 7.5 seconds.

In order to check the performance of voice / music classification for various music genres, five test files composed of each genre (metal, ballad, hip hop, classic, and jazz) and one mixed test file were created. To see the actual performance of the two methods, the actual results are compared to manual writing of 0 (silent), 1 (voice), and 2 (music) every 20ms of the test file.

The music detection probability (P _{d _ M} ) and the voice detection probability (P _{d _ S} ) for each test file were examined, and the results are summarized as in Table 1 below.

Type of noise Way Music (P _{d _M} ) Voice (P _{d _S} ) mix Traditional SMV Method 0.27 0.95 Method according to the invention 0.94 0.98 metal Traditional SMV Method 0 0.95 Method according to the invention 0.99 0.98 Ballade Traditional SMV Method 0.45 0.95 Method according to the invention 0.94 0.99 Hip hop Traditional SMV Method 0.13 0.95 Method according to the invention 0.85 0.91 classic Traditional SMV Method 0.45 0.95 Method according to the invention 0.76 0.99 jazz Traditional SMV Method 0.34 0.94 Method according to the invention 0.73 0.99

It can be seen from Table 1 that the overall detection probability of the GMM-based method according to the present invention is superior to the existing SMV method. In particular, the difference is even greater in music classification. The performance of music classification for various music genres varies greatly depending on the music genre and system. However, when the existing SMV method is compared with the GMM-based method proposed in the present invention, the performance of the proposed method is improved in all genres. You can see the excellent.

Next, a receiver operation characteristic (ROC) curve was drawn on the mixed test file that mixed data of all music genres to compare the overall performance of the two methods. FIG. 6 is a diagram illustrating a ROC curve of a conventional SMV method and a voice / music classification method according to the present invention, and shows a detection probability for voice / music while changing a value of a weight given to an inequality comparing a likelihood ratio. . As shown in FIG. 6, it can be confirmed that the voice / music classification method according to the present invention is superior in performance to the existing SMV method.

Finally, in order to evaluate the real-time voice / music classification performance of the existing SMV method and the method proposed in the present invention, FIG. Figure 7 (a) is a part of the test file is composed of voice → music → voice → music order, Figure 7 (b) and Figure 7 (c) is a real-time voice / music classification results of the existing SMV and the present invention, respectively As a result of the real-time voice / music classification of the method proposed in, 1 for voice and 2 for music are shown. Also from FIG. 7, it can be seen that there is a clear difference in real-time music classification performance between the existing SMV method and the method proposed in the present invention.

The present invention described above may be variously modified or applied by those skilled in the art, and the scope of the technical idea according to the present invention should be defined by the following claims.

According to the speech / music classification method of the present invention, by extracting feature vectors having excellent music / voice classification characteristics statistically from the feature vectors obtained in the SMV encoding process and using them as feature vectors of a Gaussian Mixture Model (GMM) It can classify music and voice in real time with excellent accuracy. In particular, since the results obtained in the existing SMV encoding process are used in large part, system resources can be effectively used.

Claims (4)

Detecting a speech signal with respect to the input signal; If it is detected that the voice is detected in the voice signal detection step, among the feature vectors obtained in the SMV (Selectable Mode Vocoder) encoding process for the input signal, a mean energy feature vector, a spectral difference (SD) feature Extracting feature vectors (hereinafter referred to as 'SMV based feature vectors') comprising a vector, a normalized pitch correlation feature vector, and a music continuity counter feature vector; And Classifying voice / music in real time using the extracted SMV-based feature vectors as feature vectors of a Gaussian Mixture Model (GMM) Method for classifying voice and music in real time comprising a. The method of claim 1, In the SMV encoding process, non-normalized feature vectors (hereinafter, referred to as 'non-SMV based feature vectors') may be applied as feature vectors of the GMM together with the SMV based feature vectors, and the pitch may be applied to the non-SMV based feature vectors. pitch) A method for classifying voice and music in real time, characterized by including a feature vector. delete delete

Images