KR20190019726A - System and method for hidden markov model based uav sound recognition using mfcc technique in practical noisy environments - Google Patents

Abstract

Disclosed are a hidden markov model (HMM)-based method and system for recognizing unmanned aerial vehicle (UAV) sound using a mel frequency cepstral coefficients (MFCC) technique in practical noise environment. The method may comprise the steps of: extracting a feature vector related to a sound signal of an UAV based on a MFCC technique in an environment where noise exists; learning a HMM model based on a learning data set generated based on the extracted feature vector; and recognizing the sound signal corresponding to the UAV based on the learned HMM model for an inputted sound signal.

Description

TECHNICAL FIELD [0001] The present invention relates to an HMM-based unmanned aerial vehicle acoustics recognition method and system using an MFCC technique in a real-

Embodiments of the present invention relate to a technique for identifying an unmanned airplane by recognizing an acoustic signal corresponding to an unmanned aerial vehicle such as a drone among various acoustic signals in a real environment where noise exists.

Unmanned Aerial Vehicle (UAV) As the use of drone is increasing, drones are widely used for military, commercial, and broadcasting applications. These unmanned aerial vehicles are mostly small, and in the case of small unmanned aerial vehicles, they operate in uncontrolled airspace, which can pose a serious threat to public safety and privacy. For example, small unmanned aerial vehicles can carry chemical weapons, biological weapons, or nuclear weapons, and can be used to smuggle drugs or perform other illegal activities across borders.

As such, there is a possibility that a small unmanned aerial vehicle such as a drone may be used for threatening human life, threatening privacy, illegal use such as drug smuggling, and threatening public safety. Therefore, Technology is becoming increasingly important. The current technology for detecting and tracking drones is based on radar.

However, when detecting and tracking the position of a drone in operation based on a radar, it is difficult to detect or track the position of the dron located near the ground when the size of the dron is small, and there is a problem in that it includes a blind spot And there is also a problem of costly installation and operation of the system.

Accordingly, there is a need for a technique that can more accurately detect and track the real-time position of the dron regardless of the size of the dron, whether the drone is small or medium. That is, there is a need for drones' location detection and tracking technology, besides the technology for tracking the location of radar-based drones.

Korean Patent Laid-Open No. 10-2013-0120176 relates to a method for tracking the position of an unmanned airplane. When the location of the unmanned airplane can not be determined by a GPS disturbance or a failure of the GPS receiver, the ground control device adjusts the image sensor (Azimuth and elevation) of the unmanned airplane and the image sensor information (pitch, roll, heading, altitude) and the fixed target (absolute coordinate) And tracking the position of the unmanned aerial vehicle in reverse with a reference point.

[1] I. Patel and Y. S. Rao, "Speech recognition using Hidden Markov Model with MFCC-Subband technique," in Proc. International Conference on Recent Trends in Information, Telecommunication and Computing, pp. 168-172, Mar. 2010. [2] G. B. Singh and H. Song, "Hidden Markov Models in Vehicular Crash Detection," IEEE Transactions on Vehicular Technology, vol. 58, no. 3, pp. 1119-1128, Mar. 2009.

The present invention can be applied to a drones which are not limited by the size of a UAV (for example, irrespective of the size of a dron) and, when various acoustic signals are inputted into the input in a real environment where noise exists, And more particularly, to a technique for accurately recognizing a sound signal to be generated. That is, the present invention relates to a technique for recognizing an acoustic signal corresponding to a dron among a plurality of acoustic signals existing in a real environment to enable detection and tracking of the dron.

A method for recognizing sound of an unmanned aerial vehicle, the method comprising: extracting acoustic signal-related feature vectors of an unmanned aerial vehicle (UAV) based on a Mel Frequency Cepstral Coefficients (MFCC) technique in the presence of noise; Learning a HMM (hidden Markov Model) model based on the extracted learning data set based on the extracted feature vector, and a step of learning an HMM model based on the learned HMM model And recognizing an acoustic signal corresponding to the UAV.

According to an aspect of the present invention, the step of extracting the feature vector may extract the feature vector based on a first MFCC and a second MFCC which are composed of different numbers of cepstral coefficients.

According to another aspect of the present invention, the extracting of the feature vector may extract acoustic signal-related feature vectors of an unmanned aerial vehicle based on an envelope of a cepstral domain for a predetermined reference time.

According to another aspect of the present invention, the step of extracting the feature vector includes a pre-emphasis processing step of performing highpass filtering on an acoustic signal corresponding to an unmanned air vehicle, a pre-emphasis processing step of performing high- Generating a spectrogram representing a feature vector corresponding to an acoustic signal of the UAV by performing framing, windowing, and fast Fourier transform (FFT) Passing a fast Fourier transformed signal through a Mel-scale filter bank, performing a discrete cosine transform (DCT) on the signal passed through the mel-scale filter bank, Calculating a Cepstral Coefficient, and constructing the MFCC based on the calculated cepstral coefficients.

A system for recognizing sound of an unmanned aerial vehicle, the method comprising: extracting acoustic signal-related feature vectors of an unmanned aerial vehicle (UAV) based on a Mel Frequency Cepstral Coefficients (MFCC) technique in the presence of noise; A model learning unit for learning an HMM (hidden Markov Model) model based on the extracted learning data set based on the extracted feature vector, and a learning unit for learning the inputted HMM model And an acoustic recognition unit for recognizing an acoustic signal corresponding to the UAV on the basis of the acoustic signal.

The present invention extracts a feature vector from an acoustic signal of an unmanned aerial vehicle (UAV) based on MFCC (Mel Frequency Cepstral Coefficient) technique, learns HMM (Hidden Markov Model) based on the extracted feature vector, (UAV) without disturbing the size of the UAV (for example, regardless of the size of the drone) by identifying the sound emitted from the propeller of the unmanned airplane among the acoustic signals inputted in the real environment based on the noise (UAV) can be recognized in a real environment where there exists an unmanned aerial vehicle (UAV).

1 is a block diagram showing the configuration of an acoustic recognition system of an unmanned aerial vehicle according to an embodiment of the present invention.
FIG. 2 is a flowchart illustrating an operation of an acoustic recognition method for an unmanned aerial vehicle according to an exemplary embodiment of the present invention.
FIG. 3 illustrates a spectrogram for various acoustic signals according to an embodiment of the present invention.
4 is a flowchart illustrating an operation of extracting a feature vector based on the MFCC technique in an embodiment of the present invention.
5 is a block diagram showing a detailed configuration of a feature vector extracting unit according to an embodiment of the present invention.
FIG. 6 is a flowchart illustrating an operation of learning a BWA (Baum-Welch Algorithm) based HMM model in an embodiment of the present invention.
7 is a block diagram illustrating an operation of recognizing a desired acoustic signal based on a learned HMM model in an embodiment of the present invention.
FIG. 8 is a graph illustrating an average recognition rate based on feature vectors extracted using the first and second MFCC techniques, according to an embodiment of the present invention. Referring to FIG.
9 is a graph showing recognition rate performance for SNR in an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

The present invention relates to a technique for recognizing an unmanned aerial vehicle such as a drone, and more particularly, to a technique for recognizing an acoustic signal (that is, acoustic related to a propeller sound) emitted as a propeller of an unmanned airplane is driven Based on the MFCC (Mel Frequency Cepstral Coefficient) technique based on the training data, and learning data of HMM (Hidden Markov Model) using the extracted feature vectors. The sound signal corresponding to the sound of the propeller of the drone or the like. That is, in the present embodiments, the HMM model can be used as a classifier for recognizing acoustic signals of a drone or the like.

In the present embodiments, the 'Unmanned Aerial Vehicle' is a flight vehicle manufactured to allow a pilot to perform a designated mission without boarding the aircraft, and may include a drone or the like.

In the present embodiment, the 'acoustic signal' represents an audio sound. For example, an acoustic signal corresponding to a drone is an acoustic sound emitted by a propeller of a drone, Lt; / RTI >

In the present embodiments, the 'second MFCC' is described as being composed of 36 polynomial coefficients (MFCC), which corresponds to the embodiment, and the 'second MFCC' Lt; / RTI >

FIG. 1 is a block diagram illustrating the configuration of an acoustic recognition system of an unmanned aerial vehicle according to an embodiment of the present invention. FIG. 2 is a flowchart illustrating an operation of the acoustic recognition method of an unmanned aerial vehicle FIG.

1, the acoustic recognition system 100 may include a feature vector extraction unit 110, a model learning unit 120, and an acoustic recognition unit 130. [ 2 may be performed by the feature vector extracting unit 110, the model learning unit 120, and the sound recognizing unit 130. [0052] FIG.

In step 210, the feature vector extraction unit 110 extracts feature vectors related to the acoustic signal of the UAV based on the Mel Frequency Cepstral Coefficients (MFCC) technique in a real environment in which noise and various acoustic signals exist. Can be extracted.

At this time, the feature vector extracting unit 110 can extract feature vectors corresponding to the UAV from acoustic signals emitted while the propeller of the UAV is rotated using the two MFCC techniques. That is, the feature vector extraction unit 110 may extract feature vectors related to the acoustic signals of the unmanned aerial vehicle based on the envelope of the cepstral domain for a predetermined reference time. Here, the two MFCC schemes may include a first MFCC and a second MFCC having different numbers of scintillation coefficients, the first MFCC may be composed of twenty-four polynomial coefficients (MFCCs), the second MFCC May be composed of 36 Stellar coefficients (MFCC). For example, the first and second MFCCs may be configured as shown in Table 1 below.

Table 1 above shows two MFCC schemes for feature vector extraction. According to Table 1, the first MFCC scheme consists of 12 standard MFCCs and 12 delta MFCCs, and the second MFCC scheme can consist of 12 standard MFCCs and 24 delta MFCCs. Here, the 24 MFCCs can be composed of 12 delta MFCCs calculated based on Equation (11) to be described later and 12 delta MFCCs calculated based on Equation (12).

In step 220, the model learning unit 120 may learn a HMM (Hidden Markov Model) model based on the learning data set generated based on the extracted feature vectors. Here, the initial learning data set can be specified in advance for each type of acoustic signal.

For example, after initializing the HMM parameters, the model learning unit 120 may learn the HMM model using the feature vectors. In other words, HMM parameters can be updated through learning.

In operation 230, the sound recognition unit 130 may recognize (or detect) acoustic signals corresponding to the UAV based on the learned HMM model, on various types of acoustic signals input to the system . That is, the learned HMM model can be used as a classifier to recognize the acoustic signal corresponding to the UAV from the input signals.

Various types of acoustic signals existing in a real environment such as an airplane sound, a bird sound, a car sound, a rain sound, a drone sound, and the like can be input through the sound input sensor of the sound recognition unit 130. Then, the sound recognition unit 130 recognizes the acoustic signal corresponding to the UAV among various types of acoustic signals based on the HMM model learned based on the extracted feature vectors in relation to the UAV have. At this time, the sound of the propeller driven by the motor may be different according to the type of the UAV, and the feature vectors may be extracted and learned according to the types. Accordingly, the sound recognition unit 130 not only recognizes that the inputted sound signal corresponds to the unmanned airplane but also recognizes the recognized UAV related information (for example, the type of the dron, the maker, the size, And the like).

The recognized UAV related information may be displayed through a display (not shown) of the system 100. For example, whether or not the input sound signal corresponds to a drone, and if so, the type, size, and affiliation information (e.g., broadcasting company affiliation, military affiliation, company affiliation, etc.) .

Hereinafter, the operation of extracting feature vectors using two MFCC techniques will be described in more detail with reference to Equations (1) to (12). At this time, three learning data sets may be set together with an increase in the acoustic type, and the learning data (for example, the acoustic signal for each type) included in each learning data set may include an HMM model based on each single cluster It can be classified into several clusters in advance for learning.

A framework (ie, a drone acoustic recognition framework) for recognizing a propeller sound of an unmanned airplane such as a drone can be divided into two steps of extracting a feature vector from a sound signal and classifying the feature vectors by type. Acoustic signals can perform their functions in the frequency domain due to time domain or frequency components due to mobility, speed variation and propagation conditions, and other factors. At this time, in order to form a feature vector set including the feature vectors, the MFCC can be configured after the spectrogram is generated by STFT (Short Time Fourier Transform) to show the acoustic characteristics in the time-frequency domain. STFT is a sort of Fourier transform, performing STFT can indicate that the signal of time is divided into shorter segments of equal length and then Fourier transform is performed separately for each shorter segment. At this time, in the case of continuous-time, the STFT of the acoustic signal can be expressed as Equation 1 below.

[Equation 1]

는 윈도우(window) 함수,

는 푸리에 변환될 신호(예컨대, 드론 등의 음향 신호)를 나타낼 수 있다. 그리고,

는 시간과 주파수에 따른 신호의 위상과 크기를 나타내는 복소 함수인

의 푸리에 변환을 나타낼 수 있다.In Equation (1)

Is a window function,

(E.g., an acoustic signal such as a drone) to be Fourier transformed. And,

Is a complex function representing the phase and magnitude of the signal with respect to time and frequency

Lt; RTI ID = 0.0 > Fourier < / RTI >

At this time, in case of discrete-time, a signal to be Fourier transformed (that is, data corresponding to a sound signal) may be divided into frames and overlap each other. When the signal is divided into several frames, the length of the frame can be analyzed based on the resolution and total duration of the signal to be analyzed. Then, at the discrete time, the Fourier transform for each frame can be expressed as shown in equation (2).

&Quot; (2) "

은 변환될 신호,

은 윈도우 함수를 나타낼 수 있다. 이때, 푸리에 변환 시 STFT 대신 FFT가 사용되는 경우, 상기 수학식 2의 변수 m과 ω는 이산적(discrete)일 수 있다.In Equation (2)

A signal to be converted,

Can represent a window function. In this case, when FFT is used instead of STFT in Fourier transform, the variables m and? In Equation (2) may be discrete.

At this time, the square of the magnitude of the STFT for calculating the spectrogram can be expressed by Equation (3) below.

&Quot; (3) "

FIG. 3 illustrates a spectrogram for various acoustic signals according to an embodiment of the present invention.

In Figure 3, a recorded sound signal of the drones (i. E., A signal 310 of a recorded sound signal emitted upon propeller rotation of the drones) 310 and an airplane 320, an automobile 330, And a spectrogram corresponding to each of various types of acoustic signals.

The acoustic signals 320, 330, and 340 corresponding to airplanes, cars, birds, etc. shown in FIG. 3 may be a major obstacle to recording and recognizing sound signals corresponding to propeller sounds of a drone. Referring to FIG. 3, it can be seen that characteristics of different types of acoustic signals appear in both time and frequency domains. For example, in the case of a drone, it can be confirmed that a strong harmonic component appears at 400 Hz to 8 KHz, and the rest of the acoustic signals 320, 330 and 340 have a harmonic component in a wide frequency range Can be confirmed. As described above, based on the spectrogram of the drone, it can be recognized whether or not the spectrogram (i.e., pattern) of the acoustic signal inputted based on the feature having the harmonic component in the region of 400 Hz to 8 KHz corresponds to the drones.

FIG. 4 is a flowchart illustrating an operation of extracting a feature vector based on an MFCC technique in an embodiment of the present invention. FIG. 5 is a flowchart illustrating an operation of extracting a feature vector Block diagram.

5, the feature vector extractor 500 includes a pre-emphasis processor 510, a framing 520, a windowing 530, an FFT processor 540, Scale filter bank 550, a logarithmic operation unit 560, a DCT (Discrete Cosine Transform) 570, and an MFCC configuration unit (MFCC) 580. 4, each of the steps 410 to 440 includes a pre-emphasis processing unit 510, a framing unit 520, a windowing unit 530, an FFT processing unit FFT 540, a Mel-scale filter bank 550, a logarithmic operation 560, a discrete cosine transform 570, and an MFCC 580 have.

First, the feature vector extractor 500 extracts the envelope of the spectrum in the logarithm region to efficiently extract a feature vector (i.e., acoustic feature) for each sound signal to generate a feature vector set for the sound signal Feature vectors can be extracted from the neural network using the MFCC technique. [1] I. Patel and YS Rao , "Speech recognition using Hidden Markov Model with MFCC - Subband technique, in Proc . International Conference on Recent Trends in Information, Telecommunication and Computing, pp. 168-172, Mar. 2010. In this paper , we propose a method to extract the envelope of the spectrum in the logarithmic domain and to extract the feature vector in the envelope domain.

At this time, various kinds of acoustic signals can be used for extracting the feature vector. For example, acoustic signals of the same property and acoustic signals of different properties may be used. Acoustic signals of the same property can represent acoustic signals corresponding to propeller sounds of different kinds of drones, and acoustic signals of different properties can represent acoustic signals such as new sounds, airplanes, and automobile sounds.

First, the feature vector extractor 500 may convert an acoustic signal into a digital signal. At this time, a digital signal representing each level of the signal in all the discrete time regions may be generated.

In operation 410, the pre-emphasis processor 510 may perform a pre-emphasis process for performing highpass filtering on a digital signal. For example, the pre-emphasis processing unit 510 performs pre-emphasis processing for increasing the total signal-to-noise ratio (SNR) by increasing the amount of energy at a high frequency compressed to a predetermined level by the recording medium with respect to the digital signal . That is, the pre-emphasis processing unit 510 may be configured as a first-order high-pass filter and may perform filtering to increase the size of a relatively higher frequency with respect to the size of the low frequency. The first-order high-pass filter can be expressed by Equation (4) below.

&Quot; (4) "

은 프리 엠퍼시스 처리부(510)로 입력되는 음향 신호(즉, 디지털 신호)를 나타내고,

은 출력 신호를 나타낼 수 있다. 그리고,

는 필터 계수를 나타낼 수 있다.In Equation (4)

(I.e., a digital signal) input to the pre-emphasis processing unit 510,

Can represent an output signal. And,

Can represent filter coefficients.

In operation 420, the feature vector extractor 500 generates a spectrogram through a framing 520, a windowing 530, and an FFT processor 540 on the pre-emphasis processed signal. Can be generated.

For example, in the framing 520 process, a frame shift may be performed and the frame shift may correspond to half of the frame length. At this time, as the frame shift is performed, overlap may occur between adjacent frames. That is, it is possible to induce an overlap between adjacent frames through the framing 520. As described above, windowing 530 may be performed for each frame, with overlapping frames (i.e., the digitized sound signal consisting of a plurality of frames) through a frame shift. At this time, the signal value may be reduced to 0 at the window boundary while avoiding discontinuity through the framing 520 and the windowing 530 process. Then, the FFT processing unit 540 can perform the FFT processing on the windowed signal to generate the spectrogram.

That is, the FFT processing unit 540 can perform fast Fourier transform (FFT) on the windowed signal, and a fast Fourier transform (FFT) output signal can be expressed by Equation (5) below.

&Quot; (5) "

는 i번째 프레임(frame)에 해당하는 음향 데이터를 나타내고,

는 i번째 프레임(frame)에 해당하는 윈도우 함수(window function)를 나타낼 수 있다. 그리고, I는 전체 프레임들의 개수를 나타내고, N은 프레임의 길이,

는 주파수 영역에서 윈도윙된 신호를 나타낼 수 있다. 예컨대, 해밍 윈도우(hamming window)가 이용될 수 있으며, N-포인트 윈도우 함수(N-point window function)는 아래의 수학식 6과 같이 표현될 수 있다.In Equation (5)

Represents the sound data corresponding to the i-th frame,

May represent a window function corresponding to the i-th frame. I denotes the total number of frames, N denotes the length of the frame,

Can represent a windowed signal in the frequency domain. For example, a hamming window may be used, and an N-point window function may be expressed by Equation (6) below.

&Quot; (6) "

In operation 430, the FFT-transformed signal may be output through a Mel-scale filter bank 550. The mel-scale filter bank 550 receives the FFT-transformed signal from the FFT processing unit 540 and can map a frequency to a Mel scale. For example, the frequency may be mapped to the mel Scale based on Equation (7) below.

&Quot; (7) "

) 필터를 구현하기 위한 방정식은 아래의 수학식 8과 같이 표현될 수 있다.Then, the m-th (

) ≪ / RTI > filter can be expressed as Equation (8) below.

&Quot; (8) "

는 멜 스케일 주파수를 나타내고, M은 주파수 대역에서 필터의 총 개수를 나타낼 수 있다.In Equation (8)

Represents the Mel Scale frequency, and M represents the total number of filters in the frequency band.

After the acoustic signal (i.e., the acoustic sequence) of each frame has passed through the melscale filter bank 550, and a weighted spectrum value is added to each filter, one single value per filter is obtained, Can form a mel-spectrum. At this time, the logarithm of the mel-spectrum can be calculated based on the following equation [9].

&Quot; (9) "

) 켑스트럴 계수가 계산될 수 있다.In step 440, the discrete cosine transform (DCT) 570 may perform discrete cosine transform (DCT) on the signal that has passed through the mel-scale filter bank 550 to calculate the cepstral coefficients. For example, by performing the discrete cosine transform based on the following equation (10), the nth (

) Stellar coefficients can be calculated.

&Quot; (10) "

In Equation (10), N may represent the total number of polynomial coefficients.

In step 440, the MFCC component (MFCC, 580) may configure the MFCC based on the calculated cepstral coefficients. For example, for the cepstral coefficients computed based on Equation (10) above, the first 12 cepstral coefficients can be maintained to construct the MFCC.

In Equation (10), the MFCC (i. E., The feature vector) configured based on the calculated cepstral coefficients represents only a power spectral envelope of a single frame, but a delta coefficient may be added to represent dynamic features . For example, the delta coefficient used in the MFCC configuration may be calculated based on Equation (11) below.

&Quot; (11) "

That is, the first 12 cepstral coefficients are calculated based on Equation (10), and then the twelve delta coefficients can be calculated based on Equation (11), and the first 12 cepstral coefficients and the following 12 delta coefficients May correspond to the first MFCC. And, the second MFCC can be calculated by twelve different delta coefficients, and the second MFCC can be constructed based on twelve delta coefficients calculated in a different manner from the coefficients corresponding to the first MFCC. For example, twelve delta coefficients different from the first MFCC can be calculated based on Equation (12) below.

&Quot; (12) "

)는 계수의 차를 계산하기 위해 이용될 수 있다.In equation (12), delta

) Can be used to calculate the difference of the coefficients.

As such, two MFCC techniques (e.g., 24 MFCCs and 36 MFCCs) may be used for feature vector extraction, and 24 MFCCs may be classified into 12 standard MFCCs and 12 And the 36 MFCCs can include 12 standard MFCCs, 12 delta MFCCs based on Equation (11), and 12 delta MFCCs based on Equation (12). Since such optimization is performed prior to the real-time recognition process, the complexity associated with feature extraction and optimization of the learning may not be a significant problem.

Hereinafter, an HMM model is learned based on extracted feature vectors based on two MFCC techniques, for example, 24 MFCCs (i.e., feature vectors) and 36 MFCCs (i.e., feature vectors) The operation of recognizing an aircraft (UAV) will be described. In the present embodiments, a multidimensional HMM (Hidden Markov Model) can be used.

The HMM model may represent a statistical model for an ordered sequence of variables characterized by a parametric random process. In HMMs, states may be hidden, while inputs may be observable / observable. The sequence of the observation vector can be expressed as Equation (13) below, and the HMM model can be expressed as Equation (14) below.

&Quot; (13) "

&Quot; (14) "

는 상태 i에서 상태 j로 천이될 확률을 나타내고, B는 상태당 방출(emission) 확률 분포를 나타낼 수 있다.

는 HMM 모델이 상태 i에 있을 때 관측 k를 방출할 확률을 나타낼 수 있다. 상태 i에서의 원소가

인 경우,

는 초기 상태 분포의 확률로서, 압축 표기법에 따라 아래의 수학식 15와 같이 표현될 수 있다. In Equation 14, N represents the number of hidden states, M represents the number of observations per state, and A represents a state transition matrix. And,

Represents the probability of transition from state i to state j, and B may represent the emission probability distribution per state.

Can represent the probability of releasing the observation k when the HMM model is in state i. The element in state i

Quot;

Is the probability of the initial state distribution, and can be expressed as shown in Equation (15) according to the compressed notation.

&Quot; (15) "

Then, a learning data set can be specified in advance to allow learning of the HMM model. For example, an initial learning data set may be specified in advance for each kind of acoustic signal. An initial learning data set may be predefined in terms of a cluster corresponding to each of a plurality of acoustic signals.

The clusters corresponding to the respective acoustic signals can be expressed by the HMM model. An HMM composed of S clusters can be expressed as Equation (16) below.

&Quot; (16) "

) 클러스터에 해당하는 s번째(

) HMM 모델을 나타낼 수 있다. 그리고, 특징 벡터들을 기반으로 생성된 학습 데이터 세트를 이용하여 HMM 모델(즉, 분류기(classifier))가 학습될 수 있다. 그러면, 학습된 HMM 모델을 사용하여 테스트 데이터 세트(예컨대, 입력되는 음향 신호) 내의 음향 신호가 드론의 프로펠러 소리에 해당하는지, 자동차, 새 소리 등에 해당하는지 여부가 인식될 수 있다.In Equation (16), S is the s-th (

) ≪ / RTI > corresponding to the cluster (

) HMM model. The HMM model (i.e., a classifier) can be learned using the learning data set generated based on the feature vectors. Then, using the learned HMM model, it can be recognized whether the acoustic signal in the test data set (for example, the inputted acoustic signal) corresponds to the sound of the propeller of the drones, or whether it corresponds to a car, a new sound or the like.

The parameters of the HMM model are determined by the learning data included in the learning data set, and the learning data is input data input to the HMM model, and can express the feature vector extracted by the feature vector extracting unit 110. The learned HMM model can be used to find the most probable, i.e., an acoustic signal most likely to match the desired signal. For example, it can be used to search for a signal that is likely to be an acoustic signal of a drones, based on the acoustic signal contained in the input test data set.

The learning data set can be expressed as Equation (17) below.

&Quot; (17) "

It may be desirable to find an HMM parameter that is likely to provide a set of observed data O. Finding such a likely HMM parameter may be an important issue in learning for HMM model optimization. Here, the observation data O can represent a feature vector extracted from a learning data set corresponding to each cluster.

FIG. 6 is a flowchart illustrating an operation of learning a BWA (Baum-Welch Algorithm) based HMM model in an embodiment of the present invention.

FIG. 6 shows the learning process of the HMM model solved based on the BWA.

6, the steps 610 to 630 may be performed by the model learning unit 120 of FIG.

In step 610, after the initial learning data set is designated, a learning data set is generated based on the feature vector, and the HMM model can be initialized.

, 및 역방향 우도(backward likelihood)

를 계산할 수 있다. 위의 비특허문헌 [2] G. B. Singh and H. Song, "Using hidden Markov models in vehicular crash detection," IEEE Transactions on Vehicular Technology, vol. 58, no. 3, pp. 1119-1128, Mar. 2009.에서는 HMM 모델의 상태(state) 별로 우도(likelihood)를 계산하는 자세한 구성을 제시하고 있다.In step 620, the model learning unit 120 calculates a forward likelihood for the state i at time t based on the Forward / Backward algorithm,

, And a backward likelihood

Can be calculated. [ Non-Patent Document 2] GB Singh and H. Song, "Using hidden Markov models in vehicular crash detection," IEEE Transactions on Vehicular Technology, vol. 58, no. 3, pp. 1119-1128, Mar. In 2009. , a detailed structure for calculating the likelihood for each HMM model state is presented.

Then, given the observed sequence O and the HMM model parameter i, the probability of being in state i at time t can be expressed as: < EMI ID = 18.0 >

&Quot; (18) "

At this time, a transition likelihood from state i to state j can be expressed by Equation 19 below.

&Quot; (19) "

In step 630, the HMM parameters may be updated based on the forward and backward likelihoods.

에 대한 학습 시퀀스의 우도(likelihood)를 증가시킬 수 있으며, 상기 새로운 모델

은 아래의 수학식 23과 같이 표현될 수 있다.For example, based on the following equations (20) to (22), the parameters of the HMM model (i.e., the HMM parameters) can be re-estimated and the HMM parameters can be updated with the re-estimated value. Here, the re-estimation is a re-

The likelihood of the learning sequence for the new model can be increased,

Can be expressed by the following equation (23).

&Quot; (20) "

&Quot; (21) "

&Quot; (21) "

&Quot; (23) "

7 is a block diagram illustrating an operation of recognizing a desired acoustic signal based on a learned HMM model in an embodiment of the present invention.

를 혼합한 분류기(classifier)가 완성될 수 있다. 그리고, 테스트 과정에서, HMM 모델을 기반으로 주어진 시퀀스(sequence)의 확률을 최대화하는 상태 시퀀스를 찾기 위해 비터비(Viterbi) 알고리즘이 이용될 수 있다. 이때, 인식 과정에서 특징 벡터

의 시퀀스로 표현되는 입력 음향 신호를 검색하는 것이 중요하며, 인식과정은 시퀀스가 주어지면 가장 높은 확률로 HMM을 찾는 과정을 나타내는 것으로서 아래의 수학식 24와 같이 표현될 수 있다.In the HMM model learning process, HMMs composed of S clusters,

A classifier may be completed. And, in the test process, a Viterbi algorithm can be used to find a state sequence that maximizes the probability of a given sequence based on the HMM model. In this case,

It is important to search for an input sound signal represented by a sequence of the HMMs. The recognition process represents a process of finding an HMM with the highest probability given a sequence, and can be expressed as Equation (24) below.

&Quot; (24) "

For example, the HMM model may use the feature vector 720 corresponding to the sound signal (i.e., sequence) included in the test data set 710 as the input data, and the feature vector 720 with the highest probability By finding a matching HMM, it can be recognized whether the input acoustic signal is a drones related acoustic signal or an airplane related acoustic signal. That is, each of the feature vectors may be classified into clusters belonging to the HMM model matching the highest probability with respect to the input various kinds of feature vectors 720 (730) It may be recognized (740) whether it corresponds to what sound.

Hereinafter, experimental results and performance evaluation results of acoustic recognition of a drone will be described. For the experiment, a database composed of a training data set and a test data set is set, and the database may include a plurality of clusters of acoustic signals for each data set. The operation of extracting feature vectors using the two MFCC techniques can be performed based on different acoustic signals for each cluster. Then, the HMM model is initialized, and the feature vector from the cluster of learning data sets is applied to the initialized HMM model to optimize the HMM parameters. In other words, the HMM model can be learned by optimizing the HMM parameters based on the feature vector and updating the HMM parameters with optimized parameters. Then, an input sound (that is, various types of acoustic signals existing in a real environment) is set as a test data set, and an HMM model is used as a classifier for a test data set, It can be recognized whether or not the signal is an acoustic signal.

At this time, a platform may be designed for recognition of a sound signal corresponding to the drones. The training data set and the test data set may include five clusters (i.e., clusters 1 through 5 corresponding to drones, aircraft, cars, birds, and noises at airports). Table 2 below shows the acoustic signals included in the test data set from the cluster viewpoint.

로 표현될 수 있으며, 3개의 학습 데이터 세트 별 차이는 클러스터 1, 클러스터 2, 클러스터 4, 클러스터 5에 학습을 위하여 1가지, 2가지 또는 3가지 종류의 음향 신호들이 포함되는지 여부에 해당할 수 있다. 여기서, 클러스터 3의 경우, 2가지, 3가지, 또는 5가지 종류의 음향 신호가 수집될 수 있으며, 자동차에 해당하는 음향 신호의 짧은 지속시간으로 인해 각 클러스터에 동일한 음향 데이터 길이가 유지될 수 있다. 그리고, 3개의 학습 데이터 세트 내의 서로 다른 클러스터에 해당하는 각 음향 신호에 대한 특징 벡터들이 HMM 모델로 입력되기 위해 추출될 수 있다.In Table 2, three sets of learning data

, And the difference between the three learning data sets may correspond to whether one, two, or three kinds of acoustic signals are included in the cluster 1, cluster 2, cluster 4, and cluster 5 for learning . Here, in the case of the cluster 3, two, three, or five types of acoustic signals may be collected, and the same acoustic data length may be maintained in each cluster due to the short duration of the acoustic signal corresponding to the vehicle . Then, feature vectors for each acoustic signal corresponding to different clusters in the three learning data sets can be extracted for input to the HMM model.

At this time, the feature vectors may be extracted using the two MFCC techniques, and Table 3 below may indicate the parameters used for extracting the feature vector.

로 표현될 수 있다. 여기서, n은 분류 기호 인덱스(index)를 나타낼 수 있으며, 각각의

내에서 5개의 클러스터에 대해 HMM 모델이 모델링될 수 있다(

). 그러면, 각 HMM 파라미터들은 상기 특징 벡터들에 기초하여 학습되고, 각각 5개의 HMM으로 구성된 3개의 분류기를 이용하여 입력되는 음향 신호가 인식될 수 있다.When the feature vectors are extracted using the parameters of Table 3, three classifiers (i.e., HMM models) corresponding to each of the three learning data sets are constructed to construct a classifier (i.e., HMM model) Can be initialized,

. ≪ / RTI > Where n may represent a classification index,

The HMM model may be modeled for five clusters

). Then, each HMM parameter is learned based on the feature vectors, and an acoustic signal input using three classifiers composed of five HMMs can be recognized.

개가 테스트 샘플인 경우, 분류기를 통해 성공적으로 인식된 샘플 수를 계산하여

로 표현하고, 인식률이라 불리는 확률은

로서 계산될 수 있다. 일반적인 음향 신호의 인식률은 아래의 수학식 25와 같이 표현될 수 있다.For example, when recognizing acoustic signals corresponding to drones based on 24 MFCCs, 100 sample data can be collected for each cluster in each learning data set. Total of the total input sound signal

If the dog is a test sample, the number of successfully recognized samples is calculated through the classifier

, And the probability called the recognition rate is

Lt; / RTI > The recognition rate of a general acoustic signal can be expressed by the following equation (25).

&Quot; (25) "

Table 4 below shows experimental results of acoustic signals recognized using the first MFCC technique.

In Table 4, if the Dron 4 model is the DJI Phantom 3 Pro (Pro) and the Plane 4 is the Airbus A320, 24 MFCCs can be used to represent recognition rates for different learning data sets.

가 클러스터 당 단일 종류의 음향 신호를 포함할 때, 드론(Drone)의 음향 신호에 대한 인식률은 64%에 불과하지만, 클러스터 1에 속하지 않는 다른 입력 음향 신호는 해당 클러스터에서 잘못 분류되지 않을 수 있다.According to Table 4, it can be seen that the recognition rate increases as the type of the acoustic signal included in the learning data set increases. Learning data set

The recognition rate for the acoustic signal of the drone is only 64%, but other input acoustic signals not belonging to cluster 1 may not be misclassified in the cluster.

내에 포함된 음향 신호의 종류가 클러스터 당 2 종류로 증가하면, 최대 정확도는 89%로 증가하고, 학습 데이터 세트

내 음향 신호의 종류가 클러스터 당 3가지 유형으로 증가하면, 정확도에 대한 확률은 100%로 됨을 확인할 수 있다. 즉, 학습 데이터 세트

의 경우, 학습 데이터 세트

및

를 사용할 때 보다 정확도가 상당히 높음을 알 수 있다. 표 4의 실험 데이터를 참조하면, 학습 데이터 세트에 포함된 학습 데이터의 양이 많이 추가될수록 인식률 성능이 향상됨을 알 수 있다.Then,

The maximum accuracy increases to 89%, and when the number of kinds of acoustic signals included in the learning data set

If the type of my acoustic signal increases to three types per cluster, the probability of accuracy is 100%. That is,

, The learning data set

And

It can be seen that the accuracy is much higher than when using Referring to the experimental data in Table 4, it can be seen that the more the amount of learning data included in the learning data set is added, the better the recognition rate performance is.

Likewise, when an acoustic signal corresponding to a sound of an airplane, an automobile, a bird, a rain, or the like of an airport is inputted, each of the sound signals can be classified into its own cluster model. As the learning data related to the acoustic signal increases, The accuracy of recognizing the acoustic signal (i.e., the recognition rate) can be similarly increased.

Table 5 below shows experimental results of acoustic signals recognized using the second MFCC technique.

Table 5 shows the recognition rates for different learning data sets using 36 MFCCs. According to Table 5, it can be seen that the recognition rate is improved by 2% to 6% when 36 MFCCs are used as compared with Table 4. [

FIG. 8 is a graph illustrating an average recognition rate based on feature vectors extracted using the first and second MFCC techniques, according to an embodiment of the present invention. Referring to FIG.

을 학습 데이터 세트(training set 1)

과 함께 사용할 때 평균 인식률이 3% 향상되는 것을 확인할 수 있다. 그리고, 분류기

를 학습 데이터 세트(training set 2)

과 함께 사용할 때 성능이 2.5% 향상되는 것을 확인할 수 있다. 그리고, 2개의 MFCC 기법 모두에서, 학습된 분류기

를 학습 데이터(training set 3)

와 함께 이용한 드론의 음향 신호에 대한 인식률 성능은 모두 최적화된 분류기로 인해 100% 달성함을 확인할 수 있다. 동일한 분류기를 사용하는 두 개의 MFCC 기법들 간의 성능 차이는 주로 음향 신호의 특징들 및 더 긴 유효 특징 벡터들의 동적인 정보와 같은 음향 신호의 특징들과 관련된 세부 사항들이 더 많이 이용되므로, 보다 효율적이고 높은 인식 정확도가 달성될 수 있다.Referring to FIG. 8, when the second MFCC scheme using 36 MFCCs is used rather than the first MFCC scheme using 24 MFCCs,

(Training set 1)

It can be confirmed that the average recognition rate is improved by 3%. Then,

(Training set 2)

The performance is improved by 2.5%. And, in both MFCC schemes,

(Training set 3)

And the recognition rate performance of the acoustic signal of the dron which is used together with the classifier achieved by the optimized classifier is 100%. The performance differences between the two MFCC schemes using the same classifier are more efficient because the details associated with the characteristics of the acoustic signal, such as the characteristics of the acoustic signal and the dynamic information of the longer valid feature vectors, High recognition accuracy can be achieved.

9 is a graph showing recognition rate performance for SNR in an embodiment of the present invention.

9, in order to test the robustness of the recognition rate with respect to the recognized acoustic signals based on the learned HMM model by extracting the feature vectors using the first and second MFCC techniques in a real environment with a lot of noises, Adaptive White Gaussian Noise (AWGN) ) Environment was considered.

According to FIG. 9, it can be confirmed that it is reasonable to increase the SNR in order to improve the performance of the recognition rate. It can be seen that it provides the best performance when using 36 MFCCs and learning data sets 3, and provides the second best performance when using the same learning data set (ie, learning data set 3) and 24 MFCCs . The performance difference between 36 MFCCs and 24 MFCCs is between 0.01% and 0.03%. For 36 MFCCs, the SNR is 20dB, and for 24 MFCCs, the SNR achieves the maximum accuracy of 16dB. Also, even if the SNR is as low as 5 dB, it can be confirmed that the recognition rate is about 80% or more. As a result, according to FIG. 9, the classifier learned using the second MFCC technique using 36 MFCCs and the HMM model having a relatively large amount of learning data is used as a UAV such as a drone in a realistic noisy environment, It can be seen that it is suitable for recognizing the acoustic signal of

As described above, the system and method for recognizing the sound of an unmanned aerial vehicle (UAV) extract feature vectors for recognizing drones using a sound of a propeller of a drone, and by using a classification approach, It is possible to recognize the sound signal corresponding to the drone more accurately in a real environment. Learning data sets and test data sets for various clusters are set as types for various types of acoustic signals, and a feature vector is extracted using 24 MFCCs and 36 MFCCs, and the HMM model is extracted using the extracted feature vectors The recognition rate of three learning data sets using a classifier based on the learned HMM model is as follows. As many types of acoustic signals are included in the learning data set, the classifiers are gradually optimized so that the average recognition rate Can reach almost 100%. In the case of 36 MFCCs, since the dynamic information is considered more than 24 MFCCs (ie, it includes 12 delta coefficients, which are dynamic information), the recognition rate can be 3 to 3.5% better on average.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI > or equivalents, even if it is replaced or replaced.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (5)

A method for recognizing sound of an unmanned aerial vehicle,
Extracting acoustic signal-related feature vectors of a UAV based on a Mel Frequency Cepstral Coefficients (MFCC) technique in an environment where noise exists;
Learning a HMM (hidden Markov Model) model based on the learning data set generated based on the extracted feature vector; And
Recognizing an acoustic signal corresponding to an unmanned aerial vehicle (UAV) on the basis of the learned acoustic signal, based on the learned HMM model
Wherein the sound recognition method comprises the steps of: The method according to claim 1,
Wherein the extracting of the feature vector comprises:
Extracting the feature vector on the basis of a first MFCC and a second MFCC composed of different numbers of cepstral coefficients
Wherein the sound recognition method comprises the steps of: The method according to claim 1,
Wherein the extracting of the feature vector comprises:
Extracting acoustic signal feature vectors of unmanned aerial vehicles based on the envelope of the cepstral domain for a predetermined reference time
Wherein the sound recognition method comprises the steps of: The method according to claim 1,
Wherein the extracting of the feature vector comprises:
A pre-emphasis processing step of performing highpass filtering on an acoustic signal corresponding to the unmanned aerial vehicle;
A spectrogram (spectrogram) representing a feature vector corresponding to an acoustic signal of the UAV is performed by performing framing, windowing, and fast Fourier transform (FFT) on the pre- );
Passing the fast Fourier transformed signal through a Mel-scale filter bank;
Performing a discrete cosine transform (DCT) on a signal passed through the mel-scale filter bank to calculate a cepstral coefficient; And
Constructing the MFCC based on the calculated cepstral coefficients;
Wherein the sound recognition method comprises the steps of: A system for recognizing sound of an unmanned aerial vehicle,
A feature vector extractor for extracting acoustic signal-related feature vectors of a UAV based on a Mel Frequency Cepstral Coefficients (MFCC) technique in an environment where noise exists;
A model learning unit that learns an HMM (hidden Markov Model) model based on the learning data set generated based on the extracted feature vector; And
An acoustic recognition unit for recognizing an acoustic signal corresponding to an unmanned air vehicle (UAV) based on the learned HMM model,
Wherein the sound recognition system comprises:

Images