KR102276964B1 - Apparatus and Method for Classifying Animal Species Noise Robust - Google Patents

Abstract

An apparatus and method for identifying animal species that are robust to a noisy environment detects an animal cry section based on the cry of the animal species, estimates noise based on a section in which no animal cry sounds, and removes noise from the received sound. , it is possible to identify animal species by learning the enhanced acoustic features in spite of various noises.

Description

Apparatus and Method for Classifying Animal Species Noise Robust

The present invention relates to an apparatus for identifying animal species, and more particularly, a noise environment capable of identifying animal species by detecting an animal cry section based on the cry of animal species despite various noises and learning enhanced acoustic characteristics. It relates to an apparatus and method for robust animal species identification.

Until now, research on automatic classification of environmental sound sources has been steadily progressing for many years. Since automatic classification of sound sources can be applied to various fields, such as voice recognition, pattern recognition, situation detection, or situation recognition, directly or indirectly, its importance is increasing.

Railway, electricity, gas, and water projects such as public works are large-scale projects and have a large impact on the natural environment. Therefore, we conduct an environmental impact assessment to identify and analyze negative factors that may affect the local environment in advance to minimize damage. possible means should be taken.

Investigating species in environmental impact assessment is an essential activity to conserve the diversity of living organisms in the area and maintain the ecosystem.

In order to identify biological species, it is possible to actually collect living things or to distinguish them by hearing the cry of living things, but it is impossible for humans to directly distinguish all living species within a specific area.

It is difficult to acquire only the cry of living things because the outdoor environment is a noisy environment unlike collecting the cry information of living things in the indoor environment.

Korean Patent Publication No. 10-2014-0122881

In order to solve this problem, the present invention detects an animal cry section based on the cry of animal species, estimates noise based on a section where no animal cry sounds, and removes the noise from the received sound. An object of the present invention is to provide an apparatus and method for identifying an animal species that is robust to a noisy environment that can identify an animal species by learning the acoustic characteristics reinforced in spite of various noises.

Another object of the present invention is to provide development of an artificial intelligence-based algorithm and system that is robust against noise that can produce good classification performance even for sound signals related to animal species identification obtained in an environment in which a lot of various noises are generated.

An apparatus for identifying an animal species robust to a noise environment according to a feature of the present invention for achieving the above object,

Receives a one-dimensional sound signal that is an acoustic signal of an animal cry, converts the one-dimensional sound signal into a log-mel spectrogram that is a two-dimensional sound feature having a frequency axis and a time axis, and converts the converted two-dimensional sound feature into sound an animal cry section detection unit for detecting a section in which the animal cries exist through section detection;

A section in which the detected animal cries are present is received as input data, a noise signal is estimated using the received input data using a neural network-based feature extraction method, and the estimated noise signal is subtracted from the input data. a feature strengthening unit for enhancing the characteristics of the acoustic signal by removing unnecessary signal components; and

Using a classification algorithm consisting of a convolutional layer and a fully connected layer (FCL) for the characteristics of the reinforced sound signal, as many results as the number of labels are calculated in the output layer, and from the calculated results It is characterized in that it includes an animal species identification unit that derives the label with the highest score as a final result.

The animal cry section detection unit calculates a specific section of the sound signal sampled at a specific frequency through Short Time Fourier Transform (STFT) transformation of the one-dimensional sound signal, and stores the specific section of the calculated sound signal in a Mel filter bank ( Mel Filter Bank), the logarithmic operation is performed to convert the log-mel spectrogram, which is the two-dimensional acoustic feature, and further comprises an acoustic feature extraction unit that selects neighboring acoustic feature vectors. .

The animal cry section detector receives the acoustic feature vectors from the acoustic feature extractor and performs a convolution operation with 32 1×1 filters, which are convolutional layers in the frequency axis direction, and , perform diated convolution operation with convolution layers that are 64 5×1 filters, and perform convolution operations with convolution layers that are 32 1×1 filters for residual connection to extract features. It is characterized in that it further comprises a convolution operation unit.

The animal cry section detection unit applies a sigmoid function having a value between 0 and 1 to the convolution operation result received from the convolution operation unit to extract the sound feature with improved animal cry frequency band. It is characterized in that it further comprises an acoustic characteristic improvement unit.

A method for identifying an animal species robust to a noise environment according to a feature of the present invention comprises:

Receives a one-dimensional sound signal that is an acoustic signal of an animal cry, converts the one-dimensional sound signal into a log-mel spectrogram that is a two-dimensional sound feature having a frequency axis and a time axis, and converts the converted two-dimensional sound feature into sound detecting a section in which the animal cry is present through section detection;

A section in which the detected animal cries are present is received as input data, a noise signal is estimated using the received input data using a neural network-based feature extraction method, and the estimated noise signal is subtracted from the input data. enhancing the characteristics of the acoustic signal by removing unnecessary signal components; and

Using a classification algorithm consisting of a convolutional layer and a fully connected layer (FCL) for the characteristics of the reinforced sound signal, as many results as the number of labels are calculated in the output layer, and from the calculated results and deriving the label with the highest score as the final result.

According to the above-described configuration, the present invention has an effect that, when a robust species identification algorithm and system are applied to a noisy environment, accurate ecosystem environment can be identified through biological species identification performance even if a signal is obtained in a noisy environment.

The present invention has the effect of being able to achieve improved performance in any noise environment by applying to an algorithm and system using voice data as well as biological species identification.

The present invention has the effect of being able to grasp the unique characteristics of living things in a given area based on accurate ecosystem environment information from the collected data, and conserving living things through environmental improvement and maintenance.

1 is a block diagram showing the configuration of an animal species identification device robust to a noise environment according to an embodiment of the present invention.
2 is a block diagram schematically illustrating an internal configuration of an animal cry section detection unit according to an embodiment of the present invention.
3 is a view showing a feature strengthening process performed in the feature strengthening unit according to an embodiment of the present invention.
4 is a diagram illustrating a noise component detection process according to an embodiment of the present invention.
5 is a diagram illustrating a domain adaptation process according to an embodiment of the present invention.
6 is a diagram illustrating a CNN structural diagram for recognizing enhanced acoustic signal features according to an embodiment of the present invention.

Hereinafter, the present invention will be described in detail by describing preferred embodiments of the present invention with reference to the drawings.

Throughout the specification of the present invention, when a part "includes" a certain component, it means that other components may be further included, rather than excluding other components, unless otherwise stated.

1 is a block diagram showing the configuration of an animal species identification device robust to a noise environment according to an embodiment of the present invention, and FIG. 2 is a block diagram schematically showing the internal configuration of an animal cry section detection unit according to an embodiment of the present invention 3 is a diagram illustrating a feature enhancement process performed by the feature enhancement unit according to an embodiment of the present invention, FIG. 4 is a diagram illustrating a noise component detection process according to an embodiment of the present invention, and FIG. 5 is a diagram of the present invention It is a diagram illustrating a domain adaptation process according to an embodiment, and FIG. 6 is a diagram illustrating a CNN structure for recognizing an enhanced acoustic signal feature according to an embodiment of the present invention.

The apparatus 100 for identifying an animal species robust to a noise environment according to an embodiment of the present invention includes a preprocessor 110 for detecting an animal cry section, and a classifier 120 for detecting a noise component and performing domain adaptation.

The animal species identification device 100 detects an animal cry section from the received sound based on the cry of the animal species, and removes the noise from the received sound by estimating the noise based on the section in which the animal cry does not exist. Then, it is possible to identify the animal species by learning the acoustic features reinforced in the presence of various noises.

The preprocessor 110 is a task performed before classifying sounds, and consists of extraction of acoustic features, detection of a sound section, and noise removal.

The preprocessor 110 performs a function of detecting a section of an animal cry and includes an acoustic signal input unit 111 , an animal cry section detecting unit 112 , and a sound quality improvement unit 113 .

The animal cry section detecting unit 112 applies a Neural Attentive Voice Activity Detection (NAVAD) algorithm to detect a section in which an animal cry is present in the input sound signal.

The animal cry section detection unit 112 includes an acoustic feature extraction unit 112a, a convolution operation unit 112b, an acoustic characteristic improvement unit 112c, an attention module unit 112d, and a final probability calculation unit 112e, This configuration module can be seen as a structural diagram of the NAVAD algorithm.

The acoustic signal input unit 111 receives an acoustic signal of a cry of an animal to be detected.

The acoustic feature extraction unit 112a is a process of converting a one-dimensional acoustic signal into a log-mel spectrogram that is a two-dimensional acoustic feature having a frequency axis and a time axis.

The acoustic feature extraction unit 112a receives a one-dimensional sound signal from the sound signal input unit 111, converts the one-dimensional sound signal to STFT (Short Time Fourier Transform), and passes the Mel filter bank After that, a logarithmic operation is performed and converted into a log-mel spectrogram, which is a two-dimensional acoustic characteristic.

The acoustic feature extraction unit 112a includes an STFT transform module and a Mel filter bank unit.

The STFT conversion module may analyze the frequency of the sound signal after segmenting the sound signal using a window of a certain length.

The Mel filter bank unit may extract a feature value of an input sound signal by using a frequency analyzed by the STFT transform module by a Mel Frequency Cepstral Coefficient (MFCC) method.

The acoustic feature extraction unit 112a converts the log-mel spectrogram, which is a two-dimensional acoustic characteristic, into a log-mel spectrogram, and then selects seven adjacent acoustic characteristic vectors among them, and transmits the selection to the convolution operation unit 112b.

In other words, the acoustic feature extraction unit 112a calculates a specific section of the acoustic signal sampled at a specific frequency through STFT transformation, and passes the calculated specific section of the acoustic signal through a Mel filter bank to a specific frequency. The Mel filterbank value of the band is obtained, and a logarithmic operation is performed on the obtained Mel filterbank value, which is converted into a log-mel spectrogram, which is a two-dimensional acoustic characteristic.

The convolution operation unit 112b performs a function of extracting a feature of the input data through a convolution operation between the input data and a filter that is a set of weights.

The convolution operation unit 112b receives seven acoustic feature vectors, that is, as a convolutional layer, which is 32 1×1 filters in the frequency axis direction, to the input acoustic feature as shown in Equation 1 below. After performing a convolution operation, ReLU (Recified Linear Unit), a non-linear activation function, is applied.

는 i번째 음향 특징 벡터,

는 1×1 크기를 갖는 j번째 필터이며,

는 i번째 입력

과 j번째 필터

의 컨볼루션(Convolution) 연산을 통해 나온 결과값이다.here,

is the ith acoustic feature vector,

is the j-th filter with a size of 1×1,

is the i-th input

and the j-th filter

It is the result obtained through the convolution operation of .

Then, the convolution operation unit 112b performs a diated convolution operation with the convolution layer, which is a convolution layer, which is a filter of 64 5 × 1 size filters, and again 32 1 × 1 size filters for residual connection with the previously extracted features. Convolution operation is performed with a convolution layer. After each convolution operation, a nonlinear activation function ReLU is applied.

The corresponding convolution operation is as shown in Equation 2 below.

는 5×1 크기를 갖는 j번째 입력 필터에 상응하는 k번째 필터이며,

는 5×1 크기를 갖는 k번째 입력 필터에 상응하는

번째 필터이다.here,

is the k-th filter corresponding to the j-th input filter having a size of 5×1,

is corresponding to the k-th input filter of size 5×1

is the second filter.

In order to match the dimensions of the input and output used in the network, a convolution operation is finally performed as shown in Equation 3 below with a filter having a size of 1×1.

는 1×1 크기를 갖는

번째 필터이고,

는 입력

와 크기와 차원이 같다.here,

has a size of 1×1

is the second filter,

is input

is the same size and dimension as

이 추출된다(수학식 4). 여기서, 시그모이드 함수는 입력을 0과 1 사이의 출력값으로 정규화시키는 활성화 함수이다.The acoustic characteristic improvement unit 112c applies a sigmoid function having a value between 0 and 1 to the convolution operation result received from the convolution operation unit 112b, so that the frequency band of the animal cry is improved.

is extracted (Equation 4). Here, the sigmoid function is an activation function that normalizes an input to an output value between 0 and 1.

을 어텐션 모듈부(112d)에 입력한다.The acoustic characteristic improvement unit 112c is the extracted improved acoustic characteristic.

is input to the attention module unit 112d.

The attention module unit 112d is a technique for finding various acoustic events by using a pattern recognition technique for acoustic signals, and is composed of a Long Short-Term Memory (LSTM) and an Attention Layer of a Recurrent Neural Network (RNN) model.

The pattern recognition technique is a prediction method using an artificial neural network, and when the result value of the output layer is predicted from the input layer, the input value can be predicted from the result values in the learning process. Since the artificial neural network does not have a one-to-one correspondence between the input value and the output value, it is impossible to restore the input layer as an output layer as it is, but the output data calculated from the result value by the Backpropagation (Backpropa) algorithm in consideration of the prediction algorithm If is different from the original input data, it can be considered that the prediction of the artificial neural network is inaccurate. Therefore, learning is trained by changing the prediction coefficients so that the output data calculated under the constraint is similar to the original input data.

A deep neural network refers to a neural network composed of several layers among neural network algorithms. A layer consists of several nodes, and actual calculations are performed at the nodes, and this calculation process is designed to mimic the processes occurring in the neurons constituting the human neural network. A typical artificial neural network is divided into an input layer, a hidden layer, and an output layer. The input data becomes the input of the input layer, and the output of the input layer becomes the input of the hidden layer and the output of the hidden layer. becomes the input of the output layer, and the output of the output layer becomes the final output.

The attention module unit 112d uses a prediction deep neural network that receives input data from an input layer and outputs a predicted value to a buffer of an output layer. The structure or shape of the prediction deep neural network is not limited, and a representative method is a Deep Neural Network (DNN). ), Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), etc., and by combining each neural network, a predictive deep neural network can be constructed to construct a deep neural network of various structures. The attention module unit 112d of the present invention uses an RNN as a structure of a prediction deep neural network.

The RNN uses the 40th order (T × 40) log Mel filter bank value of the T frame as an input, and the overall structure is 3 GRU (Gated Recurrent Unit) layers with 256 units, 4 FNN layers, and finally It consists of an output layer with a sigmoid function.

Here, in the FNN (Feedforward Neural Network), the features of the 40th order log Mel filter bank value are combined with 5 consecutive frames to form a 200-dimensional feature vector, which is used as an input to the FNN. The two hidden layers consist of 1600 units each having a Relu activation function.

The output layer has as many units as the number of acoustic event classes to be classified, and each unit uses a sigmoid activation function. Here, the output value of the sigmoid activation function is regarded as the posterior probability for each class, and after being binarized by comparing it with a reference value of 0.5, it is used for calculating the accuracy of the FNN by comparing it with the Ground Truth Table.

The RNN model is a neural network structure that processes sequential data such as speech and sentences. The state, which can be said to be the memory of the RNN, is repeatedly updated whenever input data comes in, and has information summarizing the input data from the beginning to the current time t.

As a result, after processing all the inputs, the state of the RNN becomes information that summarizes the entire input data. In the actual model training, in the state at t, the result value and state value of t are calculated with the input data at t and the state value of t-1.

RNN minimizes the cost function by using the gradient descent method based on back-propagation and finds the optimal variable value.

LSTM refers to a kind of RNN method that predicts the result using the forgetting gate instead of the weight of the RNN. When processing data sequentially in predicting time-series input data, when processing old data in the RNN method, old data is reduced according to the weight, and when it exceeds a certain stage, the value becomes 0, Regardless, there are issues that are no longer reflected.

In the case of LSTM, since addition is used instead of multiplication, there is an advantage that the recurrent input value does not become 0.

When the acoustic feature extracted from the acoustic feature improving part 112c is input, the attention module unit 112d determines which acoustic characteristic among the 7 adjacent acoustic characteristics contains an animal cry according to Equation 5 below. Attention) information is calculated as a probability.

와

는 Attention Layer의 학습 파라미터이며,

는 LSTM Layer의 내부 state,

는 개선된 i번째 음향 특징이다.where a,

Wow

is the learning parameter of the attention layer,

is the internal state of the LSTM layer,

is the improved i-th acoustic feature.

을 가중 평균 계산을 수행하여 하나의 벡터를 계산한다(수학식 6).The attention module unit 112d provides attention information calculated through Equation 5 and improved acoustic characteristics.

One vector is calculated by performing a weighted average calculation (Equation 6).

The attention module unit 112d transmits one calculated vector c to the final probability calculation unit 112e.

The neural network structure that processes sequential data, such as RNN, has a disadvantage in that the initially stored information disappears as time increases. To compensate for this, an attention algorithm is used. Attention algorithm outputs the state vector h at all times by processing sequential input data with RNN. At this time, the final result is derived by giving attention to a specific time, rather than referring to the vectors at all times at the same rate.

The final probability calculation unit 112e inputs the attention information calculated by the attention module unit 112d into two Fully Connected Layers (FCLs) having one output node and a sigmoid activation function, and finally the corresponding section The probability of the presence of an animal cry is derived (Equation 7).

Equations 1 to 7 are performed for all sections of the input sound signal.

In the acoustic signal, the noise removal step is applied to the part where the animal cry is not present, and the feature enhancement step is applied to the part where the animal cry is present.

The sound quality improvement unit 113 receives the sound signal in which the animal cry section is detected from the animal cry section detector 112 , and removes the noise signal from the received sound signal to improve the quality of the sound signal.

Since the noise signal changes with time, the variance of the noise is updated whenever there is no animal cry, and the posterior signal to noise ratio (SNR) and the prior signal to noise ratio (SNR) are estimated.

The noise cancellation technique can be expressed as a spectral gain according to the signal-to-noise ratio. Accordingly, the sound quality improvement unit 113 receives the sound signal in which the animal cry section is detected from the animal cry section detection unit 112, and the post-SNR of the received sound signal is obtained by the following Equations (8) and (9). and calculate the prior SNR.

은 잡음이 섞인 신호 대비 잡음비이고, 사전 SNR

은 잡음이 섞이지 않은 신호 대비 잡음비이다.post-mortem SNR

is the noise-to-noise ratio, and the pre-SNR

is the noise-free signal-to-noise ratio.

In a given environment, since it is impossible to know a signal without noise, a process of estimating the prior SNR is required, and the posterior SNR and the prior SNR are estimated as shown in Equations 8 and 9 below.

는 k번째 주파수 성분에서 계산된 사후 SNR,

는 음향 신호의 k번째 주파수 파워 스펙트럼,

는 잡음 신호의 k번째 주파수 파워 스펙트럼,

는 k번째 주파수 성분에서 추정된 사전 SNR,

는 이전 시간에 추정된 사전 SNR을 나타낸다. 최초의 사전 SNR 값은 0으로 초기화한다. here,

is the posterior SNR calculated from the k-th frequency component,

is the k-th frequency power spectrum of the acoustic signal,

is the kth frequency power spectrum of the noise signal,

is the pre-SNR estimated at the k-th frequency component,

denotes the prior SNR estimated at the previous time. The first prior SNR value is initialized to 0.

The sound quality improving unit 113 calculates a spectral gain by substituting the pre-SNR estimated using Equations 8 and 9 into the gain function of Equation 10 below, has a value between 0 and 1, and input Each frequency component of the acoustic signal is multiplied directly.

는 스펙트럼 이득,

는 k번째 주파수 성분에서 추정된 사전 SNR이다.here,

is the spectral gain,

is the pre-SNR estimated at the k-th frequency component.

에 직접 곱하여 잡음이 제거된 음향 신호의 k번째 주파수 파워 스펙트럼

를 계산하게 된다.The sound quality improvement unit 113 calculates the gain function Gaink as the k-th frequency power spectrum of the noisy sound signal.

The k-th frequency power spectrum of the denoised acoustic signal by directly multiplying

will be calculated

When the sound quality improvement unit 113 receives the sound signal in which the animal cry section is detected from the animal cry section detection unit 112 and performs a multiplication operation on each frequency component of the received sound signal by a spectral gain, the noise signal can be removed.

The sound quality improving unit 113 cuts a section in which an animal cry is present from which the noise signal is removed and transmits it to the feature enhancing unit 121 .

The classifier 120 includes a feature enhancement unit 121 and an animal species identification unit 122 .

The feature reinforcement unit 121 receives an input spectrogram in which an animal cry is present from the sound quality improvement unit 113 , estimates a noise signal using the received input spectrogram through a neural network-based feature extraction method, and input data By subtracting the noise signal estimated in , unnecessary components are removed to enhance the characteristics of the acoustic signal. A spectrogram is a combination of waveform and spectrum characteristics to visualize and understand sound or waves. The change of the amplitude axis according to the change of the time axis can be seen in the waveform, and the change of the amplitude axis according to the change of the frequency axis can be seen in the spectrum. The spectrogram shows the difference in amplitude according to the change of the time axis and the frequency axis as the difference in print density and display color.

The feature enhancement unit 121 processes the negative part as 0 through the nonlinear activation function ReLU after the subtraction operation, and adjusts the feature range to 0 to 1 through a normalization process. This is to maintain the positive definition and to prevent loss of features due to the subtraction operation.

The feature reinforcement unit 121 is a process of learning the neural network to be robust to domain changes such as noise and recording locations using RNN-LSTM and Auto Encoder algorithm.

In other words, learning (noise component detection and domain adaptation) of the feature reinforcement network is learned in an end-to-end manner simultaneously with the classifier 120 .

The feature enhancement unit 121 performs energy-based noise component detection (FIG. 4).

차원을 가지는 입력 스펙트로그램에서 특징 요소 간 에너지 크기를 학습하고, 에너지가 상대적으로 작으면서 학습에 불필요한 성분을 제거한다.The feature strengthening unit 121 is

In an input spectrogram having a dimension, the amount of energy between the feature elements is learned, and components unnecessary for learning are removed while the energy is relatively small.

The feature enhancement unit 121 performs dimensionality reduction and expansion, and extracts low-dimensional features for each axis of the input spectrogram using RNN-LSTM.

차원을 가지며 N > K의 조건을 만족한다. 주파수 축은

차원을 가지며 M > K의 조건을 만족한다.For the time axis,

It has dimension and satisfies the condition of N > K. the frequency axis

It has a dimension and satisfies the condition M > K.

The feature reinforcement unit 121 expresses each feature element in a range of -1 to 1 using Equation 12, which is a hyperbolic tangent function Tanh, and uses a nonlinear activation function (Rectifued Linear Unit, RELU) Negative values are made 0 through Equation (13). Through this process, the LSTM output vector for each axis always has a probability value in the range of 0 to 1.

As shown in FIG. 4 , the feature enhancement unit 121 calculates an enhanced spectrogram in which two features obtained through RNN-LSTM are divided into a noise component and a non-noise component using Equation 14 below. .

V ₁ , V ₂ , and X represent a frequency-axis LSTM output vector, a time-axis LSTM output vector, and input data, respectively.

는 각각 벡터 외적과 성분별 곱셈을 나타낸다.

는 수학식 12, 수학식 13을 연속적으로 적용한 함수이다.

represents the vector cross product and component multiplication, respectively.

is a function to which Equations 12 and 13 are successively applied.

차원의 행렬은 입력 스펙트로그램의 구성 요소에 0 내지 1 범위를 가지는 가중치를 주기 때문에 학습에 불필요한 성분을 검출하는 역할을 수행한다.obtained through the cross product of each LSTM output vector

Since the dimensional matrix gives weights having a range of 0 to 1 to the components of the input spectrogram, it plays a role of detecting components unnecessary for learning.

After detecting the noise component, the feature enhancement unit 121 performs domain adaptation. In the spectrogram calculated using Equation 14, the convolutional neural network (CNN)-based bottleneck structure as shown in FIG. Domain adaptation is performed by applying an auto encoder algorithm.

In the case of a typical CNN-based autoencoder algorithm, it serves to transform the input data into a domain suitable for subtraction operation. Here, the auto encoder algorithm is an algorithm that compresses high-dimensional input data into a low-dimensional one and then restores it back to original data. In the encoding stage, the input data is compressed, and in the process of compression, important features of the input data are extracted through a neural network.

In the decoding step, the features compressed in the encoding step are received and restored to the initial input data, and in this process, noise is removed so that only the information of the desired input data can be viewed.

CNN repeatedly performs convolution and polling operations on input data, and reduces the number of variable values to learn while preserving the characteristics of the input data while reducing the size of the data. A layer performing each operation is represented by a convolutional layer and a polling layer.

The convolutional layer performs convolutional association with a two-dimensional filter of a fixed size on a two-dimensional input. The polling layer uses the result of the convolutional layer as input and reduces the dimension by selecting the maximum value within the neighboring region.

The feature enhancement unit 121 configures four encoder layers and two decoder layers to perform domain adaptation.

In the encoder layer, features are extracted while reducing the output size and dimension through a convolutional layer of a certain size, and in the decoder layer, the output size is converted into an input spectrogram while increasing the dimension of the output size through a convolutional layer of a constant size. do the same with

Each time the encoder layer that has performed the convolution operation is turned off, the feature enhancement unit 121 performs a maximum polling layer, a batch normalization that is a normalization operation, and a RELU function that is an activation function. The maximum polling layer refers to a process of performing a convolution operation on the convolution layer and then resizing the size to make it.

Batch normalization is placed in front of the activation function, which can be learned through back propagation.

The feature enhancement unit 121 performs batch normalization, which is a normalization operation, and a RELU function, which is an activation function, whenever the decoder layer that has performed the convolution operation is turned off.

The feature enhancement unit 121 removes unnecessary components by subtracting the noise feature, which is a result of domain adaptation in the input spectrogram, and then applies a nonlinear activation function ReLU (Recified Linear Unit), min in the range of 0 to 1 -max Reinforces the characteristics of the acoustic signal by adjusting the size of the feature through normalization.

As shown in FIG. 6 , the animal species identification unit 122 is composed of a CNN-based classification algorithm and a Fully Connected Network (FCN), and finally derives a final result through a Softmax function. In other words, the animal species identification unit 122 includes five convolutional layers and two fully connected layers (FCL).

The animal species identification unit 122 performs a convolution operation on the first, second, and last convolution layers, and then performs Max Pooling.

The animal species identification unit 122 performs a convolution operation in a convolution layer when a feature of an acoustic signal reinforced from the feature reinforcement unit 121 is input, and after performing a convolution operation in the last convolution layer, FCN is connected to and yields the number of labels in the output layer

The animal species identification unit 122 displays the calculated result as a score through the Softmax function of Equation 15 below, and finally sets the label with the highest score to the final result (S). (yi)).

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improved forms of the present invention are also provided by those skilled in the art using the basic concept of the present invention as defined in the following claims. is within the scope of the right.

100: animal species identification device 110: pre-processing unit
111: sound signal input unit 112: animal cry section detection unit
112a: acoustic feature extraction unit 112b: convolution operation unit
112c: acoustic characteristic improvement unit 112d: attention module unit
112e: final probability calculation unit 113: sound quality improvement unit
120: classifier 121: feature enhancement unit
122: animal species identification unit

Claims (18)

Receives a one-dimensional sound signal that is an acoustic signal of an animal cry, converts the one-dimensional sound signal into a log-mel spectrogram that is a two-dimensional sound feature having a frequency axis and a time axis, and converts the converted two-dimensional sound feature into sound an animal cry section detection unit for detecting a section in which the animal cries exist through section detection;
A section in which the detected animal cries are present is received as input data, a noise signal is estimated using the received input data using a neural network-based feature extraction method, and the estimated noise signal is subtracted from the input data. a feature strengthening unit for enhancing the characteristics of the acoustic signal by removing unnecessary signal components; and
Using a classification algorithm consisting of a convolutional layer and a fully connected layer (FCL) for the characteristics of the reinforced sound signal, as many results as the number of labels are calculated in the output layer, and from the calculated results an animal species identifier that results in a label with the highest score as a final result;
The animal cry section detection unit,
A specific section of the sound signal sampled at a specific frequency is calculated through Short Time Fourier Transform (STFT) transformation of the one-dimensional sound signal, and a specific section of the calculated sound signal is passed through a Mel filter bank. thereafter, an acoustic feature extraction unit that performs a logarithmic operation to convert the log-mel spectrogram which is the two-dimensional acoustic feature, and selects neighboring acoustic feature vectors from among them; and
The acoustic feature vectors are received from the acoustic feature extraction unit, and a convolution operation is performed with 32 1X1-size filters in a convolutional layer in the frequency axis direction, and 64 5X1-size filters are convolutional. Animal species, characterized in that it further comprises a convolution operation unit that performs a diated convolution operation with a convolution layer, and extracts features by performing a convolution operation with a convolution layer that is again 32 1X1 size filters for residual connection identification device. delete delete According to claim 1,
The animal cry section detection unit applies a sigmoid function having a value between 0 and 1 to the convolution operation result received from the convolution operation unit, so that the animal cry frequency band is improved acoustic features are extracted Animal species identification device, characterized in that it further comprises an acoustic characteristic improvement unit. 5. The method of claim 4,
The animal cry section detection unit is composed of a Long Short-Term Memory (LSTM) and an attention layer of a Recurrent Neural Network (RNN) model, and when an acoustic characteristic extracted from the acoustic characteristic improvement unit is input, the extracted sound Animal species identification device, characterized in that it further comprises an attention module for calculating attention (Attention) information according to the following Equation (1) in which acoustic feature of the features the animal cry is present.
[Equation 1]

where a,

Wow

is the learning parameter of the attention layer,

is the internal state of the LSTM layer,

is an improved i-th acoustic feature. 6. The method of claim 5,
The animal cry section detection unit inputs the attention information calculated by the attention module unit to two fully connected layer units (FCL) having one output node and a sigmoid activation function, and finally the animal cries in the corresponding section. Animal species identification device, characterized in that it further comprises a final probability calculator for deriving the probability that the sound exists. 7. The method of claim 6,
An acoustic signal in which an animal cry section is detected is received from the final probability calculator, and a posterior signal to noise ratio (SNR) is obtained from the received acoustic signal by Equations 2 and 3 below. The prior SNR and the prior SNR are calculated, the spectral gain is calculated by substituting the calculated prior SNR into the gain function of Equation 4 below, and the spectral gain is applied to each frequency component of the received sound signal. The apparatus for identifying animal species according to claim 1, further comprising a sound quality improvement unit that removes a noise signal when a multiplication operation is performed.
[Equation 2]

[Equation 3]

here,

is the posterior SNR calculated from the k-th frequency component,

is the k-th frequency power spectrum of the acoustic signal,

is the kth frequency power spectrum of the noise signal,

is the pre-SNR estimated at the k-th frequency component,

is the prior SNR estimated at the previous time.
[Equation 4]

here,

is the spectral gain,

is the pre-SNR estimated at the k-th frequency component. 8. The method of claim 7,
The feature reinforcement unit receives an input spectrogram in which an animal cry is present from the sound quality improvement unit, and uses the received input spectrogram as a Long Short-Term Memory (LSTM) of a Recurrent Neural Network (RNN) model. An apparatus for identifying an animal species, characterized in that a feature is extracted for each axis of the input spectrogram, and the extracted feature is calculated as a spectrogram in which a noise component and a non-noise component are separated. 9. The method of claim 8,
The feature enhancement unit configures one or more Convolutional Neural Network (CNN)-based encoder layers and one or more decoder layers in the calculated spectrogram to compress high-dimensional input data to a low dimension, and then restores it back to original data. Auto encoder (Auto Encoder) An apparatus for identifying an animal species, characterized in that it performs domain adaptation by applying an algorithm. 10. The method of claim 9,
The feature enhancement unit removes unnecessary components from the received input spectrogram, which is a result of domain adaptation, by subtraction operation, and then applies a non-linear activation function ReLU (Recified Linear Unit), and ranges from 0 to 1. An apparatus for identifying an animal species, characterized in that the characteristic of the acoustic signal is reinforced by adjusting the size of the characteristic through min-max normalization. 11. The method of claim 10,
The animal species identification unit uses a CNN-based classification algorithm consisting of five convolutional layers and two fully connected layers (FCL) when the characteristics of the acoustic signal reinforced from the feature reinforcement unit are input. to calculate the number of labels in the output layer, and the calculated result is expressed as a score through the Softmax function of Equation 5 below, and the label with the highest score is Animal species identification device, characterized in that the classification by the final result (S(yi)).
[Equation 5]

An animal species identification method in which an animal species identification device identifies an animal species, the method comprising:
Receives a one-dimensional sound signal that is an acoustic signal of an animal cry, converts the one-dimensional sound signal into a log-mel spectrogram that is a two-dimensional sound feature having a frequency axis and a time axis, and converts the converted two-dimensional sound feature into sound detecting a section in which the animal cry is present through section detection;
A section in which the detected animal cries are present is received as input data, a noise signal is estimated using the received input data using a neural network-based feature extraction method, and the estimated noise signal is subtracted from the input data. enhancing the characteristics of the acoustic signal by removing unnecessary signal components; and
Using a classification algorithm consisting of a convolutional layer and a fully connected layer (FCL) for the characteristics of the reinforced sound signal, as many results as the number of labels are calculated in the output layer, and from the calculated results deriving the label with the highest score as the final result;
The step of detecting the section in which the cry of the animal is present,
A specific section of the sound signal sampled at a specific frequency is calculated through Short Time Fourier Transform (STFT) transformation of the one-dimensional sound signal, and a specific section of the calculated sound signal is passed through a Mel filter bank. Then, performing a log operation to convert the log-mel spectrogram, which is the two-dimensional acoustic feature, and selecting neighboring acoustic feature vectors from among them;
The acoustic feature vectors are received as input, and a convolution operation is performed with 32 1X1 size filters, convolutional layers, in the frequency axis direction, and diated convolution with 64 5X1 size filters, convolutional layers. Animal species identification method, characterized in that it further comprises the step of extracting features by performing an operation and performing a convolution operation with a convolution layer that is a convolution layer that is a filter of 32 1X1 size for residual connection. 13. The method of claim 12,
The step of detecting the section in which the cry of the animal is present,
By applying a sigmoid function having a value between 0 and 1 to the result of the convolution operation received from the step of extracting the feature by performing the convolution operation, an acoustic feature with an improved animal cry frequency band is extracted Animal species identification method, characterized in that it further comprises the step of. 14. The method of claim 13,
The step of detecting the section in which the cry of the animal is present,
It is composed of an LSTM (Long Short-Term Memory) and an attention layer of a Recurrent Neural Network (RNN) model, and when an acoustic feature extracted in the step of extracting the improved acoustic feature is input, any of the extracted acoustic features Animal species identification method, characterized in that it further comprises the step of calculating attention (Attention) information according to the following Equation 1 whether there is an animal cry in the acoustic characteristic.
[Equation 1]

where a,

Wow

is the learning parameter of the attention layer,

is the internal state of the LSTM layer,

is an improved i-th acoustic feature. 15. The method of claim 14,
Inputting the calculated attention information to two Fully Connected Layers (FCL) having one output node and a sigmoid activation function to finally derive the probability that an animal cry exists in the corresponding section. Animal species identification method, characterized in that. 16. The method of claim 15,
Reinforcing the characteristics of the sound signal comprises:
Receive the input spectrogram in which the cry of the animal exists, and use the received input spectrogram for each axis of the input spectrogram using Long Short-Term Memory (LSTM) of a Recurrent Neural Network (RNN) model. and calculating the extracted feature as a spectrogram in which a noise component and a non-noise component are separated. 17. The method of claim 16,
An auto encoder that configures one or more encoder layers and one or more decoder layers based on a CNN (Convolutional Neural Network) based on the calculated spectrogram to compress high-dimensional input data into a low-dimensional one, and then restores it back to the original data. performing domain adaptation by applying an algorithm; and
After removing unnecessary components from the received input spectrogram by subtracting the noise feature that is the result of performing the domain adaptation, a nonlinear activation function ReLU (Recified Linear Unit) is applied, and min-max normalization in the range of 0 to 1 (Normalization) The method of identifying an animal species, characterized in that it further comprises the step of enhancing the characteristic of the acoustic signal by adjusting the size of the characteristic. 18. The method of claim 17,
The step of deriving the final result is,
When the characteristics of the reinforced sound signal are input, the number of labels in the output layer is the result using a CNN-based classification algorithm consisting of five convolutional layers and two fully connected layers (FCL). , and the calculated result is expressed as a score through the Softmax function of Equation 2 below, and the label with the highest score is the final result (S(yi)). Animal species identification method, characterized in that it further comprises the step of classifying.
[Equation 2]

Images