CN108122562A - A kind of audio frequency classification method based on convolutional neural networks and random forest - Google Patents

Abstract

The invention discloses an audio classification method based on a convolutional neural network and a random forest. The method includes: S1: performing spectrum analysis on an original audio data set, including segmenting, framing, windowing, and Fourier transform, to obtain The spectrogram corresponding to the original audio file; S2: use the obtained spectrogram as input, train a convolutional neural network feature extractor; S3: remove the softmax layer of the convolutional neural network, and extract the high-level features of the spectrogram; S4: use the extracted The high-level features of the spectrogram train the random forest classifier; S5: Based on the high-level features extracted by the convolutional neural network, the trained random forest is used for audio classification. The present invention is based on the convolutional neural network for feature extraction, avoiding the cumbersome process of manually constructing and extracting features, and at the same time aiming at the problem of insufficient generalization ability caused by using softmax as the convolutional neural network classifier, the random forest is used to replace the convolutional neural network softmax layer, as the final classifier. High precision and recall were achieved during the test.

Description

An Audio Classification Method Based on Convolutional Neural Network and Random Forest

technical field

The invention belongs to the field of machine learning and relates to an audio classification method based on a convolutional neural network and a random forest.

Background technique

The development of the Internet and multimedia technology has filled our lives with a large amount of audio, especially various music websites, which have a large number of audio files with different styles. In the face of massive audio, audio retrieval can help us quickly and accurately find the audio files we need. Audio classification is the premise of audio retrieval, but manual classification of a large number of audio files is a very time-consuming and tedious task. With people's hearing fatigue, the accuracy of manual classification will also decrease. For a large number of audio files, fast and accurate automatic classification is very necessary. There are many studies on audio classification methods. For example, a two-level audio classification method based on a mixture of hidden Markov model and support vector machine is used. First, the hidden Markov model is used to initially classify the audio, and the two most likely classification results are determined. , and then use the corresponding support vector machine classifier to make the final decision. There is also a method of classifying audio according to the similarity between audio contents, using the pitch set of each audio to represent the audio file, and classifying the audio with the LDA topic model. There are also Gaussian mixture models, decision trees, etc. used as classifiers for classification. However, most of these methods use traditional methods to manually construct features, which is cumbersome and the extracted features are not sufficient. Moreover, the use of a single classifier leads to poor generalization ability of the model.

In recent years, deep learning has become increasingly popular. Its structure contains multiple hidden layers. By combining the underlying features to form more abstract high-level representation attributes or features, it can better mine the distributed representation features of data, which is more effective than the traditional way of manually constructing features. it is good. In view of the current situation and the above problems, it is necessary to design an audio classification method based on deep learning.

Contents of the invention

The technical problem to be solved by the present invention is to provide an audio classification method based on convolutional neural network and random forest, which uses convolutional neural network to automatically extract high-level features, and uses random forest to solve the problem that the generalization ability of a single classifier is not strong , with high precision and recall.

The technical solution of the invention is as follows:

An audio classification method based on convolutional neural network and random forest, comprising the following steps.

Step 1: Spectrum analysis is performed on the original audio file to obtain its corresponding spectrogram. Since the audio files are often long, the spectrogram obtained by directly analyzing the spectrum of the original audio is too large, resulting in the later training model occupying more system resources. Therefore, the original audio is properly segmented, and then the spectrum analysis is performed on each segment of audio, including framing, windowing, short-time Fourier transform and other processes. suppose is a long sequence, is a window function of length N, with to add Add window to get N point sequence ,Right now

In the frequency domain there are

The formula for the short-time Fourier transform is as follows:

in is the original signal, is a window function. Through spectrum analysis, the spectrum diagram corresponding to the audio is obtained.

Step 2: Use the spectrogram obtained in step 1 as a training set to train an improved convolutional neural network. The network has 14 layers, including convolutional layer, downsampling layer, Dropout layer, Flatten layer, fully connected layer, BatchNormalization layer, softmax layer, etc., using cross entropy as the loss function. The details of each layer are as follows:

Input: spectrogram with size 248*248;

Layer1: Convolutional layer, the kernel size is (5,5), 64, strides=1, the output feature map size is (244,244);

Layer2: Downsampling layer, the kernel size is (2,2), and the output feature map size is (122,122);

Layer3: convolutional layer, the kernel size is (3,3), 128, strides =2, the output feature map size is (60,60);

Layer4: Downsampling layer, the kernel size is (2,2), and the output feature map size is (30,30);

Layer5: convolutional layer, kernel size is (3,3), 256, strides =2, output feature map size is (14,14);

Layer6: downsampling layer, the kernel size is (2,2), and the output feature map size is (7,7);

Layer7: convolutional layer, kernel size is (2,2), 512, strides =1, output feature map size is (6,6);

Layer8: downsampling layer, the kernel size is (2,2), and the output feature map size is (3,3);

Layer9: Dropout layer, dropout=0.5, during the training process, the neurons will be invalidated with a certain probability to prevent overfitting;

Layer10: Flatten layer, which converts multi-dimensional data into one dimension and transitions to a fully connected layer;

Layer11: fully connected layer, the number of output neurons is 128;

Layer12: Batch Normalization, which normalizes the input signal while maintaining the expressive ability of the model;

Layer13: fully connected layer, the number of output neurons is 9, because there are 9 types of data set samples;

Layer14: softmax layer, classifier, the output is the final probability distribution, each value represents the probability of a category.

Step 3: Remove the softmax layer of the convolutional neural network trained in step 2, and use the output of the last fully connected layer as the high-level feature of the spectrogram.

Step 4: Train a random forest classifier using the high-level features extracted in step 3. Gini impurity is used as the criterion for decision tree feature selection. The algorithm is described as follows:

Input: sample set D = {(x1,y1), (x2,y2)...(xm,ym)}, weak classifier iteration number T;

Output: final strong classifier f(x);

for t = 1,2...T

a) The tth random sampling is performed from the original data set, and a total of m samples are taken to obtain the sampling set Dm;

b) Construct the mth decision tree Gm(x) by using the sampling set Dm. Randomly select some features from all the features of the sample, and then select the optimal feature from these features to divide the left and right subtrees for the decision tree.

Step 5: Perform spectral analysis on the audio to be classified to obtain a spectrogram, then use the convolutional neural network with the softmax layer removed in step 3 to extract high-level features of the spectrogram, and finally input the extracted high-level features to step 4 for training A good random forest classifier performs audio classification, and the category with the most votes cast by T weak learners is used as the final category.

The present invention proposes an audio classification method based on deep learning, which adopts a hybrid model combining convolutional neural network and random forest. Aiming at the problem of insufficient feature extraction by the traditional model, the present invention converts the audio into a spectrogram, and then uses the convolutional neural network to extract the high-level features of the spectrogram, which fully utilizes the powerful feature extraction ability of the convolutional neural network for images, and simplifies The complex process of feature extraction. Aiming at the problem that the generalization ability of a single classifier is not strong, a random forest model is used to give full play to the advantages of random forest ensemble learning, and multiple decision trees are built to classify, which makes up for the shortcomings of a single classifier. From the classification results, the present invention has higher accuracy and recall.

Description of drawings

Fig. 1 is a flow chart of an audio classification method based on convolutional neural network and random forest in the present invention.

Figure 2 Spectrum diagram obtained after spectrum analysis.

Figure 3 is a flowchart of high-level feature extraction using the improved convolutional neural network.

Detailed ways

The specific implementation method of the present invention will be further described below in conjunction with the accompanying drawings and embodiments. The following examples are only used to illustrate the present invention, but are not intended to limit the scope of the present invention.

Embodiment 1 is an example of the present invention, using "GTZAN Genre Collection" as a data set, adopting nine kinds of audio files of different genres as a training set and a test set, nine kinds of categories are: blues, C1assical, Country, Disco, Jazz, Metal, Pop, Reggae and Rock.

1. Divide the audio file into 6 segments of equal length, and each segment corresponds to the same tag. Framing, windowing, and Fourier transform each segment of audio to obtain its spectrogram. Figure 2 shows the obtained frequency spectrum. Read in the spectrogram and convert it to grayscale. Then adjust the size of each picture to 248*248. Finally, save the pixel values of the adjusted image to an array as a sample in the convolutional neural network dataset. After the above operations, the data set D (5400, 248, 248) is obtained, which means that there are 5400 spectrograms, and the width of each spectrogram is 248 and the height is 248. The data set is divided into training set and test set, 80% of which are used as training set and 20% are used as test set. Finally, training set T (4320, 248, 248) and test set V (1080, 248, 248) are obtained.

2. Use the training set T (4320, 248, 248) to train the convolutional neural network model. The network has a total of 14 layers, including convolutional layers, downsampling layers, fully connected layers, Dropout layers, Batch Normalization layers, etc.

3. When the convolutional neural network is trained, remove the last softmax layer. Use the trained convolutional neural network to perform deeper feature extraction on the spectrogram, and reconstruct the original training set T(4320,248,248) composed of the spectrogram into a new training set T'(4320,9), which will be composed of the spectrogram The original test set V(1080,248,248) is reconstructed into a new test set V'(1080,9).

4. Use the new training set T' and test set V' to train the random forest as the final classifier. Different parameter combinations are used, among which

parameter value n_estimators [10,50,100] min_samples_split [2, 3, 4] min_samples_leaf [1, 2, 3]

After selection, the best parameter combination is n_estimators:100, min_samples_split:3, min_samples_leaf:1. After the random forest training is completed, it is tested on the test set, and the results are as follows:

Classes Precision recall F1-score support 0 0.80 0.74 0.77 118 1 0.89 0.92 0.90 133 2 0.75 0.80 0.78 117 3 0.75 0.83 0.79 118 4 0.93 0.88 0.90 134 5 0.94 0.90 0.92 108 6 0.88 0.85 0.87 103 7 0.86 0.78 0.82 124 8 0.64 0.68 0.66 125 Avg/total 0.83 0.82 0.82 1080

It can be seen from the above table that this method can automatically classify audio more accurately, with an average accuracy rate of 83% and an average recall rate of 82%.

Claims (3)

1. A kind of audio classification method based on convolutional neural network and random forest, its feature comprises the steps: Step 1: carry out frequency spectrum analysis to original audio data set, at first long audio file is divided into several sections of equal length, each section of audio corresponds to the same label, then each section of audio is carried out into frames, windowed, Fourier transform, Get the spectrogram of each piece of audio as a sample of the new training set; Step 2: Utilize all spectrograms obtained in step 1 and their corresponding labels to train an improved convolutional neural network, which has 14 layers; Step 3: remove the softmax layer of the convolutional neural network learned in step 2, and then use the convolutional neural network to extract the high-level features of all spectrograms; Step 4: Utilize the high-level feature training random forest classifier of the spectrogram extracted in step 3, adopt Gini impurity as the criterion of decision tree feature selection; Step 5: Carry out the spectrum analysis in step 1 to the audio frequency to be classified to obtain the spectrogram, then use the convolutional neural network that removes the softmax layer in step 3 to extract the high-level features of the spectrogram, and finally input the high-level features extracted to step 4 for training A good random forest classifier performs audio classification, and the final classification result is obtained by voting. 2. a kind of audio classification method based on convolutional neural network and random forest according to claim 1, it is characterized in that, for audio feature, the specific implementation process of this method comprises two-stage feature extraction, and the first-level feature extraction is Through spectrum analysis, the frequency spectrum corresponding to the audio is obtained, and its low-level time-frequency features are initially extracted. The second-level feature extraction uses an improved convolutional neural network to further extract high-level features from the frequency spectrum. 3. a kind of audio classification method based on convolutional neural network and random forest according to claim 1, it is characterized in that, this method causes the problem that generalization ability is not strong in order to overcome softmax as convolutional neural network classifier, adopts Random Forest replaces the last layer of the Convolutional Neural Network as the final audio classifier.