KR20070070728A - Automatic equalizing system of audio and method thereof - Google Patents

Abstract

An automatic equalizing system of audio and a method thereof are provided to perceive characteristics of audio reproduced through an audio player so as to reproduce the audio by using equalizer setup preset according to the corresponding audio. An equalizer(100) inputs an audio signal, and controls gains of each frequency band according to a preset state of each pitch extent, and outputs the audio signal. A genre classifier(300) decides on a genre of the inputted audio signal, and controls the gains of each frequency band of the equalizer(100) by estimating the gains of each frequency band preset according to each genre.

Description

Automatic Equalizing System of Audio and Method Thereof

1 is a block diagram illustrating an automatic equalization system of audio according to the present invention.

2 is an explanatory diagram for explaining a process of extracting feature vectors in the present invention.

3 is an explanatory diagram for explaining a process of building a database using the MFC algorithm in the present invention.

4 is a flowchart illustrating an automatic equalization method of audio according to the present invention.

5 is an explanatory diagram for explaining a buffer overlap of an equalizer in the present invention;

6 is an explanatory diagram for explaining a filtering process of an equalizer in the present invention.

Explanation of symbols on the main parts of the drawings

100: equalizer

300: genre classification

The present invention relates to an audio equalizing system and method thereof, and more particularly, to an audio equalizing system and method for converting an audio file played in an audio system into a tone suitable for a genre and outputting the same.

In general, an equalizer used in audio is for converting a part of a plurality of frequency bands or a plurality of bands and adjusting the output to convert a tone to a user's favorite tone when playing music through an audio reproducing apparatus.

In particular, since it is easy to implement an equalizer in the case of a digital audio system, most digital audio devices (for example, MP3 players, CD audio, car audio, etc.) are equipped with an equalizer function.

In other words, the audio equalizer generally adjusts the characteristics of tones by frequency band by amplifying or attenuating gain values of music signals divided by frequency bands.

For example, in the case of a three-band equalizer, the low frequency band, 100 Hz, is called bass. Emphasizing the band gives the music a powerful and grand feeling. The mid-frequency band, 1 kHz, is called mid, which is the most sensitive frequency band in the human ear, so when amplified, the sound is clearly heard. The high frequency band 10 kHz is called treble, and amplifying the band gives a clear sound.

On the other hand, each person who listens to music has a different tone, and since the range of instruments or vocalists used is different according to the genre of music, it is common to set and listen to a tone suitable for each genre.

However, in order to make a desired tone using an existing equalizer, at least three to as many as twenty or more buttons must be operated, and such an operation is inconvenient because it is very complicated from a general user's point of view.

In addition, despite the fact that it is possible to produce a variety of distinctive tones with a combination of musical characteristics when playing audio, due to the inconvenience of the equalizer setting, it is necessary to listen according to the equalizer setting that is set regardless of the genre of audio being played. There is a problem that can not enjoy the audio.

An object of the present invention is to solve the problems of the prior art as described above, to grasp the characteristics of the audio to be reproduced through the audio reproducing apparatus, and to reproduce the audio using the equalizer settings preset for the corresponding audio An automatic equalization system of audio and a method thereof are provided.

Another object of the present invention is to provide an audio equalizing system and method for simplifying audio listening by automatically applying an equalizer setting suitable for the type of audio reproduced through an audio device.

In addition, another object of the present invention is to provide an automatic equalization system and method for audio that eliminates the audible inconvenience caused by listening to audio by preventing a sudden tone change in the process of applying equalizer settings according to the type of audio being played. There is.

In order to achieve the above object, the present invention comprises: an equalizer which receives an audio signal and adjusts the gain of each frequency band according to a preset sound region setting state; And a genre classification unit configured to determine a genre of the input audio signal, calculate a gain for each preset frequency band for each genre, and adjust a gain for each frequency band of the equalizer. to provide.

The equalizer includes: an input buffer for buffering an input audio signal, a filtering unit for filtering and outputting an audio signal input from the input buffer using gains set for each frequency band according to a genre classification result of the genre classification unit; It is composed of an output buffer for buffering the audio signal output from the filtering unit.

The genre classification unit analyzes the audio signal transmitted through the equalizer to extract a feature vector used for genre classification, and analyzes the feature vector of the audio signal input through the feature vector extractor to analyze the audio signal. A discriminator for determining the genre of the genre; a database in which feature vector data for each genre is stored for discriminating the genre of the audio signal by the discriminator; and when the genre of the audio signal is determined by the discriminator, the frequency unit is preset for each frequency band. It is composed of a setting unit for setting the gain for each frequency band of the equalizer as the frequency gain.

The genre classification unit may further include a down sampling unit configured to downsample the audio signal transmitted through the equalizer to a low frequency to output the feature signal to the feature vector extractor.

The feature vector extracting unit may further include a function of extracting a feature vector by analyzing audio signals corresponding to each genre, dividing the feature vector into genres, and storing the feature vector data in the database as a genre. An audio signal input for a preset time is analyzed and divided into at least two genres, and a second audio signal input for a preset time is analyzed and divided into at least four genres.

In addition, the present invention comprises the steps of (a) receiving an audio signal of the music file to be played; (b) dividing the music genres into at least two types during the first predetermined time period of the audio and equalizing each of the preset frequency bands; And (c) dividing the music genres into at least four kinds of music genres during the second predetermined time of the audio and equalizing them according to preset frequency bands.

The step (a) further includes extracting a database of feature vectors for distinguishing genres from a plurality of audio signals selected for each music genre in order to distinguish genres of music to be reproduced.

(Example)

With reference to the accompanying drawings, an automatic equalization system and method for audio according to the present invention will be described in detail.

As shown in FIG. 1, the automatic audio equalizing system of the present invention receives an audio signal and adjusts the equalizer 100 to adjust the gain of each frequency band according to a setting state for each region and outputs a genre of the input audio signal. It is composed of a classification unit 300 for determining and setting the frequency gain for each frequency band to adjust the gain for each frequency band of the equalizer 100.

The equalizer 100 receives a coefficient for classifying music genres according to a genre classification result and applies a gain (filter coefficient) value of a corresponding frequency band to equalize and buffer an input audio signal. A filtering unit 130 for filtering and outputting an audio signal input from the input buffer 110 with a gain set for each frequency band by the buffer 110 and the classification unit 300, and the filtering unit 130 It consists of an output buffer 150 for buffering the audio signal output from.

The equalizer 100 is operated by an EDMA controller, and basically equalizes 2048 pieces (STEREO) data in one frame unit.

The input buffer 110 (Ping buffer) and the output buffer 150 (Pong buffer) each have a size of 1024. Since the complex numbers are represented in the algorithm calculation process, 2048 arrays are formed per channel. As described above, a 50% overlap is taken from the data 4096 having two channels as one frame to ensure the continuity of the audio data.

The classifier 300 outputs the down sampler 310 to downsample and output an audio signal transmitted through the input buffer 110 of the equalizer 100 at a low frequency, and is output from the downsampler 310. A buffer 320 for buffering the audio signal, a feature vector extractor 340 for extracting a feature vector by analyzing the audio signal input through the buffer 320, and the feature vector extractor A discriminator 360 for determining the genre of the audio signal by analyzing the feature vector of the audio signal input through 340 and a feature vector for each genre for discriminating the genre of the audio signal by the determiner 360. When the genre of the audio signal is determined by the database 380 and the determination unit 360 in which data is stored, the filtering of the equalizer 100 is performed at a preset frequency gain for each frequency band. It consists of a setting unit 390 to set the 130.

Here, the classification unit 300 analyzes the genre of the audio signal before constructing the genre feature vector data in the database 380 before applying the equalizer 100 to the equalizer 100. It further includes a function for analyzing and storing.

That is, the classification unit 300 analyzes the pattern of the feature vector for each genre and stores the data in the database 380 in the training process and the test using the data analyzed by the training process and applying the test to the actual audio signal. Process, and classifies the genre of the audio signal that is actually reproduced and has a function of applying to the equalizer (100).

In other words, in the training process, pattern learning is performed to generate a feature vector for each genre.

The pattern learning uses feature vectors from a large number of music files to model each characteristic of an audio signal corresponding to a music genre, that is, classical music, jazz, hiphop, and rock. Statistical modeling by extracting and analyzing. When an audio signal is input to the classifier 300, the feature vector extractor 340 performs a segmentation and hamming windowing process to obtain a feature from the audio signal. After extracting the vector, we select the optimal feature vector sequence, apply the MFC algorithm to make the feature vector sequence reflecting the overall musical characteristics of the audio signal, and then build the table of feature vectors through the learning process including model matching and indexing. It generates a process of generating and storing it in the database (380).

The test process is consistent with the system operation process from the user's point of view. When the user plays the music file, the WAV file is extracted from the input audio signal, and the segmentation and hamming window is extracted by the feature vector extractor 340. Segmentation & Hamming Windowing process is performed to extract an optimal feature vector, and by using the extracted feature vector, the determiner 360 refers to a feature vector table already constructed in the database 380 with similar features. It is a process of reproducing a music file by determining the genre having a value and setting the equalizer 100 through the setting unit 390 according to each genre.

Referring to the processing of the present invention made as described above in the order of operation over time as follows.

When the music starts playing, a coarse level classification is classified into two using the audio signal for the first two seconds to determine an approximate music category, and accordingly, the equalizer 100 is set through the setting unit 390. ).

Subsequently, the equalizer 100 is set in more detail after determining the exact genre of music by classifying four fine levels using six seconds of data from the beginning of music.

As described above, the setting of the equalizer 100 is classified into two levels into the first coarse level and the second fine level to alleviate the listener's rejection due to a sudden tone change.

Referring to the characteristic configuration and operation of the present invention made as described above are as follows.

First, a feature vector extraction process performed by the feature vector extractor 340 of the classifier 300 will be described.

The feature vector is extracted from time and frequency waveforms of an audio signal of music to be reproduced. The feature vector can be classified into two types, a timbre vector and a coefficient vector.

To distinguish the timbre from the audio signal of the music being played, four of spectral centroid, spectral roll off, spectral flux, and zero crossing rates Branch feature vectors were used and 13th order MFCC and Delta-MFCC coefficients and LPC 12th order coefficients were used to extract the feature coefficients. A description of the feature vector and feature coefficients used is as follows.

Spectral Centroid is the center of the Short Time Fourier Transform. Centroid is used to measure the timbre of sound qualitatively.

Spectral Rolloff represents the frequency at which 80% of the spectral distribution is concentrated. Roll-off shows how concentrated the signal energy is in the spectral shape and the low frequency region.

Spectral Flux can be obtained by squaring the difference between normalized magnitudes in a continuous spectral distribution. Flux can calculate the amount of spectral change.

Zero Crossing Rate represents the number of times the phase of the audio signal waveform passes through the central axis. It is the simplest way to measure the frequency content of a signal. It is used to discriminate voiced and unvoiced sound from speech recognition.

Mel Frequency Cepstrum Coefficient (MFCC) is a method of modeling human auditory characteristics and converts the magnitude spectrum of the audio signal to log and then converts the FFT into Mel frequency units.

Linear Predictive Coding (LPC) is a method of encoding a speech signal based on a human vocal model. -pole) to approximate the filter.

Then, the process for extracting the feature vector from the audio signal of the music file to be played is made in two steps as shown in FIG.

First, 54 feature coefficients are obtained from an analysis window in units of 23ms (S 21), and the feature coefficient values for a total of 15 seconds are extracted to obtain the mean and variance of each feature coefficient. After combining them into rows, they are stored in the database 380 to be used for genre classification of music to be reproduced (S 22).

Here, the process for optimization of the extracted feature vector is performed as follows.

That is, the feature vector is for distinguishing the input music data, that is, the chapter of the audio signal. To this end, the feature vector extracted from the audio signal is constructed in the database 380 for each music genre, and then the audio signal is input later. Used for genre classification.

Since there are 108 feature vectors used in the present invention, 108 coefficients are generated for each music file, which acts as a factor to increase the computational complexity of the system, causing overload. In order to solve this problem, the present invention removes the correlation between the feature vectors using a sequential forward selection (SFS) algorithm, and uses only the optimized feature vector sequence by reducing the order of the feature vectors.

In other words, the feature vector extractor 340 extracts a feature vector using an MFC algorithm, and the feature vector extraction process by the MFC algorithm is as follows.

In the present invention, the feature vector extracts the total amount of 15 seconds to reflect the overall characteristics of the music to obtain the mean and variance, and is optimized by the SFS algorithm. However, there is a variation in genre classification success rate depending on which part of the music file the music data inputted by the query is extracted from, and especially since the music is not well contained at the beginning of the music. There is a tendency to lower the success rate of genre classification.

In the present invention, as shown in FIG. 3, the MFC (Multi Feature Clustering) algorithm technique is introduced to overcome the deviation of the success rate and to implement stable performance, thereby ensuring a stable success rate regardless of the position of the music query.

That is, the MFC algorithm applied to the present invention sequentially extracts the feature vectors from the beginning to the end of the music file (S 31), and finally uses one K-means clustering algorithm to finally obtain one music file. Four feature vector sequences are created per step (S 32), a music metadata database is constructed, and used for music genre classification (S 33).

In order to improve the genre classification success rate and algorithm complexity of the music to be reproduced, the discriminating unit 360 of the classifying unit 300 of the present invention performs the first reproduction of the SVM (Support Vector Machine) and the k-NN classifier, respectively. It is used to classify the coarse level, which is a stage, and the fine level, which is a second reproduction stage.

The k-NN classifier measures the similarity of data by obtaining a distance value between a feature vector sequence inputted by a query and a feature vector sequence stored in the database 380 using an Euclidean distance function. That's the way it is. The SVM obtains a high dimensional linear equation using data constructed in the database 380 during the training process, and uses the same to distinguish two genres.

Accordingly, in the case of the SVM, only two genres can be basically distinguished, but in the present invention, multiple classification is possible by combining several SVM classifiers.

A detailed process of the automatic equalization system of audio according to the present invention made as described above will be described.

The music used for the test of the present invention was selected from a total of 240 songs by selecting 60 songs from each of the four genres of Western classical music, hip hop, jazz and rock from MP3 files, music CDs, and radio. At this time, the music files used in the test were converted into files of 22,050Hz, 16bits, mono WAV format, respectively.

As shown in FIG. 4, the genre of query music is classified through a classification step of coarse level applied to the first reproduction stage and a fine level applied to the second reproduction stage. do.

In the coarse level classification step (S 41 to S 46 of FIG. 4), two genres of relatively similar sound characteristics, that is, Western classical music and jazz, are first classified into four genres of music from an audio signal input for two seconds from the beginning. The query music is classified using the hip hop and the rock as the second classification, and the gain setting value is applied to the equalizer 100 using the first classification and the second classification until a fine level is finally applied. Induce a change in timbre.

In the fine level classification step (S 47 to S 49, S 45, and S 46 of FIG. 4), music reproduced by analyzing audio signals inputted for 6 seconds by subdividing the coarse level classification is classified into one of four genres. .

As a result of the test for each coarse level and fine level classification as described above, the SVM classifier at the coarse level, which is the first stage, achieves a success rate of about 79.2% using a 38th order feature vector for a 2-second audio signal. The k-NN classifier at the second level, Fine level, showed a success rate of 81.67% using the 66th order feature vector for 6 seconds long audio data.

That is, in the equalizer 100, the music file is divided into frame units of 23ms, the equalization setting value is applied, and then reassembled and output in an overlap add method. The music file is reproduced by the classification unit 300. After the music data has been stored in the buffer 320 for 6 seconds from the time point at which it began to be processed, the operations necessary for classifying the coarse level and the fine level are sequentially performed, and then, through the setting unit 390 using the classification result. The filtering unit 130 of the equalizer 100 is adjusted.

Through this process, it took about 0.7 seconds on average for the coarse level, and about 1.1 seconds for the fine level, but the time required for two-step equalization was less than 2 seconds.

In addition, when music is played in the audio system as described above, in order to automatically set the equalizer 100 to output a tone suitable for each genre, an equalizer setting suitable for four music genres should be defined. The equalizer setting value for each music genre was set as shown in Table 1 with reference to the value set in the general music playing program such as WINAMP.

Each of the settings in Table 1 is a gain value that is multiplied by the amplitude magnitude for each frequency band of the original music source.

Here, in the case of the coarse level, genres are divided into a slow beat and a fast beat, and an equalizer setting value is a slow beat genre using an average value of classical music and jazz. The genre uses the average of hip hop and rock.

Frequency band gain of 3 band equalizer Music genre BASE MID (Senior) TREBLE (high range) Cos level Slow beat One 0.9 0.9 Fast beat 1.4 One 1.2 Fine level Western Classical Music One One 0.6 jazz One 0.8 1.2 Hip hop 1.5 1.3 One rock 1.3 0.7 1.4

On the other hand, the automatic equalization system of the audio according to the present invention was configured using a Texas Instruments (TI) DSP (TMS320C6713) for real-time implementation.

The DSP operates at an internal clock frequency of 225 MHz and has a floating point of 1350 Million Floating-point Operations per second (MFLOPS) and an integer arithmetic capability of 1800 Million instructions per second (MIPS).

After a program was created using a host PC to operate a board equipped with an audio equalizing system of the present invention implemented using the DSP, the program was stored in the board.

After storing the program on the board, using the stored program to operate the board to receive an audio signal from an external audio device, and when the feature vector is extracted by the feature vector extraction unit 340 from the input audio signal, The determination unit 360 classifies the genre having similar characteristics by referring to the feature vector table constructed in the database 380, and obtains a gain value for each genre as shown in Table 1 through the setting unit 390. The filter is applied to the filtering unit 130 of the equalizer 100 to reproduce the same.

Here, when the audio signal input to the classifier 300 is 44.1 KHz, the down sampler 310 is used to down to 22.05 KHz, which is the lowest sampling frequency required, for the purpose of preventing overload of the system and classifying genres effectively. Input through a sampling process is preferred.

In other words, as illustrated in FIG. 6, when the audio signal input (N) is input through the input buffer 110 (S 61), the equalizer 100 applies fast Fourier transform (FFT) ( S 62), the FFT-processed data FFT_input (N) is input to the filtering unit 130.

In this case, the audio signal is FFT-converted for each frequency band and classified into a low band (LPF_FFT), a mid band (BPF_FFT), and a high band (HPF_FFT) (S 631 to S 633). The low pass gain (a), the midrange gain (b), and the high pass gain (c) are determined through the setting unit 390 and set for each band (S 634 to S 636), and then set (S). 637), after reassembly by using the overlap add method (S 64, S 65), the inverse FFT transform (S 66) is output.

According to the present invention as described above, the classification success rate is improved by reducing the computational complexity of the system using the SFS technique, and stable equalization success rate is achieved regardless of the position of the music query by constructing the feature vector database of the audio signal by applying the MFC technique. to provide.

Then, after performing two classes of equalization, four kinds of equalization can be performed to induce a natural tone change.

In addition, by building a system using a DSP, the response speed of the system can be maximized.

Claims (8)

An equalizer which receives an audio signal and adjusts the gain of each frequency band according to a preset sound state setting state; And a genre classification unit configured to determine a genre of an input audio signal, calculate a gain for each preset frequency band for each genre, and adjust a gain for each frequency band of the equalizer. The method of claim 1, wherein the equalizer filters the audio signal input from the input buffer by using an input buffer for buffering an input audio signal and a gain set for each frequency band according to a genre classification result of the genre classification unit. And an output buffer configured to output an output buffer for buffering the audio signal output from the filtering unit. The feature vector extractor of claim 1, wherein the genre classifier extracts a feature vector used for genre classification by analyzing an audio signal transmitted through the equalizer, and a feature vector of an audio signal input through the feature vector extractor. A discriminator for determining the genre of the audio signal by analyzing a; a database storing feature vector data for each genre for discriminating the genre of the audio signal by the discriminator; and the genre of the audio signal is determined by the discriminator. And a setting unit for setting the gain for each frequency band of the equalizer with a preset frequency gain for each frequency band. 4. The automatic equalization system of claim 3, wherein the genre classification unit further comprises a down sampling unit which downsamples an audio signal transmitted through the equalizer to a low frequency and outputs the audio signal to the feature vector extractor. The method of claim 3, wherein the feature vector extracting unit further comprises a function of analyzing the audio signal corresponding to each genre, extracting the feature vector, classifying the genre, and storing the feature vector data by genre in the database. Audio automatic equalization system. The method of claim 3, wherein the feature vector extractor first analyzes an audio signal input for a first predetermined time and divides the audio signal into at least two genres, and analyzes an audio signal input for a second preset time for at least two genres. Automatic equalization system for audio, characterized by four genres. (a) receiving an audio signal of a music file to be played; (b) dividing the music genres into at least two types during the first predetermined time period of the audio and equalizing each of the preset frequency bands; And (c) dividing the music genres into at least four types during the second preset time of the audio and equalizing the preset music bands according to preset frequency bands; Auto equalization method of audio, characterized in that it comprises a. The method of claim 7, wherein the step (a) further comprises extracting and databaseting a feature vector for distinguishing genres from a plurality of audio signals selected for each music genre in order to distinguish genres of music to be reproduced. Featuring automatic equalization of audio.

Images