JP2008070650A - Musical composition classification method, musical composition classification device and computer program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a musical composition classification method, a musical composition classification device and a computer program, capable of classifying a musical composition by an impression value which is matched closer to real impression when a person listened to the musical piece. <P>SOLUTION: The musical composition classification device of the invention performs the steps of: sampling a sound signal from a musical composition data (S1); obtaining a fluctuation data which characterizes fluctuation of a sound volume from a sound signal (S2); detecting a tempo of the musical composition (S3); calculates a mel-cepstrum coefficient from the sound signal (S4); converting the fluctuation data and the mel-cepstrum coefficient to the impression value for showing a specific impression degree which a person receives from the musical piece (S5); correcting the tempo by using the impression value (S6); and storing the impression value and the tempo by relating them to the musical piece data (S7). By calculating the impression value from the mel-cepstrum coefficient indicating sound quality for characterizing the musical composition, the impression value which is closer to impression felt by a user who actually listened to the music piece, is obtained. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a method of classifying a large number of music data stored in a storage means, and more specifically, an impression that a person feels when listening to music and a tempo of the music are obtained from the music data, and the obtained impression and tempo are obtained. The present invention relates to a music classification method, a music classification device, and a computer program for classifying music data using the data.

  Conventionally, storage means such as a hard disk or a semiconductor memory is provided inside, music data in a data format such as MP3 (MPEG-1 Audio Layer-3) is stored in the storage means, and music is stored based on the stored music data. Music playback devices for playback are in widespread use. An example of such a music playback device is a portable music playback device. In addition, content reproduction apparatuses that can reproduce video including music are also widespread. When such a portable music playback device is used, music data created by an information processing device such as a personal computer (PC) or music data distributed to the information processing device using a communication network is usually processed. The music playback device stores the transmitted music data in the storage means.

  As a method of searching for desired music data from a large amount of music data stored in the music playback device, a method of normally searching for music data using attribute information by associating attribute information related to the content of the music with the music data Is used. For example, the music reproducing apparatus stores a plurality of types of attribute information indicating albums, artists, genres, and the like related to music in association with music data. For example, when the user designates one album name, one or a plurality of music pieces associated with the designated album name are searched, and the searched music piece can be heard.

In addition, a technique has been developed in which feature data representing the feature of a song is associated with the song data and the song data is searched using the feature data. As an example of the feature data, there is a tempo of music, and a technique for automatically detecting the tempo from a change in sound intensity sampled from the music has been developed. As another feature data, a technique has been developed that uses an impression value representing the degree of impression that a user receives from a song, such as transparency, brightness, or intensity. Patent Document 1 discloses a technique for extracting data characterizing volume fluctuation in music from music and converting the extracted data into music impression values using a hierarchical neural network. By classifying music based on characteristic data representing the characteristics of the music such as tempo or impression value, the music can be selected according to the characteristics of the music itself, such as an uplifting music, a calm music, or a relaxing music. become able to.
JP 2005-141430 A

  However, in the conventional technique for detecting the tempo of a song, when there are many types of musical instruments used for playing the song or when the rhythm in the song changes in a complicated manner, the length is half or half of the original length. There is a problem that an incorrect tempo may be detected, such as extracting the tempo of the user. In addition, the technique disclosed in Patent Document 1 has a problem in that a deviation may occur between the impression of the music represented by the obtained impression value and the impression that the user who actually listened to the music feels from the music. .

  The present invention has been made in view of such circumstances, and the object of the present invention is to obtain an impression value from data representing the characteristics of a musical piece on a scale that matches the human auditory characteristics. An object is to provide a music classification method, a music classification device, and a computer program capable of classifying music by an impression value that matches the actual impression when listening to the music.

  Another object of the present invention is to provide a music classification method, music classification apparatus, and computer program that can accurately determine the tempo of music by correcting the tempo according to the obtained impression value. There is.

  The music classification method according to the present invention includes a generation step of generating data indicating the characteristics of the music, and a conversion step of converting the data generated in the generation step into an impression value indicating a specific impression level received by a human from the music And a step of classifying the music data by associating the impression value with the music data, the generating step performs an FFT processing step of performing a fast Fourier transform (FFT) of an audio signal representing the audio of the music And a step of obtaining a mel cepstrum coefficient of the audio signal from an FFT result of the FFT processing step, and the conversion step outputs an impression value indicating a specific impression degree by inputting the mel cepstrum coefficient. Using the hierarchical neural network learned in this way, Characterized in that it comprises a step of converting the cepstral coefficients into impression values.

  In the music classification method according to the present invention, the FFT processing step includes a step of obtaining a power spectrum of the partial audio signal by performing FFT on the partial audio signal obtained by dividing the audio signal, and the generation step includes Calculating, for each of a plurality of partial audio signals included in the audio signal, an average power obtained by averaging the power of a predetermined frequency band in the power spectrum obtained in the FFT processing step; and calculating the calculated average power for each partial audio A step of obtaining an average power time series arranged on the time axis in correspondence with a temporal position of the signal in the audio signal, and performing an FFT of the obtained average power time series to thereby obtain a power spectrum of the average power time series Approximate the obtained average power time series power spectrum with a predetermined curve. A step of obtaining an approximate curve; and a step of obtaining parameters of the obtained approximate curve as fluctuation data characterizing periodic fluctuations in volume in the audio signal, wherein the conversion step inputs fluctuation data and a mel cepstrum coefficient. Step of converting fluctuation data and mel cepstrum coefficient of the audio signal obtained in the generation step into an impression value using a hierarchical neural network that has been learned to output an impression value indicating a specific impression level Is further included.

  The music classification method according to the present invention includes a step of obtaining a difference between the power spectrum of the average power time series obtained from the audio signal in the generation step and the approximate curve, and a component whose magnitude of the difference is a predetermined value or more. The step of obtaining the tempo of the music based on the longest cycle within the predetermined cycle range and the value of the impression value indicating the degree of the specific impression obtained in the conversion step were obtained. The method further includes a step of correcting the tempo and a step of associating the corrected tempo with the music data.

  The music classification apparatus according to the present invention includes a generating unit that generates data indicating the characteristics of a music, and a converting unit that converts the data generated by the generating unit into an impression value indicating a specific degree of impression received by a person from the music And a storage means for classifying and storing music data by associating impression values with music data, wherein the generating means performs fast Fourier transform (FFT) of an audio signal representing the sound of the music FFT processing means to perform, and means for obtaining a mel cepstrum coefficient of the audio signal as feature data from the FFT result by the FFT processing means, and the conversion means receives the mel cepstrum coefficient and receives a specific impression. Using the hierarchical neural network learned to output the impression value indicating the degree, the Merkepstra obtained by the generating means And having a means for converting a coefficient to impression value.

  In the music classification apparatus according to the present invention, the FFT processing means includes means for obtaining a power spectrum of the partial audio signal by performing FFT on the partial audio signal obtained by dividing the audio signal, and the generating means Means for calculating an average power obtained by averaging powers in a predetermined frequency band in a power spectrum obtained by the FFT processing means for each of a plurality of partial sound signals included in the sound signal, and an average power calculated by the means Means for obtaining an average power time series in which the partial power signals are arranged on the time axis in correspondence with the temporal positions in the voice signal, and performing FFT of the average power time series obtained by the means, Means for obtaining the power spectrum of the average power time series, and a proximity that approximates the power spectrum of the average power time series obtained by the means with a predetermined curve. Means for obtaining a curve, and means for obtaining parameters of the approximate curve obtained by the means as fluctuation data characterizing periodic fluctuations in volume in the audio signal, wherein the conversion means comprises fluctuation data and a mel cepstrum. The fluctuation data and mel cepstrum coefficient obtained by the generating means are converted into an impression value by using a hierarchical neural network that has been input to a coefficient and learned to output an impression value indicating a specific impression level. It further has a means for converting.

  The music classification apparatus according to the present invention includes: a means for obtaining a difference between the average power time-series power spectrum obtained from the audio signal by the generating means and the approximate curve; and a component whose magnitude is greater than or equal to a predetermined value. Based on the longest cycle within a predetermined cycle range, a means for obtaining the tempo of the music and an impression value indicating the degree of the specific impression obtained by the conversion means were obtained. Tempo correction means for correcting the tempo is further provided, and the storage means has means for storing music data in association with the tempo corrected by the tempo correction means.

  A computer program according to the present invention causes a computer to generate data indicating the characteristics of a song, and to convert the generated data into an impression value indicating the degree of a specific impression received by a person from a song. A procedure for performing fast Fourier transform (FFT) on an audio signal representing the sound of a song, a procedure for causing a computer to obtain a mel cepstrum coefficient of the audio signal from the FFT result, and a computer for obtaining a mel cepstrum coefficient. It includes a procedure for inputting the obtained mel cepstrum coefficient and obtaining the output impression value for a hierarchical neural network that has been input and learned to output an impression value indicating a specific impression level. It is characterized by.

  A computer program according to the present invention includes a procedure for causing a computer to obtain a power spectrum of the partial audio signal by performing FFT on the partial audio signal obtained by dividing the audio signal, and including the computer in the audio signal. For each of the plurality of partial audio signals, a procedure for calculating an average power obtained by averaging the power in a predetermined frequency band in the obtained power spectrum, and causing the computer to calculate the calculated average power in the audio signal of each partial audio signal. A procedure for obtaining an average power time series arranged on the time axis in correspondence with a temporal position, and a procedure for causing a computer to obtain an average power time series power spectrum by performing FFT of the obtained average power time series. And the computer calculates the calculated average power time series power spectrum. A procedure for obtaining an approximate curve approximated by a curve, a procedure for causing a computer to obtain parameters of the obtained approximate curve as fluctuation data characterizing periodic fluctuations in volume in the audio signal, and a computer for obtaining fluctuation data and Inputs the obtained fluctuation data and mel cepstrum coefficient of the voice signal to the hierarchical neural network that is trained to input the cepstrum coefficient and output the impression value indicating the degree of specific impression. And a procedure for acquiring an impression value.

  A computer program according to the present invention includes a procedure for causing a computer to obtain a difference between the power spectrum of the average power time series obtained from the audio signal and the approximate curve, and the computer has a magnitude of the difference greater than or equal to a predetermined value. Based on the procedure for determining the tempo of a song based on the longest period within a predetermined period within a period of a certain component and the computer's impression value indicating the degree of a specific impression, the calculated tempo The method further includes a procedure for correcting.

  In the present invention, a mel cepstrum coefficient is obtained from an audio signal representing the sound of a music piece, and a mel cepstrum coefficient is learned by inputting the mel cepstrum coefficient and learning a specific impression value. Is converted into an impression value indicating the degree of a specific impression that a human receives from the music. By obtaining the impression value from the mel cepstrum coefficient representing the sound quality that characterizes the music, an impression value closer to the impression felt by the user who actually listened to the music can be obtained.

  In the present invention, an average power time series in which average powers obtained by averaging powers within a predetermined frequency band of a power spectrum obtained by FFT of partial audio signals obtained by dividing an audio signal are arranged on a time axis is obtained, and further, average power A time-series power spectrum is obtained, and parameters of an approximate curve of the obtained power spectrum are obtained as fluctuation data indicating a frequency distribution in which the volume varies in each frequency band. For example, the slope and Y intercept of an approximate curve obtained by approximating the logarithm display power spectrum with a straight line are acquired as fluctuation data. The fluctuation data and the mel cepstrum coefficient are input and the hierarchical neural network learned to output a specific impression value is used, and the fluctuation data and the mel cepstrum coefficient indicate the degree of a specific impression that a person receives from the music. Convert to impression value.

  Furthermore, in the present invention, the tempo of the music is obtained based on the longest cycle within a predetermined cycle range among the cycles of components whose difference value between the average power time-series power spectrum and the approximate curve is equal to or greater than a predetermined value. The tempo is corrected according to the value of the impression value indicating the degree of a specific impression related to the tempo. For example, the tempo is corrected so that the tempo is faster when the impression value indicating the degree of impression of “severity” is large, and the tempo is also delayed when the impression value is small.

  In the present invention, by obtaining an impression value from a mel cepstrum coefficient that represents the sound quality that characterizes the music, the impression that the user who actually listened to the music feels from the music and the obtained impression value are shown, as compared to the prior art. The deviation from the impression of the music is reduced. Therefore, the music data is classified according to the impression that the user feels from the music, and the user can search and listen to the music that feels the desired impression according to the preference more accurately.

  In the present invention, the user who actually listened to the music compared to the conventional technique for obtaining the impression value from the fluctuation data by obtaining the impression value from the fluctuation data characterizing the periodic fluctuation of the volume and the mel cepstrum coefficient. The difference between the impression felt by the music and the impression of the music indicated by the obtained impression value becomes small.

  In the present invention, the tempo of the music is once determined, and the tempo is corrected according to the value of the impression value indicating the degree of the specific impression related to the tempo, so that the impression with a small deviation from the actual impression is small. Since the tempo is corrected based on the value, the present invention has an excellent effect, such as being able to determine the tempo of music more accurately.

Hereinafter, the present invention will be specifically described with reference to the drawings showing embodiments thereof.
FIG. 1 is a block diagram showing the configuration of a music classification device 1 and a music playback device 2 of the present invention. The music classification apparatus 1 according to the present invention is configured using a general-purpose computer such as a PC, and performs a calculation, a CPU 11 that stores calculation, a RAM 12 that stores temporary information generated by the calculation, and a data-recorded CD or CD A drive unit 13 for receiving a recording medium such as a DVD and a storage unit (storage unit) 14 such as a hard disk are provided. The storage unit 14 stores the computer program 141 of the present invention read by the drive unit 13 from a recording medium such as a CD-ROM. The computer program 141 is loaded from the storage unit 14 to the RAM 12 as necessary. Based on the loaded computer program 141, the CPU 11 executes processing necessary for the music classification device 1.

  The music classification apparatus 1 also includes an input unit 15 such as a keyboard or a pointing device for inputting information such as various processing instructions operated by the user, and a display unit 16 such as a liquid crystal display for displaying various information. It has. The music classification device 1 further includes an interface unit 17 that transmits and receives data to and from devices outside the music classification device 1 and a communication unit 18 that is connected to an external communication network N such as the Internet.

  The music classification apparatus 1 receives a recording medium such as a CD on which data is recorded by the drive unit 13, reads data recorded by the recording medium, and converts the read data into music data in a data format such as MP3 by the CPU 11. It can be carried out. The music classification device 1 can receive music data transmitted from a server device (not shown) connected to the communication network N by the communication unit 18 via the communication network N. The music data converted by the CPU 11 or the music data received by the communication unit 18 is stored in the storage unit 14. The music classification device 1 is configured to be able to transmit music data to a music playback device 2 that plays back music based on music data.

  The music reproducing device 2 includes a control unit 21 including a processor that performs calculation, a ROM that stores a control program, and a RAM. The control unit 21 is configured to control the operation of the entire music reproducing device 2. The control unit 21 is connected to a storage unit 22 such as a hard disk or a semiconductor memory, and the storage unit 22 is configured to store music data in a data format such as MP3. The control unit 21 is connected to a data processing unit 24 that performs processing for decoding music data, and an output unit 25 that DA-converts the data decoded by the data processing unit 24 and outputs the data to the outside. Headphones 26 can be connected to the output unit 25, and music is reproduced using the headphones 26. Also connected to the control unit 21 are an operation unit 23 for inputting various instructions such as a process start instruction by a user operation, and a display unit 28 for displaying information necessary for processing of the music reproducing device 2. Has been. Further, the control unit 21 is connected to an interface unit 27 that transmits / receives data to / from devices outside the music reproducing device 2.

  The interface unit 17 of the music classification device 1 and the interface unit 27 of the music reproduction device 2 can be connected to each other via a cable, and the music classification device 1 and the music reproduction device 2 are connected via the cable and the mutual interface unit. Data such as music data can be transmitted to and received from the. Note that the music classification device 1 and the music playback device 2 may be configured to transmit and receive data by wireless communication instead of wired communication. The music reproducing device 2 has a small, light and portable configuration, and is used by being carried by the user by removing the cable connected to the interface unit 27.

  Next, the music classification method of the present invention executed by the music classification device 1 of the present invention having the above configuration will be described. The music classification device 1 performs a process of classifying a plurality of music data stored in the storage unit 14 using an impression value representing a specific degree of impression that a user receives from a music and a tempo of the music. The CPU 11 of the music classification device 1 loads the computer program 141 into the RAM 12 and executes processing for obtaining the impression value and the music tempo from the music data according to the loaded computer program 141.

  FIG. 2 is a flowchart showing a procedure of processing executed by the CPU 11 in the present invention. The CPU 11 executes the following processing according to the computer program 141 loaded into the RAM 12. First, the CPU 11 reads one piece of music data stored in the storage unit 14 into the RAM 12, decodes music data in a data format such as MP3, and outputs an audio signal such as a linear PCM signal obtained by decoding the data. Sampling is performed at a frequency (S1). By this process, the CPU 11 acquires an audio signal representing the audio of the music. At this time, an audio signal having a fundamental frequency suitable for the subsequent processing is acquired by downsampling or the like.

  Next, the CPU 11 executes fluctuation data acquisition processing for acquiring fluctuation data that characterizes periodic fluctuations in volume from the audio signal acquired by sampling (S2). FIG. 3 is a flowchart showing a processing procedure in a subroutine of fluctuation data acquisition processing. CPU11 determines the starting position which starts FFT (fast Fourier transformation) in the acquired audio | voice signal (S201). At this time, the CPU 11 may set the beginning of the audio signal as the start position, and may set the position after a predetermined time from the beginning of the audio signal as the start position. Next, the CPU 11 performs FFT on the partial audio signal over a certain time length from the determined start position of the audio signal (S202).

  Next, the CPU 11 divides the power spectrum of the partial audio signal generated by performing the FFT into three frequency bands of Low (low frequency band), Middle (intermediate frequency band), and High (high frequency band). (S203). Next, the CPU 11 averages the power distributed in each of the power spectra of Low, Middle, and High, and calculates an average power obtained by averaging the power in each frequency band (S204). Next, the CPU 11 determines whether or not the process of calculating the average power up to the end of the audio signal has been executed (S205). If the process has not yet been performed to the end of the audio signal (S205: NO), the CPU 11 determines whether or not the number of partial audio signals subjected to the process of calculating the average power has reached a predetermined set value. Determination is made (S206). If the number of processed partial audio signals has not yet reached the set value (S206: NO), the CPU 11 sets the FFT start position in the audio signal for a predetermined time length in the direction in which time passes. Is shifted only by S207. The time length for shifting the FFT start position may be equal to the time length of the partial audio signal. Next, the CPU 11 returns the process to step S202, performs FFT on the partial audio signal from the new start position, and calculates the FFT and average power of the partial audio signal while shifting the start position in steps S202 to S207. repeat.

  FIG. 4 is an explanatory diagram showing an example of the result of repeating the FFT of the partial audio signal. FIG. 4A shows an example of an audio signal, where the vertical axis shows audio intensity and the horizontal axis shows time. The vertical line shown in FIG. 4 (a) indicates the FFT start position to be shifted one after another. When the time length for shifting the FFT start position is equal to the time length of the partial audio signal, the FFT start position is Each portion obtained by dividing the audio signal becomes a partial audio signal. FIG. 4B shows the power spectrum of the first partial audio signal included in the audio signal shown in FIG. 4A, and FIG. 4C shows the power spectrum of the second partial audio signal. 4B and 4C, the vertical axis indicates power, and the horizontal axis indicates frequency. The power spectrum of each partial audio signal is divided into three frequency bands, Low, Middle, and High, and the average power is calculated within each frequency band. For each of the plurality of partial audio signals included in the audio signal, a power spectrum is obtained in order, and the average power in each frequency band is calculated.

  When the processing is performed up to the end of the audio signal in step S205 (S205: YES), or when the number of partial audio signals processed in step S206 is a set value (S206: YES), the CPU 11 In each frequency band of Low, Middle, and High, an average power time series in which average powers calculated for a plurality of partial audio signals included in the audio signal are arranged on the time axis is generated (S208). At this time, the CPU 11 arranges the average power on the time axis in correspondence with the temporal position in the audio signal of each partial audio signal. For example, the CPU 11 positions the average power at the start position of each partial audio signal in the audio signal on the time axis.

  FIG. 5 is a characteristic diagram showing an example of the average power time series. FIGS. 5A, 5B, and 5C show average power time series in the Low, Middle, and High frequency bands, respectively. In the figure, the vertical axis represents average power, and the horizontal axis represents time. In this way, it is possible to obtain a state in which the average power of each frequency band varies with time in the audio signal. This corresponds to a situation in which the volume of the low sound range, the mid sound range, and the high sound range varies with time in the music.

  Next, the CPU 11 performs FFT on the average power time series in each frequency band of Low, Middle, and High (S209). Next, the CPU 11 calculates an approximate straight line of a power logarithm display power spectrum for the power spectrum obtained by FFT in each frequency band of Low, Middle, and High (S210). The approximate straight line obtained here corresponds to the approximate curve in the present invention. Next, the CPU 11 acquires the inclination and Y intercept of the approximate straight line calculated in each frequency band of Low, Middle, and High as fluctuation data characterizing periodic fluctuations in volume (S211). The CPU 11 stores the obtained fluctuation data in the RAM 12, ends the fluctuation data acquisition process, and returns the process to the main.

  FIG. 6 is a characteristic diagram illustrating an example of a power spectrum and approximate straight line in a log-log display. The vertical axis in the figure represents logarithmic power, and the power obtained by FFT of the average power time series is represented on a logarithmic scale. The horizontal axis of the figure shows the frequency on a logarithmic scale. The CPU 11 obtains a power spectrum as shown in FIG. 6 for each frequency band of Low, Middle, and High. The obtained power spectrum in each frequency band indicates a frequency distribution in which the volume of the low, middle, and high sound ranges varies in the music, and represents a periodic fluctuation of the sound volume in each sound range. The slope and Y intercept of the approximate line shown in FIG. 6 correspond to the parameters of the approximate curve in the present invention, and become fluctuation data that characterizes periodic fluctuations in the volume of the audio signal.

  Next, the CPU 11 executes tempo detection processing for detecting the tempo of the music represented by the audio signal using the obtained fluctuation data (S3). The tempo is the speed at which the music is played, and the cycle of one beat included in the music or the number of beats included in the music in a predetermined time represents the tempo. In normal music, the beat is ticked by increasing or decreasing the volume of the low frequency range at the same period as the tempo. Therefore, in the power spectrum shown in FIG. Should appear as a period of a large component. In ordinary music, the tempo period is often included between 0.3 and 1 s. In addition, the music often includes a sound whose volume changes with a rhythm obtained by doubling one beat or a rhythm obtained by dividing one beat into two or four.

  7 to 10 are flowcharts showing a processing procedure in a subroutine of tempo detection processing. The CPU 11 uses the power spectrum obtained by the FFT of the average power time series in the Low frequency band, calculates the difference value between the value of each component in the power spectrum of the logarithmic display and the approximate line, and displays the logarithmic display. In the power spectrum, it is determined whether or not there is a component having a difference value of 1.25 or more in a range of 0.3 to 2 s (S301). The period is the reciprocal of the frequency. When there is a component having a difference value of 1.25 or more (S301: YES), the CPU 11 acquires a cycle A of a component having the longest cycle among components having a difference value of 1.25 or more (S302). . Next, the CPU 11 determines whether or not the acquired period A is greater than 1.0 s (S303). If A is greater than 1.0 s (S303: YES), the tempo period should often be 1 s or less, so the CPU 11 determines that A / 2 is a period that is twice the tempo. A tempo period is set (S304). The CPU 11 stores the detected tempo cycle in the RAM 12, ends the tempo detection process, and returns the process to the main.

  When A is 1.0 s or less in step S303 (S303: NO), the CPU 11 determines whether A is smaller than 0.5 s (S305). When A is 0.5 s or more (S305: NO), since 0.5 s ≦ A ≦ 1 s and A is included between 0.3 and 1 s, the CPU 11 sets A as a tempo cycle. (S306). The CPU 11 stores the detected tempo cycle in the RAM 12, ends the tempo detection process, and returns the process to the main.

  When A is smaller than 0.5 s in step S305 (S305: YES), the CPU 11 determines whether or not the difference value of the component whose cycle is (A × 4) in the power logarithm display power spectrum is larger than 1.0. Is determined (S307). When the difference value of the component whose period is (A × 4) is larger than 1.0 (S307: YES), (A × 2) is in the range of 0.3 to 1 s and (A × 2) 2 Since the difference value is large in the double cycle, the CPU 11 sets (A × 2) as the tempo cycle (S308). The CPU 11 stores the detected tempo cycle in the RAM 12, ends the tempo detection process, and returns the process to the main. If the difference value of the component whose cycle is (A × 4) is 1.0 or less in step S307 (S307: NO), the CPU 11 advances the process to step S306 and sets A as the tempo cycle.

  When there is no component having a difference value of 1.25 or more in the range of the cycle 0.3 to 2 s in step S301 (S301: NO), the CPU 11 includes the cycle 0.3 to 2 s in the logarithmic power spectrum. It is determined whether there is a component having a difference value of 0.74 or more in the range (S309). When there is no component whose difference value is 0.74 or more (S309: NO), the CPU 11 has a difference value of 0.6 or more in the period of 0.3 to 2s in the logarithm display power spectrum. It is determined whether there is a component (S310). Note that the accuracy of detecting the tempo decreases as the reference difference value decreases.

  If there is a component whose difference value is 0.74 or more in step S309 (S309: YES), or if there is a component whose difference value is 0.6 or more in step S310 (S310: YES), the CPU 11 A component having a difference value of 0.74 or more is extracted in S309 or a component having a difference value of 0.6 or more in S310, and the longest cycle A is acquired from the extracted component cycles (S311). Next, the CPU 11 determines whether or not the acquired period A is greater than 1.2 s (S312). When A is larger than 1.2 s (S312: YES), the CPU 11 determines whether or not the difference value of the component whose cycle is (A / 8) is 1.1 or more (S313). When the difference value of the component whose period is (A / 8) is 1.1 or more (S313: YES), since the difference value is large in the period obtained by dividing (A / 4) into two, the CPU 11 4 is a tempo cycle (S315). The CPU 11 stores the detected tempo cycle in the RAM 12, ends the tempo detection process, and returns the process to the main.

  When the difference value of the component whose cycle is (A / 8) is smaller than 1.1 in step S313 (S313: NO), the CPU 11 determines that the component whose value is greater than 1.2 in the range of 0.3 s or less. Yes, whether or not the condition that the value of the component in the cycle (A / 4)> the value of the component in the cycle A and the difference value in the cycle (A / 4) is greater than 1.0 is satisfied. Determination is made (S314). If the condition of step S314 is satisfied (S314: YES), A / 4 has a small tempo period, but it is likely that the component value is large and the tempo period is small in the small period range. Advances the process to step S315 and sets A / 4 as the tempo period. When the condition of step S314 is not satisfied (S314: NO), the CPU 11 advances the process to step S304 and sets A / 2 as the tempo period.

  When A is 1.2 s or less in step S312, (S312: NO), the CPU 11 determines whether A is greater than 1.0 s (S316). If A is greater than 1.0 s (S316: YES), the CPU 11 advances the process to step S304 and sets A / 2 as the tempo period, as in step S303. When A is 1.0 s or less (S316: NO), the CPU 11 determines whether A is greater than 0.6 s and the number of extracted components is greater than 2 (S317). When the condition of step S317 is satisfied (S317: YES), the CPU 11 has two or more components that can be extracted in the period of 0.3 s to A, and the difference value is 0. 0 in the range of 2 to 3 s. It is determined whether or not the condition that there are seven or more components is satisfied (S318). When the condition of step S318 is not satisfied (S318: NO), the CPU 11 advances the process to step S304 and sets A / 2 as the tempo cycle. When the condition of step S318 is satisfied (S318: YES), the CPU 11 sets 1/4 of the period of the component whose difference value is 0.7 or more as the tempo period (S319). The CPU 11 stores the detected tempo cycle in the RAM 12, ends the tempo detection process, and returns the process to the main.

  When the condition of step S317 is not satisfied (S317: NO), the CPU 11 determines whether A is 0.6 s or more and 1.0 s or less (S320). When A is 0.6 s or more and 1.0 s or less (S320: YES), it is determined whether or not the difference value in the period (A / 4) is 1.1 or more (S321). When the difference value in the cycle (A / 4) is 1.1 or more (S321: YES), the CPU 11 advances the process to step S304 and sets A / 2 as the tempo cycle. When the difference value in the cycle (A / 4) is smaller than 1.1 (S321: NO), the CPU 11 advances the process to step S306 and sets A as the tempo cycle.

  When A is smaller than 0.6 s in step S320 (S320: NO), the CPU 11 determines whether or not A is smaller than 0.5 s (S322). When A is 0.5 s or more (S322: NO), the CPU 11 advances the processing to step S306 and sets A as a tempo cycle. When A is smaller than 0.5 s (S322: YES), the CPU 11 has no extracted component in addition to the component of the period A in the range of the period 0.3 to 1 s, and the difference value in the period A is 0.9. It is determined whether or not the condition that the difference value in the period (A / 2) is smaller than 1.1 is satisfied (S323). If the condition of step S323 is not satisfied (S323: NO), the CPU 11 advances the process to step S306 and sets A as the tempo cycle. When the condition of step S323 is satisfied (S323: YES), the CPU 11 advances the process to step S308 and sets (A × 2) as the tempo period.

  In step S310, when there is no component having a difference value of 0.6 or more in the range of the cycle 0.3 to 2s (S310: NO), the CPU 11 includes the cycle 3 to 4s in the logarithm display power spectrum. It is determined whether there is a component having a difference value of 0.6 or more in the range (S324). When there is a component having a difference value of 0.6 or more (S324: YES), the CPU 11 acquires the cycle B of the component having the longest cycle among the components having the difference value of 0.6 or more (S325). , B / 4 is the tempo period (S326). The CPU 11 stores the detected tempo cycle in the RAM 12, ends the tempo detection process, and returns the process to the main.

  If there is no component having a difference value of 0.6 or more in the range of the period 3 to 4 s in step S324 (S324: NO), the CPU 11 includes the period 0.1 to 0.3 s in the power logarithm display power spectrum. It is determined whether or not there is a component having a difference value of 0.9 or more in the range (S327). When there is a component having a difference value of 0.9 or more in the range of the cycle 0.1 to 0.3 s (S327: YES), the CPU 11 sets 0.3s as the tempo cycle (S328). The CPU 11 stores the detected tempo cycle in the RAM 12, ends the tempo detection process, and returns the process to the main.

  If there is no component having a difference value of 0.9 or more in the range of the period of 0.1 to 0.3 s in step S327 (S327: NO), the CPU 11 has a period of 0.1 It is determined whether or not there is a component having a difference value of 0.7 or more in the range of ~ 0.3 s (S329). When there is a component having a difference value of 0.7 or more (S329: YES), the CPU 11 sets a cycle obtained by quadrupling the cycle of a component having a difference value of 0.7 or more as a tempo cycle (S330). The CPU 11 stores the detected tempo cycle in the RAM 12, ends the tempo detection process, and returns the process to the main. If there is no component having a difference value of 0.7 or more in the range of the period of 0.1 to 0.3 s in step S329 (S329: NO), the CPU 11 sets 1.0 s as the tempo period (S331). The CPU 11 stores the detected tempo cycle in the RAM 12, ends the tempo detection process, and returns the process to the main.

  Next, the CPU 11 executes a mel cepstrum coefficient calculation process for calculating a mel cepstrum coefficient indicating characteristics that characterize the voice in accordance with human hearing from the voice signal (S4). Human hearing is known to have fine frequency resolution at low frequencies and coarse frequency at high frequencies. This frequency resolution shows a non-linear characteristic close to the logarithm called the Mel scale. The normal frequency is f, and the mel frequency Mel (f) is expressed by the following formula (1).

  The mel cepstrum coefficient is calculated by performing discrete cosine transform on the output of the filter bank using a triangular bank filter bank divided at equal intervals on the mel frequency axis. Since the discrete cosine transform is performed in which the absolute value of the low frequency is increased and the absolute value of the high frequency is decreased, the mel cepstrum coefficient indicates a low-order component of the speech signal, that is, a spectral envelope. The spectral envelope of the voice signal represents the sound quality that characterizes the voice, and has been conventionally used for various voice recognition processes.

FIG. 11 is a flowchart showing a processing procedure in a subroutine of mel cepstrum coefficient calculation processing. The CPU 11 determines a start position at which FFT is started in the acquired audio signal (S41), and performs FFT on the partial audio signal over a certain time length from the determined start position (S42). Next, the CPU 11 calculates the filter bank outputs of triangular windows arranged at equal intervals on the mel frequency axis with respect to the power spectrum of the partial audio signal generated by the FFT (S43). Here, ω is a normal frequency, k is a natural number of 1,..., K, and the lower limit, center, and upper limit frequencies of the kth triangular window filter are ω _lo (k), ω _c (k), ω _{hi, respectively.} If (k) is assumed and the power value in the power spectrum is Y (ω), the filter output m (k) of each triangular window is expressed by the following equation (2).

FIG. 12 is an explanatory diagram for explaining the contents of the process for obtaining the filter bank output. 12 (a) shows the power spectrum of the partial audio signal, FIG. 12 (b) shows a triangular bank filter bank arranged at equal intervals on the mel frequency axis, and the horizontal axis shows the normal frequency. Yes. The vertical axis in FIG. 12 represents power, and in FIG. 12A corresponds to Y (ω) in the expression (2), and in FIG. 12B corresponds to W (ω; k) in the expression (2). To do. Between adjacent filters in the filter bank, ω _c (k) is arranged at equal intervals on the mel frequency axis, and ω _c (k) = ω _hi (k−1) as shown in FIG. ) = Ω _lo (k + 1). The number K of filters in the filter bank, the lower limit frequency ω _lo (1) of the first filter, and the upper limit frequency ω _hi (K) of the last filter are preset. For example, if K = 40, ω _lo (1) = 166 (Hz), and ω _hi (K) = 7000 (Hz), the interval between adjacent filters is approximately 61.56 on the Mel frequency axis. As represented by the equation (2), the CPU 11 calculates the power spectrum value as shown in FIG. 12A and the triangular window filter as shown in FIG. Calculate filter bank outputs m (1),..., M (K).

  Next, the CPU 11 performs a discrete cosine transform on the calculated filter bank outputs m (1),..., M (K), thereby calculating a mel cepstrum coefficient up to a predetermined order (S44). Next, the CPU 11 determines whether or not the process of calculating the mel cepstrum coefficient is performed until the end of the audio signal (S45). If the process has not yet been performed to the end of the audio signal (S45: NO), the CPU 11 determines whether or not the number of partial audio signals subjected to the process of calculating the mel cepstrum coefficient has reached a predetermined set value. Is determined (S46). If the number of processed partial audio signals has not yet reached the set value (S46: NO), the CPU 11 sets the FFT start position in the audio signal for a predetermined time length in the direction in which time passes. Shift (S47). Next, the CPU 11 returns the processing to step S42, performs FFT on the partial audio signal from the new start position, and shifts the start position in steps S42 to S47, and calculates the FFT of the partial audio signal and the mel cepstrum coefficient. Repeat the calculation.

  When the process is performed up to the end of the audio signal in step S45 (S45: YES), or when the number of partial audio signals processed in step S46 is a set value (S46: YES), the CPU 11 Then, the mel cepstrum coefficients of the speech signal are calculated by averaging the mel cepstrum coefficients calculated from each partial speech signal up to a predetermined order among the partial speech signals (S48). In the present embodiment, first and second order mel cepstrum coefficients are calculated. The CPU 11 stores the calculated mel cepstrum coefficient in the RAM 12, ends the mel cepstrum coefficient calculation process, and returns the process to the main.

  Next, the CPU 11 inputs the fluctuation data and the mel cepstrum coefficient and uses the hierarchical neural network learned to output an impression value indicating a specific impression level. Impression value conversion processing for converting the cepstrum coefficient into an impression value indicating the degree of a specific impression received by a person from the music is executed (S5).

FIG. 13 is a schematic diagram showing a hierarchical neural network. The hierarchical neural network includes N layers, and each layer includes L _n neurons (n = 1,..., N). The first layer is an input layer, and data is input to each neuron, and each neuron weights the input data and inputs it to each neuron of the second layer. The nth layer (n = 2,..., N−1) is an intermediate layer, and each neuron performs threshold processing on the sum of data input from the n−1th layer and weights the summation of the n + 1th layer. Input to the neuron. The Nth layer is an output layer, and each neuron performs threshold processing on the sum of the data input from the (N-1) th layer and outputs the result.

  The hierarchical neural network used in the present invention is learned using an error back propagation learning method. A method for learning a hierarchical neural network will be described below. Fluctuation data and / or mel cepstrum coefficients obtained from the test audio signal are used as input signals, and an impression value indicating the degree of a specific impression received by humans from the music represented by the test audio signal is used as a teacher signal. To do. The initial value of the weight of each neuron is set to a small value in the range of about -0.1 to 0.1 by random numbers. Further, a learning rate η (0 <η ≦ 1) is set.

An input signal X _i (i = 1,..., L ₁ ) is input to the input layer of the hierarchical neural network, and calculation is performed with neurons in each layer to obtain an output from the output layer. The learning rule δ _j ^N in the output layer is calculated from the error between the output out _j ^N (j = 1,..., L _N ) of the output layer and the teacher signal y _j . Specifically, δ _j ^N is calculated using the following equation (3).

The weight between the (n−1) -th layer i-th neuron and the n-th layer j-th neuron is w _{j, i} ^{n, n−1} , the value in the n-th layer j-th neuron is out _j ⁿ , The error δ _j ⁿ in the n-th layer j-th neuron is calculated in order from n = N−1 to n = 1 using the calculated δ _j ^N. Specifically, δ _j ⁿ is calculated using the following equation (4).

Using the calculated error δ _j ⁿ , the amount of change Δw _{j, i} ^{n, n−1} of the weight w _{j, i} ^{n, n−1} of each neuron is calculated. Specifically, Δw _{j, i} ^{n, n−1} is calculated using the following equation (5).

A new weight w _{j, i} ^{n, n-1} is calculated by adding the calculated Δw _{j, i} ^{n, n-1} to w _{j, i} ^{n, n-1} . The learning of the hierarchical neural network is performed by repeating the calculation of the weights w _{j, i} ^{n, n-1} until the square error between the output obtained from the test speech signal and the teacher signal becomes sufficiently small.

  The computer program 141 includes a program for executing a hierarchical neural network that has been learned in advance so as to convert the input fluctuation data and the mel cepstrum coefficient and output an impression value. In the present embodiment, an impression value indicating the degree of “transparency” impression received by a person from a song, an impression value indicating a degree of “brightness” received by a person from a song, and a “humanity received from a song” Three types of hierarchical neural networks that output impression values indicating the degree of impression of “strength” are used. The hierarchical neural network that outputs the impression value of “transparency” receives the first-order and second-order mel cepstrum coefficients calculated in step S4, and shows one impression value that indicates the degree of the impression of transparency as a multi-stage numerical value. Is a 2-input 1-output hierarchical neural network. The hierarchical neural network that outputs the impression value of “brightness” and the hierarchical neural network that outputs the impression value of “severity” are calculated in the respective frequency bands of Low, Middle, and High acquired in step S2. The degree of impression of “brightness” and “intensity” by inputting the inclination data of the approximate straight line and the six fluctuation data as the Y-intercept and the first and second order mel cepstrum coefficients calculated in step S4. This is an 8-input 1-output hierarchical neural network that outputs one impression value that represents a multi-stage numerical value. Each hierarchical neural network is learned to output each impression value as a numerical value ranging from 0 to 1.

  FIG. 14 is a flowchart showing a processing procedure in a subroutine of impression value conversion processing. The CPU 11 normalizes the six fluctuation data acquired in step S2 and the two mel cepstrum coefficients calculated in step S4, for example, in a range of 0 to 1 (S51). Next, the CPU 11 inputs the normalized mel cepstrum coefficient to the hierarchical neural network that outputs the impression value indicating the degree of impression of “transparency”, performs the processing of the hierarchical neural network, and performs the impression of the “transparency” impression. An impression value indicating the degree is obtained (S52). Next, the CPU 11 inputs the normalized fluctuation data and the mel cepstrum coefficient to the hierarchical neural network that outputs an impression value indicating the degree of impression of “brightness”, and performs processing of the hierarchical neural network. An impression value indicating the degree of impression of “sa” is obtained (S53). Next, the CPU 11 inputs the normalized fluctuation data and the mel cepstrum coefficient to the hierarchical neural network that outputs an impression value indicating the degree of impression of “strength”, and performs processing of the hierarchical neural network. An impression value indicating the degree of impression of “Sashi” is obtained (S54). The CPU 11 stores in the RAM 12 impression values indicating the degrees of impression of the obtained “transparency”, “brightness”, and “intensity”, ends the impression value conversion process, and returns the process to the main.

  Next, the CPU 11 executes a tempo correction process for correcting the tempo of the music detected in step S3 using the impression value indicating the degree of impression of “strength” obtained in step S5 (S6). FIG. 15 is a flowchart showing a processing procedure in a subroutine of tempo correction processing. Here, the tempo is represented by a number of bpm (beat per minute) including a beat in one minute in the music. A tempo period of 0.3 s corresponds to 200 bpm, and a period of 1 s corresponds to 60 bpm. The impression value is a numerical value in the range of 0-1.

  The CPU 11 determines whether or not the impression value of intensity is greater than 0.55 (S601). If the impression value of intensity is greater than 0.55 (S601: YES), the CPU 11 determines whether or not the tempo is 75 bpm or less (S602). When the tempo is 75 bpm or less (S602: YES), since it is unnatural that the tempo is small although it is intense, the CPU 11 determines that the tempo should be larger and the CPU 11 stores the tempo stored in the RAM 12. Is corrected to double (S603). Next, the CPU 11 ends the tempo correction process and returns the process to the main.

  If the impression value of intensity is 0.55 or less in step S601 (S601: NO), or if the tempo is greater than 75 bpm in step S602 (S602: NO), the CPU 11 determines that the impression value of intensity is 0. It is determined whether it is 47 or less (S604). If the impression value of intensity is 0.47 or less (S604: YES), the CPU 11 determines whether or not the tempo is 130 bpm or more (S605). When the tempo is 130 bpm or more (S605: YES), it is unnatural that the tempo is large although the intensity is small, so the CPU 11 halves the tempo, assuming that the tempo should be smaller. Correction is performed (S606). Next, the CPU 11 ends the tempo correction process and returns the process to the main.

  If the impression value of intensity is greater than 0.47 in step S604 (S604: NO), or if the tempo is less than 130 bpm in step S605 (S605: NO), the CPU 11 determines that the impression value of intensity is 0.51. It is determined whether or not the following is true (S607). If the impression value of intensity is 0.51 or less (S607: YES), the CPU 11 determines whether or not the tempo is 150 bpm or more (S608). When the tempo is 150 bpm or more (S608: YES), the CPU 11 advances the process to step S606.

  If the impression value of intensity is greater than 0.51 in step S607 (S607: NO), or if the tempo is less than 150 bpm in step S608 (S608: NO), the CPU 11 determines that the impression value of intensity is 0.59. It is determined whether or not the following is true (S609). If the impression value of intensity is greater than 0.59 (S609: NO), the CPU 11 ends the tempo correction process without changing the tempo and returns the process to the main. If the impression value of intensity is 0.59 or less (S609: YES), the CPU 11 determines whether or not the tempo is 180 bpm or more (S610). When the tempo is 180 bpm or more (S610: YES), the CPU 11 advances the process to step S606. When the tempo is smaller than 150 bpm (S610: NO), the CPU 11 ends the tempo correction process without changing the tempo and returns the process to the main.

  Next, the CPU 11 associates the three types of impression value and tempo obtained from the audio signal with the music data that is the basis of the audio signal, and classifies the music data by associating the impression value and tempo with the music data. Information is stored in the storage unit 14 (S7). CPU11 complete | finishes the process of a music classification method above. The music classification device 1 executes the above music classification method processing for each piece of music data stored in the storage unit 14.

  FIG. 16 is a conceptual diagram illustrating an example of the contents of the classification information stored in the storage unit 14. In the classification information, the music data name is recorded, and the tempo and the impression values of transparency, brightness, and intensity are recorded numerically in association with the music data name. The music data is classified by the characteristics by associating the tempo and the impression value indicating the characteristics of the music.

  Further, the storage unit 14 stores correspondence information in which the features of the music data are associated with each item in order to classify the music data into several items. FIG. 17 is a conceptual diagram illustrating an example of the content of correspondence information. Specific numerical ranges of tempo and impression value are set for items such as Nori Nori, Healing and Relaxing. For example, with respect to the items of the nori group, the tempo is large and the impression value values of brightness and intensity are set large. For the relaxed items, the tempo is small, the transparency and brightness impression values are medium, and the intensity impression values are set small. The song A. shown in FIG. The music data of mp3 is classified into a relaxed system. The music data of mp3 is classified into a nori system.

  The music classification device 1 performs processing for transmitting the above music data, classification information, and correspondence information stored in the storage unit 14 to the music reproduction device 2. In accordance with the computer program 141 loaded in the RAM 12, the CPU 11 executes processing for causing the interface unit 17 to transmit music data, classification information, and correspondence information to the music playback device 2. The music reproducing device 2 receives the music data, classification information, and correspondence information transmitted from the music classification device 1 by the interface unit 27, and the control unit 21 stores the received music data, classification information, and correspondence information in the storage unit 22. Remember. The user removes the cable connected to the interface unit 27 and uses the music reproducing device 2 by carrying it.

  The music reproducing device 2 can search for music data based on the characteristics of music when searching for desired music data from a plurality of music data stored in the storage unit 22. When the user operates the operation unit 23 and a search instruction is input to the music playback device 2, the control unit 21 causes the display unit 28 to display a list of item names recorded in the correspondence information. The user operates the operation unit 23 to select any item from items such as a scouring system, a healing system, and a relaxing system. The control unit 21 sets the tempo and impression value stored in the classification information of the music data stored in the storage unit 22 to the tempo and impression value set in the corresponding information for the selected item. Extract music data that falls within the numerical range. The control unit 21 displays the name of the extracted music data on the display unit 28, and the user can select desired music data from the music data on which the name is displayed. When the user operates the operation unit 23 and music data is designated, the control unit 21 reads the designated music data from the storage unit 22, causes the data processing unit 24 to decode the music data, and outputs the output unit 25. To output sound. In this way, the user can search the music data based on the impression received from the music and listen to the music.

  Next, a simulation result for classifying music data using the present invention will be described. FIG. 18 is a distribution diagram showing the distribution of the impression value output for the test song by the learned hierarchical neural network and the human evaluation of the impression value for the test song. 18A shows the distribution of brightness impression values, FIG. 18B shows the distribution of transparency impression values, and FIG. 18C shows the distribution of intensity impression values. In each figure, the vertical axis represents the impression value, and the horizontal axis represents the song number corresponding to each test song. In the figure, the output from the hierarchical neural network is indicated by white diamonds, and the evaluation value of the impression value by the person is indicated by a rectangular filled area. As shown in the figure, the evaluation value of the impression value by a person is quantized in a plurality of stages.

  From the result of the simulation shown in FIG. 18, the mean square error between the output of the learned hierarchical neural network and the evaluation value by the person of the impression value was calculated. The mean square error was calculated using the following equation (6).

  Also, the mapping accuracy was calculated by converting the mean square error into a more easily understood value. The mapping accuracy was calculated using the following equation (7).

The mapping accuracy is 100% if the mean square error is (0.067) ² or less, and 0% if the mean square error is (0.3) ² or less. 0.067 is based on the quantization error when the evaluation value of the impression value by the person is quantized, and 0.3 is adjusted so that the projection accuracy of the distribution determined to be practically no problem is 90%. This is the value determined.

  FIG. 19 is a table comparing an error between the learned output of the hierarchical neural network and the evaluation value by the person of the impression value between the present invention and the prior art. FIG. 19A shows an error between the output obtained from the hierarchical neural network using the mel cepstrum coefficient by the simulation of the present invention and the evaluation value of the impression value by the person. FIG. 19B shows an error between the output obtained from the hierarchical neural network and the evaluation value of the impression value by the person by the simulation of the prior art as disclosed in Patent Document 1. In the simulation result according to the present invention, the mean square error is small in any impression value as compared with the simulation result according to the prior art. In the simulation result according to the present invention, the mapping accuracy is improved to the extent that there is no practical problem. Therefore, in the present invention, the deviation between the impression of the music indicated by the impression value obtained by the music classification device 1 and the impression that the user who actually listened to the music feels from the music is smaller than in the prior art, and the actual It is clear that the music data can be classified by the impression value that matches the impression.

  As described above in detail, in the present invention, a hierarchical neural circuit learned to obtain a mel cepstrum coefficient from an audio signal representing the sound of a song, and to input the mel cepstrum coefficient and output an impression value of “transparency”. Using the network, the mel cepstrum coefficient is converted into an impression value indicating the degree of “transparency” impression that a person receives from the music. By calculating the impression value from the mel cepstrum coefficient that represents the sound quality that characterizes the song, the impression that the user actually listening to the song feels from the song and the impression of the song indicated by the calculated impression value are different from those of the conventional technology. Get smaller. Therefore, the music data is classified according to the impression that the user feels from the music, and the user can search and listen to the music that feels the desired impression according to the preference more accurately.

  In the present invention, the average power time-series power spectrum is obtained in each frequency band of Low, Middle, and High, and the slope of the approximate straight line and the Y-intercept of the logarithm display power spectrum vary in volume in each frequency band. Acquired as fluctuation data indicating the frequency distribution. Fluctuation data and mel cepstrum coefficients are input to the brightness data and mel cepstrum coefficients, and the brightness data and mel cepstrum coefficients are learned to output impression values of "brightness" and "hardness". And an impression value indicating the degree of impression of “intensity”. Compared to the conventional technology for obtaining impression values from fluctuation data by obtaining the impression values from fluctuation data that characterizes periodic fluctuations in volume and the mel cepstrum coefficient, the impression and determination that the user who actually listened to the music feels from the music The deviation from the impression of the music indicated by the impression value is reduced.

  In the present invention, the tempo of the music is obtained based on the difference value between the average power time-series power spectrum and the approximate line. Based on the period A of the component having a difference value greater than or equal to a predetermined value in the period of 0.3 to 2 s, the tempo period is in the range of 0.3 to 1 s, and is a period that is double or half of the tempo period The tempo period is set to A / 4, A / 2, A, A × 2, or the like so that the power spectrum component at is a sufficient value. In the present invention, the tempo is increased when the impression value is large, and the tempo is decreased when the impression value is small, in accordance with the magnitude of the impression value indicating the degree of impression of “intensity”. , Correct the tempo. Since the tempo is corrected based on the impression value with a small deviation from the actual impression, the tempo of the music can be determined with higher accuracy.

  Note that in the music classification method shown in the present embodiment, an algorithm for performing FFT on an audio signal obtained by sampling music data separately in the fluctuation data acquisition process in step S2 and the mel cepstrum coefficient calculation process in step S4. However, in the music classification method of the present invention, the processing may be performed using an algorithm that collectively performs FFT processing. In the present embodiment, the first and second order mel cepstrum coefficients are used to obtain the impression value. However, in the music classification method of the present invention, the mel cepstrum coefficients of the third order or higher are used. Form may be sufficient.

  In the present embodiment, the music classification apparatus 1 uses a general-purpose computer, and the CPU 11 operates according to the computer program 141 to realize the music classification method processing of the present invention by software. However, the present invention is not limited to this, and the music classification apparatus 1 according to the present invention may be configured such that part or all of the processing of the music classification method according to the present invention is realized by hardware. For example, the music classification device 1 includes hardware that specially executes processing such as processing for acquiring an audio signal by sampling, processing for performing FFT on the audio signal, processing of a hierarchical neural network, and the like. The form which implement | achieves the process of the music classification method of this invention combining the process performed according to the program 141, and the process which a hardware performs may be sufficient.

  In the present embodiment, the music playback device 2 has been shown to search for music data using items that associate the tempo and impression value of the music, but the present invention is not limited to this, and the music playback device 2 2 may be a form in which music data is searched by specifying the tempo or impression value of the music. Further, in the present embodiment, the music classification apparatus 1 has shown the form in which the music classification method of the present invention is executed on the music data stored in advance in the storage unit 14. The apparatus 1 is not limited to this, and may be configured to execute the process of the music classification method of the present invention when performing the process of reading data from a recording medium such as a CD and generating music data.

It is a block diagram which shows the structure of the music classification device and music reproduction apparatus of this invention. It is a flowchart which shows the procedure of the process which CPU performs in this invention. It is a flowchart which shows the procedure of the process in the subroutine of fluctuation data acquisition process. It is explanatory drawing which shows the example of the result of having repeated FFT of the partial audio | voice signal. It is a characteristic view which shows the example of an average power time series. It is a characteristic view which shows the example of the power spectrum of a logarithm display, and an approximate line. It is a flowchart which shows the procedure of the process in the subroutine of a tempo detection process. It is a flowchart which shows the procedure of the process in the subroutine of a tempo detection process. It is a flowchart which shows the procedure of the process in the subroutine of a tempo detection process. It is a flowchart which shows the procedure of the process in the subroutine of a tempo detection process. It is a flowchart which shows the procedure of the process in the subroutine of a mel cepstrum coefficient calculation process. It is explanatory drawing explaining the content of the process which calculates | requires a filter bank output. It is a schematic diagram which shows a hierarchical neural network. It is a flowchart which shows the procedure of the process in the subroutine of an impression value conversion process. It is a flowchart which shows the procedure of the process in the subroutine of a tempo correction process. It is a conceptual diagram which shows the example of the content of the classification information which a memory | storage part memorize | stores. It is a conceptual diagram which shows the example of the content of corresponding | compatible information. It is a distribution map which shows distribution of the output of the impression value with respect to the test music by the learned hierarchical neural network, and the evaluation by the person of the impression value with respect to the test music. It is the table | surface which compared the difference | error of the output by the learned hierarchical neural network, and the evaluation value by the person of an impression value by this invention and the prior art.

Explanation of symbols

1 Music Classification Device 11 CPU
12 RAM
DESCRIPTION OF SYMBOLS 13 Drive part 14 Memory | storage part 141 Computer program 15 Input part 16 Display part 17 Interface part 18 Communication part 2 Music reproducing apparatus 21 Control part 22 Storage part 23 Operation part 24 Data processing part 25 Output part 26 Headphone 27 Interface part 28 Display part

Claims (9)

A generation step for generating data indicating the characteristics of the music, a conversion step for converting the data generated in the generation step into an impression value indicating the degree of a specific impression received by the human from the music, and associating the impression value with the music data A method of classifying music data, and a method of classifying music data,
The generating step includes
An FFT processing step for performing a fast Fourier transform (FFT) of an audio signal representing the audio of the music;
Obtaining a mel cepstrum coefficient of the audio signal from the FFT result of the FFT processing step,
The converting step includes
Including a step of converting the mel cepstrum coefficient obtained in the generating step into an impression value using a hierarchical neural network that is trained to receive the mel cepstrum coefficient and output an impression value indicating a specific impression level. The music classification method characterized by this. The FFT processing step includes
Obtaining a power spectrum of the partial audio signal by performing FFT on the partial audio signal obtained by dividing the audio signal;
The generating step includes
Calculating an average power obtained by averaging the power of a predetermined frequency band in the power spectrum obtained in the FFT processing step for each of a plurality of partial audio signals included in the audio signal;
Obtaining an average power time series in which the calculated average power is arranged on the time axis in correspondence with the temporal position of each partial audio signal in the audio signal;
Obtaining a power spectrum of the average power time series by performing FFT of the obtained average power time series;
Obtaining an approximate curve obtained by approximating the power spectrum of the obtained average power time series with a predetermined curve;
Further comprising the step of obtaining the obtained parameters of the approximate curve as fluctuation data characterizing periodic fluctuations in volume in the audio signal,
The converting step includes
Fluctuation data and mel cepstrum of the audio signal obtained in the generating step using a hierarchical neural network that is input with fluctuation data and a mel cepstrum coefficient and learned to output an impression value indicating a specific impression level The music classification method according to claim 1, further comprising a step of converting the coefficient into an impression value. Obtaining a difference between the power spectrum of the average power time series obtained from the audio signal in the generation step and the approximate curve;
Obtaining the tempo of the music based on the longest cycle within a predetermined cycle range among the cycles of the components whose magnitude of the difference is equal to or greater than a predetermined value;
Correcting the obtained tempo according to the value of the impression value indicating the degree of the specific impression obtained in the conversion step;
The music classification method according to claim 2, further comprising: associating the corrected tempo with music data. Generation means for generating data indicating the characteristics of the music, conversion means for converting the data generated by the generation means into an impression value indicating the degree of a specific impression received from the music, and associating the impression value with the music data In a music classification apparatus comprising storage means for classifying and storing music data by
The generating means includes
FFT processing means for performing a fast Fourier transform (FFT) of an audio signal representing the audio of the music;
Means for obtaining a mel cepstrum coefficient of the audio signal as feature data from the result of the FFT by the FFT processing means;
The converting means includes
Means for converting a mel cepstrum coefficient obtained by the generating means into an impression value using a hierarchical neural network that has been input to the mel cepstrum coefficient and learned to output an impression value indicating a specific impression level A music classification device characterized by this. The FFT processing means includes
Means for obtaining a power spectrum of the partial audio signal by performing FFT on the partial audio signal obtained by dividing the audio signal;
The generating means includes
Means for calculating an average power obtained by averaging the power of a predetermined frequency band in the power spectrum obtained by the FFT processing means for each of a plurality of partial audio signals included in the audio signal;
Means for obtaining an average power time series in which the average power calculated by the means is arranged on the time axis in correspondence with the temporal position of each partial audio signal in the audio signal;
Means for obtaining the power spectrum of the average power time series by performing FFT of the average power time series obtained by the means;
Means for obtaining an approximate curve obtained by approximating the power spectrum of the average power time series obtained by the means with a predetermined curve;
Means for obtaining the parameters of the approximate curve obtained by the means as fluctuation data characterizing periodic fluctuations in volume in the audio signal;
The converting means includes
Fluctuation data and mel cepstrum of the voice signal obtained by the generation means using a hierarchical neural network that is input with fluctuation data and a mel cepstrum coefficient and learned to output an impression value indicating a specific impression level The music classification apparatus according to claim 4, further comprising means for converting the coefficient into an impression value. Means for obtaining a difference between the power spectrum of the average power time series obtained from the audio signal by the generating means and the approximate curve;
Means for obtaining the tempo of the music based on the longest cycle within a predetermined cycle range among the cycles of the components whose magnitude of the difference is a predetermined value or more;
Tempo correction means for correcting the determined tempo according to the value of the impression value indicating the degree of the specific impression obtained by the conversion means,
The storage means
The music classification apparatus according to claim 5, further comprising means for storing music data in association with the tempo corrected by the tempo correction means. In a computer program for causing a computer to generate data indicating the characteristics of a song and converting the generated data into an impression value indicating a specific degree of impression received by a person from a song,
A procedure for causing a computer to perform a fast Fourier transform (FFT) on an audio signal representing the audio of a song;
A procedure for causing a computer to obtain a mel cepstrum coefficient of the audio signal from the result of the FFT;
Impression value that is output by inputting the obtained mel cepstrum coefficient to a hierarchical neural network that is trained to input a mel cepstrum coefficient and outputting an impression value indicating a specific impression level. The computer program characterized by including the procedure to acquire. A procedure for causing a computer to obtain a power spectrum of the partial audio signal by performing FFT on the partial audio signal obtained by dividing the audio signal;
A procedure for causing a computer to calculate an average power obtained by averaging the power of a predetermined frequency band in the obtained power spectrum for each of a plurality of partial audio signals included in the audio signal;
A procedure for causing the computer to obtain an average power time series in which the calculated average power is arranged on the time axis in correspondence with the temporal position of each partial audio signal in the audio signal;
A procedure for causing a computer to obtain a power spectrum of an average power time series by performing FFT of the obtained average power time series;
A procedure for causing a computer to obtain an approximate curve obtained by approximating a power spectrum of the obtained average power time series with a predetermined curve;
A procedure for causing the computer to obtain the parameters of the obtained approximate curve as fluctuation data characterizing periodic fluctuations in volume in the audio signal;
Fluctuation data and mel cepstrum coefficients of the obtained audio signal are obtained for a hierarchical neural network that has been input to a computer and has been trained to output an impression value indicating a specific impression level by inputting fluctuation data and mel cepstrum coefficients. The computer program according to claim 7, further comprising: a step of acquiring the impression value to be output by inputting. A procedure for causing a computer to obtain a difference between the power spectrum of the average power time series obtained from the audio signal and the approximate curve;
A procedure for causing the computer to obtain the tempo of the music based on the longest cycle within a predetermined cycle range among the cycles of the components whose magnitude of the difference is a predetermined value or more;
The computer program according to claim 8, further comprising: causing the computer to correct the obtained tempo according to an impression value indicating a specific impression level.

Images