KR101057919B1 - How to recommend customized music through analyzing playlists of users - Google Patents

Abstract

According to the present invention, a customized music recommendation method through analysis of playlists of users is performed by extracting MFCC from music data, constructing a music model using hidden Markov modeling (HMM) method, and calculating similarity between the constructed music models. Measuring music modeling and similarity comparing step (Step 1); Using the music similarity obtained in the music modeling and similarity comparison step, a music similarity graph for the music heard by the users is made, and music existing in the past playlist is clustered by the MCL graph clustering method, so that the user has listened to the past. A playlist analysis step (Step 2) of users for grouping similar music to music based on the playlist; It is equipped with a music recommendation step (Step 3) that evaluates how well the music group derived from the playlist analysis of other users matches the user's music group and provides a recommendation list to the user based on the evaluated result.

Description

A PERSONAL ADAPTIVE MUSIC RECOMMENDATION METHOD USING ANALYSIS OF PLAYLISTS OF USERS}

 The present invention relates to a customized music recommendation method through analyzing a playlist of users who grasps a music propensity of a user through content-based music analysis and recommends music suitable for the user's propensity.

Recently, with the rapid development of the Internet, various services available on the web have been created. Online music delivery is one of these new services, which has changed the way people listen to music. Conventional people listened to music through TV or radio, or purchased music media such as CDs to listen to music, whereas now the web has developed, purchase music files such as MP3 through the web to listen to music or stream on the web. More and more people are using direct listening to music provided as a service. Accordingly, online music service providers are making efforts to increase user satisfaction.

However, despite the infinite resources of the web, most service providers offer music to users in a simple way. For example, most service companies categorize music based on keywords such as the name and genre of a musician, or simply rank the songs in the order of the music they listened to. These music classification methods simply show music to users without considering current musical trends or users' personal music preferences. Therefore, in order for users to find their favorite music, they have to find and listen to the music they want by searching on their own website. This keyword search is not a problem if the user knows exactly the title of the song or the name of the artist. However, in order to find new music that suits his or her inclination without specific information, the user can repeatedly search and check the music. The disadvantage is that you have to find the music you want.

To overcome this drawback, some service providers use recommendation techniques to provide new music to their users. By giving users a list of music recommendations, users can find new music more easily.

One way to create a recommendation list is to create a recommendation list for a user based on music that has been heard by several users. One common method is collaborative filtering, which Amazon uses (see Ref. [1]). . Amazon's collaborative filtering creates a list of recommendations that a user can offer to a user by recommending other albums purchased by other users who have purchased the album. The albums on the recommendation list are the most purchased albums among other users. Collaborative filtering creates a recommendation list on the assumption that one user prefers music that many users prefer. This recommendation technique is a recommendation technique that reflects the current musical trends because it creates a list based on the selection of a large number of users. However, if the user has listened to various types of music, it is not recommended to select and recommend the music for each type. Not a suitable technique. Even these users need a different approach.

Therefore, various studies have been made on music recommendation, and a method of recommending music in consideration of the nature of music is called content-based music recommendation method. It is possible to make a recommendation even if there is no information outside of music by analyzing the music itself, and thus, various studies on content-based music recommendation have been made.

K. Kaji (see Ref. [2]) conducted a study on a system for recommending music using similarity between music and user's disposition. Reference [2] is a study on a system for recommending music to a user in consideration of both lyrics and annotation of music and a situation in which the user is currently listening to music. This has the advantage of recommending music suitable for the current user's situation, but for this purpose, the user must manually enter the music comment and the current situation.

B. Logan (see Ref. [3]) conducted a study to recommend music by judging the similarity of sound waves of music heard by the user and the rest of the music. Reference [3] is a study on a system which recommends a music having the highest average similarity by calculating an average similarity between a new music and music in a user's music playlist. This system has a high accuracy of recommendation if the user always listens to similar types of music, but does not have high accuracy of recommendation when listening to various types of music because the average similarity is obtained.

P. Cano (see Ref. [4]) conducted a study to recommend music by analyzing the sound waves of music. Reference [4] is a study on a system that applies a method that analyzes all music to express and retrieve the tone and rhythm of music numerically. The user can look at the level of music he prefers and search the system for music with similar numbers. This system provides an easy environment for recommending music similar to one music, but cannot recommend music considering various music.

The present invention has been invented in view of the above, and extracts an MFCC representing the nature of sound waveforms from music as information for constructing a music model, and makes a model using a hidden Markov modeling technique. The similarity is compared, and then the MCL graph clustering process is used to group the music listened to by similar music to analyze what type of music the user listened to. Based on the analysis result, the closest music is compared by comparing the similarity between music groups. The purpose of the present invention is to provide a customized music recommendation method through analyzing playlists of users who have made the music list as a recommendation list and provide it to the user.

Custom music recommendation method through analyzing the playlist of the user according to the present invention for achieving the above object,

Music modeling and similarity comparing step (Step 1), which extracts MFCC from music data, constructs a music model using Hidden Markov Modeling (HMM) method, calculates and measures similarity between the constructed music models,

Using the music similarity obtained in the music modeling and similarity comparison step, a music similarity graph for the music heard by the users is made, and music existing in the past playlist is clustered by the MCL graph clustering method. Playlist analysis step (Step 2) of the users of grouping the music similar music based on the playlist, and

It is equipped with a music recommendation step (Step 3) that evaluates how well the music group derived from the playlist analysis of other users matches the user's music group and provides a recommendation list to the user based on the evaluated result.

According to the present invention, when the music genres are similar, the recommendation list may be prepared and provided to the user at a ratio similar to the genre ratio of the music heard by the user.

Prior to describing the method according to the present invention, an outline of a music recommendation method to be implemented in the present invention will be described.

The music recommendation method through content-based music analysis is a method of analyzing the nature of music and using it for recommendation. This method extracts characteristic information representing the characteristics of music from music that a user listens to improve user's satisfaction, and analyzes it to recommend music that suits the individual's inclination. Content-based music analysis and collaborative filtering recommendation techniques can be used to recommend music that is similar to what the user has heard.

The present invention provides a method of grouping music listened to by a user through content-based music analysis, and recommending music suitable for a user's music inclination using information of music groups of other users similar to each group.

Elements that represent the characteristics of music include pitch of notes, timbre, rhythm, flow, and harmony of sounds (see Ref. [5]). By analyzing the characteristics of the music with respect to the music heard by the user can determine the music preference of a user.

In the present invention, a music model is constructed using feature information of music for use in music analysis. As the information for constructing the music model, MFCC (Mel Frequency Cepstral Coefficients) representing the characteristics of the waveform of sound is extracted from the music, and the model is created using the hidden Markov modeling technique. Then, how similar these models are, and grouping the similar music to the music heard by the user through the clustering step. Subsequently, music groups of other users similar to the groups are selected, and based on the selection result, new music close to the types of music heard by the user is created as a recommendation list and provided to the user.

The MFCC is one of the typical sound waves used in the speech recognition field (see Ref. [6]). Since the MFCC contains information on the waveform of sound and is useful in the field of music analysis, many researchers have conducted a study of analyzing music using the MFCC.

Tzanetakis (see Ref. [7]) conducted a study to classify the musical genre of an audio signal using characteristics of various music including MFCC, and JJ Aucouturier (see Ref. [8]) was based on MFCC. A study was conducted to measure similarity.

In the present invention, the MFCC extracted from the music is used to analyze the properties of the music. The similarity between the music is measured based on the extracted MFCC, and the similarity information is used for the music analysis and recommendation process. Since the similarity between MFCCs cannot be directly obtained, in order to construct a music model based on the extracted information, a modeling method is needed to abstract the nature of music and build it as a model. The hidden Markov modeling method is a method for modeling sequence information that changes with time, and is a modeling method suitable for building a music model (see Ref. [9]). Several researchers have used hidden Markov modeling techniques to model music using MFCC. X. Shao (see Ref. [10]) conducted a study to classify music genres using hidden Markov models and MFCC, while Y. Qi (see Ref. [11]) used hidden Markov models for music analysis. Used.

As described above, the present invention provides a music recommendation method based on music playlist analysis heard by a user. The music recommendation method according to the present invention extracts MFCC (Mel Frequency Cepstral Coefficients) representing the waveform of sound from the music for use in the music recommendation process, and extracts the extracted information from the music model using a hidden Markov modeling technique. Build. One music is modeled through one hidden Markov model, and the similarity between the music models is identified by comparing the similarities between the music models. Based on the similarity, music that the user listened to in the past is grouped with music of similar nature, and similar music groups are selected by comparing with music groups of other users. Finally, the system selects new music from another user's music group and recommends it to the user.

   In order to evaluate the method according to the present invention, experiments were conducted using information used by real users on the sound source site. Since the method according to the present invention analyzes the music playlists of the users and finally creates a music list to be recommended to the users, the system is evaluated by analyzing the similarity between the recommended music list and the user's playlist.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a schematic structural diagram of a system for implementing a music recommendation method according to the present invention. As shown in FIG. 1, the music recommendation method according to the present invention includes a music modeling and similarity comparison step (Step 1), a user playlist analysis step (Step 2), and a music recommendation step (Step 3).

In the music modeling process, all music is constructed as a music model using Hidden Markov Models (HMM). The music model thus constructed is used in playlist analysis and recommendation. The playlist analysis of users is a process of grouping all the musics that have been heard in the past among similar users. In the recommendation process, music is selected based on a comparative analysis between analyzed music groups and a final recommendation list is prepared.

Next, the music modeling and similarity comparison step (Step 1), the playlist analysis step (Step 2), and the music recommendation step (Step 3) of the user will be described in detail.

1. Music Modeling and Similarity Comparison

In the present invention, music data is used to analyze music. However, since music data only contains sound waveform information, a process of modeling this data as an abstracted model is required.

FIG. 2 is a diagram for explaining music modeling and similarity comparison steps. In order to construct a music model, MFCC is extracted from music, and a music model is constructed using a hidden Markov modeling (HMM) method. The similarity between the music is obtained by comparing the similarity between the models representing each music.

1) Build a music model

In order to construct a music model, an MFCC representing the characteristic information of music is extracted. The MFCC is made up of many indices. Higher order indices are sensitive to sound spectral changes, while lower order indices are not.

Since it is already verified (see Ref. [8]) that the sound waveform information of music can be efficiently represented by using only eight low-order indices, the present invention is insensitive to changes in the MFCC from the first to the eighth order. Only eight indices are used.

The eight exponents divide the music into small pieces of 50ms in length and extract them from each piece of music to take into account one of the other characteristics of music, the flow of music. Therefore, the MFCC for a music is represented by a sequence of 8-dimensional vectors, and in the present invention, a hidden Markov modeling method capable of modeling sequence information that changes over time to construct a model using sequence data. Use to build a music model.

The reason for building the music model is to find the similarity of the music. Hidden Markov modeling techniques generate models by learning sequential changes in sequence information as input (see Ref. [12]). Since it is difficult to directly calculate the similarity of the sequence information extracted from the sound waves of music, a Markov model is created for the music sequence extracted from one music, and the similarity between the models is calculated to calculate the similarity of the music. In order to find the similarity among all the music, one music model is constructed for each music.

2) similarity measurement

Similarity measurement is a process of measuring similarity between music. The similarity between music is measured by calculating the similarity between the music models constructed in the previous stage. The similarity between the hidden Markov models can be obtained by taking advantage of the fact that one of the features of the hidden Markov model is the observation sequence that is most similar to the model. The equation for calculating the similarity between the models is modified to define the equation (1) for obtaining the musical similarity.

---- (1)

---- (One)

In Equation (1), S _ij means the similarity between the i- th music and the j- th music, which is obtained by measuring the similarity between the two models. H _i and H _j denotes the i-th and j-th music model, respectively, and O _i and O _j is an observation sequence generated by each model. By dividing the difference between the probability that the observation sequence generated by the model itself matches the model and the probability that the observation sequence generated in the other model matches the model, divided by N , the length of the sequence, to eliminate the difference in the probability value according to the length of the sequence. Find the degree of similarity between the two models. If both models are similar because of the observed sequence of a model is created of each other becomes higher probability of matching with other models As the H _i and H _j similar to the value of the S _ij has a smaller value.

Since the similarity of music is used in different processes in the system, the similarity with all other music is obtained for each music. In the present invention, the similarity between the i- th music and the rest of the music is expressed by a similarity vector as shown in Equation (2).

---- (2)

---- (2)

In formula (2), V _i represents a similarity vector of the i th music, and | M | Represents the number of all music in the entire database. The similarity vector is stored in a database, which is used in analyzing a user's music preferences and recommending music.

2. User Playlist Analysis Stage

This step is to group the music together with similar music based on the music playlist that users have listened to in the past. In the present invention, using the music similarity obtained in the previous step for grouping, a music similarity graph for the music heard by users is made, and music existing in the past playlist using the MCL graph clustering method (see Ref. [13]) is used. Cluster them.

1) Music Similarity Graph

The music similarity graph shows the similarity relationship between the music in the user's playlist. In order to cluster the music heard by the user, information about the relationship between each music is needed. Thus, a music similarity graph as shown in FIG. 3 is created using the music similarity obtained in the previous step. 3 is an example of a music similarity graph for six pieces of music heard by a user. The node of the graph, Iai, represents each music, and the strength of the node's connecting line represents the similarity between the two musics.

2) Music Clustering

The music similarity graph is used to cluster the music that the user has listened to in order to group the music that the user has listened to. In the clustering process, the user may listen to various types of music, so a clustering technique is needed to improve the accuracy of the analysis.

However, when applying the clustering technique to the present invention, there is another problem to consider in addition to improving the accuracy. In order to cluster music efficiently, it is necessary to determine how many clusters music should be divided. Most clustering techniques require the number of clusters to be determined in advance. For example, the K-mean algorithm, one of the common techniques, requires that K, which represents the number of clusters, be set to a fixed value in advance. This is not an appropriate algorithm for use in systems that need to dynamically change the number of clusters.

Therefore, there is a need for an algorithm capable of efficiently clustering music by dynamically adjusting the number of clusters. The proposed music recommendation system (see Ref. [14]) applies an advanced K-means algorithm to the system that can dynamically change the number of clusters. This clustering technique can change the number of clusters dynamically, but the vector value used for clustering increases in dimension as the number of music heard by the user increases, which makes it impossible to efficiently cluster when many users listen to the music. Was present.

In the present invention, the MCL graph clustering technique is applied to dynamically adjust the number of clusters and to efficiently cluster regardless of the number of music.

The MCL graph clustering technique is a clustering method for creating groups of similar nodes by removing a connection line except for a connection line having a high strength based on the strength of the connection line between nodes of the graph. Techniques applied in the field of genetic information analysis were applied to enable grouping using music similarity information.

3. Music Recommendation Steps

The final purpose of the music recommendation method according to the present invention is to recommend music suitable for the user based on the user's playlist. In this step, it is evaluated how well the music group derived from the analysis of the playlists of other users matches with the music group of the user, and provides a recommendation list to the user based on the evaluated results. The similarity between two music groups is defined as shown in Equation (3).

---- (3)

---- (3)

In Equation (3), C means one music group, and C _sim (C _n , C _m ) is a value representing the similarity between music groups C _n and C _m . Max _sim (I _i , C _m ) is the similarity of the music in one group with the most similar music among the other group, and N is the smaller of the number of music in the two groups. The value to indicate. Since the average of the similarity values for N pieces of music is the value of C _sim (C _n , C _m ) , the similarity between groups is higher when there are more similar pieces of music in the two groups. . The present invention selects the groups having the highest C _sim value for each music group of the user, and provides a list of new music that the user does not listen among in the groups in the recommendation list.

Experiment and evaluation

For the experiment, the experiment was conducted using music playlists used by real users for the system. The music playlist data is a playlist of users of Last.fm (see Ref. [15]) which is a sound source providing site. The data used in the experiment was 786 playlists and 3727 songs, and the number of songs in the 786 playlists was at least 10 and at most 50. In the present invention, the metadata that distinguishes the genre of music is not used in the recommendation process, but in order to evaluate how accurate the music is recommended according to the music heard by the user, the genre information of the music is used in the experiment. Since genre information is based on tag information described by users of Last.fm for each music, the information is classified by actual users, not metadata reflecting expert opinions. The genre used in the experiment was based on the genre classification of music defined in Wikipedea (see Ref. [16]), and electronic, metal, and jazz, which were heard a lot by users, were used. The five genres, and the rest of the genres, were grouped into one, pop and rock. The 3727 pieces of music are mono sample data of 16 kHz, and the MFCC is extracted from these pieces of music to build a music model.

The experiment was conducted to check the genre ratio of music recommended by the system according to the genre ratio of music heard by the user. By classifying the users by the genre they listened to the most, we checked the genre ratio of music recommended by the system to users. For example, we selected the users who listened to Jazz as a test set and checked the ratio by comparing the average of the genre ratios heard by all users in the set with the average genre ratio recommended by the system.

Table 1 summarizes the experimental results for a set of users with a high percentage of electronic music in the playlist.

Table 1 Suggested Results for Electronic User Set

Electronic Metal Jazz Pop Rock etc LG 95.3% 0% 0% One% 3.4% 0.5% RG 67.1% 1.1% 8.9% 4.1% 12.3% 6.5%

In Table 1, LG refers to the genre of music heard by the user, and RG refers to the genre of music recommended by the system. On average, the playlists of users in the user set included electronic music at an average rate of 95.3%, and the music list recommended by the system to users in this set averaged 67.1% electronic. Music was included. Although not exactly the same as the rate they heard, the recommendation list contained a high percentage of electronic music.

The results of the system's recommendation for different user sets are shown in Tables 2, 3, 4, and 5.

Table 2 Suggested Results for Metal User Set

Electronic Metal Jazz Pop Rock etc LG 0% 77.7% 0% 0% 21.9% 0.4% RG 0% 43.6% 0% 6% 50.4% 0%

Table 3 Suggested Results for the Jazz User Set

Electronic Metal Jazz Pop Rock etc LG 0.8% 0% 82.8% 2.3% 0.9% 13.2% RG 7% 2% 80.1% 1.8% 8.3% 0.8%

Table 4 Recommended Results for Pop User Set

Electronic Metal Jazz Pop Rock etc LG 0% 0% 0% 98.8% 0.2% One% RG One% 2.8% 1.2% 65.1% 10.9% 19%

Table 5 Suggested Results for Lock Set

Electronic Metal Jazz Pop Rock etc LG 0% 0.1% 0% 1.3% 98.3% 0.3% RG One% 6.7% 1.7% 6.6% 77.6% 6.4%

The music genre ratio for the rest of the results is similar to the average genre ratio of music heard by users by user set except for the metal user set.

As can be seen from the results of Table 2, the system according to the present invention recommends music differently from the ratio of music to metal music and rock music. The average metal music rate for this user set was 77.7% and rock music was 21.9%, but the recommended average music genres were 43.6% and 56.4%, respectively. This is because in the present invention, metal music and rock are similarly obtained, and real users frequently attach rock and metal tags to a single music at the same time.

4 is a diagram illustrating a music ratio distribution for a music ratio and a recommendation result for each user set. As shown in FIG. 4, the music recommendation method according to the present invention grasps the type of music heard by the users, and creates and provides a music suitable for the type as a recommendation list.

As described above, the present invention provides a music recommendation system in consideration of music preferences of users, and for this purpose, a music model is constructed using MFCC and hidden Markov modeling techniques, and the similarity between music is measured through this model. This similarity is used to group music that users have listened to in the past to similar music groups, and to compare and compare music groups with other users to finally create and provide a recommendation list of new music similar to the music propensity of the user's playlist. do.

In addition, this invention is not limited to the above-mentioned Example, Of course, it can change and implement variously within the range which does not deviate from the summary of this invention.

references

[1] G. Linden, B. Smith, and J. York, "Amazon.com Recommendations: Item-to-item Collaborative Filtering", IEEE Internet Computing, Vol. 7, pp. 76-80, 2003.

[2] K. Kaji, K. Hirata, and K. Nagao, "A Music Recommendation System Based on Annotation about Listeners' Preferences and Situations", the 1st International Conference on Automated Production of Cross Media Content for Multi-Channel Distribution, pp . 231-234, 2005.

[3] B. Logan, "Music Recommendation from Song Sets", the 5th International Conference on Music Information Retrieval, 2004.

[4] P. Cano, M. Koppenberger, and N. Wack, "Content-based Music Audio Recommendation", the 13th annual ACM International Conference on Multimedia, 2005.

[5] N. Scaringella, G. Zoia, and D. Mlynek, "Automatic Genre Classification of Music Content", IEEE Signal Process Magazine, Vol. 23, No. 2, pp. 133-141, 2007.

[6] B. Logan, "Mel Frequency Cepstral Coefficients for music modeling", The First International Symposium on Music Information Retrieval, 2000.

[7] G. Tzanetakis, "Musical Genre Classification of Audio Signals", IEEE Transactions on Speech and Audio Processing, Vol. 10, No. 5, pp. 293-302, 2002.

[8] J.J. Aucouturier and Francois Pachet, "Music Similarity Measures: What's the Use?", International Symposium on Music Information Retrieval, pp. 157-163, 2002.

[9] L.R. Rabiner, "Fundamentals of speech recognition", Prentice-Hall, 1993.

[10] X. Shao, C. Xu, and M.S. Kankanhalli, "Unsupervised Classification of Music Genre Using Hidden Markov Model", IEEE International Conference on multimedia and expo, Vol. 3, pp. 2023-2026, 2004.

[11] Y. Qi, John W. Paisley, and L. Carin, "Music Analysis Using Hidden Markov Mixture Models", IEEE Transactions on Signal Processing, Vol. 55, No. 11, pp. 5209-5224, 2007.

[12] L. Rabiner and B. Juang, "An Introduction to Hidden Markov Models", ASSP Magazine, Vol. 3, pp. 4-16, 1986.

[13] S. V. Dongen, "Graph Clustering by Flow Simulation", PhD thesis, University of Utrecht, 2000.

[14] K.S. Kim, D.H Lee, T.B Yoon, J.H Lee, "A Music Recommendation System Based on Personal preference Analysis", The First International Conference on Applications of Digital Inforemation and Web Technologies 2008, pp. 102-106, 2008

[15] http://www.last.fm

[16] http://www.wikipedia.org

1 is a schematic structural diagram of a system implementing a music recommendation method according to the present invention;

2 is a view for explaining music modeling and similarity comparison step;

3 is a music similarity graph for six pieces of music heard by a user;

4 is a diagram illustrating a music ratio distribution for a music ratio and a recommendation result for each user set.

Claims (8)

A music modeling and similarity comparing step (Step 1) of extracting MFCC from music data, constructing a music model using a hidden Markov modeling (HMM) method, and calculating and measuring similarity between the constructed music models; Using the music similarity obtained in the music modeling and similarity comparison step, a music similarity graph for the music heard by the users is made, and music existing in the past playlist is clustered by the MCL graph clustering method, so that the user has listened to the past. A playlist analysis step (Step 2) of users for grouping similar music to music based on the playlist; By using the music similarity obtained in the music modeling and similarity comparison step, a music similarity graph for the music heard by other users is created, and the music existing in the past playlist is clustered by the MCL graph clustering method, so that the other users in the past Based on the music playlists that you have listened to, the user analyzes the playlists of other users that group similar songs together, and evaluates the similarity between the grouped music group and the user's music group, and recommends the list to the user based on the evaluated similarity results. Music recommendation step (Step 3) to provide The similarity between the music group of other users grouped through playlist analysis and the user's music group,

(Where C is a music group, C _sim (C _n , C _m ) is the similarity between music groups C _n and C _m , and Max _sim (I _i , C _m ) is a piece of music in a group The similarity of the most similar music among the music in the other group, where N is the smaller value of the number of music belonging to the two groups) Customized music recommendation method through the analysis of the playlist of the user, characterized in that defined by. delete delete delete The method of claim 1, wherein the music similarity graph indicates that each node represents each music, and the strength of the connection line of the node means the similarity between the two musics. The method of claim 1, wherein the MCL graph clustering method is configured to create groups of similar nodes based on the strength of the connection line between the nodes of the music similarity graph. . delete delete

Images