KR20230111382A - Artificial intelligence-based user-customized music recommendation service device - Google Patents

Abstract

The present invention relates to an artificial intelligence-based user-customized music recommendation service device, which analyzes sound sources based on artificial intelligence (AI), subdivides labeled genres based on BPM, musical instrument composition, and acoustic characteristics, classifies music emotions, and builds a music database by updating labels to build a music database; a user context analysis unit that collects data from a sensor of a user terminal or an external API and infers a user's context by generating meta data related to the user's context through semantic conversion of the collected data; a user emotion analysis unit for inferring the user's emotion by connecting the user's situation with the emotion; a preferred music predictor that predicts preferred music based on the user's situation and preferred music based on the user's emotion, and determines music to be recommended through mapping between the predicted preferred music; and a recommended music providing unit that provides a recommended music list to the user terminal, selects music selected by a user from the music database, and provides the selected music to the user terminal.

Description

Artificial intelligence-based user-customized music recommendation service device {ARTIFICIAL INTELLIGENCE-BASED USER-CUSTOMIZED MUSIC RECOMMENDATION SERVICE DEVICE}

The present invention relates to a music recommendation service technology, and more particularly, to an artificial intelligence-based user-customized music recommendation service device capable of providing a service of recommending a user-customized music suitable for a user's situation and emotion by analyzing a user's situation and music emotion based on artificial intelligence (AI).

With the development of various smart devices, including smartphones, many people enjoy listening to music through streaming (real-time playback) on music platforms. A representative way of listening to music is a playlist. A survey on how to use music streaming in practice (2019 Music Industry White Paper, Korea Creative Content Agency) showed the highest response rate for 'listening to edited/saved playlists' in '15 to 19 years old (60%)' and '20 to 24 years old (59.1%)'.

Music recommendation methods can be largely divided into content-based recommendation and collaborative filtering-based recommendation. Content-based recommendation is a method of recommending similar types of music that a user listens to by utilizing unique properties of music, such as a sound source including songs and lyrics. Collaborative filtering-based recommendation is a method of introducing music that other users with similar tendencies prefer by analyzing users' music consumption patterns.

However, unlike other contents, music consumes a short amount of time and consumption propensity changes several times a day depending on emotions and situations, so the existing recommendation method is not suitable. In other words, content-based recommendation and collaborative filtering-based recommendation cause a 'filter bubble' phenomenon, a phenomenon that traps users like a bubble, making it impossible to enjoy new content and various cultures.

Recently, prior arts that recommend differentiated music according to the user's situation have been proposed, but it is difficult to accurately grasp the user's current situation and still respond to changes in the situation, so the accuracy of music recommendation suitable for the user's situation is low and the user's needs are not met.

Korean Patent Registration No. 10-1943638 (2019.01.23)

An embodiment of the present invention is to provide an artificial intelligence-based user-customized music recommendation service device capable of providing a service that recommends a user-customized music suitable for the situation and emotion by analyzing the user's situation and music emotion based on artificial intelligence (AI).

An embodiment of the present invention is to provide an artificial intelligence-based user-customized music recommendation service device capable of recommending music suitable for a situation by inferring a user's current situation through situation probability inference using sensor data of a mobile device.

An embodiment of the present invention is to provide an artificial intelligence-based user-customized music recommendation service device capable of inferring a user's emotion through music emotion pattern analysis based on a user's music listening history and recommending music suitable for the user's emotion.

Among the embodiments, an artificial intelligence-based user-customized music recommendation service apparatus analyzes a sound source based on artificial intelligence (AI), subdivides a labeled genre based on BPM, musical instrument composition, and acoustic characteristics, classifies music emotion, and renews a label to build a music database; a user context analysis unit that collects data from a sensor of a user terminal or an external API and infers a user's context by generating meta data related to the user's context through semantic conversion of the collected data; a user emotion analysis unit for inferring the user's emotion by connecting the user's situation with the emotion; a preferred music predictor that predicts preferred music based on the user's situation and preferred music based on the user's emotion, and determines music to be recommended through mapping between the predicted preferred music; and a recommended music providing unit that provides a recommended music list to the user terminal, selects music selected by a user from the music database, and provides the selected music to the user terminal.

the music database constructing unit recognizing the spectral aspect of the sound source, extracting an interval between an initially extracted bit and a last extracted bit through bit extraction processing, and calculating the BPM through interpolation processing; Inferring the composition of the instrument used in the sound source through a deep learning instrument classification model; Extracting sound characteristics of the sound source through a Python spectrum extraction library; and generating a BPM, musical instrument configuration, and sound characteristics of the sound source as a dataset, and subdividing and classifying the labeled music genres.

The musical instrument classification model may be a model learned to mathematically determine the use probability of musical instruments by extracting characteristic values from the spectrum of signals for each musical instrument through a Hanning window function and finding features that distinguish musical instruments through a Bayes' classification algorithm.

The music database builder extracts data on the tempo, dynamics, noise, amplitude change, and brightness of the sound source and normalizes the extracted data based on the mean and standard deviation of each extracted data to obtain the emotion as a probability value, or extracts the feature of the sound source through a deep learning two-dimensional CNN music emotion classification model and converts the extracted feature into an emotion vector value to classify the music emotion.

The user context analyzer may pre-set meta data including the user's preferred music, activity state, place and time, which are criteria for determining the user's context, and collect raw data necessary for generating the set meta data from a sensor of the user terminal or an external API.

The user context analyzer may analyze the context of the user through context probability inference using point of interest (POI) clustering, naive Bayes, and hierarchical Bayesian networks on the collected raw data.

The user's emotion analysis unit may obtain a Gaussian mixture distribution for emotion feature vectors of listening songs based on a recent music listening history of the user, and determine an emotion having the highest probability value as the user's emotion.

The disclosed technology may have the following effects. However, it does not mean that a specific embodiment must include all of the following effects or only the following effects, so it should not be understood that the scope of rights of the disclosed technology is limited thereby.

An artificial intelligence-based user-customized music recommendation service apparatus according to an embodiment of the present invention may analyze a user's situation and music emotion based on artificial intelligence (AI) and provide a service of recommending user-customized music suitable for the situation and emotion.

An artificial intelligence-based user-customized music recommendation service device according to an embodiment of the present invention may recommend music suitable for a situation by inferring a user's current situation through situation probability inference using sensor data of a mobile device.

An artificial intelligence-based user-customized music recommendation service apparatus according to an embodiment of the present invention may recommend music suitable for the user's emotion by inferring the user's emotion through pattern analysis of the user's music emotion based on the user's music listening history.

1 is a diagram illustrating a music recommendation service system according to the present invention.
FIG. 2 is a diagram explaining the system configuration of the music recommendation service device of FIG. 1 .
FIG. 3 is a diagram explaining the functional configuration of the music recommendation service device of FIG. 1 .
4 is a flowchart illustrating an AI-based user-customized music recommendation service method according to the present invention.
5 is a diagram illustrating an embodiment of a sound source characteristic-based emotion classification method according to the present invention.
6 is a diagram for explaining another embodiment of a sound source characteristic-based emotion classification method according to the present invention.

Since the description of the present invention is only an embodiment for structural or functional description, the scope of the present invention should not be construed as being limited by the embodiments described in the text. That is, since the embodiment can be changed in various ways and can have various forms, it should be understood that the scope of the present invention includes equivalents capable of realizing the technical idea. In addition, since the object or effect presented in the present invention does not mean that a specific embodiment should include all of them or only such effects, the scope of the present invention should not be construed as being limited thereto.

Meanwhile, the meaning of terms described in this application should be understood as follows.

Terms such as "first" and "second" are used to distinguish one component from another, and the scope of rights should not be limited by these terms. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element.

It should be understood that when an element is referred to as being “connected” to another element, it may be directly connected to the other element, but other elements may exist in the middle. On the other hand, when an element is referred to as being "directly connected" to another element, it should be understood that no intervening elements exist. Meanwhile, other expressions describing the relationship between components, such as “between” and “immediately between” or “adjacent to” and “directly adjacent to” should be interpreted similarly.

Singular expressions should be understood to include plural expressions unless the context clearly dictates otherwise, and terms such as "comprise" or "having" are intended to designate that an embodied feature, number, step, operation, component, part, or combination thereof exists, and it should be understood that the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof is not precluded.

In each step, the identification code (eg, a, b, c, etc.) is used for convenience of explanation, and the identification code does not describe the order of each step, and each step is context-specific. That is, each step may occur in the same order as specified, may be performed substantially simultaneously, or may be performed in the reverse order.

The present invention can be implemented as computer readable code on a computer readable recording medium, and the computer readable recording medium includes all types of recording devices storing data that can be read by a computer system. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices. In addition, the computer-readable recording medium may be distributed to computer systems connected through a network, so that computer-readable codes may be stored and executed in a distributed manner.

All terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs, unless defined otherwise. Terms defined in commonly used dictionaries should be interpreted as consistent with meanings in the context of the related art, and cannot be interpreted as having ideal or excessively formal meanings unless explicitly defined in the present application.

1 is a diagram illustrating a music recommendation service system according to the present invention.

Referring to FIG. 1 , a music recommendation service system 100 may be implemented by including a user terminal 110 , a music recommendation service device 130 and a database 150 .

The user terminal 110 is a device capable of playing music, and may correspond to a computing device capable of using an artificial intelligence-based user-customized music recommendation service, and may be implemented as a smartphone, laptop, or tablet PC, but is not necessarily limited thereto, and may be implemented as various mobile devices including a connected car. Here, the user terminal 110 may include various sensors such as a motion sensor, a location sensor, and an environmental sensor to provide row data necessary for analyzing a user's situation. The user terminal 110 may be connected to the music recommendation service device 130 through a network, and a plurality of user terminals 1100 may be simultaneously connected to the music recommendation service device 130 .

The music recommendation service device 130 may be implemented as a server corresponding to a computer or program capable of collecting user context information through various sensor information of the user terminal 110, understanding the user's current context (Context Awareness), inferring the user's emotions according to the context and listening history, and recommending music suitable for the context and emotion. The music recommendation service device 130 may be connected to the user terminal 110 through a wired network or a wireless network such as Bluetooth, WiFi, or LTE, and may transmit/receive data with the user terminal 110 through the network.

In one embodiment, the music recommendation service device 130 may store data necessary for providing a user-customized music recommendation service based on artificial intelligence in conjunction with the database 150 . Meanwhile, unlike FIG. 1 , the music recommendation service device 130 may be implemented by including the database 150 therein. Also, the music recommendation service device 130 may be implemented by including an API module for providing a music recommendation service. More specifically, the music recommendation service device 130 may provide an external system with an API interface through which information on user profiles, music profiles, and recommended music may be accessed, and the external system may independently access various data managed by the music recommendation service device 130 through the API interface.

The database 150 may correspond to a storage device that stores various pieces of information necessary for the operation of the music recommendation service device 130 . The database 150 may store information about the user's music listening history, and may store information for situational probability inference using sensor data of the user terminal 110. The database 150 is not necessarily limited thereto, and may store information collected or processed in various forms in the process of providing the music recommendation service device 130 customized for the user based on artificial intelligence.

FIG. 2 is a diagram explaining the system configuration of the music recommendation service device of FIG. 1 .

Referring to FIG. 2 , the music recommendation service device 130 may include a processor 210 , a memory 230 , a user input/output unit 250 and a network input/output unit 270 .

The processor 210 may execute a procedure for processing each step in the process of the music recommendation service device 130 operating, manage the memory 230 read or written throughout the process, and schedule synchronization time between volatile memory and non-volatile memory in the memory 230. The processor 210 can control the overall operation of the music recommendation service device 130, and is electrically connected to the memory 230, the user input/output unit 250, and the network input/output unit 270 to control data flow between them. The processor 210 may be implemented as a central processing unit (CPU) of the music recommendation service device 130 .

The memory 230 may include an auxiliary memory implemented as a non-volatile memory such as a solid state disk (SSD) or a hard disk drive (HDD) and used to store all data necessary for the music recommendation service device 130, and may include a main memory implemented as a volatile memory such as a random access memory (RAM).

The user input/output unit 250 may include an environment for receiving user input and an environment for outputting specific information to the user. For example, the user input/output unit 250 may include an input device including an adapter such as a touch pad, a touch screen, an on-screen keyboard, or a pointing device, and an output device including an adapter such as a monitor or touch screen. In one embodiment, the user input/output unit 250 may correspond to a computing device connected through a remote connection, and in such a case, the music recommendation service device 130 may be implemented as an independent server.

The network input/output unit 270 provides a communication environment to be connected to the user terminal 110 through a network, and may include, for example, adapters for communication such as a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), and a value added network (VAN).

FIG. 3 is a diagram explaining the functional configuration of the music recommendation service device of FIG. 1 .

Referring to FIG. 3 , the music recommendation service device 130 may include a music database builder 310, a user situation analyzer 330, a user emotion analyzer 350, a preferred music predictor 370, a recommended music provider 390, and a controller (not shown in FIG. 3).

The music database building unit 310 may build a music database by analyzing sound sources based on artificial intelligence (AI). Currently, in the case of large domestic sound source providers, there are many cases where only simple meta information such as album name, sound source title, singer, and release date and simple genre are labeled, so there are limitations in recommending music. The music database building unit 310 may analyze the characteristics of the sound source and classify genres and emotions through a deep learning model based on the analyzed characteristics of the sound source to build a music database. In one embodiment, the music database builder 310 may analyze the beats per minute (BPM), musical instrument composition, and other acoustic characteristics of the sound source and classify the genre through a deep learning model. Here, BPM is a unit for expressing tempo in music. Audio files can be converted into spectra. Accordingly, the music database builder 310 may grasp the spectrum of the sound source, extract the interval between the first extracted beat and the last extracted beat through bit extraction processing, and automatically calculate the BMP through interpolation processing.

Also, the music database construction unit 310 may infer the composition of musical instruments of the sound source through a deep learning model. In a sound source, the composition of instruments can be an indicator for specifying the atmosphere of a sound source and distinguishing detailed genres. Here, the instrument classification model using deep learning extracts characteristic values from the spectrum of each instrument signal through a Hanning window function and finds features that distinguish musical instruments through Bayes' classification algorithm. The music database construction unit 310 may infer the composition of the musical instrument of the sound source by obtaining data representing the existence (probability) of the instrument used in the sound source as a real number from 0 to 1 through a deep learning instrument classification model.

In addition, the sound source database construction unit 310 may extract acoustic features such as MFCC (Mel Frequency Cepstral Coefficient), Mel-spectrogram, centroid, chromagram, and the like for the sound source using a Python spectrum extraction library. The sound source database builder 310 creates a dataset of the BPM, musical instrument composition, and acoustic characteristics of the sound source, performs deep learning model learning using the created dataset and existing genre classification data as labels, and classifies music genres by subdividing them compared to the existing ones.

Also, the sound source database construction unit 310 may classify music emotions based on sound source characteristics. In one embodiment, the sound source database construction unit 310 extracts data on the tempo, dynamics, noise, amplitude change, and brightness of the sound source, normalizes the extracted data using average and standard deviation values, and obtains each emotion with a probability. In another embodiment, the sound source database construction unit 310 may classify the emotion represented by the sound source by extracting features of the sound source through a deep learning 2D CNN model and converting the extracted features into emotion vectors.

In addition, the sound source database construction unit 310 may analyze a user's situation and emotion in SNS and apply the analysis to music classification. The sound source database construction unit 310 can grasp the user's situation or emotion about music by collecting and analyzing newsfeed, social data, and situation, emotion, and style tag data disclosed by sites provided to public APIs. In one embodiment, the sound source database construction unit 310 collects social media mention data, labels them through machine learning, analyzes the morpheme of data mentioning sound sources and singers, assigns scores according to similarities between sentences and keywords, synonyms, and contexts, and sorts and filters the data through Bayesian network learning to extract and analyze frequencies. At this time, since the reliability of data collected from social media can vary depending on the number of mentions and the influence of the place where it was posted, a social media mention collection data model can be built by considering how much the site has been cited, the influence of subscribers to the site (original article), and the number of subscribers. This can be defined as in Equation 1 below.

[Equation 1]

From here, is the total number of mentions (citations) to a particular site, is the number of link data in which a sound source is mentioned, is the number of traffic from a specific site citation (site), is the number of traffic to a particular site.

Table 1 below shows an example of a social media reference collection data model.

[Table 1]

The user context analyzer 330 may analyze the user context by collecting data from a sensor of the user terminal 110 or an external API. In one embodiment, the user context analysis unit 330 may pre-set meta data that is a criterion for judging the user's context and collect data necessary to generate the set meta data. Here, meta data may include user preference, activity state, place, time, and the like. User preference is the type of music preferred by the user. The user's activity state may include being in a vehicle such as a car (IN_VEHICLE), on a bicycle (ON_BICYLE), walking or running (ON_FOOT), running (RUNNING), standing (STILL), going up to a high place or going down to a low place (TILTING), walking (WALKING), and unknown (UNKNOWN). can be included Places can be divided into user-oriented important places and service-oriented specific places, and may include access frequency, time, location, and type of the place, and may include state changes such as exiting, entering, and already in the place (In). The time may include the time at which the user performs a specific action, the day of the week, every day, a specific time, near a specific time, etc., and a state change (IN) may be included as a state change.

The user situation analyzer 330 may collect set metadata from a sensor of the user terminal 110 or an external API based on row data capable of generating set metadata. Numerous sensors may be embedded in the user terminal 110, and an operation, location, and surrounding environment of the user terminal 110 may be measured through the built-in sensors. For example, the user terminal 110 may include an acceleration sensor, an illuminance sensor, a GPS sensor, a proximity sensor, a microphone, and a sound sensor. In one embodiment, the user situation analyzer 330 may collect only data necessary for generating meta data set from sensors in the user terminal 110, and collects data only when a change occurs in comparison with previously collected data, thereby reducing communication costs and reducing the amount of data to be processed.

The user context analyzer 330 may analyze the context of the user through context probability inference using point of interest (POI) clustering, naive Bayes, and hierarchical Bayesian networks for sensor data of the user terminal 110 . For example, the user situation analysis unit 330 may determine that the user's current situation is the way to exercise on Wednesday afternoon by determining the main movement location (home, office, etc.), weather, movement speed (exercise, drive, etc.), season, and exceptional circumstances through learning of GPS sensor information, WiFi (SSID), and Bluetooth sensor values.

The user emotion analysis unit 350 may infer the user's emotion by connecting the user's situation to the emotion. In one embodiment, the user emotion analysis unit 350 may infer the user's current emotional state by analyzing the user's activity state. Here, the user emotion analyzer 350 may analyze the user's emotion using a pre-learned emotion analysis model having at least two of the user's activity state, place, and time generated by the user situation analyzer 330 as input data.

The user emotion analyzer 350 may infer the user's emotion through an emotion pattern analysis of music recently listened to by the user. In an embodiment, the user's emotion analysis unit 350 obtains a Gaussian mixture density for the emotion feature vectors of the user's listened-to music, and determines the emotion of the GMM having the highest probability value as the user's emotion.

The preferred music predictor 370 may predict preferred music based on the user's situation and preferred music based on the user's emotion, respectively, and determine music to recommend through mapping between the predicted preferred music. In an embodiment, the preferred music predictor 370 may generate a first preferred music list by selecting N pieces of music (where N is a natural number) of genres having sound source characteristics and context tags that match the user's situation from among the pieces of music stored in the music database constructed by the music database builder 310. The preferred music predictor 370 may generate a second preferred music list by selecting N pieces of music classified as music emotion matching the user's emotion among the songs stored in the music database. The preferred music predictor 370 may determine a recommended music list by merging the first preferred music list and the second preferred music list.

The recommended music providing unit 390 may provide a recommended music list to the user terminal 110, select music selected by a user from the recommended music list from a music database, and provide the selected music to the user terminal 110. In an embodiment, the recommended music providing unit 390 may update the recommended music list according to the user's situation or emotional change while the user listens to the recommended music and provide the recommended music list to the user terminal 110 .

The controller (not shown in FIG. 3 ) controls the overall operation of the music recommendation service device 130, and can manage the control flow or data flow between the music database builder 310, the user situation analyzer 330, the user emotion analyzer 350, the preferred music predictor 370, and the recommended music provider 390.

4 is a flowchart illustrating an AI-based user-customized music recommendation service method according to the present invention.

Referring to FIG. 4 , the music recommendation service device 130 may build a music database by analyzing a sound source based on artificial intelligence through a music database construction unit 310 (step S410). The music database building unit 310 classifies sound sources into subdivided genres and emotions through a deep learning model, and analyzes sound source reference data collected from social media such as Facebook, Twitter, community comments, blogs, and news, and labels context, emotion, and style tag data to build a music database. Accordingly, it is possible to recommend subdivided music suitable for the user's needs.

In addition, the music recommendation service device 130 may collect sensor data of the user terminal 110 through the user context analyzer 330 and analyze the current context of the user through context probability inference (step S430). The music recommendation service device 130 may analyze the user's emotion through the user's emotion analysis unit 350 through the user's current situation or emotion pattern analysis of recently listened to music (step S450).

The music recommendation service device 130 may determine a recommended music list by predicting a first preferred music list according to the user's situation and a second preferred music list according to the user's emotion through the preferred music prediction unit 370 and merging them (step S470). The music recommendation service device 130 provides the user terminal 110 with a recommended music list tailored to the user's situation and emotion through the recommended music providing unit 390 (step S490).

5 is a diagram illustrating an embodiment of a sound source characteristic-based emotion classification method according to the present invention.

Referring to FIG. 5 , about 2000 pieces of uniform data can be prepared using 12 labels related to various emotions such as excited, happy, pleased, annoyed, and angry. Then, data on the tempo, dynamics, noise, amplitude change, and brightness of the sound source included in the dataset can be extracted using the Python library, and normal distribution can be performed using the average and standard deviation of each extracted data. Probability values can be derived by substituting the average and standard deviation values for the tempo, dynamics, noise, amplitude change, and brightness of the new sample sound source into the normal distribution curve obtained in advance for each part. Each probability is multiplied by the weight assigned to the X and Y axes to obtain the X (Arousal) and Y (Valence) coordinate values, and by drawing a circle centered on the obtained X and Y coordinate values, the area occupied by the plane previously divided into 12 parts can be obtained to derive the probability of each emotion.

6 is a diagram for explaining another embodiment of a sound source characteristic-based emotion classification method according to the present invention.

Referring to FIG. 6, after converting an audio stereo file into a mono file, the audio data is decomposed into frequency components through a Fourier Transform, and a spectrogram conversion process is performed with a small resolution. Two-dimensional data that can be represented most similar to what the human ear hears can be extracted. After pre-processing the two-dimensional data in the form of a spectrogram, it is applied to a music emotion classification model to extract features and convert the extracted features into emotion vectors to classify the emotions represented by the sound source. Here, the music emotion classification model is a deep learning 2D CNN model.

The present invention can increase the accuracy of emotion classification through a combination of the emotion probability value extracted in FIG. 5 and the emotion vector value extracted in FIG. 6 .

An artificial intelligence-based user-customized music recommendation service device according to an embodiment predicts a user's situation and emotion and recommends a user-customized music playlist to meet the user's needs, and furthermore, not limited to specific popular music, can provide a customized introduction of various music to consumers and provide creators with an opportunity to inform their creative activities.

Although the above has been described with reference to preferred embodiments of the present invention, those skilled in the art can variously modify and change the present invention without departing from the spirit and scope of the present invention described in the claims below. It will be understood that it can be changed.

100: AI-based user-customized music recommendation service system
110: user terminal 130: music recommendation service device
150: database
210: processor 230: memory
250: user input/output unit 270: network input/output unit
310: music database construction unit 330: user situation analysis unit
350: user emotion analysis unit 370: preferred music prediction unit
390: recommended music provision unit

Claims (7)

A music database building unit that analyzes sound sources based on artificial intelligence (AI), subdivides labeled genres based on BPM, musical instrument composition, and acoustic characteristics, classifies music emotions, and renews labels to build a music database;
a user context analysis unit that collects data from a sensor of a user terminal or an external API and infers a user's context by generating meta data related to the user's context through semantic conversion of the collected data;
a user emotion analyzer for inferring the user's emotion by connecting the user's situation to the emotion;
a preferred music predictor that predicts preferred music based on the user's situation and preferred music based on the user's emotion, and determines music to be recommended through mapping between the predicted preferred music; and
and a recommended music providing unit for providing a recommended music list to the user terminal, selecting music selected by a user from the music database, and providing the selected music to the user terminal.
The method of claim 1, wherein the music database building unit
recognizing the spectral aspect of the sound source, extracting an interval between an initially extracted bit and a last extracted bit through bit extraction processing, and calculating the BPM through interpolation processing;
Inferring the composition of the instrument used in the sound source through a deep learning instrument classification model;
Extracting sound characteristics of the sound source through a Python spectrum extraction library; and
An artificial intelligence-based user-customized music recommendation service device comprising the step of subdividing and classifying the labeled music genre by generating a BPM, musical instrument composition, and acoustic characteristics of the sound source as a dataset.
The method of claim 2, wherein the musical instrument classification model
An artificial intelligence-based user-customized music recommendation service device, characterized in that it is a model learned to mathematically determine the probability of using an instrument by extracting characteristic values from the spectrum of each instrument signal through a Hanning window function and finding a feature that distinguishes the sound of an instrument through a Bayes' classification algorithm.
The method of claim 1, wherein the music database building unit
An artificial intelligence-based user-customized music recommendation service device characterized in that it extracts data on the tempo, dynamics, noise, amplitude change, and brightness of the sound source and normalizes them based on the mean and standard deviation of each extracted data to obtain emotion as a probability value or extracts features of the sound source through a deep learning two-dimensional CNN music emotion classification model and converts the extracted features into emotion vector values to classify music emotions.
The method of claim 1, wherein the user situation analysis unit
An artificial intelligence-based user-customized music recommendation service providing device that is characterized in that it pre-sets meta data including the user's preferred music, activity state, place and time, which are criteria for determining the user's situation, and collects raw data necessary for generating the set meta data from a sensor of the user terminal or an external API.
The method of claim 5, wherein the user situation analysis unit
An artificial intelligence-based user-customized music recommendation service providing device based on artificial intelligence, characterized in that the situation analysis of the user is performed through situation probability inference using point of interest (POI) clustering, naive Bayes, and hierarchical Bayesian networks on the collected raw data.
The method of claim 1, wherein the user emotion analysis unit
An artificial intelligence-based user-customized music recommendation service providing device based on artificial intelligence, characterized in that, based on a user's recent music listening history, a Gaussian mixture distribution for emotion feature vectors of listening songs is obtained and an emotion having the highest probability value is determined as the user's emotion.