WO2023167496A1

WO2023167496A1 - Method for composing music by using artificial intelligence

Info

Publication number: WO2023167496A1
Application number: PCT/KR2023/002837
Authority: WO
Inventors: 손다영
Original assignee: 손다영
Priority date: 2022-03-02
Filing date: 2023-03-02
Publication date: 2023-09-07
Also published as: KR102492074B1

Abstract

Disclosed is a method for composing music through artificial intelligence so that the emotional state and daily life of a user can be expressed musically.

Description

How to compose music using artificial intelligence

The present invention relates to a method for composing music through artificial intelligence, and more particularly, to a technology for supporting music composition through artificial intelligence so that a user's emotional state and everyday life can be expressed musically.

Music can be the most effective way to express people's emotions or current situations. In particular, it is possible to express one's feelings or situations through writing, but there are limitations in expressing subtle feelings or situations with vocabulary, so it is common that the feelings or situations are not accurately conveyed to the reader so that they can empathize. On the other hand, musical expression can reflect various and fine elements that cannot be expressed lexically, so the degree of empathy of the listener can be further improved.

However, since ordinary people do not have music composition knowledge, there is a problem in that they cannot express their feelings or situations through musical elements. As a result, various music composing applications that allow users to compose more easily have been proposed.

Recently, with the development of artificial intelligence technology, music composition technology through artificial intelligence has been proposed. However, since these composition techniques simply generate composition results according to a predetermined pattern, there is a problem in that the user's emotion or expression of the situation is not faithfully reflected.

Prior art documents in the technical field to which the present invention belongs include Korean Patent Registration No. 10-1886534 and the like.

The present invention has been made to solve the problems of the prior art as described above, and is intended to provide a method of supporting music composition so that a user's emotional state and daily life can be expressed musically.

In particular, in the case of expressing one's feelings or situations in writing, I would like to present a technique that can overcome the limitation that the feelings or situations cannot be conveyed in a sympathetic way to the reader through music.

Furthermore, in the case of conventional technologies supporting composition, a problem in which a user's emotion or situational expression is not faithfully reflected is intended to be solved simply because a composition result is generated according to a predetermined pattern.

The object of the present invention is not limited to the above, and other objects and advantages of the present invention not mentioned can be understood by the following description.

An embodiment of a method for composing music using artificial intelligence according to the present invention includes a basic element extraction step of extracting a base rhythm according to a user's expression situation through artificial intelligence; Auxiliary element extraction step of extracting any one or more of a plurality of melodies or a plurality of harmonies corresponding to the expression situation through the artificial intelligence; an auxiliary element selection step in which one or more of a plurality of melodies or a plurality of harmonies extracted from the user is selected; and a composition generating step of generating a music composition by combining one or more melodies or harmonies selected based on the bass rhythm extracted through the artificial intelligence.

As an example, the basic element extraction step may include receiving text or humming from a user and determining the user's expression situation based on this; and extracting a bass rhythm corresponding to the user's expression situation from a database holding bass rhythms for each expression situation, wherein the extracting of auxiliary elements comprises a plurality of melodies or extracting a plurality of harmonies; and determining a plurality of melodies or a plurality of harmonies to be recommended based on the extracted bass rhythm.

As an example, the generating of the composition may include combining one or more melodies or one or more harmonies selected with the bass rhythm for each measure; and combining and arranging a plurality of measures to create a musical composition.

Preferably, it may further include a context expression information generation step of providing a list of one or more instruments suitable for the musical composition to a user, selecting one or more instruments, and generating context expression information for playing the music composition with the selected instrument. can

Furthermore, in the generating of the situation expression information, an emotion graph or an emotion image corresponding to the music composition may be generated and situation expression information including the emotional graph or emotion image may be generated.

Further, the method may further include a step of storing the situation expression information by organizing the situation expression information by date corresponding to the user's expression situation and accumulatively storing the situation expression information or linking the situation expression information to the user's social networking service (SNS).

According to the present invention, it is possible to support music composition so that the user's emotional state and daily life can be expressed musically.

In particular, in the case of expressing one's feelings or situations in writing, music can overcome the limitation that the feelings or situations cannot be conveyed in a sympathetic way to the reader.

Furthermore, various composition results can be generated to suit the emotion or situation at the time, so that the user's emotion or expression of the situation can be faithfully reflected.

In addition, by supporting the user to leave the emotions felt in a specific situation or period in the form of a diary of musical results, the user can feel the emotions felt at that time more effectively by replaying the musical results made by the user later.

In addition, by linking the musical results expressing the user's emotions to a social networking service (SNS), emotions that cannot be expressed in words or writing can be effectively conveyed to the other party.

The effects of the present invention are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 shows one embodiment of a music composition system according to the present invention.

2 shows a configuration diagram of an embodiment of a composition support service device of a music composition system according to the present invention.

3 shows a block diagram of an embodiment of a database of a music composition system according to the present invention.

Figure 4 shows a flow diagram of one embodiment of a music composition method according to the present invention.

Figure 5 shows a flow diagram of another embodiment of a music composition method according to the present invention.

6 shows a flow diagram of another embodiment of a music composition method according to the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings, but the present invention is not limited or limited by the embodiments.

In order to explain the present invention and the operational advantages of the present invention and the objects achieved by the practice of the present invention, the following describes a preferred embodiment of the present invention and references it.

First, the terms used in this application are used only to describe specific embodiments, and are not intended to limit the present invention, and singular expressions may include plural expressions unless the context clearly dictates otherwise. In addition, in this application, terms such as "comprise" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other It should be understood that the presence or addition of features, numbers, steps, operations, components, parts, or combinations thereof is not precluded.

In describing the present invention, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present invention, the detailed description will be omitted.

The present invention proposes a technology for supporting music composition through artificial intelligence so that a user's emotional state and daily life can be expressed musically.

A music composing system according to the present invention may include a user terminal 10 , a composition support service device 100 , a database 200 , and the like.

The user terminal 10 may connect to the composition support service device 100 and provide information about the user's emotional state and daily life to the composition support service device 100 and receive a music sub-grain based thereon.

As the user terminal 10, various communication terminals that can access the composition support service device 100 and allow the user to input their own expression state may be applied. As an example, a generally popular terminal such as a smart phone 10a, a laptop computer 10b, and a PC 10c may be applied.

The user may transmit various situations, such as his/her emotional state and daily life, to the composition support service device 100 through the user terminal 10 . As an example, the user may input text such as a word or sentence corresponding to a situation to be expressed or humming. Alternatively, you can also input an image such as a photo or video containing your situation.

Also, the user may select or process various pieces of information provided from the composition support service device 100 through the user terminal 10 .

The composition support service device 100 receives various situational information such as the user's emotional state and daily life through the user terminal 10 and analyzes it through artificial intelligence to obtain musical basic elements and musical auxiliary elements suitable for the user's expression situation. can be extracted and combined or processed to create a musical composition.

Here, the musical elements can include rhythm, melody, and harmony. Rhythm, the flow of beats, is set as the basic element, and the melody, melody, and harmony added to it are auxiliary elements. can be set to In addition, the composition support service device 100 may extract, process, and combine these musical elements to generate a music composition corresponding to the user's expression situation.

The composition support service device 100 may obtain various musical element information about rhythm, melody, harmony, and the like through the Internet network 30 .

For example, the composition support service device 100 collects various music information and interpretation information present on the Internet network 30 and provides rhythm, melody, harmony, etc. through artificial intelligence learning. It is possible to distinguish which situation expression can correspond to. In addition, the composition support service device 100 may classify music element information about rhythm, melody, harmony, and the like for each situation expression and store it in the database 200 .

Furthermore, the composition support service device 100 may hold various instrument information and generate situational expression information in which the created music composition is played with the instrument.

In addition to this, the composition support service device 100 may generate situation expression information by generating an emotion graph or an emotion image corresponding to a music composition and matching them.

As an example, the composition support service device 100 may generate an emotion graph that changes according to the performance of a music composition or an image corresponding to an emotional state, or a photograph input by a user corresponding to a performance state of a music composition. Situational expression information may be generated by matching images or videos.

In addition, the composition support service device 100 may organize and store the generated situational expression information in the database 200 . The composition support service device 100 may store situation expression information for each date corresponding to the user's diary format. Alternatively, the composition support service device 100 may classify the situation expression information according to situations felt by the user, classify the situation expression information according to categories, and store the situation expression information.

Situational expression information can be classified and stored in various ways. It can be classified and stored in categories by emotion, such as emotions felt by the user, or classified and stored in categories by period or date. It may be classified into categories for each subject and stored.

The user's situation expression information stored in this way may be later provided to the user terminal 10 by extracting the situation expression information stored in the database 200 from the composition support service device 100 according to the user's request.

Taking a step further, the composition support service device 100 may provide corresponding situation expression information to the social networking service 50 in association with the social networking service (SNS) 50 . For example, when a user transmits a message to a specific target, corresponding situation expression information may be transmitted together, and the user may upload the situation expression information as status information on his or her social network.

In relation to the composition support service device 100 of the music composition system according to the present invention, FIG. 2 will be described in more detail with reference to a configuration diagram of an embodiment of the composition support service device of the music composition system according to the present invention. .

The composition support service apparatus 100 may include an expression situation recognition unit 110, a music element extraction unit 120, a composition generation unit 130, a situation expression information generation unit 140, and the like.

The expression situation recognition unit 110 may receive information about various users' emotional states, daily life, etc. through the user terminal 10 and analyze them to recognize a situation that the user wants to express.

For example, in the case of information in the form of text, it is possible to recognize a situation to be expressed according to the user's emotional state by recognizing and analyzing each word and sentence through artificial intelligence. In addition, in the case of information in the form of images such as photos and videos, the image can be analyzed through artificial intelligence to recognize the situation to be expressed according to the user's emotional state.

The music element extractor 120 may extract a musical element corresponding to the analyzed expression situation of the user. As an example, a bass rhythm according to the user's expression situation may be extracted, and various melodies and harmonies corresponding to the user's expression situation may be extracted based on the bass rhythm.

The music element extractor 120 may extract music element information corresponding to the corresponding expression situation from the music element information previously stored in the database 200, or search the Internet network 30 to respond to the corresponding expression situation. You can also extract the music elements that are.

The composition generator 130 may generate a music composition by combining and processing the musical elements extracted by the music element extractor 120 . Preferably, various bass rhythms suitable for the user's expression situation are presented to the user so that the user can select the rhythm desired by the user, and various melodies and harmonies suitable for the user's expression situation are presented to the user to select the melody and harmony desired by the user. may be selected. Of course, for users who lack musical knowledge or sense, the composition generator 130 creates music compositions by combining and processing various melodies and harmonies to the bass rhythm suitable for the user's expression situation through artificial intelligence without the user's selection. You may.

Furthermore, the composition generator 130 provides the music composition as a sheet music to the user terminal 10 so that the user can check it, and when the user directly selects or processes the desired melody and harmony, dissonance, pitch, tempo It is also possible to evaluate the appropriateness of musical elements such as etc. and provide feedback on them.

In addition, the composition generator 130 may receive lyrics for a corresponding music composition through the user terminal 10 and match them to the music composition.

The context expression information generating unit 140 may generate context expression information suitable for the user's expression situation by matching additional information to the music composition generated by the composition generator 130 .

As an example, the context expression information generating unit 140 may select an instrument suitable for playing a musical composition from among various instruments or provide a list of instruments to the user to generate context expression information in which the musical composition is played with the user's selected instrument. there is.

As another example, the situation expression information generation unit 140 may generate situation expression information by generating an emotion graph or an emotion image corresponding to a music composition and matching it to the music composition. For example, situational expression information in which a picture or video of a user is matched together with a performance of a musical composition may be generated.

In addition, the situation expression information generation unit 140 may arrange and store the generated situation expression information by date corresponding to the expression situation of the user in the database 200 . As an example, the corresponding situation expression information may be maintained and stored in the database 200 in the form of a diary, or the corresponding situation expression information may be classified and stored in the database 200 for each category of situation expression.

In this way, the context expression information stored in the database 200 may be provided to the user terminal 10 by extracting the context expression information generation unit 140 upon a user's request later.

Furthermore, the context expression information generator 140 may provide context expression information in association with the social networking service 50 .

The database 200 may include a music element information storage unit 210, a situation expression information storage unit 250, and the like.

The music element information storage unit 210 may store music element information about rhythm, melody, harmony, and the like.

As an example, various rhythm information can be classified and stored according to various emotions or situations as basic element information, and various melodies and harmonies can be classified and stored according to various emotions or situations as auxiliary element information.

The situation expression information storage unit 250 may classify and store music compositions or situation expression information. The situation expression information storage unit 250 may classify and store only music compositions generated by the composition support service device 100 by emotion, situation, and date, or may classify and store situation expression information in which various information is added to music compositions. It can also be classified and stored by date, situation, and date.

In addition, the present invention proposes a method of supporting music composition using the music composition system according to the present invention described above.

Hereinafter, a music self-composing method according to the present invention will be described with reference to the embodiment of the music composition system according to the present invention described above.

The composition support service device 100 may receive information about a situation to be expressed by the user through the user terminal 10 (S110). The user's expression situation information may be text information such as words and sentences, humming information, or image information such as photos and videos.

The composition support service device 100 may recognize a situation to be expressed by the user based on the provided information on the expression situation of the user (S120). For example, artificial intelligence can determine the emotional state or situational state and recognize the situation the user wants to express through word and sentence analysis for text information, rhythm and melody analysis for humming information, or video analysis for photos or videos. can

The composition support service device 100 may extract bass rhythm information as a basic musical element according to a situation that the user wants to express (S130). The composition support service device 100 may extract bass rhythm information from the database 200 for each emotion or situation, or may extract bass rhythm information from the Internet network 30 .

In addition, the composition support service device 100 may extract a plurality of pieces of melody information or a plurality of pieces of harmony information corresponding to a situation that the user wants to express (S140). The composition support service device 100 may extract various melody rhythm information or harmony information from the database 200 for each emotion or situation through artificial intelligence, or extract various melody rhythm information or harmony information from the Internet network 30. may be

As an example, the extracted melody may be in the form of a minimalism patterned melody, and romantic music or modern musical harmony composed only of basic triads that are not too classical may be extracted by reflecting the latest trend.

Preferably, the composition support service apparatus 100 may recommend melodies and harmonies suitable for the user's expression situation to the user by determining a plurality of melodies or a plurality of harmonies to be recommended to the user based on the basic bass rhythm.

In addition, the composition support service device 100 may present the extracted plurality of melodies to the user through the user terminal 10 and support the user to set the tempo, notes, number of bars, instruments, and the number of melodies. In addition, the composition support service apparatus 100 may support a user to select a melody most similar to his or her feelings among a plurality of melodies.

The composition support service device 100 may create a music composition by combining and processing (S150) one or more melodies or harmonies selected based on the base rhythm extracted through artificial intelligence (S160).

As an example, the composition support service device 100 may generate a music composition by combining one or more melodies or one or more harmonies selected in a bass rhythm through artificial intelligence for each measure and combining and arranging a plurality of measures.

In addition, the composition support service device 100 provides a musical composition as a sheet music to the user terminal 10 so that the user can check it, and when the user directly selects or processes a desired melody and harmony, dissonance, pitch, tempo It is also possible to evaluate the appropriateness of musical elements such as etc. and provide feedback on them.

In addition, the composition support service device 100 may receive lyrics for a corresponding music composition through the user terminal 10 and match them to the music composition.

The composition support service device 100 may provide the user with a list of instruments suitable for the generated music composition through the user terminal 10 and may generate situational expression information in which the music composition is played with the user's selected instrument.

The musical composition and situation expression information generated in this way may be provided to the user through the user terminal 10 or the like.

Furthermore, in the present invention, situation expression information may be generated by reflecting various states of a user in addition to a music composition. let's take a look

The composition support service device 100 creates a music composition through the above-described embodiment (S210), additionally creates an emotion graph or emotion image corresponding to the music composition (S220), and converts the emotion graph or emotion image to the music composition. Situational expression information reflected in the performance of water may be generated (S230).

For example, an emotion graph that changes in various ways according to the playing state of the music composition or various images expressing emotions may be created and inserted along with the performance of the music composition.

In addition, situational expression information may be created by inserting a user's photo, video, or text so that it can be displayed together with the performance of the music composition.

In addition, the composition support service device 100 may accumulate and store situational expression information in the form of a diary in the database 200 . At this time, a date or a user's comment may be matched with the situation expression information and stored in the database 200 .

Alternatively, the composition support service device 100 may classify categories for each corresponding emotion or situation and store situational expression information in the database 200 .

Furthermore, the composition support service device 100 may link the generated situational expression information or the situational expression information stored in the database 200 with the social networking service 50 (S250) and reflect them on the user's social network.

The user's situation expression information stored in this way may be provided to the user terminal 10 or the like according to the user's request in the future. In this regard, FIG. show

The composition support service device 100 organizes and stores the situation expression information reflecting the user's emotion or situation, and provides a category-by-category list of situation expression information upon a user's request through the user terminal 10 (S310). can For example, a list may be provided by emotion category such as joy, anger, sadness, etc., a list may be provided by category by period or date, or a category list by subject who feels emotion or situation may be provided.

The user may select a desired situation expression from the list provided to the user terminal 10 (S320), and the composition support service device 100 corresponds to the user's selection and corresponding situation expression information in the database 200. Can be selectively extracted (S330).

In addition, the composition support service device 100 may provide the extracted context expression information to the user terminal 10 to reproduce the context expression information (S340).

In the present invention, since the situation expression information reflecting the corresponding emotion or situation can be reproduced, the user can feel the emotion or situation felt at that time again as it is by displaying the text, photo, video, etc. together with the music.

The above description is merely an example of the technical idea of the present invention, and various modifications and variations can be made to those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments described in the present invention are not intended to limit the technical spirit of the present invention, but to explain, and the technical spirit of the present invention is not limited by these embodiments. The protection scope of the present invention should be construed by the following claims, and all technical ideas within the equivalent range should be construed as being included in the scope of the present invention.

* Description of code *

10, 10a, 10b, 10c: user terminal,

100: composition support service device,

110: expression situation recognition unit,

120: music element extraction unit,

130: composition generation unit,

140: situation expression information generation unit,

200: database,

210: music element information storage unit,

250: situation expression information storage unit.

Claims

A basic element extraction step of extracting a base rhythm according to a user's expression situation through artificial intelligence;

Auxiliary element extraction step of extracting any one or more of a plurality of melodies or a plurality of harmonies corresponding to the expression situation through the artificial intelligence;

an auxiliary element selection step in which one or more of a plurality of melodies or a plurality of harmonies extracted from the user is selected; and

and a composition generating step of generating a music composition by combining one or more melodies or harmonies selected based on the bass rhythm extracted through the artificial intelligence.
According to claim 1,

The basic element extraction step,

Receiving text or humming from a user and determining an expression situation of the user based on the input; and

Extracting a bass rhythm corresponding to a user's expression situation from a database holding bass rhythms for each expression situation;

The auxiliary element extraction step,

extracting a plurality of melodies or a plurality of harmonies corresponding to expression situations from music held in the database; and

A music composition method using artificial intelligence comprising the step of determining a plurality of melodies or a plurality of harmonies to be recommended based on the extracted bass rhythm.
According to claim 1,

The step of generating the composition,

combining one or more melodies or one or more harmonies selected with the bass rhythm for each measure; and

A music composition method using artificial intelligence, comprising the step of generating a music composition by combining and arranging a plurality of measures.
According to claim 1,

Further comprising a situation expression information generation step of providing a list of one or more instruments suitable for the music composition to a user, receiving a selection of one or more instruments, and generating situation expression information for playing the music composition with the selected instrument. A method of composing music using artificial intelligence.
According to claim 4,

In the step of generating the situation expression information,

A music composition method using artificial intelligence, characterized in that for generating an emotion graph or an emotion image corresponding to the music composition and generating situation expression information including the emotional graph or emotion image.
According to claim 5,

The method of composing music using artificial intelligence, further comprising a step of storing the situation expression information by organizing and accumulatively storing the situation expression information by date corresponding to the user's expression situation, or linking the situation expression information to the user's social networking service (SNS). .