KR102472972B1 - Apparatus and method for mixing music sources based on artificial intelligence - Google Patents

Abstract

According to an embodiment of the present invention, in a method for generating a sound source based on AI, performed by a user terminal, the method comprising: (a) receiving voice information from a user; (b) selecting a comparison singer having a predetermined similarity with voice information; And (c) generating a user sound source by mixing and mastering voice information based on a pre-stored sound source adjustment value mapped to a comparison singer, wherein the voice information is a voice or song generated by the user's own utterance It is recorded information, and the sound source adjustment value includes a mixing adjustment value and a mastering adjustment value.

Description

AI-based sound source mixing device and method {APPARATUS AND METHOD FOR MIXING MUSIC SOURCES BASED ON ARTIFICIAL INTELLIGENCE}

The present invention relates to an apparatus and method for calculating a set value for mixing a sound source to suit a user's voice based on AI and mixing the sound source based on the set value.

Conventionally, platforms for sharing various videos created by individuals or companies, including YouTube, have been widely used. As these platforms develop, content creators are sharing videos with others by uploading them through the platform.

In addition, an increasing number of content creators are uploading not only videos, but also individual cover songs through video platforms, which were previously the exclusive property of singers.

In this case, the cover song refers to a song performed or sung by another person in the field of music, which was released by a specific person.

A typical content sharing platform compulsorily exposes an advertisement mapped to the content for a certain period of time before or during playback of the uploaded content. Through this process, a certain amount of revenue is paid to the content creator according to the number of views of the uploaded content and the number and exposure time of the advertisement exposed together with the reproduction time of the content.

In the case of the cover song described above, copyright issues may arise between the original authors. For example, under the Copyright Act, singing a song using an original music MR may constitute an infringement of copying rights, and arranging may constitute an infringement of a secondary copyrighted work. Furthermore, the act of uploading these videos to a content sharing platform may constitute an infringement of transmission rights.

At this time, in the case of YouTube, since 2008, the content verification technology (Content ID, hereinafter referred to as CID) has been introduced to protect the copyright of the author. YouTube's content verification technology is a system that automatically compares and verifies whether the content uploaded by the content creator matches the original content reference file of the copyright holder based on a database composed of original content reference files provided by the copyright holder.

When YouTube searches for uploaded content that matches the work of the copyright holder through this content verification technology, the copyright owner will be reported. At this time, the copyright owner can select whether to block, track, or make a profit model for the corresponding content. In other words, if the uploaded content is found to be in violation of copyright, a warning is issued according to the setting of the copyright owner, access to users is automatically blocked, the content is removed, or the copyright holder can take the revenue generated from playing the content. .

Meanwhile, YouTube is also using a bot to manage copyright. In the bot, copyright holders apply for copyright on a video sharing platform, and if other content creators other than the copyright holder upload content that matches or is similar to the copyright holder's requested work, the management bot filters it and gives the copyright holder the right to the work, and the corresponding work All of the advertising revenue from is allocated to the copyright owner.

According to statistics in 2019, it is estimated that the music usage fee paid by YouTube to the author or the music industry is a whopping 3.55 trillion won. Among them, a large portion of the advertising revenue attached to videos uploaded by famous YouTubers singing cover dances or cover songs takes up a large portion.

In the end, it can be said that there is a great need to protect the copyright of the original copyright holder in the case of simply using music or sound sources, but a new business model is needed to distribute the revenue generated from content produced by oneself, such as cover songs, with existing authors. to be.

However, there is a limit for a content creator who wants to upload content such as a cover song to a video sharing platform to personally contact the copyright holder. Even if contact is made between the content creator and the copyright holder, it is difficult to obtain permission to use the copyrighted work, and it is not easy to prepare completely from the payment of royalties to the use approval documents.

In order to solve the above problems, an object of the present invention is to implement a comprehensive platform in which users can easily create their own sound sources based on AI and generate revenue for them.

Specifically, by analyzing the recorded voice from the user, an AI mixing value suitable for the user is derived, and a unique creative sound source is created by mixing the user's voice and MR accompaniment according to the derived mixing value.

In addition, the purpose is to implement a comprehensive platform where users can not only create creative sound sources, but also disclose them to the public and generate profits.

In the method for generating a sound source based on AI, performed by a user terminal, according to an embodiment of the present invention for achieving the above technical problem, (a) receiving voice information from a user; (b) selecting a comparison singer having a predetermined similarity with voice information; And (c) generating a user sound source by mixing and mastering voice information based on a pre-stored sound source adjustment value mapped to a comparison singer, wherein the voice information is a voice or song generated by the user's own utterance It is recorded information, and the sound source adjustment value may include a mixing adjustment value and a mastering adjustment value.

In addition, before step (a), a preset MR sound source may be provided to the user, and voice information may be received by inducing the user to sing along with the song through the MR sound source.

In step (b), the voice information is compared with the previously stored singer's voice information of at least one comparison singer, and a similarity with the singer's voice information can be calculated based on the genre, timbre, and pitch of the voice information.

In addition, when preset genre information is provided for each genre of the singer's voice information, genre information associated with the user is digitized based on preferred genre information previously received from the user or genre information corresponding to the voice information, and the singer's voice information A comparison singer can be selected by quantifying and comparing each genre information given to .

In addition, the timbre of the voice information is digitized and compared with the timbre value included in the singer's voice information of the comparison singer, and the comparison singer having the lowest difference is automatically selected.

In addition, the height of the voice information is digitized and compared with the height value included in the voice information of the singer of the comparison singer, and the comparison singer having the lowest difference is automatically selected.

In addition, categories of voice information may be classified based on the comparison singer selection result, and a sound source adjustment value corresponding to the category may be set as the user's sound source adjustment value.

In addition, the sound source adjustment value is composed of a mixing condition value and a mastering adjustment value, and the sound source adjustment value may be mapped and stored differently according to the original song of the comparison singer.

In addition, as for the sound source adjustment value, at least one sound source adjustment value may be assigned to one original song of the comparison singer, and two or more types of sound source adjustment values may be divided and set according to at least one criterion of the beat or melody of the original song. .

In addition, when the genre, timbre and pitch for modifying the voice information are received from the user after step (d), the voice information can be modified by generating a custom sound source adjustment value based on the received genre, timbre and pitch.

In addition, based on the custom sound source adjustment value, a pre-stored sound source adjustment value may be corrected in conjunction with the singer's voice information of the comparison singer.

In addition, after step (d), a machine learning model is constructed that takes as input values data on genres, pitches, and timbres of a plurality of sound sources applied to a plurality of voice information, and mixing condition values and mastering conditions for generating user sound sources By setting the value as an output value, it is possible to increase the accuracy of the mixing condition value and mastering condition value in the process of generating the user sound source through the voice information of the new user.

In addition, after step (d), the user sound source may be uploaded to the central server, and the user sound source may be streamed and provided to other user terminals.

In addition, in the device for generating a sound source based on AI, a memory in which a program for generating a sound source based on AI is stored and a processor for generating a sound source based on AI by executing a program stored in the memory, the processor provides voice information from a user Receive, select a comparison singer having a predetermined similarity with the voice information, map the comparison singer and mix and master the voice information based on the pre-stored sound source adjustment value to create a user sound source, and the voice information is A voice or a song generated by direct utterance is recorded information, and the sound source adjustment value may be a device including a mixing adjustment value and a mastering adjustment value.

In addition, it may be a computer readable storage medium on which a program for performing the method of generating a sound source based on AI according to claim 1 is recorded.

According to an embodiment of the present invention, based on AI, a comprehensive platform that allows users to easily create their own sound sources and generate revenue for them is implemented.

Specifically, by analyzing the recorded voice from the user, an AI mixing value suitable for the user is derived, and a unique creative sound source is created by mixing the user's voice and MR accompaniment according to the derived mixing value.

In addition, through the comprehensive platform, users will have the opportunity to provide their created sound sources to many listeners, and distributors will have a foothold to generate revenue through created sound sources or foster excellent talent through contests. get it

1 is a diagram showing the configuration of a system for realizing a platform for generating sound sources and generating profits according to an embodiment of the present invention.
2 is a diagram showing the configuration of a central server 100 providing a platform according to an embodiment of the present invention.
3 is an operational flowchart illustrating a process of generating a sound source and performing a contest according to an embodiment of the present invention.
4 is an operation flowchart illustrating a process of producing a user sound source according to an embodiment of the present invention.
5 is an operation flowchart illustrating a process of mixing a sound source based on AI according to an embodiment of the present invention.
6 is an operational flowchart illustrating a process of mixing and contesting a sound source through an interface provided by a platform according to an embodiment of the present invention.
7 is an operational flowchart illustrating a process of generating revenue through a sound source and distributing revenue according to an embodiment of the present invention.
8 to 13 are diagrams illustrating examples of platform interfaces according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail so that those skilled in the art can easily practice the present invention with reference to the accompanying drawings. However, the present invention may be embodied in many different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part is said to be "connected" to another part, this includes not only the case where it is "directly connected" but also the case where it is "electrically connected" with another element interposed therebetween. . In addition, when a part "includes" a certain component, this means that it may further include other components, not excluding other components, unless otherwise stated, and one or more other characteristics. However, it should be understood that it does not preclude the possibility of existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

The following examples are detailed descriptions for better understanding of the present invention, and do not limit the scope of the present invention. Therefore, inventions of the same scope that perform the same functions as the present invention will also fall within the scope of the present invention.

In the platform to be implemented in the present invention, when a user selects an MR sound source and sings for it, it is combined through an AI algorithm or a custom sound source adjustment value is applied and combined to create a unique user sound source.

In addition, the platform to be implemented in the present invention not only generates user sound sources, but also plays a role in competing with other user sound sources or generating and distributing profits by disclosing them to the outside.

In the following specification, a sound source refers to an electronic file including at least one of sound and image.

In addition, in this specification, an MR sound source (music recorded, or instrumental track) refers to a sound file excluding only voices from sound sources of specific singers. Usually, it consists of only sound, but depending on the embodiment, visual information may be further included. At this time, the MR sound source may be a creative song or a ready-made song, and in the case of a ready-made song, it corresponds to a sound source for which secondary creation is possible with the permission of the original author.

In addition, in this specification, voice information is voice information generated by a user using a service through a platform through his/her terminal. That is, it corresponds to information recorded through a terminal of a song corresponding to the MR sound source (this is song lyrics). In this case, the voice information may be generated in the form of a moving picture, which is not limited to sound only, according to embodiments.

In addition, the technology disclosed in this specification combines MR sound sources and voice information. At this time, a user sound source is created by mixing the MR sound source and voice information, mixing conditions for the user sound source are reset, the user sound source is adjusted, and mastering is performed. That is, in this specification, "combination of MR sound source and voice information" includes mixing the MR sound source and voice information and performing mastering work.

1 is a diagram showing the configuration of a system for realizing a platform for generating sound sources and generating profits according to an embodiment of the present invention.

Referring to FIG. 1 , the system may include a central server 100 , a plurality of user terminals 200 and a distributor server 300 . At this time, although not shown in the drawing, each device may be interconnected in a wired or wireless manner through a communication network.

According to an embodiment of the present invention, the central server 100 stores the user information received from the user terminal 200, and then, when the user terminal 200 receives the user sound source generated through the sound source production interface, first It is stored after being matched with the stored user information.

Thereafter, the central server 100 performs a contest through user sound sources received from a plurality of user terminals 200 and provides the winning user sound source to the distributor server 300 to generate revenue.

At this time, the user terminal 200 generates a user sound source by combining the MR sound source received from the central server 100 and the voice information recorded by the user through a sound source production interface.

At this time, the sound source production interface reproduces the MR sound source selected and received by the user terminal 200 from the central server 100, receives the user's voice information, and then mixes and An interface that provides mastering functions.

Meanwhile, the platform may provide an AI mixing algorithm. Specifically, the AI mixing algorithm combines the MR sound source and voice according to pre-stored sound source adjustment values (which may include mixing adjustment values and mastering adjustment values). Therefore, the user can omit the mixing and mastering work through the sound source production interface described above.

In addition, the contest refers to a process of determining a ranking by receiving preferences from listeners who have listened to a plurality of user sound sources for a predetermined period of time. At this time, when a user sound source of a specific user wins the contest, the distributor server 300 provides the winning user sound source through the central server 100 . In addition, a platform server such as YouTube can also provide user sound sources. Thereafter, the service subject of the central server 100 and the producer of the user terminal 200 have a certain percentage of the revenue that the distributor server 300 obtains through the user sound source.

In this case, the platform application for the user terminal 200 to use the service may be an application built into the user terminal 200 or an application downloaded from an application distribution server and installed in the user terminal 200 .

Meanwhile, the user terminal 200 refers to a communication terminal capable of using a terminal application in a wired/wireless communication environment. Here, the user terminal 200 may be a user's portable terminal. In FIG. 1, the user terminal 200 is shown as a smart phone, which is a kind of portable terminal, but the spirit of the present invention is not limited thereto, and as described above, there is no limitation for terminals capable of loading terminal applications. can be borrowed

More specifically, user terminal 200 may include any type of handheld computing device (eg, PDA, email client, etc.), cell phone, or other type of computing or communication platform. However, the present invention is not limited thereto.

On the other hand, the communication network is a communication network that provides an access path so that the user terminal 200 can transmit and receive data after accessing the central server 100 or the distributor server 300 (this can be borrowed in the opposite case). it means. Communication networks include, for example, wired networks such as LANs (Local Area Networks), WANs (Wide Area Networks), MANs (Metropolitan Area Networks), and ISDNs (Integrated Service Digital Networks), wireless LANs, wireless networks such as CDMA, Bluetooth, and satellite communication. However, the scope of the present invention is not limited thereto.

2 is a diagram showing the configuration of a central server 100 providing a platform according to an embodiment of the present invention.

Referring to FIG. 2 , a central server 100 according to an embodiment of the present invention includes a communication module 101, a memory 102, a processor 103, and a database 104.

In detail, the communication module 101 provides a communication interface necessary to provide a transmission/reception signal between the central server 100, the user terminal 200, and the distributor server 300 in the form of packet data in conjunction with a communication network. Furthermore, the communication module 101 may serve to receive a data request from the user terminal 200 and transmit data as a response thereto.

Here, the communication module 101 may be a device including hardware and software necessary for transmitting and receiving a signal such as a control signal or a data signal with another network device through a wired or wireless connection.

The memory 102 stores programs for operating the platform. In addition, it performs a function of temporarily or permanently storing data processed by the processor 103. Here, the memory 102 may include magnetic storage media or flash storage media, but the scope of the present invention is not limited thereto.

The processor 103 controls the entire process of operating the platform as a kind of central processing unit. Each step performed by the processor 103 will be described later with reference to FIGS. 3 to 7 .

Here, the processor 103 may include all types of devices capable of processing data, such as a processor. Here, a 'processor' may refer to a data processing device embedded in hardware having a physically structured circuit to perform functions expressed by codes or instructions included in a program, for example. As an example of such a data processing device built into hardware, a microprocessor, a central processing unit (CPU), a processor core, a multiprocessor, an application-specific integrated (ASIC) circuit), field programmable gate array (FPGA), etc., but the scope of the present invention is not limited thereto.

The database 104 stores MR sound sources to be provided to the user terminal 200 (which may also include general sound sources according to circumstances), voice information received from the user terminal 200, and data on the user sound sources. In addition, sound source adjustment values and AI algorithms corresponding to each MR sound source may be stored.

In this case, the sound source adjustment value may correspond to a mixing adjustment value required in the process of combining the MR sound source and the user's voice, for example, a sync value, an EQ value, and the like.

Although not shown in FIG. 2, some of the MR sound source, voice information, user sound source, and data on sound source adjustment values may be stored in a database (not shown) physically or conceptually separated from the database 104. That is, as an optional embodiment, the database 104 may be omitted, and necessary data may be received and used from the database provided in the user terminal 200 .

3 is an operational flowchart illustrating a process of generating a sound source and performing a contest according to an embodiment of the present invention.

Referring to FIG. 3 , the central server 100 stores user information and user sound sources received from the user terminal 200 (S110).

Specifically, the central server 100 stores user information in advance, and when a user sound source generated by the sound source production interface is received from the user terminal 200 later, it is matched with the user information and stored.

At this time, the central server 100 provides a profile interface to the user terminal 200 . The profile interface may display at least one of user information, a user sound source, and the number of followers/following.

In addition, if a plurality of user sound sources corresponding to a specific user terminal 200 exist, they may be listed and displayed. At this time, the number of hits for each user sound source included in the list and the revenue generated by the distributor server 300 are further displayed.

At this time, the process of generating a user sound source through the sound source production interface by the user terminal 200 initiated in step S110 will be described in detail with reference to FIG. 4 to be described later.

Next, the central server 100 performs a contest through user sound sources received from different user terminals 200 (S120).

Prior to step S120, the central server 100 provides a user sound source to the listener terminal and provides a contest interface for performing the contest.

At this time, the listener terminal means a terminal that listens to or evaluates the user's sound source. That is, since it can correspond to a user who uses the service of the platform, the user terminal 200 can also be included in the listener terminal in a broad sense.

In addition, the central server 100 selects a winning sound source based on the preferences acquired by a plurality of user sound sources during a predetermined period when a listener terminal that has received a user sound source through the contest interface gives a preference to a specific user sound source. .

Finally, the central server 100 provides the winning user sound source to the distributor server 300, and the distributor server 300 generates and distributes profits (S130).

At this time, the distributor server 300 generates revenue based on the winning music source, and provides a predetermined ratio of revenue to the service subject of the central server 100 and the creator of the user terminal 200 .

4 is an operation flowchart illustrating a process of producing a user sound source according to an embodiment of the present invention.

Referring to FIG. 4 , the central server 100 provides an MR sound source according to the request of the user terminal 200 (S210).

Before step S210, the central server 100 provides a list of MR sound sources possessed to the user terminal 200, and when the user terminal 200 selects a specific MR sound source, the specific MR sound source is provided.

Next, the user terminal 200 generates a user sound source by combining the MR sound source and the sound source including the user's voice information through the sound source production interface (S220).

Prior to step S220, the central server 100 provides a voice reception interface to the user terminal 200 to receive voice information from the user terminal 200. At this time, the voice receiving interface may simply receive only voice information, or may receive voice information in the form of a video through a camera, if necessary.

In addition, in step S220, the central server 100 provides an interface for mixing and mastering voice information to be included in the user sound source through a sound source production interface.

In addition, when the user terminal 200 receives a custom sound source adjustment value from the user, it performs mixing and mastering of voice information of the user sound source based on the custom sound source adjustment value.

In this case, the custom sound source adjustment value may include a mixing adjustment value and a mastering adjustment value. Specifically, the mixing adjustment value may include at least one of a vocal sync value, a master volume value, and an editing value. Also, the mastering adjustment value may include at least one of doubling, compressor, equalizer, reverb, delay, and limiter values.

For example, the vocal sync value may correspond to a value for correcting sync by changing a reproduction time of the voice information. In addition, the master volume value may adjust the volume for each predetermined section of voice information, and the editing option value may correspond to a value for repeatedly recording or deleting a predetermined section of the voice information.

As an optional embodiment, the central server 100 may receive and store voice information generated from other terminals of the user (eg, desktop, laptop, etc.). Thereafter, voice information may be provided according to a request of the user terminal 200 - a terminal completely different from the other terminals disclosed above. That is, the user records his or her song on the desktop, generates voice information and uploads it to the central server 100, and the user later downloads the voice information uploaded to the central server 100 to the smartphone to perform editing. there will be

Finally, the central server 100 receives the user sound source from the user terminal 200 (S230).

As an optional embodiment, the MR sound source received by the user terminal 200 in step S210 may be matched with a preset sound source adjustment value and stored.

In this case, the process in step S220 may automatically combine the MR sound source and voice information according to the AI mixing algorithm and the sound source adjustment value.

A detailed description of the above embodiment will be performed through FIG. 5 to be described later.

5 is an operation flowchart illustrating a process of mixing a sound source based on AI according to an embodiment of the present invention.

Referring to FIG. 5 , the user terminal 200 receives voice information from the user (S310).

In this case, as described above, the voice information refers to information in which a voice or a song generated by a user directly uttered is recorded.

In step S310, the user terminal 200 may collect the user's voice information and extract bass, midrange, and treble from the corresponding voice information. Alternatively, as an additional embodiment, the user terminal 200 may provide a preset MR sound source to the user, induce the user to sing along through the MR sound source, and receive voice information for each bass, middle, and treble tone. .

After that, the user terminal 200 can distinguish the gender of the voice information.

Next, the user terminal 200 selects a comparison singer similar to the voice information within the corresponding gender (S320).

Specifically, the user terminal 200 compares the voice information with pre-stored singer voice information of at least one comparison singer. At this time, the similarity with the singer's voice information is calculated based on at least one of the genre, timbre, and pitch of the voice information. At this time, the singer's voice information refers to voice information of a song sung by an actual singer in accordance with his or her MR sound source.

First, a method for selecting a comparison singer based on a genre is as follows.

In addition, when preset genre information is provided for each genre of the singer's voice information, the genre information associated with the user is digitized based on either preferred genre information previously received from the user or genre information corresponding to the voice information. . Alternatively, the user terminal 200 may digitize the genre information by referring to the preferred genre information previously received from the user (ie, the value input as the preferred genre in the profile information input step).

Thereafter, the user terminal 200 digitizes each of the genre information given to the singer's voice information and compares it with the user's genre information to select a comparison singer. For example, if genres of songs owned by each singer are identified, scores for the corresponding genre are digitized and then averaged, the representative genre value of the corresponding singer can be digitized. Here, the user terminal 200 may select a comparison singer having the smallest comparison value by comparing genre information of the user and genre information of the comparison singer.

Second, a method for selecting a comparison singer based on tone is as follows.

The user terminal 200 digitizes the timbre of the voice information, compares it with the timbre value included in the singer's voice information of the comparison singer, and selects a comparison singer having the lowest difference. For example, after defining each tone, such as a husky tone, a soft tone, and a rough tone, digitization may be performed for each tone. This can be digitized through a conventional tone digitization program.

Thirdly, a method for selecting a comparison singer based on height is as follows.

The user terminal 200 digitizes the height of the voice information, compares it with the height value included in the singer voice information of the comparison singer, and selects the comparison singer having the lowest difference. In this case, the pitch value may be an average value of a plurality of pitches expressed within one singer's voice information.

A comparison singer may be selected based on at least one numerical value among the above-described timbre, genre, and pitch. If an example is given based on the tone color, at least three comparison singers having the lowest difference value from the value of the user's tone tone may be selected and sorted according to the difference value. In this way, the same method can be performed for each genre and pitch, and among them, considering all of the timbre, genre, and pitch, a comparison singer with the smallest difference from the user's numerical value can be selected. Alternatively, a comparison singer may be selected after applying a weight to any one of the three fields. For example, although singer A is derived for the genre and pitch, if the singer with the smallest difference value in the timbre part is B, and a weight is given for the timbre, B can be selected as the comparison singer.

In addition, the user terminal 200 classifies the category of voice information based on the comparison singer selection result. Thereafter, the user terminal 200 sets the sound source adjustment value corresponding to the classified category as the user's sound source adjustment value, and applies it in step S330 to be described later.

Here, the category may be defined for each comparison singer or may be defined as a separate criterion. For example, the separate criterion may be defined by genre, such as a ballad category or a hip-hop category, or defined by age, such as a teenager, 20s, or 30s. For each category, a numerical range for at least one of the genre, timbre, and pitch that can be included in the category may be defined, and if it meets these regulations, the user sound source may be defined as corresponding to the category. .

In addition, sound source adjustment values (mixing condition values, mastering adjustment values) may be preset for each category. For example, if the category is defined for each singer to be compared, and the user sound source corresponds to the category of singer A of the ballad type, the master volume ratio constituting the mixing condition value (i.e., the ratio between the singer's voice information (or MR sound source) and the user's voice volume ratio), vocal sync ratio (that is, the ratio of the alignment time difference between the reproduction position of the singer's voice information (or MR sound source) on the time axis and the reproduction position of voice information on the time axis) Since the recording is done with the MR sound source, it is recorded at the same time from the user's point of view, but in reality, the voice is recorded at a slower beat than the MR sound source. A value for ratio adjustment is required) may be preset. In addition, the mastering adjustment value is also composed of elements such as doubling, compressor, equalizer (EQ), reverb, delay, limiter, etc., and numerical values to be adjusted for each element may be preset.

Such a sound source adjustment value may be differently defined for each category. Alternatively, the sound source adjustment value may be differently defined for each genre of music. For example, a different sound source adjustment value may be applied depending on whether the user sings a ballad or a hip-hop song. In addition, even for one song, the sound source adjustment value may be set differently in the middle according to the flow (beat or melody) of the song. For example, when a song called A with a length of 2 minutes has different beats based on a play time of 1 minute, the sound source adjustment value may be set differently according to each section.

As an additional embodiment, the user terminal 200 compares a plurality of tone templates previously stored in the database 104 of the central server 100 with the user's voice information, and selects a tone template having the highest similarity. At this time, the tone template is divided into soprano, mezzo-soprano, alto, countertenor, tenor, baritone, or bass, or divided into a number of levels, for example, 0 level (low tone tone) to 10 level (high tone tone), each It has a sound source adjustment value corresponding to the option of Accordingly, a sound source adjustment value corresponding to voice information is applied in the tone template.

Finally, the user terminal 200 generates a user sound source by mixing and mastering voice information based on the previously stored sound source adjustment value mapped to a comparison singer (S330).

After step S330, the user terminal 200 may receive a new user sound source from the user. In this case, the user terminal 200 may repeatedly perform the process up to step S330. The sound source adjustment value according to the re-execution result up to step S330 may be the same as or different from the previous sound source adjustment value. This is because, for example, the genre of the MR sound source selected by the user is different, or the user's voice information may be received and analyzed slightly differently from that of the previous sound source. Alternatively, when the user manually changes the adjustment value of the sound source, it may be different from that of the preceding sound source.

In this way, a large number of voice information (tone, genre, pitch) for one user can be received, and corresponding sound source adjustment values can be generated for each corresponding voice information. If expanded for several users, there may be training data of innumerable sound source adjustment values for innumerable voice information. When training is performed by setting the input value of the training data as voice information (tone, genre, pitch) and setting the output value as the sound source adjustment value and applying it to the machine learning model, the AI algorithm adjusts the sound source optimized for the user's sound source. Algorithm learning that recommends values can be completed. In this case, the machine learning model may be any one of ANN, DNN, KNN, and CNN, or an ensemble model combining at least two of them. Or, it could be a variety of other artificial intelligence learning models.

Therefore, the more the user creates the sound source, the more the sound source adjustment value is modified little by little, so that the sound source adjustment value suitable for the user can be more accurately derived. In addition, it is possible to recommend sound source adjustment values with accuracy even for the first service for users who access after training has already been completed.

If the value is received, the user terminal 200 generates a custom sound source adjustment value based on the received genre, timbre, and pitch to correct the user's voice.

At this time, the user terminal 200 may correct the pre-stored sound source adjustment value in conjunction with the singer's voice information of the comparison singer based on the custom sound source adjustment value. Through this, it is possible to create a user sound source to suit the user's taste.

At this time, by setting the mixing condition value and the mastering condition value for generating the user sound source as output values, the accuracy of the mixing condition value and the mastering condition value is increased in the process of generating the user sound source through the voice information of the new user. .

In this case, the machine learning model may be built in the user terminal 200, but may also be implemented in the central server 100 if necessary.

On the other hand, in another optional embodiment, after step S330, the user terminal 200 uploads the user sound source to the central server, and streams the user sound source to other user terminals.

6 is an operational flowchart illustrating a process of mixing and contesting a sound source through an interface provided by a platform according to an embodiment of the present invention.

Referring to FIG. 6 , the central server 100 creates a user profile through user information (S410).

At this time, the user profile is displayed on the user terminal through a profile interface.

As shown in FIG. 8 , the profile interface displays at least one of a user's picture and the number of followers/following in the first area 110 of the profile interface.

In addition, a list of the user sound source uploaded by the user terminal is displayed in the second area 120 of the profile interface. By displaying the number of hits for each user sound source and the profit by the user sound source, the user can check which sound source is most popular or has a high profit rate among the user sound sources.

In addition, an option for selecting an interface is provided in the third profile interface area 140 .

Meanwhile, as an optional embodiment, before step S410, the central server 100 provides a survey interface for receiving user information to the user terminal 200.

At this time, the survey interface provides options for a preset question, and classifies the user's song propensity according to the selected options. Based on the classified propensity, the quality of the service to be provided in the future is improved.

Next, when the central server 100 receives the user sound source generated by the sound source production interface from the user terminal 200, it is matched with the user profile and stored (S420).

At this time, the sound source production interface may be composed of a voice reception interface, a mixing setting interface, and a mastering setting interface.

First, the voice reception interface corresponds to an interface for receiving at least one of a user's voice and video.

At this time, the MR list 210 is provided as shown in FIG. 9 as needed, and the user selects and inputs a specific MR sound source 220 corresponding to the song to be sung.

Referring to FIG. 10 , video information being recorded in real time may be provided to the first area 230 of the voice receiving interface.

Lyrics of a song corresponding to the voice information are provided in the second area 231 of the voice receiving interface, providing an environment in which the user can sing while viewing the lyrics.

At least one button for manipulating the voice reception interface may be provided in the third region 240 of the voice reception interface. For example, "recording", "camera switching", "screen effect setting", etc. may be included, and may be provided in the form of icons that users can easily recognize.

Next, the mixing setting interface may modify mixing adjustment values for mixing the MR sound source and voice information.

The mixing setting interface will be described through FIGS. 11A and 11B as follows.

At this time, in the mixing setting interface, the mixing adjustment value may include at least one of a vocal sync value, a master volume option value, and an edit value.

Specifically, an option for selecting and editing any one of a vocal sync value, a master volume option value, and an edit value may be provided in the first region 250 of the mixing setting interface. Accordingly, the user inputs and selects one of "vocal sync value", "master volume value", and "edit value" according to what he/she intends to modify.

Waveforms of the MR sound source and voice information are mapped and arranged in the second area 260 of the mixing setting interface, and a time value for determining synchronization between the MR sound source and the voice information is provided as visual information.

At least one button for adjusting any one of a vocal sync value, a master volume value, and an edit value is provided in the third area 251 of the mixing setting interface. At this time, since the shape or number of provided buttons may be provided in a form suitable for adjusting the above numerical value, the scope of the present invention is not limited.

For example, in the case of FIG. 11A, it is an interface for correcting the sync of a sound source. Accordingly, the third region 251 of the mixing setting interface is provided with an input button in the form of moving or skipping the sync of the sound source in units of time. Meanwhile, FIG. 11B is an example of an interface for adjusting voice information and master volume of an MR sound source. A pair of rotary buttons for adjusting the volume of the MR sound source or voice sound source in a specific section of the mixed sound source may be provided in the third area 251 of the mixing setting interface. As such, the shape and number of input buttons provided to set the mixing value and mastering value of the present invention can be provided in a form that the user can intuitively see and determine.

Next, the mastering setting interface may modify a mastering adjustment value for mastering at least one of the MR sound source and voice information.

In this case, the mastering adjustment value may include, for example, at least one of doubling, compressor, equalizer, reverb, delay, and limiter values.

Referring to the form of the mastering setting interface through Figure 11c, as follows.

An option for selecting and editing any one of the doubling, compressor, equalizer, reverb, delay, and limiter values is provided in the first area 252 of the mastering setting interface. The user may select a mastering setting to be modified by touch input.

The second area 261 of the mastering setting interface may be provided to visually provide a location of a section to be edited or to select a specific section where the waveform of the user sound source is arranged.

At least one button for adjusting any one of the doubling, compressor, equalizer, reverb, delay, and limiter values may be provided in the third region 253 of the mastering interface.

At this time, the shape and number of buttons provided through the third area 253 may vary according to the value to be modified, so the scope of the invention is not limited. For example, a rotary button shown in FIG. 11B may be provided in the third area 253 of the mastering interface to adjust the doubling and compressor.

Finally, a contest is performed through a plurality of user terminals received from other user terminals (S430).

At this time, the central server 100 provides the user sound source to the listener terminal and provides a contest interface for performing the contest.

Specifically, referring to FIG. 12, the contest interface will display a list 330 of one or more user sound sources in which the contest is in progress.

In addition, when the listener terminal selects a specific user sound source included in the list of user sound sources, song information and streaming of the specific user sound source are provided. At this time, the song information includes at least one of the user profile, the name of the original song, the singer's name of the original song, a description of the original song, a thumbnail, image information, and a numerical value of preference given to a specific user sound source.

In addition, if a winning sound source is selected based on the preference obtained from the listener terminal for each user sound source during a predetermined period, the selected winning sound source is displayed on the contest interface (310, 320) for a predetermined period.

At this time, as shown in FIG. 13, the user's profile information 340 and comments 350 may be received from the user terminal 200 and displayed 360 along with other user sound sources created by the user and the number of winnings. have.

7 is an operational flowchart illustrating a process of generating revenue through a sound source and distributing revenue according to an embodiment of the present invention.

The following is an example for explaining the generation and flow of revenue. In the case of conventional music, in the case of generated revenue, the music seller divides 30% and the copyright association divides 10%, and 60% of the originally generated revenue is paid again by the music distributor for 20% commission and the agency 80% commission. Since each of them is taken, the profit that the creator of the sound source has will be greatly reduced. On the other hand, even in the case of video content such as YouTube, YouTube, copyright associations, sound source (video) distributors, promoters, etc. have a portion of the revenue, so the actual content producer has a limit in that the percentage of revenue is lowered.

Referring to FIG. 7 , the distributor server 300 generates revenue of 1 million won by disclosing user sound sources to listeners (S510).

As described above, the user sound source transmitted to the distribution company server 300 in step S510 corresponds to the sound source that the central server 100 competes with a plurality of other user sound sources for a predetermined period and wins.

If the user's sound source is a media consisting of sound, the central server 100 transmits it to the distribution company server 300 corresponding to the sound source seller. If the user's sound source is a media composed of a video, the central server 100 delivers it to the distributor server 300 corresponding to the video streaming company.

Next, the distributor server 300 acquires 30% (300,000 won) of the generated revenue as a commission (or distribution margin) (S520).

At this time, as an optional embodiment, if the copyright of the MR sound source used to generate the user sound source has already been registered, the Copyright Association, etc. will obtain 10% (100,000 won) of the revenue (S5210). However, when a user sound source in which no other copyright owner's data is used for the user sound source generates revenue through YouTube or the like, the corresponding process may be omitted.

Next, the service subject of the central server 100, which receives the proceeds of 600,000 won excluding commission from the distributor server 300, provides 20% (120,000 won) to the original music distributor and 50% (240,000 won) of the remaining proceeds. is obtained (S530).

As an optional embodiment, when at least one user sound source generated by a specific user terminal 200 is provided to the distributor server 300 in step S520 or step S530, the characteristic user terminal 200 in step S540. The distribution ratio of revenue is adjusted according to the number of user sound sources.

In addition, as another optional embodiment, if a specific user sound source generates more than a predetermined amount of revenue in step S510, the distribution ratio of the revenue is adjusted according to the inquiry of the specific user sound source in step S540.

Through the above two functions, the central server 100 can be a means to support excellent creators and at the same time guarantee high profits.

Finally, the creator of the user terminal 200 obtains 50% (240,000 won) of the revenue, and thus the distribution of revenue is completed (S540).

On the other hand, in step S510, upon request of the specific user terminal 200, it is possible to request non-disclosure processing of a specific user sound source delivered to the distributor server 300. Thereafter, in step S520, when a specific listener terminal accounts for a predetermined percentage or more of the revenue generated from a plurality of user sound sources of the specific user terminal 200, the specific user sound source may be provided to the specific listener terminal. That is, when a listener B enthusiastically listens to music of a creator A, the private music source of creator A can be provided only to a listener B.

The above description of the present invention is for illustrative purposes, and those skilled in the art can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, the embodiments described above should be understood as illustrative in all respects and not limiting. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and equivalent concepts should be construed as being included in the scope of the present invention. do.

100: central server
200: user terminal
300: distributor server

Claims (15)

In the method of generating a sound source based on AI, performed by a user terminal,
(a) receiving voice information from a user;
(b) selecting a comparison singer having a predetermined similarity with the voice information; and
(c) generating a user sound source by mixing and mastering the voice information based on a pre-stored sound source adjustment value mapped to the comparison singer;
The voice information is information in which a voice or song generated by the user directly uttered is recorded,
The sound source adjustment value includes a mixing adjustment value and a mastering adjustment value,
Before step (a) above
Providing a preset MR sound source to the user and inducing the user to sing along through the MR sound source, wherein step (a) receives the voice information for each bass, middle, and treble tone from the user;
After step (c) above
Build a machine learning model that takes as input values data on the genre, pitch, and timbre of a plurality of sound sources applied to a plurality of voice information, and set the mixing condition value and mastering condition value for generating the user sound source as output values In the process of generating the user sound source through the voice information of the new user, the accuracy of the mixing condition value and the mastering condition value is increased. delete According to claim 1,
The step (b) is
Comparing the voice information with the pre-stored at least one singer's voice information of the comparison singer, and calculating a similarity with the singer's voice information based on the genre, timbre, and pitch of the voice information. How to create. According to claim 3,
If preset genre information is given for each genre of the singer's voice information,
The genre information associated with the user is digitized based on either the preferred genre information previously received from the user or the genre information corresponding to the voice information, and each genre information assigned to the singer's voice information is digitized. A method of generating a sound source based on AI, which is to select the comparison singer by comparing. According to claim 3,
The method of generating a sound source based on AI, wherein the timbre of the voice information is digitized and compared with the timbre value included in the singer voice information of the comparison singer, and the comparison singer having the lowest difference is selected. According to claim 3,
The method of generating a sound source based on AI, wherein the height of the voice information is digitized, and the comparison singer having the lowest difference is selected by comparing the height value included in the singer voice information of the comparison singer. According to claim 1,
After step (b) above
A method for generating a sound source based on AI, wherein the category of the voice information is classified based on the selection result of the comparison singer, and the sound source adjustment value corresponding to the category is set as the user's sound source adjustment value. According to claim 1,
The sound source adjustment value is composed of a mixing condition value and a mastering adjustment value,
The method of generating a sound source based on AI, wherein the sound source adjustment value is mapped and stored differently according to the original song of the comparison singer. According to claim 8,
As for the sound source adjustment value, at least one sound source adjustment value may be assigned to one original song of the comparison singer, and two or more types of sound source adjustment values are classified according to at least one criterion of time signature or melody of the original song. A method of generating a sound source based on AI, which is set. According to claim 1,
After step (c) above
Upon receiving the genre, timbre and pitch for modifying the voice information from the user, a custom sound source adjustment value is created based on the received genre, timbre and pitch to modify the voice information, based on AI. how to create it. According to claim 10,
Based on the custom sound source adjustment value, the method of generating a sound source based on AI, wherein the pre-stored sound source adjustment value is corrected in conjunction with the singer voice information of the comparison singer. delete According to claim 1,
After step (c) above
A method for generating a sound source based on AI, wherein the user sound source is uploaded to a central server and the user sound source is streamed and provided to other user terminals. In the device for generating a sound source based on AI,
A memory in which a program for generating a sound source based on the AI is stored, and
A processor for generating a sound source based on the AI by executing a program stored in the memory,
The processor receives voice information from a user, selects a comparison singer having a predetermined similarity with the voice information, mixes and masters the voice information based on a pre-stored sound source adjustment value mapped to the comparison singer, and performs user A sound source is generated, the voice information is information recorded by a voice or a song generated by the user's direct utterance, the sound source adjustment value includes a mixing adjustment value and a mastering adjustment value,
Prior to selecting the comparison singer, a preset MR sound source is provided to the user, and the user is induced to sing along through the MR sound source. Receiving the voice information for each,
After generating the user sound source, building a machine learning model that takes as input values data on genres, pitches, and timbres of a plurality of sound sources applied to a plurality of voice information, and mixing condition values and mastering for generating the user sound source A device for generating a sound source based on AI, which sets the condition value as an output value to increase the accuracy of the mixing condition value and the mastering condition value in the process of generating the user sound source through voice information of a new user. A computer-readable storage medium on which a program for performing the method of generating a sound source based on AI according to claim 1 is recorded.

Images