KR102623449B1 - User vocal evaluation result customized vocal training service providing method, apparatus and system - Google Patents

Abstract

The device according to one embodiment acquires the user's song data from the user's terminal, receives evaluation information for each item of the song data, processes the evaluation information to generate input data, and inputs the input data into an artificial neural network. The user's vocal grade can be obtained and vocal training information matched to the vocal grade can be provided.

Description

Method, device and system for providing customized vocal training service as a result of user vocal evaluation {USER VOCAL EVALUATION RESULT CUSTOMIZED VOCAL TRAINING SERVICE PROVIDING METHOD, APPARATUS AND SYSTEM}

The following embodiments relate to technology that provides customized vocal training services as a result of user vocal evaluation.

Recently, with the development of broadcasting and communication technology, the scope of celebrities' activities has expanded and their daily lives have been exposed through broadcasting programs, leading to an increase in the number of aspiring celebrities. In order for an ordinary person to become a celebrity, they must put in effort, such as attending a specialized academy or participating in an agency's offline audition. To select aspiring celebrities, agencies cast ordinary people on the street or conduct periodic auditions to select aspiring celebrities. In both cases, a lot of time, money, and effort are required, such as when aspiring celebrities make efforts on their own or when agencies directly search for and cast aspiring celebrities.

Meanwhile, as online video service technology such as YouTube has developed, opportunities for aspiring celebrities to demonstrate their talents and skills have increased. Aspiring celebrities are trying to become celebrities by recording videos of themselves singing or dancing using mobile phones or camcorders and uploading them to YouTube to attract public attention. However, in these online auditions, it is difficult to accurately evaluate the participants' skills because they upload videos of themselves singing or dancing and then attract the public's interest simply by clicking on them.

In addition, there is a problem in that popularity rankings based on keyword searches provided by portal sites, etc. are not objective due to advertising functions such as Overture. In addition, even if a person is popular online, if he or she is cast by an agency, he or she must still undergo years of professional training and the agency must invest a lot of money and time.

In this environment, the Korean music industry developed rapidly following the Korean Wave, the so-called K-POP. Accordingly, the number of aspiring singers or idols has increased, and the market for these aspiring singers is also rapidly rising. However, despite the rapid increase in the number of aspiring celebrities, the stages (i.e., auditions or agencies) where aspiring celebrities can debut as celebrities are limited, and therefore, for a successful debut, it is important to find an agency that suits one's voice.

However, most of the aspirants had difficulty selecting the vocal training that was right for them because it was difficult to obtain information about each entertainment agency and there were no appropriate indicators regarding their vocalization or voice.

Furthermore, there is a problem that agencies are spending excessive time and money to find competitive and customized candidates according to the concept planned by the agency among the rapidly increasing number of aspirants.

Republic of Korea Patent No. 10-2259612 (announced on June 1, 2021) Republic of Korea Patent No. 10-2107588 (announced on May 7, 2020) Republic of Korea Patent No. 10-1917216 (announced on November 9, 2018) Republic of Korea Patent No. 10-2139889 (announced on July 30, 2020)

Embodiments seek to provide a customized vocal training service as a result of user vocal evaluation.

Embodiments seek to obtain accurate vocal evaluation results through a process of additionally processing the input data in the process of evaluating the user's vocal grade by inputting evaluation information for each item of singing data into an artificial neural network.

Embodiments seek to recommend practice content based on the user's singing data.

According to one embodiment, the method performed by the device includes obtaining the user's song data from the user's terminal; Receiving evaluation information for each item about the song data; Processing the evaluation information to generate input data; Obtaining the user's vocal rating by inputting the input data into an artificial neural network; and providing vocal training information matched to the vocal grade.

The evaluation items included in the above evaluation information include pitch, tempo, breathing, rhythm, high pitch, expressiveness, and completeness evaluated as O or Includes levels rated as one of the numbers 1 to 8.

In the step of processing the evaluation information and generating input data, the same value as the level is input for the items that received an O among the pitch, tempo, breathing, rhythm, treble, expressiveness, and completeness, and for the items that received an Entering a value of -2 applied to the level, but entering 1 if it is less than 1; determining a grade according to the numbers entered for the pitch, tempo, breathing, rhythm, treble, expressiveness, and completeness; and It may include determining the data into grades of pitch, tempo, breathing, rhythm, high pitch, expressiveness, and completeness, a vocal range evaluated as high tone or low tone, and a voice type evaluated as one of four types.

The method performed by the device further includes the step of further processing the input data using transliteration information, and the step of further processing the input data includes obtaining the user's voice data from the user's terminal. A step of confirming the average vocal range of the user's voice using the voice data, a step of confirming the average vocal range of the user's singing using the song data, and the average vocal range of the voice and the average vocal range of the song. If the difference exceeds a preset standard, the step of lowering the grade of the pitch, tempo, breathing, rhythm, treble, expressiveness, and completeness by one level and adding a pitch correction item to the input data may be included. .

The method performed by the device further includes recommending practice content for the user based on the song data, and the step of recommending the practice content includes confirming the vocal genre of the user from the song data. , extracting main keywords from the song data, collecting vocal content of interest corresponding to the vocal genre from a vocal database, and determining whether vocal content containing the main keyword is included among the vocal content of interest. Step, if the vocal content containing the main keyword is included among the vocal content of interest, determining the vocal content containing the main keyword as practice content, the vocal content containing the main keyword among the vocal content of interest If not included, checking similar keywords related to the main keyword; if vocal content including the similar keyword is included among the vocal content of interest, determining the vocal content including the similar keyword as practice content , If vocal content including the similar keyword is not included among the vocal content of interest, determining the vocal content most registered by users with the same evaluation information as the user as practice content may be included.

The method performed by the device further includes the step of modifying the practice content based on the evaluation information, and the step of modifying the practice content includes selecting an item that received an Checking the number, if the number of items receiving X is less than a preset first standard, maintaining the practice content, while the number of items receiving Extracting similar users whose evaluation items correspond to those that received O, if the number of items that received X is more than a preset first standard but less than a preset second standard, Extracting the user's vocal content and adding it to the front of the practice content, if the number of items receiving the Extracting a good user corresponding to the number of items that received an It may include steps to add each.

The method performed by the device includes receiving content including a song desired by the user from the user's terminal; When the content is analyzed as content including a first song, classifying trainers with a history of training the first song into a matching candidate group; When it is confirmed that there is one trainer classified into the matching candidate group, setting the trainer as a matching partner for the user; When it is confirmed that there are two or more trainers classified into the matching candidate group, the trainer classified into the matching candidate group takes into account the number of training sessions for training the first song and the upload amount of content including the first song. Calculating a matching score for each individual, and if it is confirmed that the trainer has the highest matching score among the trainers classified into the matching candidate group, setting the trainer as a matching partner for the user; And when the trainer is set as a matching partner of the user, it may further include providing recommendation information about the trainer and link information of a training page including the song to the user's terminal.

The method performed by the device further includes the step of confirming the location of the trainer, wherein the step of confirming the location of the trainer includes obtaining a song history in which the user uploads the song data, the song history Based on this, generating the user's singing cycle, based on the current date and the user's singing cycle, confirming the first date, which is the user's next scheduled singing date, lessons from one of the trainers through the trainer database. Determining whether a trainer whose lesson availability period is the first date exists; if it is confirmed that a trainer whose lesson availability period is the first date exists as a first trainer, matching the first trainer to the user; Determining and confirming the location of the first trainer, if it is confirmed that there is no trainer whose lesson availability period is the first date, the second dates that fall before the preset range based on the first date are selected. A step of generating, determining whether a trainer whose lesson availability period of the trainer is the second date exists, if it is confirmed that a trainer whose lesson availability period of the trainer is the second date exists as a second trainer, determining that a second trainer will be matched to the user, and confirming the location of the second trainer, and if it is confirmed that there is no trainer whose lesson availability period is the second date, the lesson availability period among the trainers is It may include confirming a third trainer that is the trainer with the earliest date among the dates after the first date, determining that the third trainer will be matched to the user, and confirming the location of the third trainer.

The device according to one embodiment may be combined with hardware and controlled by a computer program stored in a medium to execute any one of the above-described methods.

Embodiments may provide a customized vocal training service as a result of a user's vocal evaluation.

In embodiments, accurate vocal evaluation results can be obtained through a process of additionally processing the input data in the process of evaluating the user's vocal grade by inputting evaluation information for each item of singing data into an artificial neural network.

Embodiments may recommend practice content based on the user's singing data.

1 is a diagram for explaining the configuration of a system according to an embodiment.
Figure 2 is a flowchart illustrating a process for providing a customized vocal training service as a result of a user's vocal evaluation according to an embodiment.
Figure 3 is a flow chart to explain a process for generating input data by processing evaluation information according to an embodiment.
FIG. 4 is a diagram illustrating a process for obtaining a user's vocal rating by inputting input data into an artificial neural network according to an embodiment.
Figure 5 is a diagram for explaining the learning and evaluation process of an artificial neural network according to an embodiment.
FIG. 6 is a diagram illustrating a process for determining a sound range and voice type according to an embodiment.
Figure 7 is a diagram for explaining an output interface of a user's terminal according to an embodiment.
Figure 8 is a flowchart for explaining a process of additionally processing input data according to an embodiment.
Figure 9 is a flowchart for explaining a process for recommending practice content according to an embodiment.
Figure 10 is a flowchart for explaining the process of modifying practice content according to one embodiment.
Figure 11 is a flow chart to explain the process of determining a trainer as a matching partner for a user according to an embodiment.
Figure 12 is a flow chart to explain the process of confirming the location of the trainer according to one embodiment.
Figure 13 is an exemplary diagram of the configuration of a device according to an embodiment.

Hereinafter, embodiments will be described in detail with reference to the attached drawings. However, various changes can be made to the embodiments, so the scope of the patent application is not limited or limited by these embodiments. It should be understood that all changes, equivalents, or substitutes for the embodiments are included in the scope of rights.

Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be modified and implemented in various forms. Accordingly, the embodiments are not limited to the specific disclosed form, and the scope of the present specification includes changes, equivalents, or substitutes included in the technical spirit.

Terms such as first or second may be used to describe various components, but these terms should be interpreted only for the purpose of distinguishing one component from another component. For example, a first component may be named a second component, and similarly, the second component may also be named a first component.

When a component is referred to as being “connected” to another component, it should be understood that it may be directly connected or connected to the other component, but that other components may exist in between.

The terms used in the examples are for descriptive purposes only and should not be construed as limiting. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by a person of ordinary skill in the technical field to which the embodiments belong. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and unless explicitly defined in the present application, should not be interpreted in an ideal or excessively formal sense. No.

In addition, when describing with reference to the accompanying drawings, identical components will be assigned the same reference numerals regardless of the reference numerals, and overlapping descriptions thereof will be omitted. In describing the embodiments, if it is determined that detailed descriptions of related known technologies may unnecessarily obscure the gist of the embodiments, the detailed descriptions are omitted.

Embodiments may be implemented in various types of products such as personal computers, laptop computers, tablet computers, smart phones, televisions, smart home appliances, intelligent vehicles, kiosks, and wearable devices.

An artificial intelligence (AI) system is a computer system that implements human-level intelligence, and unlike existing rule-based smart systems, it is a system in which machines learn and make decisions on their own. As artificial intelligence systems are used, their recognition rates improve and they can more accurately understand user preferences, and existing rule-based smart systems are gradually being replaced by deep learning-based artificial intelligence systems.

Artificial intelligence technology consists of machine learning and element technologies using machine learning. Machine learning is an algorithmic technology that classifies/learns the characteristics of input data on its own, and elemental technology is a technology that mimics the functions of the human brain such as cognition and judgment by utilizing machine learning algorithms such as deep learning, including linguistic understanding and visual It consists of technical areas such as understanding, reasoning/prediction, knowledge expression, and motion control.

The various fields where artificial intelligence technology is applied are as follows. Linguistic understanding is a technology that recognizes and applies/processes human language/characters and includes natural language processing, machine translation, conversation systems, question and answer, and voice recognition/synthesis. Visual understanding is a technology that recognizes and processes objects like human vision, and includes object recognition, object tracking, image search, person recognition, scene understanding, spatial understanding, and image improvement. Inferential prediction is a technology that judges information to make logical inferences and predictions, and includes knowledge/probability-based reasoning, optimization prediction, preference-based planning, and recommendations. Knowledge expression is a technology that automatically processes human experience information into knowledge data, and includes knowledge construction (data creation/classification) and knowledge management (data utilization). Motion control is a technology that controls the autonomous driving of vehicles and the movement of robots, and includes motion control (navigation, collision, driving), operation control (behavior control), etc.

Generally, in order to apply machine learning algorithms to real life, learning is performed using a trial and error method due to the nature of the basic methodology of machine learning. In particular, deep learning requires hundreds of thousands of iterations. It is impossible to execute this in an actual physical external environment, so instead, the actual physical external environment is virtually implemented on a computer and learning is performed through simulation.

1 is a diagram for explaining the configuration of a system according to an embodiment.

Referring to Figure 1, the system according to one embodiment includes a device 100, a user's terminal 200, an agency's terminal 300, an evaluator's terminal 400, and a trainer's terminal 500 that can communicate with each other through a communication network. ) may include.

First, a communication network can be configured regardless of the communication mode, such as wired or wireless, and can be implemented in various forms to enable communication between servers and between servers and terminals.

The device 100 may provide a customized vocal training service as a result of the user's vocal evaluation.

The device 100 can evaluate user vocals and provide a platform for customized vocal training services as a result of the user vocal evaluation, and manage the history of vocal evaluation results and vocal training service provision through the platform.

The device 100 may be its own server owned by a person or organization that provides services using the device 100, a cloud server, or a p2p (peer-to-peer) set of distributed nodes. It may be possible. The device 100 may be configured to perform all or part of the calculation function, storage/reference function, input/output function, and control function of a typical computer.

The device 100 may be configured to communicate wired or wirelessly with the user's terminal 200, the agency's terminal 300, the evaluator's terminal 400, and the trainer's terminal 500. Controls the operations of the terminal 300, the evaluator's terminal 400, and the trainer's terminal 500, and controls the user's terminal 200, the agency's terminal 300, the evaluator's terminal 400, and the trainer's terminal ( You can control which information is displayed on the screen 500).

According to one embodiment, a database may be provided within the device 100, but the present invention is not limited to this, and the database may be configured separately from the device 100. Device 100 may include multiple artificial neural networks for performing machine learning algorithms.

The user's terminal 200 may be a terminal used by a user who wishes to receive a vocal evaluation and a customized vocal training service according to the present invention. The user's terminal 200 may be a desktop computer, laptop, tablet, smartphone, etc. For example, as shown in FIG. 1, the user's terminal 200 may be a smartphone, and may be employed differently depending on the embodiment.

The user's terminal 200 may be configured to perform all or part of the calculation function, storage/reference function, input/output function, and control function of a typical computer. The user's terminal 200 may be configured to communicate with the device 100 wired or wirelessly.

The user's terminal 200 is connected to a web page operated by a person or organization that provides services using the device 100, or an application developed and distributed by a person or organization that provides services using the device 100. This can be installed. The user's terminal 200 may be linked to the device 100 through a web page or application.

The user's terminal 200 can access the device 100 through a web page or application provided by the device 100.

The agency's terminal 300 may be a terminal used by workers at the entertainment agency according to the present invention. The agency's terminal 300 may be a desktop computer, laptop, tablet, smartphone, etc. For example, as shown in FIG. 1, the agency's terminal 300 may be a smartphone and may be employed differently depending on the embodiment.

The agency's terminal 300 may be configured to perform all or part of the calculation function, storage/reference function, input/output function, and control function of a typical computer. The agency's terminal 300 may be configured to communicate with the device 100 in a wired or wireless manner.

The agency's terminal 300 is connected to a web page operated by a person or organization providing services using the device 100, or an application developed and distributed by a person or organization providing services using the device 100. This can be installed. The agency's terminal 300 may be linked to the device 100 through a web page or application.

The agency's terminal 300 can access the device 100 through a web page or application provided by the device 100.

The evaluator's terminal 400 may be a terminal used by an evaluator to evaluate the user's singing ability according to the present invention. The evaluator's terminal 400 may be a desktop computer, laptop, tablet, smartphone, etc. For example, as shown in FIG. 1, the evaluator's terminal 400 may be a smartphone and may be employed differently depending on the embodiment.

The evaluator's terminal 400 may be configured to perform all or part of the calculation function, storage/reference function, input/output function, and control function of a typical computer. The evaluator's terminal 400 may be configured to communicate with the device 100 wired or wirelessly.

The evaluator's terminal 400 is connected to a web page operated by a person or organization providing services using the device 100, or an application developed and distributed by a person or organization providing services using the device 100. This can be installed. The evaluator's terminal 400 may be linked to the device 100 through a web page or application.

The evaluator's terminal 400 can access the device 100 through a web page or application provided by the device 100.

The trainer's terminal 500 may be a terminal used by a trainer who wishes to provide vocal training services according to the present invention. The trainer's terminal 500 may be a desktop computer, laptop, tablet, smartphone, etc. For example, as shown in FIG. 1, the trainer's terminal 500 may be a smartphone and may be employed differently depending on the embodiment.

The trainer's terminal 500 may be configured to perform all or part of the calculation function, storage/reference function, input/output function, and control function of a typical computer. The trainer's terminal 500 may be configured to communicate with the device 100 wired or wirelessly.

The trainer's terminal 500 is connected to a web page operated by a person or organization providing services using the device 100, or an application developed and distributed by a person or organization providing services using the device 100. This can be installed. The trainer's terminal 500 may be linked to the device 100 through a web page or application.

The trainer's terminal 500 can access the device 100 through a web page or application provided by the device 100.

The singular expressions recited in the claims may be understood to include the plural.

Meanwhile, for convenience of explanation, only the user's terminal 200, the agency's terminal 300, the evaluator's terminal 400, and the trainer's terminal 500 are shown in FIG. 1, but the number of terminals varies depending on the embodiment. Anything can change. As long as the processing capacity of the device 100 allows, the number of terminals is not particularly limited.

Figure 2 is a flowchart illustrating a process for providing a customized vocal training service as a result of a user's vocal evaluation according to an embodiment.

Referring to Figure 2, first, in step S201, the device 100 may obtain the user's song data from the user's terminal 200.

Specifically, the device 100 may receive song data from the user's terminal 200 where song data in which the user sang a song was generated, and may store the user's song data by matching it with the user in a database.

The user's terminal 200 may record video or audio data of the user singing a song, and generate the user's singing data according to the recording results. At this time, the song data may be a video file or a voice file, but is not limited thereto.

The device 100 may collect voice, video, etc. directly from the user's terminal 200, or may obtain song data by collecting and processing data uploaded and registered to SNS services, etc. from the user's terminal 200.

Additionally, the user's terminal 200 may include user information related to the user in the song data. At this time, the user information may include, but is not limited to, user identification information, user gender, age, and physical characteristic information. User information may also include information about virtual people in the metaverse world.

The device 100 may provide a platform service or website for providing a customized vocal training service as a result of the user's vocal evaluation, and the user may upload singing data to the corresponding platform service or website.

In step S202, the device 100 may receive evaluation information for each item about the song data.

At this time, the evaluation items included in the evaluation information include pitch, tempo, breathing, rhythm, treble, expressiveness, and completeness evaluated as O or ) may include a voice type evaluated as one of the voices, and a level evaluated as one of the numbers 1 to 8.

According to one embodiment, the device 100 performs a vocalization classification analysis including the user's vocal level, position, and vocalization properties from the song data received from the user's terminal 200, and converts the vocalization classification analysis results into a pre-built Intuition level and item level can be calculated by applying it to the artificial intelligence learning-based evaluation model database.

In addition, the device 100 may generate vocal training information for a vocal training service based on the vocal grade including the calculated intuition level and item level, and send the generated vocal training information to the user's terminal 200, It can be transmitted to at least one of the agency's terminal 300, the evaluator's terminal 400, and the trainer's terminal 500.

More specifically, the device 100 may first perform a vocalization classification analysis including the user's vocal level, position, and vocalization attributes based on a vocalization classification analysis algorithm preset from the song data.

The device 100 performs artificial intelligence learning processing using the labeling data of the pre-learning data to calculate the intuition level and item level based on the analysis result data according to this speech classification analysis, and creates an artificial intelligence learning-based evaluation model. Pre-construction can be processed.

According to the construction of this artificial intelligence learning-based evaluation model, the device 100 can calculate a vocal rating including the intuitive level and item level for each user based on the vocalization classification analysis result, and this dual-level calculation processor Accurate evaluation becomes possible.

Here, the artificial intelligence learning-based evaluation model for vocal rating including intuition level and item level is an artificial intelligence learning-based evaluation model built by learning intuition judgment information and item evaluation information corresponding to each singing data included in the learning data. It can be determined by the evaluation model.

Intuitive judgment information can be labeled from learning data input as the evaluator using the evaluator's terminal 400 listens to or watches the song data. For example, intuitive judgment information for each judgment criterion as follows is learned. Labeling as data may be exemplified.

For example, intuition judgment information may include numbers from 1 to 8.

8: It ticks all the boxes and feels impeccably solid;

7: The more I sing, the more I listen to the song, and even when I just hear it, I instantly feel like I'm good at it.

6: A level where a lot of practice is felt, showing overall stable singing,

5: The song or K-pop feel is good, but it feels like it has some unsettling elements,

4: Gayo, starting to feel like K-pop,

3: I have the minimum basic skills, but it is difficult to get a K-Pop feel.

2: It plays a role as a singer, but the basic skills are still lacking,

1: Closer to speaking than to singing

Additionally, item judgment information can be technically determined by applying preset evaluation items to song data.

Here, the evaluation items of the item judgment information may be exemplified as follows.

Pitch: the distance between notes, whether the pitch of the sound is expressed accurately,

Sense of tempo: Whether there is a sense of keeping the BPM constant;

Breathing: Vibrato based on breathing, including vibrato when there is no basic technique, and must be expressed more than twice when singing to be recognized.

Sense of rhythm: various bending expressions, whether there is vibration that matches the rhythm,

Treble stability: Whether there is treble stability within the sound range,

Expressive power: The overall dynamics of the song, the ability to control volume dynamics, the emphasis of sarcasm diction, handling of end notes, changes in voice texture (ability to control vocal cords and breathing), whether a certain percentage of emotional expressiveness is present,

Completeness: This is a standard for items that are checked when all conditions are met and impactful parts are clear, indicating flawless skills.

The device 100 can build an evaluation model database through artificial intelligence deep learning-based learning of the intuition judgment information and item judgment information exemplified in this way, calculate the intuition level and item level based on this, and make accurate judgments. Various services related to vocals can be provided through each user's terminal (200), the agency's terminal (300), the evaluator's terminal (400), and the trainer's terminal (500).

In step S203, the device 100 may process the evaluation information to generate input data. At this time, the process of processing the evaluation information to generate input data will be described later with reference to FIG. 3.

Meanwhile, the device 100 may perform a process of further processing input data using transliteration information. At this time, a detailed description of the process of further processing the input data will be described later with reference to FIG. 8.

In step S204, the device 100 may input the input data into an artificial neural network to obtain the user's vocal rating.

The device 100 may obtain the user's song data from the user's terminal 200. Here, the device 100 can collect voice, video, etc. directly from the user's terminal 200, or collect and process data uploaded to SNS services, etc. from the user's terminal 200.

Additionally, the device 100 may transmit data collected from an artificial intelligence learning-based evaluation model database to be used as learning data. Here, intuitive judgment information and item judgment information input from each evaluator's terminal 400, input by an expert, etc., or determined based on technology may be assigned to the learning data as labeling information. Additionally, the device 100 may check the correlation between each collected song data and labeling data and remove labeling noise from the data.

And, since the length and characteristics of the collected song data are all different, the device 100 can preprocess the collected song data. For example, the preprocessing process may include cropping to a certain length (5 to 10 seconds).

Additionally, the device 100 may perform augmentation processing to expand the preprocessed data into various data, such as data with accompaniment, data without accompaniment, and data with noise. For AUGMENTATION processing, deep learning techniques can be used, traditional methods such as RoomImpulse Response can be used, and furthermore, efficient information extraction from the voice through the spectrum can be used to analyze the singing data. It can also be preprocessed.

Then, the device 100 performs a vocalization classification analysis including the user's vocal level, position, and vocalization properties from the song data, and applies the vocalization classification analysis result to a pre-built artificial intelligence learning-based evaluation model database, A vocal rating including intuition level and item level can be calculated.

Here, the vocalization classification analysis algorithm is a process of classifying each user's vocal level, position, and vocalization attributes from singing data, and is preferably performed using an artificial intelligence learning-based evaluation model database, which will be described in more detail later. do.

In addition, the artificial intelligence learning-based evaluation model database can be constructed from learning data preprocessed from the above-described song data, labeling data, and input data of the evaluator's terminal 400, and is configured according to a preset voice type labeling algorithm. It can be.

For example, an artificial intelligence learning-based evaluation model database can build a learning database by applying vocal content data transformed into a spectrum to a convolutional neural network (CNN) commonly used for images. there is. To this end, the artificial intelligence learning-based evaluation model database can achieve the highest efficiency even with small data by using pre-trained weights that are specialized for song data.

Furthermore, the artificial intelligence learning-based evaluation model database can use softmax and Cross Entropy Loss as classification models to effectively process speech classification analysis algorithms, and Mean Squared Loss for learning to determine intuition level and item level. You can use the back propagation algorithm used

The artificial intelligence learning-based evaluation model database according to an embodiment of the present invention preserves privacy by building a data flow pipeline and can continuously update the performance of the model through continuous learning, and can be used universally for various services. An applicable API may be configured. Accordingly, an evaluation process using an artificial intelligence learning-based evaluation model database can be provided through various services such as mobile phone application services and metaverse services.

The artificial intelligence learning-based evaluation model database includes four categories: pitch, tempo, breathing, rhythm, treble, expressiveness, completeness, level evaluated as one of the numbers 1 to 9, vocal range evaluated as high tone (H) or low tone (L), and The voice type evaluated as one of the types (FMSO) is composed of labeling data, and an artificial intelligence learning-based evaluation model database can be built.

According to this model building and analysis process, vocal analysis from videos uploaded from the user's terminal 200 can be automated, and the device 100 analyzes big data through an AI algorithm to classify and evaluate vocal grades. can be derived, and through an artificial intelligence learning-based evaluation model database-based evaluation for each voice range and voice type, vocal grades and vocal training services based on these are provided based on pitch, tempo, breathing, rhythm, treble, expressiveness, and completeness. can do.

The vocal classification system according to the present invention may include, but is not limited to, a vocal range evaluated as high tone (H) or low tone (L) and a voice type evaluated as one of four types (FMSO). The vocal classification system according to the present invention is proposed for effective vocal type classification according to an embodiment of the present invention. FMSO largely classifies voice types into four types (FMSO) based on the degree of vocal cord grounding and breathing. It can be composed by mixing the HL system, which is subclassified into eight types by assigning high tone (H) or low tone (L) depending on the high and low tone of the system and voice.

Accordingly, the following vocal type classification and guide information for each type can be analyzed and processed from the song data.

HF: This is the type that breathes with the strongest force, and has a high-pitched, husky voice mixed with loud breathing.

LF: It uses the largest amount of breathing and can express husky and strong emotions with a heavy low tone at high volume.

HM: He has the highest level of vocalization among idols and is the type that can handle the highest register, with a high-pitched, clear and solid voice.

LM: This type of voice is low-toned, loud and clear, showing strength in low notes and capable of digesting a wide range of sounds from low to high notes.

HS: This is the type that uses the weakest breathing power, and the high-pitched, clear voice with breathing at a low volume is attractive.

LS: Although the voice is produced with light force, it has a low tone voice with the most breathing and a pure and faint emotional tone.

HO: A clear, high-pitched sound can be produced with the power of small breathing. They often have a clear and refreshing voice.

LO: This is a type that produces a soft, low-tone voice. It uses light breathing and uses the vocal cords efficiently to produce a clear sound.

Accordingly, when the vocal type is classified, the device 100 can determine the vocal grade corresponding to the vocal type classification, and the same labeling is also processed and learned in the learning data for the determination, so that it is displayed on the artificial intelligence learning-based evaluation model database. It can be built in advance.

In step S205, the device 100 may provide vocal training information matched to the vocal grade.

Apparatus 100 may provide vocal training services based on vocal ratings, including intuition level and entry level.

For example, if the vocal grade is from 1 to 5, the database of the device 100 matches and stores information on the curriculum for vocal training in stages A, B, C, D, and E according to each vocal grade. There may be. At this time, the vocal grade and the number of steps in the curriculum for vocal training are only examples and are not limited thereto.

For example, if the user's vocal level is level 3, the device 100 may extract vocal training information including a curriculum corresponding to level C and provide it to the user.

At this time, information about the curriculum for vocal training at each stage may include, but is not limited to, information about the trainer name of the vocal curriculum, the name of the lecture, the song to be used for training, the training period, the training location, and the training cost.

The vocal training service may include a guide information provision service that guides each user's weak points or strengths according to each user's vocal level.

Additionally, the vocal training service may include a matching service that matches and provides appropriate agency information based on each user's vocal type and vocal grade.

The vocal training service may include a matching service that matches and provides appropriate lecture information based on each user's vocal type and vocal grade.

Additionally, the vocal training service may include a matching service that matches and provides appropriate trainer information based on each user's vocal type and vocal grade.

To this end, the device 100 may configure type guide service data based on the vocal rating and provide the type guide service data to the user's terminal 200.

In addition, the device 100 configures agency matching service data based on vocal ratings based on agency information pre-collected from the agency's terminal 300 and sends the agency matching service data to the agency's terminal 300 or the user's terminal ( 200).

The device 100 may configure lecture information service data based on vocal grades based on a lecture information database pre-established through a network and provide the lecture information service data to the user's terminal 200.

In addition, the device 100 configures trainer matching service data based on vocal grade based on the trainer information collected from the trainer's terminal 500, and sends the trainer matching service data to the user's terminal 200 or the trainer's terminal ( 500).

Meanwhile, in providing the service, the device 100 acquires singing data of users who participated in the audition event from the terminal 200 of the user who participated in the preset audition event, and vocals for each user calculated from the user's singing data. Based on the grade, audition service data based on user vocal evaluation may be provided to the user's terminal 200.

The vocal training service may include a service that recommends practice content to improve the user's vocal skills. According to one embodiment, the device 100 may recommend practice content for the user based on song data. At this time, a detailed description of the process of recommending practice content will be described later with reference to FIGS. 9 and 10.

FIG. 3 is a flowchart illustrating a process for generating input data by processing evaluation information according to an embodiment, and FIG. 4 is a process for obtaining a user's vocal rating by inputting input data into an artificial neural network according to an embodiment. is a diagram for explaining, and FIG. 5 is a diagram for explaining the learning and evaluation process of an artificial neural network according to an embodiment.

Referring to FIG. 3, first, in step S301, the device 100 inputs the same value as the level for the items that received an O among the sense of pitch, sense of tempo, breathing, sense of rhythm, treble, expressiveness, and completeness, and inputs the same value as the level for the item that received an X. For this, enter a value of -2 applied to the level, but if it is less than 1, you can enter 1.

For example, each level is judged based on [Pitch], [Tempo], [Breathing Control], [Rhythm], [Treble Stability], [Expression], and [Completeness] that correspond to the HL and FMSO systems and are rated from 1 to 8. Level It can be classified into levels. In addition, [Number of items receiving O] + 1 can be defined as a vocal grade, and the [Completeness] item can be an arbitrary item that can be given when the other 6 items are all positive and judged to be at a perfect level.

For example, if the user received O in pitch, tempo, expression, rhythm, and high notes, the number of O + 1 (since there is no level 0, +1 should be applied to all levels by default) = 6 levels. You will be judged on your vocal grade, and for items given O, the judged level value of 6 may be calculated, and for items given X, a level value of -2 may be calculated.

Depending on the level assigned to each evaluation item, the evaluator's terminal 400 can input intuition judgment information, and the device 100 can input item judgment information based on an evaluation based on an artificial intelligence learning-based evaluation model database. .

The device 100 calculates an intuition level based on intuition judgment information and an item level based on item judgment information, performs detailed item data adjustment such as removing contradictions between intuition levels and item levels, and determines a final vocal grade. there is.

In the case of intuition judgment information, it is calculated as input information input by intuitively determining the level as heard at the evaluator's terminal 400, and may be configured as follows.

8 Level: It ticks all the boxes and feels impeccably solid.

7 Level: The more you sing, the more you listen to the song, and even when you just listen to it, you instantly feel like you're good at it.

6 Level: This is a level where a lot of practice is felt, showing overall stable singing.

5 Level: The song or K-pop feel is good, but it feels like it has some unsettling elements.

4 Level: Starting to feel like pop music and K-pop

Level 3: Possesses the minimum basic skills, but has difficulty in feeling the K-Pop style.

2 Level: The song plays a role, but the basic skills are still lacking.

1 Level: Closer to speaking than singing

Additionally, in the case of item judgment, each item that meets the following criteria can be evaluated as O/X, corresponding to the intuitive judgment level.

Pitch: The ability to accurately express the distance between notes and the pitch of the sound.

Sense of tempo: The sense of keeping the BPM constant

Breathing control: Vibrato based on breathing (including when vibrato is used when the basic technique is not well developed, it must be expressed more than twice during the singing part to be recognized)

Sense of rhythm: Various bending expressions, vibration that matches the rhythm.

High-pitched stability: Judged by high-pitched stability within the person's vocal range, vocalization and stability in the high-pitched region.

Expressive power: When a certain number of the following are present: the overall dynamics of the song, the ability to control volume dynamics, the emphasis of negative diction, handling of end notes, changes in voice texture (ability to control vocal cords and breathing), and emotional expressiveness.

Completeness: When all conditions are met and at the same time the impact is clear and the skills are flawless.

In other words, the labeling data required for learning the artificial intelligence learning-based evaluation model database is exemplified as follows.

Sense of pitch, sense of tempo, breathing control, sense of rhythm, stability of high notes, expressiveness, completeness (O): 7 items,

Sense of pitch, sense of tempo, breathing control, sense of rhythm, stability of high notes, expressiveness, completeness (X): 7 items,

HL: 2 categories;

FMSO: 4 categories

The level measurement result data may include a level number or grade information calculated using (O) and (X) data for each item used in the above-described level determination.

In step S302, the device 100 may determine the grade according to the numbers entered for pitch, tempo, breathing, rhythm, treble, expressiveness, and completeness.

At this time, referring to FIG. 4, the device 100 may replace the numbers entered for each evaluation item including pitch, tempo, breathing, rhythm, high pitch, expressiveness, and completeness with a grade.

At this time, since it can be classified from the lowest level (e.g., level 1) to the highest level (e.g., level 8), the value corresponding to the lowest level can be replaced with an F grade, and the value corresponding to the highest level can be replaced with an SS grade. .

That is, the device 100 has F for items corresponding to level 1, E for items corresponding to level 2, D for items corresponding to level 3, C for items corresponding to level 4, and C for items corresponding to level 5. The numbers entered for each evaluation item can be replaced with a grade, with A for items corresponding to levels B and 6, S for items corresponding to level 7, and SS for items corresponding to level 8.

In step S303, the device 100 divides the input data into grades of pitch, tempo, breathing, rhythm, treble, expressiveness, and completeness, a sound range evaluated as high tone (H) or low tone (L), and four types (FMSO). It can be determined by the type of voice evaluated as one of the following.

FIG. 6 is a diagram illustrating a process for determining a sound range and voice type according to an embodiment.

Referring to Figure 6, the vocal classification system according to the present invention may include a vocal range evaluated as high tone (H) or low tone (L) and a voice type evaluated as one of four types (FMSO), but is not limited to this. does not The vocal classification system according to the present invention is proposed for effective vocal type classification according to an embodiment of the present invention. FMSO largely classifies voice types into four types (FMSO) based on the degree of vocal cord grounding and breathing. It can be composed by mixing the HL system, which is subclassified into eight types by assigning high tone (H) or low tone (L) depending on the high and low tone of the system and voice.

Accordingly, the following vocal type classification and guide information for each type can be analyzed and processed from the song data.

HF: This is the type that breathes with the strongest force, and has a high-pitched, husky voice mixed with loud breathing.

LF: It uses the largest amount of breathing and can express husky and strong emotions with a heavy low tone at high volume.

HM: He has the highest level of vocalization among idols and is the type that can handle the highest register, with a high-pitched, clear and solid voice.

LM: This type of voice is low-toned, loud and clear, showing strength in low notes and capable of digesting a wide range of sounds from low to high notes.

HS: This is the type that uses the weakest breathing power, and the high-pitched, clear voice with breathing at a low volume is attractive.

LS: Although the voice is produced with light force, it has a low tone voice with the most breathing and a pure and faint emotional tone.

HO: A clear, high-pitched sound can be produced with the power of small breathing. They often have a clear and refreshing voice.

LO: This is a type that produces a soft, low-tone voice. It uses light breathing and uses the vocal cords efficiently to produce a clear sound.

Accordingly, when the vocal type is classified, the device 100 can determine the vocal grade corresponding to the vocal type classification, and the same labeling is also processed and learned in the learning data for the determination, so that it is displayed on the artificial intelligence learning-based evaluation model database. It can be built in advance.

Figure 7 is a diagram for explaining an output interface of a user's terminal according to an embodiment.

By executing the application, the user's terminal 200 can output the analysis results of the user's singing data and the resulting vocal training information through the display.

The item level standard table according to an embodiment of the present invention can be arranged so that the leftmost items have a higher basic vocal ability weight, so guides are provided preferentially starting from the left items necessary for acquiring basic vocal skills. This has the advantage of providing more effective guides and tips.

Figure 8 is a flowchart for explaining a process of additionally processing input data according to an embodiment.

The device 100 may perform a process of further processing input data using transliteration information. The sound range information may include, but is not limited to, the user's voice data and the user's singing data.

Referring to FIG. 8, first, in step S801, the device 100 may obtain the user's voice data from the user's terminal 200.

At this time, the song data is a type of voice data generated from musical activities including the user's vocal expression, and may include voice data recorded in the process of singing or singing.

Voice data may include voice data recorded while the user is speaking. Typically, song data is a subset of voice data and is specialized for musical activities such as singing. However, in the present invention, data excluding song data among voice data may be set as voice data.

According to one embodiment, in the process of acquiring song data and voice data from the user's terminal 200, the device 100 selects the type of data, whether song data or voice data, from the user's terminal 200. A page can be created and provided to the user's terminal 200, and when whether it is song data or voice data is selected through the selection page, the tag 'vocal' is added for song data, and 'voice' is added for voice data. You can save it to the database by adding the tag '.

In step S802, the device 100 may check the average vocal range of the user's voice using voice data.

According to one embodiment, the device 100 may perform noise removal and normalization processes on the user's voice data and convert the voice data into the time-frequency domain. The process of converting voice data into the time-frequency domain may include, but is not limited to, methods such as Short-time Fourier Transform (STFT) or Mel-Frequency Cepstral Coefficients (MFCC).

The device 100 can determine the main frequency band in which speech is mainly generated in the frequency domain and calculate the average vocal range of the user's voice using the main frequency band. At this time, the device 100 may calculate the average voice range by averaging the center frequency of the frequency band, but is not limited to this.

The user's average vocal range may reflect the vocal range mainly used by the user in daily life, such as when speaking or having a conversation.

In step S803, the device 100 may check the average vocal range of the user's singing using the singing data.

According to one embodiment, the device 100 may perform noise removal and normalization processes on the user's song data and convert the song data into the time-frequency domain. The process of converting singing data into the time-frequency domain may include, but is not limited to, methods such as Short-time Fourier Transform (STFT) or Mel-Frequency Cepstral Coefficients (MFCC).

The device 100 can identify the main frequency band in which singing mainly occurs in the frequency domain and calculate the user's average singing range using the main frequency band. At this time, the device 100 may calculate the average singing range by averaging the center frequency of the frequency band, but is not limited to this.

The user's average vocal range may reflect the vocal range of the song that the user mainly uses when singing.

In step S804, if the difference between the average vocal range of the voice and the average vocal range of the song exceeds a preset standard, the device 100 lowers the grade of pitch, tempo, breathing, rhythm, treble, expressiveness, and completeness by one level, and adjusts the vocal range correction items. You can add items to the input data. At this time, the preset standard may be set differently depending on the embodiment.

The device 100 may compare the average voice range and the average singing range to determine whether the difference between the average voice range and the average singing range exceeds a preset standard.

Depending on various factors such as the user's voice characteristics, vocal range ability, and vocal habits, differences may occur between the average voice range and the average singing range.

If there is a large difference between the average voice range and the average singing range, it may be difficult to accurately evaluate the user's voice and singing ability. If the criteria for vocal evaluation are unclear or do not reflect the difference between voice ability and singing ability, the user's voice and singing ability may be difficult to evaluate. Actual vocal ability may not be assessed accurately.

Additionally, when vocal training information is provided based on vocal evaluation results that are not accurately evaluated, a problem arises in that vocal training information that does not fit the user's vocal ability and individual voice characteristics may be provided. Since vocal training information requires different methods depending on the user's vocal evaluation, a problem arises in that the user may be provided with a vocal training service that does not suit the user's vocal ability, which may reduce the usefulness and satisfaction of the service provided to the user. .

Accordingly, if the difference between the average voice range and the average singing range exceeds the preset standard, the grade of pitch, tempo, breathing, rhythm, treble, expressiveness, and completeness is lowered by one level, and a range correction item is added to the input data. This is a necessary process to accurately reflect the user's vocal evaluation and provide personalized vocal training.

If the difference between the average voice range and the average singing range exceeds the standard, there is a high probability that the user's voice and the voice when singing will not match, so the device 100 measures the pitch, tempo, breathing, rhythm, high pitch, expressiveness, and completeness. By lowering the rating, the user's actual vocal ability can be assessed more accurately.

The device 100 can supplement the evaluation results by considering the difference between voice data and song data. The device 100 adjusts the vocal grade and evaluation grade to more accurately reflect the discrepancy between the user's vocal ability and singing ability through the vocal range correction item, thereby providing more accurate evaluation results based on the user's actual vocal ability. can help

The device 100 can provide personalized vocal training by determining the difference between the average voice range and the average singing range. The device 100 can improve the user's vocal range by adding a vocal range correction item to the input data, and provide more accurate and effective vocal training by adjusting grades such as pitch, tempo, breathing, rhythm, treble, expressiveness, and completeness. there is.

The device 100 can provide more appropriate and effective vocal training by adding a pitch correction item to the input data in consideration of the difference between the average voice range and the average singing range, and provides training tailored to the user's individual abilities and goals. can do.

In other words, the device 100 can perform a process to more accurately evaluate the user's vocal ability and provide personalized vocal training to help improve vocal ability. Accurate evaluation and personalized training that take into account differences in voice and singing ability help users achieve more effective vocal improvement.

In order to derive results from artificial neural networks, a correct and well-defined input dataset is essential. Especially when dealing with voice and singing ability, such as vocal training and evaluation, the input dataset must well include the vocalist's tendencies and characteristics. The device 100 can enable the artificial neural network to produce more accurate and effective results by correctly defining the input data set.

If the difference between the average voice range and the average singing range exceeds a preset standard, the device 100 indicates that the difference between the user's voice and singing is relatively large, and reflects this difference to lower the rating in the evaluation item. By doing so, the evaluation results can be made more fair and accurate.

In addition, if the difference between the average voice range and the average singing range exceeds a preset standard, the user's vocal ability and speaking tendency may be different, so the rating of the evaluation item is lowered by one level to more accurately reflect the user's vocal tendency. and provide personalized evaluation.

Additionally, the device 100 can provide more accurate training and feedback on specific items through grade downgrading, and users can receive more specialized training and vocal training tailored to their individual needs and demands. It can be done.

The vocal range correction item may refer to an item added to improve the user's vocal range in vocal training. In the process of obtaining the user's vocal grade using input data, the device 100 enters a vocal range correction item, which is information about whether vocal range correction is necessary, by considering the difference between the average vocal range of the user's voice and the average vocal range of the singing input data. It can be added to the items in .

At this time, the vocal range correction item may be displayed as O or If the difference in the average pitch range does not exceed the preset standard, the pitch correction item can be marked as O and added to the input data.

In vocal training, vocal range refers to the range of notes a user can sing, including high and low notes. The vocal range correction item can be used to provide training if the user is having difficulty with a specific vocal range or needs improvement. For example, if a user is having difficulty with high vocal ranges, the vocal range correction item can help them train and improve their higher vocal ranges.

In addition, if the difference between the average voice range and the average singing range exceeds a preset standard, the device 100 replaces the user's average voice range with a range in which the user can comfortably sing to perform vocal evaluation. You can. This allows users to sing with natural and comfortable expression, allows users to exercise their vocal abilities more freely, and conveys emotions and expressiveness more richly.

In addition, in the process of evaluating vocals, if the evaluation is conducted in a range that the user can sing comfortably, the evaluation will focus on the range that the user frequently sings, so vocal evaluation can be conducted more realistically.

If the user is evaluated in a comfortable vocal range, vocal abilities in that vocal range can be emphasized and brought to life. This allows for more effective vocal evaluation by focusing on the areas where the user can sing with the most confidence.

In addition, by performing an evaluation by replacing the vocal range to suit the individual, a vocal evaluation that reflects the user's personal tendency and style can be made, personalized vocal training and evaluation is possible, and training tailored to the user's abilities and goals is provided. You can let it go.

The device 100 can provide a positive experience by allowing the user to be evaluated in a comfortable vocal range, increase the user's confidence, promote positive motivation for vocal training, and contribute to continuous training and development.

Figure 9 is a flowchart for explaining a process for recommending practice content according to an embodiment.

Referring to Figure 9, first, in step S901, the device 100 may check the user's vocal genre from the song data.

According to one embodiment, the device 100 may provide a vocal training service customized as a result of the user's vocal evaluation, including a service that recommends the user's practice content based on singing data.

The song data may include vocal genre, singing time, song title, song lyrics, etc., and the device 100 may receive the song data from the user's terminal 200 and store it in a database.

The device 100 can determine which vocal genre the user's song data corresponds to from the song data. For example, vocal genres may include, but are not limited to, pop, jazz, classical, rock, R&B, hip hop, ballad, etc.

In step S902, the device 100 may extract main keywords from the song data.

According to one embodiment, the device 100 may extract key keywords from song data using natural language processing technology. The device 100 can extract text from song data and perform a text pre-processing process to remove unnecessary information from the song data. The text pre-processing process includes a process of removing punctuation marks, stop words, noise, etc. It can be included.

The device 100 can extract keywords from song data by dividing sentences into words through morphological analysis, and among the extracted keywords, keywords mentioned above a preset standard can be set as main keywords. At this time, the preset standard may be set differently depending on the embodiment. The process of extracting main keywords from song data is not limited to this and can be performed in various ways.

In step S903, the device 100 may collect vocal content of interest corresponding to a vocal genre from the vocal database.

Specifically, the device 100 may additionally include a vocal database or communicate with the vocal database wired or wirelessly, and the device 100 may extract vocal content from the vocal database.

At this time, the vocal database may refer to a database in which vocal content information is matched and stored, including information on the title of the vocal content, vocal genre, included songs, lyrics, form, time, source, etc.

The device 100 may check the vocal genre of the song sung by the user from the user's singing data and collect vocal content of interest corresponding to the vocal genre from the vocal database.

For example, if the song genre from the user's song data is a ballad, the device 100 may extract vocal content of interest, which is vocal content corresponding to the ballad, from the vocal database.

In step S904, the device 100 may determine whether vocal content including a main keyword is included among the vocal content of interest.

In step S905, if vocal content including a main keyword is included among the vocal content of interest, the device 100 may determine the vocal content including the main keyword as practice content.

That is, if there is vocal content that corresponds to the user's vocal genre and includes the main keyword, the device 100 may determine the vocal content that corresponds to the user's vocal genre and includes the main keyword as practice content.

The device 100 can provide the user with customized practice content tailored to the user's personal musical tastes and needs by recommending practice content that includes key keywords and belongs to a vocal genre in which the user is already interested, thereby allowing the user to Vocal content determined as practice content can help you feel more satisfied and practice more passionately and fruitfully.

In addition, it may take a lot of time and effort for users to find vocal content for practice. If vocal content that corresponds to the user's vocal genre and contains key keywords is selected in advance and determined as practice content, the user can use it more efficiently. You can learn and practice and get the musical experience you want while saving time.

In other words, determining vocal content that corresponds to the user's vocal genre and contains key keywords as practice content can help improve the user's learning experience and lead to more effective learning, and helps users improve their vocal skills. You can further maximize your musical enjoyment.

In step S906, if the vocal content of interest does not include vocal content including the main keyword, the device 100 checks similar keywords related to the main keyword, and determines whether vocal content containing the similar keyword is included among the vocal content of interest. You can judge whether it is or not.

If vocal content including a main keyword is not included among the vocal content of interest, the device 100 may check similar keywords related to the main keyword. For example, the device 100 may search a portal site for a main keyword and determine related search words that are displayed together when searching the portal site as similar keywords to the main keyword. Related searches that appear together when searching a portal site may indicate other keywords that are similar to or related to the main keyword.

For example, if the main keyword is love, the device 100 can access portal sites including Naver, Google, Daum, etc., search for 'love' on the portal site, and display it on the search results page of the portal site. You can check the automatically generated and displayed related search terms related to 'love' and determine them as similar keywords.

At this time, the process of determining similar keywords related to the main keyword is not limited to this and can be performed in various ways.

In step S907, if vocal content including a similar keyword is included among the vocal content of interest, the device 100 may determine the vocal content including the similar keyword as practice content.

The device 100 may determine vocal content including similar keywords among vocal content of interest as practice content.

If vocal content containing a main keyword is not included among the vocal contents of interest, the device 100 determines vocal content containing similar keywords related to the main keyword as practice content, thereby ensuring that the practice content is relevant to the user's interests and has a scope of practice. By providing expanded practice content to users, it is possible to provide a practice experience that stimulates the user's musical curiosity and arouses interest.

In step S908, if vocal content containing similar keywords is not included among the vocal content of interest, the device 100 may determine the vocal content most frequently registered by users with the same evaluation information as the user as practice content.

According to one embodiment, the device 100 checks the user's evaluation information when vocal content including similar keywords is not included among the vocal content of interest, and the number of items receiving X among the user's evaluation information is the same or , Vocal content registered the most by a user with the same vocal range or voice type can be determined as practice content.

Figure 10 is a flowchart for explaining the process of modifying practice content according to one embodiment.

Referring to FIG. 10, first, in step S1001, the device 100 can check the number of evaluation items that received an X from the evaluation information.

From the evaluation information, the device 100 can check the number of items that received an

In step S1002, the device 100 may determine whether the number of items receiving X is less than a preset first standard. At this time, the preset first standard may be set differently depending on the embodiment.

In step S1003, the device 100 may maintain practice content when the number of items receiving X is less than a preset first standard.

In step S1004, if the number of items receiving X is greater than or equal to a preset first standard, the device 100 may determine whether the number of items receiving At this time, the preset second standard may be set differently depending on the embodiment.

In step S1005, if the number of items receiving You can add

The apparatus 100 may extract similar users among all users who have the same number of items for which the user received an

For example, if the number of items for which a user received an Similar users who received O in the evaluation items of breathing and rhythm can be extracted.

The device 100 may extract vocal content from similar users and add it to the front of the practice content.

The device 100 extracts the vocal content of similar users who received an O for the items in which the user received an X while the number of items receiving an You can help users recognize their own shortcomings by referring to the vocal content of users who are at a similar level but have something to learn about areas where their skills are lacking, and by referring to the vocal content of other users, you can gain a new musical perspective. .

In addition, compared to providing vocal content from users who are much better than themselves, when providing vocal content from similar users, users can develop musical confidence and gain musical motivation, allowing users to create richer and more effective music. Learning experiences can be provided.

In step S1006, if the number of items receiving You can add

If the number of items receiving an By modifying the practice content, it is possible to provide practice content that allows users with relatively limited skills to experience and refer to a variety of vocal content.

Figure 11 is a flow chart to explain the process of determining a trainer as a matching partner for a user according to an embodiment.

According to one embodiment, when the first song is identified as the user's needs, the device 100 provides a training page including the song, finds and matches a trainer who trained the song, and then transfers the matched trainer to the user. I can recommend it to you. Here, the trainer may mean a trainer who trains a song.

Referring to FIG. 11, first, in step S1101, the device 100 may receive content including a song desired by the user from the user's terminal 200. Here, the content may be content created by being photographed through the user's terminal 200, or may be content acquired externally and stored in the memory of the user's terminal 200.

Specifically, the user can select content including a song he or she wants through the user's terminal 200, and when the content is selected, the user's terminal 200 can transmit the content to the device 100.

In step S1102, the device 100 may analyze the content as content including the first song.

That is, when the device 100 receives content, it can analyze which song the content includes. For example, as a result of comparing the content learned for each song with the content, if it is confirmed that the content matches to some extent the content of the first song, the device 100 analyzes the content as content including the first song. You can.

Additionally, the device 100 may compare the content with the content learned for each song, and if it is confirmed that the content matches the content of the first song to some extent, it may analyze the content as content including the first song.

As described above, the device 100 may analyze whether the content includes the first song based on the content learned in advance.

Additionally, if the first song is recognized in the content as a result of analyzing the content, the device 100 may analyze the content as content including the first song.

In step S1103, if the content is analyzed as content including the first song, the device 100 may classify trainers with a history of training the first song into the matching candidate group. To this end, trainers can register the songs they trained on during song training in the training history, and the device 100 can store and manage the training history classified by trainer in the database. The training history is accumulated information on the details of training tasks performed and completed by the trainer, and may consist of a list of information including the trained song, song training date, training location, training site, and training results.

For example, when the first trainer trains the first song, the trainer can generate the first training result by shooting a video of the song training completed through the terminal, and request registration of the generated first result. can be transmitted to the device 100, and the device 100 can recognize the first song in the first result and then register the first song in the training history as the song trained during training.

As described above, when the trainer completes song training, a training result can be generated by shooting a video of the song training completed, and the song used during training is recognized through the generated training result, and the recognized song is used during training. A song can be registered in the training history, and when this process is performed for each trainer, each trainer's training history can be updated.

Based on the trainers' training history, the device 100 checks whether the first song is registered as a song used during training in the trainer's training history and determines whether there is a history of training using the first song. This can be confirmed, and trainers who have a history of performing training using the first song can be classified into the matching candidate group.

In step S1104, the device 100 may check whether there is a trainer classified into the matching candidate group.

If it is confirmed in step S1104 that there is no trainer classified as a matching candidate group, the device 100 sends a notification message requesting selection of other content to the user's terminal (200) because the user and trainer matching service cannot be provided through the content. ), and if other content replacing the content is selected through confirmation of the notification message, the process returns to step S1101 and the device 100 can perform the process again from receiving the other selected content.

If it is confirmed in step S1104 that there is a trainer classified as a matching candidate group, in step S1105, the device 100 may check whether there is only one trainer classified as a matching candidate group.

If it is confirmed in step S1105 that the first trainer is the only trainer classified as a matching candidate group, in step S1107, the device 100 may set the first trainer as the user's matching partner.

That is, when it is confirmed that there is only one trainer classified into the matching candidate group, the device 100 may set the first trainer as the user's matching partner.

If it is confirmed in step S1105 that there are two or more trainers classified into the matching candidate group instead of one, the device 100 may calculate a matching score for each trainer classified into the matching candidate group in step S1106. At this time, the device 100 may calculate a matching score for each trainer classified into the matching candidate group by considering the number of training sessions performed using the first song and the upload amount of content including the first song. The number of training sessions performed using the first song can be confirmed through the training history, and the upload amount of content including the first song can be confirmed through the upload history. To this end, the device 100 may store and retain upload details classified by trainer in a database. The upload history can indicate the upload status of each song uploaded to the trainer's SNS or website, and can be updated at the trainer's request.

Specifically, the device 100 sets the first score to a higher score as the number of training sessions using the first song increases, and sets the second score to a higher score as the upload amount of content including the first song increases. After setting, the matching score can be calculated by adding the first score and the second score.

For example, the first trainer and the second trainer are classified as a matching candidate group, the number of times training was performed using the first song is registered in the first trainer's training history 5 times, and the first trainer's upload The upload amount of the first song is registered as 10 in the history, the number of times training was performed using the first song is registered as 3 in the training history of the second trainer, and the number of uploads of the first song is registered in the upload history of the second trainer. If the upload amount is registered as 8, the device 100 calculates the first score as 50 points through the number of trainings performed 5 times using the first song for the first trainer and calculates the upload amount of the first song After calculating the second score as 100 points through 10, you can calculate the second trainer's matching score as 150 points by adding 50 points and 100 points, and train the second trainer using the first song. The first score is calculated as 30 points through the number of training performed 3 times, and the second score is calculated as 80 points through the upload amount of 8 songs of the first song, and then the 30 points and 80 points are added together to determine the second trainer. The matching score can be calculated as 110 points.

That is, when it is confirmed that there are two or more trainers classified into the matching candidate group, the device 100 takes into account the number of training sessions performed using the first song and the upload amount of content including the first song, Matching scores can be calculated for each trainer classified into a matching candidate group.

When the matching score is calculated for each trainer classified into the matching candidate group, the device 100 can compare the calculated matching scores to check which trainer's matching score is the highest.

If it is confirmed that the first trainer has the highest matching score among the trainers classified into the matching candidate group, in step S1107, the device 100 may set the first trainer as the user's matching partner.

In step S1108, if the first trainer is set as the user's matching partner, the device 100 may provide recommendation information about the first trainer and link information of a training page including the first song to the user's terminal 200. there is.

When the first trainer is set as the user's matching partner, the process of providing recommendation information about the first trainer and link information to the training page including the song to the user's terminal 200 improves the user's convenience and personalized experience. This allows the song's trainer selection process to be performed smoothly.

The process of providing recommended information about the trainer of the device 100 and link information to the training page of the first song can improve convenience and accessibility for the user, and the user can easily obtain recommended information and songs related to the first trainer. This allows you to purchase the songs you want more quickly and efficiently.

Additionally, the device 100 may provide an opportunity to discover and select a trainer related to the first song that suits the user by providing recommendation information, and the user can select a trainer that more accurately reflects the user's preferences.

Additionally, the device 100 provides link information for the training page so that the user can directly go to the training page of the first song and check the trainer's information. In addition, it has the effect of encouraging active singing training.

Figure 12 is a flow chart to explain the process of confirming the location of the trainer according to one embodiment.

Referring to FIG. 12, first, in step S1201, the device 100 may obtain a song history in which the user uploads song data from the user's terminal 200. Here, the song history is a history of when the user uploaded song data in the past and how much song data was uploaded.

The device 100 may receive a singing history from the user's terminal 200, match the received singing history with the user, and store it in a database.

Specifically, when the user seeks a trainer and wants to match the trainer to the user, the device 100 can obtain a singing history, which is a history of the user singing a song in the past and uploading singing data. At this time, the device 100 may obtain the singing history from the user's terminal 200, and the device 100 may obtain the singing history through a user database provided by the device 100.

The user database stores the singing history, which is the history of the user uploading singing data in the past by matching with the user. That is, the user database stores which users uploaded which songs, when and how much they sang in the past, and reviews about trainers. Information about when and how much content was created is stored.

That is, the device 100 provides information about which users uploaded which songs, when and how much they sang in the past, and when and how much review content about the trainer was written through the user's terminal 200 or the user database. You can obtain singing history.

In step S1202, the device 100 may generate the user's singing cycle based on the singing history.

Specifically, the device 100 may check the date on which the song data was uploaded based on the song history and generate the user's song cycle.

For example, when a user seeks a trainer and wants to match the trainer, the device 100 can obtain a song history, which is a history of the user uploading song data, through the user's terminal 200 or a user database, and Through the history, you can see that the date the song data was uploaded is February 3, 2023, March 3, 2023, April 3, 2023, and the current date, May 3, 2023, respectively, and based on the created date As a result, the song cycle, which is the upload cycle of song content, can be created as a month.

In step S1203, the device 100 may check the first date, which is the user's next scheduled singing date, based on the current date and the user's singing cycle.

For example, if the user wants to match a trainer, the current date is May 3, 2023, and the user's singing cycle is one month, the device 100 selects 6, 2023 as the first date, which is the user's next scheduled singing date. You can check the 3rd day of the month.

In step S1204, the device 100 may determine whether a trainer exists whose lesson availability period is the first date.

Specifically, the device 100 may determine whether there is a trainer whose lesson availability period is the first date among trainers through the trainer database.

The device 100 can check the trainer's location through the trainer database.

Specifically, the device 100 matches the user among the trainers through the trainer database based on the user's trainer matching information including the name of the trainer the user wishes to match and the number of trainers obtained from the user's terminal 200. You can check the location of the trainer, which is information about where the trainer you want to use is located.

Here, the device 100 may be provided with a trainer database or may communicate with a separate trainer database wired or wirelessly. The trainer database may store information about the trainer by matching with the trainer, and the information about the trainer includes the trainer's information. This may include the name, the trainer's lesson availability period, the number of trainers, the trainer's location, the trainer's training location, and the trainer's vocal genre.

If it is confirmed in step S1204 that the trainer whose lesson availability period is the first date exists as the first trainer, in step S1205, the device 100 determines to match the first trainer to the user, and determines the location of the first trainer. You can check.

Specifically, the device 100 determines whether there is a trainer whose lesson availability period is the first date among the trainers through the trainer database. As a result, the trainer whose lesson availability period is the first date among the trainers is the first trainer. If it is confirmed that it exists, it is determined that the first trainer will be matched to the user, and the location of the first trainer can be confirmed through the trainer database.

For example, if the user wants to match a trainer, the current date is May 3, 2023, and the singing cycle is one month, the device 100 selects June 3, 2023 as the first date, which is the user's next scheduled singing date. You can check the work, and through the trainer database, check whether there is a trainer whose lesson available period is June 3, 2023, and confirm that the trainer whose lesson available period is June 3, 2023 exists as the first trainer. If so, it is determined that the first trainer will be matched to the user, and the location of the first trainer can be confirmed through the trainer database.

If it is confirmed in step S1204 that there is no trainer whose lesson availability period is the first date, in step S1206, the device 100 may generate second dates that fall before a preset range based on the first date. . Here, the preset range may vary depending on the embodiment, and the range may become wider as the user's singing cycle becomes longer, and may become narrower as the user's singing cycle becomes shorter, depending on the user's singing cycle.

Specifically, the device 100 determines whether there is a trainer whose lesson availability period is the first date among the trainers through the trainer database, and determines that there is no trainer among the trainers whose lesson availability period is the first date. Once confirmed, second dates that fall before the preset range from the first date can be generated.

For example, if the user wants to match a trainer, the current date is May 3, 2023, the singing cycle is one month, and the preset range is 3 days, the device 100 selects the first scheduled singing date of the user. You can check the date as June 3, 2023, and check through the trainer database whether there are any trainers whose lesson availability period is June 3, 2023. Among the trainers, which trainers have the lesson availability period on June 3, 2023? If it is confirmed that does not exist, the second dates that fall within 3 days, a preset range based on the first date, June 3, 2023, are May 31, 2023, June 1, 2023, and June 6, 2023. You can create 2 days in a month.

In step S1207, the device 100 may determine whether a trainer exists whose lesson availability period is the second date.

Specifically, the device 100 may determine whether there is a trainer whose lesson availability period is a second date, one of the second dates, among trainers through the trainer database.

If it is confirmed in step S1207 that the trainer whose lesson availability period is the second date exists as a second trainer, in step S1208, the device 100 determines to match the second trainer to the user, and determines the location of the second trainer. You can check.

Specifically, the device 100 determines whether there is a trainer whose lesson availability period is the second date among the trainers through the trainer database, and as a result, the trainer whose lesson availability period is the second date among the trainers is the second trainer. If it is confirmed that it exists, it is determined that the second trainer will be matched to the user, and the location of the second trainer can be confirmed through the trainer database.

For example, if the user wants to match a trainer, the current date is May 3, 2023, the singing cycle is one month, and the preset range is 3 days, the device 100 selects the first scheduled singing date of the user. You can check the date as June 3, 2023, and check through the trainer database whether there are any trainers whose lesson availability period is June 3, 2023. Among the trainers, which trainers have the lesson availability period on June 3, 2023? If it is confirmed that does not exist, the second dates that fall within 3 days, a preset range based on the first date, June 3, 2023, are May 31, 2023, June 1, 2023, and June 6, 2023. You can create 2 days per month, and check through the trainer database whether there is a trainer whose lesson availability period is one of May 31, 2023, June 1, 2023, or June 2, 2023. If it is confirmed that a trainer with a lesson availability period of May 31, 2023, June 1, 2023, or June 2, 2023 exists as a second trainer, it is determined that the second trainer will be matched to the user. You can check the location of the second trainer through the trainer database. In this process, the device 100 determines that there are all trainers whose lesson availability periods are May 31, 2023, June 1, 2023, and June 2, 2023. It is possible to match a trainer to a user as of May 31, but is not limited to this.

If it is confirmed in step S1207 that there is no trainer whose lesson availability period is the second date, in step S1209, the device 100 selects the trainer whose lesson availability period is the earliest of the dates after the first date among the trainers. 3 trainers can be checked, it is determined that a third trainer will be matched to the user, and the location of the third trainer can be confirmed.

Specifically, the device 100 determines whether there is a trainer whose lesson availability period is the second date among the trainers through the trainer database, and determines that there is no trainer among the trainers whose lesson availability period is the second date. Once confirmed, the third trainer whose lesson availability period is the earliest after the first date among the trainers is confirmed, it is determined that the third trainer will be matched to the user, and the location of the third trainer can be confirmed. .

For example, if the user wants to match a trainer, the current date is May 3, 2023, the singing cycle is one month, and the preset range is 3 days, the device 100 selects the first scheduled singing date of the user. You can check the date as June 3, 2023, and check through the trainer database whether there are any trainers whose lesson availability period is June 3, 2023. Among the trainers, which trainers have the lesson availability period on June 3, 2023? If it is confirmed that does not exist, the second dates that fall within 3 days, a preset range based on the first date, June 3, 2023, are May 31, 2023, June 1, 2023, and June 6, 2023. You can create 2 days per month, and check through the trainer database whether there is a trainer whose lesson availability period is one of May 31, 2023, June 1, 2023, or June 2, 2023. If it is confirmed that there is no trainer whose lesson availability period is one of May 31, 2023, June 1, 2023, or June 2, 2023, among the trainers, the lesson availability period is after June 3, 2023. The third trainer, which is the trainer with the earliest date among the dates, is checked, it is determined that the third trainer will be matched to the user, and the location of the third trainer can be confirmed through the trainer database.

In addition, if it is determined in this process that there is no trainer whose lesson available period is after the first date, the device 100 selects the trainer with the longest lesson available period among trainers whose lesson available period is before the first date to the user. can be provided.

For this reason, the device 100 determines the date on which the user next wants to match a trainer through the user's singing cycle, and matches the trainer with a lesson availability period up to that date, thereby providing a trainer with a lesson availability period suitable for the user. There is a matching effect.

Figure 13 is an exemplary diagram of the configuration of the device 100 according to one embodiment.

Device 100 according to one embodiment includes a processor 101 and memory 102. Device 100 according to one embodiment may be the server or terminal described above. The processor 101 may include at least one device described above with reference to FIGS. 1 to 12 or may perform at least one method described with reference to FIGS. 1 to 12 . The memory 102 may store information related to the above-described method or store a program in which the above-described method is implemented. Memory 102 may be volatile memory or non-volatile memory.

The processor 101 can execute programs and control the device 100. The code of the program executed by the processor 101 may be stored in the memory 102. The device 100 is connected to an external device (eg, a personal computer or a network) through an input/output device (not shown) and can exchange data.

The embodiments described above may be implemented with hardware components, software components, and/or a combination of hardware components and software components. For example, the devices, methods, and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, and a field programmable gate (FPGA). It may be implemented using one or more general-purpose or special-purpose computers, such as an array, programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. A processing device may execute an operating system (OS) and one or more software applications that run on the operating system. Additionally, a processing device may access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a single processing device may be described as being used; however, those skilled in the art will understand that a processing device includes multiple processing elements and/or multiple types of processing elements. It can be seen that it may include. For example, a processing device may include a plurality of processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are possible.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the medium may be specially designed and configured for the embodiment or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes optical media (magneto-optical media) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing unit to operate as desired, or may be processed independently or collectively. You can command the device. Software and/or data may be used on any type of machine, component, physical device, virtual equipment, computer storage medium or device to be interpreted by or to provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

Although the embodiments have been described with limited drawings as described above, those skilled in the art can apply various technical modifications and variations based on the above. For example, the described techniques are performed in a different order than the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form than the described method, or other components are used. Alternatively, appropriate results may be achieved even if substituted or substituted by an equivalent.

Therefore, other implementations, other embodiments, and equivalents of the claims also fall within the scope of the following claims.

Claims (3)

In a method performed by a device,
Obtaining the user's song data from the user's terminal;
Receiving evaluation information for each item about the song data;
Processing the evaluation information to generate input data;
Obtaining the user's vocal rating by inputting the input data into an artificial neural network; and
Including; providing vocal training information matched to the vocal grade,
The evaluation items included in the above evaluation information are:
Pitch, tempo, breathing, rhythm, treble, expressiveness, and completeness evaluated as O or Including,
Further comprising recommending practice content for the user based on the singing data,
The step of recommending the practice content is,
Confirming the user's vocal genre from the song data,
Extracting main keywords from the song data,
Collecting vocal content of interest corresponding to the vocal genre from a vocal database,
determining whether vocal content including the main keyword is included among the vocal content of interest;
If vocal content including the main keyword is included among the vocal content of interest, determining the vocal content including the main keyword as practice content;
If vocal content including the main keyword is not included among the vocal content of interest, checking similar keywords related to the main keyword,
If vocal content including the similar keyword is included among the vocal content of interest, determining the vocal content including the similar keyword as practice content;
If vocal content including the similar keyword is not included among the vocal content of interest, determining vocal content most frequently registered by users with the same evaluation information as the user as practice content,
Based on the evaluation information, further comprising modifying the practice content,
The step of modifying the practice content is,
From the evaluation information, confirming the number of evaluation items that received X,
If the number of items receiving X is less than a first preset standard, maintaining the practice content;
Extracting similar users among users whose evaluation items corresponding to the items for which the user received
If the number of items receiving the
If the number of items receiving X is greater than the second standard, extracting excellent users corresponding to the number of items receiving step, and
Comprising the step of extracting the vocal content of the similar user and the vocal content of the excellent user corresponding to the practice content and adding them to the front and back of the practice content, respectively.
Method of providing customized vocal training service as a result of user vocal evaluation. According to paragraph 1,
The step of processing the evaluation information to generate input data is,
Among the above-mentioned pitch, tempo, breathing, rhythm, treble, expressiveness, and completeness, enter the same value as the level for the items that received an O, and for the items that received an Entering 1,
A step of determining a grade according to the numbers entered for the pitch, tempo, breathing, rhythm, treble, expressiveness, and completeness, and
A step of determining the input data into grades of pitch, tempo, breathing, rhythm, high pitch, expressiveness, and completeness, a vocal range evaluated as high tone or low tone, and a voice type evaluated as one of four types,
Further comprising the step of further processing the input data using transliteration information,
The step of further processing the input data is,
Obtaining the user's voice data from the user's terminal,
Confirming the average vocal range of the user's voice using the voice data,
Using the song data, confirming the user's average singing range, and
If the difference between the average vocal range of the voice and the average vocal range of the song exceeds a preset standard, the grade of the pitch, tempo, breathing, rhythm, treble, expressiveness, and completeness is lowered by one level, and a vocal range correction item is added to the input data. steps of adding to, including,
Method of providing customized vocal training service as a result of user vocal evaluation. delete