KR100989867B1 - An automatic song transcription method - Google Patents

Abstract

Disclosed is a method for automatic song taking, extracting a pitch from a voice signal input through a voice signal input unit such as a microphone, and dividing a continuous voice signal into syllable units based on the extracted pitch data. After preprocessing to remove the noise of the syllables and provide a representative pitch value of the syllables, and convert the divided syllables into effective unit sections capable of sound field recognition, perform clustering by each note unit to perform sound field recognition, and then divide the first syllables. After reconstructing the relative pitch frequency table based on the representative pitch value of, map recognition is performed by mapping the representative pitch values of syllables based on the reconstructed relative pitch frequency table, and the threshold value is applied to the extracted pitch data. After extracting the rest period which is the non-voice interval of, using the extracted rest information, the node is detected and the sound field is And it generates the music data based on the expression information, pitch information recognition, word detection information. Therefore, the present invention can perform noise reduction and effective region division when the syllable is divided, and can recognize the sound field irrespective of the individual's speech rate, and it is possible to recognize the pitch suitable for the individual pitch through the pitch recognition through the relative pitch. By detecting the nodes, accurate scores can be generated.

Song, Grading, Pitch Extraction, Syllable Splitting, Sound Field Recognition, Pitch Recognition, Node Detection

Description

An automatic song transcription method

The present invention relates to an automatic song retrieval method.

More specifically, the present invention relates to an automatic song retrieval method for automatically generating sheet music data from human song.

For mankind, song is a cultural phenomenon that has existed for a long time, a means of expressing emotions of individuals and social groups and an instrument of amusement. Songs belong to the category of voice and are similar to ordinary words in that they are expressed through vocal organs and have a linguistic appearance, but songs, pitch, volume, tone, tone, etc. It has a musical attribute, so it is different from general words. Music recognition is one of the fields of speech recognition, which is the study of recognizing the pitch, time, and lyrics of music played by human songs or instruments, or by recognizing the name of a song by comparing existing song data with input songs. Each feature of the song is used to finally express it in the form of the desired data.

This music recognition can be used not only in the field of sight education through the song and in the entertainment field for leisure activities, but also as a tool for plagiarism check to protect the copyright of the song which is recently important. However, despite the development of computer performance for various information means, studies on tune recognition have been minimal.

An automatic grading system, which is a field of tune recognition in speech processing, recognizes the height (pitch, interval), length (sound field, duration), and lyrics of a song played by human voices and musical instruments. As a result of expressing the result in the form of sheet music, the system automatically recognizes the musical characteristics of the song of the viewer and compares the score with the expert who is familiar with the existing music. It can be used easily even by the general public.

[0003] The presently known automatic grading system combines segmented sections by phoneme to be used as boundary information of syllables to divide feature information extracted from continuous speech signals into individual notes. There is a method of forming a syllable segment using energy information obtained by connecting the maximum value of the voice generated every time.

However, in the above-described conventional automatic grading system, due to the continuous characteristics of the speech signal, the efficiency is remarkably decreased in the part where the boundary is ambiguous when the syllable is divided, and the pitch information is represented to recognize the pitch at each section of the predicted syllable boundary. The process of finding the value should be added, and since it is impossible to detect the node, it can not represent the result of the song in the form of a complete score, and thus has a problem in that it cannot provide satisfactory results.

In addition, although the speed of a person's song usually has different speed and time, the individual difference is very large. However, the conventional method of sound field recognition uses a method of mapping to standard notes based on generalized standard data. Because of this, there is a disadvantage in that each person cannot adapt to the speed of different song input.

SUMMARY OF THE INVENTION An object of the present invention is to provide an automatic song retrieval method for automatically recognizing a sound field and a pitch from a voice signal such as a person's song, and automatically detecting a measure to finally generate sheet music data so as to solve the above problems. To provide.

Another object of the present invention is to convert the song of a person into a voice signal, and then to perform pitch recognition based on efficient pitch extraction and syllable division, and to recognize the sound field based on the relative pitch and the relative pitch. The present invention provides an automatic song collecting method for generating accurate sheet music data by detecting nodes using pause information of a signal.

The automatic song taking method according to the present invention for achieving this object comprises the steps of: (1) extracting the pitch indicating the height of the voice from the voice signal input through the voice signal input unit, such as a microphone, (2) (1) Based on the pitch data extracted in the step, the continuous speech signal is divided into syllable units, the noise of the speech signal is removed, a representative pitch value of the syllables is provided, and (3) (2) are divided. Performing preprocessing to convert the syllables into effective unit sections capable of sound field recognition, clustering the preprocessed sound field data into units of each note, and performing sound field recognition; and (4) step (2) of the first syllable divided Reconstructing a relative pitch frequency table based on the representative pitch value, and performing pitch recognition by mapping a representative pitch value of a syllable based on the reconstructed relative pitch frequency table; (5) applying a threshold value to the pitch data extracted in step (1) to extract a rest period, which is a non-speech interval below the threshold value, detecting a node using the extracted rest period information, and (6) (3 And generating sheet music data based on the sound field recognition information, the pitch recognition information, and the node detection information performed through the steps (4) and (5).

delete

As described above, according to the automatic song picking method of the present invention, the syllable segmentation is performed based on the stabilized despreading equation to remove the noise and the effective region segmentation, and at the same time obtain the representative pitch value of the segmented syllable, and the genetic algorithm By recognizing the note length, clustering each note with a similar utterance time enables note length recognition regardless of the voice speed of the speaker, and when recognizing the note based on the standardized data, It is possible to recognize the pitch suitable for the pitcher's pitch by mapping the basic information of the pitch obtained at the time of division through relative pitch, and detecting node using pause information of voice signal unlike the conventional grading method which composed the score based on the pitch and beat information. So that when you're compiling a song It is possible to create sheet music.

In addition, the present invention can be applied to a personal communication terminal such as a mobile phone, such as an automatic composition system through voice, and furthermore, it can be applied to an emotional search system that analyzes a user's voice feature amount.

Hereinafter, with reference to the accompanying drawings will be described in detail the automatic song taking method of the present invention.

1 is a block diagram schematically showing a configuration to which the automatic song picking method according to the present invention is applied.

As shown in the drawing, the apparatus to which the automatic song collection method for automatically generating the music data from the song of a person is applied includes a voice signal input unit 10, a pitch extractor 20, a syllable divider 30, and a preprocessor ( 40, the sound field recognition unit 50, the pitch recognition unit 60, the pause extraction unit 70, the node detection unit 80, the score generation unit 90 and the like.

The voice signal input unit 10 includes a microphone, and converts an analog voice signal input from the outside into a digital voice signal, and then outputs the digital voice signal to the pitch extraction unit 20.

The pitch extractor 20 is a part for extracting feature values of a voice signal input through the voice signal input unit 10. The pitch extractor 20 extracts a pitch representing a height of the voice signal and divides the extracted pitch data into a syllable divider 30. ) And the pause extractor 70.

To extract the pitch, we use autocorrelation function, which is used to detect periodicity in general periodic signals. The autocorrelation function represents the correlation between the signal value at one time and the signal value at another time. The signal resulting from the calculation highlights the periodic part of the signal, allowing the pitch to be measured.

The autocorrelation function is defined by Equation 1 below.

Therefore, the autocorrelation coefficient can be regarded as the sum of all n values of the sample x (n) multiplied by the value of the sample at a certain point n from each other.

FIG. 2 is a diagram illustrating an example of extracting a pitch by applying an autocorrelation function of Equation 1 to an input voice signal, wherein the detected pitch becomes basic information of an automatic song collecting method of the present invention. A series of steps are performed, such as recognition, pitch recognition, node detection, and the like.

The syllable divider 30 removes noise of a continuous voice signal based on the pitch data extracted by the pitch extractor 20 and at the same time a syllable (the syllables represent one note each). It is divided into a) unit, to recognize the representative pitch value of the syllable, and outputs the divided syllable data to the preprocessor 40, the pitch recognition unit 60.

In this case, the syllable divider 30 uses a stabilized inverse diffusion equation to separate the pitch data extracted through the pitch extractor 20 into units capable of recognizing the pitch and the sound field of the note.

The stabilized despreading equation is an algorithm generally used for object segmentation in the field of image processing, and is very robust against noise and exhibits stable segmentation results. Therefore, in the present invention, in order to remove noise of a speech signal and divide a continuous signal into syllable units use.

The F function used in the stabilized despreading equation to divide the syllables can be defined by Equation 2 below.

In the present invention, the following equation (3) satisfying equation (2) is used as the F function.

The multi-scale filtering method based on the stabilized despreading equation is for the purpose of region division and an equation for obtaining a change in region value is used as shown in Equation 4 below.

The syllable segmentation based on the stabilized despreading equation performs the following processes sequentially. The number of iterations I of the algorithm, which is an initial condition, can be arbitrarily set. However, the number of repetitions (I) is preferably set to an optimized value through several experiments, and the experiment sequentially increases the value of the number of repetitions (I) while the syllable division according to each repetition number (I). Sensory test methods can be applied in which the person hears the results and selects the most natural one.

Step 1—Pitch data extracted by the pitch extractor 20 is set as an independent region. At this time, the area m _i of each area, the value u _i of the area, and the number N of divided areas are calculated.

Step 2-The change rate value according to the scale is updated through Equation 4 when filtering of each area is performed until two or more values of adjacent areas are equal.

Step 3-Merges adjacent areas with the same value. At this time, the area m _i of each region, the value u _i of the region, and the number N of divided regions are updated.

Perform steps 4-2 and beyond. At this time, if the repetition number I satisfies the initial setting value, the algorithm is terminated.

The segmented results show that when the segmentation is based on the stabilized despreading equation, the results differ depending on the σ used in the F function. As can be seen from Equations 3 and 4, if the value of σ is large, it is characterized by decreasing the number of repetitions, noise reduction, and edge loss.However, if the value of σ is small, it is characterized by increasing the number of repetitions, noise preservation, and edge preservation. Have Therefore, for efficient syllable segmentation, we apply the method to increase the σ value at the beginning to remove noise, improve convergence speed, and increase the performance by decreasing σ value as the segmentation progresses.

FIG. 3 is a diagram illustrating an example of a result of performing syllable division using the stabilized despreading equation of the present invention based on the pitch data of FIG. 2, which effectively divides and merges an ambiguous syllable boundary in a speech signal, and produces noise. At the same time, we can find the representative value of the pitch needed to recognize the pitch. In addition, the performance can be improved by reducing the number of repetitions so that the time required can be improved, and it can be seen that an area which is not close to a predetermined threshold after division is considered invalid as a note and is included in the preceding syllable.

The preprocessor 40 recognizes the syllables divided by the syllable divider 30 in order to recognize the sound field of the note (represented along with the pitch and expressed according to a certain rule within a word and is essential information in recognizing the tune). Converted to the basic unit and output to the sound field recognition unit 50.

4 is a view for explaining the concept of the duration, pause and IOI of the pre-processing process of the present invention, the voice signal is a pause (duration) when the voice is spoken and a pause caused by the person breathing or pronunciation reasons ( pause). Even when a person sings, a voice break occurs, and the sum of the talk duration and the pause can be defined as an inter onset interval (IOI). Here, the start time of the utterance duration and the end time of the pause period means the difference between the start time of each note and the start time of the next note. In the sound field recognition of the present invention, since the performance obtained by using the IOI-modified data is better than the existing duration data, it is used through the preprocessing process of converting the syllable data into the IOI. FIG. 5 is a diagram illustrating an example of syllable length information transformed into an IOI based on syllable segmentation data of FIG. 3.

The sound field recognition unit 50 performs a sound field recognition by clustering the preprocessed sound field data input from the preprocessing unit 40 into each note unit by using a genetic algorithm, and generates music score recognition information. Output to the unit 90.

Genetic algorithms are a general class of search methods that balance the possible areas of search and solution. While other existing search methods are at risk of falling into local minimum before finding the optimal value in the search space, genetic algorithms The present invention has been applied in that the risk of falling into local minima can be solved by maintaining a population that is likely to be harmful and all of them simultaneously find optimal values.

The clustering procedure using the genetic algorithm performed in the sound field recognition unit 50 is as follows.

Step 1-Construct chromosomes and initialize the population based on the extracted syllable data.

와 클러스터 센터간의 거리(d_min)를 최소화하는 클러스터링을 수행한다. 거리를 최소화하는 수학식은 다음의 수학식 5와 같다.Step 2-Each object based on the center of the initial cluster

Clustering is performed to minimize the distance d _min between the cluster and the cluster center. Equation 5 for minimizing the distance is shown in Equation 5 below.

In order to evaluate the goodness of fit of the results of Step 3 and Step 2, D _inter and D _{intra, which} are the distance between clusters, are defined using Equations 6 and 7 below.

이고,

이다.At this time,

ego,

to be.

는 현재 클러스터와 그 센터,

는 나머지 클러스터와 각각의 센터를 말하며, 적합도 함수는 다음의 수학식 8로 정의한다.

Is the current cluster and its center,

Denotes the remaining clusters and their respective centers, and the fitness function is defined by Equation 8 below.

The result of the goodness-of-fit function computed in steps 4-3 performs a series of steps: The roulette wheel method is used to select one point crossover the resulting chromosomes, and finally, a new generation is created using a given mutation coefficient. Then, the algorithm after step 2 is performed again.

The pitch recognition unit 60 reconstructs the relative pitch frequency table based on the representative pitch value of the first syllable divided by the syllable divider 30, and maps the representative pitch value of the syllable based on the reconstructed relative pitch frequency table. Pitch recognition is performed, and the pitch recognition information is output to the score generation unit 90.

Conventional pitch recognition method recognizes human voice using standard frequency table based on absolute pitch, but this method does not adapt to change of human pitch. Therefore, the pitch recognition of the present invention uses the concept of a relative interval (relative interval). Relative pitch is a method of determining the pitch by measuring the relative change with the previous note. It measures how much the previous note has changed through the relative pitch, and approximates the change to be applied to the 12th scale.

In order to recognize the relative pitch, the representative pitch value of the first syllable is regarded as the reference sound, and the relative pitch frequency table for each song is reconstructed using the ratio of the average rate scale. The average rate scale is based on the 12 scales used in the determination of the pitch, as shown in FIG.

The 12 scale system is closely related to the human auditory limits, and the relationship between frequencies within the 12 scale is a linear multiple of the relationship. The frequency ratio of the 1 degree difference sound is expressed by the following equation (9).

Therefore, if the octaves are 12 semitones and all semitones are exactly the same size, the pitches of C to C # are the same as the pitches of C # to D. If the pitch value is a and the frequency of C is 1, C # is the relative frequency a, and D is a ² multiplied by a.

As shown in FIG. 7, the average rate scale based on the frequency ratio may be divided into a total of 12 according to each pitch by Equation 9 described above.

The procedure of pitch recognition performed by the pitch recognizer 60 using the average rate scale of FIG. 7 is as follows.

First, the first syllable divided by the syllable divider 30 is regarded as a reference sound, and the pitch is confirmed through the representative pitch value of the first syllable. Then, the determined pitch is divided by the average rate scale shown in FIG. Check it. After checking the 12 scales of the relative pitch frequency table by multiplying the confirmed C frequency by the average rate scale, the relative pitch frequency table is reconstructed as shown in FIG. 8 is a diagram showing an example of a table of relative pitch frequencies when the first note is a 'sol' in a standard pitch and the corresponding frequency is 200 Hz. The pitch of the syllables may be recognized by mapping the representative pitch values of the syllables divided by the syllable divider 30 based on the reconstructed relative pitch frequency table. As such, when the relative pitch is used, there is an advantage that the individual difference caused by the difference in the ranges of each individual does not have to be considered.

The pause extractor 70 applies a proper threshold value to the pitch data extracted by the pitch extractor 20 to extract a pause that is a non-speech interval below the threshold value, and sorts the extracted pause information to measure the node 80. Will output

The node detector 80 detects a node of the voice signal input through the pause information extracted by the pause extractor 70, and outputs the node detection information to the score generation unit 90.

In music, the measure indicates the effective length of the beat between the vertical bars drawn vertically above the stave. Since the time signature that distinguishes the temporal flow of music is determined by the type of the underlying note and the number of notes in one measure, detecting a measure is important for constructing a more accurate score.

The procedure of node detection performed in the node detection unit 80 is as follows.

First, the pause extractor 70 determines a speech / non-voice section by applying a threshold value to the pitch data input from the pitch extractor 30, and the non-voice section is called the pause of speech and the candidate position of the node on the score. Assume When a person sings a song composed of sheet music, the longest rest period is generally present in the middle of the song, which is called a reference node, and the longest rest period of the rest information input from the rest period extraction unit 70 is defined as a reference node (N = 1). In each song, each measure has notes corresponding to the time signature, which means that the vocalization time of one measure and the other measure is distributed in a similar band, so a constant multiple (0.25, 0.5, 0.75…) is added to the reference measure. The multiplication operation is performed to generate prediction nodes using reference nodes as shown in FIG. 9. Since the predicted nodes are valid only when the total time of the speech signal is shorter, the progress and end of node detection are determined by comparing the predicted nodes with the total time. As a result of the node detection, candidate pauses close to the prediction node are detected. The detection of the candidate pauses indicates that the detection of the Nth node is completed. Then, the N = N + 1 operation is performed. Perform detection. The node detection information thus performed is output to the sheet music generating unit 90.

The score generation unit 90 generates score data based on the sound field recognition information, the pitch recognition information, and the node detection information input from the sound field recognition unit 50, the pitch recognition unit 60, and the node detector 80.

Next, an embodiment of the automatic song picking method according to the present invention configured as described above will be described in detail with reference to FIGS. 10 to 14.

10 to 14 are flowcharts showing the operation of one embodiment of the automatic song picking method according to the present invention.

First, the voice signal input unit 10 converts an analog voice signal (human song) input from the outside into a digital voice signal and outputs it to the pitch extractor 20 (S100). After extracting the pitch indicating the height of the speech signal using the autocorrelation function of Equation 1 described above, the speech signal input from the signal input unit 10 is divided into the syllable divider 30 and the pause extractor 70. Output to (S200).

The syllable divider 30 receives the pitch data from the pitch extractor 20 and divides the syllable data into syllable units based on the stabilized despreading equation. The syllable data is divided into the preprocessor 40 and the note recognizer 60. Output to (S300). At this time, the syllable data divided by the syllable divider 30 is in a state where noise is removed and a representative pitch value of the syllable is confirmed.

If the syllable division of step S300 will be described in more detail with reference to FIG. 11, the syllable division unit 30 sets the respective pitch data extracted in step S100 as an independent area (in this case, the area of each area, the value of the area, The number of divided regions is calculated (S310), and the value of each region is updated through the above-described Equation 4 until two or more adjacent regions have the same value (S320).

Adjacent areas having the same value are merged (in this case, the area of each area, the value of the area, and the number of divided areas are updated) (S330), and it is determined whether the number of repetitions satisfies the initial setting value (S340).

As a result of the determination, if the number of repetitions satisfies the initial set value, the syllable division unit 30 outputs the divided syllable data to the preprocessor 40 and the pitch recognition unit 60 after finishing the syllable division (S350). Otherwise, the process is repeated after step S320.

After the syllable division in step S300, the preprocessor 40 performs preprocessing to convert the syllable divided by the syllable divider 30 into an effective unit section capable of sound field recognition, and outputs the result to the sound field recognition unit 50 (S400). .

The sound field recognition unit 50 performs sound field recognition by clustering the sound field data preprocessed by the preprocessor 40 in units of each note based on a genetic algorithm, and outputs the sound field recognition information to the score generation unit 90 ( S500).

Referring to FIG. 12 in more detail with reference to FIG. 12, a chromosome is constructed and a population is initialized based on pre-processed sound field data for converting a divided syllable into an effective unit section capable of sound field recognition (S510). On the basis of the center value of the initial cluster of sound field data, clustering is performed to minimize the distance between each object and the cluster center as shown in Equation 5 (S520).

The distance between the clusters and the internal distance of the clusters are defined using Equations 6 and 7 (S530), and the evaluation of the clustered result is performed in step S520 using the fitness function of Equation 8 (S540).

After the evaluation of the clustered results, the result of the fitness function performed in step S540 is selected by using a known roulette wheel method, and one point crosses the chromosome constructed based on the result, and finally, a new generation is generated using a given mutation coefficient. (S550). And after the step S520 is repeated until the clustering of the sound field data is completed.

After performing the sound field recognition through the step S500, the pitch recognizer 60 reconstructs the relative pitch frequency table based on the representative pitch value of the first syllable divided by the step S300, and then performs a syllable based on the reconstructed relative pitch frequency table. Pitch recognition is performed by mapping a representative pitch value of S, and outputs the pitch recognition result to the score generation unit 90 (S600).

When the sound field recognition in step S600 is described in more detail with reference to FIG. 13, the pitch recognition unit 60 determines the pitch through the representative pitch value of the first syllable by viewing the first syllable divided in step S300 as a reference sound (S610). ), Dividing the checked pitch by the average rate scale to confirm the C tone of the relative pitch frequency table (S620).

Subsequently, multiply the frequency of the C sound identified in step S620 by the average rate scale to check the 12th scale of the relative pitch frequency table (S630), and reconstruct the relative pitch frequency table based on this (S640).

The pitch recognition unit 60 maps representative pitch values of the syllables divided in step S300 based on the relative pitch frequency table reconstructed in step S640 (S650), and processes pitch recognition based on the mapping result of each syllable. The result is provided to the sheet music generating unit 90 (S660).

After performing the sound field recognition and the pitch recognition through the steps S500 and S600, the pause extractor 70 applies a threshold value to the pitch data extracted in the step S200 to extract a pause that is a non-speech interval below the threshold value. The node detector 80 detects the node using the resting period information extracted by the pause extractor 70, and then outputs the node detection information to the score generation unit 90 (S700).

If the node detection of the step S700 will be described in more detail with reference to FIG. 14, the pause extractor 70 checks the voice section and the non-voice section by applying an appropriate threshold value to the pitch data extracted at step S200 (S710). In operation S720, a pause period that is a non-voice interval below a threshold value is extracted.

Then, the node detector 80 defines the longest resting period among the pause information extracted in step S720 as the reference node N = 1 (S730), and multiplies the defined reference node by a predetermined multiple (0.25, 0.5, 0.75…). In operation S740, a prediction node is generated.

In operation S740, it is determined whether the generated prediction node is smaller than the total time of the voice signal by comparing the predicted node generated in step S740 with the total time of the voice signal (S750). After completing the node detection, the node detection information is output to the score generation unit 90.

However, if the predicted node generated as a result of the determination in step S750 is smaller than the total time of the speech signal, the node detector 80 detects a candidate dormant close to the predicted node (S770), and if the candidate dormant close to the predicted node is detected, the Nth node is detected. After the detection of the node is completed, the N = N + 1 operation is performed to detect the next node, and the process is repeated after step S740 (S780).

Finally, the sheet music generation unit 90 generates sheet music data based on the sound field recognition information, the pitch recognition information, and the node detection information performed through steps S500, S600, and S700 (S800).

Herein, while the present invention has been described with reference to the preferred embodiments, those skilled in the art will variously modify the present invention without departing from the spirit and scope of the invention as set forth in the claims below. And can be changed.

1 is a block diagram schematically showing a configuration to which the automatic song picking method according to the present invention is applied;

2 is a diagram illustrating an example of extracting a pitch by applying an autocorrelation function to an input voice signal;

3 is a diagram showing an example of a result of performing syllable division using a stabilized despreading equation;

4 is a view for explaining the concept of the duration of the pretreatment process, the rest period and the IOI of the present invention,

5 is a diagram illustrating an example of syllable length information transformed into an IOI;

6 is a diagram showing 12 scales used when determining pitches;

7 is a diagram illustrating an average rate scale;

8 is a diagram illustrating an example of a relative pitch frequency table used for pitch recognition;

9 is a view showing an example of prediction nodes found using reference nodes;

10 is a flowchart showing the operation of an embodiment of the automatic song picking method according to the present invention;

11 to 14 are flowcharts illustrating the operation of each subroutine of FIG. 10 in more detail.

Explanation of symbols on the main parts of the drawings

10: voice signal input unit 20: pitch extraction unit

30: syllable divider 40: preprocessor

50: sound field recognition unit 60: pitch recognition unit

70: pause extraction unit 80: node detection unit

90: music score generation unit

Claims (12)

(1) extracting the pitch indicating the height of the voice from the voice signal input through the voice signal input unit such as a microphone, (2) dividing the continuous voice signal into syllable units based on the pitch data extracted in step (1), removing noise of the voice signal, and providing a representative pitch value of the syllable; (3) performing preprocessing to convert the syllables divided in step (2) into effective unit sections capable of sound field recognition, and performing sound field recognition by clustering the preprocessed sound field data in units of respective notes, (4) reconstructing the relative pitch frequency table based on the representative pitch value of the first syllable divided in step (2), and performing pitch recognition by mapping the representative pitch value of the syllable based on the reconstructed relative pitch frequency table , (5) applying a threshold value to the pitch data extracted in step (1) to extract a rest period which is a non-speech interval below the threshold value, detecting a node using the extracted rest period information, and (6) generating sheet music data based on sound field recognition information, pitch recognition information, and node detection information performed through steps (3), (4), and (5) above; The syllable division of the step (2), Perform syllable segmentation based on a stabilized despreading equation, (2-1) setting each pitch data extracted in the step (1) as an independent area, and calculating the area, the value of each area, and the number of divided areas, (2-2) updating the rate of change value according to scale as the filtering of each area is performed until two or more values of adjacent areas are equal, (2-3) merging adjacent regions having the same value to update the area of each region, the value of the region, and the number of divided regions, (2-4) determining whether the repetition number satisfies the initial setting value, and (2-5) If the number of repetitions satisfies the initial setting value as a result of the determination in the step (2-4), the syllable division is terminated, or the step (2-2) or later is performed. Automatic song picking method, including. The method of claim 1, Pitch extraction of the step (1),

Automatic song collection method using autocorrelation function. delete The method of claim 1, Sound field recognition of the step (3), It performs sound field recognition based on clustering based on genetic algorithm. (3-1) constructing a chromosome and initializing a population based on pre-processed sound field data for converting the syllables divided in step (2) into effective unit sections capable of sound field recognition; (3-2) performing clustering to minimize the distance between each object and the cluster center based on the center value of the initial cluster of the sound field data; (3-3) defining a distance between clusters and an intracluster distance, and performing evaluation of the clustered result in the step (3-2) using a goodness-of-fit function, (3-4) crossing one chromosome based on the result of the fitness function performed in step (3-3), and finally creating a new generation using a given mutation coefficient, and (3-5) repeating step (3-2) and later until clustering of the sound field data ends Automatic song picking method, including. The method of claim 1, Pitch recognition of the step (4), (4-1) confirming the pitch through the representative pitch value of the first syllable by viewing the first syllable divided in the step (2) as a reference sound, (4-2) dividing the pitches identified in step (4-1) by the average rate scale to confirm the C sound of the relative pitch frequency table, (4-3) checking the 12th scale of the relative pitch frequency table by multiplying the frequency of the C tone identified in the step (4-2) by the average rate scale, and reconstructing the relative pitch frequency table based on this, (4-4) mapping representative pitch values of the syllables divided in step (2) based on the relative pitch frequency table reconstructed in step (4-3), and (4-5) the step of processing the speech recognition based on the mapping result for each syllable in the step (4-4) Automatic song picking method, including. The method of claim 1, Extracting the node of step (5), (5-1) determining a speech section and a non-voice section by applying a threshold value to the pitch data extracted in step (1), and extracting a non-speech pause period; (5-2) defining the longest resting period among the resting period information extracted in step (5-1) as a reference node N = 1, (5-3) generating a prediction node by performing an operation of multiplying the reference node defined in step (5-2) by a predetermined multiple, (5-4) determining whether the generated prediction node is smaller than the total time of the voice signal by comparing the prediction node generated in step (5-3) with the total time of the voice signal; (5-5) if the prediction node generated as a result of the determination of step (5-4) is smaller than the total time of the speech signal, detecting a candidate pause in proximity to the prediction node, and (5-6) In step (5-5), if the candidate rest period close to the prediction node is detected, the detection of the Nth node is completed, and after performing N = N + 1 operation to detect the next node, the above (5 -3) repeat the steps after Automatic song picking method, including. delete delete delete delete delete delete

Images