KR101520621B1 - / Method and apparatus for query by singing/huming - Google Patents

Abstract

The present invention relates to a query method by song / humming, wherein a method according to an embodiment of the present invention includes inputting a query sequence of a frame level extracted from arbitrary song / humming, inputting a set of template sequences divided by a bar Performs matching between the query sequence and the template sequence, and outputs the query candidates according to the matching result.

Song, humming, query

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and apparatus for querying by singing /

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an automatic search system and method, and more particularly, to a method and apparatus for querying by song / humming in which a user can quickly search for a desired song or music from a music database.

Conventionally, when a user wishes to listen to music, in order to query a roulette in a music database, the user must type the title of the music, the name of the singer, or other information related to the specific music. However, as the number of songs stored in the music database increases, it becomes difficult to memorize a lot of information about the songs in order to find the music the user wants to listen to.

A query based on a content-based search method, i.e., song / humming, is proposed. The QBSH system is specifically aimed at retrieving music by accepting queries of songs, humming or whistling and helping the user find songs that have read detailed information from the music database.

In U. S. Patent Publication No. 2007/131094, a QBH system is disclosed. It focuses on music information search using 3D search algorithm. It computes a measure of similarity between the query and the database independently, indicating the similarity between the melody and the lyrics. The search is performed in three-dimensional space, t for time, S. for acoustic-phonetic features and H for UDS string. This invention combines the standard speech recognizer with additional 'da' words and the QBH system.

In another US 6,188,010, a query method by song / humming is disclosed. This makes it possible to search for song titles only when the song's melody is known. A piano roll music rotation interface or a piano keyboard interface can be used to input the melody.

US 6,121,530 and US 5,784,686 relate to music retrieval methods.

In general, the conventional QBSH includes four main components: (a) the melody is extracted from the query by the pitching track and converted to a semitone (note). (b) Note splitting is performed on the melody and the note length is ignored. (c) The melody is converted into the same forms as the relative note difference, UDS string. (d) The search method includes string matching, DTW and viterbi search, and the like.

However, in the conventional QBSH system, the recognition rate for a large-capacity music database is reduced in size, which makes it impossible to apply it to practical applications. Also, the processing speed is longer for a large-capacity music database. Therefore, there is a need for a QBSH method and system that can quickly find a desired song by song / humming for a large music database.

In order to solve the above problems, the present invention provides a method and apparatus for querying by song / humming, whereby a user can quickly find a desired song by singing / humming in a large music database, You do not need to know much, you can ask for lyrics, or you can call some of the songs or whistles to ask questions.

In a method of querying by song / humming according to an embodiment of the present invention, a method of generating a song template includes extracting a main melody contour from a song, performing bar division on the extracted main melody contour, Converts the main melody contour into a frame-level note sequence, and stores the template sequence as a template sequence.

The step of performing the bar division according to an embodiment of the present invention may include acquiring information related to the song melody, searching for a start position and an end position of each bar according to the obtained information, This is done by labeling the end point.

The converting of the main melody contour into a frame-level note sequence may include sampling the main melody contour at a predetermined frame shift to obtain the melody note sequence.

The method according to an embodiment of the present invention may further include converting the melody note contour into an average subtracted melody note sequence.

A method for querying by song / humming according to an embodiment of the present invention includes inputting a frame-level query sequence extracted from a song / humming, inputting a bar-divided template sequence, matching between the query sequence and the template sequence And outputs the query candidates according to the matching result.

Wherein matching between the query sequence and the template sequence comprises:

Extracting a low resolution query sequence from the input query sequence at a first frame shift, extracting a low tone template sequence from the input template sequence at the first frame shift, Obtaining a first set of candidate songs from the tune database according to the matching result, and obtaining a sequence of high-tone-quality queries from the input query sequence with a second frame shift smaller than the first frame shift Extracting a high-tone template sequence from the input template sequences included in the candidate templates at the second frame shift, and outputting the high-tone tone sequence and the high-tone tone template based on the first set of candidate songs, Performs a second matching of the sequence, and, in accordance with the second matching result, Characterized in that obtaining candidate songs in the final set from the database.

Here, the matching is performed by using a linear scaling algorithm of fast tempo scaling.

Wherein the step of matching the query sequence with the template sequence comprises the steps of: (a) inputting a query sequence (Q) and a template sequence (T); (b) causing the scaling r to be r ₀ and the offset 隆 of the scaling r to have 隆₀ ; (c) calculating Q ₁ = Q x r and a distance value d = | Q ₁ -T | of the sequence Q ₁ and the template sequence T, using a linear scaling algorithm; (d) calculating Q _high = Q x (r +?) and the distance value d _hign = | Q _high- T | of the sequence Q _high and the template sequence T, using the linear scaling algorithm; (e) calculating Q _low = Q x (r-delta) using the linear scaling algorithm and a distance value d _low = | Q _low- T | of the sequence Q _low and the template sequence T; (f) comparing the distance values d, d _high , d _low ; (g) If d _high is the smallest, r is r + delta and d is d _high , and if d _low is the smallest, r is r-delta and d is d _low , r and d; (h) determining whether the? is not less than a predetermined value; (i) if? is equal to or greater than a predetermined value,? is? / 2, and performing the step (d); (j) outputting the distance value d when? is less than or equal to a predetermined value; And (k) determining whether the query sequence and the template sequence match according to whether the distance value d is less than or equal to a predetermined threshold.

Here, r ₀ and 隆₀ are 1.4 and 0.4, respectively, and the predetermined value is 0.09.

Here, when the query sequence and the template sequence are matched, a distance value between the query sequence and the template sequence is calculated using a recursive alignment algorithm. If the distance value is less than a predetermined threshold value And outputs a song corresponding to the template sequence as a result of the query. Otherwise, the template is discarded, and matching between the query sequence and another template sequence is performed.

(Q ₁ , q ₂ , ..., q _N ) and the template sequence T = (t ₁ , t ₂ , ..., q _N ) and the template sequence t _M ), where N and M are the number of frames of Q and T, respectively; (b) inputting a recursive depth D and having j = [N / 2] and i = 0; (c) at the point sequence Q j, 2 sequences of _{Q 1 = (q 1, q} 2, ..., q i), Q 2 = (q j +1, q j +2, ... , q _N ); (d) calculating a sum sum (Q1) of Q1 and a sum (Q2) of Q2 and calculating a ratio sum (Q1) / sum (Q); (e) k = [M / 2] to obtain, for the template T sequence at point k, the two sequence _{T 1 = (t 1, t} 2, ..., t k) and T ₂ (t _{+ K 1} , t _{K +2} , ..., t _M ); (f) the method comprising using a linear scaling algorithm, calculating the distance value d ₂ between Q ₁ and distance value between T ₁ d ₁ and Q ₂ and T _2, and so as to have a _{S 1 = d 1 + d 2} ; (g) from the point h to the sequence of the template T, a sequence of 2 _{T 3 = (t 1, t} 2, ..., t h) and _{T 4 = (t h +1,} t h +2, .. ., t _M ), so that the ratio of the sum of T ₃ and T is R ₀ ; (h) the step of using the linear scaling algorithm, and calculates the Q ₁ and distance value d ₃ T _3, the distance value of Q ₂ and T ₄ d _4, S ₂ to have a d ₃ + d _4; (i) comparing S ₁ and S ₂ , and if S ₁ is less than S ₂ , S = S ₁ and i = k, otherwise S = S ₂ and i = h; (j) If the D is 0, S, and outputs, D is not 0, the two sequences to the template sequences _{T T 1 = (t 1,} t 2, ... t i) and T ₂ = (t _{i +1} , t _{i +2} , ..., t _M , returning to step (a) to have Q = Q ₁ and T = T ₁ and repeat the operation to obtain S ₁ , and Q = Q ₂ and T = T ₂ and returning to step (a) to obtain S ₂ , then having S = S ₁ + S ₂ and outputting S; And (k) outputting a song corresponding to the template as a query result if S is less than a predetermined value, and if not, performing matching between the query sequence and another template sequence.

According to another embodiment of the present invention, an apparatus for generating a song template in a query by song / humming includes an extraction unit for extracting a main melody contour from each piece of music database, a bar division on the extracted main melody contour And a conversion unit for converting the melody contour into melody contours to be performed and a melody contour in which the bar division is performed into a melody contour in a frame level and storing the melody contour as a template sequence.

According to another embodiment of the present invention, a query device by song / humming includes a query sequence input unit for inputting a frame-level query sequence extracted from a song / humming, a template sequence input unit for inputting a template sequence of a frame- A matching unit for performing matching between the query sequence and the template sequence, and an output unit for outputting query candidates according to a result of the matching.

According to still another embodiment of the present invention, there is provided a method for generating a melody contour, comprising the steps of: generating a template set by extracting a main melody of each song of a song database and performing bar division on the melody contour; A melody extracting unit for extracting melody contours, a melody extracting unit for extracting melody contours, and a melody matching unit for performing matching between respective templates of the query melody contour and template sets, and outputting candidate songs according to the matching result.

According to another embodiment of the present invention, the apparatus further includes a polyphonic tone template sequence generator for extracting a note sequence from the main melody with a predetermined frame shift to generate a polyphonic frame level melody template sequence.

According to another embodiment of the present invention, the apparatus further includes a multi-tone tone query sequence generation unit for extracting a note sequence from the query melody contour, to generate a multi-tone tone frame-level query sequence.

According to another embodiment of the present invention, the melody matching unit includes a distance value calculating unit for calculating a distance between the multi-tone frame-level melody template sequence and the multi-tone frame frame-level resequencing, And if the distance value is less than or equal to the threshold value, determining the template sequence to match the query sequence and outputting a song corresponding to the template sequence; otherwise, matching the query sequence with another template sequence For example.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a schematic block diagram of a song / humming query device 100 according to an embodiment of the present invention.

Referring to FIG. 1, a song / humming inquiry apparatus 100 includes a template generating unit 110, a melody extracting unit 120, and a melody matching unit 130.

The template generation unit 110 generates a template set by extracting a main melody of each piece of the music database and performing bar division on the melody contour. Here, template set generation is performed, and templates for the next matching processing are created. In addition, template set generation can be performed offline. The main melody of each song in the music database is extracted, and then the bar division is performed on the extracted main melody to create a set of multi-tone templates. With reference to FIGS. 3 and 4, the template generation will be described later.

The melody extracting unit 120 extracts a query melody contour from the song / humming of the user. That is, the melody contour is obtained from the song / humming of the user for the next matching process.

First, the user's song / humming is input, which may be a query of a person with or without lyrics, a song of a song of a song, or a whistle. These query types can be stored in a wave format file.

Next, pitch tracking is performed. In this process, pitches are extracted from the chord of the highest frame, i. E. The smallest frame shift. If a melodic contour of a lower chord is desired, the corresponding melody sequence is extracted from the extracted melody sequence with the smallest frame shift. For example, if the smallest frame shift is 0.1s and the audio length is 8s, data with 80 frames is obtained. If a melody sequence of a frame shift of 0.2s is required, even (or odd) frames are obtained from the data of 80 frames. The algorithm for extracting the tone is a conventional auto-correlation algorithm, and a detailed description thereof is omitted.

Next, melody contour smoothing is performed. Since it is impossible to extract the tonal frequency value accurately, it may be a semi-tonal frequency or multiple tonal frequencies, or some isolated points, and it is necessary to smooth the contour to the extracted tonalities. In this process, median filtering and linear filtering are employed to smoothing the tone contour in order to remove non-ideal tonal frames in tonal tracking.

After tonality tracking, conversion from tonality to semitone is performed. Each tonal frequency is converted into a semitone as shown in the following equation (1).

Here, freq is the pitch frequency.

Next, an average subtraction method is performed on the converted semitone. Since it is difficult for different people to sing or hum with different key notes and to hum from notes in the music database accurately, in order to reduce the influence on the matching process, the note sequence is changed to a predetermined representative value for the matching process There is a need. In one embodiment of the invention, the semitone sequence is converted to an averaged subtracted sequence, which is then used in the next matching process. In particular, the semitone sequence is transformed into a sequence used in an algorithm for matching. That is, for the semitone sequence, the sum of non-zero values in the sequence is obtained, and an average value is obtained. Subsequently, the average value is subtracted from the sum of each non-zero value, and a sequence for matching is obtained. The same values as zero in the semitone sequence are discarded. Also, the main melody extraction is performed in the same manner.

Finally, a frame-level melody contour is output. In this process, a simple sampling method is used to obtain the frame level note contours of the other chords from the frame levels of the highest chord note contours. This operation is similar to creating a template with different chords for each song and is not described further.

The melody matching unit 130 performs matching between the query melody contour and each template of the template set, and outputs candidate songs according to the matching result. After the melody contour is extracted from the user ' s song / humming, matching between the melody contour and the template set is performed to obtain the result that the user wishes to query. The matching operation will be described later with reference to Fig.

FIG. 2 is a specific block diagram of the song / humming query device 200 shown in FIG.

2, the song / humming query device 200 includes a multiplexed tone template sequence generator 210, a multiplexed tone query sequence generator 220, and a melody matching unit 130, A distance value calculation unit 230 and an output unit 240. [

The multi-chord template sequence generation unit 210 extracts a note sequence from the main melody with a predetermined frame shift to generate a multi-chord frame level melody template sequence.

The multi-tone pitch query sequence generator 220 extracts a note sequence from the query melody contour to generate a multi-tone frame-level query sequence.

The distance value calculation unit 230 calculates the distance between the multiplexed tone frame level melody template sequence output from the multiplexed tone template sequence generation unit 210 and the multiplexed tone frame level query sequence output from the multiplexed tone query sequence generation unit 220 .

The output unit 240 determines whether the distance value output from the distance value calculation unit 230 is equal to or less than a predetermined threshold value. Here, if the distance value is equal to or less than the threshold value, a template sequence matching with the query sequence is determined and a song corresponding to the template sequence is output. Otherwise, the query sequence is matched with another template sequence.

FIG. 3 is a specific block diagram of the song template generating unit 110 shown in FIG.

3, the song template generating unit 110 includes an extracting unit 300, a bar dividing unit 310, and a converting unit 320. [

The extraction unit (300) extracts the main melody contour from each piece of music database. The bar division unit 310 performs bar division on the main melody contour extracted by the extraction unit 300. The transforming unit 320 transforms the melody contour subjected to the bar division into a frame-level note sequence, and stores the transformed melody contour as a template sequence.

The specific operation of the extracting unit 300, the bar dividing unit 310, and the converting unit 320 will be described with reference to FIG.

4 is a flowchart illustrating a method of generating a song template according to another embodiment of the present invention.

Referring to FIG. 4, first, a music database is input. In step 400, the main melody of each song is extracted. The method of extracting the melody varies according to the MIDI type. Because a mono MIDI type song has only one track, the melody stored in the track is the main melody. The next (polyphonic) MIDI type song usually has the main melody on the first track. However, some of the following MIDI types do not. The following tracks can be determined as tracks having the main melody. (1) Track titles with words "MELODIES", "VOCAL", "SING", "SOLO", "LEAD", "VOICE" in the track title; (2) a track having a maximum mean note intensity; (3) a track having a maximum total note duration, and note length means the time from the start time of the first note to the end time of the last note. A track satisfying one of the above three conditions is regarded as a track in which a main melody is present, and note information is extracted from the track as a main melody.

In step 402, other information is extracted from the track. For one song, tempo, time signature (4/4 and 2/4 time), and quarter note length are very useful information, and are generally written in the header of each track. Thus, this information can be obtained from the tracks of each MIDI song.

In step 404, the start time and the end time of each bar can be known in accordance with the information obtained in step 402. In particular, the beginning and ending positions of each bar in MIDI are labeled with length and time signature.

The operations of steps 402 and 404 are collectively referred to as bar segmentation. If the user is singing / humming a song, it typically starts at the beginning of the bar and stops at the end of the bar. These habits can increase the accuracy of the query and speed up the query. Also, if the user is performing a query, the query can start from any part of the song. That is, the user can sing / hum from any part of the song. Therefore, automatic bar division is adopted in the present invention.

In step 406, a frame-level multiplex tone melody contour is generated. Melody note sequences with different chords can be obtained according to different frame shifts. A low chord means sampling the notes of each song with a large frame shift. In this case, the number of obtained notes is small, and the representation of the melody is relatively rough. A high tone means sampling the notes of each song with a small frame shift. In this case, the number of acquired notes is relatively large, and the representative of the melody is relatively elaborate. Generally, two or more frame level melody sequences can be obtained from one song. The average subtraction process is performed on the finally obtained melody note sequences.

In step 408, a template set for the matching process is output. I. E., Melody sequences with each song, and other chords extracted from the set of templates for matching.

5 is a flowchart illustrating a query method by song / humming according to another embodiment of the present invention.

Referring to FIG. 5, in step 500, a frame level melody contour extracted from song / humming is input. In the next matching process, the frame level melody contour may be referred to as a song / humming query sequence. In step 502, the template set obtained from the template generating unit is input. In the next matching process, each template in the template set may be referred to as a template sequence.

At step 504, low-tone melody contours are generated. In this process, the melodic contours of the same chord are obtained from the frame-level polyphonic melody with the highest polyphony and the highest polyphonic frame-level polyphonic sound template set, respectively. Because the melody contour of the low-pitched tone is small, fast matching can be performed between the query sequence and the template sequence to obtain candidates in a short period.

At step 506, a fast matching process is performed to select a small set of candidates for the next comparison. In particular, an extracted melody contour (i.e., a song / humming query sequence Q) and a template set (i.e., a template sequence T) are performed. All kinds of matching algorithms can be used to perform the matching process. In one embodiment of the present invention, linear scaling (LS) with fast tempo scaling is proposed, which is a conventional linear scaling algorithm with fast tempo scaling operation. Here, the proposed algorithm is referred to as Algorithm 1 and is described in detail in Table 1 below.

Where r is the scaling, δ is the offset of r, and r ₀ and δ ₀ are the two preset values set at 1.4 and 0.4. Q x r means scaling the query sequence from N to N x r. If the length of Q is 100 and r is 1,5, then the transformed Q is 150, LS (Q, T) is the linear scaling function, and the distance function of the query and template is defined as d = | qt |.

LS (Q, T) will be described in detail with reference to Fig. As shown in FIG. 6, the horizontal axis represents a template sequence, and the vertical axis represents a query sequence. The distance is calculated using diagonal points, that is, to calculate the absolute value of the difference of the two sequences.

After Algorithm 1 is performed and the distance is output, four different filters are used to further reduce the size of the candidates. These filters are variance, sub-segment mean comparison, center comparison, and note variance comparison.

If the linear scaling filter of fast tempo scaling is less than the predetermined value, then the corresponding template sequence will remain, otherwise it will be discarded.

The variance comparison filter compares the variance between the query sequence and the template sequence, and if the variance comparison filter is less than the predetermined threshold of variance, the corresponding template sequence will remain, otherwise it will be discarded.

The sub-segment comparison filter averages queries and templates evenly into four segments, and calculates the average of each sub-segment to construct each of the four-dimensional vectors. Then, the distance between the query sequence vector and the template sequence vector is compared. If the distance value is less than the predetermined threshold value, the corresponding template sequence will remain, otherwise it will be discarded.

The center comparison filter computes the query sequence and the centers of the template sequence and compares the distances between the centers. If the distance value is less than the predetermined threshold value, the corresponding template sequence will remain, otherwise it will be discarded.

The formula for calculating the center is shown in Equation 2 below.

Where S (i) is the i-th value in the sequence and N is the length of the sequence.

The note distribution variance filter calculates the variance of the query sequence and the notes of the template sequence, then arranges them according to the values from the largest value to the lowest value, constructs the three-dimensional vector using the three notes having the maximum value, And the distance between the vector of the template sequence and the vector of the template sequence. If the distance value is less than the predetermined threshold value, the template sequence will remain, otherwise it will be discarded.

The five filters are mainly used to filter the template sequences for the next exact match.

After filtering in step 506, the size of the candidate songs is reduced, and in step 508, a small scale template set is output.

In step 510, a high tone melody contour is generated. In this process, a high-tone template with the same chord as the high-chord quality sequence is output for final correct matching.

In step 512, an exact match is performed. In this process, a fast recursive sort with an average shift is used. A fast recursive sort with an average shift is based on a recursive sort (RA) algorithm, which is named Algorithm 2, as follows.

7A and 7B are diagrams for explaining a conventional RA algorithm. As shown in Figure 7A, the bold line is the optimal alignment path between the query sequence and the template sequence. In the first phase of recursion, the three alignment paths, the solid line, the dotted line, and the dashed line are compared. If the score obtained along the dotted line is the smallest, in the next recursion step, the path is divided into two as shown in Fig. 7B. The calculation operation is then repeated for the two path groups, and the smallest value is obtained in each of the two groups. The final path is then obtained by combining the two paths with the smallest score.

The fast RA algorithm differs from the conventional RA algorithm in that only the path with the same bending direction is calculated after the bending direction of the optimal path is determined to store the calculation part. For example, in Figure 7A, only the path corresponding to the dashed line is calculated by determining that the best path is bent upward to store the remaining path calculations.

Generally, female singers have higher tone frequencies than men. That is, different people have different tone frequency values (note values) for the same song. This is a problem regarding the accuracy of the matching process. To address these problems, an average shift is implemented to achieve a minimum score between the query and the template. Here, a high-speed RA with an average shift is proposed and will be described in Algorithm 3.

In Algorithm 3, δ ₀ is a value set to 2. Q ± δ are all values that add or subtract δ to Q, and δ is the shift value.

Here, the average shift is the same algorithm as the previously described average subtraction. Because song / humming is not accurate, there is a difference between the average of the query sequence and the average of the template sequence. To eliminate this difference, an average shift algorithm is used. That is, the average of the query sequence is shifted to an appropriate position, up or down so as to have a distance between the average of the query sequence and the average of the template sequence.

8 is a diagram for explaining an average shift algorithm. In FIG. 8, t denotes the average of the query sequence and? Is set to 2. The distance value between the query sequence and the template sequence is calculated when the average of the query sequence is t, t-2 and t + 2, respectively. The calculated distances are then compared in size. If the distance at t + 2 is the smallest, t + 2 is denoted by t 'and δ is divided by 2. Next, the distance values are compared in size at t ', t'-1 and t'+ 1, and the smallest distance is the final result.

In step 308, a matching result is output. In particular, if the distance between the query sequence and the template sequence is less than a predetermined value, the two sequences are considered to be matched to each other. A song in the database corresponding to the template is considered a candidate song. Through accurate matching, the size of the candidates is further reduced. The user can then select a song he or she wishes to hear from the candidates output in a short time.

According to one embodiment of the present invention, matching processes are performed on different chords. First, matching is performed between the melodic contours of the low-tones to select small candidate sets from the large-volume database. Then, based on the small candidate sets, an exact match is performed between the melody contours in the high tones to obtain the query result. Such a process increases the speed of the query.

The present invention has the following advantages. In melody extraction, the melody is performed with automatic bar division, the query is performed as needed, and the accuracy is increased. Matching is performed on different chords to increase processing speed. The user may have a problem with the song, and an error is allowed in the song / hum. The present invention can be applied to PCs, mobile phones, and MP3 players.

Meanwhile, the present invention can be embodied in computer readable code on a computer readable recording medium. A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored.

Examples of the computer-readable recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device and the like, and also a carrier wave (for example, transmission via the Internet) . In addition, the computer-readable recording medium may be distributed over network-connected computer systems so that computer readable codes can be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the present invention can be easily deduced by programmers skilled in the art to which the present invention belongs.

The present invention has been described above with reference to preferred embodiments. It will be understood by those skilled in the art that the present invention may be embodied in various other forms without departing from the spirit or essential characteristics thereof. Therefore, the above-described embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the present invention is indicated by the appended claims rather than by the foregoing description, and all differences within the scope of equivalents thereof should be construed as being included in the present invention.

1 is a schematic block diagram of a song / humming query device 100 according to an embodiment of the present invention.

FIG. 2 is a specific block diagram of the song / humming query device 200 shown in FIG.

FIG. 3 is a specific block diagram of the song template generating unit 110 shown in FIG.

4 is a flowchart illustrating a method of generating a song template according to another embodiment of the present invention.

5 is a flowchart illustrating a query method by song / humming according to another embodiment of the present invention.

6 is a diagram for explaining a conventional LS algorithm.

7 is a diagram for explaining a conventional RA algorithm.

8 is a diagram for explaining an average shift algorithm.

Claims (19)

CLAIMS 1. A method for generating a song template in a query by song or humming, Extracting a main melody contour from an arbitrary piece of music; Performing bar segmentation on the extracted main melody contour; And Converting the melody contour into a frame-level note sequence and storing the converted melody contour as a template sequence, Converting the melody contour into a frame-level note sequence and storing the melody contour as a template sequence, And sampling the melody contour with a predetermined frame shift to obtain the frame-level note sequence. The method according to claim 1, Wherein the performing the bar segmentation comprises: Obtaining information on the melody of the tune; Searching for a start position and an end position of each bar according to the obtained information; And And labeling the start point and end point of each bar in the melody contour. delete The method according to claim 1, And converting the frame-level note sequence into an average subtracted melody note sequence. A method of querying by song or humming, Inputting a query sequence of a frame level extracted from any song or humming; Performing a matching between the query sequence stored in the tune database and corresponding to the respective divided frame level template sequences; And And if the template sequence matches the query sequence, outputting a song corresponding to the template as a result, Wherein the template sequence comprises: And a melody contour of each of the plurality of songs is sampled at a predetermined frame shift. 6. The method of claim 5, Wherein the predetermined frame shift includes a first frame shift and a second frame shift, Wherein the query sequence and the template sequence match, Sampling the input query sequence with the first frame shift to extract a low-tone preview sequence; Extracting a hypothesis template sequence sampled at the first frame shift of the template sequences; Performing a first matching of the low tone tone query sequence and the low tone template sequence and obtaining candidate template sequences from the template sequences according to the result of the matching; Sampling the input query sequence with the second frame shift smaller than the first frame shift and extracting a sequence of high tone quality queries; Extracting a high-tone template sequence sampled at the second frame shift among the candidate template sequences; And And performing a second matching of the high tone tone query sequence and the high tone tone template sequence. 6. The method of claim 5, Wherein the matching of the query sequence and the template sequence is performed using a linear scaling algorithm. 6. The method of claim 5, Wherein the step of matching the query sequence with the template sequence comprises: (a) inputting a query sequence (Q) and a template sequence (T); (b) the scaling r is r ₀ and the offset 隆 of the scaling r has 隆₀ ; (c) calculating Q ₁ = Q x r and a distance value d = | Q ₁ -T | of the sequence Q ₁ and the template sequence T, using a linear scaling algorithm; (d) calculating Q _high = Q x (r +?) and the distance value d _hign = | Q _high- T | of the sequence Q _high and the template sequence T, using the linear scaling algorithm; (e) calculating Q _low = Q x (r-delta) using the linear scaling algorithm and a distance value d _low = | Q _low- T | of the sequence Q _low and the template sequence T; (f) comparing the distance values d, d _high , d _low ; (g) If d _high is the smallest, r is r + delta and d is d _high , and if d _low is the smallest, r is r-delta and d is d _low , r and d; (h) determining whether the? is not less than a predetermined value; (i) if? is equal to or greater than a predetermined value,? is? / 2, and performing the step (d); (j) outputting the distance value d when? is less than or equal to a predetermined value; And (k) determining whether the query sequence and the template sequence match according to whether the distance value d is less than or equal to a predetermined threshold value. 9. The method of claim 8, Wherein r ₀ and 隆₀ are 1.4 and 0.4, respectively, and the predetermined value is 0.09. 6. The method of claim 5, When the query sequence and the template sequence are matched, Calculating a distance value between the query sequence and the template sequence using a recursive sorting algorithm and outputting a query corresponding to the template sequence as a query result if the distance value is less than or equal to a predetermined threshold value; And discarding the template and performing a matching between the query sequence and another template sequence. 6. The method of claim 5, Wherein the query sequence and the template sequence match, (a) inputting the query sequence Q = (q ₁ , q ₂ , ..., q _N ) and the template sequence T = (t ₁ , t ₂ , ... t _M ) M is the number of frames of Q and T, respectively); (b) inputting a recursive depth D and having j = [N / 2] and i = 0; (c) at the point sequence Q j, 2 sequences of _{Q 1 = (q 1, q} 2, ..., q i), Q 2 = (q j + 1, q j + 2, ... , q _N ); (d) calculating a sum sum (Q1) of Q1 and a sum (Q2) of Q2 and calculating a ratio sum (Q1) / sum (Q); (e) k = [M / 2] to obtain, for the template T sequence at point k, the two sequence _{T 1 = (t 1, t} 2, ..., t k) and T ₂ (t _{+ K 1} , t _{K +2} , ..., t _M ); (f) the method comprising using a linear scaling algorithm, calculating the distance value d ₂ between Q ₁ and distance value between T ₁ d ₁ and Q ₂ and T _2, and so as to have a _{S 1 = d 1 + d 2} ; (g) from the point h to the sequence of the template T, a sequence of 2 _{T 3 = (t 1, t} 2, ..., t h) and _{T 4 = (t h +1,} t h +2, .. ., t _M ), so that the ratio of the sum of T ₃ and T is R ₀ ; (h) the step of using the linear scaling algorithm, and calculates the Q ₁ and distance value d ₃ T _3, the distance value of Q ₂ and T ₄ d _4, S ₂ to have a d ₃ + d _4; (i) comparing S ₁ and S ₂ , and if S ₁ is less than S ₂ , S = S ₁ and i = k, otherwise S = S ₂ and i = h; (j) If the D is 0, S, and outputs, D is not 0, the two sequences to the template sequences _{T T 1 = (t 1,} t 2, ... t i) and T ₂ = (t _{i +1} , t _{i +2} , ..., t _M , returning to step (a) to have Q = Q ₁ and T = T ₁ and repeat the operation to obtain S ₁ , and Q = Q ₂ and T = T ₂ and returning to step (a) to obtain S ₂ , then having S = S ₁ + S ₂ and outputting S; And (k) outputting a song corresponding to the template as a query result if S is less than a predetermined value, and if not, performing a matching between the query sequence and another template sequence. An apparatus for generating a song template in a query by a song or a humming, An extracting unit for extracting a main melody contour from each piece of music in the music database; A bar division unit for performing bar division on the extracted main melody contour; And And a conversion unit for converting the melody contour into a frame level note sequence and storing the converted melody contour as a template sequence, Wherein, And samples the melody contour at a predetermined frame shift to obtain the frame-level note sequence. As a query device by song or humming, A query sequence input unit for inputting a query sequence of a frame level extracted from a song or a humming; A matching unit that performs matching between the query sequence and the divided frame level template sequence corresponding to each of the tunes; And And an output unit outputting query candidates according to a result of the matching, Wherein the template sequence comprises: And a melody contour of each of the songs is sampled at a predetermined frame shift. As a query device by song or humming, A template generation unit for generating a template set by extracting a main melody of each piece of music in a music database and performing bar division on a melody contour; A melody extracting unit for extracting a query melody contour from the song or humming; And And a melody matching unit performing matching between the query melody contour and each template of the template set and outputting candidate songs according to the matching result, Further comprising a polyphonic tone template sequence generator for extracting a note sequence from the main melody with a predetermined frame shift to generate a polyphonic frame level melody template sequence. 15. The method of claim 14, And a multi-tone pitch query sequence generator for extracting a note sequence from the query melody contour to generate a multi-chord frame level query sequence. 16. The apparatus of claim 15, wherein the melody matching unit comprises: A distance value calculator for calculating a distance between the multi-tone frame-level melody template sequence and the multi-tone frame-level melody query sequence; And Determining whether the distance value is less than or equal to a predetermined threshold value and determining the template sequence to match the query sequence if the distance value is less than or equal to the threshold value and outputting a song corresponding to the template sequence; And an output unit for performing a matching between the sequence and another template sequence. The method according to claim 1, Wherein the predetermined frame shift includes a first frame shift and a second frame shift, Wherein sampling at the predetermined frame shift comprises: Sampling the melody contour with the first frame shift to obtain a first frame level melody note sequence; And  And sampling the melody contour with the second frame shift to obtain a second frame-level melody note sequence. 13. The method of claim 12, Wherein the predetermined frame shift includes a first frame shift and a second frame shift, Wherein, Sampling the melody contour with the first frame shift to obtain a first frame level melody note sequence, and Wherein the melody contour is sampled at the second frame shift to obtain a second frame-level melody note sequence. 14. The method of claim 13, Wherein the predetermined frame shift includes a first frame shift and a second frame shift, The matching unit, Sampling the input query sequence with the first frame shift to extract a low-tone preview sequence, Extracting a low-tone template sequence sampled at the first frame shift among the template sequences, Performing a first matching of the low-tone-tone query sequence and the low-tone template sequence, obtaining candidate template sequences from the template sequences according to a result of the matching, Sampling the sequence of input queries with the second frame shift smaller than the first frame shift to extract a sequence of high- Extracting a high-tone template sequence sampled at the second frame shift among the candidate template sequences, and And performs a second matching of the high-tone-quality query sequence and the high-tone-tone template sequence.

Images