KR20220125389A - Music search method and device using inflection point of melody line - Google Patents

Abstract

The disclosed technology relates to a music search method and device using an inflection point of a melody line. The method includes: a step of selecting, by the music search device, a plurality of inflection points included in an input music source; a step of extracting, by the music search device, inflection point information based on a gradient variation for each of the plurality of inflection points and a distance ratio between each of the inflection points; a step of calculating a similarity between sequences by comparing the inflection point information of original music with the inflection point information; and a step of dividing the input music source into a length of representative phrase of the original music to be normalized.

Description

Music search method and device using inflection point of melody line {MUSIC SEARCH METHOD AND DEVICE USING INFLECTION POINT OF MELODY LINE}

The disclosed technology relates to a method and apparatus for searching music using an inflection point of a melody line.

Existing song search algorithms are being studied focusing on pitch changes or length analysis. Typically, a chromatogram is used to calculate the components with an octave difference in the sound source data and put the entire frequency in one octave. Calculation of similarity between chromatograms is an essential component in implementing a music cover song search system.

On the other hand, if the chromatogram is used, it is possible to accurately grasp the information on the name of the gye name, but there is a disadvantage that the relationship such as the relative pitch between the pitches and the change in the speed of the song cannot be considered. For example, when searching for a cover song with a modified original song, if the composition, speed, and overall composition are modified based on a certain standard, the search accuracy can be guaranteed, but it is possible to freely transform the change in composition step by step or freely change the speed of the song. In this case, there was a problem that the search accuracy was lowered.

On the other hand, a search system using a chromagram requires a chromagram coding method to reduce the amount of storage and calculation required for chromatogram comparison as well as accuracy for a search result.

Korean Patent Publication No. 10-2018-0103639

The disclosed technology is to provide a method and apparatus for searching music using an inflection point of a melody line.

A first aspect of the disclosed technology to achieve the above technical problem is the steps of selecting a plurality of inflection points included in an input sound source by a search apparatus, and the search apparatus changes inclination of each of the plurality of inflection points and between the inflection points. Extracting the inflection point information based on the distance ratio, calculating the similarity between sequences by comparing the inflection point information of the original song with the inflection point information, and normalizing the input sound source by dividing the length of the representative passage of the original song An object of the present invention is to provide a music search method using an inflection point of a melody line.

A second aspect of the disclosed technology to achieve the above technical task is to select an input device for receiving a sound source, a storage device for storing a plurality of sound sources, and a plurality of inflection points included in the input sound source, A processing device for extracting inflection point information based on the gradient change and the distance ratio between each inflection point, and comparing the inflection point information of each of the plurality of sound sources with the inflection point information of the input sound source to search for the original song for the input sound source. An object of the present invention is to provide a music retrieval device using an inflection point of a melody line.

Embodiments of the disclosed technology may have effects including the following advantages. However, since it does not mean that the embodiments of the disclosed technology should include all of them, the scope of the disclosed technology should not be understood as being limited thereby.

According to an embodiment of the disclosed technology, a music search method and apparatus using an inflection point of a melody line has an effect of searching for an original song using a cover song or humming.

In addition, there is an effect of protecting the copyright of the sound source author by searching for the original song of the sound source inserted in the broadcast content.

In addition, there is an effect of determining plagiarism by comparing the original song and the cover song.

In addition, there is an effect of reducing the amount of storage and calculation required compared to the conventional chromatogram search system.

1 is a diagram illustrating a music search process using an inflection point of a melody line according to an embodiment of the disclosed technology.
2 is a flowchart of a music search method using an inflection point of a melody line according to an embodiment of the disclosed technology.
3 is a block diagram of a music search apparatus using an inflection point of a melody line according to an embodiment of the disclosed technology.
4 is a diagram schematically illustrating a plurality of inflection points included in a sound source.
5 is a diagram illustrating calculating the similarity in a sliding manner.
6 is a view showing a result of selecting an inflection point.

Since the present invention can have various changes and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention.

Terms such as first, second, A, and B may be used to describe various components, but the components are not limited by the above terms, and only for the purpose of distinguishing one component from other components. used only as For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. and/or includes a combination of a plurality of related listed items or any of a plurality of related listed items.

A singular expression in terms used herein should be understood to include a plural expression unless the context clearly dictates otherwise. And terms such as "comprising" mean that the specified feature, number, step, operation, component, part, or a combination thereof exists, but one or more other features or number, step operation component, part It is to be understood that this does not exclude the possibility of the presence or addition of or combinations thereof.

Prior to a detailed description of the drawings, it is intended to clarify that the classification of the constituent parts in the present specification is merely a division according to the main function that each constituent unit is responsible for. That is, two or more components to be described below may be combined into one component, or one component may be divided into two or more for each more subdivided function.

In addition, each of the constituent units to be described below may additionally perform some or all of the functions of other constituent units in addition to the main function it is responsible for. Of course, it can also be performed by being dedicated to it. Therefore, the existence or non-existence of each component described through the present specification should be interpreted functionally.

1 is a diagram illustrating a music search process using an inflection point of a melody line according to an embodiment of the disclosed technology. Referring to FIG. 1 , the search apparatus may search for an original song for an input cover song using an inflection point instead of a sequence of a melody line, and may output a search result through an output device. Here, the inflection point is a point where the pitch changes on the score, and the points representing each inflection point are connected with a line to make it easier to understand. The search apparatus may generate inflection point information by selecting some inflection points at which significant data changes appear among the extracted inflection points. And by comparing the inflection point information, it is possible to compare the similarity between the input sound source and the original song.

In FIG. 1 , the sound source for the cover song is input as an example, but data recorded by humming the melody of a specific song, such as humming, may be input as the sound source. The sound source input to the search device may use a melody line obtained by converting the music score into data. Such a melody line may be expressed as a frequency value or may be expressed in the form of a MIDI note number. The search apparatus includes a database for storing a plurality of sound sources, and may calculate a similarity by comparing sound sources stored in the database with an input sound source. The search apparatus may calculate similarity between two inflection point sequences by comparing inflection point information extracted from the input cover song sound source and inflection point information extracted from a pre-stored sound source, respectively. For example, it may be performed in a manner of calculating whether the representative phrase of the original song is included in the sound source by sliding the sequence for the inflection point information of the representative phrase of the original song one by one with respect to the sequence regarding the inflection point information of the input sound source. Here, the stored sound source means the sound source for the original song. When the similarity is calculated in this way, the input sound source is divided by the length of the representative phrase of the stored original song and normalized, so that the one with the highest similarity can be searched for as the original song for the input sound source.

In the conventional music search algorithm, major causes of performance degradation are changes due to transposition, changes in the overall speed of the sound source, changes in the composition of songs, and memory problems caused by extracting feature values included in the sound source for every frame. is designated as The most representative of these problems, the research on the search algorithm to cope with the change caused by transposition was intensively conducted, and as a result, the Levenshtein Distance, Edit Distance algorithm and the Longest Common Substring (LSC) algorithm were conducted. A search system using a sequence similarity measurement algorithm such as Although these algorithms are string-based algorithms, since the string can be processed as a kind of sequence, it can also be used to analyze sequence data for a sound source. The edit distance algorithm defines the number of insertions, deletions, and changes required for two strings to be the same as the edit distance, calculates it, and returns the minimum value as a result. That is, the smaller the editing distance, the more similar the two strings. The longest common substring algorithm is an algorithm that finds the longest among substrings that a given string has in common. Substrings are included even if they are not contiguous, and as the length of the longest common substring is longer, two strings can be considered similar.

On the other hand, it is possible to measure the similarity of two time series data using not only these algorithms but also a dynamic time warping technique. For example, by increasing the x-axis of data to be compared by n times and matching it with specific data, it is possible to compare whether the graphs of the two data match. It can be determined that the shorter the movement distance on the matrix between the two data, the higher the similarity.

Meanwhile, the disclosed technology uses an editing cost algorithm modified from an editing distance algorithm. The search apparatus may calculate the similarity by comparing the characteristics of the inflection point, which is a characteristic part of the sequence, using the editing cost algorithm. The main characteristics of the editing cost algorithm are that a different weight is given to the editing distance algorithm when calculating the similarity, and the result value is taken as the minimum value in the last row rather than the last value. Here, the other weight means a change in slope for each of the inflection points and a ratio of a distance between the inflection points. For example, a feature value of a specific inflection point at which a change in slope and distance ratio among a plurality of inflection points starts may be extracted and used as inflection point information.

After extracting a plurality of inflection points included in the input cover song sound source, the search apparatus may extract inflection point information according to the above-described weight. In addition, the similarity between sequences may be calculated by comparing the inflection point information of the input sound source with the previously stored inflection point information of the original song.

On the other hand, the conventional editing distance algorithm takes the (M,N) value of the matrix calculated when the length of the input data is M and N, respectively, as the result value, but in the editing cost algorithm, the representative phrase of the original song is included in the cover song. The purpose is to search for whether the song is included, so after calculating all the editing costs with the cover song based on the length of the representative passage of the original song, the smallest value among them can be taken as the result.

Meanwhile, as described above, instead of using the sequence of the melody line as it is, the search apparatus may extract an inflection point and calculate the sequence similarity between the cover song and the original song based on the characteristic. Since the similarity is compared based on the inflection point, the memory usage is lower compared to the conventional algorithm because it is not necessary to store only the starting point or the end part of the part where the same note continues for a long time or the part where the degree of change is kept constant. has the advantage of being reduced. In addition, since the front and rear slope of the inflection point and the distance relation between the inflection points can be considered, it is possible to infer even omitted information not stored in the memory. Accordingly, the memory usage and calculation amount are reduced, but the search accuracy can be increased compared to the prior art. In addition, since a relative criterion can be applied to each sequence based on the inflection point, search accuracy can be maintained even if there is a part with a different speed for each song.

2 is a flowchart of a music search method using an inflection point of a melody line according to an embodiment of the disclosed technology. Referring to FIG. 2 , the music search method 200 may be sequentially performed through a search device, and includes a step 210 of selecting a plurality of inflection points included in an input sound source, a change in slope for each of the plurality of inflection points, and each Step of extracting inflection point information based on the distance ratio between the inflection points (220) Comparing the inflection point information of the original song with the inflection point information of the input sound source and calculating the similarity between sequences (230) and using the input sound source as a representative phrase of the original song and normalizing (240) by dividing by length.

In step 210, the detection device selects a plurality of inflection points included in the input sound source. The plurality of inflection points selected in step 210 include those having a constant height change or a constant distance ratio. That is, all feature points determined to be inflection points may be extracted through the detection device.

In step 220, the detection apparatus extracts inflection point information based on a change in slope of each of the plurality of inflection points and a distance ratio between the respective inflection points. The inflection points extracted here may be those in which the change in slope and the change in distance ratio are different from the previous ones. For example, a feature value of a specific inflection point at which a change in slope and a change in distance ratio among the plurality of inflection points starts may be extracted as inflection point information. Inflection point information in the form of sequence data can be extracted by selecting these inflection points and arranging them in time series order.

In step 230, the detection device compares the inflection point information of the original song with the inflection point information of the input sound source to calculate the similarity between the sequences. Since the inflection point information is converted into sequence data in step 220 above, it is possible to compare whether the sequence of the original song and the sequence of the input sound source are similar to each other. In order to compare the similarity of the two sequences, the detection device slides the sequence for the inflection point information of the representative phrase of the original song one by one with respect to the sequence for the inflection point information of the input sound source. can In order to process this calculation process, the detection device may use an editing cost algorithm.

In step 240, the input sound source is normalized by dividing it by the length of the representative passage of the original song. Here, normalization means dividing the input sound source by the length of the representative passage of the stored original song, and it means normalizing the editing cost for the input sound source. For example, the sound source having the lowest editing cost may be determined as the original music for the input sound source. The search apparatus may determine that the lowest editing cost for the input sound source among the plurality of original songs stored in the database is the original song of the input sound source, and output information about the song as a search result.

3 is a block diagram of a music search apparatus using an inflection point of a melody line according to an embodiment of the disclosed technology. Referring to FIG. 3 , the music search device 300 includes an input device 310 , a storage device 320 , and a processing device 330 .

The input device 310 receives a sound source. The input device 310 may receive a sound source input from a user or may receive a sound source transmitted from an external terminal. The input device 310 may be implemented in the form of an input interface of the music search device 300 . For example, it may be an interface capable of inputting sound source data, such as a keyboard or a mouse.

The storage device 320 stores a plurality of sound sources. The storage device may be implemented as a memory having a capacity to store a plurality of sound sources. Such a memory may be a non-volatile memory capable of continuously maintaining stored information even when power is not supplied, and some of the plurality of stored sound sources may be updated according to an input of a manager.

The processing device 330 selects a plurality of inflection points included in the input sound source, extracts inflection point information based on a change in slope for each of the plurality of inflection points and a distance ratio between each inflection point, and extracts the inflection point information of each of the plurality of sound sources and The original song for the input sound source is searched by comparing the inflection point information of the input sound source. The processing device 330 may be implemented as a processor or an AP of the music search device 300 . The processing device 330 may search for an original song among the plurality of sound sources by calculating a sequence similarity between the inflection point information of each of the plurality of sound sources stored in the storage device 320 and the inflection point information of the input sound source.

Meanwhile, the music search apparatus 300 may further include an output apparatus for outputting a search result. The output device may output information about a sound source having the highest similarity with the input sound source among the plurality of sound sources as a search result.

Meanwhile, the music search apparatus 300 as described above may be implemented as a program (or application) including an executable algorithm that can be executed in a computer. The program may be provided by being stored in a temporary or non-transitory computer readable medium.

The non-transitory readable medium refers to a medium that stores data semi-permanently, rather than a medium that stores data for a short moment, such as a register, cache, memory, etc., and can be read by a device. Specifically, the various applications or programs described above are CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM (read-only memory), PROM (programmable read only memory), EPROM (Erasable PROM, EPROM) Alternatively, it may be provided while being stored in a non-transitory readable medium such as an EEPROM (Electrically EPROM) or a flash memory.

Temporarily readable media include: Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (Enhanced) SDRAM, ESDRAM), Synchronous DRAM (Synclink DRAM, SLDRAM) and Direct Rambus RAM (Direct Rambus RAM, DRRAM) refers to a variety of RAM.

4 is a diagram schematically illustrating a plurality of inflection points included in a sound source. In FIG. 4 , the x-axis represents the timestamp and the y-axis represents the MIDI note number. As indicated by a circle, a part in which the aspect of data changes can be defined as an inflection point. For example, the same difference in MIDI note numbers as in the 5th and 6th lines means that the same value is maintained or that the slope is constantly increased, decreased or sloped without changing the slope, so these inflection points can be excluded from being included in the inflection point information. . According to this process, unnecessary memory consumption may be reduced.

5 is a diagram illustrating calculating the similarity in a sliding manner. In FIG. 5 , the x-axis represents a timestamp and the y-axis represents an arbitrary data value. You can compare the representative verse data of the original song by sliding it sideways one by one in order to explore whether the cover song data indicated by the black line contains the representative verse data of the original song indicated by the blue line at the bottom. As shown in FIG. 5 , since there are phrases that match 18 to 22 parts on the timestamp, it can be confirmed that the cover song input into the search device includes the representative phrases of the original song.

6 is a view showing a result of selecting an inflection point. In the editing cost algorithm described above with reference to FIGS. 4 and 5 , when the same value continues through several frames, both the beginning and the end are recognized as inflection points, so there may be a deviation from the inflection point recognized when listening to an actual song. In addition, if the slope change amount is constant, such as when the value increases or decreases to a certain degree, if the value changes per frame, it is not included in the inflection point information as originally intended. All of them may be included in the inflection point information. That is, unnecessary memory consumption may occur.

Therefore, in the disclosed technology, in order to overcome this limitation, the selection of the inflection point can be performed by dividing it into two steps. For example, as shown in FIG. 6 , lines 1 to 11 may be regarded as candidates for inflection points because they are the starting portions where the values change primarily and the portions in which the slope changes before and after. Naturally, the beginning of line 1 and the end of line 11 are characteristics that are necessarily included in the inflection point information. Here, lines 3 and 4 can be excluded from screening because the slope does not change. In FIG. 6 , an inflection point denoted by O denotes a selected candidate inflection point, and an inflection point denoted by an X denotes an inflection point excluded from the screening. This can be processed as the first step of selecting the inflection point.

Meanwhile, in step 2, the search apparatus selects candidate inflection points again and finally extracts inflection point information. The inflection points finally selected may be selected only if the slopes are not constant before and after the inflection points. For example, the inflection point of row 3 in FIG. 6 may be excluded because the front and rear slopes are maintained the same. According to this process, it is possible to generate sequence-type inflection point information by dividing inflection points into two steps. The search apparatus may search for an original song by extracting a sequence of a plurality of original songs stored in the database according to the same process and comparing the sequence with the input sound source sequence, respectively.

A method and apparatus for searching music using an inflection point of a melody line according to an embodiment of the disclosed technology has been described with reference to the embodiment shown in the drawings for better understanding, but this is only an example, and those of ordinary skill in the art It will be appreciated that various modifications and equivalent other embodiments are possible therefrom. Accordingly, the true technical protection scope of the disclosed technology should be defined by the appended claims.

Claims (12)

selecting, by a search device, a plurality of inflection points included in the input sound source;
extracting, by the search apparatus, information on inflection points based on a change in slope of each of the plurality of inflection points and a distance ratio between the respective inflection points;
calculating a similarity between sequences by comparing the inflection point information of the original song with the inflection point information; and
and normalizing the input sound source by dividing it by the length of a representative passage of the original song. The method of claim 1, wherein calculating the similarity comprises:
Music search using the inflection point of a melody line that calculates whether the representative phrase of the original song is included in the sound source by sliding the sequence for the inflection point information of the representative phrase of the original song one by one with respect to the input sequence for the inflection point information of the sound source Way. The method of claim 1,
The search apparatus calculates the similarity between the sequences by using an editing cost algorithm. A music search method using an inflection point of a melody line. The method of claim 1,
The search apparatus extracts, as the inflection point information, a feature value of a specific inflection point where a change in slope and a distance ratio among the plurality of inflection points starts, as the inflection point information. The method of claim 1,
The search apparatus extracts a plurality of inflection point information for a plurality of sound sources stored in a database, respectively, and compares the plurality of inflection point information with the inflection point information of the input sound source. 6. The method of claim 5,
The search apparatus outputs information on a sound source having the highest similarity with the input sound source among the plurality of sound sources as a search result. A music search method using an inflection point of a melody line. an input device for receiving a sound source;
a storage device for storing a plurality of sound sources; and
Selecting a plurality of inflection points included in the input sound source, extracting inflection point information based on a change in slope for each of the plurality of inflection points and a distance ratio between the respective inflection points, and extracting the inflection point information of each of the plurality of sound sources and the input A music search apparatus using an inflection point of a melody line comprising a; a processing unit that compares the inflection point information of the sound source to search for an original song for the input sound source. 8. The method of claim 7,
The processing device calculates a degree of sequence similarity between the inflection point information of each of the plurality of sound sources and the inflection point information of the input sound source, and searches for the original song among the plurality of sound sources by using an inflection point of a melody line. 9. The method of claim 8,
The processing device divides the input sound source by the length of the representative phrase of the original song and normalizes the music search device using the inflection point of the melody line. 8. The method of claim 7,
The processing device slides the sequence for the inflection point information of the representative phrase of the original song one by one with respect to the sequence for the inflection point information of the input sound source, and calculates whether the representative phrase of the original song is included in the sound source. music retrieval device. 8. The method of claim 7,
The processing device extracts, as the inflection point information, a feature value of a specific inflection point, at which a change in slope and a distance ratio, among the plurality of inflection points, as the inflection point information. 8. The method of claim 7,
The music search device further comprises an output device,
The output device is a music search device using an inflection point of a melody line for outputting information on a sound source having the highest similarity with the input sound source among the plurality of sound sources as the search result.

Images