KR20060019096A - Hummed-based audio source query/retrieval system and method - Google Patents

Abstract

1. TECHNICAL FIELD OF THE INVENTION

The present invention relates to a humming-based sound source query / search system and a method thereof, and a computer-readable recording medium having recorded thereon a program for realizing the method.

2. The technical problem to be solved by the invention

The present invention is intended to find a user from the audio information (sound source information) stored in the server using the humming of the remote user as a search key in consideration of the different pitches and tempo, individual singing, network load, etc. It is an object of the present invention to provide a humming-based sound source retrieval system and a method for searching and providing a corresponding sound source, and a computer-readable recording medium recording a program for realizing the method.

3. Summary of Solution to Invention

According to an aspect of the present invention, there is provided a sound source query / search system comprising: a humming input unit for receiving a part of a piece of music to be searched by a user by humming; Section detecting means for identifying a boundary between a start point and an end point of a note from the input humming signal; Feature amount extracting means for extracting pitch information of the note detected by said section detecting means and feature information of a sound field; Symbol melody representation means for taking a relative value between each note to generate a symbol melody sequence; Metadata generating means for generating metadata information of each audio information (sound source information); Metadata storage means for storing the metadata information; Similarity for receiving the symbol melody sequence from the symbol melody representation means and calculating the similarity between the symbol melody sequence and the feature amount of each sound source stored in the metadata storage means using dynamic programming (DP) matching. Calculation means; Distance-based classification means for arranging the distance information calculated by the similarity calculation means and extracting the metadata information from the metadata storage means; And search result output means for outputting a search result from the distance based classification means.

4. Important uses of the invention

The present invention is used for sound source query / search.

Humming, Audio, Audio, Query, Search, Metadata, Similarity

Description

Humming-based audio source query / retrieval system and method             

1 is a configuration diagram of an embodiment of a humming-based sound source query / search system according to the present invention;

2 is an example of application record configuration of the metadata DB used in the present invention;

3 is an exemplary view of the sound field (Duration, IOI) characteristics used in the present invention,

4 is an exemplary diagram of music of a sound source DB according to an embodiment of the present invention;

5 is an exemplary DP matching diagram used in the present invention;

6 is a flowchart illustrating an embodiment of a humming-based sound source query / search method according to the present invention.

* Explanation of symbols for the main parts of the drawings

10: input humming query 20: melody extractor

21: interval detector 22: feature extractor

23: symbol melody presenter 30: search results output unit

40: Search Engine 41: Melody Similarity Calculator                 

42: distance-based classifier 50: metadata generator

60: sound source DB 70: metadata DB

The present invention relates to a humming-based sound source query / search system and a method thereof, and a computer-readable recording medium recording a program for realizing the method, wherein a humming of a remote user is used as a search key. Humming-based sound source query / search system and method thereof capable of searching for and providing a corresponding sound source to be searched for by a user from audio information (sound source information) stored in a server, and can be read by a computer recording a program for realizing the method. To a recording medium.

In recent years, with the development of the Internet, wireless networks, and high-speed networks, the use of digital multimedia contents (audio, video, images, music videos, etc.) is rapidly increasing, and various contents are developed and developed in many fields such as education, entertainment, and science. It is being used in applications such as multimedia database, digital TV, internet broadcasting and music video. The rise of such multimedia information has a great influence on modern life, and in particular, it is changing a lot of ways of thinking and using media.

In order to efficiently use and manage a large amount of multimedia contents, an information retrieval system is required.                         

However, the multimedia information retrieval method has been mainly developed based on texts and images, but there is an increasing demand for retrieval of video and audio information. In particular, as the Internet becomes more popular and audio encryption and decryption technologies are developed, services for providing music using the Internet are increasing. These Internet sites, called so-called Music On Demand (MOD) or Audio On Demand (AOD), store a vast amount of music (music) and selectively provide music to the user's needs.

Such audio information retrieval methods can be classified into text-based retrieval and contents-based retrieval.

The text-based search is a method of searching by using bibliographic information representing characteristics of a piece of music, such as the name of a song, a singer's name, lyrics, a composer, and a player's name, as keywords.

On the other hand, the content-based search is to use the content of the music, such as the search key, there is a QBH (Query-by-Humming) method using the humming of the user (users) as the input information used. The present invention is a technique of searching for audio information (sound source information) desired by a user by singing a melody, which is a predominance rate for a piece of music, as a search clue.

Now, let's take a look at the background of introduction of audio search method by humming query.

In general, a user may search for a desired song by selectively following a web page previously classified and organized by a service provider using a specific criterion, or sometimes by searching for a part of a song title by entering a string. . However, such a search method by dictionary classification or search method by song title is possible only when the user knows in advance the title of the song to be searched for. In addition, as the amount of music material to be searched is enormous, it becomes difficult and complicated to classify the music material, and also faces the limitations of the title-based search technique.

In the face of so many popular songs, modern people sometimes don't remember song titles or lyrics, but they often remember one or two rhythms or rhythms. For example, a few rhythms hover over the head, but often the title does not come to mind. In such a case, the conventional music data searching method by the title or the searching method by dictionary classification is hardly helpful. What you really need is a way to search for music based content.

The content-based means the search by the song itself, that is, the melody, not the song title or lyrics as described above. Advances in digital signal processing, music data representation and storage techniques, and computer hardware technologies have made this method of content-based music data retrieval possible. For example, the music library often calls for one or two measures to sing or hum to find a particular piece of music. However, given the vast amount of records, searching for a particular piece of music would be enormous.

Considering that the search for such a huge amount of data is made possible through the Internet as the electronic library is developed, an effective method of searching music based on melody is essential. Through this content-based music data search, users can search and find all songs including the same rhythm by simply singing or humming a few measures. Thus, such a search method is helpful for both music hobbyists and professionals.

Content-based music data retrieval techniques will be a very important component in future electronic music libraries. Professional musicians may analyze the music of a particular composer. For example, you can find and analyze recurring themes or melodies that appear frequently.

Of course, the content-based audio retrieval method does not exist in the prior art, but in particular, in order to provide a high efficiency, high speed audio retrieval service by humming in a real client-server environment, special consideration should be given. For this reason, it is necessary to solve singing errors such as the insertion and dropping of notes due to vague memories at the time of accommodating different pitches and tempos for each individual. Another consideration is that transmitting the humming sound signal to the server side as it is, the transmission data size is huge, and if many users request the search at the server side at the same time, the increase in the traffic on the network and the excessive calculation amount on the server side, etc. In consideration of the size of the transmission data is an important factor. Therefore, more efficient and effective melody extraction method is required.

The full-fledged study of audio information retrieval system using humming started in the 1990s, but is more active in recent years. Examples include systems such as Query-by-Humming, MiDiLib, MELDEX, SuperMBox, and SoundCompus.

In order to implement an efficient audio retrieval system, as mentioned above, individual pitches and tempos are accommodated by individuals, and a song mistake such as insertion or dropping of notes due to ambiguous memory at the time of humming is considered. Apart from this, it is important to consider that even if the hummed query is perfect, it is difficult to accurately convert the input hum signal into the music notation used for melody matching.

Although various methods for melody expression and matching have been attempted at various angles in the related art, performances have not yet been satisfactory. In addition, in order to operate the implemented retrieval system in the Internet environment, it is necessary to reduce the size of transmission data considering the load of the network. In order to solve such a wide variety of problems, not only an efficient method of expressing melody that is efficient for network resources and resistant to user error is required, but also a more effective method of extracting information in melody extraction.

Therefore, in consideration of the different pitches and tempo, the number of songs, and the network load for each individual, there is an urgent need for a method of efficiently expressing the melody, which is the mainline rate, from the humming query, and efficiently searching for and providing the corresponding sound source.

The present invention has been proposed to meet the above demands, and is stored in a server using a humming of a remote user as a search key in consideration of different pitches and tempo, individual singing, and network load. To provide a humming-based sound source retrieval system and method and a computer-readable recording medium recording a program for realizing the method to search and provide the sound source that the user wants to find from the audio information (sound source information) The purpose is.

Other objects and advantages of the present invention can be understood by the following description, and will be more clearly understood by the embodiments of the present invention. In addition, it will be readily appreciated that the objects and advantages of the present invention may be realized by the means and combinations thereof indicated in the claims.

In order to achieve the above object, the present invention provides a sound source query system of a client, comprising: a humming input means for receiving a part of a piece of music to be searched by a user as a humming; Section detecting means for identifying a boundary between a start point and an end point of a note from the input humming signal; Feature amount extracting means for extracting pitch information of the note detected by said section detecting means and feature information of a sound field; Symbol melody expression means for taking a relative value between each note and generating a symbol melody sequence and transmitting the same to a server; And outputting "metadata information according to a similarity measurement result between" symbol melody sequence "using" dynamic programming (DP) matching "and" feature amount of each sound source stored in the database "from the server (search results). Characterized in that it comprises a search result output means for.

The present invention provides a sound source retrieval system of a server, comprising: metadata generating means for generating metadata information of each audio information (sound information); Metadata storage means for storing the metadata information; Receives the "symbol melody sequence which takes the relative value of each note in the user's humming signal" from the client, and uses the dynamic programming (DP) matching of each symbol melody sequence and the metadata stored in the metadata storage means. Similarity calculating means for calculating a similarity with the feature amount; And distance-based classification means for arranging the distance information calculated by the similarity calculation means, extracting the metadata information from the metadata storage means, and transmitting the metadata information to the client.

In addition, the present invention provides a sound source query / search system, comprising: humming input means for receiving a part of a piece of music to be searched from a user by humming; Section detecting means for identifying a boundary between a start point and an end point of a note from the input humming signal; Feature amount extracting means for extracting pitch information of the note detected by said section detecting means and feature information of a sound field; Symbol melody representation means for taking a relative value between each note to generate a symbol melody sequence; Metadata generating means for generating metadata information of each audio information (sound source information); Metadata storage means for storing the metadata information; Similarity for receiving the symbol melody sequence from the symbol melody representation means and calculating the similarity between the symbol melody sequence and the feature amount of each sound source stored in the metadata storage means using dynamic programming (DP) matching. Calculation means; Distance-based classification means for arranging the distance information calculated by the similarity calculation means and extracting the metadata information from the metadata storage means; And search result output means for outputting a search result from the distance-based classification means.

On the other hand, the present invention for achieving the above object, in the sound source query method applied to the sound source query / search system, a humming quality input step of receiving a part of the music to be searched from the user by humming (humming); An interval detection step of identifying a boundary between a start point and an end point of a note from the input humming signal; A feature amount extraction step of extracting feature information of the pitch and sound field of the detected note; A symbol melody representation step of generating a symbol melody sequence by taking a relative value between each note and transmitting it to a server; And "metadata information according to a similarity measurement result between" symbol melody sequence "using" dynamic programming (DP) matching "and" feature amount of each sound source stored in the database "from the server (search results). Characterized in that it comprises a search result output step.

The present invention provides a sound source search method applied to a sound source query / search system, the method comprising: generating and storing metadata information of each audio information (sound source information) and storing the metadata information in a metadata DB; A feature of each sound source stored in the symbol melody sequence and the metadata DB using dynamic programming (DP) by receiving a "symbol melody sequence from each user's humming signal from the client". A similarity calculation step of calculating the similarity with the quantity; And sorting the calculated distance information, extracting the corresponding metadata information from the metadata DB, and transmitting the distance-based classification and result transmission to the client.

The present invention also provides a sound source query / search method applied to a sound source query / search system, comprising: a humming input step of receiving a part of a piece of music to be searched by a user as a humming; An interval detection step of identifying a boundary between a start point and an end point of a note from the input humming signal; A feature amount extraction step of extracting feature information of the pitch and sound field of the detected note; A symbol melody expression step of generating a symbol melody sequence by taking a relative value between each note; A similarity calculation step of receiving the symbol melody sequence and calculating similarity between the symbol melody sequence and metadata information (characteristic amount of each sound source) of each audio information (sound source information) using dynamic programming (DP) matching; And sorting the calculated distance information, and outputting corresponding metadata information and outputting a search result.

On the other hand, the present invention for achieving the above object, for the humming-based sound source query, a humming quality input function for receiving a part of the music to be searched by the user in the sound source query system having a processor as a humming (humming) ; A section detection function for identifying a boundary between a start point and an end point of a note from the input humming signal; A feature amount extraction function for extracting pitch information of the detected note and feature information of a sound field; A symbol melody expression function for generating a symbol melody sequence by taking a relative value between each note and transmitting it to a server; And "metadata information according to the result of similarity measurement between" symbol melody sequence "using" dynamic programming (DP) matching "and" feature amount of each sound source stored in the database "from the server (search results). A computer readable recording medium having recorded thereon a program for realizing a search result output function is provided.

The present invention provides a metadata generation and storage function of generating metadata information of each audio information (sound source information) and storing the metadata information in a metadata DB in a sound source search system having a processor for searching for a humming-based sound source; A feature of each sound source stored in the symbol melody sequence and the metadata DB using dynamic programming (DP) by receiving a "symbol melody sequence from each user's humming signal from the client". A similarity calculation function for calculating similarity with the quantity; And arranging the calculated distance information, extracting the metadata information from the metadata DB, and transmitting the program to the client to provide a distance-based classification and result transmission function. do.

In addition to the development of the Internet, wireless networks, and high-speed networks, multimedia contents have been developed and produced in many fields such as education, entertainment, and science, and are being used in applications such as multimedia databases, digital TV, Internet broadcasting, and music videos. The present invention can be widely used in the technical field of searching for and using audio information (sound information) desired by a user in such a multimedia database.

The above objects, features and advantages will become more apparent from the following detailed description taken in conjunction with the accompanying drawings, whereby those skilled in the art may easily implement the technical idea of the present invention. There will be. In addition, in describing the present invention, when it is determined that the detailed description of the known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a configuration diagram of an embodiment of a humming-based sound source query / search system according to the present invention, and illustrates an example of searching for audio information (sound source information) for a humming query in a client-server environment.

As illustrated in FIG. 1, a humming-based sound source query / search system according to the present invention includes a humming input unit 10 for receiving a part of music to be searched by a user as a humming, and an input humming. Characteristic amount for extracting characteristic information of pitch and sound field (IOI) of the notes detected by the interval detector 21, and interval detector 21 for identifying the boundary between the start and end points of a note from the interrogation signal An extractor 22, a symbol melody presenter 23 for generating a symbol melody sequence by taking a relative value between each note, and a metadata generator 50 for generating metadata information of each audio information (sound source information) And a symbol melody sequence from the metadata database 70 storing metadata information and the symbol melody presenter 23, and using symbol programming using dynamic programming (DP) matching. The melody similarity calculator 41 for calculating the similarity between the desequence and the feature amount of each sound source stored in the metadata database 70 and the distance information calculated by the melody similarity calculator 41 are arranged, A distance-based classifier 42 for extracting data information from the metadata database 70 and a search result output unit 30 for outputting a search result from the distance-based classifier 42.

The server constructs a database 60 of various audio information (sound source information) through music score, piano, electronic musical instrument, MP3, music CD, etc., and extracts the melody information which is the predominant rate, in the form of metadata, The metadata DB 70 of the relative sound equipment series is created and constructed. Therefore, metadata for each audio (sound source) is stored in the metadata DB 70 as shown in FIG. 2, such as symbol melody sequence (metadata), audio name, file name, file size, playback time, and the like. Various information is recorded.

The sound source query / search system according to the present invention has a client-server type structure, and the melody extractor 20, which is the core thereof, is composed of a range detector 21, a feature extractor 22, and a symbol melody expresser 23. The search engine 40 includes a melody similarity calculator 41 and a distance-based classifier 42. In addition, the sound source DB 60 and the metadata DB 70 can be configured as one or a separate DB.

The client side is connected to the server side via an information communication network such as the Internet (WEB), wireless Internet (WAP), LAN (LAN), fixed wireless LAN (LAN), mobile wireless LAN (portable Internet).

Referring to the humming query / search process schematically, first, when a user freely hums a melody of a piece of music to be searched for, the melody extractor 20 of the client detects a note from the input hum and then features necessary for the search. Extract the amount. Then, a symbol melody sequence that is a relative feature value is generated.                     

The search engine 40 on the server side then measures the similarity between the input hum and the music (sound source) of the database by using dynamic programming (DP) matching. At this time, the audio information list (sound source information list) as a search result is presented to the user with various kinds of information about the audio such as a song name / similarity value / matched portion sorted in order of high similarity.

Then, the user may select and listen to a desired song name from the list presented as a search result or perform a subordinate operation such as downloading.

Then, the function of each component of the humming query / search system according to the present invention will be described in detail.

Melody input as a humming query is accomplished by the user singing through a microphone a portion of the piece of music he remembers at free pitch, free tempo. The humming method uses a method of singing a single note in the form of syllables (eg, tatata, cha cha cha, etc.) consisting of whistle or unvoiced sound, beginning with burst sound and vowel voice sound. The advantage of this method is that it is easy to extract stable melody information for the whole system, and it is a melody input method that is easy for the user.

The interval detector 21 serves to identify a boundary between an onset and an offset of a note from the input humming signal.

The interval detection method based on amplitude, which is mainly used in the related art, is a method of performing event detection by setting a threshold value suitable for detection of a single note interval. However, this method is likely to detect one note separated into two or three notes, and conversely, it is also highly likely that a plurality of notes are combined into one. Therefore, the error of the section detection may seriously affect the performance of the audio retrieval system.

The sound field, which is a characteristic used for interval detection, may be defined as "Duration" and IOI (Inter Onset Interval) as shown in FIG. Here, Duration means the difference between the start time of a note and the end time of that note. On the other hand, IOI or span means the difference between the start time of a note and the start time of the next note.

In the present invention, IOI is used as the feature amount of the sound field. In most cases, when a user sings, the IOI is often cut off in staccato form for long notes (e.g., more than half a note) rather than the exact length of the actual sound field. Using it yields better search results than using Duration. In addition, since the last note of the humming does not have any information about the IOI, it is better not to use it.

The feature extractor 22 extracts the feature through the pitch and identification of the sound field IIO. At this time, the sound field information is obtained in the above-described section detection process. In addition, the pitch is obtained by extracting the fundamental frequency through the Cepstrum analysis for the period of stable amplitude in the detected note.

The symbol melody presenter 23 converts a sequence of the identified pitch and the sound field IIO into respective deltaPitch and relative sound equipment, thereby representing a symbol melody expression sequence. Create and send it to the server side. In this case, the relative pitch (deltaPitch) is normalized so that the difference of the semitone is 100 cents, and the relative sound equipment (IOIratio) expresses the ratio with the immediately previous sound field as a percentage (%). In this case, the reason for converting only the information necessary for the search into the symbol melody sequence without transmitting the sound signal as it is to transmit to the server side is to minimize the amount of traffic required.

The server has a sound source (audio) DB 60 and a metadata DB 70, and performs searching processing of music pieces. At this time, the search uses DP matching to find the similarity between the time series input from the client and each music sequence stored in the metadata DB 70 as the sum of Euclidean distance or absolute difference value. Then, the song having the closest sequence to the matching result input sequence is transmitted to the client as a candidate for the correct answer.

Although humming the same song in the sound source database 60, the following points should be taken into consideration for the user's humming. First, individual differences can cause pitch height and speed to differ from the reference patterns included in real databases when users humming. Secondly, due to the ambiguous memory of the user, insertion and dropping of notes during humming occur.

Therefore, in order to solve the first problem, normalization of the humming data is required. A typical melody expression method normalizes data using a span ratio (IOIratio) and a relative pitch difference (deltaPitch) relative to successive notes. In addition, the second problem can be solved by using a DP matching algorithm.

As a detailed example of the DP matching, it is assumed that the following time series values are obtained in the feature extractor 22 from the input acoustic signal.

-Humming sound field time series: {245, 123, 247, 250, 249, 124, 125, 510} [unit: ms]-humming sound time series: {260, 292, 331, 393, 392, 348, 330, 261} [Unit: Hz]

Regarding the identified sound field and time series of pitches, the symbol melody presenter 23 performs the relativity as follows.

-Humming relative sound equipment (IOIratio): {50.2 200.8 101.2 99.2 49.8 100.8 408.0} [unit:%]-delta pitch: {198 201 297 2 -203 -98 -394} [unit: cent]

When any song of the sound source database 60 is composed of a combination of notes as shown in FIG. 4, this is shown as time series as follows.

-DB Song Sound Field Time Series: {Quarter Note, Eighth Note, Quarter Note, Quarter Note, Quarter Note, Eighth Note, Eighth Note, Quarter Note}-DB Song Note Time Series: {Do Remi Brush brush wave

The relative value of the metadata DB 70 is as follows.

-DB song relative sound equipment (IOIratio): {50 200 100 100 50 100 400} [unit:%]-DB song relative pitch (deltaPitch): {200 200 300 0 -200 -100 -400} [unit: cent ]

The search engine 40 outputs a search result by calculating a similarity degree using DP matching on the relative value of the input humming and the relative value of the DB song.

This can be generalized and expressed as a mathematical expression as follows.

The DP matching matrix g (i, j) at any point i, j for a time series R of a piece of music having a note length I and a time series Q of a note length of an input hum J in the sound source database 60 is shown in FIG. Obtained as

With that the cost of the distance of the element Q and R database d, emergency database length of time series of notes I _{R = {i 0, i 1} , i 2, ... … , _{I i-1}} of the input and Hamming length of time series of notes J _{Q = {j 0, j 1} , j 2, ... … , J _J-1} has been configured, if a relative sound equipment and the relative sets of higher order negative, the degree of similarity d (i, j) for the two time series i with j = | ij | Can be calculated repeatedly, and based on this assumption, the distance calculation for the similarity can be obtained by Equation 1 below.

Looking at the search process by the humming query described above in more detail with reference to FIG.

First, when the user humming query (601), the melody extractor 20 on the client side extracts the symbol melody sequence for this (602 ~ 604), to the search engine 40 on the server side through the information communication network, such as the Internet To pass.

That is, at the client side, the humming query input from the user is automatically detected as one note interval through the interval detector 21 (602). At this time, the feature extractor 22 extracts pitch and span information which are components of the note from the detected note (603). Subsequently, in the symbol melody presenter 23, since the actual humming music has a different tempo and tonality with respect to the music in the database, a symbol melody sequence having a compact information amount is obtained by taking a normalization value between the notes. Generate (604) and send it to the server side.

Subsequently, the symbol melody sequence transmitted through the network and the metadata information of the metadata DB 70 are measured through the similarity measurement of the search engine 40 (605). The data is transmitted to the client through the information communication network (607).

That is, on the server side, the melody similarity calculator 41 transmits the audio file (sound source file) of the sound source DB (audio DB) 60 to the metadata DB 70 via the metadata generator 50 and the metadata of FIG. 2. Record it in the form. The melody similarity calculator 41 calculates the similarity between the input symbol melody sequence and the feature amount of each audio in the metadata DB 70 (605). In this case, the similarity may be a method such as an Euclidian distance or a sum of absolute difference that measures a distance between two vectors using DP matching.

Thereafter, the distance-based classifier 42 sorts the calculated distance information in the order of the smallest distance value (606), extracts the corresponding metadata information from the metadata DB 70, and retrieves the Internet as search result information. It passes to the client side via (607).

Then, the user can download and enjoy the audio (sound source) from the link of the searched result list.

As described above, the method of the present invention may be implemented as a program and stored in a recording medium (CD-ROM, RAM, ROM, floppy disk, hard disk, magneto-optical disk, etc.) in a computer-readable form. Since this process can be easily implemented by those skilled in the art will not be described in detail any more.

The present invention described above is capable of various substitutions, modifications, and changes without departing from the technical spirit of the present invention for those skilled in the art to which the present invention pertains. It is not limited by the drawings.

The present invention as described above, the sound information (audio) information can be easily and efficiently searched by using humming through an information communication network such as the Internet, there is an effect that can be efficiently used for applications such as content-based audio search.

Claims (14)

In the sound source query system of the client, Humming input means for receiving a portion of a piece of music to be searched by a user for humming; Section detecting means for identifying a boundary between a start point and an end point of a note from the input humming signal; Feature amount extracting means for extracting pitch information of the note detected by said section detecting means and feature information of a sound field; Symbol melody expression means for taking a relative value between each note and generating a symbol melody sequence and transmitting the same to a server; And "Metadata information according to the result of similarity measurement between" symbol melody sequence "using" dynamic programming (DP) matching "and" feature amount of each sound source stored in the database "from the server (search results) Search result output means Humming-based sound source query system comprising a. The method of claim 1, The characteristic sound field used in the section detection means, A Humming-based sound source query system, characterized by using an Inter Onset Interval (IOI), which is a difference between a start time of a note and a start time of the note and a start time of the next note with respect to a start time of the note. The method of claim 2, The feature amount extraction means, Feature amount is extracted through identification of pitch and sound field (IOI), where sound field information is obtained from the interval detecting means, and pitch information is subjected to a spectral analysis for a section with stable amplitude in the detected note. Humming-based sound source query system, characterized in that obtained by extracting the fundamental frequency through. The method of claim 1, The symbol melody representation means, Humming, characterized by generating a symbol melody representation sequence by converting an identified pitch and a sequence of sound field (IOI) into respective deltaPitch and relative sound equipment (IOIratio). Sound source query system based. The method according to any one of claims 1 to 4, The server, Using DP matching, the similarity between the time series input from the sound source query system and each music sequence stored in the metadata DB is obtained by the sum of Euclidean distance or absolute difference value, and as a result of the matching, the sequence closest to the input sequence is obtained. Humming-based sound source query system, characterized in that the transmission to the sound source query system as a candidate for the correct answer. In the sound source search system of the server, Metadata generating means for generating metadata information of each audio information (sound source information); Metadata storage means for storing the metadata information; Receives the "symbol melody sequence which takes the relative value of each note in the user's humming signal" from the client, and uses the dynamic programming (DP) matching of each symbol melody sequence and the metadata stored in the metadata storage means. Similarity calculating means for calculating a similarity with the feature amount; And Distance-based classification means for arranging the distance information calculated by the similarity calculation means, extracting the metadata information from the metadata storage means, and transmitting the metadata information to the client. Humming-based sound source search system comprising a. The method of claim 6, The similarity calculation means, Using the Euclidian distance or the sum of absolute difference, which measures the distance between two vectors using DP matching, the inputted symbol melody sequence and each of the metadata storage means Humming-based sound source retrieval system, characterized in that it calculates the similarity with the feature amount of the sound source. The method according to claim 6 or 7, The distance-based classification means, Sorting the calculated distance information in the order of the smallest distance value, extracts the corresponding metadata information from the metadata storage means and delivers the metadata information to the client as search result information. Way. In the sound source query / search system, Humming input means for receiving a portion of a piece of music to be searched by a user for humming; Section detecting means for identifying a boundary between a start point and an end point of a note from an input humming signal; Feature amount extracting means for extracting pitch information of the note detected by said section detecting means and feature information of a sound field; Symbol melody representation means for taking a relative value between each note to generate a symbol melody sequence; Metadata generating means for generating metadata information of each audio information (sound source information); Metadata storage means for storing the metadata information; Similarity for receiving the symbol melody sequence from the symbol melody representation means and calculating the similarity between the symbol melody sequence and the feature amount of each sound source stored in the metadata storage means using dynamic programming (DP) matching. Calculation means; Distance-based classification means for arranging the distance information calculated by the similarity calculation means and extracting the metadata information from the metadata storage means; And Search result output means for outputting a search result from the distance-based classification means Humming-based sound source query / search system comprising a. In the sound source query method applied to the sound source query / search system, A humming input step of receiving a portion of a piece of music to be searched by a user through humming; An interval detection step of identifying a boundary between a start point and an end point of a note from the input humming signal; A feature amount extraction step of extracting feature information of the pitch and sound field of the detected note; A symbol melody representation step of generating a symbol melody sequence by taking a relative value between each note and transmitting it to a server; And A search for outputting "metadata information according to a similarity measurement result between" symbol melody sequence "using" dynamic programming (DP) matching "and" feature amount of each sound source stored in the database "from the server (search result) Result output stage Humming-based sound source query method comprising a. In the sound source search method applied to the sound source query / search system, A metadata generation and storage step of generating metadata information of each audio information (sound source information) and storing the metadata information in a metadata DB; A feature of each sound source stored in the symbol melody sequence and the metadata DB using dynamic programming (DP) by receiving a "symbol melody sequence from each user's humming signal from the client". A similarity calculation step of calculating the similarity with the quantity; And Sorting the calculated distance information, the distance-based classification and result transmission step of extracting the corresponding metadata information from the metadata DB to transmit to the client Humming-based sound source search method comprising a. In the sound source query / search method applied to the sound source query / search system, A humming input step of receiving a portion of a piece of music to be searched by a user through humming; An interval detection step of identifying a boundary between a start point and an end point of a note from the input humming signal; A feature amount extraction step of extracting feature information of the pitch and sound field of the detected note; A symbol melody expression step of generating a symbol melody sequence by taking a relative value between each note; A similarity calculation step of receiving the symbol melody sequence and calculating similarity between the symbol melody sequence and metadata information (characteristic amount of each sound source) of each audio information (sound source information) using dynamic programming (DP) matching; And Distance-based classification and search result outputting step of sorting the calculated distance information and outputting corresponding metadata information Humming-based sound source query / search method comprising a. For a humming-based sound source query, in a sound source query system having a processor, A humming input function for receiving a part of a piece of music to be searched from a user by humming; A section detection function for identifying a boundary between a start point and an end point of a note from the input humming signal; A feature amount extraction function for extracting pitch information of the detected note and feature information of a sound field; A symbol melody expression function for generating a symbol melody sequence by taking a relative value between each note and transmitting it to a server; And A search for outputting "metadata information according to a similarity measurement result between" symbol melody sequence "using" dynamic programming (DP) matching "and" feature amount of each sound source stored in the database "from the server (search result) Result output function A computer-readable recording medium having recorded thereon a program for realizing this. In the sound source search system equipped with a processor for a humming-based sound source search, A metadata generation and storage function for generating metadata information of each audio information (sound source information) and storing the metadata information in a metadata DB; A feature of each sound source stored in the symbol melody sequence and the metadata DB using dynamic programming (DP) by receiving a "symbol melody sequence from each user's humming signal from the client". A similarity calculation function for calculating similarity with the quantity; And The distance-based classification and result transmission function of sorting the calculated distance information, extracting the corresponding metadata information from the metadata DB, and transmitting the extracted metadata information to the client A computer-readable recording medium having recorded thereon a program for realizing this.

Images