CN101271457B - Music retrieval method and device based on rhythm - Google Patents
Music retrieval method and device based on rhythm Download PDFInfo
- Publication number
- CN101271457B CN101271457B CN2007100646076A CN200710064607A CN101271457B CN 101271457 B CN101271457 B CN 101271457B CN 2007100646076 A CN2007100646076 A CN 2007100646076A CN 200710064607 A CN200710064607 A CN 200710064607A CN 101271457 B CN101271457 B CN 101271457B
- Authority
- CN
- China
- Prior art keywords
- music
- melody
- user
- client
- search
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000000034 method Methods 0.000 title claims abstract description 30
- 230000033764 rhythmic process Effects 0.000 title claims description 34
- 230000005236 sound signal Effects 0.000 claims abstract description 22
- 239000000284 extract Substances 0.000 claims abstract description 7
- 239000012634 fragment Substances 0.000 claims description 13
- 238000012545 processing Methods 0.000 claims description 13
- 238000001914 filtration Methods 0.000 claims description 7
- 238000005520 cutting process Methods 0.000 claims description 6
- 230000003993 interaction Effects 0.000 claims description 4
- 230000005540 biological transmission Effects 0.000 claims description 3
- 230000008676 import Effects 0.000 claims description 3
- 230000008030 elimination Effects 0.000 claims description 2
- 238000003379 elimination reaction Methods 0.000 claims description 2
- 238000010606 normalization Methods 0.000 claims description 2
- 238000001208 nuclear magnetic resonance pulse sequence Methods 0.000 claims description 2
- 238000003672 processing method Methods 0.000 abstract 1
- 230000006870 function Effects 0.000 description 7
- 241000196324 Embryophyta Species 0.000 description 4
- 230000008878 coupling Effects 0.000 description 4
- 238000010168 coupling process Methods 0.000 description 4
- 238000005859 coupling reaction Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000005070 sampling Methods 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 230000007812 deficiency Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 244000287680 Garcinia dulcis Species 0.000 description 1
- 206010038743 Restlessness Diseases 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 230000002950 deficient Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 238000012797 qualification Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- GOLXNESZZPUPJE-UHFFFAOYSA-N spiromesifen Chemical compound CC1=CC(C)=CC(C)=C1C(C(O1)=O)=C(OC(=O)CC(C)(C)C)C11CCCC1 GOLXNESZZPUPJE-UHFFFAOYSA-N 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
- 238000005303 weighing Methods 0.000 description 1
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Electrophonic Musical Instruments (AREA)
- Reverberation, Karaoke And Other Acoustics (AREA)
Abstract
The present invention discloses a digital music search method and a device therefore, which can search the music with the designated melody by taking musical melody as the keyword. The present invention provides two melody input methods to a user: playing and humming. The humming input method adopts a series of signal processing methods to analyze the humming audio signal and extract the melodic information. A musical database adopts a reverse algorithm to compile an index, thus increasing the search efficiency. The device provided by the present invention is divided into a server terminal and a client terminal; the server terminal is to maintain the musical database and the index thereof and responds the search query of the client terminal; the client terminal is to collect the melodic input of the user, receive and display the search result of the server. The present invention utilizes the musical melody to search the music, thus compensating the shortcoming of the traditional text-based search method and helping the user to search the wanted music without text information. The user can use computers, mobile phones and other common equipment for music search.
Description
Technical field
The invention belongs to the computer technology application, be specifically related to digital music with melody, and make computer hardware and the communication apparatus device that this method can trouble-free operation as the search method of key word.
Background technology
Along with the growth of the geometric series of internet information amount, the information that how to find us to need from the information bank of magnanimity rapidly and exactly becomes a big bottleneck of people's internet usage.Content-based multimedia retrieval is an emerging research field, and it provides brand-new way of search to people: come searching multimedia information with multimedia itself.Multimedia messages has various ways such as audio frequency, video, image, animation, and wherein audio-frequency information occupies sizable ratio.And in the middle of audio frequency, music is again modal form.Present music retrieval is mainly searched for according to text keyword, music name for example, and the author sings the singer, special edition, school, the lyrics etc.But music itself is essentially different with text keyword, and the user uses key word search, and precondition is that the user must have gained some understanding to the target music, is familiar with associated text message.If the user is just interested in music rhythm itself, and to title of the song, text messages such as the lyrics are known nothing, existing method for searching music is just powerless.
Summary of the invention
Existing music key search technology, if do not know the text keyword of target music, this text keyword searching method is just powerless, in order to solve prior art problems, the purpose of this invention is to provide a kind of digital music search method and device based on melody.
In order to realize described purpose, first aspect present invention provides the music retrieval method based on melody, and step is as described below:
Step S1: specify one section melody waiting to look in the music as the melody key word of searching for;
Step S2:, obtain the digitizing melody signal through handling with specified melody key word input inquiry client device;
Step S3: the music in the music libraries is set up index, and this index embodies the melody characteristics of music, forms the musical database of indexation;
Step S4: by search engine the melody in the musical database of digitizing melody signal and generation is compared, select one group of one group of music that comprises the nominal key music rhythm from musical database;
Step S5: with the music selected according to the similarity degree sort descending of melody key word.
Described music input mode comprises: play input and humming input.
Described index is the produce index at the melody characteristics of melody fragment.
Described for the humming input mode, take following steps to obtain digitized melody signal:
Step S21: use audio collecting device to gather user's humming input;
Step S22: the sound signal of user's input is carried out pre-filtering handle, comprise direct current elimination, gain normalization, low-pass filtering treatment, obtain the audio frame sequence signal;
Step S23: the audio frame sequence signal is carried out time domain or frequency-domain analysis, extract the fundamental frequency sequence;
Step S24: the fundamental frequency sequence is further handled, comprised linearization, ask poor, obtain digitized melody signal.
In order to realize described purpose, second aspect present invention provides the music retrieval device based on melody, comprising:
At least one station server provides online music rhythm retrieval service;
Send online music rhythm retrieval request with at least one client terminal device, and the result of reception server query music melody.
Described client comprises:
Load module is used to import the music rhythm information that need search, and sends it to server end; The display module of Search Results, client obtains Search Results by network or other transmission modes from server end, and presents to the user.
Described load module comprises:
The audio collection unit is used to gather user's humming sound signal; The note collecting unit is used to gather the note melody signal that the user plays; Audio signal processing unit is converted into the music rhythm signal with the sound signal of audio collection unit collection.
Described server comprises:
Music data source interface unit is used to provide the various data sources of visit to obtain the interface of original music data; Data are obtained and analytic unit, be used to collect original music data, and music data is analyzed, and therefrom extract music rhythm information; The authorized index unit is used for that data are obtained the original music data of obtaining with analytic unit and sets up index according to its melody characteristics; Search unit, be used to receive the query requests of client load module, and the music of the identical or close melody of the melody key word that provides with the client load module is provided in search in the index that the authorized index unit generates, search result list is pressed the ordering of similarity degree inverted order, and feed back to the search result display module of client.
Described music data source interface unit provides the interface of following one or more data obtain manners:
Web: the mode of taking the Web network to grasp, music file and the information relevant with this music file are grasped in roaming on the internet automatically; File: the music file of storing in this locality or the network file system(NFS) is grasped and analyzes; Database: the music file that writes down in the database is extracted and analyzes.
Described client is one or more in the following equipment:
PC; The intelligence mobile device comprises: mobile phone, personal digital assistant, vehicle intelligent terminal etc.; Phone; Audio frequency and video amusement equipment with media-on-demand function: comprise Karaoke program request equipment.
When described client is selected personal computer equipment, the PC client is installed specific Web browser plug-in software from downloaded, during music retrieval Web website that user access server provides, the user interface that is used to the user to provide audio collection input and note to gather melody, and gather user's inquiry input, be sent to server by the internet.
When described client was selected intelligent mobile device, client was installed specific software, the user interface that this software provides audio collection and note to gather for the user, and gather user's inquiry input, be sent to server by wireless network.
When described client is selected telephone plant, server provides specific phone information service center, client is dialed this information service center number, utilize the phone numbers keyboard, or using telephone receiver respectively as note collection and audio collection input equipment, server and client are carried out information interaction by PSTN.
When described client is selected to have the audio frequency and video amusement equipment of media-on-demand function, client is equipped with digital piano keyboard equipment, or the fingerboard note that virtual piano keyboard software collection user is installed is imported, utilize microphone of carok collection user's humming input, server is a dedicated local server, and the scope of search is the local musical database of Karaoke.
The music list that described server is chosen for Search Results according to the similarity sort descending of Search Results with inquiry input melody, and sends it back client and shows.
The present invention provides a kind of new way of search for the user, uses the music rhythm search for music that is:.It has remedied the deficiency of tradition based on the text search mode, makes the user in the music of not knowing that search is wanted under the situation of text message; The present invention also is implemented on concrete hardware platform with this way of search, makes the user can use common equipment such as computer, and mobile phone etc. carry out music searching.
Description of drawings
Fig. 1 structural representation of the present invention
Embodiment
Below in conjunction with accompanying drawing the present invention and advantage are described in detail, be to be noted that described embodiment only is intended to be convenient to the understanding of the present invention, and it is not played any qualification effect.
The present invention mainly studies content-based music retrieval (Content based MusicRetrieval), and a kind of mode of coming search for music with music itself is provided.Specifically, be exactly that search engine returns one group of one group of music that comprises the designated key melody with the key word of a bit of music rhythm as search.Melody is as key word, and it is different from text keyword, the user can't be directly from the keyboard input, and need provide a kind of method of special input melody.The method that meets most people's custom is hummed input exactly, and the user as microphone, hums one section melody that needs are searched as long as use the audio collection input equipment.In addition, the user can also pass through virtual fingerboard, plays input.
Embodiments of the invention provide a complete computer technology application system platform, its function provides the music searching service based on melody, this platform has realized that simultaneously the music raw data obtains, the music raw data is analyzed, the musical database authorized index, online query, Audio Signal Processing, functions such as information feedback.This system platform has possessed the input of humming on terminal devices such as ordinary individual's computer, intelligent mobile device, phone, Karaoke program request equipment and fingerboard is played the condition of input music rhythm, and has possessed the condition that shows or reproduce Search Results on above these terminal devices to the user.
The present invention is organically combined by a plurality of functional modules and forms, and each functional module is finished specific function.The structure of system complete as shown in Figure 1.The present invention is based on the music retrieval device of melody, comprise that at least one computing machine provides online music retrieval service as server 2, send online music retrieval request with at least one client 1 terminal device, and the Query Result of reception server 2, server 2 has obtained and has stored the music rhythm database that comprises a large amount of music rhythm features from the several data source, and database is set up index.When receiving the query requests of client 1, the inquiry melody fragment of 2 pairs of user's inputs of server and the melody in the database compare, and filter out and inquire about the incoherent music of melody fragment, several remaining candidate's music are sorted according to the similarity degree with inquiry melody fragment, the music list after the ordering is returned client.Client 1 provides two kinds of inputting interfaces for the user, receives user's melody input and is translated into the digitizing melody signal that can be used for inquiring about.
In the structural drawing shown in Figure 1, parts in the frame of broken lines of left side are the modules in client 1 terminal device, comprise: load module 11 is gathered user's input and is sent to server 2, and search result display module 12 is presented to the user with the Query Result that server 2 returns.
Described load module 11 comprises: audio collection unit 111 and note collecting unit 113 are respectively applied for the humming input of gathering the user and play input; Audio signal processing unit 112, the sound signal that audio collection unit 111 is gathered is converted into music rhythm information.
User's humming input is gathered in audio collection unit 111.It is made up of audio collecting device and one section recorded program software.Audio collecting device microphone normally on PC and Karaoke program request terminal is generally on communicating terminals such as mobile phone and is subjected to microphone.It is driven by recording software, and the simulating signal of audio volume control is carried out digital collection by the sample frequency of recording software appointment, and the digital pulse sequence of gathering is stored in the storer of client 1.Because the fundamental frequency (first harmonic) of voice is usually in 2000Hz, according to the Nyquist sampling thheorem, be the digital signal that guarantees to gather occurrence frequency aliasing not, sample frequency should be greater than 2 times of the highest effective frequency.Because the present invention need analyze the harmonic wave of voice, is 8000Hz or 11025Hz so get sample frequency.Audio collection unit 111 each time spans of gathering are defaulted as 10 seconds, can according to circumstances set up on their own.
Audio signal processing unit 112, the sound signal that it gathers audio collection unit 111 is converted into the melodic information of music.112 pairs of sound signals of audio signal processing unit are carried out following processing:
The sound signal that step 1), audio collection unit 111 are collected contains DC component usually, and DC component causes the skew of signal-balanced position current potential, causes error for the low-frequency spectra analysis of signal.Therefore be necessary the DC component of erasure signal.Because direct current signal is invariant feature sometimes, makes all sampled point potential values deduct the equilibrium point potential value of the sampled signal overall situation, can eliminate DC component.The error of bringing for the difference of erasure signal power, audio signal processing unit 112 has also carried out standardization to signal intensity, method is the energy maximal value for a sampled signal, it is made as 1, all the other have a fews are that standard is amplified pro rata or dwindled with this point, guarantee that any once energy maximal value of sampling all equates.In addition, sampled signal is handled by low-pass filter, can be suppressed high frequency noise, improve signal to noise ratio (S/N ratio).
Step 2), signal that step 1) is handled gets frame, has certain overlappingly between the consecutive frame, in voice signal was handled, common every frame signal length was regarded stationary signal as so that each frame signal can be similar in 200 milliseconds.Every frame data are carried out the windowing Filtering Processing.Hanning window filtering formula is as follows:
Step 3), Fourier Tranform (Fourier Transform) are a kind of time-domain signal to be transformed to the method for frequency-region signal, and in frequency domain, the energy distribution of signal on the different frequency component can be reproduced clear and intuitively.Adopting fast fourier transform (FFT) algorithm in this step with step 2) every frame signal after handling transforms to complex frequency domain, obtains the complex vector of each frequency component.Each complex vector comprises real axis and two components of the imaginary axis, gets its square sum, obtains energy value, has promptly represented the power of this frame signal on each frequency component.It is 2 that fast fourier transform requires the sampling number of input
N, if step 2) in every frame sampling less than 2 of counting out
N, then the point of deficiency is mended 0.
During step 4), the every frame frequency territory after step 3) is handled distribute, if can find the peak value of energy at people's audio frequency range, and significantly surpassed the energy of ground unrest, the frequency of first peak value correspondence that then satisfies condition is the fundamental frequency value of voice.The fundamental frequency value of consecutive frame is compared,, then think same note,, then think the conversion of note if variation is bigger if change not quite.In addition, quiet frame also can be used as the boundary of note.
Step 5), between adjacent two notes, ask the logarithmic difference of its frequency, obtain the difference characteristic sequence of melody note.Frequency values is taken the logarithm, be exactly will make the scale difference be directly proportional with the frequency values linearization of scale exponential increase with the logarithmic difference of its frequency.With the logarithm difference on the frequency of note feature as melody, in the time of can eliminating the different user humming, the difference that different keynotes brings.
Through above 5 steps, the audio frequency of voice humming has changed into melody characteristics information, can be used as key feature and sends to server end search.In above fundamental frequency extraction step, can adopt the method for time domain equally, for example correlation method etc.
Because common telephone plant does not have data-handling capacity, therefore in telephone terminal, audio signal processing unit 112 runs on server 2 ends, and client 1 telephone plant only is responsible for collecting user's input.In the humming input mode, the user uses phone to be subjected to microphone as audio collection unit 111, and sound signal is to be sent to server end by Public Switched Telephony Network (PSTN); In the fingerboard input mode, the user makes telephonic digit dialling keyboard, mode with music numerical notation is imported melody, after server 2 ends are received the telephone key-press signal, server 2 carries out information interaction with client 1 by PSTN (PSTN), be translated into corresponding musical tones, feed back to the user so that the user revises.
The display module 12 of Search Results, client 1 obtains the required music rhythm information result of search by network or other transmission modes from server 2 ends.Search Results presents with the form of tabulation, and each in the tabulation is a piece of music name (title), and the author, information such as singer.Music in the tabulation is pressed the similarity degree sort descending.
In the structural drawing of Fig. 1, in the empty frame in the right is server 2, comprise: music data source interface unit 21, data are obtained and analytic unit 22, authorized index unit 23, search unit 24, and they are finished on the backstage and collect data, analyze data, produce index, and the online search arithmetic of carrying out.
Data are obtained and analytic unit 22, and it is responsible for collecting original music data file, and music data file is analyzed, and therefrom extracts music rhythm information; The music file formats that the present invention directly supports is a midi format, and therefore, data are obtained with analytic unit 22 and mainly the MIDI music file analyzed.The MIDI file layout is the key element with the form storage music of digital command, as pitch, and duration, tone color, rhythm etc.By parsing to music digital command sequence in the MIDI file, can be easily and accurately extract the parameter of music.The MIDI music file can be regarded the structure of a layering as.Common MIDI file has two kinds of forms: single track form (Type0) and multiple form (Type1).In the single track form, each file comprises a track (track), and 16 passages (channel) are arranged in each track, and each passage can be deposited a kind of musical instrument.When playing, 16 passages are play simultaneously.The single track form has 16 kinds of musical instruments to play simultaneously at most, can satisfy the needs of general digital music.In the multiple form music file, each file comprises a plurality of tracks (track), and each track also comprises 16 passages, is movable but each track has only a passage, and other passages all are empty.A plurality of tracks also are to play simultaneously.The multiple form can be play simultaneously more than 16 kinds of musical instruments, so the abundant digital music of some expressive forces often adopts this form.Data are obtained with analytic unit 22 two kinds of file layouts are unified, and set up hierarchy: MIDI-track-four levels of passage-note, the upper strata element is made up of the set of lower floor's element.The passage of each non-NULL all comprises one section sequence of notes.Data are obtained with analytic unit 22 each music file are converted into an object with hierarchy, and have preserved the finger print information of this music, title, relevant informations such as author.
Any one search engine, its work are exactly to return an information list that mates with this user inquiring in the time at an acceptable.Have three notions should be noted that here:
1) the acceptable time.This refers to the response time.For the software that service is provided to users on Internet, this time can not be oversize, usually just in " second " this magnitude.This is a basic index weighing the search engine availability, also is a difference with conventional ir system.Further, such response time requirement not only wants to satisfy the inquiry of unique user, and wants to satisfy all users under the situation of system design load.That is to say that system should guarantee a second level response time under the situation of specified throughput.
2) coupling.With the webpage is example, refers to the content that includes the key word of the inquiry of user's input in the webpage with certain form, perhaps occurs and the very close content of key word of the inquiry.In the music searching automotive engine system based on melody, what coupling referred to is exactly the melody key word that comprises user's input in the theme of music.The input of user's melody and target melody be deviation to some extent, and therefore, coupling is not only wanted and can accurately be mated, but also needs certain fault-tolerant ability.
3) tabulation.Return to user's Search Results at search engine, normally tabulation that comprises multinomial result, each element in this tabulation all has to a certain degree similar or relevant with the key word of user's input.Yet most users only are concerned about the element that comes in the results list first page, therefore, are essential to the similar relevance ranking of element in the search result list.This ordering is called Rank.At present different search engines has been taked the Ranking algorithm that is not quite similar.What adopt as Google is the PageRank algorithm, and it sorts to the importance of the page among the result, and Baidu has adopted the method for bid ranking etc.
In search engine system, the quality of Index Algorithm has fundamental influence to above three performance index.In the present music searching engine based on melody, what majority adopted is the algorithm of linear matched.This algorithm is exactly that the melody in user's input melody and the music file is regarded as two strings respectively, the similarity of going here and there contrast.In content-based music searching field, relatively commonly used have methods such as Suffix Tree, Suffix Array, Linear Alignment.Yet linear search has a common defective, in search procedure, need each element in the database be scanned, to determine whether coupling.When this data volume at raw data base is little is acceptable, but increase along with the data of database amount, under optimal situation, the time of search also can be growth linearly, promptly Sou Suo time complexity is at least O (n), for example, in Suffix Array algorithm, its time complexity is O (nlogn).The data volume of present large-scale search engine is usually 10
8To 10
9The order of magnitude, if huge database like this is carried out linear sweep, be that the user is unacceptable operation time.Therefore, large-scale search engine, the general algorithm that all adopts inverted index.
In numerous searching algorithms, inverted index (Inverted Index) is with flexibly, and is efficient, has characteristics such as versatility, obtains widespread use rapidly.It is a kind of Index Algorithm based on word, can be according to the key word of user's input, direct filtration is fallen incoherent content in the database, and can the correlativity of related content be sorted, and good fault freedom is arranged, and content that can pairing approximation is discerned.
In the text of most language, natural separator is all arranged between speech and the speech, as the space, punctuation mark etc.Do not have in the language of natural participle at Chinese etc., the participle technique of comparative maturity is also arranged.Inverted index appears at the frequency difference according to each word exactly in article, the same speech that occurs in the different articles is classified as a class, and with the major key of word as index, the article that contains this word is as element list.Like this, several particular words occurred in an inquiry, system will directly remove to search the article element under these several certain words, and will be fallen by automatic fitration with the irrelevant article of inquiry.This automatic fitration does not need to take cpu resource, so efficient is very high.The mechanism of this efficient automatic fitration irrelevant information is exactly the advantage place of the data structure of this uniqueness of inverted index.
In the music searching automotive engine system, search to as if music rhythm, rather than text.Therefore need make some to text based inverted index model and revise, make it to adapt to the authorized index of music rhythm.
Music rhythm is to be made of continuous sequence of notes.In music,, in the MIDI music format, do not have the sign that tangible trifle is separated though there is trifle melody can be divided into segment yet.In addition, rest is very similar to the space in the text, just in the music of different-style, the appearance of rest very at random, neither one has the rule of obvious characteristic.Therefore, the natural separator of this class music of trifle and rest itself all is not suitable for dividing melody.
Because music rhythm itself does not find a kind of good participle mechanism at present, so the present invention adopts melody fragment cutting method.With one section continuous melody cutting is segment, and every segment comprises 3～4 notes, section with section between have certain overlapping.The present invention is with the participle of melody fragment as music rhythm, and utilization is arranged algorithm and carried out authorized index.When new music track need add index, only need be to the division of this Qu Jinhang melody fragment, and should add respectively in the element set of each melody fragment by song.
Authorized index unit 23 is used for according to above method the music rhythm information segment being carried out authorized index as the participle of music rhythm, the music data that provides with analytic unit 22 is provided data sets up index.
Search unit 24, be used to receive the query requests of client load module 11, and the online search arithmetic of carrying out of music rhythm information music identical or close melody that search and client 1 sound intermediate frequency collecting unit 111 or note collecting unit 113 are inquired about in the index that authorized index unit 23 generates, be used for search result list is sorted by the similarity degree inverted order, and feed back to the search result display module 12 of client.
Above mentioning, by similarity degree Search Results is sorted, is important function of search engine.Search unit 24 calculates similarity according to the number of identical note in the melody string in client query string and the music libraries, and identical note is many more, and it is similar more that both are described.
Search unit 24 adopts different interactive modes according to different clients 1 equipment.
When being personal computer equipment for client 1, the PC client is installed specific Web browser plug-in software, the recorded program in the audio collection module 111 that this plug-in software is integrated and the virtual piano keyboard program of note acquisition module 113 from downloaded.During music retrieval Web website that user access server provides, the user interface that is used to the user to provide audio collection input and note to gather melody, and the inquiry of gathering the user imports, and is sent to server by the internet.
When being intelligent mobile device for client 1, client 1 is installed specific software, the mobile device operation system platform that this software uses based on the user is developed (as Windows Mobile platform, the Linux platform, Nokia S60 platform, Java platform etc.), for the user provides the user interface that audio collection is imported and note is gathered melody, and gather user's inquiry input, be sent to server by wireless network.
When selecting telephone plant for client 1, server 2 provides specific phone information service center, client 1 is dialed this information service center number, utilize the phone numbers keyboard, or using telephone receiver as the audio collection input equipment, server 2 carries out information interaction with client 1 by PSTN (PSTN).
The audio frequency and video amusement equipment that 1 selection has the media-on-demand function for client, client 1 is equipped with hardware digital piano keyboard equipment, or the fingerboard note that virtual piano keyboard software collection user is installed is imported, utilize microphone of carok collection user's humming input, server 2 is a dedicated local server, and the scope of search is the local music libraries of Karaoke.
For computer and intelligent movable equipment, Search Results is presented to the user with tabular form, and the user can download under the situation of not invading the musical works intellecture property, operations such as broadcast.For the client 1 of phone, server 2 ends will be read aloud search result list with voice prompting mode, and user's available phone button is chosen.For program request device clients 1, after the user chooses, can preengage operations such as program request.
Music data source interface unit 21 is used to provide multiple different data source access interface, and make server obtain the original music data, and musical database is expanded according to concrete purposes and demand from different data sources, for example:
1. the mode of taking the Web network to grasp, music file and the information relevant with this music file are grasped in roaming on the internet automatically; Or
2. take the file of storing in this locality or the network file system(NFS) is grasped and analyzes; Or
3. take the musical recording in the database is extracted and analyzes.
The present invention is not limited to above three kinds of data sources, but and provided the application programming interfaces (API) of secondary development, can further expand data source.
Describing above is to be used to realize embodiments of the invention, it should be appreciated by those skilled in the art, in any modification or partial replacement that does not depart from the scope of the present invention, all belongs to claim of the present invention and comes restricted portion.
Claims (13)
1. the music retrieval method based on melody is characterized in that,
Step S1: specify one section melody waiting to look in the music as the melody key word of searching for;
Step S2:, obtain the digitizing melody signal that is used to inquire about through handling with specified melody key word input inquiry client device;
Step S3: adopting melody fragment cutting method, is segment with one section continuous melody cutting, and every segment comprises 3～4 notes, section with section between have certain overlapping; With the participle of music rhythm fragment as music rhythm, use Inversed File Retrieval Algorithm to carry out authorized index again, the music data in the musical database is set up index, this index embodies the melody characteristics of music, forms the musical database of indexation;
Step S4: by search engine the melody in the musical database of digitizing melody signal and generation is compared, and filter out and the incoherent music of melody key word, several remaining candidate's music are sorted according to the similarity degree with the melody key word; Number according to identical note in the melody in digitizing melody signal and the musical database is calculated similarity, and selects one group of music according to similarity from musical database;
Step S5: with the music selected according to the similarity degree sort descending of melody key word.
2. music retrieval method according to claim 1 is characterized in that, the described input mode of looking into music of waiting comprises: play input and humming input.
3. music retrieval method according to claim 1 is characterized in that, described index is the index of working out at the melody characteristics of melody fragment.
4. music retrieval method according to claim 2 is characterized in that, for the humming input mode, takes following steps to obtain digitized melody signal:
Step S21: use audio collecting device to gather user's humming input;
Step S22: the sound signal of user's input is carried out pre-filtering handle, comprise direct current elimination, gain normalization, low-pass filtering treatment, obtain the audio frame sequence signal;
Step S23: the audio frame sequence signal is carried out time domain or frequency-domain analysis, extract the fundamental frequency sequence;
Step S24: the fundamental frequency sequence is further handled, comprised linearization, ask poor, obtain digitized melody signal.
5. the music retrieval device based on melody is characterized in that, comprising:
At least one station server (2) provides online music rhythm retrieval service, the melody key word of user's input and the melody in the musical database are compared, and filter out and the incoherent music of melody key word, several remaining candidate's music are sorted according to the similarity degree with the melody key word, the music list after the ordering is returned client (1);
Described server (2) comprising:
Music data source interface unit (21) is used to provide the various data sources of visit to obtain the interface of original music data;
Data are obtained and analytic unit (22), be used to collect original music data, and music data is analyzed, and therefrom extract music rhythm information;
Authorized index unit (23) adopts melody fragment cutting method, is segment with one section continuous melody cutting, and every segment comprises 3～4 notes, section with section between have certain overlapping; With the participle of music rhythm fragment as music rhythm, use Inversed File Retrieval Algorithm to carry out authorized index again, be used for that data are obtained the original music data of obtaining with analytic unit 22 and set up index according to its melody characteristics;
Search unit (24), be used to receive the query requests of client load module (11), and the music of the identical or close melody of the melody key word that provides with client load module (11) is provided in search in the index that authorized index unit (23) generate, search result list is pressed the ordering of similarity degree inverted order, and feed back to the search result display module (12) of client (1);
Send online music rhythm retrieval request with at least one client (1) terminal device, specify one section melody waiting to look in the music as the melody key word of searching for; With specified melody key word input inquiry client device, obtain the digitizing melody signal through handling, send to server end, and the result of reception server query music melody.
6. music retrieval device according to claim 5 is characterized in that, described client (1) comprising:
Load module (11) is used to import the music rhythm information that need search, and sends it to server (2);
The display module of Search Results (12), client (1) obtains Search Results by network or other transmission modes from server (2), and Search Results presents with the form of tabulation, and the music in the tabulation is pressed the similarity degree sort descending, and presents to the user.
7. music retrieval device according to claim 6 is characterized in that, described load module 11 comprises:
Audio collection unit (111) is used to gather user's humming sound signal, drive by recording software, the simulating signal of audio volume control is carried out digital collection by the sample frequency of recording software appointment, the digital pulse sequence of gathering is stored in the storer of client (1); Sample frequency is that 8000Hz or 11025Hz analyze the harmonic wave of voice;
Audio signal processing unit (112), the sound signal that audio collection unit (111) are gathered is converted into the music rhythm signal;
Note collecting unit (113) is used to gather the note melody signal that the user plays, adopt the mode of fingerboard input that the interface of playing the input melody is provided, go up the demonstration fingerboard in client (1), the user can click corresponding key input melody with mouse or other contact equipment, note collecting unit (113) is pressed the pitch serial number with each key of piano, ID as each key, it is poor that the difference of the ID of adjacent two keys that the user clicked is the note identical with the output implication of audio signal processing unit (112), is sent to server (2) as melody characteristics.
8. music retrieval device according to claim 5 is characterized in that, described music data source interface unit (21) provides the interface of following one or more data obtain manners:
Web: the mode of taking the Web network to grasp, music file and the information relevant with this music file are grasped in roaming on the internet automatically;
File: the music file of storing in this locality or the network file system(NFS) is grasped and analyzes;
Database: the music file that writes down in the database is extracted and analyzes.
9. music retrieval device according to claim 7 is characterized in that, described client (1) is one or more in the following equipment:
PC, intelligent mobile device, phone, audio frequency and video amusement equipment with media-on-demand function.
10. music retrieval device according to claim 9, it is characterized in that, when described client (1) is selected personal computer equipment, PC client (1) downloads and installs specific Web browser plug-in software from server (2), during music retrieval Web website that user access server (2) provides, the user interface that is used to the user to provide audio collection input and note to gather melody, and collection user's inquiry input are sent to server (2) by the internet.
11. music retrieval device according to claim 9, it is characterized in that, when described client (1) is selected intelligent mobile device, client (1) is installed specific software, the user interface that this software provides audio collection and note to gather for the user, and gather user's inquiry input, be sent to server by wireless network.
12. music retrieval device according to claim 9, it is characterized in that, when described client (1) is selected telephone plant, server (2) provides specific phone information service center, client (1) is dialed this information service center number, utilize the phone numbers keyboard, or use telephone receiver respectively as note collection and audio collection input equipment, server (2) carries out information interaction with client (1) by PSTN.
13. music retrieval device according to claim 9, it is characterized in that, when described client (1) is selected to have the audio frequency and video amusement equipment of media-on-demand function, client (1) is equipped with digital piano keyboard equipment, or the fingerboard note that virtual piano keyboard software collection user is installed is imported, utilize microphone of carok collection user's humming input, server (2) is a dedicated local server, and the scope of search is the local musical database of Karaoke.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2007100646076A CN101271457B (en) | 2007-03-21 | 2007-03-21 | Music retrieval method and device based on rhythm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2007100646076A CN101271457B (en) | 2007-03-21 | 2007-03-21 | Music retrieval method and device based on rhythm |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101271457A CN101271457A (en) | 2008-09-24 |
| CN101271457B true CN101271457B (en) | 2010-09-29 |
Family
ID=40005434
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2007100646076A Expired - Fee Related CN101271457B (en) | 2007-03-21 | 2007-03-21 | Music retrieval method and device based on rhythm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101271457B (en) |
Families Citing this family (38)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101552001B (en) * | 2009-02-25 | 2012-07-04 | 北京派瑞根科技开发有限公司 | Network searching system and information searching method |
| CN101552000B (en) * | 2009-02-25 | 2012-07-04 | 北京派瑞根科技开发有限公司 | Music similarity processing method |
| CN101552003B (en) * | 2009-02-25 | 2012-07-04 | 北京派瑞根科技开发有限公司 | Media information processing method |
| US20110077756A1 (en) * | 2009-09-30 | 2011-03-31 | Sony Ericsson Mobile Communications Ab | Method for identifying and playing back an audio recording |
| CN101916250B (en) * | 2010-04-12 | 2011-10-19 | 电子科技大学 | Humming-based music retrieving method |
| CN102375834B (en) * | 2010-08-17 | 2016-01-20 | 腾讯科技（深圳）有限公司 | Audio file search method, system and audio file type recognition methods, system |
| CN102411578A (en) * | 2010-09-25 | 2012-04-11 | 盛乐信息技术（上海）有限公司 | Multimedia playing system and method |
| CN101980197B (en) * | 2010-10-29 | 2012-10-31 | 北京邮电大学 | Long time structure vocal print-based multi-layer filtering audio frequency search method and device |
| CN102332262B (en) * | 2011-09-23 | 2012-12-19 | 哈尔滨工业大学深圳研究生院 | Method for intelligently identifying songs based on audio features |
| CN102522083B (en) * | 2011-11-29 | 2014-03-05 | 北京百纳威尔科技有限公司 | Method for searching hummed song by using mobile terminal and mobile terminal thereof |
| CN102497400A (en) * | 2011-11-30 | 2012-06-13 | 上海博泰悦臻电子设备制造有限公司 | Music media information obtaining method of vehicle-mounted radio equipment and obtaining system thereof |
| CN102420910A (en) * | 2011-12-16 | 2012-04-18 | 广东步步高电子工业有限公司 | Handheld mobile terminal capable of playing music and synchronously displaying music score and realization method thereof |
| CN103812917A (en) * | 2012-11-15 | 2014-05-21 | 佛山市顺德区顺达电脑厂有限公司 | Information collecting system and method thereof |
| CN103970793B (en) | 2013-02-04 | 2020-03-03 | 腾讯科技（深圳）有限公司 | Information query method, client and server |
| CN103108229A (en) * | 2013-02-06 | 2013-05-15 | 上海云联广告有限公司 | Method for identifying video contents in cross-screen mode through audio frequency |
| CN103218454A (en) * | 2013-05-06 | 2013-07-24 | 百度在线网络技术（北京）有限公司 | Voice-data-based file searching method, voice-data-based file device and voice-data-based file system |
| JP2014219607A (en) * | 2013-05-09 | 2014-11-20 | ソニー株式会社 | Music signal processing apparatus and method, and program |
| CN103258033A (en) * | 2013-05-15 | 2013-08-21 | 江苏奇异点网络有限公司 | Song automatic searching system |
| CN103559312B (en) * | 2013-11-19 | 2017-01-18 | 北京航空航天大学 | GPU (graphics processing unit) based melody matching parallelization method |
| CN104679778B (en) | 2013-11-29 | 2019-03-26 | 腾讯科技（深圳）有限公司 | A kind of generation method and device of search result |
| WO2017028115A1 (en) * | 2015-08-16 | 2017-02-23 | 胡丹丽 | Intelligent desktop speaker and method for controlling intelligent desktop speaker |
| CN105069146B (en) * | 2015-08-20 | 2019-04-02 | 百度在线网络技术（北京）有限公司 | Sound searching method and device |
| CN105244021B (en) * | 2015-11-04 | 2019-02-12 | 厦门大学 | Conversion method of the humming melody to MIDI melody |
| CN105895079B (en) * | 2015-12-14 | 2022-07-29 | 天津智融创新科技发展有限公司 | Voice data processing method and device |
| CN107146631B (en) * | 2016-02-29 | 2020-11-10 | 北京搜狗科技发展有限公司 | Music identification method, note identification model establishment method, device and electronic equipment |
| WO2018018283A1 (en) * | 2016-07-24 | 2018-02-01 | 张鹏华 | Counting method for usage condition of song information recognition technique and recognition system |
| CN106776977A (en) * | 2016-12-06 | 2017-05-31 | 深圳前海勇艺达机器人有限公司 | Search for the method and device of music |
| CN108268530B (en) * | 2016-12-30 | 2022-04-29 | 阿里巴巴集团控股有限公司 | Lyric score generation method and related device |
| CN108574771A (en) * | 2017-03-10 | 2018-09-25 | 峰范（北京）科技有限公司 | Collecting and processing of information system and its voice playing device, processing method |
| CN107205043A (en) * | 2017-07-03 | 2017-09-26 | 武汉理工大学 | A kind of violin class network virtual musical instrument |
| CN107436953B (en) * | 2017-08-15 | 2020-07-10 | 中国联合网络通信集团有限公司 | Music searching method and system |
| CN108665903B (en) * | 2018-05-11 | 2021-04-30 | 复旦大学 | Automatic detection method and system for audio signal similarity |
| CN108806392A (en) * | 2018-07-03 | 2018-11-13 | 东北石油大学 | A kind of vocal music pronunciation training apparatus and system |
| CN109346043B (en) * | 2018-10-26 | 2023-09-19 | 平安科技（深圳）有限公司 | Music generation method and device based on generation countermeasure network |
| CN110472094B (en) * | 2019-08-06 | 2023-03-31 | 沈阳大学 | Traditional music recording method |
| CN110853457B (en) * | 2019-10-31 | 2021-09-21 | 中科南京人工智能创新研究院 | Interactive music teaching guidance method |
| CN111627410B (en) * | 2020-05-12 | 2022-08-09 | 浙江大学 | MIDI multi-track sequence representation method and application |
| CN112015942A (en) * | 2020-08-28 | 2020-12-01 | 上海掌门科技有限公司 | Audio processing method and device |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1703734A (en) * | 2002-10-11 | 2005-11-30 | 松下电器产业株式会社 | Method and apparatus for determining musical notes from sounds |
-
2007
- 2007-03-21 CN CN2007100646076A patent/CN101271457B/en not_active Expired - Fee Related
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1703734A (en) * | 2002-10-11 | 2005-11-30 | 松下电器产业株式会社 | Method and apparatus for determining musical notes from sounds |
Non-Patent Citations (4)
| Title |
|---|
| 张静,朱悦心.采用人声输入的网络音乐检索系统.微电子学与计算机23 5.2006,23(5),173-178. |
| 张静,朱悦心.采用人声输入的网络音乐检索系统.微电子学与计算机23 5.2006,23(5),173-178. * |
| 金毅,黄敏.基于旋律的音乐检索研究--旋律特征的输入识别*.现代图书情报技术 106.2004,(106),41-45. |
| 金毅,黄敏.基于旋律的音乐检索研究——旋律特征的输入识别*.现代图书情报技术 106.2004,(106),41-45. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101271457A (en) | 2008-09-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101271457B (en) | Music retrieval method and device based on rhythm | |
| CN100419742C (en) | Internet browser | |
| CN102053998A (en) | Method and system device for retrieving songs based on voice modes | |
| Typke | Music retrieval based on melodic similarity | |
| Lidy et al. | On the suitability of state-of-the-art music information retrieval methods for analyzing, categorizing and accessing non-western and ethnic music collections | |
| Cornelis et al. | Access to ethnic music: Advances and perspectives in content-based music information retrieval | |
| US20100223223A1 (en) | Method of analyzing audio, music or video data | |
| CN101657817A (en) | Search engine based on music | |
| CN101014953A (en) | Audio fingerprinting system and method | |
| JP5066963B2 (en) | Database construction device | |
| KR20080054393A (en) | Music analysis | |
| WO2008101130A2 (en) | Music-based search engine | |
| Futrelle et al. | Interdisciplinary research issues in music information retrieval: ISMIR 2000–2002 | |
| Baggi et al. | Music navigation with symbols and layers: Toward content browsing with IEEE 1599 XML encoding | |
| CN100501738C (en) | Searching method, system and apparatus for playing media file | |
| CN111192601A (en) | Music labeling method and device, electronic equipment and medium | |
| Gurjar et al. | Comparative Analysis of Music Similarity Measures in Music Information Retrieval Systems. | |
| Pachet et al. | The cuidado music browser: an end-to-end electronic music distribution system | |
| Kurth et al. | Syncplayer-An Advanced System for Multimodal Music Access. | |
| KR100702059B1 (en) | Ubiquitous music information retrieval system and method based on query pool with feedback of customer characteristics | |
| KR102165940B1 (en) | System and method for providing cbmr based music identifying serivce using note | |
| KR20020053979A (en) | Apparatus and method for contents-based musical data searching | |
| JP2009151541A (en) | Optimum information presentation method in retrieval system | |
| Lin | Design of the violin performance evaluation system based on mobile terminal technology | |
| Arentz et al. | Retrieving musical information based on rhythm and pitch correlations |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20100929 Termination date: 20180321 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |