CN101663708B - System and method for searching audio fingerprint by index information - Google Patents

System and method for searching audio fingerprint by index information Download PDF

Info

Publication number
CN101663708B
CN101663708B CN2008800126394A CN200880012639A CN101663708B CN 101663708 B CN101663708 B CN 101663708B CN 2008800126394 A CN2008800126394 A CN 2008800126394A CN 200880012639 A CN200880012639 A CN 200880012639A CN 101663708 B CN101663708 B CN 101663708B
Authority
CN
China
Prior art keywords
fingerprint
audio
index
frequency
search
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN2008800126394A
Other languages
Chinese (zh)
Other versions
CN101663708A (en
Inventor
李承宰
徐镇洙
李相光
俞元英
尹英锡
徐庸硕
吴元根
徐泳浩
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Electronics and Telecommunications Research Institute ETRI
Original Assignee
Electronics and Telecommunications Research Institute ETRI
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Electronics and Telecommunications Research Institute ETRI filed Critical Electronics and Telecommunications Research Institute ETRI
Publication of CN101663708A publication Critical patent/CN101663708A/en
Application granted granted Critical
Publication of CN101663708B publication Critical patent/CN101663708B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G11B27/11—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information not detectable on the record carrier
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/60—Information retrieval; Database structures therefor; File system structures therefor of audio data
    • G06F16/61—Indexing; Data structures therefor; Storage structures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/60—Information retrieval; Database structures therefor; File system structures therefor of audio data
    • G06F16/68—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/683—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B20/00—Signal processing not specific to the method of recording or reproducing; Circuits therefor
    • G11B20/10—Digital recording or reproducing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • General Physics & Mathematics (AREA)
  • Library & Information Science (AREA)
  • Software Systems (AREA)
  • Signal Processing (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Collating Specific Patterns (AREA)

Abstract

Provided are a system and method for searching audio fingerprint by index information. The system includes a DB group for generating an index based on statistical characteristics of an audio fingerprint for an audio file and consecutively matching the index, the audio fingerprint, and music information, and an audio fingerprint searching apparatus for generating a new index based on statistical characteristic of an audio fingerprint for a new input audio file and searching corresponding music information for the new input audio file by searching the new index from the DB group.

Description

The system and method that is used for searching audio fingerprint by index information
Technical field
The present invention relates to a kind of audio-frequency fingerprint (fingerprint) search technique; And more specifically; Relate to following system and method; This system and method is used for coming searching audio fingerprint by index information, to improve recognition performance and to increase search speed through audio-frequency fingerprint is indexed (indexing), based on the predetermined audio-frequency fingerprint of search of indexing, and the audio-frequency fingerprint searched for of checking.
This work is supported by infotech (IT) the research and development plans [2007-S-017-01, " Development of user-centriccontents protection and distribution technology (exploitation of customer center content protecting and distribution technique) "] of MIC (Korea S information communication portion)/IITA (Korea S's infotech promotes association).
Background technology
The target of audio fingerprint system is through received audio signal and utilizes the audio-frequency fingerprint database search corresponding audio that makes up in advance to discern predetermined audio frequency.According to application, audio fingerprint system has been used for commentator's monitor, CF identification and file filter.In order in said application, to use audio fingerprint system effectively, even under various distortion situation, also need high discrimination and fast search speed.Particularly, for kill file in P2P or UCC field, need to search for rapidly and exactly by it audio-frequency fingerprint data that each hundreds of thousands audio file with the copyright of oneself forms.For in broadcasting supervision of operating based on high capacity audio frequency fingerprint database and the real-time processing in the file filter field, recognition speed is one of most important factor.
In addition, according to the application of audio fingerprint system, also require audio fingerprint system to have high recognition performance (although voice data is out of shape through resampling, filtration, equilibrium and compression) and fast recognition speed.
The Korean Patent that is called " Method for search in audio database (method that is used for the search of audio database) " in name discloses the searching method of having introduced in 2003-7001489 number according to prior art.In the method for the search that is used for audio database, extract boundary mark (landmark) and fingerprint, and use the corresponding relation of boundary mark and fingerprint to search for predetermined voice data.In said method, except fingerprint, also calculate boundary mark, the boundary mark of being calculated is stored as index and in boundary mark and use the candidate list between the music ID of the fingerprint in the boundary position.Then, discern audio frequency based on its linear relationship.Yet,, do not consider the characteristic of fingerprint although in said method, search for sound signal based on fingerprint.In addition, as complementary features, said method needs boundary mark to discern predetermined audio.
The Korean Patent that is called " Fingerprint producing method and audio fingerprinting systembased on normalized spectral subband centroids (based on the fingerprint production method and the audio fingerprinting system of normalized music score band centre of moment) " in name discloses the audio search system of having introduced in 2007-0031765 number according to prior art.Said fingerprint production method and audio fingerprinting system generate fingerprint based on normalized music score band centre of moment, and search for predetermined audio frequency through the distance that compares fingerprint.Said fingerprint production method and audio fingerprinting system do not have to consider to be used for the characteristic of the fingerprint of audio search, although said fingerprint production method and audio fingerprinting system have MFCC and tone, equilibrium and random start better recognition performance than the typical fingerprint of MP3.
Summary of the invention
Technical matters
Correspondingly, the present invention is intended to a kind of system and method that is used for searching for according to index information audio-frequency fingerprint, and it has been got rid of in fact because the restriction of prior art and one or more problems that shortcoming is caused.
Target of the present invention provides a kind of system and method that uses index information search audio-frequency fingerprint; To generate index through the statistical property of using the audio-frequency fingerprint characteristic information and to use the index that generates to search for predetermined audio, improve the audio identification performance and increase search speed.
Technical solution
In order to realize that these targets are with other advantage and according to the object of the invention; Such as here enforcement and broadly described; A kind of system that is used to search for audio-frequency fingerprint is provided; This system comprises: DB group, be used for generating index based on the statistical property of the audio-frequency fingerprint of audio file, and match index, audio-frequency fingerprint and music information continuously; And audio fingerprint search apparatus, be used for generating new index, and be used for newly importing the corresponding music information of audio file through the new index search of search from the DB group based on the statistical property of the audio-frequency fingerprint of new input audio file.
According to another object of the present invention; Providing a kind of is used to use index information to search for the method for audio-frequency fingerprint; Comprise the steps: a) to generate index based on the statistical property of the audio-frequency fingerprint of audio file; And prepare to be used for the DB group of stored position information, this DB group is the index, audio-frequency fingerprint and the music information that are generated of coupling continuously; B) statistical property based on the audio-frequency fingerprint of new input audio file generates index; And c) generates the corresponding music information that index search is used for newly importing audio file through what search from DB group generated in step b).
Beneficial effect
The system and method that is used for searching for according to index information audio-frequency fingerprint according to the present invention uses the statistical property of audio-frequency fingerprint to generate index, and searches for audio-frequency fingerprint based on the index that is generated.Therefore, the system and method that is used for searching for according to index information audio-frequency fingerprint according to the present invention can be supported fast search time, and can be applied to filter and keep watch on the file in the Large Volume Data storehouse.In addition; Be used for searching for candidate index that the system and method for audio-frequency fingerprint created the index bit that comprises variable position so that compensating distortion according to index information; If this is index directly is used for search and does not have compensation, then reduce discrimination sharp owing to distortion.Therefore, be used for according to index information search for the system and method for audio-frequency fingerprint can be through proofreading and correct because bit index and the error that can generate improved discrimination.
Description of drawings
In the part of this application, merged accompanying drawing; And said accompanying drawing constitutes the part of this application; Comprise said accompanying drawing so that further understanding of the present invention to be provided, said accompanying drawing illustrates embodiments of the invention, and is used to explain principle of the present invention with describing.In the accompanying drawings:
Fig. 1 is the block diagram that illustrates according to an embodiment of the invention, is used to search for the system of audio-frequency fingerprint;
Fig. 2 illustrates the block diagram of index processor according to an embodiment of the invention;
Fig. 3 is the figure that is used for searching for the relation in the DB file that the system of audio-frequency fingerprint uses that illustrates according to the embodiment of the invention;
Fig. 4 illustrates the figure that is used to generate the employed probability distribution of fingerprint index;
Fig. 5 is the figure that illustrates that fingerprint that use handle to extract from audio search generates fingerprint index and search for the process of predetermined audio based on the fingerprint index that is generated;
Fig. 6 illustrates the figure that in audio search is handled, generates the process of candidate index; And
Fig. 7 is that the candidate index that illustrates use generation in audio search is handled is searched for the figure of the process of net result.
Embodiment
Now, detailed reference is illustrated in the accompanying drawings the preferred embodiments of the present invention of its example.
Fig. 1 is the block diagram that illustrates according to an embodiment of the invention, is used to search for the system of audio-frequency fingerprint.
With reference to figure 1, comprise audio fingerprint search apparatus 1 according to the system that is used to search for audio-frequency fingerprint of present embodiment.Said audio fingerprint search apparatus 1 comprises: fingerprint extraction unit 11 is used to extract the audio-frequency fingerprint that is used for audio file; Candidate index search unit 12; Be used for through the value of the fingerprint that extracted that sorts of the ascending order according to the absolute value of the difference between audio finger value of being extracted and the mean value (mean value) that when generating index, uses, and consider variable position ground generation candidate index; Fingerprint matching unit 13 is used for mating audio-frequency fingerprint accordingly and having extracted audio-frequency fingerprint with candidate index; And result verification unit 14, if the distance that is used between the audio-frequency fingerprint is in predetermined range, then the verification search result is corresponding music information.
Because index is used to search, also comprises so be used for the system of search fingerprint: DB group 2, be used for store audio fingerprints, said audio-frequency fingerprint has the manipulative indexing with said audio-frequency fingerprint coupling.In order to operate, at first need set up relevant database according to audio fingerprint search apparatus 1 of the present invention.Therefore, preferably, DB group 2 formed have fingerprint DB 21, music information DB 22 and fingerprint index DB 23.In addition, in order to mate and store audio fingerprints and corresponding index, need carry out pre-service to audio file.Preferably, also comprise: index processor 3, be used for receiving audio file and music id information in pre-service, and with audio-frequency fingerprint and corresponding index coupling.
As stated, be divided into according to the system that is used to search for audio-frequency fingerprint of present embodiment: DB generates the zone, and be used to generate fingerprint index and set up its database, and the DB region of search, be used for searching for through indexing.That is to say that generate in the step at DB, relevant information is stored among fingerprint DB 21, music information DB 22 and the fingerprint index DB 23.In the step of searching for, carry out fingerprint extraction, the candidate search of carrying out and coupling and result verification through indexing based on index.Although the system that is used to search for audio-frequency fingerprint according to present embodiment is divided into two zones, be apparent that to those skilled in the art said two zones can be carried out in a zone.
Fig. 2 illustrates the block diagram of index processor according to an embodiment of the invention.
With reference to figure 2, index processor 3 comprises: fingerprint extractor 31, fingerprint statistical analyzer 32, fingerprint binarizer 33 and the fingerprint device (indexer) 34 of indexing.
Fingerprint extractor 31 uses the fingerprint extraction algorithm from the audio file of input, to extract audio-frequency fingerprint, and the statistical property of fingerprint statistical analyzer 32 analyzing audio fingerprints and calculating probability distribution.That is to say that fingerprint statistical analyzer 32 is approximately probability model through the average (mean) and the fingerprint distribution of calculating audio-frequency fingerprint with the audio-frequency fingerprint that is extracted.Then, the statistical property of fingerprint binarizer 33 through the analyzing audio fingerprint is binarized as fingerprint has distribution same on the probabilistic, and the fingerprint device 34 of indexing generates index.
Here, the fingerprint extraction algorithm can be in zero-crossing rate (ZCR:zero crossing rate), energy difference, spectrum flatness, Mei Er frequency cepstral coefficient (MFCC:mel frequency cepstral coefficients) and the frequency centroid (frequency centroide).
Fig. 3 be illustrate with at the figure of relation of structure that is used for searching for the DB file that the system of audio-frequency fingerprint uses according to the embodiment of the invention.
With reference to figure 3, comprise according to the DB of present embodiment group 2: fingerprint DB 21, music information DB 22 and fingerprint index DB 23.
Fingerprint audio values and positional information in the fingerprint DB 21 storage music.Music information DB 22 storages are about the information of music ID and the number of fingerprint.Fingerprint index DB 23 stores the information about the position among the fingerprint DB 21 according to the fingerprint value of binarization.
When according to the system that is used to search for audio-frequency fingerprint of present embodiment after each DB storage corresponding informance, when receiving the request of search predetermined audio file; Said system generates candidate index through the step of indexing shown in Fig. 2; The candidate index that search is generated from fingerprint index DB 23, and the fingerprint positions information of detection candidate index.Then, said system detects finger print information storage, corresponding with fingerprint positions information in fingerprint DB 21, and output music information storage, corresponding with the positional information of music in music information DB 22.
Fig. 4 illustrates the curve map that is used to generate the employed probability distribution of fingerprint index.
With reference to figure 4, said curve map shows the histogram distribution of normalized frequency centroid values.Said curve map clearly show that mean value approaches 0.
Fig. 5 is the figure that illustrates that fingerprint that use handle to extract from audio search generates fingerprint index and search for the process of predetermined audio based on the fingerprint index that is generated.
The same in the image pattern 3, from audio file, prepare three DB files.Said audio fingerprint search apparatus 1 uses three DB files to carry out search service.
With reference to figure 5; When the predetermined audio file is imported, with the step that is used for the samely carrying out the step of extracting audio-frequency fingerprint successively, coming the step of calculated candidate fingerprint, audio-frequency fingerprint that coupling is extracted and come the verification search result based on the index that obtains from the fingerprint that is extracted with the step and the use matching result of the candidate fingerprint index corresponding audio fingerprint that is calculated from the method that audio file generates DB.These steps will be described as follows.
If the input audio file then generates index through using said method to extract audio-frequency fingerprint.That is to say that fingerprint extractor 11 uses the fingerprint extraction algorithm from the audio file of input, to extract audio-frequency fingerprint, and the statistical property of fingerprint statistical analyzer 32 analyzing audio fingerprints, and calculating has the probability distribution of distribution same on the probabilistic.That is to say that fingerprint statistical analyzer 32 is approximately probability model with distributing with fingerprint through the average that calculates audio-frequency fingerprint.Then, the statistical property of fingerprint binarizer 33 analyzing audio fingerprints, and audio-frequency fingerprint is binarized as has distribution same on the probabilistic.Then, index device 34 of fingerprint generates index.
In order to obtain candidate fingerprint value to be used for audio search, obtain the information among the fingerprint DB 21 about the position, said positional information has the manipulative indexing value of fingerprint index DB 23.Therebetween, if audio distortion, then also might distortion from the fingerprint of its extraction.Correspondingly, the index value of fingerprint possibly change.Said index value maybe be owing to noise, equilibrium, compression, analog to digital conversion and digital-to-analog conversion change.For changing, index value generates candidate index.
After generating candidate index, the information that fingerprint index DB 23 obtains about the position with the corresponding candidate index value among the fingerprint DB 21.Then, export the corresponding music result of information of search through the step of coupling and checking fingerprint.
Fig. 6 illustrates the figure that in audio search is handled, generates the process of candidate index.
As shown in Figure 6, if there is N dimension fingerprint value, then based on the absolute value of the difference of the mean value that is used to generate index, arrange N dimension fingerprint value according to ascending order.Decide threshold value according to probability distribution shape, the decision variable position, and consider that variable position generates candidate index.
Fig. 7 illustrates the candidate index that is generated that uses in the audio search step to search for the figure of the step of net result.
As shown in Figure 7, call the fingerprint value with the candidate index value that generated coupling, and calculate the distance between the target audio fingerprint that will search for and position with the fingerprint value that is called.Then, institute's calculated distance and predetermined threshold are compared.If minimum value, is then stored a result less than threshold value.For result highly reliably is provided, the fingerprint place in diverse location repeatedly carries out above-mentioned steps.Then, through being verified, the result exports said net result.
In the present embodiment, suppose that the audio-frequency fingerprint that is extracted has the floating point real number value, and the estimated probability model of fingerprint has bell-shaped distribution as shown in Figure 4, this bell-shaped distribution has mean value " 0 ".Here, shown in equality 1, can use mean value " 0 " that N is tieed up fingerprint and be expressed as N binary number.
[equality 1]
Figure G2008800126394D00061
To convert decimal number into the fingerprint of binary number representation through equality 2.Said decimal system fingerprint number is used as the index of database.
[equality 2]
Index [ i ] = Σ m = 0 N AF B i · [ m ] 2 N - m
For example, be that 16 dimension audio-frequency fingerprints are represented as a value between 0 to 65535, and this value is used as the index in the database under 16 the situation at N.
Audio fingerprint system according to present embodiment generates three data library files that are used for audio search.As shown in Figure 3, three data library files are formed fingerprint DB 21, music information DB 22 and fingerprint index DB 23.
The fingerprint value that fingerprint DB 21 storages are extracted.That is to say that fingerprint DB 21 stores the fingerprint value that is extracted as it is.Music information DB 22 storage is about the information of following music, promptly is based on the information that provides when generating fingerprint and from its music that takes the fingerprint.For example, but music information DB 22 storing various information, such as the length of music ID, copyright information, fingerprint.Fingerprint index DB 23 is transformed to index through equality 1 and equality 2 with fingerprint, and stores the fingerprint value as index according to the positional information among the fingerprint DB 21.
For example, under the situation of 16 dimension fingerprints, as shown in Figure 3, store fingerprint successively with positional information.Also store music information and finger print information with them.Through storing fingerprint index, and said fingerprint index is used for audio search with the positional information of fingerprint with manipulative indexing value shown in Figure 3.
After audio file is prepared three DB files, audio fingerprint system uses the DB file of being prepared to carry out search service.That is to say; If imported predetermined audio file, then execution is used for extracting the step of audio-frequency fingerprint, the step through come the calculated candidate fingerprint from the fingerprint computation index of being extracted, the step of matching candidate fingerprint and the step of using matching result to verify from the said method of audio file generation DB successively.These steps are with as described in more detail below.
When the input audio file, use the audio-frequency fingerprint method for distilling to extract audio-frequency fingerprint, and generate index based on the audio-frequency fingerprint that is extracted.
In order to obtain candidate fingerprint value to be used for audio search, obtain information about the position among the fingerprint DB 21, it has the manipulative indexing value of fingerprint index DB 23.Therebetween, if audio distortion, then also might distortion from the fingerprint of its extraction.Correspondingly, the index value of fingerprint possibly change.Said index value maybe be owing to noise, equilibrium, compression, analog to digital conversion and digital-to-analog conversion change.For changing, index value generates candidate index.
After generating candidate index, with candidate index obtain accordingly among the fingerprint DB 21 positional information, this positional information has the corresponding candidate index value among the fingerprint index DB 23.Then, after fingerprint matching and verification step, export the Search Results of corresponding music information.
Hereinafter, with describing the step that generates candidate index in more detail.Under the situation of N dimension fingerprint, according to arranging N dimension value with the order of mean value (for example, the mean value in the present embodiment " the 0 ") minimum distance of the probability distribution that is used to generate index.Here, can calculate the position that has the big probability of change according to probability distribution successively.Can be chosen in through the position in the predetermined distance range of probability distribution.In addition, selection predetermined number destination locations under the situation of condition can had no.
Equality [3]
sort ascend{|F i-mean i|}
Be decided to be constant if having the threshold value of bell probability distribution in the audio-frequency fingerprint that formerly uses, then can obtain information about the position that can change according to fingerprint.After the decision position, use the index generation method that in the step that takes the fingerprint, uses to generate index.In addition, generate all possible index accordingly with variable position.
For example, if 4 dimension audio finger value are (0.2,0.1,0.4; 0.2), if the decision index value be 0, and if second position have the big probability that will change; Then its index is 0111, and candidate index is 0011, and this is because second position possibly change.
After calculated candidate index as described above, from the index of correspondence, obtain candidate fingerprint with reference to the positional information in fingerprint DB 21, and through according to positional information and comparison is arranged the candidate fingerprint that obtains apart from the distance of the target audio fingerprint that will search for.
Here, remove redundancy, and calculate distance from K predetermined audio fingerprint to the fingerprint value of fingerprint DB 21 with predetermined length based on the value corresponding with reference position as a reference.For example, calculate Euclidean distance, and with the Euclidean distance of being calculated and threshold ratio.If institute's calculated distance is less than threshold value, then search for music information in music information DB 22, and output Search Results.If be not less than, then as a result of export and indicate the basic information value that does not search music.Come to have described well above-mentioned step with reference to figure 7.Shown in equality 4, as the end value of reliability by the Music ID (music ID) of the music information in the referred database, instruction time the position Position (position) and the Distance (distance) of indication range difference form.
[equality 4]
R[n]＝{MusicID[n]，Position[n]，Distance[n]}
When the value of music ID during less than threshold value, the value of using the positional information among the fingerprint index DB 23 to come computational music ID.Because music information DB 22 has stored the fingerprint number of each music, so if m music ID be the result, then the value of position greater than last fingerprint number sum to (m-1) individual music.In addition, the value of position is less than the fingerprint number sum of last music to (m+1) individual music.Utilize this fact, the value of computational music ID.
[equality 5]
&Sigma; k = 1 m - 1 feat _ num [ k ] < position [ n ] < &Sigma; k = 1 m + 1 feat _ num [ k ]
General System can repeatedly be carried out search step, so that improve the reliability of search.After the candidate index that calculates the pre-position, can search for the candidate index of another position once more, and repeatedly carry out step, to obtain the result in order to the search for candidate fingerprint.
Parameter value based on being selected by system decides such result, and with the number of times of carrying out search step memory search result as many.After verification step, the result that output is stored is as final Search Results.
In verification step, the fingerprint that from the predetermined audio signal that will search for, extracts is the successive value on the time domain.That is to say, take the fingerprint successively in time.Similarly, take the fingerprint successively in time and it is stored among the fingerprint DB 21 that is generated.
That is to say that result that the fingerprint of use pre-position is searched for and the result that the fingerprint that uses the next position place is searched for have last same range difference of time.Based on this fact, use equality 6 verification search results.
[equality 6]
If MusicID [n+p]=MusicID [n],
P-1≤Position [n+p]-Position [n]≤p+1 then
That is to say that if the Search Results of p position is identical, then the difference of positional information must be greater than p-1 and less than p+1.Based on this fact, verified the result who repeats to obtain.If satisfy said condition, then this result is output as net result.
Each has 27,000 audio files generation test databases of 40 seconds length for it, to be used for checking.Then, from test database search be compressed to its each have 100 audio files of MP3 audio file of the 32kbps of 20 seconds length.Extract the fingerprint of describing in the present embodiment that is used for 16 dimensions, and use the fingerprint that is extracted.In order to compare distance, use 52 fingerprints.In addition, search step is carried out 5 times to be used for checking.In order to compare performance, carry out search successively under the same conditions.In searching for successively, all fingerprints of search in DB, and will be through relatively confirming as net result apart from the result with minimum value.
Form 1
[form 1]
[form]
Discrimination (%) Be used to search for the T.T. (second) (comprising extraction time and search time) of 100 audio files
Search successively 99％ 368 seconds
Search (candidate index threshold value=1) based on index 87％ 31 seconds
Search (candidate index threshold value=5) based on index 96％ 41 seconds
Shown in form 1, more a lot of soon than the search speed of search successively according to the search speed based on the search of index of present embodiment.In addition, the deterioration that is used for the discrimination of bit index values can overcome through the parameter value of adjustment decision candidate index, although Zong expand a little search time.
That is to say, although Zong expand search time about 10 seconds long, but discrimination can be improved to 96% from 87%.Can reduce search time and reach search successively total search time 1/9.
Based on fast search time and high discrimination, can be applicable to filtration and supervision at the file in Large Volume Data storehouse according to the system and method that is used for searching audio fingerprint by index information of present embodiment.Particularly, can be applicable to file filter, to solve content (UCC) field that the user creates or the copyright problem in the P2P field according to the system and method that is used for searching audio fingerprint by index information of present embodiment.
Obvious to those skilled in the art is to carry out various modifications and variation in the present invention.Like this, as long as modification of the present invention and variation fall in the scope of accompanying claims and their equivalent, the present invention just is intended to cover these modifications and variation.

Claims (11)

1. system that is used to search for audio-frequency fingerprint comprises:
Index processor is used for generating index based on the statistical property of the audio-frequency fingerprint of audio file;
Database D B group is used for store audio fingerprints, and match index, audio-frequency fingerprint and music information continuously; With
Audio fingerprint search apparatus is used for generating index based on the statistical property of the audio-frequency fingerprint of new input audio file, and is used for newly importing the corresponding music information of audio file through the index search that search from the DB group is generated by said audio fingerprint search apparatus,
Wherein in said index processor and audio fingerprint search apparatus; Realize that through following steps index generates: use the fingerprint extraction algorithm to extract audio-frequency fingerprint; The audio-frequency fingerprint of mean value through calculate to(for) the audio-frequency fingerprint that is extracted is approximately probability model with distributing with fingerprint; And carry out binarization based on this mean value and generate index to have distribution same on the probabilistic
Wherein said audio fingerprint search apparatus comprises:
Fingerprint extractor is used to extract the audio-frequency fingerprint of new audio file;
The candidate index search unit; Be used for through according to the audio finger value of being extracted and the ascending order that is used to generate the absolute value of the difference between the mean value of the audio finger value that new index uses sort the audio-frequency fingerprint that extracted value and consider variable position, generate candidate index;
The fingerprint matching unit is used for coupling and candidate index corresponding audio fingerprint and has extracted audio-frequency fingerprint; And
The result verification unit is used to measure and candidate index corresponding audio fingerprint and extracted the distance between the audio-frequency fingerprint, and if measured distance be in the predetermined range, then through temporal information checking result.
2. according to the system of claim 1, wherein said DB group comprises:
Fingerprint DB is used for the positional information of the audio-frequency fingerprint and the music information of storing audio files;
Music information DB is used to store the music ID of music information and about the information of fingerprint number; And
Fingerprint index DB, the information that is used for storing the fingerprint DB corresponding about the audio-frequency fingerprint position with index.
3. according to the system of claim 2, audio-frequency fingerprint that wherein said index processor will be extracted from audio file and music information are transferred to corresponding DB.
4. according to the system of claim 3, wherein said index processor comprises:
Fingerprint extractor is used to use the fingerprint extraction algorithm to extract said audio-frequency fingerprint;
The fingerprint statistical analyzer is used for distributing fingerprint being approximately said probability model through the average that obtains audio-frequency fingerprint for the audio-frequency fingerprint that is extracted;
The fingerprint binarizer is used to utilize the mean value of said audio-frequency fingerprint to carry out binarization, to have distribution same on the said probabilistic; And
The fingerprint device of indexing is used to generate index.
5. according to the system of claim 4, wherein said fingerprint extraction algorithm is one of zero-crossing rate, energy difference, spectrum flatness, Mei Er frequency cepstral coefficient and frequency centroid.
6. according to the system of claim 1, wherein said variable position is to decide through the threshold value setting in probability distribution shape.
7. one kind is used to use index information to search for the method for audio-frequency fingerprint, comprises the steps:
A) statistical property based on the audio-frequency fingerprint of audio file generates index, and prepares to be used for the database D B group of store audio fingerprints, and this DB group is the index, audio-frequency fingerprint and the music information that are generated of coupling continuously;
B) statistical property based on the audio-frequency fingerprint of new input audio file generates index; And
C) generate the corresponding music information that index search is used for newly importing audio file through what search from DB group generated in step b),
Wherein at step a) and b) in index comprise the steps: in generating
Use the fingerprint extraction algorithm to extract audio-frequency fingerprint;
The audio-frequency fingerprint of mean value through calculate to(for) the audio-frequency fingerprint that is extracted is approximately probability model with distributing with fingerprint; And
Through carrying out binarization to have distribution same on the probabilistic based on this mean value, generate index,
Wherein said step c) comprises the steps:
Through according to the audio finger value of being extracted and the ascending order that in step b), is used to generate the absolute value of the difference between the mean value of the audio finger value that index uses sort the audio-frequency fingerprint that extracted value and consider variable position, generate candidate index;
Coupling is with candidate index corresponding audio fingerprint and extracted audio-frequency fingerprint; And
Through measuring with candidate index corresponding audio fingerprint and extracted the distance between the audio-frequency fingerprint, and measured distance is in the predetermined range, utilizes temporal information to verify the result.
8. according to the method for claim 7, wherein step a) comprises the steps:
The audio-frequency fingerprint of storing audio files and the positional information of music information at fingerprint DB place;
Storage is as the music ID and the information that comprises fingerprint number of unique ID of music information at music information DB place; And
Information in the fingerprint index DB place storage fingerprint DB corresponding about the audio-frequency fingerprint position with index.
9. according to the method for claim 7; Wherein tie up the situation of fingerprint value for N; Sort according to ascending order based on absolute value with the difference of the mean value of the probability distribution that is used to generate index N dimension fingerprint value and change the bit value of correspondence position through the position of approaching the one-dimensional finger print value of mean value in the middle of according to this N dimension fingerprint value of threshold value decision after, generate candidate index.
10. according to the method for claim 9; Wherein reference fingerprint DB and pre-determined number as many calculate from the fingerprint positions information of candidate index calculating and from the Euclidean distance between the fingerprint of new input audio file acquisition, and search for the music information with minor increment.
11.,, then as a result of export the music information of fingerprint positions with minor increment if wherein institute's calculated distance is in the threshold range in the candidate index according to the method for claim 10.
CN2008800126394A 2007-04-17 2008-04-14 System and method for searching audio fingerprint by index information Expired - Fee Related CN101663708B (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
KR1020070037399 2007-04-17
KR1020070037399A KR100862616B1 (en) 2007-04-17 2007-04-17 Searching system and method of audio fingerprint by index information
KR10-2007-0037399 2007-04-17
PCT/KR2008/002085 WO2008127052A1 (en) 2007-04-17 2008-04-14 System and method for searching audio fingerprint by index information

Publications (2)

Publication Number Publication Date
CN101663708A CN101663708A (en) 2010-03-03
CN101663708B true CN101663708B (en) 2012-10-10

Family

ID=39864101

Family Applications (1)

Application Number Title Priority Date Filing Date
CN2008800126394A Expired - Fee Related CN101663708B (en) 2007-04-17 2008-04-14 System and method for searching audio fingerprint by index information

Country Status (3)

Country Link
KR (1) KR100862616B1 (en)
CN (1) CN101663708B (en)
WO (1) WO2008127052A1 (en)

Families Citing this family (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101833579B (en) * 2010-05-11 2012-09-05 同方知网(北京)技术有限公司 Method and system for automatically detecting academic misconduct literature
US8584198B2 (en) 2010-11-12 2013-11-12 Google Inc. Syndication including melody recognition and opt out
US8584197B2 (en) * 2010-11-12 2013-11-12 Google Inc. Media rights management using melody identification
GB2487795A (en) * 2011-02-07 2012-08-08 Slowink Ltd Indexing media files based on frequency content
CN102314875B (en) * 2011-08-01 2016-04-27 北京音之邦文化科技有限公司 Audio file identification method and device
US8949872B2 (en) * 2011-12-20 2015-02-03 Yahoo! Inc. Audio fingerprint for content identification
CN103179430A (en) * 2011-12-20 2013-06-26 中国电信股份有限公司 Method, device and server for audio and video content transcoding on basis of cloud computing
US9552607B2 (en) * 2012-03-21 2017-01-24 Beatport, LLC Systems and methods for selling sounds
CN103995890A (en) * 2014-05-30 2014-08-20 杭州智屏软件有限公司 Method for updating and searching for data of real-time audio fingerprint search library
US9881083B2 (en) 2014-08-14 2018-01-30 Yandex Europe Ag Method of and a system for indexing audio tracks using chromaprints
WO2016024172A1 (en) 2014-08-14 2016-02-18 Yandex Europe Ag Method of and a system for matching audio tracks using chromaprints with a fast candidate selection routine
CN105138541B (en) * 2015-07-08 2018-02-06 广州酷狗计算机科技有限公司 The method and apparatus of audio-frequency fingerprint matching inquiry
KR101661666B1 (en) * 2015-11-20 2016-09-30 광운대학교 산학협력단 Hybrid audio fingerprinting apparatus and method
KR102037220B1 (en) 2017-11-06 2019-10-29 주식회사 아이티밥 Audio finger print matching system
KR102037221B1 (en) 2017-11-06 2019-10-29 주식회사 아이티밥 Audio finger print matching method
CN110322886A (en) * 2018-03-29 2019-10-11 北京字节跳动网络技术有限公司 A kind of audio-frequency fingerprint extracting method and device
CN113536026B (en) * 2020-04-13 2024-01-23 阿里巴巴集团控股有限公司 Audio searching method, device and equipment
KR102337677B1 (en) * 2020-07-16 2021-12-09 (주)휴먼스케이프 System for embedding digital verification fingerprint and Method thereof

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1708758A (en) * 2002-11-01 2005-12-14 皇家飞利浦电子股份有限公司 Improved audio data fingerprint searching

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE19930518C1 (en) * 1999-07-05 2000-10-12 Thyssenkrupp Stahl Ag Production of a non grain-oriented electric sheet used as core material in motors and generators comprises producing a hot strip from a steel pre-material, hot rolling and spooling
DE10133333C1 (en) * 2001-07-10 2002-12-05 Fraunhofer Ges Forschung Producing fingerprint of audio signal involves setting first predefined fingerprint mode from number of modes and computing a fingerprint in accordance with set predefined mode
KR100473163B1 (en) * 2002-01-15 2005-03-08 주식회사 에듀미디어텍 A storage medium storing multimedia contents and apparatus and method for reproducing the same
WO2005101243A1 (en) * 2004-04-13 2005-10-27 Matsushita Electric Industrial Co. Ltd. Method and apparatus for identifying audio such as music

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1708758A (en) * 2002-11-01 2005-12-14 皇家飞利浦电子股份有限公司 Improved audio data fingerprint searching

Also Published As

Publication number Publication date
WO2008127052A1 (en) 2008-10-23
CN101663708A (en) 2010-03-03
KR100862616B1 (en) 2008-10-09

Similar Documents

Publication Publication Date Title
CN101663708B (en) System and method for searching audio fingerprint by index information
AU2020200997B2 (en) Optimization of audio fingerprint search
CN102959624B (en) System and method for audio media recognition
CN101523415B (en) Method of deriving representation of image, method for recognizing and comparing image and device thereof
KR100838674B1 (en) Audio fingerprinting system and method
Seo et al. Audio fingerprinting based on normalized spectral subband centroids
CN101620677B (en) Fingerprint identification method based on triangulation and LOD technology
US20020133499A1 (en) System and method for acoustic fingerprinting
CN101189658A (en) Automatic identification of repeated material in audio signals
CN1997989A (en) Method and apparatus for automatic detection and identification of broadcast audio or video programming signal
KR100733145B1 (en) Fingerprint Producing Method and Audio Fingerprinting System Based on Normalized Spectral Subband Centroids
WO2005101243A1 (en) Method and apparatus for identifying audio such as music
CN101042870B (en) Method and apparatus for estimating degree of similarity between voices
CN101980197B (en) Long time structure vocal print-based multi-layer filtering audio frequency search method and device
CN101594527B (en) Two-stage method for detecting templates in audio and video streams with high accuracy
US10776420B2 (en) Fingerprint clustering for content-based audio recognition
CN103294696A (en) Audio and video content retrieval method and system
Yao et al. Enhancing sampling and counting method for audio retrieval with time-stretch resistance
Ribbrock et al. A full-text retrieval approach to content-based audio identification
Cai et al. Two-layer large-scale cover song identification system based on music structure segmentation
US20220335082A1 (en) Method for audio track data retrieval, method for identifying audio clip, and mobile device
CN112784099B (en) Sampling counting audio retrieval method resisting tonal modification interference
CN103065661A (en) Signal detection method for recording medium
Kampelopoulos et al. Byzantine Hymn Recognition with Audio Fingerprints Resistant to Noise, Tempo and Scale Changes
Kamesh et al. Audio fingerprinting with higher matching depth at reduced computational complexity

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20121010

Termination date: 20200414

CF01 Termination of patent right due to non-payment of annual fee