EP1687806A1 - Matching data objects by matching derived fingerprints - Google Patents
Matching data objects by matching derived fingerprintsInfo
- Publication number
- EP1687806A1 EP1687806A1 EP04799078A EP04799078A EP1687806A1 EP 1687806 A1 EP1687806 A1 EP 1687806A1 EP 04799078 A EP04799078 A EP 04799078A EP 04799078 A EP04799078 A EP 04799078A EP 1687806 A1 EP1687806 A1 EP 1687806A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- fingeφrint
- query
- candidate
- fingerprint
- matching
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/60—Information retrieval; Database structures therefor; File system structures therefor of audio data
- G06F16/68—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/683—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/783—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/48—Matching video sequences
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
Definitions
- the invention relates to a method and apparatus for matching fingerprints.
- Fingerprinting technology is used to identify media content (such as audio or video).
- An audio or video segment is identified by extracting a fingerprint from it, and searching the extracted fingerprint in a database in which fingerprints of known contents are stored. Content is identified if the similarity between the extracted fingerprint and the stored finge ⁇ rint is deemed sufficient.
- the prime objective of multimedia fingerprinting is an efficient mechanism to establish the perceptual equality of two multimedia objects: not by comparing the (typically large) objects themselves, but by comparing the associated fingerprints (small by design). In most systems using fingerprinting technology, the fingerprints of a large number of multimedia objects along with its associated metadata (e.g. in the case of song information, name of artist, title and album) are stored in a database.
- the fingerprints serve as an index to the metadata.
- the metadata of unidentified multimedia content are then retrieved by computing a fingerprint and using this as a query in the fingerprint/metadata database.
- the advantage of using fingerprints instead of the multimedia content itself is three-fold: reduced memory/storage requirements as fingerprints are relatively small; efficient comparison as perceptual irrelevancies have already been removed from fingerprints; and efficient searching as the data set to be searched is smaller.
- a fingerprint can be regarded as a short summary of an object. Therefore, a fingerprint function should map an object X consisting of a large number of bits to a fingerprint F of only a limited number of bits.
- the degree of robustness of a system determines whether a particular object can be correctly identified from a fingerprint in cases where signal degradation is present.
- the fingerprint F should be based on perceptual features which are invariant (at least to a certain degree) with respect to signal degradations.
- a severely degraded signal will still yield a similar finge ⁇ rint to a fmge ⁇ rint of an original undegraded signal.
- the "false rejection rate" (FRR) is generally used to express the measure of robustness of the fingerprinting system. A false rejection occurs when the finge ⁇ rints of perceptually similar objects are too different to lead to a positive identification.
- the reliability of a finge ⁇ rinting system refers to how often an object is identified falsely. In other words, reliability relates to a "false acceptance rate" (FAR) - i.e. the probability that two different objects may be falsely declared to be the same.
- FAR false acceptance rate
- fingerprint size is important to any finge ⁇ rinting system. In general, the smaller the fingerprint size, the more finge ⁇ rints can be stored in a database. Finge ⁇ rint size is often expressed in bits per second and determines to a large degree the memory resources that are needed for a finge ⁇ rint database server. Granularity is a parameter that can depend on the application and relates to how long (large) a particular sample of an object is required in order to identify it.
- Search speed refers to the time needed in order to find a fingerprint in a finge ⁇ rint database.
- the above five basic parameters have a large impact on each other. For instance, to achieve a lower granularity, one needs to extract a larger finge ⁇ rint to obtain the same reliability. This is due to the fact that the false acceptance rate is inversely related to finge ⁇ rint size.
- search speed will generally increase when one designs a more robust finge ⁇ rint.
- a finge ⁇ rint may be based on extracting a feature- vector from an originating audio or video signal. Such vectors are stored in a database with reference to the relevant metadata (e.g. title, author, etc.). Upon reception of an unknown signal, a feature- vector is extracted from the unknown signal, which is subsequently used as a query on the fingerprint database. If the distance between the query feature-vector and its best match in the database is below a given threshold, then the two items are declared equal and the associated metadata are returned: i.e. the received content has been identified.
- the threshold that is used in the matching process is a trade-off between the false acceptance rate (FAR) and the false rejection rate (FRR). For instance, increasing the threshold (i.e.
- a method of comparing a query fmge ⁇ rint to a candidate finge ⁇ rint the method being characterised by comprising: determining a statistical model of the query fingerprint and/or a candidate finge ⁇ rint; and on the basis of the statistical model, deriving a threshold distance within which the query fingerprint and the candidate fingerprint may be declared similar.
- a second aspect of the invention provides a method of matching a query object to a known object, wherein a plurality of candidate finge ⁇ rints representing a plurality of candidate objects are pre-stored in a database, the method comprising receiving an information signal forming part of the query object and constructing a query finge ⁇ rint therefrom and comparing the query finge ⁇ rint to a candidate finge ⁇ rint in the database, the method being characterised in that it further comprises the steps of: determining a statistical model for the query fingerprint and/or the candidate fingerprint; and on the basis of the statistical model, deriving a threshold distance within which the query finge ⁇ rint and the candidate finge ⁇ rint may be declared similar.
- the derivation of a threshold based upon a statistical model of the particular finge ⁇ rint provides adaptive threshold setting which may optimise the F.A.R. according to query fingerprint type/ internal characteristics giving improved matching qualities over the application of an arbitrary thresholding system.
- the candidate finge ⁇ rint is declared the best matching candidate finge ⁇ rint and the candidate object represented by the best matching candidate fingerprint and the query object represented by the query finge ⁇ rint are deemed to be the same.
- the statistical model comprises the result of performing an internal correlation on the query finge ⁇ rint and/or the candidate finge ⁇ rint.
- the finge ⁇ rints comprise binary values and the statistical model is computed for the query finge ⁇ rint by determining a transition probability q for the query fingerprint by determining how many bits of a query fingerprint frame F(m,k) are different from their corresponding bit in their preceding finge ⁇ rint frame F(m,k-1) and dividing the number of transitions by a maximum value M*(k-1), which would be obtained if all finge ⁇ rint bits were of an opposite state to their corresponding preceding bit, where each finge ⁇ rint comprises M bits per frame and spans K frames, in which k is the frame index (ranging from 0 to K) and m is the bit-index within a frame (ranging from 0 to M).
- the invention provides apparatus for matching a query object to a known object, the apparatus comprising a finge ⁇ rint extaction module for receiving an information signal forming part of a query object and constructing a query finge ⁇ rint therefrom and a finge ⁇ rint matching module for comparing the query finge ⁇ rint to candidate fingerprints stored in a database to one or more candidate finge ⁇ rints, the apparatus being characterised in that it further comprises: a statistical module for determining a statistical model of the query f ⁇ nge ⁇ rint and/or one or more of the one or more candidate finge ⁇ rints; a threshold determiner ,deriving on the basis of the statistical model, a threshold distance T within which the query finge ⁇ rint and a candidate fingerprint may be declared similar; and an identification module arranged such that if a candidate finge ⁇ rint
- the candidate fingerprint is declared the best matching candidate finge ⁇ rint and the candidate object represented by the best matching candidate finge ⁇ rint and the query object represented by the query finge ⁇ rint are deemed to be the same.
- Figure 1 shows a functional block diagram illustrating a finge ⁇ rinting method with an adaptive threshold in accordance with an embodiment of the invention
- Figure 2 is a flow diagram explaining in general the process involved in finding and matching finge ⁇ rints in accordance with an embodiment of the invention
- Figure 3 is a flow diagram illustrating in general the methodology for determining an adaptive threshold in accordance with an embodiment of the present invention
- Figure 4 is a flow diagram illustrating a specific adaptive threshold setting methodology in accordance with embodiments of the invention.
- FIG. 1 there is shown a functional block diagram divided into a client side 100 and a database server side 200.
- an object is received by a finge ⁇ rint extraction module 110 and a query fingerprint F computed for the object.
- the query f ⁇ nge ⁇ rint F is, on the one hand, passed to an statistical module 120 and, on the other hand, also passed to the database server side 200.
- the statistical module 120 determines a measure of randomness/correlation (for instance, it may determine the internal correlation) of the query fingerprint F and passes this information to a threshold determiner 130.
- the threshold determiner 130 on the basis of the information from the module 120 adaptively sets a threshold level T and passes this threshold level T to the database server side 200.
- a matching module 210 receives the query finge ⁇ rint F from the client side 100 and looks for the best match of that finge ⁇ rint within a database of known finge ⁇ rints. The best match information is then passed to a threshold comparison module 220 to determine whether a best matching candidate fingerprint is close enough (within threshold distance T) to the query finge ⁇ rint to determine the identity of the input object with the matched object corresponding to the candidate finge ⁇ rint.
- the threshold comparison module 220 might, for instance, compare the Hamming distance between a finge ⁇ rint block Hi and a finge ⁇ rint block H 2 relating to the best match in the database 210 and check to see whether the Hamming distance between the two blocks is below the threshold distance T, supplied to the comparison module 220 from the threshold determining module 130. An identification decision is made by identification module 230 so that if the Hamming distance between the two derived finge ⁇ rint blocks is below the threshold distance T then the unidentified query object is declared similar to the object found in the database and the relevant metadata is returned.
- the query finge ⁇ rint F and the threshold T are sent by the client side 100 to the database server side 200.
- the threshold T could also be determined at the database server side 200 and that, therefore, modifications of the aforementioned block diagram are of course possible.
- FIG 2 there is shown a flow diagram which explains, in general, the operation of the components of the block diagram of Figure 1 in finding and matching finge ⁇ rints.
- an object sample e.g. in the case of video a short "clip
- a query fingerprint dete ⁇ nined based upon the sample.
- This query finge ⁇ rint may be determined in accordance with any suitable prior art method (such as disclosed in US 2002/0178410 Al).
- a threshold for the query finge ⁇ rint is determined in accordance with the particular characteristics (randomness/correlation) of the query finge ⁇ rint.
- the query finge ⁇ rint is matched to finge ⁇ rints held on the database server side 200, to return a best matching candidate. Again, this matching process may be performed conventionally, so as to return the closest match to the query finge ⁇ rint.
- step S300 the "distance" between the query finge ⁇ rint and the best match candidate will be determined and, in a step S400, it is checked whether or not the "distance" is less than the threshold distance determined in step S200. If the distance between the query finge ⁇ rint and the best match candidate is found in step S400 to be greater than the threshold, then in step S500 the result is returned that no matching object to the query object has been found. On the other hand, if the distance between query fingerprint and best match candidate fingerprint is less than the threshold distance in step S400, then in step S600 a match is declared between the query object and the object in the database relating to the best matching candidate. Metadata etc., of the best matching object may then be returned to a user.
- This possibility is denoted by the alternative pathway B from S300 to S200.
- the threshold T may be set based upon a combination of the characteristics of both the query finge ⁇ rint and the best matching candidate finge ⁇ rint e.g. by setting a threshold at the average between two derived adaptive thresholds TI, T2.
- Figure 3 is a flow diagram illustrating the general methodology for adaptively determining a given threshold T.
- step S210 the query candidate finge ⁇ rint is received and a measure of randomness of the finge ⁇ rint determined, then in step S220 a threshold distance is set according to the measure of randomness found in step S210.
- the threshold value T (TI or T2) used in the comparison is adapted to the randomness/correlation in either the query-finge ⁇ rint or/and the best matching candidate. More specifically, in the case of threshold determination for a query f ⁇ nge ⁇ rint, the correlation of the query finge ⁇ rint is determined and, from this correlation, the threshold to be used during matching is computed.
- the threshold is determined upon the internal correlation of the query fingerprint, a best matching candidate finge ⁇ rint or a combination of the two.
- a solution can be derived for adaptively setting the threshold. The solution to the adaptive threshold setting problem is shown in Figure 4.
- a step S221 the internal correlation of the finge ⁇ rint in question is determined, in step S222 the transition probability for the finge ⁇ rint is determined based upon the internal correlation and in step S223, the threshold distance is set adaptively, based upon both the transition probability (explained below) and a desired false acceptance rate.
- the fingerprint consist of M bits per frame and span K frames.
- the fingerprint can be denoted F(m,k), where k is the frame index (ranging from 0 to K-l) and m is the bit-index within a frame (ranging from 0 to M-l).
- Extract finge ⁇ rint F Determine the transition probability q for finge ⁇ rint F, as follows:
- the threshold distance is set adaptively based on the internal characteristics of a particular query sample or, indeed, of a particular candidate sample or set of samples.
- the invention can also be applied using so-called "pruning" techniques in which certain candidates within the database can be immediately discarded if it is obvious that they can never make a match - searching/matching can then be done within a much reduced search space.
- methods and apparatus for setting an adaptive threshold are disclosed, in which the threshold depends upon specific characteristics of a fingerprint.
- the particular method is very suitable for use in matching of video content, but is not limited to this.
- the techniques described may be applied to various different areas of technology and various different signal types, including, but not limited to, audio signals, video signals, multimedia signals. The skilled man will realise that the processes described may be implemented in software, hardware, or any suitable combination.
- the invention relates to methods and apparatus for finge ⁇ rint matching.
- apparatus comprising a finge ⁇ rint extraction module (110), a finge ⁇ rint matching module (210), a statistical module (120) and an identification module.
- the finge ⁇ rint extraction module (110) receives an information signal forming part of a query object and constructs a query finge ⁇ rint.
- the fingerprint matching module (210) compares the query finge ⁇ rint to candidates stored in a database (215) to find at least one potentially best matching candidate.
- the statistical module determines a statistical model of the query finge ⁇ rint so as to, for instance, determine the statistical distribution of the query fmge ⁇ rint.
- the threshold determiner (120) is arranged, on the basis of the distribution of the query finge ⁇ rint to derive an adaptive threshold distance T within which the query finge ⁇ rint and a potentially best matching candidate may be declared similar by the identification module (130).
- an improved false acceptance rate F.A.R. and other advantages may be achieved.
Abstract
The invention relates to methods and apparatus for matching a query data object with a candidate data object by esetracting and comparing fingerprints of said data objects. In an embodiment of the invention apparatus comprising a fingerprint extraction module (110), a fingerprint matching module (210), a statistical module (120) and an identification module is provided. The fingerprint extraction module (110) receives an information signal forming part of a query object and constructs a query fingerprint. The fingerprint matching module (210) compares the query fingerprint to candidates stored in a database (215) to find at least on potentially best matching candidate. Meanwhile, the statistical module determines a statistical model of the query fingerprint so as to, for instance, determine the statistical distribution of certain information inside the query fingerprint. The threshold determiner (120) is arranged, on the basis of the distribution of the query fingerprint to derive an adaptive threshold distance T within which the query fingerprint and a potentially best matching candidate may be declared similar by the identification module (130). By setting a threshold which ma depand on satatistical data derived from the query and/or candidate fingerprint , an improved false acceptance rate F.A.R. may be achieved.
Description
MATCHING DATA OBJECTS BY MATCHING DERIVED FINGERPRINTS
FIELD OF THE INVENTION The invention relates to a method and apparatus for matching fingerprints.
BACKGROUND OF THE INVENTION Fingerprinting technology is used to identify media content (such as audio or video). An audio or video segment is identified by extracting a fingerprint from it, and searching the extracted fingerprint in a database in which fingerprints of known contents are stored. Content is identified if the similarity between the extracted fingerprint and the stored fingeφrint is deemed sufficient. The prime objective of multimedia fingerprinting is an efficient mechanism to establish the perceptual equality of two multimedia objects: not by comparing the (typically large) objects themselves, but by comparing the associated fingerprints (small by design). In most systems using fingerprinting technology, the fingerprints of a large number of multimedia objects along with its associated metadata (e.g. in the case of song information, name of artist, title and album) are stored in a database. The fingerprints serve as an index to the metadata. The metadata of unidentified multimedia content are then retrieved by computing a fingerprint and using this as a query in the fingerprint/metadata database. The advantage of using fingerprints instead of the multimedia content itself is three-fold: reduced memory/storage requirements as fingerprints are relatively small; efficient comparison as perceptual irrelevancies have already been removed from fingerprints; and efficient searching as the data set to be searched is smaller. A fingerprint can be regarded as a short summary of an object. Therefore, a fingerprint function should map an object X consisting of a large number of bits to a fingerprint F of only a limited number of bits. There are five main parameters of a fingerprint system: robustness; reliability; fingerprint size; granularity; and search speed (or scalability). The degree of robustness of a system determines whether a particular object can be correctly identified from a fingerprint in cases where signal degradation is present. In order to achieve high robustness the fingerprint F should be based on perceptual features which are invariant (at least to a certain degree) with respect to signal degradations.
Preferably, a severely degraded signal will still yield a similar fingeφrint to a fmgeφrint of an original undegraded signal. The "false rejection rate" (FRR) is generally used to express the measure of robustness of the fingerprinting system. A false rejection occurs when the fingeφrints of perceptually similar objects are too different to lead to a positive identification. The reliability of a fingeφrinting system refers to how often an object is identified falsely. In other words, reliability relates to a "false acceptance rate" (FAR) - i.e. the probability that two different objects may be falsely declared to be the same. Obviously, fingerprint size is important to any fingeφrinting system. In general, the smaller the fingerprint size, the more fingeφrints can be stored in a database. Fingeφrint size is often expressed in bits per second and determines to a large degree the memory resources that are needed for a fingeφrint database server. Granularity is a parameter that can depend on the application and relates to how long (large) a particular sample of an object is required in order to identify it. Search speed (or scalability), as it sounds, refers to the time needed in order to find a fingerprint in a fingeφrint database. The above five basic parameters have a large impact on each other. For instance, to achieve a lower granularity, one needs to extract a larger fingeφrint to obtain the same reliability. This is due to the fact that the false acceptance rate is inversely related to fingeφrint size. Another example: search speed will generally increase when one designs a more robust fingeφrint. Having discussed the basic parameters of a fingeφrinting system, a general description of a typical fingeφrinting system is now made. A fingeφrint may be based on extracting a feature- vector from an originating audio or video signal. Such vectors are stored in a database with reference to the relevant metadata (e.g. title, author, etc.). Upon reception of an unknown signal, a feature- vector is extracted from the unknown signal, which is subsequently used as a query on the fingerprint database. If the distance between the query feature-vector and its best match in the database is below a given threshold, then the two items are declared equal and the associated metadata are returned: i.e. the received content has been identified. The threshold that is used in the matching process is a trade-off between the false acceptance rate (FAR) and the false rejection rate (FRR). For instance, increasing the threshold (i.e. increasing the acceptable "distance" between two fingerprints for those fingeφrints to still be judged similar) increases the FAR, but at the same time it reduces the
FRR. The trade-off between FAR and FRR is usually done via the so-called Neyman-Pearson approach. This means that the threshold is selected to have the smallest value which keeps the FAR below a pre-specified, allowable level. The FRR is not used for determining the threshold, but merely results from the selected threshold value. US 2002/0178410 Al (Haitsma, Kalker, Baggen and Oostveen) discloses a method and apparatus for generating and matching fingeφrints of multimedia content. In this document, it is described on page 4 thereof how two 3 second audio clips are declared similar if the Hamming distance between two derived fingeφrint blocks Hi and H2 is less than a certain threshold value T. In order to analyse the choice of the threshold T, the authors of US
2002/0178410 assume that the fingeφrint extraction process yields random i.i.d. (independent and identically distributed) bits. The number of bit errors will then have a binomial distribution with parameters (n, p) where n equals the number of bits extracted and p (=0.5) is the probability that a 0 or 1 bit is extracted. Since n is large, the binomial distribution can be approximated by a normal distribution with a mean μ=np and a standard deviation σ = sjnp(l - p). Given a fingeφrint block Hi, the probability that a randomly selected fingerprint block H2 has less than T= a n errors with respect to Hi is then given by:
However, in practice robust fingeφrints have high correlation along the time axis. This may be due to the large time correlation of the underlying video sequence, or the overlap of audio frames. Experiments for audio fingeφrints show that the number of erroneous bits is normally distributed, but that the standard deviation is approximately 3 times larger than the i.i.d. case. Equation (1) therefore is modified to include this factor 3.
The above approach assumes that the distribution between the fingerprints is stationary. Although this seems to be a reasonable assumption for certain technologies, this is definitely not the case for video fingeφrinting. In video fingeφrinting, the amount of "activity" in the video is directly reflected in the correlation of the fingeφrint bits: prolonged stills lead to constant (i.e., very highly correlated) fingeφrints, whereas a "flashy" music clip will lead to a very low correlation between the fingeφrint bits. This non-stationarity leads to problems in determining an appropriate value for the threshold.
OBJECT AND SUMMARY OF THE INVENTION It is an aim of embodiments of the present invention to propose an arrangement for providing an adaptive thresholding technique. According to a first aspect of the invention, there is provided a method of comparing a query fmgeφrint to a candidate fingeφrint, the method being characterised by comprising: determining a statistical model of the query fingerprint and/or a candidate fingeφrint; and on the basis of the statistical model, deriving a threshold distance within which the query fingerprint and the candidate fingerprint may be declared similar. A second aspect of the invention provides a method of matching a query object to a known object, wherein a plurality of candidate fingeφrints representing a plurality of candidate objects are pre-stored in a database, the method comprising receiving an information signal forming part of the query object and constructing a query fingeφrint therefrom and comparing the query fingeφrint to a candidate fingeφrint in the database, the method being characterised in that it further comprises the steps of: determining a statistical model for the query fingerprint and/or the candidate fingerprint; and on the basis of the statistical model, deriving a threshold distance within which the query fingeφrint and the candidate fingeφrint may be declared similar. In the methods of the first and second aspects, the derivation of a threshold based upon a statistical model of the particular fingeφrint provides adaptive threshold setting which may optimise the F.A.R. according to query fingerprint type/ internal characteristics giving improved matching qualities over the application of an arbitrary thresholding system. Preferably, if a candidate fingeφrint is found to be separated from the query fingerprint by a distance less than the threshold distance, and the distance between the candidate and the query fingeφrint is less than the distance between any other candidate fingerprint and the query fingeφrint, then the candidate fingeφrint is declared the best matching candidate fingeφrint and the candidate object represented by the best matching candidate fingerprint and the query object represented by the query fingeφrint are deemed to be the same. Preferably, the statistical model comprises the result of performing an internal correlation on the query fingeφrint and/or the candidate fingeφrint. Preferably, the fingeφrints comprise binary values and the statistical model is computed for the query fingeφrint by determining a transition probability q for the query fingerprint by determining how many bits of a query fingerprint frame F(m,k) are different
from their corresponding bit in their preceding fingeφrint frame F(m,k-1) and dividing the number of transitions by a maximum value M*(k-1), which would be obtained if all fingeφrint bits were of an opposite state to their corresponding preceding bit, where each fingeφrint comprises M bits per frame and spans K frames, in which k is the frame index (ranging from 0 to K) and m is the bit-index within a frame (ranging from 0 to M). The threshold distance T may then be computed from the following equation based on a desired False Acceptance Rate (FAR):
In a third aspect, the invention provides apparatus for matching a query object to a known object, the apparatus comprising a fingeφrint extaction module for receiving an information signal forming part of a query object and constructing a query fingeφrint therefrom and a fingeφrint matching module for comparing the query fingeφrint to candidate fingerprints stored in a database to one or more candidate fingeφrints, the apparatus being characterised in that it further comprises: a statistical module for determining a statistical model of the query fϊngeφrint and/or one or more of the one or more candidate fingeφrints; a threshold determiner ,deriving on the basis of the statistical model, a threshold distance T within which the query fingeφrint and a candidate fingerprint may be declared similar; and an identification module arranged such that if a candidate fingeφrint is found to . be separated from the query fingeφrint by a distance less than the threshold distance T, and the distance between the candidate and the query fingeφrint is less than the distance between any other candidate fingerprint and the query fingeφrint, then the candidate fingerprint is declared the best matching candidate fingeφrint and the candidate object represented by the best matching candidate fingeφrint and the query object represented by the query fingeφrint are deemed to be the same.
BRIEF DESCRIPTION OF THE DRAWINGS For a better understanding of the invention, and to show how embodiments of the same may be carried into effect, reference will now be made, by way of example, to the accompanying diagrammatic drawings in which: Figure 1 shows a functional block diagram illustrating a fingeφrinting method with an adaptive threshold in accordance with an embodiment of the invention;
Figure 2 is a flow diagram explaining in general the process involved in finding and matching fingeφrints in accordance with an embodiment of the invention; Figure 3 is a flow diagram illustrating in general the methodology for determining an adaptive threshold in accordance with an embodiment of the present invention; and Figure 4 is a flow diagram illustrating a specific adaptive threshold setting methodology in accordance with embodiments of the invention.
DESCRIPTION OF EMBODIMENTS Referring to Figure 1, there is shown a functional block diagram divided into a client side 100 and a database server side 200. At the client side, an object is received by a fingeφrint extraction module 110 and a query fingerprint F computed for the object. The query fϊngeφrint F is, on the one hand, passed to an statistical module 120 and, on the other hand, also passed to the database server side 200. The statistical module 120 determines a measure of randomness/correlation (for instance, it may determine the internal correlation) of the query fingerprint F and passes this information to a threshold determiner 130. The threshold determiner 130, on the basis of the information from the module 120 adaptively sets a threshold level T and passes this threshold level T to the database server side 200. At the database server side 200, a matching module 210 receives the query fingeφrint F from the client side 100 and looks for the best match of that fingeφrint within a database of known fingeφrints. The best match information is then passed to a threshold comparison module 220 to determine whether a best matching candidate fingerprint is close enough (within threshold distance T) to the query fingeφrint to determine the identity of the input object with the matched object corresponding to the candidate fingeφrint. In the case where the fingeφrint F takes binary values, the threshold comparison module 220 might, for instance, compare the Hamming distance between a fingeφrint block Hi and a fingeφrint block H2 relating to the best match in the database 210 and check to see whether the Hamming distance between the two blocks is below the threshold distance T, supplied to the comparison module 220 from the threshold determining module 130. An identification decision is made by identification module 230 so that if the Hamming distance between the two derived fingeφrint blocks is below the threshold distance T then the unidentified query object is declared similar to the object found in the database and the relevant metadata is returned.
In the above description the query fingeφrint F and the threshold T are sent by the client side 100 to the database server side 200. Here, of course, it could be noted that the threshold T could also be determined at the database server side 200 and that, therefore, modifications of the aforementioned block diagram are of course possible. Referring now to Figure 2, there is shown a flow diagram which explains, in general, the operation of the components of the block diagram of Figure 1 in finding and matching fingeφrints. In a step S100, an object sample ( e.g. in the case of video a short "clip") is received and a query fingerprint deteπnined based upon the sample. This query fingeφrint may be determined in accordance with any suitable prior art method (such as disclosed in US 2002/0178410 Al). In a step S200 (reached by pathway "A"), a threshold for the query fingeφrint is determined in accordance with the particular characteristics (randomness/correlation) of the query fingeφrint. In a step S300, which may be carried out in parallel with step S200, the query fingeφrint is matched to fingeφrints held on the database server side 200, to return a best matching candidate. Again, this matching process may be performed conventionally, so as to return the closest match to the query fingeφrint. In the step S300, the "distance" between the query fingeφrint and the best match candidate will be determined and, in a step S400, it is checked whether or not the "distance" is less than the threshold distance determined in step S200. If the distance between the query fingeφrint and the best match candidate is found in step S400 to be greater than the threshold, then in step S500 the result is returned that no matching object to the query object has been found. On the other hand, if the distance between query fingerprint and best match candidate fingerprint is less than the threshold distance in step S400, then in step S600 a match is declared between the query object and the object in the database relating to the best matching candidate. Metadata etc., of the best matching object may then be returned to a user. In Figure 2, the pathway "A" denoted by the broken lines leading to step S200 from SI 00 denote one option for setting a threshold T=T1 based on the query fingerprint. Alternatively however, pathway "A" may be disregarded and a threshold T=T2 may be based upon the characteristics of the best matching candidate. This possibility is denoted by the alternative pathway B from S300 to S200.
In a further alternative, the threshold T may be set based upon a combination of the characteristics of both the query fingeφrint and the best matching candidate fingeφrint e.g. by setting a threshold at the average between two derived adaptive thresholds TI, T2. Figure 3 is a flow diagram illustrating the general methodology for adaptively determining a given threshold T. In step S210, the query candidate fingeφrint is received and a measure of randomness of the fingeφrint determined, then in step S220 a threshold distance is set according to the measure of randomness found in step S210. As will be appreciated from the above and from the explanation in relation to Figure 1, the threshold value T (TI or T2) used in the comparison is adapted to the randomness/correlation in either the query-fingeφrint or/and the best matching candidate. More specifically, in the case of threshold determination for a query fϊngeφrint, the correlation of the query fingeφrint is determined and, from this correlation, the threshold to be used during matching is computed. The less random the internal correlation is found to be, the smaller the threshold distance T can be set without adversely affecting the FRR. As stated, the threshold is determined upon the internal correlation of the query fingerprint, a best matching candidate fingeφrint or a combination of the two. In cases where the fingeφrint is binary and the fingeφrint-bits behave like a Markov-process, a solution can be derived for adaptively setting the threshold. The solution to the adaptive threshold setting problem is shown in Figure 4. In a step S221, the internal correlation of the fingeφrint in question is determined, in step S222 the transition probability for the fingeφrint is determined based upon the internal correlation and in step S223, the threshold distance is set adaptively, based upon both the transition probability (explained below) and a desired false acceptance rate. Let the fingerprint consist of M bits per frame and span K frames. In this case, the fingerprint can be denoted F(m,k), where k is the frame index (ranging from 0 to K-l) and m is the bit-index within a frame (ranging from 0 to M-l). Let q denote the probability that a fingeφrint-bit extracted from frame k is unequal to the corresponding fingerprint bit from frame k-l by (q=Prob[bit(m,k) ≠ bit(m,k-l) ]). This probability q is called the transition probability. In this case the correlation increases (compared to the case of purely random bits, in which q=l/2) by a factor
As a consequence, the False Acceptance Rate FAR is described by the relation
Use of the above relation for computing an adaptive threshold from the desired FAR and the computed transition probability q may be summarised as follows: Extract fingeφrint F Determine the transition probability q for fingeφrint F, as follows:
(a) Determine how many of the fingeφrint bits F(m,k) are different from their predecessor F(m,k-1).
(b) Divide the number of transitions, as computed in step (a) by the theoretical maximum M*(K-1), which would be obtained if for each frame, all fingerprint bits would be the opposite from the bits in the previous frame to determine the transition probability q = (number ojbit-transitions)/(M*(K-l)). Determine the threshold T which is to be used for matching this specific query fingeφrint F from the computed value q, and a defined pre-agreed False Acceptance Rate using relation (4). From the above, the threshold T may be adaptively set for T=T1 (based on correlation of query fingerprint above), or T=T2 (based on correlation of best match (7T + TI) fingeφrint above), or T=T3 (based on a combination of TI, T2 [e.g.T = ]. Then, in the decision stage if the Hamming distance is less than T, declare the underlying objects to be the same. In the above specific examples of the present invention the threshold distance is set adaptively based on the internal characteristics of a particular query sample or, indeed, of a particular candidate sample or set of samples. However, whilst the specific examples described take the internal characteristics in question to be randomness/correlation, it will be realised that other types of statistical distribution might apply to certain types of information signal and that, therefore, the invention may be legitimately extended to providing adaptive thresholds according to any given applicable "statistical model" to which a query sample or a candidate sample fingerprint is expected to conform. Further, the skilled man will realise that whilst the Figure 2 through 4 flow diagrams show one arrangement for implementing the invention, other arrangements are possible. For instance, rather than returning a single best match candidate in step S300 of figure 2, a plurality of close matching candidates within a threshold distance may be returned
and processed in parallel (or less advantageously in series) to thereafter calculate the "best" match. The invention can also be applied using so-called "pruning" techniques in which certain candidates within the database can be immediately discarded if it is obvious that they can never make a match - searching/matching can then be done within a much reduced search space. In accordance with embodiments of the invention, methods and apparatus for setting an adaptive threshold are disclosed, in which the threshold depends upon specific characteristics of a fingerprint. The particular method is very suitable for use in matching of video content, but is not limited to this. The techniques described may be applied to various different areas of technology and various different signal types, including, but not limited to, audio signals, video signals, multimedia signals. The skilled man will realise that the processes described may be implemented in software, hardware, or any suitable combination. In summary, the invention relates to methods and apparatus for fingeφrint matching. In an embodiment of the invention apparatus comprising a fingeφrint extraction module (110), a fingeφrint matching module (210), a statistical module (120) and an identification module is provided. The fingeφrint extraction module (110) receives an information signal forming part of a query object and constructs a query fingeφrint. The fingerprint matching module (210) compares the query fingeφrint to candidates stored in a database (215) to find at least one potentially best matching candidate. Meanwhile, the statistical module determines a statistical model of the query fingeφrint so as to, for instance, determine the statistical distribution of the query fmgeφrint. The threshold determiner (120) is arranged, on the basis of the distribution of the query fingeφrint to derive an adaptive threshold distance T within which the query fingeφrint and a potentially best matching candidate may be declared similar by the identification module (130). By setting a threshold in an adaptive manner according to the statistical distribution of the query fingeφrint, an improved false acceptance rate F.A.R. and other advantages may be achieved.
Claims
1. A method of comparing a query fingeφrint to a candidate fingeφrint, the method being characterised by comprising: deteπnining a statistical model of the query fingeφrint and/or a candidate fingeφrint and, on the basis of the statistical model, deriving a threshold distance within which the query fingeφrint and the candidate fingeφrint may be declared similar.
2. A method of matching a query object to a known object, wherein a plurality of candidate fingeφrints representing a plurality of candidate objects are pre-stored in a database, the method comprising receiving an information signal forming part of the query object and constructing a query fingeφrint therefrom and comparing the query fingeφrint to a candidate fingeφrint in the database, the method being characterised by the further steps of: determining a statistical model for the query fingeφrint and/or the candidate fingerprint; and on the basis of the statistical model, deriving a threshold distance within which the query fingeφrint and the candidate fingerprint may be declared similar.
3. The method of claim 1 or 2, wherein if a candidate fingeφrint is found to be separated from the query fingerprint by a distance less than the threshold distance, and the distance between the candidate and the query fingerprint is less than the distance between any other candidate fingeφrint and the query fingeφrint, then the candidate fingeφrint is declared the best matching candidate fingeφrint and the candidate object represented by the best matching candidate fingeφrint and the query object represented by the query fingerprint are deemed to be the same.
4. The method of claim 1, 2 or 3, wherein the statistical model comprises the result of performing an internal correlation on the query fingerprint and/or the candidate fingerprint.
5. The method of claim 4, wherein the fingeφrints comprise a plurality of frames containing binary values and the statistical model is computed for the query fingeφrint by determining a transition probability q for the query fingeφrint by determining how many bits of a frame of the query fingeφrint F(m,k) are different from their corresponding bit in their preceding fingerprint frame F(m,k-1) and dividing the number of transitions by a maximum value M*(k-1), which would be obtained if all fingerprint bits were of an opposite state to their corresponding preceding bit, where each fingerprint comprises M bits per frame and spans K frames, in which k is the frame index (ranging from 0 to K) and m is the bit-index within a frame (ranging from 0 to M).
6. The method of claim 5, wherein the threshold distance T is computed from the following equation based on a desired False Acceptance Rate (FAR):
7. Apparatus for matching a query object to a known object, the apparatus comprising a fingeφrint extraction module (110) for receiving an information signal forming part of a query object and constructing a query fingeφrint therefrom and a fingerprint matching module (210) for comparing the query fingeφrint to candidate fingerprints stored in a database (215) to one or more candidate fingeφrints, the apparatus being characterised by also comprising: a statistical module (120) for determining a statistical model of the query fingeφrint and/or one or more of the one or more candidate fingeφrints; a threshold determiner (120) deriving, on the basis of the statistical model, a threshold distance T within which the query fingeφrint and a potentially best matching candidate fingeφrint may be declared similar; and an identification module (230) arranged such that if a candidate fingeφrint is found to be separated from the query fingerprint by a distance less than the threshold distance T, and the distance between the candidate and the query fingeφrint is less than the distance between any other candidate fingeφrint and the query fingerprint, then the candidate fingeφrint is declared the best matching candidate fingerprint and the candidate object represented by the best matching candidate fingerprint and the query object represented by the query fingerprint are deemed to be the same.
8. The apparatus of claim 7, wherein the statistical module (120) performs an internal correlation on the query fingeφrint and/or the one or more candidate fingerprints.
9. The method of claim 8, wherein the fingeφrints comprise a plurality of frames containing binary values and the statistical module (120) computes the statistical model for the query fingeφrint or/and the candidate fingeφrint by determining a transition probability q by determining how many bits of a frame of the query fingeφrint F(m,k) are different from their corresponding bit in the preceding fingeφrint frame F(m,k-1) and dividing the number of transitions by a maximum value M*(k-1), which would be obtained if all fingeφrint bits were of an opposite state to their corresponding preceding bit, where each fϊngeφrint comprises M bits per frame and spans K frames, in which k is the frame index (ranging from 0 to K) and m is the bit-index within a frame (ranging from 0 to M).
10. The method of claim 9, wherein the threshold determiner (130) computes the threshold distance T from the following equation based on a desired False Acceptance Rate (FAR):
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP04799078A EP1687806A1 (en) | 2003-11-18 | 2004-11-08 | Matching data objects by matching derived fingerprints |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP03104250 | 2003-11-18 | ||
EP04799078A EP1687806A1 (en) | 2003-11-18 | 2004-11-08 | Matching data objects by matching derived fingerprints |
PCT/IB2004/052334 WO2005050620A1 (en) | 2003-11-18 | 2004-11-08 | Matching data objects by matching derived fingerprints |
Publications (1)
Publication Number | Publication Date |
---|---|
EP1687806A1 true EP1687806A1 (en) | 2006-08-09 |
Family
ID=34610093
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP04799078A Ceased EP1687806A1 (en) | 2003-11-18 | 2004-11-08 | Matching data objects by matching derived fingerprints |
Country Status (6)
Country | Link |
---|---|
US (1) | US20070071330A1 (en) |
EP (1) | EP1687806A1 (en) |
JP (1) | JP2007519986A (en) |
KR (1) | KR20060118493A (en) |
CN (1) | CN1882984A (en) |
WO (1) | WO2005050620A1 (en) |
Families Citing this family (103)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
ES2414650T3 (en) * | 2000-08-23 | 2013-07-22 | Gracenote, Inc. | Procedure and system for obtaining information |
US8205237B2 (en) | 2000-09-14 | 2012-06-19 | Cox Ingemar J | Identifying works, using a sub-linear time search, such as an approximate nearest neighbor search, for initiating a work-based action, such as an action on the internet |
DE60228202D1 (en) * | 2001-02-12 | 2008-09-25 | Gracenote Inc | METHOD FOR GENERATING AN IDENTIFICATION HASH FROM THE CONTENTS OF A MULTIMEDIA FILE |
JP2005517211A (en) * | 2002-02-05 | 2005-06-09 | コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ | Efficient storage of fingerprints |
ATE426297T1 (en) * | 2002-09-30 | 2009-04-15 | Gracenote Inc | FINGERPRINT EXTRACTION |
CN1708758A (en) * | 2002-11-01 | 2005-12-14 | 皇家飞利浦电子股份有限公司 | Improved audio data fingerprint searching |
WO2004044820A1 (en) * | 2002-11-12 | 2004-05-27 | Koninklijke Philips Electronics N.V. | Fingerprinting multimedia contents |
US8719779B2 (en) * | 2004-12-28 | 2014-05-06 | Sap Ag | Data object association based on graph theory techniques |
US20070106405A1 (en) * | 2005-08-19 | 2007-05-10 | Gracenote, Inc. | Method and system to provide reference data for identification of digital content |
US11216498B2 (en) | 2005-10-26 | 2022-01-04 | Cortica, Ltd. | System and method for generating signatures to three-dimensional multimedia data elements |
US8326775B2 (en) | 2005-10-26 | 2012-12-04 | Cortica Ltd. | Signature generation for multimedia deep-content-classification by a large-scale matching system and method thereof |
US11403336B2 (en) | 2005-10-26 | 2022-08-02 | Cortica Ltd. | System and method for removing contextually identical multimedia content elements |
US11003706B2 (en) | 2005-10-26 | 2021-05-11 | Cortica Ltd | System and methods for determining access permissions on personalized clusters of multimedia content elements |
US11019161B2 (en) | 2005-10-26 | 2021-05-25 | Cortica, Ltd. | System and method for profiling users interest based on multimedia content analysis |
US10360253B2 (en) | 2005-10-26 | 2019-07-23 | Cortica, Ltd. | Systems and methods for generation of searchable structures respective of multimedia data content |
US10607355B2 (en) | 2005-10-26 | 2020-03-31 | Cortica, Ltd. | Method and system for determining the dimensions of an object shown in a multimedia content item |
US11361014B2 (en) | 2005-10-26 | 2022-06-14 | Cortica Ltd. | System and method for completing a user profile |
US10585934B2 (en) | 2005-10-26 | 2020-03-10 | Cortica Ltd. | Method and system for populating a concept database with respect to user identifiers |
US9953032B2 (en) | 2005-10-26 | 2018-04-24 | Cortica, Ltd. | System and method for characterization of multimedia content signals using cores of a natural liquid architecture system |
US9558449B2 (en) | 2005-10-26 | 2017-01-31 | Cortica, Ltd. | System and method for identifying a target area in a multimedia content element |
US9489431B2 (en) | 2005-10-26 | 2016-11-08 | Cortica, Ltd. | System and method for distributed search-by-content |
US10635640B2 (en) | 2005-10-26 | 2020-04-28 | Cortica, Ltd. | System and method for enriching a concept database |
US9477658B2 (en) | 2005-10-26 | 2016-10-25 | Cortica, Ltd. | Systems and method for speech to speech translation using cores of a natural liquid architecture system |
US10380623B2 (en) | 2005-10-26 | 2019-08-13 | Cortica, Ltd. | System and method for generating an advertisement effectiveness performance score |
US10776585B2 (en) | 2005-10-26 | 2020-09-15 | Cortica, Ltd. | System and method for recognizing characters in multimedia content |
US10621988B2 (en) | 2005-10-26 | 2020-04-14 | Cortica Ltd | System and method for speech to text translation using cores of a natural liquid architecture system |
US9639532B2 (en) | 2005-10-26 | 2017-05-02 | Cortica, Ltd. | Context-based analysis of multimedia content items using signatures of multimedia elements and matching concepts |
US10848590B2 (en) | 2005-10-26 | 2020-11-24 | Cortica Ltd | System and method for determining a contextual insight and providing recommendations based thereon |
US9031999B2 (en) | 2005-10-26 | 2015-05-12 | Cortica, Ltd. | System and methods for generation of a concept based database |
US10372746B2 (en) | 2005-10-26 | 2019-08-06 | Cortica, Ltd. | System and method for searching applications using multimedia content elements |
US9767143B2 (en) | 2005-10-26 | 2017-09-19 | Cortica, Ltd. | System and method for caching of concept structures |
US9191626B2 (en) | 2005-10-26 | 2015-11-17 | Cortica, Ltd. | System and methods thereof for visual analysis of an image on a web-page and matching an advertisement thereto |
US10691642B2 (en) | 2005-10-26 | 2020-06-23 | Cortica Ltd | System and method for enriching a concept database with homogenous concepts |
US10380164B2 (en) | 2005-10-26 | 2019-08-13 | Cortica, Ltd. | System and method for using on-image gestures and multimedia content elements as search queries |
US8818916B2 (en) | 2005-10-26 | 2014-08-26 | Cortica, Ltd. | System and method for linking multimedia data elements to web pages |
US9646005B2 (en) | 2005-10-26 | 2017-05-09 | Cortica, Ltd. | System and method for creating a database of multimedia content elements assigned to users |
US10193990B2 (en) | 2005-10-26 | 2019-01-29 | Cortica Ltd. | System and method for creating user profiles based on multimedia content |
US10698939B2 (en) | 2005-10-26 | 2020-06-30 | Cortica Ltd | System and method for customizing images |
US9218606B2 (en) | 2005-10-26 | 2015-12-22 | Cortica, Ltd. | System and method for brand monitoring and trend analysis based on deep-content-classification |
US11604847B2 (en) | 2005-10-26 | 2023-03-14 | Cortica Ltd. | System and method for overlaying content on a multimedia content element based on user interest |
US10535192B2 (en) | 2005-10-26 | 2020-01-14 | Cortica Ltd. | System and method for generating a customized augmented reality environment to a user |
US11386139B2 (en) | 2005-10-26 | 2022-07-12 | Cortica Ltd. | System and method for generating analytics for entities depicted in multimedia content |
US10380267B2 (en) | 2005-10-26 | 2019-08-13 | Cortica, Ltd. | System and method for tagging multimedia content elements |
US10614626B2 (en) | 2005-10-26 | 2020-04-07 | Cortica Ltd. | System and method for providing augmented reality challenges |
US9529984B2 (en) | 2005-10-26 | 2016-12-27 | Cortica, Ltd. | System and method for verification of user identification based on multimedia content elements |
US11620327B2 (en) | 2005-10-26 | 2023-04-04 | Cortica Ltd | System and method for determining a contextual insight and generating an interface with recommendations based thereon |
US10180942B2 (en) | 2005-10-26 | 2019-01-15 | Cortica Ltd. | System and method for generation of concept structures based on sub-concepts |
US9384196B2 (en) | 2005-10-26 | 2016-07-05 | Cortica, Ltd. | Signature generation for multimedia deep-content-classification by a large-scale matching system and method thereof |
US9466068B2 (en) | 2005-10-26 | 2016-10-11 | Cortica, Ltd. | System and method for determining a pupillary response to a multimedia data element |
US8312031B2 (en) | 2005-10-26 | 2012-11-13 | Cortica Ltd. | System and method for generation of complex signatures for multimedia data content |
US10949773B2 (en) | 2005-10-26 | 2021-03-16 | Cortica, Ltd. | System and methods thereof for recommending tags for multimedia content elements based on context |
US11032017B2 (en) | 2005-10-26 | 2021-06-08 | Cortica, Ltd. | System and method for identifying the context of multimedia content elements |
US10191976B2 (en) | 2005-10-26 | 2019-01-29 | Cortica, Ltd. | System and method of detecting common patterns within unstructured data elements retrieved from big data sources |
US10742340B2 (en) | 2005-10-26 | 2020-08-11 | Cortica Ltd. | System and method for identifying the context of multimedia content elements displayed in a web-page and providing contextual filters respective thereto |
US10387914B2 (en) | 2005-10-26 | 2019-08-20 | Cortica, Ltd. | Method for identification of multimedia content elements and adding advertising content respective thereof |
US9372940B2 (en) | 2005-10-26 | 2016-06-21 | Cortica, Ltd. | Apparatus and method for determining user attention using a deep-content-classification (DCC) system |
US8266185B2 (en) | 2005-10-26 | 2012-09-11 | Cortica Ltd. | System and methods thereof for generation of searchable structures respective of multimedia data content |
WO2007053112A1 (en) * | 2005-11-07 | 2007-05-10 | Agency For Science, Technology And Research | Repeat clip identification in video data |
JP2007171772A (en) * | 2005-12-26 | 2007-07-05 | Clarion Co Ltd | Music information processing device, music information processing method, and control program |
WO2007144813A2 (en) | 2006-06-13 | 2007-12-21 | Koninklijke Philips Electronics N.V. | Fingerprint, apparatus, method for identifying and synchronizing video |
JP2009541908A (en) | 2006-06-23 | 2009-11-26 | コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ | Method for navigating items in a media player |
US10733326B2 (en) | 2006-10-26 | 2020-08-04 | Cortica Ltd. | System and method for identification of inappropriate multimedia content |
US20080274687A1 (en) * | 2007-05-02 | 2008-11-06 | Roberts Dale T | Dynamic mixed media package |
US8266142B2 (en) * | 2007-06-06 | 2012-09-11 | Dolby Laboratories Licensing Corporation | Audio/Video fingerprint search accuracy using multiple search combining |
US8275681B2 (en) | 2007-06-12 | 2012-09-25 | Media Forum, Inc. | Desktop extension for readily-sharable and accessible media playlist and media |
US8447032B1 (en) | 2007-08-22 | 2013-05-21 | Google Inc. | Generation of min-hash signatures |
US8238669B2 (en) * | 2007-08-22 | 2012-08-07 | Google Inc. | Detection and classification of matches between time-based media |
US8437555B2 (en) * | 2007-08-27 | 2013-05-07 | Yuvad Technologies, Inc. | Method for identifying motion video content |
WO2009087511A1 (en) * | 2008-01-04 | 2009-07-16 | Koninklijke Philips Electronics N.V. | A method and a system for identifying elementary content portions from an edited content |
TWI506565B (en) | 2008-03-03 | 2015-11-01 | Avo Usa Holding 2 Corp | Dynamic object classification |
US8611701B2 (en) * | 2008-05-21 | 2013-12-17 | Yuvad Technologies Co., Ltd. | System for facilitating the search of video content |
US8488835B2 (en) * | 2008-05-21 | 2013-07-16 | Yuvad Technologies Co., Ltd. | System for extracting a fingerprint data from video/audio signals |
WO2009140817A1 (en) * | 2008-05-21 | 2009-11-26 | Yuvad Technologies Co., Ltd. | A method for facilitating the search of video content |
WO2009140818A1 (en) * | 2008-05-21 | 2009-11-26 | Yuvad Technologies Co., Ltd. | A system for facilitating the archiving of video content |
WO2009140822A1 (en) * | 2008-05-22 | 2009-11-26 | Yuvad Technologies Co., Ltd. | A method for extracting a fingerprint data from video/audio signals |
WO2009140824A1 (en) * | 2008-05-22 | 2009-11-26 | Yuvad Technologies Co., Ltd. | A system for identifying motion video/audio content |
WO2009143667A1 (en) * | 2008-05-26 | 2009-12-03 | Yuvad Technologies Co., Ltd. | A system for automatically monitoring viewing activities of television signals |
US8335786B2 (en) * | 2009-05-28 | 2012-12-18 | Zeitera, Llc | Multi-media content identification using multi-level content signature correlation and fast similarity search |
US8195689B2 (en) | 2009-06-10 | 2012-06-05 | Zeitera, Llc | Media fingerprinting and identification system |
US10334324B2 (en) | 2008-11-26 | 2019-06-25 | Free Stream Media Corp. | Relevant advertisement generation based on a user operating a client device communicatively coupled with a networked media device |
US8180891B1 (en) | 2008-11-26 | 2012-05-15 | Free Stream Media Corp. | Discovery, access control, and communication with networked services from within a security sandbox |
US9961388B2 (en) | 2008-11-26 | 2018-05-01 | David Harrison | Exposure of public internet protocol addresses in an advertising exchange server to improve relevancy of advertisements |
US10567823B2 (en) | 2008-11-26 | 2020-02-18 | Free Stream Media Corp. | Relevant advertisement generation based on a user operating a client device communicatively coupled with a networked media device |
US10631068B2 (en) | 2008-11-26 | 2020-04-21 | Free Stream Media Corp. | Content exposure attribution based on renderings of related content across multiple devices |
US10977693B2 (en) | 2008-11-26 | 2021-04-13 | Free Stream Media Corp. | Association of content identifier of audio-visual data with additional data through capture infrastructure |
US10419541B2 (en) | 2008-11-26 | 2019-09-17 | Free Stream Media Corp. | Remotely control devices over a network without authentication or registration |
US9986279B2 (en) | 2008-11-26 | 2018-05-29 | Free Stream Media Corp. | Discovery, access control, and communication with networked services |
US10880340B2 (en) | 2008-11-26 | 2020-12-29 | Free Stream Media Corp. | Relevancy improvement through targeting of information based on data gathered from a networked device associated with a security sandbox of a client device |
US9519772B2 (en) | 2008-11-26 | 2016-12-13 | Free Stream Media Corp. | Relevancy improvement through targeting of information based on data gathered from a networked device associated with a security sandbox of a client device |
US9154942B2 (en) | 2008-11-26 | 2015-10-06 | Free Stream Media Corp. | Zero configuration communication between a browser and a networked media device |
CN102411578A (en) * | 2010-09-25 | 2012-04-11 | 盛乐信息技术(上海)有限公司 | Multimedia playing system and method |
JP4999981B2 (en) * | 2010-12-20 | 2012-08-15 | 株式会社エヌ・ティ・ティ・ドコモ | Information reception notification device and information reception notification method |
CN102413007B (en) * | 2011-10-12 | 2014-03-26 | 上海奇微通讯技术有限公司 | Deep packet inspection method and equipment |
CN103093761B (en) * | 2011-11-01 | 2017-02-01 | 深圳市世纪光速信息技术有限公司 | Audio fingerprint retrieval method and retrieval device |
KR101315970B1 (en) * | 2012-05-23 | 2013-10-08 | (주)엔써즈 | Apparatus and method for recognizing content using audio signal |
US9141676B2 (en) * | 2013-12-02 | 2015-09-22 | Rakuten Usa, Inc. | Systems and methods of modeling object networks |
US9986280B2 (en) * | 2015-04-11 | 2018-05-29 | Google Llc | Identifying reference content that includes third party content |
CN106407226B (en) * | 2015-07-31 | 2019-09-13 | 华为技术有限公司 | A kind of data processing method, backup server and storage system |
CN106910494B (en) | 2016-06-28 | 2020-11-13 | 创新先进技术有限公司 | Audio identification method and device |
CN106446802A (en) * | 2016-09-07 | 2017-02-22 | 深圳市金立通信设备有限公司 | Fingerprint identification method and terminal |
US10771828B2 (en) | 2018-09-18 | 2020-09-08 | Free Stream Media Corp. | Content consensus management |
US20210056085A1 (en) * | 2019-08-19 | 2021-02-25 | Gsi Technology Inc. | Deduplication of data via associative similarity search |
US11417099B1 (en) * | 2021-11-08 | 2022-08-16 | 9219-1568 Quebec Inc. | System and method for digital fingerprinting of media content |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7013301B2 (en) * | 2003-09-23 | 2006-03-14 | Predixis Corporation | Audio fingerprinting system and method |
DE60228202D1 (en) * | 2001-02-12 | 2008-09-25 | Gracenote Inc | METHOD FOR GENERATING AN IDENTIFICATION HASH FROM THE CONTENTS OF A MULTIMEDIA FILE |
WO2003009277A2 (en) * | 2001-07-20 | 2003-01-30 | Gracenote, Inc. | Automatic identification of sound recordings |
US7142699B2 (en) * | 2001-12-14 | 2006-11-28 | Siemens Corporate Research, Inc. | Fingerprint matching using ridge feature maps |
-
2004
- 2004-11-08 CN CNA200480033941XA patent/CN1882984A/en active Pending
- 2004-11-08 KR KR1020067009641A patent/KR20060118493A/en not_active Application Discontinuation
- 2004-11-08 EP EP04799078A patent/EP1687806A1/en not_active Ceased
- 2004-11-08 US US10/579,412 patent/US20070071330A1/en not_active Abandoned
- 2004-11-08 WO PCT/IB2004/052334 patent/WO2005050620A1/en not_active Application Discontinuation
- 2004-11-08 JP JP2006540687A patent/JP2007519986A/en not_active Withdrawn
Non-Patent Citations (3)
Title |
---|
DUDA R.O.; HART P.E.; STORK D.G.: "PATTERN CLASSIFICATION", 2001, JOHN WILEY & SONS, US, NEW YORK, US, ISBN: 0-471-05669-3, 276860 * |
JOHN DAUGHMAN: "How Iris Recognition Works", IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, vol. 14, no. 1, January 2004 (2004-01-01), US, pages 21 - 30, XP011105913 * |
See also references of WO2005050620A1 * |
Also Published As
Publication number | Publication date |
---|---|
US20070071330A1 (en) | 2007-03-29 |
JP2007519986A (en) | 2007-07-19 |
WO2005050620A1 (en) | 2005-06-02 |
KR20060118493A (en) | 2006-11-23 |
CN1882984A (en) | 2006-12-20 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2005050620A1 (en) | Matching data objects by matching derived fingerprints | |
US11294955B2 (en) | System and method for optimization of audio fingerprint search | |
US7260439B2 (en) | Systems and methods for the automatic extraction of audio excerpts | |
EP2685450B1 (en) | Device and method for recognizing content using audio signals | |
EP2323046A1 (en) | Method for detecting audio and video copy in multimedia streams | |
US7477739B2 (en) | Efficient storage of fingerprints | |
CN108881947B (en) | Method and device for detecting infringement of live stream | |
JP2004519015A (en) | Generating and matching hashes of multimedia content | |
KR20050061594A (en) | Improved audio data fingerprint searching | |
WO2003096337A2 (en) | Watermark embedding and retrieval | |
WO2002011123A2 (en) | Method for search in an audio database | |
Saracoglu et al. | Content based copy detection with coarse audio-visual fingerprints | |
WO2005122141A1 (en) | Effective audio segmentation and classification | |
KR101841985B1 (en) | Method and Apparatus for Extracting Audio Fingerprint | |
CN107204183B (en) | Audio file detection method and device | |
CN114598933B (en) | Video content processing method, system, terminal and storage medium | |
KR102334018B1 (en) | Apparatus and method for validating self-propagated unethical text | |
US20070101354A1 (en) | Method and device for discriminating obscene video using time-based feature value | |
Haitsma et al. | An efficient database search strategy for audio fingerprinting | |
CN109543511B (en) | Video identification method, system and device based on pattern mutation frame and feature calculation | |
CN111291224A (en) | Video stream data processing method, device, server and storage medium | |
Mapelli et al. | Robust audio fingerprinting for song identification | |
Bauer et al. | Optimal configuration of hash table based multimedia fingerprint databases using weak bits | |
CN109524026B (en) | Method and device for determining prompt tone, storage medium and electronic device | |
CN115495728A (en) | Unsupervised unknown attack detection method and system based on automatic encoder |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20060619 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LU MC NL PL PT RO SE SI SK TR |
|
17Q | First examination report despatched |
Effective date: 20060904 |
|
DAX | Request for extension of the european patent (deleted) | ||
17Q | First examination report despatched |
Effective date: 20060904 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION HAS BEEN REFUSED |
|
18R | Application refused |
Effective date: 20081022 |