KR101507709B1 - fingerprinting methods for providing additional information service related in broadcasting - Google Patents

Abstract

The present invention relates to audio fingerprinting. According to the present invention, characteristic values of audio are stored in a DB in advance; later, the audio is recognized as contents by searching a relevant portion of the audio in the DB if the audio to be recognized has been stored; and a replay time point can be identified. Two or more characteristics values (DNAs) are used to enable real-time broadcasting recognition service and large-scale VOD recognition service to be provided. It is difficult to apply the contents recognition services stated earlier on a practical system with one DNA. However, when two or more DNAs are used, the contents recognition services with increased recognition performance can be more accurately provided.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a fingerprinting method for providing additional information related to a broadcast program,

The present invention relates to an audio fingerprinting method in which audio feature points are stored in a database (DB) in advance and audio portions to be recognized later are already stored in the DB, the contents are recognized by DB search, .

Audio fingerprinting is a method of storing audio feature points in a database (DB) in advance and recognizing the contents by DB search if the audio to be recognized later is already stored in the DB . A method of extracting a feature point (DNA) of an audio content and recognizing a broadcast signal has been developed and used. Herein, the term " DNA " refers to a characteristic value that enables a content to be distinguished from other contents.

The fingerprinting algorithm is implemented using a unique feature point (DNA) of the content. When a feature point is used, a satisfactory performance is often not obtained. In other words, arbitrary DNA A may be characterized by a small amount of DNA in a noisy environment, which quickly distinguishes it roughly, but it is difficult to accurately discriminate the contents. Another arbitrary DNA, B, can discriminate the content accurately, but there may be a DNA having a disadvantage that a large amount of DNA must be used. There is a problem that it is difficult to apply such DNA to any application such as a large-capacity VOD service.

The present invention relates to a method for enabling actual broadcast recognition and large capacity VOD recognition service using two or more DNAs. Although it is difficult to apply each DNA as a practical system, it is intended to make the content recognition service more accurate and improve the recognition performance by using two or more DNAs.

BRIEF DESCRIPTION OF THE DRAWINGS The above and other features and advantages of the present invention will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings. This summary is not an extensive overview of all contemplated aspects, nor is it intended to identify key or critical elements of all aspects nor to precisely describe the scope of any or all aspects. Its sole purpose is to present some concepts of the aspects in a simplified form as a prelude to a more detailed description presented later.

In order to solve the above problems, there is provided a method of recognizing a content with a content recognition system using two minutiae in a large capacity VOD service, comprising: searching a first minutiae point of the input content in a database; Selecting a candidate group using the first feature point, the candidate group including candidate content IDs in the database that have passed a predetermined criterion; Determining whether the candidate group has IDs of the candidate content; Outputting a recognition failure signal when IDs of the candidate contents do not exist in the candidate group; Searching candidate candidates for a second characteristic point of the input content if the candidates for the candidate content are present in the candidate group; Selecting a candidate content most similar to the second feature point in the candidate content within the candidate group; Performing a False Positive Error (FPE) check on the most similar candidate content; Outputting a recognition error signal if the FPE test result is failure; And outputting a recognition completion signal when the FPE test result is successful. In the large capacity VOD service, a method of recognizing a content using two minutiae points is used.

The foregoing aspects will be fully described hereinafter and will include in particular the features pointed out in the claims. The following description and the annexed drawings set forth in detail certain illustrative aspects of one or more aspects. It is to be understood, however, that these embodiments represent only some of the various ways in which the principles of the various aspects may be utilized, and that the described aspects include all such aspects and their equivalents.

The present invention provides a content recognition service more accurately using two or more DNAs and improves recognition performance. Also, the database of contents can be categorized into two or more groups, and efficient recognition performance can be realized with a low search system cost.

1 is a block diagram of an embodiment showing a process of recognizing contents using two feature points (DNA) of the present invention.
FIG. 2 illustrates an embodiment of a recognition algorithm for recognizing a large-capacity content.
FIG. 3 illustrates an embodiment of a flow for recognizing real-time broadcast contents.
Fig. 4 illustrates an embodiment showing an example of the overall structure of a DNA search system.
5 is a block diagram of one embodiment illustrating more detailed operation of the indexing system of FIG.

BRIEF DESCRIPTION OF THE DRAWINGS The present invention is capable of various modifications and various embodiments, and specific embodiments are illustrated in the drawings and described in detail by way of preferred embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The following detailed description, together with the accompanying drawings, is intended to illustrate exemplary embodiments of the invention and is not intended to represent the only embodiments in which the invention may be practiced. The following detailed description includes specific details in order to provide a thorough understanding of the present invention. However, those skilled in the art will appreciate that the present invention may be practiced without these specific details.

The present invention relates to a method of using content feature points (DNA) to perform a supplementary information interlocking service using a smart terminal (smart pad, smart phone, etc.) of a viewer in real broadcast and large capacity VOD (Video On Demand) . Combining the DNAs with two characteristics to perform the desired additional information interworking service.

1 is an example showing a process of recognizing contents using two DNAs. Reference numeral 101 denotes a database (DB) search for contents inputted using the first DNA. Reference numeral 102 denotes a candidate group. In this case, the degree of similarity is measured and arranged in descending order of the degree of similarity between the DNA stored in the DB and the search DNA used for the search, and arranged in the order of the content IDs which can be judged as the closest. The candidate content ID that has passed the criterion that is determined to be similar as the candidate group is selected as the candidate group. The criterion that is judged here may be different for each algorithm, but usually the degree of similarity is numerically expressed and selected as a candidate group if it exceeds a certain set value. If a DNA that does not exist in the established DB is searched, the content ID selected as the candidate is usually not available. Reference numeral 103 denotes a function for judging whether or not a content ID exists in a candidate group. The case where the candidates do not exist in the candidate group 104 and the content recognition fails. Reference numeral 105 denotes a search unit for searching for a second DNA corresponding to the candidate content selected in the candidate group when candidate contents exist in the candidate group, and selects the most similar content. The characteristic of the second DNA is that the DNA can be used to accurately and accurately measure the similarity of the content but can not be used for DNA search because of the large amount of computation. Reference numeral 106 denotes a FPE (False Positive Error) test for the selected content candidate. FPE is a case where it should not occur as a case of recognizing an invalid content ID. If FPE occurs, it can produce very poor results in recognition services depending on application usage. Reference numeral 107 denotes a function for judging whether there is an error as a result of the FPE inspection. If it is an error, a recognition error is returned (108). If the FPE test result is not an error, the normal recognition is completed (109).

2 is an example showing a process of a recognition algorithm for recognizing a large-capacity content. Reference numeral 130 denotes a query first DNA which is recorded in the smart terminal of the listener and sent to the search server for content recognition. Reference numeral 131 denotes a first DNA DB search process using query first DNA data. A histogram table is created in a DNA file generated for each content using a lookup table. The value counted in the histogram is the number of matches between the query first DNA and the DNA values in the DNA file. Reference numeral 132 denotes a list of candidate recognition content candidates arranged in descending order of numbers by using a histogram table created by the reference numeral 131. The content with the largest count value becomes the candidate with the highest probability. Reference numeral 133 denotes a FPE (False Positive Error) discriminator, which performs a final verification of the candidate selected at 132. Here, if the similarity value measured using the second DNA is equal to or larger than the predetermined set value Th, the content candidate is judged as the recognized content and transmitted as the recognition result. If the set value is not exceeded, it is determined that the content is not recognized. The method of measuring the similarity value is a method of calculating various distances (hamming distance, euclidean distance, etc.), but the correlation value is used here. The equation below is a general formula for calculating the correlation value. Where x and y are the DNA vector values to be compared, respectively.

---- 수식 1

----

3 is an example showing a flow of recognizing real-time broadcast contents. Since real-time broadcasting can not DB-format the DNA for contents, there is a restriction to construct a DB in real time and use it immediately. Reference numeral 140 denotes an actual broadcasting system of the broadcasting station. Real-time DNA is extracted from the reference numeral 141 using the sound source of the image being on-air. Reference numeral 142 denotes a system for converting a DNA extracted in real time into a DB. Here, Live DB is called. Reference numeral 1421 accumulates about two minutes of the extracted DNA. The amount of time here must be set so as to overcome the actual broadcast signal delay time. And, to keep the DNA of 2 minutes at all times, when the latest DNA 1 block data is inputted, the oldest one DNA block is discarded. The correlator 1422 obtains the correlation value by taking the size of the query DNA 147 for retrieval from the DNA DB for two minutes of the reference numeral 1421. Reference numeral 1423 denotes a determiner, which determines that a correlation value is greater than the set value when the threshold value is Th and returns corresponding channel information and broadcast time offset information. If it is smaller than the set value, Live DB search is judged as failed. That is, there is no broadcast signal coinciding with the search DNA value. Reference numeral 143 denotes a device for recording audio using a smart device so that a viewer can recognize a broadcast signal. Reference numeral 144 denotes a web service server for searching broadcast signals. Reference numeral 145 denotes a matching system that plays a role of receiving a search request and performing matching. Reference numeral 146 denotes a function for extracting the Live DB search DNA sent from the reference numeral 143 for the matching search. Reference numeral 147 denotes a Live DB for retrieving the search DNA sent from the reference numeral 146 to the Live DB.

4 shows an example of the overall structure of a DNA search system. Reference numeral 150 denotes a portion in which a general listener makes a content recognition request using a smart terminal. Reference numeral 151 denotes a portion for performing a DNA search request sent from the reference numeral 150. Reference numeral 1510 denotes a search query request input as a web server. Reference numeral 1511 denotes a search command through a DNA DB matching for each query request received at a reference numeral 1510. Reference numeral 1512 denotes the overall DNA DB structure. The entire DNA DB is composed of a Preceding DNA DB 1533 and a DNA DB for VOD 1544. The preceding DNA DB 1533 is structured to perform fast searching, and is mainly used for a live-channel-recognition DNA database, a CF-recognition DNA database that can be applied to the most applications, and a terrestrial re- And a DNA database for hot content recognition to recognize at a high speed. The DNA DB for VOD (1544) is a DNA database configured to recognize a large amount of contents, and is mainly used mainly in VOD service. The classification into two DNA DB groups is to realize efficient recognition performance with a low system cost for a search system. First, when a recognition query is inputted, the preceding DNA DB 1533 having the highest recognition possibility is first searched for and recognized. If not, the VOD DNA DB 1544 is searched to perform a search service for content recognition. In case of generating such a large capacity query, the CF content can be mainly used when the live broadcast contents, the cable TV are retransmitted terrestrial contents, or the service using CF is activated.

5 is an illustration of a more detailed operation flow for the indexing system in FIG. The indexing system consists of two DNA DB structures. First, a Preceding DNA DB 1533 and a VOD DNA DB 1544 are constructed. The Preceding DNA DB 1533 is a DB provided for fast searching, and it mainly consists of a DNA DB for recognizing live broadcast contents and a CF DNA DB and a Hot Content DNA DB, which are thought to be frequently used for searching. The Hot Content DB is a content located in the Preceding DNA DB 1533 so that a content having a high viewership among the contents broadcast by a terrestrial broadcasting station can be searched first. Hot Content DB selection methods can be targeted to content that has a high audience rating in recent times. The CF DNA DB is provided to perform additional information providing function for the advertisement contents, and it can make access to the information of the goods which can not be shown in the short advertisement time or the means to purchase directly. In this way, the user first performs a search for a content likely to be searched by the user, and searches for a large-capacity VOD DNA DB (1544) for a query that does not pass the search, thereby proceeding to a search process for searching VOD contents. If all the retrieval fails in the second retrieval, the query returns unrecognized results as unrecognizable contents. If it recognizes, it returns the content recognition result.

The description of the disclosed technique is merely an example for structural or functional explanation and the scope of the disclosed technology should not be construed as being limited by the embodiments described in the text. That is, the embodiments are to be construed as being variously embodied and having various forms, so that the scope of the disclosed technology should be understood to include equivalents capable of realizing technical ideas.

Meanwhile, the meaning of the terms described in the present application should be understood as follows.

It should be understood that the singular " include "or" have "are to be construed as including a stated feature, number, step, operation, component, It is to be understood that the combination is intended to specify that it is present and not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

All terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the disclosed technology belongs, unless otherwise defined. Commonly used predefined terms should be interpreted to be consistent with the meanings in the context of the related art and can not be interpreted as having ideal or overly formal meaning unless explicitly defined in the present application.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the present invention as defined by the following claims It can be understood that

Claims (10)

A method of recognizing contents with a contents recognition system using two minutiae points in a large-capacity VOD (Video On Demand) service,
Retrieving a first feature point of the input content in a database;
Selecting a candidate group using the first feature point,
Wherein the candidate group includes candidate content IDs in the database that have passed a predetermined criterion;
Determining whether the candidate group has IDs of the candidate content;
Outputting a recognition failure signal when IDs of the candidate contents do not exist in the candidate group;
Searching candidate candidates for a second characteristic point of the input content if the candidates for the candidate content are present in the candidate group;
Selecting a candidate content most similar to the second feature point in the candidate content within the candidate group;
Performing a False Positive Error (FPE) check on the most similar candidate content;
Outputting a recognition error signal if the FPE test result is failure; And
And outputting a recognition completion signal if the FPE test result is successful,
Wherein the second feature point can accurately measure the degree of similarity with respect to the content than the first feature point, but the amount of computation is larger than the first feature point. The method according to claim 1,
Wherein searching the first feature point of the input content in the database comprises:
Generating a histogram table in a minutia file created for each content in a database using a lookup table;
Counting the number of coincidence of the first feature point and the feature point in the feature point file and displaying the counted number in the histogram table. The method according to claim 1,
Wherein the step of selecting a candidate group using the first feature point comprises:
Measuring a degree of similarity between the first feature point and a feature point of the content in the database;
Arranging IDs of contents in the database in order of the measured similarity; And
Further comprising selecting candidate IDs of candidate contents in the database satisfying the predetermined criterion in which the degree of similarity is equal to or larger than a predetermined set value. delete The method according to claim 1,
Wherein performing FPE checking for the most similar candidate content comprises:
Further comprising determining that the FPE check is successful if the similarity value to the most similar candidate content measured using the second feature point is greater than or equal to a predetermined set value. 6. The method of claim 5,
Wherein the similarity value measurement is performed by at least one of Hamming distance, Euclidean distance, and correlation. The method according to claim 1,
The method of claim 1, wherein if the input content is a real-time broadcast content, a step of converting the minutiae point of the real-time broadcast content into a database before searching the first minutia point in a database,
Extracting characteristic points of real-time broadcasting contents in real time;
Accumulating feature points of the real-time broadcast contents for a predetermined time; And
And discarding old feature points among the feature points accumulated in the predetermined time period, accumulating new feature points, and maintaining the accumulated feature points at the predetermined amount of time. 8. The method of claim 7,
Wherein the predetermined amount of time is two minutes. The method according to claim 1,
Wherein,
A preceding feature point database provided for fast searching, and
A feature point database for a VOD for a content that is less likely to be searched than a content in a pre-syndication feature point database,
Wherein the VOD feature point database is used when the recognition failure or the recognition error signal is output as a result of performing the method of recognizing the contents using the pre-syndication feature point database. 10. The method of claim 9,
The pre-syndication feature point database includes a feature point database for live channel recognition, a feature point database for advertisement (CF) recognition, and hot content recognition for quickly recognizing contents such as terrestrial re-transmission in a cable TV broadcast company And a feature point database.

Images