EP2850838A1 - Methods and systems for providing broadcast ad identification - Google Patents

Methods and systems for providing broadcast ad identification

Info

Publication number
EP2850838A1
EP2850838A1 EP13785191.1A EP13785191A EP2850838A1 EP 2850838 A1 EP2850838 A1 EP 2850838A1 EP 13785191 A EP13785191 A EP 13785191A EP 2850838 A1 EP2850838 A1 EP 2850838A1
Authority
EP
European Patent Office
Prior art keywords
fingerprint
signatures
fingerprint signatures
repeat
identification
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP13785191.1A
Other languages
German (de)
French (fr)
Other versions
EP2850838A4 (en
Inventor
Adi Shavit
Dana Shavit
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tvtak Ltd
Original Assignee
Tvtak Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from PCT/IB2012/051584 external-priority patent/WO2012131653A2/en
Priority claimed from US13/832,083 external-priority patent/US20140013352A1/en
Application filed by Tvtak Ltd filed Critical Tvtak Ltd
Publication of EP2850838A1 publication Critical patent/EP2850838A1/en
Publication of EP2850838A4 publication Critical patent/EP2850838A4/en
Withdrawn legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04H—BROADCAST COMMUNICATION
    • H04H20/00—Arrangements for broadcast or for distribution combined with broadcast
    • H04H20/12—Arrangements for observation, testing or troubleshooting
    • H04H20/14—Arrangements for observation, testing or troubleshooting for monitoring programmes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04H—BROADCAST COMMUNICATION
    • H04H60/00—Arrangements for broadcast applications with a direct linking to broadcast information or broadcast space-time; Broadcast-related systems
    • H04H60/35—Arrangements for identifying or recognising characteristics with a direct linkage to broadcast information or to broadcast space-time, e.g. for identifying broadcast stations or for identifying users
    • H04H60/37—Arrangements for identifying or recognising characteristics with a direct linkage to broadcast information or to broadcast space-time, e.g. for identifying broadcast stations or for identifying users for identifying segments of broadcast information, e.g. scenes or extracting programme ID
    • H04H60/375—Commercial
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04H—BROADCAST COMMUNICATION
    • H04H60/00—Arrangements for broadcast applications with a direct linking to broadcast information or broadcast space-time; Broadcast-related systems
    • H04H60/56—Arrangements characterised by components specially adapted for monitoring, identification or recognition covered by groups H04H60/29-H04H60/54
    • H04H60/59—Arrangements characterised by components specially adapted for monitoring, identification or recognition covered by groups H04H60/29-H04H60/54 of video
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof
    • H04N21/812—Monomedia components thereof involving advertisement data
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/835—Generation of protective data, e.g. certificates
    • H04N21/8352—Generation of protective data, e.g. certificates involving content or source identification data, e.g. Unique Material Identifier [UMID]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/835—Generation of protective data, e.g. certificates
    • H04N21/8358—Generation of protective data, e.g. certificates involving watermark
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845—Structuring of content, e.g. decomposing content into time segments
    • H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments

Definitions

  • the present invention relates to methods and systems for providing broadcast ad identification.
  • the present invention relates to systems that can identify when (i.e., in time and duration) and where (e.g., on which TV channel) each TV ad is broadcast.
  • Tian ⁇ 67 discloses method for identifying repeat clip instances in video data by partitioning the data into ordered video units using content-based keyframe sampling, with each video unit having a sequence interval between two consecutive key- frames.
  • the methods of Tian ⁇ 67 create a fingerprint for each video unit, and group consecutive video units into time-indexed video segments in order to identify repeat clip instances based on correlation of the video segments.
  • Tian ⁇ 67 are based on utilizing sequential frame similarity relying on multi-frame fingerprints, not the reoccurrence of single-frame fingerprints.
  • the fingerprints are unit fingerprints of a segment.
  • Such a "bottom-up" approach has inherent limitations with regard to frame-boundary issues, detection accuracy, and real-time detection.
  • Ads are relatively short in duration (i.e., typically about 5-30 seconds) - distinguishing the ads from other longer video content, or from certain, very short, repeating slides or frames.
  • Ads are repeatedly broadcast during the day - distinguishing the ads from show-specific video clips which may repeat (e.g., title sequences and re-entry excerpts from commercials), since TV shows are not broadcast throughout the day but are limited to a designated time slot.
  • Ads are repeatedly broadcast at varying intervals during the day (i.e., there is a minimum time period between two broadcasts of an ad).
  • Ads are broadcast on multiple channels - distinguishing the ads from channel and show promotion, since competing channels won't usually broadcast the same promotions.
  • Ads typically appear within a batch of other commercials.
  • video In the interest of clarity, several terms which follow are specifically defined for use herein.
  • video video feed
  • TV feed Broadcast feed
  • video clip are used herein to refer to various technological embodiments of electronically processing and reconstructing a sequence of still images representing scenes in motion.
  • video server is used herein to refer to a computer hardware system dedicated to run one or more video-feed services (detailed below) as a host in order to serve the needs of other components in the ad detection system of the present invention.
  • fingerprint signature is used herein to refer to a compact representation of a frame of a video feed.
  • exemplary is used herein to refer to examples of embodiments and/or implementations, and is not meant to necessarily convey a more- desirable use-case.
  • preferred is used herein to refer to an example out of an assortment of contemplated embodiments and/or implementations, and is not meant to necessarily convey a more-desirable use-case. Therefore, it is understood from the above that “exemplary” and “preferred” may be applied herein to multiple embodiments and/or implementations.
  • a non-transitory computer-readable medium includes a hard drive, a compact disc, flash memory, volatile memory, non-volatile memory, and similar device memory, but does not include a transitory signal per se.
  • Preferred embodiments of the present invention enable recognition of TV ads that have not been previously detected, and therefore are not pre-indexed.
  • inventions of the present invention enable recognition of TV ads.
  • the system identifies in real time a commercial being broadcast. Utilizing the distinguishing ad characteristics mentioned above, the detection of repeating segments that aren't typically ads is enabled.
  • Such implementations enable revenue generation via real-time ad recognition, competitor analysis, and affiliate call-to-actions.
  • Ad broadcast statistics including ad count, frequency, and time-of-day airing can be used in business-to-business (B2B) relationships.
  • Real-time ad notification of an ad broadcast while it is being aired also provides enhanced B2B value.
  • a method for providing broadcast ad identification including the steps of: (a) providing fingerprint signatures of each frame in a broadcast video for ad identification; and (b) designating at least two repeat fingerprint signatures upon detecting at least one fingerprint-signature match from the fingerprint signatures.
  • the method further includes the step of: (c) prior to the step of designating, determining whether the fingerprint signatures correspond to a known ad based upon detecting at least one fingerprint- signature match of the fingerprint signatures with pre-indexed fingerprint signatures of pre-indexed ads.
  • the method further includes the step of: (c) creating segments of the fingerprint signatures, ordered according to a timeline temporal proximity of the fingerprint signatures, by grouping at least two fingerprint signatures based on a repeat temporal proximity of at least two repeat fingerprint signatures respective of at least two fingerprint signatures.
  • the method further includes the step of: (d) detecting at least one ad candidate based on an occurrence of at least one repeat segment.
  • the step of detecting at least one ad candidate is checked for consistency based on a segment-to- segment mapping.
  • a system for providing broadcast ad identification including: (a) a server including: (i) a CPU for performing computational operations; and (ii) a memory module for storing data; and (b) a processing module, residing on the server, configured for: (i) processing fingerprint signatures of each frame in a broadcast video for ad identification; and (ii) designating at least two repeat fingerprint signatures upon detecting at least one fingerprint-signature match from the fingerprint signatures.
  • the processing module is further configured for: (iii) prior to the designating, determining whether the fingerprint signatures correspond to a known ad based upon detecting at least one fingerprint- signature match of the fingerprint signatures with pre-indexed fingerprint signatures of pre-indexed ads.
  • the processing module is further configured for: (iii) creating segments of the fingerprint signatures, ordered according to a timeline temporal proximity of the fingerprint signatures, by grouping at least two fingerprint signatures based on a repeat temporal proximity of at least two repeat fingerprint signatures respective of at least two fingerprint signatures.
  • the processing module is further configured for: (iv) detecting at least one ad candidate based on an occurrence of at least one repeat segment.
  • the detecting at least one ad candidate is checked for consistency based on a segment-to- segment mapping.
  • Figure 1 is a simplified schematic diagram of the system architecture for providing broadcast ad identification, according to preferred embodiments of the present invention
  • FIG. 2A is a simplified flowchart of the major process steps on a video server for providing broadcast ad identification, according to preferred embodiments of the present invention
  • Figure 2B is a simplified flowchart of the major process steps on an ad- candidate identification unit for providing broadcast ad identification, according to preferred embodiments of the present invention
  • Figure 2C is a simplified flowchart of the major process steps on an ad identification engine for providing broadcast ad identification, according to preferred embodiments of the present invention
  • Figure 2D is a simplified flowchart of the major process steps on an annotation site for providing broadcast ad identification, according to preferred embodiments of the present invention
  • Figure 3A is a simplified flowchart of the major process steps on a video server for providing broadcast ad identification, according to alternate embodiments of the present invention
  • Figure 3B is a simplified flowchart of the major process steps on an ad- candidate identification unit for providing broadcast ad identification, according to alternate embodiments of the present invention.
  • the present invention relates to methods and systems for providing broadcast ad identification.
  • the principles and operation for such methods and systems, according to the present invention, may be better understood with reference to the accompanying description and drawings.
  • FIG. 1 is a simplified schematic diagram of the system architecture for providing broadcast ad identification, according to preferred embodiments of the present invention.
  • a video server 2 is shown having a broadcast-video & signature session storage 4.
  • the designation "session” is used herein to indicate that the storage of the bulk data is temporary. Once the bulk data of a session (e.g., a day's worth of video feeds) has been processed, the data will be removed to make room for the next session batch.
  • Video server 2 is operationally connected to an ad-candidate identification unit 6 having an ad-candidate session storage 8.
  • Ad-candidate identification unit 6 automatically detects all new, unindexed ads, and extracts the ads as ad candidates for external tagging.
  • Ad-candidate identification unit 6 processes in bulk from fingerprint session signatures.
  • Fingerprint signatures are computed from the TV feeds, and, optionally, relevant metadata is added to the fingerprint signatures.
  • fingerprint-signature comparison is performed using the well-known Hamming distance between bit-strings (i.e., the number of bits that are different between 2 bit- strings). The larger the Hamming distance is, the worse the match is between the signatures.
  • Bulk processing by ad-candidate identification unit 6 includes the processing of all sequential fingerprint signatures of multiple broadcast channels. Such bulk data of fingerprint signatures are required to be from a fairly-long duration of time (e.g., a day's worth of multi-channel broadcast feed) in order to identify ad candidates which have repeating elements. Processing can be performed on all the extracted fingerprint signatures (as described below with regard to Figure 2B), or only on the fingerprint signatures corresponding to unidentified new content (as described below with regard to Figure 3B).
  • Ad-candidate identification unit 6 is operationally connected to an ad identification engine 10 having an ad-identifier module 12 and an ad storage repository 14.
  • Ad identification engine 10 is also operationally connected to an ad monitoring unit 16 which monitors all broadcast channels in real time, and detects the start and end of a known, pre-indexed ad (i.e., an ad that had previously been detected and tagged). Once a new, unindexed ad candidate is identified (and tagged at an annotation site 18 described below) as an ad by ad-candidate identification unit 6, the ad is added to ad storage repository 14, which allows the ad to be detected by ad monitoring unit 16 in real time. It is noted that in some implementations ad monitoring unit 16 resides on video server 2.
  • the ad-candidate extraction process employs a fingerprint-to-fingerprint (F2F) matching method. For each frame fingerprint signature, all - if any - matching fingerprint signatures (i.e., repeats) are found in the new session data in which ad identification engine 10 matches the bulk data to itself. Such operations need to be very fast and algorithmic ally efficient, since self-matching requires matching millions of fingerprint signatures to the same millions of fingerprint signatures.
  • F2F fingerprint-to-fingerprint
  • Segments are then created by grouping together all consecutive fingerprint signatures which have repeats that are close together in time.
  • the segment-creation process is sequentially performed over all fingerprint signatures. Fingerprint signatures are added to a current segment if the signatures' matches are close (in time) to the matches of previously-added fingerprint signatures in the current segment. If the signatures' matches aren't close to the matches of previously- added fingerprint signatures in the current segment, a new segment (having a new segment ID) is created. Fingerprint signatures with no matches are grouped into zero- segments.
  • each segment is a sequence of consecutive fingerprint signatures. All fingerprint signatures possibly have matching fingerprint signatures in other segments. Since the segments are non- overlapping with no gaps between them, each fingerprint signature belongs to one and only one segment (and segment ID).
  • a segment- to- segment (S2S) mapping method is then employed in which each segment is mapped to all other segments that contain matching fingerprint signatures to fingerprint signatures within a given segment.
  • Mapping consistency is checked by creating a consistent S2S mapping that identifies true repeats for each segment.
  • IDs the other segments
  • Short segments are optionally removed.
  • N(s) is the set of neighboring segments previously mapped, all mapped segments t belonging to N(s) are then iterated. If s belongs to any of N(t), and the segment durations of s and t are similar, then the match between s and t is considered consistent - a type of bidirectional mapping.
  • segment filtering based on several segment types including:
  • Ad-candidate identification unit 6 is also operationally connected to an annotation site 18, which is also operationally connected to ad identification engine 10.
  • Annotation site 18 enables each ad candidate to be identified and tagged. Once the broadcast times, durations, and channels of all ad candidates are determined, each ad candidate is identified and tagged.
  • Such identifying and tagging can be done manually by displaying the relevant video clip at the appropriate time to a user, and letting the user select the appropriate brand, and assign any relevant metadata to the clip. Alternatively, such identifying and tagging can be done by using additional metadata associated with the video feed.
  • Such additional metadata may include (but is not limited to):
  • closed-caption text often aired in-sync with the video feed, and which may contain brand names and other valuable ad information (in such a case, the video feed corresponding to the bulk data must be stored for the viewing process);
  • logo recognition - certain brands may be identified by automatic logo identification within a video feed.
  • the ad candidate can be added to ad identification engine 10 for later identification in real time via ad- identifier module 12. All appearances of ads in the bulk data are also uploaded (retroactively) into ad statistics logs on ad identification engine 10.
  • FIG. 2A is a simplified flowchart of the major process steps on a video server for providing broadcast ad identification, according to preferred embodiments of the present invention.
  • the process starts on video server 2 by receiving a broadcast video feed (Step 20). Session signatures are then generated (Step 22), and the broadcast video and signatures are stored in broadcast- video & signature session storage 4 (Step 24).
  • FIG. 2B is a simplified flowchart of the major process steps on an ad- candidate identification unit for providing broadcast ad identification, according to preferred embodiments of the present invention.
  • the process starts when ad-candidate identification unit 6 receives the stored session signatures from video server 2.
  • the stored session signatures are analyzed (Step 26), and ad candidates associated with repeating segments within and across channels are detected (Step 28).
  • Ad-candidate identification unit 6 determines whether the ad candidates are known ads (Step 30).
  • the ad candidates are stored in ad- candidate session storage 8 (Step 32). If the ad candidates are known ads, the ad candidates are transferred to ad identification engine 10.
  • FIG. 2C is a simplified flowchart of the major process steps on an ad identification engine for providing broadcast ad identification, according to preferred embodiments of the present invention.
  • the process starts when ad-candidate identification unit 6 transfers the ad candidates that are known ads to ad identification engine 10.
  • Ad identification engine 10 updates the ad broadcast statistics, stored in ad storage repository 14, and used by ad-identifier module 12 (Step 34).
  • Ad identification engine 10 then provides the pre-indexed ad-identification service to ad monitoring unit 16 for real-time ad broadcast detection (Step 36).
  • FIG 2D is a simplified flowchart of the major process steps on an annotation site for providing broadcast ad identification, according to preferred embodiments of the present invention.
  • the process starts when the ad candidates are retrieved from ad-candidate identification unit 6 by annotation site 18 (Step 38).
  • Corresponding video segments and/or additional metadata are retrieved (Step 40), and any metadata is processed (Step 42).
  • the new ads are then stored (Step 44), and the ad candidates and video are removed from storage (Step 46).
  • FIG 3A is a simplified flowchart of the major process steps on a video server for providing broadcast ad identification, according to alternate embodiments of the present invention.
  • video server 2 receives a broadcast video feed (Step 50). Session signatures are then generated (Step 52) as in the process of Figure 2A. Video server 2 then determines whether the ad candidates are known ads (Step 54).
  • the ad candidates are stored in broadcast- video & signature session storage 4 (Step 56). If the ad candidates are known ads, the ad candidates are transferred to ad identification engine 10 for further processing as in Figure 2C.
  • FIG 3B is a simplified flowchart of the major process steps on an ad- candidate identification unit for providing broadcast ad identification, according to alternate embodiments of the present invention.
  • the process starts when ad-candidate identification unit 6 receives the stored session signatures from video server 2.
  • the stored session signatures are analyzed (Step 58), and ad candidates associated with repeating segments within and across channels are detected (Step 60) as in the process of Figure 2B.
  • the ad candidates are then stored in ad-candidate session storage 8 (Step 62), and the ad candidates are retrieved by annotation site 18 for further processing as in Figure 2D. While the present invention has been described with respect to a limited number of embodiments, it will be appreciated that many variations, modifications, and other applications of the present invention may be made.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Computer Security & Cryptography (AREA)
  • Business, Economics & Management (AREA)
  • Marketing (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Collating Specific Patterns (AREA)

Abstract

The present invention discloses methods and systems for providing broadcast ad identification. Methods include the steps of: providing fingerprint signatures of each frame in a broadcast video; and designating at least two repeat fingerprint signatures upon detecting at least one fingerprint-signature match from the signatures. Preferably, methods further include: prior to the designating, determining whether the fingerprint signatures correspond to a known ad based upon detecting at least one fingerprint- signature match of the fingerprint signatures with pre-indexed fingerprint signatures of pre-indexed ads. Preferably, method further include: creating segments of the fingerprint signatures, ordered according to a timeline temporal proximity of the fingerprint signatures, by grouping at least two fingerprint signatures based on a repeat temporal proximity of at least two repeat fingerprint signatures respective of at least two fingerprint signatures. Preferably, methods further include detecting at least one ad candidate based on an occurrence of at least one repeat segment.

Description

METHODS AND SYSTEMS FOR PROVIDING BROADCAST AD IDENTIFICATION
CROSS REFERENCE TO RELATED APPLICATIONS
This patent application claims priority to U.S. Patent Application No.
13/832,083, filed on March 15, 2013, which therein claims priority to U.S. Patent
Application No. 13/821,983, filed on March 10, 2013, which therein claims priority U.S. Provisional Patent Application No. 61/469,875 filed March 31, 2011, and to PCT
Patent Application No. IB2012/051584 filed April 1, 2012, and which are hereby incorporated by reference in their entirety.
This patent application further claims priority U.S. Provisional Patent
Application No. 61/669,132 filed July 9, 2012, which is hereby incorporated by reference in its entirety.
FIELD AND BACKGROUND OF THE INVENTION
The present invention relates to methods and systems for providing broadcast ad identification. In particular, the present invention relates to systems that can identify when (i.e., in time and duration) and where (e.g., on which TV channel) each TV ad is broadcast.
Televised broadcast advertising has been a staple of consumer marketing and brand recognition for countless retailers since the advent of the television. Over the decades, while changing in content, TV ads have largely maintained a fairly-standard format. For numerous applications, the ability to accurately and precisely detect TV ads is considered to be valuable. However, the identification of TV ads from broadcasts is a task that is fraught with complexity. Adequate methods for detection are not known in the art.
In the art, US Patent Publication No. 2012/0114167 by Tian et al. (hereinafter referred to as Tian Ί67) discloses method for identifying repeat clip instances in video data by partitioning the data into ordered video units using content-based keyframe sampling, with each video unit having a sequence interval between two consecutive key- frames. The methods of Tian Ί67 create a fingerprint for each video unit, and group consecutive video units into time-indexed video segments in order to identify repeat clip instances based on correlation of the video segments.
The methods of Tian Ί67 are based on utilizing sequential frame similarity relying on multi-frame fingerprints, not the reoccurrence of single-frame fingerprints. The fingerprints are unit fingerprints of a segment. Such a "bottom-up" approach has inherent limitations with regard to frame-boundary issues, detection accuracy, and real-time detection.
Some important observations can be generally made regarding the nature of TV ads in comparison to the rest of the broadcasts that the ads are presented within. Ads obey a certain patterns that distinguish them from other broadcast video content.
(1) Ads are relatively short in duration (i.e., typically about 5-30 seconds) - distinguishing the ads from other longer video content, or from certain, very short, repeating slides or frames.
(2) Ads are repeatedly broadcast during the day - distinguishing the ads from show-specific video clips which may repeat (e.g., title sequences and re-entry excerpts from commercials), since TV shows are not broadcast throughout the day but are limited to a designated time slot. (3) Ads are repeatedly broadcast at varying intervals during the day (i.e., there is a minimum time period between two broadcasts of an ad).
(4) Ads are broadcast on multiple channels - distinguishing the ads from channel and show promotion, since competing channels won't usually broadcast the same promotions.
(5) Ads typically appear within a batch of other commercials.
It would be desirable to have such methods and systems for providing broadcast ad identification. Such methods and systems would, inter alia, overcome the various limitations mentioned above utilizing the distinguishing ad characteristics mentioned above.
SUMMARY OF THE INVENTION
It is the purpose of the present invention to provide methods and systems for providing broadcast ad identification.
In the interest of clarity, several terms which follow are specifically defined for use herein. The terms "video," "video feed," "TV feed," "broadcast feed," and "video clip" are used herein to refer to various technological embodiments of electronically processing and reconstructing a sequence of still images representing scenes in motion. The term "video server" is used herein to refer to a computer hardware system dedicated to run one or more video-feed services (detailed below) as a host in order to serve the needs of other components in the ad detection system of the present invention. The term "fingerprint signature" is used herein to refer to a compact representation of a frame of a video feed.
It is noted that the term "exemplary" is used herein to refer to examples of embodiments and/or implementations, and is not meant to necessarily convey a more- desirable use-case. Similarly, the term "preferred" is used herein to refer to an example out of an assortment of contemplated embodiments and/or implementations, and is not meant to necessarily convey a more-desirable use-case. Therefore, it is understood from the above that "exemplary" and "preferred" may be applied herein to multiple embodiments and/or implementations.
It is noted that a non-transitory computer-readable medium includes a hard drive, a compact disc, flash memory, volatile memory, non-volatile memory, and similar device memory, but does not include a transitory signal per se.
Preferred embodiments of the present invention enable recognition of TV ads that have not been previously detected, and therefore are not pre-indexed.
Other embodiments of the present invention enable recognition of TV ads. In preferred implementations of such embodiments, the system identifies in real time a commercial being broadcast. Utilizing the distinguishing ad characteristics mentioned above, the detection of repeating segments that aren't typically ads is enabled.
Such implementations enable revenue generation via real-time ad recognition, competitor analysis, and affiliate call-to-actions. Ad broadcast statistics including ad count, frequency, and time-of-day airing can be used in business-to-business (B2B) relationships. Real-time ad notification of an ad broadcast while it is being aired also provides enhanced B2B value.
Therefore, according to the present invention, there is provided for the first time a method for providing broadcast ad identification, the method including the steps of: (a) providing fingerprint signatures of each frame in a broadcast video for ad identification; and (b) designating at least two repeat fingerprint signatures upon detecting at least one fingerprint-signature match from the fingerprint signatures. Preferably, the method further includes the step of: (c) prior to the step of designating, determining whether the fingerprint signatures correspond to a known ad based upon detecting at least one fingerprint- signature match of the fingerprint signatures with pre-indexed fingerprint signatures of pre-indexed ads.
Preferably, the method further includes the step of: (c) creating segments of the fingerprint signatures, ordered according to a timeline temporal proximity of the fingerprint signatures, by grouping at least two fingerprint signatures based on a repeat temporal proximity of at least two repeat fingerprint signatures respective of at least two fingerprint signatures.
More preferably, the method further includes the step of: (d) detecting at least one ad candidate based on an occurrence of at least one repeat segment.
Most preferably, the step of detecting at least one ad candidate is checked for consistency based on a segment-to- segment mapping.
According to the present invention, there is provided for the first time a system for providing broadcast ad identification, the system including: (a) a server including: (i) a CPU for performing computational operations; and (ii) a memory module for storing data; and (b) a processing module, residing on the server, configured for: (i) processing fingerprint signatures of each frame in a broadcast video for ad identification; and (ii) designating at least two repeat fingerprint signatures upon detecting at least one fingerprint-signature match from the fingerprint signatures.
Preferably, the processing module is further configured for: (iii) prior to the designating, determining whether the fingerprint signatures correspond to a known ad based upon detecting at least one fingerprint- signature match of the fingerprint signatures with pre-indexed fingerprint signatures of pre-indexed ads. Preferably, the processing module is further configured for: (iii) creating segments of the fingerprint signatures, ordered according to a timeline temporal proximity of the fingerprint signatures, by grouping at least two fingerprint signatures based on a repeat temporal proximity of at least two repeat fingerprint signatures respective of at least two fingerprint signatures.
More preferably, the processing module is further configured for: (iv) detecting at least one ad candidate based on an occurrence of at least one repeat segment.
Most preferably, the detecting at least one ad candidate is checked for consistency based on a segment-to- segment mapping.
These and further embodiments will be apparent from the detailed description and examples that follow.
BRIEF DESCRIPTION OF THE DRAWINGS
The present invention is herein described, by way of example only, with reference to the accompanying drawings, wherein:
Figure 1 is a simplified schematic diagram of the system architecture for providing broadcast ad identification, according to preferred embodiments of the present invention;
Figure 2A is a simplified flowchart of the major process steps on a video server for providing broadcast ad identification, according to preferred embodiments of the present invention;
Figure 2B is a simplified flowchart of the major process steps on an ad- candidate identification unit for providing broadcast ad identification, according to preferred embodiments of the present invention; Figure 2C is a simplified flowchart of the major process steps on an ad identification engine for providing broadcast ad identification, according to preferred embodiments of the present invention;
Figure 2D is a simplified flowchart of the major process steps on an annotation site for providing broadcast ad identification, according to preferred embodiments of the present invention;
Figure 3A is a simplified flowchart of the major process steps on a video server for providing broadcast ad identification, according to alternate embodiments of the present invention;
Figure 3B is a simplified flowchart of the major process steps on an ad- candidate identification unit for providing broadcast ad identification, according to alternate embodiments of the present invention.
DESCRIPTION OF THE PREFERRED EMBODIMENTS
The present invention relates to methods and systems for providing broadcast ad identification. The principles and operation for such methods and systems, according to the present invention, may be better understood with reference to the accompanying description and drawings.
Referring now to the drawings, Figure 1 is a simplified schematic diagram of the system architecture for providing broadcast ad identification, according to preferred embodiments of the present invention. A video server 2 is shown having a broadcast-video & signature session storage 4. The designation "session" is used herein to indicate that the storage of the bulk data is temporary. Once the bulk data of a session (e.g., a day's worth of video feeds) has been processed, the data will be removed to make room for the next session batch. Video server 2 is operationally connected to an ad-candidate identification unit 6 having an ad-candidate session storage 8. Ad-candidate identification unit 6 automatically detects all new, unindexed ads, and extracts the ads as ad candidates for external tagging. Ad-candidate identification unit 6 processes in bulk from fingerprint session signatures.
Fingerprint signatures are computed from the TV feeds, and, optionally, relevant metadata is added to the fingerprint signatures. In such embodiments, fingerprint-signature comparison is performed using the well-known Hamming distance between bit-strings (i.e., the number of bits that are different between 2 bit- strings). The larger the Hamming distance is, the worse the match is between the signatures.
Bulk processing by ad-candidate identification unit 6 includes the processing of all sequential fingerprint signatures of multiple broadcast channels. Such bulk data of fingerprint signatures are required to be from a fairly-long duration of time (e.g., a day's worth of multi-channel broadcast feed) in order to identify ad candidates which have repeating elements. Processing can be performed on all the extracted fingerprint signatures (as described below with regard to Figure 2B), or only on the fingerprint signatures corresponding to unidentified new content (as described below with regard to Figure 3B).
Ad-candidate identification unit 6 is operationally connected to an ad identification engine 10 having an ad-identifier module 12 and an ad storage repository 14. Ad identification engine 10 is also operationally connected to an ad monitoring unit 16 which monitors all broadcast channels in real time, and detects the start and end of a known, pre-indexed ad (i.e., an ad that had previously been detected and tagged). Once a new, unindexed ad candidate is identified (and tagged at an annotation site 18 described below) as an ad by ad-candidate identification unit 6, the ad is added to ad storage repository 14, which allows the ad to be detected by ad monitoring unit 16 in real time. It is noted that in some implementations ad monitoring unit 16 resides on video server 2.
The ad-candidate extraction process employs a fingerprint-to-fingerprint (F2F) matching method. For each frame fingerprint signature, all - if any - matching fingerprint signatures (i.e., repeats) are found in the new session data in which ad identification engine 10 matches the bulk data to itself. Such operations need to be very fast and algorithmic ally efficient, since self-matching requires matching millions of fingerprint signatures to the same millions of fingerprint signatures.
For example, 24 hours of video, indexed at 15 frames/sec, generates 15x60x60x24 = 1.3 million fingerprint signatures. If matched naively against each other, such operations would require 1.3Mxl.3M = 1.678 billion signature comparisons. With a speedy 1 ms/comparison, more than 53 years of computing time would be required to complete the task.
Segments are then created by grouping together all consecutive fingerprint signatures which have repeats that are close together in time. The segment-creation process is sequentially performed over all fingerprint signatures. Fingerprint signatures are added to a current segment if the signatures' matches are close (in time) to the matches of previously-added fingerprint signatures in the current segment. If the signatures' matches aren't close to the matches of previously- added fingerprint signatures in the current segment, a new segment (having a new segment ID) is created. Fingerprint signatures with no matches are grouped into zero- segments.
The data is thus split into consecutive segments, with each segment being a sequence of consecutive fingerprint signatures. All fingerprint signatures possibly have matching fingerprint signatures in other segments. Since the segments are non- overlapping with no gaps between them, each fingerprint signature belongs to one and only one segment (and segment ID).
Optionally, a segment- to- segment (S2S) mapping method is then employed in which each segment is mapped to all other segments that contain matching fingerprint signatures to fingerprint signatures within a given segment. Mapping consistency is checked by creating a consistent S2S mapping that identifies true repeats for each segment. For each segment, all the other segments (IDs) that are from a different broadcast of the same video feed are found. Short segments are optionally removed. For each segment s, given N(s) is the set of neighboring segments previously mapped, all mapped segments t belonging to N(s) are then iterated. If s belongs to any of N(t), and the segment durations of s and t are similar, then the match between s and t is considered consistent - a type of bidirectional mapping.
Once all consistent repeating segments are found, the segments are classified using a segment filtering based on several segment types including:
(1) unique segments which appear only once;
(2) segments which repeat only on a single channel; and
(3) segments which repeat across multiple channels.
Ad-candidate identification unit 6 is also operationally connected to an annotation site 18, which is also operationally connected to ad identification engine 10. Annotation site 18 enables each ad candidate to be identified and tagged. Once the broadcast times, durations, and channels of all ad candidates are determined, each ad candidate is identified and tagged. Such identifying and tagging can be done manually by displaying the relevant video clip at the appropriate time to a user, and letting the user select the appropriate brand, and assign any relevant metadata to the clip. Alternatively, such identifying and tagging can be done by using additional metadata associated with the video feed. Such additional metadata may include (but is not limited to):
(1) closed-caption text - often aired in-sync with the video feed, and which may contain brand names and other valuable ad information (in such a case, the video feed corresponding to the bulk data must be stored for the viewing process);
(2) speech-to-text - involving converting the audio track of the video feed into text, such captured text extracted from the video feed itself can contain valuable information and metadata (in such a case, there is no need to store the video feed itself, but only the additional metadata such as the audio track, the converted text, and/or the closed-caption text); and
(3) logo recognition - certain brands may be identified by automatic logo identification within a video feed.
Note that none of the identifying and tagging methods involving additional associated metadata allow for the precise extraction of the ad boundaries and repeats, nor are such approaches always feasible to perform in real time once an ad has been indexed.
Once an ad candidate has been identified and tagged, the ad candidate can be added to ad identification engine 10 for later identification in real time via ad- identifier module 12. All appearances of ads in the bulk data are also uploaded (retroactively) into ad statistics logs on ad identification engine 10.
Figure 2A is a simplified flowchart of the major process steps on a video server for providing broadcast ad identification, according to preferred embodiments of the present invention. The process starts on video server 2 by receiving a broadcast video feed (Step 20). Session signatures are then generated (Step 22), and the broadcast video and signatures are stored in broadcast- video & signature session storage 4 (Step 24).
Figure 2B is a simplified flowchart of the major process steps on an ad- candidate identification unit for providing broadcast ad identification, according to preferred embodiments of the present invention. The process starts when ad-candidate identification unit 6 receives the stored session signatures from video server 2. The stored session signatures are analyzed (Step 26), and ad candidates associated with repeating segments within and across channels are detected (Step 28). Ad-candidate identification unit 6 then determines whether the ad candidates are known ads (Step 30).
If the ad candidates aren't known ads, the ad candidates are stored in ad- candidate session storage 8 (Step 32). If the ad candidates are known ads, the ad candidates are transferred to ad identification engine 10.
Figure 2C is a simplified flowchart of the major process steps on an ad identification engine for providing broadcast ad identification, according to preferred embodiments of the present invention. The process starts when ad-candidate identification unit 6 transfers the ad candidates that are known ads to ad identification engine 10. Ad identification engine 10 updates the ad broadcast statistics, stored in ad storage repository 14, and used by ad-identifier module 12 (Step 34). Ad identification engine 10 then provides the pre-indexed ad-identification service to ad monitoring unit 16 for real-time ad broadcast detection (Step 36).
Figure 2D is a simplified flowchart of the major process steps on an annotation site for providing broadcast ad identification, according to preferred embodiments of the present invention. The process starts when the ad candidates are retrieved from ad-candidate identification unit 6 by annotation site 18 (Step 38). Corresponding video segments and/or additional metadata are retrieved (Step 40), and any metadata is processed (Step 42). The new ads are then stored (Step 44), and the ad candidates and video are removed from storage (Step 46).
Figure 3A is a simplified flowchart of the major process steps on a video server for providing broadcast ad identification, according to alternate embodiments of the present invention. In alternate embodiments, video server 2 receives a broadcast video feed (Step 50). Session signatures are then generated (Step 52) as in the process of Figure 2A. Video server 2 then determines whether the ad candidates are known ads (Step 54).
If the ad candidates aren't known ads, the ad candidates are stored in broadcast- video & signature session storage 4 (Step 56). If the ad candidates are known ads, the ad candidates are transferred to ad identification engine 10 for further processing as in Figure 2C.
Figure 3B is a simplified flowchart of the major process steps on an ad- candidate identification unit for providing broadcast ad identification, according to alternate embodiments of the present invention. The process starts when ad-candidate identification unit 6 receives the stored session signatures from video server 2. The stored session signatures are analyzed (Step 58), and ad candidates associated with repeating segments within and across channels are detected (Step 60) as in the process of Figure 2B. The ad candidates are then stored in ad-candidate session storage 8 (Step 62), and the ad candidates are retrieved by annotation site 18 for further processing as in Figure 2D. While the present invention has been described with respect to a limited number of embodiments, it will be appreciated that many variations, modifications, and other applications of the present invention may be made.

Claims

WHAT IS CLAIMED IS:
1. A method for providing broadcast ad identification, the method comprising the steps of:
(a) providing fingerprint signatures of each frame in a broadcast video for ad identification; and
(b) designating at least two repeat fingerprint signatures upon detecting at least one fingerprint-signature match from said fingerprint signatures.
2. The method of claim 1, the method further comprising the step of:
(c) prior to said step of designating, determining whether said fingerprint signatures correspond to a known ad based upon detecting at least one fingerprint-signature match of said fingerprint signatures with pre-indexed fingerprint signatures of pre-indexed ads.
3. The method of claim 1, the method further comprising the step of:
(c) creating segments of said fingerprint signatures, ordered according to a timeline temporal proximity of said fingerprint signatures, by grouping at least two said fingerprint signatures based on a repeat temporal proximity of said at least two repeat fingerprint signatures respective of said at least two fingerprint signatures.
4. The method of claim 3, the method further comprising the step of: (d) detecting at least one ad candidate based on an occurrence of at least one repeat segment.
5. The method of claim 4, wherein said step of detecting said at least one ad candidate is checked for consistency based on a segment-to-segment mapping.
6. A system for providing broadcast ad identification, the system comprising:
(a) a server including:
(i) a CPU for performing computational operations; and
(ii) a memory module for storing data; and
(b) a processing module, residing on said server, configured for:
(i) processing fingerprint signatures of each frame in a broadcast video for ad identification; and
(ii) designating at least two repeat fingerprint signatures upon detecting at least one fingerprint-signature match from said fingerprint signatures.
7. The system of claim 6, wherein said processing module is further configured for:
(iii) prior to said designating, determining whether said fingerprint signatures correspond to a known ad based upon detecting at least one fingerprint-signature match of said fingerprint signatures with pre-indexed fingerprint signatures of pre-indexed ads.
8. The system of claim 6, wherein said processing module is further configured for:
(iii) creating segments of said fingerprint signatures, ordered according to a timeline temporal proximity of said fingerprint signatures, by grouping at least two said fingerprint signatures based on a repeat temporal proximity of said at least two repeat fingerprint signatures respective of said at least two fingerprint signatures.
9. The system of claim 8, wherein said processing module is further configured for:
(iv) detecting at least one ad candidate based on an occurrence of at least one repeat segment.
10. The system of claim 9, wherein said detecting said at least one ad candidate is checked for consistency based on a segment-to-segment mapping.
EP13785191.1A 2012-04-01 2013-04-16 Methods and systems for providing broadcast ad identification Withdrawn EP2850838A4 (en)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
PCT/IB2012/051584 WO2012131653A2 (en) 2011-03-31 2012-04-01 Devices, systems, methods, and media for detecting, indexing, and comparing video signals from a video display in a background scene using a camera-enabled device
US201261669132P 2012-07-09 2012-07-09
US201313821983A 2013-03-10 2013-03-10
US13/832,083 US20140013352A1 (en) 2012-07-09 2013-03-15 Methods and systems for providing broadcast ad identification
PCT/IL2013/050328 WO2013164817A1 (en) 2012-04-01 2013-04-16 Methods and systems for providing broadcast ad identification

Publications (2)

Publication Number Publication Date
EP2850838A1 true EP2850838A1 (en) 2015-03-25
EP2850838A4 EP2850838A4 (en) 2016-01-20

Family

ID=52469544

Family Applications (1)

Application Number Title Priority Date Filing Date
EP13785191.1A Withdrawn EP2850838A4 (en) 2012-04-01 2013-04-16 Methods and systems for providing broadcast ad identification

Country Status (4)

Country Link
EP (1) EP2850838A4 (en)
HK (1) HK1203259A1 (en)
IN (1) IN2014DN11195A (en)
WO (1) WO2013164817A1 (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9369780B2 (en) * 2014-07-31 2016-06-14 Verizon Patent And Licensing Inc. Methods and systems for detecting one or more advertisement breaks in a media content stream
CN104967894B (en) * 2014-09-04 2019-01-08 腾讯科技（深圳）有限公司 The data processing method and client of video playing, server
US11528525B1 (en) * 2018-08-01 2022-12-13 Amazon Technologies, Inc. Automated detection of repeated content within a media series
US11037304B1 (en) 2018-09-10 2021-06-15 Amazon Technologies, Inc. Automated detection of static content within portions of media content
CA3162906A1 (en) 2021-06-15 2022-12-15 Evertz Microsystems Ltd. Method and system for content aware monitoring of media channel output by a media system

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6469749B1 (en) * 1999-10-13 2002-10-22 Koninklijke Philips Electronics N.V. Automatic signature-based spotting, learning and extracting of commercials and other video content
US20120114167A1 (en) * 2005-11-07 2012-05-10 Nanyang Technological University Repeat clip identification in video data
US20080066098A1 (en) * 2006-08-25 2008-03-13 Skyclix, Inc. Phone-based targeted advertisement delivery
GB2460844B (en) * 2008-06-10 2012-06-06 Half Minute Media Ltd Automatic detection of repeating video sequences
JP2012500585A (en) * 2008-08-18 2012-01-05 アイファロ メディア ゲーエムベーハー Supplementary information distribution
US9264758B2 (en) * 2009-09-14 2016-02-16 Tivo Inc. Method and an apparatus for detecting media content recordings

Also Published As

Publication number Publication date
WO2013164817A1 (en) 2013-11-07
HK1203259A1 (en) 2015-10-23
EP2850838A4 (en) 2016-01-20
IN2014DN11195A (en) 2015-10-02

Similar Documents

Publication Publication Date Title
US20140013352A1 (en) Methods and systems for providing broadcast ad identification
CN100531362C (en) Method for marking and parting repeating objects embedded in a stream
CA2906192C (en) Systems and methods for real-time television ad detection using an automated content recognition database
US10614137B2 (en) System and method for detecting repeating content, including commercials, in a video data stream
EP1138151B1 (en) Automatic signature-based spotting, learning and extracting of commercials and other video content
US9628836B1 (en) System and method for removing erroneously identified TV commercials detected using automatic content recognition
US10462536B2 (en) System for low-latency detection of known audio video content using audio fingerprinting and audio watermarking
US20140278845A1 (en) Methods and Systems for Identifying Target Media Content and Determining Supplemental Information about the Target Media Content
US20140282673A1 (en) Systems and methods for real-time television ad detection using an automated content recognition database
EP2982131B1 (en) Systems and methods for real-time television ad detection using an automated content recognition database
US20150082349A1 (en) Content Based Video Content Segmentation
EP3776263B1 (en) System and method for detecting repeating content, including commercials, in a video data stream using audio-based and video-based automated content recognition
US20060271947A1 (en) Creating fingerprints
EP2850838A1 (en) Methods and systems for providing broadcast ad identification
Berrani et al. A non-supervised approach for repeated sequence detection in TV broadcast streams
JP2018530273A (en) Common media segment detection
US10965965B2 (en) Detecting of graphical objects to identify video demarcations
EP3140991B1 (en) Synchronizing broadcast timeline metadata
US10805681B2 (en) System and method for detecting unknown TV commercials from a live TV stream
JP6082716B2 (en) Broadcast verification system and method
US10178415B2 (en) Chapter detection in multimedia streams via alignment of multiple airings
GB2444094A (en) Identifying repeating video sections by comparing video fingerprints from detected candidate video sequences
US10945030B2 (en) Detection of potential commercial by detection and analysis of transitions in video content
CN107844578B (en) Method and device for identifying repeated segments in audio stream
CN110929095A (en) Video abstract playback method and device, electronic equipment and readable storage medium

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20150202

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAX Request for extension of the european patent (deleted)
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 1203259

Country of ref document: HK

RA4 Supplementary search report drawn up and despatched (corrected)

Effective date: 20151217

RIC1 Information provided on ipc code assigned before grant

Ipc: H04N 21/8358 20110101ALI20151211BHEP

Ipc: H04H 60/37 20080101AFI20151211BHEP

Ipc: H04N 21/8352 20110101ALI20151211BHEP

Ipc: H04N 21/81 20110101ALI20151211BHEP

Ipc: H04H 20/14 20080101ALI20151211BHEP

Ipc: H04N 21/234 20110101ALI20151211BHEP

Ipc: G06F 17/30 20060101ALI20151211BHEP

Ipc: H04N 21/845 20110101ALI20151211BHEP

Ipc: H04H 60/59 20080101ALI20151211BHEP

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN

18D Application deemed to be withdrawn

Effective date: 20160722

REG Reference to a national code

Ref country code: HK

Ref legal event code: WD

Ref document number: 1203259

Country of ref document: HK