EP2021976A2 - Application efficace de technologies de marquage video - Google Patents

Application efficace de technologies de marquage video

Info

Publication number
EP2021976A2
EP2021976A2 EP07783843A EP07783843A EP2021976A2 EP 2021976 A2 EP2021976 A2 EP 2021976A2 EP 07783843 A EP07783843 A EP 07783843A EP 07783843 A EP07783843 A EP 07783843A EP 2021976 A2 EP2021976 A2 EP 2021976A2
Authority
EP
European Patent Office
Prior art keywords
information
media content
location
image
video
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP07783843A
Other languages
German (de)
English (en)
Other versions
EP2021976A4 (fr
Inventor
Niels Thorwirth
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Verimatrix Inc
Original Assignee
Verimatrix Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Verimatrix Inc filed Critical Verimatrix Inc
Publication of EP2021976A2 publication Critical patent/EP2021976A2/fr
Publication of EP2021976A4 publication Critical patent/EP2021976A4/fr
Withdrawn legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/0021—Image watermarking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition

Definitions

  • the invention relates generally to marking video, and more particularly to efficient application of marking technologies for digital video.
  • the information can, for example, be embedded during playback or reception by an individual, and relate to the time of reception and the individual recipient by means of an identification number. If the copy is later found distributed illegally, the information can be retrieved and the original recipient, who is responsible for the illegal distribution can be identified.
  • This technology can be used for tracking of individual media asset copies and enforcement of copyright laws.
  • Content marking is an important component of digital media distribution and enables digital delivery of copyright protected content by limiting the risk of illegal distribution e.g., through peer to peer file sharing sites. Because the distribution can be tracked to the last legal recipient, the risk of being held responsible for copyright violations increases, as the person distributing the content can be identified.
  • tracking number can be embedded using obvious and visible modifications, they are less destructive to the content and better secured against removal, if applied in an invisible fashion.
  • transformations allow for invisible marking modifications in a domain that remains largely intact after modifications to the video, and allows aggregation of modifications that are distributed over a number of frames when the media is observed. These transformations also enable a strong signal in a transformed domain to be embedded. The signal is largely imperceptible in the domain that is used to present the media to the user.
  • the information can be embedded at the receiving end, so that the sender is not burdened with marking and sending individual streams for every recipient.
  • the sender delivers the identical media copy to all recipients and the media is marked at the time it is received at the recipient's end. Thereafter, each user is presented with a copy that is individually marked.
  • the limitation of timely processing is even greater, since the machine on the receiving end often has very limited processing power available to apply the mark. This limitation of processing power makes it prohibitive to apply forensic marks that contain information about the recipient in media in most distribution environments in use today.
  • Systems and methods are described for rendering information to be embedded in media content at a first location and for embedding the rendered information into the media content at a second location, hi many embodiments, the embedding process is less processor intensive than the rendering process and can be performed on a consumer electronics device such as a set top box.
  • One embodiment of the invention includes rendering the information into an image at a first location and embedding the image in the media at a second location in order to achieve efficient marking of the media content.
  • the information embedded in media content is transformed from a frequency domain to a rendered image.
  • the information is modified according to perceptual characteristics of the media content.
  • the perceptual characteristics of the media content are determined from a compressed electronic file.
  • the information comprises metadata related to the media content.
  • the information identifies a time and location of media content playback.
  • the information identifies a copyright owner or recipient of the media content.
  • the information represents a database index.
  • the embedding of the image is accomplished using an on-screen display of a set top box.
  • the first location is a video delivery head end and the second location is a consumer electronics set top box receiving the video.
  • the image created in the first location is stored for repeated or subsequent use.
  • An embodiment of the invention includes a server configured to render the information at a first location and to communicate the rendered information to a device connected to the server via a network.
  • the device is configured to embed the image in the media content at a second location.
  • the network is a cable network and the device is a digital set top box.
  • the server is configured to transform the information that is to be embedded in the media content from the frequency domain to create the rendered image.
  • the server is configured to determine the perceptual characteristics of the content and the server is configured to modify the information according to the perceptual characteristics of the media content.
  • the server is configured to determine the perceptual characteristics of the media from a compressed electronic file.
  • the device includes hardware configured to generate on screen displays on an output device, the device is configured to embed the rendered information in the media content by displaying the rendered information as an on screen display.
  • the first location is a video delivery head end and the second location is a user site.
  • the server is configured to store the rendered image for subsequent use.
  • the device is configured to receive the media content via the network.
  • FIG. 1 is an illustrative flow diagram of a forensic marking process, according to one embodiment of the invention.
  • FIG. 2 is an illustrative diagram of a forensic mark being created as an image, converted into an overlay image, and subsequently applied to a video frame, thereby marking the video frame with the overlay image, according to one embodiment of the invention.
  • the present invention involves a system and method of rendering a forensic mark into a baseband, uncompressed, spatial transformation image or video frame, in which modifications to media content (video) are expressed in modifications to single pixels.
  • the application of these pre-processed representations of the mark can subsequently be applied very efficiently in a different location or system by a simple combination of the mark rendered into an image with the image or video frame to be marked.
  • the combination is applied via addition, in another via alpha blending that can be performed on a per pixel basis.
  • the actual manipulation can typically be adjusted with an alpha blending value that adjusts the strength of the overlaid graphic when combined with the underlying video display.
  • the common use for this on screen display is the display of menus and information about the video playing or as a user interface for other applications running in the device. This on screen display can be used to modify the media content/video in a way that allows embedding of imperceptible forensic marking information.
  • the application of the mark is further simplified by performing the processes of adding the mark by using an efficient image application method provided in many set top boxes, referred to as on screen display.
  • the on screen display adds a given image (in this case the transformed mark) with a given strength (defined by an "alpha value") to each frame of the video.
  • the preparation of the mark to be represented as an image can be performed once per movie, e.g., in the head end that delivers the content. In this case, the calculation has to be performed only once, but the data has to be transmitted to the location where the mark is embedded. Alternatively the preparation can occur in the location where it is applied although the processing resources might be limited.
  • Systems for forensic marking often use perceptual models that indicate the positions in space and time where modifications to the video are less visible.
  • the information to be embedded as a mark is varied according to the transmitted content. Examples of the use of perceptual properties and perceptual models in the embedding of marks are described in U.S. Patent Application Serial No. 11/489,754.
  • the perceptual properties can be derived from the compressed content in simple measurements such as compression ratio and bit rate.
  • the video to be sent is analyzed one time and the information of perceptibility in certain locations and frames is stored with the movie. While the movie is delivered to a device, the marking information is modified accordingly.
  • the overlay image that is applied to the video in accordance with many embodiments of the invention can be applied using an often existing alpha blending mechanism that allows the image or individual pixels to be applied in a translucent fashion, hi one embodiment of the invention this approach is used to allow very slight, invisible manipulation to the image, while embedding the mark.
  • the overlay image is opaque and is largely identical to the video frame that it is covering. It is updated with every frame. The typically slight difference between the image and the video that it is covering constitutes the mark.
  • Systems for marking media content in accordance with embodiments of the invention can require that the information to be embedded be transformed into a different domain, such as the frequency domain (e.g., DCT, wavelet, or Fast Fourier).
  • the transformation is required in order to prepare the information to be embedded, the information can generally be separated in a way that the transformation is only required once and the information can be assembled to create variations of different data to be embedded in a simple manner.
  • the modification is then applied in the spatial domain by addition of images to video frames. This can be applied by a remote device possessing less processing power
  • Many systems for marking media content in accordance with embodiments of the present invention embed static information that is independent of the frame content.
  • the transformation is applicable without knowledge of the underlying media that the modification is applied to.
  • Other systems require a modification applied to the actual media and the media content is modified according to its content.
  • These systems analyze the content before creating the overlay. The process of analyzing the video is only required to be performed once, while it can be applied multiple times for every copy that is marked with different information. A significant gain in performance is archived if the analysis is not performed on every frame but on every N th frame assuming that the video content does not change significantly over a short period of time.
  • a flow diagram of a forensic marking process is shown.
  • the typically compressed original video content 100 is used for distribution of several copies that should ultimately be marked with unique information.
  • the same file is digitally delivered 101 to the consumer electronic device of the recipients 145 where it is actually marked.
  • the file Prior to delivery, the file is analyzed and a perceptual model is determined that identifies locations of the video that are suitable to bide imperceptible information. This can be useful to reduce the required processing power at the receiving end.
  • the overlay image is prepared by applying the marking procedure and creating components that can be used to assemble the information to be embedded 104.
  • the modification required to embed the mark is derived 106.
  • the information is subsequently compiled to create an overlay image that can be applied to at least one of the video frames.
  • the preparation of the mark above and the following steps are applied in the consumer electronics device. In another embodiment, they are implemented before transmission and the result is transmitted to the consumer electronics device 145.
  • the consumer electronics device applies the overlay image to the video frame 150 to produce a combined frame of the uniquely marked video 160.
  • a diagram of a forensic mark being created as an image, converted into an overlay image and subsequently applied to a video frame is shown.
  • the information to be embedded in this example is "ABC 12 ⁇ " 201. It is assembled 210 from preprocessed marking information into an image representation 220.
  • the image representation holds the marking modifications that could be human readable or machine readable.
  • the image representation is converted 240 into an overlay image 250 with an alpha blending parameter that indicates the strength with which the image 220 is to be combined with a frame 260.
  • the combination 270 of the image representation 250 with the frame 260 to produce a marked video frame 280 is typically performed with an alpha blending operation that is defined as follows:
  • S x,y>f is the pixel at position x,y in frame number f of the marked video.
  • I x , y is the pixel at position x,y of the overlay image.
  • ⁇ Xjy alpha value indicating the strength of the overlay image at position x,y.
  • O x>yif is the pixel at position x,y in frame number f of the original unmarked video.
  • the embedding and extraction of the present invention is implemented in the form of apparatuses for practicing the previously described methods.
  • apparatuses may include, but are not limited to, set top boxes that receive, decode, and display video content, VHS tape players, DVD players, TV sets, video projectors, cameras, digital video camcorders, personal computers that process media data, hand held video playback devices, and personal organizers that process video (see for example the systems and set top boxes described in U.S. Patent Application Serial No. 11/489,754).
  • the present invention is implemented in the form of program code embodied in tangible media, discs, memory, or other machine-readable storage medium.
  • the present invention is embodied in the form of program code, whether stored in a storage medium, loaded into and/or executed by a machine, or transmitted over some transmission medium or carrier, such as over electrical wiring or cabling, through fiber optics, or via electromagnetic radiation.
  • the present invention is implemented as a circuit-based system.
  • various functions of circuit elements can be implemented as processing steps in a software program.
  • Such software may be employed in, for example, a digital signal processor, micro-controller, or general-purpose computer.
  • Variations, modifications, and other implementations of what is described herein may occur to those of ordinary skill in the art without departing from the spirit and scope of the invention. Accordingly, the invention is not to be defined only by the preceding illustrative description.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Artificial Intelligence (AREA)
  • Editing Of Facsimile Originals (AREA)
  • Image Processing (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

L'invention concerne des systèmes et des procédés pour restituer des informations destinées à être imbriquées dans un contenu multimédia à un premier emplacement, et pour imbriquer les informations restituées dans le contenu multimédia à un second emplacement. Dans plusieurs modes de réalisation, le processus d'imbrication suppose une charge moindre pour le processeur que le processus de restitution, et ledit processus d'imbrication peut être réalisé avec un appareil électronique grand public, tel qu'un boîtier décodeur, en utilisant des mécanismes de traitement existants. Un mode de réalisation de l'invention comporte la restitution des informations dans une image à un premier emplacement et l'imbrication de l'image dans le contenu multimédia à un second emplacement afin d'obtenir un marquage efficace du contenu multimédia.
EP07783843A 2006-05-17 2007-05-16 Application efficace de technologies de marquage video Withdrawn EP2021976A4 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US80094106P 2006-05-17 2006-05-17
PCT/US2007/069076 WO2007137091A2 (fr) 2006-05-17 2007-05-16 Application efficace de technologies de marquage vidéo

Publications (2)

Publication Number Publication Date
EP2021976A2 true EP2021976A2 (fr) 2009-02-11
EP2021976A4 EP2021976A4 (fr) 2010-09-29

Family

ID=38723999

Family Applications (1)

Application Number Title Priority Date Filing Date
EP07783843A Withdrawn EP2021976A4 (fr) 2006-05-17 2007-05-16 Application efficace de technologies de marquage video

Country Status (4)

Country Link
EP (1) EP2021976A4 (fr)
KR (1) KR101352830B1 (fr)
CN (1) CN101479742B (fr)
WO (1) WO2007137091A2 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108491774B (zh) * 2018-03-12 2020-06-26 北京地平线机器人技术研发有限公司 对视频中的多个目标进行跟踪标注的方法和装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20010054150A1 (en) * 2000-03-18 2001-12-20 Levy Kenneth L. Watermark embedding functions in rendering description files
US20030167392A1 (en) * 2000-06-16 2003-09-04 Fransdonk Robert W. Method and system to secure content for distribution via a network
US20050135656A1 (en) * 1994-11-16 2005-06-23 Digimarc Corporation Authentication of physical and electronic media objects using digital watermarks
US20060017747A1 (en) * 2004-07-26 2006-01-26 Dawson Thomas P Copy protection arrangement

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3773482B2 (ja) * 2002-10-30 2006-05-10 独立行政法人科学技術振興機構 改ざん検出方法、改ざん検出プログラム及びそのプログラムを記録した記録媒体
CN100367274C (zh) * 2005-06-24 2008-02-06 清华大学 一种在英文文本中嵌入和提取水印的方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050135656A1 (en) * 1994-11-16 2005-06-23 Digimarc Corporation Authentication of physical and electronic media objects using digital watermarks
US20010054150A1 (en) * 2000-03-18 2001-12-20 Levy Kenneth L. Watermark embedding functions in rendering description files
US20030167392A1 (en) * 2000-06-16 2003-09-04 Fransdonk Robert W. Method and system to secure content for distribution via a network
US20060017747A1 (en) * 2004-07-26 2006-01-26 Dawson Thomas P Copy protection arrangement

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of WO2007137091A2 *

Also Published As

Publication number Publication date
EP2021976A4 (fr) 2010-09-29
CN101479742A (zh) 2009-07-08
WO2007137091A2 (fr) 2007-11-29
CN101479742B (zh) 2016-10-19
WO2007137091A3 (fr) 2008-07-31
KR20090018108A (ko) 2009-02-19
KR101352830B1 (ko) 2014-01-17

Similar Documents

Publication Publication Date Title
KR102433684B1 (ko) 시간 기반 동적 워터마크를 생성하기 위한 시스템 및 방법
US7607017B2 (en) Embedding data in material
Swanson et al. Multimedia data-embedding and watermarking technologies
JP4785317B2 (ja) レンダリングコマンドとしてのトランスマーキング透かし埋め込み機能およびマルチメディア信号の特徴に基づく透かし埋め込み処理
US7757089B2 (en) Apparatus, method and computer program for distributing and rendering content
US7738677B2 (en) Efficient application of video marking technologies
WO2008014024A2 (fr) filigrane visible par un utilisateur
JP4542411B2 (ja) 媒体のデコード後透かし付けシステム
US7496197B2 (en) Method and system for robust embedding of watermarks and steganograms in digital video content
KR101677110B1 (ko) 제로널리지 기반의 영상 위변조 방지와 탐지를 위한 워터마킹 삽입과 추출 장치 및 그 방법
Su et al. A practical design of digital watermarking for video streaming services
Katariya Digital watermarking
Lakshmi et al. Digital video watermarking tools: an overview
Shirali-Shahreza A new method for real-time steganography
Verma Review of steganography techniques
EP2021976A2 (fr) Application efficace de technologies de marquage video
Maloo et al. Study of digital watermarking techniques for against security attacks
KR100659870B1 (ko) 이동 단말기 및 콘텐츠 서비스 방법
WO2010044102A2 (fr) Système manuel expert, unique et robuste à base de filigrane numérique visiblement non intrusif pour détection judiciaire du point de piratage (pop) d'un contenu vidéo numérique, protégé par le droit d'auteur
Chen et al. ACM Multimedia 2023 Grand Challenge Report: Invisible Video Watermark
Gangarde et al. Application of Video Watermarking Using Subdivision Technique to Enhance Information Security.
Iwamura A Benchmark Tool for Digital Watermarking
Ahuja et al. Video data security: analysis, relevance and open challenges
US20120134529A1 (en) Method and apparatus for applying of a watermark to a video during download
Bansal et al. Use of textual compression in steganography & cryptography

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20081121

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC MT NL PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA HR MK RS

RIN1 Information on inventor provided before grant (corrected)

Inventor name: THORWIRTH, NIELS

DAX Request for extension of the european patent (deleted)
A4 Supplementary search report drawn up and despatched

Effective date: 20100827

17Q First examination report despatched

Effective date: 20130213

17Q First examination report despatched

Effective date: 20130305

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN

18D Application deemed to be withdrawn

Effective date: 20181127