WO2001013642A1

WO2001013642A1 - Watermarking data streams at multiple distribution stages

Info

Publication number: WO2001013642A1
Application number: PCT/US2000/022144
Authority: WO
Inventors: Dinei A. F. Florencio; Michael Tinker
Original assignee: Sarnoff Corporation
Priority date: 1999-08-12
Filing date: 2000-08-11
Publication date: 2001-02-22

Abstract

Watermarks are added to data streams (e.g., video streams) to enable improved detection and thereby prevention of the theft of such data streams. In one embodiment, different watermarks are added at two or more different stages along a primary distribution path in order to determine at what location a particular copy of the video stream diverged, if at all, from the primary distribution path prior to the end of the path. In this way, a particular copy of a video stream can be analyzed to determine who was the intended end user of the video stream. These features can be implemented to aid in the detection of content theft and, through related deterrent effects, aid in the prevention of such theft in the first place.

Description

WATERMARKING DATA STREAMS AT MULTIPLE DISTRIBUTION STAGES

BACKGROUND OF THE INVENTION Field of the Invention The invention relates to data processing and, more particularly, to digital watermarking of data streams, such as digital image sequences.

Cross-Reference to Related Applications

The subject matter of this application is related to U.S. patent application no. 09/001,205 ("the '205 application") filed 12/30/97 as attorney docket no. 12602, the teachings of which are incorporated herein by reference. The '205 application describes techniques for inserting a watermark into a compressed video bitstream without first having to fully decode the bitstream.

Description of the Related Art With the advent of digitization of images, digital image distribution, and digital video availability, copyright protection of such digital imagery has become a substantial issue for image publishers and authors. One technique used to identify digital video ownership is to embed into an image sequence a digital "watermark" containing information identifying the owner. Such watermarks should (a) be secure and robust to intentional corruption and to compression processing, (b) not be unreasonably complex to embed and extract, and (c) be compatible and interoperable with conventional image processing systems. For authentication applications, the watermark is generally invisible to a viewer of a decoded image sequence. However, in some applications, it is desirable to produce a visible watermark that can be removed by an authorized image decoder, but cannot be removed by an unauthorized decoder. Illegal copying of video content is a serious problem. Such copying may occur at any of various different stages along the distribution path of video content as it is transmitted from the original producer of the video content to the final end user (i.e., the viewer) of that content.

SUMMARY OF THE INVENTION The present invention is directed to techniques for using digital watermarking to improve the ability to detect where along the distribution path from producer to viewer illegally obtained video content was pirated. As such, the present invention may provide a deterrent effect that will aid in the prevention of such content theft in the first place. According to one embodiment, the present invention is a method for processing a data stream, comprising the steps of adding a plurality of different watermarks to the data stream at a plurality of different stages along a primary distribution path for the data stream thereby enabling detection of a location along the primary distribution path of divergence of the data stream from the primary distribution path by analyzing the watermarks in the data stream.

According to an alternative embodiment, the present invention is a method for processing a data stream, comprising the steps of (a) analyzing the data stream to identify a set of one or more different watermarks contained in the data stream, wherein each different watermark corresponds to a different stage along a primary distribution path for the data stream; and (b) determining where the data stream diverged from the primary distribution path based on the identified set of watermarks.

BRIEF DESCRIPTION OF THE DRAWINGS Other aspects, features, and advantages of the present invention will become more fully apparent from the following detailed description, the appended claims, and the accompanying drawings in which Fig. 1 shows a set of four different distribution paths for video streams, according to one embodiment of the present invention.

DETAILED DESCRIPTION Fig. 1 shows a set of four different distribution paths for video streams, according to one embodiment of the present invention. Each distribution path begins with the same three stages: video source 102, encoder 104, and storage device 106. Video source 102, which may be a digital video camera or other suitable device, generates the original video stream in a pixel domain. Encoder 104 applies a suitable digital video compression algorithm to generate a compressed video bitstream for the video stream, which bitstream is then stored on storage device 106. At appropriate times, the compressed video bitstream is read from storage device 106 for transmission by an appropriate transmission device 108 to a decoder 110, where the compressed video bitstream is decoded back into a pixel domain for display on a suitable display device 112 (e.g., a television set or computer monitor).

As shown in Fig. 1, at each stage along the four different distribution paths, a different watermark (i.e., one of watermarks WM#1 through WM#13) is added to the video stream. In general, a watermark may contain any useful information, but will preferably contain at least (1) a stage ID that identifies the stage at which the watermark was added and (2) a time tag that identifies the time and date at which the watermark was added to the video stream. Video source 102, encoder 104, each decoder 110, and each display device 112 all preferably add their corresponding watermarks into the video stream in the pixel domain, while storage device 106 and each transmission device 108 all preferably add their corresponding watermarks into the corresponding compressed video bitstream in the compressed domain, e.g., relying on the teachings in the '205 patent application.

After completely traversing one of the four distribution paths in Fig. 1, a particular copy of the video stream will have one of the following four sets of watermarks: o Distribution Path A consisting of video source 102, encoder 104, storage device 106, transmission device #1, decoder #1, and display device #1: {WM#1, WM#2, WM#3, WM#4, WM#6, and WM#10}; o Distribution Path B consisting of video source 102, encoder 104, storage device 106, transmission device #1, decoder #2, and display device #2: {WM#1, WM#2, WM#3, WM#4, WM#7, and WM#l l }; o Distribution Path C consisting of video source 102, encoder 104, storage device 106, transmission device #2, decoder #3, and display device #3: {WM#1, WM#2, WM#3, WM#5, WM#8, and WM#12}; and o Distribution Path D consisting of video source 102, encoder 104, storage device 106, transmission device #2, decoder #4, and display device #4: {WM#1, WM#2, WM#3, WM#5, WM#9, and WM#13}. Each of the four distribution paths shown in Fig. 1 may be referred to as a primary distribution path. It may be that the four primary distribution paths shown in Fig. 1 are the only proper (i.e., legitimate or authorized) paths for distribution of a particular video stream. A copy of the video stream that follows one of these four primary distribution paths from beginning (i.e., video source 102) to end (i.e., one of the four display devices 112) will have a corresponding one of the four sets of watermarks listed above.

If a copy of the video stream is diverted from one of the four primary distribution paths prior to reaching one of the four display devices 112, that copy will not have a complete set of watermarks, and the watermarks that are contained in the copy will help identify exactly where (i.e., after what stage) along one of the distribution paths the video stream was diverted. For example, if a particular copy of the video stream contains the set of watermarks consisting of {WM#1, WM#2, WM#3, WM#4}, then it would be known that the video stream diverged from either Distribution Path A or Distribution Path B after being processed by transmission device #1 108. In this way, the present invention can be used to detect where along the distribution path from producer to viewer illegally obtained video content was pirated. As such, the present invention may also provide a deterrent effect that will aid in the prevention of such content theft in the first place.

In one possible implementation, each transmission device 108 could be a different microwave transmitter broadcasting a video stream corresponding to a pay-per-view or other controlled or regulated broadcast to a different set of authorized viewers. If someone were improperly capturing and illegally distributing copies of the pirated video stream, an analysis of one of the pirated copies containing the watermarks {WM#1, WM#2, WM#3, WM#4} could at least identify that the theft occurred during a particular microwave transmission of the video stream somewhere within the coverage area of transmission device #1 at a known date and time. This information may be very helpful in tracking down the individuals responsible for the content theft and illegal distribution.

The exact types of watermarks that are added at the various stages may determine whether — and what kind of — special equipment is needed to perform the analysis of the video stream to determine which watermarks are present. In preferred embodiments, the watermarks are not visible to a viewer when the decoded video stream is played back at normal speeds. This can be achieved, for example, by placing watermarks only into a known subset of frames in the video stream (e.g., as few as a single frame) and/or by encoding the watermark information so that it is only decipherable by special equipment implementing proprietary decode processing.

Note that the processing along any one distribution path in Fig. 1 could be continuous in time, where all stages process different portions of the video stream at the same time (perhaps bypassing storage device 106), or discontinuous in time, where at least one stage (e.g., storage device 106) completes its processing of the video stream before the next stage (e.g., a transmission device 108) begins its processing. The distribution paths shown in Fig. 1 are merely exemplary; it will be understood that the present invention can be applied to any suitable distribution path having a plurality of different processing stages, including suitable types of processing stages not shown in Fig. 1. For example, a particular distribution path might include a video retailer who sells or rents video content that contains a watermark identifying the video retailer. Such a watermark could be added by the video wholesaler who distributes copies of the video content to the video retailer. Alternatively, the watermark could be added by the video retailer at the time of sale/lease by the customer, thereby adding the ability to include a more relevant time tag into the watermark. The present invention is not limited to video streams, but can probably be applied to the distribution of any other type of data stream as well, in order to determine the source of illegal copying of that data stream. When applied to a video stream, a watermark corresponds to any of the standard variety of techniques for encoding watermark information into a video stream, including those techniques in which the watermark is virtually — if not completely ~ undetectable to viewers during normal playback of the video stream. When applied to other types of data streams, such as audio data, text data, or any other suitable type of data, a "watermark" corresponds to information encoded into those data stream in a manner analogous to that used for video streams. For example, when applied to audio streams, a "watermark" may correspond to information encoded into audio stream in a manner such that the information is imperceptible to a listener during normal audio playback of the audio stream. Similarly, when applied to streams of text, a "watermark" may correspond to information encoded into the text stream in a manner such that the information does not appear on either (a) the graphical display presented to the user or (b) the corresponding printed hardcopy. In all such cases, including that of video watermarking, special processing equipment may be needed to analyze the data stream to detect and identify the different watermarks that are in fact present in the stream to determine if and where the data stream diverged from a primary distribution path.

The present invention can be embodied in the form of methods and apparatuses for practicing those methods. The present invention can also be embodied in the form of program code embodied in tangible media, such as floppy diskettes, CD-ROMs, hard drives, or any other machine-readable storage medium, wherein, when the program code is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the invention. The present invention can also be embodied in the form of program code, for example, whether stored in a storage medium, loaded into and/or executed by a machine, or transmitted over some transmission medium or carrier, such as over electrical wiring or cabling, through fiber optics, or via electromagnetic radiation, wherein, when the program code is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the invention. When implemented on a general-purpose processor, the program code segments combine with the processor to provide a unique device that operates analogously to specific logic circuits. It will be further understood that various changes in the details, materials, and arrangements of the parts which have been described and illustrated in order to explain the nature of this invention may be made by those skilled in the art without departing from the principle and scope of the invention as expressed in the following claims.

Claims

CLAIMS What is claimed is: 1. A method for processing a data stream, comprising the steps of adding a plurality of different watermarks to the data stream at a plurality of different stages along a primary distribution path for the data stream thereby enabling detection of a location along the primary distribution path of divergence of the data stream from the primary distribution path by analyzing the watermarks in the data stream.

2. The invention of claim 1, wherein the data stream is a video stream.

3. The invention of claim 2, wherein, along the primary distribution path, at least one watermark is added to a compressed bitstream for the video stream without having to fully decode the compressed bitstream.

4. The invention of claim 3, wherein, at two or more different stages along the primary distribution path, two or more different watermarks are added to the compressed bitstream without having to fully decode the compressed bitstream.

5. The invention of claim 4, wherein at least one watermark is added to the video stream in an original pixel domain near a beginning of the primary distribution path.

6. The invention of claim 5, wherein at least one watermark is added to the video stream in a decompressed pixel domain near an end of the primary distribution path.

7. The invention of claim 6, wherein each watermark identifies a stage of the primary distribution path and a time tag corresponding to when processing of the data stream occurred at the corresponding stage.

8. The invention of claim 3, wherein at least one watermark is added to the video stream in an original pixel domain near a beginning of the primary distribution path.

9. The invention of claim 8, wherein at least one watermark is added to the video stream in a decompressed pixel domain near an end of the primary distribution path.

10. The invention of claim 1, wherein each watermark identifies a stage of the primary distribution path and a time tag corresponding to when processing of the data stream occurred at the corresponding stage.

11. A method for processing a data stream, comprising the steps of: (a) analyzing the data stream to identify a set of one or more different watermarks contained in the data stream, wherein each different watermark corresponds to a different stage along a primary distribution path for the data stream; and (b) determining where the data stream diverged from the primary distribution path based on the identified set of watermarks.

12. The invention of claim 11, wherein the data stream is a video stream.

13. The invention of claim 12, wherein, along the primary distribution path, at least one watermark was added to a compressed bitstream for the video stream without having to fully decode the compressed bitstream.

14. The invention of claim 13, wherein, at two or more different stages along the primary distribution path, two or more different watermarks were added to the compressed bitstream without having to fully decode the compressed bitstream.

15. The invention of claim 14, wherein at least one watermark was added to the video stream in an original pixel domain near a beginning of the primary distribution path.

16. The invention of claim 15, wherein at least one watermark was added to the video stream in a decompressed pixel domain near an end of the primary distribution path.

17. The invention of claim 16, wherein each watermark identifies a stage of the primary distribution path and a time tag corresponding to when processing of the data stream occurred at the corresponding stage.

18. The invention of claim 13, wherein at least one watermark was added to the video stream in an original pixel domain near a beginning of the primary distribution path.

19. The invention of claim 18, wherein at least one watermark was added to the video stream in a decompressed pixel domain near an end of the primary distribution path.

20. The invention of claim 11, wherein each watermark identifies a stage of the primary distribution path and a time tag corresponding to when processing of the data stream occurred at the corresponding stage.

21. A machine-readable medium, having encoded thereon program code, wherein, when the program code is executed by a machine, the machine implements the steps of: (a) analyzing the data stream to identify a set of one or more different watermarks contained in the data stream, wherein each different watermark corresponds to a different stage along a primary distribution path for the data stream; and (b) determining where the data stream diverged from the primary distribution path based on the identified set of watermarks.