EP1151614A1

EP1151614A1 - Apparatus and method for video signal watermarking

Info

Publication number: EP1151614A1
Application number: EP00902787A
Authority: EP
Inventors: Jia Hong Yin
Original assignee: Central Research Laboratories Ltd
Current assignee: Central Research Laboratories Ltd
Priority date: 1999-02-11
Filing date: 2000-02-11
Publication date: 2001-11-07
Also published as: KR20010102075A; AU2452600A; GB2347295A; JP2002537697A; WO2000048401A1; GB9903126D0

Abstract

An apparatus for, and method of, encoding information into and decoding information from, a sequence of moving images, such as video images. It is known to insert codes into images for purposes of identifying the owner of the images. However, existing schemes have suffered from the disadvantage that certain data compression techniques have removed or degraded the code so that the code cannot be later recognised. The present invention solves the problem by identifying moving and static portions of an image in an image sequence and providing two separate channels, one for the moving portion of an image and the other for a static portion of an image. One or more codes are then inserted into both moving and static image channels, so that moving image sequences including the code are, preserved even after image compression or low pass filtering.

Description

APPARATUS AND METHOD FOR VIDEOSIGNAL WATERMARKING

The present invention relates to an apparatus for, and method of, encoding information into and decoding information from a sequence of moving images. The technique of coding information into a sequence of video or cinematographic images is also known as electronic watermarking. The invention is therefore particularly, but not exclusively, suitable for electronic watermarking all types of media having moving images stored thereon or involving the transmission or broadcast of moving image data.

In video broadcasting, data can be in either digital or analogue form. Increasingly data is transmitted in digital form. One reason for this is because larger amounts of data can be transmitted in digital form than in analogue form. Another is because digital signals are less prone to interference and can easily be recovered by using error correction techniques if received images are distorted. However, as side effects of facile access to digital data, complaints of copyright infringement and for tampering or modification of content are increasing.

It is known to insert codes into images for the purpose of identifying the owner of the images. The images may be in the form of video transmissions, or video clips or stills, for example for transmission across a telecommunication link. There are a number of known schemes for inserting identification codes into so called 'sync' periods in transmitted video images. More recently it has been proposed to insert identification codes into the image itself in such a way that the code cannot be detected by a human eye. However, such schemes may suffer from the disadvantage that low pass filtering and other image processes, such as data compression, may remove the code or degrade it to an extent that it cannot be recognised.

UK Patent Application GB-A-2 305 803 (Philips) describes a method of correcting errors which occur in a telecine scanning process. The method involves sampling an area of an image which includes a structure and comparing this sampled structure with subsequently sampled structures obtained from subsequent image frames. Use of motion vectors obtained is then made to correct picture steadiness.

All current schemes suffer from the disadvantage that coded digital information cannot easily be transformed and maintain its integrity. Low pass filtering and other processes, such as data compression, may occur as a result of image compression algorithms or transmission of audio signals across a telecommunication link. Such transforms may remove the code or degrade it to an extent where it cannot be recognised.

In the Applicant's published International Patent Application WO-A 1-9625005 (Todd), there is described a method of coding data into an image. The method of coding and decoding information into an image, comprises: dividing the image into MxN blocks, selectively encoding (decoding) information into selected blocks in such a way as not to be visible to a human eye, wherein in a decoding stage, the decoding is synchronised to the occurrence of the blocks for analysis of image information. In a preferred embodiment the size of insertions and their positions are fixed by processing the image in a block-by- block manner, typically with a block size of 8 by 8 pixels.

In MPEG compression standards, temporal redundancy in image sequences is reduced by block-based motion compensation. Any change in an image from its previous frame caused by object movement is reserved to recover the image in the process of image decompression. Whilst previous encoding techniques have been successful, sometimes embedded codes have not survived MPEG compression.

The present invention arose to overcome this and associated problems.

According to the present invention there is provided a method of encoding information into a sequence of images; the sequence of images comprising a first image and a subsequent image; the method comprising the steps of: locating an edge in the first image, locating a corresponding edge in a subsequent image, comparing relative positions of the corresponding edges in said images, thereby identifying at least one relatively moving edge; inserting a first code into said moving edge and inserting a second code into said static edge. A corresponding apparatus is also provided.

The invention thus solves the problem of preserving both static and moving encoded information during subsequent image compression. Another advantage of the present invention over the arrangement described in UK Patent Application GB-A-2305 803 (Philips) is the fact that the present invention is capable of diagrammatically encoding watermark code into a signal to be stored, or broadcast.

Preferably the apparatus and method identify moving, and therefore static edges, in an image sequence and treat moving and static edges as separate channels each carrying its own code(s). Hence identification information is encoded into both moving edges (which are not compressed to any extent) and static edges independently. Both channels may carry identical codes if required. Alternatively different codes may be inserted into moving or static edges. The code insertion technique of the present invention may be used to embed codes in moving edge pixels.

Means is advantageously provided to preselect said relative moving edges so that a code can be inserted into switchable sequences. If there is not movement in a sequence of images, no code is inserted and a different sequence may be selected.

Preferably coded information is inserted into an image in so called strongly featured regions of an image in such a way that the code is resistant to image compression and/or low pass filtering, but is not visible to the human eye. Examples of strongly featured regions of images are textured regions or lines, or edges between two regions of different luminance or contrast. In such regions, it is possible to insert a relatively large amount of information without significantly altering the image.

Information encoded into an image may be used for a variety of purposes, for example:

i) to insert copyright or identification information into video clips or still images; ii) to monitor when advertisements or films are played in broadcasts, for monitoring royalty payment purposes; iii) to identify a master copy of a data storage medium, such as a CD or DVD or video disc or similar medium, from which pirated copies may be produced.

Codes are preferably inserted in edges within an image. Edge regions are known to have masking properties because of the way the human visual system works. In particular, the local orientation of edges are important, and there are specific structures in the primary visual cortex of the human brain, for detecting the presence of an edge and its local orientation.

Coded information is preferably inserted into an image so that it does not alter the local orientation of certain features. The insertions are preferably made along the length of a local section of edge. The insertions are preferably made as a 2D function, by using for example an ellipse which is aligned to the local orientation of the edge.

According to another aspect of the present invention there is provided apparatus for decoding information from an encoded moving image sequence, comprising: means for identifying a region in an image in the sequence, means for determining whether the said region is a moving region or static region and means for recovering code from at least said moving image region.

Corresponding to this further aspect of the invention, there is also provided a method of decoding information from an encoded moving image sequence. There is correspondingly also provided a method of encoding information into, and decoding information from, a moving image sequence, having one or more of the aforementioned preferred features.

Embodiments of the invention will now be described, by way of examples only, with reference to the Figures, in which:-

Figure 1 shows diagrammatically an embodiment of an encoder according to the invention;

Figure 2 shows an embodiment of a decoder according to the invention; Figure 3 shows a sketch with edges of images in bold;

Figure 4 shows the sketch of Figure 3, with only moving edges highlighted in bold;

Figure 5 shows diagrammatically an embodiment of the invention, incorporated in a surveillance camera;

Figure 6 is a diagrammatical view of a supermarket in which the surveillance camera of Figure 5 may be used;

Figure 7 is an example of an image of pedestrian flow through a supermarket check-out area; and

Figure 8 is a vector diagram representative of the pedestrian flow of Figure 7.

Referring to the Figures, Figures 1 and 2 show block diagrams of an encoder 10 and decoder 20 respectively. The invention will now be described in the encoding sequence with reference to Figure 1. The signal is split into two portions: a moving image signal 22 and a static image signal 23. This can be according to a standard MPEG protocol or a different proprietary image analysis (compression) protocol.

A first code is inserted into the static image signal and a second code is inserted into the moving image signal. These signals are then transmitted via separate channels.

The moving edge encoding sequence is now described with reference to the following Equations 1 to 3 and Figures 3 and 4.

Moving edges and static edges may be defined as the following:

Let (x,y), h(x,y) and (x,y) denote three consecutive images in an image sequence. Let E (x,y) be an edge image from image h(x,y). The moving edge image ME(x,y) is defined as follows: ME(x,y) = D₁₂(x,y).E₂(x,y).D23(x,y) - (Eqn 1) where D₁₂(x,y) = \ (x,y)-I₂(x,y)\, D₂₃(x,y) = \h(x,y) - h(x,y)\. - (Eqn 2)

The static edge image SE(x,y) may be obtained using the following equation:

SE(x,y) = / E₂(x,y), ifME(x,y) = 0

Once inserted the coded information is treated the same as a moving image by any compression algorithm or protocol such as MPEG. Static code is included in early sequences of a series of encoded images so that when compressed the encoded information remains in tact.

Brief reference will now be made to Figure 2, which depicts a decoder 20. Decoder 20 receives encoded images signal 24. The image signal 24 is split into a moving image signal 26. Edge diffraction then occurs. Edge detector 27 detects edges in moving images. Edge detector 28 detects edges in static images. Detected edge signals 29 and 30 are subtracted one from another at subtractor 31 and a static edge is decoded at static edge decoder 32. Moving edges are decoded directly from a signal presented by moving edge detector 27 to the moving edge decoder 33. Static edge decoded signals 34 and moving edge decoded signals 35 are added at 36 to provide an electronic watermark signal which indicates for example, the origin of a signal or the owner of copyright in an image sequence or piece of video footage.

The invention may be used to encode information onto all forms of recording media.

These may include videotape, video disc, compact disc (CD or DVD), or any other form of video storage medium. Similarly the invention may be incorporated into video broadcasting systems, video editing equipment, video monitoring equipment, televisions, computers or any other piece of electronic equipment used to produce or view video images, including a video cassette recorder and/or a set-top box.

The invention has been described, by way of example only, and it will be understood that variation may be made to the embodiments described without departing from the scope of the invention. In an alternative embodiment, shown in Figures 5 to 8, the invention may be deployed within shops or other public places in order to monitor activity of purchasers or members of the public. The invention allows for the automation of data capture relating to motion of, say, shoppers and optionally, its subsequent encoding for transmission.

Figure 6 represents the layout of a typical supermarket. The system to be implemented uses real-time video data from appropriately positioned Close Circuit Television (CCTV) cameras to monitor the customers arriving at entrance 102, check-outs 106 and exit 104. The system uses information extracted from video data to estimate net arrival/departure rate of customers and from this maintains an estimate of the number of customers inside the store at any time. This together with the information from the aforementioned areas enables the system to maintain estimates of the number of customers:

a) within the store shelf area b) queuing at the check-outs

In addition a correlation between the number of customers arriving at the check-outs and the number of customers arriving at the store at some earlier time allows the system to adaptively optimise parameters of a prediction model of the store. This optimised prediction model enables the system to estimate impending customer demand for checkouts in advance of the need becoming apparent. This allows for timely staff redeployment to avoid any reduction in customer service.

Output from the system is preferably in two main forms:

1. Real-time data concerning conditions in the store as they happen. This data in turn may be presented either a) On a screen in some management area, or b) As immediately useful instructions relayed through radio headsets. These instructions need to be unambiguous. It is envisaged at this stage that the instructions consist of messages to open or close a number of check-outs.

2. The system will be capable of producing a statistical report concerning the arrival rate of customers and the length of time they stay in the store. This data will be suitable for planning overall staff levels and will be a useful source of marketing information.

Imaging equipment may be used in conjunction with a microprocessor dedicated to determining length of queues or regions of a store visited by an unusually large or small number of customers. This may be particularly advantageous in predicting employee allocation at check-outs for example. Similarly the system may be configured to monitor automatically a specific region of a store or even a specific aisle or display. By adapting control and analytical software the system may be arranged to provide a statistical assessment of how popular or unpopular a particular type of marketing or location of a product is performing.

Claims

Apparatus for encoding information into a moving image sequence, comprising: means for identifying a region in an image in the sequence, means for determining whether the said region is a moving or static region and means for inserting a coded into at least said moving image region.

2. Apparatus for decoding information from an encoded moving image sequence, comprising: means for identifying a region in an image in the sequence, means for determining whether the said region is a moving or static region and means for recovering code from at least said moving image region.

3. Apparatus according to claim 1 or 2 wherein means is provided to identify moving and static edges and signals representative of said moving and static image regions are sent via first and second channels so that code is inserted into, or recovered from, either or both of said channels.

4. Apparatus according to claim 1, or claim 3 when dependant on claim 1, wherein a first code is inserted into signals in a first channel and a second code is inserted into signals in said second channel.

5. Apparatus according to claim 4 wherein the first and second codes are identical.

6. Apparatus according to claim 4 wherein the first and second codes are different.

7. Apparatus according to any claim 1 wherein code is inserted into a moving image region of a moving image sequence in such a way that the code is resistant to image compression.

8. Apparatus according to claim 1 wherein code is inserted into a region of a moving image so that the code is resistant to low pass filtering.

. Apparatus according to claims 1, 7 or 8 wherein the code is inserted into boundaries between regions of different luminance, chrominance or contrast.

10. Apparatus substantially as herein described and with reference to the Figures.

11. Apparatus according to any of claims 1 to 9 which is included in video recording, video broadcasting, video viewing equipment; or a television receiving or a set-top box.

12. Apparatus according to any of claims 1 to 9, including a digital imager, arranged to obtain images of moving objects on a static background and to transmit signals indicative of the objects to a location remote from said imager.

13. Apparatus according to claim 12 operating under control of software for statistically analysing said code, so as to provide an indicator of the amount of movement of the objects during a chosen time interval.

14. A method of encoding information into, or decoding information from, a moving image sequence using the apparatus of claims 1 to 9.

15. A method of encoding information into a sequence of images; the sequence of images comprising: a first image and a subsequent image, the method comprising the steps of: locating an edge in the first image, locating a corresponding edge in the subsequent image, comparing relative positions of the corresponding edges in said first and second images, thereby identifying either a relatively static edge or two relatively moving images; inserting a first code into said moving edge and inserting a second code into said static edge.

16. A medium storing a video or cinematographic image or sequence of images thereon, characterised in that a code has been inserted into selective portions of said images, according to the method of claim 14, when dependant on claims 1 or claims 4 to 9, or claim 15.