GB2369943A

GB2369943A - Perceptible modification of material

Info

Publication number: GB2369943A
Application number: GB0029852A
Authority: GB
Inventors: Jason Charles Pelly; Daniel Tapson
Original assignee: Sony United Kingdom Ltd
Current assignee: Sony Europe BV United Kingdom Branch
Priority date: 2000-12-07
Filing date: 2000-12-07
Publication date: 2002-06-12
Also published as: GB0029852D0

Abstract

Material (audio, video or data) (52), represented by digital samples is perceptibly modified, for example watermarked, which can protect from copyright theft. A selector selects a part or parts of the material in which sample values are to be modified (56). A processor (51) applies an invertible function, or algorithm (58), to the bits of the samples in the selected part or parts to modify the sample values, thereby perceptibly modifying the material. The total number of bits in the samples is kept constant so that the modified material has no more bits after modification than before.

Description

PERCEPTIBLE MODIFICATION OF MATERIAL FIELD OF THE INVENTION The present invention relates to the perceptible modification of material.

"Material"means one or more of video material, audio material and data material. Video, in this context, is generic to still and moving images.

BACKGROUND OF THE INVENTION It is known to apply watermarks to material such as video material, audio material and data material. A watermark may be imperceptible or perceptible in the material. Watermarks are typically embedded in material to identify the ownership of the material and/or to trace the provenance of the material to detect and protect against copyright infringement. Such a watermark comprises data which is embedded in the material.

Watermarks may be"robust"in that they are difficult to remove from the material. Robust watermarks are useful to trace the provenance of material which is processed in some way either in an attempt to remove the mark or to effect legitimate processing such as video editing or compression for storage and/or transmission.

Watermarks may be"fragile"in that they are easily damaged by processing which is useful to detect attempts to remove the mark or process the material.

In the case of images, a visible watermark is useful to allow e. g. a customer to view the image, e. g. over the Internet, to determine whether they wish to buy it but without allowing the customer access to the unmarked image they would buy. The watermark degrades the image and the mark is preferably not removable by the customer. Visible watermarks are also used to determine the provenance of the material into which they are embedded.

It is desirable perceptibly mark material in a way which is difficult to remove by an unauthorised person. It is also desirable to provide a mark which may be fully removed by an authorised person.

SUMMARY OF THE INVENTION

According to one aspect of the present invention there is provided an apparatus for generating a perceptibly modified material, which material is represented by digital samples, the apparatus comprising: a selector for selecting a part or parts of the material in which sample values are to be modified; and an encoder for applying an invertible function to the bits of the said samples in the selected part or parts so as to modify the sample values, thereby perceptibly modifying the material; wherein the modified material has no more bits after modification than before modification.

Hence, perceptibly marked material is provided. The marking is done by applying an invertible function to the samples. By using an invertible function. the marking may be completely removed by applying the inverse of the function, assuming that the material has not been processed in a manner which irreversibly changes the content of the material between modification and restoration.

It will be appreciated that if the material is video material, i. e. still or moving images, the samples are spatial domain samples also known as pixels. If the material is audio material the samples are time domain audio samples.

The modification is such that the modified sample values have no more bits after modification than before modification. That is the modification does not increase the number of bits but it may reduce the number of bits provided all the original bits can be restored by applying the inverse of the invertible function. That allows the modified material to be stored in the same data space, and/or transferred via a communications channel of the same bandwidth, as the unmodified material.

The invertible function, which is preferably not known to unauthorised people, makes it difficult to remove the modification without authorisation. The function may encrypt the bits of the samples. The function is for example a predetermined algorithm, which may: 1) shuffle bits of individual samples; or 2) encrypting those bits according to an encryption algorithm; or 3) shuffle the less bits of a group of samples amongst the samples. Other functions may be used.

The alteration of the material occurs only within a predetermined part or parts

of the material. In the case of an image the part (s) is/are defined by a map of the image. This allows such part (s) of the material to be altered as much as is desired but allows the material to be recognised from the unaltered parts. This allows the altered part (s) to be made more robust against processing which may inadvertently or deliberately remove or at least reduce the mark.

Preferably, the sample values have most significant bits (MSBs) and less significant bits (LSBs), and the encoder applies the said function to the LSBs but not to the MSBs. If the material is for example an image, the image can perceptibly altered or degraded. Because the LSBs are altered, but not the Most Significant bits (MSBs), the image is still recognisable provided the degree of alteration is chosen carefully.

In the example of audio material, the audio can be degraded in parts but may be recognised provided the degree of alteration is chosen carefully.

Other aspects and features of the present invention are defined in the appended claims.

BRIEF DESCRIPTION OF THE DRAWINGS For a better understanding of the present invention, reference will now be made by way of example to the accompanying drawings in which: Figure lA is a schematic diagram of an original image; Figure 1B is a schematic diagram of a bit map of a perceptible watermark ; Figure 1C is a schematic diagram of a combination of the perceptible watermark and the original image; Figure ID is a photograph of an actual example of a combination of a perceptible watermark and an original image; Figures 2A and 2B illustrate a pixel modification technique according to an embodiment of the present invention; Figure 3 illustrates another pixel modification technique according to an embodiment of the present invention; Figure 4A and 4B illustrate a pixel modification technique according to yet another embodiment of the present invention; Figure 5 is a block diagram of an illustrative watermarking system of the

present invention ; and Figure 6 is a block diagram of an illustrative system of the present invention for removing a watermark.

DESCRIPTION OF PREFERRED EMBODIMENTS Overview Referring to Figure lA, the luminance of an original image in the spatial domain is represented by pixels as is conventional. Each pixel is represented by an n bit digital number. Referring to Figure 1B, a bitmap, which is referred to herein after as a template, contains data indicating which pixels of the original image are to be modified and which are to remain unmodified. In the example shown in Figure 1B, pixels m within the shaded"S"shaped area are to be modified and the other pixels remain unmodified."S"is only an example and any other shape can be used. Also, a plurality of areas may be provided and they can be of different shapes. See for example Figure ID.

In the example of the present invention, as shown in Figure 1C, pixels in the area of the original image corresponding to the"S"shaped area of the template are modified. The modification changes the value of the pixels. Examples of modifications are described below with reference to Figure 2, 3 and 4. A photograph of an actual image modified in accordance with the invention is shown in Figure ID.

The modification of the image degrades the image. The modification is such that it cannot be easily removed by an unauthorised person.

-. The owner of an image may wish to offer the image for sale. The owner modifies the image in accordance with the present invention. The modification is such that the image, or sufficient of the image, is recognisable so that a potential buyer can determine whether they wish to buy the image. However the buyer cannot remove the modification. The buyer pays for the image. Once the buyer has paid the owner either: 1) sends an unmodified copy of the original image to the buyer; or 2) sends to the buyer removal data which allows the buyer to remove the modification so as to restore the unmodified image.

Preferably, if the owner sends the removal data, it is done in such known manner that the data is kept secure.

Examples of modifying pixels.

Referring to Figures 2 to 4, a pixel represented by an 8 bit number is shown by way of example. The number n of bits may be other than 8. The number has Most Significant Bits (MSBs) and Less Significant Bits (LSBs). In this example there are 3 MSBs and 5 LSBs. The number of bits allocated to LSBs is a matter of choice. Also, the number of LSBs may be changed pseudo-randomly from pixel to pixel.

Example 1.

Referring to Figure 2, the pixel is modified by rearranging, in a predetermined manner, the LSBs of a pixel within the pixel. An example of such a rearrangement is shown in Figure 2B. The manner of rearrangement is defined by an invertible function. It will be appreciated that the rearrangement involves at least two bits.

Example 2.

Referring to Figure 3, the pixel is modified by encrypting the LSBs in a predetermined manner. The encryption may be achieved by combining the LSBs with an encryption code having the same number of bits as the LSBs according to an EXCLUSIVE-OR function. The manner of encryption is defined by an invertible function. The encrypting may involve at least three bits, increasing the number of bits increases the level of encryption.

Example 3.

Referring to Figures 4A and B, pixels within a group of pixels are modified by rearranging, in a predetermined manner, the LSBs between the pixels of the group.

Thus for example, a group may have four pixels 1 to 4. Each pixel has five LSBs A, B, C, D and E. Bit Al of pixel I is moved to the position of bit A of pixel 2; bit A2 of pixel 2 is moved to the position of AI : Bl moves to B3 and B3 moves to BI ; and so on. In the example shown in Figure 4B, the bits are moved but retain their original ranking: that is a bit A of one pixel is moved to an A position in another pixel. In other examples, a bit such as A may be moved to the position of a B, C, D or E bit in another pixel. The manner of rearrangement is defined by an invertible function. The

rearrangement involves a minimum of one bit from each sample and at least 2 samples from a group. Preferably, each sample has at least 3 LSBs.

The result of the modifications illustrated by Figures 2 to 4 is an effective change in the values represented by the pixels. It will be noted that the number of bits has not changed. The application of the invertible function does not increase the number of bits in each sample. That modifies the image. Figure ID shows an example of the result of modifying an image by method 3.

By way of comparison with the above three examples, if only the single least significant bit of each individual sample was changed, then the original image can be restored simply by removing the least significant bit, and because that bit contributes least to the image, the loss in quality is likely to be minimal. Also, if only the two least significant bits of each individual sample were modified, the modification could be damaged or partially removed.

As shown in Figures 1B and ID, the modification of the pixel values occurs in a predetermined area or areas of the image defined by the template. The size, shape and number of the areas in which modifications are to be made is a matter of choice. Preferably large changes can be made in those areas making the changes more robust against unauthorised removal, whilst leaving sufficient of the image unchanged for a potential buyer to decide whether they wish to buy it Illustrative Apparatus Referring to Figure 5, a source 50 produces a digital image I. The image may be a still image or a frame of a movie. The image I is stored in a frame store 52. The means for writing data into the frame store are well known and are not shown in Figure 5. A template, defining the pixels to be modified and those to be left unmodified is stored in another store 56 which in this example is a frame store identical to store 52.

A read address generator 54 generates identical read addresses for the stores 52 and 54. For each address, the image store 52 reads out a pixel value and the template store 56 reads out data indicating whether that pixel is to be modified or not. A memory 58 stores an algorithm defining the manner in which pixels are to be modified. Storing the algorithm in memory 58 enables the algorithm to be configurable or changeable. A processor 51 modifies the pixels indicated by the template according to the algorithm.

The processor 51 may modify individual pixel values (if so indicated by the template) according to the predetermined algorithm stored in a memory 58 and operating in the manner shown in Figures 2 and 3.

Referring to Figure 4, the processor 51 may create groups of pixels which the template indicate are to be modified and rearranges the LSB bits of the pixels amongst the pixels of the group. In this illustrative example the processor 51 creates mutually exclusive groups of adjacent pixels. However, it will be appreciated that the groups need not be mutually exclusive nor need the pixels be adjacent. Instead, any grouping may be specified by the algorithm, including a pseudo-random selection.

Figure 6 shows an apparatus for removing modifications from an image. Such an apparatus would be used by a buyer after the buyer has paid for the image. It will be appreciated that such an apparatus is not essential, because the owner of the image could simply send the buyer an unmodified image once the buyer has paid for it.

Referring to Figure 6, a frame store 62 stores a modified digital image 1'. The image may be a still image or a frame of a movie. The means for writing data into the frame store are well known and are not shown in Figure 6. A template, as described above with reference to Figure 1 B is stored in another store 66 which in this example is a frame store identical to store 62. A read address generator 64 generates identical read addresses for the stores 62 and 64. For each address, the image store 62 reads out a pixel value and the template store 66 reads out data indicating whether that pixel is to be modified or not. A memory 68 stores an algorithm defining the manner in which modified pixels are to be restored to their original values. Storing the algorithm in memory 68 enables the algorithm to be configurable or changeable. A processor 61 modifies the pixels indicated by the template according to the algorithm.

Figure 6 assumes that the modified image I'has been received over a communications channel 69. For example the modified image may have been received

via the Internet. The channel in this example communicates via the processor 61. The processor 61 receives the modified image and causes it to be stored in the store 62. The buyer reviews the image and indicates to the owner via the channel 69 that he wishes to buy it. The owner then transmits the template and the decoding algorithm to the buyer. The read address generator 64 reads out the pixel values form the store 62

and the template data from the store 66. The processor then decodes the modified image using the template and the decoding algorithm in the inverse of manner by which the image was modified by the apparatus of Figure 5.

Modifications The image could be a color image in which color is represented by digital values. The image could be modified by modifying the values representing color in the manner described above.

The preceding description assumes that the same algorithm is applied to all pixels or groups of pixels. However the algorithm may vary from pixel to pixel or from group to group. For that purpose the template store may store plural-bit numbers for each pixel or group. For example if the plural-bit number has two bits then for example 00 = no change to a pixel or group, 01 = algorithm 1, 10 = algorithm 2, and 11 =algorithm 3. A random order of the three algorithms is preferably stored in template.

Any one or more of the original image, the modified image and the authorised copies thereof may have data imperceptibly embedded therein for copyright protection and/or to identify the image.

Whilst the invention has been illustrated by reference to image samples, it may be applied to audio samples or to samples of other data.

Although particular embodiments have been described herein, it will be appreciated that the invention is not limited thereto and that many modifications and additions thereto may be made within the scope of the invention. For example, various combinations of the features of the following dependent claims could be made with the features of the independent claims without departing from the scope of the present invention.

Claims

1. An apparatus for generating a perceptibly modified material, which material is represented by digital samples of the material, the apparatus comprising : a selector for selecting a part or parts of the material in which sample values are to be modified; and an encoder for applying an invertible function to the bits of the said samples in the selected part or parts so as to modify the sample values, thereby perceptibly modifying the material; wherein the modified material has no more bits after modification than before modification.

2. Apparatus according to claim 1, wherein the encoder applies the said function to individual samples.

3. Apparatus according to claim 2, wherein the encoder is arranged to rearrange, in a predetermined manner defined by the said function, bits within individual samples.

4. Apparatus according to claim 1, wherein the encoder applies the said function to groups of samples.

5.-Apparatus according to claim 4, wherein the encoder is arranged to rearrange, in a predetermined manner defined by the said function, bits between the samples of a group of samples.

6. Apparatus according to any preceding claim, wherein the said function encrypts the bits of the samples.

7. Apparatus according to claim 6, wherein the encoder is arranged to combine the bits of a said sample with an encrypting code.

8. Apparatus according to any one of claims 1 to 7, wherein the sample values have most significant bits (MSBs) and less significant bits (LSBs), and the encoder applies the said function to the LSBs but not to the MSBs.

9. Apparatus according to any preceding claim, wherein the said material is video material and the said samples are spatial domain samples of the video material.

10. Apparatus according to any one of claims 1 to 8, wherein the said material is audio material and the said samples are time domain samples of the audio material.

11. Apparatus according to any one of claims 1 to 8, wherein the said material is data material.

12. Apparatus according to any one of claims 1 to 8, wherein the said material is audio/visual material.

13. Apparatus according to any one of claims 1 to 9 and 12, wherein the material comprises an image and the said selector comprises a map of the image indicating the said selected part or parts.

14. Apparatus for regenerating unmodified material which material is represented by digitally encoded samples, the bits of samples in selected part or parts being modified by the application thereto of an invertible function, thereby perceptibly modifying the material, the regenerating apparatus comprising: a selector for selecting the said part or parts of the modified material; and a decoder for applying an inverse function, which is the inverse of the invertible function, to the sample values within the said part or parts so as to regenerate the unmodified material.

15. Apparatus according to claim 14, wherein the decoder applies the said inverse function to individual samples.

16. Apparatus according to claim 15, wherein the decoder is arranged to rearrange, in a predetermined manner defined by the said inverse function, the bits within individual samples.

17. Apparatus according to claim 14, wherein the decoder applies the said inverse function to groups of samples.

18. Apparatus according to claim 17, wherein the decoder is arranged to rearrange, in a predetermined manner defined by the said inverse function, the bits between the samples of a group of samples.

19. Apparatus according to any one of claims 14 to 18, wherein the said invertible function encrypted the bits of the samples and the decoder applies the said inverse function to decrypt the bits of the samples.

20. Apparatus according to claim 19, wherein the decoder is arranged to decrypt the bits of a said sample by combining the bits with a decryption code.

21. Apparatus according to any one of claims 14 to 20, wherein the sample values

have most significant bits (MSBs) and less significant bits (LSBs), the material being perceptibly modified by applying the invertible function to the LSBs, but not the MSBs, and wherein the decoder applies the inverse function to the LSBs, but not the MSBs, of those samples which have been modified.

22. Apparatus according to any one of claims 14 to 21, wherein the said material is video material and the said samples are spatial domain samples of the video material.

23. Apparatus according to any one of claims 14 to 21, wherein the said material is audio material and the said samples are time domain samples of the audio material.

24. Apparatus according to any one of claims 14 to 21, wherein the said material is data material.

25. Apparatus according to any one of claims 14 to 21, wherein the said material is audio/visual material.

26. Apparatus according to any one of claims 14 to 22 and 25, wherein the material comprises an image and the said selector comprises a map of the image indicating the said part or parts.

27. A method of generating perceptibly modified material, which material is represented by digitally encoded samples, the method comprising: selecting a part or parts of the material in which sample values are to be modified ; and applying an invertible function to the bits of the said samples in the selected part or parts so as to modify the sample values, thereby perceptibly modifying the material; wherein the modified material has no more bits after modification than before modification.

28. A method according to claim 27, wherein the applying step applies the said invertible function to individual samples.

29. A method according to claim 28, wherein the applying step rearranges, in a predetermined manner defined by the said invertible function, bits within individual samples.

30 A method according to claim 27, wherein the applying step applies the said invertible function to groups of samples.

31. A method according to claim 30, wherein the applying step rearranges, in a predetermined manner defined by the said invertible function, bits between the samples of a group of samples.

32. A method according to any one of claims 27 to 31, wherein the said invertible function encrypts the bits of the samples.

33. A method according to any one of claims 27 to 32, wherein the said material is video material and the said samples are spatial domain samples..

34. A method according to any one of claims 27 to 32, wherein the said material is audio material and the said samples are time domain samples.

35. A method according to any one of claims 27 to 32, wherein the said material is data material.

36. A method according to any one of claims 27 to 32, wherein the said material is audio/visual material.

37. A method according to any one of claims 27 to 36, wherein the material comprises an image and the said step of defining a predetermined part of the material defines a map of the image indicating the said part or parts.

38. A method of regenerating unmodified material, which material is represented by digitally encoded samples, the bits of samples in selected part or parts being modified by the application thereto of an invertible function, thereby perceptibly modifying the material, the regenerating method comprising the steps of : selecting the said part or parts of the modified material; and applying an inverse function, which is the inverse of the invertible function, to the sample values within the said part or parts so as to regenerate the unmodified material.

39. A method according to claim 38, wherein the applying step applies the said inverse function to individual samples.

40. A method according to claim 39, wherein the applying step rearranges, in a predetermined manner defined by the said inverse function, the bits within individual samples.

41. A method according to claim 38, wherein the applying step applies the said inverse function to groups of samples.

42. A method according to claim 41, wherein the applying step rearranges, in a predetermined manner defined by the said inverse function, the bits between the samples of a group of samples.

43. A method according to any one of claims 38to 42, wherein the said invertible function encrypted the bits of the samples and the applying step applies the said inverse function to decrypt the bits of the samples.

44. A method according to claim 43, wherein the applying step decrypts the bits of a said sample by combining the bits with a decryption code.

45. A method according to any one of claims 38 to 44, wherein the sample values have most significant bits (MSBs) and less significant bits (LSBs), the material being perceptibly modified by applying the invertible function to the LSBs, but not the MSBs, and wherein the applying step applies the inverse function to the LSBs, but not the MSBs, of those samples which have been modified.

46. A method according to any one of claims 38 to 45, wherein the said material is video material and the said samples are spatial domain samples of the video material.

47. A method according to any one of claims 38 to 45, wherein the said material is audio material and the said samples are time domain samples of the audio material.

48 A method according to any one of claims 38 to 45, wherein the said material is data material.

49. A method according to any one of claims 38 to 45, wherein the said material is audio/visual material.

50. A method according to any one of claims 38 to 46 and 49, wherein the material comprises an image and the said step of defining a predetermined part of the material defines a map of the image indicating the said predetermined part.

51. An apparatus for generating a perceptibly modified material as claimed in any one of claims 1 to 13 or a method of generating perceptibly modified material as claimed in any one of claims 27 to 37 wherein each modified sample values have no more bits after modification than before modification.

52. A computer program product arranged to carry out the method of any one of claims 27 to 50 or claim 51 when dependent on claims 27 to 37 when run on a computer.

53. An apparatus for generating a perceptibly modified material substantially as herein before described with reference to Figures 1 to 5 of the accompanying drawings.

54. Apparatus for regenerating unmodified material substantially as herein before described with reference to Figures 1 to 4 and 6 of the accompanying drawings.

55. A method of generating a perceptibly modified material substantially as herein before described with reference to Figures 1 to 5 of the accompanying drawings.

56. A method of regenerating unmodified material substantially as herein before described with reference to Figures 1 to 4 and 6 of the accompanying drawings.

57. A system comprising an apparatus according to any one of claims I to 13 and 51 when dependent on any one of claims 1 to 13 and 53 in combination with an apparatus according to any one of claims 14 to 26 and 54.