GB2425924A - Watermarking an audio or audio-visual signal - Google Patents

Abstract

A method of labelling an audio or audio-visual signal is described comprising inserting a watermark code. The watermark code is split into several sections, and timing information is encoded by providing a variable time delay between inserting adjacent sections of the watermark. The time delays are varied in a pattern or sequence which does not repeat over a time scale of up to a few minutes, and which remains recognisable even when a significant proportion of the watermarks are lost. The pattern or sequence is preferably not random.

Description

A METHOD OF LABELLING AN AUDIO OR AUDIO-VISUAL SIGNAL

[001] This invention relates to a method of labelling an audio or audiovisual signal by inserting a watermark code.

It also relates to apparatus for inserting a watermark code into an audio or audio-visual signal, and apparatus for retrieving said code.

[002] Methods of inserting a substantially inaudible identification code or watermark code into an audio signal such that it can be read by a suitable decoder are well known. Such systems are often used in broadcasting to identify whether an advertisement was broadcast at an agreed time on an agreed programme, although many other uses for such codes have been proposed - including identifying the source of an illegally copied track.

[003] It is sometimes useful for the advertiser or a monitoring body to be able to tell from the watermark data received whether or not a specific track was broadcast in its entirety, or whether the start or end of the track was missing. One solution to this problem is to incorporate a time stamp in the watermark code. However, the data channel comprising the watermark code has a low bandwidth, and the time stamp can consume quite a large part of this available bandwidth.

[004] In another application, it is useful when fingerprinting audio tracks for recognition purposes to include timing information such as the elapsed time, so that it is less demanding to recognise which track is being sampled a random time after the start.

[005] It is an object of the present invention to provide a method which can mitigate the above disadvantage and/or provide additional advantages over known methods.

[006] According to a first aspect of the invention there is provided a method as specified in claims 1 - 13.

[007] According to a second aspect of the invention there is provided a computer readable medium as specified in claim 14.

[008] According to a third aspect of the invention there is provided a system as specified in claim 15.

[009] According to a fourth aspect of the invention there is provided a decoder as specified in claims 16 or 17.

0] Embodiments of the invention will now be described, by way of example only, with reference to the accompanying schematic drawings in which:- [0011] Figure 1 shows a block diagram of apparatus according to the present invention.

2] Figure 1 shows a simplified apparatus according to one aspect of the present invention. In this embodiment, an audio file to be broadcast is, prior to broadcast, encoded by a watermark encoder 10, with the time delay between consecutive watermarks being controlled by a timing sequence generator 20 coupled or connected thereto. The encoded audio file may then be broadcast immediately, or stored for broadcast at a later time. The timing sequence generator 20 is reset at the start of the file, and controls the time delay between the start of the file and the insertion of the first watermark, and the time delay between the insertion of subsequent watermarks.

3] In the present example, the watermark is inserted into a notch having a narrow frequency of about 150 Hz at a frequency of 2 or 3 kHz, as described in more detail in EP 0 245 037. Of course, other known methods of inserting a watermark code may be used as an alternative.

4] Although in Figure 1, the encoder connects directly to a decoder (30), there may in practice be a period of weeks or months before the decoder is exposed to the audio.

5] In the embodiment shown in Figure 1 a second timing sequence generator 32, identical to the one coupled to the encoder, is provided at the decoder. When a signal is received by the decoder, the decoder timing sequence generator is run and the received retrieved timing sequence is compared with the original sequence provided by timing sequence generator 32. This original sequence is identical to the first sequence generated by timing sequence generator during encoding. The comparison may be made, for example, using cross correlation techniques which are effective if some parts of the sequence are missing, at comparator 34.

6] Correlation techniques are simplified if values from the timing sequence generator are quantized in steps that are larger than the timing resolution of the decoder. Each timing delay recovered by the decoder can then be placed in a quantizing bin, for ease of matching the original and retrieved sequences.

7] As an alternative, the timing sequence generator 20 can provide the first sequence as a look up table. This is particularly advantageous if for any reason a watermark cannot be inserted when the timing sequence generator indicates it should be inserted (for example if the programme material is too quiet so that inserting the watermark code at that time might be audible, or the frequency content of the programme material is such that masking is unlikely to occur) . The look up table can then be passed to the decoder as a separate file, or in some other way, so that a decoder timing sequence generator may not be required in the decoder.

8] In a further embodiment, a method of inserting a watermark code into an audio or audio-visual signal is described In which the watermark code is split into several sections, and timing information is encoded by providing a variable time delay between inserting adjacent sections of the watermark. The time delays are varied in a pattern or sequence which does not repeat over a time scale of up to a few minutes, and which remains recognisable even when a significant proportion of the watermarks are lost.

9] One example of a suitable sequence generator will now be described in more detail, for the specific case where each watermark is encoded in four distinct sections or "chunks" whose timing is controlled by the encoder. The watermark code will In this example use two bits of information as an index to identify which section Is being encoded and/or detected.

0] The four quantlsed inter-chunk timing intervals can be expressed as a four digit integer. A requirement is that the timing interval sequence should not repeat over the length of the audio file (which is the "given time interval" in the present example) . It is also desirable that consecutive numbers in the sequence share few digits. An example of such a sequence is shown in table 1 below.

INCREMENTING SEQUENCE PREFERRED SEQUENCE

0168 0168 0169 1279 2390 0171 3501 0172 4612 0173 5723

Table 1

1] The left hand column of this table is an incrementing sequence of four-digit integers. Each digit has ten possible values (i.e. decimal), although numbers in other bases such as base 4 or base 12 may be used as an alternative. In this sequence the least significant bit always changes for adjacent integers in the sequence: the others change less often - only when there is a carry. In such sequences each integer is unique and so the sequence does not repeat in the short term, but uniqueness is destroyed locally if the least significant bit is not recovered.

2] In contrast, the right hand column of table 1 shows a preferred sequence for use in the present invention. In this sequence each digit is changed (in the present example incremented) between adjacent integers in the sequence. Thus there is no most significant or least significant digit. This sequence will cycle through the same number of combinations (i.e. 10,000 for a 4 digit decimal integer), but in a way which preserves uniqueness locally even when a significant number of digits are missing.

3] Of course other sequences are possible which have this property for example 1234, 5678, 4123, 8567, 3412, 7856, 2341, 6785, 4321, 8765, 1432, 5876, etc. - In this sequence (in number base 8 - the decimal digits 0 and 9 are not being used) the digits change, but do not increment.

4] If a chunk is not recovered in a sequence as shown in the right hand column of table 1, this is not actually equivalent to having a missing digit. It is equivalent to knowing the sum of two adjacent digits rather than their individual values. Such distortions can be overcome by using conventional correlation techniques.

5] The actual sequence of time intervals in the present embodiment has the added property that future members of the sequence can be predicted from previous ones.

6] The sequence of time intervals is preferably not random, although quasi-random sequences may be employed. The sequence is advantageously predetermined. However this is not always possible.

7] The decoder shown in Figure 1 is provided with a timing sequence generator which follows the same sequence as that of the encoder. The sequence can either be the same for all audio files or different for different files.

8] Of course, a decoder for use in the present system need not have a timing sequence generator at all. For example, if the decoder recovers a code which identifies a file in some way it could be provided with means to look up the sequence used for that particular file in a database or look-up table, and use those values as the sequence to be used when matching timings. A particular example of this is when the watermarking technology being used does not allow the watermark code to be inserted at will, but sometimes requires additional delays - for example to ensure inaudibility. In that case the detailed encoding sequence can be saved by the encoder for future use by the decoder.

9] In another embodiment of the present invention, the watermark code is split into 4 sections, and one of three possible time delays is used in the sequence (i.e. the integers in the sequence are effectively in number base 3) This will give a sequence of 81 possible unique combinations before the sequence repeats. Using the watermarking technology as described in EP 0 245 037, a 4-section watermark occupies approximately 3 seconds of audio signal.

Thus a sequence of 81 insertions will occupy 4 minutes before repeating, which is adequate for tracking advertisements which never exceed this duration. If longer files are to be used, the number of possible delays can be increased to 8 or 10 or 12, thus providing correspondingly longer periods before the sequence repeats.

0] In practice, it is important that the time interval used is adapted to have a value larger than the minimum timing resolution of the decoding apparatus. For example, if the data rate within a watermark is 50 bits per second, then each bit has a duration of about 20 ms. To be able to recover this data it is typically necessary to have a decoder timing resolution of better than 5 ms (preferably of the order of 2 ms, or better) [0031] The methods described can be applied to digital or analogue signals. The resulting watermarks are analogue in nature in that the watermark survives if the digital signal is converted to analogue - for example for routing to loudspeakers.

2] In summary, a method of labelling an audio or audio- visual signal has been described comprising inserting a watermark code. The watermark code may be split into several sections, and timing information is encoded by providing a variable time delay between inserting adjacent sections of the watermark. The time delays are varied in a pattern or sequence which does not repeat over a time scale of up to a few minutes, and which remains recognisable even when a significant proportion of the watermarks are lost. The pattern or sequence is preferably not random.

3] It is to be understood that many minor and/or obvious modifications to the above embodiments can be made whilst still falling within the scope of the invention as specified in the following claims. In addition, means equivalent to those specified in the claims may be substituted if such equivalent means achieve the same results.

Claims (17)

1. A method of labelling an audio or audio-visual signal or file by inserting a watermark code therein on a multiplicity of successive occasions during a given time period, characterised in that the time interval between successive insertions of the watermark code, or sections thereof, is varied according to a first sequence which does not repeat in the given time period, thereby providing timing information associated with the said signal or file.

2. A method as claimed in claim 1 in which the timing information is such that if one or more parts of the original audio-visual signal or file are missing, the location in time and/or duration of such missing parts can be determined from comparison of a retrieved sequence with the first sequence.

3. A method as claimed in claim 1 or claim 2 in which the timing information is such that the elapsed playing time of the signal or file from a starting time can be determined from comparison of a retrieved sequence with the first sequence.

4. A method as claimed in any preceding claim in which each time interval between insertions of successive watermark codes, or sections thereof, is a multiple of a minimum time interval.

5. A method as claimed in claim 4 in which the watermark code is subsequently read by decoding apparatus, and the minimum time interval is arranged to have a value larger than the minimum timing resolution of the decoding apparatus.

6. A method as claimed in any preceding claim in which the watermark code is split into two or more sections which are inserted sequentially, the time interval between the insertion of successive sections being varied according to the first sequence.

7. A method as claimed in claim 6 in which the first sequence is such that the set of time intervals between sections of a given watermark on one insertion and the set of time intervals between sections of the said watermark on the next insertion have two or fewer members in common.

8. A method as claimed in claim 6 or claim 7 in which the watermark code is split into n sections, such that there are n time intervals between successive insertions of the first section of a given watermark code, each time interval being represented by a digit such that the first sequence comprises a sequence of n-digit numbers.

9. A method as claimed in claim 8 in which each of the n digits of the ndigit number changes, on average, at approximately the same rate.

10. A method as claimed in claim 8 or 9 in which adjacent n-digit numbers in the first sequence have two or fewer digits in common.

11. A method as claimed in claim 8, 9 or 10 in which at least n-2 of the n digits change between adjacent numbers in the first sequence.

12. A method as claimed in any one of claims 8 - 11 in which n is 4 or more, and each time interval can take one of three or more values.

13. A method as claimed in any preceding claim in which the first sequence is not random.

14. A computer readable medium having a computer program stored therein which, when loaded into a computer, performs a method as claimed in any preceding claim.

15. A system for watermarking an audio or audio-visual - 10 - signal or file using a method as claimed in any one of claims 1 - 13, comprising an encoder (10) for inserting watermarking code and a decoder for retrieving said watermarking code, the encoder including a timing sequence generator (20) for generating a first sequence of time intervals between successive insertions of the watermark code, or sections thereof, in a given time period, which sequence does not repeat in the given time period, thereby providing timing information associated with the said signal or file, and the decoder including means for comparing the retrieved sequence of time intervals with said first sequence.

16. A decoder for use in a system as claimed in claim 15, including means for comparing the retrieved sequence of time intervals with said first sequence to derive timing information associated with a watermarked audio or audio-visual signal or file, the timing information being such that if one or more parts of the original audio-visual signal or file are missing, the location in time and/or duration of such missing parts can be determined from comparison of the retrieved sequence with the first sequence.

17. A decoder for use in the system as claimed in claim 15, including means for comparing the retrieved sequence of time intervals with said first sequence to derive timing information associated with a watermarked audio or audio-visual signal or file, the timing information being such that the elapsed duration of the signal or file from a starting time can be determined from comparison of a retrieved sequence with the first sequence.