MXPA99007705A - Embedding supplemental data in an encoded signal - Google Patents

Abstract

Method and arrangement for watermarking an audio or video signal. The signal is encoded by an encoder (6) which includes a feedback loop (64) to control the encoding process, such as a DPCM encoder or a (sigma-)delta modulator. The watermark (3) is embedded by modifying (2) selected samples of the encoded signal. Said modifying is carried out before the encoded signal is fed back so that quantization errors which are introduced by the embedded watermark will be eliminated by subsequent encoding operations. In addition, one or more samples preceding the selected samples are also modified in such a way that the error induced by the watermark is further reduced. This is achieved by"looking ahead"which (combination of) preceding sample modifications yields the best encoding quality.

Description

INCLUSION OF SUPPLEMENTARY DATA IN A CODED SIGN FIELD OF THE INVENTION The invention relates to a method and an arrangement for including supplementary data in a signal, comprising the steps of encoding the signal in accordance with and the coding process including the step of feeding back the encoded signal to control the encoding, and modifying selected samples of the encoded signal to represent such supplementary data.

BACKGROUND OF THE INVENTION There is a growing need to accommodate supplementary data in audio and video signals in a perceptually invisible manner. For example, watermarks to be included in multiple media devices to identify the origin or copyright status of audiovisual documents and programs. The watermark provides legal proof of the copyright holder, allows piracy to be tracked, and helps protect intellectual property. A known method for marking a video signal with a watermark as described in the opening paragraph is described in F. Hartung and B. Turns: "Digital atermarking of Raw and Compressed Video", SPIE Vol. 2952, pp. 205-213. The watermark is achieved here by modifying the DCT coefficients selected in the output bit stream of an MPEG2 encoder. As is generally known, an MPEG2 encoder is a predictive encoder that includes a feedback loop to control the encoding process. A prediction error (the difference between the input signal and a prediction of it) is encoded instead of the input signal itself. The prediction signal is obtained by decoding the coded signal locally. In the method of the prior art, watermarks are inserted after conventional coding. The capacity available to mark the signal coded in this way with a watermark seems to be restricted.

OBJECT AND BRIEF DESCRIPTION OF THE INVENTION An object of the invention is to provide a method for including supplementary data in an encoded audio or video signal, which allows most of the bits of the encoded signal to be altered without substantially affecting the perceptual quality . For this purpose, the method according to the invention is characterized in that the step of modifying the selected samples is carried out before the feedback of the coded signal and includes modifying at least one additional sample of the coded signal preceding the sample selected, if it is found that such modification of the additional sample, improves the quality of the coding process. The step of including supplementary data before the feedback of the signal has also been proposed in the European Patent Application of the Applicant, unpublished, No. 97200197.8 (file of proxy PHN 16.209). With this step, it is achieved that the adverse effects of a modification of the sample are compensated in the subsequent operations of the encoder. However, the initial disturbance of the sample modification remains. The invention is based on the recognition that the coding quality is further improved by deliberately modifying one or more signal samples before a selected sample. In fact, the encoded signal is slightly predistorted to minimize encoding errors yet to come. The invention is particularly useful for including supplementary data in the unit bit encoding signals. Unit bit encoders such as delta modulators, sigma-delta modulators, and noise form encoders produce a one-bit output sample at each coding step. The encoded signal is very vulnerable to being marked with a watermark. Sigma-delta modulators, for example, which were designed to record high-quality audio on audio DVDs at a sampling rate of 2, 822,400 (64 * 44100) Hz, has a signal-to-noise ratio of 115 dB. Watermarked labeling such as the sigma-delta modulated signal in a form as taught by the prior art, i.e., after conventional coding, appears to result in a considerable quantization noise. For example, replacing every 100th bit of a sigma-delta modulated audio signal with a bit marked with a watermark will raise the quantization noise to -60 dB, which is clearly unacceptable. Watermarked marking as proposed in Applicant's European Patent Application, copending, No. 97200197.8 allows each 100th bit to be replaced at the expense of only a 1 dB decrease in quantization noise. The invention not only improves the coding quality from the point of view of signal-to-noise reduction. It is well known that sigma-delta modulators with a circuit filter of the order >; 2 lead to instability problems for large input signals. This instability is usually prevented by prohibiting the input signal from exceeding a predetermined interval. The invention also provides a solution to those types of instability problems and problems related to limiting the amplitude of the output signal.

Other objects, features and advantages of the present invention will become apparent upon reading the following description, in conjunction with the accompanying drawings, in which: BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 shows an arrangement for including supplementary data in a signal modulated by delta modulation according to the invention. Figures 2-4 show waveforms of the signal to illustrate the operation of the arrangement shown in Figure 1. Figure 5 shows a flow chart to illustrate the operation of a modified circuit, which is shown in Figure 1. Figure 6 shows an array for including supplementary data in a signal modulated by sigma-delta modulation according to the invention. Figure 7 shows a third-order sigma-delta modulator filter, which is used in the arrangement shown in Figure 6. Figures 8, 9, 10A-10D and 11A-11C show signal waveforms to illustrate the operation of the arrangement shown in Figure 6.

DESCRIPTION OF THE MODALITIES The invention will be described with reference to the unit bit encoders, but it should be understood that teaching can also be applied to other types of predictive encoders, such as DPCM encoders (e.g., MPEG). A delta modulator array will be described first because of its easy to understand operation. Subsequently, an array of sigma-delta modulator will be described, which will most likely be used in the practical coding system. Figure 1 shows an array for including supplementary data to the delta-modulated signal according to the invention. The array comprises a conventional delta modulator 1, which includes a subtractor 11, a polarity detector 12 and a decoding filter 13. The subtractor 11 generates a production error signal and subtracts a prediction signal x from the input signal x . The prediction error e is applied to the polarity detector 12, the c, ual produces, at a rate determined by a sampling frequency fs (not shown), an output sample "+1" for x > x and an output sample "-1" for? < x. A feedback circuit 14 includes a local decoder 23 (an adder or integrator) to obtain the prediction signal x.

In a conventional delta modulator, the feedback circuit 14 is connected to the output of the polarity detector 12. Figure 2 shows waveforms of such a conventional delta modulator. More particularly, Figure 2 shows the input signal x, the prediction signal x and the coded output signal and the conventional delta modulator. Note that the prediction signal i is also the output signal of a receiver (not shown in Figure 1). With reference to Figure 1, again, the arrangement according to the invention comprises a modification circuit 2, which is connected between the polarity detector 12 and the feedback circuit 14. The modification circuit modifies the output bits of the Polarity detector selected in response to a selection signal s. For example, the modification circuit replaces every 100th bit of the encoded signal and by one bit of a watermark data pattern w, which is stored in a watermark data record 3. Alternatively, the modification circuit inverts the selected bits, the number of bit periods between the inverted bits that represent the data pattern of the watermark.

Figure 3 reveals the effect of modifying a selected bit 20 of the encoded signal and by a data bit of the watermark w. The input signal x (the same as the signal in Figure 2), the production signal i and the coded signal, modified z, are shown in this Figure. The numeric reference 21 denotes the bit of the included watermark. As it has been tried to show in the Figure, the bit of the included watermark has the value of "-1", which differs from the value of "+1" of the bit of the encoded signal 20. Recalling that the prediction signal x is also the output signal of the receiver, it allows to easily see that the modification of the bit increases the quantization noise. Because the modified signal z is fed back to the input of the • encoder, the quantization error will be later compensated and eventually eliminated. According to the invention, the modification circuit 2 (Figure 1) is arranged to also modify at least one of the bits preceding the watermark if it is found that this improves the coding quality. An example of the same is shown in Figure 4. Again, the input signal x, the prediction signal x, the encoded, modified signal, z and the watermark bit 21 are shown. In addition, a bit 22 was also modified, which precedes the bit of the watermark 21. A comparison of Figure 3 with Figure 2 immediately shows that the total quantization error is therefore further reduced. The quality of coding is, in this way, considerably improved. In the example shown in Figure 3, good operation is obtained by modifying the bit immediately preceding the bit of the watermark. This is not always the case. The modification of the 2nd, 3rd, ... etc. bit that precedes the bit of the watermark, or a combination of them, can improve the operation - even more. An example of them will be given later. To obtain the effect described above, the arrangement shown in Figure 1 was adapted to execute the delta modulation process for several combinations of preceding bits, and to select the combination of the best result. The test of various bit combinations is also referred to herein as "advance registration" and the bits that precede the watermark bit that are being considered for the modification are referred to as "advance registration" bits. The modification process is carried out under the control of the modification circuit 2. The circuit can be implemented in programs and programming systems or physical computing components, depending on practical aspects such as the speed and complexity of the physical computing components. Figure 5 shows a flow diagram to illustrate the operation of the circuit. It is assumed that the input signal x is stored on a storage medium (not shown in Figure 1) and that every 100th bit of the signal coded y, must be replaced by a bit of water maca w. For this purpose, the input signal x is divided into two segments, each of which comprises 100 input samples X0 ... X99. For each segment, the output signal z comprises 100 bits zo ... z99 / in which, z0 ... z2 are three advance registration bits and z3 is the bit of the watermark. In a step 50, an encoded, binary, 3-bit number, c, is given an initial value of zero. The number c represents a current combination of three bits between anticipation records. In step 51, the three bits of c are assigned to z0..z2. That is, that Zi is set to "+1", if the corresponding bit of c is "1" and zi is set to "-1" if the corresponding bit of c is "0". Also, in step 51, the bit of the watermark w to be included is assigned to z3 in the same way. In a subroutine 52, the delta modulation process is applied to a given number of input samples, say xo ... x2o, to observe the behavior of the circuit or core for the preassigned values of z0 ... z3. The sequence of corresponding output bits z0 ... z20 is stored in a buffer (not shown in Figure 1). In a step 53, the Q (c) coding quality of the delta modulation process for the current combination c of the anticipation register bits is determined and stored in the buffer. In this example, the coding quality is represented by the mean square error (MSE) between the input signal and the prediction signal: The number c is then increased by one (step 54) to create a new combination of forward registration bits z0 ... z2 and calculate the corresponding value of MSE (c). As long as not all combinations have been processed (step 55), the delta modulation of the sequence x0 ... x2o is repeated. Obviously (and therefore not shown in the Figure), the same signals of the initial integrator are used each time. If all the combinations have been processed, the maximum coding quality Q (c) is determined in step 56. For this purpose, the number c for which the MSE (c) is minimum, is searched or queried in the buffer . In a step 57, the encoded sequence z0 ... z2o corresponding to the minimum MSE is read from the buffer and applied to the output terminal of the encoder. Next, in a subroutine 58, the X21 ... X99 of the input sample segment is coded and, in step 59, the encoder output is applied. Having thus encoded a segment of 100 input samples, the array returns to step 50 to process the next segment. It will be appreciated that a number of parameter values in the encoding process described above, such as the length of the segment (here 100), the number of advance registration bits (here 3), and the number of output bits that are evaluated (here 20), are given by way of example only. It should also be noted that the quality of coding can be expressed by other parameters, for example, the greater difference between an input sample xn and the corresponding prediction of x. Now a sigma-delta modulator according to the invention will be described. The sigma-delta modulation was designed to record high-quality audio over the audio version of the Digital Versatile Disc (DVD-Audio). This differs from delta modulation, in that the input signal x is filtered, before coding, by the same filter as the filter in the prediction circuit of a delta modulator.

The filters in the input path and the feedback path are then replaced by a single filter in the path of the encoder circuit. In Figure 6 an arrangement for including supplementary data in a signal modulated by sigma-delta modulation according to the invention is shown. The array comprises a conventional sigma-delta modulator 6, which includes a subtracter 61, a circuit filter 62, a polarity detector 63 and a feedback loop 64. The subtractor 61 subtracts the coded output signal z (which has the value of "+1" or "-1") of the input signal x. The difference signal d is filtered by the filter 62. The filtered signal f is applied to the polarity detector 63, which produces, at a speed determined by a sampling frequency fs (not shown), an output bit "+1"for f > 0 and an output bit "-1" for f < 0. The same modification circuit 2, as shown in Figure 1, is connected between the polarity detector 63 and the feedback circuit 64. In response to the selection signal s, the circuit 2 replaces a bit of the signal encoded and by a bit of the watermark w, which is stored in the register 3. Various modalities of the circuit filter 62 are used in the practical sigma-delta modulators. Through this description, a third order filter was used as an example. For the purpose of completion, this is shown in Figure 7. The filter comprises three integrators, which are connected in cascade. The output signals of the three integrators were denoted as a, b, c, respectively. The output signal of the filter f is a weighted combination of the integrator's signals In the figure, the whole number preceded by # is shown by each integrator, such an integer denotes the maximum value that the relevant integrator can maintain - and store Signal samples that exceed the maximum value are eliminated, as will become evident later, the elimination is relevant for the sigma-delta modulator modalities. Figure 8 shows waveforms to explain the operation of the array if the modification circuit 2 is inactive. More particularly, the Figure shows the input signal x, the encoded signal z, the difference signal d, the filtered signal f. The three output signals of the integrator a, b and c are also shown. The average value of the sigma-delta modulator's output signal represents the input level. In this example, the input signal x is at a level of 0.5V dc, which is encoded as a bit stream comprising (on average) three "+1" bits and one "-1" bit, according to with: 3x (+ l) + lx (-l) = 0.5 4 Figure 9 shows waveforms to illustrate the effect of the inclusion of a watermark bit 90 on the output coded signal z. The same signals as in Figure 8 are shown. A comparison of both Figures shows that the watermark bit introduces longer runs of the same bit values in the z-coded signal, which is an indication of an increase in the quantization noise. The watermark also causes large signal amplitudes to occur in the integrators, particularly in the output signal of the third integrator c. Obviously, this is true only if the bit of the watermark and the "regular" output bit have opposite values. Figures 10A-10D show the coded signal z and the output signal of the third integrator c under various conditions. The waveforms shown in Figures 10A and 10B are the same as the corresponding waveforms already shown in Figures 8 and 9, ie, without and with the watermark bit 90, respectively. Figure 10C illustrates the effect of setting the forward register bits 91 and 92 to "+1" and "-1", respectively. By comparison, Figure 10B can be seen that the amplitude of the output signal of the third integrator was reduced and the length of 1 successive in the encoded signal was shortened. As a result, the quantization error was reduced. Figure 10D shows that the operation of the sigma-delta modulator is further improved by setting other forward registration bits, viz, by setting both forward registration bits 91 and 92 to "+1". The algorithm to determine which combination of bits of the anticipation register produces the best coding quality, can be the same as that already described for the delta modulators with reference to Figure 5. That is, the quality Q (c) for coding a given sequence of input samples (e.g., xo -... x2o) was determined for various combinations c of anticipation register bits (e.g. z0..z2). The output sequence corresponding to the highest coding quality Q is then selected. Because the decoded signal is not available in a sigma-delta encoder, the mean square error is a less attractive criterion for coding quality. It has been found that the following parameters are very suitable for representing Q-coding quality. They have the additional advantage that they can be easily calculated. * The longest run of the same successive values in the sequence z0..z2o. The longer runs are denoted by R in Figures 10B-10D. Therefore, the sequence that has the "shortest" run is selected.

Obviously, the sequence for which R = 4 (Figure 10D) is the best choice in the present example. * The peak-to-peak amplitude that occurs in a given integrator. The peak-to-peak amplitude in the third integrator is denoted by V in Figures 10B-10D. The sequence that has the lowest amplitude is selected. Again, the sequence shown in Figure 10D seems to be the best choice. It has been found that the third integrator is very suitable, even when a higher order filter is used (> 3). * The average deviation of the values of the signal in a given integrator. A further criterion for selecting a combination of bits in the anticipation record may be the presence (or absence) of overflow in a given integrator. Because sigma-delta modulators are very sensitive to the levels of the input signal (in contrast to delta modulators, which are sensitive to the slopes of the input signal), overflow can easily occur in response to the inclusion of a watermark bit. As already mentioned with reference to Figure 7, the integrators are protected against overflow by a mechanism limiting the amplitude of the output signal, which keeps each output signal of the integrator at a maximum value.

Figures 11A-C show the coded signal z and the output signal of the third integrator c under conditions of limitation of the amplitude of the output signal. Again, the input signal is 0.5V cd. For reference, Figure HA shows a signal without watermark. In Figure 11B, a watermark bit 95 has been included in the encoded signal. Its position differs slightly from the position of the bit of the watermark 90 in the previous examples. The reference numeral 96 denotes the limitation of the amplitude of the output signal of the third integrator due to the inclusion of the bit of the watermark 90. In one embodiment of the modification circuit, different combinations of bits of the anticipation register were tested up to that a combination was found in which the limitation of the amplitude of the output signal does not occur anymore. An example of the same is shown in Figure 11B, which shows the effect of setting the bit of the advance register 97 to "+1". In summary, a method and an arrangement for marking an audio or video signal with a watermark was described. The signal is encoded by an encoder, which includes a feedback loop to control the coding process, such as a DPCM encoder or a modulator (sigma-) delta. The watermark is included by modifying selected samples of the encoded signal. Such modification is carried out before the encoded signal is fed back, so that the quantization errors that are introduced by the included watermark will be eliminated by the subsequent coding operations. In addition, one or more samples that precede the selected sample are also modified, in such a way that the error induced by the watermark is further reduced. This is achieved through the "advance registration", which (combination of which) precedes the - sample modifications, produce the best coding quality.

Claims (15)

CHAPTER CLAIMEDICATORÍO Having described the invention, it is considered as a novelty and, therefore, the content is claimed in the following CLAIMS:

1. A method for including supplementary data in a signal, comprising the steps of encoding the signal according to an encoding process, which includes the step of feedback to the encoded signal to control such encoding, and modifying the selected samples of the encoded signal to represent such supplementary data, characterized in that the step of modifying the selected samples is carried out before the feedback of the coded signal, and that includes modifying at least one additional sample of the coded signal that precedes the selected sample, if finds that modifying the additional sample improves the quality of the coding process.

The method according to claim 1, characterized in that the step of further modification comprises successively coding a segment of the signal with different combinations of the samples further modified, until a combination corresponding to a higher coding quality has been found. .

The method according to claim 1, characterized in that the step of further modification comprises successively coding a segment of the signal with different combinations of the samples further modified, determining the quality of the coding for each combination, and selecting the combination that corresponds to a higher coding quality. .

The method of compliance - with the claim 1, characterized in that it comprises the step of decoding the encoded signal and determining the amount of the quantization error in the decoded signal and the input signal, where the quality of the coding is represented by the amount of the quantization error.

5. The method according to claim 1, characterized in that the coding is the coding of unit bits.

6. The method according to claim 5, characterized in that the coding is the sigma-delta modulation.

The method according to claim 6, characterized in that the coding quality is represented by the longest run of the same successive values of the encoded signal with the supplementary data included.

8. The method according to claim 6, characterized in that the coding quality is represented by the peak-to-peak amplitude that occurs in a selected stage of a circuit filter in the sigma-delta modulator.

9. The method according to claim 6, characterized in that the coding quality is represented by the average deviation of the amplitudes in a selected stage of a filter of the circuit in the sigma-delta modulator.

The method according to claim 6, characterized in that the coding quality is represented by the occurrence of a maximum amplitude value in a selected stage of a filter of the circuit in the sigma-delta modulator.

11. An arrangement for including supplementary data in a signal, comprising an encoder for encoding the signal including the feedback circuit for feedback of the encoded signal to control the encoder, and means for modifying the selected samples of the encoded signal to represent the supplementary data, characterized in that the feedback circuit is connected to feed back the modified, coded signal, the modification means are arranged to modify at least one additional sample of the coded signal preceding the selected sample, if it is found that the modification of the additional sample improves the quality of the coding process.

12. The arrangement according to claim 11, characterized in that the encoder is a unit bit encoder.

13. The arrangement according to claim 11, characterized in that the encoder is a sigma-delta modulator.

14. A signal with supplementary data included, the signal is coded according to a given coding process and selected samples of the signal, represent the supplementary data, characterized in that at least one of the samples preceding the selected samples is different from the sample that corresponds to the given coding process.

15. A storage medium, characterized in that it has stored in it a signal according to claim 14.