Classifications

• • G—PHYSICS
  • G10—MUSICAL INSTRUMENTS; ACOUSTICS
  • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
  • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
  • G10L19/018—Audio watermarking, i.e. embedding inaudible data in the audio signal
• • G—PHYSICS
  • G10—MUSICAL INSTRUMENTS; ACOUSTICS
  • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
  • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
  • G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders

Definitions

• the present invention relates to the field of audio coding and in particular to methods and devices for embedding a watermark in an audio signal.
• Modern audio coding methods process discrete-time audio samples to provide a bit stream that is compressed relative to the original audio signal.
• the stream of discrete-time audio samples is first windowed to generate successive blocks of windowed audio samples from the stream of audio samples. Further processing takes place in blocks.
• a block of audio samples generated by windowing is typically converted into a spectral representation by means of an analysis filter bank.
• the spectral representation comprises spectral values lying next to one another from frequency 0 to the maximum audio frequency, which can be, for example, 16 kHz.
• the audio spectral values are grouped and quantized in scale factor bands. The quantization takes place in such a way that the quantization noise introduced by the quantization is dimensioned such that it is masked by the audio signal.
• a psychoacoustic model which, on the basis of the audio signal, supplies an energy value for each scale factor band, which indicates the energy level up to which quantization noise is masked, ie it will not be audible in the decoded audio signal. If, on the other hand, the quantization noise introduced by the quantizer lies above the psychoacoustic masking threshold, the re-decoded audio signal will contain audible interference.
• the quantizer levels of the quantizer are dependent on the masking threshold calculated. When the quantization levels are calculated, the audio spectral values are quantized using these quantization levels to obtain quantized audio spectral values. For data efficiency reasons, the quantized audio spectral values of an entropy coding, such as B.
• Huffman coding to provide a bit stream with code words that represent the audio spectral values.
• page information is added to the stream of code words, which includes, among other things, the scale factors on the basis of which an audio decoder can determine the quantization levels that have been used in the encoder.
• the bit stream including side information is split up into code words on the one hand and side information on the other using a bit stream demultiplexer.
• the entropy coding is first undone. Then the entropy decoded, values quantized audio spectral values, an inverse quantization that is subjected to inversely quantized spectral values ⁇ to get. These are then converted from the frequency domain to the time domain using a synthesis filter bank. The decoded audio signal is present at the output of the synthesis filter bank.
• the psychoacoustic masking threshold of an audio signal section depends on the actual input audio signal. If the audio signal changes over time, the masking properties also change over time. For reasons of data efficiency, it is preferred to always introduce as much quantization noise into the audio signal as possible, ie the quantization noise should correspond as closely as possible to the psychoacoustic masking threshold. Audio signal sections with good masking properties can therefore be encoded with a relatively low bit cost, while on the other hand audio signal sections with relatively poor masking properties, such as. B. tonal audio signal sections, must be quantized very finely, which in turn means that a large number of bits must be used to encode these audio signal sections.
• bit savings bank function is usually used.
• the bit savings bank is filled when easy-to-encode audio sections are encoded such that bits that are not needed to encode these easy-to-encode sections are not simply "wasted” by more subtle than necessary quantization, but nevertheless a coarser quantization is used and the surplus bits are "inserted" into the bit savings bank.
• bit savings bank function thus acts as a buffer
• This method also has the advantage of high audio quality, since the quantization noise and the watermark noise can be matched to one another if the energy introduced into the audio signal by the watermark is below the psychoacoustic masking threshold lies.
• the method is also characterized by a high level of robustness, since the watermark cannot be removed from the re-decoded audio signal, for example by an illegal distributor of the audio signal, without impairing the audio quality.
• a disadvantage of the described method is the fact that the watermark may be quantized or weakened under certain circumstances by the quantization of the signal applied to the watermark. This is due to the fact that the energy of the watermark signal is sometimes in the range of the quantization interval. Furthermore, there is only limited control over the interference introduced by the watermark. B. can affect an audio quality loss.
• Another watermarking method is to embed the watermark during the compression of the audio signal. This concept is described in the specialist publication "Combined Compression / Watermarking for Audio Signals", Frank Siebenhaar, Christian Neubauer and Jürgen Herre, 110th AES Convention, May 12-15, 2001, Amsterdam, Preprint 5344.
• First, an uncompressed audio signal is fed to a psychoacoustic model in order to determine the masking threshold.
• the audio signal is then transformed into the frequency domain.
• the spread, spectrally represented watermark signal is weighted based on the masking threshold in the frequency domain and added to the spectrum of the input audio signal.
• the parameters for the quantization are determined on the basis of the masking threshold, whereupon the watermarked signal is quantized and encoded.
• This method is also characterized by a low level of computational complexity, since certain operations, such as e.g. B. the calculation of the masking model and the conversion of the audio signal into a spectral representation only have to be carried out once.
• the method usually also provides one good audio quality, since quantization noise and watermark noise can be coordinated.
• a disadvantage of this method is also the fact that the watermark may be quantized or weakened under certain circumstances by the quantization of the signal applied to the watermark. This is again due to the fact that the energy of the watermark signal is sometimes in the range of the quantization interval. Furthermore, there is only limited control over the interference introduced by the watermark. B. can affect an audio quality loss.
• the spread watermark signal is also characterized by a large number of spectral lines. However, so that the watermark does not lead to audible interference in the decoded audio signal, the height of the watermark spectral lines is significantly less than the height of the audio signal spectral lines.
• the combined spectrum is only slightly changed compared to the original spectrum. The subsequent quantization of the combined spectrum will always remove the watermark without replacement if the quantization step size is greater than the height of the watermark spectral lines that are quantized with this quantizer step size. If too many watermark spectral lines are "quantized away" by the subsequent quantization, the watermark detector can no longer extract a clear watermark.
• the object of the present invention is to provide an improved concept for embedding a watermark in an audio signal which on the one hand provides good audio quality and on the other hand also ensures good watermark detectability.
• This object is achieved by a method for embedding a watermark in an audio signal according to claim 1 or by a device for embedding a watermark in an audio signal according to claim 16.
• the present invention is based on the finding that better watermark detectability is achieved if the fact that the audio signal including the watermark is subjected to quantization is taken into account in the watermark embedding.
• a watermark will only be detectable if a spectral line, which represents the watermark and audio signal, falls through the watermark into a different quantization level than if no watermark is embedded. Only in this case will a watermark detector, which only receives quantized information, be able to detect a watermark.
• the spectral representation of the watermark signal is therefore processed in such a way that it is ensured that the watermark signal processed by the processing step is designed in such a way that it will still be present after quantization.
• a predetermined watermark starting value is selected, which depends on the spectral representation of the watermark signal.
• the watermark must not lead to any or only a very slight interference in the audio signal.
• a disturbance introduced by the predetermined watermark start value into the spectral representation of the audio signal is determined, but in which the conditions after a quantization of the spectral representation of the audio signal can be used. On the one hand, this makes it possible to see whether some of the watermark remains after quantization.
• the watermark initial value error is greater than a predetermined Storungsschwelle
• the watermark initial value is changed until the system introduced by a modified watermark initial value in the spectral representation after quantization error less than or equal to the Restaurant ⁇ voted fault threshold is.
• the resulting changed watermark starting value is then combined with the audio signal in order to obtain the watermarked audio signal in which the watermark is embedded.
• An advantage of the present invention is that ratios that ultimately do not correspond to the initial ratios are no longer taken into account, namely the audio signal / watermark ratios before the quantization, but that the watermark z. B. iteratively changed until a desired watermark "interference energy" is found.
• the conditions after the quantizer are now taken into account, i. H. the ratios that are relevant for the audio signal decoder and for the watermark extractor.
• the watermark energy has usually been set in the prior art in such a way that the watermark energy is less than or equal to the psychoacoustic masking threshold, the unpredictability of what happens to the watermark signal during quantization still remains.
• the case could occur that the watermark is quantized away, which means that no watermark or only a very weak watermark could be extracted in the decoded signal.
• the case could also arise that, although the watermark What has been important is that it is below the masking threshold, yet noise has been introduced that was audible in the decoded signal.
• the method according to the invention has the advantage that for cases in which good detectability is particularly important, certain - tolerable - disturbances are deliberately introduced into the audio signal in favor of a higher watermark detectability, while in other cases in which the watermarks -Detectability does not have to be ensured at all times under all circumstances, compromises with regard to watermark detectability can be made in order to meet the highest audio quality requirements.
• the watermark signal is added to the audio signal prior to quantization to obtain a combined signal.
• the combined signal is then quantized and again inversely quantized and compared with the original audio signal. The comparison determines whether the interference introduced by the watermark is tolerable. If it is determined that the disturbance is not tolerable, the spectrum of the watermark signal is weighted iteratively using certain strategies in order then to carry out quantization and inverse quantization again until it is determined that the disturbance is now tolerable.
• the watermark spectrum obtained by this processing is then added to the original audio spectrum.
• the added or combined signal is then quantized, entropy-encoded and provided with side information to obtain an audio bit stream in which the watermark is present.
• the original audio signal is quantized.
• a quantized watermark is added to the audio signal to obtain the combined signal.
• the combined signal is then no longer quantized again, as in the first exemplary embodiment, but is directly entropy-coded.
• the "quantized" watermark signal added to the quantized audio signal is set in such a way that on the one hand the requirement for tolerable interference is met and on the other hand a desired watermark detectability is achieved.
• FIG. 1 shows a block diagram of an inventive device for embedding a watermark in an audio signal
• FIG. 2 shows a block diagram of a device according to the invention for introducing a watermark into an audio signal according to a first exemplary embodiment
• FIG. 3 shows an inventive device for embedding a watermark in an audio signal according to a second exemplary embodiment; and 4a to 4d a schematic explanation of the line selection algorithm in the second exemplary embodiment of the present invention.
• the device according to the invention shown in FIG. 1 comprises an audio input 10 and a watermark input 12. Both the audio signal at the audio input 10 and the watermark signal at the watermark input 12 are converted into a spectral representation by means of a device 14 or 16.
• the spectral representation of the audio signal comprises audio spectral values, while the spectral representation of the watermark signal has watermark spectral values.
• a device 18 for combining the audio spectral values are combined with changed watermark spectral values in order to obtain the combined audio signal in an output 20, in which the watermark is embedded.
• a device 22 for processing the spectral representation of the watermark signal is provided depending on a psychoacoustic masking threshold supplied via an input 24.
• the spectral representation of the watermark signal is processed depending on the psychoacoustic masking threshold obtained via the input 24 in order to obtain a processed watermark signal so that a disturbance introduced into the audio signal by the processed watermark signal is below a predetermined disturbance threshold which is different from that psychoacoustic masking threshold depends.
• the device 22 for processing the spectral representation of the watermark signal comprises a device 26 for selecting a predetermined watermark starting value, which depends on the spectral representation of the watermark signal.
• a disturbance introduced by the predetermined watermark start value into the spectral representation of the audio signal after quantization of the spectral representation of the audio signal is detected.
• quantization information is supplied by a device 30 for supplying quantization information.
• the device 30 provides quantization information which depends on the original audio signal, that is to say the audio signal without a watermark.
• a device 32 examines whether the ascertained disturbance is greater than the predetermined disturbance threshold. If not, i. H. if the disturbance is acceptable, the watermark start value is fed directly to the device 18 for combining. If this is the case, however. H. if the introduced interference is too great or different than desired, a device 34 for changing the watermark start value is activated until the interference introduced by a changed watermark start value in the spectral representation of the audio signal after quantization is less than or equal to the predetermined one Interference threshold is.
• the loop sketched in the processing device 22 may have to be iterated several times in order to at some point obtain a changed watermark starting value at the output of the device 32, which is used as a processed watermark signal and fed to the combining device 18 in order to to receive the audio signal at the output 20 in which the watermark is embedded.
• the combination is carried out by means of an addition 18 before the quantization.
• the device 28 for determining the initial value introduced into the audio signal by the block watermark weighting 26 is determined by first quantizing and inversely quantizing the combined signal in a quantizer / inverse quantizer device 28a.
• the disturbance introduced by the watermark is then calculated in a device 28b, for example by forming the difference and squaring the difference values, and then in the device 32 with the psy- choacoustic masking threshold 24 compared. If the disturbance is too great, the device 34, which is labeled "weight control" in FIG. 2, is actuated in order to supply changed weighting factors to the block 26, in order then to change the weighted spectrum of watermarks in the device 18 with the original audio signal in to combine spectral representation and to go through the iteration loop again.
• the watermark spectrum with a watermark spectrum weighted equally for all spectral lines as the initial watermark value.
• the weighting factor for each spectral line is therefore equal to a constant for all spectral lines, which is chosen such that the watermark energy lies above the masking threshold. Then the watermark energy is gradually reduced in order to then "iterate" the energy of the watermark below the masking threshold.
• the device 34 is designed to control the weighting factors in order to reduce all weighting factors, e.g. B. cut in half. If the disturbance is then still too large, all current weighting factors could be halved again in a next iteration step, etc. This can continue until the device 32 determines that the disturbance is now OK.
• the spectrally represented watermark signal is thus spectrally weighted with the current weighting factors provided by block 34 by means of a weighting filter bank, which can be contained in block 26, as has been explained with reference to FIG. 2.
• the resulting signal is added to the original audio signal.
• the combined signal at the output of device 18 is quantized and inversely quantized and results in the signal present at the output of device 28a which is fed into device 28b in the same way as the original audio signal.
• the device 28b now compares the original signal with the quantized and again inversely quantized signal and uses this to determine the quantization error signal which is supplied to the device 32.
• the weighting control in block 34 is activated in order to determine new, better weighting factors.
• the masking threshold determined by the masking model is available, which indicates how much interference in the signal is "allowed" at a certain point in the signal spectrum. If the weight control block 34 has determined optimal weighting factors with regard to the desired audio signal interference and the desired watermark detectability, ie watermark energy, the method terminates.
• the quantized spectral values of the combination signal last determined by block 28a are then passed on to the bitstream multiplexer as a result, in order to be formed there together with the side information into an audio bitstream. 3 is discussed below in order to illustrate a device for embedding a watermark in an audio signal according to a second exemplary embodiment of the present invention. In contrast to the first exemplary embodiment shown in FIG.
• this combination 18 in FIG. 3 takes place in the "quantization range", ie it becomes a quantized audio signal with a combined quantized watermark.
• This can be achieved either by using a quantizer 42 to calculate the quantizer stages by quantizing the original audio signal, or by extracting the quantizer stages from a coded audio signal.
• means 40a for calculating the quantized audio signal minus a predetermined number of n quantization levels and means 40b for calculating the quantized audio signal plus a predetermined number of n quantization levels are operated.
• a so-called “maximum” watermark is first calculated as a predetermined watermark starting value by means of a device 36. To calculate the predetermined maximum watermark, only the signs of the watermark spectrum are used. If the watermark spectrum has a positive sign, the corresponding spectral value of the original quantized audio signal is increased by n quantization levels, n being an integer greater than or is 1.
• the sign of a watermark spectral value is negative
• the corresponding quantized spectral value ie the spectral value of the audio signal at the same frequency as the spectral value of the watermark signal whose sign is currently being viewed
• n quantization levels are reduced by n quantization levels.
• a device 38 that implements a line selection algorithm is provided.
• the device 38 determines the interference introduced into the audio signal by the maximum watermark provided by the device 36. If the disturbance is greater than the predetermined disturbance threshold, the device 38 changes the "maximum" watermark by selecting individual lines until the disturbance introduced by the watermark is less than or equal to the predetermined disturbance threshold. If this condition is fulfilled, the watermark, which is already in quantized form, and the quantized original audio signal are fed to the adder 18 in order to obtain the quantized watermarked audio signal on the output side.
• 4a shows, by way of example, a quantized audio signal which, because of the clarity of the illustration, only shows three spectral values 50a-50c.
• an audio spectrum has e.g. B. 1024 spectral values.
• the number of non-zero quantized spectral values depends on how many audio spectral values have been quantized to zero. Of course, the quantized audio spectral values have different heights in the real case.
• 4b shows an audio spectrum with plus or minus n quantization levels (depending on the sign of the watermark spectral values).
• the spectral component of the watermark corresponding to the audio spectral value 50a of FIG. 4a has a negative sign for the example shown in FIG. 4b.
• the spectral component of the watermark, which corresponds to the audio spectral value 50b of FIG. 4a, has a positive sign in the example shown in FIG. 4b, while the third spectral component of the watermark again had a negative sign.
• the amount of the watermark spectral components is initially irrelevant, since it is assumed that watermark detection is already possible if the quantized audio spectral values 50a-50c are changed by the watermark.
• the maximum watermark which is determined by the device 36 from FIG. 3, is shown in FIG. 4c for the case shown in FIG. 4b. It has a spectrum that is characterized in that each quantized original audio spectral value is changed by a quantization level, either enlarged if the watermark has a positive sign, or decreased if the watermark had a negative sign.
• the amount of a watermark spectral line could be taken into account in such a way that not only is incremented or decremented by one quantization level, but that is incremented or decremented by several quantization levels if the amount of Watermark spectral line is correspondingly large.
• the function of the device 38 from FIG. 3 will now be described with reference to FIG. 4d.
• the facility 38 If the left quantized audio spectral component 50a is reduced by a quantization level, as is represented by the spectral component 50a ', the situation is such that the situation is such that the interference introduced by the watermark is too large, as is represented by the spectral component 50a', this becomes the spectral component not selected by the device 38, which is so noticeable in the changed watermark spectral values after the line selection that the changed watermark has a spectral line of 0 at this point.
• the quantized audio spectral values are now based on the watermark signal, ie using the sign of the watermark signal by z. B. plus or minus changed a quantization level.
• This procedure has advantages in that computing time can be saved since the quantization and inverse quantization (device 28a of FIG. 2) and the weighting of the watermark (device 26 of FIG. 2) can be omitted without replacement.
• the maximum watermark (FIG. 4c) is determined line by line on the basis of the audio spectra already precalculated, ie the original spectrum and the original spectrum minus n quantization levels or the original spectrum plus n quantization levels. This results as the difference between the original spectrum (FIG. 4a) and the audio spectrum changed by a number of n quantization levels (FIG. 4b), the difference having the same sign as the unweighted watermark.
• the line selection algorithm which is executed in the device 38, takes into account the amount of the unweighted watermark spectral lines, the masking threshold 24 and possibly a bit savings bank function 44 of the audio encoder.
• the lines of the maximum watermark in such a way that the watermark spreading band signal is embedded in a broadband manner, i. H. that as many lines of the quantized audio signal as possible are changed.
• the structure of the watermark should be changed as little as possible within a frequency band.
• bit savings bank function which can provide additional bits to later signal blocks, as has been carried out.
• the line selection strategy is preferably adapted to the fill level of the bit savings bank, so as to allow, for example, when the bit savings bank is filled, that even quantized audio spectral values of the original audio signal that have the value 0 are watermarked, which would normally not be permitted due to the bit requirement , This can noticeably improve watermark detection.
• the original values are also available after the transformation into the frequency range.
• the quantization of the original audio spectral values can also be seen as a kind of watermark embedding, since a certain disturbance of the audio spectrum results in both the quantization and the addition of a watermark signal. Due to its random nature, the disturbance introduced by the quantization should not be regarded as a watermark. However, if the introduced disturbance is correct with the watermark due to the quantization, the quantization noise supports the detectability of the watermark. The following cases result from this.
• the watermark is introduced with the correct sign.
• the device 38 of FIG. 3 is preferably arranged such that, due to the fact that the quantization is already in phase with the watermark spectral value for one certain frequency a disturbance has been introduced, arranged to dispense with the introduction of another watermark disturbance.
• a quantization level could be added to further improve the detectability.
• the psychoacoustic masking threshold is not calculated line by line, but by scale factor band. This means that energies of individual spectral lines are not considered, but the total energies z. B. 20 spectral lines in a scale factor band. However, in a scale factor band in which many watermark spectral lines are tolerable, a few lines in the sense of good audio quality can be dispensed with without the watermark detectability suffering significantly.
• This functionality can also be achieved in the exemplary embodiment shown in FIG. 2 in that the weighting control 34 of FIG.
• the concept according to the invention is such that a spectrally represented watermark signal is first generated. This is weighted using weighting factors. The weighted signal is added to the original audio signal, which is available in a spectral representation. Alternatively, a change in the lines of the original audio signal, which is available in a spectral representation, is carried out on the basis of the watermark signal. The disturbance introduced after the quantization is then determined, the disturbance being determined by quantizing, inverse quantizing and forming the difference from the original, or the disturbance being pre-calculated.
• new weighting factors are determined, using the masking threshold, using a line selection strategy, or using a line selection strategy in particular in such a way that the sign and amount of the spectral lines of the unweighted watermark are used, and that the sum of the watermark line and the original spectral line is such it is determined that this new spectral line falls within a different quantization interval than the original spectral line.
• the concept according to the invention is advantageous in that it can be used both for bitstream watermarking methods and for methods which carry out audio coding and watermark embedding in one step. Another advantage of the concept according to the invention is that full control over the introduced disturbance can be achieved. This makes it possible to specifically set the method in favor of optimal watermark detection or optimal audio quality.
• Another advantage of the concept according to the invention is full control over the frequency distribution of the watermark spreading band signal into the audio signal.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• Health & Medical Sciences (AREA)
• Signal Processing (AREA)
• Audiology, Speech & Language Pathology (AREA)
• Human Computer Interaction (AREA)
• Computational Linguistics (AREA)
• Acoustics & Sound (AREA)
• Multimedia (AREA)
• Spectroscopy & Molecular Physics (AREA)
• Compression, Expansion, Code Conversion, And Decoders (AREA)
• Signal Processing For Digital Recording And Reproducing (AREA)
• Reverberation, Karaoke And Other Acoustics (AREA)

Applications Claiming Priority (3)

Publications (2)

Family

ID=7688519

Family Applications (1)

Country Status (5)

Families Citing this family (62)

Family Cites Families (12)

• 2001
  • 2001-06-18 DE DE10129239A patent/DE10129239C1/de not_active Expired - Lifetime
• 2002
  • 2002-05-10 US US10/481,860 patent/US7346514B2/en not_active Expired - Lifetime
  • 2002-05-10 EP EP02740586A patent/EP1382038B1/fr not_active Expired - Lifetime
  • 2002-05-10 DE DE50203511T patent/DE50203511D1/de not_active Expired - Lifetime
  • 2002-05-10 AT AT02740586T patent/ATE298921T1/de active
  • 2002-05-10 WO PCT/EP2002/005173 patent/WO2002103695A2/fr active IP Right Grant

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events