KR100685974B1 - Apparatus and method for watermark insertion/detection - Google Patents

Abstract

The present invention relates to an apparatus and method for inserting and detecting a watermark during audio or video encoding and decoding. In particular, the present invention can effectively insert and detect watermarks by inserting and transmitting watermark information in the Huffman encoding process of the audio or video signal, and extracting the watermark information in the Huffman decoding process of the audio or video signal. In this case, the audio or video decoding apparatus having the watermark extraction function may decode the audio or video signal at the same time as the watermark information is extracted with respect to the bit string into which the watermark is inserted. It is possible.

Watermark, bitstream, Huffman coding, codebook

Description

Apparatus and method for watermark insertion / detection}

1 is a schematic diagram showing a system for embedding and detecting a general digital watermark

2 is a schematic diagram showing a system for embedding and detecting a digital watermark according to the present invention;

3 is a block diagram showing an embodiment of an MP3 encoding apparatus including a watermark insertion unit according to the present invention;

4 shows an example of a Huffman table used for MP3 encoding.

5A and 5B illustrate an example of dividing Huffman codebooks in the Huffman table of FIG. 4 into two groups according to watermark information to be inserted.

6 is a block diagram showing an embodiment of an MP3 decoding apparatus including a watermark extractor according to the present invention;

7 is a block diagram showing an embodiment of an AAC encoding apparatus including a watermark inserter according to the present invention;

8 shows an example of a Huffman table used for AAC encoding.

9A and 9B illustrate examples of dividing Huffman codebooks in the Huffman table of FIG. 8 into two groups according to watermark information to be inserted;

10 is a block diagram showing an embodiment of an AAC decoding apparatus including a watermark extractor according to the present invention;

Explanation of symbols for main parts of the drawings

210: audio encoder 230: audio decoder

211: watermark inserting unit 231: watermark extracting unit

310: subband filter bank 320: MDCT unit

330: distortion prevention unit 331: quantization unit

332: Huffman coding and watermark inserting unit

333: bit allocation unit 334: additional information encoding unit

340: FFT unit 350: psychoacoustic model unit

360: bit string generator 710: psychoacoustic model unit

720: preprocessor 730: filter bank

740: TNS unit 750: strength / coupling unit

760: prediction unit 770: M / S stereo processing unit

780: data recovery unit 781: compression ratio / distortion control

782: scale factor extraction unit 783: quantization unit

784: Huffman coding and watermark inserting unit

790: bit string generator

The present invention relates to digital watermarking, which is a kind of data concealment, and more particularly, to an apparatus and method for inserting and detecting watermark information in an audio or video encoding and decoding process.

In general, watermarking refers to 'hiding' secret information called 'watermark' inside media such as video, image, audio, and text. The hidden watermark information can be extracted only to those who know it, and the medium into which the watermark is inserted is recognized in the same way as a normal medium by general users.

In particular, digital media has created a new problem called 'protection of copyright' due to the advantages that it is easier to access, transmit, edit, and archive compared to analog, and that there is no deterioration of data during distribution through radio waves or communication networks. Marking is drawing attention as a means to protect this copyright.

The digital watermarking is used for copy protection, distribution process check, broadcast monitoring, etc. by inserting control information in addition to the purpose of copyright protection in which information for distinguishing owners is inserted. Or, it may be inserted into a real-time medium such as audio and video, and may be used for transmitting information such as playback time control information, audio / video sync (lip sync / lip sync), content information, lyrics, and the like.

In addition, although the characteristics of digital watermarking are different according to various purposes of use, non-perception and toughness are essential characteristics.

The non-perceptive means that the source medium before the watermark is inserted and the medium after the insertion should not be distinguished by people to see or hear, which is the most basic requirement of watermarking. The robustness means that the inserted watermark should be preserved even if the medium having the watermark embedded therein is subjected to modifications such as filtering, compression, noise addition, and degradation during distribution and transmission. In particular, in case of watermarking for copyright protection and copy protection, it should be tough to cope with intentional attacks that want to remove watermark, while in case of watermarking for anti-counterfeiting, watermark that easily disappears when modified or manipulated Sometimes inserted. At this time, in case of watermarking concealing additional information such as playback time control, lip syncing, content information, lyrics, and the like, the requirements for robustness against intentional attack or distortion are relatively low.

1 shows an apparatus for inserting and extracting a general digital watermark.

Referring to FIG. 1, watermark data is concealed and transmitted to a digital medium (eg, at least one of audio, video, image, and text) to which a watermark is to be inserted through a watermark embedding apparatus. In this case, depending on the algorithm, a secret or public key for security may be additionally used.

The watermark extraction apparatus basically extracts the watermark from the medium into which the watermark is inserted. In this case, the original medium may be required according to the algorithm, and it may be decrypted only if the secret key used to insert the watermark is included.

In the watermark extraction process, the system that does not require the original is called blind watermarking.

As a method of inserting such a watermark, various methods such as a Least Significant Bit (LSB) encoding method, an echo hiding method, and spread spectrum communication are provided.

The LSB encoding method is a method of inserting desired information by transforming least significant bits of quantized audio and video samples. This method takes advantage of the fact that the least significant bit distortion in the audio and video signals has little effect on the quality. It is simple to insert and detect and has low distortion, but it is very vulnerable to signal processing such as lossy compression and filtering. There is this.

The echo insertion method is particularly a technique applied to an audio signal and inserts an echo of a size small enough to be inaudible to the human ear. This method inserts and encodes echoes with different time delays according to binary watermark information to be inserted into audio signals segmented at predetermined time intervals, and detects echo time delays in each subdivided section during decoding. A method of decoding information. In this case, the added signal is not the noise but the audio signal itself, which has the same characteristics as the original signal. Therefore, even if the inserted signal is heard, it is not recognized as a distortion. Suitable. However, since the watermark is detected through the Ceptstrum operation, the computational amount of the decoding process is very high, and if the synchronization for the section to be divided in the time domain is missed, the decoding is not performed.

The spread spectrum communication based method is a representative watermarking method that is most studied in audio and video watermarking. This method converts an A / V signal to a frequency through DCT (Discrete Cosine Transform) or DFT (Discrete Fourier Transform), and then spreads the binary watermark information into PN (Pseudo Noise) sequence to apply the frequency-converted signal. Insert a watermark by adding. The inserted watermark can be detected by a correlator using a high auto-correlation characteristic of the PN sequence, and has a feature of being strong against interference and excellent in encryption. However, when inserting with a large energy to improve the robustness, the quality of the original signal is deteriorated, the computational amount of the insertion and detection process is very high, and the robustness of the compression coding is not perfect.

As a result of combining the prior arts of audio and video watermarking, in general, by using a method of inserting watermark information into a power signal to be compressed and encoded, an implementation method is complicated, and thus, a large amount of computation is required. However, it had a problem of being easily deformed during the compression process.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and an object of the present invention is to insert and detect watermark information during compression encoding and decoding of digital audio or video signals, thereby easily inserting and detecting watermark information. Rather, the present invention provides an apparatus and method for preventing distortion of an original signal and an embedded watermark.

Another object of the present invention is to insert and detect watermark information during Huffman encoding and decoding, thereby efficiently inserting and detecting watermark without causing additional noise or distortion during encoding and decoding of digital audio or video signals. An apparatus and a method thereof are provided.

In order to achieve the above object, an apparatus and method for embedding / detecting a watermark according to the present invention change a part of the Huffman encoding process to insert a watermark, and change a part of the Huffman decoding process to change the inserted watermark. It is characterized in that the extraction.

In particular, the watermark-embedded audio or video bit stream can be decoded without distortion through a conventional audio and video decoder, and the general audio or video decoder does not recognize whether or not the watermark is inserted. That is, the decoding method and apparatus having the above-described watermark extraction function can decode the audio signal at the same time as the watermark information is extracted with respect to the bit string in which the watermark is inserted, and decode the conventional general bit string in which the watermark is not inserted. It is also possible.

An apparatus for embedding / detecting watermarks according to the present invention which specifically implements such a feature may include classifying a plurality of Huffman codebooks used in Huffman coding into at least a plurality of groups, and sorting the groups according to the watermark information to be inserted. An encoder which selects one group and selects a Huffman codebook to be used for Huffman encoding of an input signal from the selected group, and then transmits the Huffman encoding after Huffman encoding; And a decoder for performing Huffman decoding of the transmitted signal using the Huffman codebook used for Huffman coding of the signal transmitted from the encoder and extracting watermark information based on the group to which the Huffman codebook belongs. It features.

The group and Huffman codebook information selected by the encoder is transmitted to the decoder as side information.

The coder uses an MP3 encoder and classifies the Huffman codebooks into a plurality of groups using at least one of a maximum value for each Huffman codebook, statistical characteristics of each Huffman codebook, and linbits values in the Huffman codebook.

The coder uses an AAC coder and classifies the Huffman codebooks into a plurality of groups based on the maximum absolute value of each Huffman codebook.

When the AAC encoder encodes an input signal that cannot be represented by the Huffman codebook, the AAC encoder does not insert a watermark for the selected Huffman codebook itself, and inserts the watermark in the process of performing exception processing (ESC). .

The method for embedding / detecting watermarks according to the present invention comprises: classifying a plurality of Huffman codebooks used for Huffman coding into at least a plurality of groups, and selecting one group among groups classified according to the watermark information to be inserted. Selecting a Huffman codebook to be used for Huffman encoding of an input signal from the selected group, and transmitting the Huffman encoding after Huffman encoding; And receiving a Huffman decoding of the transmitted signal using the Huffman codebook used for Huffman coding of the transmitted signal and extracting watermark information based on the group to which the Huffman codebook belongs. It features.

The group and Huffman codebook information selected in the transmitting step is transmitted as additional information.

The transmitting step performs MP3 encoding on the input signal and classifies the Huffman codebooks into a plurality of groups using at least one of a maximum value for each Huffman codebook, statistical characteristics of each Huffman codebook, and linbits values in the Huffman codebook. It is characterized by.

The transmitting step performs AAC encoding and classifies the Huffman codebooks into a plurality of groups based on the maximum absolute value of each Huffman codebook.

Other objects, features and advantages of the present invention will become apparent from the following detailed description of embodiments taken in conjunction with the accompanying drawings.

Hereinafter, with reference to the accompanying drawings, preferred embodiments of the present invention that can specifically realize the above object will be described. At this time, the configuration and operation of the present invention shown in the drawings and described by it will be described as at least one embodiment, by which the technical spirit of the present invention and its core configuration and operation is not limited.

The present invention relates to an apparatus and method for inserting a watermark in a compression encoding process of an image or an audio encoder using Huffman encoding, and extracting a watermark inserted in an image or audio decoding process. In particular, the present invention is to insert and detect the watermark information in the bit stream during the Huffman encoding and decoding process.

2 is a schematic diagram showing an embodiment of an encoding and decoding apparatus to which a watermark embedding and extracting apparatus according to the present invention is applied, which is combined with a high quality audio encoder and a decoder to insert and extract a watermark. do.

Referring to FIG. 2, the encoder 210 in which audio encoding and watermark information is inserted simultaneously performs encoding of a high quality audio signal to be compressed and watermark information to be inserted. At this time, the watermark embedding 211 is performed by changing a part of the conventional Huffman encoding process.

The decoder 230, which performs audio decoding and watermarking extraction, decodes the compressed bit string to restore the high quality audio signal in its original state and simultaneously extracts the watermark inserted by the encoder 210. The watermark extraction 231 is performed by changing a part of the Huffman decoding process in the decoder.

In this case, even in the conventional high quality audio decoder without the watermark extraction apparatus, the output audio signal can be obtained by decoding the audio bit string normally. In addition, when an unwatermarked bit string is inputted, only an audio signal can be decoded.

In the present invention, a watermark insertion and detection process will be described based on an MPEG third layer (MP3) encoding and decoding process and an AAC (Advanced Audio Coding) encoding and decoding process. That is, the encoder 210 including the watermark embedding apparatus performs MP3 encoding or AAC encoding. On the contrary, the decoder 230 including the watermark extraction apparatus performs MP3 decoding or AAC decoding.

The MP3 and AAC coding methods use psychoacoustic models based on the auditory characteristics of the human ear to remove perceptual redundancy of audio signals as with other high quality audio coding techniques, and to remove the statistical redundancy of signals. It has a combination of data compression methods.

3 is a block diagram showing an embodiment of an MP3 encoding apparatus including a watermark inserter according to the present invention, wherein watermark insertion is performed during the Huffman encoding process.

Referring to FIG. 3, the subband filter bank 310, the Modified Discrete Cosine Transform (MDCT) unit 320, the distortion control unit 330, the FFT (Fast Fourier Transform) unit 340, the psychoacoustic model unit 350, And a bit string generator 360.

The distortion control unit 330 is a repetitive loop and includes a nonlinear quantization unit 331, a bit allocation unit 332, a Huffman encoding and watermark inserting unit 333, and an additional information encoding unit 334. .

First, the audio signal is input to the subband filter bank 310 to remove statistical redundancy, and simultaneously to the FFT unit 340 to remove perceptual redundancy due to human auditory characteristics.

The subband filter bank 310 is composed of 32 weighted overlap addition scheme equal interval filter banks, and when the audio signal passes through the subband filter bank 310, the subband filter bank 310 is converted into subband samples. The subband sample from which the statistical redundancy of the audio signal is removed from the subband filter bank 310 is output to the MDCT unit 320 and the MDCT is performed, thereby using a hybrid transform coding method for increasing the frequency resolution.

In other words, in order to effectively use the auditory characteristics, it is necessary to divide the signal into frequency components. Therefore, the subband filter bank 310 and the MDCT 320 are used as MDCT coefficients having up to 576 frequency resolutions for the input signal. Convert.

The fast fourier transform unit 340 converts the input audio signal into a frequency domain through the FFT and outputs the converted audio signal to the psychoacoustic model unit 350. In the psychoacoustic model unit 350 using the FFT, a masking threshold, which is an inaudible noise level from an FFT frequency signal, is used to remove perceptual redundancy due to human auditory characteristics from an input audio signal. Get

Here, the masking phenomenon is one of important characteristics of sound perception, and a phenomenon in which a small sound below a certain threshold is covered by a large sound, that is, a phenomenon in which one sound suppresses the perception of another sound. Among these masking phenomena, the frequency masking phenomena deals with the case where two sounds exist at the same time. In other words, frequency masking means that when a pure sound of a certain frequency masks sound of another frequency, the masked nagging should be heard only when it has energy above a certain threshold. The threshold used here is called a masking threshold, and an absolute audible threshold ( Distinguish from Absolute threshold. The absolute audible threshold refers to a threshold capable of perceiving any sound.

The bit allocator 332 of the distortion controller 330 allocates the quantized bits of the MDCT coefficients based on the masking thresholds of the frames and outputs the quantized bits to the quantizer 331. At this time, a minimum bit is allocated to each scale factor band so that the quantization noise can be masked by the audio signal.

Bit allocation, quantization, and Huffman coding processes in the distortion control unit 330 are processed through an iterative loop. In this case, when the noise cannot be completely masked, the bit allocation unit 332 allocates bits for each scale factor band to minimize subjective noise. The nonlinear quantized MDCT coefficients are input to the Huffman encoding and watermark inserting unit 333 based on the allocated bits, and are output to the bit string generator 360 after Huffman encoding and watermarking are inserted. do. The Huffman-coded and watermark-embedded audio signal is transmitted in a bit string together with additional information by the bit string generator 360.

Huffman coding is a method of statistical compression, in which a bit number representing each unit information is allocated based on the frequency of occurrence of the unit information. That is, high frequency information is represented using a small number of bits, and low frequency information is represented using a large number of bits to reduce the amount of bits necessary for representing the entire data. Therefore, it is a variable length code in which the length of the code varies according to the unit information, and is widely used for compressing audio and video. The data compressed by the Huffman coding is a kind of lossless coding method because original information can be recovered without error by Huffman decoding.

In the MP3 and AAC encoding, Huffman coding is performed on the quantized signal, and a Huffman codebook is used to find a codebook index corresponding to each unit information and transmit the corresponding codeword.

First, the Huffman encoding process of MP3 will be described in detail.

Huffman coding of the MP3 is performed in an iterative loop. The iterative loop corresponding to the distortion control unit 330 performs nonlinear quantization of MDCT coefficients according to the allocated bits and the outer loop for allocating bits based on a masking threshold value for the corresponding input signal, and Huffman quantizes the quantized MDCT coefficients. It consists of an inner loop that encodes.

In the Huffman coding in the inner loop, a variable length coding method is used, which is divided into three regions according to the size of quantized MDCT coefficients. That is, it is the rzero area where all MDCT coefficients are 0, the count1 area where only 0, 1, and -1 exist, and the big_value area having other sizes.

In this case, the interval from the highest frequency spectrum coefficient to the first non-zero value, that is, the rzero region in which all MDCT coefficients are 0 is not encoded.

Next, one codeword in the Huffman table is transmitted in quadruple pairs of four MDCT coefficients for the count1 interval where only 0, 1, and -1 exist. The Huffman tables used at this time are, for example, Huffman code table for quadruples (A) and Huffman code table for quadruples (B) of the Huffman codes for Layer III. The Huffman tables A and B for the count1 region may be selectively used to represent the corresponding information, and the selected table information is transmitted as included in the bit string as additional side information.

In the big_value region having absolute values greater than 1, Huffman coding is performed by pairing two MDCT coefficients. That is, the big_value region is further divided into three subregions. Huffman tables 0 to 31 for each sub-area, for example, are selected from Huffman codes table 0 to 31 among Huffman codes for Layer III and used for Huffman coding. . The Huffman table selected for each subregion is also included in the bit string and transmitted through additional information.

In this case, since the maximum value (absolute value) that can be expressed for each codebook is different from Huffman codebooks 0 to 31 used for encoding the MPCT coefficients of the big_value region, it is selected from among Huffman codebooks that can represent the maximum value of MDCT coefficients for each subregion. Should.

FIG. 4 summarizes the maximum absolute value of each Huffman codebook used in the MP3.

Referring to FIG. 4, the maximum value that can be expressed through the Huffman codebook of MP3 is 15 as in the codebooks 13 to 31. However, the quantized MDCT coefficients have more values, and exceptions are expressed to express these values. That is, if there is a large value of 15 or more, Huffman is coded by a codeword corresponding to the maximum value of 15, and the coefficient value corresponding to the above value is represented by a linear PCM coding method and added after the codeword. At this time, the number of bits used for linear PCM encoding is linbits.

For example, to express information of 18, codebook 17 (linbits = 2) is selected, a codeword corresponding to 15 is inserted into the corresponding MDCT coefficients, and then 18-15, that is, 3 using 2 bits. This is a method of appending binary 11 after the codeword. Codebooks 16-23 use the same Huffman codebook (# 16), but show different linbits. Similarly, codebooks 24 through 31 use the same # 24 codebook, with different linbits.

In this case, the Huffman encoding and watermark inserting unit 333 simultaneously performs not only Huffman encoding but also watermark information encoding. That is, the Huffman encoding and watermark inserting unit 333 receives a watermark signal as an additional input, performs watermarking, and performs Huffman encoding to output the bitmap generator 360.

In other words, the Huffman encoding and watermark inserting unit 333 in which the watermark is inserted is modified to select the Huffman codebook for each region in the conventional Huffman encoding process to insert the desired binary number watermark.

The detailed principle is as follows.

That is, as described above, Huffman coding in the MP3 encoding process is divided into three regions, that is, rzero, count1, and big_value regions according to the size of the MDCT coefficients, and different lossless coding is performed for each region. Among the three regions, one of the predetermined Huffman codebooks is selected for the count1 region and the big_value region to perform Huffman encoding using the corresponding codebook. The Huffman codebook selected for each region is transmitted as additional information in the MP3 bit string so that the MP3 decoder can Huffman decode the MDCT coefficients with reference to the MP3 decoder.

At this time, when looking at the Huffman codebooks (see FIG. 4) that can be selected and used for each region used, it can be seen that there are two or three codebooks having the same maximum absolute value. This is for selecting and coding a codebook suitable for the statistical characteristics of the spectral distribution, that is, codeable with the least bits.

For example, if the maximum absolute value of the MDCT coefficients in a particular sub-region among the big_value regions is 5, Huffman codebooks are performed using Huffman codebooks 7,8, and 9, respectively, and then the Huffman codebook having the smallest number of bits. Select. In fact, when the maximum absolute value is 5, the Huffman codebook having a maximum value other than the 7,8, and 9 codebooks may be used. That is, the codebook can be encoded using codebooks 10, 11, 12 or more.

In the present invention, watermarking is performed by dividing the selectable Huffman codebooks into two groups in order to express the same information, first selecting a group according to the binary watermark information to be inserted, and making a selection from among the codebooks in the group. . For example, when a binary watermark '1' is to be inserted into a subregion of which the maximum absolute value is 5, the codebook is selected from 9, 10, 11 times and a binary watermark '0' is inserted. In this case, the codebook may be selected from among 7,8, and 12 codebooks.

5A and 5B illustrate an example in which the Huffman codebooks selectable according to binary watermarks '0' and '1' to be inserted by the Huffman encoding and watermark inserting unit 333 according to the present invention are classified into two groups.

This is an example in which the Huffman codebook is divided into two and selected according to the watermarks '0' and '1'. Other combinations based on the same principle are possible.

In the present invention, Huffman codebooks may be classified into three, four, or more groups according to binary watermark information to be inserted. In this case, watermarking is performed by first selecting a group according to the binary watermark information to be inserted and then selecting the codebook from the corresponding group.

The classification of FIGS. 5A and 5B is a case in which the maximum value and linbits for each codebook and statistical characteristics according to each codebook are divided so that the binary numbers '0' and '1' are well distributed.

At this time, the watermark is not inserted in the areas where codebooks 0 and 1 should be used.

Codebooks 16-23 and 24-31, which use the same Huffman table and are distinguished by linbits values, are also separately assigned to FIGS. 5A and 5B according to linbits values.

Thus, by distributing the Huffman codebooks in the same Huffman table as shown in FIGS. 5A and 5B in consideration of the statistical characteristics and linbits size of the Huffman codebook, it is necessary to restrict the Huffman codebooks that can be used for each region for the insertion of watermarks. This can minimize problems that can reduce compression efficiency.

Accordingly, when the Huffman coding is performed during the MP3 encoding process, the Huffman codebook is selected from among FIG. 5A and FIG. 5B when the Huffman codebook is selected in the MP3 encoding process. It is explained by the process.

When the watermark information is inserted and transmitted at the time of Huffman encoding in the encoder 210 as described above, the decoder 230 transmits the information using the transmitted section value of each region and information on the Huffman table selected for each section. The quantized MDCT coefficients are reconstructed from the codewords of the bit strings, and watermark information is extracted according to the Huffman codebook used during the reconstruction.

FIG. 6 is a block diagram illustrating an MP3 decoder to which the watermark extracting apparatus according to the present invention is applied, and operates in the reverse order of the MP3 encoder to which the watermark inserting apparatus of FIG. 3 is applied.

6, a bit string demultiplexing and watermark extractor 610, a Huffman decoder 620, an inverse quantizer 630, an IMDCT unit 640, and a subband filter bank 650 are configured. do.

That is, the bit string demultiplexing and watermark extractor 610 extracts binary watermark information in a demultiplexing process to unpack information necessary for decoding from the transmitted bit string.

The method of extracting watermark information from the bit string demultiplexing and watermark extractor 610 is very simple. That is, the binary watermark information '0' and '1' are extracted by reading the codebook of the Huffman table (table_select) selected for each region among the transmitted additional information and comparing it with FIGS. 5A and 5B. For example, if the Huffman codebook for a subregion is 10, it is present in FIG. 5B, and thus watermark information inserted at a corresponding position may be determined as '1'.

In the MP3 bit string of one frame (1152 time samples), 12 table_select values and 4 count1table_select values exist in the case of two channels, and a binary watermark of up to 16 bits can be inserted per frame.

The additional information and the audio bit string separated by the bit string demultiplexing and watermark extractor 610 are output to the Huffman decoder 620.

The Huffman decoder 620 obtains the quantized MDCT coefficients from the Huffman coded audio bit stream and outputs the quantized MDCT coefficients to the inverse quantizer 630. The dequantization unit 630 dequantizes the quantized MDCT coefficients to calculate the real MDCT coefficients. The real MDCT coefficient is generated as an output PCM signal which is a signal in the time domain through the IMDCT unit 640 and the subband filter bank 650.

At this time, the decoded PCM audio signal is not distinguished from the case where it is decoded from the general MP3 bit string without the watermark inserted.

FIG. 7 is a block diagram showing an embodiment of an AAC encoding apparatus to which a watermark embedding apparatus according to the present invention is applied. As in the MP3 encoding process, watermark insertion is performed during the Huffman encoding process.

Referring to FIG. 7, a psychoacoustic model unit 710, a preprocessor 720, a subband filter bank 730, a temporal noise shaping unit 740, an intensity / coupling unit 750, and a prediction A unit 760, a M / S (Mid / Side) stereo processing unit 770, a bit allocation and quantization unit 780, and a bit string generation unit 790.

The bit allocation and quantization unit 780 has an iterative loop structure and includes a compression / distortion control unit 781, a scale factor extraction unit 782, a quantization unit 783, and a Huffman encoding and watermark insertion unit 784. do.

That is, the audio signal in the form of pulse code modulation (PCM) is input to the psychoacoustic model unit 710 and the preprocessor 720. The preprocessor 720 changes and outputs a sampling frequency of an input audio signal according to a bit rate to be transmitted.

The psychoacoustic model unit 710 binds the input audio signals into signals of an appropriate scale factor band, and masks each scale using a masking phenomenon generated by interaction of the respective signals. Calculate the masking threshold. The output of the psychoacoustic model unit 710 is input to the filter bank 730, the TNS unit 740, the intensity / coupling unit 750, and the M / S stereo processing unit 770.

The filter bank 730 removes the statistical redundancy of the audio signal using the signal passed through the preprocessor 720 and the masking threshold of the psychoacoustic model unit 710 and outputs the statistical signal to the TNS unit 740. That is, the filter bank 730 generates MDCT coefficients having a maximum frequency resolution of 1024 by MDCT transform.

The MDCT coefficients generated by the filter bank 730 are allocated through the TNS unit 740, the strength / coupling unit 750, the prediction unit 760, and the M / S stereo processor 770. Is entered. The TNS unit 740, the strength / coupling unit 750, the prediction unit 760, and the M / S stereo processing unit 770 may be selectively used by the encoder.

That is, the TNS unit 740 receives the output of the filter bank 730 and the output of the psychoacoustic model unit 710 to control the temporal shape of the quantization noise in each window of the transform. In this case, time domain noise shaping is possible by applying a filtering process of frequency data.

The intensity / coupling unit 750 is a module for more efficiently processing a stereo signal. The intensity / coupling unit 750 receives an output of the psychoacoustic model unit 710 and an output of the TNS unit 740, and outputs one of two channels. Only quantized information about the scale factor band is encoded, and the remaining channels transmit only relative size information. The strength / coupling unit 750 is not a module that must be used in the encoder, and may determine whether to use each scale factor band in consideration of various matters.

The prediction unit 760 predicts the MDCT coefficient value (that is, the output value of the strength / coupling unit 750) of the current frame using the MDCT coefficients of the previous frame.

The amount of bits used can be reduced by quantizing and encoding the difference between the predicted value and the actual frequency component. In this case, the prediction unit 760 may also be selectively used in units of frames. In other words, when the predicting unit 760 is used, the complexity of predicting the next frequency coefficient may not be used. In addition, in some cases, since the difference due to the prediction has a greater probability than the original signal, the predicted bit generation may be larger than when the prediction is not performed. In this case, the prediction unit 760 is not used.

The M / S stereo processor 770 is also a module for processing stereo signals more efficiently. The M / S stereo processor 770 receives the output of the psychoacoustic model unit 710 and the output of the predictor 760 and receives the left channel signal and the right channel. The signals are then converted into plus and minus signals, respectively, and processed. The M / S stereo processor 770 is also not necessarily used in the encoder, but may determine whether the encoder is used in each scale factor band unit in consideration of various matters.

The output of the M / S stereo processor 770 is input to the scale factor extractor 782 in the bit allocation and quantization unit 780.

The scale factor extractor 782 extracts the scale factor under the control of the compression ratio / distortion control unit 781, and then normalizes the subband samples output from the M / S stereo processor 770 to the quantizer 783. .

The quantizer 783 quantizes the subband samples normalized by the scale factor extractor 782 and is input to the Huffman encoding and watermark inserting unit 784. That is, the quantization unit 783 quantizes subband samples of each band such that the quantization noise of each band is smaller than a masking threshold so that a human cannot feel it.

The signal quantized by the quantizer 783 is Huffman encoded and watermarked by the Huffman encoding and watermark inserting unit 784, and then output to the compression / distortion control unit 781 and the bit string generator 790. The bit string generator 790 collects information generated by each module 720 to 780 and generates and transmits a bit string.

As described above, in the AAC encoding process of FIG. 7, the structure of obtaining a masking threshold in the frequency domain through the psychoacoustic model and performing bit allocation so that quantization noise is not audible based on the psychoacoustic model is the same as that of the MP3 encoding process. However, in the AAC encoding process, the frequency conversion process performs a 2048-point MDCT having a higher frequency resolution than the MP3 (filter bank) and uses temporal noise shaping (TNS), a technique for shaping noise in the time domain. Using prediction to remove statistical redundancy between frames is very different from MP3. In addition, the AAC encoding process has a preprocessor 720 that can vary the sampling frequency according to the bit rate to be transmitted.

In this structure, Huffman coding of AAC is similar to MP3 in that two or four MDCT coefficient pairs (n-tupples) are encoded using the Huffman codebook. However, areas using the same codebook are divided into units called sections, except that there is no division such as an rzero area and a count1 area. That is, in the AAC encoding process, Huffman coding is performed by dividing sections so that regions having similar statistical characteristics with respect to MDCT spectral coefficients are grouped, and selecting the most suitable codebook for each section. In this case, a total of 12 Huffman codebooks are used in the AAC, and features such as a maximum value that can be expressed for each codebook are shown in FIG. 8.

In the case of the AAC, the maximum value that can be represented by the Huffman table is up to 15. To express more than that, an exception coding (ESC) must be performed. The exception handling method used in the AAC is different from that of the MP3 and is as follows.

That is, to encode 16 or more information, first Huffman encodes a codeword corresponding to 16, and then adds escape_sequence configured as shown in Equation 1 below.

escape_sequence = <escape_prefix> <escape_separator> <escape_word>

Where <escape_prefix> represents N bits of binary number '1', <escape_separator> represents binary number '0', and <escape_word> represents an N + 4 bit unsigned integer. Indicates. Therefore, N is selected as the minimum value that can represent the MDCT coefficient where 2 ^{(N + 4)} is placed in the <escape_word> position.

In MP3, the number of bits used for encoding by exception processing is determined by selection of the Huffman codebook, whereas in AAC, the number of bits used for each exception processing coefficient is different.

In this case, the watermark embedding process in the Huffman encoding and watermark embedding unit 784 is similar to the watermark embedding process in the MP3 encoder. That is, the entire Huffman codebook used in the AAC of FIG. 8 is divided into two groups, the group is first selected according to the binary watermark information to be inserted, and the appropriate Huffman codebook is found in the group to insert the watermark. do.

Referring to the Huffman codebook of FIG. 8, except for codebooks 0 and 11, two codebooks exist for each case having the same maximum absolute value. Therefore, classifying them to correspond to binary '0' and '1' It is possible.

9A and 9B are AAC Huffman codebooks classified corresponding to binary watermark information '0' and '1' used for watermark insertion in the AAC encoding process according to the present invention. In this case, the watermark is not inserted in the case of using Huffman codebook 0.

On the other hand, when using the codebook 11, the watermark is not inserted into the selected codebook itself. In this case, the watermark is inserted in the process of performing the exception coding. Detailed description thereof is as follows.

That is, among the escape_sequences used for encoding 16 or more information in the AAC, when the <escape_prefix> is set, the N value is determined according to the watermark bit to be inserted. In the conventional method, 2 ^{(N + 4)} was determined to be the minimum N value that can sufficiently express the <escape_word> value. However, in order to insert the watermark information according to the present invention, when the binary watermark to be inserted is '0', the minimum even number that can represent <escape_word> is 2 ^{(N + 4)} . If the binary watermark to be inserted is '1', ^N is the minimum odd number N that can represent <escape_word> as 2 ^{(N + 4)} .

In other words, the embedded binary watermark information is encoded from the N value of the number of <escape_prefix>.

In the AAC encoder, the number of sections related to the number of Huffman codebooks used is not fixed. In addition, the number of escape_sequence is used according to the characteristics of the input audio signal. Therefore, the number of embedable watermark bits varies depending on the frame.

FIG. 10 is a block diagram illustrating an AAC decoder to which the watermark extracting apparatus according to the present invention is applied, and operates in the reverse order of the AAC encoder to which the watermark inserting apparatus of FIG. 7 is applied.

Referring to FIG. 10, a bit string demultiplexing and watermark extractor 910, a Huffman decoding and watermark extractor 920, an inverse quantizer 930, an M / S stereo processor 940, and a predictor 950. , The strength / coupling unit 960, the TNS unit 970, the filter bank 980, and the AAC gain control unit 990.

That is, when the AAC bit string is input, the information necessary for decoding is extracted through the bit string demultiplexing process, and Huffman decoding, dequantization, M / S stereo reconstruction, prediction, and intensity / coupling are performed. The PCM audio signal is decoded through stereo reconstruction, TNS filtering, and filter bank. This process is the same as the conventional AAC decoding process.

In this case, when a bit string having a watermark inserted into the bit string demultiplexing and watermark extractor 910 is received, information on a section Huffman codebook transmitted with reference to FIGS. 9A and 9B during a bit string demultiplexing process. The watermark can be extracted primarily from. That is, for a signal that can be represented by the Huffman codebook, watermark information may be extracted from the Huffman codebook used for Huffman coding and a group including the Huffman codebook. However, in the case of a signal that cannot be represented by the Huffman codebook, watermark information may be extracted from an N value of the number of <escape_prefix> used for exception processing.

Accordingly, the watermark information extracted by the bit string demultiplexing and watermark extractor 910 is output to the Huffman decoding and watermark extractor 920. That is, the watermark inserted in the escape_sequence is extracted by decoding the number value N of <escape_prefix> during Huffman decoding. As such, the watermark information is decoded in two parts of the AAC decoding process, and the actual processing method is very simple.

Meanwhile, the watermark-embedded bit string can be decoded through a conventional AAC decoder, and the decoded PCM audio signal is not distinguished from the case where no watermark is inserted. In addition, the AAC decoder according to the present invention can decode an audio signal in the same manner as in the conventional method even for a bit string without a watermark embedded therein.

The embodiment of the present invention has been described based on the application of MP3 and AAC encoding among high quality audio encoding methods. However, it can be seen that watermarks can be inserted by applying the same principle to other audio and video encoding methods for performing data to be transmitted through Huffman coding using a codebook.

On the other hand, the terms used in the present invention (terminology) are terms defined in consideration of the functions in the present invention may vary according to the intention or practice of those skilled in the art, the definitions are the overall contents of the present invention It should be based on.

The present invention is not limited to the above-described embodiments, and as can be seen in the appended claims, modifications can be made by those skilled in the art to which the invention pertains, and such modifications are within the scope of the present invention.

An apparatus and method for embedding / detecting a watermark according to the present invention described above are described as follows.

First, by inserting and detecting watermark information in the Huffman encoding and decoding process, the watermark can be inserted and detected without affecting the quality of the original signal. Therefore, separate information different from the original signal can be transmitted through an audio and video compressed bit stream compatible with the conventional art.

Second, when the watermark information is inserted and transmitted as described above, it may be used for copyright protection of the corresponding content, or may be used as control information for controlling access such as decoding, copying, and playback, and for monitoring. It can also be used to transmit information, synchronization information (LipSync) between audio and video signals, additional information such as song names, lyrics, and subtitles. In addition, the bit string may be used for the purpose of determining whether the bit string is damaged during the channel transmission process or for determining whether the bit string is damaged. In addition, when a method of extracting a watermark is disclosed only to a specific person, the corresponding watermark may be used for secret communication.

Third, the bit string in which the watermark is inserted can be decoded without distortion through a conventional decoder. In this case, the conventional decoder can maintain compatibility because it does not recognize whether or not the watermark is inserted. That is, a separate information transmission channel is secured while maintaining compatibility with the conventional decoder, and it can be used for various applications.

Fourth, the insertion and extraction of the watermark has an advantage that it can be implemented only by adding a negligible amount of computation in the conventional audio and video encoding process. Therefore, the method of embedding and extracting the watermark according to the present invention requires very small calculation amount and is easy to implement.

Those skilled in the art will appreciate that various changes and modifications can be made without departing from the spirit of the present invention.

Therefore, the technical scope of the present invention should not be limited to the contents described in the embodiments, but should be defined by the claims.

Claims (17)

Classify a plurality of Huffman codebooks used for Huffman coding into at least a plurality of groups, select one of the groups classified according to the watermark information to be inserted, and select a Huffman codebook to be used for Huffman coding of an input signal from the selected group. An encoder for selecting and transmitting Huffman encoding; And And a decoder for performing Huffman decoding of the transmitted signal using the Huffman codebook used for Huffman coding of the signal transmitted from the encoder and extracting watermark information based on the group to which the Huffman codebook belongs. A device for embedding / detecting watermarks. The method of claim 1, And the group and Huffman codebook information selected by the encoder is transmitted to the decoder as side information. The method of claim 1, The coder uses an MP3 encoder and classifies the Huffman codebooks into a plurality of groups using at least one of a maximum value for each Huffman codebook, statistical characteristics of each Huffman codebook, and linbits values in the Huffman codebook. Device for mark insertion / detection. The method of claim 3, wherein And the MP3 encoder does not insert a watermark when Huffman encodes an input signal using Huffman codebooks 0 and 1. The method of claim 1, And the encoder uses an AAC encoder, and classifies the Huffman codebooks into a plurality of groups based on the maximum absolute value of each Huffman codebook. The method of claim 5, wherein Wherein the AAC encoder does not insert a watermark when Huffman encodes an input signal using Huffman codebook # 0. The method of claim 5, wherein When the AAC encoder uses the Huffman codebook 11 to Huffman encode the input signal, the AAC encoder does not insert a watermark for the selected Huffman codebook itself, and inserts the watermark in the process of performing exception processing (ESC). A device for embedding / detecting watermarks. The method of claim 7, wherein The AAC encoder adds escape_sequence (= <escape_prefix> <escape_separator> <escape_word>) to a codeword corresponding to the maximum absolute value for an input signal having a size that cannot be represented by the Huffman codebook. And an N value which is the number of <escape_prefix> according to the mark information. Here, <escape_prefix> represents N-bit binary '1', <escape_separator> represents binary '0', and <escape_word> represents a coefficient expressed as an unsigned integer value of N + 4 bits. The method of claim 8, If the binary watermark to be inserted is '0', 2 ^{(N + 4)} determines the minimum even number N that can represent <escape_word>, and if it is '1', 2 ^{(N + 4). )} as a watermark unit for embedding / detection, characterized in that to determine the minimum of the odd number, N to express <escape_word>. The method of claim 8, And the decoder extracts watermark information from a group to which the Huffman codebook used for Huffman coding of the signal transmitted from the encoder belongs and N value of the number of <escape_prefix>. Classify a plurality of Huffman codebooks used for Huffman coding into at least a plurality of groups, select one of the groups classified according to the watermark information to be inserted, and select a Huffman codebook to be used for Huffman coding of an input signal from the selected group. A transmission step of selecting and transmitting Huffman encoding; And And receiving a Huffman decoding of the transmitted signal using the Huffman codebook used for Huffman coding of the transmitted signal and extracting watermark information based on the group to which the Huffman codebook belongs. A method for embedding / detecting watermarks. The method of claim 11, The group and Huffman codebook information selected in the transmitting step is transmitted as side information. The method of claim 11, The transmitting step performs MP3 encoding on the input signal and classifies the Huffman codebooks into a plurality of groups using at least one of a maximum value for each Huffman codebook, statistical characteristics of each Huffman codebook, and linbits values in the Huffman codebook. Method for embedding / detecting watermarks, characterized in that. The method of claim 11, The transmitting step performs AAC encoding and classifies the Huffman codebooks into a plurality of groups based on the maximum absolute value of each Huffman codebook. The method of claim 14, In the transmitting step, when Huffman encodes an input signal using Huffman codebook 11, the watermark is not inserted into the selected Huffman codebook itself, and the watermark is inserted in the process of performing exception processing (ESC). A method for embedding / detecting watermarks. The method of claim 15, In the transmitting step, Huffman encoding is performed by adding escape_sequence (= <escape_prefix> <escape_separator> <escape_word>) to a codeword corresponding to the maximum absolute value that can be expressed for an input signal having a size that cannot be represented by the Huffman codebook. The method for embedding / detecting a watermark according to claim information, characterized by determining an N value which is the number of <escape_prefix> according to the mark information. Here, <escape_prefix> represents N-bit binary '1', <escape_separator> represents binary '0', and <escape_word> represents a coefficient expressed as an unsigned integer value of N + 4 bits. The method of claim 16, wherein the receiving step Extracting first watermark information from a group to which the Huffman codebook used for Huffman coding of the signal transmitted in the transmitting step belongs; And extracting second watermark information from an N value of the number of <escape_prefix>.

Images