KR100355268B1 - Digital Watermark Extraction Apparatus for Copyright Protection of Digital Audio/Video/Multimedia - Google Patents

Abstract

The present invention relates to a method of extracting digital watermark, which is copyright protection information included in digital audio / video, using time-varying prediction modeling, and extracts a watermark from audio and video having watermark embedded therein without the original audio or video. Even if various attacks are applied to remove the watermark on the watermark embedded audio / video signal, the watermark information can be extracted without being removed.

As the use of computer networks and computers such as the Internet is rapidly increasing, the use and dissemination of digital multimedia data is becoming common. However, such multimedia data can be easily copied due to the digital property, and anyone can easily obtain digital data through illegal copying. As a result, there is a demand to prevent illegal copying and to protect the copyright and ownership of the author who produced the multimedia contents.

The present invention can accurately extract a watermark without using the original audio / video and multimedia using a time varying predictive analyzer and a time varying predictive filter in a watermark-embedded audio / video and multimedia. This watermark extraction technique can be applied to various fields.

Description

Digital Watermark Extraction Apparatus for Copyright Protection of Digital Audio / Video / Multimedia}

The present invention relates to a device for extracting a digital watermark including copyright information embedded in digital audio / video / multimedia, and more specifically, to a digital watermark without using original audio / video / multimedia using time-varying prediction modeling. The present invention relates to a device for accurately extracting copyright information.

As the use of computer networks and computers such as the Internet is rapidly increasing, the use and dissemination of digital multimedia data is becoming common. However, such multimedia data can be easily copied due to the digital property, and anyone can easily obtain digital data through illegal copying. As a result, there has been a demand to prevent illegal copying and to protect copyrights and ownership of writers producing multimedia contents.

To date, the most widely used data protection method is the data encryption method. This data encryption method is impossible to access the data without knowing the password, but once decrypted, there is a problem that can be illegally copied and distributed without any sanction.

To solve this problem, a method of inserting a watermark, which is a kind of digital signature, into the original signal has been proposed. A digital watermark is a kind of copyright protection technology that solves a copyright problem by inserting a signal of an author unrecognizable in a multimedia and extracting a watermark embedded in the multimedia later. Watermark may be inserted in the form of a logo that can be seen by humans, but due to the ease of deletion of watermarks and the deterioration of the multimedia quality, there are many ways to insert a watermark in a form that cannot be seen in recent years. Research is used.

There are some requirements for embedding invisible watermarks. First, the inserted watermark must have a non-perceptibility that cannot be distinguished by the human eye. That is, the degradation of the quality of the original multimedia caused by the insertion of the watermark should not occur, and should almost never occur. Second, watermarks must be robust. In other words, the inserted watermark must have robustness that is not deleted even when subjected to various data processing or deliberate attacks by others who want to delete the watermark. Third, the watermark must be unambiguous. Inhomogeneity is the property of being able to identify exactly who the inserted watermark is. That is, there should be no problem such as determining that an uninserted watermark is inserted or misinterpreting the inserted watermark as a watermark of another person.

Most digital watermark extraction methods published to date require original data when extracting watermark. Correlation between watermark embedded audio / video and original audio / video is calculated and correlated with a known watermark signal. A method of detecting the copyright information included in the watermark was obtained.

Since the conventional watermark extraction method requires original data when extracting the watermark, many limitations are imposed on the use of real life where watermark should be detected without original data such as audio or video playback or digital broadcasting or communication in a portable device. As a result, a lot of time is required for watermark detection, making it difficult to detect in real time. In addition, when a random attack is applied to the audio or video embedded with the watermark, watermark extraction is impossible.

Therefore, an object of the present invention for solving the above problems of the prior art does not require the original data when extracting the watermark, it is possible to accurately extract the copyright information contained in the watermark even if an arbitrary attack is applied from the outside The present invention provides a digital watermark extraction apparatus for digital audio / video / multimedia.

1 is an overall configuration diagram of an apparatus for extracting digital watermark of digital multimedia according to an embodiment of the present invention;

FIG. 2 is a detailed block diagram of an audio watermark extractor shown in FIG. 1;

FIG. 3 is a detailed block diagram of the video watermark extracting unit shown in FIG. 1;

FIG. 4 is a diagram illustrating various methods of scanning video data by the 1D / 2D time varying prediction analysis unit to the 1D / 2D time varying prediction filter of FIG. 3.

※ Explanation of code about main part of drawing ※

100: demultiplexing unit

200: audio decoder

300: audio watermark extraction unit

301: one-dimensional time-varying prediction analysis unit 302: synchronization information detection unit

303: 1-dimensional time-varying prediction filter unit 304: pseudo noise sequence generator

305: 1-dimensional time-variable prediction part 306: Matching filter part

307: error correction decoder

400: video decoder

500: video watermark extraction unit

501: frequency converter 502: one-dimensional and two-dimensional time-varying prediction analysis unit

503: synchronization information extraction unit 504: one-dimensional and two-dimensional time-varying prediction filter unit

505: pseudo noise sequence generator 506: one-dimensional / two-dimensional time-varying prediction filter

507: matching filter unit 508: error correction decoding unit,

510: Zig-zag scanning method 520: Horizontal scanning method

530: vertical scanning method 540: two-dimensional spectrum estimation method

600: audio output control unit

700: video output control unit

According to an aspect of the present invention, there is provided an apparatus for extracting digital watermark of digital multimedia, comprising: a demultiplexer for receiving a watermark-embedded digital multimedia signal into an audio bitstream and a video bitstream;

An audio watermark extraction unit for extracting a watermark including audio copyright information from the audio bitstream;

A video watermark extractor for extracting a watermark including video copyright information from the video bitstream;

An audio output control unit which controls the output of the audio bitstream using the audio copyright information;

And a video output control unit which controls the output of the video bitstream using the video copyright information.

In addition, the digital watermark extraction apparatus of digital audio according to the present invention, the synchronization information extracting unit for extracting the synchronization information from the audio bitstream embedded with the watermark;

A time-varying prediction analyzer for extracting filter values by analyzing the spectrum of the audio bitstream using the synchronization information;

A first time-varying prediction filter unit extracting a watermark signal component by filtering the spectrum of the audio bitstream using the filter value;

A pseudo noise sequence generator for generating a pseudo noise sequence using the synchronization information;

A second time-varying prediction filter unit for filtering the pseudo noise sequence using the filter value, and

And a matched filter unit configured to receive the first time-varying prediction filter unit and the second time-varying prediction filter unit to extract copyright information included in the watermark.

In addition, the digital watermark extraction apparatus of the digital video according to the present invention, the synchronization information extraction unit for extracting the synchronization information from the video bitstream embedded with the watermark;

A time-varying prediction analyzer for extracting filter values by analyzing the spectrum of the video bitstream using the synchronization information;

A first time-varying prediction filter unit extracting a watermark signal component by filtering the spectrum of the video bitstream using the filter value;

A pseudo noise sequence generator for generating a pseudo noise sequence using the synchronization information;

A second time-varying prediction filter unit for filtering the pseudo noise sequence using the filter value, and

And a matched filter unit configured to receive the first time-varying prediction filter unit and the second time-varying prediction filter unit to extract copyright information included in the watermark.

Hereinafter, an apparatus for extracting digital watermark of digital audio / video / multimedia according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram showing an apparatus for extracting digital watermark of digital multimedia according to the present invention.

In the drawing, 100 is a demultiplexer, 200 is an audio decoder, 300 is an audio watermark extractor, 400 is a video decoder, 500 is a video watermark extractor, 600 is an audio output controller, and 700 is a video output controller.

Referring to FIG. 1, an apparatus for extracting digital watermark of digital multimedia may include a demultiplexer 100 which receives a compressed audio / video signal and separates the compressed audio / video signal into an audio and video bit stream, and decodes the compressed audio bit stream. A decoder 200, an audio watermark extractor 300 for extracting audio copyright information from the decoded audio bit stream, a video decoder 400 for decoding the compressed video bit stream, and a decoded video bit stream. A video watermark extractor 500 for extracting video copyright information from the audio output unit, an audio output control unit 600 for receiving audio copyright information from the audio watermark extractor 300, and controlling audio output; And a video output controller 700 for receiving video copyright information from the 500 and controlling the video output.

When the audio and video compressed in FIG. 1 are input, the audio watermark extractor 300 and the video watermark extract are passed through the demultiplexer 100, the audio decoder 200, and the video decoder 400, respectively. When the audio and video are input to the unit 500 but are not compressed, the audio watermark extractor 300 and the video watermark extractor 500 are directly input.

In addition, when the audio and video copyright information extracted by the audio watermark extractor 300 and the video watermark extractor 500 includes an output control signal such as copy control information in FIG. 1, the audio output controller 600. And the video output controller 700 controls the audio and video output using the output control signal information.

FIG. 2 is a detailed configuration diagram of the audio watermark extractor 300 shown in FIG. 1. In the figure, 301 is a one-dimensional time-varying prediction analysis unit, 302 is a synchronization information extracting unit, 303 is a one-dimensional time-varying prediction filter unit, 304 is a pseudo noise sequence generating unit, 305 is a one-dimensional time-varying prediction filter unit, 306 is a matched filter unit, 307 is an error correction decoding unit.

Referring to FIG. 2, an audio signal having a watermark embedded therein is input to the synchronization information extracting unit 302 to extract synchronization information. This is because when the attack is applied to the audio signal in which the watermark is inserted, the synchronization of the watermark is not synchronized with the position where the watermark is initially inserted. Therefore, the watermark may not be detected. Therefore, the synchronization information extracting unit 302 extracts the synchronization information and extracts the synchronization information. To keep them motivated.

The one-dimensional time-varying prediction analysis unit 301 analyzes the spectrum of the input audio signal by performing AR (Autoregressive) modeling or ARMA (Autoregressive Moving Average) modeling using the synchronization information of the synchronization information extracting unit 301. The filter value is transmitted to the one-dimensional time-varying prediction filter unit 303. The AR (Autoregressive) modeling or the Autoregressive Moving Average (ARMA) modeling is a kind of method for predicting a signal, and the spectrum predicted by the filter obtained using the AR or the ARMA corresponds to the spectrum of the audio signal.

The 1D time varying prediction filter unit 303 filters the spectral components of the input audio signal using the filter value received from the 1D time varying prediction analysis unit 301. The inherent spectrum of the audio signal is removed from the audio signal having the watermark inserted through the one-dimensional time-varying prediction filter unit 303, and the watermark signal component remains.

The watermarked audio signal consists of the original audio component and the watermark signal component. Therefore, since the spectrum of the audio signal can be predicted using the AR or ARMA filter value, removing it from the watermarked audio signal leaves only the watermark signal component.

The pseudo-noise sequence generated by the pseudo-noise sequence generator 304 is filtered by the one-dimensional time-varying prediction filter unit 305 using the filter value obtained by the one-dimensional time-varying prediction analyzer 301 and then matched by the matching filter unit 306. Is entered. When filtering the pseudo noise sequence, the output of the 1D time-varying prediction filter unit 303 and the filtered pseudo noise sequence have maximum correlation, so that the matched filter unit 306 can accurately extract copyright information. The copyright information obtained through the matched filter unit 306 corrects a bit error that may occur if it occurs through the error correction decoder 307.

The matched filter unit 306 determines how much correlation the two input signals have to each other. If there is a correlation, a large value is output, and if there is no correlation, a very small value is output. The signal passing through the one-dimensional time-varying prediction filter unit 303 is a signal from which an audio component has been removed and only the watermark component remains, and this signal is a signal having a component such as a pseudo noise signal passing through the one-dimensional time-varying prediction filter unit 305. In this case, the matching filter unit can obtain the maximum correlation.

FIG. 3 is a detailed block diagram of the video watermark extractor 500 shown in FIG. 1. In the drawing, 501 is a frequency converter, 502 is a 1D / 2D time varying prediction analysis unit, 503 is a synchronization information extracting unit, 504 is a 1D / 2D time varying prediction filter unit, 505 is a pseudo noise sequence generator, and 506 is 1 The dimensional / two-dimensional time-varying prediction filter unit, 507 is a matching filter unit, and 508 is an error correction decoding unit.

Referring to FIG. 3, a video signal having a watermark embedded therein is converted into data in a two-dimensional frequency domain through a frequency converter 501.

The frequency converter 501 may be omitted when the watermark extraction process is performed in the time domain. The video signal having the watermark embedded therein is input to the synchronization information extracting unit 503 to extract the synchronization information. When an attack is applied to a video signal containing a watermark, watermark detection may not be possible because it is not synchronized with the position where the watermark is inserted in the first place. Therefore, synchronization is extracted using the synchronization information extracted by the synchronization information extraction unit 503. Fit.

The 1D / 2D time varying prediction analysis unit 502 inputs video through 1D or 2D AR (Autoregressive) modeling or ARMA (Autoregressive Moving Average) modeling using the synchronization information of the synchronization information extractor 503. The spectrum of the signal is analyzed and the corresponding filter value is transmitted to the 1D / 2D time varying prediction filter unit 504. The AR (Autoregressive) modeling or the Autoregressive Moving Average (ARMA) modeling is a kind of method for predicting a signal, and the spectrum predicted by the filter obtained using the AR or the ARMA corresponds to the spectrum of the audio signal.

The 1D / 2D time varying prediction filter unit 504 filters the spectral components of the input video signal using a 1D or 2D filter value received from the 1D / 2D time varying prediction analysis unit 502. The watermark-embedded video signal passing through the 1D / 2D time-varying prediction filter unit 504 removes an inherent spectrum of the video signal and leaves a watermark signal component.

The pseudonoise sequence generated by the pseudonoise sequence generation unit 505 is filtered by the 1D / 2D time varying prediction filter unit 506 using the filter value obtained by the 1D / 2D time varying prediction analysis unit 502. The filter is passed to the matching filter unit 507. When filtering the pseudo noise sequence, the output of the 1D / 2D time-varying prediction filter unit 504 has a maximum correlation with the filtered pseudo noise sequence, and the matching filter unit 507 can accurately extract copyright information. The copyright information obtained through the matched filter unit 507 corrects a bit error that may occur if it occurs through the error correction decoder 508.

The one-dimensional and two-dimensional time-varying prediction analysis unit 502, the one-dimensional and two-dimensional time varying prediction filter unit 504, and the one-dimensional and two-dimensional time varying prediction filter unit 506 perform one-dimensional time-varying prediction analysis and filtering. Or in a two-dimensional region.

In case of time-varying predictive analysis and filtering in one-dimensional domain, two-dimensional video data should be made in one dimension. 4 illustrates various scanning methods for making two-dimensional video data into one-dimensional data.

That is, when time-varying prediction analysis and filtering are performed in the 1-dimensional region, the 2D video is zigzag-scanned in one dimension by using a zigzag scanning method 510, a horizontal scanning method 520, or a vertical scanning method 530. After obtaining the filter value which is a predictive coefficient through 1D AR modeling or ARMA modeling, 1D time varying prediction filtering is performed to filter the spectrum of the video signal so that only watermark signal components remain.

On the other hand, in time-variable prediction analysis and filtering in the two-dimensional region, the filter value which is a prediction coefficient is obtained through two-dimensional AR modeling or ARMA modeling, which is a two-dimensional spectrum estimation method 540, and then the video signal is subjected to two-dimensional time-varying prediction filtering. Filter the spectrum of so that only the watermark signal components remain.

While the invention has been described above based on the preferred embodiments thereof, these embodiments are intended to illustrate rather than limit the invention. It will be apparent to those skilled in the art that various changes, modifications, or adjustments to the above embodiments can be made without departing from the spirit of the invention. Therefore, the protection scope of the present invention will be limited only by the appended claims, and should be construed as including all such changes, modifications or adjustments.

As described above, the present invention can be used as a core technology for copyright protection and copy protection of digital audio / video and multimedia, and watermark can be extracted from original audio and video signals without watermark data without watermark. It is easy to use in real life. In addition, even if any attack is applied to the audio and video signal embedded in the watermark, the watermark can be efficiently extracted, and can also be used for authentication that can confirm whether the digital audio and video are manipulated or modified. . In addition, when the digital data is reproduced through a portable device or other audio and video equipment, the allowable number of copies can be adjusted by using a watermark, which can be used for usage control.

Claims (20)

A demultiplexer for receiving a watermark-embedded digital multimedia signal into an audio bitstream and a video bitstream; An audio watermark extraction unit for extracting a watermark including audio copyright information from the audio bitstream; A video watermark extractor for extracting a watermark including video copyright information from the video bitstream; An audio output control unit which controls the output of the audio bitstream using the audio copyright information; And a video output control unit for controlling the output of the video bitstream by using the video copyright information. The method of claim 1, When the multimedia signal in which the watermark is inserted is input in a compressed state, An audio decoder which decodes the audio bitstream separated by the demultiplexer and provides the decoded audio bitstream to the audio watermark extractor; And a video decoder which decodes the video bitstream separated by the demultiplexer and provides the decoded video bitstream to the video watermark extractor. The method according to claim 1 or 2, The audio watermark extraction unit, A synchronization information extraction unit for extracting synchronization information from the audio bitstream; A time-varying prediction analyzer for extracting filter values by analyzing the spectrum of the audio bitstream using the synchronization information; A first time-varying prediction filter unit extracting a watermark signal component by filtering the spectrum of the audio bitstream using the filter value; A pseudo noise sequence generator for generating a pseudo noise sequence using the synchronization information; A second time-varying prediction filter unit for filtering the pseudo noise sequence using the filter value, and And a matched filter unit for receiving the first time-varying prediction filter unit and the second time-varying prediction filter unit to extract copyright information included in the watermark. The method of claim 3, wherein The audio watermark extraction unit, And an error correction decoding unit for correcting a bit error of copyright information output from the matching filter unit. The method of claim 3, wherein The time-varying prediction analyzer extracts the filter value, which is a prediction coefficient, by using a one-dimensional spectral estimation method, And the first time-varying prediction filter unit and the second time-varying prediction filter unit filter the spectrum of the video bitstream through one-dimensional time-varying prediction filtering. The method according to claim 1 or 2, The video watermark extractor, A synchronization information extraction unit for extracting synchronization information from the video bitstream; A time-varying prediction analyzer for extracting filter values by analyzing the spectrum of the video bitstream using the synchronization information; A first time-varying prediction filter unit extracting a watermark signal component by filtering the spectrum of the video bitstream using the filter value; A pseudo noise sequence generator for generating a pseudo noise sequence using the synchronization information; A second time-varying prediction filter unit for filtering the pseudo noise sequence using the filter value, and And a matched filter unit for receiving the first time-varying prediction filter unit and the second time-varying prediction filter unit to extract copyright information included in the watermark. The method of claim 6, The video watermark extractor, And a frequency converter for converting the video bitstream into a two-dimensional frequency domain and providing the synchronization information extractor, the time-varying prediction analyzing unit and the first time-varying prediction filter unit. The method of claim 6, The video watermark extractor, And an error correction decoding unit for correcting a bit error of copyright information output from the matching filter unit. The method of claim 6, The time-varying prediction analyzer extracts the filter value that is a prediction coefficient by using a two-dimensional spectrum estimation method, And the first time-varying prediction filter unit and the second time-varying prediction filter unit filter the spectrum of the video bitstream through two-dimensional time-varying prediction filtering. The method of claim 6, The time-varying prediction analyzer extracts the filter value, which is a prediction coefficient, by converting a 2D video into 1D and then using a 1D spectrum estimation method. The first time-varying prediction filter unit and the second time-varying prediction filter unit convert the 2D video into 1D and filter the spectrum of the video bitstream through 1D time varying prediction filtering. Device. The method of claim 10, The method of converting the two-dimensional video to one-dimensional, And converting the 2D video into one dimension by scanning by one of a zigzag scanning method, a horizontal scanning method, and a vertical scanning method. A synchronization information extracting unit for extracting synchronization information from the audio bitstream having the watermark embedded therein; A time-varying prediction analyzer for extracting filter values by analyzing the spectrum of the audio bitstream using the synchronization information; A first time-varying prediction filter unit extracting a watermark signal component by filtering the spectrum of the audio bitstream using the filter value; A pseudo noise sequence generator for generating a pseudo noise sequence using the synchronization information; A second time-varying prediction filter unit for filtering the pseudo noise sequence using the filter value, and And a matched filter unit configured to receive the first time-varying prediction filter unit and the second time-varying prediction filter unit to extract copyright information included in the watermark. The method of claim 12, And an error correction decoding unit for correcting a bit error of copyright information outputted from the matching filter unit. The method of claim 12, The time-varying prediction analyzer extracts the filter value, which is a prediction coefficient, by using a one-dimensional spectral estimation method, And the first time-varying prediction filter unit and the second time-varying prediction filter unit filter the spectrum of the video bitstream through one-dimensional time-varying prediction filtering. A synchronization information extracting unit for extracting synchronization information from the video bitstream having the watermark embedded therein; A time-varying prediction analyzer for extracting filter values by analyzing the spectrum of the video bitstream using the synchronization information; A first time-varying prediction filter unit extracting a watermark signal component by filtering the spectrum of the video bitstream using the filter value; A pseudo noise sequence generator for generating a pseudo noise sequence using the synchronization information; A second time-varying prediction filter unit for filtering the pseudo noise sequence using the filter value, and And a matched filter unit which receives the first time-varying prediction filter unit and the second time-varying prediction filter unit to extract copyright information included in the watermark. The method of claim 15, And a frequency converter for converting the video bitstream into a two-dimensional frequency domain and providing the synchronization information extractor, the time-varying prediction analyzing unit, and the first time-varying prediction filter unit. The method of claim 15, And an error correction decoding unit for correcting bit errors of copyright information output from the matching filter unit. The method of claim 15, The time-varying prediction analyzing unit extracts the filter value, which is a prediction coefficient, by using a 2D spectrum estimation method, and the first time-varying prediction filter unit and the second time-varying prediction filter unit perform a 2D time-varying prediction filtering on the spectrum of the video bitstream. Digital watermark extraction apparatus for digital video, characterized in that for filtering the. The method of claim 15, The time-varying prediction analyzer extracts the filter value, which is a prediction coefficient, by converting a 2D video into 1D and then using a 1D spectrum estimation method. The first time-varying prediction filter unit and the second time-varying prediction filter unit transform the 2D video into 1D and filter the spectrum of the video bitstream through 1D time varying prediction filtering. Device. The method of claim 19, The method of converting the two-dimensional video to one-dimensional, And converting the 2D video into one dimension by scanning by one of a zigzag scanning method, a horizontal scanning method, and a vertical scanning method.