KR101271197B1 - Watermarking system using horizontal noise mean shifting - Google Patents

Abstract

The present invention relates to a watermarking system using the horizontal movement of the mean of the horizontal axis. The present invention not only satisfies the invisibility by inserting the watermark in the form of noise, but also protects the copyright of the center, left eye, and right eye images. It is an object of the present invention to detect an inserted watermark anywhere in depth information arbitrarily set by a user by inserting a watermark using an average shift of horizontal axis noise, which is an invariant feature in three-dimensional conversion.
According to an aspect of the present invention, there is provided a watermark inserter for inserting a watermark signal by shifting an average of a noise distribution of a center image; The watermark signal is extracted from the noise column extracted from the detection target region by averaging, and the watermark signal is detected by comparing the extracted watermark signal with a normal correlation coefficient between the reference watermark signal and a threshold value. Mark detector; .

Description

WATERMARKING SYSTEM USING HORIZONTAL NOISE MEAN SHIFTING}

The present invention relates to a watermarking system for protecting content in a DIBR method, and more particularly, by arbitrarily setting a watermark by inserting a watermark using an average shift of horizontal axis noise, which is an invariant feature of two-dimensional and three-dimensional conversion. The present invention relates to a system capable of detecting an embedded watermark in any depth information.

Due to the rapid spread of three-dimensional image playback equipment, many content producers began to produce three-dimensional content. These contents are represented in various ways, of which two three-dimensional expression methods are most widely used. The first method is stereo image recording, which attaches left and right eye images directly.

This method has the advantage of simple storage and playback, but has a problem that the transmission bandwidth of the stereoscopic image is twice the bandwidth of the 2D image.

Second, DIBR (Depth-Image-Based Rendering) has the center image and the depth image, so that the user converts the image into left and right eyes before playing it [1]. The human eye is not sensitive to slight changes in depth, so even if you compress the depth image strongly, you will not notice it. The transmission efficiency is much better than that of the first method because this characteristic is utilized in the DIBR method.

In addition, since the DIBR method generates the left and right eye images directly from the device using the center image and the depth image, the user can easily adjust the depth sense desired by the user and selectively view the 2D and 3D images as the user wants. It is another advantage. Because of these many advantages, the DIBR method has attracted much attention in recent years in the academic and industrial world.

However, copyright protection of images stored in the DIBR method is not easy. Not only the center image but also the converted left eye and right eye images have sufficient value as two-dimensional contents, so if any one of them leaks, a problem occurs. Therefore, all three images must be protected. In other words, there is a need for a new type of watermark technology that can detect watermark signals embedded in the center image in all three images of the center, left eye and right eye without the help of a depth image.

Recently, several watermarking techniques have been proposed for the protection of contents stored in the DIBR method [2-3]. The LEE study proposed a method of embedding a watermark signal into a center image and extracting from the center image using depth information [2]. The main concern of this method was image quality, so when the center image was converted into left and right eye images, it had very good invisibility, but the watermark signal was destroyed in the left and right eye images, which prevented watermark detection. Had a problem.

Lin has also proposed a watermarking technique that takes into account two-dimensional to three-dimensional transformations [3] . Watermarking that protects not only the center image but also the left and right eyes by inserting watermark signals for each image so that they can be detected from the left and right eyes to be generated in a predefined depth environment using the center and depth images. Introduced the technology. This method is meaningful as a stereoscopic watermarking technique that considers two-dimensional to three-dimensional transform for the first time.

However, this method has a problem in that the image quality deteriorates after the watermark information is inserted because the watermark signals for the center, left eye, and right eye images are inserted all at once. Also, since watermark information inserted only in a fixed depth environment can be detected, the user cannot arbitrarily adjust the depth.

Reference

[1] C. Fehn, "Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3-DTV," Proc. SPIE, vol. 5029 p. 93, San jose, CA, USA, January 2004.

[2] M. J. Lee, J. W. Lee, and H. K. Lee, "perceptualwatermarking for stereoscopic video using depth information," Proc. IIH-MSP 2011, To be published

[3] Y. H. Lin, and J. L. Wu, "A digital blind watermarking scheme for depth-image-based rendering 3D images," IEEE Trans. on Broadcasting, vol. 57, no. 2, pp. 602 ?? 611, 2011.

The present invention has been made in view of the above problems, and the first object of the present invention is not only to satisfy the invisibility by inserting a watermark in the form of noise, but also to protect the copyright of all of the center, left and right eye images. It's in the ship.

A second object of the present invention is to insert a watermark using the average shift of horizontal axis noise, which is an invariant feature in two-dimensional to three-dimensional conversion, so that the inserted watermark can be detected anywhere in the depth information arbitrarily set by the user. It is in making it.

The present invention for achieving the above technical problem relates to a watermarking system using the horizontal axis noise mean shift, a watermark inserter for inserting a watermark signal by moving the average of the noise distribution of the center image; The watermark signal is extracted from the noise column extracted from the detection target region by averaging, and the watermark signal is detected by comparing the extracted watermark signal with a normal correlation coefficient between the reference watermark signal and a threshold value. Mark detector; .

인 가우시안 분포를 따르는 참조 워터마크신호를 비밀키와 메시지를 이용하여 생성하는 참조 워터마크 신호 생성부; 평균이 이동된 워터마크 노이즈를 이용하여, 중앙 영상에서 추출한 노이즈 기둥에 더함으로써 워터마크 신호를 삽입하는 워터마크 삽입부; 및 워터마크 신호가 삽입된 노이즈를 다시 노이즈를 추출했던 원본 영상에 더해줌으로써, 워터마크가 삽입된 영상을 취득하는 삽입영상 취득부; 를 포함하는 것을 특징으로 한다. The watermark inserter may include: a noise image extractor for extracting a noise image from a center image and dividing the noise image into k horizontal noise columns; Mean 0 and variance

A reference watermark signal generator for generating a reference watermark signal according to the Gaussian distribution using a secret key and a message; A watermark inserting unit inserting a watermark signal by adding the averaged watermark noise to the noise column extracted from the center image; And an embedded image acquiring unit for adding the watermark signal-embedded noise to the original image from which the noise is extracted, to acquire the watermark-embedded image. And a control unit.

The watermark detector may further include: a signal extractor configured to extract noise pillars from a detection target image, and then extract an inserted watermark signal by calculating an average at each noise pillar; And a correlation coefficient calculator for determining whether a watermark signal is detected by calculating a normal correlation coefficient between the reference watermark signal and the watermark signal extracted by the signal extractor. Characterized in that it comprises a.

The watermark detector may further include an image restoring unit configured to adjust an average of noise posts after detecting a watermark signal in order to restore an image having a watermark embedded therein; And further comprising:

The correlation coefficient calculator may determine that the watermark signal inserted through the watermark inserter is detected when the calculated normal correlation coefficient exceeds a threshold.

The present invention is a method that can protect the contents of the DIBR method, which was not covered by the existing three-dimensional stereoscopic image watermarking technology, and according to the present invention as described above, even if only one of the three-dimensional image is converted, Since the inserted watermark can be detected, there is an effect that the source can be blocked from illegal distribution.

According to the present invention, the left and right eye images converted through the DIBR method can detect the inserted watermark signal irrespective of any depth, thereby protecting even the image which the DIBR watermarking technology could not protect. .

1 is an exemplary view showing a left eye and right eye image generation process of the DIBR system.
2 is an overall configuration of a watermarking system using the horizontal axis noise mean shift in accordance with the present invention.

Specific features and advantages of the present invention will become more apparent from the following detailed description based on the accompanying drawings. It is to be noted that the detailed description of known functions and constructions related to the present invention is omitted when it is determined that the gist of the present invention may be unnecessarily blurred.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will now be described in detail with reference to the accompanying drawings.

Referring to the watermarking system using the horizontal axis average moving according to the present invention.

Prior to this, a brief description of the Depth-Image-Based Rendering (DIBR) system will be provided with reference to FIG. 1. As shown in FIG. 1, the DIBR system uses a center image and a depth image corresponding to the center image to detect left and right eyes. Will be created.

Here, the depth image is obtained from the two-dimensional image by using a camera or a depth measuring technique specially manufactured to measure the depth. The depth image generally has a value between 0 and 255. In this case, the close distance from the user has a value close to 255 and vice versa.

The formula for converting the center image into the left eye and right eye images using the depth image is shown in Equation 1 below.

[Equation 1]

과

중앙 영상의 가로축 값 중,

에 해당하는 위치가 2차원에서 3차원으로 변환된 후의 좌안 및 우안 영상에 맺히는 위치이다.

는 카메라의 초점거리를 의미하며,

는 기준선 길이를 의미한다. 그리고

는 상대적 깊이를 의미하며 다음의 [수식 2] 와 같이 계산할 수 있다.here,

and

Among the horizontal axis values of the center image,

This position corresponds to the left eye and right eye images after the two-dimensional to three-dimensional image is converted.

Means the focal length of the camera,

Means baseline length. And

Denotes the relative depth and can be calculated as shown in Equation 2 below.

[Equation 2]

는 중앙 영상의 해당위치에 해당하는 깊이값을 의미하고,

와

는 화면의 앞과 뒤에 맺히는 깊이의 범위를 각각 의미하게 된다. here,

Means a depth value corresponding to the corresponding position of the center image.

Wow

Means a range of depths before and after the screen.

It should be noted that, in the three-dimensional transformation, the pixel values are only moved along the horizontal axis. In the present invention, this point is intended to be combined with watermarking technology.

The left and right eye images after 3D transformation using the above equation still remain a problem. After the objects in the image are moved through the conversion process, an empty space without pixel values remains. This is called a hole, and the central image does not have information to fill the hole. Thus, much research is being done to fill this hole naturally, and in general, the hole is filled by using interpolation using surrounding pixel values.

Since not only the central image but also the left and right images converted from the central image are sufficiently valuable as two-dimensional images, these three images must be protected from illegal copying. To this end, the present invention proposes a stereoscopic image watermarking technique that is robust to the two-dimensional to three-dimensional transformation of DIBR. The proposed technique is made using two characteristics of DIBR contents that are invariant to 3D transformation.

1. The noise distribution of any image follows the Laplace distribution with a mean of zero.

2. When the center image is converted to the left and right eye images, the objects in it move only on the horizontal axis.

2 is an overall configuration diagram of the watermarking system S using the horizontal axis noise averaging according to the present invention, and includes a watermark inserter 100 and a watermark detector 200 as shown.

The watermark inserter 100 inserts a watermark signal by shifting the average of the noise distribution of the center image, as shown in FIG. 2, the noise image extractor 110 and the reference watermark signal generator 120. ), A watermark embedding unit 130 and an embedded image obtaining unit 140.

In detail, the noise image extractor 110 extracts a noise image from a center image of m ㅧ n. Thereafter, the noise image is divided into k horizontal noise pillars.

을 만족해야 한다.
In this case, each noise column has a size of m ㅧ (n / k), where k is the length of the watermark signal.

Must be satisfied.

인 가우시안 분포를 따르는 참조 워터마크신호(

)를 비밀키와 메시지를 이용하여 생성한다. The reference watermark signal generator 120 has an average of 0 and a variance

Reference watermark signal along the Gaussian distribution

) With a private key and a message.

Since each noise pixel follows the Laplace distribution with an average of zero, each extracted noise column also follows the Laplace distribution with an average of zero.

The watermark inserting unit 130 inserts the watermark signal by using the watermark noise of which the average is shifted and adding it to the noise column extracted from the center image.

)의

번째 원소(

) 를

번째 노이즈 기둥에 삽입하기 위하여, 참조 워터마크신호(

)의

번째 원소를 평균으로 갖는 워터마크 노이즈를, 상기 중앙 영상에서 추출한

번째 노이즈 기둥에 더한다.More specifically, the reference watermark signal (

)of

Element

)

To insert the first noise column, the reference watermark signal (

)of

The watermark noise having an average of the first element is extracted from the center image.

To the first noise column.

At this time, the added watermark noise is generated as shown in Equation 3 below.

[Equation 3]

는

번째 워터마크 노이즈를 의미하며, 이때 x와 y의 범위는

를 만족해야 한다. here,

The

The second watermark noise, where x and y range

.

는 평균 0, 분산 c를 가진 가우시안 분포를 가진 랜덤 변수를 의미한다. 그리고, 인지 크기 성분(

)은 다음의 [수식 4] 를 통해 계산할 수 있다. Also,

Denotes a random variable with a Gaussian distribution with mean 0 and variance c. And the perceived size component (

) Can be calculated by the following Equation 4.

[Equation 4]

는 픽셀의 NVF 값을 이용한 지역 가중치 성분이며,

는 JNDD 를 이용한 지역 가중치 성분이다. NVF는 사람의 눈이 거친 지역이나 경계에서 노이즈를 잘 인식하지 못한다는 사실을 활용한 함수로써 다음의 [수식 5] 를 통해 계산할 수 있다. here,

Is the local weighting component using the pixel's NVF value,

Is a regional weighting component using JNDD. NVF is a function that takes advantage of the fact that the human eye does not recognize noise in rough areas or boundaries. It can be calculated by Equation 5 below.

[Equation 5]

는

크기를 갖는 윈도우의 지역 분산을 의미하며,

는 각각의 영상마다 조절해야하는 조율 변수이다. 그리고

는 주어진 영상의 지역 변수 중 가장 큰 지역 변수를 의미한다.

은 실험적으로 결정해야하는 변수이다.

는 평평한 지역에서는 1에 가까운 값을 갖게 되고 거칠거나 경계 지역에서는 0에 가까운 값을 갖게 된다. 지역 가중치 성분

는

값과 두 개의 가중치 성분

을 이용하여 다음의 [수식 6] 과 같이 계산할 수 있다.here,

The

Means the local distribution of a window of size,

Is the tuning parameter that needs to be adjusted for each image. And

Denotes the largest local variable among the local variables of a given image.

Is a variable that needs to be determined experimentally.

Has a value close to 1 in flat areas and close to 0 in rough or bordered areas. Geo weight component

The

Value and two weight components

It can be calculated using Equation 6 below.

[Equation 6]

의 윈도우 크기

은 3ㅧ3 을 사용했으며,

는 50을 사용했으며,

는 각각 1.5, 1 로 설정하였다. In this insertion process, regional dispersion of NVF

Window size

Used 3 ㅧ 3,

Used 50,

Was set to 1.5 and 1, respectively.

는 다음의 [수식 7] 과 같이 계산할 수 있다.The human eye is also less likely to notice small differences when an object is deep or shallow. Using this property, the local weighting component using JNDD

Can be calculated as shown in Equation 7 below.

[Equation 7]

은 각각 4, -4, 그리고 1.5의 값을 가지며,

은 중앙 영상에 대응하는 깊이 값을 정규화한 값을 의미한다. here,

Have values of 4, -4, and 1.5, respectively,

Denotes a value obtained by normalizing a depth value corresponding to the center image.

) 의 각 원소 값들과 동일한 평균을 각 노이즈 기둥들이 갖게 된다.
In this way, the watermark signal is inserted through the method of adding the moved watermark noise to the extracted noise pillars. By fine-tuning the noise column in which the watermark noise is inserted, the reference watermark signal (

Each noise column has an average equal to each element value of

Thereafter, the embedded image acquiring unit 140 acquires an image having the watermark embedded by adding the noise having the watermark signal embedded therein to the original image from which the noise has been extracted.

)와, 상기 참조 워터마크 신호(

) 사이의 정규상관계수(

)와 임계값(

)과의 비교를 통해 워터마크 신호를 검출하는 바, 상기 도 1 에 도시된 바와 같이, 신호 추출부(210), 상관계수 계산부(220) 및 영상 복원부(230)를 포함한다. The watermark detector 200 extracts the watermark signal from the noise column extracted from the detection target region through averaging and extracts the extracted watermark signal (

) And the reference watermark signal (

Normal correlation between

) And threshold (

As shown in FIG. 1, the watermark signal is detected by comparison with the reference numeral), and includes a signal extractor 210, a correlation coefficient calculator 220, and an image reconstructor 230.

)를 추출한다.
Specifically, the signal extractor 210 extracts noise pillars from the detection target image using a Wiener filter, calculates an average of each noise pillar, and inserts the watermark signal (

).

)와, 신호 추출부(210)를 통해 추출한 워터마크 신호(

) 사이의 정규상관계수(

)를 다음의 [수식 8] 을 통해 계산함으로써 워터마크 신호가 검출되는지 판단한다. The correlation coefficient calculator 220 may generate the reference watermark signal (

) And the watermark signal extracted by the signal extractor 210 (

Normal correlation between

) Is determined by the following Equation 8 to determine whether a watermark signal is detected.

[Equation 8]

)가 임계값(

)을 초과하면, 상기 워터마크 삽입기(100)를 통해 삽입한 워터마크 신호가 검출된 것으로 판단한다. That is, the correlation coefficient calculation unit 220 calculates the normal correlation coefficient (

) Is the threshold (

Is exceeded, it is determined that the watermark signal inserted through the watermark inserter 100 has been detected.

)는 평균이

인 가우시안 분포를 따르게 된다. 이때, 평균

및 표준편차

는 다음의 [수식 9] 으로 계산한다.On the other hand, considering the central limit theorem,

) Has an average

Will follow the Gaussian distribution. Where average

And standard deviation

Is calculated by the following formula (9).

[Equation 9]

)는 평균이 0인 가우시안 분포를 따르고 임의의 영상에서부터 추출된 추출 워터마크 신호(

) 역시 앞에서 언급했던 DIBR 방식의 컨텐츠의 첫 번째 조건인 평균 0을 따르므로

는 0이 된다. 그리고, 참조 워터마크 신호(

)의 차원의 수는 k이므로 참조 워터마크 신호(

)의 크기는 대략적으로 다음의 [수식 10] 과 같이 계산할 수 있다. Reference watermark signal (

) Follows a Gaussian distribution with a mean of 0 and extracts watermark signals extracted from random images (

) Also follows the average condition of 0, which is the first condition of DIBR content.

Becomes zero. And, the reference watermark signal (

), The number of dimensions is k, so the reference watermark signal (

) Can be roughly calculated as Equation 10 below.

[Equation 10]

)의 분산은 다음의 [수식 11] 과 같다. Therefore, by the above two equations,

) Is as shown in Equation 11 below.

[Equation 11]

)는

를 따르므로, 거짓 양성 오류의 확률은 다음의 [수식 12] 와 같이 계산된다.At this time, the normal correlation coefficient (

)

Therefore, the probability of false positive error is calculated as follows.

[Equation 12]

)은 거짓양성오류가 10^-6을 가질 수 있도록 정하도록 한다.
That is, the threshold value in the present invention (

) Specifies that false positive errors can have 10 ^-6 .

The image reconstructor 230 reconstructs the watermark-embedded image into a better image by adjusting the average of the noise posts to 0 after detecting the watermark signal.

The DIBR method, which is one of the three-dimensional image representation methods, has recently attracted attention in various fields because of its many advantages. Two of the biggest advantages are that it is possible to selectively view the 2D-3D image, and in the case of the 3D image, the user can adjust the desired depth. However, these two big advantages are two big constraints in terms of copyright protection.

In other words, the watermarking system for DIBR-type images should be robust to 2D-3D transformation and should be detectable in various 3D depth environments.

The present invention proposes a stereoscopic image watermarking technique that solves two constraints that the watermarking system for DIBR image has to solve. The watermarking system S using the horizontal axis noise averaging as described above can insert a watermark signal using a method of changing the average of horizontal axis noise, which is an invariant feature of two-dimensional to three-dimensional conversion, in various depth environments. It was able to be robust to 2D-3D transformation.

While the present invention has been particularly shown and described with reference to preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. It will be appreciated by those skilled in the art that numerous changes and modifications may be made without departing from the invention. Accordingly, all such appropriate modifications and changes, and equivalents thereof, should be regarded as within the scope of the present invention.

S: Watermarking system using horizontal noise mean shift
100: watermark inserter 110: noise image extraction unit
120: reference watermark signal generation unit 130: watermark insertion unit
140: embedded image acquisition unit 200: watermark detector
210: signal extraction unit 220: correlation coefficient calculation unit
230: image restoration unit

Claims (5)

A watermark inserter 100 for inserting a watermark signal by shifting an average of noise distribution of a center image;
The watermark signal is extracted from the noise column extracted from the detection target region by averaging, and the extracted watermark signal (

) And the reference watermark signal (

Normal correlation between

) And threshold (

A watermark detector 200 for detecting the watermark signal through comparison with the < RTI ID = 0.0 > Watermarking system using a horizontal noise mean shift comprising a.
The method of claim 1,
The watermark inserter 100,
A noise image extractor 110 for extracting a noise image from the center image and dividing the noise image into k horizontal noise columns;
Mean 0 and variance

A reference watermark signal generation unit 120 for generating a reference watermark signal according to the Gaussian distribution using a secret key and a message;
A watermark inserting unit 130 for inserting a watermark signal by adding the averaged watermark noise to the noise column extracted from the center image; And
An embedded image acquiring unit 140 for acquiring the watermark-embedded image by adding the noise having the watermark signal added thereto to the original image from which the noise has been extracted; Watermarking system using the horizontal axis noise moving average, characterized in that it comprises a.
The method of claim 1,
The watermark detector 200,
After extracting the noise pillars from the detection target image, the average of the noise pillars is calculated, and the embedded watermark signal (

A signal extracting unit 210 for extracting; And
The reference watermark signal (

) And the watermark signal extracted by the signal extractor 210

Normal correlation between

A correlation coefficient calculation unit 220 for determining whether a watermark signal is detected by calculating a); Watermarking system using the horizontal axis noise moving average, characterized in that it comprises a.
The method of claim 3, wherein
The watermark detector 200,
An image restoration unit 230 for adjusting an average of noise posts after detecting the watermark signal in order to restore an image having a watermark embedded therein; Watermarking system using the horizontal axis noise mean shift, characterized in that it further comprises.
The method of claim 3, wherein
The correlation coefficient calculation unit 220,
Calculated normal correlation

) Is the threshold (

), It is determined that the watermark signal inserted through the watermark inserter (100) is detected.

Images