KR100383836B1 - Watermarking insert and analysis method of robust motion based on multi-resolution analysis - Google Patents

Abstract

The present invention relates to a watermarking insertion and analysis method based on multiresolution analysis of motion data. In particular, the present invention relates to a watermarking insertion and analysis method that enables analysis of unauthorized copying of motion data by substituting a value of a largest value among the detailed coefficients of a motion signal expressed in multi-resolution.

In the conventional motion data display, when watermarks are displayed in a three-dimensional space by using rotation matrices or unit quaternions, a group is formed in a lith, so that watermarks are not easily extracted. Generating a watermark to be inserted into various operation signals constituting the data, inserting the watermark by substituting the generated watermark with coefficients of an operation displacement map among the operation signals, Aligning the motion signal of the dragon represented by the multiresolution constituting the data with the original motion signal; extracting a watermark from the aligned motion signal of the dragon represented by the multi resolution; Comparing and analyzing the watermark of the motion signal and the watermark of the extracted dragon motion signal with the multi-resolution representation. It is provided to enable the operation of the copyright protection data.

Description

Watermarking insert and analysis method of robust motion based on multi-resolution analysis

The present invention relates to a watermarking insertion and analysis method based on multiresolution analysis of motion data. In particular, the present invention relates to a watermarking insertion and analysis method that enables analysis of unauthorized copying of motion data by substituting a value of a largest value among the detailed coefficients of a motion signal expressed in multi-resolution.

In recent years, with the spread of Internet services and the development of large-capacity media, the use of digital multimedia media represented by a combination of text, still images, moving pictures, and voices has been increasing rapidly. However, since the digital multimedia medium can be easily copied by an unauthorized user, copyright protection of the digital multimedia medium is required. As an alternative, a digital watermark is presented. lost.

Digital watermark inserts watermark which is invisible additional information in data area to be copyrighted, extracts watermark inserted from digital multimedia data suspected of unauthorized copying and judges whether it is unauthorized copying. Suggest ways to protect

The digital watermark is classified into two methods. First, a forensic watermarking technique for extracting an inserted watermark by contrasting suspect data and original data, and second, It is divided into a blind detection technique that extracts a watermark from suspect data using only a secret key.

The first watermarking technique is advantageous in that it is not easy to remove the watermark, and the second watermarking technique does not require original data, and thus is suitable for network distribution or product monitoring of a broadcasting system.

In addition, the watermark embedded in the digital multimedia data presents a copyright protection function for unauthorized copying as well as unauthorized copying because of the property that the data is not destroyed even if the data is not damaged so much that it is difficult to reuse. Give it.

Herein, a description will be given in relation to rhetorical watermarking, which is a related art of the present invention.

As described above, due to the functional characteristics of the watermark described above, a number of watermarking techniques have been proposed for image, audio, video, and 3D geometric models, which are various media that are currently used.

Conventional watermarking techniques insert additional information into the least significant bit (LSB) of data, and also use multiresolution analysis, Fourier transform, Or spread-spectrum watermarking using discrete cosine transform or the like. [I. J. Cox, J. Kilian, F. T. Leighton, and T. Shamoon. Secure spread spectrum watermarking for images, audio, and video. In IEEE Proceedings of International Conference on Image Processing, volume 3, pages 243-246, 1996 .; I. J. Cox, J. Kilian, F. T. Leighton, and T. Shamoon. Secure spread spectrum watermarking for multimedia. IEEE Transactions on Image Processing, 6 (12): 1673-1687, December 1997 .; E. Praun, H. Hoppe, and A. Finkelstein. Robust mesh watermarking. In Computer Graphics (Proceedings of SIGGRAPH 99), pages 49-56, August 1999 .; J. J. K. O. Ruanaidh, F. M. Boland, and O. Sinnen. Wa-termarking digital images for copyright protection. In Proceedings of the Electronic Imaging and Visual Arts Conference, pages 243-246, February 1996.].

In the above watermarking techniques, the technique of inserting additional information into the least significant bit (LSB) of the data can insert a watermark in a large area of the original data with little damage to the original data, In order to ensure that the watermark remains in the data as long as there is no damage, the frequency domain is analyzed to obtain important parts of the data, and the watermark is preferentially inserted on the obtained data area.

In addition, the spread spectrum watermarking technique has been extended to be applied to 3D meshes by Praun et al., And to improve the robustness of watermarks for audio and visual data, human auditory system or human visual system. ) Was also used [L. Boney, A. H. Tewfik, and K. N. Hamdy. Digital watermarks for audio signals. In Proceedings of the Third IEEE International Conference on Multimedia Computing Systems, 1996, pages 473-480, 1996 .; C. I. Podilchuk and W. Zeng. Image-adaptive watermarking using visual models. IEEE Journal on Selected Areas in Communications, 16 (4): 525-539, May 1998.].

Moreover, due to the trend of increasing the production of realistic human animation, which is realized by a large number of motion data, due to the spread of motion capture technology in the field of computer graphics, the watermarking technique described above is used. If watermarking techniques for copyright protection are also applied to motion data that implements human animation, motion data of human animation, which consists of bundles of motion signals, is first applied to the position or orientation of each joint of the human animation. Unlike the position of the three-dimensional object, the direction of each joint of the animation is caused by a singularity problem, and thus cannot be represented as a vector in three-dimensional space because it is represented by a continuous array of corresponding values.

In order to solve this problem, when displaying vectors in three-dimensional space using a rotation matrix or unit quaternion, which prevents singularity problems, a Lie group is formed. It is difficult to obtain frequency information from motion data.

However, in the case of displaying watermarking on a plurality of motion data used for human animation using the conventional watermarking technique, when a vector of a three-dimensional space is displayed by using a rotation matrix or unit quaternary, a group is formed in Li, The acquisition of the frequency information on the operation data is not easy, so there is a problem that it is difficult to insert and extract the watermark.

The present invention is to solve the problems of the prior art, an object of the present invention by using a multi-resolution representation of the various motion signals constituting the motion data of the human body animation, the motion displacement map of the part constituting the motion signal The present invention provides a watermarking technique by substituting a coefficient of water into a watermark to insert a watermark, thereby facilitating extraction of a watermarked display and enabling an analysis on whether or not to copy unauthorized watermarked motion data.

In the technical concept for achieving the present invention, in the method of embedding and analyzing the watermark in the digital data,

Generating a watermark to be inserted into various motion signals constituting motion data of a human body animation, replacing the generated watermark with coefficients of a motion displacement map among the motion signals, and inserting a watermark; Aligning the dragon's motion signal represented by the multi-resolution constituting the dragon's data to be analyzed with the original motion signal, and extracting a watermark from the dragon's motion signal represented by the aligned multi-resolution And a watermark insertion and analysis method based on the multiresolution analysis of motion data comprising the step of comparing the watermark of the original motion signal with the watermark of the extracted dragon motion signal. present.

Each element of the generated watermark is replaced with the largest m coefficients among the coefficients of the motion displacement map among the parts constituting the motion signal.

In addition, in order to enable extraction of watermarks embedded in various motion signals constituting motion data of the human body animation, a multi-resolution representation based on a reduction operation and a diffusion operation is applied to the motion signal. Protected.

1 is a flow chart illustrating the steps of generating, inserting, extracting, and comparing similarities of watermarking using multiresolution representations of various motion signals constituting motion data of a human body animation.

In order to insert a watermark on the motion data, first, a multi-resolution analysis of the motion signal is required. For this, reduction and expansion operations described in the following literature are used. [Jehee Lee. A Hierarchical Approach to Motion Analysis and Synthesis for Articulated Figures. PhD thesis, DCS955295, Department of Computer Science, KAIST, 2000.]

Further, the motion data is composed of position and direction data.

Hereinafter, the analysis of the multi-resolution in detail by mathematical definition and theory are as follows.

The multi-resolution representation of the motion signal is composed of a base signal and a displacement map that represent the overall shape of the signal.

The displacement mapping process using the motion displacement map is a process that is performed to deform the overall shape while maintaining the detailed characteristics of the motion. In other words, during the multi-resolution analysis, the layer of the displacement map is applied to the base signal. It is used to restore the original behavior by adding details in order.

First, in order to obtain the motion displacement map, two operations, reduction and expansion, are used.

Here, the expansion operation is performed by a smoothing process that is smoothed by subdivision using a smoothing filter after an up-sampling process for sampling the motion signal more densely. In addition, the reduction operation is the inverse of the extension operation. That is, after the smoothing process, it is a down-sampling process of sampling more coarsely.

More specifically, multiresolution analysis of motion signals based on reduction and diffusion operations is represented by motion displacement functions, motion filtering, and multiresolution representations.

1. Motion Displacement Function

The posture of the articulated body can be specified by the position and direction of the root joint and the composition of each joint, and for the sake of consistency, assuming that the composition of each joint is given by three-dimensional rigid deformation, Each part can be described by the vector of R ³ and the unit number of S ³ .

That is, the motion data of the articulated body is composed of a plurality of motion signals, and the individual motion signals are represented by a series of frames corresponding to the position and the direction of each joint. The frame is represented by equation (1).

Here, the equation (1) frame _{((p i, q i)} ) of the rigid body transformation may be represented as corresponding to a point in R ³ to a different point on the R ^3. The rigid body transformation is represented by equation (1).

In Equation 2, u representing the position of an arbitrary point is expressed as Equation 3.

In addition, Equation 3 is again regarded as a pure unit of imaginary quaternary number. Here, the pure unit imaginary quaternary number is expressed as in Equation (4).

Here, for the two operation signals m and m '(Equation 5 below), the operation displacement d (Equation 6 below) is defined to satisfy Equation 7, or Equation 8 below.

Here, the displacement function d may be derived by using Equations 4 to 8 and a composite transformation (Equation 9).

In Equation 9, exp (v) represents a three-dimensional rotation by Equation 11 based on Equation 10.

2. Motion Filtering

Motion filtering is described as follows.

Given a vector signal (Equation 12 below) and a spatial mask (Equation 13 below), the basic approach of spatial filtering can be represented by multiplying and then adding mask coefficients and sample values at specific locations of the signal.

In equations (12) and (13), the i-th filter response is represented by equation (14).

For a vector signal, the kind of filtering as in Equation 14 is frequently used.

However, if the above mask is applied to a unit quaternary signal, the response to the mask may not generally be a quaternary with unit length.

This is because the unit quaternary spaces are not closed for sum or constant product operations.

However, given a filter mask where the sum of the coefficients is 1, the directional filter is defined as in Equation 16 below.

In Equation 17, b _m may be represented by Equation 16 below.

Since the unit quaternary space is closed with respect to the product of the quaternary numbers, the filter response using the filter mask is certainly a pure unit quaternary, and frequency information can be obtained from the motion data.

In addition, the spatial filter for the directional data (hereinafter referred to as H) also inherits the important properties of the equation for the spatial filter for the vector. That is, since H has an invariant property for local and global coordinate transformations, H gives the same result regardless of the coordinate system in which the direction data is expressed.

In addition, H has a time-invariant property that exhibits a filter response that does not depend on a specific position of the signal, and H is also symmetric if the mask coefficient is symmetric.

3. Multi-resolution representation

The multiresolution representation of the operating signal M is defined by a series of continuous detailed signals (Equation 18 below) and a series of displacement functions (Equation 19 below).

In order to obtain the multi-resolution representation, first, a reduction operation is applied to the original operation M = M ^(N) to obtain a more gentle and smooth signal M ^(N-1) .

Here ^, the expansion of M ^(N-1) attaches insufficient information through interpolatory subdivision. In the present invention, in the interpolation division, by using four-point interpolation, the even frame in the step "n + 1" (Equation 20 below) is a step "n" frame (Equation 21 below), and the odd frame (following) (22) is configured to insert a new one between the previous frames.

Here ^, the difference between M ^(N) and M ^(N-1 ) is represented by a displacement function (Equation 23 below).

In Equation 23, S is an extension operator.

In this case, if the process of Equations 20 to 23 is repeated until only a few frames remain on the operation signal, a multiresolution representation including M ⁽⁰⁾ and a series of displacement functions, which are the estimated base signals, can be obtained. .

In addition, the original operation M ^(N) can be obtained by recursively adding the displacement function in each step to the extended signal in the same step using Equation (24 ⁾ below.

The above is a process of multiresolution analysis of a required motion signal in order to insert a watermark on the motion data, and a robust watermarking technique that uses the analyzed multiresolution representation to insert widely on the area of the overall motion signal. Spread spectrum watermarking is applied.

In order to insert a watermark by applying the spread spectrum watermarking technique, a method of inserting a watermark by finding parts to be recognized as important throughout the operation signal is required, and the importance of the operation signal into which the watermark is inserted is required. The part looks like this:

The multiresolution representation of the motion signal has a hierarchy of base signals and motion displacement maps, each of which is represented as a different detail of motion.

Here, the watermark is inserted by perturbing the values of the coefficients of the motion displacement map. Changing the coefficients at lower resolutions deforms a large portion of the operating signal, and changing the coefficients at higher resolutions results in a narrower range of the operating signals.

In this way, the deformation shape of the operation signal according to the distortion of the coefficient is a symmetric vibration type, and the width of the vibration pattern depends on the resolution of the displacement function including the distorted coefficient. In addition, the amplitude has a peak value at the center position and rapidly decreases at both edges.

In the above, since only the author of the motion signal knows the degree of deformation of the motion signal due to the insertion of the watermark, the attacker will make as much modification as possible to remove the watermark, thereby reducing the quality of the motion signal. Is generated.

Here, the watermark inserted into the motion data has a tradeoff between the strength and the imperceptibility. In other words, the larger the distortion of the coefficient by the watermark embedding, the stronger the watermark is, but it is more prominent. The smaller the distortion, the less difficult it is to visually identify the inserted watermark.

For this reason, in order to increase the robustness of the watermark, a method of embedding the watermark in an important part of the data is required if possible, and the most of the coefficients of the motion displacement function for inserting the watermark in the important part of the motion signal. Transform large ones first. At this time, it is preferable to apply a deformation within a range such that the inserted watermark is not noticeable.

Hereinafter, with reference to the accompanying drawings, the configuration and operation of the embodiment of the present invention will be described in detail.

FIG. 1 is a flow chart illustrating the steps of generating, inserting, extracting, and comparing similarity of watermarks using multiresolution representations of various motion signals constituting motion data of a human body animation.

As shown in FIG. 1, generating a watermark inserted into various motion signals constituting motion data of a human body animation (S100);

Inserting a watermark by substituting the generated watermark with a coefficient of an operation displacement map in the operation signal (S110);

Aligning the motion signal of the dragon represented by the multi-resolution constituting the dragon data to be analyzed with the original motion signal (S120);

Extracting a watermark from the operation signal of the dragon represented by the arranged multi-resolution (S130);

And comparing (S140) the watermark of the original operation signal with the watermark of the extracted operation signal.

In step S100, a watermark (Equation 25) is first generated before embedding the watermark in the operation signal.

In Equation 25, individual watermarks w _i are independently extracted from a normal distribution having an average of 0 and a standard deviation of 1. Here, in order for this watermarking technique to be non-invertible, a random number is obtained from the value obtained by applying the MD5 cryptographic hash function after connecting the original operation signal and the author's secret key. To generate individual watermark w _i values. For this purpose the MD5 cryptographic hash function described in the literature is used.

[B. Schneier. Applied Cryptography: protocols, algorithms, and source code in C. John Wiley Sons, Inc., 2nd edition, 1996.]

Each element of the generated watermark is replaced with the largest m coefficients among the coefficients of the motion displacement map, which are the parts constituting the motion signal.

In step S110, the watermark w is inserted into the coefficient of the motion displacement map among the original motion signals to obtain the watermarked motion signal M '.

Here, when inserting a watermark (w) into the coefficient of the motion displacement map, select the largest coefficient, m m of the motion displacement map, and replace D ^(l) to replace each of them using the watermark (w ⁾ . A displacement function at the lth level of the multiresolution representation of the motion displacement map, and (u _i , v _i ) is the i-th largest coefficient among the coefficients of the displacement function. _{u 'i, v' i)} ( only, i = 1, ..., m ) has been is obtained using the equation (26).

In Equation 26, α is a scaling parameter.

The watermarked operation signal M 'may be obtained by sequentially adding the displaced displacement function (Equation 26) to the base signal through the above process.

In the step S120, the operation signal for the dragon to be analyzed may be cropped or partially scaled with respect to the time axis. Therefore, before extracting the watermark, the dragon's motion signal is aligned and aligned with the original motion signal.

Here, to align the dragon's motion signal, the dragon's motion signal is aligned to match the original motion signal by resampling the dragon's motion signal using dynamic time warping. The dynamic time variant described in the following documents was used for this.

[A. Bruderlin and L. Williams. Motion signal processing. In Computer Graphics (Proceedings of SIGGRAPH 95), pages 97 104, August 1995 .; X. D. Huang, Y. Ariki, and M. A. Jack. Hidden Markov Models for Speech Recognition. Edinburgh University Press, 1990 .; T. W. Sederberg and E. Greenwood. A physically based approach to 2-D shape blending. In Proceedings of the 19th annual conference on Computer graphics, pages 25-34, July 27-31 1992.]

In the step S130, the watermark is extracted from the dragon's operation signal in the reverse order of the process of inserting the watermark.

In other words, the multi-resolution representation is generated from the dragon's operation signal M ^* and the watermark is inserted into the dragon's operation signal by obtaining a watermark w that minimizes the coefficient of substitution (Equation 26). The watermark w _i ^* is extracted.

In step S140, the watermark of the original motion signal and the watermark of the dragon's motion signal represented by the extracted multi-resolution expression are to be noted when comparing and analyzing the false-positive probability, that is, the dragon. It is important to minimize the likelihood that a watermark will be mistaken, even though the data does not actually contain the watermark you want to obtain. To this end, it is configured to statistically compare the watermark inserted by the author and the watermark obtained from the dragon data.

In the present invention, a linear correlation between the watermark w inserted by the author and the watermark w ^* extracted from the dragon data is obtained. At this time, it is configured to obtain the false positive probability by using the Student's t-test from the linear correlation value obtained by removing the outlier elements that are far from the mean among the watermark elements for stable comparison. .

EXAMPLE

In order to test the robustness of the inserted watermarking technique according to the present invention, four motion data such as Fly Spin Kick, Back Flip, Broad Jump, and Blown Back were measured.

In the present invention, since the watermarking technique is affected by the random number used in generating the watermark, after five experiments with each motion data, the median of the false positive probability is shown.

As the α parameter, which is a scaling parameter used for watermark insertion, 10 ⁻⁴ is used for the motion signal for the pelvis joint and 10 ⁻² for the motion signal for the other joints. The α values are obtained by experiments so that the inserted watermark is robust without being noticeable.

In addition, the number of inserted watermarks is 20, and values that are 2.5 times or more of the standard deviation from the average of the extracted watermark values when the similarity of watermarks are compared are regarded as the outer layer. Table 1 shows data showing the robustness measurement results for various attacks of the present watermarking technique.

Attack type Fly spin kick Back flip Broad jump Blown back No attack 10 ^-99 10 ^-99 10 ^-99 10 ^-99 0.2% noise 10 ^-18 10 ^-17 10 ^-52 10 ^-26 0.7% noise 10 ^-9 10 ^-6 10 ^-21 10 ^-16 cut 10 ^-9 10 ^-6 10 ^-21 10 ^-16 Euphemism 1 10 ^-24 10 ^-9 10 ^-15 10 ^-11 Euphemism 2 10 ^-7 10 ^-5 10 ^-5 10 ^-12 Simplify 1 10 ^-35 10 ^-35 10 ^-35 10 ^-35 Simplified 2 10 ^-27 10 ^-38 10 ^-27 10 ^-17 Simplified 3 10 ^-15 10 ^-28 10 ^-32 10 ^-15 Time warping 1 10 ^-8 10 ^-27 10 ^-13 10 ^-24 Time warping 2 10 ^-10 10 ^-3 10 ^-30 10 ^-36 Exaggeration 10 ^-4 10 ^-4 10 ^-12 10 ^-4 Motion mitigation 10 ^-7 10 ^-5 10 ^-14 10 ^-15 Secondary watermarking 10 ^-73 10 ^-80 10 ^-74 10 ^-76

As shown in Table 1, the false positive discrimination probability value for the motion signal corresponding to the head joint is shown in the motion data. The noise attack during the attack used in this embodiment is an attack that mixes noise having an amplitude within the range of 0.2% to 0.7% with respect to the motion signal.

In addition, a truncation attack is a case where a part of operation signal is cut | disconnected. Equations 1 and 2 are cases in which the average filter and the softening filter proposed by Lee are applied to the operation signal, respectively.

Simplifications 1, 2, and 3 are attacks that remove detailed information between level 0, level 0 and 1, and level 0 through level 2 through multiresolution analysis, respectively. The time distortions 1 and 2 are cases where the operating signal is scaled uniformly or nonuniformly with respect to the time axis.

Motion exaggeration and motion mitigation also transform the values of the coefficients of detail at low resolution in the multiresolution representation of the motion signal.

In this case, the experiment for the second watermarking is an experiment for checking whether the first inserted watermark remains even when another person's watermark is inserted in the watermarked data. The proposed watermarking technique showed very low false positive discrimination probability, and the experimental results show that it is robust.

As described above, the present invention enables the analysis of the unauthorized copying of the watermarked motion data by applying a multi-resolution representation to various motion signals constituting the motion data of the human body animation, thereby ensuring the copyright of the human motion data. It has the effect of providing protection.

Claims (2)

In the method of embedding and analyzing watermark in digital data, Generating a watermark to be inserted into various motion signals constituting motion data of a human body animation; Inserting a watermark by substituting the generated watermark with a coefficient of an operation displacement map of the motion signal; Arranging the dragon's motion signal represented by the multi-resolution constituting the dragon's data to be analyzed with the original motion signal; Extracting a watermark from the operation signal of the dragon represented by the arranged multi-resolution; A watermark insertion and analysis method based on multiresolution analysis of motion data, comprising comparing and analyzing a watermark of an original motion signal and a watermark of the extracted dragon motion signal. The method according to claim 1, In the step of inserting a watermark by substituting the generated watermark with coefficients of a motion displacement map among the motion signals, the coefficients of the motion displacement map to be substituted are the largest coefficients. Based watermarking insertion and analysis method.