JP3536891B2 - Information multiplexing method and information extraction method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION The present invention relates to a method for multiplexing information so as not to be perceived by human perception when different information is multiplexed to digital moving pictures according to the present invention. about the method for extracting the multiplexed information to the
It is those that. Today, such information multiplexing and extraction technologies are widely used in systems for protecting copyrights and preventing unauthorized duplication of digital information contents by secretly multiplexing copyright information and user IDs into information contents. Have been.

[0002]

2. Description of the Related Art Conventionally, when different information is multiplexed on a digital moving image so as not to be perceived by human perception,
A method of multiplexing information into units called subdivided blocks has been common. However, this method has a problem that much noise such as block distortion is observed in an information multiplexed image, and the multiplexed information is easily erased due to compression processing such as reducing a bit rate of a moving image. There was a point.

[0003]

SUMMARY OF THE INVENTION An object of the present invention is to significantly improve the quality of an image in comparison with editing of image quality and compression processing, which is a problem of a conventional information multiplexing system which is not perceived by humans. In addition, the multiplexed information can withstand extreme compression processing, and information can be extracted.

[0004]

In order to achieve the above object, according to the present invention, a digital moving image is framed for a unit time T.
And decompose it into any group of M × N × T (M × N is the maximum
Size) Performs orthogonal transformation of size and performs M × N × T orthogonal transformation.
Generates a permutation coefficient matrix and sequentially generates random numbers from the information multiplexing key
And the random number is calculated from the M × N × T orthogonal transformation coefficient matrix.
Sequentially select any orthogonal transform coefficient using the selected
The orthogonal transform coefficient value based on the information to be multiplexed
Repeat for each unit time frame group
Multiplex different information to digital video in closed format
In an information multiplexing method, M × N ×
Inverse orthogonal transformation of a T-size orthogonal transformation coefficient matrix is performed to obtain M × N ×
Obtain a pixel size matrix of size T, the pixel value indicated by the pixel value matrix
Multiplexing is performed when the
The amplitude of the AC component value other than the orthogonal transformation coefficient
Multiplexing by changing the value of the DC component.
While maintaining the values of the orthogonal transform coefficients obtained by inverse orthogonal transform.
The pixel value indicated by the pixel value matrix falls within a predetermined domain.
Information multiplexing that avoids over-range of pixel values
An orthogonal transform coefficient matrix is obtained.

[0005] Also, orthogonal transforms of a plurality of sizes are performed, and
Information multiplexing for each orthogonal transform coefficient matrix separately
By doing so, multiple information can be multiplexed.
You.

On the other hand, on the information extraction side, digital moving images
Information multiplexed digital video with different information multiplexed
Is decomposed into frames of unit time T, and
Performs orthogonal transformation of the same M × N × T size and information multiplexing.
A random number is sequentially generated using the same information extraction key as the
Are used to sequentially select orthogonal transform coefficients, and the selected orthogonal
Extract multiplexed information based on the value of the transform coefficient,
Multiplexing information from the frame group of each unit time using majority vote
Determine information. During this information extraction process, multiple
Perform orthogonal transform of size and obtain each orthogonal transform coefficient matrix
Information multiplexing is performed separately for
Multiple pieces of information are multiplexed on the digitalized moving image
In this case, multiple orthogonal transforms of the same size as
By extracting multiplexed information separately,
Extract a plurality of pieces of multiplexed information.

In the process of extracting information, the generated
Only the orthogonal transform coefficients selected from the random numbers
Calculate the value of the crossover coefficient to extract the multiplexed information
Is also possible.

[0008]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a processing and data flow diagram of a moving image processing apparatus on the information multiplexing side according to the present invention. The moving image processing apparatus 1 receives the coded original moving image 2, the information multiplexing key 3, and the multiplexing information 4 as inputs,
The information-multiplexed encoded moving image 5 is output.

The moving image processing apparatus 1 receives the encoded original moving image 2, interprets the format of the image by a decoding processing unit 6, and performs a decoding process. The decryption processing unit 6
First, the number of horizontal and vertical pixels and the number of unit images in the time axis direction 7 are extracted from the received moving image, and are sent to the information multiplexing unit 8. Here, the number of unit images in the time axis direction is, for example, MPEG (Moving Picture Image C).
In the digital moving picture coding method of the oroding experts (groove), it is the number of frames in a GOP (groove of picture). Thereafter, the decoding processing unit 6 decomposes the moving image data into attribute information 9 of the moving image and frame images (for example, every 15 frame images) 10 of the number of unit images in the time axis direction, and the attribute information 9 is encoded. The frame images 10 of the number of unit images in the time axis direction are sent to the processing unit 11 and sequentially sent to the information multiplexing unit 8. The information multiplexing unit 8 receives the information multiplexing key 3, the multiplexing information 4, the number of horizontal and vertical pixels of the image, the number of unit images 7 in the time axis direction, and the frame image 10 of the number of unit images in the time axis direction as inputs. The multiplexing process is performed to output the information-multiplexed frame images 12 of the number of unit images in the time axis direction. Encoding processing unit 11
Performs encoding while synchronizing the attribute information 9 of the moving image with the frame images 12 of the information multiplexed unit images in the time axis direction, and outputs the encoded information multiplexed moving image 5.

Next, the information multiplexing unit 8 will be specifically described. FIG. 2 is a configuration diagram of the information multiplexing unit 8 using the M × N × T orthogonal transform. Here, M is the number of horizontal pixels of the image, N is the number of vertical pixels of the image, and T is the number of unit images in the time axis direction, but these are only examples.

In FIG. 2, in the information multiplexing unit 8, first, the M × N × T orthogonal transform processing unit 14 inputs the number of horizontal and vertical pixels and the number of unit images in the time axis direction (M × N × T) as inputs. 7 and the number of frame images 10 in the number of unit images in the time axis direction are converted into an M × N × T coefficient matrix 15. This coefficient matrix 15 is stored in a matrix buffer 16.

On the other hand, the multiplex information 4 is stored in the buffer 18. Here, the input multiplexing information 4 is converted into a bit string b.
_0, b _1, assume ···, b _n-1 (bit length n) and.
The head control unit 18 fixes the head position in the first bit b ₀ whenever there is an input of a new multiplexed information 4.

The random number generator 19 generates a random number (ri _{, k} ) 20 from the first half of the input information multiplexing key 3 and multiplexes the multiplexed information 4
Information multiplexing processing unit 2
Send to 1. Here, it is assumed that the random number generator 19 generates a sufficiently large number so as not to collide. The latter half of the input information multiplexing key 3 is sent to the information multiplexing processing unit 21 and is converted into the information multiplexing strength embedding_range.

In the following, of the multiplexing information 4 stored in the buffer 17, the k-th multiplexing information bit b _k {0,1} whose header has been read out is stored in the matrix buffer 1
The case of multiplexing into an M × N × T coefficient matrix [c _{(h, v, t)} ] _i stored in No. 6 will be described.

The information multiplexing processing unit 21 includes a random number generator 19
Using the random number r _{i, k} sent from
One of the M × N × T coefficient matrices [c _{(h, v, t)} ] _i stored in

[0016]

[Outside 1]

Is selected by a one-to-one mapping, and its value is

[0018]

(Equation 1)

[0019]

(Equation 2)

[0020]

[Equation 3]

By multiplexing, the bit k in the multiplex information 4 is multiplexed. The coordinates of the multiplexed coefficients

[0022]

[Outside 2]

Are stored in the coordinate buffer 22.

The head controller 18 shifts the head position to the right by one to the head position b ₀ when the head position is b _n , and shifts it from b _k to b _{k + 1} otherwise. In synchronization with this, the random number generator 19 generates the next random number.

In the information multiplexing processing section 21, the buffer 17
, And the random number 20 are sequentially input to the matrix buffer 1
6, the above multiplexing process is performed n times for the bit length of the multiplexing information 4 for the M × N × T coefficient matrix, and this is indicated by the secret information in the information multiplexing processing unit 21. The multiplexed information 4 is M × N by repeating l times
Multiplexing is performed a plurality of times (n × l) in a closed format on a × T frame image.

In the embodiment shown in FIG. 2, the information multiplexing processing section 21
Is repeated l × n times, the coefficient matrix [c _{(h, v, t)} ] _i stored in the matrix buffer 16 is sent to the range over avoidance processing unit 23. The range over avoiding processing unit 23 includes an M × N × T inverse orthogonal transform processing unit 24 and M
This processing unit cooperates with the × N × T orthogonal transformation processing unit 14 to avoid over-range of pixel values.

Here, a pixel value matrix obtained by inversely transforming the coefficient matrix [c _{(h, v, t)} ] _i inputted to the range over avoiding processing unit 23 by using the M × N × T inverse orthogonal transformation processing unit 24. [P
_{(h, v, t)} ] _i . Further, the range over avoiding processing unit 23
The coefficient matrix [c _{(h, v, t)} ] _i input to
The (0,0,0) component value (for example, as an orthogonal transform, when the (0,0,0) component (DC component) is subjected to an M × N × T orthogonal transform of all the pixel values at the lowest value. DC
When T is selected, its value is −Lm × ｍ (M × N × T)
(Lm is an intermediate value of the pixel values) and the values of all the components of the 1 × n coordinates in the coordinate buffer 22 are set to 0, and the other is (0, 0, 0) A pixel value matrix obtained by inversely transforming the component values and the values other than the l × n coordinate components in the coordinate buffer 22 to 0 using the M × N × T inverse orthogonal transform processing unit 24 As [p1 _{(h, v, t)} ] _i and [p2 _{(h, v, t)} ] _i , respectively.

The range over avoiding processing unit 23 includes a set A _i
= ｛(X, y, z) | p _{(x, y, z) i} <L _min orp
_{(x, y, z) i} > L _max } (L _min is the minimum pixel value,
L _max is the maximum value of the pixel value), and only when A _i is not the empty set, using [p 1 _{(x, y, z)} ] _i and [p 2 _{(x, y, z)} ] _i ,

[0029]

(Equation 4)

[0030]

(Equation 5)

[0031]

(Equation 6)

[0032]

(Equation 7)

The pixel value matrix [p ' _{(h, v, t)} ] _i is obtained by performing the above calculation, and it is orthogonally transformed using the M.times.N.times.T orthogonal transformation processing unit 14 to obtain the pixel value matrix. A coefficient matrix [c ' _{(h, v, t)} ] _{i for} which the range over of the value has been taken is obtained. However, in the pixel value matrix [p2 _{(h, v, t)} ] _i , L
_{When a} pixel value smaller than _{min and} a pixel value larger than _Lmax are mixed, the algorithm for avoiding the range over cannot be applied.

In the range over avoiding processing unit 23, the coefficient matrix [c ′ _{(h, v, t)} ] for which the above range over countermeasures have been taken is again subjected to inverse orthogonal transform using the M × N × T inverse orthogonal transform processing unit 24. Then, it outputs the frame images 25 of the information multiplexed unit images in the time axis direction.

The above multiplexing process is repeated for each moving image in the time axis direction, n times, which is the information bit length to be multiplexed, once for each piece of secret information in the information multiplexing processing unit 21 to obtain one moving image. The multiplexing process ends.

FIG. 3 is a processing and data flow diagram of the moving image processing apparatus on the information extraction side according to the present invention. The moving image processing apparatus 26 receives a moving image 27 in which information has been multiplexed and encoded and an information extraction key 28 as input, and outputs information 29 multiplexed in the moving image.

The moving picture processing device 26 receives the moving picture 27 in which information has already been multiplexed and encoded, interprets the picture format by the decoding processing section 30, and starts decoding processing. The decoding processing unit 30 first extracts the number of horizontal and vertical pixels and the number of unit images 31 in the time axis direction from the received moving image, and sends them to the information extraction unit 32. After that, the decoding processing unit 30 decomposes the moving image data into frame images 33 of the number of unit images in the time axis direction that have been subjected to the information multiplexing, and sends the frame images 33 to the information extracting unit 32. Information extraction unit 3
2 receives as input the information extraction key 28, the number of horizontal and vertical pixels of the image and the number of unit images 31 in the time axis direction,
Information extraction processing is performed, and information 34 multiplexed on each time-axis direction unit image is output. Multiplexing information determination processing unit 35
Determines multiplexed information by majority decision from information multiplexed in each time-axis direction unit image given as a sequential input, and outputs the result as information 29 multiplexed in a moving image.

Next, the information extracting device 32 will be specifically described. FIG. 4 is a configuration diagram of the information extraction unit 32 using the M × N × T orthogonal transform. Here, M, N, and T are the same values as the orthogonal transform used when performing the information multiplexing.

In FIG. 4, a random number generator 37 uses the information of the first half of the input information extraction key 28 to generate a random number (r
_{i, k} ) 38 are generated one by one and sent to the information extraction processing unit 39. The latter half of the input information extraction key 28 is sent to the information extraction processing unit 39 as it is, and the information extraction intensity verifying_
Converted to range.

The input information extraction key 28 is the same as the information multiplexing key 3 used when multiplexing information so that correct information cannot be extracted even if other keys are input. Therefore, it is assumed that the random number generator 37 generates a sufficiently large number so as not to collide. Also, the frame images 33 of the number of unit images in the time axis direction after the information multiplexing have been performed.
It is assumed that n-bit information is repeatedly multiplexed l times.

In the following, the k-th (k: an integer greater than or equal to 0) information bit b _k multiplexed on the frame images 33 of the number of unit images in the time axis direction that have been information-multiplexed. in {0,1}
Will be described.

The information extraction processing unit 39 uses the random number r _ik sent from the random number generator 37 to calculate the orthogonal transform coefficient component position in a relatively low frequency region.

[0043]

[Outside 3]

Is selected by a one-to-one mapping and sent to an M × N × T orthogonal transformation processing unit 40 which calculates only a single component of the orthogonal transformation coefficient. As a result, the M × N × T orthogonal transformation Number

[0045]

[Outside 4]

This M × N × T orthogonal transformation processing unit 40
By calculating only a single component of the orthogonal transformation coefficient, high-speed information extraction is possible, and real-time processing such as during reproduction of a moving image can be performed.

In the information extraction processing section 39, the obtained M × N
× T orthogonal transform coefficient component value

[0048]

[Outside 5]

On the other hand,

[0050]

(Equation 8)

The k-th information multiplexed on the frame images 33 of the information multiplexed unit images in the time axis direction is extracted by calculating
Save to th. The information of the buffer 41 is k mod n
When n = n−1, it is sent to the majority decision processing unit 43.

The above information extraction process is performed n times, which is the bit length of the information multiplexed in each unit image in the time axis direction, and is repeated l times indicated by the secret information in the information extraction processing unit 39. By repeating, one set of multiplexed n-bit information is extracted for each bit. By inputting this to the majority decision processing unit 42 and deciding one by majority decision,
Information 34 multiplexed in one time axis unit image
Is output.

As described above, one embodiment of the present invention has been described. However, the present invention sets T = 1 when multiplexing information in a moving image, and multiplexes closed information for each frame, thereby obtaining a moving image. It is possible to simultaneously multiplex information on separate media, that is, the entire image and each still image, and extract the respective information.

Further, orthogonal transforms of a plurality of sizes having a small influence on each other due to the orthogonal transform are performed, and information multiplexing is performed separately for each orthogonal transform matrix.
Multiple pieces of information can be multiplexed and extracted.

Further, it is possible to detect the presence or absence of a tampering fact of the multiplexed moving image by making the plurality of pieces of information to be multiplexed the same and comparing each of the plurality of pieces of extracted information.

[0056]

According to the present invention, the following effects can be obtained. (1) By performing an inverse orthogonal transform on the information-multiplexed orthogonal transform coefficient matrix and taking measures to over-range the pixel values,
It is possible to measure the bottom-up of the borderline between the quality of the multiplexed information and the survival rate of the multiplexed information, which are in a trade-off relationship.

(2) Regardless of the presence or absence of encoding, the moving image, the key information, and the multiplexed information are input, and the multiplexed information is multiplexed into the moving image while maintaining the original image quality. Can be extracted.

(3) When multiplexing information into a moving image,
A video is decomposed into a group of frames of a unit time, and information multiplexing is performed by using an M × N × T size orthogonal transform, thereby eliminating block distortion and the like from the multiplexed image and quantizing by information compression. By multiplexing information into a frequency domain that does not affect the data, it is possible to extract information even when extreme compression is performed.

(4) When multiplexing information into a moving image,
By multiplexing closed information for each frame,
It is possible to simultaneously multiplex information on separate media of the entire moving image and each still image, and extract the multiplexed information.

(5) At the time of extracting information from a moving image, by calculating only a single component of a necessary orthogonal transform coefficient, information can be extracted in real time during reproduction of the moving image. High-speed extraction is possible.

(6) By utilizing the property that information content and multiplexed information cannot be separated, it can be used as an elemental technology of an advanced copyright protection system.

[Brief description of the drawings]

FIG. 1 is a schematic block diagram of an embodiment of an information multiplexing processing side of a moving image processing apparatus according to the present invention.

FIG. 2 is a detailed block diagram of the information multiplexing unit of FIG.

FIG. 3 is a schematic block diagram of an embodiment of an information extraction processing side of the moving image processing apparatus according to the present invention.

FIG. 4 is a detailed block diagram of the information extraction unit of FIG. 3;

[Explanation of symbols]

Reference Signs List 1 moving image processing device (information multiplexing processing device) 2 original moving image 3 information multiplexing key 4 multiplexed information 5 information multiplexed coded moving image 6 decoding processing unit 7 number of horizontal and vertical pixels and number of unit images in time axis direction Reference Signs List 8 Information multiplexing unit 9 Attribute information 10 Frame images of the number of unit images in the time axis direction 11 Encoding processing unit 12 Frame images of the number of unit images in the time axis direction after information multiplexing 26 Moving image processing device (information extraction processing device) 27 Information multiplexed encoded video 28 Information extraction key 29 Information multiplexed in video 30 Decoding processing unit 31 Number of horizontal and vertical pixels and number of unit images in time axis direction 32 Information extraction unit 33 Information multiplexed Frame images 34 with the number of unit images in the time axis direction of the information 35 information multiplexed on each unit image in the time axis direction 35

──────────────────────────────────────────────────続 き Continuation of the front page (56) References JP-A-6-189286 (JP, A) JP-A-6-141301 (JP, A) JP-A-5-236424 (JP, A) International publication 97/000578 (WO, A1) Jian Zhao, Eckhar d Koch, mbedding Ro bust Labels Into I mages For Copyrigh t Protection, Inter national Congress on Intellectual Pr operty Rights for Specialised Inform ation, Knowledge a nd Ne, 8 May 1995 (58) survey Field (Int.Cl. ⁷ , DB name) H04N ^7/ 00-7/088 H04N 7/12 H04N ^7/ 24-7/68 H04N 1/38-1/393 JSTPlus File

Claims (4)

(57) [Claims] 1. A digital moving image is framed in a unit time T.
Decompose into groups, and select any M × N × T (M × N is the maximum
I) Perform orthogonal transformation of size and perform M × N × T orthogonal transformation
Generates a coefficient matrix and sequentially generates random numbers from the information multiplexing key
And the random number is calculated from the M × N × T orthogonal transformation coefficient matrix.
Sequentially select any orthogonal transform coefficient using the selected
The orthogonal transform coefficient value based on the information to be multiplexed
Repeat for each unit time frame group
Multiplex different information to digital video in closed format
In an information multiplexing method, an orthogonal transform coefficient matrix of M × N × T size after information multiplexing is used.
An inverse orthogonal transform is performed to obtain an M × N × T size pixel value matrix.
If the pixel value indicated by the pixel value matrix exceeds the predetermined domain,
In this case, the AC components other than the multiplexed orthogonal transform coefficients
Change the amplitude of the value or the value of the DC component.
And maintain the values of the multiplexed orthogonal transform coefficients.
First, the pixel value indicated by the pixel value matrix obtained by the inverse orthogonal transform is
Over-range of pixel values so that
To obtain an information multiplexing orthogonal transform coefficient matrix in which
Characteristic information multiplexing method. 2. An information multiplexing method according to claim 1, wherein
Te performs orthogonal transformation of a plurality of sizes, each orthogonal transform coefficient of
By separately performing information multiplexing on the number matrix,
Information multiplexing method characterized by multiplexing multiple pieces of information
Law. 3. An information multiplexing method according to claim 1 is applied.
Multiplexing where different information is multiplexed into digital video
Digital video into a group of frames with unit time T
Then, the same orthogonal transformation of M × N × T size as in the information multiplexing is performed.
And a random number using the same information extraction key as the information multiplexing key.
Are sequentially generated, and the orthogonal transform coefficients are sequentially selected using the random numbers.
Multiplexed based on the value of the selected orthogonal transform coefficient.
Information extracted from each group of frames for each unit time.
An information extraction method that determines multiplexed information using voting.
Te performs orthogonal transformation of a plurality of sizes when multiplexing, each straight
Performing information multiplexing separately for the cross-transformation coefficient matrix
More information in digital multiplexed video
When multiplexing, multiple sizes of the same
Perform orthogonal transform and extract multiplexed information separately
Extracting a plurality of pieces of multiplexed information.
Information extraction method. 4. An information extraction method according to claim 3 , wherein only orthogonal transform coefficients selected from random numbers are extracted.
Calculate the value of the conversion coefficient to extract the multiplexed information.
An information extraction method characterized by the following.