JP2001292428A - Data hiding method of moving picture and data extract method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To imbed attached information in a moving picture without deteriorating the image quality and losing a compression efficiency. SOLUTION: This invention relates a data hiding method that imbeds information in a moving picture consisting of frames. Macro blocks to which the information is imbedded are specified. Imbedding rules are defined where contents of imbedded data are cross-referenced with an inter-frame prediction type of the macro blocks. The rules define that a 2-way prediction is adopted for a prediction type in the case of imbedding, e.g. '1', and forward or backward prediction is adopted when '0' is imbedded. The prediction type of macro blocks specified for an object of imbed processing is forcibly decided according to the data going to be imbedded according to the rules. In the case of extraction, extraction rules in reverse relation to the imbed rules are defined, and the imbedded bits are extracted on the basis of the prediction type of the specified macro blocks as the object of extraction processing by referencing the extraction rules.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a data hiding method for hiding message data in media data and a data extracting method for extracting hiding data.

[0002]

2. Description of the Related Art With the development of the multimedia society, many digitized image information and audio information are distributed on the Internet or as CD-ROM software. Since anyone can easily make a complete copy of such digital information without deterioration, its unauthorized use and protection of its copyright have become a problem. In order to prevent a third party from illegally copying media data such as image data and audio data, other information such as the author's signature is hidden as message data in the original media data (Hiding ) Technology has begun to attract attention. When digitized image data or the like is illegally copied, by confirming the signature hidden in the copy and specifying its source, it is possible to know whether or not it is caused by an illegal act. Such concealment technology is called "data hiding".

FIG. 1 is a halftone image in which digitized data is displayed on a display. The media data, which is a digitized image in FIG. 7A, includes a photo explanation (message) such as “hobo,” “river,” “child,” and “bird” as shown in FIG. It is hidden. Media data is obtained by dividing an image into small pieces based on a photograph or the like, and digitizing brightness and color for each small point. At that time, the original numerical value of the image is intentionally slightly changed. If the numerical value changes very little, there is almost no image distortion and humans do not notice. By taking advantage of this property, completely different information (message data) can be hidden in the original video. The message data may be any information, such as a lattice pattern, a ruler, or the signature of the creator of the image. The message data hidden in the media data can be extracted by processing with a special program. Therefore, based on this extracted message data,
It is possible to check whether the media data has been tampered with.

One of the well-known moving image compression methods is MPEG (Moving Pictur).
e Experts Group). When some additional information (message data) is included in a video bit stream (media data) using MPEG,
A method of preparing a data field and hiding additional information in this field is generally used.
However, such a method has a problem that the field can be easily separated from the media data, so that it is easy to detect and remove hidden additional information.

[0005]

SUMMARY OF THE INVENTION It is an object of the present invention to provide a novel method for embedding additional information in a moving image compressed using inter-frame prediction.

Another object of the present invention is to provide a method that hardly deteriorates the image quality even when additional information is embedded in a moving image.

Another object of the present invention is to make it difficult to remove embedded information from a moving image.

[0008]

According to a first aspect of the present invention, there is provided a data hiding method for embedding information in a moving image composed of a plurality of frames. The step of specifying one embedding area in a frame and the embedding rule corresponding to the type of the inter-frame prediction of the embedding area are referred to according to the embedding information. Determining a type.

The frame in which the embedment area exists is
Preferably, the frame is a bidirectional predictive encoded frame. In this case, the type of inter-frame prediction is preferably selected from forward prediction, backward prediction, bidirectional prediction, and intra-frame prediction.

The embedding rule preferably associates one bit with a bidirectional prediction type and associates the other bit with either forward prediction or backward prediction. Further, the embedding rule may associate the prohibition of data embedding with the type of intra-frame prediction.

When the prediction error of the inter-frame prediction type determined based on the embedding rule exceeds a predetermined threshold value, embedding of data in the embedding area is prohibited. Is preferred. Since the prediction type of the embedding area is forcibly determined based on the embedding rule, there is a possibility that image quality may deteriorate. Therefore, it is effective to set a threshold value as a criterion of image quality, and to prevent data from being embedded in a place exceeding the threshold value by applying an embedding rule.

When the reference number of forward prediction or the reference number of backward prediction in a bidirectional prediction coded frame is smaller than a predetermined number, embedding of data in an embedment area in this frame is prohibited. Good.
For example, when a scene change or the like occurs, the number of forward references or the number of backward references of the frame related to the scene change significantly decreases. In such a case, applying the embedding rule,
Forcibly determining the type of prediction can significantly degrade image quality. Therefore, it is preferable that the number of references is counted and the frame is not subjected to data embedding if the number is equal to or smaller than a predetermined threshold. Therefore, another invention provides a data hiding method for embedding information in a moving image composed of a plurality of frames, the reference number of forward prediction or the reference number of backward prediction in a frame having an embedding area for embedding information. Counting, and if the reference number is larger than a predetermined number, refer to an embedding rule that associates the content of the embedding data with the characteristics of the embedding area, and according to the embedding information, There is provided a data hiding method including a step of determining characteristics and a step of prohibiting embedding of data in an embedding area in the frame when the number of references is smaller than a predetermined number.

The second invention relates to a data hiding method for embedding information in an image with redundancy. First, a plurality of embedding regions for embedding the same data in an image are specified. Next, referring to the embedding rules,
The same data is embedded in each embedding area according to the information to be embedded. For example, if the bit “1” is 3
Consider the case of embedding in one embedding area. By referring to an embedding rule that defines the value of the bit and the characteristics of the embedding region (for example, the type of prediction), all three embedding regions are set to the characteristics corresponding to the bit “1”. As a result, information can be embedded in an image with redundancy.

[0014] The third invention relates to a data extracting method for extracting information embedded in an encoded moving image. At least one embedding area in which the information is embedded is specified in the frame. The data embedded in the specified embedding area is extracted by referring to an extraction rule that associates the type of inter-frame prediction with the content of the data to be embedded.

The frame in which the embedding area exists is:
Preferably, the frame is a bidirectionally predicted coded frame, and the types of inter-frame prediction are forward prediction, backward prediction,
Preferably, it is selected from bidirectional prediction and intra-frame prediction.

The extraction rule may be such that one bit is associated with bidirectional prediction and the other bit is associated with forward prediction or backward prediction. Furthermore, the extraction rules are:
Intra-frame prediction may be associated with prohibition of data embedding.

The fourth invention relates to a data extracting method for extracting information having redundancy embedded in an image.
First, a plurality of embedding areas in which the same data is embedded are specified in an image. Then, the data embedded in each embedding area is extracted with reference to an extraction rule that associates the characteristics of the embedding area with the data to be extracted. At this time, if different data is extracted from each of the embedding areas, the number of the embedding areas is compared for each of the extracted data, and the data with the larger number is specified as the embedded information. For example, it is assumed that data extracted from three embedding areas A, B, and C in which the same data (bit “1”) should be embedded are bit “1”, bit “1”, and bit “0”, respectively. I do. In this case, the number of embedded areas from which the bit “1” is extracted is two, and the number of the embedded areas from which the bit “0” is extracted is one. The bit with the larger number is
Since it is more probable, it is recognized that the bit “1” has been embedded. Originally, if the bit “1” is embedded, the bits to be extracted from the three embedding areas should be “1”. However, the embedded information may change for some reason. Therefore, a method for extracting information having redundancy embedded in an image as in the fourth invention is effective.

The fifth invention relates to a moving picture coding system comprising a plurality of frames and embedding information in a moving picture using inter-frame prediction. The system uses forward prediction to embed information in a first frame based on an embedding region specified in a first frame and a reference region in a second frame referred to by the embedding region to embed information.
, A second prediction error based on the embedding region and a reference region in a third frame referred to by the embedding region is calculated using backward prediction, and the second prediction error is calculated using bidirectional prediction. Error calculating means for calculating a third prediction error based on the region and the reference regions in the second and third frames referred to by the embedding region; The type of inter-frame prediction specifies that either forward prediction or backward prediction is used, and when embedding other data, the type of inter-frame prediction refers to an embedding rule that specifies that bidirectional prediction is used. The type of inter-frame prediction in the embedding area is determined according to the content of the information to be embedded, and the determined inter-frame prediction is determined. Depending on the type, and a determining means buried region and specifying the reference area referenced further first, to identify the one of the second or third prediction error.

Here, when the prediction error of the type of inter-frame prediction determined based on the embedding rule for a certain embedding region exceeds a predetermined threshold value, the deciding means determines the data of the embedding region. A first prohibition unit for prohibiting embedding may be included.
When the reference number of forward prediction or the reference number of backward prediction in the bidirectional prediction coded frame is smaller than a predetermined number, the determining unit prohibits embedding of data in the embedment area in the frame. A second prohibition means may be included.

The sixth invention relates to a moving picture decoding system for extracting information embedded in a coded moving picture. The system shall have at least one embedded information
By referring to the specifying means for specifying one embedding area in the frame and the extraction rule corresponding to the content of the data to embed the inter-frame prediction type in the embedding area, the embedding is performed based on the inter-frame prediction type in the embedding area. Extraction means for extracting information.

According to a seventh aspect of the present invention, in order to execute a data hiding process for embedding information in a moving image composed of a plurality of frames, a computer specifies at least one embedding area in which information is to be embedded in a frame. And determining the type of inter-frame prediction in the embedding area according to the information to be embedded by referring to an embedding rule associated with the content of the data in which the type of inter-frame prediction in the embedding area is to be embedded. This is a medium on which the program is recorded.

According to an eighth aspect of the present invention, in order for a computer to execute a data extraction process for extracting information embedded in an encoded moving image, at least one embedded area in which information is embedded is included in a frame. Specifying the type of inter-frame prediction in the embedding area and extracting the embedded information from the type of inter-frame prediction in the embedding area with reference to an extraction rule associated with the content of the data to be embedded. It is a medium on which a program is recorded.

[0023]

DESCRIPTION OF THE PREFERRED EMBODIMENTS [Embedding Data] First, MP
A case will be described in which some additional information (message data) is embedded in a video bitstream (media data) using EG. In MPEG, forward prediction from a past reproduced image, backward prediction from a future reproduced image, and bidirectional prediction using both forward prediction and backward prediction are used. FIG. 2 is a diagram for explaining an arrangement state of pictures in MPEG. As shown in the figure, in order to realize bidirectional prediction, MPEG defines three types of frames, namely, a Ι picture, a P picture, and a B picture.

Here, the Ι picture is an image which has been intra-coded (intra-coded), and all macroblocks in this picture have been intra-coded (intra-frame predictive coding). The P picture is an image that has been subjected to inter-frame forward prediction coding, and some macroblocks in this picture may be intra-coded in some cases. A B picture is an image that has been subjected to inter-frame bidirectional predictive coding. A macroblock in a B picture is basically coded by forward prediction, backward prediction, or bidirectional prediction, but may include intra coding. A screen for intra-coding all of the screen is a Ι picture, and the Ι picture and the P picture are coded in the same order as the moving picture. In contrast, a B picture is
(4) After processing the picture and the P picture, encode the B picture inserted between them.

The embedding area in which information (message data) is embedded is a macroblock of a B picture,
One bit of information can be embedded in one macroblock. Therefore, if the message data is multi-bit, it is necessary to perform the embedding process on the corresponding number of macro blocks. FIG. 3 is a diagram showing a state of a macro block arranged in a B picture. A macroblock is a unit to be encoded. For each macroblock, motion compensation is performed on a luminance block of 16 × 16 pixels, and a motion compensation inter-frame prediction method is performed on a macroblock basis, and information compression based on temporal screen correlation is performed.

Macroblocks in a B picture can be classified into the following four types of prediction. -Intra macroblock (intra-frame predicted macroblock) A macroblock that is coded using only the information of the screen itself without performing interframe prediction.

Forward-predicted macroblock A macroblock to be forward-predicted and coded by referring to a past Ι-picture or P-picture (reference frame). Specifically, a square area of 16 × 16 pixels that is most similar in the past reference frame is searched, and a prediction error (ΔP) which is a difference from the square area and a spatial relative position (movement) are calculated. Vector). Here, the prediction error ΔP is expressed as a difference between luminance and chrominance of 16 pixels × 16 pixels. The criterion for selecting a similar square area is left to the encoder.

Backward predicted macroblock A backward predicted macroblock is a macroblock that is backward predicted and coded by referring to a future reference frame in display order. The most similar area is searched for in the future reference frame, and information on the prediction error (ΔN), which is the difference from the area, and the spatial relative position (motion vector) is stored.

A bidirectionally predicted macroblock is a macroblock that is bidirectionally predicted coded by referring to past and future reference frames. The most similar area in the past reference frame and the most similar area in the future reference frame are searched for, and a prediction error (( ΔN + ΔP) / 2) and their spatial relative positions (two motion vectors).

In order to embed message data, at least one macroblock to be embedded must be specified in a B picture. This may be defined, for example, as each macroblock (embedded area) existing from the first line to the third line of the B picture, or may be the entire macroblock in a certain frame. In addition to the format defined in advance as described above, the format can be determined by using an algorithm for generating a position sequence. The algorithm for generating the position series is described in, for example, Japanese Patent Application No. 8-159330.
No. (in-house reference number JA996-044).

Next, based on the embedding rule, the macro block specified as the object of the embedding process is
One-bit data is embedded in one macroblock. This embedding rule associates bit information with a macroblock prediction type, and includes, for example, the following rules.

(Embedding Rule) Bit information to be embedded Type of inter-frame prediction of macroblock Bit “1” Bidirectionally predicted macroblock (represented by B) Bit “0” Forward prediction (represented by P) or backward prediction macroblock (N Embed prohibited intra macro block

For example, message data "1010"
Consider embedding. The 4-bit data is sequentially embedded in four embedding areas (macroblocks) from the first to the fourth left of the first line shown in FIG. First, since the first data is bit “1”, the prediction type of the leftmost macroblock (first embedded area) is determined to be bidirectional prediction (B) according to the embedding rule.
The prediction error in this case is a prediction error that is a difference between the average of the most similar area in the past reference frame and the average of the most similar area in the future reference frame.

The next data is bit "0". Therefore, according to the embedding rule, the prediction type of the second macroblock (second embedding region) is one of the forward prediction (P) and the backward prediction macroblock (N). In this case, in order to suppress image quality degradation, a prediction error in forward prediction and a prediction error in backward prediction are compared, and a type having a small prediction error is selected. In the example of FIG. 3, since the prediction error of the second macroblock is smaller in forward prediction than in backward prediction, forward prediction (P) is selected.

A similar procedure is repeatedly applied to the third and fourth buried regions. That is, the prediction type of the third macroblock from the left is bidirectional prediction (B).
The prediction type of the macroblock is determined to be backward prediction (N) because the prediction error in backward prediction is smaller.

As described above, the types of the inter-frame prediction of the first to fourth embedding regions are respectively set to “BPBN”.
As a result, the message data "1010" is embedded in these areas. If bits are to be embedded in a certain embedding area, the image quality may be significantly degraded. In this case, no bits are embedded in the embedding area, and the prediction type of the area is
It is assumed to be an intra indicating “embedding prohibited”. This will be described later.

[Extraction of Data] A method of extracting the message data embedded by the above procedure will be described. When extracting message data, first, information for specifying the macroblock in which the message data is embedded must be provided. This can be given as information from the outside or embedded in the data itself in advance. In addition, if the position of the embedding area is standardized, or if the algorithm for generating the position sequence is known, the message data can be extracted. As a method for extracting message data using the position sequence, for example, the technique disclosed in Japanese Patent Application No. 8-159330 can be used.

Next, the information embedded in the specified embedding area is extracted by referring to the extraction rule based on the type of prediction in the specified embedding area. This extraction rule associates the type of prediction of a macroblock with bit information, and must be given as information at the time of extraction. The rules include, for example, the following rules. It should be noted that the association between the prediction type and the bit information in this extraction rule is the same as that of the above-described embedding rule. When the prediction type is intra, it is determined that “bits are not embedded” in the embedding area.

(Extraction Rule) Type of Interframe Prediction of Macroblock Extracted Bit Information Bidirectionally Predicted Macroblock (B) Bit “1” Forward Prediction (Represented by P) or Bit “0” Backward Predicted Macroblock Intra Macroblock bits are not embedded

The case where message data is embedded as shown in FIG. 3 will be described. It is assumed that it is known that the message data is embedded in the first to fourth macroblocks on the left of the first line in FIG. Since the prediction type of the leftmost macroblock is bidirectional prediction (B), referring to the extraction rule,
Bit "1" is extracted. Since the prediction type of the second macroblock is forward prediction (P), bit “0” is extracted according to the extraction rule. By repeatedly applying the same procedure to other macro blocks, bits "1" and "0" are sequentially extracted. As a result, message data "1010" is extracted from these areas.

If the prediction type of the rightmost macroblock is intra, the above extraction rule gives
It is determined that "bits are not embedded in this macroblock". As a result, the message data becomes "101".

[Measures for Image Quality (Introduction of Threshold Value)] MPE
The type of macroblock prediction in G can be freely selected by the encoder within the range allowed for each picture. Generally, the prediction type of the macroblock is selected to have the smallest prediction error. However, a feature of this embodiment is that the prediction type of the macroblock is selected according to the embedding rule.
Since the relationship between the prediction type and the bit information in the extraction rule is the same as the relationship defined by the embedding rule, by referring to this extraction rule, the embedded data can be accurately extracted.

However, if the prediction type is determined according to the embedding rule, there is a possibility that a prediction type having a large prediction error is selected so that the image quality deterioration can be visually recognized. For the prediction error, the sum of the absolute values of the differences for each pixel or the sum of squares is often used.
Since it is not specified in PEG, the encoder is free. However, no matter what criterion, if a prediction type having a large prediction error is selected, image quality will be degraded. Therefore, a certain threshold value is set in advance for the prediction error, and if the prediction error in the selected prediction type is larger than this threshold value, no bits are embedded in the macro block. It is preferable to do so. In this case, the prediction type of the macro block is set to “intra” according to the above embedding rule. This will be described in more detail with reference to FIG.

FIG. 4 is a diagram for explaining the relationship between the prediction type of the macroblock and the prediction error. In this figure, the vertical axis represents the prediction error, and indicates that the larger the prediction error, the greater the deterioration of the image quality. In addition, a threshold is set as a degree of allowable prediction error, that is, a criterion for determining that image quality does not deteriorate. In the figure, three short horizontal bars (i), (ii), (iii) connected by one vertical bar
Indicates one of three prediction error values of forward prediction, backward prediction, and bidirectional prediction of a macroblock. Focusing on the relationship between the threshold and the three prediction errors, macroblocks can be classified into four types (a), (b), (c), and (d). That is, when all three prediction errors are smaller than the threshold value (type (a)), and when one of the prediction errors exceeds the threshold value (type (b)), two prediction errors are generated. The case where the threshold value is exceeded (type (c)) and the case where any prediction error exceeds the threshold value (type (d)).

For a macroblock of type (d),
Whatever type is selected, the prediction error exceeds the threshold value and the image quality is greatly deteriorated. Therefore, it is not preferable to use this type of block as the embedding area. In this case, this macroblock becomes an intra macroblock according to the embedding rules. However, MPEG
In most cases, most macroblocks in a B-picture are inter-frame predictions (forward prediction, backward prediction, or bidirectional prediction), and thus the probability that macroblocks of this type actually appear is not high.

Regarding the type (a) macroblock, no matter what prediction type is selected, the prediction error does not exceed the threshold value. That is, no matter what data is embedded, the deterioration of the image quality is not noticeable, so that this macro block can be used as an embedded area. Also, regarding type (b), even if this block is used as an embedding area, it is considered that image quality does not deteriorate. Normally, the bidirectional prediction error is not the worst of the three prediction errors (ie,
It will not be a horizontal bar (i)). According to the above embedding rules, both the bit “1” and the bit “0” can be embedded in this type of block without exceeding the threshold value. Therefore, when the above embedding rule is used, the type (a)
And bits can be embedded in a macroblock of type (b).

It should be noted that, with respect to the type (c), it is not preferable in principle to use this block as an embedding area. This is because, when the bidirectional prediction error is the horizontal bar (iii), the forward prediction error and the backward prediction error both exceed the threshold, so that image quality is deteriorated depending on the bits to be embedded. However, even in this case, when the prediction error in the prediction type corresponding to the bit to be actually embedded does not exceed the threshold value (for example, when a certain data is embedded according to the embedding rule, it becomes horizontal). Rod (iii)
In the case where the prediction error is equal to or smaller than the threshold value indicated by), it is possible to use this type of block as the embedding area.

In view of the above, three prediction errors are obtained for a certain macroblock into which data is to be embedded, and the prediction error of the prediction type determined based on the embedding rule is determined by a predetermined threshold value. In the case where the number exceeds the limit, it is preferable to prohibit embedding of data in this macro block. In this case, the prediction type of the macroblock for which embedding is prohibited is determined according to the embedding rule described above.
Become an intra.

The type (d) intra macroblock is not used for data embedding and is made an invalid bit. However, as described above, since the appearance rate is not high in practice, the information to be embedded is redundant. If the performance is allowed, it can be compensated by error correction coding.

According to the present embodiment, when coding a moving image, the prediction type of the macroblock and the bits to be embedded are determined in association with each other. Therefore, the message data can be embedded in the moving image with almost no effect on the compression efficiency of the moving image and with almost no deterioration in image quality. Further, it is very difficult to remove the message data embedded in this manner from the moving image. Further, since the amount of information to be embedded hardly depends on the content of the image, the message data can be efficiently embedded.

[Measures for Image Quality (Measures for Scene Change)] When a scene change occurs, most of the macroblocks in the B picture sandwiched between the Ι and P pictures before and after the scene change have the type shown in FIG. It is known to be like c). FIG. 5 is a diagram for explaining a reference image when a scene change has occurred. FIG.
The figure shows a case where there is no scene change, and FIG.
The case where a scene change has occurred between picture 2 and picture 3 is shown. In both FIGS. (A) and (b), the two pictures at both ends are あ り or P pictures, and the two central pictures are B pictures. The illustrated arrows indicate the reference relationship between the frames.

If there is no scene change, the B picture has many bidirectionally predicted macroblocks. However, when a scene change as shown in FIG. 4B occurs, the number of backward predicted macroblocks in picture 2 that has an error smaller than the threshold value in FIG. It becomes a forward prediction macro block. Also, the number of forward prediction macroblocks in the picture 3 in which the error is smaller than the threshold value in FIG. 4 is extremely reduced, and most of them become backward prediction macroblocks. Therefore, it is not preferable to embed data in such a picture. Therefore, the number of forward-predicted macroblocks and backward-predicted macroblocks is monitored, and when these numbers fall below a certain threshold, it is determined that a scene change has occurred, and such a picture change is detected. It is preferable not to embed data in (in accordance with embedding rules, the prediction type is set to “intra”).

[Measures for Image Quality (Measures for Occlusion)] Occlusion means that when a certain object moves, an object hidden behind it suddenly appears or hides. When occlusion occurs, the macroblock related to occlusion in the entire picture is of the type shown in FIG. in this case,
As described above, there is no problem if the prediction error of the prediction type determined by the embedding rule is smaller than the threshold value, but otherwise the image quality is deteriorated. When the image quality is emphasized, this deterioration can be avoided by using an error correction code. In other words, one bit of information is not represented by one macroblock, but information is given redundancy, and information of one bit is represented by a plurality of macroblocks. In this case, one embedding area is constituted by a set of a plurality of these macro blocks.

For example, consider a case where three macroblocks represent information equivalent to one bit. In this case, even if the prediction type is opposite to the bit that one of the three macroblocks wants to represent, the bits can be correctly represented from the remaining two. If a set of macroblocks representing one bit of information (here, three macroblocks) includes a predetermined number or more of intra macroblocks, data is not embedded in the set. Conversely, if two or more of the three macroblocks are of the type shown in FIG. 4 (c), it is necessary to make one of them intra and explicitly indicate that "data is not embedded".

When occlusion occurs, it is conceivable that a plurality of adjacent macroblocks existing in a portion related to occlusion are simultaneously in the state of type (c) in FIG. In view of such a point, it is desirable that a set of macroblocks constituting the error correction code use macroblocks located at distant positions in the picture.

Although the above description has been made by taking MPEG as an example, the present invention is not limited to MPEG, and can be applied to other image compression methods using inter-frame predictive coding. It is natural. In that sense,
The embedding area referred to in this specification is not limited to a macro block.

The above-described embedding rules and extraction rules are merely examples, and the present invention is not limited to these rules, and various rules can be used. For example, ternary data is embedded in one macro block by associating a forward prediction macro block with data "0", a backward prediction macro block with data "1", and a bidirectional prediction macro block with data "2". It is also possible.

Further, in the above example, a B picture, that is, a bidirectional prediction frame has been described as an example, but data can be embedded in a P picture. Since the types of macroblocks constituting the P picture are forward prediction and intra, the bits may be associated with these bits. However, from the viewpoint of suppressing the deterioration of the image quality and the increase of the data amount, it is better to embed in the B picture as described above.
This is because if a macroblock to be an intra is forcibly referred to forward according to an embedding rule, the image quality is degraded, and conversely, the data amount is increased.

[0059]

FIG. 6 is a block diagram of a moving picture coding system using the present invention. In the memory 61, moving image data composed of a plurality of frames is stored. The frame memory 62 stores data relating to past frames, and is a frame. The memory 63 stores data on future frames in the order of display.

The area specifying unit 64 specifies an embedding location of data to be embedded as additional information. Thereby, at least one macroblock is specified in the frame.

The error calculator 65 is provided in the frame memory 6
The forward prediction error, the backward prediction error, and the bidirectional prediction error are obtained based on the data stored in the second and the third data. The forward prediction error is calculated using the forward prediction from the embedding region and the reference region in the second preceding frame referred to by the embedding region. The backward prediction error is calculated from the embedded region and a reference region of a frame behind by the embedded region using backward prediction. Then, the bidirectional prediction error is calculated from the embedding area and the reference areas in both the front and rear frames referred to by the embedding area using the bidirectional prediction.

The determiner 66 refers to the embedding rule, and
The data to be embedded is embedded in the embedding area by manipulating the characteristics of the macro block that is the embedding area. Specifically, the embedding rule specifies that when one piece of data is embedded in an embedding area, the prediction type in the embedding area uses one of forward prediction and backward prediction. Further, when embedding other data, it is specified that the prediction type uses bidirectional prediction. Decider 66
Determines the type of inter-frame prediction in the embedding area according to the content of the information to be embedded, and specifies the reference area to be referred to by the embedding area according to the determined type of inter-frame prediction. First, one of the second and third prediction errors is specified.

Thereafter, the encoder 67 encodes the signal output from the determiner 66.

Here, when the prediction error in the type of inter-frame prediction determined based on the embedding rule exceeds a predetermined threshold value for a certain embedding area, the determiner 66 determines the data of the embedding area. Designed to prohibit embedding. This prevents the image quality from deteriorating due to embedding. Further, the determiner 66 is designed to prohibit embedding of data in an embedding area in the frame when the reference number of forward prediction or the reference number of backward prediction in the bidirectional prediction coding frame is smaller than a predetermined number. ing. By counting these reference numbers, a scene change can be detected. When a scene change occurs, embedding is not performed in a frame related to the scene change, so that image quality deterioration can be prevented.

[Moving Picture Decoding System] FIG. 7 is a block diagram of a moving picture decoding system using the present invention. The memory 71 stores encoded moving image data in which additional information is embedded. The area specifying unit 72 specifies at least one embedded area in which additional information is embedded in a frame. The extraction means 73 extracts additional information embedded therein from the type of inter-frame prediction in the embedded area with reference to the extraction rule. Then, the extractor 7 is decoded by the decoder 74.
The encoded data output from 3 is decoded, whereby the moving image is reproduced.

[Embed Fingerprint] MP
In the EG, since the B picture is not referred to by other pictures, changing the prediction type of the macroblock in the B picture does not affect other pictures. This can be used to embed a fingerprint. The fingerprint is unique information that differs for each owner. A typical example of the use of a fingerprint is that when a moving image data is issued to a third party, a mark is embedded in the moving image in advance so that the issuer can identify the third party to which the data is issued. Is the case. In this way, if an illegal act such as piracy is performed, the source of the copy can be specified. Can be charged. It is also conceivable to give valid owner registration information to the encrypted video product and embed a fingerprint corresponding to the information.

When a fingerprint is embedded, basically, a “bidirectionally predicted macroblock” and a “forward or backward predicted macroblock having a smaller prediction error” generated at the time of MPEG encoding.
Keep both. Then, an appropriate person is selected according to the third party to which the certificate is issued. This does not affect any other picture or a data layer (for example, a slice) higher than the macroblock of the picture.

[0068]

As described above, according to the present invention, message data can be embedded in a moving image with almost no effect on the compression efficiency of the moving image and with almost no deterioration in image quality. Further, since the message data is embedded in a part necessary for the code of the moving image, it is difficult to remove the message data from the moving image without deteriorating the image quality. Further, since the amount of information to be embedded hardly depends on the content of the image, the message data can be efficiently embedded.

[Brief description of the drawings]

FIG. 1 is a halftone image in which digitized data is displayed on a display.

FIG. 2 is a diagram for explaining an arrangement state of pictures in MPEG.

FIG. 3 is a diagram illustrating a state of a macro block arranged in a B picture.

FIG. 4 is a diagram for explaining a relationship between a prediction type of a macroblock and a prediction error.

FIG. 5 is a diagram for explaining a reference image when a scene change occurs.

FIG. 6 is a block diagram of a video encoding system.

FIG. 7 is a block diagram of a video decoding system.

[Explanation of symbols]

 61 ... memory 62, 63 ... frame memory 64 ... area specifying device 65 ... error calculator 66 ... determiner 67 ... encoder

Continued on the front page (72) Inventor Junji Maeda 1623-14 Shimotsuruma, Yamato-shi, Kanagawa Prefecture Inside the Tokyo Research Laboratory, IBM Japan, Ltd. 14 Inside the Tokyo Research Laboratory, IBM Japan, Ltd.

Claims (5)

[Claims] 1. A data hiding method for embedding information in an image, comprising the steps of specifying a plurality of embedding areas for embedding the same data in the image; By referring to the embedding rules associated with the characteristics,
Determining each of the embedding areas to have the same characteristics, and embedding the same data in each of the embedding areas, thereby embedding the information in an image with redundancy.・ Hiding method. 2. A data extraction method for extracting information having redundancy embedded in an image, comprising:
Identifying a plurality of embedment areas in which the same data is embedded, and referring to an extraction rule in which the characteristics of the embedment area are associated with the data to be extracted, from the characteristics of each embedment area, Extracting the data embedded in the embedding area; and, when different data is extracted from each of the embedding areas, comparing the number of the embedding areas for each of the extracted data. Identifying the data as embedded information. 3. The method according to claim 2, wherein the determining unit determines whether or not the reference number of the forward prediction or the reference number of the backward prediction in the bidirectionally coded frame is smaller than a predetermined number. 5. The moving picture coding system according to claim 4, further comprising a second prohibiting unit for prohibiting embedding of the moving image. 4. A computer-readable recording medium including a program for performing a data hiding process for embedding information in an image, wherein the program is stored in a computer for embedding the same data in the image. A function of specifying the embedding area of the embedding area, a function of referencing an embedding rule that associates the content of the embedding data with the characteristic of the embedding area, and determining each of the embedding areas to have the same characteristic according to the embedding information A recording medium that realizes a function of embedding the same data in each of the embedding areas, thereby embedding the information in an image with redundancy. 5. A computer-readable recording medium including a program for performing a process of extracting redundant information embedded in an image, wherein the program is stored in a computer. Referring to a function for specifying a plurality of embedment areas in which data is embedded and an extraction rule in which the characteristics of the embedment area are associated with the data to be extracted, the characteristics of each embedment area are used to determine each of the embedment areas. Function to extract data embedded in
When different data is extracted from each of the embedding areas, a function of comparing the number of the embedding areas for each of the extracted data and specifying the data having the larger number as embedded information is provided. A recording medium to be realized.