JP4018335B2 - Image decoding apparatus and image decoding method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image encoding apparatus and method for inputting and encoding a moving image, and an image decoding apparatus and method for decoding the encoded code.
[0002]
[Prior art]
Conventionally, as an image encoding method, an intra-frame encoding method such as Motion JPEG or Digital Video, or H.264 using inter-frame predictive encoding is used. 261, H.M. Coding schemes such as H.263, MPEG-1, and MPEG-2 are known. These encoding methods are internationally standardized by ISO (International Organization for Standardization) and ITU (International Telecommunication Union). The intra-frame coding method performs coding independently for each frame, and is easy to manage the frames, so that it is most suitable for an apparatus that requires editing of moving images and special reproduction. In addition, the inter-frame predictive coding method has a feature that coding efficiency is high because inter-frame prediction based on a difference in image data between frames is used.
[0003]
Furthermore, an international standardization work for MPEG-4 is in progress as a general-purpose next-generation multimedia coding standard that can be used in many fields such as computers, broadcasting, and communication.
[0004]
With the spread of such digital coding standards, the content industry has strongly raised the problem of copyright protection. That is, there is a problem that the content cannot be provided with peace of mind for a standard whose copyright is not sufficiently protected.
[0005]
For this reason, in MPEG-4, IPMP (Intellectual Property Management and Protect) technology has been introduced, and a function of interrupting or resuming reproduction of an image is being studied in order to protect copyright. In this method, copyright protection is realized by not reproducing a frame that requires copyright protection.
[0006]
On the other hand, methods and services have been started to provide viewers with images that can be recognized to some extent by scrambling the images. Specifically, this is realized by replacing an arbitrary scanning line or pixel in the television signal. There is also a method for converting an output reproduced image by a reproducing apparatus.
[0007]
Scalability functions are also being studied, and there is a method for encoding and decoding images while having multiple levels of image time and spatial resolution.
[0008]
[Problems to be solved by the invention]
Therefore, the following problems occur.
{Circle around (1)} With conventional IPMP technology, it is impossible to provide information to the viewer at all because the decoding is stopped or the reproduction of the image is stopped for the image whose copyright is to be protected. This means that information of the content (for example, an image) cannot be provided to a viewer who does not have the right to view the video or the like. Originally, the content provider wants to spread the content to a larger number of viewers as a business, and to that end, it provides a certain amount of content information to viewers who do not have the right to view. There is a need to.
(2) In addition, in the above-described series of image encoding schemes, when conventional scrambling is applied to the entire bitstream, a viewer who has a decoder that cannot scramble or a viewer who does not have the right to view normal decoding. Cannot be recognized at all.
(3) Furthermore, a series of image coding schemes achieve high coding efficiency by utilizing the spatial and temporal correlation of the image, but when conventional scrambling is applied to the input image at the time of coding, The correlation between the image space and the time direction is lost, and the coding efficiency is significantly reduced.
(4) Furthermore, even if a part of the bit stream is scrambled, in a reproduced image of a moving image coding method using interframe predictive coding, distortion in one frame propagates to the next frame. It will gradually accumulate. For this reason, distortion does not occur constantly, and when the reproduced image is viewed on the decoding side, it is difficult to determine whether it is distortion for scrambling or another symptom of malfunction.
(5) In recent years, the processing of an image encoding / decoding apparatus has become complicated, and it has been assumed that encoding and decoding by software are assumed. In such a case, there is a problem that the performance of the entire apparatus is lowered when the load of the scramble process other than the image encoding / decoding process is large.
[0009]
The present invention has been made in view of the above-described conventional example. An image code that can scramble a part of a target bitstream to be protected at the time of image encoding and encode an image without lowering the encoding efficiency. An object of the present invention is to provide an apparatus and a method thereof.
[0010]
It is another object of the present invention to perform normal reproduction in an image decoding device of a viewer who has viewing rights, and to perform reproduction so that an approximate overview of an image can be recognized in an image decoding device of a viewer who does not have viewing rights. It is an object to provide an image encoding apparatus and method for encoding an image so that the image can be transmitted.
[0011]
It is another object of the present invention to provide an image decoding apparatus that can decode and reproduce an image encoded by the image encoding apparatus.
[0012]
[Means for Solving the Problems]
  In order to achieve the above object, an image decoding apparatus according to an aspect of the present invention has the following arrangement. That is,
  An encoded bitstream is input and the bitstream is protected with a protection code for protecting intellectual property and one or moreExpansionWith layersA base layer having a lower resolution than the enhancement layerDistribution means for distributing to,
  Authentication data input means for inputting authentication data from the outside,
  An authentication means for checking consistency between the authentication data and the protection code;
  SaidOne or moreExpansionA descrambling means for descrambling the layer;
  Enhancement layer decoding means for decoding the enhancement layer distributed by the distribution means or the enhancement layer descrambled by the descrambling means;
  Base layer decoding means for decoding the base layer distributed by the distributing means;
  If the authentication result by the authentication means is coincident and the enhancement layer is scrambled, select an image obtained by decoding the enhancement layer released by the descrambling means,
  When the authentication result by the authentication means does not match and the enhancement layer is scrambled, or when the authentication result by the authentication means matches and the extension layer is not scrambled Selects an image obtained by decoding the enhancement layer,
  If the authentication result by the authentication unit is inconsistent and the enhancement layer has not been scrambled, a selection unit that selects an image obtained by decoding the base layer;
  SaidChoiceTo the meansChooseImage output means for outputting the processed image.
[0013]
  An image decoding method according to another aspect of the present invention for achieving the above object is as follows:
  An encoded bitstream is input and the bitstream is protected with a protection code for protecting intellectual property and one or moreExpansionWith layersA base layer having a lower resolution than the enhancement layerA distribution step of distributing to
  An authentication data input process for inputting authentication data from the outside,
  An authentication step for checking the consistency between the authentication data and the protection code;
  SaidOne or moreExpansionA descrambling step for descrambling the layer;
  An enhancement layer decoding step of decoding the enhancement layer distributed by the distribution step or the enhancement layer descrambled by the descrambling step;
  A base layer decoding step of decoding the base layer distributed by the distributing step;
  If the authentication result by the authentication step is the same and the enhancement layer has been scrambled, select an image obtained by decoding the enhancement layer released by the descrambling step,
  When the authentication result by the authentication step is inconsistent and the enhancement layer is scrambled, or when the authentication result by the authentication step is coincident and the extension layer is not scrambled Outputs an image obtained by decoding the enhancement layer,
  If the authentication result of the authentication step is inconsistent and the enhancement layer is not scrambled, a selection step of selecting an image obtained by decoding the base layer;
  SaidChoiceIn the processChoiceAn image output step for outputting the processed image.
[0016]
DETAILED DESCRIPTION OF THE INVENTION
Preferred embodiments of the present invention will be described below in detail with reference to the accompanying drawings.
[0017]
[Embodiment 1]
FIG. 1 is a block diagram showing the configuration of a video encoding device according to Embodiment 1 of the present invention, and FIG. 3 shows the configuration of a video decoding device that decodes a code encoded by this encoding device. It is a block diagram. In the first embodiment, in the MPEG-4 encoding system, the spatial scalability function is provided, and the enhancement layer 6001 is scrambled by inverting the sign bit of the Huffman code obtained by encoding the DCT coefficient. Will be described. Refer to the ISO / IEC recommendation for details of the MPEG-4 encoding method.
[0018]
In FIG. 1, reference numeral 100 denotes a frame memory (FM) which stores input image data for one frame and outputs it as a macroblock which is a coding unit. Here, the macro block has a luminance of 16 × 16 pixels, and the color differences Cb and Cr are both 8 × 8 pixels, the luminance is 4 blocks, and the color difference is 1 block each. Reference numeral 101 denotes a DCT device, which performs two-dimensional discrete cosine transform (DCT) in units of 8 × 8 pixels (blocks), sequentially converts input macroblocks for each block, and outputs DCT coefficients. . Reference numeral 102 denotes a quantizer that sequentially quantizes the DCT coefficients for each block and outputs the quantized representative value. Reference numeral 103 denotes an inverse quantizer that outputs the quantized representative value as a DCT coefficient. Reference numeral 104 denotes an inverse DCT unit that converts the inversely quantized DCT coefficients into original image data. Reference numeral 105 denotes a frame memory for storing locally decoded images. A motion compensator 106 receives input image data from the frame memory 100 and locally decoded image data from the frame memory 105 and an upsampling unit 301 described later, and performs motion vector detection for each macroblock to perform prediction. Output an image.
[0019]
A variable length encoder 107 performs Huffman coding on the quantized representative value and outputs a Huffman code. Reference numeral 108 denotes a DCT code inverter that inverts the Huffman code from the variable length encoder 107. The sign bit of the Huffman code in MPEG-4 is the last bit of the bit string, which is “0” for positive and “1” for negative. Therefore, inversion of this sign bit is the processing of the DCT code inverter 108. Note that when DCT [i] (i = 0 to 63) is a continuous column in the DCT coefficient block in the zigzag scan order, the Huffman code inverted in the first embodiment is i = 3 to 63. .
[0020]
Reference numeral 109 denotes a selector that selects either the output from the variable length encoder 107 or the output from the DCT code inverter 108 in accordance with a scramble ON / OFF flag input from the outside. A multiplexer 110 multiplexes a Huffman code output from the selector 109, a scramble ON / OFF flag input from the outside, and an IP encoded code output from the IP encoder 111 as user data, and a bit stream. Output as. The IP encoder 111 receives information for protecting the copyright IP (Intellectual Property) of the image from the outside, and outputs an IP encoded code. In the first embodiment, this IP is a password.
[0021]
Next, the configuration between layers in FIG. 1 will be described.
[0022]
Reference numeral 300 denotes a downsampler, which downsamples an input image. In the first embodiment, the downsampling rate in the downsampler 300 is set to “1/2”. An upsampling unit 301 upsamples a locally decoded image in a frame memory 205 described later. In the first embodiment, the upsampling rate in the upsampler 301 is set to “2”. A multiplexer 302 multiplexes the bit streams of the enhancement layer 6001 and the base layer 6000 in spatial scalability.
[0023]
Next, the configuration of the base layer 6000 in FIG. 1 will be described.
[0024]
In the basic layer 6000, each functional block with the same name is the same as the above-described enhancement layer 6001 except that the input is the output of the downsampler 300. Reference numeral 200 denotes a frame memory which stores one frame of an input image and outputs it as a macroblock which is a coding unit. A DCT unit 201 performs two-dimensional discrete cosine transform in units of 8 × 8 pixels (blocks). Reference numeral 202 denotes a quantizer that performs quantization for each block and outputs a quantized representative value. Reference numeral 203 denotes an inverse quantizer that outputs a quantized representative value as a DCT coefficient. Reference numeral 204 denotes an inverse DCT device that performs inverse DCT on the DCT coefficient to convert it into image data. A frame memory 205 stores a locally decoded image. A motion compensator 206 receives an input image from the frame memory 200 and a locally decoded image from the frame memory 205, detects a motion vector for each macroblock, and outputs a predicted image. A variable length encoder 207 performs Huffman coding on the quantized representative value output from the quantizer 202 and outputs a Huffman code. A multiplexer 208 multiplexes the Huffman code from the variable length encoder 207 and outputs it as a bit stream.
[0025]
First, the operation of the upper extension layer 6001 in FIG. 1 will be described.
[0026]
Hereinafter, in the first embodiment, intra-frame coding is performed in an I-VOP (Video Object Plane) coding mode, inter-frame prediction coding that is predicted from one predicted image is performed in a P-VOP coding mode, and two predicted images. The inter-frame prediction coding predicted from the B-VOP coding mode is used.
[0027]
The frame memory 100 converts the input image into a macro block which is a coding unit and outputs it. Prediction image data from the motion compensator 106 is subtracted from the image data output from the frame memory 100 by a subtracter and input to the DCT unit 101 as prediction error image data. The DCT unit 101 converts the input macroblock prediction error into DCT coefficients for each block. The quantizer 102 outputs the DCT coefficient as a desired quantization representative value for each block. The quantized representative value is decoded as prediction error image data via the inverse quantizer 103 and the inverse DCT unit 104. The prediction error image data is added to the prediction image data from the motion compensator 106 by an adder, and then stored in the frame memory 105 as locally decoded image data. Note that the motion compensator 106 performs prediction according to the encoding mode 1 for each frame designated from the outside and outputs predicted image data.
[0028]
The variable length encoder 107 that receives the quantized representative value encodes the quantized representative value by Huffman coding and outputs a Huffman code. The selector 109 directly inputs the Huffman code to one terminal (a), and inputs the Huffman code whose sign bit is inverted (scrambled) by the DCT code inverter 108 to the other terminal (b). . This selector 109 selects the terminal (a), that is, the output of the variable length encoder 107 when the scramble ON / OFF flag is OFF in accordance with the scramble ON / OFF flag input from the outside, and selects the scramble ON / OFF When the OFF flag is on, the terminal (b), that is, the Huffman code with the sign bit inverted is selected and output. The multiplexer 110 multiplexes and outputs the output of the selector 109, the scramble ON / OFF flag, and the IP code data output from the IP encoder 111.
[0029]
Next, the operation of the base layer 6000 shown in FIG. 1 will be described.
[0030]
The image downsampled by the downsampler 300 is input to the frame memory 200 and stored therein. The DCT unit 201, the quantizer 202, the inverse quantizer 203, the inverse DCT unit 204, the frame memory 205, the motion compensator 206 that inputs the frame-by-frame encoding mode 2, and the variable length encoder 207 are described above. It operates in the same manner as the corresponding part in the enhancement layer 6001. The multiplexer 208 multiplexes the output of the variable length encoder 207.
[0031]
Next, the operation between the base layer 6000 and the enhancement layer 6001 will be described including the upsampler 301.
[0032]
FIG. 2 is a diagram for explaining the relationship of each VOP between the base layer and the enhancement layer based on the spatial scalability according to the first embodiment.
[0033]
In the first frame of the input image, first, the downsampler 300 downsamples the input image data, performs I-VOP (intraframe) encoding in the base layer, and stores the locally decoded image in the frame memory 205.
[0034]
The enhancement layer 6001 up-samples the image in the frame memory 205 with the up-sampler 301, and then inputs the output of the up-sampler 301 as a reference image to the motion compensator 106 to generate a P-VOP (from one predicted image). Predict interframe prediction) encoding.
[0035]
Next, the second second frame is P-VOP encoded by the base layer 6000 with reference to the locally decoded image stored in the frame memory 205 when the first frame is encoded. On the other hand, in enhancement layer 6001, the locally decoded image stored in frame memory 105 at the time of encoding the first frame and the data obtained by up-sampling image data in frame memory 205 by up-sampler 301 are input to motion compensator 106. Then, B-VOP (interframe prediction predicted from two prediction images) is encoded.
[0036]
The third frame is performed in the same manner as the second frame, and thereafter, the operation for these three frames is repeated. In FIG. 2, “I” indicates I-VOP (intraframe) encoding, “P” indicates P-VOP encoding, and “B” indicates B-VOP encoding.
[0037]
FIG. 3 is a block diagram showing a configuration of the image decoding apparatus according to the first embodiment. First, the configuration of the enhancement layer 7001 in FIG. 3 will be described.
[0038]
Reference numeral 1000 denotes a distributor, which distributes the Huffman code, the encoding mode 1, the scramble ON / OFF flag, and the IP encoded code from the bit stream input to the enhancement layer 7001. An IP decoder 1010 decodes the IP encoded code from the distributor 1000 into IP. Reference numeral 1011 denotes an IP authenticator, which performs authentication by checking the match between the IP decrypted by the IP decoder and the authentication IP input from the outside. A security controller 1012 controls a selector 1002 and a selector 1009 described later based on a scramble ON / OFF flag from the distributor 1000 and an authentication result from the IP authenticator 1011.
[0039]
Reference numeral 1001 denotes a DCT code inverter that reverses the sign of the DCT coefficient of the Huffman code. The selector 1002 selects and outputs either the output of the distributor 1000 (a input) or the output of the DCT code inverter 1001 (b input) according to the selection signal from the security controller 1012. Reference numerals 1003 and 1006 denote variable length decoders that convert the Huffman codes into quantized representative values. Reference numerals 1004 and 1007 denote inverse quantizers that output quantized representative values as DCT coefficients. Reference numerals 1005 and 1008 denote inverse DCT units which convert DCT coefficients into images. Reference numeral 1014 denotes a frame memory for storing locally decoded images. A motion compensator 1013 refers to an output from the frame memory 1014 and a locally decoded image from an upsampler 1301 described later, and performs motion compensation for each macroblock to output a predicted image. The selector 1009 selects and outputs either the output (a input) from the inverse DCT device 1005 or the output (b input) from the inverse DCT device 1008 by the security controller 1012.
[0040]
Next, the configuration between the base layer and the enhancement layer in FIG. 3 will be described.
[0041]
Reference numeral 1300 denotes a distributor that distributes an input bit stream to a base layer and an enhancement layer. Reference numeral 1301 denotes an upsampling device that inputs a locally decoded image from a frame memory 1105 described later and upsamples it. The selector 1302 selects either the input from the enhancement layer 7001 (a input) or the input from the base layer 7000 (b input) based on the selection signal from the security controller 1012.
[0042]
Next, the configuration of the base layer 7000 in FIG. 3 will be described.
[0043]
Reference numeral 1100 denotes a distributor which inputs a base layer bit stream and distributes the Huffman code to the encoding mode 2 and outputs the Huffman code to the variable length decoder 1101 and the encoding mode 2 to the motion compensator 1104. Yes. The variable length decoder 1101 converts the Huffman code into a quantized representative value. Reference numeral 1102 denotes an inverse quantizer that outputs a quantized representative value subjected to variable length decoding as a DCT coefficient. Reference numeral 1103 denotes an inverse DCT device which converts DCT coefficients into original image data. A frame memory 1105 stores locally decoded image data. Reference numeral 1104 denotes a motion compensator, which receives locally decoded image data from the frame memory 1105, performs motion compensation for each macroblock, and outputs a predicted image.
[0044]
An operation based on the above configuration will be described.
[0045]
The encoded input bit stream is distributed by the distributor 1300 to the enhancement layer and the base layer. In the base layer 7000, the distributor 1100 distributes the Huffman code and encoding mode 2, and the Huffman code is decoded into image data via the variable length encoder 1101, the inverse quantizer 1102, and the inverse DCT 1103. In the case of I-VOP (intraframe) encoding, the locally decoded image data is directly stored in the frame memory 1105 and is supplied to the b input of the selector 1302 together with this. In the case of P-VOP (interframe prediction) encoding, the predicted image data output from the motion compensator 1104 is added to the output of the inverse DCT unit 1103, and then stored in the frame memory 1105. Supply to b input.
[0046]
On the other hand, in the enhancement layer 7001, the distributor 1000 distributes the Huffman code, the scramble ON / OFF flag, the IP, and the encoding mode 1. The DCT code inverter 1001 outputs the Huffman code with the sign inverted.
[0047]
Hereinafter, a normal Huffman code is referred to as a normal code, and a Huffman code whose sign is inverted is referred to as an inverted code. When a DCT [i] (i = 0 to 63) is a continuous column in the DCT coefficient block in the zigzag scan order, the Huffman code inverted in the first embodiment corresponds to the encoder. i = 3 to 63.
[0048]
The relationship between the VOPs of the base layer and the enhancement layer due to spatial scalability in the first embodiment is the same as that in the encoder in FIG. 1 described above.
[0049]
(A) First, the case where the scramble is on and the IP authentication result is OK will be described.
The selector 1002 inputs the Huffman code whose sign is inverted to the a input, and the normal code whose sign has been restored by the sign inverter 1001 to the b input. In order to scramble on, the selector 1002 selects a normal code of b input. The variable length decoder 1006, the inverse quantizer 1007, and the inverse DCT unit 1008 receive the normal code and output normal prediction error image data as a result.
[0050]
Next, the motion compensator 1013 outputs a prediction image from the output of the frame memory 1014 and the output of the upsampler 1301 in accordance with the encoding mode 1. The prediction error image from the inverse DCT unit 1008 and the prediction image data from the motion compensator 1013 are added to supply normal image data to the b input of the selector 1009. At the same time, the normal image data is supplied to the frame memory. 1014. The selector 1009 selects and outputs the b input according to the selection signal from the security controller 1012, and the selector 1302 selects the a input that is the output of the enhancement layer 7001. As a result, the image decoding apparatus according to the first embodiment can reproduce a high spatial resolution image.
[0051]
(B) A case where the scramble is ON and the IP authentication result is NO will be described.
In this case, because of scrambling on, the selector 1002 selects a normal code in which the code of b is restored. The variable length encoder 1003, the inverse quantizer 1004, and the inverse DCT device 1005 receive the code-inverted code and output the decoded image.
[0052]
Next, the motion compensator 1013 outputs a prediction image from the output of the frame memory 1014 and the output of the upsampler 1301 in accordance with the encoding mode 1. The predicted image from the inverse DCT device 1005 is added to the predicted image data from the motion compensator 1013 and input to a of the selector 1009. The selector 1009 selects and outputs the a input based on the control data from the security controller 1012. The selector 1302 selects the enhancement layer a based on the control data from the security controller 1012. As a result, the decoding apparatus according to the first embodiment reproduces an image having distortion due to scrambling.
[0053]
(C) Describes the case where the scramble is off and the IP authentication result is OK.
For scrambling off, the selector 1002 selects the a input and outputs a normal code. The variable length decoder 1006, the inverse quantizer 1007, and the inverse DCT device 1008 input this normal code and output a normal image. At this time, since the input bit stream is not scrambled, the same normal image data is input to both the a and b inputs of the selector 1009. Next, the selector 1009 selects and outputs the a input or the b input. The selector 1302 selects the a input that is the output of the enhancement layer 7001. As a result, a high spatial resolution image can be reproduced.
[0054]
(D) A case where the scramble is off and the IP authentication result is NO will be described.
Since the authentication result is NO in the security controller 1012, the selector 1302 selects and outputs the b input that is the output of the base layer 7000. As a result, the image decoding apparatus according to the first embodiment decodes and reproduces only the intra-frame encoded image and the P-VOP (inter-frame prediction) encoded image of the base layer. A resolution image is reproduced.
[0055]
FIG. 4 is a diagram showing the relationship between the selector 1302, the selector 1009, the selection state of the selector 1002, and the playback image by the security controller 1012 according to the first embodiment.
[0056]
Here, the three selectors 1002, 1009, and 1302 are controlled based on the scramble ON / OFF flag and the IP authentication result, and as a result, three types of reproduced image states (high resolution, low resolution, and distortion) are generated. Is possible.
[0057]
[Multi-layer spatial scalability configuration]
FIG. 5 is a diagram showing a multi-layered configuration of the encoding and decoding apparatus having the spatial scalability function in FIGS. 1 and 3 described above. Here, the number of layers is arbitrary, and is “n” in the figure.
[0058]
In the figure, 6000 corresponds to the base layer portion of FIG. 1, and 6001 corresponds to the enhancement layer portion of FIG. Reference numeral 7000 corresponds to the base layer portion in FIG. 2, and 7001 corresponds to the enhancement layer portion in FIG. Reference numerals 8000 and 8001 denote downsamplers, and 8002 and 8003 denote corresponding upsamplers. These can set the sampling rate according to the number of layers. However, each layer must correspond to the sampling rate. For example, the downsampling rate is “1/2” and the upsampling rate is “2”. Reference numeral 8004 denotes a multiplexer that multiplexes the bit streams from the multi-layer part into one. Reference numeral 8005 denotes a separator that separates one bit stream for each layer. Reference numeral 8006 denotes a selector that selects a layer to be reproduced according to the authentication result of each enhancement layer.
[0059]
[Multi-layer spatial scalability operation]
In the encoding apparatus of FIG. 5, copyright information and scramble ON / OFF are designated for each enhancement layer. Further, the decoding device decodes and reproduces an image having a resolution according to the scramble ON / OFF and the copyright authentication result. However, when a certain enhancement layer is scrambled, the higher layer must be scrambled.
[0060]
Each function of the first embodiment may be realized by software.
[0061]
FIG. 6 is a flowchart showing an image encoding process according to the first embodiment, and FIG. 7 is a flowchart showing the decoding process.
[0062]
First, the encoding process of FIG. 6 will be described. This process is started by inputting one frame of image data. First, in step S1, both the counter fr for counting the number of frames and the value of the counter n are set to “0”. These counters do not have to have the signals of the encoding modes 1 and 2 in FIG. In step S2, the frame counter fr is incremented by one. In step S3, it is checked whether the value of the frame counter fr is “3n + 1”, that is, whether the frame numbers in FIG. 2 are “1”, “4”, “7”... “3n + 1”. In the case of the layer, n is incremented by 1 in step S4, and then the process proceeds to step S5. In step S5, intra-frame coding (I-VOP) is executed. In step S6, inter-frame predictive coding (P-VOP) is performed in which prediction is performed from one predicted image based on the code encoded in step S5, and the process proceeds to step S10 described later. On the other hand, the frame image data encoded in step S5 is multiplexed in step S7 and output as an output bit stream. Then, the process returns to step S2.
[0063]
On the other hand, if the value of the frame counter fr is not “3n + 1” and it is the base layer in step S3, that is, if it is a frame that is interframe predictively encoded in the base layer, the process proceeds to step S8. In step S8, inter-frame predictive coding (P-VOP) predicted from one predicted image is executed, and the process proceeds to step S7. On the other hand, in the case of a frame subjected to interframe prediction encoding in the enhancement layer, interframe prediction encoding (B-VOP) for predicting from two prediction images is executed in step S9. Next, in step S10, it is checked whether or not the scramble ON / OFF flag is on. If it is on, the process proceeds to step S11 to invert the sign bit of the Huffman code. However, the Huffman code to be inverted in the DCT coefficient sequence (i = 0 to 63) is a DCT coefficient of i = 3 to 63 here. After executing step S11 or if scrambling is off in step S10, the process proceeds to step S12, where an IP code encoding IP, a scramble ON / OFF flag, an encoding mode (I-VOP, P-VOP, B-VOP) and the inter-frame prediction encoded code are multiplexed, and in step S7, multiplexed with the code encoded in the base layer and output. Then, the process returns to step S2.
[0064]
FIG. 7 is a flowchart showing an image decoding process according to the first embodiment.
[0065]
This process is started by inputting the code stream encoded by the encoding apparatus of FIG. 1 described above. First, in step S21, the input bit stream is distributed to the base layer and the enhancement layer. In step S22, the base layer code is subjected to variable-length decoding, inverse quantization, inverse DCT, and prediction code decoding processing by motion compensation.
[0066]
On the other hand, in the case of the enhancement layer, it is checked in step S23 whether or not the scramble ON / OFF flag is on. If it is on, the process proceeds to step S24 to check whether or not the IP authentication is OK. If OK, for example, since the user has permission to view such as copyright, the process proceeds to step S25, and the sign of the Huffman code is inverted to release the scramble. In step S26, variable length decoding, inverse quantization, inverse DCT, and P-VOP and B-VOP decoding processing by a motion compensator are executed. In step S27, the decoded and reproduced image data is output and displayed. In this case, a high resolution image can be displayed.
[0067]
If it is determined in step S24 that the IP authentication is not OK, the process proceeds to step S26, where the scrambled image is decoded and reproduced in step S27. In this case, an image having distortion due to scramble is reproduced.
[0068]
On the other hand, if the scramble flag is off in step S23, the process proceeds to step S29 to check whether the IP authentication is OK. If OK, the process proceeds to step S26, and the unscrambled image is decoded and reproduced.
[0069]
If it is determined in step S28 that the IP authentication is not OK, the process proceeds to step S22, in which decoding processing by I-VOP decoding or P-VOP decoding is executed, and image reproduction (reproduction at a low resolution) is performed in step S27. Thus, in step S28, the above-described processing is repeatedly executed until all the received image decoding processing is completed.
[0070]
In the first embodiment described above, the scramble target of the Huffman code is set to i = 3 to 63 as described above, but other ranges may be used. By setting the range in this way, the scramble distortion can be adjusted.
[0071]
In addition, the present embodiment is the case of the color signal configuration 420, but other color signal configurations may be used. Further, the frame mode configuration in FIG.
[0072]
In this embodiment, the downsampling is set to “1/2” and the upsampling rate is set to “2”. However, these values may be any values as long as they correspond.
[0073]
As described above, according to the first embodiment, in the image encoder and decoder having a spatial scalability function, by performing scrambling on the enhancement layer, distortion is performed on the decoder side as necessary. The copyright of the moving image can be protected.
[0074]
Further, by performing scrambling on the sign bit of the Huffman code instead of the entire bit stream, the processing load on the apparatus can be reduced.
[0075]
Further, since the Huffman code is inverted for each block, when the scrambled bit stream is directly decoded, the distortion is closed in the block. In the case of the first embodiment, an image in which quantization distortion called block distortion or mosquito distortion is generated more than a normal image is reproduced, and as a result, a viewer who cannot unscramble the overview of the image. It will remain to recognize.
[0076]
In addition, the moving image decoding apparatus is provided with two systems of variable length decoders, inverse quantizers, and inverse DCT units for the frame memory 1014 for storing locally decoded images. The variable length decoder 1006 is provided in the frame memory 1014. Since the normal decoded image of the path of the inverse quantizer 1007 and the inverse DCT unit 1008 is stored, distortion due to scrambling generated from the path of the other variable length decoder 1003, the inverse quantizer 1004, and the inverse DCT unit 1005 is caused. Accumulation for each frame can be prevented.
[0077]
It can also be realized with the number of layers of 3 or more.
[0078]
[Embodiment 2]
FIG. 8 is a block diagram showing the configuration of the video encoding apparatus according to Embodiment 2 of the present invention, and FIG. 11 is a block diagram showing the configuration of the video decoding apparatus corresponding to this. In the second embodiment, the MPEG-2 encoding method has a temporal scalability function and scrambles by inverting the sign bit of the Huffman code obtained by encoding the DCT coefficient for the enhancement layer. explain. Refer to the ISO / IEC recommendation for details of the MPEG-2 encoding method.
[0079]
FIG. 8 is a block diagram showing the configuration of the moving picture coding apparatus according to Embodiment 2 of the present invention. In FIG. 8, the same components as those in the first embodiment are denoted by the same reference numerals, and detailed description thereof is omitted. Therefore, in the second embodiment, differences from the first embodiment will be described in detail.
[0080]
A distributor 700 assigns an input image to the enhancement layer and the base layer for each frame, and at the same time performs ordering (frame rearrangement). A motion compensator 702 receives an input image from the frame memory 100 and local decoded image data from the frame memory 205 of the base layer, performs motion vector detection for each macroblock, and outputs a predicted image. A multiplexer 701 multiplexes the bit stream of the enhancement layer and the base layer in time scalability.
[0081]
An operation based on the above configuration will be described.
[0082]
In the second embodiment, intra-frame coding is performed in I-Picture coding mode, inter-frame prediction coding predicted from one predicted image is performed in P-Picture coding mode, and inter-frame prediction code is predicted from two predicted images. This is called B-Picture encoding mode.
[0083]
The input image is distributed by the distributor 700 to the base layer and the enhancement layer for each frame according to the encoding mode 1 input from the outside.
[0084]
FIG. 9 and FIG. 10 are diagrams illustrating the relationship between the frames of the base layer and the enhancement layer based on temporal scalability in the second embodiment. Here, FIG. 9 shows the order of the input images, and FIG. 10 is a diagram showing the configuration of the frames after the order of the frames is rearranged by the distributor 700. Here, the input image is assigned to each layer in the order shown in FIG.
[0085]
The first frame of the input image is first input to the base layer to perform I-Picture encoding, and the locally decoded image is stored in the frame memory 205. Next, the second frame is input to the base layer, and the local frame is stored in the frame memory 205. P-Picture encoding is performed with reference to the decoded image. Next, the third frame is input to the enhancement layer, and the input image is converted into a macroblock in the frame memory 100.
[0086]
The motion compensator 702 receives the output of the frame memory 100 and performs motion compensation from the reference images of the first and second frames already stored in the frame memory 205 of the base layer. That is, B-Picture encoding is performed. The next fourth frame is B-Picture encoded in the same manner as the third frame, and thereafter the operation is repeated according to the frame mode.
[0087]
Next, the video decoding device shown in FIG. 11 will be described. Among the constituent elements in FIG. 11, the same constituent elements as those in the first embodiment are given the same reference numerals, and detailed description thereof is omitted. Therefore, the second embodiment will describe points that are different from the first embodiment.
[0088]
Reference numeral 2000 denotes a distributor that distributes an input bit stream to a base layer and an enhancement layer. 2002 is a motion compensator, which receives locally decoded image data from the frame memory 1105 in the base layer and outputs a predicted image for each macroblock. Reference numeral 2001 denotes a selector that selects an enhancement layer output (a input) and a base layer output (b input) in accordance with the encoding mode 1 from the distributor 1000, and simultaneously performs reordering (frame rearrangement). To rearrange the images in the display order.
[0089]
In the above configuration, the input bit stream is distributed by the distributor 2000 to the enhancement layer and the base layer. The relationship of each frame between the base layer and the enhancement layer in the present embodiment is the same as in FIG.
[0090]
First, in the first to second frames, a bit stream is input to the base layer, and the first frame is decoded as I-Picture and the second frame is decoded as P-Picture. In the third frame, a bit stream is input to the enhancement layer, and the motion compensator 2002 refers to the two images of the first and second frames already stored in the frame memory 1105 of the base layer as B-Picture. Decrypt. In the fourth frame, a bit stream is input to the enhancement layer, and decoding similar to that in the third frame is performed. Thereafter, the operation is repeated according to the frame mode.
[0091]
The selector 2001 selects an enhancement layer (a input) and a base layer (b input) according to the encoding mode 1 from the distributor 1000, performs reordering, and outputs a decoded image.
[0092]
In FIG. 4 described above, if the selector 1302 is the selector 2001, the relationship between each selector and the reproduced image is the same as in FIG. However, the resolution of the reproduced image in this case is relative to the time frequency. That is, the low resolution image indicates an image of only the basic layer, and the high resolution image indicates an image including an enhancement layer.
[0093]
As described above, according to the second embodiment, in the image encoder and decoder having a temporal scalability function, by performing scrambling on the enhancement layer, the decoder side performs each frame as necessary. Distortion can be generated, and the copyright of the moving image can be protected.
[0094]
Further, by performing scrambling on the sign bit of the Huffman code instead of the entire bit stream, the processing load on the apparatus can be reduced.
[0095]
In addition, since scrambling is performed only on the enhancement layer, when both layers are reproduced continuously and an enhancement layer image with scramble distortion is reproduced, an image without distortion and distortion are present. Images are reproduced alternately. As a result, viewers who cannot unscramble only recognize the appearance of the image.
[0096]
In addition, similar to the spatial scalability, it can be realized with the number of layers of 3 or more.
[0097]
Note that the processing according to the second embodiment can also be processed by software as in the first embodiment. The flowchart showing the processing flow in this case is basically the same as that of the first embodiment, and the description thereof is omitted.
[0098]
[Embodiment 3]
FIG. 12 is a block diagram showing a configuration of an image encoding device according to Embodiment 3 of the present invention, and FIG. 13 is a block diagram showing a configuration of an image decoding device corresponding to this. In the third embodiment, the intra-frame coding method has a function similar to the spatial scalability of the MPEG coding method, and by inverting the sign bit of the Huffman code obtained by coding the DCT coefficient for the enhancement layer. A case where scramble is applied will be described.
[0099]
In FIG. 12, the same components as those in the first embodiment are denoted by the same reference numerals, and detailed description thereof is omitted. Therefore, the third embodiment will describe differences from the first embodiment.
[0100]
A separator 3000 separates the DCT low frequency component and high frequency component from the output of the variable length encoder 207 corresponding to the position in the block. Reference numeral 3001 denotes a multiplexer that multiplexes the codes of each layer separated by the separator 3000.
[0101]
When a DCT [i] (i = 0 to 63) is a continuous column in the DCT coefficient block in the zigzag scan order, the Huffman code output to the enhancement layer by the separator 3000 according to the third embodiment is i = 3 to 63.
[0102]
In FIG. 12, an input image is converted into a macroblock in the frame memory 200 and becomes a variable-length encoded code via a DCT unit 201, a quantizer 202, and a variable-length encoder 207. This encoded code is separated for each component in the block by the separator 3000, the low frequency component is output to the multiplexer 3001, and the other components are output to the DCT code inverter 108 and the selector 109.
[0103]
In the image decoding apparatus in FIG. 13, the same components as those in the first embodiment are given the same numbers, and detailed descriptions thereof are omitted. Therefore, the third embodiment will describe points that are different from the first embodiment.
[0104]
Reference numeral 4000 denotes a separator that separates an input bit stream into an enhancement layer and a base layer. Reference numeral 4001 denotes a multiplexer that multiplexes codes in both layers. A selector 4002 selects a black image (a input) or an output of the inverse DCT unit 1103 according to a selection signal from the security controller 1012.
[0105]
The operation of the decoding device based on the above configuration will be described.
[0106]
First, the input bit stream is separated into an enhancement layer and a base layer by a separator 4000. Hereinafter, a normal Huffman code is referred to as a normal code, and a Huffman code whose sign is inverted is referred to as an inverted code. Note that when DCT [i] (i = 0 to 63) is a continuous column in the block of DCT coefficients in the zigzag scan order, the Huffman code output to the enhancement layer by the separator 4000 of the third embodiment is i = 3 to 63.
[0107]
(A) A case where the scramble is on and the IP authentication result is OK will be described.
Since the authentication result is OK in the security controller 1012, the selector 1002 selects the normal code with b input. The multiplexer 4001 multiplexes the low frequency component code from the separator 4000 and the high frequency component code from the selector 1002. The variable length decoder 1101, the inverse quantizer 1102, and the inverse DCT device 1103 receive normal codes. As a result, a normal image can be decoded. Here, the selector 4002 selects the b input according to the selection signal from the security controller 1012 and outputs it. As a result, the present decoding device reproduces a normal image (high resolution).
[0108]
(B) A case where the scramble is ON and the IP authentication result is NO will be described.
In the security controller 1012, since the authentication result is NO, the selector 1002 selects the inverted code of the a input. For this reason, the high frequency component is output from the inverse DCT unit 1103 while being in the inverted code. The selector 4002 selects the b input according to the selection signal from the security controller 1012 and outputs it. As a result, an image having distortion due to scramble is reproduced.
[0109]
(C) Describes the case where the scramble is off and the IP authentication result is OK.
Since the authentication result is OK in the security controller 1012, the selector 1002 selects the a input and outputs a normal code. For this reason, all frequency components become normal and are output from the inverse DCT unit 1103. The selector 4002 selects the b input according to the selection signal from the security controller 1012 and outputs it. As a result, a normal image can be reproduced.
[0110]
(D) Describes the case where the scramble is off and the IP authentication result is NO.
Since the authentication result is NO in the security controller 1012, the selector 4002 selects and outputs a black image (a input). As a result, a black image is reproduced.
[0111]
FIG. 14 is a diagram illustrating the relationship between the selection state of the selector 4002 and the selector 1002 by the security controller 1012 and the playback image according to the third embodiment.
[0112]
As described above, it is possible to control the two selectors 1002 and 4002 based on the scramble ON / OFF flag and the IP authentication result, and as a result, three types of reproduced image states can be generated.
[0113]
As described above, according to the third embodiment, since it is an intra-frame coding method, it can be realized by a still image and moving image coding / decoding device.
[0114]
In an image encoder and decoder having a spatial scalability function, by performing scrambling on the enhancement layer, distortion can be generated on the decoder side as needed, and the copyright of the image can be protected.
[0115]
Further, by performing scrambling on the sign bit of the Huffman code instead of the entire bit stream, the processing load on the apparatus can be reduced.
[0116]
Further, since the Huffman code is inverted for each block, when the scrambled bit stream is directly decoded, the distortion is closed in the block. In the case of this embodiment, an image in which quantization distortion called block distortion or mosquito distortion is generated more than a normal image is reproduced, and as a result, a viewer who cannot unscramble recognizes the appearance of the image. Will stay.
[0117]
Note that the present invention can be applied to a system (for example, a copier, a facsimile machine, etc.) consisting of a single device even if it is applied to a system composed of a plurality of devices (eg, a host computer, interface device, reader, printer, etc.). You may apply.
[0118]
Another object of the present invention is to supply a storage medium (or recording medium) that records a program code of software that realizes the functions of the above-described embodiments to a system or apparatus, and to perform computer (or CPU or MPU) of the system or apparatus. ) Is also achieved by reading and executing the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an operating system (OS) running on the computer based on the instruction of the program code. A case where part or all of the actual processing is performed and the functions of the above-described embodiments are realized by the processing is also included.
[0119]
Furthermore, after the program code read from the storage medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer, the function is determined based on the instruction of the program code. This includes a case where the CPU or the like provided in the expansion card or the function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.
[0120]
As described above, according to the present embodiment, an IP encoding code and additional information for protecting copyright are provided in a bitstream in an image encoding device and decoding device having a scalability function. Thus, it is possible to protect the copyright of the content provider for a part of the desired image.
[0121]
Further, according to the present embodiment, since the limited code of a part of the bit stream is scrambled, the processing load is less than that for the entire encoded code. Further, since the distortion is closed in the block, it is possible to create a state in which a viewer who cannot unscramble recognizes the appearance of the image.
[0122]
【The invention's effect】
As described above, according to the present invention, it is possible to scramble a part of a target bit stream to be protected for copyright at the time of image encoding, and to encode an image without lowering the encoding efficiency.
[0123]
In addition, according to the present invention, normal reproduction is performed in the image decoding device of the viewer who has the viewing right, and reproduction is performed so that an approximate overview of the image can be recognized in the image decoding device of the viewer who does not have the viewing right. There is an effect that an image can be encoded so that
[0124]
According to the present invention, the image encoded by the image encoding device is decoded,Three types of images (high resolution, low resolution, and distortion) can be obtained by combining the authentication result of whether or not the user has the right to view and the presence or absence of scrambling.There is an effect that it can be reproduced.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a main configuration of an image coding apparatus according to Embodiment 1 of the present invention.
FIG. 2 is a diagram showing a relationship of each VOP between a base layer and an enhancement layer based on spatial scalability according to the first embodiment.
FIG. 3 is a block diagram showing a main configuration of an image decoding apparatus according to Embodiment 1 of the present invention.
FIG. 4 is a diagram for explaining the relationship between the input / output of the security controller and the reproduced image in the first embodiment of the present invention.
FIG. 5 is a diagram showing a multi-layer spatial scalability configuration according to Embodiment 1 of the present invention;
FIG. 6 is a flowchart illustrating an image encoding process according to Embodiment 1 of the present invention.
FIG. 7 is a flowchart showing an image decoding process according to the first embodiment of the present invention.
FIG. 8 is a block diagram showing the main configuration of an image coding apparatus according to Embodiment 2 of the present invention.
[Fig. 9] Fig. 9 is a diagram illustrating a relationship in frame display order between a base layer and an enhancement layer based on spatial scalability according to the second embodiment.
[Fig. 10] Fig. 10 is a diagram illustrating a relationship of frame encoding orders in a base layer and an enhancement layer based on spatial scalability according to the second embodiment.
FIG. 11 is a block diagram showing the main configuration of an image decoding apparatus according to Embodiment 2 of the present invention.
FIG. 12 is a block diagram showing a configuration of a main part of an image coding apparatus according to Embodiment 3 of the present invention.
FIG. 13 is a block diagram showing the main configuration of an image decoding apparatus according to the third embodiment.
FIG. 14 is a diagram illustrating a relationship between input / output of a security controller and a reproduced image according to the third embodiment.

Claims (6)

Input an encoded bitstream, and distribute the bitstream to a protection code for protecting intellectual property, one or more enhancement layers, and a base layer having a lower resolution than the enhancement layer Distribution means to
Authentication data input means for inputting authentication data from the outside,
An authentication means for checking consistency between the authentication data and the protection code;
And descrambling means for descrambling the one or more enhancement layers,
Enhancement layer decoding means for decoding the enhancement layer distributed by the distribution means or the enhancement layer descrambled by the descrambling means;
Base layer decoding means for decoding the base layer distributed by the distributing means;
If the authentication result by the authentication means is coincident and the enhancement layer is scrambled, select an image obtained by decoding the enhancement layer released by the descrambling means,
When the authentication result by the authentication means does not match and the enhancement layer is scrambled, or when the authentication result by the authentication means matches and the extension layer is not scrambled Selects an image obtained by decoding the enhancement layer,
If the authentication result by the authentication unit is inconsistent and the enhancement layer has not been scrambled, a selection unit that selects an image obtained by decoding the base layer;
Image decoding apparatus characterized by comprising an image output means for outputting the result selected image to the selection unit. The image decoding apparatus according to claim 1 , wherein the image output unit outputs an image frame assigned to a reproduction order for each layer. The image decoding apparatus according to claim 1 , wherein the descrambling means replaces the bitstream with another code having the same code length. The base layer decoding means or the enhancement layer decoding means
Variable-length decoding means for variable-length decoding the encoded code for each block;
Inverse quantization means for inversely quantizing the decoding result by the variable length decoding means;
The image decoding apparatus according to any one of claims 1 to 3 , further comprising: an inverse orthogonal transform unit that performs an inverse orthogonal transform on an inverse quantization result obtained by the inverse quantization unit. Input an encoded bitstream, and distribute the bitstream to a protection code for protecting intellectual property, one or more enhancement layers, and a base layer having a lower resolution than the enhancement layer A dispensing process to
An authentication data input process for inputting authentication data from the outside,
An authentication step for checking the consistency between the authentication data and the protection code;
A descrambling step for descrambling the one or more enhancement layers,
An enhancement layer decoding step of decoding the enhancement layer distributed by the distribution step or the enhancement layer descrambled by the descrambling step;
A base layer decoding step of decoding the base layer distributed by the distributing step;
If the authentication result by the authentication step is the same and the enhancement layer has been scrambled, select an image obtained by decoding the enhancement layer released by the descrambling step,
When the authentication result by the authentication step is inconsistent and the enhancement layer is scrambled, or when the authentication result by the authentication step is coincident and the extension layer is not scrambled Outputs an image obtained by decoding the enhancement layer,
If the authentication result of the authentication step is inconsistent and the enhancement layer is not scrambled, a selection step of selecting an image obtained by decoding the base layer;
An image decoding method comprising: an image output step of outputting the image selected in the selection step. A computer-readable storage medium storing a program for causing a computer to execute the method according to claim 5 .

Images