KR100483676B1 - Flexible shape information encoding apparatus and method - Google Patents

Abstract

According to the present invention, a shape information encoding method of a shape information encoding method (SHAPE INFORMATION CODING METHOD) which separates and processes a video applied through an image input apparatus into each object image having a shape information and a background image is processed. (SCALABLE SHAPE CODING), and particularly, when encoding a low resolution image (base layer image) to a high resolution image (enhancement layer image), encoding error ( By extracting the information on ENCOBING ERROR and correcting during decoding, it is possible to obtain a lossless high resolution image from the lossy coded low resolution image.

Description

Flexible shape information encoding apparatus and method

According to the present invention, a shape information encoding method of a shape information encoding method (SHAPE INFORMATION CODING METHOD) which separates and processes a video applied through an image input apparatus into each object image having a shape information and a background image is processed. (SCALABLE SHAPE CODING), and particularly, when encoding a low resolution image (base layer image) to a high resolution image (enhancement layer image), encoding error ( By extracting information about ENCODING ERROR and correcting during decoding, it is possible to obtain a lossless high resolution image from a lossy coded low resolution image.

As is well known, the recent image processing technology breaks away from a method of compressing and encoding an image corresponding to one frame (FRAME) as a whole, and divides it into predetermined unit blocks having arbitrary shape information and compresses and encodes each of them. In the direction of transmission.

In other words, the applied shape is separated into each object image and the background image, and only the change of the object image is transmitted to improve compression efficiency and encoding efficiency, and an international standard for this is prepared.

For example, WG11 under ISO / IEC, a world standardization organization, has arbitrary shape information unlike MPEG (MOVING PICTURE EXPERTS GROUP). MPEG-4, which is a standardization work on the method of encoding an object, is in progress, and MPEG-4, which is being standardized, is a unit block having arbitrary shape information. The concept of VOP (VIDEO OBJECT PLANE) is called. Is based on.

Here, the VOP is defined as a quadrangle that divides the applied image into a background image and each object image, and includes the separated background image and the object image. In MPEG-4, a predetermined object or a predetermined object is included in the image. If there is a region of the object consisting of the region of, the video of the object is divided into respective VOPs, and the separated VOPs are encoded respectively.

The VOP has the advantage of freely synthesizing or decomposing a natural image or an artificial image as a unit of an object image, and is fundamental to processing images of an object in the field of computer graphics and multimedia.

FIG. 1 shows the configuration of an encoder (ENCODER) 100 that is primarily determined by the International Standards Organization (ISO / IEC JTC1 / SC29 / WG11 MPEG96 / N1172 JANUARY). The block diagram shown.

Here, when the VOP formation unit 110 receives the image sequence (SEQUENCE) to be transmitted or stored, the VOP formation unit 110 divides it into a target image unit to form different VOPs.

2 shows an example in which one VOP is formed by setting an image of a "cat" as an object image.

Here, the horizontal size of the VOP is defined by the width of the VOP, the vertical size is defined by the height of the VOP, the formed VOP is M and N respectively in the X-axis and Y-axis with the upper left as the grid (GRID) starting point. It is partitioned into an M x N macroblock having two pixels. For example, it is divided into 16 x 16 macroblocks each having 16 pixels on the X and Y axes.

In this case, when there are not M and N pixels of the X and Y axes of the macroblock formed on the right and the bottom of the VOP, the size of the VOP is extended to M and both of the X and Y axes pixels of each macro block are expanded. And N.

In addition, M and N are set to an even number in order to perform encoding in units of sub-blocks in an object coding unit (TEXTURE CODING) to be described later.

On the other hand, each VOP formed in the VOP forming unit 110 is input to the VOP encoder 120a, 120b, ..., 120n, respectively, are encoded for each VOP, multiplexed by the multiplexer 130 into a bit string (BIT STREAM) Is sent.

FIG. 4 is a block diagram showing the configuration of the VOP encoders 120a, 120b, ..., 120n of the VM encoder 100 determined primarily by the international standard subdivision.

First, when a VOP for each object image formed by the VOP forming unit 110 is input to a motion estimation unit 121, the motion estimation unit 121 moves in units of macro blocks from an applied VOP. Will be estimated.

In addition, the motion information estimated by the motion estimation unit 121 is input to a motion compensation unit 122 to compensate for the motion.

In addition, the VOP whose motion is compensated by the motion compensator 122 is input to the subtractor 123 together with the VOP formed by the VOP forming unit 110 to detect a difference value, and the difference detected by the subtractor 123. The value is input to the object internal encoding unit 124 to encode the internal information of the object in units of sub blocks of the macro block.

For example, after the X and Y axes of the macroblock are subdivided into 8 × 8 subblocks having 8 pixels each with M / 2 × N / 2, the object internal information is encoded.

On the other hand, the VOP whose motion is compensated by the motion compensator 122 and the internal information of the object encoded by the object internal encoder 124 are added to the adder 125, and the output of the adder 125 is added. The signal is input to a PREVIOUS RECONSTRUCTED VOP 126 to detect a previous VOP which is a VOP of the image immediately before the current image.

In addition, the previous VOP detected by the previous VOP detector 126 is input to the motion estimation unit 121 and the motion compensation unit 122 and used for motion estimation and motion compensation.

The VOP formed by the VOP forming unit 110 is input to a SHAPE CODING BLOCK 127 to encode shape information.

Here, the output signal of the shape coding unit 127 is used or not depending on the field to which the VOP coders 120a, 120b, ..., 120n are applied. As shown by a dotted line, the shape coding unit ( The output signal of 127 may be input to the motion estimation unit 121, the motion compensator 122, and the object internal encoding unit 124, and used to encode motion estimation, motion compensation, and internal information of the object.

In addition, motion information estimated by the motion estimation unit 121, object internal information encoded by the object internal encoding unit 124, and shape information encoded by the shape encoder 127 are applied to the multiplexer 128. After being multiplexed and multiplexed, it is output to the multiplexer 130 of FIG. 1 through the buffer 129 and transmitted in a bit string.

3 is a block diagram showing the configuration of a VM decoder (DECODER) 200 determined primarily by the International Standards Organization.

The encoded signal of the VOP, which is encoded through the VM encoder 100 and transmitted for each bit, is separated for each VOP in the demultiplexer 210 of the VM decoder 200.

In addition, the separated VOP coded signals are respectively decoded by the VOP decoders 220a, 220b, ..., 220n, and the decoded signals output from the VOP decoders 220a, 220b, ..., 220n are synthesized by the synthesizer 230. ) Is synthesized and output as the original image.

In the MPEG-4, a technique applied to the shape encoding unit 127 for encoding each VOP transmitted from the VOP forming unit 110 is N × N blocks (N = 16, 8, 4). MMR SHAPE CODING TECHNIQUE, which encodes shape information based on the VERIFEX-BASED SHAPE CODING TECHNIQUE, which encodes shape information based on the vertices, and the baseline. BASELINE-BASED SHAPE CODING TECHNIQUE and CONTEXT-BASED ARITHMETIC CODING.

On the other hand, the stretched shape information encoding technology converts an image having a high amount of information (still image or a video) into a low resolution image and transmits it, and then converts it to an original image having a high resolution. It is a technology to reduce.

That is, when the encoder converts a high resolution image into a low resolution image having a small amount of information, encodes the image, and then transmits the encoded image to the decoder, the decoder decodes the low resolution image transmitted by the decoder to restore the high resolution image. The encoding process and the decoding process will be described as follows.

In the present description, for convenience of description, as shown in FIGS. 5 and 6, an image of a 4 × 4 enhancement layer is converted into an image of a base layer, encoded, transmitted, and then, again, is scanned by a scan interleaving method. The process of encoding the image of the health layer will be described in detail as follows.

In the encoder, the image of the 4x4 enhancement layer as shown in Fig. 5A is divided into four sections, and the pixels A5, A7, A13, and A15 at the lower right side are selected. Create an image of the base layer with a lossy encoding or lossless encoding and transmit the image to the decoder.

When the image of the base layer is extracted, the sample of the base layer is compared with the image of the base layer (reduced image) and the image of the enhancement layer (original image). And EXCEPTIONAL SAMPLE DATA (hereinafter referred to as ESD) are detected. In this case, when a TSD or ESD is detected, the context-based arithmetic encoding (CONTEXT-BASED ARITHMETIC ENCODING: hereinafter referred to as CAE) is transmitted to the decoder along with the image of the base layer.

Here, the reason for detecting and encoding the TSD and ESD is as follows.

FIG. 7 illustrates a horizontal search in a process of comparing an image of a base layer with an enhancement image, and a reference scan line (hereinafter referred to as RSL) in the drawing is a base layer image. A pixel column including horizontal pixels, and a code scan line (hereinafter referred to as CSL) represents a horizontal pixel column of an enhancement layer image that is excluded from the base layer image extraction process. In addition, in order to encode the CSL, an RSL existing under and above the pixel to be encoded is used. As shown in FIG. 6, it can be seen that there are certain rules.

That is, when two neighboring pixels on the RSL existing under and above the pixel to be encoded on the CSL have the same value, the pixel values of the current position to be encoded may have the same value. If the pixel value at the current position is also the same as the two neighboring pixel values, no encoding is performed.

However, when two neighboring pixel values are different, the pixel values of the current position are likely to be different. Therefore, the pixel values of the current position to be encoded should be encoded. In this case, the transitional sample (TRANSITIONAL SAMPLE) ( Ellipses in solid lines).

In addition, when two neighboring pixel values are the same but the pixel values of the current position are different, encoding should be performed. This case is called an EXCEPTIONAL SAMPLE (ellipse indicated by a dotted line in FIG. 6).

Accordingly, in order to encode the enhancement layer, two types of data, that is, TSD and ESD, may exist. When the TSD and ESD exist, information about the TSD and ESD must be transmitted to the decoder to display the image of the enhancement layer. It can be restored accurately. CAE is a technique used to reduce the number of bits required when encoding the information on the TSD and ESD.

On the other hand, the decoder decodes the coded signal for the base layer image transmitted from the encoder and the coded signal for TSD or ESD, thereby reconstructing the image of the enhancement layer, which is a horizontal search and a vertical search (VERTICAL). SCANNING) divided in detail as follows.

FIG. 6A is a view for explaining a horizontal direction search method. When the pixel at the current point "X (A10)" is encoded, the pixels adjacent to the point "X" are shown as "A5" in the drawing. , A6, A7, A13, A14, A15, and B0 (A9) ". In this case, the "A5, A6, A7, A13, A14, A15, and B0" pixels are obtained by the scan interleaving method of the previous step. The pixel at the "X" point to be obtained is the "A5, A6, The values of A7, A13, A14, A15, and B0 "are obtained by performing CAE. That is, CAE is encoded by using a CONTEXT of seven pixels in the horizontal and vertical directions.

Similarly, in the vertical search method, as shown in Fig. 6B, the pixel value of the "Y" point to be obtained is obtained from the pixel values of "C0, C1, C2, A5, A7, A13, A15".

Here, the pixels "B0, C0, C2" represent the values obtained by the horizontal search method, the "C1" pixels represent the values obtained by the vertical direction search method, and the hatched pixels indicate the next pixel to be obtained. It is shown.

Therefore, when the horizontal direction search is performed, the pixels in the vertical direction are doubled, and when the vertical search is performed, the pixels in the horizontal direction are doubled, resulting in a 4 × 4 enhancement layer image from the 2 × 2 base layer image. Will be restored.

In addition, in the case of a video, MPEG-4 supports elastic shape information encoding. In the case of MPEG-4, as shown in FIG. 8, the I-VOP (INTRA VIDEO) of the base layer video is displayed due to its video transmission characteristics. Reconstruction of the enhancement layer image using OBJECT PLANE (hereinafter referred to as I-VOP), and P-VOP (PREDICTED VIDEO OBJECT PLANE: referred to as P-VOP). There is a method of restoring an image and a method of restoring an image of an enhancement layer using a B-VOP (BY-DIRECTION VIDEO OBJECT PLANE: P-VOP).

Here, when the base layer image is encoded into the enhancement layer image by using the I-VOP of the base layer, the scan interleaving method as described above is used.

However, in the conventional stretched shape information encoding technique as described above, when the image of the enhancement layer is encoded using the image of the base layer, the image of the base layer is referred to as a reference layer (ie, RSL). In order to use a lossless code with the image of the enhancement layer, since the image of the base layer must also be lossless coded, there is a problem that the amount of information is increased and the coding efficiency is lowered.

That is, in case of loss coding, since the exact location of TSD and ESD is not known in the scan interleaving process, an error occurs in the enhancement layer, so that the image of the lossless enhancement layer cannot be restored. there was.

In other words, when encoding the image of the base layer, when a loss coding method that allows a certain percentage (%) loss is used to increase the information amount compression efficiency, the loss of the base layer is restored from the signal transmitted by the loss coding. The image is an image having the same error rate as that of the original image. When the image of the enhancement layer is reconstructed using the lost base layer image, the image of the enhancement layer is naturally lost. You will have no choice but to get it.

Therefore, in order to reconstruct a lossless enhancement layer image, a lossless base layer image must be reconstructed. To this end, when encoding the base layer image, lossless encoding must be performed to increase the amount of information.

SUMMARY OF THE INVENTION An object of the present invention is to solve the above-mentioned conventional problems. In particular, when encoding a low resolution image (base rare image) into a high resolution image (enhancement rare image) The present invention provides a " extended shape information encoding apparatus and method " capable of obtaining a lossless high resolution image from a lossy coded low resolution image by extracting and correcting the information.

In order to achieve the above object, the present invention "contractable shape information encoding apparatus" converts a high resolution image into a low resolution image, encodes it through an encoder, and transmits the same to a decoder. In the stretchable shape information encoding apparatus for restoring an image of

When encoding a low-resolution video and transmitting it to a decoder, information about encoding errors generated during encoding is detected and transmitted together.In decoding, information about encoding errors transmitted from an encoder is detected to correct errors generated during encoding. Therefore, the technical configuration features that it is possible to restore a lossless high resolution image from a lossy coded low resolution image.

In addition, the present invention "contractable shape information coding method", when converting a high-resolution video to a low-resolution video, encoded through an encoder and then transmitted to the decoder, the decoder reconstructs the high-resolution video from the transmitted low-resolution video In the type shape information encoding method,

When encoding a low-resolution video and transmitting it to a decoder, information about encoding errors generated during encoding is detected and transmitted together.In decoding, information about encoding errors transmitted from an encoder is detected to correct errors generated during encoding. Therefore, a feature of the method configuration is that it is possible to recover a lossless high resolution image from a lossy coded low resolution image.

According to the present invention, the apparatus and method for elastic shape information encoding may change a high resolution image into a low resolution image and then perform loss loss encoding, and if the error generated during loss encoding is detected and transmitted to the decoder, the decoder may detect the error. By reconstructing the low resolution video and the high resolution video by analyzing the information, a lossless high resolution video (enhancement layer video) can be obtained from the loss coded low resolution video (base layer video).

Hereinafter, with reference to the accompanying drawings, the configuration, operation and effect of the embodiment according to the technical idea of the present invention "stretch shape information coding apparatus and method configured as described above in detail as follows.

Example 1

In the first embodiment, when the lossy encoding of the image of the base layer, information corresponding to the error is transmitted to the decoder, and the decoder losslessly decodes the image of the base layer using the error information transmitted from the encoder. A method of decoding an image of an enhancement layer.

That is, as shown in FIG. 9, the lossy encoding of the original image of the base layer is performed, and error information is extracted using the lossy encoding result of the original image of the base layer.

The TSD and the ESD are encoded by using the original image of the base layer and the image of the enhancement layer.

In addition, the encoder side transmits the loss coded information, the error extraction information of the base layer, and the TSD and ESD encoding information of the enhancement layer.

On the decoder side, the right-side image of the base layer is reconstructed using the lossy coded information and the error information, and the image of the enhancement layer that is lost by the conventional scan interleaving method is decoded using the reconstructed image, TSD, and ESD information. .

Example 2

In Embodiment 2, as in Embodiment 1, the encoder does not use the original image as the RSL for loss coding the base layer image and encoding the TSD and ESD information for the enhancement layer image. As shown in Fig. 10, an image reconstructed from the lossy coded base layer image is used as RSL. This allows lossless coding on the part inserted by the scan interleaving method, but the lossless coding is not possible because the error is included in the RSL.

Therefore, in the second embodiment, in order to recover the RSL having an error without error, as shown in FIG. 10, the error information is extracted through the error information extraction and encoding unit 31 to extract error information about the RSL.

Error extraction is encoded in the same way that the TSD and ESD are encoded in the conventional scan interleaving method.

Here, the difference from the existing method is not to use the image of the base layer as the RSL, but to use the portion inserted by the lossless coding as the new RSL.

Based on this, the error of the previously encoded lossy RSL portion is encoded and transmitted.

In addition, the decoder loss-decodes an image of the enhancement layer by using the encoded information of the base layer image and the enhancement layer image encoded by the lossy coding.

The encoder performs error correction by using the transmitted error information and the lost decoded information.

The error correction used here is the same as the conventional scan interleaving method. However, the difference is that the portion inserted by lossless decoding is used as the RSL, not the image of the base layer as the RSL.

Thus, lossless decoding is possible by eliminating errors generated in the error correction step.

As described above, the present invention provides an apparatus and method for elastic shape information encoding. In particular, when encoding a low resolution image into a high resolution image, information about encoding errors is extracted and corrected during decoding. It is possible to obtain a lossless high resolution image from a lossy coded low resolution image.

1 is a block diagram showing the configuration of a VM encoder primarily determined by the International Standards Organization.

2 is a diagram illustrating a VOP having shape information divided into macro blocks;

3 is a block diagram showing the configuration of a VM decoder primarily determined by the International Standards Organization;

4 is a block diagram showing a configuration of a VOP encoder determined primarily by an international standard computing instrument;

5 to 7 are diagrams for explaining a scan interleaving method in flexible encoding;

8 is a diagram for explaining spatial stretching coding;

9 is a block diagram showing an embodiment of an "extended shape information encoding apparatus" of the present invention;

9 is a block diagram showing another embodiment of the "extended shape information encoding apparatus" of the present invention.

<Explanation of symbols for main parts of drawing>

11,32: TSD / ESD encoder 12,31: Error information extraction and encoder

13,33: Loss Code Encoding 14,34: Loss Code Encoding

21,43: High resolution image decoding unit 22,41: Error information decoding unit

23,42: error correction 24,44: loss and decryption department

Claims (17)

In the flexible shape information encoding apparatus that converts a high resolution image into a low resolution image, encodes it through an encoder, and then transmits the same to a decoder. The decoder reconstructs a high resolution image from the transmitted low resolution image. When a low-resolution video is encoded by a lossy coding method and transmitted to a decoder, information about an error generated during encoding by the lossy coding method is detected and transmitted together.In decoding, the error transmitted from the encoder is detected and corrected. And reconstructing a lossless high resolution image from the lossy coded low resolution image. The apparatus of claim 1, wherein the apparatus for detecting and transmitting information on an error generated during encoding by the lossy coding method is provided. A lossy encoding unit for lossy encoding the low resolution image and transmitting the lossy image to the decoder; A lossy decoding unit which decodes the low resolution image loss-coded by the lossy coding unit; An error information extraction and encoding unit which compares an original low resolution image that is not loss coded with a low resolution image that is loss coded and then decoded through the loss encoding unit and the loss decoding unit, and extracts and encodes an error during loss encoding; ; And a TSD / ESD encoding unit for detecting and encoding TSD and ESD by comparing an original low resolution image that is not loss coded with a high resolution image. The apparatus of claim 1, wherein the apparatus for reconstructing a lossless high resolution image from the loss coded low resolution image by detecting and correcting the error information during loss encoding transmitted from the encoder is performed. A lossy decoding unit for decoding the low resolution image transmitted by loss coding to restore the lost low resolution image; An error information decoding unit for decoding the coded signal with respect to the error information generated during the lossy coding to extract the error information; An error correction unit for restoring a lossless low resolution image by correcting the loss coded low resolution image applied by the loss decoding unit according to the error information applied by the error information decoding unit; Stretchable shape information encoding comprising a high resolution image decoding unit for restoring a lossless high resolution image by scanning interleaving the lossless low resolution image applied by the error correction unit based on the transmitted TSD and ESD information Device. The apparatus of claim 1, wherein the apparatus for detecting and transmitting information on an error generated during encoding by the lossy coding method is provided. A lossy encoding unit for lossy encoding the low resolution image and transmitting the lossy image to the decoder; A lossy decoding unit which decodes the low resolution image loss-coded by the lossy coding unit; A TSD / ESD encoding unit for detecting and encoding TSD and ESD by comparing a high resolution image with a low resolution image that has been loss coded and decoded through the lossy coding unit and the lossy decoding unit; An elastic shape comprising an error information extraction and encoding unit for detecting and encoding an error between an original low resolution image that is not loss coded and a low resolution image that is loss coded and decoded and used as an RSL of a high resolution image. Information encoding apparatus. The apparatus of claim 1, wherein the apparatus for reconstructing a lossless high resolution image from the loss coded low resolution image by detecting and correcting the error information during loss encoding transmitted from the encoder is performed. A lossy decoding unit for decoding the low resolution image transmitted by loss coding to restore the lost low resolution image; A high resolution image decoding unit for reconstructing a high resolution image lost by scan interleaving the loss coded low resolution image applied by the loss decoding unit based on the transmitted TSD and ESD information; An error information decoding unit for decoding the coded signal with respect to the error information generated during the lossy coding to extract the error information; The error information decoding unit converts the lost low resolution image applied by the high resolution image decoding unit. An elastic shape information encoding apparatus comprising an error correction unit for restoring a lossless high resolution image by correcting according to the error information applied by the. In the flexible shape information encoding method of converting a high-resolution video into a low-resolution video, encoding it through an encoder and then transmitting it to a decoder, the decoder restores a high-resolution video from the transmitted low-resolution video. When encoding a low-resolution video and transmitting it to a decoder, information about encoding errors generated during encoding is detected and transmitted together.In decoding, information about encoding errors transmitted from an encoder is detected to correct errors generated during encoding. And reconstructing a lossless high resolution image from the lossy coded low resolution image. The method of claim 6, wherein the encoding method is a case in which shape information (hereinafter referred to as BL) of "(2X) x (Y)" resolution, which is already transmitted by INTERLACDED SHAPE CODING, is loss coded. And shape information encoding (" EL " hereinafter) of &quot; (2X) × (2Y) &quot; resolution. The method of claim 6, wherein the encoding method encodes and transmits error information of a loss coded BL to encode an EL, restores a lossless BL using the encoded BL, and then uses the same to lossless encode the EL. Flexible shape information encoding method characterized in that. The flexible shape information encoding according to claim 8, wherein the encoder extracts error information of the loss coded BL, and the decoder side decodes the lossless BL by adding the loss-decoded BL and uses the same for lossless EL decoding. Way. The method of claim 6, wherein the encoding method encodes and transmits error information of a loss coded BL in order to lossless encode the EL in interlaced shape information encoding, restores the lossless BL, and then uses the same to lossless encode the EL. Flexible shape information encoding method characterized in that. The method of claim 10, wherein the encoding method extracts error information of the loss coded BL from the encoder side, decodes the lossless BL by adding the same to the loss decoded BL, and uses the same for lossless EL decoding. Flexible shape information coding method. 7. The stretchable shape information encoding method according to claim 6, wherein the encoding method decodes the EL with a loss coded BL to encode the EL in the stretchable encoding. The stretch method according to claim 12, wherein the pixel inserted into the cap is losslessly encoded, and the lossy decoding of the EL is consequently performed by re-coding the error of the lossy coded BL using the new lossless BL. Information coding method. The elastic shape information encoding method according to claim 6, wherein the encoding method decodes the EL with a loss coded BL in order to encode the EL in interlaced shape information encoding. 15. The stretch method according to claim 14, wherein the pixel inserted into the cap is losslessly encoded, and the lossy decoding of the EL is performed as a new lossless BL to re-encode the error of the lossy coded BL. Information coding method. 15. The elastic shape information encoding method according to any one of claims 7 to 14, wherein the encoding method uses a scan interleaving method. 15. The elastic shape information encoding method according to any one of claims 7 to 14, wherein the decoding method uses a scan interleaving method.

Images