KR100728222B1 - Hierarchical Video Encoding/Decoding Method for Complete Spatial Scalability and Apparatus thereof - Google Patents

Abstract

The present invention relates to a scalable video encoding / decoding method and apparatus. More particularly, in order to provide complete scalability in a spatial domain, multi regeions having different spatial resolutions and various positions according to resolution may be used. A method and apparatus for hierarchically encoding / decoding.

According to an aspect of the present invention, there is provided an encoder for encoding a video image, and receives encoding region information about a plurality of regions (ROIs) to be encoded in the video image, and overlaps the region between the ROIs. OR detection unit for detecting OR); A region arranging unit which arranges the video image, the ROI, and the detected OR in a plurality of layers according to a resolution to be encoded; And an area encoder which encodes the video image, the ROI, and the detected OR according to the resolution of the layer arranged in the area arrangement unit. The encoding region information includes position information in a video image of the ROI and encoding resolution information of the ROI.

The present invention has the effect of providing a video encoding / decoding apparatus that enables full scalability not only in resolution but also spatially by defining an ROI in an image. In addition, the present invention has the effect of increasing the coding efficiency by considering the inter-layer redundancy between the plurality of ROI regions.

Scalable Video Coding (SVC), MPEG-4, Resolution, Layer, ROI, inter-layer coding, intra-layer coding

Description

Hierarchical Video Encoding / Decoding Method for Complete Spatial Scalability and Apparatus

1 is a block diagram of an embodiment of an image processing system for providing spatial scalability according to the present invention;

2 is a block diagram of an embodiment of the encoder 110 of FIG. 1 according to the present invention;

3 is a flowchart illustrating an operation of the overlapped area detector 210 according to the present invention;

4 is a flowchart illustrating an example of a process for detecting overlapping areas according to the present invention;

FIG. 5 is a layout diagram according to layers for regions and overlapped regions when a multi-layer based coding scheme is applied according to an embodiment of the present invention; FIG.

6 is a layout diagram according to layers for regions and overlapped regions when a layer-based coding scheme is applied according to an embodiment of the present invention;

7 is a flowchart illustrating a method of arranging layers according to layers in a multi-layer based coding scheme;

8 is a flowchart illustrating an example of a layer-based arrangement method for each region in the case of a layer-based encoding scheme;

9 is a diagram illustrating an embodiment of a region encoding unit when a multi-layer based encoding scheme is applied according to the present invention;

10 is a diagram illustrating an embodiment of a region encoding unit in the case where a one-layer based encoding scheme is applied according to the present invention;

11 is a flowchart illustrating an operation of an area encoder in a case where a multi-layer based coding scheme is applied according to the present invention;

12 is a flowchart illustrating a method of determining an encoding mode of a stream output for a block of a selection region when a multi-layer based encoding scheme is applied according to the present invention;

FIG. 13 is a flowchart illustrating an operation of an area coder when one layer-based encoding is applied according to the present invention; FIG.

14 is a flowchart illustrating a method of determining an encoding mode of a stream output for a block of a selected region when a hierarchical coding scheme is applied according to the present invention;

FIG. 15 illustrates an interpolation method used in a motion prediction / compensation mode and an inter-layer coding mode according to an embodiment of the present invention; FIG.

FIG. 16 is a view showing an SVC bitstream added with a flag indicating that there is an ROI in an image according to a first embodiment of the present invention; FIG.

FIG. 17 is a view showing an SVC bitstream to which a flag indicating that there is an ROI in an image according to a second embodiment of the present invention; FIG.

18 is a view showing an SVC bitstream to which a flag indicating that there is an ROI in an image according to a third embodiment of the present invention;

19 is a diagram illustrating encoding dependencies between regions in inter-layer coding when a multi-layer coding scheme is applied;

20 is a diagram illustrating encoding dependencies between regions in inter-layer coding when one layer coding scheme is applied;

21 is a diagram illustrating a syntax structure of an encoded stream including information about an associated lower layer region according to an embodiment of the present invention;

22 is a block diagram of an embodiment of the decoder 120 of FIG. 1 according to the present invention;

23 is a flowchart illustrating an operation of an area decoding unit when a multi-layer based method is applied according to the present invention;

24 is a flowchart illustrating a method of decoding an encoded stream for a block of a selected region when a multi-layer based scheme is applied according to the present invention;

25 is a flowchart illustrating an operation of an area decoder in a case where a one-layer based scheme is applied according to the present invention;

FIG. 26 is a flowchart illustrating a method of decoding an encoded stream for a block of a selected region when a one-layer based scheme is applied according to the present invention.

* Explanation of symbols for the main parts of the drawings

110: encoder 120: decoder

130: user interface unit 140: display unit

210: overlapped area (OR) detection unit 220: region arrangement unit

230: Region encoding unit 2210: Decoding region extraction unit

2220: region decoder

The present invention relates to a scalable video encoding / decoding method and apparatus. More particularly, in order to provide complete scalability in a spatial domain, multi regeions having different spatial resolutions and various positions according to resolution may be used. A method and apparatus for hierarchically encoding / decoding.

JPEG 2000 standard [ISO / IEC JTC1 15444-2: JPEG 2000 Image Coding System; Extensions, 2004] supports encoding of different parts of an image at various bit rates. That is, a specific region in the image may be encoded at a higher bit rate than other portions of the image. Coding of specific regions in these images has been a key issue in the last few years. In relation to this, in JPEG 2000, a specific region in an image is defined as a region of interest (ROI) and encoded before the other part of the image by using a scaling based method, so that the ROI is spatially scaled in the decoder. It has the ability to independently decode. However, since the encoding method of JPEG 2000 is basically a wavelet-based still image encoding method, it cannot be applied to MPEG-based video image encoding.

The MPEG-based encoding method is a general method for encoding a video image, and in particular, the object-based encoding method of MPEG-4 [ISO / IEC JTC1 14496-2: Coding of Audio-Visual Objects, Part 2, 1998] based encoding) provides an encoding method capable of satisfying a condition that each region may be independently decoded for an image including one or more regions. The plurality of regions included in the image are two-dimensional regions having an arbitrary shape, and are independently encoded or encoded through motion estimation, residual coding, and SADCT (Shape-Adaptive DCT). . However, there is a problem in that the object-based encoding method of MPEG-4 cannot provide an encoding method for a case where a plurality of regions included in an image have a different spatial resolution than the image.

Scalable video coding for decoding video signals having various resolutions and frame rates without encoding signals that are presumed by extracting low resolution video signals or encoding signals having different frame rates from the currently encoded video signal Video coding is being actively researched. H.262 / MPEG-2 Visual [ISO / IEC JTC1 13818-2: Generic Coding of Moving Pictures and Associated Audio Information, Part 2; Video, 1994], H.263 [ITU-T: Video Coding for Low Bitrate Communication, 1995 (version 1), 1998 (version 2)], MPEG-4 Visual [ISO / IEC JTC1 14496-2: Coding of Audio- Visual Objects, Part 2, 1998] supports the same image to be decoded at different spatial resolutions by using a layer-based encoding method for spatial resolution variable encoding. This layer-based encoding method encodes an image made with a lower resolution than the original image into a base layer, and encodes a higher spatial resolution image generated using information of the encoded image into a next layer. (next layer). Recently, with the advent of various video terminals such as PDAs, mobile phones, DMB phones, SDTVs, and HDTVs, the need for various video services is increasing. Accordingly, video that supports decoding of images at various locations as well as various resolutions There is a need for an encoding method. However, the above-described conventional layer-based coding methods have a problem in that it is difficult to apply to the case where regions having different spatial resolutions in the image or regions having various positions in the image exist. In addition, when the above-described conventional layer-based coding method is applied to a plurality of regions in an image as it is, overlapping regions between regions are repeatedly encoded, and thus, coding in a scalable video coding scheme is possible. There is a problem that the efficiency is greatly reduced.

The present invention has been proposed to solve the above problems, and provides an encoding / decoding apparatus and method for providing full scalability in a spatial domain so that regions having various positions and various resolutions within an image can be independently decoded. There is a purpose.

Another object of the present invention is to provide an efficient hierarchical encoding / decoding apparatus and method considering the redundancy between layers in encoding of the above areas.

Another object of the present invention is to provide a scalable encoding / decoding apparatus for selecting a user's desired region and viewing the image at a desired resolution.

Other objects and advantages of the invention can be understood by the following description, and become apparent with reference to the embodiments of the present invention. Also, it will be readily appreciated that the objects and advantages of the present invention may be realized by the means and combinations thereof indicated in the claims.

According to an aspect of the present invention, an encoder for encoding a video image receives input encoding region information about a plurality of regions (ROIs) to be encoded in the video image, and overlaps the region between the ROIs. OR detection unit for detecting the; A region arranging unit which arranges the video image, the ROI, and the detected OR in a plurality of layers according to a resolution to be encoded; And an area encoder which encodes the video image, the ROI, and the detected OR according to the resolution of the layer arranged in the area arrangement unit. The encoding region information includes position information in a video image of the ROI and encoding resolution information of the ROI. The area encoder performs inter-layer coding for encoding in block units by using an area disposed in a lower layer. In the inter-layer coding, when interpolation is performed using pixels outside the ROI region, interpolation is performed after determining pixel values outside the ROI region through extrapolation. A flag indicating that there is an ROI in the video image is added to the encoded stream. In the inter-layer coding, when interpolation is to be performed using pixels outside the ROI region, intra-layer coding may be performed without performing interpolation. have.

In addition, the decoder of the present invention for achieving the above object is required for decoding the ROI corresponding to the selection region from the encoding stream that receives the selection information for the region to be decoded and includes encoding information for a plurality of ROI. A decoding area extraction unit for extracting area information; And an area decoder configured to recover the image signal of the ROI in the entire video by decoding the received area information. The selection information includes position information and resolution information of the region to be decoded. The decoding region extractor extracts an encoded stream of an ROI corresponding to the selected region from the encoded stream, and extracts relevant region information of a lower layer necessary to decode the ROI. The decoding region extractor performs inter-layer decoding on a block-by-block basis using a region of a lower layer in which a region of low resolution is disposed. In the inter-layer decoding, a block located at an ROI boundary performs intra-layer decoding. On the other hand, the plurality of ROI may be composed of a small rectangular area to support interactive decoding.

The method of the present invention provides an encoding method for providing spatial scalability, comprising: an input step of receiving position information and resolution information of a plurality of ROIs to be encoded with respect to an input video image; Disposing the ROI for each layer according to a resolution; And an encoding step of performing encoding on a block basis with respect to the arranged ROI. In the encoding step, when inter-layer coding using ROI information of a lower layer is performed, intra-layer coding is performed on a block requiring interpolation using pixels outside the ROI. In addition, when performing motion prediction / compensation coding using ROI information of the same layer having a temporal correlation, the motion information of a block requiring interpolaton using pixels outside the ROI is included. Integer pel unit prediction is performed with respect to each other. In addition, when inter-layer coding using ROI information of a lower layer is performed, an external pixel value is determined through extrapolation for an block requiring interpolaton using pixels outside the ROI and then interpolated. . In addition, when performing motion prediction / compensation coding using ROI information of the same layer having a temporal correlation, extrapolation (block) that requires interpolaton using pixels outside the ROI is performed. After determining the external pixel value through extrapolaton), interpolation is performed. The location information of the ROI is represented by a macro block scan number for the video image.

The present invention also provides a decoding method for providing spatial scalability, comprising: an input step of receiving position information of a region to be decoded and resolution information to be decoded in an input video image; Extracting ROI information corresponding to the region to be decoded from the transmitted encoded stream; And a decoding step of performing decoding on the ROI using the extracted information. In the decoding step, when interlayer decoding using ROI information of a lower layer is performed, intra layer decoding is performed on a block requiring interpolation using pixels outside the ROI. In addition, when performing motion prediction / compensation decoding using ROI information of the same layer having a temporal correlation, the motion information of a block requiring interpolaton using pixels outside the ROI is included. Integer pel unit prediction is performed with respect to each other. In addition, when interlayer decoding using ROI information of a lower layer is performed, an external pixel value is determined through extrapolation for an block requiring interpolaton using pixels outside the ROI and then interpolated. . In addition, when performing motion prediction / compensation decoding using ROI information of the same layer having a temporal correlation, extrapolation (block) that requires interpolaton using pixels outside the ROI is performed. After determining the external pixel value through extrapolaton), interpolation is performed. In the decoding step, it is determined whether a ROI region in a video image exists in the encoded stream through a flag included in the encoded stream. The location information of the ROI is represented by a macro block scan number for the video image.

The above-described contents of the present invention will become more apparent through the following detailed description with reference to the accompanying drawings, and thus, those skilled in the art to which the present invention pertains may easily implement the technical idea of the present invention. will be. In addition, in describing the present invention, when it is determined that the detailed description of the known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating an embodiment of an image processing system that provides spatial scalability for an input video image and regions included in the image according to the present invention. As shown in FIG. 1, the image processing system of the present invention includes an encoder 110, a decoder 120, a user interface 130, and a display unit 140.

The encoder 110 receives a video image and information about a region of interest (ROI) in the image and outputs the encoded bit stream by performing encoding according to the present invention. The information on the encoding region may include location information of a region of interest (ROI) to be encoded in the image and information on encoding resolution, and may further include a region index of the ROI. The encoded stream output from the encoder 110 is transmitted to the decoder 120 through a channel, and the decoder 120 inputs decoding region information selected by the user from the user interface 130. Receiving the decoding according to the present invention restores the video signal for the area selected by the user. The decryption area information may include position information of an area to be decoded by a user, and may further include information regarding a resolution to be decoded. In addition, the display unit 140 receives the image signal reconstructed by the decoder 120 and displays an image of the area selected by the user on the display.

2 is a diagram illustrating an embodiment of the encoder 110 of FIG. 1 for encoding an input video image and regions within an image according to the present invention. As shown in FIG. 2, the encoder 110 according to the present invention receives information about a video image and an encoding region in the image, and detects an overlapped region (OR) between the image and the regions. A region detector 210, a region arrangement unit 220 for arranging regions in an image to be encoded, including the detected overlap region OR, by layer, and the input video image and region. And a region encoder 230 for generating an encoded stream by encoding the signals. Hereinafter, the configuration and operation of the overlapping region detector 210, the region arrangement unit 220, and the image / region encoder 230 of the encoder 110 will be described with reference to the drawings.

The overlapped region (OR) detector 210 detects an overlapped region (OR) between regions by using the input position information of the encoded region, and generates a new region index for the detected overlapped region. ). Referring to FIG. 5, a duplicate region detected between Region 1 and Region 2 is defined as OR 1, and is detected between Region 2 and Region 3. Overlapping regions are defined as OR 2. On the other hand, the overlapping region between the region 3 (Region 3) and the region 4 (Region 4) includes a region also overlapping with the region 2 (Region 2). In this case, first, a duplicate region between the region 2 and the region 3 having the same spatial resolution is detected and defined as OR 2, and then the region 3 and the region 4 having different spatial resolutions are defined. Among the overlapping regions detected between (Region 4), the region except for the OR 2 is defined as OR 3. As a result, the overlapping region detected between the region 3 (Region 3) and the region 4 (Region 4) is composed of OR 2 and OR 3.

3 is a flowchart illustrating an operation of the overlapped area detector 210 according to the present invention. As shown in the drawing, the overlapped area detector 210 detects overlapped areas of all regions in the image using the encoded area information (S310), and also detects all overlapped areas between the detected overlapped areas (S320). ). A new region index is defined for the detected overlapping region (S330). As described in the index definition method for OR 3, in defining a new region index in step S330, an overlapping region (OR) detected between regions having different spatial resolutions is detected between regions having the same spatial resolution (OR). 2), a new region index (OR 3) for the region except for the overlapping region (OR 2) detected between the regions having the same spatial resolution among the overlapping regions detected between the regions having different spatial resolutions. ).

FIG. 4 is a flowchart illustrating an example of detecting a duplicate area in step S310. The same applies to detection of the duplicate area in step S320. First, when any two regions (Region) are selected using the input encoding region information (ROI region information) (S410), it is determined whether a duplicate region (OR) exists between the two selected regions (S420). If the overlapping region exists, it is determined whether the spatial resolutions of the two regions are the same (S430). If the overlapping region is the same, the overlapping region is configured as the overlapping region OR (S460). On the other hand, if the spatial resolutions of the two areas are different from each other, it is determined whether the region having the low spatial resolution is completely included in the region having the high spatial resolution (S440). A region overlapping with a region having a resolution is configured as a redundant region OR (S450). Then, the process is repeated by selecting a pair of two regions that are not selected (S470 and S480).

The region placement unit 220 arranges regions and overlapped regions for each layer according to the resolution. The input video image is treated as one area having the highest resolution in the hierarchical arrangement.

The present invention is divided into two types according to a method of processing an overlapped area OR. In the video processing system of FIG. 1, operations of the encoder 110, the decoder 120, and the display 140 are duplicated. It is made different according to the processing method of the area OR. In the first of the two methods, when there is a duplicate area, a multi-layer based encoding using a plurality of related lower layers in configuring a new layer and arranging the duplicate area and encoding one layer is performed. Multi-layer based coding method. The second method is a one-layer based coding method that does not form a separate redundant layer for the overlapping region and uses only lower layer information one step below in encoding one layer.

FIG. 5 is a layout diagram according to layers for regions (video image, regions, and overlapped regions) when a multi-layer based coding scheme is applied according to an embodiment of the present invention. FIG. 6 is an embodiment of the present invention. According to an example, this is a layout diagram for each layer of regions (Video Image, Region, and Overlapped Regions) when one layer-based encoding is applied. 5 and 6, region 2 and region 3 have the same resolution, and region 4 has a higher resolution than the two regions 2 and 3. Also, region 1 has a higher resolution than region 4, and the input video image is treated as the region having the highest resolution. In addition, the first layer (Layer 1), the second layer (Layer 2), the third layer (Layer 3), the fourth layer (Layer 4), and the fifth layer (Layer 5) has a layer index, respectively 1 to 5, and a higher layer index corresponds to a higher layer having a higher spatial resolution.

A method of arranging regions (Video Image, Region, and Overlapped Regions) by layer will be described with reference to FIGS. 5 and 6, where a region having a high resolution is disposed in a higher layer and has a lower resolution. Is placed in the lower layer. In addition, regions having the same resolution are arranged in the same layer, and an input video image having the highest resolution is treated as one region and arranged in the uppermost layer.

On the other hand, when there is a duplicate region, the method of disposing the duplicate region differs depending on whether a multi-layer based encoding scheme (see FIG. 5) or a one-layer based encoding scheme (see FIG. 6) is used. First, in the case of a multi-layer based coding scheme, as shown in FIG. 5, an overlapped region OR is formed between a layer in which a region having the same resolution as that of the overlapping region is disposed and a layer below one level. ) Is newly defined, and an overlapping area (OR) is disposed in the overlapping layer (OR layer). On the other hand, in the case of the hierarchical coding scheme, as shown in FIG. 6, the overlapping region OR is disposed in a layer in which a region having the same resolution as that of the overlapping region is disposed.

FIG. 7 is a flowchart illustrating a method of arranging regions according to layers in a multi-layer based encoding scheme. First, a region having the lowest spatial resolution among a region to be encoded (including an input video image and a duplicate region) is selected (S710), and an index of the current layer is initialized to 1 (S720). In operation S730, it is determined whether a redundant region OR having the same spatial resolution as the currently selected region exists. As a result of the check, when there are overlapping regions having the same spatial resolution, the overlapping region OR is disposed in the current layer and then an area which is not the overlapping region is disposed in an upper layer (S750, S760). On the other hand, if there is no overlapping area OR having the same resolution, regions having the same resolution as the selected area are arranged in the current layer (S760). Subsequently, it is checked whether there is an area having a higher spatial resolution than the currently selected area (S770), and if present, a region having the lowest resolution among the unplaced areas is selected (S780), and a layer index is selected. After increasing by 1 (S790), the area arrangement process is repeated.

8 is a flowchart illustrating a method of arranging regions according to layers in the case of a layer-based encoding scheme. First, a region having the lowest spatial resolution among a region to be encoded (including an input video image and a duplicate region) is selected (S810), and the index of the current layer is initialized to 1 (S820). Regions having the same spatial resolution are placed in the current layer (S830). Subsequently, after increasing the layer index by 1 (S840), it is checked whether there is an area having a higher spatial resolution than the currently selected area (S850), and if present, the area having the lowest resolution among the unplaced areas ( Region (S8600) and repeats the region arrangement process.

The region encoder 230 generates an encoded stream by encoding the input video image and the regions. Since the configuration and operation of the region encoder 230 are made differently according to the method of processing the overlapped region OR, it is divided into a case where the multi-layer based coding scheme is applied and a case where the one-layer based coding scheme is applied. Explain.

FIG. 9 is a diagram illustrating an embodiment of the region encoder 230 when a multi-layer based encoding scheme is applied according to the present invention. When the video image and the region information are input from the region arranging unit 220, the region encoder 230 performs encoding on a layer-by-layer region according to the procedures illustrated in FIGS. 11 and 12. Coding is performed). The input region information includes information on a region (including a duplicate region) to be encoded and layout information for each layer of the region. The encoders 920, 940, and 950 encoding regions of each layer receive the region information and determine a coding mode for a block of a selected region according to the procedure illustrated in FIG. 12. The encoded stream according to the determined mode is output to a stream multiplexer (Stream MUX). In FIG. 9, the encoder for Layer 1 does not have a portion where a block to be encoded overlaps with regions of a lower layer, and thus, an inter-layer coding mode is excluded in selecting a coding mode. Accordingly, the encoder 1 downsampler 910 receives a downsampled video image at a resolution suitable for the layer 1, and the motion prediction / compensation mode is applied to the block of the selected region. A coded stream encoded in one of a motion prediction / compensation mode and an intra mode is output. On the other hand, the encoder (Layer 2 encoder, Layer 3 encoder) for the other layer receives a video image corresponding to the resolution of each layer, the inter-layer coding mode, motion prediction / A coded stream encoded in one of a compensation mode (motion prediction / compensation mode) and an intra mode is output. When the encoder performs inter-layer coding, motion information and texture information of a region overlapping a block to be encoded among regions of a lower layer. information, and motion compensated residual information are upsampled and used. On the other hand, the stream multiplexer (Stream MUX) 960 receives and multiplexes an encoded stream output from an encoder for each layer.

FIG. 10 is a diagram illustrating an embodiment of the region encoder 230 when a hierarchical coding scheme is applied according to the present invention. When the video image and the region information are input from the region arranging unit 220, the image and region encoding unit 230 performs encoding on the layered region according to the procedures illustrated in FIGS. 13 and 14. The input region information includes information on a region (including a duplicate region) to be encoded and layout information for each layer of the region. The encoders 1020, 1040, and 1050 encoding regions of each layer receive the region information, determine an encoding mode for a block of a selected region according to the procedure illustrated in FIG. 14, and determine an encoding stream according to the determined mode. Output to the stream multiplexer. Operation of the encoder (Layer 1 encoder) for the layer 1 in FIG. 10 is as described with reference to FIG. On the other hand, when an encoder for another layer (Layer 2 encoder, Layer 3 encoder) performs inter-layer coding (inter-layer coding), unlike FIG. 9 that can use the information of the region present in a plurality of lower layers, The motion information, the texture information, and the motion compensated residual information of a region existing in a hierarchy below one level are upsampled and used. In this case, when there is no overlapping region in the lower layer, inter-layer coding is performed using an intermediate region. On the other hand, the stream multiplexer (Stream MUX) 1060 receives an encoded stream output from an encoder for each layer and multiplexes it.

11 is a flowchart illustrating an operation of the region encoder 230 when a multi-layer based coding scheme is applied according to the present invention. When the layout information for each layer of the region (including the video image and the overlapped region) is input from the region arrangement unit 220, the region of the lowest layer is selected and encoded (S1110 and S1120). After the encoding is finished, it is checked whether another region exists in the same layer (S1130), and if present, the region is selected and encoded (S1140). After encoding the region of the same layer, the region existing in the upper layer is selected and encoded (S1150, S1160).

The encoding in step S1120 follows a block-based video coding scheme in blocks, and is the same layer as inter-layer coding using area information of a lower layer. A coding method of one of intra-layer coding using region information of the C-based apparatus is used. Inter-layer coding is a coding method that is also proposed in the MPEG-4 standard [ISO / IEC 14496-2 (1998)] and includes motion information and texture information of a lower layer. A coding method using upsampling of information and motion compensated residual information. In addition, intra-layer coding is a coding method proposed by MPEG-4 AVC [ISO / IEC 14496-10: Advanced Video Coding, 2003], and is a motion prediction / compensation mode. It is divided into and intra mode. As a result, the region encoder 230 may select one of an inter-layer coding mode, a motion prediction / compensation mode, and an intra mode for each block of the selected region. Outputs an encoded stream encoded in one mode.

12 is a flowchart illustrating a method of determining an encoding mode of an encoded stream output for a block of a selected region when a multi-layer based encoding scheme is applied according to the present invention. As described above, the encoding of the selection region performed in step S1120 follows a block-based video coding scheme, and three blocks (inter-layer coding) for each block are performed. A coded stream encoded in one of modes (motion mode, motion prediction / compensation mode, intra mode) is output. First, in order to apply the inter-layer coding mode, it is necessary to know which region of each layer should be used for each block. Therefore, it is checked whether there is an area portion in which a block of a selection area to be encoded overlaps with regions of lower layers using the layer arrangement information for each layer transmitted from the area arrangement unit 220. In operation S1210, an area having a layer index having the largest layer index among the areas having overlapping portions is selected (S1220). Subsequently, it is checked whether the overlapped region OR is defined for the region (1230), and if defined, inter-later coding is performed using the defined overlapped region OR (S1240). ). On the contrary, when the overlapped region OR is not defined, inter-later coding is performed using a region existing in the layer having the largest index among regions having overlapping portions (S1250). Meanwhile, in step S1260, the motion prediction / compensation mode (motion) is performed on the block regardless of whether there is a region portion in which the block of the selection region to be encoded overlaps with the regions of the lower layers. Intra-layer coding of prediction / compensation mode and intra mode is applied. Finally, in step S1270, one mode of minimizing bit rate is selected by using the encoding results of the three modes. In this case, when it is determined in step S1210 that there is no overlapping area between the blocks to be encoded and the areas of the lower layers, the inter-layer coding mode is excluded in mode selection. Let's do it.

FIG. 13 is a flowchart illustrating an example of an operation of an area encoder when a hierarchical coding scheme is applied according to the present invention. When the layout information for each layer of the region (including the video image and the overlapped region) is input from the region arrangement unit 220, the region of the lowest layer is selected and encoded (S1310 and S1320). After the encoding is finished, it is checked whether another region exists in the same layer (S1330), and if present, the region is selected and encoded (S1340). If no other area exists in the same layer, it is checked whether there exists an area that does not overlap with the area of the current layer among the areas existing in the hierarchy below one step (S1350). Intermediate regions are configured for regions that are not spatially matched with regions of the current hierarchy among the regions of the hierarchy (S1360). The intermediate region is a region that is referred to when inter-layer coding is performed in an upper layer. On the other hand, if it is determined in step S1350 that there is no area that does not overlap with the area of the current layer among areas existing in the layer below one step, the area existing in the upper layer (Layer) is selected and encoded (S1370, S1380).

The intermediate region configured in step S1360 includes motion information, texture information, and motion compensation residual information of a region existing in a lower layer below one step. Residual) is a region configured with the spatial resolution of the current layer, and encoding is not performed on the intermediate region. The intermediate region configured as described above is used when performing inter-layer coding encoding in an upper layer. Since the encoding according to the present invention follows a block-based video coding scheme, even in the configuration of an intermediate region, motion information, texture information, and motion in units of blocks are used. Motion compensated residual information is needed. Motion information, texture information, and motion compensated residual information upsampled up to the spatial resolution of the current layer through interpolation are higher layers. If there is no area matching the mediation area in the layer, it may be used to construct a new mediation area through interpolation of the same method in the upper layer. On the other hand, the motion information includes a motion vector and information about a coding mode (inter-layer coding mode, motion prediction / compensation mode, intra mode) of the corresponding block.

The encoding performed in step S1320 follows a block-based video coding scheme made in units of blocks, as described in the case where the multi-layer based coding scheme is applied, and the region information of the lower layer. One coding method of inter-layer coding using intra-layer coding and intra-layer coding using region information of the same layer is used. Inter-layer coding is a coding method that is also proposed in the MPEG-4 standard [ISO / IEC 14496-2 (1998)] and includes motion information and texture information of a lower layer. A coding method using upsampling of information and motion compensated residual information. In addition, intra-layer coding is a coding method proposed by MPEG-4 AVC [ISO / IEC 14496-10: Advanced Video Coding, 2003], and is a motion prediction / compensation mode. It is divided into and intra mode. As a result, the image and region encoder 230 performs an inter-layer coding mode, a motion prediction / compensation mode, and an intra mode for each block of the selected region. mode) outputs an encoded stream encoded in one mode.

FIG. 14 is a flowchart illustrating a method of determining an encoding mode of an encoded stream output for a block of a selection region when a hierarchical encoding scheme is applied according to the present invention. As described above, the encoding of the selection region performed in step S1320 follows a block-based video coding scheme, and three blocks (inter-layer coding) for each block are performed. A coded stream encoded in one of modes (motion mode, motion prediction / compensation mode, intra mode) is output. First, in order to apply the inter-layer coding mode, it is necessary to know which region of each layer should be used for each block. Accordingly, a region in which a block of a selection region to be encoded by using the region arrangement information for each layer transmitted from the region arrangement unit 220 overlaps with regions of layers in one step below. In operation S1410, inter-later coding is performed by using an overlapping region of a layer below one step, in operation S1420. If the overlapping region does not exist, it is checked whether a block of the selection region overlaps with an intermediate region of the hierarchy below one step, and if the region overlaps, the intermediate region In operation S1450, inter-later coding is performed. Meanwhile, in step S1460, the motion prediction / compensation mode (motion) is performed on the block regardless of whether there is a region part in which the block of the selection area to be encoded overlaps with the regions of the lower layers. Intra-layer coding of prediction / compensation mode and intra mode is applied. Finally, in step S1270, a mode for minimizing bit rate is selected using the encoding results of the three modes. In this case, when it is determined in step S1440 that there is no overlapping region between the blocks to be encoded and the areas of the lower layers, the inter-layer coding mode is selected in selecting the mode in step S1430. mode).

FIG. 15 illustrates a motion prediction / compensation mode performed in steps S1260 and S1460 and an inter-layer coding mode performed in steps S1240, S1250, S1420, and S1450 according to an embodiment of the present invention. A diagram for describing an interpolation method used in a layer coding mode. The grayed out small rectangular area represents the pixels of the original image, and in order to double the resolution of the original image, a half-pixel value between the pixels must be generated by interpolation. . MPEG-4 Advanced Video Coding (AVC) and Scalable Video Coding (SVC) use an FIR filter with six taps for half pel motion estimation, and the same filter uses the interpolation of the present invention. interpolation). Equation 1 is an example of a filter equation used for interpolation of a motion prediction / compensation mode and an inter-layer coding mode of the present invention. The interpolation method is basically a method of doubling the resolution of an image through half-pel interpolation. [Equation 1] is applicable to the above interpolation considering the amount of computation and performance. One of the filter expressions.

In Equation 1, I (x, y) is the pixel value of the (x, y) coordinate in the input image, and O (x, y) is the pixel value of the (x, y) coordinate in the interpolated output image. Means.

However, in performing the interpolation as described above, a region in which encoding is performed (hereinafter referred to as ROI) should not be decoded using region information other than the ROI when decoding. Therefore, when interpolation of the ROI boundary region is performed by the region encoder 230, it is necessary not to use a pixel value outside the ROI region. To this end, the present invention uses one of two methods.

The first method is to determine the pixel value outside the boundary region by extrapolation before performing interpolation at the ROI boundary and then perform half-pel interpolation. In the extrapolation, a zero order extrapolation method or a specific constant value substitution method may be used. Equation 2 below is a filter formula considering the ROI interface by applying zero order extrapolation.

As described above, when the pixel value outside the boundary area is determined through extrapolation at the ROI boundary and then half-pel interpolation is performed, the ROI in the image transmitted through the channel is decoded by the decoder. You must know that it exists. Therefore, a flag is added to the encoded stream output from the image and region encoder to indicate that there is an ROI region that can be independently decoded in the transmitted bit stream such as roi_flag (or roi_enable or boundary_handling or multiple_roi_flag). The decoder receiving the bit stream to which the above flag is added checks whether the ROI is defined through the flag value, and if so, the decoder is encoded in consideration of the ROI boundary region as described above. Enables the ability to decode bit streams. 16 to 18 are diagrams for describing an example in which a flag for indicating that an ROI in an image exists as described above is added to an encoded stream. First, FIG. 16 shows an example of an SVC bit stream in which roi_flag (or roi_enable or boundary_handling or multiple_roi_flag) is added 1610 when the ROI is generated as slice_group_map_type 2. 17 shows an example of generating a new slice group map type (slice_group_map_type) for ROI and enabling 1720 roi_flag (or roi_enable or boundary_handling or multiple_roi_flag) in the slice group map type. That is, the slice group map type (slice_group_map_type) is expanded to generate a slice group map type as an integer greater than 6, and the roi_flag (or roi_enable or boundary_handling or multiple_roi_flag) is disabled before determining the slice group map type. 1710 and enable 1720 the flag in the new slice group map type for the ROI. At this time, the generation of the slice group map is the same as the case of the conventional slice group map type (slice_group_map_type). FIG. 18 illustrates a case in which an ROI is generated as a slice group map type 2 1820 as in FIG. 16. In FIG. 18, when roi_flag (or roi_enable or boundary_handling or multiple_roi_flag) is enabled, the num_rois_minus1, roi_top_left, and roi_bottom_right variables read geometric information such as the number and coordinates of the ROI, and slices matching the ROI boundary. Interpolation (i.e., upsampling) taking into account the ROI boundary is performed on the slice group boundary. In order to express the position of the ROI, the macro block number where the ROI region starts after the numbers of macro blocks of the entire image are assigned in a raster scan order without using coordinates. The position of the ROI may be expressed by using a macro block number at which the ROI region ends.

On the other hand, a second method for the decoder not to decode by using the information of the ROI outer region is encoding by a motion prediction / compensation mode and an inter-layer coding mode This method adds a time constraint. More specifically, interpolation is performed when interpolation should be performed by referring to external pixels outside the ROI boundary in the motion prediction / compensation mode performed in steps S1260 and S1460. I never do that. Therefore, motion information referring to the ROI boundary is predicted in integer pixels. In addition, inter-layer prediction is not performed on blocks located at ROI boundaries in the inter-layer coding mode performed in steps S1240, S1250, S1420, and S1450. Therefore, a block located at the ROI boundary can be encoded using only information of the current layer, not a lower layer, that is, a motion prediction / compensation mode or an intra mode. encoded in (intra mode).

19 and 20 are diagrams illustrating coding dependencies between regions in inter-layer coding when the multi-layer coding scheme is applied and when the one-layer coding scheme is applied, respectively. In FIG. 19 and FIG. 20, a region where an arrow starts is needed to encode a region indicated by an arrow.

First, a case in which the multi-layer coding scheme is applied will be described with reference to FIGS. 19 and 12. In encoding of region 1, regions 1910 where regions of the lower layer and regions that do not overlap exist do not exist. The belonging blocks are encoded in a mode that minimizes the bit rate among a motion prediction / compensation mode and an intra mode. On the other hand, blocks belonging to the region 1920 where the region of the lower layer and the overlapped region exist are inter-layer coding mode, motion prediction / compensation mode. Are encoded in a mode that minimizes a bit rate among, and intra modes. In this case, when the inter-layer coding mode is applied, region 1 has a region portion overlapping with region 2, and overlap region OR 1 with respect to the overlapping portion. Since is defined, blocks belonging to the region 1920 in which the region of the lower layer and the overlapped region exist are encoded using OR 1.

Next, a case in which the one-layer coding scheme is applied will be described with reference to FIGS. 20 and 14. In the encoding of region 1, a region where a region of a lower layer does not exist and a duplicate region do not exist (2010). Blocks belonging to the Rx are encoded in a mode that minimizes bit rate among a motion prediction / compensation mode and an intra mode, as in the case where a multi-layer coding scheme is applied. Meanwhile, blocks belonging to the region 2020 in which the region of the lower layer and the overlapped region exist may be inter-layer coding mode or motion prediction / compensation mode. Are encoded in a mode that minimizes a bit rate among, and intra modes. In this case, when an inter-layer coding mode is applied, region 1 has a region portion overlapping with region 2, and the overlapping portion exists under two steps. It is defined as a duplicate area OR 1 in the layer 1. However, in the one-layer coding scheme, since a layer below one step performs inter-layer coding using a region, first, motion information, texture information, and motion residual information of OR 1 in Layer 2 are performed. The intermediate region 2030 is configured by upsampling motion compensated information. In Layer 3, blocks belonging to a region 2020 in which a region of a lower layer and a duplicate region exist are encoded by using the information of the intermediate region 2030.

FIG. 21 is a diagram illustrating a syntax structure of an encoded stream including information about an associated lower layer region, according to an embodiment of the present invention. FIG. In step S1120 of FIG. 11 and step S1320 of FIG. 13, information about regions of a lower layer used for encoding a selected region is added to an encoding stream. Referring to FIG. 21, an encoding stream for a selection region includes a region index 2110 of the selection region, a layer index 2120 of the layer to which the selection region belongs, and the selection region. Number of related Regions 2130 used for the encoding of the information, information on the regions used for the encoding (related Region information) 2140, and a video signal encoded for the selected region (encoded). Region video signal) 2150. In the information (region related information) 2140 of regions used for encoding the selected region, information 2160, 2170, and 2180 for each region used for encoding the selected region is included in the number of regions used. (Number of related Regions) 2130. Further, the information on the region used for encoding is included in the region index 2171 of the region used for encoding, the layer index 2172 of the layer to which the region used for encoding belongs, and the selection region. The horizontal axis position (h_org) 2173 of the region used for the encoding for the selected region, the vertical axis position (v_org) 2174 of the region used for the encoding for the selected region, and the horizontal width (width) of the region used for the encoding ( 2175, and a vertical length 2176 of the region used for the encoding. In representing the location of the region, macro block numbers, not coordinates, may be used. That is, the macro blocks of the entire image are numbered in the order of the raster scan, and then the macro block number at which the region starts and the macro block number at which the region ends. Can be used to represent the location of the region.

Meanwhile, regions having the same position in the input video image have the same region index even when the spatial resolutions of the regions are different. Through this, the decoder can decode the specific region at various spatial resolutions by extracting an encoding stream of a region having the same region index in each layer to decode the specific region. have.

22 is a block diagram of an embodiment of the decoder 120 of FIG. 1 for decoding an encoded stream according to the present invention. As illustrated in FIG. 22, the decoder 120 including the decoding area extraction unit 2210 and the region decoding unit 2220 may receive the decoding area information selected by the user from the user interface unit 130. Video signal is recovered by performing decoding according to the present invention on the encoded stream received from the encoder 110 and transmitted from the encoder 110. The decoding area information includes location information of an area within an image selected by the user for decoding.

The decoding region extraction unit 2210 receives decoding region information from the user interface 130, and extracts and configures information necessary for decoding the decoding region from an encoded stream transmitted through a channel. That is, the decoding region extracting unit 2210 may include a number of related regions 2130 and information about each lower layer region required for decoding an ROI including a region selected by a user from an encoded stream. Region information 2140 is read to form a list of regions necessary for the ROI decoding, and a region portion necessary for ROI decoding is extracted from the encoded stream to configure information about each region.

In this case, when the encoder 110 divides and encodes an entire image into a plurality of non-overlapping small regions, interactive decoding may be implemented in which the decoder 120 may select and decode an arbitrary region. Such encoding may be performed on the entire image in a plurality of layers. More specifically, in the current Scalable Video Coding (SVC) scheme, when configuring a region through a slice group map type 2, the region is configured to be a very small rectangular region and decoded. It is possible to enable interactive decoding by decoding regions corresponding to the region selected by the user. In this case, when the user selects an arbitrary area that the user wants to decode through the user interface 130, the decoding area extractor 2210 configures the ROI area into areas corresponding to the selected area selected by the user. The number of related regions 2130 and the related region information 2140 of the lower layer regions required for decoding the ROI region are read and decoded from the encoded stream transmitted through the channel. A list of regions required for the configuration is constructed, and region information necessary for decoding the ROI region is extracted from the encoded stream.

The region decoder 2220 receives region information necessary for decoding the ROI region from the decoding region extractor 2210 and restores the image signal by performing decoding. Hereinafter, an operation of the region decoder 2220 will be described as a case where a multi-layer based method and a one-layer based method are applied.

FIG. 23 is a flowchart illustrating an operation of an area decoder when a multi-layer based method is applied according to the present invention. When region information necessary for decoding is input from the decoding region extracting unit 2210, the region of the lowest layer among the regions related to decoding is selected and decoded (S2310 and S2320). After decoding, it is checked whether another region exists in the same layer (S2330), and if present, the region is selected and decoded (S2340). After decoding the region of the same layer, the region existing in the upper layer is selected and decoded (S2350, S2360, and S2320).

Decoding in step S2320 follows a block-based video coding scheme in blocks, and is the same layer as inter-layer coding using area information of a lower layer. A decoding method corresponding to one of the intra-layer coding methods using intra-layer coding is used. Intra-layer coding is divided into a motion prediction / compensation mode and an intra mode. As a result, the region decoder 2220 may interleave each block of the selected region. Decoding is performed by one of inter-layer decoding, motion prediction / compensation decoding, and intra mode decoding. The motion prediction / compensation decoding and intra mode decoding methods are presented in MPEG-4 AVC [ISO / IEC 14496-10: Advanced Video Coding, 2003]. The inter-layer decoding method is a decoding method proposed in the MPEG-4 standard [ISO / IEC 14496-2 (1998)] and includes motion information and texture of a lower layer. It is a decoding method using upsampling of texture information and motion compensated residual information.

24 is a flowchart illustrating a method of decoding an encoded stream for a block of a selected region when a multi-layer based scheme is applied according to the present invention. As described above, the decoding of the selection region performed in step S2320 follows a block-based scheme, and the encoding stream of each block has three methods (inter-layer decoding, motion prediction / compensation). decoding, intra mode decoding). Referring to FIG. 24, a decoding method for a block of a selection area according to the present invention will be described. First, the encoding modes of a block to be decoded are checked (S2410 and S2440). When the coding mode of the block to be decoded is the inter-layer coding mode, the layer having the largest layer index among the regions of the lower layer overlapping the current block to be decoded. An area present in the layer is selected (S2420) and inter-layer decoding is performed (S2430). If the encoding mode of the selection block to be decoded is a motion prediction / compensation mode, motion prediction / compensation decoding is performed on the block, and the encoding mode of the block is motion. In the case of neither the motion prediction / compensation mode nor the inter-layer coding mode, intra mode decoding is performed on the block.

25 is a flowchart illustrating an operation of an area decoder in the case where a one-layer based scheme is applied according to the present invention. When region information necessary for decoding is input from the decoding region extracting unit 2210, a region of a lower layer among the regions related to decoding is selected (S2510), and the selected region exists in the uppermost layer. In this case, decoding is performed on the selected region (S2550). If the selected area does not exist in the top layer, it is checked whether there exists an area that does not overlap with the area of the current layer among areas existing in the hierarchy below one step (S2330). Decode the selected region (S2550). If present, an intermediate region is formed for a region that is not spatially matched with the region of the current layer among the regions of the layer below the one step. Decryption is performed (S2540, S2550). After decoding, it is checked whether another region exists in the same layer (S2560), and if present, the region is selected and decoded (S2570). After decoding the region of the same layer, the region existing in the upper layer is selected and decoded (S2580, S2590, and S2520).

The intermediate region configured in step S2540 includes motion information, texture information, and motion compensation residual information of a region existing in a lower layer below one step. A region formed by interpolating residual information and used when inter-layer decoding is performed in an upper layer. Since the decoding of the present invention follows a block-based scheme, the structure of an intermediate region is also formed in units of blocks. The motion information includes a motion vector and information about a coding mode (inter-layer coding mode, motion prediction / compensation mode, intra mode) of the corresponding block.

Decoding in step S2550 is performed in units of blocks as described in the case where the multi-layer-based scheme is applied, and each block of the selection region is inter-layer decoding, motion prediction / compensation decoding. decoding in one of compensation compensation and intra mode decoding. The motion prediction / compensation decoding and intra mode decoding methods are presented in MPEG-4 AVC [ISO / IEC 14496-10: Advanced Video Coding, 2003]. The inter-layer decoding method is a decoding method proposed in the MPEG-4 standard [ISO / IEC 14496-2 (1998)], which is motion information and texture information of a lower layer. And up-sampling of motion compensated residual information.

FIG. 26 is a flowchart illustrating a method of decoding an encoded stream for a block of a selected region when a one-layer based scheme is applied according to the present invention. The decoding scheme when the one-layer based scheme is applied is basically the same as the case where the multi-layer based scheme is applied, except that only a layer below one layer is referred to when performing inter-layer decoding. In one layer-based method, when there is no overlapping region part in a layer below one step, an intermediate area is configured as shown in step S2540 to refer only to the layer below one step. Perform decryption. Referring to FIG. 26, a decoding method for a block of a selection area according to the present invention will be described. First, the encoding modes of a block to be decoded are checked (S2610 and S2640). When the coding mode of the block to be decoded is an inter-layer coding mode, the region of one layer below or one region having a region portion overlapping with the block to be decoded is configured in step S2540. Inter-layer decoding is performed using an intermediate region (S2620 and S2630). If the encoding mode of the selection block to be decoded is a motion prediction / compensation mode, motion prediction / compensation decoding is performed on the block, and the encoding mode of the block is motion. In the case of neither the motion prediction / compensation mode nor the inter-layer coding mode, intra mode decoding is performed on the block.

A method of performing half pel interpolation in the motion prediction / compensation decoding performed in steps S2450 and S2650, and in the inter-layer decoding in steps S2430 and S2630, The method of performing upsampling is as described above with reference to FIG. 15 in steps S1260, S1460, S1240, S1250, S1420, and S1450. As described in the steps S1260, S1460, S1240, S1250, S1420, and S1450, the encoder 110 uses one of two processing methods for the ROI boundary region and is used by the encoder 110. The operation of the decoder 120 is made differently according to the method.

First, before performing interpolation on the ROI boundary in the encoder 110, the pixel value outside the boundary area is first determined by extrapolation, and then half-pel interpolation or upsampling is performed. In the case of performing upsampling, the decoder 120 can independently decode the bit stream through a flag such as roi_flag (or roi_enable or boundary_handling or multiple_roi_flag) included in the encoded stream transmitted through the channel. Check whether ROI area is defined. As a result of the check, when an independently decodable ROI region is defined, the region decoder 2220 may decode the encoded bit stream considering the ROI boundary region as described above (for example, [Equation 2]). Enable filter).

On the other hand, the encoder 110 does not interpolate a boundary region that should refer to an external pixel outside the ROI boundary so as not to refer to a pixel outside the ROI boundary region. For this, constraints for encoding in a mode (motion prediction / compensation mode or intra mode) that can be encoded using only information on the current layer may be added. When the method of adding the above constraints in the encoder 110 is used, the region decoder 2220 may motion information referring to the ROI boundary during motion prediction / compensation decoding. Is predicted in integer pixel units, and the block located at the ROI boundary can be decoded using only the information of the current layer, not inter-layer decoding. Decoding is performed using a method (motion prediction / compensation decoding or intra mode decoding).

As described above, the method of the present invention may be implemented as a program and stored in a recording medium (CD-ROM, RAM, ROM, floppy disk, hard disk, magneto-optical disk, etc.) in a computer-readable form. Since this process can be easily implemented by those skilled in the art will not be described in more detail.

The present invention described above is capable of various substitutions, modifications, and changes without departing from the technical spirit of the present invention for those skilled in the art to which the present invention pertains. It is not limited by the drawings.

As described above, the present invention has the effect of providing a scalable video encoding / decoding apparatus that is not only in resolution but also in space by defining ROI in a video image. In addition, the present invention has the effect of increasing the coding efficiency by considering the inter-layer redundancy between the plurality of ROI regions.

In addition, the present invention enables a decoder to easily obtain region information necessary for decoding a corresponding region through an encoded stream having a syntax structure capable of efficiently transmitting the information on the plurality of ROI regions as described above. In addition, the present invention provides a method of adding an ROI flag to a syntax structure so that a decoder can recognize that there is an ROI region that can be independently decoded and activate an associated decoding function.

In addition, the present invention has the effect of allowing the decoder to independently decode the ROI by suggesting a processing method in the ROI boundary region during region encoding that should refer to information of another region. In particular, the present invention has an effect of easily decoding the ROI without any additional configuration by adding a constraint that the interpolation is not performed when the interpolation is to be performed using a pixel of the ROI outer region during encoding. have.

Claims (98)

An encoder for encoding a video image, An OR detector which receives encoding region information on a plurality of regions (ROIs) to be encoded in the video image and detects overlapping regions (ORs) between the ROIs; A region arranging unit which arranges the video image, the ROI, and the detected OR in a plurality of layers according to a resolution to be encoded; And And an area encoder which encodes the video image, the ROI, and the detected OR according to the resolution of the layer arranged in the area arranging unit. The method of claim 1, The coding region information, And location information in the video image of the ROI and encoding resolution information of the ROI. The method of claim 1, The OR detection unit, And if the low resolution ROI is not spatially included in the high resolution ROI, detect an OR in the low resolution ROI. The method of claim 3, wherein The OR detection unit, And if the low resolution ROI is spatially included in the high resolution ROI, the encoder is not detected. The method of claim 3, wherein The OR detection unit, When the overlapping part between ROIs having different resolutions includes the overlapping part between ROIs having the same resolution, the part except the overlapping part between ROIs having the same resolution in the overlapping part between ROIs having different resolutions Is an OR. The method of claim 3, wherein The area arrangement unit, And a region having a higher resolution among the video image, the ROI, and the OR treated as one region, and a region having a lower resolution in a lower layer. The method of claim 6, The area arrangement unit, And generating an OR layer under a layer having an ROI having the same resolution as the OR and arranging an OR that is spatially overlapped with the ROI in the generated OR layer. The method of claim 6, The area arrangement unit, And an OR that is spatially overlapped with the ROI in a layer having an ROI having the same resolution as the OR. The method of claim 6, The area encoder, And an inter-layer coding for encoding in units of blocks by using a region arranged in a lower layer. The method of claim 6, The area encoder, And an intra-layer coding for performing motion prediction / compensation on a block basis using a region of the same layer. The method of claim 7, wherein The area encoder, And performing inter-layer coding on the block by using a region disposed in a higher layer among regions of a lower layer having overlapping blocks of a region to be encoded. The method of claim 11, The area encoder, And if an OR is detected with respect to the overlapped part, inter-layer coding of the block is performed using the detected OR. The method of claim 11, The area encoder, And a coded stream encoded in a mode that minimizes a bit rate among inter-layer coding mode, motion prediction / compensation mode, and intra mode for a block of a region to be encoded. The method of claim 8, The area encoder, And performing inter-layer coding on the block by using an area of a first-level lower layer having an overlapping part with a block of an area to be encoded. The method of claim 14, The area encoder, And an intermediary region for reference when performing inter-layer coding in a higher layer by upsampling motion information and texture information of a lower layer region. The method of claim 15, The area encoder, And a coded stream encoded in a mode that minimizes a bit rate among inter-layer coding mode, motion prediction / compensation mode, and intra mode for a block of a region to be encoded. The method of claim 9, The area encoder, In the inter-layer coding, when interpolation is to be performed using pixels outside the ROI region, interpolation is performed after determining pixel values outside the ROI region through extrapolation. Encoder. The method of claim 10, The area encoder, In case of performing interpolation using pixels outside the ROI region in the intra-layer coding for performing the motion prediction / compensation, after determining the pixel value outside the ROI region through extrapolation An encoder that performs interpolation. The method of claim 17 or 18, The extrapolation is an encoder using a zero-order extrapolation or a constant constant assignment. The method of claim 17 or 18, The area encoder, And a flag indicating that there is an ROI in the video image to an encoded stream. The method of claim 20, And the flag is one of roi_flag, roi_enable, boundary_handling, and multiple_roi_flag. The method of claim 20, The ROI is generated as a slice group map type 2 (slice_group_map_type 2). The method of claim 20, And the ROI is generated as an extended slice group map type (slice_group_map_type) having a type identifier of 6 or more. The method of claim 20, The area encoder, And adding geometric information of the ROI in the video image to an encoded stream. The method of claim 24, The geometric information is represented by the num_rois_minus1, roi_top_left, and roi_bottom_right variables. The method of claim 24, And the geometric information of the ROI is expressed through a scan number of a macroblock constituting the ROI. The method of claim 26, Wherein the scan number is a number generated through a raster scan of a macroblock of the video image. The method of claim 9, The area encoder, And the interpolation is not performed when interpolation is to be performed using pixels outside the ROI region in the inter-layer coding. The method of claim 9, And a block located at a ROI boundary in the inter-layer coding performs intra-layer coding. The method of claim 10, The area encoder, And the interpolation is not performed when interpolation is performed using pixels outside the ROI region in the intra-layer coding for performing the motion prediction / compensation. The method of claim 10, In case of performing interpolation using pixels outside the ROI region in the intra-layer coding for performing motion prediction / compensation, motion information is predicted in units of integer pels. Encoder. The method of claim 9, The area encoder, And an encoder for outputting an encoding stream including information on lower layer regions required for region encoding. The method of claim 32, The encoded stream, And an information about a region index of an encoding region, a hierarchical index of the encoding region, the number of related regions required for encoding, and a related region required for encoding. The method of claim 33, wherein The information about the relevant region required for the encoding is And an area index, a layer index, and position information of the related region. The method of claim 34, wherein The location information of the related area is And a horizontal axis position, a vertical axis position, a horizontal length, and a vertical length with respect to the encoding region. The method of claim 34, wherein The location information of the related area is And an encoder is represented by a raster scan number of a macroblock in a video image. A decoding region extracting unit which receives selection information about a region to be decoded and extracts region information necessary for decoding the ROI corresponding to the selection region from an encoding stream including encoding information about a plurality of ROIs; And And a region decoder configured to receive the extracted region information and to decode the image signal for the ROI in the entire video. The method of claim 37, The selection information, And a position information in the video image of the region to be decoded and resolution information to be decoded. The method of claim 37, The decoding region extraction unit, And extracting the encoded stream of the ROI corresponding to the selected region from the encoded stream, and extracting relevant region information of a lower layer necessary to decode the ROI. The method of claim 39, The extracted encoded stream, And information about the region index of the ROI, the layer index of the ROI, the number of the related regions, and the related regions. The method of claim 39, The relevant area information, And a region index, a layer index, and position information of the related region. 42. The method of claim 41 wherein The location information of the related area is And a horizontal axis position, a vertical axis position, a horizontal length, and a vertical length with respect to the ROI region to be decoded. 42. The method of claim 41 wherein The location information of the related area is A decoder as represented by a raster scan number for a macroblock in a video image. The method of claim 39, The area decoder, And performing inter-layer decoding on a block-by-block basis using an area of a lower layer in which a region of low resolution is disposed. The method of claim 39, The area decoder, And an intra-layer decoding for performing motion prediction / compensation on a block-by-block basis using an area of the same layer in which an area of the same resolution is disposed. The method of claim 44, The area decoder, And the interpolation is not performed when interpolation is to be performed using pixels outside the ROI region in the inter-layer decoding. The method of claim 44, And a block located at an ROI boundary in the inter-layer decoding performs intra-layer decoding. The method of claim 45, The area decoder, The decoder does not perform interpolation when it is necessary to perform interpolation using pixels outside the ROI region in intra-layer decoding for performing motion prediction / compensation. . The method of claim 45, In the case of intra-layer decoding for performing motion prediction / compensation, when interpolation is to be performed using a pixel outside the ROI region, motion information is predicted in units of integer pels. Decoder. 46. The method of claim 44 or 45, The area decoder, When the flag indicating that there is an ROI in the encoded stream is activated, the geometric information of the fraudulent ROI included in the encoded stream is determined, and the pixel value outside the ROI region is determined through extrapolation and then interpolated. Decoder characterized by. The method of claim 39, The area decoder, If the encoding mode of the block in the region to be decoded is the inter-layer coding mode, the inter-layer of the block is made by using a region disposed at the highest layer among the related regions of the lower layer having an overlapping portion with the block of the region to be decoded. And a decoder for performing inter-layer decoding. The method of claim 39, The area decoder, And a mediation region for reference when performing inter-layer decoding in a higher layer by upsampling motion information and texture information of a lower layer region. The method of claim 52, wherein The area decoder, If the encoding mode of a block in a region to be decoded is an interlayer coding mode, inter-layer decoding of the block using an area of a first-level lower layer having an overlapping portion with a block of the region to be decoded. And a decoder. The method of claim 37, And the plurality of ROIs is configured of a small rectangular area to support interactive decoding. Input means for receiving encoding region information about a plurality of ROIs to encode the input video image; And Encoding means for performing encoding on the ROI, The interpolation is not performed when the interpolation is to be performed with reference to the pixels outside the ROI in encoding the ROI. The method of claim 55, The encoding means, Performing motion prediction / compensation coding on the ROI using temporal correlation Video encoding apparatus characterized in that. The method of claim 56, wherein The encoding means, In the case of performing the motion prediction / compensation coding, when interpolation is to be performed with reference to a pixel outside the ROI, motion information is predicted in an integer pixel unit. Video encoding apparatus characterized in that. The method of claim 55, The coding region information, And position information in the video image of the ROI and encoding resolution information of the ROI. The method of claim 55 or 58, And means for arranging the ROI for each layer according to the resolution to be encoded. The method of claim 59, The encoding means, And an inter-layer coding for encoding an ROI disposed in a higher resolution upper layer by using an area disposed in a lower layer having a lower resolution. The method of claim 60, When the encoding means performs interlayer coding on the ROI in units of blocks, a block that needs to be interpolated using pixels outside the ROI region is encoded using intra-layer coding. And a video encoding apparatus. The method of claim 55, And the plurality of ROIs comprises a small rectangular area to support interactive decoding. The method of claim 55, And the ROI position information included in the encoded stream output from the encoding means is represented by a coordinate value. The method of claim 55, ROI position information included in the encoded stream output from the encoding means is represented by a macroblock scan number in the entire video image. The method of claim 60, The inter-layer coding is a video encoding apparatus characterized by using an overlapping region in the lower layer and the ROI to be encoded. 66. The method of claim 65, The overlapping region is configured as OR in a newly generated layer below the lower layer. 66. The method of claim 65, The overlapping area is configured as OR in the lower layer. The method of claim 67 wherein The encoding means, If there is no spatially overlapping ROI in the upper layer of the ROI or OR, upsampling the ROI or OR texture information of the lower layer to form a mediation area suitable for the resolution of the upper layer. Video encoding apparatus characterized in that. The method of claim 68, wherein The encoding means, A video encoding apparatus characterized by performing inter-layer coding using an area of a layer below one step. Input means for receiving selection information on an area to be decoded in the input video image; Decoding means for performing decoding on the ROI corresponding to the region to be decoded, The interpolation is not performed when the interpolation is to be performed with reference to a pixel outside the ROI in the decoding of the ROI. The method of claim 70, The decoding means, Performing motion prediction / compensation decoding using temporal correlation on the ROI Video decoding apparatus characterized in that. The method of claim 71 wherein The decoding means, In the case of performing the motion prediction / compensation decoding, when interpolation is to be performed by referring to a pixel outside the ROI, motion information is predicted in an integer pixel unit. Video decoding apparatus characterized in that. The method of claim 70, The selection information, And position information in the video image of the region to be encoded and resolution information to encode the region. The method of claim 70, The decoding means, And an inter-layer decoding for decoding an ROI disposed in a higher resolution upper layer by using an area disposed in a lower layer having a lower resolution. The method of claim 74, wherein When the decoding means performs interlayer decoding on a block-by-block basis with respect to the ROI, intra-layer coding is performed on a block that needs to be interpolated using pixels outside the ROI region. A video decoding apparatus. The method of claim 70, And a plurality of small rectangular ROI regions for decoding the region to be decoded. The method of claim 70, And the ROI location information is represented by a macroblock scan number in the entire video image. The method of claim 74, wherein The inter-layer decoding is a video decoding apparatus, characterized in that using the ROI to perform decoding and the region (OR) overlapping in the lower layer. The method of claim 78, The decoding means, If there is no spatially overlapping ROI in the upper layer of the ROI or OR, upsampling the ROI or OR texture information of the lower layer to form a mediation area suitable for the resolution of the upper layer. Video decoding apparatus characterized in that. 80. The method of claim 79 wherein The decoding means, And an inter-layer decoding using an area of a layer below one step. An input step of receiving position information and resolution information of a plurality of ROIs to be encoded with respect to an input video image; Disposing the ROI for each layer according to a resolution; And Encoding step of performing block unit encoding on the arranged ROI An encoding method for providing spatial scalability comprising a. 82. The method of claim 81 wherein The encoding step, In case of performing inter-layer coding using ROI information of lower layer, performing intra-layer coding on a block requiring interpolaton using pixels outside the ROI. An encoding method for providing spatial scalability, characterized in that. 82. The method of claim 81 wherein The encoding step, In case of performing motion prediction / compensation coding using ROI information of the same layer having temporal correlation, an integer is obtained for motion information of a block requiring interpolaton using pixels outside the ROI. Performing pixel-by-pixel prediction An encoding method for providing spatial scalability, characterized in that. 82. The method of claim 81 wherein The encoding step, In case of performing inter-layer coding using ROI information of a lower layer, performing interpolation after determining an external pixel value through extrapolation for a block requiring interpolaton using pixels outside the ROI. An encoding method for providing spatial scalability, characterized in that. 82. The method of claim 81 wherein The encoding step, When performing motion prediction / compensation coding using ROI information of the same layer having temporal correlation, extrapolation is performed on blocks requiring interpolaton using pixels outside the ROI. To interpolate after determining external pixel values An encoding method for providing spatial scalability, characterized in that. 86. The method of claim 84 or 85, The encoding step, Adding a flag to the encoded stream to indicate that there is an ROI region in the video image; A coding method that provides spatial scalability. 82. The method of claim 81 wherein The location information of the ROI may be represented by a macroblock scan number for the video image. A coding method that provides spatial scalability. 82. The method of claim 81 wherein Detecting a duplicate area OR between ROIs; The encoding step And inter-layer coding on the ROI using the OR. 89. The method of claim 88 wherein Configuring an intermediary region through upsampling with respect to a region that is not spatially overlapped with an upper layer of the ROI or the OR; The encoding step A coding method for providing spatial scalability, wherein the inter-layer coding is performed by using information of a region disposed in a layer below one step. An input step of receiving position information on a region to be decoded and resolution information to be decoded in an input video image; Extracting ROI information corresponding to the region to be decoded from the transmitted encoded stream; And And a decoding step of performing decoding on the ROI using the extracted information. 92. The method of claim 90, The decoding step, When performing interlayer decoding using lower layer ROI information, performing intra layer decoding on a block requiring interpolaton using pixels outside the ROI. A decoding method for providing spatial scalability, characterized in that. 92. The method of claim 90, The decoding step, When performing motion prediction / compensation decoding using ROI information of the same layer having a temporal correlation, an integer is obtained for motion information of a block requiring interpolaton using pixels outside the ROI. Performing pixel-by-pixel prediction A decoding method for providing spatial scalability, characterized in that. 92. The method of claim 90, The decoding step, In the case of performing inter-layer decoding using ROI information of a lower layer, performing interpolation after determining an external pixel value through extrapolation for a block requiring interpolaton using pixels outside the ROI. A decoding method for providing spatial scalability, characterized in that. 92. The method of claim 90, The decoding step, In case of performing motion prediction / compensation decoding using ROI information of the same layer having temporal correlation, extrapolation is performed on blocks requiring interpolaton using pixels outside the ROI. To interpolate after determining external pixel values A decoding method for providing spatial scalability, characterized in that. 95. The method of claim 93 or 94, The decoding step, Determining whether a ROI region in a video image exists in the encoded stream through a flag indicating that there is a ROI in the video image included in the encoded stream. Decoding Method Provides Spatial Scalability 92. The method of claim 90, The location information of the ROI may be represented by a macroblock scan number for the video image. Decoding method that provides spatial scalability. 92. The method of claim 90, The decryption step And performing interlayer decoding on the ROI using the overlapped region OR between the ROIs. 97. The method of claim 97, Configuring an intermediate region through upsampling with respect to an area that is not spatially overlapped with an upper layer of the ROI or the OR; The decryption step A decoding method for providing spatial scalability, wherein the inter-layer decoding is performed using information of a region disposed in a layer below one step.

Images