WO2011049392A2

WO2011049392A2 - Image encoding and decoding apparatus and method

Info

Publication number: WO2011049392A2
Application number: PCT/KR2010/007252
Authority: WO
Inventors: 송진한; 김수년; 임정연; 문주희; 이영렬; 김해광; 전병우; 한종기; 박형미; 박민철; 김동원; 김기백; 이주옥
Original assignee: 에스케이텔레콤 주식회사
Priority date: 2009-10-21
Filing date: 2010-10-21
Publication date: 2011-04-28
Also published as: WO2011049392A3

Abstract

The present invention provides an image encoding and decoding apparatus and method. According to one embodiment of the present invention, the image encoding and decoding apparatus comprises: an image encoder which reduces horizontal and vertical sizes of a variable macro block into the set size or ratio so that the variable macro block may be reset in a smaller size than the current size, if the variable macro block of the current encoding target is not positioned inside an input image, and performs a prediction encoding process on the basis of the reset variable macro block; and an image decoder which performs a decoding process by reducing the size of a variable macro block of the current decoding target into the set size or ratio so that the variable macro block may be reset in a smaller size than the current size, if the variable macro block of the current decoding target is not positioned inside an input image.

Description

Image encoding / decoding apparatus and method

The present invention relates to an image encoding / decoding apparatus and method. More specifically, an apparatus and method for efficiently encoding and decoding an outer portion of an input image when the size of the input image is not an integer multiple of the macroblock size. ) Are coded / decoded on the assumption of the mode that generates the least bits, or the part (block) that crosses the image boundary does not encode / decode data such as prediction information or transform information or uses variable macroblock size. The present invention relates to a method and an apparatus for encoding and decoding an image.

The contents described in this section merely provide background information on the embodiments of the present invention and do not constitute a prior art.

Standardized techniques for compressing video data currently include H.261, H.263, H.264, MPEG-2, and MPEG-4. According to the existing video compression technology, each image is divided into macroblocks of fixed size and encoded in macroblock units.

If the size of the input image is not an integer multiple of the size of the macroblock, the input image is padded such that the input image is an integer multiple of the macroblock, and then all luma components of each macroblock in the padded input image are padded. The chroma components are predicted spatially and temporally, and the predicted residuals are compressed through a process of transform, quantization, and entropy coding.

However, in a conventional video compression technique, if the size of the input image and the size of the macroblock are not integer multiples, the input image is padded to an integer multiple of the macroblock size, and then the type and prediction mode of the macroblock including the padded area Determination and encoding make it difficult to determine an optimal macroblock type and prediction mode, and encoding efficiency may be degraded because a bitstream including data of a padded region is generated.

In addition, since the decoding of the image is performed corresponding to the encoding of the image, it is difficult to expect high-efficiency decoding in a state where the compression efficiency for encoding of the high resolution image is deteriorated.

An embodiment of the present invention was devised to efficiently solve the above-described problem. When the size of an input image is not an integer multiple of the size of a macroblock of an arbitrary size, the boundary of an image is crossed when the outer portion is encoded and decoded. The parts (blocks) are coded / decoded on the assumption that they are the least-occurring mode, or the parts (blocks) crossing the image boundary do not encode / decode data such as prediction information or transform information, or macroblocks of variable size. The main purpose is to improve the compression efficiency and the reconstruction efficiency by encoding and decoding the image using the.

An image encoding / decoding apparatus according to an embodiment of the present invention for achieving the above object is to set the horizontal and vertical sizes of the variable macroblock when the variable macroblock of the current encoding target is not located inside the input image. An image encoder configured to reduce the size or the ratio to a size smaller than the current size and perform prediction encoding based on the reset variable macroblock; And when the variable macroblock of the current decoding target is not located inside the input image, the image decoding is performed by reducing the size of the variable macroblock of the current decoding target to a set size or ratio and resetting it to a size smaller than the current size. It is characterized by including a group.

According to an embodiment of the present invention for achieving the above object, the size of the horizontal and vertical size of the macroblock of the current encoding target, if the macroblock of the current encoding target does not reach a predetermined ratio or more A macroblock size setter configured to reset the size of the macroblock by reducing the ratio to a set size or ratio; A predictor for performing intra prediction or inter prediction on a macroblock of a current encoding target; And an encoder for transforming, quantizing, and entropy encoding the residual signal obtained by intra prediction or inter prediction.

According to another aspect of the present invention, when the macroblock of the current encoding target is not located inside the input image, the image encoding apparatus according to another embodiment of the present invention may be used. A macroblock size setter for resetting the size of the macroblock of the current encoding target to the largest size located; A predictor for performing intra prediction or inter prediction on a macroblock of a current encoding target; And an encoder for transforming, quantizing, and entropy encoding the residual signal obtained by intra prediction or inter prediction.

According to an embodiment of the present invention for achieving the above object, the size of the horizontal and vertical size of the macroblock of the current decoding target, if the macroblock of the current decoding target does not span more than a predetermined ratio to the input video A macroblock size setter configured to reset the size of the macroblock by reducing the ratio to a set ratio; A decoder which extracts and decodes prediction information including type information and transform coefficients of a macroblock to be decoded from the bitstream; A predictor for performing intra prediction or inter prediction on a macroblock to be decoded based on the type information; And a reconstruction unit for inversely quantizing and inversely transforming a transform coefficient to generate a residual signal, and generating a reconstructed macroblock by adding a residual signal generated to a value predicted by the predictor.

According to another aspect of the present invention, when the macroblock to be decoded currently is not located inside the input image, the image decoding apparatus according to another embodiment of the present invention may be used. A macroblock size setter for resetting the size of the macroblock to be decoded to the largest size to be located; A decoder which extracts and decodes prediction information including type information and transform coefficients of a macroblock to be decoded from the bitstream; A predictor for performing intra prediction or inter prediction on a macroblock to be decoded based on the type information; And a reconstruction unit for generating a residual signal by inversely quantizing and inversely transforming a transform coefficient, and generating a reconstructed macroblock by adding the residual signal generated to a value predicted by a predictor.

The video encoding / decoding method according to an embodiment of the present invention for achieving the above object is to set the horizontal and vertical sizes of the variable macroblock when the variable macroblock of the current encoding target is not located inside the input video. Reducing to a size or a ratio and resetting to a size smaller than a current size, and performing predictive encoding based on the reset variable macroblock; And when the variable macroblock of the current decoding target is not located inside the input image, reducing the size of the variable macroblock of the current decoding target by a set size or ratio and resetting the variable macroblock to a size smaller than the current size to perform decoding. It is characterized by including.

According to an embodiment of the present invention for achieving the above object, the size of the horizontal and vertical size of the macroblock of the current encoding target when the macroblock of the current encoding target does not reach a predetermined ratio or more. Resetting the size of the macroblock by reducing to a predetermined size or ratio; Performing intra prediction or inter prediction on a macroblock of a current encoding target; And transforming, quantizing, and entropy encoding the residual signal obtained by intra prediction or inter prediction.

Here, in the resetting step, the size of the macroblock may be reset by repeatedly reducing the horizontal and vertical sizes to a predetermined ratio until the macroblock of the current encoding target reaches a predetermined ratio or more.

According to another exemplary embodiment of the present invention for achieving the above object, when the macroblock of the current encoding target is not located inside the input image, the image encoding method may be performed within the input image. Resetting the size of the macroblock of the current encoding target to the largest size located; Performing intra prediction or inter prediction on a macroblock of a current encoding target; And transforming, quantizing, and entropy encoding the residual signal obtained by intra prediction or inter prediction.

Here, the size of the available macroblock is preferably an integer multiple of the size of the available transform.

According to an embodiment of the present invention for achieving the above object, the size of the horizontal and vertical size of the current macroblock to be decoded, if the current macroblock to be decoded does not span a predetermined ratio or more Resetting the size of the macroblock by reducing the ratio to a set ratio; Extracting and decoding prediction information including type information and transform coefficients of the macroblock to be decoded from the bitstream; Performing intra prediction or inter prediction on the current macroblock to be decoded based on the type information; And generating a residual signal by inversely quantizing and inversely transforming the transform coefficient, and generating a reconstructed macroblock by adding the residual signal generated to the predicted value.

According to another exemplary embodiment of the present invention for achieving the above object, when the macroblock to be decoded currently is not located inside the input image, the image decoding method may be performed in the input image. Resetting the size of the macroblock to be decoded to the largest size to be located; Extracting and decoding prediction information including type information and transform coefficients of the macroblock to be decoded from the bitstream; Performing intra prediction or inter prediction on the current macroblock to be decoded based on the type information; And inversely quantizing and inversely transforming the transform coefficient to generate a residual signal, and generating a reconstructed macroblock by adding the residual signal generated to the value predicted by the predictor.

As described above, according to the exemplary embodiment of the present invention, when the size of the input image is not an integer multiple of the size of the macroblock, the image is encoded and used by using the variable macroblock size when encoding and decoding the outer portion of the input image. By decoding, the compression efficiency and the recovery efficiency can be improved.

1 and 2 are diagrams for explaining image encoding and decoding when an input image is padded to be an integer multiple of a macroblock size according to a conventional encoding / decoding method.

3 and 4 are diagrams for explaining image encoding and decoding using variable size macroblocks according to an embodiment of the present invention.

5 is a flowchart illustrating a video encoding method according to an embodiment of the present invention.

6 is a flowchart illustrating an image decoding method according to an embodiment of the present invention.

7 is a flowchart illustrating an image encoding method according to another embodiment of the present invention.

8 is a flowchart illustrating an image decoding method according to another embodiment of the present invention.

9 is a diagram schematically illustrating an image encoding apparatus according to an embodiment of the present invention.

10 is a diagram schematically illustrating an image decoding apparatus according to an embodiment of the present invention.

Hereinafter, some embodiments of the present invention will be described in detail through exemplary drawings. In adding reference numerals to the components of each drawing, it should be noted that the same reference numerals are assigned to the same components as much as possible even though they are shown in different drawings. In addition, in describing the present invention, when it is determined that the detailed description of the related well-known configuration or function may obscure the gist of the present invention, the detailed description thereof will be omitted.

In addition, in describing the component of this invention, terms, such as 1st, 2nd, A, B, (a), (b), can be used. These terms are only for distinguishing the components from other components, and the nature, order or order of the components are not limited by the terms. If a component is described as being "connected", "coupled" or "connected" to another component, that component may be directly connected or connected to that other component, but between components It will be understood that may be "connected", "coupled" or "connected".

A video encoding apparatus (Video Encoding Apparatus), a video decoding apparatus (Video Decoding Apparatus) to be described below is a personal computer (PC), notebook computer, personal digital assistant (PDA), portable multimedia player (PMP) : Portable Multimedia Player (PSP), PlayStation Portable (PSP: PlayStation Portable), Mobile Communication Terminal (Mobile Communication Terminal), and the like, and may be used to encode a video or a communication device such as a communication modem for communicating with various devices or a wired or wireless communication network. It refers to various devices having various programs for decoding and a memory for storing data, a microprocessor for executing and controlling a program.

In addition, the image encoded in the bitstream by the video encoding apparatus is real-time or non-real-time through the wired or wireless communication network, such as the Internet, local area wireless communication network, wireless LAN network, WiBro network, mobile communication network, or the like, or a cable, universal serial bus (USB: Universal) The image decoding apparatus may be transmitted to a video decoding apparatus through a communication interface such as a serial bus, decoded by the video decoding apparatus, reconstructed, and played back.

An embodiment of the present invention describes a method of encoding / decoding an outer portion of an input image over a portion of a macroblock when the size of the input image is not an integer multiple of the size of the macroblock.

1 and 2 are diagrams for explaining image encoding and decoding when an input image is padded to an integer multiple of a macroblock size according to a conventional encoding / decoding method, and FIGS. 3 and 4 illustrate the present invention. FIG. Is a diagram for describing image encoding and decoding using a variable size macroblock according to an embodiment of the present invention.

In the case where the size of the input image is 1280x720 and the size of the macroblock is 64x64, the existing encoding / decoding method divides the input image into 1280x768, which is an integer multiple of the macroblock size as shown in FIG. Encoding and decoding are performed in units of macroblocks. Here, the 20 macroblocks located at the bottom are 64x16 that is included in the actual input image and 64x48 is a portion not included in the input image.

As another example, when the size of the input image is 1920x1080 and the size of the macroblock is 128x128, the conventional encoding / decoding method is as shown in FIG. 2 and pads the input image to 1920x1152, which is an integer multiple of the macroblock size, as shown in FIG. Encoding and decoding are performed by dividing into two macroblocks. Here, for the 15 macroblocks located at the lowest level, the 128x56 pixel area in each macroblock is a part included in the actual input image, and the 128x72 pixel area is an area belonging to the padded area.

If the horizontal size of the input image is not an integer multiple of the macroblock horizontal size or the vertical size of the input image is not an integer multiple of the macroblock vertical size, the macroblock is positioned at the boundary of the input image. In the example of FIG. 1 and FIG. 2, the macroblock is located at the lowermost boundary of the input image, and the macroblocks located at the lowermost end have a lot of padded area data than the actual image area in the macroblock. Can be determined. That is, the prediction mode and the macroblock type determined using only the actual image area may be different. In addition, since the bitstream is generated to include the data of the padded region, encoding and decoding performance may be degraded.

As a solution to this, a small sized macroblock such as 16x16 may be encoded, but since a high resolution image is efficiently encoded in a large block unit, a method of encoding / decoding using a small sized macroblock for the entire image is performed. Is not effective.

Therefore, in the embodiment of the present invention, a macroblock having a large size is used for the inside of the input image, and a macroblock having a variable size is used to minimize padding area data when encoding / decoding an outer portion of the input image. Effective encoding / decoding is performed on the outer portion of the input image while maintaining the encoding / decoding efficiency obtained by using the macroblock of. In this case, the size of the macroblock for encoding / decoding the outer portion of the input image can be obtained by using the same method as the encoder and the decoder if the size of the input image and the size of the macroblock are known, so additional information indicating the size of the macroblock is necessary. Not.

According to the video encoding / decoding method according to the embodiment of the present invention shown in FIG. 3, when the size of the input image is 1280x720 and the size of the macroblock is 64x64, the lowest 1280x16 area in which the input image spans a part of the macroblock is shown. For the encoding, the 16x16 macroblock is encoded / decoded so that data other than the input image is not used for encoding / decoding. In addition, the data encoded in the bitstream may use only pixel values belonging to the actual input image.

4 illustrates an example of another invention in which the size of an input image is 1920x1080 and the size of a macroblock is 128x128, the 1920x1024 pixel area of the input image is encoded / decoded using a macroblock having a size of 128x128. The size of the macroblock is reduced by a predetermined ratio or a predetermined size for the lowermost 1920x56 pixel area covering a part of the macroblock. After resetting the size of the macroblock, even when encoding using the reset macroblock, it is determined whether or not the part of the input image is covered by the macroblock and to what extent, and encoding using the reset macroblock. In the case where a portion of the input image that extends within the macroblock becomes a region of a predetermined size, for example, 1/2 or more, the encoding / decoding is performed using the reset macroblock. In this case, when the area of the macroblock that covers the input image is a region of a predetermined size, for example, 1/2 or more, and only a portion of the macroblock is covered, padding is performed and image encoding / decoding is performed.

Even when the macroblock of the reset size is used, if the portion of the input image that extends to the macroblock is smaller than an area of a predetermined size, for example, 1/2, the macroblock is resized again and encoded / decoded using the above-described method. Do this.

For example, in FIG. 4, it is determined whether the size of the macroblock should be reset when encoding / decoding the lowest 1920x56 pixel region of the input image using 1/2 as the reduction ratio of the macroblock. Since the area of the 128x128 macroblock is 128x56, which is smaller than 1/2 of the macroblock, the size of the macroblock is reset using a predetermined reduction ratio, for example, 1/2. Since the area of the reset 64x64 macroblock is 64x56 and is 1/2 or more of the reset macroblock, the lowest region of the input image is encoded / decoded using a 64x64 macroblock. At this time, since the input image exists only for 64x56 pixels which are a part of the macroblock, padding is performed for the 64x16 pixel area.

According to the video encoding method of the embodiment of the present invention, the video encoder sets the size and shape of one or more usable macroblocks (S502). The size and shape of the usable macroblocks may be determined by the type or transformation of subblocks in the usable macroblocks or input by the user, or may be a defined candidate group input from other devices. In addition, the size and shape of the available macroblock may be one or more, such as 64x64, 32x32, 16x16, 8x8. Provided that the size of the available macroblocks must be a positive integer multiple of the size of the available transforms.

For example, the size of available macroblocks is determined by the size of the subblocks or transforms in the macroblocks, and the size of subblocks available for prediction is 4x4, 4x8, 8x4, 8x8, 16x8, 8x16, 16x16, If 16x32, 32x16, 32x32, 32x16, 32x32, 32x64, 64x32, 64x64 and the available transform sizes are 4x4, 8x8, 16x16, the size of the available macroblock may be the same as the type of subblocks in the macroblock. . That is, the size of the available macroblock may be 4x4, 4x8, 8x4, 8x8, 16x8, 8x16, 16x16, 16x32, 32x16, 32x32, 32x16, 32x32, 32x64, 64x32, 64x64.

The size and shape information of the macroblock usable herein may be included in sequence header, picture or slice header information, and may be encoded in a bitstream.

In addition, the size and shape of the available macroblocks may be determined internally according to the type of subblocks used for prediction and the type of subblocks used by the encoder and the decoder. In this case, the size and shape information of the usable macroblock is not encoded in the bitstream.

In addition, the size information of the input image is encoded in the sequence header, and the decoder extracts and decodes the encoded input image size information from the bitstream, and performs each macroblock decoding using the reconstructed size of the input image.

In the following embodiments, for convenience of description, a description of a process of encoding and decoding size information of an input image in a sequence header is omitted.

When the current encoding target macroblock is located in the input image when the input image is encoded in macroblock units using the size and shape of the available macroblocks promised by the encoder and the decoder (S504), the macroblock is divided into a plurality of subblocks. Predictive encoding of the current macroblock having a reference size is performed to generate encoded image data, and the encoding of the current macroblock to be encoded is terminated and the next macroblock encoding is started (S506).

If the current encoding target macroblock spans the input video, the size of the current encoding target macroblock is reset (S508). The method of resetting the size of the macroblock may reset the size of the macroblock by reducing the width and height of the current macroblock by a predetermined ratio, such as 1/2, and may be used in the input image among the available macroblock sizes. It can also be reset to the macroblock size of the largest size that allows the macroblock to be located. However, it should be one of the sizes of available macroblocks and smaller than the size of the current macroblock.

In addition, when resetting the macroblock size, the largest size that allows the macroblock to be positioned inside the input image among the available macroblock sizes may be determined as the macroblock size. Referring to the case of Figure 3 as an example. If the available macroblock sizes are 64x64, 64x32, 64x16, 64x8, 64x4, 32x64, 16x64, 8x64, 4x64, 32x32, 32x16, 32x8, 32x4, 16x32, 8x32, 4x32, 16x16, 16x8, 16x4, 8x16, 4x16, 8x8 For macroblocks of 8x4, 4x8, and 4x4 and the input image in the macroblock is 64x16, the size of the macroblock is reset. Among the available macroblock sizes, the macroblock sizes that allow the macroblock to be placed inside the input image are 64x16, 64x8, 64x4, 32x16, 32x8, 32x4, 16x16, 16x8, 16x4, 8x16, 4x16, 8x8, 8x4, 4x8. , 4x4, which is 64x16, the largest of which is the size of the current macroblock, and predictively encodes the current coded macroblock to generate coded image data, thereby ending the current macroblock encoding and determining the size of the macroblock as the reference macroblock size. After resetting to 64x64, the next macroblock coding starts.

If the available macroblock sizes are 64x64, 64x32, 64x16, 64x8, 64x4, 32x64, 16x64, 8x64, 4x64, 32x32, 32x16, 32x8, 32x4, 16x32, 8x32, 4x32, 16x16, 16x8, 16x4, 8x16, 4x16, 8x8 , 8x4, 4x8, 4x4 and 64x48 of the input image in the macroblock, the macroblock sizes that allow the macroblock to be located inside the input image are 64x32, 64x16, 64x8, 64x4, 32x32 , 32x16, 32x8, 32x4, 16x32, 8x32, 4x32, 16x16, 16x8, 16x4, 8x16, 4x16, 8x8, 8x4, 4x8, 4x4, the largest of which is 64x32 as the size of the current macroblock and the current encoded macroblock Predictively encode to generate encoded image data, end the current macroblock encoding, reset the macroblock size to 64x64 as the reference macroblock size, and start the next macroblock encoding.

When the size of the available macroblocks is 64x64, 32x32, 16x16, 8x8, 4x4 and 1/2 is used as a predetermined size and ratio for resetting the size of the macroblock, the encoding method will be described with reference to FIG. 3 as follows. .

First, set 64x64, 32x32, 16x16, 8x8, and 4x4 as size information of available macroblocks. It is determined whether the current macroblock to be encoded is located inside the input image, and when the macroblock is located inside the input image, the image data is generated by encoding in 64x64 units. In the case of a 64x64 macroblock located at the bottom of the input image, since the macroblock is not located inside the input image and only partially covers the input image, after resetting the macroblock size to 32x32, the size of the reset current macroblock is set to the input image. Re-determine whether it is located inside or only part of the video. Even for 32x32 sized macroblocks, only a portion of the input image is reset, so reset the size of the macroblock to 16x16, which is half the size of the existing horizontal and vertical sizes, and then reset the size of the current macroblock within the input image. It is determined whether only part of the input image is covered. Since the macroblock having the reset 16x16 size is included in the input image, the size of the current macroblock is determined to be 16x16, the current encoded macroblock is predictively encoded to generate encoded image data, and the current macroblock encoding is terminated. Start encoding.

According to the image decoding method of the embodiment of the present invention, the image decoder sets one or more usable macroblock sizes (S602). The size of the usable macroblock may be determined by the size of the usable subblock or the type of transform, or may be set by extracting usable macroblock size information from the bitstream. It may also be a predetermined candidate group input from other devices. In this case, a method for setting a usable macroblock size uses a method promised with an encoder among the aforementioned methods.

If the size of the available macroblock is determined by the information extracted from the bitstream, the information is extracted from a position promised with an encoder such as a sequence header, a picture header, a slice header, or a macroblock header, and then decoded and set.

In addition, when the size of the usable macroblock is determined by the type of subblock used for prediction or transformation, the decoder internally sets the usable macroblock size information in a manner promised by the encoder.

After extracting and decoding the sequence header and the macroblock header from the bitstream, the size and shape of the available macroblocks are set using the method promised by the encoder, and then the current in the input image using the current decoding target macroblock number. The position of the decoding target macroblock is calculated (S604). If the current decoding target macroblock is located inside the input image (S606), the macroblock data encoded from the bitstream is extracted and decoded using the reference macroblock size, and the macroblock decoding is performed using the decoded macroblock data. It performs (S608). If the current decoding target macroblock is not located inside the input (S606), after resetting the macroblock size using the method promised with the encoder, the current decoding target macroblock is input using the reset macroblock size. After determining whether it is located inside the video or only part of the input video, if it is located inside the input video, macroblock decoding is performed using the reset macroblock, and if it is over the input video, the macroblock size is reset again. (S610).

7 is a flowchart illustrating a video encoding method according to another embodiment of the present invention.

According to the image encoding method of another embodiment of the present invention, after the image encoder sets one or more available macroblock sizes and shapes by using the above-described method (S702), the current encoding target macroblock is positioned inside the input image. In operation S704, the current macroblock divided into the plurality of subblocks is predictively encoded to generate encoded image data to end encoding of the current macroblock to be encoded, and then the next macroblock encoding is started (S710).

If the current macroblock to be encoded is not located inside the input video, but covers a predetermined area, for example, 1/2 or more (S704 and S706), the input video is recorded as much as the size of the region located outside the input video. After padding (S708), the current encoding target macroblock is predictively encoded to generate encoded image data to terminate encoding of the current encoding target macroblock, and the next macroblock encoding is started (S710).

If the current macroblock to be encoded covers a predetermined area, for example, an area smaller than 1/2 without being located inside the input image (S704, S706), the size of the macroblock is reset using the above-described method. Then, it is determined whether the current macroblock of the reset size is located inside the input image and how long it is if not located within the input image, and then the encoding or the input image using the macroblock of the current size described above is padded. After that, the encoding or macroblock size reset process using the macroblock of the current size is repeated (S712).

In the case of encoding the block including the portion or blocks that cross the input image boundary in the macroblock, for example, when encoding the macroblocks located at the bottom of FIG. 4, denotes the portions beyond the actual input image boundary in the macroblock, That is, the method of encoding macroblocks including a padded region is encoded / decoded on the assumption that the portions (blocks) crossing the image boundary in the macroblock are the least generated mode, or the data of these blocks are decoded. Do not encode / decode.

Hereinafter, a method of encoding a macroblock including portions (blocks) crossing an image boundary in a macroblock will be described.

Each macroblock is divided into a plurality of subblocks and intra prediction or inter prediction encoding is performed.

In this case, the subblocks located in the input image may be selected from the intra prediction mode or the inter prediction mode (that is, the motion vector and the reference picture information) in consideration of an error value between the encoding target block and the reconstruction block and the number of bits required to encode the encoding target block. Determine and encode the determined mode information into a bitstream, and reconstruct each block according to the determined mode information. The subblocks located outside the input image determine the intra prediction mode or the inter prediction mode (that is, the motion vector and the reference picture information) by considering only the number of bits necessary to encode the encoding target block, and encode the determined mode information into a bitstream. Each block is restored according to the determined mode information. That is, the current subblock is encoded in the mode having the least generation bits in consideration of only generation bits necessary for data encoding such as prediction mode information, transformation information, and transformation coefficient information during prediction encoding.

Alternatively, the subblocks located in the input image may be encoded using the above-described method, and the subblocks located outside the input image may be an intra prediction mode or an inter prediction mode (that is, a motion vector) in consideration of only the number of bits necessary for encoding the encoding target block. And reference picture information), and encodes the determined mode information into a bitstream and reconstructs each block by using a method of padding the input image when generating the reconstructed block. That is, when padding is performed by filling 128 pixel values of portions crossing the image boundary when the input image is padded, all the pixel values of the reconstructed blocks of the subblocks located outside the input image are 128.

In addition, the subblocks located in the input image are encoded using the above-described method, and data such as prediction information, transform information, or transform coefficient information, etc. of subblocks located outside the input image are encoded into a bitstream and input when generating a reconstructed block. Each block may be reconstructed using a method of padding an image.

The above-described method of encoding a macroblock including portions (blocks) crossing an image boundary in the macroblock may be used in combination with an existing encoding method.

First, the image decoder sets the size of one or more usable macroblocks using the method promised with the encoder (S802). If the size of the available macroblock is determined by the information extracted from the bitstream, the information is extracted from a position promised with an encoder such as a sequence header, a picture header, a slice header, or a macroblock header, and then decoded and set.

Alternatively, when the size of the usable macroblock is determined by the type of subblock used for prediction or transformation, the decoder internally sets the usable macroblock size information in a manner promised by the encoder.

After extracting and decoding the sequence header and the macroblock header from the bitstream, the size and shape of the available macroblocks are set using the method promised by the encoder, and then the current in the input image using the current decoding target macroblock number. The position of the decoding target macroblock is calculated (S804).

If the current decoding target macroblock is located inside the input video or not within the input video, and covers a predetermined area, for example, 1/2 or more (S806), the encoded macroblock data is extracted and decoded from the bitstream. Then, macroblock decoding is performed using the decoded macroblock data (S808).

If the current decoding target macroblock is not located inside the input image and covers a predetermined area, for example, an area smaller than 1/2 (S806), after resetting the size of the macroblock using the above-described method, After determining how much the current macroblock of the reset size extends to the input image, the macroblock size is reset again or macroblock decoding is performed on the macroblock of the currently set size (S810).

Hereinafter, an apparatus for encoding and decoding an image in units of blocks will be described as an example. Here, the block may be a macroblock of size MxN (where M and N may be an integer greater than or equal to 16) or a subblock or subblock of size OxP (where O and P are integers less than or equal to M or N). For example, encoding and decoding in units of blocks is exemplary, and an image may be encoded and decoded into a standardized area or an unstructured area such as a block. However, the video encoding / decoding apparatus described below may use blocks of any size, but the size of the blocks is the size promised by the video encoding apparatus and the video decoding apparatus.

9 is a block diagram schematically illustrating an image encoding apparatus according to an embodiment of the present invention.

An image encoding apparatus according to an embodiment of the present invention is an apparatus for encoding an image using a macroblock of 16x16 or larger size, and includes a macroblock size setter 902, a predictor 903, an encoder 910, and a reconstructor ( 912, filter 914, and frame memory 916. Here, the reconstructor 912, the filter 914, and the frame memory 916 may be selectively omitted or included in other components according to an implementation method.

The macroblock size setter 902 resets the size of the macroblock when the current encoding target macroblock is not located inside the input image.

The method of resetting the size of the macroblock is only when the current macroblock to be encoded is not located inside the input image, while reducing the horizontal and vertical sizes of the current macroblock by a predetermined ratio, for example, 1/2. The size of may be reset or may be reset to the macroblock size of the largest size that allows the macroblock to be positioned inside the input image among the available macroblock sizes.

Alternatively, when the macroblock to be encoded currently covers a predetermined area, for example, an area smaller than 1/2 without being located inside the input image, the size of the macroblock may be reset using the above-described method.

The predictor 903 may include a motion estimator 904, a motion compensator 906, and an intra predictor 908, and predict a macroblock of an input image. Here, the macroblock refers to a macroblock of 16x16 or more size (ie, an MxN sized macroblock, provided that M and an integer of N≥16).

The motion estimator 904 generates a motion vector by estimating the motion of the macroblock by comparing the macroblock to be predicted with the reference picture stored in the frame memory.

The motion compensator 906 refers to a motion vector generated by the motion estimator 904 to obtain a block corresponding to the size of the macroblock to be predicted from the reference picture stored in the frame memory. The macroblock obtained by the motion compensator 906 becomes a prediction macroblock having a prediction value of the macroblock to be predicted.

The intra predictor 908 intra predicts a macroblock to be predicted. To this end, the intra prediction unit 908 generates a reference block by using the neighboring pixel information that is already encoded, decoded, and reconstructed, compares the reference block with the macroblock to be encoded, and determines an intra prediction mode, and determines the intra prediction mode. Therefore, the macroblock is intra predicted. The macroblock predicted by the intra prediction unit 908 becomes a prediction macroblock having a prediction value of the target macroblock.

The encoder 910 encodes a residual signal that is a difference between pixel values of the target macroblock and the predictive macroblock. The encoder 910 may transform, quantize, and entropy encode the residual signal. In addition, when inter prediction of a target macroblock to be encoded, the encoder 908 may encode motion information such as a motion vector generated by the motion estimation unit 904 and macroblock mode information such as a size of a macroblock. have. When the encoder intra prediction the target macroblock to be encoded, the encoder may encode prediction mode information such as an intra prediction mode and macroblock mode information such as a size of the macroblock.

The reconstructor 912 reconstructs the target macroblock by inversely quantizing and inversely transforming the changed and quantized residual signal and adding the prediction macroblock output from the predictor.

The filter 914 filters the reconstructed target macroblock using a filter such as a deblocking filter. The filtered reconstructed macroblock is stored in the frame memory 916 and used by the predictor 903 to predict the next macroblock or macroblock of the next picture.

10 is a block diagram schematically illustrating an image decoding apparatus according to an embodiment of the present invention.

An image decoding apparatus according to an embodiment of the present invention includes a macroblock size setter 1001, a decoder 1002, a predictor 1003, a reconstructor 1008, a filter 1010, and a frame memory 1012. Can be configured.

The macroblock size setter 1001 obtains the position of the decoding target macroblock by using the current decoding target macroblock number and resets the size of the macroblock when the decoding target macroblock is not located inside the input image. In this case, the macroblock number can be calculated by increasing the number of macroblocks internally decoded by the decoder by 1, and the size of the input image can be obtained by extracting and decoding the sequence header from the bitstream.

The method of resetting the size of the macroblock is only when the current macroblock to be decoded is not located inside the input image, while reducing the horizontal and vertical sizes of the current macroblock by a predetermined ratio, for example, 1/2. The size of may be reset or may be reset to the macroblock size of the largest size that allows the macroblock to be positioned inside the input image among the available macroblock sizes.

Alternatively, when the current decoding target macroblock covers a predetermined area, for example, an area smaller than 1/2, without being located inside the input image, the size of the macroblock may be reset using the above-described method.

The decoder 1002 extracts three types of information required for macroblock decoding from the input bitstream.

First, macroblock type information and subblock mode information indicating whether a macroblock to be currently decoded is an inter macroblock or an intra macroblock and a subblock mode of the macroblock are extracted by entropy decoding.

Second, information necessary for prediction is extracted by entropy decoding. In this case, the type of prediction data to be decoded and the method of decoding the prediction data vary depending on whether each block is an intra block or an inter block. When the block to be restored is an inter macro block, a reference picture required for motion compensation of each subblock from the bitstream Information about a motion such as information and a motion vector is extracted and decoded. In the case of an intra block, information about an intra prediction mode of a luminance component and a chrominance component is extracted and decoded from a bitstream.

Finally, the information necessary for decoding the residual signal is decoded. First, information indicating whether a non-zero transform coefficient is present in each subblock (for example, CBP) is decoded. Decode the transform information and the quantized transform coefficients.

In this case, when the encoder does not encode data of blocks crossing the image boundary in the macroblock into a bitstream, the decoder 1002 calculates the position of each subblock in the macroblock and whether the current subblock crosses the image boundary. After determining, the mode information of the subblock, information necessary for prediction, and information necessary for decoding the residual signal are not decoded for the subblock crossing the image boundary (that is, the decoder does not encode the bitstream into the bitstream. Extraction and decryption from

The predictor 1003 predicts a current block to be currently decoded, and may include a motion compensator 1004 and an intra predictor 1006. If the current block is an inter block, the motion compensator 1004 predicts by bringing pixels of the size of the current macroblock from the reference picture stored in the frame memory using the motion vector decoded and decoded by the decoder 1002. Create a macroblock. If the current block is an intra block, the intra predictor 1006 predicts the current macroblock according to the intra prediction mode decoded and reconstructed by the decoder 1002 to generate a predicted macroblock.

In this case, the encoder determines the intra prediction mode or the inter prediction mode (that is, the motion vector and the reference picture information) by considering only the number of bits required as the encoding method of blocks crossing the image boundary in the macroblock, and encodes the determined mode information into the bitstream. In the case of reconstructing each block using a method of padding the input image when generating the reconstructed block, the predictor 1006 calculates the position of each subblock in the macroblock and determines whether the current subblock crosses the image boundary. After the determination, the subblock crossing the image boundary is generated without using the mode information extracted from the bitstream by using the padding method promised with the encoder.

Also, even when the encoder does not encode data of blocks crossing the image boundary in the macroblock into a bitstream, it is determined whether the current subblock crosses the image boundary, and then the padding promised to the encoder for the subblock crossing the image boundary. The prediction block is generated using the method.

The decompressor 1008 inversely quantizes the quantized transform coefficients decoded by the decoder, and inversely transforms the dequantized transform coefficients using the transform type extracted and reconstructed by the decoder 1002 to generate a residual signal. The reconstructed macroblock is generated by adding the predicted macroblock generated by the predictor to the received residual signal. The generated reconstruction macroblock is filtered by the filter 1010 and stored in the frame memory 1012 and used to reconstruct the next macroblock or the next picture.

In this case, the encoder determines the intra prediction mode or the inter prediction mode (that is, the motion vector and the reference picture information) by considering only the number of bits required as the encoding method of blocks crossing the image boundary in the macroblock, and encodes the determined mode information into the bitstream. In the case of reconstructing each block using a method of padding an input image when generating a reconstructed block, or when the encoder does not encode data of blocks that cross the image boundary in a macroblock into a bitstream, the current subblock is an image. After determining whether to cross the boundary, the predicted block generated by using the padding method promised with the encoder is generated as a reconstructed block for the subblock crossing the image boundary. (I.e. no residual signal is assumed)

The above-described method of decoding a macroblock including portions (blocks) crossing an image boundary in the macroblock may be used in combination with an existing decoding method.

In the above description, it is described that all the components constituting the embodiments of the present invention are combined or operated in one, but the present invention is not necessarily limited to these embodiments. In other words, within the scope of the present invention, all of the components may be selectively operated in combination with one or more. In addition, although all of the components may be implemented in one independent hardware, each or all of the components may be selectively combined to perform some or all functions combined in one or a plurality of hardware. It may be implemented as a computer program having a. Codes and code segments constituting the computer program may be easily inferred by those skilled in the art. Such a computer program may be stored in a computer readable storage medium and read and executed by a computer, thereby implementing embodiments of the present invention. The storage medium of the computer program may include a magnetic recording medium, an optical recording medium, a carrier wave medium, and the like.

In addition, the terms "comprise", "comprise" or "having" described above mean that the corresponding component may be inherent unless specifically stated otherwise, and thus excludes other components. It should be construed that it may further include other components instead. All terms, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. Terms commonly used, such as terms defined in a dictionary, should be interpreted to coincide with the contextual meaning of the related art, and shall not be interpreted in an ideal or excessively formal sense unless explicitly defined in the present invention.

The above description is merely illustrative of the technical idea of the present invention, and those skilled in the art to which the present invention pertains may make various modifications and changes without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention but to describe the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of the present invention.

As described above, in the embodiment of the present invention, when the size of the input image is not an integer multiple of the size of the macroblock of any size, the image is encoded and used by using a variable size macroblock when encoding and decoding the outer part of the image. It is a very useful invention to generate an effect that can improve the compression efficiency and the recovery efficiency by decoding.

CROSS-REFERENCE TO RELATED APPLICATION

This patent application is filed with the patent application number 10-2009-0100443 filed in Korea on October 21, 2009, and the patent application number 10-2010-0035446 filed on April 16, 2010, April 16, 2010. Patent Application No. 10-2010-0035456, April 16, 2010 Patent Application No. 10-2010-0035536, Patent Application No. 10-2010-0103075, filed October 21, 2010 If a priority is claimed in accordance with US Patent § 119 (a) (35 USC § 119 (a)), all of which is incorporated by reference in this patent application. In addition, if this patent application claims priority for the same reason as above for a country other than the United States, all the contents thereof are incorporated into this patent application by reference.

Claims

In the video encoding / decoding device,

When the variable macroblock of the current encoding target is not located inside the input image, the horizontal and vertical sizes of the variable macroblock are reduced to a set size or ratio and reset to a size smaller than the current size. An image encoder for performing predictive encoding based on the block; And

When the variable macroblock of the current decoding target is not located inside the input image, the image decoding is performed by reducing the size of the variable macroblock of the current decoding target to a set size or ratio and resetting it to a size smaller than the current size to perform decoding. group

Image encoding / decoding apparatus comprising a.
In the video encoding / decoding device,

In the case of encoding including a portion or blocks beyond the boundary of the input image in the macroblock to be encoded, the portion (block) crossing the image boundary is encoded / decoded on the assumption that the generation bit is the mode with the least generation bit. A video encoding / decoding device.
In the video encoding / decoding device,

In the case where the encoding includes a portion or blocks crossing the boundary of the input image in the macroblock to be encoded, the portion (block) crossing the image boundary omits encoding / decoding of data including prediction information or transform information. An image encoding / decoding apparatus, characterized in that.
In the video encoding apparatus,

If the macroblock of the current encoding target does not reach a predetermined ratio or more, the macroblock size setter resets the size of the macroblock by reducing the horizontal and vertical sizes of the macroblock of the current encoding target to a predetermined size or ratio. ;

A predictor for performing intra prediction or inter prediction on the macroblock of the current encoding target; And

An encoder that transforms, quantizes, and entropy encodes a residual signal obtained by the intra prediction or the inter prediction.

An image encoding apparatus comprising a.
In the video encoding apparatus,

If the macroblock of the current encoding target is not located inside the input image, the macro which resets the size of the macroblock of the current encoding target to the largest size among the available macroblocks located inside the input image. Block size setter;

A predictor for performing intra prediction or inter prediction on the macroblock of the current encoding target; And

An encoder that transforms, quantizes, and entropy encodes a residual signal obtained by the intra prediction or the inter prediction.

An image encoding apparatus comprising a.
In the video decoding apparatus,

A macroblock size setter configured to reset the size of the macroblock by reducing the horizontal and vertical sizes of the current macroblock to be decoded to a predetermined ratio when the macroblock to be decoded does not reach the input image more than a predetermined ratio;

A decoder which extracts and decodes prediction information including type information and transform coefficients of the current macroblock to be decoded from a bitstream;

A predictor for performing intra prediction or inter prediction on the macroblock of the current decoding target based on the type information; And

A reconstruction unit for generating a residual signal by inversely quantizing the transform coefficients and inversely transforming the residual coefficient, and generating a reconstructed macroblock by adding the residual signal generated to a value predicted by the predictor

Video decoding apparatus comprising a.
In the video decoding apparatus,

If the macroblock to be decoded currently is not located inside the input image, the macro which resets the size of the macroblock to be decoded to the largest size among the available macroblocks located inside the input image. Block size setter;

A decoder which extracts and decodes prediction information including type information and transform coefficients of the current macroblock to be decoded from a bitstream;

A predictor for performing intra prediction or inter prediction on the macroblock of the current decoding target based on the type information; And

A reconstruction unit for generating a residual signal by inversely quantizing the transform coefficients and inversely transforming the residual coefficient, and generating a reconstructed macroblock by adding the residual signal generated to a value predicted by the predictor

Video decoding apparatus comprising a.
In the video encoding / decoding method,

When the variable macroblock of the current encoding target is not located inside the input image, the horizontal and vertical sizes of the variable macroblock are reduced to a set size or ratio and reset to a size smaller than the current size. Performing predictive encoding based on the block; And

Performing decoding by resetting the size of the variable macroblock of the current decoding target to a size or a ratio smaller than the current size when the variable macroblock of the current decoding target is not located inside the input image.

Image encoding / decoding method comprising a.
In the video encoding method,

Resetting the size of the macroblock by reducing the horizontal and vertical sizes of the macroblock of the current encoding target to a predetermined size or ratio when the macroblock of the current encoding target does not reach a predetermined ratio or more;

Performing intra prediction or inter prediction on the macroblock of the current encoding target; And

Transforming, quantizing, and entropy encoding a residual signal obtained by the intra prediction or the inter prediction

Image encoding method comprising a.
The method of claim 9,

The reset step,

And resetting the size of the macroblock by repeatedly reducing the horizontal and vertical sizes to the set ratio until the macroblock of the current encoding target is over the predetermined ratio.
In the video encoding method,

If the macroblock of the current encoding target is not located inside the input image, resetting the size of the macroblock of the current encoding target to the largest size among the available macroblocks located inside the input image. ;

Performing intra prediction or inter prediction on the macroblock of the current encoding target; And

Transforming, quantizing, and entropy encoding a residual signal obtained by the intra prediction or the inter prediction

Image encoding method comprising a.
The method of claim 11,

And the size of the usable macroblock is a positive integer multiple of the size of the usable transform.
In the video decoding method,

Resetting the size of the macroblock by reducing the horizontal and vertical sizes of the current macroblock to be decoded to a predetermined ratio when the macroblock to be currently decoded does not reach a predetermined ratio over the input image;

Extracting and decoding prediction information including type information and transform coefficients of the current macroblock to be decoded from the bitstream;

Performing intra prediction or inter prediction on the current macroblock to be decoded based on the type information; And

Inversely quantizing and inversely transforming the transform coefficients to generate a residual signal, and generating a reconstructed macroblock by adding the residual signal generated to the value predicted by the prediction step

Image decoding method comprising a.
In the video decoding method,

Resetting the size of the macroblock of the current decoding target to the largest size located inside the input video among the available sized macroblocks if the macroblock of the current decoding target is not located inside the input video; ;

Extracting and decoding prediction information including type information and transform coefficients of the current macroblock to be decoded from the bitstream;

Performing intra prediction or inter prediction on the current macroblock to be decoded based on the type information; And

Inversely quantizing and inversely transforming the transform coefficients to generate a residual signal, and generating a reconstructed macroblock by adding the residual signal generated to the value predicted by the prediction step

Image decoding method comprising a.