KR100681920B1 - Method and apparatus for a spatial scalable encoding/decoding based on multiple reference frames - Google Patents

Abstract

A spatial scalable encoding/decoding method and apparatus referring to a plurality of frames are provided to improve a compression rate by allowing macroblocks constructing the same enhancement frame to individually select reference frames. A basic frame for watching basic texture and an enhancement frame for watching high definition texture are extracted for moving picture information. The basic frame is encoded macroblock by macroblock to generate basic frame information and the enhancement frame is encoded macroblock by macroblock to generate enhancement frame information. The basic frame information and the enhancement frame information are generated in a manner that macroblocks constructing the same enhancement frame individually select reference frames so that a single frame refers to a plurality of frames.

Description

Spatial scalable encoding and decoding method for referencing a plurality of frames, and apparatus therefor {Method and Apparatus for a Spatial Scalable Encoding / Decoding Based on Multiple Reference Frames}

1 is a schematic diagram of a digital broadcasting system;

2 is a schematic diagram of a spatial scalable encoding / decoding method for referring to a single frame according to the prior art;

3 is a view for explaining a frame structure used in the present invention,

4 is a schematic diagram of a spatial scalable encoding / decoding method for referring to a plurality of frames according to the present invention;

FIG. 5 is a diagram illustrating a process in which one frame refers to a plurality of frames in macroblock units; FIG.

6 is a block diagram of a spatial scalable encoding apparatus for referring to a plurality of frames according to the present invention;

7 is a configuration diagram of a spatial scalable decoding apparatus for referring to a plurality of frames according to the present invention;

8 is a diagram illustrating a process of transmitting frame information through a heterogeneous network.

<Description of Symbols for Main Parts of Drawings>

100, 110: digital broadcasting center 200: communication network

210: first communication network 220: second communication network

310: first terminal 320: second terminal

400: Frame 500, 510, 520, 530: Macroblock

600: encoding device 610: video information input unit

620: frame extraction unit 630: frame information generation unit

640: frame information output unit 700: decoding device

710: frame information input unit 720: frame generation unit

730: frame output unit

The present invention relates to a method and apparatus for spatially scalable encoding and decoding referring to a plurality of frames. More particularly, the present invention relates to a video for having spatial scalability for digital broadcasting services such as satellite digital multimedia broadcasting (DMB). A spatial scalable encoding and decoding method and apparatus for referencing a plurality of frames that can improve a compression rate by enabling reference to a plurality of frames when compressing information (data).

In general, since video information (data) has a large capacity, it is effectively compressed and transmitted or stored. As a standard of the video information (data) compression technology, moving picture MPEG (Moving Picture) of ISO / IEC (International Data Communication / International Electrotechnical Commission) Experts Group) and ITU-T (International Telecommunication Union-Telecommunication Session) H.26x series. In relation to digital broadcasting, "H.264 / MPEG4 part 10" proposed by JVT (Joint Video Team) jointly formed by "MPEG4 part2 video" or MPEG group of ISO / IEC and VCEG group of ITU-T. AVC "is mainly used.

Video compression technology takes advantage of the fact that the statistical characteristics of video data have a lot of spatial and temporal redundancy.In general, to reduce spatial redundancy, a discrete cosine transform (DCT) is performed on a frame-by-frame basis. In order to reduce temporal redundancy, differential pulse coded modulation (DPCM) is performed by searching for continuity and similarity between images from the reference image that has been encoded for each frame.

As encoding modes for video compression, there are an intra mode and an inter mode. Intra mode is a mode that converts and encodes based on continuity and similarity included in the original image itself. In addition, the inter mode is a mode for encoding based on continuity and similarity between images, which is a predictive mode that predicts motion from an existing image in time and compares it with the original image to encode a prediction error. It may be classified into a bi-predictive mode, which predicts a motion from an image existing before or after time, and compares it with an original image to encode a prediction error.

1 and 2, the outline of the conventional digital broadcasting system and the spatial scalable encoding / decoding technology will be described briefly. As shown in FIG. 1, in the digital broadcasting center 100, a satellite DMB network or a terrestrial DMB network is used. Digital broadcasting has been performed in such a manner that the video information is transmitted using the network 200 and the DMB receiving terminals 310 and 320 receive and reproduce the same. On the other hand, although not commercially available in current digital broadcasting, there is a technology for having spatial scalability in encoding / decoding so that resolution can be selected in video playback for high quality broadcasting.

Referring to FIG. 2, the frames constituting the video include a plurality of basic frames B '(0), ..., B' (n-4), B '(n-2), and B for basic quality viewing. '(n), B' (n + 2), ...) and multiple extended frames S '(0), ..., S' (n- 4), S '(n-2), S' (n), S '(n + 2), ...), and these basic frames and extended frames are encoded by encoding the basic frame information and extended frame information, respectively. Create In addition, during reproduction, spatial scalability is achieved by decoding only basic frame information or by decoding and combining both basic frame information and extended frame information.

Looking at the processing technique for the frame for this purpose, the first frame (B '(0)) of the basic frame is processed in the intra mode from the original image, the subsequent basic frames (B' (2), ..., B ' (n-4), B '(n-2), B' (n), B '(n + 2), ...) are mainly used in inter mode by comparing continuity and similarity with previous or subsequent frames. If the continuity and similarity with the processed or neighboring frames is low (e.g., when the scene is switched), it is processed in intra mode.

Meanwhile, separate extended frame information is generated for high-definition viewing with higher resolution than the reference frame. As shown in FIG. 2, the extended frame S '(n) generally corresponds to the previous basic frame B' (n). It is generated with reference to (inter mode processing) or with reference to previous extended frames S '(n-4) and S' (n-2) (inter mode processing).

However, the spatial scalable encoding / decoding technique according to the prior art has decided a frame to refer to extended frames on a frame basis in inter mode (eg, forward prediction mode, bidirectional prediction mode) processing. there was.

The present invention was devised to solve the above problems of the prior art, and an object of the present invention is to individually refer to macroblocks constituting the same extended frame when compressing moving picture information (data) to have spatial scalability. The present invention provides a spatial scalable encoding and decoding method and apparatus capable of improving a compression ratio by referring to a plurality of frames.

For this purpose, a spatially scalable encoding method for referring to a plurality of frames according to the present invention includes: a) a basic frame for viewing a basic quality of video information, and a resolution corresponding to the basic frame and having a resolution higher than the basic quality; Extracting an extended frame for high definition viewing; And b) generating basic frame information by encoding the basic frame in macroblock units and generating extended frame information by encoding the extended frame in macroblock units, wherein macroblocks constituting the same extended frame are individually referenced frames. And generating the basic frame information and extended frame information so that one frame refers to a plurality of frames by selecting.

In addition, the spatial scalable decoding method of referring to a plurality of frames according to the present invention includes: i) basic frame information capable of reproducing a basic frame for viewing a basic image quality, and a resolution corresponding to the basic frame and having a higher resolution than the basic image quality. Receiving extended frame information capable of playing extended frames for high quality viewing; And ii) generating a basic frame by decoding the basic frame information in macroblock units and generating an extended frame by decoding the extended frame information in macroblock units, wherein macroblocks constituting the same extended frame are individually referenced frames. The method may include generating a base frame and an extended frame by referring to a plurality of frames.

On the other hand, the spatial scalable encoding apparatus for referring to a plurality of frames according to the present invention, a video input unit for inputting video information; A frame extracting unit for extracting a basic frame for viewing a basic image from the video information and an extended frame corresponding to the basic frame and having a higher resolution than the basic frame; By generating the basic frame information by encoding the basic frame in macroblock units and generating the extended frame information by encoding the extended frame in macroblock units, macroblocks constituting the same extended frame are individually selected by reference frames. A frame information generation unit generating the basic frame information and the extended frame information so that one frame refers to a plurality of frames; And a frame information output unit configured to output the basic frame information and extended frame information.

In addition, the spatial scalable decoding apparatus referencing a plurality of frames according to the present invention includes basic frame information capable of reproducing a basic frame for viewing a basic image quality, and high definition viewing corresponding to the basic frame and having a higher resolution than the basic image quality. A frame information input unit configured to receive extended frame information capable of playing an extended frame for the mobile device; By generating the basic frame by decoding the basic frame information in macroblock units, and generating an extended frame by decoding the extended frame information in macroblock units, macroblocks constituting the same extended frame are individually selected by reference frames. A frame generator configured to generate the basic frame and the extended frame with reference to a plurality of frames by one frame; And a frame output unit for selectively outputting the generated basic frame or extended frame.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings and preferred embodiments. In the following description, the same or similar components are denoted by the same reference numerals as much as possible, and detailed descriptions of well-known functions and configurations that may unnecessarily obscure the subject matter of the present invention are omitted.

Prior to the detailed description, the terms used in the present invention are simply defined, and the term "basic quality" refers to a minimum quality (or standard quality) provided for watching a video. For example, the resolution and The frame rate is QVGA (320 x 240) and 15 fps, respectively. In addition, the image quality higher in resolution than the basic image quality is referred to as "high definition". Similarly, in the case of satellite DMB, the image quality such as VGA / 15fps, XGA / 15fps, SVGA / 15fps, and the like is referred to.

3 is a view for explaining a frame structure used in the present invention. A frame representing one screen (image) is divided into a plurality of macroblocks. In FIG. 3, four macroblocks MB_0 (500), MB_1 (510), BM_2 (520), and MB_3 (530) are one. An example of configuring the frame 500 is shown.

The main technical idea of the present invention is that video information (data) is encoded / decoded for each macroblock constituting a frame as described above, which will be described in detail below.

4 is a schematic diagram of a spatial scalable encoding / decoding method for referring to a plurality of frames according to the present invention.

The input or transmitted video information is extracted as a base frame for viewing a basic picture for spatial scalability and an extended frame corresponding to the base frame and having a higher resolution than the basic picture. These base frames form a so-called "Base Layer" and extension frames form a so-called "Enhancement Layer".

The first frame of the basic frame is generally processed in the intra mode from the original image, and subsequent frames are mainly processed in the inter mode by comparing the continuity and similarity with previous or subsequent frames, and the continuity and similarity with neighboring frames are compared. If it is low (eg when the scene is switched), it is processed in intra mode.

Although not shown in detail with reference to FIG. 4, assuming that B (0) is the first frame, B (0) is composed of four macroblocks, and each macroblock can be referred to as a frame (reference frame). Since it does not exist, it is processed in intra mode from the original image itself. On the other hand, B (n), which is the nth basic frame, is similarly composed of four macroblocks, and each macroblock has neighboring basic frames B (n-4), B (n-2), and B (n + 2). ) And continuity and similarity are compared with the macroblocks constituting the &lt; RTI ID = 0.0 &gt; etc., &Lt; / RTI &gt;

FIG. 5 illustrates a process in which one frame refers to a plurality of frames in macroblock units, and macroblocks constituting the n-th frame are compared with the macroblocks constituting the previous frame and continuity and similarity, respectively. The processing in the inter mode is shown with reference to the most similar macroblock. In FIG. 5, for example, a first macroblock of macroblocks constituting an nth frame refers to a macroblock constituting an n-2nd frame, and a second macroblock refers to a macroblock constituting an n-1th frame. For reference, the third macroblock refers to a macroblock constituting the n-4th frame, whereby macroblocks constituting the same frame (eg, the nth frame of FIG. 5) are individually selected by selecting a reference frame. Frames are encoded / decoded with reference to a plurality of frames.

To this end, in the basic frame information, frame information representing the position or order of the frame among all the basic frames, reference macroblock information representing a macroblock referred to by the macroblock during encoding / decoding, and reference macroblocks A motion vector for moving to a position and prediction error information for reconstructing the macroblock by performing motion compensation on the reference macroblock are included.

Meanwhile, separate extended frame information is generated for high-definition viewing with higher resolution than the basic picture quality. The extended frame provides the same screen (image) as the reference frame and corresponds one-to-one to the reference frame. For example, the extended frames S (n-4), S (n-2), S (n), and S (n + 2) shown in FIG. 4 are the basic frames B (n-4) and B (n-2), respectively. ), B (n) and B (n + 2) correspond one-to-one, and the resolution is twice as wide and vertical respectively.

The extended frames forming the enhancement layer are processed in an inter mode from a base frame representing the same screen (picture) or a previous extended frame in time, and each extended frame is composed of macroblocks as in the base frame. For example, the extended frame S (n) is composed of four macroblocks, each of which refers to a macroblock constituting a basic frame B (n) representing the same picture (picture) or is located earlier in time. It is processed in inter mode with reference to the macroblock constituting the extended frame S (n-4) or S (n-2).

Similarly, in the extended frame information, frame information, reference macroblock information representing a macroblock referenced by the corresponding macroblock during encoding / decoding, and movement for moving the reference macroblock to the correct position so that extended frames can be sequentially formed. A vector and prediction error information for reconstructing the macroblock by performing motion compensation on the reference macroblock are included.

6 is a block diagram of a spatial scalable encoding apparatus for referring to a plurality of frames according to the present invention. The encoding apparatus 600 according to the present invention includes a video input unit 610 to which video information (data) is largely input, a frame extractor 620 which extracts a basic frame and an extended frame from the video information, and the basic frame and an extended frame. Frame information generation unit 630 for generating basic frame information and extended frame information by encoding the macroblock in macroblock units, and a frame information output unit 640 for outputting the basic frame information and extended frame information.

The moving picture information or moving picture data is information (data) generated by a video camera, a movie shooting device, or the like, and, in principle, refers to information that has not yet been compressed (encoded). However, even if data compressed (encoded) information is decompressed (decoded) when inputted to the video input unit 610 or information that can generate (restore) an original frame, the information is included in the video information according to the present invention. something to do.

The video information input to the video input unit 610 is transmitted to the frame extracting unit 620, and the frame extracting unit 620 corresponds to a basic frame for viewing a basic image from the video information and the basic frame, and the basic image quality. Extract the extended frame for higher resolution high quality viewing. The basic frame and the extended frame are transmitted to the frame information generator 630, and the frame information generator 630 encodes them in units of macroblocks to generate basic frame information and extended frame information. As described in detail above, macroblocks constituting the same frame at encoding select reference frames individually, so that frame information is generated with reference to a plurality of frames in one frame. On the other hand, the frame information generation unit 630, the basic frame information generation unit for generating the basic frame information by encoding the basic frame in macroblock units and the extended frame information for generating extended frame information by encoding the extended frame in macroblock units It is also preferably divided into a generation unit, and configured to generate basic frame information and extended frame information, respectively. Finally, the frame information output unit 640 outputs the basic frame information and the extended frame information generated in this way to the outside.

The detailed process of generating the basic frame information and the extended frame information from the basic frame and the extended frame may refer to the above-described encoding method, and thus redundant description thereof will be omitted.

So far, a method and apparatus for encoding spatial scalable referencing a plurality of frames have been described. Hereinafter, a method and apparatus for decoding basic frame information and extended frame information encoded by the above-described process will be described.

For convenience of description, a decoding method and apparatus for referring to a plurality of frames according to the present invention will be described with reference to FIG. 7.

The decoding apparatus 700 according to the present invention is largely composed of a frame information input unit 710, a frame generator 720, and a frame output unit 730.

The frame information input unit 710 receives or receives from the outside basic frame information capable of reproducing a basic frame for viewing a basic quality and extended frame information capable of reproducing an extended frame for a high definition viewing having a higher resolution than the basic quality. The received (input) basic frame information and extended frame information are decoded in units of macroblocks by the frame generator 720, and are restored to the basic frame and the extended frame before encoding. That is, during decoding, the frame is restored based on the reference macroblock information, the motion vector, and the prediction error information included in each frame information. In this case, the macroblocks constituting the same extended frame are individually separated by the reference macroblock information. By selecting a reference frame (exactly a macroblock), one frame is decoded with reference to a plurality of frames. The frame generator 720 is divided into a basic frame generator for generating basic frames by decoding basic frame information in macroblock units and an extended frame generator for generating extended frames by decoding extended frame information in macroblock units. It is also preferred that each of them be configured to generate a base frame and an extended frame separately.

Finally, the frame output unit 730 outputs a basic frame or an extended frame according to the image quality selection information input by the user.

Since the spatial scalable decoding method and apparatus for referring to the plurality of frames described above are to decode the basic frame information and the extended frame information generated in the spatial scalable encoding method and apparatus for referring to the plurality of frames described above, And details may refer to the contents of the encoding method and apparatus.

Meanwhile, as shown in FIG. 8, when the present invention is applied to digital broadcasting such as satellite DMB, basic frame information and extended frame information generated by the encoding apparatus are respectively transmitted to the user terminal through different communication networks. desirable. For example, basic frame information is transmitted through a first communication network specialized for digital broadcasting (eg, satellite DMB network, terrestrial DMB network, etc.) 210, and extended frame information is not a professional digital broadcasting network but is a commercially available system. By transmitting through the two communication networks (eg, wireless Internet, wired Internet, mobile communication network, etc.) 220, it is possible to reduce the load of the first communication network.

In addition, the DMB receiving terminals 310 and 320 receive only basic frame information according to its performance (function) or user selection to provide basic quality broadcasting or receive both basic frame information and extended frame information to broadcast in high quality. Can be provided.

Although the present invention has been described in detail with reference to preferred embodiments, it will be apparent to those skilled in the art that the present invention can be implemented in other specific forms without changing the technical spirit or essential features thereof. One embodiment is to be understood in all respects as illustrative and not restrictive.

In addition, the scope of the present invention is specified by the appended claims rather than the detailed description, and all changes or modifications derived from the meaning and scope of the claims and equivalent concepts are included in the scope of the present invention. Should be interpreted as

According to the present invention, when encoding / decoding video information so as to have spatial scalability, the macroblock constituting the same extended frame is determined by determining the frame to be referred to by the extended frame in units of macroblocks constituting the frame, rather than in units of frames. Each of them has an effect of further improving the compression ratio by selecting reference frames individually.

Claims (10)

a) extracting a basic frame for viewing a basic image with respect to the video information, and an extended frame corresponding to the basic frame and having a higher resolution than the basic frame; And b) generating basic frame information by encoding the basic frame in macroblock units, and generating extended frame information by encoding the extended frame in macroblock units, wherein macroblocks constituting the same extended frame are individually Generating the basic frame information and extended frame information such that one frame refers to a plurality of frames by selecting. The method of claim 1, and c) outputting the basic frame information and the extended frame information to different communication networks. The method according to claim 1 or 2, The basic frame information includes reference macroblock information, a motion vector, and prediction error information. The method according to claim 1 or 2, The extended frame information includes reference macroblock information, a motion vector, and prediction error information. Iii) receiving basic frame information for reproducing a basic frame for viewing a basic picture quality and extended frame information for reproducing an extended frame for high quality viewing corresponding to the basic frame and having a higher resolution than the basic frame; And Ii) generating a basic frame by decoding the basic frame information in units of macroblocks, and generating an extended frame by decoding the extended frame information in units of macroblocks, wherein macroblocks constituting the same extended frame each separately refer to a reference frame; And selecting one frame by referring to the plurality of frames to generate the base frame and the extended frame. A video input unit to which video information is input; A frame extracting unit for extracting a basic frame for viewing a basic quality from the video information, and an extended frame corresponding to the basic frame and having a higher resolution than the basic frame; By generating the basic frame information by encoding the basic frame in macroblock units and generating the extended frame information by encoding the extended frame in macroblock units, macroblocks constituting the same extended frame are individually selected by reference frames. A frame information generation unit generating the basic frame information and the extended frame information so that one frame refers to a plurality of frames; And And a spatial scalable encoding apparatus including a frame information output unit configured to output the basic frame information and extended frame information. The method of claim 6, The frame information generator is divided into a basic frame information generator for generating basic frame information by encoding the basic frame in macroblock units and an extended frame information generator for generating extended frame information by encoding the extended frame in macroblock units. Spatial scalable encoding apparatus for referencing a plurality of frames, characterized in that. A frame information input unit configured to receive basic frame information capable of reproducing a basic frame for viewing a basic quality and extended frame information corresponding to the basic frame and reproducing an extended frame for high quality viewing having a higher resolution than the basic frame; By generating the basic frame by decoding the basic frame information in macroblock units, and generating an extended frame by decoding the extended frame information in macroblock units, macroblocks constituting the same extended frame are individually selected by reference frames. A frame generator configured to generate the basic frame and the extended frame with reference to a plurality of frames by one frame; And A spatial scalable decoding apparatus for referring to a plurality of frames including a frame output unit for selectively outputting the generated basic frame or extended frame. The method of claim 8, The frame generator is divided into a basic frame generator for generating basic frames by decoding basic frame information in macroblock units and an extended frame generator for generating extended frames by decoding extended frame information in macroblock units. A spatial scalable decoding apparatus for referring to a plurality of frames. The method according to claim 8 or 9, The frame output unit refers to a plurality of frames, characterized in that for selectively outputting the basic frame or extended frame according to the image quality selection information.

Images