JP4660550B2 - Multi-layer video coding and decoding method, video encoder and decoder - Google Patents

Description

  The present invention relates to a multi-layer video coding algorithm, and more particularly to a multi-layer video coding algorithm that codes a predetermined resolution layer by a plurality of coding algorithms.

  With the development of information communication technology including the Internet, not only text and voice but also image communication is increasing. Since existing character-centric communication methods cannot satisfy the diverse needs of consumers, multimedia services that can accommodate various forms of information such as characters, video, and music are increasing. Since the amount of multimedia data is enormous, it requires a large-capacity storage medium and requires a wide bandwidth during transmission. For example, a 24-bit true color image having a resolution of 640 * 480 requires a capacity of 640 * 480 * 24 bits per frame, that is, about 7.37 Mbits of data. When this is transmitted at 30 frames per second, a bandwidth of 221 Mbit / sec is required, and a storage space of about 1200 Gbit is required to store a movie to be screened for 90 minutes. Therefore, it is essential to use a compression coding technique to transmit multimedia data including characters, video, and audio.

  The basic principle of compressing video data is the process of eliminating data duplication (redundancy). Consider spatial overlap where the same color or object repeats in the image, temporal overlap where there is little change between adjacent frames in the video frame, or insensitivity to frequencies with high human visual and perceptual capabilities Video data can be compressed by eliminating duplicated psychovisual vision.

FIG. 1 illustrates an environment where video compression is applied.
Video data is compressed by the video encoder 110. Currently known video compression schemes are MPEG-2, MPEG-4, H.264. 263, H.M. H.264 and the like, and such a compression method is based on DCT (Discrete Cosine Transform). Recently, research on scalable video coding based on wavelet transform has become active. The compressed video data is transmitted to the video decoder 130 via the network 120. The video decoder 130 restores the video data by decoding the compressed video data.

  If the video encoder 110 does not compress the video data so as not to exceed the bandwidth of the network 120, the video decoder 130 cannot decode the compressed video data. However, the communication bandwidth of the network 120 varies depending on the type of network. For example, the communication bandwidth when using Ethernet (registered trademark) is different from the communication bandwidth when using a wireless run. In addition, when using a cellular communication network, the communication bandwidth is very narrow. For this reason, research on a method capable of obtaining compressed video data of various bit rates from one compressed video data, in particular, scalable video coding is active.

  Scalable video coding refers to a video compression scheme that codes video data to have scalability. Scalability means the characteristic that various video sequences having different resolutions, frame rates and image quality can be restored from a bit stream obtained by compressing one video sequence. The temporal scalability can be obtained by an algorithm such as MCTF (Motion Compensation Temporal Filtering), UMCTF (Unconstrained MCTF), or STAR (Successive Temporal Application and Referencing). Spatial scalability can be obtained by the wavelet transform algorithm, but recently, research on multi-layer techniques is active. SNR (Signal to Noise Ratio) scalability can be obtained by quantization algorithms such as EZW, SPIHT, EZBC, EBCOT.

2 and 3 show examples of the multi-layer video stream structure.
Referring to FIG. 2, the multi-layer video encoder codes each layer according to an MPEG-4 AVC (Advanced Video Coding) algorithm having the highest coding efficiency known to date. In the MPEG-4 AVC algorithm, temporal overlap between frames is removed, and then a frame from which temporal overlap is removed is converted by a DCT (Discrete Cosine Transform) method and then quantized.

  Referring to FIG. 2, the hierarchies are different from each other in at least one of resolution, frame rate, and transmission rate. The video coding process codes the enhancement layer with reference to the coded base layer frame after coding the base layer frame having the lowest resolution, the frame rate, and the transmission rate. As described above, the AVC-based multi-layer video coding scheme has excellent coding efficiency because the technology based on AVC is used in each layer. In particular, if the intra prediction and deblocking technique used by the AVC algorithm is used, it is possible to remove most of the artifacts generated in the video coding process. Each layer is optimized from the viewpoint of rate-distortion. However, the generated bitstream is not flexible in scalability characteristics. A bit stream generated by a multi-layer AVC video coding scheme cannot easily realize fine unit scalability (hereinafter referred to as FGS), and cannot easily obtain combined scalability characteristics. . This is because each scalability is subordinate to each other. When there are many layers actually used, the multi-layer coding scheme of FIG. 2 must perform AVC coding for all layers.

  Referring to FIG. 3, after coding the base layer having the lowest resolution, the frame rate, and the transmission rate by the AVC method, the wavelet coding method is used to determine the highest resolution, the frame rate, and the image quality layer by referring to the coded base layer. Use and code.

  Referring to FIG. 3, since a layer having the highest resolution, frame rate, and transmission rate is coded by the wavelet coding method, a bit stream having sufficient scalability can be obtained. In addition, since the lowest resolution layer is coded according to the AVC method, the video decoder can reconstruct a video frame with satisfactory image quality at the minimum resolution by the bit stream.

  Comparing FIG. 2 and FIG. 3, although the bitstream of FIG. 2 is optimized for all layers, the scalability property is weak, and FIG. 3 shows a single wavelet-coded layer with good scalability properties. Therefore, the image quality is inferior because all the layers excluding the AVC layer having the lowest resolution must be reconfigured.

  The present invention has been devised to solve the problems described in detail, and an object of the present invention is to provide a multi-layer video coding method and decoding method, a multi-layer video encoder, and a multi-layer video encoder that are excellent in coding efficiency and scalability. The purpose is to provide a decoder.

  The objects of the present invention are not limited to the objects mentioned above, and further objects not mentioned can be clearly understood by those skilled in the art by the following.

  To achieve the object, a multi-layer video coding method according to an embodiment of the present invention includes: (a) coding a predetermined resolution video frame according to a first video coding scheme; and (b) the first video coding. Referencing a frame coded by a scheme, coding the video frame by a second video coding scheme at the same resolution as the resolution, and (c) a frame coded by the first and second video coding schemes Generating a bitstream containing.

  To achieve the object, a multi-layer video coding method according to another embodiment of the present invention includes: (a) down-sampling a video frame to generate a low-resolution video frame; and (b) the low-resolution video. Video coding a frame; (c) video coding the video frame with reference to the coded frame of the low resolution layer; and (d) coding the coded frame of the low resolution and high resolution layer. Generating a bitstream that includes: (b1) coding the low-resolution video frame according to a first video coding scheme; and (b2) coding according to the first video coding scheme. See frame To and a step of coding the second video coding scheme the low-resolution video frame.

  To achieve the above object, a multi-layer video coding method according to another embodiment of the present invention includes: (a) coding a video frame of a predetermined resolution layer according to a first video coding scheme; and (b) Coding the video frame with a second video coding scheme with the same resolution as the coded frame with reference to a frame coded with the first video coding scheme, and (c) coding of all resolution layers The steps (a) and (b) are cyclically performed from the low resolution layer to the high resolution layer in all resolution layers.

  To achieve the above object, a multi-layer video encoder according to an embodiment of the present invention includes a downsampler that downsamples a high-resolution video frame to generate a low-resolution video frame, and a low-sample that video-codes the low-resolution video frame. A resolution video encoding unit, a high-resolution video encoding unit that video-codes the high-resolution video frame with reference to a frame coded at a low resolution by the low-resolution video encoding unit, and A bit stream generation unit for generating a bit stream including frames coded in resolution, wherein the low resolution video encoding unit converts the low resolution video frame into a first video coding method; The coding, to generate the first video coding scheme frames the low-resolution coding by coding the second video coding scheme the video frame with reference to the coded frame at the same resolution as the resolution by.

  To achieve the above object, a multi-layer video decoding method according to an embodiment of the present invention includes: (a) extracting first and second coded frames from a bitstream; and (b) the first Decoding a coded frame according to a first video decoding scheme to reconstruct the first frame; and (c) second coded from the information with reference to the reconstructed first frame. Decoding a second frame with a second video decoding scheme at the same resolution as the reconstructed first frame.

  To achieve the above object, a multi-layer video decoding method according to another embodiment of the present invention includes: (a) extracting first and second coded frames from a bitstream; and (b) the first. Reconstructing a first frame by decoding one coded frame according to a first video decoding scheme; and (c) reconfiguring the second coded frame at the same resolution as the reconstructed first frame. Decoding with a second video decoding scheme to reconstruct a second frame; and (d) adding the reconstructed first and second frames to reconstruct a video frame.

  To achieve the above object, a multi-layer video decoding method according to another embodiment of the present invention includes: (a) extracting a low-resolution and high-resolution layer coded frame from a bitstream; ) Decoding the low-resolution layer coded frame to reconstruct a low-resolution layer frame; and (c) a high-resolution layer obtained from the information with reference to the reconstructed low-resolution layer frame. And reconstructing a high-resolution layer frame by decoding the coded frame of the low-resolution layer, wherein the low-resolution layer coded frame is coded by the first video coding scheme and the second video coding scheme. Frame composed of The step (b) includes (b1) reconstructing a first frame by decoding a frame coded according to the first video coding scheme according to a first video decoding scheme, and (b2) reconstructing the frame. And reconstructing a second frame (low resolution frame) by decoding the frame coded by the second video coding scheme with reference to the first frame by the second video decoding scheme.

  To achieve the above object, a multi-layer video decoding method according to another embodiment of the present invention includes: (a) extracting information about low-resolution and high-resolution layer coded frames from a bitstream; (B) decoding a low-resolution layer coded frame obtained from the information to reconstruct a low-resolution layer frame; and (c) referring to the reconstructed low-resolution layer frame, the information. And reconstructing a high-resolution frame by decoding a high-resolution layer coded frame obtained from the low-resolution layer coded frame and a second frame coded by a first video coding scheme. Coded by video coding scheme (B1) comprises (b1) reconstructing the first frame by decoding the frame coded by the first video coding scheme by the first video decoding scheme; and (b2). ) Decoding a frame coded by the second video coding scheme by a second video decoding scheme to reconstruct a second frame; and (b3) reconstructing the reconstructed first and second frames. In addition, reconstructing the low resolution video frame is included.

  In order to achieve the above object, a multi-layer video decoding method according to another embodiment of the present invention extracts information about coded frames of a low resolution and a high resolution layer from a bitstream, and the low resolution from the information. And a multi-layer video decoding method for reconstructing a video frame by decoding a coded frame of a high-resolution layer, wherein the coded video frame of each resolution layer is coded by a first video coding scheme. 1 frame and a second frame coded by the second video coding scheme, and (a) the first frame coded by the predetermined resolution layer is coded by the first video decoding scheme to reconstruct the first frame. Stee And (b) reconstructing a second frame by decoding a second frame coded in the resolution hierarchy with reference to the reconstructed first frame according to a second video decoding scheme. The steps (a) and (b) are cyclically performed in the order from the low resolution layer to the high resolution layer in all resolution layers.

  To achieve the above object, a multi-layer video decoding method according to another embodiment of the present invention extracts information about coded frames of a low resolution and a high resolution layer from a bitstream, A multi-layer video decoding method for reconstructing a video frame by decoding a coded frame of a high-resolution layer, wherein the coded video frame of each resolution layer is encoded by a first video coding scheme. A frame and a second frame coded by a second video coding scheme; (a) a first frame coded by a predetermined resolution layer is coded by a first video decoding scheme to reconstruct the first frame; Step And (b) reconstructing a second frame by decoding a second frame coded in the resolution hierarchy according to a second video decoding scheme, and (c) the reconstructed first and second Adding a frame to reconstruct the frame of the resolution hierarchy, and the steps (a), (b) and (c) are cyclically performed in the order of the low resolution hierarchy to the high resolution hierarchy in all resolution hierarchy. .

To achieve the above object, a multi-layer video decoder according to an embodiment of the present invention analyzes a bit stream to extract a coded frame of a high-resolution and low-resolution layer, and the low-resolution coding. A low-resolution video decoding unit for decoding the encoded frame, and a high-resolution video for decoding the coded frame of the high-resolution hierarchy with reference to the low-resolution frame reconstructed by the low-resolution video decoding unit A low-resolution video decoding unit that decodes a first coded frame constituting a coded frame of the low-resolution layer according to a first video decoding scheme; Referring to the first frame decoded by the decoding method, the second coded frame constituting the coded frame of the low resolution layer is decoded by the second video decoding method to obtain the low resolution frame. Reconfigure.
In addition, the specific matter of embodiment is contained in detailed description and drawing.

  According to the embodiment of the present invention, it is possible to code a predetermined resolution layer by a plurality of video coding schemes having different characteristics, and perform video coding with excellent scalability and video coding efficiency.

  Advantages and features of the present invention, and methods for achieving them will be apparent with reference to the embodiments described below in detail with reference to the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, and can be embodied in various different forms. The present embodiments merely complete the disclosure of the present invention, and are ordinary in the technical field to which the present invention belongs. It is provided to provide those skilled in the art with a full understanding of the scope of the invention and is defined only by the scope of the claims.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

FIG. 4 is a diagram illustrating a structure of a bitstream generated by multi-layer video coding according to an embodiment of the present invention.
In this embodiment, the bit stream has two layers for each resolution. One is a layer coded by the AVC method, and the other is a layer coded by the wavelet coding method. In this specification, AVC coding or AVC layer means coding or layer employing DCT transform and quantization of AVC algorithm, and wavelet coding method or wavelet layer means coding or layer employing wavelet transform and embedded quantization. . In order to make the generated bit stream have temporal scalability, all the AVC coding and wavelet coding adopt an algorithm having temporal scalability such as MCTF, UMCTF, or STAR.

  At each resolution, the AVC layer ensures coding efficiency in terms of the resolution-time-image quality of the AVC layer, while the wavelet layer ensures FGS characteristics. The pre-decoder can generate a bit stream having an image quality between the AVC layer and the wavelet layer by simply cutting out a part of the bit stream of the wavelet layer. In addition, the cutting process can be similarly applied to multiple layers.

  For example, in the bit stream of FIG. 4, the pre-decoder can generate a bit stream having an image quality of 32 to 64 kbp having a QCIF resolution. For this purpose, the predecoder may cut out all of the CIF and SD resolution layers and cut out part or all of the wavelet layer of QCIF resolution.

  An example of a video encoder for generating a multi-layer bitstream in this way will be described with reference to FIG. For convenience, a video encoder having two resolution hierarchies will be described, but this is exemplary.

FIG. 5 is a block diagram showing a configuration of a multi-layer video encoder according to an embodiment of the present invention. The multi-layer video encoder includes a down-sampler 550, an AVC coding unit 510 and a wavelet coding unit 520 for low-resolution layer video coding, an AVC coding unit 530 and a wavelet coding unit 540 for high-resolution layer video coding, A bit stream generation unit 560 that generates a bit stream is included.
Downsampler 550 downsamples the video frame to generate a low resolution video frame.

The multi-layer video encoder has two coding units for each resolution layer. One is an AVC coding unit, and the other is a wavelet coding unit. That is, for low-resolution layer coding, the multi-layer video encoder includes an AVC coding unit 510 and a wavelet coding unit 520, and for high-resolution layer coding, the multi-layer video encoder includes an AVC coding unit 530 and a wavelet coding unit 540. Including.
The bitstream generation unit 560 generates a bitstream including coded frames of a low resolution and a high resolution layer.

The process of generating the bitstream is as follows.
First, the downsampler 550 downsamples the video frame 500 to generate a low resolution video frame having half the resolution. The generated low resolution video frame is provided to the AVC coding unit 510 and the wavelet coding unit 520 in the low resolution layer, and the video frame 500 is provided to the AVC coding unit 530 and the wavelet coding unit 540 in the high resolution layer.

  The AVC coding unit 510 in the low resolution layer removes temporal overlap of the low resolution frame, performs DCT conversion on the low resolution frame from which temporal overlap has been removed, and quantizes and codes the low resolution frame that has been DCT transformed. For this, the AVC coding unit 510 includes a temporal filtering unit 511, a DCT transform unit 512, and a quantization unit 513. The low resolution hierarchy AVC coded data is provided to the low resolution wavelet coding process.

  The wavelet coding unit 520 of the low resolution layer refers to the AVC coded frame of the low resolution layer, removes temporal overlap of the low resolution frame, performs wavelet transform on the low resolution frame from which the temporal overlap is removed, Quantize and code the converted low-resolution frame. For this, the wavelet coding unit 520 includes a temporal filtering unit 521, a wavelet transform unit 522, and a quantization unit 523. The low resolution layer wavelet coded data is provided to the high resolution AVC coding process.

  The AVC coding unit 530 of the high resolution layer refers to the wavelet-coded frame of the low resolution layer, removes temporal duplication of the high resolution frame, performs DCT conversion on the high resolution frame from which temporal duplication is removed, and performs DCT The converted high-resolution frame is quantized and coded. For this, the AVC coding unit 530 includes a temporal filtering unit 531, a DCT conversion unit 532, and a quantization unit 533. The high resolution hierarchy AVC coded data is provided to the high resolution wavelet coding process.

  The wavelet coding unit 540 of the high resolution layer refers to the AVC coded frame of the high resolution layer, removes temporal overlap of the high resolution frame, performs wavelet transform on the high resolution frame from which the temporal overlap is removed, The converted high-resolution frame is quantized and coded. For this, the wavelet coding unit 520 includes a temporal filtering unit 521, a wavelet transform unit 522, and a quantization unit 523.

  The bitstream generation unit 560 generates a bitstream including an AVC-coded frame and a wavelet-coded frame in a low-resolution layer, and an AVC-coded frame and a wavelet-coded frame in a high-resolution layer. In addition to information about the coded frame, the bitstream further includes header information such as a sequence header, a GOP (Group Of Picture) header, and a frame header, and other information such as a motion vector obtained from a temporal filtering process.

  The generated bit stream is pre-decoded by a pre-decoder (not shown) and transmitted to the multi-layer video decoder. For example, for devices with small display screens, such as mobile phones and PDAs, the predecoder may truncate the high resolution layer in the bitstream to generate a bitstream that includes only the coded frames in the low resolution layer. it can. The predecoder can generate a bitstream having a low bit rate by cutting a part of the bitstream when the network condition is not good. On the other hand, when the required frame rate is small, the predecoder can cut out some frames from the bitstream to generate a bitstream having a low frame rate.

FIG. 6 is a flowchart illustrating a multi-layer video encoding process.
First, a video frame is input to the multi-layer video encoder (S610). The multi-layer video encoder downsamples the input video frame (S620). In one embodiment, the multi-layer video encoder utilizes an MPEG downsampler during the downsampling process. This is because the MPEG downsampler is softer in the down-sampled low resolution image than the wavelet downsampler known so far. However, this is exemplary and any filter capable of obtaining a downsampled image can be used. To generate a bitstream with three resolution layers, the multi-layer video encoder downsamples the input video frame to generate low resolution frames downsampled to 1/2 and 1/4 resolution. To generate a bitstream with four resolution layers, the multi-layer video encoder downsamples the input video frame to generate low resolution frames downsampled to 1/2, 1/4, and 1/8 resolution To do.

  The multi-layer video encoder codes a low-resolution video frame according to the AVC method (S630). Then, the low resolution video frame is coded by the wavelet coding method with reference to the video frame coded by the low resolution AVC method (S640). For example, after performing AVC coding so that the resolution is QCIF, the frame rate is 15 Hz, and the bit rate is 32 kbps, the AVC coded frame is referred to have the same resolution and frame rate, and the bit rate is 64 kbps. Wavelet coding is performed so that

  The multi-layer video encoder codes a low-resolution frame and then codes a high-resolution frame with reference to the coded frame of the low-resolution layer.

  Specifically, the multi-layer video encoder codes a high-resolution video frame according to the AVC method (S650). Then, the multi-layer video encoder codes a high-resolution video frame according to the wavelet coding method with reference to the video frame coded according to the AVC method (S660). For example, a multi-layer video encoder performs AVC coding so that the resolution is CIF, the frame rate is 30 Hz, and the bit rate is 256 kbps, and then the video frame coded by the QCIF resolution AVC method and the wavelet coding method are used. The wavelet coding process is performed so that the bit rate is 750 kbps with the CIF resolution and the frame rate of 30 Hz with reference to the video frame and the AVC coded frame of the CIF resolution.

  If video coding is completed for all resolutions, the multi-layer video encoder generates a bitstream using the coded video frames (S670).

  A specific example of a multi-layer video coding process will be described with reference to FIGS. For convenience, the case where two resolution layers are coded will be described, and the case where there are three or more resolution layers can be coded in the same manner.

First, the embodiment of FIG. 7 will be described.
The multi-layer video encoder downsamples the video frame 700 to generate a low resolution video frame 710.

  The multi-layer video encoder AVC codes the low-resolution video frame 710 to generate a low-resolution layer AVC-coded frame included in the bitstream. The multi-layer video encoder then decodes the AVC-coded frame of the low resolution layer to obtain a decoded frame 720 and obtains a low resolution difference frame 730 compared to the low resolution video frame 710.

  The multi-layer video encoder wavelet-codes the low-resolution difference frame 730 to generate a low-resolution layer wavelet-coded frame. The multi-layer video encoder then decodes the low-resolution layer wavelet-coded frame to obtain a decoded frame 740 that is combined with the decoded frame 720 to generate the low-resolution layer decoded video frame 750. obtain.

  The multi-layer video encoder upsamples the decoded video frame 750 of the low resolution layer and compares the upsampled frame 760 with the video frame 700 to obtain a high resolution layer frame 770. The multi-layer video encoder performs AVC coding on the high-resolution layer frame 770 to generate a high-resolution layer AVC-coded frame included in the bitstream. The multi-layer video encoder then decodes the AVC coded frame of the high resolution layer to obtain a decoded frame 780, and obtains a high resolution difference frame 780 compared with the high resolution layer frame 770.

  The multi-layer video encoder wavelet-codes the high-resolution difference frame 780 to generate a high-resolution layer wavelet-coded frame included in the bitstream.

  The multi-layer video encoder generates a bitstream including the generated low-resolution layer AVC-coded and wavelet-coded frames, and the high-resolution layer AVC-coded and wavelet-coded frames.

Next, the embodiment of FIG. 8 will be described.
First, the coding process will be described. The multi-layer video encoder generates a low-resolution video frame by down-sampling the high-resolution video frame. The multi-layer video encoder performs AVC coding on the low-resolution video frame, and wavelet-codes the video frame on the low-resolution layer with reference to the AVC-coded video frame on the low-resolution layer.

  First, when coding the Nth low-resolution video frame 812, the low-resolution video frame 811 and the video frame 813 are referred to. Frames referenced in open-loop video coding are low-resolution video frames 811 and 813, but frames referenced in closed-loop video coding are coded after AVC coding of low-resolution video frames 811 and 813. Refer to the reconstructed frame by decoding the received frame.

When the low resolution AVC coding is completed, the multi-layer video encoder performs the low resolution wavelet coding. When the multi-layer video encoder codes the frame 822, the multi-layer video encoder can refer to the frame 821 and the frame 823. However, the multi-layer video encoder can also refer to a frame reconstructed by decoding a frame obtained by AVC coding the frame 812.
When video coding of the low resolution layer is completed, the multi-layer video encoder performs video coding of the high resolution layer.

  When AVC coding the video frame 842 of the Nth high-resolution layer, the multi-layer video encoder can refer to the video frame 841 and the video frame 843, but can also refer to the video frame obtained by reconstructing the video frame 822. it can. For this purpose, the multi-layer video encoder generates a video frame 832 by up-sampling the video frame obtained by reconstructing the video frame 822 and refers to it.

  When the Nth high-resolution layer video frame 852 is wavelet-coded, the multi-layer video encoder can refer to a frame obtained by reconstructing the frames 851 and 853 and the frame 842.

  If the embodiment of FIG. 7 is compared with the embodiment of FIG. 8, in the embodiment of FIG. 7, reference is made between layers after temporal filtering is completed. In the embodiment of FIG. References between them are made. The embodiment of FIG. 7 shows good coding performance because there is a possibility that the spatial linkage is larger than the temporal linkage between frames when there is a lot of motion, but the frame is shown when there is little motion. Since the temporal association may be greater than the spatial association between, the embodiment of FIG. 8 may exhibit better coding performance.

Next, a process of assigning a bit rate at each layer will be described.
FIG. 9 is a diagram illustrating a process of assigning a bit rate of each layer in a multi-layer video coding process according to an embodiment of the present invention. For convenience, the description will be made based on the fact that there are three resolution layers (QCIF, CIF, SD layer).

  Scalability requirements for video coding are 96 to 192 kbps at a 15 Hz frame rate in the QCIF layer 930, 192 to 768 kbps at a frame rate of 7.5 to 30 Hz in the CIF layer 920, and 15 to 60 Hz frames in the SD layer 910 Description will be made assuming that the rate is 768-3072 kbps.

  First, video coding of the QCIF layer 930 will be described. The multi-layer video encoder performs AVC coding of the QCIF frame so as to have a bit rate of 96 kbps and a frame rate of 15 Hz. The QCIF frame is then wavelet coded with reference to the AVC frame coded to have a bit rate of 192 kbps and a frame rate of 15 Hz.

The video coding of the CIF layer 920 will be described.
The multi-layer video encoder performs AVC coding on the CIF frame so as to have a frame rate of 30 Hz, which is the maximum frame rate of the CIF layer 920. To reconstruct a video frame having a bit rate of 192 kpbs and a frame rate of 7.5 Hz, one of the AVC coded and wavelet coded frames of the QCIF layer 930 and one of the AVC coded frames of the CIF layer 920 Department is necessary.

  The multi-layer video encoder wavelet codes the CIF frame so as to have a frame rate of 30 Hz, which is the maximum frame rate of the CIF layer 920. To reconstruct the 384 kbps to 768 kbps frame of the CIF layer 920, one of the AVC coded frame and wavelet coded frame of the QCIF layer 930 and one of the AVC coded frame and wavelet coded frame of the CIF layer 920 Department is necessary.

The video coding of the SD layer 910 will be described.
The multi-layer video encoder performs AVC coding on the SD frame so as to have a frame rate of 60 Hz, which is the maximum frame rate of the SD layer 910. To reconstruct a video frame having a bit rate of 768 kpbs and a frame rate of 15 Hz, an AVC coded frame and wavelet coded frame of QCIF layer 930 and an AVC coded frame and wavelet coded of CIF layer 920 And a part of the AVC coded frame of the SD layer 910 is required.

  The multi-layer video encoder wavelet-codes the CIF frame so as to have a frame rate of 60 Hz which is the maximum frame rate of the SD layer 910. To reconstruct a 1536 kbps to 3072 kbps frame in SD layer 910, an AVC-coded and wavelet-coded frame in QCIF layer 930, an AVC-coded and wavelet-coded frame in CIF layer 920, and A portion of the SD layer 910 AVC coded frame and wavelet coded frame is required.

  The above description of multi-layer video coding is exemplary, and various multi-layer video coding can be implemented. 10 and 11 show an example of a bit stream structure different from the above-described embodiment.

  Unlike FIG. 4, the embodiment of FIG. 10 codes only one coding scheme in the SD layer. It easily reconstructs video frames with a relatively low bit rate (eg 1.5 Mbps) from a wavelet coded bitstream if it has a high resolution and a sufficient bit rate (eg 3.0 MHz bit rate) This is because it can be done.

  FIG. 12 is a block diagram showing a configuration of a multi-layer video decoder according to an embodiment of the present invention. For convenience of explanation, the description will be made with reference to a video decoder that reconstructs a video frame from a bit stream having two resolution layers.

  The multi-layer video decoder includes a bitstream analysis unit 1250, an AVC decoding unit 1210 and a wavelet decoding unit 1220 for decoding a coded video frame of a low resolution layer, and a coded video frame of a high resolution layer. An AVC decoding unit 1230 and a wavelet decoding unit 1240 for coding are included.

  The bitstream analysis unit 1250 extracts coded frames of the high resolution layer and the low resolution layer included in the input bitstream. The low resolution layer coded frames include AVC coded frames and wavelet coded frames, and the high resolution layer coded frames also include AVC coded frames and wavelet coded frames.

  The AVC decoding unit 1210 in the low resolution layer performs inverse quantization on the AVC-coded frame in the low resolution layer, performs inverse DCT conversion on the inversely quantized frame, and then performs inverse temporal filtering on the frame subjected to inverse DCT conversion. For this, the AVC decoding unit 1210 of the low resolution layer includes an inverse quantization unit 1211, an inverse DCT transform unit 1212, and an inverse temporal filtering unit 1213.

  The wavelet decoding unit 1220 in the low resolution layer refers to the video frame reconstructed by the AVC decoding unit 1210, dequantizes the wavelet coded frame in the low resolution layer, and performs inverse wavelet transform on the dequantized frame. After that, the inverse wavelet transformed frame is subjected to inverse temporal filtering. For this purpose, the wavelet decoding unit 1220 in the low resolution layer includes an inverse quantization unit 1221, an inverse wavelet transform unit 1222, and an inverse temporal filtering unit 1223.

  The AVC decoding unit 1230 of the high resolution layer refers to the video frame reconstructed by the wavelet decoding unit 1220 of the low resolution layer, and inversely quantizes the AVC coded frame of the high resolution layer, thereby dequantizing the frame. After the inverse DCT transform, the inverse DCT transform frame is subjected to inverse temporal filtering. For this, the AVC decoding unit 1230 of the high resolution layer includes an inverse quantization unit 1231, an inverse DCT transform unit 1232, and an inverse temporal filtering unit 1233.

  The wavelet decoding unit 1240 of the high resolution layer refers to the video frame reconstructed by the AVC decoding unit 1230, and inversely quantizes the wavelet coded frame of the high resolution layer, and inverse wavelet transforms the dequantized frame. After that, the inverse wavelet transformed frame is subjected to inverse temporal filtering. For this, the wavelet decoding unit 1240 of the high resolution layer includes an inverse quantization unit 1241, an inverse wavelet transform unit 1242, and an inverse temporal filtering unit 1243.

  Each component in the specification means a software or hardware component such as FPGA or ASIC, and each component plays a role. However, the components are not limited to software or hardware. Each component can be configured to reside on an addressable storage medium and can be configured to run one or more processors. Thus, by way of example, a component includes components such as software components, object-oriented software components, class components and task components, and processes, functions, attributes, procedures, subroutines, segments of program code, drivers, Includes firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functionality provided by such components can be combined with a smaller number of components and modules or further separated into additional components and modules. The components can also be implemented to run one or more computers for a video encoding and decoding system.

FIG. 13 is a flowchart illustrating a multi-layer video decoding process according to an embodiment of the present invention.
First, if a bit stream is input to the multi-layer video decoder, the multi-layer video decoder analyzes the bit stream (S1310). The multi-layer video decoder analyzes the bitstream and extracts high-resolution layer and low-resolution layer coded frames from the bitstream.

  After extracting the coded frame, the multi-layer video decoder decodes the low-resolution AVC layer by AVC decoding the low-resolution layer AVC-coded frame among the coded frames (S1220). The video frame reconstructed by decoding the low resolution AVC layer is provided to the decoding process of the low resolution wavelet layer.

  The multi-layer video decoder decodes the low-resolution wavelet layer with reference to the video frame reconstructed by the decoding process of the low-resolution AVC layer (S1330). In order to decode the low-resolution wavelet layer, the multi-layer video decoder refers to the video frame reconstructed by the decoding process of the low-resolution AVC layer, and the wavelet-coded frame of the low-resolution layer among the coded frames. Wavelet decoding. The video frame reconstructed by decoding the low resolution wavelet layer is provided to the decoding process of the high resolution AVC layer.

  The multi-layer video decoder decodes the high-resolution AVC layer with reference to the video frame reconstructed by the decoding process of the low-resolution wavelet layer (S1340). In order to decode the high-resolution AVC layer, the multi-layer video decoder refers to the video frame reconstructed by the decoding process of the low-resolution wavelet layer, and among the coded frames, the AVC-coded frame of the high-resolution layer Are AVC-decoded. The video frame reconstructed by decoding the high resolution AVC layer is provided to the decoding process of the high resolution wavelet layer.

  The multi-layer video decoder decodes the high-resolution wavelet layer with reference to the video frame reconstructed by the decoding process of the high-resolution AVC layer (S1350). In order to decode the high-resolution wavelet layer, the multi-layer video decoder refers to the video frame reconstructed by the decoding process of the high-resolution AVC layer and is wavelet-coded in the high-resolution layer among the coded frames. Wavelet decoding the frame.

  If the decoding process for all layers is completed, the multi-layer video decoder generates a video signal using the decoded video frame (S1360). The generated video signal is displayed via a display device.

  The embodiments and drawings disclosed in this specification are exemplary, and the technical idea of the present invention is not limited thereto. For example, in the detailed description, one resolution layer is composed of an AVC layer and a wavelet layer. However, this is only an example, and may be composed of two layers using other coding algorithms. Further, although one resolution layer is coded by two video coding schemes, it is also possible to code by a plurality of video coding schemes exceeding two. Therefore, the above description is exemplary, and the technical idea of the present invention is more clearly limited by the scope of the claims to be described later.

It is a figure which shows the environment where video compression is applied. It is a figure which shows the example of the structure of a multi-layer video bit stream. It is a figure which shows the example of the structure of a multi-layer video bit stream. FIG. 3 is a diagram illustrating a structure of a multi-layer video bitstream according to an embodiment of the present invention. It is a block diagram which shows the structure of the multi-layer video encoder by one Embodiment of this invention. 5 is a flowchart illustrating a multi-layer video coding process according to an embodiment of the present invention. FIG. 3 illustrates a multi-layer video coding process according to an embodiment of the present invention in more detail. FIG. 3 illustrates a multi-layer video coding process according to an embodiment of the present invention in more detail. FIG. 5 is a diagram illustrating a process of assigning a bit rate of each layer in a multi-layer video coding process according to an embodiment of the present invention. FIG. 6 is a diagram illustrating a structure of a multi-layer video bitstream according to another embodiment of the present invention. FIG. 6 is a diagram illustrating a structure of a multi-layer video bitstream according to another embodiment of the present invention. It is a block diagram which shows the structure of the multi-layer video decoder by one Embodiment of this invention. 4 is a flowchart illustrating a multi-layer video decoding process according to an embodiment of the present invention.

Claims (15)

(A) generating a low-resolution video frame by down-sampling a video frame of a predetermined resolution;
(B) coding the low-resolution video frame according to a first video coding scheme;
(C) referring to a low-resolution video frame coded according to the first video coding scheme, coding the low-resolution video frame according to a second video coding scheme different from the first video coding scheme;
; (D) a decoded frame obtained by decoding the low resolution video frame coded by the first video coding scheme, and decoding the low resolution video frame coded by the second video coding scheme Coding the video frame of the predetermined resolution with the first video coding scheme with reference to the obtained frame ;
(E) referring to a predetermined resolution video frame coded according to the first video coding scheme, coding the predetermined resolution video frame according to the second video coding scheme;
(F) generating a bitstream including a low resolution video frame coded according to the first and second video coding schemes and a predetermined resolution video frame coded according to the first and second video coding schemes. Multi-layer video coding method.   The multi-layer video coding method according to claim 1, wherein the first video coding scheme is an AVC coding scheme and the second video coding scheme is a wavelet coding scheme.   The multi-layer video coding method according to claim 1, wherein the first video coding scheme and the second video coding scheme perform a video coding process at the same frame rate. The step (c) includes:
(C1) decoding a low-resolution video frame coded according to the first video coding scheme;
(C2) obtaining a difference frame between the low resolution video frame and the frame decoded in the step (c1);
The multi-layer video coding method according to claim 1, further comprising: (c3) coding the difference frame according to the second video coding scheme. The step (d) includes:
(D1) decoding a differential frame coded according to the second video coding scheme;
(D2) obtaining a sum frame by combining the frame decoded in step (c1) and the frame decoded in step (d1);
(D3) upsampling the sum frame to generate a high resolution video frame;
(D4) obtaining a second difference frame between the video frame of the predetermined resolution and the high resolution video frame, and coding the second difference frame according to the first video coding method. Multi-layer video coding method. The step (e) includes:
(E1) decoding a predetermined resolution video frame coded according to the first video coding scheme;
(E2) obtaining a third difference frame between the video frame of the predetermined resolution and the frame decoded in the step (e1);
The multi-layer video coding method according to claim 1, further comprising: (e3) coding the third differential frame according to the second video coding scheme. The step (c) includes:
(C4) decoding a low-resolution video frame coded according to the first video coding scheme;
(C5) coding the low-resolution video frame according to the second video coding scheme;
The multi-layer video coding method according to claim 1, wherein the frame decoded by the step (c4) is referred to in the temporal filtering process of the step (c5). A downsampler that downsamples high resolution video frames to generate low resolution video frames;
A low-resolution video encoding unit for video-coding the low-resolution video frame;
A high-resolution video encoding unit that video-codes the high-resolution video frame with reference to a frame that is low-resolution coded by the low-resolution video encoding unit;
A bitstream generation unit that generates a bitstream including the frame coded at the low resolution and the frame coded at the high resolution;
The low-resolution video encoding unit codes the low-resolution video frame according to a first video coding scheme, refers to the low-resolution video frame coded according to the first video coding scheme, and converts the low-resolution video frame into the first resolution. Coding by the second video coding scheme different from the one video coding scheme to generate the low-resolution coded frame;
The high-resolution video encoding unit includes a decoded frame obtained by decoding a low-resolution video frame coded by the first video coding scheme, and a low-resolution video frame coded by the second video coding scheme. The high-resolution video frame is coded by the first video coding scheme, and the high-resolution video frame coded by the first video coding scheme is referred to. A multi-layer video encoder that codes a high-resolution video frame according to the second video coding scheme to generate a frame coded at the high resolution.   The multi-layer video encoder according to claim 8, wherein the first video coding scheme is an AVC coding scheme and the second video coding scheme is a wavelet coding scheme. (A) Extracting a high-resolution video frame and a low-resolution video frame coded by the first video coding scheme and a high-resolution video frame and a low-resolution video frame coded by the second video coding scheme, respectively, from the bitstream. And steps to
(B) decoding a low-resolution video frame coded according to the first video coding scheme according to a first video decoding scheme to reconstruct a first low-resolution video frame;
(C) Referring to the reconstructed first low-resolution video frame, the low-resolution video frame coded by the second video coding scheme is decoded by the second video decoding scheme to obtain the second low-resolution video. Reconstructing the frame;
(D) Referring to the reconstructed second low-resolution video frame, the high-resolution video frame coded by the first video coding scheme is decoded by the first video decoding scheme to obtain a first high-resolution Reconstructing the video frame;
(E) Referring to the reconstructed first high resolution video frame, the high resolution video frame coded by the second video coding scheme is decoded by the second video decoding scheme to obtain a second high resolution. A multi-layer video decoding method comprising: reconstructing a video frame.   The method of claim 10, wherein the first video decoding scheme is an AVC decoding scheme and the second video decoding scheme is a wavelet decoding scheme.   The multi-layer video decoding according to claim 10 or 11, further comprising: (f) restoring a final video frame based on the first and second low resolution video frames and the first and second high resolution video frames. Method. A bitstream analyzer that parses the bitstream and extracts coded frames of high and low resolution layers;
A low resolution video decoding unit for decoding the low resolution coded frame;
A high-resolution video decoding unit that decodes the coded frame of the high-resolution layer with reference to the low-resolution frame reconstructed by the low-resolution video decoding unit;
The low-resolution video decoding unit decodes a first coded frame constituting a coded frame of the low-resolution layer according to a first video decoding scheme, and is decoded according to the first video decoding scheme. And reconstructing the low-resolution frame by decoding the second coded frame constituting the coded frame of the low-resolution layer according to a second video decoding scheme.
The high resolution video decoding unit refers to a second low resolution frame decoded by the second video decoding scheme with respect to a first coded frame constituting a coded frame of the high resolution layer. Decoding according to the first video decoding scheme, a second coded frame constituting a coded frame of the high resolution layer is represented as a first high resolution frame decoded according to the first video decoding scheme. A multi-layer video decoder that reconstructs the high-resolution frame by referring to and decoding by the second video decoding method.   The multi-layer video decoder according to claim 13, wherein the first video decoding scheme is an AVC decoding scheme and the second video decoding scheme is a wavelet decoding scheme.   A medium in which a program for causing a computer to execute the method according to any one of claims 1 to 7 and claims 10 to 12 is recorded.

Images