KR20040076428A - Apparatus for temporal scalable video coding and method thereof - Google Patents

Abstract

PURPOSE: A temporal scalable video coding system and method thereof are provided to recover an error within a short period of time when the error is generated while video service such as wired/wireless video phone and VOD(Video On Demand) is provided. CONSTITUTION: A temporal scalable video coding system includes a scalatility pre-processor(101), a base layer(104), and at least one enhancement layer(102). The scalability pre-processor divides an input image in response to a frame rate. The base layer generates a lower layer bitstream from the first input data. The enhancement layer generates an upper layer bitstream from the second input data using an encoder mid-processor(103). The temporal scalable video coding system includes a base layer for generating the first output from a lower bitstream using a decoder mid=processor when an error is generated in the lower layer bitstream, at least one enhancement layer for generating the second output from an upper bitstream using a mid-processor when an error is generated in the upper layer bitstream, and a scalability post-processor for generating an output image from the first and second outputs.

Description

Apparatus for temporal scalable video coding and method

The present invention relates to temporal scalable video coding, and more particularly, to an apparatus and method for temporal scalable video coding that is resistant to errors.

Hereinafter, the conventional and the present invention will be described.

In general, in order to improve video quality in an error-prone environment, when an error occurs in a video service such as a wired / wireless videophone or a VOD, the error should be restored in a short time.

However, in the related art, a video service using a wired / wireless communication network has temporal scalable video coding as one of methods corresponding thereto because an allocated bandwidth is variable.

However, transmission errors occur in wired / wireless communication networks. In the conventional temporal scalability, error propagation can be prevented, but the error cannot be recovered.

Therefore, the present invention is to solve the above problems, the temporal scalability of the present invention is composed of a base layer and one or more enhancement layers so that all the layers can restore information generated in the upper or lower layers If an error occurs in an arbitrary layer, we propose to restore the error from the upper and lower layers.

1 is a high level block diagram of a temporal scalable video encoder

2 is a high level block diagram of a temporal scalable video decoder

FIG. 3 is a diagram illustrating in detail the temporal scalable video encoder of FIG. 1. FIG.

4 is a diagram illustrating in detail the temporal scalable video decoder of FIG.

FIG. 5 illustrates a case of having N ELs in a high level structure of a temporal scalable video encoder

FIG. 6 is a diagram illustrating a case of having N ELs in a high level structure of a temporal scalable video decoder. FIG.

7 is a flowchart illustrating a temporal scalable video coding method in an encoder of the present invention.

8 is a flowchart illustrating a temporal scalable video coding method in a decoder of the present invention.

A temporal scalable video coding apparatus according to the present invention comprises: a scalability pre-processor for dividing an input image differently according to a frame rate; A base layer for generating a lower layer bitstream from the first input data; At least one enhancement layer for generating an upper layer bitstream from the second input data using an encoder mid-processor; It is configured to include.

The present invention also provides a base layer for generating a first output from a lower bitstream using a decoder mid-processor when an error occurs in a lower layer bitstream, and a second from a higher bitstream using a mid-processor when an error occurs in an upper layer bitstream. One or more enhancement layers for generating output; And a scalability post-processor for producing an output image from the first and second outputs.

In addition, the Temporal scalable video coding method in the encoder of the present invention comprises the steps of: configuring a base layer which is a lower layer and at least one enhancement layer; Coding a sync frame that is the same frame at the same time as a frame of the lower layer periodically in all enhancement layers; The sync frame of the upper layer includes coding a difference between the previous frame of the same layer and the sync frame of the lower layer.

In addition, the Temporal scalable video coding method in the decoder of the present invention, comprising the steps of configuring a base layer which is a lower layer and at least one enhancement layer; And when an error occurs in any frame in any layer, recovering the error from the sync frame of the upper layer without errors or the frame of the lower layer without errors.

Other objects and features of the present invention will become apparent from the detailed description of the embodiments with reference to the accompanying drawings.

Hereinafter, a temporal scalable video coding apparatus and method according to the present invention will be described with reference to the accompanying drawings.

First, the features of the present invention will be described.

Temporal scalability consists of one base layer and one or more enhancement layers.

1) For each enhancement layer, the same frame is coded periodically at the same time as the frame of the lower layer. In this case, two identical frames at the same time are called sync frames .

2) The sync frame of the upper layer among the pair of sync frames codes the difference between the previous frame of the same layer and the sync frame of the lower layer.

3) If an error occurs in any frame in any layer, the error is recovered from the sync frame of the upper layer where there is no error.

4) Or, if an error occurs in any frame in any layer, the error is recovered from the frame of the lower layer without errors.

Hereinafter, the drawings will be described with reference.

1 is a high level block diagram of a temporal scalable video encoder.

As shown in the figure, this shows when the output bitstream (1,0) consists of one base layer (BL) 104 and one enhancement layer (EL) 102.

When the video encoder has N ELs, the BL encoder of FIG. 1 represents the EL n encoder, and the EL 1 encoder of FIG. 1 represents the EL n + 1 encoder (n = 1 to N-1).

Scalability pre-processor 101 divides the input image into input 0 and input 1 according to the frame rate.

That is, a frame matching the BL frame rate is divided by input 0, and a frame that is not otherwise divided by input 1.

The BL encoder creates a lower layer bitstream (bitstream 0) from input zero.

The EL 1 encoder creates an upper layer bitstream (bitstream 1) from input 1 using the encoder mid-processor (103).

2 is a high level block diagram of a temporal scalable video decoder.

As shown in the figure, this shows when the input bitstream (1,0) consists of one BL and one EL.

When the video decoder has N ELs, the BL decoder 203 of FIG. 2 represents an EL n decoder, and the EL 1 decoder 201 of FIG. 2 represents an EL n + 1 decoder. (n = 1 through N-1).

The BL decoder generates output 0 from the lower layer bitstream (bitstream 0).

If an error occurs in Bitstream 0, the BL decoder uses the decoder mid-processor 202 to generate output 0 from the lower layer bitstream (bitstream 0).

On the other hand, the EL 1 decoder 201 produces an output 1 from an upper layer bitstream (bitstream 1) using a decoder mid-processor.

If an error occurs in Bitstream 1, the EL 1 decoder 201 uses the decoder mid-processor 202 to make output 1 from the upper layer bitstream (bitstream 1).

Scalability post-processor 204 produces an output image from output 0 and output 1.

FIG. 3 is a detailed diagram illustrating the temporal scalable video encoder of FIG. 1.

As shown in the figure, the operation blocks of the BL encoder and the EL 1 encoder are basically the same as in FIG.

The BL encoder 303 encodes a BL frame like a general video encoder. For example, input 0 is encoded as an I-frame or a P-frame according to the frame rate of the BL.

The EL 1 encoder 301 encodes input 1 using the encoder mid-processor 302.

At this time, the encoder reference frame selector 302a determines whether to select the reference frame of the EL 1 encoder in BL or EL.

For example, when all the frames of input 1 are encoded as P-frames, the frame most recently encoded in BL or the frame most recently encoded in EL is referred to as a reference frame.

The encoder sync frame selector 302b also encodes the EL frame.

On the other hand, since the scalability pre-processor divides the input image according to the frame rate of the BL, the input 0 of the BL and the input 1 of the EL do not have the same frame in time.

However, the encoder sync frame selector periodically encodes the EL frame at the same time as the BL frame.

At this time, the BL frame and the EL frame existing at the same time are defined as sync frames , and this time is defined as sync time .

This sync frame is used for error recovery in a temporal scalable video decoder. To create a sync frame, we use the BL frame of sync time and the most recent EL frame. The quantized coefficient of the EL sync frame is calculated from the quantized coefficient of each frame.

Q_BL_sync_frame = quantized coefficient of BL sync frame

Q_EL_latest_frame = quantized coefficient of the most recent EL frame

Q_EL_sync_frame = quantized coefficient of EL sync frame

Q_EL_sync_frame = Q_EL_latest_frame-Q_BL_sync_frame

FIG. 4 illustrates the temporal scalable video decoder of FIG. 2 in detail.

As shown in the figure, the operation blocks of the BL decoder and the EL 1 decoder are basically as shown in FIG.

The BL decoder 403 decodes the BL frame from bitstream 0 like a general video decoder.

For example, the I-frame and the P-frame are decoded from bitstream 0 encoded as described with reference to FIG. 3.

The EL 1 decoder 401 decodes the EL frame from bitstream 1 using the decoder mid-processor 402.

At this time, the decoder reference frame selector 402a analyzes bitstream 1 and determines whether to select a reference frame from a BL frame or an EL frame.

For example, if the current EL frame is a P-frame, the reference frame becomes the most recent I-frame or P-frame in BL, or the most recent P-frame in EL, which is determined after analyzing bitstream 1.

On the other hand, if an error occurs in BL bitstream 0, decoding is performed again from the next I-frame of the BL. This is a way to overcome the error from the BL's frame when an error occurs in the BL (lower layer).

Or, if an error occurs in BL bitstream 0, overcome this by using a sync frame. This is a way to overcome the error from the sync frame of the EL (upper layer) when an error occurs in the BL (lower layer).

However, if the BL frame in which the error occurs is not a sync frame, the next sync frame of the EL is decoded as a reference frame.

At this time, an EL sync frame used as a reference frame by the decoder sync frame selector is transmitted to the BL decoder.

On the other hand, if the BL frame in which the error occurs is a sync frame, the current decoding frame of the EL is decoded as a reference frame. At this time, an EL sync frame used as a reference frame by the decoder sync frame selector is transmitted to the BL decoder.

If an error occurs in EL 1 bitstream 1, decoding starts from the next frame of BL.

This is a way to overcome an error from the next frame of that BL (lower layer) when an error occurs in the EL (higher layer).

5 is a diagram illustrating a case of having N ELs in a high level structure of a temporal scalable video encoder.

The encoding is performed as shown in Figs. 1 and 3 by pairs of two adjacent layers.

For example, FIG. 1 and FIG. 3 are applied in pairs BL and EL 1, and FIGS. 1 and 3 are applied in pairs EL 1 and EL 2.

At this time, the sync frame can be matched in two or more adjacent layers.

For example, the sync time of BL and EL 1 may be the same as the sync time of EL 1 and EL 2.

FIG. 6 is a diagram illustrating a case of having N ELs in a high level structure of a temporal scalable video decoder.

Decoding is performed as shown in FIGS. 2 and 4 by a pair of two adjacent layers.

For example, FIGS. 2 and 4 are applied in pairs BL and EL 1, and FIGS. 2 and 4 are applied in pairs EL 1 and EL 2.

If an error occurs in any layer, the error is recovered using the next I-frame of the layer as in the case of FIG.

Alternatively, if an error occurs in an arbitrary layer, the error is restored using the sync frame of the upper layer as in the case of FIG.

If an error occurs in an arbitrary layer, the error is recovered using the next frame of the lower layer as in the case of FIG.

7 is a flowchart illustrating a temporal scalable video coding method in an encoder of the present invention.

It configures a base layer as a lower layer and one or more enhancement layers as an upper layer. (S 701).

In every enhancement layer, a sync frame, which is the same frame at the same time as a frame of the lower layer, is coded. (S 702).

The sync frame of the upper layer codes the difference between the sync frame of the lower layer and the previous frame of the same layer. (S 703).

In other words, the two layers repeat the above steps S702 and S703 in pairs.

That is, steps S702 and S703 are performed in pairs with the base layer and EL1, S702 and S703 are paired again with EL1 and EL2, and S702 and S703 are performed again with pairs EL2 and EL3.

8 is a flowchart illustrating a temporal scalable video coding method in a decoder of the present invention.

It configures a base layer as a lower layer and one or more enhancement layers as an upper layer. (S 801).

When an error occurs in any frame in any layer, the error is recovered from the sync frame of the upper layer without errors or the frame of the lower layer without errors. (S 802).

Although the preferred embodiment of the present invention has been described above, the present invention may use various changes, modifications, and equivalents. It is clear that the present invention can be applied in the same manner by appropriately modifying the above embodiments.

That is, in order to be compatible with the standard codec, the sync frame of the upper layer among the pair of sync frames is encoded using code not used in the corresponding profile and level of the standard codec.

In this case, the standard decoder may regard the sync frame of the upper layer among the pair of sync frames as an error and ignore it.

Accordingly, the above description does not limit the scope of the invention as defined by the limitations of the following claims.

According to the present invention, when an error occurs in a video service such as a wired / wireless video telephone or a VOD, the error can be restored in a short time.

Therefore, since the error is restored in a short time, there is an advantage of improving the image quality in an environment where many errors occur.

Claims (8)

A scalability pre-processor for dividing the input image differently according to the frame rate; A base layer for generating a lower layer bitstream from the first input data; At least one enhancement layer for generating an upper layer bitstream from the second input data using an encoder mid-processor; Temporal scalable video coding device characterized in that it comprises a. The apparatus of claim 1, wherein the encoder reference frame selector determines whether to select a reference frame of the EL 1 encoder in BL or EL. Constructing a base layer, which is a lower layer, and at least one enhancement layer, which is an upper layer; Coding a sync frame that is the same frame at the same time as a frame of the lower layer periodically in all enhancement layers; The sync frame of the upper layer may include coding a difference between a previous frame of the same layer and a sync frame of the lower layer; Temporal scalable video coding method comprising a. 4. The method of claim 3, wherein a BL frame of sync time and a most recent EL frame are used to create a sync frame. 5. The method of claim 3 or 4, wherein the quantized coefficient of the EL sync frame is calculated from the quantized coefficient of each frame. A base layer for generating a first output from the lower bitstream using a decoder mid-processor when an error occurs in the lower layer bitstream; At least one enhancement layer for generating a second output from the upper bitstream using a mid-processor when an error occurs in the upper layer bitstream; And a scalability post-processor for producing an output image from the first and second outputs. 7. The apparatus of claim 6, wherein the decoder reference frame selector determines whether to select a reference frame from a BL frame or an EL frame after analyzing bitstream 1. Constructing a base layer, which is a lower layer, and at least one enhancement layer, which is an upper layer; When an error occurs in any frame in any layer, recovering the error from the sync frame of the upper layer without errors or the frame of the lower layer without errors; and temporal scalable video coding method.