Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
  • H04N19/187—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a scalable video layer
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
  • H04N19/103—Selection of coding mode or of prediction mode
  • H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/46—Embedding additional information in the video signal during the compression process

Definitions

• the present invention relates to a method for providing and using information about inter-layer prediction when providing a video signal to a plurality of layers by encoding the video signal .
• a video signal is encoded with a best video quality in such a manner that a low quality video can be presented even though parts of picture sequences (frame sequences intermittently selected from among the picture sequences) derived from the encoding are decoded.
• a motion compensated temporal filter (or filtering) (MCTF) is one of encoding schemes suggested for the SVC scheme .
• an additional assistant picture sequence having a low transmission rate such as a small-sized video and/or a ' picture sequence having the smaller number of frames per second, may be provided in a hierarchy structure having at least one layer
• an assistance sequence (a lower layer sequence) and a main picture sequence (an upper layer sequence) are called a base layer and an enhancement layer, respectively.
• a base layer and an enhancement layer are obtained by encoding the same video source, redundancy information exists in video signals of both layers. Accordingly, in order to improve the coding rate of the enhanced layer, the video signal of the enhanced layer is coded using coding information of the base layer. This is one of inter-layer prediction schemes.
• the current macro block is coded into residual data using image data of a corresponding block in an intra mode of the base layer (this corresponding block temporarily simultaneous with and identical to the macro block of the enhanced layer) .
• the coded macro block is called an 'intra BL mode block', and the specification of the intra BL mode for the macro block may be limited or not.
• the inter-layer prediction may be constrained by allowing the inter-layer prediction only when the corresponding block of the base layer is coded in the intra mode.
• the inter-layer prediction may not be constrained by allowing the inter-layer prediction when the corresponding block of the base layer is coded in an inter mode as well as in an intra mode .
• a encoding mode (in the side of an encoder) or a decoding mode (in the side of a decoder) is determined from one of modes shown in FIGs .
• IA and IB based on the value of a ⁇ constrained_intra_pred_flag' of the base layer.
• both the upper layer (Ll) and the middle layer (L2) are coded by employing the lower layer (L3) as a prediction layer and then are provided to the decoder
• both the upper layer (Ll) and the middle layer (L2) are constrained in a coding scheme for an intra mode block of the lower layer (L3) .
• the constrained_intra__pred_flag of the lower layer (L3) is ⁇ l'
• prediction-constraint is assigned to the upper layer (Dl) and the middle layer (L2) so that the upper layer (Ll) and the middle layer (L2) are coded through a constrained scheme as shown in PIG. IA.
• prediction-unconstraint is assigned to the upper layer (Ll) and the middle layer (L2) so that the upper layer (Ll) and the middle layer (L2) are coded through an unconstraint scheme as shown in FIG. IB.
• the upper layers of the lower layer cannot individually select the inter-layer prediction constraint or the inter-layer prediction unconstraint.
• inter-layer prediction constraint (unconstraint) is determined for "constrained_intra_pred_flag" having a different attribute
• the upper layer may loss the degree of freedom for the inter-layer prediction constraint. In other words, only one of inter-layer prediction constraint and inter-layer prediction unconstraint is forcibly determined for the upper layer.
• an object of the present invention is to provide a method for transmitting information about inter-layer prediction constraint for a video signal through a prediction coded layer and decoding the video signal using the transmitted information.
• a method for encoding an image signal to bit streams of a first layer and a second layer comprising the steps of (1) coding an image block in a picture of the first layer into residual data based on image data of a block of the second layer corresponding to the vide block; and (2) providing indication information through the bit stream of the first layer, the indication information representing whether the coding is achieved in inter-layer prediction constraint or inter-layer prediction unconstraint .
• a method for decoding encoded bit streams of a first layer and a second layer by receiving the encoded bit streams comprising the steps of:(l) determining indication information representing that all or a part of data of the first layer are coded in an inter-layer prediction constraint state or in an inter-layer prediction unconstraint state, the indication information being provided through the bit stream of the first layer,- and (2) determining whether an inter mode block of the second layer is reconstructed first or not in order to reconstruct a corresponding block having a specific mode in a picture of the first layer.
• a method for decoding encoded bit streams of a first layer and a second layer by receiving the encoded bit streams comprising the steps of: (1) determining indication information indicating that use of adjacent inter mode block data is constrained when an intra mode block is reconstructed, the flag being included in the bit stream of the second layer; and (2) determining indication information representing that all or a part of data of the first layer are coded in an inter-layer prediction constraint state or in an inter-layer prediction unconstraint state, the indication information being provided through the bit stream of the first layer if the determined indication information of step (1) indicates that the use of the adjacent inter mode block data is constrained.
• the indication information is allocated with different values depending on types of pictures .
• the types of the pictures are classified into two types including a key picture (or a picture having a low band component) and a non-key picture (or a picture having a high band component) .
• the indication information is classified according to the types of the pictures and has a value representing that the pictures are coded in an inter-layer prediction constraint/unconstraint .
• FIG. IA is a view illustrating a scheme of predicting an intra mode block based on a lower layer according to inter-layer prediction constraint
• FIG. IB is a view illustrating a scheme of predicting an intra mode block based on a lower layer according to inter-layer prediction unconstraint
• FIG. 2 is a view illustrating an example of an inter-layer prediction scheme of upper layers when a video signal is encoded and transmitted through a plurality of layers including at least three layers
• FIG. 3 is a block diagram illustrating the structure of a video signal encoding device employing a scalable coding scheme for a video signal according to the present invention
• FIG. 4 is a block diagram illustrating the structure of a decoder for decoding a data stream transmitted and encoded by the encoding device shown in FIG. 3 ;
• FIG. 5 is a view illustrating an example of syntax used for determining by a decoder whether or not inter-layer prediction is constrained according to an embodiment of the present invention.
• FIG. 6 is a view illustrating an example of syntax used for determining by a decoder whether or not inter-layer prediction is constrained according to another embodiment of the present invention.
• FIG. 3 is a block diagram illustrating the structure of a video signal encoding device employing a video signal scalable coding scheme according to the present invention.
• the device shown in FIG. 3 codes an input video signal in two layers, the following principle of the present invention is applicable between layers even if the device codes the input signal in three layers or more.
• the video signal encoding device shown in FIG. 3 codes an input video signal in two layers, the following principle of the present invention is applicable between layers even if the device codes the input signal in three layers or more.
• an enhanced layer (EL) encoder 1 for scalably encoding an input video signal through a specification scheme (e.g., MCTF or AVC) based on each macro block unit and generating suitable management information
• a texture coding unit 2 for converting the encoded information of each macro block into a compressed bit string
• a motion coding unit 3 for coding motion vectors of a video block obtained from the EL encoder 1 into a compressed bit string through a specific scheme
• a base layer encoder 4 for encoding an input video signal through a specification scheme such as an AVC scheme and generating the sequence of small-sized videos, for example, the sequence of pictures having 25% of an original video size or the sequence of pictures having the same size as the enhanced layer and a low frame rate
• a muxer 5 for encapsulating the output data of the texture coding unit 2 , the encoded video sequence of the BL encoder 4 , and an output vector data of the motion coding unit 120 in a predetermined format, multiplexing the data
• the EL encoder 1 performs a motion estimation operation and a prediction operation with respect to a macro block in a predetermined image frame.
• the EL encoder 1 performs an inter-layer prediction coding using video data and coding information provided from the BL encoder 4, inserts coding information according to the inter-layer prediction coding as macro block information into a suitable header, and transmits the coding information if coding efficiency is improved.
• a current macro block to be coded is determined as an intra-mode macro block, it is determined that a block corresponding to the current macro block is in an intra-mode or in an inter-mode based on information provided from the BL encoder 4.
• the current macro block is specified as an intra BL mode block and coded into residual data using image data of the corresponding block in the base layer.
• the current macro block may be specified as an intra BL mode block (in a case of unconstraint) according to an inter-layer prediction constraint/unconstraint state predetermined for encoding, and coded into the residual data using image data of the corresponding block of the base layer as described above.
• the current macro block may be specified as an intra mode block according to the inter-layer prediction constraint/unconstraint state and coded into residual data based on pixel values of an adjacent macro block.
• specific information including "constrained_inter_layer_pred_flag" maybe set to ⁇ l' . If the constraint is not set, the constrained_inter_layer_jpred_flag may be set to x 0'.
• the set specific information is inserted into a header (a sequence header, a picture or picture group header, or a slice header) to be transmitted.
• the EL encoder 1 codes an image of the current macro block into residual data and specifies the mode of the image of the current macro block as an intra BL mode based on a reconstructed image obtained by performing a reconstruction operation (MC: Motion Compensation or inverse-MCTF) for an inter mode block of the base layer corresponding to the current macro block.
• MC Motion Compensation or inverse-MCTF
• the reconstructed image may be received by requesting the image from the BL encoder 4, and, if the BL encoder 4 temporarily stores image data before the current macro block is coded into the residual data, the stored image data may be used.
• the EL encoder 1 may insert the constraint flag into one of several headers. If the EL encoder 1 inserts the constraint flag into a picture header (PPS; Picture Parameter Set and PPSE; PPS extension) or a slice header, the EL encoder 1 may specify inter-layer prediction constraint according to slices. In addition, the EL encoder 1 may insert the constraint flag into a header for a sequence or a header for a group of pictures (GOP) . In this case, all slices belonging to the sequence or the GOP equally undergo constraint or unconstraint specified by the constraint flag. Description about encoding procedures for a macro block in several block modes will be omitted because the encoding procedures do not directly relate to the present invention.
• PPS Picture Parameter Set and PPSE
• PPS extension Picture Parameter Set and PPSE
• a slice header the EL encoder 1 may specify inter-layer prediction constraint according to slices.
• the EL encoder 1 may insert the constraint flag into a header for a sequence or
• Input image signals are stored in an additional storage unit through encoding procedures including the above-described procedure. Then, all or a part of the image signals are read-out from the storage unit according to the request of a decoder and are transmitted through a transmission channel. The selection for an amount of the image signals is determined according to the capacity of the transmission channel, and description about the selection for the amount of the image signals will be omitted because the selection is regardless of the present invention.
• FIG. 4 is a block diagram illustrating the structure of the decoder for decoding a data stream encoded and transmitted as described above.
• the decoder shown in FIG. 4 includes a de-muxer 15 for dividing the received data stream into a compressed motion vector stream and a compressed macro block information stream, a texture decoding unit 12 for recovering an original uncompressed information stream from the compressed macro block stream, a motion decoding unit 13 for recovering an original uncompressed stream from a compressed motion vector stream, an enhanced layer
• EL EL
• BL base layer
• the BL decoder 14 provides header information included in an input base layer stream to the EL decoder 11 while decoding the input base layer stream such that the EL decoder 11 can use required encoding information (in the header of the base layer stream) of the base layer including motion vector information and macro block mode information.
• the BL decoder 14 provides image data of the decoded base layer picture or a picture, in which a part of blocks is decoded, to the EL decoder 11 according to the direction of the EL decoder 11.
• the EL decoder 11 determines the constrained__inter_layer_jpred_flag, which is an inter-layer prediction constraint flag, in a header of a GOP or a sequence having a picture to be currently decoded among received picture sequences of the enhanced layer. As described through the above encoding procedure, the inter-layer prediction constraint flag may be checked in the picture header or the slice header.
• the EL decoder 11 receives decoded picture data provided by the BL decoder 14 (or reads the decoded picture data from a memory shared between the EL decoder 11 and the BL decoder 14) and reconstructs original image data from residual data of the intra BL mode block based on reconstructed data of blocks in the decoded picture data corresponding to the intra BL mode blocks.
• the EL decoder 11 directs the BL decoder 14 to reconstruct all macro blocks in a slice of the base layer corresponding to the slice to be decoded and provide the reconstructed macro blocks to the EL decoder 11. If the BL decoder 14 is set in such a manner that the BL decoder 14 always provides decoded data for all pictures, it is unnecessary for the EL decoder 11 to request the direction according to conditions. However, since residual prediction is used between layers in addition to inter-layer prediction based on an intra BL mode, it is preferred that the
• 5 BL decoder 14 reconstructs only intra mode block data instead of all data within a picture so as to provide the reconstructed data to the EL decoder 11 and, if there is additional direction, performs a reconstruction operation (the MC or the inverse_MCTF) for inter mode blocks within the picture so as to provide a slice
• the EL decoder 11 reconstructs original image data from residual data of a current intra BL mode block based on reconstructed data of a block corresponding to the BL mode block in a slice having reconstructed data received from the BL decoder
• information about inter-layer prediction constraint may be classified and set according to types of pictures.
• a type of a picture is a key picture or a picture coded
• the EL encoder 1 shown in FIG. 3 may set or reset flag information about ⁇ constrained_inter_layer_pred_L_flag" , thereby transmitting the flag information to the decoder. If the type of the picture is
• the EL encoder 1 shown in FIG. 3 may set or reset flag information about "constrained_inter_layer_pred_H_flag" , thereby transmitting the flag information to the decoder.
• the key picture denotes a picture coded into residual data by performing a prediction operation based on only a quality-base picture.
• the non-key picture denotes a picture coded into residual data based on SNR enhance picture data in addition to the quality-base picture.
• the definition of the key picture is only one example, and the present invention is limited to the definition.
• the two flags have the same usage as the constrained_inter_layer_jpred_flag. Accordingly, the EL decoder 11 shown in FIG. 4 determines the value of the constrained_inter_layer_pred_L_flag if a picture to be decoded is the key picture or the picture having a low band component. If the picture to be decoded is the non-key picture or the picture having a high band component, the EL decoder 11 determines the value of the constrained_inter_layer_jpred_H_flag.
• the reconstruction operation of a BL mode macro block according to the determined value is identical to that of the previous embodiment of the present invention.
• the constrained_intra_j?red_flag which indicates whether or not data in an adjacent inter mode block are used in the reconstruction of an intra mode block
• the BL decoder 14 shown in FIG. 4 performs a reconstruction operation through an MC scheme (an inverse-MCTF scheme) with respect to the adjacent inter mode block in order to reconstruct residual data in the intra mode block of the picture.
• the EL encoder 1 shown in FIG. 1 encodes a macro block to be encoded of the enhanced layer without distinguishing if a block of the base layer corresponding to the macro block is in an inter mode or an intra mode.
• the EL decoder 11 shown in FIG. 4 can obtain reconstructed image data from the BL decoder 14 through the operation such as the MC even for the inter mode block. Accordingly, when the constrained_intra__pred__flag of the base layer is set to ⁇ 0' by the BL encoder 4 (that is, in a case in which constraint is not set with respect to the adjacent block) , the EL encoder 1 shown in FIG.
• the EL encoder 1 codes a specific block to an intra BL mode block according to a predetermined prediction constraint condition for the enhanced layer and transmits the specific block by setting the constrained_inter_layer_pred_flag to ⁇ 0' or ⁇ l' .
• the EL decoder 11 requests a reconstruction operation for the inter mode block from the BL decoder 14 as described above according to the previous embodiment. Accordingly, in this case, the MC operation (or the inverse-MCTF) for data reconstruction must be performed in both layers.
• the EL decoder 11 shown in FIG. 4 determines whether or not constraint is set for inter-layer prediction through a scheme shown in FIG. 5 because the constrained_inter_layer_pred_flag may not be transmitted from the encoder.
• the EL decoder 11 initializes "interlayer_constrained" which is a variable for inter-layer prediction, as a unconstraint state value (i.e., ⁇ 0') and then determines the value of the constrained_intra_j?red_flag transmitted from the BL decoder 14.
• the EL decoder copies the value of the constrained_inter_layer__pred_flag received through a header of the enhanced layer to the value of the interlayer_constrained only if the value of the constrained_intra_pred_flag has ⁇ l'.
• the EL decoder 11 shown in FIG. 4 can recognize based on the value of the interlayer_constrained if constraint is set for inter-layer prediction.
• a procedure of reconstructing original image data from residual data of an intra BL mode macro block according to the state of the inter-layer prediction is identical to the procedure described according to the previous embodiment .
• (constrained_inter__layer_pred_flag) indicating the selected constraint state may be classified and allocated according to types of pictures. For example, the value relating to inter-layer prediction constraint may be carried through the constrained_inter_layer_pred_L_flag which corresponds to a key picture or a picture coded using a low band component or the constrained_inter_layer_pred_H_flag which corresponds to a non-key picture or a picture coded using a high band component .
• both the flags may not be transmitted according to the value of the constrained_intra_pred_flag of the base layer or may be transmitted with the value of ⁇ 0' or x l' as described above.
• the EL decoder 11 shown in FIG. 4 reconstructs an intra BL mode macro block according to the transmission state. If the constrained_intra_jpred_flag of the base layer has the value of ⁇ 0', and if the constrained_inter_layer_jpred_L_flag, or the constrained_inter_layerj?red_H_flag is not transmitted, the EL decoder 11 shown in FIG. 4 performs an operation according to a syntax shown in FIG. 6 and determines whether or not the constraint is set for inter-layer prediction according to the final value of the interlayer_constrained obtained through the operation.
• Information about inter-layer prediction constraint may be delivered using one integrated indicator such as "constrained_inter_layer_pred_idc" without classification and allocation according to types of pictures as described according to the previous embodiment.
• the constrained_inter_layer_pred_idc is set to x 0 ' when representing the constraint inter-layer prediction regardless of the types of pictures.
• the constrained__inter_layer_pred_idc is set to ⁇ l' when representing an inter-layer prediction unconstraint state for a first picture type (e.g., a key picture or a picture having a low band component) , and an inter-layer prediction constraint state for a second picture type (e.g., a non-key picture or a picture having a high band component) .
• the constrained_inter_layer_pred_idc is set to S 2' when representing the inter-layer prediction unconstraint state regardless of the types of pictures.
• the EL decoder 11 shown in FIG. 4 determines whether or not constraint is set for the inter-layer prediction according to types of pictures based on the indication information depending on the indication value defined above.
• the value indicating the case is excluded.
• a decoder of determining information representing inter-layer prediction constraint and performing a reconstruction operation for a corresponding BL mode macro block according to the determination result may be installed at a mobile communication terminal or a device of reproducing recorded media.
• the inter-layer prediction constraint/unconstraint state is directly delivered through a prediction-coded layer instead of a base layer for the inter-layer prediction, thereby individually setting or releasing prediction constraint of each layer when a video signal is encoded and transmitted based on a plurality of layers. Accordingly, coding can be freely performed, and layer coding suitable for an encoding environment can be achieved.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Compression Or Coding Systems Of Tv Signals (AREA)

Applications Claiming Priority (4)

Publications (2)

Family

ID=37452221

Family Applications (1)

Country Status (2)

Cited By (1)

Families Citing this family (9)

• 2006
  • 2006-05-25 EP EP06747456A patent/EP1897377A4/de not_active Withdrawn
  • 2006-05-25 WO PCT/KR2006/001980 patent/WO2006126841A1/en active Application Filing

Non-Patent Citations (6)

Also Published As

Similar Documents

Legal Events