EP2684369A1 - A method for decoding video - Google Patents
A method for decoding videoInfo
- Publication number
- EP2684369A1 EP2684369A1 EP12755140.6A EP12755140A EP2684369A1 EP 2684369 A1 EP2684369 A1 EP 2684369A1 EP 12755140 A EP12755140 A EP 12755140A EP 2684369 A1 EP2684369 A1 EP 2684369A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- slice
- tile
- entropy
- frame
- decoding
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
- 238000000034 method Methods 0.000 title claims abstract description 69
- 230000008859 change Effects 0.000 claims description 2
- 230000008569 process Effects 0.000 description 16
- 238000012545 processing Methods 0.000 description 4
- 230000001131 transforming effect Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 239000011229 interlayer Substances 0.000 description 2
- 239000003550 marker Substances 0.000 description 2
- 238000013139 quantization Methods 0.000 description 2
- 230000011664 signaling Effects 0.000 description 2
- 230000002123 temporal effect Effects 0.000 description 2
- 241000023320 Luma <angiosperm> Species 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 230000003139 buffering effect Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000011010 flushing procedure Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- OSWPMRLSEDHDFF-UHFFFAOYSA-N methyl salicylate Chemical compound COC(=O)C1=CC=CC=C1O OSWPMRLSEDHDFF-UHFFFAOYSA-N 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
- H04N19/436—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation using parallelised computational arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/174—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a slice, e.g. a line of blocks or a group of blocks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/46—Embedding additional information in the video signal during the compression process
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
- H04N19/91—Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
Definitions
- the present invention relates to a method for decoding video .
- Digital video is typically represented as a series of images or frames, each of which contains an array of pixels.
- Each pixel includes information, such as intensity and / or color information.
- each pixel is represented as a set of three colors, each of which is defined by eight bit color values .
- Video-coding techniques for example H .264 / MPEG-4 AVC (H .264 /AVC) , typically provide higher coding efficiency at the expense of increasing complexity. Increasing image quality requirements and increasing image resolution requirements for video coding techniques also increase the coding complexity.
- Video decoders that are suitable for parallel decoding may improve the speed of the decoding process and reduce memory requirements; video encoders that are suitable for parallel encoding may improve the speed of the encoding process and reduce memory requirements .
- H .264 / MPEG-4 AVC Joint Video Team of ITU-T VCEG and ISO / IEC MPEG, "H .264 : Advanced video coding for generic audiovisual services," ITU-T Rec.
- JCT-VC ["Draft Test Model Under Consideration” , JCTVC-A205 , JCT-VC Meeting, Dresden, April 2010 (JCT-VC) ] , both of which are incorporated by reference herein in their entirety, are video codec (encoder/ decoder) specifications that use macroblock prediction followed by residual coding to reduce temporal and spatial redundancy in a video sequence for compression efficiency.
- One embodiment of the present invention disclose a method for decoding video .
- the method comprising: (a) receiving a frame of said video that includes at least one slice and at least one tile , wherein each of said at least one slice is characterized that it is decoded independently of the other said at least one slice , wherein each of said at least one tile is characterized that it is a rectangular region of said frame and having coding units for said decoding arranged in a raster scan order, wherein said at least one tile of said frame are collectively arranged in a raster scan order of said frame ; (b) receiving entropy information suitable for decoding at least one of said tiles; (c) receiving information indicating that location of at least one tile is transmitted within a slice; (d) receiving information indicating said location and information indicating number of said at least one tile.
- One embodiment of the present invention disclose a method for decoding video .
- the method comprises (a) receiving a frame of said video that includes at least one slice and at least one tile, where each of said at least one slice and said at least one tile are not all aligned with one another, wherein each of said at least one slice is characterized that it is decoded independently of the other said at least one slice, wherein each of said at least one tile is characterized that it is a rectangular region of said frame and having coding units for said decoding arranged in a raster scan order, wherein said at least one tile of said frame are collectively arranged in a raster scan order of said frame; (b) receiving entropy information suitable for decoding at least one of said tiles that is not aligned with any of said at least one slice .
- One embodiment of the present invention discloses a method for decoding video .
- the method comprises (a) receiving a frame of said video that includes at least one slice and at least one tile, where each of said at least one slice and said at least one tile are not all aligned with one another, wherein each of said at least one slice is characterized that it is decoded independently of the other said at least one slice, wherein each of said at least one tile is characterized that it is a rectangular region of said frame and having coding units for said decoding arranged in a raster scan order, wherein said at least one tile of said frame are collectively arranged in a raster scan order of said frame; (b) identifying at least one of said tiles that is not aligned with any of said at least one slice based upon signal within a bitstream of said frame without requiring entropy decoding to identify said signal.
- FIG . 1 illustrates a H .264 / AVC video encoder.
- FIG. 2 illustrates a H .264 /AVC video decoder.
- FIG . 3 illustrates an exemplary slice structure .
- FIG . 4 illustrates another exemplary slice structure .
- FIG . 5 illustrates reconstruction of an entropy slice .
- FIG . 6 illustrates parallel reconstruction of an entropy slice .
- FIG . 7 illustrates a frame with a slice and 9 tiles.
- FIG . 8 illustrates a frame with three slices and 3 tiles.
- FIGS . 9A and 9B illustrate entropy selection for a tile .
- FIGS . 10A and 1 0B illustrates another entropy selection for a tile .
- FIG . 1 1 illustrates yet another entropy selection for a tile .
- FIGS . 12A and 12B illustrates exemplary syntax.
- any video coder/ decoder (codec) that uses entropy encoding/ decoding may be accommodated by embodiments described herein, exemplary embodiments are described in relation to an H .264 / AVC encoder and an H .264 /AVC decoder merely for purposes of illustration.
- Many video coding techniques are based on a block-based hybrid video-coding approach, wherein the source-coding technique is a hybrid of inter-picture, also considered inter-frame , prediction , intra-picture , also considered intra-frame, prediction and transform coding of a prediction residual. Inter-frame prediction may exploit temporal redundancies, and intra-frame and transform coding of the prediction residual may exploit spatial redundancies.
- FIG . 1 shows a block diagram of an exemplary H .264 / AVC video encoder 2.
- An input picture 4 also considered a frame, may be presented for encoding.
- a predicted signal 6 and a residual signal 8 may be produced, wherein the predicted signal 6 may be based on either an inter-frame prediction 10 or an intra-frame prediction 12.
- the inter-frame prediction 10 may be determined by motion compensation section 14 one or more stored, reference pictures 16 , also considered reference frames, using motion information 19 determined by a motion estimation section 18 process between the input frame 4 and the reference frames 16.
- the intra-frame prediction 12 may be determined by an intra-frame prediction section 20 using a decoded signal 22.
- the residual signal 8 may be determined by subtracting the input frame 4 from the predicted signal 6.
- the residual signal 8 is transformed, scaled and quantized by a transforming/ scaling/ quantizing section 24 , thereby producing quantized, transform coefficients 26.
- the decoded signal 22 may be generated by adding the predicted signal 6 to a signal 28 generated by an inverse (transforming/ scaling/ quantizing) section 30 using the quantized, transform coefficients 26.
- the motion information 19 and the quantized, transform coefficients 26 may be entropy coded by an entropy coding section 32 and written to the compressed-video bitstream 34.
- An output image region 38 for example a portion of the reference frame, may be generated at the encoder 2 by a de-blocking filter 36 using the reconstructed, pre-filtered signal 22. This output frame may be used as a reference frame for the encoding of subsequent input pictures.
- FIG. 2 shows a block diagram of an exemplary H .264 /AVC video decoder 50.
- An input signal 52 also considered a bitstream, may be presented for decoding.
- Received symbols may be entropy decoded by an entropy decoding section 54 , thereby producing motion information 56, intra-prediction information 57 , and quantized, scaled, transform coefficients 58.
- the motion information 56 may be combined by a motion compensation section 60 with a portion of one or more reference frames 84 which may reside in frame memory 64 , and an inter-frame prediction 68 may be generated.
- the quantized, scaled, transform coefficients 58 may be inverse quantized, scaled and inverse transformed by an inverse (transforming/ scaling/ quantizing) section 62 , thereby producing a decoded residual signal 70.
- the residual signal 70 may be added to a prediction signal 78 : either the inter-frame prediction signal 68 or an intra-frame prediction signal 76.
- the intra-frame prediction signal 76 may be predicted by an intra-frame prediction section 74 from previously decoded information in the current frame 72.
- the combined signal 72 may be filtered by a de-blocking filter 80 and the filtered signal 82 may be written to frame memory 64.
- an input picture may be partitioned into fixed-size macroblocks, wherein each macroblock covers a rectangular picture area of 16 x 16 samples of the luma component and 8 x 8 samples of each of the two chroma components .
- the decoding process of the H .264 /AVC standard is specified for processing units which are macroblocks.
- the entropy decoder 54 parses the syntax elements of the compressed-video bitstream 52 and de-multiplexes them.
- H .264 / AVC specifies two alternative methods of entropy decoding: a low-complexity technique that is based on the usage of context-adaptively switched sets of variable length codes, referred to as CAVLC , and the computationally more demanding technique of context-based adaptively binary arithmetic coding, referred to as CABAC .
- CABAC context-based adaptively binary arithmetic coding
- decoding of a current symbol may rely on previously, correctly decoded symbols and adaptively updated context models .
- different data information for example , prediction data information, residual data information and different color planes, may be multiplexed together. De-multiplexing may wait until elements are entropy decoded.
- a macroblock may be reconstructed by obtaining: the residual signal through inverse quantization and the inverse transform, and the prediction signal, either the intra-frame prediction signal or the inter-frame prediction signal.
- Blocking distortion may be reduced by applying a de-blocking filter to decoded macroblocks.
- such subsequent processing begins after the input signal is entropy decoded, thereby resulting in entropy decoding as a potential bottleneck in decoding.
- entropy decoding may be requisite prior to processing at the decoder, thereby making entropy decoding a potential bottleneck.
- An input picture comprising a plurality of macroblocks may be partitioned into one or several slices .
- the values of the samples in the area of the picture that a slice represents may be properly decoded without the use of data from other slices provided that the reference pictures used at the encoder and the decoder are the same and that de-blocking filtering does not use information across slice boundaries . Therefore, entropy decoding and macroblock reconstruction for a slice does not depend on other slices .
- the entropy coding state may be reset at the start of each slice .
- the data in other slices may be marked as unavailable when defining neighborhood availability for both entropy decoding and reconstruction .
- the slices may be entropy decoded and reconstructed in parallel. No intra prediction and motion-vector prediction is preferably allowed across the boundary of a slice . In contrast, de-blocking filtering may use information across slice boundaries.
- FIG . 3 illustrates an exemplary video picture 90 comprising eleven macroblocks in the horizontal direction and nine macroblocks in the vertical direction (nine exemplary macroblocks labeled 9 1 -99) .
- FIG. 3 illustrates three exemplary slices : a first slice denoted "SLICE #0" 1 00, a second slice denoted “SLICE # 1 " 10 1 and a third slice denoted "SLICE #2" 102.
- An H .264 /AVC decoder may decode and reconstruct the three slices 100, 1 0 1 , 102 in parallel. Each of the slices may be transmitted in scan line order in a sequential manner.
- context models are initialized or reset and macroblocks in other slices are marked as unavailable for both entropy decoding and macroblock reconstruction.
- a macroblock for example, the macroblock labeled 93 , in “SLICE # 1 ,” macroblocks (for example , macroblocks labeled 9 1 and 92) in “SLICE #0" may not be used for context model selection or reconstruction.
- the macroblock labeled 95 for example, the macroblock labeled 95 , in "SLICE # 1 ”
- other macroblocks for example , macroblocks labeled 93 and 94
- SLICE # 1 may be used for context model selection or reconstruction. Therefore , entropy decoding and macroblock reconstruction proceeds serially within a slice .
- FMO flexible macroblock ordering
- Flexible macroblock ordering defines a slice group to modify how a picture is partitioned into slices .
- the macroblocks in a slice group are defined by a macroblock-to-slice-group map, which is signaled by the content of the picture parameter set and additional information in the slice headers .
- the macroblock-to-slice-group map consists of a slice-group identification number for each macroblock in the picture .
- the slice-group identification number specifies to which slice group the associated macroblock belongs.
- Each slice group may be partitioned into one or more slices, wherein a slice is a sequence of macroblocks within the same slice group that is processed in the order of a raster scan within the set of macroblocks of a particular slice group. Entropy decoding and macroblock reconstruction proceeds serially within a slice group.
- FIG . 4 depicts an exemplary macroblock allocation into three slice groups: a first slice group denoted "SLICE GROUP #0" 103 , a second slice group denoted “ SLICE GROUP # 1 " 104 and a third slice group denoted "SLICE GROUP #2” 105.
- These slice groups 103 , 104 , 105 may be associated with two foreground regions and a background region, respectively, in the picture 90.
- a picture may be partitioned into one or more reconstruction slices, wherein a reconstruction slice may be self-contained in the respect that values of the samples in the area of the picture that the reconstruction slice represents may be correctly reconstructed without use of data from other reconstruction slices, provided that the references pictures used are identical at the encoder and the decoder. All reconstructed macroblocks within a reconstruction slice may be available in the neighborhood definition for reconstruction .
- a reconstruction slice may be partitioned into more than one entropy slice, wherein an entropy slice may be self-contained in the respect that symbol values in the area of the picture that the entropy slice represents may be correctly entropy decoded without the use of data from other entropy slices.
- the entropy coding state may be reset at the decoding start of each entropy slice .
- the data in other entropy slices may be marked as unavailable when defining neighborhood availability for entropy decoding.
- Macroblocks in other entropy slices may not be used in a current block' s context model selection.
- the context models may be updated only within an entropy slice . Accordingly, each entropy decoder associated with an entropy slice may maintain its own set of context models.
- An encoder may determine whether or not to partition a reconstruction slice into entropy slices, and the encoder may signal the decision in the bitstream.
- the signal may comprise an entropy- slice flag, which may be denoted "entropy_slice_flag" .
- an entropy-slice flag may be examined 130 , and if the entropy-slice flag indicates that there are no 132 entropy slices associated with a picture, or a reconstruction slice , then the header may be parsed 134 as a regular slice header.
- the entropy decoder state may be reset 136, and the neighbor information for the entropy decoding and the reconstruction may be defined 138.
- the slice data may then be entropy decoded 140, and the slice may be reconstructed 142. If the entropy-slice flag indicates there are 146 entropy slices associated with a picture, or a reconstruction slice, then the header may be parsed 148 as an entropy-slice header.
- the entropy decoder state may be reset 150, the neighbor information for entropy decoding may be defined 1 52 and the entropy-slice data may be entropy decoded 1 54.
- the neighbor information for reconstruction may then be defined 1 56, and the slice may be reconstructed 142. After slice reconstruction 142 , the next slice , or picture, may be examined 1 58.
- the decoder may be capable of parallel decoding and may define its own degree of parallelism, for example , consider a decoder comprising the capability of decoding N entropy slices in parallel.
- the decoder may identify 170 N entropy slices . If fewer than N entropy slices are available in the current picture, or reconstruction slice , the decoder may decode entropy slices from subsequent pictures, or reconstruction slices, if they are available. Alternatively, the decoder may wait until the current picture , or reconstruction slice, is completely processed before decoding portions of a subsequent picture, or reconstruction slice . After identifying 170 up to N entropy slices, each of the identified entropy slices may be independently entropy decoded .
- a first entropy slice may be decoded 172 - 176.
- the decoding 172 - 176 of the first entropy slice may comprise resetting the decoder state 172. If CABAC entropy decoding is used, the CABAC state may be reset.
- the neighbor information for the entropy decoding of the first entropy slice may be defined 1 74 , and the first entropy slice data may be decoded 176. For each of the up to N entropy slices, these steps may be performed ( 178- 182 for the Nth entropy slice) .
- the decoder may reconstruct 184 the entropy slices when all, or a portion of, the entropy slices are entropy decoded.
- a decode thread may begin entropy decoding a next entropy slice upon the completion of entropy decoding of an entropy slice .
- the thread may commence decoding additional entropy slices without waiting for other threads to finish their decoding.
- the arrangement of slices may be limited to defining each slice between a pair of macroblocks in the image scan order, also known as raster scan or a raster scan order.
- This arrangement of scan order slices is computationally efficient but does not tend to lend itself to the highly efficient parallel encoding and decoding. Moreover, this scan order definition of slices also does not tend to group smaller localized regions of the image together that are likely to have common characteristics highly suitable for coding efficiency.
- the arrangement of slices, as illustrated in FIG . 4 is highly flexible in its arrangement but does not tend to lend itself to high efficient parallel encoding or decoding. Moreover, this highly flexible definition of slices is computationally complex to implement in a decoder.
- a tile technique divides an image into a set of rectangular (inclusive of square) regions .
- the macroblocks e.g. , largest coding units
- the arrangement of tiles are likewise encoded and decoded in a raster scan order. Accordingly, there may be any suitable number of column boundaries (e. g. , 0 or more) and there may be any suitable number of row boundaries (e . g. , 0 or more) .
- the frame may define one or more slices, such as the one slice illustrated in FIG . 7.
- macroblocks located in different tiles are not available for intra-prediction, motion compensation, entropy coding context selection or other processes that rely on neighboring macroblock information.
- the tile technique is shown dividing an image into a set of three rectangular columns .
- the macroblocks e . g. , largest coding units
- the tiles are likewise encoded and decoded in a raster scan order.
- One or more slices may be defined in the scan order of the tiles. Each of the slices are independently decodable . For example , slice 1 may be defined as including macroblocks 1 -9 , slice 2 may be defined as including macroblocks 10-28 , and slice 3 may be defined as including macroblocks 29- 126 which spans three tiles.
- the use of tiles facilitates coding efficiency by processing data in more localized regions of a frame .
- the entropy encoding and decoding process is initialized at the beginning of each tile .
- this initialization may include the process of writing remaining information in the entropy encoder to the bitstream, a process known as flushing, padding the bitstream with additional data to reach one of a pre-defined set of bitstream positions, and setting the entropy encoder to a known state that is pre-defined or known to both the encoder and decoder.
- the known state is in the form of a matrix of values.
- a pre-defined bitstream location may be a position that is aligned with a multiple number of bits, e . g. byte aligned .
- this initialization process may include the process of setting the entropy decoder to a known state that is known to both the encoder and decoder and ignoring bits in the bitstream until reading from a pre-defined set of bitstream positions .
- multiple known states are available to the encoder and decoder and may be used for initializing the entropy encoding and/ or decoding processes .
- the known state to be used for initialization is signaled in a slice header with an entropy initialization indicator value .
- the tile technique illustrated in FIG. 7 and FIG. 8 tiles and slices are not aligned with one another.
- macroblock 1 is initialized using the entropy initialization indicator value that is transmitted in the slice header but there is no similar entropy initialization indicator value for macroblock 16 of the next tile . Similar entropy initialization indicator information is not typically present for macroblocks 34 , 43 , 63 , 87 , 99 , 109 , and 12 1 for the corresponding tiles for the single slice (which has a slice header for macroblock 1 ) .
- an entropy initialization indicator value is provided in the slice headers for macroblock 1 of slice 1 , provided in the slice header for macroblock 10 of slice 2 , and provided in the slice header for macroblock 29 of slice 3.
- the entropy initialization indicator value for the middle and right hand tiles it is problematic to efficiently encode and decode the macroblocks of the tiles in a parallel fashion and with high coding efficiency.
- the entropy initialization indicator value is provided to explicitly select the entropy initialization information.
- the explicit determination may use any suitable technique, such as for example , indicate that a previous entropy initialization indicator value should be used, such as that in a previous slice header, or otherwise send the entropy initialization indicator value associated with the respective macroblock / tile .
- the slices may include a header that includes an entropy index value
- the first macroblock in a tile may likewise include an entropy initialization indicator value.
- the encoding of this additional information may be as follows:
- tile_cabac_init_idc_present_flag If (num_column_minus 1 >0 && num_rows_minus 1 >0) then tile_cabac_init_idc_present_flag
- the tile_cabac_init_idc_present_flag is a flag indicating how the entropy initialization indicator values are communicated from an encoder to a decoder. For example, if the flag is set to a first value then a first option may be selected such as using a previously communicated entropy initialization indicator value .
- this previously communicated entropy initialization indicator value may be equal to the entropy initialization indicator value transmitted in the slice header corresponding to the slice containing the first macroblock of the tile .
- a second option may be selected such as the entropy initialization indicator value is being provided in the bitstream for the corresponding tile .
- the entropy initialization indicator value is provided within in the data corresponding to the first macro-block of the tile .
- the syntax for signaling the flag indication how the entropy initialization indicator values are communicated from an encoder to a decoder may be as follows:
- a flag in a sequence parameter set e. g. , information regarding a sequence of frames
- a picture parameter set e. g. , information regarding a particular frame
- the syntax may be as follows:
- tile_enable_flag determines if tiles are used in the current picture.
- a technique to provide a suitable entropy initialization indicator value information for a tile may be as follows.
- the technique determines the first macroblock of a tile that may include an entropy initialization indicator value.
- this refers to macroblocks 1, 16, 34, 43, 63, 87, 99, 109, and 121.
- this refers to macroblocks 1, 37, and 100.
- the technique identifies additional tiles within the slice. Referring to FIG. 7, this refers to macroblocks 16, 34, 43, 63, 87, 99, 109, and 121. Referring to FIG.8, this refers to macroblocks 37 and 100.
- tile_cabac_init_idc_flag is equal to a first value and if tiles are enabled. In one specific embodiment, this value is equal to 0. In a second embodiment, this value is equal to 1. In an additional embodiment, tiles are enabled when (num_column_minus 1 >0 && num_rows__minus 1 >0) . In another embodiment, tiles are enabled when tile_enable_flag equal to 1 .
- the cabac_init_idc_present_flag may be set.
- the system may only signal cabac_init_idc_flag if tile_cabac_init_idc_flag is present and if
- the system only sends the entropy information if tiles are being used and the flag indicates the entropy information is being sent (i. e. , cabac_init_idc flag) .
- the coding syntax may be as follows:
- one or more flag(s) associated with the first macroblock (e . g. , coding unit) of a tile not associated with the first macroblock of a slice may define an entropy initialization indicator value .
- a flag may indicate whether the entropy initialization indicator value is previously provided information, a default value, or otherwise entropy initialization indicator value to be provided.
- the decoder knows the location of macroblock 16 in the picture frame but due to entropy encoding is not aware of the positions of bits describing macroblock 16 in the bitstream until macroblock 15 is entropy decoded. This manner of decoding and identifying the next macroblock maintains a low bit overhead, which is desirable . However, it does not facilitate tiles to be decoded in parallel. To increase the ability to identify a specific position in the bitstream for a specific tile in a frame, so that the different tiles may be simultaneously decoded in parallel in the decoder without waiting for completion of the entropy decoding, a signal may be included in the bitstream identifying the location of tiles in the bitstream. Referring to FIG.
- the signaling of the location of tiles in the bitstream is preferably provided in the header of a slice . If a flag indicates that the location of tiles in the bitstream is transmitted within the slice , then in addition to the location within the first macroblock of each of the tile(s) within the slice it also preferably includes the number of such tiles within the frame . Further, the location information may be included for only a selected set of tiles, if desired.
- the coding syntax may be as follows:
- tile_locations_flag signals if the tile locations are transmitted in the bitstream.
- the tile_offset[i] may be signaled using absolute location values or differential size values (change in tile size with respect to previously coded tile) or any suitable technique .
- the encoder can not generally transmit the bitstream until all the tiles are encoded.
- the encoder can transmit only the number of bits necessary to support the identified largest value; the decoder can receive only the number of bits necessary to support the identified largest value . For example, with a relatively small largest value only a small bit depth is necessary for the tile location information . For example, with a relatively large largest value, a large bit depth is necessary for the tile location information .
- markers within the bitstream associated with the start of each tile may be used . These tile markers are included within the bitstream in such a manner that they can be identified without entropy decoding of that particular portion of the bitstream.
- the markers may begin with a start code, which is a sequence of bits that is only present in the bitstream as marker data.
- the marker may include additional headers associated with a tile and / or the first macroblock of the tile . In this manner the encoder can write each tile to the bitstream after it is encoded without waiting until all the tiles are encoded, although the bit rate is increased as a result.
- the decoder can parse the bitstream to identify the different tiles in a more efficient manner, especially when used in conjunction with buffering.
- the tile headers may be similar to the slice headers, although less information is typically included .
- the principal information required is the macroblock number of the next block and entropy initialization data and slice index (indicating, to which slice the starting CU in the tile belongs) .
- the coding syntax of such a tile header may be as illustrated in FIG. 12A.
- the principal information may also include the initial quantization parameter.
- the coding syntax of such a tile header may be as illustrated in FIG. 12B . Values that is not transmitted in the slice header and not in the tile header may be reset to the values transmitted in the slice header.
- markers are included in the bitstream and associated with the start of a tile . However, markers may not be included for every tile in the bitstream. This facilitates and encoder and decoder to operate a different levels of parallelism. For example, an encoder could use 64 tiles while only including 4 markers in the bitstream. This enables parallel encoding with 64 processes and parallel decoding with 4 processes.
- the number of markers in the bitstream is specified in a manner known both to the encoder and decoder. For example, the number of markers may be signaled in the bitstream ⁇ or " defined with a profile or level.
- location data is included in the bitstream and associated with the start of a tile .
- location data may not be included for every tile in the bitstream .
- This facilitates and encoder and decoder to operate a different levels of parallelism.
- an encoder could use 64 tiles while only including 4 locations in the bitstream. This enables parallel encoding with 64 processes and parallel decoding with 4 processes .
- the number of locations in the bitstream is specified in a manner known both to the encoder and decoder. For example, the number of markers may be signaled in the bitstream or defined with a profile or level.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computing Systems (AREA)
- Theoretical Computer Science (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
Description
Claims
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP24150561.9A EP4362459A2 (en) | 2011-03-10 | 2012-03-09 | A method for decoding video |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/045,425 US20120230398A1 (en) | 2011-03-10 | 2011-03-10 | Video decoder parallelization including slices |
US13/045,442 US20120230399A1 (en) | 2011-03-10 | 2011-03-10 | Video decoder parallelization including a bitstream signal |
PCT/JP2012/056786 WO2012121420A1 (en) | 2011-03-10 | 2012-03-09 | A method for decoding video |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP24150561.9A Division EP4362459A2 (en) | 2011-03-10 | 2012-03-09 | A method for decoding video |
Publications (2)
Publication Number | Publication Date |
---|---|
EP2684369A1 true EP2684369A1 (en) | 2014-01-15 |
EP2684369A4 EP2684369A4 (en) | 2014-08-27 |
Family
ID=46798361
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP12755140.6A Ceased EP2684369A4 (en) | 2011-03-10 | 2012-03-09 | A method for decoding video |
EP24150561.9A Pending EP4362459A2 (en) | 2011-03-10 | 2012-03-09 | A method for decoding video |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP24150561.9A Pending EP4362459A2 (en) | 2011-03-10 | 2012-03-09 | A method for decoding video |
Country Status (6)
Country | Link |
---|---|
EP (2) | EP2684369A4 (en) |
JP (4) | JP5947820B2 (en) |
CN (2) | CN106851290B (en) |
MX (2) | MX352139B (en) |
MY (2) | MY163983A (en) |
WO (1) | WO2012121420A1 (en) |
Families Citing this family (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2684369A4 (en) * | 2011-03-10 | 2014-08-27 | Sharp Kk | A method for decoding video |
LT3793200T (en) | 2012-04-13 | 2023-02-27 | Ge Video Compression, Llc | Low delay picture coding |
AU2013283173B2 (en) | 2012-06-29 | 2016-03-24 | Ge Video Compression, Llc | Video data stream concept |
KR102549562B1 (en) * | 2018-07-09 | 2023-06-29 | 프라운호퍼 게젤샤프트 쭈르 푀르데룽 데어 안겐반텐 포르슝 에. 베. | Encoders and decoders, encoding methods and decoding methods for various spatial divisions of coded pictures |
EP3891986A4 (en) * | 2018-12-17 | 2022-03-30 | Huawei Technologies Co., Ltd. | Tile group assignment for raster scan and rectangular tile groups in video coding |
KR102619997B1 (en) * | 2019-01-02 | 2024-01-02 | 애플 인크. | Method for encodign/decodign video signal and apparatus therefor |
US11159795B2 (en) | 2019-03-04 | 2021-10-26 | Tencent America LLC | Max transform size control |
CN112307700B (en) * | 2019-12-17 | 2022-07-29 | 成都华微电子科技股份有限公司 | Bit stream parallel generation method and system of programmable device |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20100232504A1 (en) * | 2009-03-13 | 2010-09-16 | The State of Oregon acting by and through the State Board of Higher Education on behalf of the | Supporting region-of-interest cropping through constrained compression |
Family Cites Families (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP3034172B2 (en) * | 1994-10-25 | 2000-04-17 | 株式会社グラフィックス・コミュニケーション・ラボラトリーズ | Image data recording and playback device |
US6208692B1 (en) * | 1997-12-31 | 2001-03-27 | Sarnoff Corporation | Apparatus and method for performing scalable hierarchical motion estimation |
US8064520B2 (en) * | 2003-09-07 | 2011-11-22 | Microsoft Corporation | Advanced bi-directional predictive coding of interlaced video |
JP4241417B2 (en) * | 2004-02-04 | 2009-03-18 | 日本ビクター株式会社 | Arithmetic decoding device and arithmetic decoding program |
US8213511B2 (en) * | 2007-04-30 | 2012-07-03 | Texas Instruments Incorporated | Video encoder software architecture for VLIW cores incorporating inter prediction and intra prediction |
CN101309405B (en) * | 2007-05-14 | 2011-04-20 | 华为技术有限公司 | Reference data loading method and device |
CN101394550B (en) * | 2007-09-17 | 2010-08-11 | 华为技术有限公司 | Packing method and device for video data |
US8542748B2 (en) * | 2008-03-28 | 2013-09-24 | Sharp Laboratories Of America, Inc. | Methods and systems for parallel video encoding and decoding |
US9445121B2 (en) * | 2008-08-04 | 2016-09-13 | Dolby Laboratories Licensing Corporation | Overlapped block disparity estimation and compensation architecture |
EP2684369A4 (en) * | 2011-03-10 | 2014-08-27 | Sharp Kk | A method for decoding video |
-
2012
- 2012-03-09 EP EP12755140.6A patent/EP2684369A4/en not_active Ceased
- 2012-03-09 CN CN201610837101.3A patent/CN106851290B/en active Active
- 2012-03-09 CN CN201280012388.6A patent/CN103563388B/en active Active
- 2012-03-09 MX MX2016001058A patent/MX352139B/en unknown
- 2012-03-09 EP EP24150561.9A patent/EP4362459A2/en active Pending
- 2012-03-09 WO PCT/JP2012/056786 patent/WO2012121420A1/en active Application Filing
- 2012-03-09 MY MYPI2013003299A patent/MY163983A/en unknown
- 2012-03-09 MX MX2013010310A patent/MX336707B/en unknown
- 2012-03-09 MY MYPI2016001737A patent/MY188970A/en unknown
- 2012-03-09 JP JP2013557348A patent/JP5947820B2/en active Active
-
2016
- 2016-06-02 JP JP2016110704A patent/JP6180588B2/en active Active
-
2017
- 2017-07-18 JP JP2017138979A patent/JP6588507B2/en active Active
-
2019
- 2019-09-12 JP JP2019166423A patent/JP6792685B2/en active Active
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20100232504A1 (en) * | 2009-03-13 | 2010-09-16 | The State of Oregon acting by and through the State Board of Higher Education on behalf of the | Supporting region-of-interest cropping through constrained compression |
Non-Patent Citations (7)
Title |
---|
HOROWITZ M ET AL: "Generalized slices", 95. MPEG MEETING; 24-1-2011 - 28-1-2011; DAEGU; (MOTION PICTURE EXPERT GROUP OR ISO/IEC JTC1/SC29/WG11),, no. m19155, 20 January 2011 (2011-01-20), XP030047722, * |
KIRAN MISRA ET AL: "Lightweight slicing for entropy coding", 4. JCT-VC MEETING; 95. MPEG MEETING; 20-1-2011 - 28-1-2011; DAEGU;(JOINT COLLABORATIVE TEAM ON VIDEO CODING OF ISO/IEC JTC1/SC29/WG11AND ITU-T SG.16 ); URL: HTTP://WFTP3.ITU.INT/AV-ARCH/JCTVC-SITE/,, no. JCTVC-D070, 16 January 2011 (2011-01-16), XP030008110, ISSN: 0000-0015 * |
KIRAN MISRA ET AL: "Periodic inits for wavefront coding functionality", 4. JCT-VC MEETING; 95. MPEG MEETING; 20-1-2011 - 28-1-2011; DAEGU;(JOINT COLLABORATIVE TEAM ON VIDEO CODING OF ISO/IEC JTC1/SC29/WG11AND ITU-T SG.16 ); URL: HTTP://WFTP3.ITU.INT/AV-ARCH/JCTVC-SITE/,, no. JCTVC-D073, 16 January 2011 (2011-01-16), XP030008113, ISSN: 0000-0015 * |
KIRAN MISRA ET AL: "Tiles for Parallel Decoding", 5. JCT-VC MEETING; 96. MPEG MEETING; 16-3-2011 - 23-3-2011; GENEVA;(JOINT COLLABORATIVE TEAM ON VIDEO CODING OF ISO/IEC JTC1/SC29/WG11AND ITU-T SG.16 ); URL: HTTP://WFTP3.ITU.INT/AV-ARCH/JCTVC-SITE/,, no. JCTVC-E412, 12 March 2011 (2011-03-12) , XP030008918, ISSN: 0000-0005 * |
MISRA K ET AL: "New results for entropy slices for highly parallel coding", 3. JCT-VC MEETING; 94. MPEG MEETING; 7-10-2010 - 15-10-2010;GUANGZHOU; (JOINT COLLABORATIVE TEAM ON VIDEO CODING OF ISO/IECJTC1/SC29/WG11 AND ITU-T SG.16 ); URL: HTTP://WFTP3.ITU.INT/AV-ARCH/JCTVC-SITE/,, no. JCTVC-C256, 2 October 2010 (2010-10-02), XP030007963, ISSN: 0000-0019 * |
See also references of WO2012121420A1 * |
WIEGAND T ET AL: "High Efficiency Video Coding (HEVC) text specification Working Draft 1", 3. JCT-VC MEETING; 95. MPEG MEETING; 7-10-2010 - 15-10-2010;GUANGZHOU; (JOINT COLLABORATIVE TEAM ON VIDEO CODING OF ISO/IECJTC1/SC29/WG11 AND ITU-T SG.16 ); URL: HTTP://WFTP3.ITU.INT/AV-ARCH/JCTVC-SITE/,, no. JCTVC-C403, 6 January 2011 (2011-01-06), XP030008032, ISSN: 0000-0018 * |
Also Published As
Publication number | Publication date |
---|---|
WO2012121420A1 (en) | 2012-09-13 |
EP4362459A2 (en) | 2024-05-01 |
MX336707B (en) | 2016-01-28 |
MX2013010310A (en) | 2013-12-02 |
EP2684369A4 (en) | 2014-08-27 |
JP2020010381A (en) | 2020-01-16 |
JP5947820B2 (en) | 2016-07-06 |
CN103563388B (en) | 2016-11-02 |
JP2017208846A (en) | 2017-11-24 |
CN103563388A (en) | 2014-02-05 |
CN106851290A (en) | 2017-06-13 |
JP6792685B2 (en) | 2020-11-25 |
MY163983A (en) | 2017-11-15 |
JP2014511644A (en) | 2014-05-15 |
JP6180588B2 (en) | 2017-08-16 |
MY188970A (en) | 2022-01-16 |
JP2016174414A (en) | 2016-09-29 |
JP6588507B2 (en) | 2019-10-09 |
CN106851290B (en) | 2020-11-03 |
MX352139B (en) | 2017-11-10 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11805253B2 (en) | Processing a video frame having slices and tiles | |
US9667971B2 (en) | Video decoder for slices | |
US9398307B2 (en) | Video decoder for tiles | |
US20120230399A1 (en) | Video decoder parallelization including a bitstream signal | |
US20120230398A1 (en) | Video decoder parallelization including slices | |
EP4362459A2 (en) | A method for decoding video |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20130919 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
DAX | Request for extension of the european patent (deleted) | ||
A4 | Supplementary search report drawn up and despatched |
Effective date: 20140729 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: H04N 19/00 20140101AFI20140806BHEP |
|
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: VELOS MEDIA INTERNATIONAL LIMITED |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
17Q | First examination report despatched |
Effective date: 20180122 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
APBK | Appeal reference recorded |
Free format text: ORIGINAL CODE: EPIDOSNREFNE |
|
APBN | Date of receipt of notice of appeal recorded |
Free format text: ORIGINAL CODE: EPIDOSNNOA2E |
|
APBR | Date of receipt of statement of grounds of appeal recorded |
Free format text: ORIGINAL CODE: EPIDOSNNOA3E |
|
APAF | Appeal reference modified |
Free format text: ORIGINAL CODE: EPIDOSCREFNE |
|
P01 | Opt-out of the competence of the unified patent court (upc) registered |
Effective date: 20230328 |
|
APBT | Appeal procedure closed |
Free format text: ORIGINAL CODE: EPIDOSNNOA9E |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R003 |
|
APAM | Information on closure of appeal procedure modified |
Free format text: ORIGINAL CODE: EPIDOSCNOA9E |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION HAS BEEN REFUSED |
|
18R | Application refused |
Effective date: 20240112 |