JP2015146599A - Inter-layer pixel sample prediction - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a system, an apparatus, and a method, performing scalable video coding by using Inter-layer pixel sample prediction.SOLUTION: The inter-layer pixel sample prediction at an extended layer coding unit, a prediction unit, or a conversion unit uses a reconstituted pixel sample acquired from a basic layer or a lower level extended layer. The pixel sample receives up-sample filtering and/or improved filtering. An up-sample of an improved filtering coefficient is predetermined or adaptively determined.

Description

  The present invention relates to pixel sample prediction between layers.

  High-efficiency video currently under development by the Joint Working Group on Video Coding (JCT-VC) formed by the ISO / IEC Video Experts Group (MPEG) and ITU-T Video Coding Experts Group (VCEG) Encoding (HEVC) is a video compression standard planned to be finalized in 2012. Similar to previous video coding standards, HEVC includes basic functional modules such as intra prediction / inter prediction, transform, quantization, in-loop / filtering, and entropy coding.

  HEVC defines a coding unit (CU) as a sub-partition of a picture that takes the form of a rectangular block having a variable size. Within each CU, a split scheme based on quad-tree specifies a CU partition pattern. HEVC is how it should be partitioned prediction unit that specifies each for a given CU predicted objects and transform objects (PU) and conversion unit (TU) is also defined. After intra prediction or inter prediction, a transform operation is applied to the residual block to generate coefficients. The coefficients are then quantized, scanned to a one-dimensional order, and finally entropy encoded.

  HEVC is expected to include scalable video coding (SVC) extensions. The HEVC SVC bitstream provides several subset bitstreams that represent the source video content with different spatial resolution, frame rate, quality, bit depth, etc. Scalability is then generally achieved using a multi-layer coding structure that includes a base layer (BL) and at least one enhancement layer (EL). This allows a part of a picture belonging to EL or a part of a picture like a CU to be predicted from a lower layer picture (eg BL picture) or from an already encoded picture in the same layer To.

  In the conventional approach, inter prediction and / or intra prediction for the current CU is performed for CUs of pictures inside the same layer. As an example, pixel sample prediction for an EL CU is performed conventionally for the same EL CU and not for another EL or BL CU.

  A method comprising determining, at an enhancement layer (EL) video decoder, predicted pixels of a block of EL frames based at least in part on pixel samples obtained from at least one of a lower EL frame or a base layer (BL) frame.

The subject matter described in this specification is illustrated by way of example and not limitation in the accompanying drawings. For simplicity and clarity of illustration, elements shown in the figures are not necessarily drawn to scale. For example, the dimensions of one element may be exaggerated relative to other elements for clarity. Further, where considered appropriate, reference numerals have been repeated among the figures to indicate corresponding or similar elements.
1 shows a diagram according to an embodiment of an encoding system. 1 shows a diagram according to an embodiment of an encoding system. 3 shows a flowchart according to an embodiment of the process. 1 shows a diagram according to an embodiment of the system. Fig. 4 shows a diagram according to an embodiment of an encoding scheme. Fig. 3 shows a diagram according to an embodiment of a bitstream. 1 shows a diagram according to an embodiment of a decoding system. 3 shows a flowchart according to an embodiment of the process. 1 shows a diagram according to an embodiment of the system. Fig. 4 illustrates an example of an all-placed device in accordance with at least some implementations of the present disclosure.

  One or more embodiments or implementations will now be described with reference to the accompanying drawings. Although specific configurations and arrangements are contemplated, it should be understood that this is done for illustrative purposes only. Those skilled in the art will appreciate that other configurations and arrangements may be utilized without departing from the spirit and scope of the description. It will be apparent to those skilled in the art that the techniques and / or arrangements described herein may be utilized for various other systems and applications that are not described herein.

  The following description describes various implementations that may be manifested in an architecture such as, for example, a system on chip (SoC) architecture, but implementations of the techniques and / or arrangements described herein are Without being limited to a particular architecture and / or computing system, it may be implemented by the architecture and / or computing system for similar purposes. One example is a plurality of integrated circuit (IC) chips and / or packages and / or various computing devices and / or consumer electronics (CE) such as set top boxes, smartphones, etc. Various architectures that utilize the devices may implement the techniques and / or arrangements described herein. Further, the following description may set forth numerous specific details such as the logical implementation, type and interrelationships of the system components, logical partitioning / integration selection, etc., but the claimed subject matter is May be implemented without any specific details. In other cases, certain materials such as, for example, control structures and complete software instruction sequences may not be revealed in detail in order not to obscure the material published herein.

  The subject matter published herein may be implemented in hardware, firmware, software, or any combination thereof. The subject matter published herein may be implemented as instructions stored on a machine-readable medium that may be read and executed by one or more processors. A machine-readable medium may include any medium and / or mechanism for storing or transmitting information in a form readable by a machine (eg, a computing device). For example, the machine readable medium may be read only memory (ROM), random access memory (RAM), magnetic disk storage medium, optical storage medium, flash memory device, electrical, optical, acoustic or other In the form of propagation signals (eg, carrier waves, infrared signals, digitel signals, etc.), and others.

  References to “one implementation”, “implementation”, “example of implementation”, etc. in the specification may include specific features, structures, or characteristics of the described implementation, but every embodiment may have this special feature. , Suggests not necessarily including structure, or property. Moreover, such phrases may not refer to the same implementation. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, such feature in connection with other implementations, whether or not explicitly described herein. It is within the knowledge of one of ordinary skill in the art to achieve any structure, property, or characteristic.

  Systems, apparatus, products, and methods that include operations for video coding that utilize inter-layer pixel sample prediction are described below.

  As used herein, the term “encoder” may refer to an encode and / or a decoder. Similarly, as used herein, the term “encoding” may refer to encoding by an encoder and / or decoding by a decoder. For example, a video encoder and a video decoder may both be examples of encoders that are capable of encoding. Moreover, as used herein, the term “codec” refers to any process such as, for example, some combination of software, firmware, and / or hardware that may implement an encoder and / or decoder, May refer to a set of programs or operations.

  In scalable video coding systems, multi-layer coding is used to support several types of scalability including spatial scalability, temporal scalability, bit depth scalability, and the like. In accordance with the present disclosure, various inter-layer pixel sample prediction schemes may be used to increase coding efficiency and / or coding flexibility in a scalable video coding system.

  FIG. 1 illustrates an example scalable video coding (SVC) coding system 100 in accordance with the present disclosure. In various implementations, the system 100 may be implemented, for example, by the High Efficiency Video Coding (HEVC) standard (ISO / IEC JTC / SC29 / WG11 and ITU-T SG16 WP3, “High efficiency video coding (HEVC) test specification 8” JCTVC-J1003_d7), July 2012) and undertake video compression and decompression according to one or more standards or specifications such as these and some scalable video coding (SVC) extensions and / or video codecs Is implemented. Although the system 100 and / or other systems, schemes or processors may be described herein in the context of the SVC extension of the HEVC standard, the present disclosure is not limited to any particular video encoding standard or specification or these It is not limited to the extension.

  As illustrated, the system 100 includes a layer 0 or base layer (BL) encoder 102, a layer 1 or first enhancement layer (EL) encoder 104, and a layer 2 or second EL encoder 106. An encoder subsystem 101 having a plurality of video encoders. The system 100 further includes a corresponding video decoder in a decoder subsystem 103 that includes a layer 0 (BL) decoder 108, a layer 1 (EL) decoder 110, and a layer 2 (EL) decoder 112. In general, BL may be HEVC compatible encoded. When encoding an EL with a layer identification number (ID) equal to N, the SVC encoding scheme is used when a picture belonging to a particular EL starts from a lower layer picture (eg, in a BL or EL with a lower layer ID). ), Or that any coding layer with a layer ID less than N is available for use in the inter-layer prediction scheme, as may be predicted from pre-coded pictures in the same layer. Guarantee.

  In various implementations, HEVC then specifies a maximum coding unit (LCU) for a picture that may be partitioned into coding units (CUs) that take the form of rectangular blocks having variable sizes. Within each LCU, a quadtree based partitioning scheme specifies a CU partition pattern. HEVC further define how it should be partitioned prediction unit that specifies each for a given CU predicted objects and transform objects (PU) and conversion unit (TU). A CU typically includes one luminance coding block (CB) and two chromaticity CBs with associated syntax, and a PU is a prediction block (ranging from 64 × 64 to 4 × 4 samples in size). PB) may be further divided. As used herein, the term “block” may refer to any partition or subpartition of a video picture. For example, a block may refer to PC or PB.

  In accordance with the present disclosure, as described in more detail below, one or both of EL encoders 104 and 106 were obtained from either encoding 102 or 104 to perform inter-layer pixel sample prediction. Pixel samples may be used. For example, in certain implementations, the encoder 104 may perform inter-layer pixel sample prediction using the pixel samples 114 obtained from the encoder 102 and processed by the inter-layer prediction module 116. Moreover, in certain implementations, the encoder 106 receives either the pixel sample 114 or the pixel sample 118 obtained from the encoder 102 or encoder 104, respectively, and processed by the inter-layer prediction module 120 or inter-layer prediction module 122, respectively. May be used to perform inter-layer pixel sample prediction.

  As used herein, the term “interlayer pixel sample prediction” refers to inter prediction of enhancement layer pictures using pixel samples obtained from a reference layer picture. Further, as used herein, a “pixel sample” refers to a reconstructed pixel. By re-using coding information such as pixel samples reconstructed by a reference layer, inter-layer pixel prediction can be used for SVC systems such as system 100 and / or compression efficiency and coding flexibility of codec designs. May improve. In various implementations according to the present disclosure, inter-layer pixel sample prediction may be applied to temporal scalable video coding applications, spatial scalable video coding applications, and / or quality scalable video coding applications.

  Utilizing any one or more of the inter-layer prediction modules 116, 120 and / or 122 may provide a separate bitstream to the entropy encoder 124. The entropy encoder 124 may then provide a compressed bitstream 126 that includes multiple layers of scalable video content to the entropy decoder 128 of the decoder subsystem 103. In accordance with the present disclosure, as described in more detail below, one or both of EL decoders 110 and 112 were obtained from either decoder 108 or 110 to perform inter-layer pixel sample prediction. Pixel samples may be used. For example, in certain implementations, the decoder 110 may perform inter-layer pixel sample prediction using pixel samples obtained from the decoder 108 and processed by the inter-layer prediction module 130. Moreover, in certain implementations, the decoder 112 uses inter-layer samples using pixel samples obtained from either the decoder 108 or the decoder 110 and processed by the inter-layer prediction module 132 or the inter-layer prediction module 134, respectively. Pixel sample prediction may be performed.

  FIG. 1 illustrates an example system 100 as utilizing three layers of scalable video content and a corresponding set of three encoders in subsystem 101 and three decoders in subsystem 103, Any number of scalable video coding layers and corresponding encoders and decoders may be utilized in accordance with the present disclosure. Further, although system 100 depicts certain components, the present disclosure is not limited to the particular components shown in FIG. 1 and / or the manner in which the various components of system 100 are arranged. In one example, in various implementations, certain elements of system 100, such as, for example, inter-layer prediction modules 116, 120, and 122 of encoder subsystem 101 are coupled to all three encoders 102, 104, and 106, etc. Implemented by a single inter-layer prediction module.

  Further, the encoder subsystem 101 may be associated with, for example, a content provider system that includes a server system, and the bitstream 126 may be a transceiver, antenna, network system, etc. not depicted in FIG. It may be appreciated that various communication components or systems such as may be transmitted or communicated to the decoder subsystem 103. The decoder subsystem 103 is a computing device (eg, a computer, a smartphone, etc.) that receives the bitstream 126 via various communication components or systems such as transceivers, antennas, network systems, etc. that are also not depicted in FIG. May be associated with a client subsystem such as

  FIG. 2 shows an example SVC encoding system 200 according to the present disclosure. System 200 includes a reference BL encoder 202 and a target EL encoder 204 that may correspond to, for example, encoder 102 and encoder 104 of system 100, respectively. System 200 includes only two encoders 202 and 204 corresponding to two SVC coding layers, but any number of SVC coding layers and corresponding encoders can be added to those depicted in FIG. May be used in accordance with the disclosure of the invention. For example, additional encoders corresponding to additional enhancement layers may be incorporated into system 200 and may interact with BL encoder 202 in the same manner as described below with respect to EL encoder 204.

  When utilizing the system 200 to undertake SVC encoding, various blocks of a picture or image frame in the enhancement layer, such as the EL input frame 206, are still processed by the BL encoder 202. Or from other pictures in the same enhancement layer that were pre-encoded by the EL encoder 204. As described in more detail below, when using system 200 to undertake inter-layer pixel sample prediction operations, the pixels of a picture in layer 20, such as EL input frame 206, are pixels provided by BL encoder 202. May be predicted using sample 210. As described above, the EL input frame 206 may be encoded in units corresponding to one or more blocks of pixel values, and the encoded block is in the form of a CU, PU, or TU. There is. Further, encoding may be applied at the slice, picture, or layer level.

  The pixel samples 210 used for inter-layer pixel sample prediction are a transform and quantization module 212, an inverse transform and quantization module 214, an intra prediction module 216, an inter prediction module 218, and an in-loop filtering module. 220 may be obtained from the processing of the BL input frame 208 using a coding loop comprising 220. Specifically, when operating the BL encoder 202 to perform inter prediction using the inter prediction module 218, pixel samples 210 may be obtained from the output of the in-loop filtering module 220. The functionality of modules 212, 214, 216, 218, and 220 is well recognized in the art and will not be described in greater detail herein.

  As described in more detail below, in certain implementations, pixel samples 210 may be processed by upsampling module 222 and / or refinement module 224 before being applied to EL encoder 204. In various implementations, upsampling module 22 and refinement module 224 may be components of an inter-layer prediction module (eg, inter-layer prediction module 116 of system 100). Further, in various embodiments, at least a portion of upsampling module 222 and refinement module 224 may be provided by hardware logic, such as fixed function circuitry.

  In EL encoder 204, the filtered pixel samples 226 provided by refinement module 224 are encoded into EL input frame 206 using an encoding loop that includes transform and quantization module 228 and inverse transform and quantization module 230. May be used to predict pixel samples. When operated to undertake inter-layer pixel sample prediction for EL input frame 206, EL encoder 204 performs intra-prediction module 232 in the coding loop used to compress EL input frame 206, inter prediction. Neither module 234 nor in-loop filtering module 236 may be utilized. Again, the functionality of modules 228, 232, 234, and 236 is well recognized in the art and will not be described in more detail herein. The EL encoder 204 may then use the filtered pixel samples 226 to predict various pixels in various CUs of the EL input frame 206.

  In various implementations, one or both of BL encoder 202 and EL encoder 204 are compressed corresponding to respective encoded residuals of at least a portion of BL input frame 208 and at least a portion of EL input frame 206. The coefficients may be supplied to the entropy encoder module 238. Module 238 then performs lossless compression of the residual and provides, as output from system 200, the multiplexed SVC bitstream contained in the encoded residual. is there. In other implementations, one or both of the BL encoder 202 and EL encoder 204 may use the entropy encoder module 238 to encode the encoded residuals of at least a portion of the BL input frame 208 and at least a portion of the EL input frame 206, respectively. May not supply.

  FIG. 3 shows a flow diagram of an example process 300 according to various implementations of the present disclosure. Process 300 may include one or more operations, functions, or acts as illustrated by one or more of blocks 301, 302, 304, 306, 308, 310, 312, 314, and 316 of FIG. . As a non-limiting example, process 300 may form at least part of a scalable video encoding process for a portion of an EL layer (eg, a CU in this implementation) to be undertaken by encoder system 200.

  Further, the process 300 is described herein in connection with encoding the enhancement layer CU using the scalable video encoding system 400 of FIG. 4 where the system 400 includes a processor 402, an SVC codec module 406, and a memory 408. Will be described in the same way. The processor 402 may instantiate the SVC codec module 406 to perform inter-layer pixel sample prediction in accordance with the present disclosure. In an embodiment of the system 400, the memory 408 includes video content including at least a portion of the BL input frame 208 and / or at least a portion of the EL input frame 206, and filter coefficients as described in more detail below. And other items such as The SVC codec module 406 may be provided by any combination of software logic, firmware logic, and / or hardware logic suitable for implementing the encoding system 200. Memory 408 can be any type of volatile memory (eg, static random access memory (SRAM), dynamic random access memory (DRAM), etc.) or non-volatile memory (eg, flash memory, etc.). It may be a memory. In a non-limiting example, memory 480 may be implemented with a cache memory.

  Process 300 may begin at block 301 where inter-layer pixel prediction is to be performed for the current EL CU. In various implementations, the determination of whether to perform inter-layer pixel prediction may be based on rate-distortion costs. For example, the SVC codec 406 may determine whether to perform inter-layer pixel prediction for the EL CU 502 based on known rate distortion cost techniques. If inter-layer pixel prediction is to be performed, process 300 may continue at block 302, and if inter-layer pixel prediction is not to be performed, process 300 may end.

  Process 300 may continue at block 302. For example, for the current EL CU, one or more co-located blocks of BL corresponding to this EL CU may be determined. For example, FIG. 5 shows the current CU 502 of the EL input frame 206 when the CU 502 spatially corresponds to the co-located block 504 of the BL input frame 208. In this example, the CU 502 corresponds to four co-located blocks 504 in the BL input frame 208. However, in various implementations, depending on the spatial scaling between EL and BL or lower level EL, any number of BL or lower level EL blocks may be co-located with respect to a particular EL CU. In other implementations, only a portion of the BL or lower level EL block may be co-located with the EL CU. Furthermore, in some scalable video coding implementations where spatial scaling is not applied such that the EL and sub-EL or BL have a spatial ratio of 1 (eg when quality scaling is applied between layers without spatial scaling) There is a one-to-one correspondence between the blocks in and the blocks in the lower EL or BL.

  With respect to the embodiment of FIG. 5, determining the co-located block at block 302 may include marking block 504 as currently co-located with respect to EL CU 502 or otherwise labeling. is there. Further, in various implementations, co-located blocks in the BL or lower EL layer may be intra-coded, inter-coded, and / or intra / inter-hybrid coded. Sometimes. Moreover, different blocks in the co-located block set may have different block encodings. In one example, in a non-limiting example, co-location block 504 may have two of the co-location blocks encoded intra and one of the co-location blocks may be inter-coded. The coding modes are mixed so that the remaining co-located blocks may be intra / inter hybrid coded.

  Process 300 may continue to block 304 where pixel samples corresponding to the same location block may be accessed. As an example, referring to FIGS. 4 and 5, block 304 may include an SVC codec 406 using processor 402 to obtain pixel samples corresponding to co-located block 504 from memory 408. As an example, memory 408 may serve as a frame buffer that temporarily stores video content, such as a portion of BL input frame 208 that corresponds to the reconstructed pixel value of block 504.

  At block 306, a determination may be made regarding performing upsampling of the pixel samples obtained at block 304. As an example, in the example of FIG. 5 where CU 502 corresponds to four co-located blocks 504, the pixel samples corresponding to block 504 need to be upsampled to match the size of CU 502. The process 300 may proceed to block 308. In other embodiments, for example, when no spatial scalability is provided, upsampling may not be performed, and process 300 may omit from block 306 to block 310 such that block 308 may not be performed. is there.

  In various implementations, pixel upsampling in block 308 may be performed by applying an interpolation filter to the pixel samples, but the present disclosure is limited to some special form of upsampling filtering. It will not be done. In various implementations, upsampling pixel samples may improve the accuracy of inter-layer pixel prediction and may result in better compression performance for EL. In various implementations, a fixed interpolation filter or an adaptive interpolation filter may be applied at block 308. For implementations that utilize a fixed upsampling filter, the filter coefficients may be predetermined and are used by both the encoder (eg, system 200) and the decoder (described in more detail below). Sometimes. For implementations that utilize an adaptive upsampling filter, the filter coefficients may be determined adaptively (eg, by training) at the encoder, and then as described further below in the bitstream May be sent to the decoder as part of. In various implementations, the interpolation filter applied at block 308 may be a multi-tap multi-complementary filter. Further, in various implementations, block 308 includes an SVC codec 406 that uses hardware logic, such as fixed function circuitry, in processor 402 to apply the interpolation filter coefficients obtained from memory 407 to the pixel samples. There is.

  Process 300 may continue to block 310 where a determination of either the pixel sample obtained at block 304 or an up-sampled pixel obtained from block 308 may be made. In various implementations, applying an improved filter to pixel samples may improve the accuracy of inter-layer pixel prediction and may result in better compression performance for EL. If improvement is selected, the process 300 may proceed to block 212 where, in certain implementations, a two-dimensional (2D) spatial filter may be applied as the improvement filter, It is not limited to a special type of improved filter. In other implementations, spatial filtering may not be performed and process 300 may omit from block 310 to block 314 so that block 312 is not performed. Further, in various implementations, block 312 includes an SVC codec 406 that uses hardware logic, such as fixed function circuitry, in processor 402 to apply the improved filter coefficients obtained from memory 408 to the pixel samples. There is.

In various implementations, if p represents the center pixel before filtering and the surrounding pixels q _{i, j} (i, j = 0,..., N) represent a 2D filter window, then applied in block 312. The improved filter may determine the corresponding filtered center pixel p ′ according to the following equation:

p ′ = Σ _{i, j = 0} ^N q _{i, j} × a _{i, j} + b
In the formula, a _{i, j} (i, j = 0,..., N) is a filter coefficient, and b is an offset factor.

In various implementations, the filter coefficients a _{i, j} may be fixed or may be adaptable. In implementations where the filter coefficients are fixed, the filter coefficients a _{i, j} and offset factor b may be predetermined and utilized by both the encoder and decoder. In implementations where the filter coefficients can be adapted, the filter coefficients a _{i, j} and the offset factor b may be determined adaptively (eg, by training) at the encoder and then the bitstream as further described below. May be sent to the decoder as part of.

  Referring to system 400, in various implementations, the application of the filters at blocks 308 and 312 may be undertaken by SVC codec 406 that uses processor 402 to obtain filter coefficients from memory 408. In an adaptive filter implementation, the SVC codec 406 may use the processor 402 to adaptively determine filter coefficients that may or may not be stored in the memory 408.

  The processor 300 may continue to block 314 where the pixel samples may be used to determine a predicted pixel for the current EL CU. In one example, the SVC codec 406 may use pixel samples that may be obtained at block 304 and filtered at block 308 and / or block 310 to form a prediction signal for the CU 502. In various implementations, block 314 may further include generating a residual for the current EL CU. In one example, the SVC codec 406 generates a residual corresponding to the difference between the pixel sample obtained from block 504 (filtered or otherwise processed) and the pixel value of the CU 502. Sometimes. In other implementations, block 314 may not include generating a residual for the current EL CL. In various implementations, block 314 may be undertaken using hardware logic such as fixed function circuitry to perform the arithmetic operations required to determine the predicted pixel. Further, such hardware logic may allow parallel determination of predicted pixels for various parts of the current PU and / or for multiple PUs. As used herein, a “predicted pixel” is a current PU's that may be predicted based on one or more pixel samples obtained from one or more PUs in one or more lower layers. May refer to the value of the middle pixel sample.

  Process 300 may end at block 316 where a bitstream for the current EL CL may be formed. In various implementations, block 316 forms a bitstream portion corresponding to the current EL CU that includes a header portion and a data portion that may or may not include a compression residual for the current EL CU. May be included. In various implementations, the header portion indicates whether to perform inter-layer pixel prediction for the current EL CL and / or to indicate whether a residual exists for the current CU. May include one or more indicators (eg, one or more flags).

  Further, in various implementations, filter coefficients corresponding to either the upsampling filter coefficients in block 308 or the improved filter coefficients in block 312 are indicated or included in the bitstream formed in block 316. Sometimes. In other implementations, the utilized filter coefficients may be indicated in the header portion of the bitstream formed at block 316, for example.

  FIG. 6 illustrates an example bitstream portion 600 corresponding to an EL CU according to various implementations of the present disclosure. The bit stream unit 600 includes a header unit 602 and a data unit 604. The header part 602 includes one or more indicators 606. In one example, the index 606 may include an index 608 having a value that specifies whether to perform inter-layer pixel prediction for the EL CU. The indicator 606 may or may not include an indicator 610 that specifies whether there is a residual for the EL CU. For example, in one implementation, the indicator 608 may be labeled as “inter_layer_pixel_prediction_flag” and a CU syntax table to indicate whether the current CU (eg, CU 502) uses inter-layer pixel prediction. May be added. Further, when inter_layer_pixel_prediction_flag has a special value (eg, equal to 1), the bitstream header portion 602 may subsequently be labeled as “inter_layer_residual_flag” and there is a residual for the current CU. May include an indicator 610 that may indicate whether or not.

  In various implementations, the indicators 608 and 610 are two scalable video coding modes according to the present disclosure: an inter-layer skip mode in which inter-layer pixel prediction is used without residual, and an inter-layer pixel prediction is residual. It may be used to specify the inter-layer direct mode used with. In this way, in the inter-layer skip mode, the SVC codec 406 may not transmit a residual in the data portion 604 of the bitstream 600. Conversely, in the inter-layer direct mode, the SVC codec 406 may transmit a residual in the data portion 604 of the bitstream 600.

  Although the process 300 is described herein in connection with FIG. 5, the present disclosure discloses inter-layer prediction where all pixels of the EL CU are predicted from samples derived from the same co-located BL block. It is not limited to the execution of pixel prediction. Thus, in various embodiments, a portion of a CU (eg, only certain blocks of the CU) may be predicted for only a portion of the co-located block, while another portion of the CU is identical. May be predicted for other parts of the location block. For example, for a CU corresponding to four co-located BL blocks, the first upper portion of the CU may be predicted based on two upper horizontally adjacent co-located BL blocks, The second lower part of the CU may be predicted based on two lower horizontally adjacent co-located BL blocks.

  Further, in various implementations, different parts of the CU may be predicted from different co-located BL blocks with different encodings. Continuing with the above example, two upper horizontally adjacent identical position BL blocks may be intra-coded, but two lower horizontally adjacent identical position BL blocks may be inter-coded. It may be encoded. In this way, the first part of the CU may be predicted from intra-coded pixel samples, while the second part of the CU may be predicted from inter-coded pixel samples. is there.

  FIG. 7 shows an example SVC decoding system 700 according to the present disclosure. System 700 includes a reference BL decoder 702 and a target EL decoder 704 that may correspond to, for example, decoder 108 and decoder 110 of system 100, respectively. System 700 includes only two decoders 702 and 704 corresponding to the two SVC coding layers, which are depicted in FIG. 7 according to the present disclosure. In addition to these, any number may be used. For example, additional decoders corresponding to additional enhancement layers may be included in system 700 and may interact with BL decoder 702 as described below with respect to EL decoder 704.

  When utilizing the system 700 to undertake SVC encoding, a picture or image frame in an enhancement layer, such as the EL output frame 706, is processed by the EL decoder 704 as it is processed by the BL decoder 702. Or from other pictures pre-encoded by the EL decoder 704 in the same enhancement layer. As described in more detail below, the pixels of a picture in layer 704, such as EL output frame 706, were provided by BL decoder 702 when undertaking inter-layer pixel sample prediction operations using system 700. May be predicted using reconstructed BL pixel samples 710. Sample 710 may be obtained using inverse transform and quantization module 712, inter prediction module 714, and in-loop filtering module 718. In particular, samples 710 may be obtained from the output of the in-loop filtering module 718 when operating the BL decoder 702 to perform inter prediction using the inter prediction module 714.

  As described in more detail below, reconstructed pixel samples 710 may be processed by upsampling module 720 and refinement module 722 before being provided to EL decoder 704. In various implementations, upsampling module 720 and refinement module 722 may be components of an inter-layer prediction module (eg, inter-layer prediction module 130 of system 100). In EL decoder 704, the filtered pixel samples 724 provided by refinement module 322 are combined with inverse transform and quantization module 726 and in-loop filtering module 728 to produce pixels in EL output frame 706. Sometimes used to predict a sample. When operated to assume inter-layer pixel sample prediction for EL output frame 706, EL decoder 704 does not utilize either intra prediction module 730 or inter prediction module 732. The EL decoder 704 may then use the filtered pixel samples 724 to predict various pixels in the various CUs of the EL output frame 706.

  The various components of the systems described herein may be implemented with software logic, firmware logic, and / or hardware logic, and / or any combination thereof. For example, the various components of system 700 may be provided, at least in part, by system on chip (SoC) hardware, such as may be found in a computing system such as a smartphone, for example. is there. Those skilled in the art may recognize that the systems described herein may include additional components not depicted in the corresponding figures. For example, systems 200 and 700 may include additional components such as a bitstream multiplexer module that is not depicted in FIGS. 2 and 7 for clarity.

  FIG. 8 shows a flow diagram of an example process 800 according to various implementations of the present disclosure. Process 800 may include one or more operations, functions or actions as indicated by one or more of blocks 802, 804, 806, 808, 810, 812, 814, 816 and 818 of FIG. . As a non-limiting example, process 800 may form at least part of a scalable video encoding process for a portion of an EL layer (eg, a CU in this implementation), as undertaken by decoder system 700. . Further, the process 800 encodes the enhancement layer CU using the scalable video coding system 400 of FIG. 4 where the SVC codec module 406 may instantiate the decoder system 700 and the example bitstream of FIG. 600 may be described herein in connection with FIG.

  The processor 800 may begin at block 802 where a determination may be made regarding whether to perform inter-layer pixel prediction for the current CU. In various implementations, the SVC codec 406 may assume the block 802 depending on the value of the indicator 608 received in the header portion 602 of the bitstream 600. As an example, if the indicator 608 has a first value (eg, 1), the SVC codec 406 may decide to undertake inter-layer pixel sample prediction for the current CU. Conversely, if the index 608 has a second value (eg, 2), the SVC codec 406 may determine not to assume inter-layer pixel sample prediction for the current CU.

  If block 802 results in a negative decision, process 800 determines whether to apply a skip mode that would result in the current CU being decoded based on one or more pre-decoded CUs. Proceed to block 804, which may occur. If the skip mode is invoked at block 804, the process 800 may return to block 814 where pixel values for the current CU may be reconstructed based on one or more pre-decoded CUs for the skip mode implementation. May go forward.

  Conversely, if the skip mode is not invoked at block 804, the process 800 may make a decision regarding whether intra prediction or inter prediction should be performed for the CU. The process may proceed to 806. If intra prediction is selected, process 800 may proceed to block 808 where intra prediction may be performed using known intra-layer intra prediction techniques. If inter prediction is selected, process 800 may proceed to block 810 where inter prediction may be performed using known inter-layer inter prediction techniques. In various implementations, the SVC codec 406 uses, for example, the intra prediction module 730 of the decoder 704 to assume the block 808 and the inter prediction module 732 of the decoder 704 to assume the block 810, Blocks 804, 806, 808 and 810 may be undertaken.

  Process 800 may continue to block 812 where residual decoding may be undertaken using a known residual decoding technique and the result of either block 808 or 810. Process 800 may then end at block 814 where the CU pixel values may be reconstructed using known techniques and the results of block 812.

  Returning to the review of block 802, if block 802 results in a positive decision, the process 800 may include pixel samples from which the predicted pixel was obtained from the lower layer (s) as described above with respect to the process 300. May proceed to block 816, which may be determined based on In various implementations, the SVC codec 406 may assume block 816 in response to an indicator 608 having a first value (eg, 1). The SVC codec 406 may then obtain, for example, pixel samples corresponding to co-located sub-EL and / or BL blocks, and may or may not apply up-sample filtering to the pixel samples, Then, improved filtering may or may not be applied to the pixel samples. Further, when undertaking upsample filtering and / or refinement filtering of the reconstructed pixel samples, the SVC codec 406 uses the filter coefficients indicated by the bitstream 600 or communicated in the bitstream 600. Do it that way.

  Process 800 may continue to block 818 where a determination may be made as to whether a residual is available for the CU. In various implementations, the bitstream associated with the CU and received by the decoder may include an indicator that specifies whether the bitstream includes a residual for the CU. For example, SVC codec 406 may assume block 818 depending on the value of index 610 in bitstream 600. As an example, if the indicator 610 has a first value (eg, 1), an inter-layer direct mode may be invoked where inter-layer pixel prediction is used with the residual to reconstruct the CU pixel value. Yes, process 800 may proceed to block 812 and hence block 814. Otherwise, if the index 610 has a second value (eg, 0), an inter-layer skip mode is invoked in which inter-layer pixel prediction is used without residual to reconstruct the CU pixel value. The process 800 may proceed directly to block 814 without calling block 812.

  Although process 800 is described herein as a decoding process for an EL CU, the present disclosure is not limited to performing inter-layer pixel sample prediction at the CU level. Thus, in various implementations, the process 800 may be further applied to an EL PU or EL TU. Further, as described above, any inter-layer pixel sample prediction process described herein may be applied in the context of temporal scalable video coding, spatial scalable video coding, and / or quality scalable video coding. is there.

  As shown in FIGS. 3 and 8, implementations of example processes 300 and 800 may include underwriting of any block represented in the illustrated order, but the present disclosure is limited in this regard. Rather, in various embodiments, implementations of processes 200 and 800 may include underwriting only a subset of blocks in the order shown and / or in a different order than the illustrated order.

  Moreover, any one or more of the blocks of FIGS. 3 and 8 may be undertaken in response to instructions provided by one or more computer program products. Such a program product may include, for example, a signal bearing media that provides instructions that, when executed by a processor, may provide the functionality described herein. . A computer program product may be provided on one or more machine-readable media in any form. Thus, for example, a processor including one or more processor core (s) may be responsive to program code and / or instructions or instruction sets communicated to a processor by one or more machine-readable media. , One or more of the blocks represented in FIGS. 3 and 8 may be undertaken. Generally, the machine-readable medium is a program code that causes any of the devices and / or systems described herein to implement at least a portion of the video systems 100, 200, and 700, and / or the SVC codec module 406. May transmit software in the form of instructions or instruction sets.

  As used in any of the implementations described herein, the term “module” refers to software, firmware, and / or configured to provide the functionality described herein. Refers to any combination of hardware. Software may be embodied as a software package, code and / or instruction set or instructions, and as used in any of the implementations described herein, “hardware” may be, for example, a single Or, in some combination, may include firmware that stores hardwired circuitry, programmable circuitry, state machine circuitry, and / or instructions executed by the programmable circuitry. A module may be embodied collectively or individually as a circuit configuration that forms part of a larger system, such as an integrated circuit (IC), system on chip (SoC), and the like.

  FIG. 9 illustrates an example system 900 according to the present disclosure. In various implementations, the system 900 may be a media system, but the system 900 is not limited to this situation. For example, the system 900 can be a personal computer (PC), laptop computer, ultra laptop computer, tablet, touchpad, portable computer, handheld computer, palmtop computer, personal information terminal (PDA), cellular Telephone, combination cellular phone / PDA, television, smart device (eg, smart phone, smart tablet or smart television), mobile internet device (MID), messaging device, data communication device, camera (eg, auto focus / auto) It may be incorporated in an exposure camera, a super telephoto camera, a super zoom camera, a digital single lens reflex (DSLR) camera, or the like.

  In various implementations, the system 900 includes a platform 902 coupled to the display 900. Platform 902 may receive content from content devices such as content service device (s) 930 or content distribution device (s) 940 or from other similar content sources. A navigation controller 950 having one or more navigation feature portions may be used, for example, to interact with platform 902 and / or display 920. Each of these components is described in more detail below.

  In various implementations, platform 902 may include any combination of chipset 905, processor 910, memory 912, storage 914, graphics subsystem 915, application 916, and / or radio 918. Chipset 905 may communicate between processor 910 and memory 912, storage 914, graphics subsystem 915, application 916, and / or radio 918. For example, chipset 905 may include a storage adapter (not shown) that is capable of intercommunication with storage 914.

  The processor 910 may be implemented as a complex instruction set computer (CISC) or reduced instruction set computer (RISC) processor, x86 instruction set compatible processor, multi-core, or other microprocessor or central processing unit (CPU). In various implementations, the processor 910 may be a dual-core processor (s), a dual-core mobile processor (s), etc.

  Memory 912 may be implemented as a volatile memory device such as, but not limited to, random access memory (RAM), dynamic random access memory (DRAM), or static RAM (SRAM). is there.

  Storage 914 includes, but is not limited to, a magnetic disk drive, optical disk drive, tape drive, internal storage device, external storage device, flash memory, battery-backed SDRAM (synchronous DRAM), and / or network accessible It may be implemented as a non-volatile storage device such as a storage device. In various implementations, the storage 914 may include technology that enhances storage performance-enhanced protection for valuable digital media, for example when multiple hard devices are included.

  The graphics subsystem 915 may perform processing of images such as display still images or moving images. Graphics subsystem 915 may be, for example, a graphics processing unit (GPU) or a visual processing unit (VPU). An analog or digital interface may be used to communicatively couple graphics subsystem 915 and display 920. For example, the interface may be any of a high definition multimedia interface, a display port, wireless HDMI, and / or wireless HD compliant technology. Graphics subsystem 915 may be integrated into processor 910 or chipset 905. In certain implementations, graphics subsystem 915 may be a stand-alone device that is communicatively coupled to chipset 905.

  The graphics and / or video processing techniques described herein may be implemented with various hardware architectures. For example, graphics and / or video functionality may be integrated inside the chipset. Alternatively, discrete graphics and / or video processors may be used. As yet another implementation, graphics and / or video functionality may be provided by a general purpose processor including a multi-core processor. In further embodiments, the functionality may be implemented in a consumer electronics device.

  Radio 918 may include one or more radios capable of transmitting and receiving signals using various suitable wireless communication technologies. Such techniques may involve communication across one or more wireless networks. Examples of wireless networks include (but are not limited to) wireless local area networks (WLAN), wireless personal area networks (WPAN), wireless metropolitan area networks (WMAN), cellular networks, and satellite networks. . In communicating across such networks, the radio 918 may operate in accordance with one or more applicable standards of any version.

  In various implementations, the display 920 may include any television type monitor or display. Display 920 may include, for example, a computer display screen, a touch screen display, a video monitor, a device such as a television, and / or a television. The display 920 may be digital and / or analog. In various implementations, the display 920 may be a holographic display. Further, the display 920 may be a transparent surface that receives a visual projection. Such projections may convey various types of information, images, and / or objects. For example, such a projection may be a visual overlay for mobile augmented reality (MAR) applications. Under the control of one or more software applications 916, the platform 902 may display a user interface 922 on the display 920.

  In various implementations, the content service device (s) 930 may be hosted by national, international, and / or stand-alone services, so that the platform 902 may be available over the Internet, for example. Content service device (s) 930 may be coupled to platform 902 and / or display 920. The tablet form 902 and / or content service device (s) 930 may be coupled to the network 960 for communicating (eg, transmitting and / or receiving) media information to and from the network 960. Content distribution device (s) 940 may be further coupled to platform 902 and / or display 920.

  In various implementations, the content service device (s) 930 is a cable television box, a personal computer, a network, a telephone, and an internet connectable device capable of distributing digital information and / or content. Or an electrical appliance and other similar devices capable of one-way or two-way communication of content between the content provider and the platform 902 and / or the display 920 directly via the network 960. May contain. It will be appreciated that content may be communicated unidirectionally and / or bidirectionally via a network 960 between any one of the components of the system 980 and a content provider. Examples of content may include any media information including, for example, video, music, medical and gaming information, and the like.

  Content service device (s) 930 may receive content such as cable television programming, digital information, and / or other content. Examples of content providers may include any cable or satellite television or radio or internet content provider. The provided examples are in no way meant to limit implementation according to the present disclosure.

  In various implementations, the platform 902 may receive control signals from a navigation controller 950 having one or more navigation feature portions. The navigation feature portion of the controller 950 may be used to interact with the user interface 922, for example. In various embodiments, the navigation controller 950 is a computer hardware component (specifically, a human interface device) that allows a user to enter spatial (eg, continuous and multidimensional) data into a computer. It may be a positioning device. Many systems, such as graphical user interfaces (GUIs) and televisions and monitors, allow users to use physical gestures to control and feed data to a computer or television.

  The movement of the navigation feature portion of controller 950 may be replicated on a display (eg, display 920) by movement of a pointer, cursor, focus ring, or other visual indicator displayed on the display. For example, a navigation feature portion located in the navigation controller 950 under the control of the software application 916 may be mapped to a virtual navigation feature portion displayed on the user interface 922, for example. In various embodiments, controller 950 may be integrated into platform 902 and / or display 920 rather than as a separate component. The present disclosure, however, is not limited to the elements or circumstances that are disclosed or described herein.

  In various implementations, a driver (not shown), for example, when enabled, allows a user to instantly turn on and off a television-like platform 902 with a touch of a button after initial activation. May include technology. Program logic allows the platform 902 to stream content to a media adapter or other content service device (s) 930 or content distribution device (s) 940 when the platform is “off”. Sometimes. Additionally, chip 905 may include hardware and / or software support for, for example, 5.1 surround sound audio and / or high definition 7.1 surround sound audio. The driver may include a graphics driver for the integrated graphics platform. In various embodiments, the graphics driver may comprise a peripheral component interconnect (PCI) express graphics card.

  In various implementations, one or more of the components represented in system 900 may be integrated. For example, platform 902 and content service device (s) 930 may be integrated, or platform 902 and content distribution device (s) 940 may be integrated, or, for example, platform 902, content The service device (group) 930 and the content distribution device (group) 940 may be integrated. In various embodiments, platform 902 and display 920 may be an integrated unit. For example, display 920 and content service device (s) 930 may be integrated, or display 920 and content distribution device (s) 940 may be integrated. These examples are not meant to limit the present disclosure.

  In various embodiments, system 900 may be implemented as a wireless system, a wired system, or a combination of both. When implemented as a wireless system, the system 900 includes components and interfaces suitable for communicating over a wireless shared medium, such as one or more antennas, transmitters, receivers, transceivers, amplifiers, filters, control logic, and the like. May contain. Examples of wireless shared media may include portions of the wireless spectrum, such as the RF spectrum. When implemented as a wired system, the system 900 includes an input / output (I / O) adapter, a physical connector that connects the I / O adapter to a corresponding wired communication medium, a network interface card (NIC), a disk controller. May include components and interfaces suitable for communicating via a wired communication medium, such as a video controller, an audio controller, and the like. Examples of wired communication media may include wiring, cables, metal leads, printed circuit boards (PCBs), backplanes, switch fabrics, semiconductor materials, twisted pair wiring, coaxial cables, optical fibers, and the like.

  Platform 902 may establish one or more logical or physical channels to communicate information. The information may include media information and control information. Media information may refer to some data representing content intended for the user. Examples of content may include, for example, voice conversations, audio conferencing, streaming video, electronic mail (“e-mail”) messages, voice mail messages, alphanumeric symbols, graphics, images, video, text, etc. . Data from a voice conversation may be, for example, speech information, silence time, background noise, comfort noise, tone, and the like. Control information may refer to any data that represents a command, instruction or control word directed to an automated system. For example, control information may be used to guide media information through the system or to instruct a node to process media information in a predetermined manner. Embodiments, however, are not limited to the elements or situations depicted or described in FIG.

  As previously mentioned, the system 900 may be embodied in various physical styles or form factors. FIG. 10 shows an implementation of a small form factor device 1000 in which the system 1000 may be implemented. In various embodiments, for example, device 1000 may be implemented as a mobile computing device with wireless capabilities. A mobile computing device may refer to a processing system and any device having a mobile power supply or power supply system, such as one or more batteries.

  As mentioned above, examples of mobile computing devices include personal computers (PCs), laptop computers, ultra laptop computers, tablets, touchpads, portable computers, handheld computers, palmtop computers, Personal information terminal (PDA), mobile phone, combination cellular phone / PDA, television, smart device (eg, smart phone, smart tablet or smart television), mobile internet device (MID), messaging device, data communication device, Cameras (eg, autofocus and autoexposure cameras, super telephoto cameras, super zoom cameras, DSLR cameras) may be included.

  Examples of mobile computing devices include wrist computers, finger computers, ring computers, eyeglass computers, belt clip computers, armband computers, shoe computers, closing computers, and other wearable computers. It may further include a computer worn by such a person. In various embodiments, for example, a mobile computing device may be implemented as a smart phone with the ability to perform computer applications and voice and / or data communications. Certain embodiments may be described using, as an example, a mobile computing device that may be implemented as a smartphone, but other embodiments may use other wireless mobile computing devices as well. It may be observed that it may be used and implemented. Embodiments are not limited to this situation.

  As shown in FIG. 10, device 1000 may include a housing 1002, a display 1004, an input / output (I / O) device 1006, and an antenna 1008. The apparatus 1000 may further include a navigation feature portion 1012. Display 1004 may include any suitable display unit that displays appropriate information for the mobile computing device. I / O device 1006 may include any suitable I / O device that inputs information to a mobile computing device. Examples of I / O device 1006 may include alphanumeric keyboards, numeric keypads, touchpads, input keys, buttons, switches, rocker switches, microphones, speakers, voice recognition devices, software, and the like. Information may be further input to device 1000 using a microphone (not shown). Such information may be digitized by a voice recognition device (not shown). Embodiments are not limited to this situation.

  Various embodiments may be implemented using hardware elements, software elements, or a combination of both. Examples of hardware elements include processors, microprocessors, circuits, circuit elements (eg, transistors, resistors, capacitors, inductors, etc.), integrated circuits, application specific integrated circuits (ASICs), programmable logic devices (PLDs), May include digital signal processors (DSPs), field programmable gate arrays (FPGAs), logic gates, registers, semiconductor devices, chips, microchips, chipsets, and the like. Examples of software elements include software components, programs, applications, computer programs, application programs, system programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, functions, methods, procedures, software It may include an interface, application program interface (API), instruction set, computing code, computer code, code segment, computer code segment, word, value, symbol, or some combination thereof. Determining whether an embodiment is implemented using hardware and / or software depends on the desired calculation rate, power level, heat resistance, processing cycle quota, input data rate, output data rate, memory It can vary according to a number of factors such as resources, data bus speed, and other design or performance constraints.

  One or more aspects of at least one embodiment represent various logic within the processor and, when read by the machine, cause the machine to assemble logic to perform the techniques described herein. May be implemented by representative instructions stored on a machine-readable medium. Such representations, known as “IP cores”, are stored on a tangible machine-readable medium and supplied to various customers or manufacturing facilities for loading into the manufacturing machine that actually creates the logic or processor. May be.

  Although certain feature portions described herein have been described with reference to various implementations, this description is not intended to be construed in a limiting sense. Accordingly, various modifications to the implementation described herein are intended to be within the scope of the present disclosure, as well as other implementations that will be apparent to those of ordinary skill in the art related to the present disclosure. And is considered to be included in the scope.

  According to the present disclosure, pixel samples obtained from a base layer video frame are accessed from an enhancement layer video decoder, and inter-layer pixel prediction of at least a portion of the enhancement layer frame is at least partially responsive to the pixel sample. May be executed. Performing inter-layer pixel prediction may include performing inter-layer pixel prediction at least partially in response to the pixel samples and residuals.

  According to the present disclosure, the enhancement layer frame may be one of a temporal enhancement layer frame, a spatial enhancement layer frame, or a quality enhancement layer frame. Performing inter-layer pixel prediction may include performing inter-layer pixel prediction for at least one of a slice level, a picture level, or a layer level. The enhancement layer frame portion may be one of a coding unit (CU), a prediction unit (PU), or a transform unit (TU).

  According to the present disclosure, performing inter-layer pixel prediction may include performing inter-layer pixel prediction according to an index in the bitstream received by the enhancement layer video decoder. In the first state, the indicator may specify that the enhancement layer video decoder should perform inter-layer pixel prediction, and in the second state, the indicator performs the enhancement layer video decoder performing inter-layer pixel prediction. It may be specified that it should not. The indicator may be placed in one of the first state or the second state depending on the rate distortion cost. A portion of the enhancement layer frame may include one or more blocks of the enhancement layer frame, and a pixel sample may correspond to one or more co-located blocks of the base layer frame. The co-located block of the base layer frame may be an intra coded block, an inter coded block, or an intra / inter hybrid coded block.

  In accordance with the present disclosure, upsample filters may be applied to pixel samples before performing inter-layer pixel prediction. The upsample filter may have a fixed upsample coefficient or may have an adaptive upsample coefficient. Furthermore, the refinement filter may be applied to the pixel samples before performing inter-layer pixel prediction. The improvement filter may have a fixed improvement factor or may have an adaptive improvement factor.

  In accordance with the present disclosure, pixel samples obtained from a base layer video frame may be accessed by an enhancement layer video decoder, and inter-layer pixel prediction of at least a portion of the enhancement layer frame is at least partially pixelated. May be executed depending on the sample. Performing inter-layer pixel prediction includes performing inter-layer pixel prediction based at least in part on pixel samples and residuals. Further, the enhancement layer frame may be entropy encoded after performing inter-layer pixel prediction, and a bitstream including the entropy encoded enhancement layer frame may be generated. An entropy encoded enhancement layer frame may contain residuals.

In accordance with the present disclosure, an index may be generated, and in the first state, the index specifies that inter-layer pixel prediction should be performed on a portion of the enhancement layer frame, and a second In the state, the indicator specifies that inter-layer pixel prediction should not be performed on a portion of the enhancement layer frame. The indicator may then be placed in the bitstream. The indicator may be placed in one of the first state or the second state depending on the rate distortion cost.
The following additional notes are left for the above embodiment.
(Appendix 1)
A method comprising determining, at an enhancement layer (EL) video decoder, predicted pixels of a block of EL frames based at least in part on pixel samples obtained from at least one of a lower EL frame or a base layer (BL) frame.
(Appendix 2)
The method of claim 1, further comprising accessing the pixel sample in memory.
(Appendix 3)
The method of claim 1, wherein the EL frame comprises at least one of a temporal EL frame, a spatial EL frame, or a quality EL frame.
(Appendix 4)
The method of claim 1, wherein determining the prediction pixel comprises determining a prediction pixel for at least one of a slice level, a picture level, or a layer level.
(Appendix 5)
The method of claim 1, wherein the block comprises one of a coding unit (CU), a prediction unit (PU), or a transform unit (TU).
(Appendix 6)
The method of claim 1, wherein determining a prediction pixel comprises determining a prediction pixel in response to an indicator included in a bitstream received by the EL video decoder.
(Appendix 7)
In the first state, the indicator specifies that the EL video decoder is to perform inter-layer pixel prediction, and in the second state, the indicator is that the EL video decoder performs inter-layer pixel prediction. The method of appendix 6, specifying that it should not.
(Appendix 8)
The method of claim 7, wherein the indicator is placed in one of the first state or the second state depending on a rate distortion cost.
(Appendix 9)
The method of claim 1, wherein the pixel sample corresponds to one or more co-located blocks of the lower EL frame or the BL frame.
(Appendix 10)
The method of claim 9, wherein the one or more co-located blocks of the base layer frame comprise at least one of an intra coded block, an inter coded block, or an intra / inter hybrid coded block.
(Appendix 11)
The method of claim 1, further comprising applying an upsample filter to the pixel samples before determining the predicted pixel.
(Appendix 12)
The method of claim 11, wherein the upsampling filter includes one of a fixed upsampling factor or an adaptive upsampling factor.
(Appendix 13)
The method of claim 1 (Appendix 14) further comprising applying an improved filter to the pixel sample prior to determining the predicted pixel.
The method of claim 13, wherein the improvement filter includes one of a fixed improvement factor or an adaptive improvement factor.
(Appendix 15)
The method of claim 1, wherein determining the predicted pixel comprises determining the predicted pixel in response to the pixel sample and residual, at least in part.
(Appendix 16)
16. At least one machine-readable medium comprising a plurality of instructions that, when executed on a computing device, cause the computing device to perform the method of any one of appendices 1-15.
(Appendix 17)
An apparatus configured to perform the method of any one of appendices 1 to 15 when available.
(Appendix 18)
An apparatus comprising means for executing the method according to any one of appendices 1 to 15.
(Appendix 19)
An enhancement layer (EL) video encoder comprises determining predicted pixels of a block of an EL frame in response to pixel samples obtained at least in part from at least one of a lower EL frame or a base layer (BL) frame. Method.
(Appendix 20)
The method of claim 19, further comprising accessing the pixel sample in memory.
(Appendix 21)
Entropy encoding the EL frame after determining the prediction pixel;
Generating a bitstream including the entropy encoded EL frame;
The method according to appendix 19, further comprising:
(Appendix 22)
The method of claim 21, wherein the entropy encoded EL frame includes a residual.
(Appendix 23)
The first state specifies that inter-layer pixel prediction will be performed on the block, and the second state indicates that inter-layer pixel prediction will not be performed on the block. Generating an indicator to specify;
Placing the indicator in the bitstream;
The method according to appendix 21, further comprising:
(Appendix 24)
24. The method of claim 23, further comprising placing the indicator in one of the first state or the second state depending on a rate distortion cost.
(Appendix 25)
The method of claim 19, wherein the EL frame comprises at least one of a temporal EL frame, a spatial EL frame, or a quality EL frame.
(Appendix 26)
The method of clause 19, wherein determining the prediction pixel comprises determining a prediction pixel for at least one of a slice level, a picture level, or a layer level.
(Appendix 27)
The method of claim 19, wherein the block comprises one of a coding unit (CU), a prediction unit (PU), or a transform unit (TU).
(Appendix 28)
The method of claim 19, wherein the pixel sample corresponds to one or more co-located blocks of the lower EL frame or the BL frame.
(Appendix 29)
29. The method of clause 28, wherein the one or more co-location blocks comprise at least one of an intra-coded block, an inter-coded block, or an intra / inter-coded block.
(Appendix 30)
The method of claim 19, further comprising applying an upsample filter to the pixel samples prior to determining the predicted pixel.
(Appendix 31)
The method of claim 30, wherein the upsample filter comprises one of a fixed upsample coefficient or an adaptive upsample coefficient.
(Appendix 32)
The method of claim 19, further comprising applying an improved filter to the pixel samples prior to determining the predicted pixel.
(Appendix 33)
The method of claim 32, wherein the refinement filter comprises one of a fixed improvement factor or an adaptive improvement factor.
(Appendix 34)
The method of claim 19, wherein determining the prediction pixel comprises determining the prediction pixel in response to the pixel sample and a residual, at least in part.
(Appendix 35)
35. At least one machine-readable medium comprising a plurality of instructions that, when executed on a computing device, cause the computing device to perform the method of any one of appendices 19 to 34.
(Appendix 36)
An apparatus configured to perform the method of any one of appendices 19 to 34 when available.
(Appendix 37)
An apparatus comprising means for executing the method according to any one of appendices 19 to 34.

102 Layer 0 (BL) encoder 104 Layer 1 (EL) encoder 106 Layer 2 (EL) encoder 108 Layer 0 (BL) decoder 110 Layer 1 (EL) decoder 112 Layer 2 (EL) decoder 116 Inter-layer prediction 120 Inter-layer prediction 122 Inter-layer prediction 124 Entropy encoder 128 Entropy decoder 130 Inter-layer prediction 132 Inter-layer prediction 134 Inter-layer prediction 202 BL encoder 204 EL encoder 206 EL input frame 208 BL input frame 210 Pixel sample 212 Transformation and quantization 214 Inverse transformation and inverse quantum 216 Intra Prediction 218 Inter Prediction 220 In-Loop Filtering 222 Upsampling 224 Improvement 228 Transform and Quantize 230 Inverse Transform and Inverse Quantization 232 Intra prediction 234 Inter prediction 236 In-loop filtering 238 Entropy encoder 402 Processor 406 SVC codec 408 Memory 602 Header part 604 Data part 701 Entropy decoder 702 BL decoder 704 EL decoder 706 EL output frame 708 BL input frame 710 Reconstructed BL sample 712 Inverse transformation and inverse quantization 714 Inter prediction 716 Intra prediction 718 In-loop filtering 720 Upsampling 722 Improvement 726 Inverse transformation and inverse quantization 728 In-loop filtering 730 Intra prediction 732 Inter prediction 902 Platform 905 Chipset 910 Processor 912 Memory 914 Storage 915 Graphics Subsystem 91 6 Application 918 Radio 920 Display 922 User interface 930 Content service device (group)
940 Content distribution device (s)
960 network

Claims (30)

A circuit that selectively upsamples the chromaticity and luminance samples of the base layer region regardless of whether inter-layer prediction should be used to determine the enhancement layer region, wherein the enhancement layer region comprises the base layer A circuit corresponding to at least a portion of the region;
A circuit that selectively determines the enhancement layer using inter-layer prediction based at least in part on a portion of the base layer region related to the upsampled chromaticity and luminance samples;
A video decoder device. Upsampling the chromaticity and luminance samples of the base layer region includes the use of a multi-tap filter with a fixed coefficient,
The video decoder device according to claim 1. Circuitry for selectively determining the enhancement layer region using intra prediction;
Circuitry that selectively determines the enhancement layer region using inter prediction;
The video decoder device according to claim 1, comprising: Circuitry for improving the upsampled chromaticity and luminance samples;
The video decoder device according to claim 1, comprising: Including a circuit for receiving the bitstream portion;
The bitstream part includes a flag that indicates whether to use inter-layer prediction to determine the enhancement layer region,
The video decoder device according to claim 1. The bitstream part includes a residual and a flag indicating whether to use the residual for determining the enhancement layer region.
The video decoder device according to claim 5. The circuit that selectively determines the enhancement layer using inter-layer prediction based at least in part on a portion of the base layer region that is related to the up-sampled chromaticity and luminance samples. In response to the flag instructing to use the residual to determine a layer region, the residual is also used.
The video decoder device according to claim 6. Radio and
At least one antenna communicatively coupled to the radio;
The video decoder device according to claim 1, comprising: One or more processor circuits;
At least one memory communicatively coupled to the one or more processor circuits;
A display communicatively coupled to the one or more processor circuits, the display displaying video based at least in part on the determined enhancement layer region;
The video decoder device according to claim 8. The base layer region includes a picture;
The video decoder device according to claim 1. The enhancement layer region corresponding to at least a portion of the base layer region includes the enhancement layer region co-located with at least a portion of the base layer region;
The video decoder device according to claim 1. One or more of the circuits include one or more processors, hardware, software executed by firmware, or combinations thereof,
The video decoder device according to claim 1. At least one fixed computer-readable medium comprising instructions stored on the medium, wherein when executed by the computer, the computer comprises:
Regardless of whether or not inter-layer prediction should be used to determine the enhancement layer region, the chromaticity and luminance samples of the base layer region are selectively upsampled, and the enhancement layer region includes at least the base layer region It corresponds to a part, and
Selectively determining the enhancement layer using inter-layer prediction based at least in part on a portion of the base layer region related to the up-sampled chromaticity and luminance samples.
A computer-readable medium. When the computer upsamples the chromaticity and luminance samples of the base layer region, the computer uses a multi-tap filtering technique that includes fixed coefficients.
The computer readable medium of claim 13. When the instructions stored on the medium are executed by a computer, the computer further includes:
Selectively determining the enhancement layer region using intra prediction; and
Selectively determining the enhancement layer region using inter prediction;
The computer readable medium of claim 13. When the instructions stored on the medium are executed by a computer, the computer further includes:
Improving the upsampled chromaticity and brightness samples;
The computer readable medium of claim 13. When the instructions stored on the medium are executed by a computer, the computer further includes:
Receive the bitstream part,
The bitstream part includes a flag indicating whether to use inter-layer prediction to determine the enhancement layer region;
The computer readable medium of claim 13. The bitstream part includes a residual and a flag indicating whether to use the residual for determining the enhancement layer region.
The computer readable medium of claim 17. Selectively determining the enhancement layer using inter-layer prediction based at least in part on a portion of the base layer region that is related to the up-sampled chromaticity and luminance samples; Is also based on the residual in response to the flag instructing to use the residual to determine a region;
The computer readable medium of claim 18. The base layer region includes a picture;
A computer readable medium according to any one of claims 13 to 19. The enhancement layer region corresponding to at least a portion of the base layer region includes the enhancement layer region co-located with at least a portion of the base layer region;
A computer readable medium according to any one of claims 13 to 19. A computer-implemented method comprising:
Regardless of whether inter-layer prediction should be used to determine the enhancement layer region, selectively upsampling chromaticity and luminance samples of the base layer region, wherein the enhancement layer region comprises the base layer A step corresponding to at least a portion of the region;
Selectively determining the enhancement layer using inter-layer prediction based at least in part on a portion of the base layer region related to the upsampled chromaticity and luminance samples;
Including a method. Up-sampling the chromaticity and luminance samples of the base layer region includes using a multi-tap filtering technique that includes fixed coefficients,
The method of claim 22. The method further comprises:
Selectively determining the enhancement layer region using intra prediction;
Selectively determining the enhancement layer region using inter prediction;
23. The method of claim 22, comprising: The method further comprises:
Improving the upsampled chromaticity and brightness samples;
23. The method of claim 22, comprising: The method further comprises:
Receiving a bitstream portion, and
The bitstream part includes a flag that indicates whether to use inter-layer prediction to determine the enhancement layer region,
The method of claim 22. The bitstream part includes a residual and a flag indicating whether to use the residual for determining the enhancement layer region.
27. The method of claim 26. The step of selectively determining the enhancement layer using inter-layer prediction based at least in part on a portion of the base layer region that is related to the upsampled chromaticity and luminance samples is also the enhancement layer. Is also based on the residual in response to the flag instructing to use the residual to determine a region;
28. The method of claim 27. The base layer region includes a picture;
29. A method according to any one of claims 22 to 28. The enhancement layer region corresponding to at least a portion of the base layer region includes the enhancement layer region co-located with at least a portion of the base layer region;
29. A method according to any one of claims 22 to 28.

Images