US9270871B2 - Optimized filter selection for reference picture processing - Google Patents

Optimized filter selection for reference picture processing Download PDF

Info

Publication number
US9270871B2
US9270871B2 US13/877,140 US201113877140A US9270871B2 US 9270871 B2 US9270871 B2 US 9270871B2 US 201113877140 A US201113877140 A US 201113877140A US 9270871 B2 US9270871 B2 US 9270871B2
Authority
US
United States
Prior art keywords
reference picture
information
picture
filter
disparity
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related, expires
Application number
US13/877,140
Other languages
English (en)
Other versions
US20130194505A1 (en
Inventor
Peshala V. Pahalawatta
Yuwen He
Alexandros Tourapis
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Dolby Laboratories Licensing Corp
Original Assignee
Dolby Laboratories Licensing Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Dolby Laboratories Licensing Corp filed Critical Dolby Laboratories Licensing Corp
Priority to US13/877,140 priority Critical patent/US9270871B2/en
Assigned to DOLBY LABORATORIES LICENSING CORPORATION reassignment DOLBY LABORATORIES LICENSING CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: TOURAPIS, ALEXANDROS, HE, YUWEN, PAHALAWATTA, PESHALA
Publication of US20130194505A1 publication Critical patent/US20130194505A1/en
Application granted granted Critical
Publication of US9270871B2 publication Critical patent/US9270871B2/en
Expired - Fee Related legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/14—Picture signal circuitry for video frequency region
    • H04N5/21—Circuitry for suppressing or minimising disturbance, e.g. moiré or halo
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/117—Filters, e.g. for pre-processing or post-processing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H04N19/147—Data rate or code amount at the encoder output according to rate distortion criteria
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
    • H04N19/39—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability involving multiple description coding [MDC], i.e. with separate layers being structured as independently decodable descriptions of input picture data
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/513—Processing of motion vectors
    • H04N19/517—Processing of motion vectors by encoding
    • H04N19/52—Processing of motion vectors by encoding by predictive encoding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
    • H04N19/82—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation involving filtering within a prediction loop

Definitions

  • the present invention relates generally to video processing. More specifically, an embodiment of the present invention relates to optimized reference processing filter selection methods.
  • Multi-layered video codecs provide, for instance, scalability in spatial and temporal resolution, bit-depth, color gamut, and quality.
  • a number of multi-layered video codecs has been standardized by the video coding community.
  • the standardized multi-layered video codecs are the Multiview Video Coding extension (MVC) and the Scalable Video Coding (SVC) extension of the MPEG-4 AVC/H.264 standard.
  • MVC Multiview Video Coding extension
  • SVC Scalable Video Coding
  • FIG. 1 shows an implementation of a multi-layered video codec system.
  • FIG. 2 shows an implementation of a multi-layered video codec system that utilizes reference processing.
  • FIG. 3 shows an implementation of a frame compatible codec system.
  • FIG. 4 shows an implementation of a frame compatible encoder.
  • FIG. 5 shows an embodiment of a filter selection method.
  • FIG. 6 shows an embodiment of a filter selection method that involves single pass encoding.
  • FIG. 7 shows an embodiment of a filter selection method that utilizes information from temporal references.
  • FIG. 8 shows an embodiment of a filter selection method that utilizes motion information from a base layer.
  • a method for selecting a particular filter from among a plurality of filters, the particular filter adapted for use in a reference picture processing unit in a multi-layered video coding system, the multi-layered video coding system comprising a base layer and one or more enhancement layers comprising: a) providing a reference picture and an enhancement layer source picture; b) filtering copies of the reference picture using at least one filter from the plurality of filters to obtain at least one filtered reference picture, wherein each filter is applied to a corresponding copy of the reference picture; c) performing disparity estimation based on the enhancement layer source picture and a full set or subset of the at least one filtered reference picture, wherein the disparity estimation is adapted to generate disparity information; and d) selecting the particular filter based on comparing the disparity information generated in step c), wherein the disparity information is a function of at least one of motion vectors, prediction distortion, intra prediction information, illumination parameters, luma components, chroma components,
  • a method for selecting a particular filter from among a plurality of filters, the particular filter adapted for use in a reference picture processing unit in a multi-layered video coding system, the multi-layered video coding system comprising a base layer and one or more enhancement layers comprising: a) providing a reference picture and an enhancement layer source picture; b) filtering copies of the reference picture using at least one filter from the plurality of filters to obtain at least one filtered reference picture, wherein each filter is applied to a corresponding copy of the reference picture; c) performing disparity estimation based on the enhancement layer source picture and a full set or subset of the at least one filtered reference picture, wherein the disparity estimation is adapted to generate disparity information; d) obtaining distortion information based on the disparity information; and e) selecting the particular filter based on comparing the distortion information generated in step d), wherein the disparity information is a function of at least one of motion vectors, prediction distortion, intra prediction information,
  • a method for selecting a particular filter from among a plurality of filters, the particular filter adapted for use in a reference picture processing unit in a multi-layered video coding system, the multi-layered video coding system comprising a base layer and one or more enhancement layers comprising: a) providing an enhancement layer source picture; b) performing disparity estimation based on the enhancement layer source picture and motion information from a particular layer, wherein the disparity estimation is adapted to generate disparity information; c) obtaining distortion information based on the enhancement layer source picture and the motion information from the particular layer; and d) selecting the particular filter based on comparing the distortion information acquired in step c), wherein the disparity information is a function of at least one of motion vectors, prediction distortion, intra prediction information, illumination parameters, luma components, chroma components, and tone mapping parameters, and wherein the motion information from the particular layer is based on temporal reference pictures of the particular layer.
  • a method for selecting a particular filter from among a plurality of filters, the particular filter adapted for use in a reference picture processing unit in a multi-layered video coding system, the multi-layered video coding system comprising a base layer and one or more enhancement layers comprising: a) providing a reference picture and an enhancement layer source picture; b) filtering copies of the reference picture using at least one filter from the plurality of filters to obtain at least one filtered reference picture, wherein each filter is applied to a corresponding copy of the reference picture; c) performing disparity estimation based on the enhancement layer source picture, a full set or subset of the at least one filtered reference picture, and motion information from a particular layer, wherein the disparity estimation is adapted to generate disparity information; d) obtaining distortion information based on the enhancement layer source picture, the full set or subset of the at least one filtered reference picture, and motion information from the particular layer; and e) selecting the particular filter based on comparing the
  • a method for selecting a particular filter from among a plurality of filters, the particular filter adapted for use in a reference picture processing unit in a video coding system, the coding system comprising a layer comprising: a) providing a reference picture and a source picture, wherein both the reference picture and the source picture are from the same layer; b) filtering copies of the reference picture using at least one filter from the plurality of filters to obtain at least one filtered reference picture, wherein each filter is applied to a corresponding copy of the reference picture; c) performing disparity estimation based on the source picture and a full set or subset of the at least one filtered reference picture, wherein the disparity estimation is adapted to generate disparity information; d) obtaining distortion information based on the disparity information; and e) selecting the particular filter based on comparing the distortion information generated in step d), wherein the disparity information is a function of at least one of motion vectors, prediction distortion, intra prediction information, illumination
  • a filter selector adapted for selecting a particular filter from among a plurality of filters, the particular filter adapted for use in a reference processing unit in a multi-layered video coding system, the multi-layered video coding system comprising a base layer and one or more enhancement layers
  • the filter selector comprising: a full set or subset of the plurality of filters for processing a reference picture or a region of the reference picture to obtain one or more processed reference pictures; and a disparity estimator adapted to generate disparity information based on an enhancement layer source picture and at least one processed reference picture from the one or more processed reference pictures, wherein the disparity information is a function of at least one of motion vectors, prediction distortion, intra prediction information, illumination parameters, luma components, chroma components, and tone mapping parameters, wherein the particular filter is selectable based on the disparity information.
  • a filter selector adapted for selecting a particular filter from among a plurality of filters, the particular filter adapted for use in a reference processing unit in a multi-layered video coding system, the multi-layered video coding system comprising a base layer and one or more enhancement layers
  • the filter selector comprising: a full set or subset of the plurality of filters for processing a reference picture or a region of the reference picture to obtain one or more processed reference pictures; and a disparity estimator adapted to generate disparity information based on an enhancement layer source picture and at least one processed reference picture from the one or more processed reference pictures, wherein the disparity information is a function of at least one of motion vectors, prediction distortion, intra prediction information, illumination parameters, luma components, chroma components, and tone mapping parameters; and a distortion information computation module adapted to generate distortion information based on the enhancement layer source picture and the at least one processed reference picture from the plurality of processed reference pictures, wherein the particular filter is selectable based on the
  • a filter selector adapted for selecting a particular filter from among a plurality of filters, the particular filter adapted for use in a reference processing unit in a multi-layered video coding system, the multi-layered video coding system comprising a base layer and one or more enhancement layers
  • the filter selector comprising: a disparity estimator adapted to generate disparity information based on an enhancement layer source picture and motion information from a particular layer, wherein the disparity information is a function of at least one of motion vectors, prediction distortion, intra prediction information, illumination parameters, luma components, chroma components, and tone mapping parameters, intra prediction information, illumination parameters, and tone mapping parameters; and a distortion information computation module adapted to generate distortion information based on the enhancement layer source picture and the motion information from the particular layer, wherein the particular filter is selected based on the distortion information.
  • multi-layered video codecs such as MVC
  • several encoding/decoding processes may be used in order to encode/decode a base layer and several enhancement layer image sequences, where each enhancement layer usually corresponds to a different view.
  • each enhancement layer usually corresponds to a different view.
  • several independent and dependent encoding/decoding processes may be used in combination in order to encode/decode different views.
  • the independently coded views are typically called the base layer views while the dependently coded views are called the enhancement layer views.
  • FIG. 1 shows an implementation of a video codec system supporting at least a base layer ( 100 ) and an enhancement layer ( 110 ).
  • a reconstructed output picture ( 120 , 122 ) of a base layer encoder ( 102 ) and a base layer decoder ( 104 ) is inserted, along with temporal references ( 106 , 116 ) of each layer ( 100 , 110 ), into a reference picture buffer of an enhancement layer encoder ( 112 ) and an enhancement layer decoder ( 114 ), respectively, prior to encoding/decoding of the corresponding picture ( 120 , 122 ) in the enhancement layer ( 110 ).
  • This particular implementation enables the enhancement layer ( 110 ) to use both its own temporal references ( 116 ) as well as references ( 106 ) from a previously decoded layer for purposes such as motion estimation and compensation.
  • the references ( 106 ) from the previous layer ( 100 ) can improve coding efficiency of the enhancement layer ( 110 ).
  • the video codec system implementation of FIG. 1 could also support additional layers ( 152 , 154 ), where the additional layers also comprise corresponding encoders and decoders.
  • the enhancement layer ( 110 ) shown in FIG. 1 which may also be called an inter-layer (among a plurality of other inter-layers) in such a multi-layer implementation, may provide enhancement layer references for use in subsequent enhancement layers ( 152 , 154 ).
  • FIG. 2 shows a second implementation of a system supporting at least a base layer ( 200 ) and an enhancement layer ( 210 ).
  • a previous layer's references ( 206 , 220 , 222 ) are processed, prior to insertion into the enhancement layer's ( 210 ) reference picture buffer, using a Reference Processing Unit (RPU) ( 230 , 240 ).
  • the RPU ( 230 , 240 ) derives a new reference picture ( 232 , 242 ) for each corresponding layer that is generally better correlated with the enhancement layer source image ( 216 ) to be encoded than the case with no RPUs ( 230 , 240 ) such as the implementation shown in FIG. 1 .
  • the better correlation generally yields better compression efficiency for the enhancement layer ( 210 ).
  • the enhancement layer ( 210 ) may serve as an inter-layer for, and thus provide inter-layer references for, subsequent enhancement layers ( 252 , 254 ). Consequently, an RPU may be used to derive new reference pictures based on references from one layer as well as references from previous layers.
  • Additional processing provided by the RPU ( 230 ) may, by way of example and not of limitation, involve linear/non-linear filtering, motion transformation, motion compensation, illumination compensation, scaling, inverse and forward tone mapping, color format conversion, and gamma correction.
  • the processing may be applied at a region-level on a reference picture, thereby enabling processing with methods of different characteristics to be used for different portions of the reference picture.
  • Processing parameters can be derived in the encoder ( 202 , 212 ), taking into account final reconstructed quality of an output video as well as available bandwidth and computing power, and then signaled to the decoder ( 204 , 214 ). It should be noted that the term “processing”, as used in this disclosure, is equivalent to the term “filtering”. Consequently, processing on a reference picture may be performed by applying filters to the reference picture.
  • Frame compatible stereoscopic 3D delivery refers to delivery of stereoscopic content in which original left and right eye images are first downsampled, with or without filtering, to a lower resolution (typically half the original resolution) and then packed together into a single image frame (typically of the original resolution) prior to encoding.
  • Many subsampling (e.g., horizontal, vertical, and quincunx) and packing (e.g., side-by-side, over-under, line-by-line, and checkerboard) methods are used for frame compatible stereoscopic video delivery.
  • FIG. 3 shows an implementation of a frame compatible full resolution 3D delivery architecture.
  • This implementation encodes a base layer ( 300 ), or an inter-layer, as a frame compatible bitstream ( 308 ).
  • a reference processing unit ( 330 ) is used to process the base layer ( 306 ), or inter-layer reference, and generate a prediction ( 332 ) for an enhancement layer ( 310 ).
  • the enhancement layer contains information that, when combined with base layer decoded images ( 322 ), as well as decoded images from other layers (not shown), generally improves resolution of final decoded left eye view ( 350 ) and right eye view ( 360 ).
  • reference processing filters generally for use within a reference processing unit of an encoder, that conform to the architecture presented in FIGS. 2 and 3 .
  • the encoder generally signals information pertaining to the RPU filter used in the encoder to a corresponding decoder.
  • the decoder generally uses the signaled information to select its RPU filter.
  • the signaling need not necessarily include specific filter coefficients. Instead, the encoder may signal a method by which the decoder is to derive the filter coefficients.
  • the reference processing filters can be derived as shown in FIG. 4 .
  • a base layer reference picture from a base layer reference picture buffer ( 420 ) is provided as an input to an RPU ( 430 ).
  • the input to the RPU ( 430 ) could also be, or include in addition to or instead of the base layer reference picture, reference pictures from other layers.
  • left and right eye source images ( 400 , 410 ) are also input into the RPU ( 430 ).
  • An optimized filter selection (not shown) within the RPU ( 430 ) comprises comparing reference processed images or processed image regions from each filter to a source enhancement layer image ( 400 , 410 ) and choosing the filters that provide optimal cost-distortion trade-off.
  • the filter selection may be obtained using a Lagrangian optimization technique.
  • a Lagrangian optimization technique shown below in Equation ( 1 ) can be used to derive a filter based on cost-distortion criteria:
  • the distortion D f can be computed using a variety of techniques including sum of absolute or squared error between pixels, mean square error, PSNR, weighted sum of transformed square errors, sum of transformed absolute errors, SSIM, multiscale SSIM, and other perceptual image/video quality metrics.
  • the cost C f is generally a function of number of bits used to signal the filter parameters. However, the cost may also be a function of other factors. For instance, in a power/complexity constrained application, the cost may also consider computational complexity and power requirements of applying the filter. In the case that the cost includes multiple factors including but not limited to number of bits, computational complexity, and power requirements, multiple lambda parameters may be used in order to separately tune influence of each factor on overall cost.
  • the filter selection process provided above is independent of motion/disparity estimation and mode decision process of an enhancement layer encoder.
  • the mode decision process generally involves selecting an encoding mode for blocks or macroblocks (such as inter, intra, and skip blocks) to be used by the enhancement layer encoder.
  • the mode decision process is based on the rate-distortion optimization ( 446 ).
  • the mode decision process may comprise, for instance, rate-distortion-complexity optimization.
  • the enhancement layer encoder comprises a motion compensator ( 442 ), a motion estimator ( 444 ), and a rate distortion optimization module ( 446 ), which perform corresponding motion compensation ( 442 ), motion estimation ( 444 ), and rate distortion optimization ( 446 ) processes.
  • the motion compensation ( 442 ), motion estimation ( 444 ), and rate distortion optimization ( 446 ) processes of the enhancement layer encoder are applied based on reference pictures from the enhancement layer reference picture buffer ( 440 ).
  • the processes ( 442 , 444 , 446 ) are applied using processed images from the base layer, or previous enhancement layers, as well as the temporal reference pictures from the enhancement layer reference picture buffer ( 440 ).
  • coding efficiency of the RPU filters (not shown) in the enhancement layer does not take into account the processes of motion compensation ( 442 ), motion estimation ( 444 ), and rate distortion optimization ( 446 ).
  • FIG. 5 shows an embodiment of a filter selection process that takes into account motion estimation ( 525 ) and rate distortion optimization ( 530 ) processes of an enhancement layer encoder.
  • the motion estimation ( 525 ) and rate distortion optimization ( 530 ) processes are performed using a motion estimator ( 525 ) and a rate distortion optimization module ( 530 ), respectively.
  • each region ( 505 ) of a previously encoded base layer picture ( 500 ), also referred to as a base layer reference picture is filtered using each of possible filter ( 510 ) in an RPU (not shown) to obtain a filtered base layer reference picture ( 515 ).
  • references or regions of references may also come from previous inter-layers and be filtered by filters in the RPU to obtain filtered inter-layer reference pictures.
  • the filtered picture ( 515 ) or pictures are then inserted into an enhancement layer reference picture buffer ( 520 ).
  • the enhancement layer encoder then performs motion/disparity estimation ( 525 ) and rate distortion optimization ( 530 ) on the filtered picture ( 515 ) based on the filtered base layer reference picture ( 515 ) and an enhancement layer source image ( 545 ).
  • the motion estimation ( 525 ) and mode decision, which occurs along with the rate distortion optimization ( 530 ), of the enhancement layer encoder will use the filtered base layer picture ( 515 ) as a potential reference, in addition to other temporal reference pictures, and generate distortion and cost estimates of using a particular RPU filter ( 510 ).
  • Factors that determine whether a potential reference is actually used as a reference picture may depend on, for instance, distortion, cost, and computational complexity that result from utilization of the potential reference.
  • the cost estimates may include, for instance, motion costs such as costs for signaling of motion vectors that refer to each reference picture as well as costs for encoding prediction residuals in addition to filter costs.
  • Filter costs may include cost of signaling filter parameters such as filter types, filter coefficients, and so forth.
  • the distortion may be computed using final reconstructed pixel values after motion compensation and coding of enhancement layer source images.
  • the rate-distortion optimization ( 530 ) After performing the motion estimation ( 525 ) and rate-distortion optimization ( 530 ) processes for each of the possible filters ( 510 ), the rate-distortion optimization ( 530 ) outputs rate-distortion cost to a filter selector ( 550 ), which selects one or more filters based on the rate-distortion cost.
  • one filter can be selected for each region of the base layer reference picture ( 500 ).
  • coding modes for an enhancement layer picture can be chosen, where the coding modes are used by the enhancement layer encoder to code blocks, macroblocks, and/or regions into the enhancement layer picture.
  • the filtered base layer picture ( 515 ) may be removed ( 535 ) from the enhancement layer reference buffer ( 520 ).
  • the same filtered base layer picture ( 515 ) may be subsequently re-used in a case of multi-pass encoding or re-evaluated using different criteria.
  • FIG. 5 describes a sequential scheme in which a single enhancement layer picture buffer ( 520 ) is used for the rate-distortion optimization ( 530 ) process.
  • a parallel scheme is also possible in which a plurality of enhancement layer reference picture buffers are generated to store each possible RPU filtered picture ( 515 ) prior to motion estimation and mode selection, such as mode selection based on rate-distortion optimization ( 530 ), at the enhancement layer encoder.
  • each possible RPU filtered picture ( 515 ) could come from other enhancement layer reference picture buffers.
  • the filter selection is based on the reconstructed base layer picture ( 515 ).
  • the filter selection can be performed using original non-encoded base layer, or alternatively inter-layer, images ( 500 ).
  • the original non-encoded enhancement layer images which are used as the input source images ( 545 ) in the filter selection process, may or may not have been processed ( 540 ). Processing ( 540 ) may be performed, for instance, on the original non-encoded enhancement layer images to remove noise and improve the motion estimation process ( 525 ).
  • the filter selection process can perform a single pass encoding and base the filter selection decision on the distortion between the filtered images ( 500 ) and the source images ( 545 ) and the cost of motion estimation and mode decision. Additionally, motion estimation may be performed on a subsampled image or a smaller region within the image to reduce computational complexity.
  • FIG. 5 may involve multi-pass encoding.
  • the rate-distortion optimization process ( 530 ) may involve transforms and/or entropy coding as well as loop filtering. Consequently, the filtered references ( 515 ) may have been encoded multiple times during the filter selection process.
  • motion estimation ( 525 ) and rate distortion optimization ( 530 ) is used in this particular embodiment shown in FIG. 5 .
  • the motion estimation ( 525 ) may be replaced with, for instance, a disparity estimation process performed by a disparity estimator whereas rate distortion optimization ( 530 ) may be replaced with, for instance, rate-distortion-cost optimization.
  • Disparity information may include, by way of example and not of limitation, motion information (e.g., motion vectors and prediction distortion), intra prediction information, illumination parameters, luma components, chroma components, and tone mapping parameters.
  • each of the motion estimation and rate distortion optimization processes given in the following FIGS. 6-8 may also be replaced with, for instance, disparity estimation and rate-distortion-cost optimization processes.
  • FIG. 6 shows an embodiment of a filter selection method that involves single pass encoding. This embodiment captures the effect of motion estimation/compensation on overall performance of an RPU.
  • a base layer reference picture ( 670 ) from a base layer reference picture buffer ( 600 ) is utilized as an input to a plurality of filters ( 610 ). Each filter ( 610 ) outputs a filtered base layer reference picture ( 612 ).
  • the embodiment need not involve the base layer and could instead involve an inter-layer.
  • an inter-layer reference picture from a corresponding inter-layer reference picture buffer may be filtered to output filtered inter-layer reference pictures.
  • motion estimation ( 620 ) is performed on each filtered base layer or inter-layer reference ( 612 ) in order to obtain a prediction for an enhancement layer.
  • the motion estimation ( 620 ) is performed using a motion estimator ( 620 ).
  • the distortion and cost of applying each filter ( 610 ) is then computed.
  • the distortion quantifies, for instance, losses due to motion compensation processes and the cost takes into consideration, for instance, number of bits for signaling motion vectors and/or signaling filter parameters as well as computational complexity of the filter.
  • the distortion and cost of each filter may be computed using a distortion computation module ( 622 ) and a cost computation module ( 622 ), respectively.
  • block size used for the motion estimation process ( 620 ) can be chosen to be constant.
  • the block size can be chosen based on image characteristics such as edge features, texture, color, shape, and size of elements in the image under consideration.
  • a mode decision process (not shown) may also be performed in order to determine size and type of a block, illumination parameters, transformation parameters, quantization parameters, and so forth for each block.
  • the motion estimation may involve the luma and/or chroma components.
  • the distortion may be computed based on distortion of one or more components or may include a combination of the motion estimated distortion of the one or more components and distortion assuming zero motion for other components (whether it be a luma component or other chroma components).
  • the motion estimation may be performed using only a subset of the components. The motion vectors derived from the subset of the components may be used to determine the motion estimated distortion of the other components. The distortion obtained from the different components may also be weighted differently for each component when obtaining the combined distortion over all the components.
  • FIG. 7 shows an embodiment of a filter selection method that captures the effect of temporal references on the performance impact of an RPU on coding processes of video codec systems.
  • an enhancement layer source picture 750
  • not all encoded blocks in an enhancement layer source picture will use a filtered base layer reference picture ( 712 ), or a filtered reference picture from an inter-layer (not shown), as a reference.
  • Some of the blocks may instead use temporal references from an enhancement layer picture buffer ( 702 ) while other blocks may be intra coded.
  • reference pictures ( 712 ) in addition to the filtered base layer, or inter-layer (not shown), enhancement layer temporal references ( 730 ) are also made available to the RPU at an enhancement layer encoder.
  • the RPU can perform motion estimation ( 720 ) using all or some of the available references ( 712 , 730 ) and a reference selector ( 760 ) will determine, for each block of the encoded picture ( 750 ), the reference picture ( 712 , 730 ) that is likely to be used by the enhancement layer encoder. Additionally, the reference selector ( 760 ) determines motion vectors between the selected reference picture ( 712 , 730 ) and the encoded picture as well as makes a mode decision.
  • a smaller set of temporal references ( 730 ) from the enhancement layer picture buffer ( 702 ) than the ones actually available to the encoder may be selected in a temporal region selector ( 740 ), which may be designed to ignore less important temporal references ( 730 ).
  • the importance of the temporal references ( 730 ) may be determined based on a temporal distance from a current enhancement layer picture ( 750 ) to be encoded.
  • the importance of the temporal references ( 730 ) may be determined based on a correlation metric that determines correlation between each of the temporal references ( 730 ) in the reference picture buffer ( 702 ) and the current picture ( 750 ) to be encoded and only M most correlated temporal references will be used in RPU filter decision process, where M is an arbitrary number.
  • a bi-predictive or multi-hypothesis search may also be performed between the base layer filtered references ( 712 ) and the temporal references ( 730 ) in order to generate a more accurate prediction of a subsequent enhancement layer picture.
  • FIG. 8 shows an embodiment for a case where motion information from a base layer, or an inter-layer, is used in place of or in conjunction with enhancement layer temporal references.
  • the base layer could also be one of a plurality of inter-layers.
  • the motion vectors and/or mode decisions used for the base or inter-layer may also be used for the enhancement layer since the different layers share similar motion characteristics.
  • a temporal distortion/cost estimator in the enhancement layer can re-use the mode decisions and motion vectors from the base layer, in order to generate an estimate of temporal distortion and cost of predicting motion vectors and making mode decisions for the enhancement layer.
  • the base layer information ( 880 ) such as motion vectors and mode decisions can be similarly transformed prior to using the motion vectors and mode decisions from the base layer to generate the estimates for the enhancement layer.
  • the relation or transformation, applied on motion vectors of the base layer (or inter-layer) in order to derive the estimated motion vectors for the enhancement layer may be determined based on the motion vectors derived in previously coded pictures or regions for the enhancement layer. Differences between the motion vectors of each layer in previously coded pictures can also be used as a guide to determine a confidence level in the motion estimates. Similarly, the distortion estimates obtained by reusing the motion vectors may be weighted by a confidence level of the distortion estimates.
  • a reference selection ( 860 ) and mode decision ( 890 ) process can be performed that takes into account both the temporal references ( 830 ) and the filtered base layer references ( 812 ) (or inter-layer references) without necessarily increasing computational complexity of the filter selection process.
  • the filter selection methods can be simplified by constraining the motion estimation process ( 820 ) to a lower complexity scheme. For example, in one embodiment, only integer pixel motion estimation is performed on the reference pictures ( 830 , 870 ), which comprises the base layer reference pictures ( 830 ) or inter-layer reference pictures (not shown) and the enhancement layer temporal references ( 870 ).
  • sub-pixel accurate motion estimation may be performed on the reference pictures ( 830 , 870 ) generated using a subset of a plurality of filters ( 810 ) or a subset of the temporal references ( 830 ), while integer pixel motion estimation is performed on the remaining references ( 830 , 870 ).
  • Such simplifications can reduce both computational complexity as well as memory requirements for the enhancement layer encoder.
  • a motion search range may also be adapted based on filter type. Computational complexity of motion estimation can also be reduced by providing more accurate motion vector predictors, which may be derived from motion vectors, around which a motion search is performed.
  • the motion vectors can be extracted from the base layer, inter-layers, or temporally from the enhancement layer itself. Spatial (intra prediction) may also be used.
  • the motion vector predictors can also be derived based on filter type used to generate the reference picture ( 712 , 812 ).
  • filter type used to generate the reference picture ( 712 , 812 ).
  • one possible “filter” that can be used in the RPU is that of simply copying a base layer reference picture without any additional filtering.
  • a motion vector predictor can be used that accounts for the phase offset. Additional reduction in computational complexity can also be obtained by reusing information from previously coded pictures, where the previously coded pictures could include temporal pictures and pictures from various layers. A number of techniques for reusing information from previously coded pictures for filter selection can be applied in this case.
  • the RPU filter decision process will provide a map of the distortion and cost of using each RPU processed image ( 712 , 812 ) as a reference as well as the distortion and cost of using a temporal reference ( 730 , 830 ).
  • the map may be used for selecting a filter for use in the RPU. Additionally, the map may also be used for performing a number of other encoder optimizations that further improve coding efficiency of the enhancement layer.
  • encoding decisions generally made by the RPU may include determining number and shape of the regions over which a base layer reference picture ( 870 ) is divided prior to filtering. Since the cost of signaling the filter parameters increases with the number of regions, region sizes in the RPU are generally assigned to be larger than the block sizes used for motion estimation/compensation.
  • the filter selection process can choose the region size to be equal to the size of a smallest contiguous set of blocks for which the selected filter remains the same.
  • region sizes and shapes may be tested in the filter selector ( 890 ) for both filter distortion and filter cost, and the best performing shape and size for a given distortion and/or cost criteria may be chosen.
  • edge detection may be performed on the base layer reference picture ( 770 , 870 ) in order to determine the number and shape of the regions that are likely to be used by the enhancement layer encoder.
  • the RPU may determine that use of the temporal references ( 730 , 830 ) provides a more desirable trade-off between distortion and cost for an entire picture or a slice of the picture.
  • the RPU may determine that use of the base layer filtered reference ( 770 , 870 ) provides the more desirable trade-off.
  • the RPU may signal to the enhancement layer encoder to modify ordering of the reference pictures ( 730 , 770 , 830 , 870 ) within reference picture buffers ( 700 , 702 , 800 , 802 ) such that more important references, given by the references ( 730 , 770 , 830 , 870 ) that provide the more desirable distortion/cost trade-off, can be signaled using fewer bits than less important references.
  • the more important references which are used more frequently for prediction purposes than the less important references, are generally encoded such that the more important references take fewer bits to signal than the less important references.
  • the RPU can be disabled for the current picture to be encoded, saving both computational time and memory.
  • embodiments of the present disclosure provide a set of schemes for accomplishing filter selection of a filter for use in reference processing units in a multi-layered codec.
  • the selected filter may be used to provide a filtered previously coded layer picture, which is used as a reference picture for an enhancement layer.
  • the single layered codec could use temporal references that may be used for prediction in the single layer. Prior to the prediction, the temporal references may be processed utilizing global or regional motion compensation, prefiltering, and so forth.
  • the methods and systems described in the present disclosure may be implemented in hardware, software, firmware, or combination thereof.
  • Features described as blocks, modules, or components may be implemented together (e.g., in a logic device such as an integrated logic device) or separately (e.g., as separate connected logic devices).
  • the software portion of the methods of the present disclosure may comprise a computer-readable medium which comprises instructions that, when executed, perform, at least in part, the described methods.
  • the computer-readable medium may comprise, for example, a random access memory (RAM) and/or a read-only memory (ROM).
  • the instructions may be executed by a processor (e.g., a digital signal processor (DSP), an application specific integrated circuit (ASIC), or a field programmable logic array (FPGA)).
  • DSP digital signal processor
  • ASIC application specific integrated circuit
  • FPGA field programmable logic array
  • an embodiment of the present invention may thus relate to one or more of the example embodiments that are enumerated in Table 1, below. Accordingly, the invention may be embodied in any of the forms described herein, including, but not limited to the following Enumerated Example Embodiments (EEEs) which described structure, features, and functionality of some portions of the present invention.
  • EEEs Enumerated Example Embodiments
  • a method for selecting a particular filter from among a plurality of filters, the particular filter adapted for use in a reference picture processing unit in a multi-layered video coding system, the multi-layered video coding system comprising a base layer and one or more enhancement layers comprising: a) providing a reference picture and an enhancement layer source picture; b) filtering copies of the reference picture using at least one filter from the plurality of filters to obtain at least one filtered reference picture, wherein each filter is applied to a corresponding copy of the reference picture; c) performing disparity estimation based on the enhancement layer source picture and a full set or subset of the at least one filtered reference picture, wherein the disparity estimation is adapted to generate disparity information; and d) selecting the particular filter based on comparing the disparity information generated in step c), wherein the disparity information is a function of at least one of motion vectors, prediction distortion, intra prediction information, illumination parameters, luma components
  • a method for selecting a particular filter from among a plurality of filters, the particular filter adapted for use in a reference picture processing unit in a multi-layered video coding system, the multi-layered video coding system comprising a base layer and one or more enhancement layers comprising: a) providing a reference picture and an enhancement layer source picture; b) filtering copies of the reference picture using at least one filter from the plurality of filters to obtain at least one filtered reference picture, wherein each filter is applied to a corresponding copy of the reference picture; c) performing disparity estimation based on the enhancement layer source picture and a full set or subset of the at least one filtered reference picture, wherein the disparity estimation is adapted to generate disparity information; d) obtaining distortion information based on the disparity information; and e) selecting the particular filter based on comparing the distortion information generated in step d), wherein the disparity information is a function of at least one of motion vectors, prediction distortion, intra prediction information, illumination parameters, luma components, chro
  • a method for selecting a particular filter from among a plurality of filters, the particular filter adapted for use in a reference picture processing unit in a multi-layered video coding system, the multi-layered video coding system comprising a base layer and one or more enhancement layers comprising: a) providing an enhancement layer source picture; b) performing disparity estimation based on the enhancement layer source picture and motion information from a particular layer, wherein the disparity estimation is adapted to generate disparity information; c) obtaining distortion information based on the enhancement layer source picture and the motion information from the particular layer; and d) selecting the particular filter based on comparing the distortion information acquired in step c), wherein the disparity information is a function of at least one of motion vectors, prediction distortion, intra prediction information, illumination parameters, luma components, chroma components, and tone mapping parameters, and wherein the motion information from the particular layer is based on temporal reference pictures of the particular layer.
  • a method for selecting a particular filter from among a plurality of filters, the particular filter adapted for use in a reference picture processing unit in a multi-layered video coding system, the multi-layered video coding system comprising a base layer and one or more enhancement layers comprising: a) providing a reference picture and an enhancement layer source picture; b) filtering copies of the reference picture using at least one filter from the plurality of filters to obtain at least one filtered reference picture, wherein each filter is applied to a corresponding copy of the reference picture; c) performing disparity estimation based on the enhancement layer source picture, a full set or subset of the at least one filtered reference picture, and motion information from a particular layer, wherein the disparity estimation is adapted to generate disparity information; d) obtaining distortion information based on the enhancement layer source picture, the full set or subset of the at least one filtered reference picture, and motion information from the particular layer; and e) selecting the particular filter based on comparing the distortion information acquired in step d), wherein
  • a method for selecting a particular filter from among a plurality of filters, the particular filter adapted for use in a reference picture processing unit in a video coding system, the coding system comprising a layer comprising: a) providing a reference picture and a source picture, wherein both the reference picture and the source picture are from the same layer; b) filtering copies of the reference picture using at least one filter from the plurality of filters to obtain at least one filtered reference picture, wherein each filter is applied to a corresponding copy of the reference picture; c) performing disparity estimation based on the source picture and a full set or subset of the at least one filtered reference picture, wherein the disparity estimation is adapted to generate disparity information; d) obtaining distortion information based on the disparity information; and e) selecting the particular filter based on comparing the distortion information generated in step d), wherein the disparity information is a function of at least one of motion vectors, prediction distortion, intra prediction information, illumination parameters, luma components, chrom
  • EEE6 The method of any one of Enumerated Example Embodiments 1-2 or 4-5, wherein the reference picture is not previously encoded prior to the step of filtering.
  • EEE7 The method of any one of Enumerated Example Embodiments 1, 2, or 4, wherein the reference picture is a base layer reference picture or an inter-layer reference picture, wherein an inter-layer is a layer from among the one or more enhancement layers.
  • EEE8 The method of any one of Enumerated Example Embodiments 1-2 or 4, wherein the reference picture is a spatial reference picture or a temporal reference picture from the enhancement layer.
  • EEE10 The method of Enumerated Example Embodiment 5, wherein the step of providing further comprises processing the source picture, wherein the processing removes noise in the source picture.
  • EEE11 The method of any one Enumerated Example Embodiments 1-4 or 6-9, wherein the step of providing further comprises processing the enhancement layer source picture, wherein the processing involves applying at least one of filtering, motion transformation, motion compensation, illumination compensation, scaling, inverse and forward tone mapping, color format conversion, and gamma correction.
  • step of providing further comprises processing the enhancement layer source picture, wherein the processing involves applying at least one of filtering, motion transformation, motion compensation, illumination compensation, scaling, inverse and forward tone mapping, color format conversion, and gamma correction.
  • processing involves applying at least one of filtering, motion transformation, motion compensation, illumination compensation, scaling, inverse and forward tone mapping, color format conversion, and gamma correction.
  • the step of performing disparity estimation further comprises obtaining cost information for the at least one filter from among the plurality of filters used in the step of filtering, and wherein the step of selecting is further based on the cost information, wherein: the cost information is a function of the disparity information, number of bits to be used in signaling filter parameters of each filter, number of bits to be used in signaling the motion vectors corresponding to each filtered reference picture, number of bits to be used in signaling the prediction distortion corresponding to each filtered reference picture, computational complexity in applying each filter, and power consumption of each filter.
  • the cost information is a function of the disparity information, number of bits to be used in signaling filter parameters of each filter, number of bits to be used in signaling the motion vectors corresponding to each filtered reference picture, number of bits to be used in signaling the prediction distortion corresponding to each filtered reference picture, computational complexity in applying each filter, and power consumption of each filter.
  • the step of performing disparity estimation further comprises obtaining cost information for the at least one filter from among the plurality of filters used in step b), and wherein the step of selecting is further based on the cost information, wherein: the cost information is a function of at least one of the disparity information, distortion between the enhancement layer source picture and each filtered reference picture, number of bits to be used in signaling filter parameters of each filter, number of bits to be used in signaling the motion vectors corresponding to each filtered reference picture, number of bits to be used in signalizing the prediction distortion corresponding to each filtered reference picture, computational complexity in applying each filter, and power consumption of the filter.
  • the cost information is a function of at least one of the disparity information, distortion between the enhancement layer source picture and each filtered reference picture, number of bits to be used in signaling filter parameters of each filter, number of bits to be used in signaling the motion vectors corresponding to each filtered reference picture, number of bits to be used in signalizing the prediction distortion corresponding to each filtered reference picture, computational complexity in applying each filter, and power
  • the step of obtaining distortion information further comprises obtaining cost information and the step of selecting is further based on the cost information, and wherein the cost estimation is a function of at least one of number of bits to be used in signaling filter parameters of a full set or subset of filters in the plurality of filters, number of bits to be used in signaling the motion information from the particular layer, and power consumption of the full set or subset of filters in the plurality of filters.
  • the cost estimation is a function of at least one of number of bits to be used in signaling filter parameters of a full set or subset of filters in the plurality of filters, number of bits to be used in signaling the motion information from the particular layer, and power consumption of the full set or subset of filters in the plurality of filters.
  • Example Embodiment 21 wherein the reference picture is decomposed using a plurality of region sizes and region shapes to obtain a plurality of reconstructed reference pictures, and wherein each of the steps are performed on the plurality of reconstructed reference pictures.
  • EEE23 The method of Enumerated Example Embodiment 22, wherein the region sizes and the region shapes are determined based on performing edge detection on the reference picture.
  • EEE24 The method of any one of the preceding Enumerated Example Embodiments, wherein the disparity estimation comprises block-based motion estimation.
  • EEE25 The method of Enumerated Example Embodiment 24, wherein motion vectors corresponding to a particular block are adapted to be predicted by motion vectors of blocks neighboring the particular block.
  • EEE26 The method of Enumerated Example Embodiment 21, wherein the reference picture is decomposed using a plurality of region sizes and region shapes to obtain a plurality of reconstructed reference pictures, and wherein each of the steps are performed on the plurality of reconstructed reference pictures
  • Example Embodiment 24 or 25 wherein block size is based on image characteristics of the reference picture, wherein the image characteristics are a function of at least one of a luma component, a chroma component, and edge characteristics of the reference picture and texture, color, shape, and size of elements in the reference picture.
  • EEE27 The method of any one of Enumerated Example Embodiments 24-26, wherein the step of performing disparity estimation or the step of obtaining distortion information also determines at least one of block size and block shape.
  • EEE28 The method of any one of the preceding Enumerated Example Embodiments, wherein the performing disparity estimation comprises integer pixel motion estimation.
  • EEE29 The method of any one of the preceding Enumerated Example Embodiments, wherein the performing disparity estimation comprises integer pixel motion estimation.
  • Example Embodiment 32 wherein the step of selecting temporal reference pictures is based on time difference between the temporal reference pictures and the enhancement layer source picture.
  • EEE34 The method of Enumerated Example Embodiment 33, wherein the step of selecting temporal reference pictures is based on correlation between each of the temporal reference pictures and the enhancement layer source picture.
  • EEE35 The method of any one of Enumerated Example Embodiments 31-34, further comprising a step of prioritizing importance of reference pictures, from high importance to low importance, based on comparing the cost information and the distortion information between the reference pictures to obtain a prioritized set of reference pictures.
  • EEE36 The method of any one of Enumerated Example Embodiments 31-34, further comprising a step of prioritizing importance of reference pictures, from high importance to low importance, based on comparing the cost information and the distortion information between the reference pictures to obtain a prioritized set of reference pictures.
  • Example Embodiment 35 wherein fewer bits are used for signaling pictures in the prioritized set of reference pictures of higher importance than for pictures in the prioritized set of reference pictures of lower importance.
  • EEE37 The method of any one of Enumerated Example Embodiments 31-36, wherein sub- pixel accurate motion estimation is performed on a subset of the at least one filtered reference picture and a subset of the temporal reference pictures while integer pixel motion estimation is performed on remaining filtered reference pictures and remaining temporal reference pictures.
  • EEE38 The method of Enumerated Example Embodiment 35 or 36, wherein reference pictures of importance below a pre-defined threshold are not used in the step of obtaining distortion information.
  • EEE39 The method of Enumerated Example Embodiment 35 or 36, wherein reference pictures of importance below a pre-defined threshold are not used in the step of obtaining distortion information.
  • Example Embodiment 35 or 36 wherein the disparity estimation is performed at higher precision for reference pictures of higher importance and the disparity estimation is performed at lower precision for reference pictures of lower importance.
  • EEE40 The method of Enumerated Example Embodiment 3 or 4, wherein: motion information of the enhancement layer is based on the motion information from the particular layer, differences between the enhancement layer's motion information and the particular layer's motion information are used to compute a confidence level, and motion information comprises motion vectors and prediction distortion.
  • EEE41 The method of Enumerated Example Embodiment 40, wherein the distortion information is based on the differences between the enhancement layer's motion information and the particular layer's motion information, and the distortion information is weighted by the confidence level.
  • EEE42 The method of Enumerated Example Embodiment 35 or 36, wherein the disparity estimation is performed at higher precision for reference pictures of higher importance and the disparity estimation is performed at lower precision for reference pictures of lower importance.
  • Example Embodiment 3 or 4 wherein the motion information is based on at least one of motion vectors obtained from the particular layer, motion models used in the particular layer, and motion vectors from previously coded pictures of the particular layer or regions of the previously coded pictures of the particular layer.
  • step of providing comprises providing a plurality of reference pictures and the step of filtering is performed on a full set or subset of the plurality of reference pictures to obtain at least one filtered reference picture, and wherein each filter is applied to each reference picture of the full set or subset of the plurality of reference pictures.
  • a filter selector adapted for selecting a particular filter from among a plurality of filters, the particular filter adapted for use in a reference processing unit in a multi-layered video coding system, the multi-layered video coding system comprising a base layer and one or more enhancement layers, the filter selector comprising: a full set or subset of the plurality of filters for processing a reference picture or a region of the reference picture to obtain one or more processed reference pictures; and a disparity estimator adapted to generate disparity information based on an enhancement layer source picture and at least one processed reference picture from the one or more processed reference pictures, wherein the disparity information is a function of at least one of motion vectors, prediction distortion, intra prediction information, illumination parameters, luma components, chroma components, and tone mapping parameters, wherein the particular filter is selectable based on the disparity information.
  • a filter selector adapted for selecting a particular filter from among a plurality of filters, the particular filter adapted for use in a reference processing unit in a multi-layered video coding system, the multi-layered video coding system comprising a base layer and one or more enhancement layers, the filter selector comprising: a full set or subset of the plurality of filters for processing a reference picture or a region of the reference picture to obtain one or more processed reference pictures; and a disparity estimator adapted to generate disparity information based on an enhancement layer source picture and at least one processed reference picture from the one or more processed reference pictures, wherein the disparity information is a function of at least one of motion vectors, prediction distortion, intra prediction information, illumination parameters, luma components, chroma components, and tone mapping parameters; and a distortion information computation module adapted to generate distortion information based on the enhancement layer source picture and the at least one processed reference picture from the plurality of processed reference pictures, wherein the particular filter is selectable based on the distortion information.
  • the filter selector of Enumerated Example Embodiment 47 or 48 further comprising a cost information computation module adapted to generate cost information, wherein the cost information is a function of at least one of distortion between the enhancement layer source picture and the at least one filtered reference picture, number of bits to be used in signaling filter parameters of each filter, number of bits to be used in signaling the motion vectors corresponding to each filtered reference picture, number of bits to be used in signaling the prediction distortion corresponding to each filtered reference picture, computational complexity in applying each filter, and power consumption of the filter.
  • the cost information is a function of at least one of distortion between the enhancement layer source picture and the at least one filtered reference picture, number of bits to be used in signaling filter parameters of each filter, number of bits to be used in signaling the motion vectors corresponding to each filtered reference picture, number of bits to be used in signaling the prediction distortion corresponding to each filtered reference picture, computational complexity in applying each filter, and power consumption of the filter.
  • a filter selector adapted for selecting a particular filter from among a plurality of filters, the particular filter adapted for use in a reference processing unit in a multi-layered video coding system, the multi-layered video coding system comprising a base layer and one or more enhancement layers
  • the filter selector comprising: a disparity estimator adapted to generate disparity information based on an enhancement layer source picture and motion information from a particular layer, wherein the disparity information is a function of at least one of motion vectors, prediction distortion, intra prediction information, illumination parameters, luma components, chroma components, and tone mapping parameters, intra prediction information, illumination parameters, and tone mapping parameters; and a distortion information computation module adapted to generate distortion information based on the enhancement layer source picture and the motion information from the particular layer, wherein the particular filter is selected based on the distortion information.
  • EEE51 The filter selector of any one of Enumerated Example Embodiments 47-50, wherein the particular filter is selected based on the disparity information and temporal reference pictures from an enhancement layer reference picture buffer.
  • EEE52 The filter selector of Enumerated Example Embodiment 51, further comprising a reference selector that selects the temporal reference pictures for use in selecting the particular filter.
  • EEE53 The filter selector of Enumerated Example Embodiment 52, wherein the reference selector prioritizes the temporal reference pictures based on comparing the disparity information and the cost information of the temporal reference pictures.
  • EEE54 The filter selector of any one of Enumerated Example Embodiments 47-50, wherein the particular filter is selected based on the disparity information and temporal reference pictures from an enhancement layer reference picture buffer.
  • the distortion information computation module computes the distortion information using the group consisting of sum of absolute or squared errors between pixels, mean square error, PSNR, weighted sum of transformed square errors, sum of transformed absolute errors, SSIM, and multiscale SSIM.
  • a reference processing unit comprising a plurality of filters and the filter selector of Enumerated Example Embodiment 55, wherein the filter selector selects a particular filter from the plurality of filters for use in the reference processing unit.
  • EEE57. A computer-readable storage medium containing a set of instructions that causes a computer to perform the method recited in any one of Enumerated Example Embodiments 1- 46.
  • EEE58. An encoder for encoding a video signal comprising a reference processing unit, wherein the reference processing unit selects a filter based on the method recited in any one of Enumerated Example Embodiments 1-46.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
US13/877,140 2009-04-20 2011-09-19 Optimized filter selection for reference picture processing Expired - Fee Related US9270871B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US13/877,140 US9270871B2 (en) 2009-04-20 2011-09-19 Optimized filter selection for reference picture processing

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
US17099509P 2009-04-20 2009-04-20
US22302709P 2009-07-04 2009-07-04
US30042710P 2010-02-01 2010-02-01
US38918010P 2010-10-01 2010-10-01
PCT/US2011/052210 WO2012044487A1 (en) 2010-10-01 2011-09-19 Optimized filter selection for reference picture processing
US13/877,140 US9270871B2 (en) 2009-04-20 2011-09-19 Optimized filter selection for reference picture processing

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2011/052210 A-371-Of-International WO2012044487A1 (en) 2009-04-20 2011-09-19 Optimized filter selection for reference picture processing

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US15/004,582 Continuation US9521413B2 (en) 2009-04-20 2016-01-22 Optimized filter selection for reference picture processing

Publications (2)

Publication Number Publication Date
US20130194505A1 US20130194505A1 (en) 2013-08-01
US9270871B2 true US9270871B2 (en) 2016-02-23

Family

ID=44681446

Family Applications (2)

Application Number Title Priority Date Filing Date
US13/877,140 Expired - Fee Related US9270871B2 (en) 2009-04-20 2011-09-19 Optimized filter selection for reference picture processing
US15/004,582 Expired - Fee Related US9521413B2 (en) 2009-04-20 2016-01-22 Optimized filter selection for reference picture processing

Family Applications After (1)

Application Number Title Priority Date Filing Date
US15/004,582 Expired - Fee Related US9521413B2 (en) 2009-04-20 2016-01-22 Optimized filter selection for reference picture processing

Country Status (5)

Country Link
US (2) US9270871B2 (zh)
EP (1) EP2622857A1 (zh)
JP (1) JP5663093B2 (zh)
CN (1) CN103141099B (zh)
WO (1) WO2012044487A1 (zh)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150055702A1 (en) * 2012-03-06 2015-02-26 Lg Electronics Inc. Method of processing video signals
US20160037173A1 (en) * 2011-10-26 2016-02-04 Intellectual Discovery Co., Ltd. Scalable video coding method and apparatus using intra prediction mode
US20160295219A1 (en) * 2013-09-20 2016-10-06 Vid Scale, Inc. Systems and methods for providing 3d look-up table coding for color gamut scalability
US9912925B2 (en) 2013-12-13 2018-03-06 Vid Scale, Inc. Providing 3D look-up table (LUT) estimation for color gamut scalability
US10218971B2 (en) * 2012-09-28 2019-02-26 Vid Scale, Inc. Adaptive upsampling for multi-layer video coding
US10334263B2 (en) * 2013-04-05 2019-06-25 Samsung Electronics Co., Ltd. Interlayer video encoding method and apparatus and interlayer video decoding method and apparatus for compensating luminance difference

Families Citing this family (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130194386A1 (en) * 2010-10-12 2013-08-01 Dolby Laboratories Licensing Corporation Joint Layer Optimization for a Frame-Compatible Video Delivery
US20140003504A1 (en) * 2012-07-02 2014-01-02 Nokia Corporation Apparatus, a Method and a Computer Program for Video Coding and Decoding
US20140247878A1 (en) * 2012-09-21 2014-09-04 Lidong Xu Cross-layer motion vector prediction
KR101835360B1 (ko) 2012-10-01 2018-03-08 지이 비디오 컴프레션, 엘엘씨 향상 레이어에서 변환 계수 블록들의 서브블록-기반 코딩을 이용한 스케일러블 비디오 코딩
US10080064B2 (en) * 2013-03-15 2018-09-18 Oath Inc. Geo, segment, uniques distributed computing system
US9648353B2 (en) * 2013-04-04 2017-05-09 Qualcomm Incorporated Multiple base layer reference pictures for SHVC
US9380305B2 (en) * 2013-04-05 2016-06-28 Qualcomm Incorporated Generalized residual prediction in high-level syntax only SHVC and signaling and management thereof
CN105308970B (zh) * 2013-04-05 2018-11-23 三星电子株式会社 针对整数像素的位置对视频进行编码和解码的方法和设备
US9860529B2 (en) * 2013-07-16 2018-01-02 Qualcomm Incorporated Processing illumination compensation for video coding
KR101946039B1 (ko) * 2013-09-24 2019-02-08 브이아이디 스케일, 인크. 스케일가능한 비디오 코딩을 위한 계층간 예측
WO2015056961A1 (ko) * 2013-10-14 2015-04-23 삼성전자 주식회사 휘도 보상 여부에 따른 뷰 합성 예측 적용 방법 및 장치
FR3027480B1 (fr) * 2014-10-17 2020-10-23 Institut Nat Des Sciences Appliquees De Rennes Decodeur, procede et systeme de decodage de flux multimedia
JP6409516B2 (ja) * 2014-11-13 2018-10-24 富士通株式会社 ピクチャ符号化プログラム、ピクチャ符号化方法及びピクチャ符号化装置
US10462480B2 (en) * 2014-12-31 2019-10-29 Microsoft Technology Licensing, Llc Computationally efficient motion estimation
US10554968B2 (en) 2015-06-10 2020-02-04 Lg Electronics Inc. Method and apparatus for inter prediction on basis of virtual reference picture in video coding system
US9807339B2 (en) * 2015-06-12 2017-10-31 Sharp Laboratories Of America, Inc. Frame rate conversion system
US20180316914A1 (en) * 2015-10-30 2018-11-01 Sony Corporation Image processing apparatus and method
US10075671B2 (en) 2016-09-26 2018-09-11 Samsung Display Co., Ltd. System and method for electronic data communication
US10469857B2 (en) 2016-09-26 2019-11-05 Samsung Display Co., Ltd. System and method for electronic data communication
US10523895B2 (en) 2016-09-26 2019-12-31 Samsung Display Co., Ltd. System and method for electronic data communication
US10616383B2 (en) * 2016-09-26 2020-04-07 Samsung Display Co., Ltd. System and method for electronic data communication
CN108154480B (zh) * 2016-12-05 2020-11-06 广东精点数据科技股份有限公司 一种基于Adaboosting算法思想的图像去噪方法及装置
CN110999296B (zh) * 2017-06-21 2022-09-02 Vid拓展公司 解码360度视频的方法、设备及计算机可读介质
KR102030630B1 (ko) * 2018-04-30 2019-11-08 서울과학기술대학교 산학협력단 하이브리드 망 기반의 스테레오스코픽 3차원 360도 가상 현실 서비스 제공 시스템

Citations (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1493157A (zh) 2001-09-12 2004-04-28 ���µ�����ҵ��ʽ���� 图像编码方法和图像解码方法
US20040179620A1 (en) * 2002-07-11 2004-09-16 Foo Teck Wee Filtering intensity decision method, moving picture encoding method, and moving picture decoding method
US20060146941A1 (en) * 2005-01-04 2006-07-06 Samsung Electronics Co., Ltd. Deblocking control method considering intra BL mode and multilayer video encoder/decoder using the same
JP2006295913A (ja) 2005-04-11 2006-10-26 Sharp Corp 空間的スケーラブルコーディングのためのアダプティブアップサンプリング方法および装置
CN101589625A (zh) 2006-10-25 2009-11-25 弗劳恩霍夫应用研究促进协会 质量可缩放编码
US20100002770A1 (en) * 2008-07-07 2010-01-07 Qualcomm Incorporated Video encoding by filter selection
JP2010507961A (ja) 2006-10-25 2010-03-11 韓國電子通信研究院 マルチビュービデオのスケーラブルコーディングおよびデコーディング方法、ならびにコーディングおよびデコーディング装置
WO2010075348A2 (en) 2008-12-22 2010-07-01 Unisys Corporation Java enterprise resource management system and method
US20110243231A1 (en) * 2010-04-02 2011-10-06 National Chiao Tung University Selective motion vector prediction method, motion estimation method and device thereof applicable to scalable video coding system
US20120027079A1 (en) 2009-04-20 2012-02-02 Dolby Laboratories Licensing Corporation Adaptive Interpolation Filters for Multi-Layered Video Delivery
US20120026288A1 (en) 2009-04-20 2012-02-02 Dolby Laboratories Licensing Corporation Directed Interpolation and Data Post-Processing
US20120092452A1 (en) 2009-07-04 2012-04-19 Dolby Laboratories Licensing Corporation Encoding and Decoding Architectures for Format Compatible 3D Video Delivery
JP2012516657A (ja) 2009-01-29 2012-07-19 ドルビー ラボラトリーズ ライセンシング コーポレイション 立体画像である複数の画像をサブサンプリング及びインタリーブする方法及び装置
JP2012521734A (ja) 2009-04-20 2012-09-13 ドルビー ラボラトリーズ ライセンシング コーポレイション フィルタを選択する方法及び装置
JP2013518465A (ja) 2010-01-27 2013-05-20 ドルビー ラボラトリーズ ライセンシング コーポレイション 画像及び映像コーデックにおける参照処理のための方法及びシステム
US20130136371A1 (en) * 2010-06-17 2013-05-30 Sharp Kabushiki Kaisha Image filter device, decoding apparatus, encoding apparatus, and data structure
JP5235035B2 (ja) 2008-09-23 2013-07-10 ドルビー ラボラトリーズ ライセンシング コーポレイション チェッカーボード多重化したイメージデータのエンコーディング構造及びデコーディング構造
JP2013534117A (ja) 2010-07-08 2013-08-29 ドルビー ラボラトリーズ ライセンシング コーポレイション 参照処理信号を使った多層画像およびビデオ送達のためのシステムおよび方法
JP2013535901A (ja) 2010-07-21 2013-09-12 ドルビー ラボラトリーズ ライセンシング コーポレイション ビデオ符号化のための高度な動きモデルを使った参照処理
JP2013538487A (ja) 2010-07-21 2013-10-10 ドルビー ラボラトリーズ ライセンシング コーポレイション マルチレイヤフレーム準拠ビデオ配信のためのシステム及び方法
JP2013542648A (ja) 2010-09-24 2013-11-21 クゥアルコム・インコーポレイテッド ステレオビデオデータの符号化

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
IT1212993B (it) * 1983-08-11 1989-12-07 Sip Equalizzatore adattivo per segnali numerici soggetti a distorsioni variabili nel tempo
US6453069B1 (en) * 1996-11-20 2002-09-17 Canon Kabushiki Kaisha Method of extracting image from input image using reference image
US7957466B2 (en) * 2005-09-16 2011-06-07 Sony Corporation Adaptive area of influence filter for moving object boundaries
CN101491101B (zh) * 2006-07-18 2011-11-16 汤姆森许可贸易公司 用于自适应参考滤波的方法和装置
KR100815486B1 (ko) * 2006-08-18 2008-03-20 광주과학기술원 다시점 비디오의 인코더, 인코딩하는 방법, 및 이를 이용한 저장 매체
US9014280B2 (en) * 2006-10-13 2015-04-21 Qualcomm Incorporated Video coding with adaptive filtering for motion compensated prediction
EP2104358A4 (en) * 2006-11-30 2016-04-27 Ntt Docomo Inc DEVICE, METHOD AND PROGRAM FOR DYNAMIC IMAGE DEFINITION, DEVICE, METHOD AND PROGRAM FOR DYNAMIC IMAGE DECODING
US8270472B2 (en) * 2007-11-09 2012-09-18 Thomson Licensing Methods and apparatus for adaptive reference filtering (ARF) of bi-predictive pictures in multi-view coded video
KR101385884B1 (ko) * 2008-01-30 2014-04-16 고려대학교 산학협력단 다시점 동영상의 부호화 및 복호화 방법 및 장치
EP2152009A1 (en) * 2008-08-06 2010-02-10 Thomson Licensing Method for predicting a lost or damaged block of an enhanced spatial layer frame and SVC-decoder adapted therefore
TWI468020B (zh) * 2009-02-19 2015-01-01 Sony Corp Image processing apparatus and method

Patent Citations (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1493157A (zh) 2001-09-12 2004-04-28 ���µ�����ҵ��ʽ���� 图像编码方法和图像解码方法
US20040179620A1 (en) * 2002-07-11 2004-09-16 Foo Teck Wee Filtering intensity decision method, moving picture encoding method, and moving picture decoding method
US20060146941A1 (en) * 2005-01-04 2006-07-06 Samsung Electronics Co., Ltd. Deblocking control method considering intra BL mode and multilayer video encoder/decoder using the same
JP2006295913A (ja) 2005-04-11 2006-10-26 Sharp Corp 空間的スケーラブルコーディングのためのアダプティブアップサンプリング方法および装置
EP1720358A2 (en) 2005-04-11 2006-11-08 Sharp Kabushiki Kaisha Method and apparatus for adaptive up-sampling for spatially scalable coding
CN101589625A (zh) 2006-10-25 2009-11-25 弗劳恩霍夫应用研究促进协会 质量可缩放编码
JP2010507961A (ja) 2006-10-25 2010-03-11 韓國電子通信研究院 マルチビュービデオのスケーラブルコーディングおよびデコーディング方法、ならびにコーディングおよびデコーディング装置
US20100002770A1 (en) * 2008-07-07 2010-01-07 Qualcomm Incorporated Video encoding by filter selection
JP5235035B2 (ja) 2008-09-23 2013-07-10 ドルビー ラボラトリーズ ライセンシング コーポレイション チェッカーボード多重化したイメージデータのエンコーディング構造及びデコーディング構造
WO2010075348A2 (en) 2008-12-22 2010-07-01 Unisys Corporation Java enterprise resource management system and method
JP2012516657A (ja) 2009-01-29 2012-07-19 ドルビー ラボラトリーズ ライセンシング コーポレイション 立体画像である複数の画像をサブサンプリング及びインタリーブする方法及び装置
US20120027079A1 (en) 2009-04-20 2012-02-02 Dolby Laboratories Licensing Corporation Adaptive Interpolation Filters for Multi-Layered Video Delivery
US20120026288A1 (en) 2009-04-20 2012-02-02 Dolby Laboratories Licensing Corporation Directed Interpolation and Data Post-Processing
JP2012521734A (ja) 2009-04-20 2012-09-13 ドルビー ラボラトリーズ ライセンシング コーポレイション フィルタを選択する方法及び装置
US20120092452A1 (en) 2009-07-04 2012-04-19 Dolby Laboratories Licensing Corporation Encoding and Decoding Architectures for Format Compatible 3D Video Delivery
JP2013518465A (ja) 2010-01-27 2013-05-20 ドルビー ラボラトリーズ ライセンシング コーポレイション 画像及び映像コーデックにおける参照処理のための方法及びシステム
US20110243231A1 (en) * 2010-04-02 2011-10-06 National Chiao Tung University Selective motion vector prediction method, motion estimation method and device thereof applicable to scalable video coding system
US20130136371A1 (en) * 2010-06-17 2013-05-30 Sharp Kabushiki Kaisha Image filter device, decoding apparatus, encoding apparatus, and data structure
JP2013534117A (ja) 2010-07-08 2013-08-29 ドルビー ラボラトリーズ ライセンシング コーポレイション 参照処理信号を使った多層画像およびビデオ送達のためのシステムおよび方法
JP2013535901A (ja) 2010-07-21 2013-09-12 ドルビー ラボラトリーズ ライセンシング コーポレイション ビデオ符号化のための高度な動きモデルを使った参照処理
JP2013538487A (ja) 2010-07-21 2013-10-10 ドルビー ラボラトリーズ ライセンシング コーポレイション マルチレイヤフレーム準拠ビデオ配信のためのシステム及び方法
JP2013542648A (ja) 2010-09-24 2013-11-21 クゥアルコム・インコーポレイテッド ステレオビデオデータの符号化

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
Kim, Jae Hoon et al. "New Coding Tools for Illumination and Focus Mismatch Compensation in Multiview Video Coding" IEEE Transactions on Circuits and Systems for Video Technology, IEEE Service Center, Piscataway, NJ, US, vol. 17, No. 11, Nov. 1, 2007, pp. 1519-1535.
Tourapis, A.M et al. "A Frame Compatible System for 3D Delivery" MPEG Meeting, Motion Picture Expert Group or ISO/IEC JTC 1, Jul. 30, 2010.
T-REC-H.264 "Advanced Video Coding for Generic Audiovisual Services" Mar. 2010.

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160037173A1 (en) * 2011-10-26 2016-02-04 Intellectual Discovery Co., Ltd. Scalable video coding method and apparatus using intra prediction mode
US9532064B2 (en) * 2011-10-26 2016-12-27 Intellectual Discovery Co., Ltd. Scalable video coding method and apparatus using intra prediction mode
US20170064323A1 (en) * 2011-10-26 2017-03-02 Intellectual Discovery Co., Ltd. Scalable video coding method and apparatus using intra prediction mode
US9762923B2 (en) * 2011-10-26 2017-09-12 Intellectual Discovery Co., Ltd. Scalable video coding method and apparatus using intra prediction mode
US9936218B2 (en) 2011-10-26 2018-04-03 Intellectual Discovery Co., Ltd. Scalable video coding method and apparatus using intra prediction mode
US20150055702A1 (en) * 2012-03-06 2015-02-26 Lg Electronics Inc. Method of processing video signals
US9924169B2 (en) * 2012-03-06 2018-03-20 Lg Electronics Inc. Method and apparatus for selecting a filter based on a variance of depth data
US10218971B2 (en) * 2012-09-28 2019-02-26 Vid Scale, Inc. Adaptive upsampling for multi-layer video coding
US10334263B2 (en) * 2013-04-05 2019-06-25 Samsung Electronics Co., Ltd. Interlayer video encoding method and apparatus and interlayer video decoding method and apparatus for compensating luminance difference
US9955174B2 (en) * 2013-09-20 2018-04-24 Vid Scale, Inc. Systems and methods for providing 3D look-up table coding for color gamut scalability
US20160295219A1 (en) * 2013-09-20 2016-10-06 Vid Scale, Inc. Systems and methods for providing 3d look-up table coding for color gamut scalability
US10390029B2 (en) 2013-09-20 2019-08-20 Vid Scale, Inc. Systems and methods for providing 3D look-up table coding for color gamut scalability
US9912925B2 (en) 2013-12-13 2018-03-06 Vid Scale, Inc. Providing 3D look-up table (LUT) estimation for color gamut scalability
US10440340B2 (en) 2013-12-13 2019-10-08 Vid Scale, Inc. Providing 3D look-up table (LUT) estimation for color gamut scalability

Also Published As

Publication number Publication date
WO2012044487A1 (en) 2012-04-05
CN103141099B (zh) 2016-10-26
EP2622857A1 (en) 2013-08-07
CN103141099A (zh) 2013-06-05
US20130194505A1 (en) 2013-08-01
JP5663093B2 (ja) 2015-02-04
JP2014504459A (ja) 2014-02-20
US20160142709A1 (en) 2016-05-19
US9521413B2 (en) 2016-12-13

Similar Documents

Publication Publication Date Title
US9521413B2 (en) Optimized filter selection for reference picture processing
JP6633694B2 (ja) 多視点信号コーデック
US9078008B2 (en) Adaptive inter-layer interpolation filters for multi-layered video delivery
US9241160B2 (en) Reference processing using advanced motion models for video coding
EP2422521B1 (en) Filter selection for video pre-processing in video applications
US8457201B2 (en) Inter-layer prediction method for video signal
US11303900B2 (en) Method and apparatus for motion boundary processing
US8902976B2 (en) Hybrid encoding and decoding methods for single and multiple layered video coding systems
CN108028931B (zh) 用于视频编解码的自适应帧间预测的方法及装置
WO2016066028A1 (en) Method of guided cross-component prediction for video coding
JP6157614B2 (ja) エンコーダ、デコーダ、方法、及びプログラム
KR102287414B1 (ko) 손실 비디오 코딩을 위한 저복잡도 혼합 도메인 협력 인-루프 필터
US20180332292A1 (en) Method and apparatus for intra prediction mode using intra prediction filter in video and image compression
US20150043639A1 (en) Method and device for coding scalable video on basis of coding unit of tree structure, and method and device for decoding scalable video on basis of coding unit of tree structure
US20150085933A1 (en) Method and apparatus for encoding multi-view images, and method and apparatus for decoding multi-view images
EP4128770A1 (en) Methods and devices for prediction dependent residual scaling for video coding
AU2015255215B2 (en) Image processing apparatus and method

Legal Events

Date Code Title Description
AS Assignment

Owner name: DOLBY LABORATORIES LICENSING CORPORATION, CALIFORN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:PAHALAWATTA, PESHALA;HE, YUWEN;TOURAPIS, ALEXANDROS;SIGNING DATES FROM 20110301 TO 20110323;REEL/FRAME:030121/0160

STCF Information on status: patent grant

Free format text: PATENTED CASE

FEPP Fee payment procedure

Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

LAPS Lapse for failure to pay maintenance fees

Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STCH Information on status: patent discontinuation

Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362

FP Lapsed due to failure to pay maintenance fee

Effective date: 20200223