EP2594074A1 - Décodeur vidéo pour réduction de puissance à basse résolution utilisant des données à basse résolution - Google Patents

Décodeur vidéo pour réduction de puissance à basse résolution utilisant des données à basse résolution

Info

Publication number
EP2594074A1
EP2594074A1 EP11806941.8A EP11806941A EP2594074A1 EP 2594074 A1 EP2594074 A1 EP 2594074A1 EP 11806941 A EP11806941 A EP 11806941A EP 2594074 A1 EP2594074 A1 EP 2594074A1
Authority
EP
European Patent Office
Prior art keywords
resolution
low resolution
resolution data
decoder
low
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP11806941.8A
Other languages
German (de)
English (en)
Other versions
EP2594074A4 (fr
Inventor
Zhan MA
Christopher A. Segall
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sharp Corp
Original Assignee
Sharp Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US12/838,381 external-priority patent/US8767828B2/en
Priority claimed from US12/838,335 external-priority patent/US20120014445A1/en
Application filed by Sharp Corp filed Critical Sharp Corp
Publication of EP2594074A1 publication Critical patent/EP2594074A1/fr
Publication of EP2594074A4 publication Critical patent/EP2594074A4/fr
Withdrawn legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/117—Filters, e.g. for pre-processing or post-processing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/156—Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/182—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a pixel
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/423—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation characterised by memory arrangements
    • H04N19/426—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation characterised by memory arrangements using memory downsizing methods
    • H04N19/428—Recompression, e.g. by spatial or temporal decimation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/46—Embedding additional information in the video signal during the compression process
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/59—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution

Definitions

  • the present invention relates to a video decoder with power reduction.
  • Power reduction is generally achieved by using two primary techniques.
  • the first technique for power reduction is opportunistic, where a video coding system reduces its processing capability when operating on a sequence that is easy to decode. This reduction in processing capability may be achieved by frequency scaling, voltage scaling, on-chip data pre-fetching (caching) , and/ or a systematic idling strategy. In many cases the resulting decoder operation conforms to the standard.
  • the second technique for power reduction is to discard frame or image data during the decoding process. This typically allows for more significant power savings but generally at the expense of visible degradation in the image quality. In addition, in many cases the resulting decoder operation does not conform to the standard.
  • One embodiment of the present invention discloses a video decoder that decodes video comprising: (a) an entropy decoder that decodes a bitstream defining said video; (b) an inverse transformation that transforms said decoded bitstream; (c) a predictor that selectively performs an intra- prediction and a motion compensated prediction based on said decoded bitstream; (d) a buffer comprising compressed image data used for said motion compensated prediction, including low-resolution data and high-resolution data, where said predictor predicts both a low-resolution data set and a high-resolution data set based upon said low resolution data using high-resolution prediction information decoded from said bitstream without using said high resolution data.
  • a video decoder that decodes video comprising: (a) an entropy decoder that decodes a bitstream defining said video; (b) an inverse transformation that transforms said decoded bitstream; (c) a predictor that selectively performs an intra- prediction and a motion compensated prediction on said decoded bitstream; (d) a buffer comprising compressed image data used for said motion compensated prediction; (e) wherein said compressed image data includes a low-resolution data set and a high-resolution data set, where said low resolution data set is independent of said high resolution data set and used for decoding in a low resolution mode, and wherein said low resolution data set and said high resolution data set are both used when decoding in a high resolution mode.
  • FIG. 1 illustrates a decoder
  • FIG. 2 illustrates low resolution prediction
  • FIGS. 3A and 3B illustrate a decoder and data flow for the decoder.
  • FIG. 4 illustrates a sampling structure of the frame buffer.
  • FIG. 5 illustrates integration of the frame buffer in the decoder.
  • FIG. 6A and 6B illustrates representative pixel values of two blocks.
  • the system may be used with minimal impact on coding efficiency.
  • the system should operate alternatively on low resolution data and high resolution data.
  • the combination of low resolution data and high resolution data may result in full resolution data.
  • the use of low resolution data is particularly suitable when the display has a resolution lower than the resolution of the transmitted content.
  • Power is a factor when designing higher resolution decoders.
  • One major contributor to power usage is memory bandwidth. Memory bandwidth traditionally increases with higher resolutions and frame rates, and it is often a significant bottleneck and cost factor in system design.
  • a second major contributor to power usage is high pixel counts. High pixel counts are directly determined by the resolution of the image frame and increase the amount of pixel processing and computation. The amount of power required for each pixel operation is determined by the complexity of the decoding process. Historically, the decoding complexity has increased in each "improved" video coding standard.
  • the system may include an entropy decoding module 10, a transformation module (such as inverse transformation using a dequant IDCT) 20, an intra prediction module 30, a motion compensated prediction module 40, an adder 80, a deblocking module 50, an adaptive loop filter module 60, and a memory compression / decompression module associated with a frame buffer 70.
  • the arrangement and selection of the different modules for the video system may be modified, as desired.
  • the system in one aspect, preferably reduces the power requirements of both memory bandwidth and high pixel counts of the frame buffer.
  • the memory bandwidth is reduced by incorporating a frame buffer compression technique within a video coder design.
  • the purpose of the frame buffer compression technique is to reduce the memory bandwidth (and power) required to access data in the reference picture buffer. Given that the reference picture buffer is itself a compressed version of the original image data, compressing the reference frames can be achieved without significant coding loss for many applications.
  • the video codec should support a low resolution processing mode without drift.
  • the decoder may switch between low-resolution and full-resolution operating points and be compliant with the standard. This may be accomplished by performing prediction of both the low-resolution and high-resolution data using the full-resolution prediction information but only the low-resolution image data. Additionally, this may be improved using a de-blocking process that makes de-blocking decisions using only the low-resolution data. De-blocking is applied to the low-resolution data and, also if desired, the high- resolution data. The de-blocking of the low-resolution pixels does not depend on the high-resolution pixels. The low resolution deblocking and high resolution deblocking may be performed serially and/ or in parallel.
  • the deblocking of the high resolution pixels may depend on the low- resolution pixels.
  • the low resolution process is independent of the high resolution process, thus enabling a power savings mode, while the high resolution process may depend on the low resolution process, thus enabling greater image quality when desired.
  • a decoder when operating in the low-resolution mode (S 10) , may exploit the properties of low- resolution prediction and modified de-blocking to significantly reduce the number of pixels to be processed. This may be accomplished by predicting only the low-resolution data (S 12) . Then after predicting the low resolution data, computing the residual data for only the low-resolution pixels (i. e. , pixel locations) and not the high resolution pixels (i.e. , pixel locations) (S 14) . The residual data is typically transmitted in a bit-stream. The residual data computed for the low- resolution pixel values has the same pixel values as the full resolution residual data at the low-resolution pixel locations.
  • the residual data needs to only be calculated at the position of the low-resolution pixels .
  • the low-resolution residual is added to the low-resolution prediction (S 16) , to provide the low resolution pixel values .
  • the resulting signal is then de-blocked .
  • the de-blocking is preferably performed at only the low-resolution sample locations (S 18) to reduce power consumption.
  • the result may be stored in the reference picture frame buffer for future prediction .
  • the result may be processed with an adaptive loop filter.
  • the adaptive loop filter may be related to the adaptive loop filter for the full resolution data, or it may be signaled independently, or it may be omitted.
  • FIGS . 3A and 3B An exemplary depiction of the system operating in low- resolution mode is shown in FIGS . 3A and 3B .
  • the system may likewise include a mode that operates in full resolution mode.
  • entropy decoding 100 may be performed at full resolution, while the inverse transform (Dequant IDCT) 200 and prediction (Intra Prediction 300; Motion Compensated Prediction (MCP) 400) are preferably performed at low resolution.
  • the de-blocking 500 is preferably performed in a cascade fashion so that the de-blocking of the low resolution pixels do not depend on the additional, high resolution data.
  • a frame buffer that includes memory compression stores the low-resolution data used for future prediction.
  • the entropy decoding 100 shown in FIG. 3A entropy decodes the residual data for full-resolution pixels ( 10 1 ) .
  • the shaded pixels in the residual 10 1 represent low resolution positions, while the un-shaded pixels represent high resolution positions .
  • the Dequant IDCT 200 inverse transforms only the low resolution pixel data in the residual 10 1 , so as to produce a residual-after-Dequant-and-IDCT 201.
  • the Intra Prediction 300 produces a prediction 30 1 only for the low resolution positions (depicted by the shaded pixels) .
  • Adder 800 adds the low resolution pixel data in the residual-after-Dequant-and- IDCT 201 to the low resolution pixel data in the prediction 30 1 , so as to produce a reconstruction 80 1 only for the low resolution positions (depicted by the shaded pixels) .
  • the MCP 400 shown in FIG. 3B reads out the low resolution pixel data of the reference picture (depicted by the shaded pixels in the reference picture data 702) from the Memory 700, and produces by interpolation the high resolution pixel data which have been removed.
  • the MCP 400 produces by interpolation the high resolution pixel data C from low resolution pixel data of the neighboring pixels. Taking an average of low resolution pixel data of pixels located in the upper and bottom side of C, taking an average of low resolution pixel data of pixels located in the left and right side of C, taking an average of low resolution pixel data of pixels located in the upper, bottom, left and right side of C may be employed as the interpolation.
  • Deblocking 500 is performed in a cascade fashion.
  • the Deblocking 500 filters the low resolution data in the first time (501 ) , while it filters the high resolution data in the second time (502) . More specifically, Deblocking 500 is performed in the following manner.
  • Deblocking 500 applies only to the low resolution data using the low resolution data and the high resolution data by interpolation.
  • Deblocking 500 applies only to the high resolution data using the low resolution data and the high resolution data by interpolation.
  • a picture 502 after the Deblocking 500 is a full resolution picture, which may be referred to as a picture 701 .
  • the picture 701 after the Deblocking 500 is decimated (702) in a checker-board pattern such that only the low resolution positions are remained and stored in the Memory 700.
  • the decimated high resolution pixel data (depicted by the unshaded pixels of 702) is interpolated and the interpolated picture is used for producing a predicted picture.
  • the frame buffer compression technique is preferably a component of the low resolution functionality.
  • the frame buffer compression technique preferably divides the image pixel data into multiple sets, and that a first set of the pixel data does not depend on other sets.
  • the system employs a checker-board pattern as shown in FIG. 4.
  • the shaded pixel locations belong to the first set and the un-shaded pixels belong to the second set.
  • Other sampling structures may be used, as desired. For example, every other column of pixels may be assigned to the first set. Alternatively, every other row of pixels may be assigned to the first set. Similarly, every other column and row of pixels may be assigned to the first set. Any suitable partition into multiple sets of pixels may be used .
  • the frame buffer compression technique preferably has the pixels in a second set of pixels be linearly predicted from pixels in the first set of pixels.
  • the prediction may be pre-defined. Alternatively, it may be spatially varying or determined using any other suitable technique.
  • the pixels in the first set of pixels are coded.
  • This coding may use any suitable technique, such as for example, block truncation coding (BTC) , such as described by Healy, D . ; Mitchell, O . , " Digital Video Bandwidth Compression Using Block Truncation Coding, " IEEE Transactions on Communications [legacy, pre - 1988] , vol.29 , no. 12 pp. 1809- 1817 , Dec 1981 , absolute moment block truncation coding (AMBTC) , such as described by Lema, M . ; Mitchell, O . , "Absolute Moment Block Truncation Coding and Its A pp.
  • BTC block truncation coding
  • AMBTC absolute moment block truncation coding
  • the pixels in the second set of pixels may be coded and predicted using any suitable technique, such as for example being predicted using a linear process known to the frame buffer compression encoder and frame buffer compression decoder. Then the difference between the prediction and the pixel value may be computed. Finally, the difference may be compressed.
  • the system may use block truncation coding (BTC) to compress the first set of pixels.
  • AMBTC absolute moment block truncation coding
  • the system may use quantization to compress the first set of pixels .
  • the system may use bi-linear interpolation to predict the pixel values in the second set of pixels .
  • the system may use bi-cubic interpolation to predict the pixel values in the second set of pixels.
  • the system may use bi-linear interpolation to predict the pixel values in the second set of pixels and absolute moment block truncation coding (AMBTC) to compress the residual difference between the predicted pixel values in the second set and the pixel value in the second set.
  • AMBTC absolute moment block truncation coding
  • a property of the frame buffer compression technique is that it is controlled with a flag to signal low resolution processing capability.
  • this flag does not signal low resolution processing capability, then the frame buffer decoder produces output frames that contain the first set of pixel values (i. e . low resolution pixel data) , possibly compressed, and the second set of pixel values (i. e . high resolution pixel data) that are predicted from the first set of pixel values and refined with optional residual data.
  • this flag does signal low resolution processing capability
  • the frame buffer decoder produces output frames that contain the first set of pixel values, possibly compressed, and the second set of pixel values that are predicted from the first set of pixel values but not refined with optional residual data. Accordingly, the flag indicates whether or not to use the optional residual data.
  • the residual data may represent the differences between the predicted pixel values and the actual pixel values.
  • the encoder when the flag does not signal low resolution processing capability, then the encoder stores the first set of pixel values, possibly in compressed form. Then, the encoder predicts the second set of pixel values from the first set of pixel values. In some embodiments, the encoder determines the residual difference between the prediction and actual pixel value and stores the residual difference, possibly in compressed form. In some embodiments, the encoder selects from multiple prediction mechanisms a preferred prediction mechanism for the second set pixels. The encoder then stores the selected prediction mechanism in the frame buffer. In one embodiment, the multiple prediction mechanisms consist of multiple linear filters and the encoder selects the prediction mechanism by computing the predicted pixel value for each linear filter and selecting the linear filter that computes a predicted pixel value that is closest to the pixel value .
  • the multiple prediction mechanisms consist of multiple linear filters and the encoder selects the prediction mechanism by computing the predicted pixel values for each linear filter for a block of pixel locations and selecting the linear filter that computes a block of predicted pixel value that are closest to the block of pixel values.
  • a block of pixels is a set of pixels within an image . The determination of the block of pixel predicted pixel values that are closest to the block of pixel values may be determined by selecting the block of predicted pixel values that result in the smallest sum of absolute differences between the block of predicted pixels values and block of pixels values. Alternatively, the sum of squared differences may be used to select the block. In other embodiments, the residual difference is compressed with block truncation coding (BTC) .
  • BTC block truncation coding
  • the residual difference is compressed with the absolute moment block truncation coding (AMBTC) .
  • the parameters used for the compression of the second set pixels are determined from the parameters used for the compression of the first set of pixels.
  • the first set of pixels and second set of pixels use AMBTC, and a first parameter used for the AMBTC method of the first set of pixels is related to a first parameter used for the AMBTC method for the second set of pixels.
  • said first parameter used for the second set of pixels is equal to said first parameter used for the first set of pixels and not stored .
  • said first parameter used for the second set of pixels is related to said first parameter used for the first set of pixels.
  • the relationship may be defined as a scale factor, and the scale factor stored in place of said first parameter used for the second set of pixels. In other embodiments, the relationship may be defined as an index into a look-up-table of scale factors, the index stored in place of said first parameter used for the second set of pixels. In other embodiments, the relationship may be predefined.
  • the encoder combines the selected prediction mechanism and residual difference determination step. By comparison, when the flag signals low resolution processing capability, then the encoder stores the first set of pixel values, possibly in compressed form. However, the encoder does not store residual information. In embodiments described above that determine a selected prediction mechanism, the encoder does not compute the selected prediction mechanism from the reconstructed data. Instead, any selected prediction mechanism is signaled from the encoder to the decoder.
  • the signaling of a flag enables low resolution decoding capability.
  • the decoder is not required to decode a low resolution sequence even when the flag signals a low resolution decoding capability. Instead, it may decode either a full resolution or low resolution sequence . These sequences will have the same decoded pixel values for pixel locations on the low resolution grid. The sequences may or may not have the same decoded pixel values for pixel locations on the high resolution grid.
  • the signaling the flag may be on a frame-by- frame basis, on a sequence-by-sequence basis, or any other basis.
  • the decoder When the flag appears in the bit-stream, the decoder preferably performs the following steps:
  • (a) Disables the residual calculation in the frame buffer compression technique. This includes disabling the calculation of residual data during the loading of reference frames as well as disabling the calculation of residual data during the storage of reference frames, as illustrated in FIG . 5.
  • the decoder may continue to operate in full resolution mode. Specifically, for future frames, it can retrieve the full resolution frame from the compressed reference buffer, perform motion compensation, residual addition, de-blocking, and loop filter. The result will be a full resolution frame . This frame can still contain frequency content that occupies the entire range of the full resolution pixel grid.
  • the decoder may choose to operate only on the low-resolution data. This is possible due to the independence of the lower resolution grid on the higher resolution grid in the buffer compression structure .
  • the interpolation process is modified to exploit the fact that high resolution pixels are linearly related to the low-resolution data.
  • the motion estimation process may be performed at low resolution with modified interpolation filters.
  • the system may exploit the fact that the low resolution data does not rely on the high resolution data in subsequent steps of the decoder.
  • the system uses a reduced inverse transformation process that only computes the low resolution pixels from the full resolution transform coefficients.
  • the system employs a de-blocking filter that de-blocks the low-resolution data independent from the high-resolution pixels (the high-resolution may be dependent on the low- resolution) . This is again due to the linear relationship between the high-resolution and lower-resolution data.
  • d is a variable and pij and qij are pixel values.
  • the location of the pixel values are depicted in FIG. 6.
  • two 4x4 coding units are shown .
  • the pixel values may be determined from any block size by considering the location of the pixels relative to the block boundary.
  • the value computed for d is compared to a threshold. If the value d is less than the threshold, the deblocking filter is engaged . If the value d is greater than or equal to the threshold, then no filtering is applied and the deblocked pixels have the same values as the input pixel values.
  • the threshold may be a function of a quantization parameter, and it may be described as beta(QP) .
  • the deblocking decision is made independently for horizontal and vertical boundaries.
  • the process continues to determine the type of filter to apply.
  • the de-blocking operation uses either strong or weak filter types.
  • the choice of filtering strength is based on the previously computed d, beta(QP) and also additional local differences. This is computed for each line (row or column) of the de-blocked boundary. For example, for the first row of the pixel locations shown in FIG. 6, the calculation is computed as
  • StrongFilterFlag ((d ⁇ beta(QP)) 8585 ((
  • the filtering process may be described as follows. Here, this is described by the filtering process for the boundary between block A and block B in FIG. 6. The process is:
  • ⁇ is an offset
  • Clipo-25s() is an operator that maps the input value to the range [0,255].
  • the operator may map the input values to alternative ranges, such as [16,235], [0,1023] or other ranges.
  • the filtering process may be described as follows. Here, this is described by the filtering process for the boundary between block A and block B in FIG. 6. The process is:
  • Clipo-255() is an operator that maps the input value to the range [0,255].
  • the operator may map the input values to alternative ranges, such as [16,235], [0,1023] or other ranges.
  • ⁇ also referred to as delta in this specification, is an offset
  • Clipo-255 () is an operator that maps the input value to the range [0,255] .
  • the operator may map the input values to alternative ranges, such as [ 16,235] , [0, 1023] or other ranges.
  • the pixel locations within an image frame may be partitioned into two or more sets.
  • a flag is signaled in the bit-stream, or communicated in any manner, the system enables the processing of the first set of pixel locations without the pixel values at the second set of pixel locations.
  • FIG. 4 An example of this partitioning is shown in FIG. 4.
  • a block is divided into two sets of pixels. The first set corresponds to the shaded locations; the second set corresponds to the unshaded locations.
  • the system may modify the previous de-blocking operations as follows:
  • the system uses the previously described equations, or other suitable equations. However, for the pixel values corresponding to pixel locations that are not in the first set of pixels, the system may use pixel values that are derived from the first set of pixel locations.
  • pOl, p03, p05, p07, qOO, q02, q04, q06 in FIG. 6A and 6B are first set of pixels which are calculated by entropy decoding, inverse transformation and prediction.
  • pOO, p02, p04, p06, qOl, q03, q05, q07 are second set of pixels which are calculated by such an equation shown in FIG.3B or FIG.5,
  • Eq.l, Eq 2, Eqs 3, Eqs 4, and Eqs 5 are calculated using these pixel values.
  • the system derives the pixel values as a linear summation of neighboring pixel values located in the first set of pixels.
  • the system uses bi-linear interpolation of the pixel values located in the first set of pixels.
  • the system computes the linear average of the pixel value located in the first set of pixels that is above the current pixel location and the pixel value located in the first set of pixels that is below the current pixel location.
  • the system is operating on a vertical block boundary (and applying horizontal de-blocking). For the case that the system is operating on a horizontal block boundary (and applying vertical de-blocking) , then the system computes the average of the pixel to the left and right of the current location.
  • the system may restrict the average calculation to pixel values within the same block. For example, if the pixel value located above a current pixel is not in the same block but the pixel value located below the current pixel is in the same block, then the current pixel is set equal to the pixel value below the current pixel.
  • the system may use the same approach as described above. Namely, the pixels values that do not correspond to the first set of pixels are derived from the first set of pixels. After computing the above decision, the system may use the decision for the processing of the first set of pixels. Decoders processing subsequent sets of pixels use the same decision to process the subsequent sets of pixels.
  • the system may use the weak filtering process described above .
  • the system does not use the pixel values that correspond to the set of pixels subsequent to the first set. Instead, the system may derive the pixel values as discussed above .
  • the value for ⁇ is then applied to the actual pixel values in the first set and the delta value is applied to the actual pixel values in the second set.
  • the system may do the following:
  • the system may use the equations for the luma strong filter described above. However, for the pixel values not located in the first set of pixel locations, the system may derive the pixel values from the first set of pixel locations as described above. The system then store the results of the filter process for the first set of pixel locations. Subsequently, for decoders generating the subsequent pixel locations as output, the system uses the equations for the luma strong filter described above with the previously computed strong filtered results for the first pixel locations and the reconstructed (not filtered) results for the subsequent pixel locations. The system then applies the filter at the subsequent pixel locations only. The output are filtered first pixel locations corresponding to the first filter operation and filtered subsequent pixel locations corresponding to the additional filter passes.
  • the system takes the first pixel values and interpolates the missing pixel vales, computes the strong filter result for the first pixel values, updates the missing pixel values to be the actual reconstructed values, and computes the strong filter result for the missing pixel locations.
  • the system uses the equations for the strong luma filter described above. For the pixel values not located in the first set of pixel locations, the system derives the pixel values from the first set of pixel locations as described above . The system then computes the strong filter result for both the first and subsequent sets of pixel locations using the derived values. Finally, the system computes a weighted average of the reconstructed pixel values at the subsequent locations and the output of the strong filter at the subsequent locations. In one embodiment, the weight is transmitted from the encoder to the decoder. In an alternative embodiment, the weight is fixed.
  • the system uses the weak filtering process for chroma as described above .
  • the system does not use the pixel values that correspond to the set of pixels subsequent to the first set. Instead, the system preferably derives the pixel values as in the previously described.
  • the value for ⁇ is then applied to the actual pixel values in the first set and the delta value is applied to the actual pixel values in the second set.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computing Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Abstract

L'invention porte sur un décodeur vidéo qui utilise des techniques de réduction de puissance. Le décodeur vidéo comprend : (a) un décodeur entropique qui décode un train de bits définissant ladite vidéo; (b) une unité de transformation inverse qui transforme ledit train de bits décodé; (c) un prédicteur qui effectue sélectivement une prédiction intra et une prédiction par compensation de mouvement sur la base dudit train de bits décodé; (d) un tampon comprenant des données d'image compressées utilisées pour ladite prédiction par compensation de mouvement, comprenant des données à basse résolution et des données à haute résolution, ledit prédicteur prédisant un ensemble de données à basse résolution et un ensemble de données à haute résolution sur la base desdites données à basse résolution en utilisant des informations de prédiction à haute résolution décodées à partir dudit train de bits sans utiliser lesdites données à haute résolution.
EP11806941.8A 2010-07-16 2011-07-14 Décodeur vidéo pour réduction de puissance à basse résolution utilisant des données à basse résolution Withdrawn EP2594074A4 (fr)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US12/838,381 US8767828B2 (en) 2010-07-16 2010-07-16 System for low resolution power reduction with compressed image
US12/838,335 US20120014445A1 (en) 2010-07-16 2010-07-16 System for low resolution power reduction using low resolution data
PCT/JP2011/066636 WO2012008614A1 (fr) 2010-07-16 2011-07-14 Décodeur vidéo pour réduction de puissance à basse résolution utilisant des données à basse résolution

Publications (2)

Publication Number Publication Date
EP2594074A1 true EP2594074A1 (fr) 2013-05-22
EP2594074A4 EP2594074A4 (fr) 2014-08-06

Family

ID=45469597

Family Applications (1)

Application Number Title Priority Date Filing Date
EP11806941.8A Withdrawn EP2594074A4 (fr) 2010-07-16 2011-07-14 Décodeur vidéo pour réduction de puissance à basse résolution utilisant des données à basse résolution

Country Status (5)

Country Link
EP (1) EP2594074A4 (fr)
JP (1) JP5732125B2 (fr)
CN (1) CN103004202A (fr)
EA (1) EA024279B1 (fr)
WO (1) WO2012008614A1 (fr)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2014519212A (ja) * 2011-05-26 2014-08-07 シャープ株式会社 ビデオデコーダ
JP6512741B2 (ja) * 2014-03-28 2019-05-15 株式会社メガチップス 画像処理装置および画像処理方法
WO2024136203A1 (fr) * 2022-12-22 2024-06-27 삼성전자 주식회사 Procédé de réalisation d'inférence par division d'une pluralité de modèles de réseau neuronal, et dispositif électronique

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0883299A2 (fr) * 1997-06-06 1998-12-09 Fujitsu Limited Dispositif et méthode de codage et décodage d'images vidéo

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3575508B2 (ja) * 1996-03-04 2004-10-13 Ｋｄｄｉ株式会社 符号化動画像再生装置
JP3384727B2 (ja) * 1997-11-05 2003-03-10 三洋電機株式会社 画像復号装置
US6959116B2 (en) * 2001-09-18 2005-10-25 Emc Corporation Largest magnitude indices selection for (run, level) encoding of a block coded picture
US7397858B2 (en) * 2002-05-29 2008-07-08 Innovation Management Sciences, Llc Maintaining a plurality of codebooks related to a video signal
EP1728393A2 (fr) * 2004-03-08 2006-12-06 Koninklijke Philips Electronics N.V. Decodeur video presentant une compression pouvant etre mise a l'echelle et un tampon pour stocker et pour extraire des donnees de trame de reference
KR101117840B1 (ko) * 2004-04-02 2012-03-15 톰슨 라이센싱 복잡도 가변 비디오 디코더를 위한 방법 및 장치
KR100679035B1 (ko) * 2005-01-04 2007-02-06 삼성전자주식회사 인트라 ｂｌ 모드를 고려한 디블록 필터링 방법, 및 상기방법을 이용하는 다 계층 비디오 인코더/디코더
JP2008536451A (ja) * 2005-04-14 2008-09-04 トムソン ライセンシング 空間スケーラブルビデオ符号化及び復号化向けスライス適応型動きベクトル符号化のための方法及び装置
JP4774315B2 (ja) * 2006-02-20 2011-09-14 株式会社日立製作所 画像復号化装置及び画像復号化方法
CN101202906A (zh) * 2006-12-11 2008-06-18 国际商业机器公司 在数字视频广播系统中对视频流进行处理的方法和设备
JP2009296300A (ja) * 2008-06-05 2009-12-17 Panasonic Corp 画像符号化装置および方法

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0883299A2 (fr) * 1997-06-06 1998-12-09 Fujitsu Limited Dispositif et méthode de codage et décodage d'images vidéo

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
MA Z ET AL: "System for graceful power degradation", 2. JCT-VC MEETING; 21-7-2010 - 28-7-2010; GENEVA; (JOINT COLLABORATIVETEAM ON VIDEO CODING OF ISO/IEC JTC1/SC29/WG11 AND ITU-T SG.16 ); URL:HTTP://WFTP3.ITU.INT/AV-ARCH/JCTVC-SIT E/,, no. JCTVC-B114, 23 July 2010 (2010-07-23), XP030007693, ISSN: 0000-0046 *
See also references of WO2012008614A1 *

Also Published As

Publication number Publication date
CN103004202A (zh) 2013-03-27
JP2013531401A (ja) 2013-08-01
EA201390113A1 (ru) 2013-12-30
EP2594074A4 (fr) 2014-08-06
JP5732125B2 (ja) 2015-06-10
EA024279B1 (ru) 2016-09-30
WO2012008614A1 (fr) 2012-01-19

Similar Documents

Publication Publication Date Title
US8548062B2 (en) System for low resolution power reduction with deblocking flag
KR101530832B1 (ko) 영상의 재구성된 샘플 세트에 대한 보상 오프셋들의 인코딩/디코딩을 최적화하는 방법 및 장치
CA2755889C (fr) Appareil et procede de traitement d'image
EP2813079B1 (fr) Procédé et appareil de prédiction intercouche pour codage vidéo évolutif
US8767828B2 (en) System for low resolution power reduction with compressed image
KR101628797B1 (ko) Ｈｅｖｃ에서 무손실 코딩의 개선된 인트라 예측
US20120300850A1 (en) Image encoding/decoding apparatus and method
US20080240252A1 (en) Simplified deblock filtering for reduced memory access and computational complexity
US20120076203A1 (en) Video encoding device, video decoding device, video encoding method, and video decoding method
WO2010144406A1 (fr) Compression d'image numérique par décimation du résidu
WO2010144408A9 (fr) Compression d'image numérique par codage de macroblocs avec résolution adaptative
KR101646856B1 (ko) 감소된 해상도 화소 보간
US11438596B2 (en) Method and device for processing video signal by using cross-component linear model
US9313523B2 (en) System for low resolution power reduction using deblocking
US20120014445A1 (en) System for low resolution power reduction using low resolution data
EP2594074A1 (fr) Décodeur vidéo pour réduction de puissance à basse résolution utilisant des données à basse résolution
US20120300844A1 (en) Cascaded motion compensation
WO2012008616A1 (fr) Décodeur vidéo pour réduction de puissance à basse résolution utilisant des données à basse résolution
US20120300838A1 (en) Low resolution intra prediction
US20120014447A1 (en) System for low resolution power reduction with high resolution deblocking
JP2013524659A (ja) ハンドヘルド及びモバイル用途に適した低減された記憶容量及び処理要求を有するブロードキャストビデオレコーダ
WO2012161345A1 (fr) Décodeur vidéo
EP4289139A1 (fr) Métadonnées de signalisation d'informations représentatives d'une consommation d'énergie d'un processus de décodage
GB2516225A (en) Method, device, and computer program for block filtering in a video encoder and decoder
KR20090076019A (ko) 보간 필터, 그 보간 필터를 이용한 멀티 코덱 디코더 및디코딩 방법

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20130128

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 1180155

Country of ref document: HK

DAX Request for extension of the european patent (deleted)
A4 Supplementary search report drawn up and despatched

Effective date: 20140707

RIC1 Information provided on ipc code assigned before grant

Ipc: H04N 19/33 20140101ALI20140701BHEP

Ipc: H04N 19/50 20140101AFI20140701BHEP

17Q First examination report despatched

Effective date: 20160930

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN

18D Application deemed to be withdrawn

Effective date: 20190201

REG Reference to a national code

Ref country code: HK

Ref legal event code: WD

Ref document number: 1180155

Country of ref document: HK