Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
  • H04N19/103—Selection of coding mode or of prediction mode
  • H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
  • H04N19/103—Selection of coding mode or of prediction mode
  • H04N19/11—Selection of coding mode or of prediction mode among a plurality of spatial predictive coding modes
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
  • H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
  • H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
  • H04N19/159—Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
  • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
  • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
  • H04N19/182—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a pixel
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
  • H04N19/593—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques

Definitions

• This invention relates to digital video compression and, more particularly, to a method for improved intra prediction in the absence of neighboring reference samples required for the intra prediction.
• Digital video compression technology is replete with a number of technical standards and competing techniques. For example, there are the following exemplary techniques:
• High Efficiency Video Coding also known as H.265 and MPEG-H Part 2, published in 2014 as ITU-T, High Efficiency Video Coding (HEVC), Recommendation ITU-T H.265
• AVC Advanced Video Coding
• H.264 also known as H.264 or MPEG-4 Part 10, published in 2002 as ITU-T, Rec. H264
• VP9 which is said to be an open video compression format presently in development by a Google partnership, described in a submission by A. Grange et al. that was entitled "A VP9 Bitstream Overview draft-grange-vp9-bitstream-
• Intra prediction can be described in quite generalized terms as involving: filling a neighboring shape of reference samples that are either previously decoded or reconstructed samples that are currently available or padded samples when decoded/reconstructed samples are unavailable; and then propagating these reference samples within the neighboring shape according to the intra prediction directional mode in use.
• the intra prediction directional modes propagate neighboring pixels to predict a current block, which is also known as a Prediction Unit (PU). Since intra frame prediction relies on previously decoded samples (pixels) around the current, some samples from neighbors may not be available for use as reference samples.
• PU Prediction Unit
• Figure 1 shows a stage of prediction for a block (i.e., prediction unit PU) where pixels on the left side are missing. In this case, all those pixels are assigned the value of the first available pixel when scanning the reference pixel L-shape from bottom left to top right, as depicted on the right side of Figure 1 .
• the directional modes ranging from 2 to 9, which are shown in Figure 1 are rendered substantially useless because all the reference pixels being employed will have the same value.
• directional modes 27 to 34 shown in Figure 1 are rendered substantially useless when the neighbors above are not available.
• directional modes are recovered by filling pixel locations in the prediction unit with samples from the portion of the unit that is determined to have samples available for use. In certain cases, reference samples may still be missing or unavailable for use, but padding in accordance with the principles of the present method will employ better pixel values corresponding to the considered intra directional mode.
• a method for compression of digital video images including an operation for intra frame prediction, each said digital video image including one or more prediction units, wherein said intra frame prediction comprises determining whether one or more reference samples are unavailable or missing for a prediction unit or portion thereof given a particular directional mode. If it is determined that one or more reference samples are unavailable or missing, the method further comprises determining whether one or more reference samples are available along another portion of the prediction unit, wherein the another portion of the prediction unit is along a side of the prediction unit at an opposite end of the particular directional mode.
• the method further comprises replacing the particular directional mode with a counterpart directional mode that is oppositely directed to the particular directional mode, and filling the prediction unit with the available reference samples using the counterpart directional mode.
• an apparatus for compression of digital video images, each said digital video image including one or more prediction units, the apparatus comprising: a memory that stores a plurality of instructions; and a processor coupled to the memory and configured to perform said video image compression including an operation for intra frame prediction.
• the processor is configured to perform said intra frame prediction by: determining whether one or more reference samples are unavailable or missing for a prediction unit or portion thereof given a particular directional mode. If it is determined that one or more reference samples are unavailable or missing, determining whether one or more reference samples are available along another portion of the prediction unit, wherein the another portion of the prediction unit is along a side of the prediction unit at an opposite end of the particular directional mode.
• a nontransitory computer readable medium having one or more executable instructions stored thereon, which when executed by a processor cause the processor to perform a method for compression of digital video images including an operation for intra frame prediction, each said digital video image including one or more prediction units.
• the intra frame prediction comprises: determining whether one or more reference samples are unavailable or missing for a prediction unit or portion thereof given a particular directional mode. If it is determined that one or more reference samples are unavailable or missing, determining whether one or more reference samples are available along another portion of the prediction unit, wherein the another portion of the prediction unit is along a side of the prediction unit at an opposite end of the particular directional mode.
• an apparatus for transmitting a video image block comprising: a memory and a processor.
• the processor is configured to perform a method for compression of digital video images including an operation for intra frame prediction, each said digital video image including one or more prediction units.
• the intra frame prediction comprises determining whether one or more reference samples are unavailable or missing for a prediction unit or portion thereof given a particular directional mode. If it is determined that one or more reference samples are unavailable or missing, determining whether one or more reference samples are available along another portion of the prediction unit, wherein the another portion of the prediction unit is along a side of the prediction unit at an opposite end of the particular directional mode.
• the present method improves intra prediction over the prior art techniques in those cases where reference samples are missing or unavailable for use and where the best directional mode of propagation for a prediction unit is not covered by classical padding of unavailable samples.
• Figure 1 depicts a current prediction unit (PU) or block in which certain reference samples are missing or unavailable for use in prediction; and Figures 2-6 depict exemplary scenarios showing the presence of useless directional modes and their replacement in accordance with the principles of the present method.
• PU current prediction unit
• Figures 2-6 depict exemplary scenarios showing the presence of useless directional modes and their replacement in accordance with the principles of the present method.
• the subject matter disclosed herein describes a method for improving intra frame prediction in digital video compression when reference samples are missing or unavailable.
• the present invention may at times be described primarily within the context of a particular digital video compression technique, the specific embodiments of the present inventive method should not be treated as limiting the scope of the inventive subject matter. It will be appreciated by those skilled in the art and informed by the teachings herein that the concepts of the present invention can be advantageously applied for improving substantially any intra frame prediction for compression of digital video images.
• reference sample and “reference pixel” are intended to be used interchangeably without limitation to the meaning.
• reference sample will be used almost exclusively in the description below.
• Recovery of some intra directional modes is realized by determining whether, for particular directional mode(s), there exist reference samples available for use along another portion of the prediction unit - the other portion being along the side of the prediction unit at an opposite end of the particular directional mode - and then by replacing the particular directional modes that are rendered useless (because of the unavailability of reference samples) with their counterpart directional modes that are oppositely directed.
• the method can involve initially determining whether reference samples are unavailable or missing for a particular prediction unit or portion thereof given a particular directional mode.
• this initial determination would identify that reference samples are missing and unavailable all along the left side of the prediction unit. These missing reference samples would be used in the scanning performed along intra directional modes 2-9, for example. But since the reference samples are missing and unavailable, the above-identified directional modes 2-9 are rendered effectively useless thereby undermining the accuracy and effectiveness of the intra frame prediction process.
• the prediction unit size is generally the same in height and width since prediction units are squares in intra mode. Given these dimensions, it can be appreciated that, contrary to the classical filling techniques in the art, all the pixels of the current prediction unit cannot be filled with accurate reference samples using these directional modes. More generally, even if a large number of reference samples were available on the above left of the prediction unit, some of these new directions shown in Figure 2 would not be able cover the whole prediction unit with meaningful samples.
• the oppositely directed mode to directional mode 6 of HEVC will not be able to cover all the pixels for effective prediction, leaving the question-marked area of the prediction unit below the solid sloped line uncovered.
• the method proceeds by extending the number of intra directional modes by replacing the original particular directional modes that are rendered useless, because of the absence or unavailability of reference samples, with their counterpart directional modes that are oppositely directed to the original particular directional modes.
• the counterpart directional mode(s) allow the use of the newly determined available reference samples in the intra frame prediction process.
• directional mode 6 is replaced by its oppositely directed counterpart directional mode shown on the right side of the prediction unit. This allows the prediction process to proceed using the reference sample(s) existing along the top edge of the prediction unit.
• the present method improves intra frame prediction over the prior art techniques in those cases where reference samples are missing or unavailable for use and where the best directional mode of propagation for a prediction unit is not covered by classical padding of unavailable samples.
• the improvement is realized from the use of a better reference sample than the missing or unavailable sample, which is found at the opposite end of the original directional mode.
• Several enhancements are contemplated and disclosed below for performing the filling operation the predicted block (i.e., prediction unit) with relevant available sample values.
• Figure 4 shows a prediction unit that has been filled according to the method disclosed above. No reference samples were available for use along the left edge of the prediction unit.
• the counterpart directional mode shown on the right side of the unit was substituted for - that is, it replaced - the original directional mode that was rendered effectively useless.
• An extrapolated solid line parallel to the counterpart directional mode is shown separating the filled area of the prediction unit from the unfilled area of the prediction unit. From experimental practice, it has been found that when all the available reference pixels have been propagated along the considered direction of the counterpart directional mode in the area above the extrapolated solid line, the method can be enhanced by filling the remaining unfilled region with the last reference sample's value. The value of this reference sample is shown as the box at the far end of the extrapolated line bordering the unfilled region.
• Figure 5 shows a prediction unit that has been filled according to the method disclosed above. No reference samples were available for use along the left edge of the prediction unit.
• the counterpart directional mode shown on the right side of the unit replaced the original directional mode that was rendered effectively useless by the missing reference samples.
• An extrapolated solid line parallel to the counterpart directional mode is shown separating the filled area of the prediction unit from the unfilled area of the prediction unit. From experimental practice, it has been found that when all the available reference pixels have been propagated along the considered direction of the counterpart directional mode in the area above the extrapolated solid line, the method can be enhanced by filling the remaining unfilled region with sample values that account for the changes in the sample values occurring in a direction normal to the extrapolated line.
• the values being assigned for the pixels in the prediction unit into the area below the extrapolated line are computed reflect the continuous decrease in shading along the directions that are depicted as normal to the extrapolated line.
• the sample values for the pixels are changed in accordance with the changes observed along the normal directions. Whether the sample values are computed by extrapolation or some other form of modeling, the pixel values below the extrapolated line find their basis in available reference samples. When the prediction unit is completely filled with the available reference sample values, the intra frame prediction process can proceed.
• Figure 6 shows a prediction unit that has been filled according to the method disclosed above. No reference samples were available for use along the left edge of the prediction unit.
• the counterpart directional mode shown on the right side of the unit replaced the original directional mode that was rendered effectively useless by the missing reference samples.
• An extrapolated solid line parallel to the counterpart directional mode is shown separating the filled area of the prediction unit from the unfilled area of the prediction unit.
• the method can be enhanced by filling the remaining unfilled region with the same sample value derived as an average of the available reference samples used in filling the area above the extrapolated line using the counterpart directional mode. This corresponds to using a DC-like mode for the area below the line.
• the prediction unit is completely filled with the available reference sample values, the intra frame prediction process can proceed.
• Figure 7 shows one embodiment of a method 700 for compression of digital video images including an operation for intra frame prediction.
• the method commences at Start block 701 and proceeds to block 710 for determining whether one or more reference samples are unavailable or missing for a prediction unit or portion thereof given a particular directional mode. If these samples are unavailable, control proceeds to block 720 for determining whether one or more reference samples are available along another portion of the prediction unit, wherein the another portion of the prediction unit is along a side of the prediction unit at an opposite end of the particular directional mode. If these samples are available, control proceeds back to the Start block 701 .
• control proceeds to block 730 for replacing the particular directional mode with a counterpart directional mode that is oppositely directed to the particular directional mode. If it is not determined that one or more reference samples are available along another portion of the prediction unit, control proceeds back to Start block 701 . After block 730, control proceeds to block 740 for filling the prediction unit with the available reference samples using the counterpart directional mode.
• Figure 8 shows one embodiment of an apparatus 800 for compression of digital video images including an operation for intra frame prediction.
• Apparatus 800 comprises Processor 810 which receives at least one input comprising video image data and is in signal communication with Memory 820 whose output port may also be available.
• Processor 810 may also comprise at least one output port.
• Processor 810 is configured to perform the method of Figure 7.
• processor or “controller” should not be construed to refer exclusively to hardware capable of executing software, and can implicitly include, without limitation, digital signal processor (“DSP”) hardware, read-only memory (“ROM”) for storing software, random access memory (“RAM”), and nonvolatile storage.
• DSP digital signal processor
• ROM read-only memory
• RAM random access memory

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Compression Or Coding Systems Of Tv Signals (AREA)

Priority Applications (12)

Applications Claiming Priority (2)

Related Child Applications (2)

Publications (1)

Family

ID=57130321

Family Applications (1)

Country Status (6)

Families Citing this family (5)

Citations (3)

Family Cites Families (22)

• 2016
  • 2016-09-27 EP EP16306238.3A patent/EP3300362A1/en not_active Withdrawn
• 2017
  • 2017-09-18 CN CN202310512613.2A patent/CN116647674A/zh active Pending
  • 2017-09-18 US US16/336,228 patent/US10834387B2/en active Active
  • 2017-09-18 KR KR1020197011869A patent/KR20190053257A/ko not_active Withdrawn
  • 2017-09-18 EP EP17768462.8A patent/EP3520398A1/en not_active Withdrawn
  • 2017-09-18 JP JP2019515624A patent/JP7365234B2/ja active Active
  • 2017-09-18 KR KR1020247012428A patent/KR20240052895A/ko not_active Withdrawn
  • 2017-09-18 CN CN201780059589.4A patent/CN109804623A/zh active Pending
  • 2017-09-18 KR KR1020227039000A patent/KR102658747B1/ko active Active
  • 2017-09-18 WO PCT/EP2017/073494 patent/WO2018059992A1/en not_active Ceased
• 2020
  • 2020-09-09 US US17/015,456 patent/US11363254B2/en active Active
• 2022
  • 2022-06-13 US US17/838,693 patent/US11863739B2/en active Active
• 2023
  • 2023-10-06 JP JP2023174089A patent/JP2023175946A/ja active Pending
  • 2023-11-27 US US18/519,573 patent/US20240163424A1/en not_active Abandoned

Patent Citations (4)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events