WO2019072918A1 - Onedimensional bilateral filtering of intra prediction reference pixels - Google Patents

Onedimensional bilateral filtering of intra prediction reference pixels Download PDF

Info

Publication number
WO2019072918A1
WO2019072918A1 PCT/EP2018/077604 EP2018077604W WO2019072918A1 WO 2019072918 A1 WO2019072918 A1 WO 2019072918A1 EP 2018077604 W EP2018077604 W EP 2018077604W WO 2019072918 A1 WO2019072918 A1 WO 2019072918A1
Authority
WO
WIPO (PCT)
Prior art keywords
reference sample
dimensional
block
reference samples
predictor
Prior art date
Application number
PCT/EP2018/077604
Other languages
French (fr)
Inventor
Philipp Merkle
Heiko Schwarz
Detlev Marpe
Thomas Wiegand
Original Assignee
Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. filed Critical Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.
Priority to EP18782086.5A priority Critical patent/EP3695598A1/en
Publication of WO2019072918A1 publication Critical patent/WO2019072918A1/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/11—Selection of coding mode or of prediction mode among a plurality of spatial predictive coding modes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/117—Filters, e.g. for pre-processing or post-processing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/182—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a pixel
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/189—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
    • H04N19/192—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding the adaptation method, adaptation tool or adaptation type being iterative or recursive
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/593—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression

Definitions

  • Embodiments of the present invention relate to a predictor for intra predicting a block of a picture. Further embodiments of the present invention relate to video decoders or to video encoders.
  • the components of a video frame are predicted either by motion compensated prediction, using the reconstructed color components of previous pictures, or by intra prediction, using previously reconstructed blocks of the same picture [1].
  • the basic concept of intra-picture prediction is that sample values of the current block are predicted using reference samples of previous coded blocks in the same picture. Usually, reference samples are adjacent samples left and above of the current block [2].
  • Intra-picture prediction either uses original (un- filtered) or modified (filtered) reference samples.
  • High-Efficiency Video Cod- ing uses low-pass filtered reference samples for larger blocks having a square shape and strong intra smoothing for 32x32 blocks.
  • the regular reference sample low-pass filter for larger blocks having a square shape is a 3-tap smoothing filter with positive kernel values, i.e. 1 ⁇ 4 [1 , 2, 1].
  • strong intra smoothing reference sample values are derived as the linear interpola- tion between the values of the corner and the end samples. [3].
  • Filtering reference samples in video coding using intra-picture prediction may improve a compression efficiency of the transmitted information and may reduce a computational load in video coding.
  • the quality of a reconstructed video may be reduced by such filtering, especially when the strength of the filter is increased.
  • the invention provides a predictor for intra predicting a predicted block for a currently processed block of a picture from reference samples in a two-dimensional coordinate system which are derived from one or more previously processed blocks of the picture, the predictor comprising: an arranger configured for arranging the reference samples as a one- dimensional reference sample sequence in a one-dimensional coordinate system; a smoothing filter configured for filtering the one-dimensional reference sample sequence based on a bilateral filter function with a one-dimensional window in order to obtain a one-dimensional filtered reference sample sequence comprising filtered reference samples; and a determiner configured for determining for each of a plurality of predicted image samples of the predicted block a respective prediction value by using the filtered reference sample sequence in order to produce the predicted block.
  • the predictor according to the invention is, in particular, suitable for a video encoder or a video decoder.
  • processing may refer to encoding or to decoding.
  • a block is a part of the picture which may have a square shape or a rectangular shape.
  • a currently processed block is that block which is currently to be processed. In case that the predictor is used within a video encoder, the term currently processed block describes the block currently encoded. If, however, the predictor is used within a video decoder, the term currently processed block describes the block currently to be decoded.
  • Previously processed blocks are such blocks which have been processed before the processing of the currently processed block has been started.
  • a predicted block is a block which comprises predicted image samples predicted by the predictor which are derived from the reference samples and are an approximation of the original image samples of the respective block.
  • a sample can be regarded as a pixel of the picture and is defined by its position and an intensity value.
  • the reference samples for intra predicting a currently processed block of a picture are derived from previously processed blocks of the same picture, which are positioned adjacent to the currently processed block.
  • Each of the reference samples is defined by a two-dimensional position in a two- dimensional coordinate system of the picture and by an intensity value sometimes referred to as sample value.
  • the reference samples may, for example, be stored in a memory array. In case that an intensity value of one of the reference samples is not available, the intensity value of the one reference sample may be interpolated using intensity values of other reference samples.
  • the arranger is configured for arranging such reference samples into a one- dimensional reference sample sequence so that each reference sample may be identified by a one-dimensional position in a one-dimensional coordinate system.
  • a one-dimensional window may be used for subsequent filtering of the reference sample sequence which requires less computational effort.
  • the arranger does usually not change the intensity values of the reference samples.
  • the arranger may be configured for copying the reference samples from a memory array to a line buffer.
  • the determiner is capable of determining for each of a plurality of predicted image samples of the currently processed block a respective prediction value by using the filtered reference sample sequence in order to produce the predicted block.
  • the determiner may be capable of executing different prediction modes such as a planar intra-prediction mode, a DC intra- prediction mode and/or an angular intra-prediction mode as used in High Efficient Video Coding.
  • the determiner may be configured for determining for each of a plurality of predicted image samples of the currently processed block a respective prediction value by, depending on a prediction mode used for producing the predicted block, a channel type and/or depending on a size of the currently processed block, either using the filtered reference sample sequence or the (un- filtered) reference samples in order to produce the predicted block.
  • a prediction mode used for producing the predicted block
  • a channel type and/or depending on a size of the currently processed block
  • a luma channel represents the brightness in an image (the "black-and-white" or achromatic portion of the image).
  • a luma channel may be used alone, but is typically paired with a chroma channel, wherein the luma represents the achromatic image, while the chroma channel represents the color information.
  • the smoothing filter has a one-dimensional window, which requires a lower computational effort compared to a two-dimensional window.
  • the smoothing filter uses a bilateral filter function.
  • a bilateral filter function combines a range kernel function, sometimes referred to as intensity function, with a spatial kernel function such that an outputted sample value for a specific position, depends on an intensity difference between an inputted intensity value for the specific position and an inputted intensity value for a further specific position and on a position difference between the specific position and the further specific position multiplied by the intensity value for the further position.
  • low-pass filters as used in references [2] and [3] only comprise a spatial kernel function such that an outputted intensity value only depends on the position difference between the specific position and the further specific position multiplied by the intensity value for the further position.
  • the bilateral filter function is defined as with normalization term where
  • Each of the filtered reference samples has a position in the one-dimensional coordinate system so that each of the filtered reference samples has one- dimensional coordinates being the same as the one-dimensional coordinates of the reference sample corresponding to the respective filtered reference sample.
  • the intensity value of a filtered reference sample may, and usually will, differ from an intensity value of the reference sample having the same one-dimensional coordinates as the respective filtered reference sample.
  • filtering the reference samples takes place prior to the prediction process, so that filtering has an effect on the predicted values of the predicted image samples for the currently processed block, but no effect on the predicted values of previously processed blocks.
  • the invention improves a compression efficiency of the transmitted information and reduces a computational load in video coding, while the quality of a reconstructed video is maintained.
  • the invention avoids small local details being predicted over several rows/columns of samples.
  • the invention preserves sharp edges.
  • bilateral filtering according to the invention in contrast to conventional filtering, is also beneficial for very large blocks of different shapes, in particular, bilateral filtering is also beneficial for rectangular blocks which have a large difference between the number of the rows and the number of the columns, wherein conventional filtering is only suitable for blocks having a square shape.
  • At least a part of the reference samples in the two-dimensional coordinate system sequentially extend along two rays starting at a common intersection point and pointing into different directions, wherein the arranger is configured for arranging the reference samples into a one-dimensional reference sample sequence by anti-parallelly traveling along a first one of the two rays and paralle!ly traveling along a second one of the two rays.
  • the reference samples in the two-dimensional coordinate system comprise first edge reference samples bordering on a first side of the currently processed block.
  • the reference samples in the two-dimensional coordinate system comprise second edge reference samples bordering on a second side of the currently processed block.
  • a range kernel function of the bilateral filter function is a Gaussian range kernel function
  • a spatial kernel function of the bilateral filter function is a Gaussian spatial kernel function
  • a Gaussian filter function is defined as so that a Gaussian range kernel function can be written as
  • the bilateral filter function (1) can be reformulated as using the weight function
  • the smoothing filter is configured for iteratively filtering the one-dimensional reference sample sequence in order to obtain the one-dimensional filtered reference sample se- quence comprising the filtered reference samples. Iteratively filtering means, that a result of a filtering step is used as an intermediate result which is used as an input for a further filtering step, wherein a arbitrary number of further filtering steps may be executed before the result of the last filtering step is outputted by the filter.
  • the bilateral filter function (6) can be reformulated as
  • a reference sample sequence can be understood as a one-dimensional array of values such that the bilateral filter function (8) can be reformulated as
  • the smoothing filter is configured for outputting the one-dimensiona! filtered reference sample sequence as a sequence of integer numbers.
  • the determiner comprises a shifter configured for shifting the filtered reference samples in order to obtain shifted filtered reference samples, wherein one of the filtered reference samples is shifted to a position of one of the reference samples in the two-dimensional coordinate system which corresponds to the one of the filtered reference samples in order to obtain one of the shifted filtered reference samples.
  • each of the shifted filtered reference samples has two- dimensional coordinates in the two-dimensional coordinate system which are the same as the two-dimensional coordinates of the reference sample which corresponds to the one of the filtered reference samples. This may be achieved by copying the filtered reference samples from a line buffer to a memory array which corresponds to a memory array in which the original (unfiltered) reference samples previously has been stored.
  • the determiner comprises a prediction finder configured for determining the prediction value for the predicted image sample of the currently processed block by using the shifted filtered reference sample sequence.
  • the prediction finder is that part of the determiner which predicts the prediction values for the predicted block based on the filtered reference samples.
  • the predictor comprises a lookup table configured for receiving from the smoothing filter a spatial difference between a currently processed filtered reference sample of the one-dimensional filtered reference sample sequence and a reference sample of the one- dimensional reference sample sequence, configured for receiving from the smoothing filter an intensity difference between the currently processed filtered reference sample and the reference sample, and configured for transmitting to the smoothing filter a weight for calculating an intensity value for the currently processed filtered reference sample, wherein the weight is selected from a set of weights stored at the look-up table depending on the received spatial difference and the received intensity difference, and wherein the set of weights is pre-calculated based on a weight function of the bilateral filter function.
  • the filter weights only depend on spatial difference and intensity difference with bit depth b.
  • the weights are stored as integer numbers.
  • the bilateral filter function (10) can be reformulated as
  • the predictor comprises a lookup table arrangement comprising a first lookup table, a second lookup table and a multiplier, wherein the first lookup table is configured for receiving from the smoothing filter a spatial difference between a currently processed filtered reference sample of the one-dimensional filtered reference sample sequence and one reference sample of the one-dimensional reference sample sequence, wherein the second lookup table is configured for receiving from the smoothing filter an intensity difference between the currently processed filtered reference sample and the one reference sample, wherein the lookup table arrangement is configured for transmitting to the smoothing filter a weight for calculating an intensity value for the currently processed filtered reference sample, wherein the multiplier is configured for calculating the weight by multiplying a first weight factor and a second weight factor, wherein the first weight factor is selected from a set of first weight factors stored at the first look-up table depending on
  • two one-dimensional lookup tables may be used in order to determine the filter weights w n (x).
  • the set of first weight factors w n i(xi) stored at the first iook-up table may be pre-calculated using a spatial weight function of the bilateral filter function which is the spatial term of the weight function (1 1 ).
  • the set of second weight factors stored at the second look-up table may be pre-calculated using an intensity weight function of the bilateral filter function which is the intensity term of the weight function (1 1 ).
  • the filter weights may then be calculated by
  • the invention provides a video encoder for encoding a picture into a data stream, wherein the video encoder comprises a predictor according to the invention.
  • the video encoder comprises a residual information determiner configured for comparing the predicted image samples of the predicted block with original image samples of the currently processed block of the picture in order to determine a residual information, a transformer configured for quantizing the residual information in order to obtain a transform coefficient block, a data stream inserter configured for inserting the transform coefficient block and intra prediction parameters received from the predictor into the data stream, an inverse transformer configured for dequantizing the transform coefficient block in order to obtain a reconstructed residual information and a reconstructor configured for reconstructing subsequent reference samples for intra predicting a subsequent predicted block for a subsequently processed block of the picture based on the predicted block and based on the reconstructed residual information.
  • a residual information determiner configured for comparing the predicted image samples of the predicted block with original image samples of the currently processed block of the picture in order to determine a residual information
  • a transformer configured for quantizing the residual information in order to obtain a transform coefficient block
  • a data stream inserter configured for inserting the transform coefficient block and intra prediction parameters received from the predictor into the data stream
  • the invention provides a video decoder for decoding a data stream in order to reconstruct a picture, wherein the video decoder comprises a predictor according to the invention.
  • the video decoder comprises an extractor configured for extracting a transform coefficient block for the currently processed block of the picture and intra prediction parameters for the currently processed block of the picture from the data stream and configured to send the intra prediction parameters to the predictor, an inverse transformer configured for invers transforming the transform coefficient block in order to obtain a reconstructed residual information and a reconstructor configured for producing the reconstructed picture based on the predicted block and based on the reconstructed residual information and configured for reconstructing further reference samples for intra predicting a following predicted block based on the predicted block and based on the reconstructed residual information.
  • the invention provides a method for intra predicting a predicted block for a currently processed block of a picture from reference samples in a two-dimensional coordinate system which are derived from one or more previously processed blocks of the picture, the method comprising the steps: arranging, by using an arranger, the reference samples as a one-dimensional reference sample sequence in a one-dimensional coordinate system; filtering, by using a smoothing filter, the one-dimensional reference sample sequence based on a bilateral filter function with a one-dimensional window in order to obtain a one-dimensional filtered reference sample sequence comprising filtered reference samples; and determining, by using a determiner, for each of a plurality of predicted image samples of the predicted block a respective prediction value by using the filtered reference sample sequence in order to produce the predicted block.
  • the invention provides a computer program for, when run- ning on a processor, executing the method according to the invention.
  • FIG. 1 illustrates an embodiment of a predictor according to the invention in a schematic view
  • Fig. 2 illustrates an example of a currently processed block, a plurality of previously processed blocks and reference samples given in a two-dimensional coordinate system which are derived from the previous processed blocks in a schematic view;
  • Fig. 3 illustrates an example of a one-dimensional reference sample sequence given in a one-dimensional coordinate system derived from the reference samples illustrated in Fig. 2 in a schematic view;
  • Fig. 4 illustrates an example of a predicted block comprising predicted image samples, wherein each image sample comprises a prediction value, in a schematic view
  • Fig. 5 illustrates an embodiment of a determiner for a predictor
  • Fig. 6 illustrates an example of a one-dimensional filtered reference sample sequence derived from the reference sample sequence illustrated in Fig. 3 in a schematic view;
  • Fig. 7 illustrates an example of shifted filtered reference samples derived from the filtered reference sample sequence illustrated in Fig. 6 in a schematic view
  • Fig. 8 illustrates an embodiment of a predictor according to the invention in a schematic partial view
  • Fig. 9 illustrates an embodiment of a video encoder according to the invention in a schematic view
  • Fig. 10 illustrates an embodiment of a video decoder according to the invention in a schematic view
  • Fig. 11 illustrates a further embodiment of a predictor according to the invention in a schematic partial view.
  • Fig. 1 illustrates an embodiment of a predictor 1 according to the invention in a schematic view.
  • the inventive predictor 1 is configured for intra predicting a predicted block PB for a currently processed block CPB of a picture PIC from reference samples RS in a two-dimensional coordinate system which are derived from previously processed blocks PPB of the picture PIC.
  • the predictor 1 comprises an arranger 2 configured for arranging the reference samples RS as a one- dimensional reference sample sequence RSS in a one-dimensional coordinate system; a smoothing filter 3 configured for filtering the one-dimensional reference sample sequence RRS based on a bilateral filter function with a one- dimensional window Wl in order to obtain a one-dimensional filtered refer- ence sample sequence FRSS comprising filtered reference samples FRS; and a determiner 4 configured for determining for each of a plurality of predicted image samples PIS of the predicted block PB a respective prediction value PV by using the filtered reference sample sequence FRSS in order to produce the predicted block PB.
  • a range kernel function of the bilateral filter function is a Gaussian range kernel function
  • a spatial kernel function of the bilateral filter function is a Gaussian spatial kernel function
  • the smoothing filter 3 is configured for iteratively filtering the one-dimensional reference sample se- quence RSS in order to obtain the one-dimensional filtered reference sample sequence FRSS comprising the filtered reference samples FRS.
  • the smoothing filter 3 is configured for outputting the one-dimensional filtered reference sample se- quence FRSS as a sequence of integer numbers.
  • the invention provides a method for intra predicting a predicted block PB for a currently processed block CPB of a picture PIC from reference samples RS in a two-dimensional coordinate system which are derived from one or more previously processed blocks PPB of the picture PIC, the method comprising the steps: arranging, by using an arranger 2, the reference samples RS as a one- dimensional reference sample sequence RSS in a one-dimensional coord i- nate system; filtering, by using a smoothing filter 3, the one-dimensional reference sample sequence RRS based on a bilateral filter function with a one-dimensional window Wl in order to obtain a one-dimensional filtered reference sample sequence FRSS comprising filtered reference samples FRS; and determining, by using a determiner 4, for each of a plurality of predicted image samples PIS of the predicted block PB a respective prediction value PV by using the filtered reference sample sequence FRSS in order to produce the predicted block PB.
  • the invention provides a computer, program for, when running on a processor, executing the inventive method.
  • the predictor 1 may be employed in a hybrid video decoder or in a hybrid video decoder.
  • An advantage of using a smoothing filter 3 based on data bilateral filter function for filtering the reference samples RS is that outliers within the reference samples RS do not negatively affect the intra prediction result, whereas edges are preserved. The computational overhead may be kept reasonable.
  • Fig. 2 illustrates an example of a currently processed block CBP, a plurality of previously processed blocks PPB and reference samples RS given in a two-dimensional coordinate system, which are derived from the previous processed blocks PPB, in a schematic view.
  • the currently processed block CBP has a width W and a height H.
  • the reference samples RS in the two-dimensional coordinate system sequentially extend along two rays R1 , R2 starting at a common intersection point IP and pointing into different directions, wherein the arranger 2 is configured for arranging the reference samples RS into a one-dimensional reference sample sequence RSS by anti-parallelly traveling along a first one R1 of the two rays R1 , R2 and parallelly traveling along a second one R2 of the two rays R1 , R2.
  • the reference samples RS in the two-dimensional coordinate system comprise first edge reference samples ERS1 bordering a first side of the currently processed block CPB.
  • the reference samples RS in the two-dimensional coordinate system comprise second edge reference samples ERS2 bordering a second side of the currently processed block CPB.
  • first edge reference samples ERS1 are bordering the left side of the currently processed block CBP and second edge reference samples ERS2 are bordering the upper side of the currently processed block CBP.
  • the reference samples RS are given in a two- dimensional coordinate system.
  • each of the reference samples RS has two-dimensional coordinates.
  • Fig. 2 illustrates the references samples RS, before they are processed by the arranger 2.
  • the bilateral filtering may be applied to all blocks which are intra predicted.
  • the bilateral filtering may be applied to blocks containing luma in- formation (e.g. blocks of a luma channel).
  • blocks containing chroma information e.g. blocks of a chroma channel.
  • bilaterai filtering may be applied to all blocks having a width and/or a height being equal or greater than 16 samples. Depending on the block size, following parameters may be used:
  • Fig. 3 illustrates an example of a one-dimensional reference sample se- quence RSS given in a one-dimensional coordinate system derived from the reference samples RS illustrated in Fig. 2 in a schematic view.
  • the reference sample sequence RSS comprises 2(W+H) + 1 reference samples RS so that all angular prediction modes may be performed.
  • Fig. 3 illustrates the reference samples RS, after they are processed by the arranger 2.
  • a filtered reference sample FRS for the position ka one- dimensional window Wl depending on the filter size s is used in that way, that all reference samples RS having a position between k-s ⁇ o k+s are considered.
  • the filtered reference sample FRS for the position kmay be calculated by using equation (10) using the window Wl.
  • Fig. 4 illustrates an example of a predicted block PTB comprising predicted image samples PIS, wherein each image sample PIS comprises a prediction value PV, in a schematic view.
  • the predicted block PB consists of predicted image samples PIS, wherein each of the predicted image samples PIS has a position indicated by two indices and the predicted value having the same two indices.
  • Fig. 5 illustrates an embodiment of a determiner 4 for a predictor 1 according to the invention in a schematic view.
  • the determiner 4 comprises a shifter 5 configured for shifting the filtered reference samples FRS in order to obtain shifted filtered reference samples SFRS, wherein one of the filtered reference samples FRS is shifted to a position of one of the reference samples RS in the two-dimensional coordinate system which corresponds to the one of the filtered reference samples FRS in order to obtain one of the shifted filtered reference samples SFRS.
  • the determiner 4 comprises a prediction finder 6 configured for determining the prediction value PV for the predicted image sample PIS of the currently processed block CPB by using the shifted filtered reference sample sequence SFRS.
  • Fig. 6 illustrates an example of a one-dimensional filtered reference sample sequence FRSS derived from the reference sample sequence RSS illustrated in Fig. 3 in a schematic view.
  • the one-dimensional filtered reference sample sequence FRSS may be inputted into a shifter 5.
  • Fig. 7 illustrates an example of shifted filtered reference samples SFRS derived from the filtered reference sample sequence FRSS illustrated in Fig. 6 in a schematic view.
  • the shifted filtered reference samples SFRS maybe outputted by the shifter 5.
  • Fig. 8 illustrates an embodiment of a predictor 1 according to the invention in a schematic partial view.
  • the predictor 1 comprises a lookup table 7 configured for receiving from the smoothing filter 3 a spatial difference SPD between a currently processed filtered reference sample FRS of the one- dimensional filtered reference sample sequence FRSS and one reference sample RS of the one-dimensional reference sample sequence RSS, configured for receiving from the smoothing filter 3 an intensity difference ITD between the currently processed filtered reference sample FRS and the one reference sample RS, and configured for transmitting to the smoothing filter 3 a weight WE for calculating an intensity value for the currently processed filtered reference sample FRS, wherein the weight WE is selected from a set of weights stored at the look-up table depending on the received spatial difference SPD and the received intensity difference ITD, and wherein the set of weights is p recalculated based on a weight function of the bilateral filter function.
  • Fig. 9 illustrates an embodiment of a video encoder 8 according to the invention in a schematic view.
  • the invention provides encoder for encoding a picture PIC into a data stream DS, the video encoder 8 comprising a predictor 1 according to the invention.
  • the video encoder 8 comprises a residual information determiner 9 configured for comparing the predicted image samples PIS of the predicted block PB with original image samples OIS of the currently processed block CBP of the picture PIC in order to determine a residual information RIN, a transformer 10 configured for quantizing the residual information RIN in order to obtain a transform coefficient block TCB, a data stream inserter 11 configured for inserting the transform coefficient block TCB and intra prediction parameters IPP received from the predictor 1 into the data stream DS, an inverse transformer 12 configured for dequantizing the transform coefficient block TCB in order to obtain a reconstructed residual information RRIN and a reconstructor 13 configured for reconstructing subsequent reference samples SURS for intra predicting a subsequent predicted block for a subsequently processed block of the picture based on the predicted block PB and based on the reconstructed residual information RRIN.
  • the video encoder 8 may comprise a block determiner 14 configured for splitting up the picture PIC into a plurality of blocks. The blocks may have a rectangular
  • a predicted block PB of a picture PIC may be predicted using intra prediction.
  • the predictor 1 of the video encoder 8 may be configured to determine a predicted block PB for a currently processed block CBP which has to be coded of the current picture PIC of a video.
  • the predictor 1 determines the predicted block PB for the currently processed block CBP based on reference samples RS, which have been encoded before the currently processed block CBR.
  • the reference samples RS are bordering the currently processed block CBR of the same picture PIC like the currently processed block CBR.
  • the reference samples RS are filtered using the smoothing filter 3, which uses the bilateral filter function, to sup- press outliers within the reference samples RS, such that these outliers do not negatively affect (or only insignificantly negatively affect) the predicted block PB for the currently processed block CPB and such that sharp edges are preserved.
  • the residual information determiner 9 is configured to compare the predicted block PB for the currently processed block CPB with the (original) currently processed block CPB in order to determine a residual information RIN for the currently processed block CPB.
  • the residual information RIN determines the difference of the (original) currently processed block CPB and its predicted block PB.
  • the transformer 10 may transform the residual information RIN and quantize the resulting transform coefficients to obtain a transform coefficient block TCB.
  • a quality loss (if any) in the video encoder 8 occurs in the quantization of the transform coefficients of the transform coefficient block TCB.
  • the transform coefficient block TCB may then together with the intra prediction parameter IPP be inserted into a data stream DS by the data stream inserter 1 1.
  • the data stream DS may be transmitted to a video decoder in which the currently processed block CBP may be reconstructed.
  • the transform coefficient block TCB comprising the quantized transform coefficients representing the residual information RIN may be inverse transformed by the inverse transformer 12 to obtain reconstructed residual information RRIN for the currently processed block CPB.
  • the reconstructed residual information RRIN may show a loss compared to the residual information RIN because of the quantization of the transform coefficients in the transformer 10.
  • the reconstructed residual information RRIN may be used by the recon- structor 13 in conjunction with the predicted block PB for currently processed block CBP to obtain reference samples RS from the currently processed block CPB, which then may be used as reference samples RS for a subsequently processed block.
  • predictor 1 of the video encoder 8 may be configured to include into the intra- prediction parameters IPP an information, whether bilateral filtering has been used or have not been used for encoding the currently processed block CBP. For the same reason, predictor 1 of the video encoder 8 may be configured to include into the intra-prediction parameters IPP parameters of the bilateral filter function. Also, predictor 1 of the video encoder 8 may be configured to include into the intra-prediction parameters IPP information regarding the prediction mode.
  • Fig. 10 illustrates an embodiment of a video decoder 15 according to the invention in a schematic view.
  • the invention provides a video decoder 15 for decoding a data stream DS in order to reconstruct a picture PIC, wherein the video decoder 15 comprises a predictor 1 according to the invention.
  • the video decoder 15 comprises an extractor 16 configured for extracting a transform coefficient block TCB for the currently processed block CPB of the picture PIC and intra prediction parameters IPP for the currently processed block CPB of the picture PIC from the data stream DS and configured to send the intra prediction parameters IPP to the predictor 1 , an inverse transformer 12 configured for in vers transforming the transform coefficient block TCB in order to obtain a reconstructed residual information RRIN and a reconstructor 13 configured for producing the currently processed block CPB based on the predicted block PB and based on the reconstructed residual information RRIN and configured for producing subsequent reference samples SURS for intra predicting a subsequent predicted block for a subsequently processed block of the picture PIC based on the predicted block PB and based on the reconstructed residual information RRIN.
  • an extractor 16 configured for extracting a transform coefficient block TCB for the currently processed block CPB of the picture PIC and intra prediction parameters IPP for the currently processed block CPB of the picture PIC from the data stream DS and configured to send
  • the video decoder 15 may comprise a picture determiner configured for reconstruction the picture PIC by adding the currently processed block CBP to previously processed blocks PPB.
  • the video decoder 15 may be configured to decode the data stream DS, which may have been produced with the video encoder 8 according to Fig. 9.
  • Fig. 10 same reference numerals as in Fig. 9 are used, in order to denote elements in the video decoder 15 which are equal or similar elements of the video encoder 8.
  • the extractor 16 may be configured to extract the transform coefficient block TCB for the currently processed block CBP and the intra prediction parameter IPP for the currently processed block CBP from the data stream DS (which, for example, is provided by the video encoder 8).
  • the inverse transformer 12 obtains (like the inverse transformer 12 of the video encoder 8) the reconstructed residual information RRIN for the currently processed block CPB.
  • the predictor 1 is configured to determine the predicted block PB for the currently processed block CPB by using the reference samples RS (which, for example, are derived from a previously processed block PPB).
  • the reference samples RS are filtered by using the (bilateral) smooth- ing filter 3 of the predictor 1 in order to obtain the filtered reference samples FRS.
  • a determiner 4 of the predictor 1 may then obtain the predicted block PB for the currently processed block CPB based on the intra prediction parameter IPP for the currently processed block CPB and the filtered reference samples FRS.
  • the determiner for may determine the predicted block PB for the currently processed block CPB on the same way like the determiner for of the video encoder 8.
  • the determiner for may therefore be a reduced version of the determiner for of the video encoder 8, as the determiner for of the video encoder 8 is, as mentioned before, further configured to obtain the intra prediction parameter IPP for the currently processed block .
  • the predicted block PB for the currently processed block CPB may then together with the reconstructed residual information RRIN for the currently processed block CPB be combined by the reconstructor 13 to obtain a reconstructed version of the currently processed block CPB and reconstructed ref- erence samples RS of the currently processed block CPB.
  • This reconstructed reference samples RS of the currently processed block CPB may be used again as reference samples RS for subsequently processed blocks.
  • the reconstructed currently processed block CBP may be sent to the picture determiner 17.
  • Fig. 1 1 illustrates a further embodiment of a predictor 1 according to the invention in a schematic partial view.
  • the predictor 1 com- prises a lookup table arrangement 18 comprising a first lookup table 19, a second lookup table 20 and a multiplier 21
  • the first lookup table 19 is configured for receiving from the smoothing filter 3 a spatial difference SPD between a currently processed filtered reference sample FRS of the one-dimensional filtered reference sample sequence FRSS and one reference sample RS of the one-dimensional reference sample sequence RSS
  • the second lookup table 20 is configured for receiving from the smoothing filter 3 an intensity difference ITD between the currently processed filtered reference sample FRS and the one reference sample RS
  • the lookup table arrangement 18 is configured for transmitting to the smoothing filter 3 a weight WE for calculating an intensity value for the cur- rently processed filtered reference sample FRS
  • the multiplier 21 is configured for calculating the weight WE by multiplying a first weight factor FWF and a second weight factor SWF, wherein the first weight factor FWF is selected from a set of first weight factors stored at the first look-
  • predictor 1 of the video decoder 15 may be configured to extract from the intra-prediction parameters IPP an information, whether bilateral filtering has been used or have not been used for encoding the currently processed block CBP. For the same reason, predictor 1 of the video decoder 15 may be configured to extract from the intra-prediction parameters IPP parameters of the bilateral filter function. Also, predictor 1 of the video decoder 15 may be configured to extract from the intra-prediction parameters IPP information regarding the prediction mode.
  • embodiments of the inventive device and system can be implemented in hardware and/or in software.
  • the implementation can be performed using a digital storage medium, for example a floppy disk, a DVD, a Blu-ray Disc, a CD, a ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that one or more or all of the functionalities of the inventive device or system is performed.
  • a digital storage medium for example a floppy disk, a DVD, a Blu-ray Disc, a CD, a ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that one or more or all of the functionalities of the inventive device or system is performed.
  • a programmable logic device for example a field programmable gate array
  • a field programmable gate array may cooperate with a microprocessor in order to perform one or more or all of the functionalities of the devices and systems described herein.
  • aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus.
  • embodiments of the inventive method can be implemented using an apparatus comprising hardware and/or software.
  • the implementation can be performed using a digital storage medium, for example a floppy disk, a DVD, a Blu-ray Disc, a CD, a ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed.
  • embodiments of the inventive method can be implemented using an apparatus comprising hardware and/or software.
  • Some or all of the method steps may be executed by (or using) a hardware apparatus, like a microprocessor, a programmable computer or an electronic circuit. Some one or more of the most important method steps may be executed by such an apparatus.
  • Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system such that one of the methods described herein is performed.
  • embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer.
  • the program code may for example be stored on a machine readable carrier.
  • inventions comprise the computer program for performing one of the methods described herein, which is stored on a machine readable carrier or a non-transitory storage medium.
  • a further embodiment comprises a processing means, for example a computer, or a programmable logic device, in particular a processor comprising hardware, configured or adapted to perform one of the methods described herein.
  • a further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.
  • the methods are advantageously performed by any apparatus comprising hardware and or software.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

Predictor for intra predicting a predicted block for a currently processed block of a picture from reference samples in a two-dimensional coordinate system which are derived from one or more previously processed blocks of the picture, the predictor comprising: an arranger configured for arranging the reference samples as a one-dimensional reference sample sequence in a one-dimensional coordinate system; a smoothing filter configured for filtering the one-dimensional reference sample sequence based on a bilateral filter function with a one-dimensional window in order to obtain a one-dimensional filtered reference sample sequence comprising filtered reference samples; and a determiner configured for determining for each of a plurality of predicted image samples of the currently processed block a respective prediction value by using the filtered reference sample sequence in order to produce the predicted block.

Description

ONEDIMENSIONAL BILATERAL FILTERING OF INTRA PREDICTION
REFERENCE PIXELS
Description
Embodiments of the present invention relate to a predictor for intra predicting a block of a picture. Further embodiments of the present invention relate to video decoders or to video encoders. In the prior art video coding, the components of a video frame are predicted either by motion compensated prediction, using the reconstructed color components of previous pictures, or by intra prediction, using previously reconstructed blocks of the same picture [1]. The basic concept of intra-picture prediction is that sample values of the current block are predicted using reference samples of previous coded blocks in the same picture. Usually, reference samples are adjacent samples left and above of the current block [2]. Intra-picture prediction either uses original (un- filtered) or modified (filtered) reference samples. High-Efficiency Video Cod- ing (HEVC) uses low-pass filtered reference samples for larger blocks having a square shape and strong intra smoothing for 32x32 blocks. The regular reference sample low-pass filter for larger blocks having a square shape is a 3-tap smoothing filter with positive kernel values, i.e. ¼ [1 , 2, 1]. In strong intra smoothing reference sample values are derived as the linear interpola- tion between the values of the corner and the end samples. [3].
Filtering reference samples in video coding using intra-picture prediction according to prior art may improve a compression efficiency of the transmitted information and may reduce a computational load in video coding. However, the quality of a reconstructed video may be reduced by such filtering, especially when the strength of the filter is increased.
It is an objective of the present invention to provide a concept for improving video coding using intra prediction. In a first aspect the invention provides a predictor for intra predicting a predicted block for a currently processed block of a picture from reference samples in a two-dimensional coordinate system which are derived from one or more previously processed blocks of the picture, the predictor comprising: an arranger configured for arranging the reference samples as a one- dimensional reference sample sequence in a one-dimensional coordinate system; a smoothing filter configured for filtering the one-dimensional reference sample sequence based on a bilateral filter function with a one-dimensional window in order to obtain a one-dimensional filtered reference sample sequence comprising filtered reference samples; and a determiner configured for determining for each of a plurality of predicted image samples of the predicted block a respective prediction value by using the filtered reference sample sequence in order to produce the predicted block.
The predictor according to the invention is, in particular, suitable for a video encoder or a video decoder. Thus, herein the term processing may refer to encoding or to decoding. A block is a part of the picture which may have a square shape or a rectangular shape. A currently processed block is that block which is currently to be processed. In case that the predictor is used within a video encoder, the term currently processed block describes the block currently encoded. If, however, the predictor is used within a video decoder, the term currently processed block describes the block currently to be decoded. Previously processed blocks are such blocks which have been processed before the processing of the currently processed block has been started. A predicted block is a block which comprises predicted image samples predicted by the predictor which are derived from the reference samples and are an approximation of the original image samples of the respective block. A sample can be regarded as a pixel of the picture and is defined by its position and an intensity value. In general, the reference samples for intra predicting a currently processed block of a picture are derived from previously processed blocks of the same picture, which are positioned adjacent to the currently processed block. Each of the reference samples is defined by a two-dimensional position in a two- dimensional coordinate system of the picture and by an intensity value sometimes referred to as sample value. The reference samples may, for example, be stored in a memory array. In case that an intensity value of one of the reference samples is not available, the intensity value of the one reference sample may be interpolated using intensity values of other reference samples.
The arranger is configured for arranging such reference samples into a one- dimensional reference sample sequence so that each reference sample may be identified by a one-dimensional position in a one-dimensional coordinate system. As a result, a one-dimensional window may be used for subsequent filtering of the reference sample sequence which requires less computational effort. However, the arranger does usually not change the intensity values of the reference samples. The arranger may be configured for copying the reference samples from a memory array to a line buffer.
The determiner is capable of determining for each of a plurality of predicted image samples of the currently processed block a respective prediction value by using the filtered reference sample sequence in order to produce the predicted block. For this purpose, the determiner may be capable of executing different prediction modes such as a planar intra-prediction mode, a DC intra- prediction mode and/or an angular intra-prediction mode as used in High Efficient Video Coding.
The determiner may be configured for determining for each of a plurality of predicted image samples of the currently processed block a respective prediction value by, depending on a prediction mode used for producing the predicted block, a channel type and/or depending on a size of the currently processed block, either using the filtered reference sample sequence or the (un- filtered) reference samples in order to produce the predicted block. In video technology luma channels and chroma channels are known. A luma channel, represents the brightness in an image (the "black-and-white" or achromatic portion of the image). A luma channel may be used alone, but is typically paired with a chroma channel, wherein the luma represents the achromatic image, while the chroma channel represents the color information.
The smoothing filter has a one-dimensional window, which requires a lower computational effort compared to a two-dimensional window. The smoothing filter uses a bilateral filter function. A bilateral filter function combines a range kernel function, sometimes referred to as intensity function, with a spatial kernel function such that an outputted sample value for a specific position, depends on an intensity difference between an inputted intensity value for the specific position and an inputted intensity value for a further specific position and on a position difference between the specific position and the further specific position multiplied by the intensity value for the further position.
In contrast to that, low-pass filters as used in references [2] and [3] only comprise a spatial kernel function such that an outputted intensity value only depends on the position difference between the specific position and the further specific position multiplied by the intensity value for the further position.
In general, the bilateral filter function is defined as
Figure imgf000006_0002
with normalization term
Figure imgf000006_0001
where
Figure imgf000007_0001
Each of the filtered reference samples has a position in the one-dimensional coordinate system so that each of the filtered reference samples has one- dimensional coordinates being the same as the one-dimensional coordinates of the reference sample corresponding to the respective filtered reference sample. However, the intensity value of a filtered reference sample may, and usually will, differ from an intensity value of the reference sample having the same one-dimensional coordinates as the respective filtered reference sample.
It has to be noted that filtering the reference samples takes place prior to the prediction process, so that filtering has an effect on the predicted values of the predicted image samples for the currently processed block, but no effect on the predicted values of previously processed blocks.
The invention improves a compression efficiency of the transmitted information and reduces a computational load in video coding, while the quality of a reconstructed video is maintained. In particular, the invention avoids small local details being predicted over several rows/columns of samples. In contrast to conventional strong low-pass filtering, the invention preserves sharp edges. Furthermore, it has turned out that bilateral filtering according to the invention, in contrast to conventional filtering, is also beneficial for very large blocks of different shapes, in particular, bilateral filtering is also beneficial for rectangular blocks which have a large difference between the number of the rows and the number of the columns, wherein conventional filtering is only suitable for blocks having a square shape.
According to a preferred embodiment of the invention at least a part of the reference samples in the two-dimensional coordinate system sequentially extend along two rays starting at a common intersection point and pointing into different directions, wherein the arranger is configured for arranging the reference samples into a one-dimensional reference sample sequence by anti-parallelly traveling along a first one of the two rays and paralle!ly traveling along a second one of the two rays.
According to a preferred embodiment of the invention the reference samples in the two-dimensional coordinate system comprise first edge reference samples bordering on a first side of the currently processed block.
According to a preferred embodiment of the invention the reference samples in the two-dimensional coordinate system comprise second edge reference samples bordering on a second side of the currently processed block.
According to a preferred embodiment of the invention a range kernel function of the bilateral filter function is a Gaussian range kernel function, and wherein a spatial kernel function of the bilateral filter function is a Gaussian spatial kernel function.
In general, a Gaussian filter function is defined as
Figure imgf000008_0001
so that a Gaussian range kernel function can be written as
Figure imgf000008_0003
and that a Gaussian spatial kernel function can be written as
Figure imgf000008_0002
For Gaussian kernel functions (4) and (5), the bilateral filter function (1) can be reformulated as
Figure imgf000009_0001
using the weight function
Figure imgf000009_0002
According to a preferred embodiment of the invention the smoothing filter is configured for iteratively filtering the one-dimensional reference sample sequence in order to obtain the one-dimensional filtered reference sample se- quence comprising the filtered reference samples. Iteratively filtering means, that a result of a filtering step is used as an intermediate result which is used as an input for a further filtering step, wherein a arbitrary number of further filtering steps may be executed before the result of the last filtering step is outputted by the filter.
For iterative filtering, the bilateral filter function (6) can be reformulated as
Figure imgf000009_0003
with weight function
Figure imgf000009_0004
using iteration index
Figure imgf000009_0005
number of iterations
Figure imgf000009_0006
and
Figure imgf000010_0002
Consequently, the final filtered value is
Figure imgf000010_0003
A reference sample sequence can be understood as a one-dimensional array of values
Figure imgf000010_0006
such that the bilateral filter function (8) can be reformulated as
Figure imgf000010_0004
with weight function
Figure imgf000010_0005
and parameters
Figure imgf000010_0001
In the following, an example is given: Using the parameters
Figure imgf000011_0001
and the filter function is
Figure imgf000011_0002
According to a preferred embodiment of the invention the smoothing filter is configured for outputting the one-dimensiona! filtered reference sample sequence as a sequence of integer numbers. These features reduce the computational complexity.
According to a preferred embodiment of the invention the determiner comprises a shifter configured for shifting the filtered reference samples in order to obtain shifted filtered reference samples, wherein one of the filtered reference samples is shifted to a position of one of the reference samples in the two-dimensional coordinate system which corresponds to the one of the filtered reference samples in order to obtain one of the shifted filtered reference samples. Thus, each of the shifted filtered reference samples has two- dimensional coordinates in the two-dimensional coordinate system which are the same as the two-dimensional coordinates of the reference sample which corresponds to the one of the filtered reference samples. This may be achieved by copying the filtered reference samples from a line buffer to a memory array which corresponds to a memory array in which the original (unfiltered) reference samples previously has been stored.
According to a preferred embodiment of the invention the determiner comprises a prediction finder configured for determining the prediction value for the predicted image sample of the currently processed block by using the shifted filtered reference sample sequence. The prediction finder is that part of the determiner which predicts the prediction values for the predicted block based on the filtered reference samples.
According to a preferred embodiment of the invention the predictor comprises a lookup table configured for receiving from the smoothing filter a spatial difference between a currently processed filtered reference sample of the one-dimensional filtered reference sample sequence and a reference sample of the one- dimensional reference sample sequence, configured for receiving from the smoothing filter an intensity difference between the currently processed filtered reference sample and the reference sample, and configured for transmitting to the smoothing filter a weight for calculating an intensity value for the currently processed filtered reference sample, wherein the weight is selected from a set of weights stored at the look-up table depending on the received spatial difference and the received intensity difference, and wherein the set of weights is pre-calculated based on a weight function of the bilateral filter function.
Introducing a lookup table for the filter weights helps solving the following two implementation-related issues:
• Calculation of filtered sample values In(x) according to equation (10) requires floating point arithmetic. • Calculation of filter weights
Figure imgf000013_0006
) according to equation (11) requires using an exponential function and tends to be computationally complex.
The filter weights
Figure imgf000013_0005
only depend on spatial difference
Figure imgf000013_0004
and intensity difference
Figure imgf000013_0001
with bit depth b.
According to a preferred embodiment of the invention the weights are stored as integer numbers.
Given the values of
Figure imgf000013_0007
and
Figure imgf000013_0008
the weights can be pre-calculated as integers and stored in a lookup table as follows:
Figure imgf000013_0003
where is the integer precision factor and
Figure imgf000013_0009
is the round to nearest integer function.
Given such a table of pre-calculated integer weights, the bilateral filter function (10) can be reformulated as
Figure imgf000013_0002
where both Wmt and / are integer values in order to avoid floating point arithmetic and Wnt may be looked up in the look up table in order to avoid complex calculation. According to a preferred embodiment of the invention the predictor comprises a lookup table arrangement comprising a first lookup table, a second lookup table and a multiplier, wherein the first lookup table is configured for receiving from the smoothing filter a spatial difference between a currently processed filtered reference sample of the one-dimensional filtered reference sample sequence and one reference sample of the one-dimensional reference sample sequence, wherein the second lookup table is configured for receiving from the smoothing filter an intensity difference between the currently processed filtered reference sample and the one reference sample, wherein the lookup table arrangement is configured for transmitting to the smoothing filter a weight for calculating an intensity value for the currently processed filtered reference sample, wherein the multiplier is configured for calculating the weight by multiplying a first weight factor and a second weight factor, wherein the first weight factor is selected from a set of first weight factors stored at the first look-up table depending on the received spatial difference, wherein the set of first weight factors is pre-caiculated based on a spatial weight function of the bilateral filter function, and wherein the second weight factor is selected from a set of second weight factors stored at the second lookup table depending on the received intensity difference, wherein the set of weights is pre-calculated based on an intensity weight function of the bilateral filter function.
Instead of using one two-dimensional lookup table as described above, two one-dimensional lookup tables may be used in order to determine the filter weights wn(x). The set of first weight factors wni(xi) stored at the first iook-up table may be pre-calculated using a spatial weight function of the bilateral filter function
Figure imgf000015_0002
which is the spatial term of the weight function (1 1 ).
The set of second weight factors
Figure imgf000015_0005
stored at the second look-up table may be pre-calculated using an intensity weight function of the bilateral filter function
Figure imgf000015_0001
which is the intensity term of the weight function (1 1 ).
The filter weights
Figure imgf000015_0004
may then be calculated by
Figure imgf000015_0003
In a further aspect the invention provides a video encoder for encoding a picture into a data stream, wherein the video encoder comprises a predictor according to the invention.
According to a preferred embodiment of the invention the video encoder comprises a residual information determiner configured for comparing the predicted image samples of the predicted block with original image samples of the currently processed block of the picture in order to determine a residual information, a transformer configured for quantizing the residual information in order to obtain a transform coefficient block, a data stream inserter configured for inserting the transform coefficient block and intra prediction parameters received from the predictor into the data stream, an inverse transformer configured for dequantizing the transform coefficient block in order to obtain a reconstructed residual information and a reconstructor configured for reconstructing subsequent reference samples for intra predicting a subsequent predicted block for a subsequently processed block of the picture based on the predicted block and based on the reconstructed residual information.
In a further aspect the invention provides a video decoder for decoding a data stream in order to reconstruct a picture, wherein the video decoder comprises a predictor according to the invention.
According to a preferred embodiment of the invention the video decoder comprises an extractor configured for extracting a transform coefficient block for the currently processed block of the picture and intra prediction parameters for the currently processed block of the picture from the data stream and configured to send the intra prediction parameters to the predictor, an inverse transformer configured for invers transforming the transform coefficient block in order to obtain a reconstructed residual information and a reconstructor configured for producing the reconstructed picture based on the predicted block and based on the reconstructed residual information and configured for reconstructing further reference samples for intra predicting a following predicted block based on the predicted block and based on the reconstructed residual information.
In a further aspect the invention provides a method for intra predicting a predicted block for a currently processed block of a picture from reference samples in a two-dimensional coordinate system which are derived from one or more previously processed blocks of the picture, the method comprising the steps: arranging, by using an arranger, the reference samples as a one-dimensional reference sample sequence in a one-dimensional coordinate system; filtering, by using a smoothing filter, the one-dimensional reference sample sequence based on a bilateral filter function with a one-dimensional window in order to obtain a one-dimensional filtered reference sample sequence comprising filtered reference samples; and determining, by using a determiner, for each of a plurality of predicted image samples of the predicted block a respective prediction value by using the filtered reference sample sequence in order to produce the predicted block.
In a further aspect the invention provides a computer program for, when run- ning on a processor, executing the method according to the invention.
Preferred embodiments of the invention are subsequently discussed with respect to the accompanying drawings, in which: Fig. 1 illustrates an embodiment of a predictor according to the invention in a schematic view;
Fig. 2 illustrates an example of a currently processed block, a plurality of previously processed blocks and reference samples given in a two-dimensional coordinate system which are derived from the previous processed blocks in a schematic view;
Fig. 3 illustrates an example of a one-dimensional reference sample sequence given in a one-dimensional coordinate system derived from the reference samples illustrated in Fig. 2 in a schematic view;
Fig. 4 illustrates an example of a predicted block comprising predicted image samples, wherein each image sample comprises a prediction value, in a schematic view;
Fig. 5 illustrates an embodiment of a determiner for a predictor
cording to the invention in a schematic view; Fig. 6 illustrates an example of a one-dimensional filtered reference sample sequence derived from the reference sample sequence illustrated in Fig. 3 in a schematic view;
Fig. 7 illustrates an example of shifted filtered reference samples derived from the filtered reference sample sequence illustrated in Fig. 6 in a schematic view; Fig. 8 illustrates an embodiment of a predictor according to the invention in a schematic partial view;
Fig. 9 illustrates an embodiment of a video encoder according to the invention in a schematic view;
Fig. 10 illustrates an embodiment of a video decoder according to the invention in a schematic view; and
Fig. 11 illustrates a further embodiment of a predictor according to the invention in a schematic partial view.
Equal or equivalent elements or elements with equal or equivalent functionality are denoted in the following description by equal or equivalent reference numerals.
In the following description, a plurality of details is set forth to provide a more thorough explanation of embodiments of the present invention. However, it will be apparent to one skilled in the art that embodiments of the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form rather than in detail in order to avoid obscuring embodiments of the present invention. In addition, features of the different embodiments described hereinafter may be combined with each other, unless specifically noted otherwise.
Fig. 1 illustrates an embodiment of a predictor 1 according to the invention in a schematic view. The inventive predictor 1 is configured for intra predicting a predicted block PB for a currently processed block CPB of a picture PIC from reference samples RS in a two-dimensional coordinate system which are derived from previously processed blocks PPB of the picture PIC. The predictor 1 comprises an arranger 2 configured for arranging the reference samples RS as a one- dimensional reference sample sequence RSS in a one-dimensional coordinate system; a smoothing filter 3 configured for filtering the one-dimensional reference sample sequence RRS based on a bilateral filter function with a one- dimensional window Wl in order to obtain a one-dimensional filtered refer- ence sample sequence FRSS comprising filtered reference samples FRS; and a determiner 4 configured for determining for each of a plurality of predicted image samples PIS of the predicted block PB a respective prediction value PV by using the filtered reference sample sequence FRSS in order to produce the predicted block PB.
According to preferred embodiment of the invention a range kernel function of the bilateral filter function is a Gaussian range kernel function, and wherein a spatial kernel function of the bilateral filter function is a Gaussian spatial kernel function.
According to a preferred embodiment of the invention the smoothing filter 3 is configured for iteratively filtering the one-dimensional reference sample se- quence RSS in order to obtain the one-dimensional filtered reference sample sequence FRSS comprising the filtered reference samples FRS.
According to a preferred embodiment of the invention the smoothing filter 3 is configured for outputting the one-dimensional filtered reference sample se- quence FRSS as a sequence of integer numbers.
In a further aspect the invention provides a method for intra predicting a predicted block PB for a currently processed block CPB of a picture PIC from reference samples RS in a two-dimensional coordinate system which are derived from one or more previously processed blocks PPB of the picture PIC, the method comprising the steps: arranging, by using an arranger 2, the reference samples RS as a one- dimensional reference sample sequence RSS in a one-dimensional coord i- nate system; filtering, by using a smoothing filter 3, the one-dimensional reference sample sequence RRS based on a bilateral filter function with a one-dimensional window Wl in order to obtain a one-dimensional filtered reference sample sequence FRSS comprising filtered reference samples FRS; and determining, by using a determiner 4, for each of a plurality of predicted image samples PIS of the predicted block PB a respective prediction value PV by using the filtered reference sample sequence FRSS in order to produce the predicted block PB.
In a further aspect the invention provides a computer, program for, when running on a processor, executing the inventive method.
The predictor 1 may be employed in a hybrid video decoder or in a hybrid video decoder. An advantage of using a smoothing filter 3 based on data bilateral filter function for filtering the reference samples RS is that outliers within the reference samples RS do not negatively affect the intra prediction result, whereas edges are preserved. The computational overhead may be kept reasonable.
Fig. 2 illustrates an example of a currently processed block CBP, a plurality of previously processed blocks PPB and reference samples RS given in a two-dimensional coordinate system, which are derived from the previous processed blocks PPB, in a schematic view. The currently processed block CBP has a width W and a height H.
According to a preferred embodiment of the invention at least a part of the reference samples RS in the two-dimensional coordinate system sequentially extend along two rays R1 , R2 starting at a common intersection point IP and pointing into different directions, wherein the arranger 2 is configured for arranging the reference samples RS into a one-dimensional reference sample sequence RSS by anti-parallelly traveling along a first one R1 of the two rays R1 , R2 and parallelly traveling along a second one R2 of the two rays R1 , R2. According to a preferred embodiment of the invention the reference samples RS in the two-dimensional coordinate system comprise first edge reference samples ERS1 bordering a first side of the currently processed block CPB. According to a preferred embodiment of the invention the reference samples RS in the two-dimensional coordinate system comprise second edge reference samples ERS2 bordering a second side of the currently processed block CPB. In the example of Fig. 2, first edge reference samples ERS1 are bordering the left side of the currently processed block CBP and second edge reference samples ERS2 are bordering the upper side of the currently processed block CBP. In Fig. 2 the reference samples RS are given in a two- dimensional coordinate system. Thus, each of the reference samples RS has two-dimensional coordinates. Fig. 2 illustrates the references samples RS, before they are processed by the arranger 2.
The bilateral filtering may be applied to all blocks which are intra predicted. In particular the bilateral filtering may be applied to blocks containing luma in- formation (e.g. blocks of a luma channel). However, it's also possible to apply the bilateral filtering to blocks containing chroma information (e.g. blocks of a chroma channel). In particular, bilaterai filtering may be applied to all blocks having a width and/or a height being equal or greater than 16 samples. Depending on the block size, following parameters may be used:
Figure imgf000021_0001
Fig. 3 illustrates an example of a one-dimensional reference sample se- quence RSS given in a one-dimensional coordinate system derived from the reference samples RS illustrated in Fig. 2 in a schematic view. The reference sample sequence RSS comprises 2(W+H) + 1 reference samples RS so that all angular prediction modes may be performed.
In Fig. 3 the reference samples RS are given in a one-dimensional coordinate system. Thus, each of the reference samples RS has one-dimensional coordinates. Fig. 3 illustrates the references samples RS, after they are processed by the arranger 2.
In order to calculate a filtered reference sample FRS for the position ka one- dimensional window Wl depending on the filter size s is used in that way, that all reference samples RS having a position between k-s\o k+s are considered. In particular, the filtered reference sample FRS for the position kmay be calculated by using equation (10) using the window Wl.
Fig. 4 illustrates an example of a predicted block PTB comprising predicted image samples PIS, wherein each image sample PIS comprises a prediction value PV, in a schematic view. The predicted block PB consists of predicted image samples PIS, wherein each of the predicted image samples PIS has a position indicated by two indices and the predicted value having the same two indices.
Fig. 5 illustrates an embodiment of a determiner 4 for a predictor 1 according to the invention in a schematic view.
According to a preferred embodiment of the invention the determiner 4 comprises a shifter 5 configured for shifting the filtered reference samples FRS in order to obtain shifted filtered reference samples SFRS, wherein one of the filtered reference samples FRS is shifted to a position of one of the reference samples RS in the two-dimensional coordinate system which corresponds to the one of the filtered reference samples FRS in order to obtain one of the shifted filtered reference samples SFRS.
According to a preferred embodiment of the invention the determiner 4 comprises a prediction finder 6 configured for determining the prediction value PV for the predicted image sample PIS of the currently processed block CPB by using the shifted filtered reference sample sequence SFRS. Fig. 6 illustrates an example of a one-dimensional filtered reference sample sequence FRSS derived from the reference sample sequence RSS illustrated in Fig. 3 in a schematic view. The one-dimensional filtered reference sample sequence FRSS may be inputted into a shifter 5.
Fig. 7 illustrates an example of shifted filtered reference samples SFRS derived from the filtered reference sample sequence FRSS illustrated in Fig. 6 in a schematic view. The shifted filtered reference samples SFRS maybe outputted by the shifter 5.
Fig. 8 illustrates an embodiment of a predictor 1 according to the invention in a schematic partial view.
According to a preferred embodiment of the invention the predictor 1 comprises a lookup table 7 configured for receiving from the smoothing filter 3 a spatial difference SPD between a currently processed filtered reference sample FRS of the one- dimensional filtered reference sample sequence FRSS and one reference sample RS of the one-dimensional reference sample sequence RSS, configured for receiving from the smoothing filter 3 an intensity difference ITD between the currently processed filtered reference sample FRS and the one reference sample RS, and configured for transmitting to the smoothing filter 3 a weight WE for calculating an intensity value for the currently processed filtered reference sample FRS, wherein the weight WE is selected from a set of weights stored at the look-up table depending on the received spatial difference SPD and the received intensity difference ITD, and wherein the set of weights is p recalculated based on a weight function of the bilateral filter function.
According to a preferred embodiment of the invention the weights WE are stored as integer numbers. Fig. 9 illustrates an embodiment of a video encoder 8 according to the invention in a schematic view.
In a further aspect the invention provides encoder for encoding a picture PIC into a data stream DS, the video encoder 8 comprising a predictor 1 according to the invention.
According to a preferred embodiment of the invention the video encoder 8 comprises a residual information determiner 9 configured for comparing the predicted image samples PIS of the predicted block PB with original image samples OIS of the currently processed block CBP of the picture PIC in order to determine a residual information RIN, a transformer 10 configured for quantizing the residual information RIN in order to obtain a transform coefficient block TCB, a data stream inserter 11 configured for inserting the transform coefficient block TCB and intra prediction parameters IPP received from the predictor 1 into the data stream DS, an inverse transformer 12 configured for dequantizing the transform coefficient block TCB in order to obtain a reconstructed residual information RRIN and a reconstructor 13 configured for reconstructing subsequent reference samples SURS for intra predicting a subsequent predicted block for a subsequently processed block of the picture based on the predicted block PB and based on the reconstructed residual information RRIN. Furthermore, the video encoder 8 may comprise a block determiner 14 configured for splitting up the picture PIC into a plurality of blocks. The blocks may have a rectangular or square shape, wherein different clocks may have different sizes.
As mentioned above, a predicted block PB of a picture PIC may be predicted using intra prediction. The predictor 1 of the video encoder 8 may be configured to determine a predicted block PB for a currently processed block CBP which has to be coded of the current picture PIC of a video. The predictor 1 determines the predicted block PB for the currently processed block CBP based on reference samples RS, which have been encoded before the currently processed block CBR. The reference samples RS are bordering the currently processed block CBR of the same picture PIC like the currently processed block CBR. In the predictor 1 the reference samples RS are filtered using the smoothing filter 3, which uses the bilateral filter function, to sup- press outliers within the reference samples RS, such that these outliers do not negatively affect (or only insignificantly negatively affect) the predicted block PB for the currently processed block CPB and such that sharp edges are preserved.
The residual information determiner 9 is configured to compare the predicted block PB for the currently processed block CPB with the (original) currently processed block CPB in order to determine a residual information RIN for the currently processed block CPB. The residual information RIN determines the difference of the (original) currently processed block CPB and its predicted block PB. By using the smoothing filter for to filter the reference samples RS the residual information RIN can be kept smaller, than without using the smoothing filter 3. The residual information RIN can, therefore, be represented by fewer bits. The transformer 10 may transform the residual information RIN and quantize the resulting transform coefficients to obtain a transform coefficient block TCB. A quality loss (if any) in the video encoder 8 occurs in the quantization of the transform coefficients of the transform coefficient block TCB. The transform coefficient block TCB may then together with the intra prediction parameter IPP be inserted into a data stream DS by the data stream inserter 1 1. The data stream DS may be transmitted to a video decoder in which the currently processed block CBP may be reconstructed. The transform coefficient block TCB comprising the quantized transform coefficients representing the residual information RIN may be inverse transformed by the inverse transformer 12 to obtain reconstructed residual information RRIN for the currently processed block CPB. The reconstructed residual information RRIN may show a loss compared to the residual information RIN because of the quantization of the transform coefficients in the transformer 10. The reconstructed residual information RRIN may be used by the recon- structor 13 in conjunction with the predicted block PB for currently processed block CBP to obtain reference samples RS from the currently processed block CPB, which then may be used as reference samples RS for a subsequently processed block.
As it is essential for correctly decoding the data stream DS that intra predicting of the predicted blocks PB at the video decoder 15 is done in the same way as intra predicting of the predicted blocks PB at the video encoder 8, predictor 1 of the video encoder 8 may be configured to include into the intra- prediction parameters IPP an information, whether bilateral filtering has been used or have not been used for encoding the currently processed block CBP. For the same reason, predictor 1 of the video encoder 8 may be configured to include into the intra-prediction parameters IPP parameters of the bilateral filter function. Also, predictor 1 of the video encoder 8 may be configured to include into the intra-prediction parameters IPP information regarding the prediction mode.
Fig. 10 illustrates an embodiment of a video decoder 15 according to the invention in a schematic view.
In a further aspect the invention provides a video decoder 15 for decoding a data stream DS in order to reconstruct a picture PIC, wherein the video decoder 15 comprises a predictor 1 according to the invention.
In a preferred embodiment of the invention the video decoder 15 comprises an extractor 16 configured for extracting a transform coefficient block TCB for the currently processed block CPB of the picture PIC and intra prediction parameters IPP for the currently processed block CPB of the picture PIC from the data stream DS and configured to send the intra prediction parameters IPP to the predictor 1 , an inverse transformer 12 configured for in vers transforming the transform coefficient block TCB in order to obtain a reconstructed residual information RRIN and a reconstructor 13 configured for producing the currently processed block CPB based on the predicted block PB and based on the reconstructed residual information RRIN and configured for producing subsequent reference samples SURS for intra predicting a subsequent predicted block for a subsequently processed block of the picture PIC based on the predicted block PB and based on the reconstructed residual information RRIN.
Additionally, the video decoder 15 may comprise a picture determiner configured for reconstruction the picture PIC by adding the currently processed block CBP to previously processed blocks PPB. The video decoder 15 may be configured to decode the data stream DS, which may have been produced with the video encoder 8 according to Fig. 9. In Fig. 10 same reference numerals as in Fig. 9 are used, in order to denote elements in the video decoder 15 which are equal or similar elements of the video encoder 8. The extractor 16 may be configured to extract the transform coefficient block TCB for the currently processed block CBP and the intra prediction parameter IPP for the currently processed block CBP from the data stream DS (which, for example, is provided by the video encoder 8). The inverse transformer 12 obtains (like the inverse transformer 12 of the video encoder 8) the reconstructed residual information RRIN for the currently processed block CPB. The predictor 1 is configured to determine the predicted block PB for the currently processed block CPB by using the reference samples RS (which, for example, are derived from a previously processed block PPB). The reference samples RS are filtered by using the (bilateral) smooth- ing filter 3 of the predictor 1 in order to obtain the filtered reference samples FRS. A determiner 4 of the predictor 1 may then obtain the predicted block PB for the currently processed block CPB based on the intra prediction parameter IPP for the currently processed block CPB and the filtered reference samples FRS. The determiner for may determine the predicted block PB for the currently processed block CPB on the same way like the determiner for of the video encoder 8. The determiner for may therefore be a reduced version of the determiner for of the video encoder 8, as the determiner for of the video encoder 8 is, as mentioned before, further configured to obtain the intra prediction parameter IPP for the currently processed block .
The predicted block PB for the currently processed block CPB may then together with the reconstructed residual information RRIN for the currently processed block CPB be combined by the reconstructor 13 to obtain a reconstructed version of the currently processed block CPB and reconstructed ref- erence samples RS of the currently processed block CPB. This reconstructed reference samples RS of the currently processed block CPB, may be used again as reference samples RS for subsequently processed blocks. Furthermore, the reconstructed currently processed block CBP may be sent to the picture determiner 17. Fig. 1 1 illustrates a further embodiment of a predictor 1 according to the invention in a schematic partial view.
According to a preferred embodiment of the invention the predictor 1 com- prises a lookup table arrangement 18 comprising a first lookup table 19, a second lookup table 20 and a multiplier 21 , wherein the first lookup table 19 is configured for receiving from the smoothing filter 3 a spatial difference SPD between a currently processed filtered reference sample FRS of the one-dimensional filtered reference sample sequence FRSS and one reference sample RS of the one-dimensional reference sample sequence RSS, wherein the second lookup table 20 is configured for receiving from the smoothing filter 3 an intensity difference ITD between the currently processed filtered reference sample FRS and the one reference sample RS, wherein the lookup table arrangement 18 is configured for transmitting to the smoothing filter 3 a weight WE for calculating an intensity value for the cur- rently processed filtered reference sample FRS, wherein the multiplier 21 is configured for calculating the weight WE by multiplying a first weight factor FWF and a second weight factor SWF, wherein the first weight factor FWF is selected from a set of first weight factors stored at the first look-up table 19 depending on the received spatial difference SPD, wherein the set of first weight factors is pre-calculated based on a spatial weight function of the bilateral filter function, and wherein the second weight factor SWF is selected from a set of second weight factors stored at the second lookup table 20 depending on the received intensity difference ITD, wherein the set of weights is pre-calculated based on an intensity weight function of the bilateral filter function. As it is essential for correctly decoding the data stream DS that intra predicting of the predicted blocks PB at the video decoder 15 is done in the same way as intra predicting of the predicted blocks PB at the video encoder 8, predictor 1 of the video decoder 15 may be configured to extract from the intra-prediction parameters IPP an information, whether bilateral filtering has been used or have not been used for encoding the currently processed block CBP. For the same reason, predictor 1 of the video decoder 15 may be configured to extract from the intra-prediction parameters IPP parameters of the bilateral filter function. Also, predictor 1 of the video decoder 15 may be configured to extract from the intra-prediction parameters IPP information regarding the prediction mode.
Depending on certain implementation requirements, embodiments of the inventive device and system can be implemented in hardware and/or in software. The implementation can be performed using a digital storage medium, for example a floppy disk, a DVD, a Blu-ray Disc, a CD, a ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that one or more or all of the functionalities of the inventive device or system is performed.
In some embodiments, a programmable logic device (for example a field programmable gate array) may be used to perform one or more or all of the functionalities of the devices and systems described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one or more or all of the functionalities of the devices and systems described herein.
Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus.
Depending on certain implementation requirements, embodiments of the inventive method can be implemented using an apparatus comprising hardware and/or software. The implementation can be performed using a digital storage medium, for example a floppy disk, a DVD, a Blu-ray Disc, a CD, a ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed.
Depending on certain implementation requirements, embodiments of the inventive method can be implemented using an apparatus comprising hardware and/or software.
Some or all of the method steps may be executed by (or using) a hardware apparatus, like a microprocessor, a programmable computer or an electronic circuit. Some one or more of the most important method steps may be executed by such an apparatus.
Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system such that one of the methods described herein is performed.
Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may for example be stored on a machine readable carrier.
Other embodiments comprise the computer program for performing one of the methods described herein, which is stored on a machine readable carrier or a non-transitory storage medium.
A further embodiment comprises a processing means, for example a computer, or a programmable logic device, in particular a processor comprising hardware, configured or adapted to perform one of the methods described herein. A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.
Generally, the methods are advantageously performed by any apparatus comprising hardware and or software.
While this invention has been described in terms of several embodiments, there are alterations, permutations, and equivalents which fall within the scope of this invention. It should also be noted that there are many alternative ways of implementing the methods and compositions of the present invention. It is therefore intended that the following appended claims be interpreted as including all such alterations, permutations and equivalents as fall within the true spirit and scope of the present invention.
Figure imgf000031_0001
Figure imgf000032_0001
References:
Thomas Wiegand, Gary J. Sullivan, Gisle Bjontegaard, and Ajay Luthra: Overview of the H.264/AVC Video Coding Standard, IEEE Transactions on Circuits and Systems for Video Technology, Vol. 13, No. 7, pp. 560-576, July 2003.
Figure imgf000033_0001
[3] Gary J. Sullivan, Jens-Rainer Ohm, Woo-Jin Han, Thomas Wiegand:
Overview of the High Efficiency Video Coding (HEVC) Standard, IEEE
Transactions on Circuits and Systems for Video Technology, Vol. 22, No. 12, pp. 1649-1668, November 2012.

Claims

Claims
1. Predictor for intra predicting a predicted block (PB) for a currently processed block (CPB) of a picture (PIC) from reference samples (RS) in a two-dimensional coordinate system which are derived from one or more previously processed blocks (PPB) of the picture (PIC), the predictor (1) comprising: an arranger (2) configured for arranging the reference samples (RS) as a one-dimensional reference sample sequence (RSS) in a one-dimensional coordinate system; a smoothing filter (3) configured for filtering the one-dimensional reference sample sequence (RRS) based on a bilateral filter function with a one-dimensional window (Wl) in order to obtain a one-dimensional filtered reference sample sequence (FRSS) comprising filtered reference samples (FRS); and a determiner (4) configured for determining for each of a plurality of predicted image samples (PIS) of the predicted block (PB) a respective prediction value (PV) by using the filtered reference sample sequence (FRSS) in order to produce the predicted block (PB).
2. Predictor according to the preceding claim, wherein at least a part of the reference samples (RS) in the two-dimensional coordinate system sequentially extend along two rays (R1 , R2) starting at a common intersection point (IP) and pointing into different directions, wherein the arranger (2) is configured for arranging the reference samples (RS) into a one- dimensional reference sample sequence (RSS) by anti-parallelly traveling along a first one (R1) of the two rays (R1 , R2) and parallelly traveling along a second one (R2) of the two rays (R1 , R2).
3. Predictor according to the preceding claim, wherein the reference samples (RS) in the two-dimensional coordinate system comprise first edge reference samples (ERS1 ) bordering on a first side of the currently processed block (CPB).
4. Predictor according to the preceding claim, wherein the reference samples (RS) in the two-dimensional coordinate system comprise second edge reference samples (ERS2) bordering on a second side of the currently processed block (CPB).
5. Predictor according to one of the preceding claims, wherein a range kernel function of the bilateral filter function is a Gaussian range kernel function, and wherein a spatial kernel function of the bilateral filter function is a Gaussian spatial kernel function.
6. Predictor according to one of the preceding claims, wherein the smoothing filter (3) is configured for iteratively filtering the one-dimensional reference sample sequence (RSS) in order to obtain the one-dimensional filtered reference sample sequence (FRSS) comprising the filtered reference samples (FRS).
7. Predictor according to one of the preceding claims, wherein the smoothing filter (3) is configured for outputting the one-dimensional filtered reference sample sequence (FRSS) as a sequence of integer numbers.
8. Predictor according to one of the preceding claims, wherein the determiner (4) comprises a shifter (5) configured for shifting the filtered reference samples (FRS) in order to obtain shifted filtered reference samples (SFRS), wherein one of the filtered reference samples (FRS) is shifted to a position of one of the reference samples (RS) in the two-dimensional coordinate system which corresponds to the one of the filtered reference samples (FRS) in order to obtain one of the shifted filtered reference samples (SFRS).
9. Predictor according to the preceding claim, wherein the determiner (4) comprises a prediction finder (6) configured for determining the prediction value (PV) for the predicted image sample (PIS) of the currently processed block (CPB) by using the shifted filtered reference sample sequence (SFRS).
10. Predictor according to one of the preceding claims, wherein the predictor (1) comprises a lookup table (7) configured for receiving from the smoothing filter (3) a spatial difference (SPD) between a currently processed filtered reference sample (FRS) of the one-dimensional filtered reference sample sequence (FRSS) and one reference sample (RS) of the one-dimensional reference sample sequence (RSS), configured for receiving from the smoothing filter (3) an intensity difference (ITD) between the currently processed filtered reference sample (FRS) and the one reference sample (RS), and configured for transmitting to the smoothing filter (3) a weight (WE) for calculating an intensity value for the currently processed filtered reference sample (FRS), wherein the weight (WE) is selected from a set of weights stored at the lookup table (7) depending on the received spatial difference (SPD) and the received intensity difference (ITD), and wherein the set of weights is pre-calculated based on a weight function of the bilateral filter function.
11. Predictor according to the preceding claim, wherein the weights (WE) are stored as integer numbers.
12. Predictor according to one of the claims 1 to 9, wherein the predictor (1) comprises a lookup table arrangement (18) comprising a first lookup table (19), a second lookup table (20) and a multiplier (21), wherein the first lookup table (19) is configured for receiving from the smoothing filter (3) a spatial difference (SPD) between a currently processed filtered reference sample (FRS) of the one-dimensional filtered reference sample sequence (FRSS) and one reference sample (RS) of the one-dimensional reference sample sequence (RSS), wherein the second lookup table (20) is configured for receiving from the smoothing filter (3) an intensity difference (ITD) between the currently processed filtered reference sample (FRS) and the one reference sample
(RS), wherein the lookup table arrangement (18) is configured for transmitting to the smoothing filter (3) a weight (WE) for calculating an intensity value for the currently processed filtered reference sample (FRS), wherein the multiplier (21) is configured for calculating the weight (WE) by multiplying a first weight factor (FWF) and a second weight factor (SWF), wherein the first weight factor (FWF) is selected from a set of first weight factors stored at the first look-up table (19) depending on the received spatial difference (SPD), wherein the set of first weight factors is pre- calculated based on a spatial weight function of the bilateral filter function, and wherein the second weight factor (SWF) is selected from a set of second weight factors stored at the second lookup table (20) depending on the received intensity difference (ITD), wherein the set of weights is pre- calculated based on an intensity weight function of the bilateral filter function.
13. Video encoder for encoding a picture (PIC) into a data stream (DS), the video encoder (8) comprising a predictor (1) according to one of the claims 1 to 12.
14. Video encoder according to the preceding claim, wherein the video encoder (8) comprises a residual information determiner (9) configured for comparing the predicted image samples (PIS) of the predicted block (PB) with original image samples (OIS) of the currently processed block (CBP) of the picture (PIC) in order to determine a residual information (RIN), a transformer (10) configured for quantizing the residual information (RIN) in order to obtain a transform coefficient block (TCB), a data stream inserter (11) configured for inserting the transform coefficient block (TCB) and intra prediction parameters (IPP) received from the predictor (1) into the data stream (DS), an inverse transformer (12) configured for dequan- tizing the transform coefficient block (TCB) in order to obtain a reconstructed residual information (RRIN) and a reconstructor (13) configured for reconstructing subsequent reference samples (SURS) for intra predicting a subsequent predicted block for a subsequently processed block of the picture based on the predicted block (PB) and based on the reconstructed residual information (RRIN).
15. Video decoder for decoding a data stream (DS) in order to reconstruct a picture (PIC), the video decoder (15) comprising a predictor (1) according to one of the claims 1 to 12.
16. Video decoder according to the preceding claim, wherein the video decoder (15) comprises an extractor (16) configured for extracting a transform coefficient block (TCB) for the currently processed block (CPB) of the picture (PIC) and intra prediction parameters (IPP) for the currently processed block (CPB) of the picture (PIC) from the data stream (DS) and configured to send the intra prediction parameters (IPP) to the predictor (1), an inverse transformer (12) configured for invers transforming the transform coefficient block (TCB) in order to obtain a reconstructed residual information (RRIN) and a reconstructor (13) configured for producing the currently processed block (CPB) based on the predicted block (PB) and based on the reconstructed residual information (RRIN) and configured for producing subsequent reference samples (SURS) for intra predicting a subsequent predicted block for a subsequently processed block of the picture (PIC) based on the predicted block (PB) and based on the reconstructed residual information (RRIN).
17. Method for intra predicting a predicted block (PB) for a currently processed block (CPB) of a picture (PIC) from reference samples (RS) in a two-dimensional coordinate system which are derived from one or more previously processed blocks (PPB) of the picture (PIC), the method comprising the steps: arranging, by using an arranger (2), the reference samples (RS) as a one- dimensional reference sample sequence (RSS) in a one-dimensional coordinate system; filtering, by using a smoothing filter (3), the one-dimensional reference sample sequence (RRS) based on a bilateral filter function with a one- dimensional window (Wl) in order to obtain a one-dimensional filtered reference sample sequence (FRSS) comprising filtered reference samples (FRS); and determining, by using a determiner (4), for each of a plurality of predicted image samples (PIS) of the predicted block (PB) a respective prediction value (PV) by using the filtered reference sample sequence (FRSS) in order to produce the predicted block (PB).
18. Computer program for, when running on a processor, executing the
method according to the preceding claim.
PCT/EP2018/077604 2017-10-11 2018-10-10 Onedimensional bilateral filtering of intra prediction reference pixels WO2019072918A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP18782086.5A EP3695598A1 (en) 2017-10-11 2018-10-10 Onedimensional bilateral filtering of intra prediction reference pixels

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
EP17195973 2017-10-11
EP17195973.7 2017-10-11

Publications (1)

Publication Number Publication Date
WO2019072918A1 true WO2019072918A1 (en) 2019-04-18

Family

ID=60083141

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2018/077604 WO2019072918A1 (en) 2017-10-11 2018-10-10 Onedimensional bilateral filtering of intra prediction reference pixels

Country Status (2)

Country Link
EP (1) EP3695598A1 (en)
WO (1) WO2019072918A1 (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011127964A2 (en) 2010-04-13 2011-10-20 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Apparatus for intra predicting a block, apparatus for reconstructing a block of a picture, apparatus for reconstructing a block of a picture by intra prediction
US20130003832A1 (en) * 2011-06-30 2013-01-03 Futurewei Technologies, Inc. Simplified Bilateral Intra Smoothing Filter

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011127964A2 (en) 2010-04-13 2011-10-20 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Apparatus for intra predicting a block, apparatus for reconstructing a block of a picture, apparatus for reconstructing a block of a picture by intra prediction
US20130003832A1 (en) * 2011-06-30 2013-01-03 Futurewei Technologies, Inc. Simplified Bilateral Intra Smoothing Filter

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
ALBRECHT M ET AL: "Description of SDR, HDR and 360° video coding technology proposal by Fraunhofer HHI", 10. JVET MEETING; 10-4-2018 - 20-4-2018; SAN DIEGO; (THE JOINT VIDEO EXPLORATION TEAM OF ISO/IEC JTC1/SC29/WG11 AND ITU-T SG.16 ); URL: HTTP://PHENIX.INT-EVRY.FR/JVET/,, no. JVET-J0014-v4, 12 April 2018 (2018-04-12), XP030151173 *
C. TOMASI ET AL: "Bilateral filtering for gray and color images", SIXTH INTERNATIONAL CONFERENCE ON COMPUTER VISION (IEEE CAT. NO.98CH36271), 1 January 1998 (1998-01-01), pages 839 - 846, XP055166574, DOI: 10.1109/ICCV.1998.710815 *
GARY J. SULLIVAN; JENS-RAINER OHM; WOO-JIN HAN; THOMAS WIEGAND: "Overview of the High Efficiency Video Coding (HEVC) Standard", IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, vol. 22, no. 12, November 2012 (2012-11-01), pages 1649 - 1668, XP011487803, DOI: doi:10.1109/TCSVT.2012.2221191
STRÖM J ET AL: "EE2-JVET-E0032 Bilateral filter Test 1, Test2", 6. JVET MEETING; 31-3-2017 - 7-4-2017; HOBART; (THE JOINT VIDEO EXPLORATION TEAM OF ISO/IEC JTC1/SC29/WG11 AND ITU-T SG.16 ); URL: HTTP://PHENIX.INT-EVRY.FR/JVET/,, no. JVET-F0034, 23 March 2017 (2017-03-23), XP030150687 *
THOMAS WIEGAND; GARY J. SULLIVAN; GISLE BJONTEGAARD; AJAY LUTHRA: "Overview of the H.264/AVC Video Coding Standard", IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, vol. 13, no. 7, July 2003 (2003-07-01), pages 560 - 576, XP008129745, DOI: doi:10.1109/TCSVT.2003.815165

Also Published As

Publication number Publication date
EP3695598A1 (en) 2020-08-19

Similar Documents

Publication Publication Date Title
US9344744B2 (en) Apparatus for intra predicting a block, apparatus for reconstructing a block of a picture, apparatus for reconstructing a block of a picture by intra prediction
RU2683165C1 (en) Intra block copy prediction with asymmetric partitions and encoder-side search patterns, search ranges and approaches to partitioning
WO2017206826A1 (en) Methods and apparatuses of video data processing with conditionally quantization parameter information signaling
JP6847211B2 (en) Distance-weighted bidirectional intra-prediction
EP2536151A2 (en) Method and apparatus for processing a video signal
JP2022130647A (en) Bit width control method and device for bidirectional optical flow
US11102474B2 (en) Devices and methods for intra prediction video coding based on a plurality of reference pixel values
EP3891990A1 (en) Bit-width control for bi-directional optical flow
US20210014484A1 (en) Picture encoding and decoding, picture encoder, and picture decoder
US11962786B2 (en) Multi-stage block coding
WO2019008366A1 (en) Image data encoding and decoding
KR20140124448A (en) Method for encoding and decoding video using intra prediction, and apparatus thereof
CN111279698B (en) Asymmetric division apparatus and method for image coding
CN113068026B (en) Coding prediction method, device and computer storage medium
CN110771166B (en) Intra-frame prediction device and method, encoding device, decoding device, and storage medium
US20220038688A1 (en) Method and Apparatus of Encoding or Decoding Using Reference Samples Determined by Predefined Criteria
WO2019072918A1 (en) Onedimensional bilateral filtering of intra prediction reference pixels
WO2020264221A1 (en) Apparatuses and methods for bit-width control of bi-directional optical flow
RU2817405C1 (en) Method, equipment and devices for encoding and decoding
EP4068776A1 (en) Decoding device and method for predicting block partitioned into random shapes
Kamisli Lossless compression in HEVC with integer-to-integer transforms
KR20140124447A (en) Method for encoding and decoding video using intra prediction, and apparatus thereof
WO2022191947A1 (en) State based dependent quantization and residual coding in video coding
WO2020198543A1 (en) Methods and devices for bit-depth control for bi-directional optical flow
GB2564150A (en) Image data encoding and decoding

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18782086

Country of ref document: EP

Kind code of ref document: A1

DPE1 Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101)
NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2018782086

Country of ref document: EP

Effective date: 20200511