WO2015196333A1 - Segmental prediction for video coding - Google Patents

Segmental prediction for video coding Download PDF

Info

Publication number
WO2015196333A1
WO2015196333A1 PCT/CN2014/080515 CN2014080515W WO2015196333A1 WO 2015196333 A1 WO2015196333 A1 WO 2015196333A1 CN 2014080515 W CN2014080515 W CN 2014080515W WO 2015196333 A1 WO2015196333 A1 WO 2015196333A1
Authority
WO
WIPO (PCT)
Prior art keywords
segment
prediction
segmental
pixel
value
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2014/080515
Other languages
French (fr)
Inventor
Kai Zhang
Jicheng An
Xianguo Zhang
Han HUANG
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
MediaTek Singapore Pte Ltd
Original Assignee
MediaTek Singapore Pte Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by MediaTek Singapore Pte Ltd filed Critical MediaTek Singapore Pte Ltd
Priority to PCT/CN2014/080515 priority Critical patent/WO2015196333A1/en
Priority to US15/032,205 priority patent/US10244258B2/en
Priority to PCT/CN2015/082074 priority patent/WO2015196966A1/en
Priority to CN201580001847.4A priority patent/CN105556968B/en
Publication of WO2015196333A1 publication Critical patent/WO2015196333A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/117—Filters, e.g. for pre-processing or post-processing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H04N19/14—Coding unit complexity, e.g. amount of activity or edge presence estimation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/167—Position within a video image, e.g. region of interest [ROI]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
    • H04N19/86—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving reduction of coding artifacts, e.g. of blockiness
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding

Definitions

  • the invention relates generally to video/image processing.
  • Prediction takes a critical role in video coding.
  • a prediction block which is generated by intra-prediction or inter-prediction, is obtained first before producing residues at encoder or reconstructing the reconstruction samples at decoder.
  • IBC intra-block copy
  • SCC screen content coding
  • HEVC high efficiency video coding
  • Inter simplified depth coding is adopted into 3D-HEVC as a special prediction mode for depth coding.
  • InterSDC Inter simplified depth coding
  • a normal inter- prediction is performed for the current block first.
  • each pixel in the prediction block is added by a coded offset.
  • P Lj represents the prediction value at pixel position ( i, j) after performing the normal inter-prediction
  • Offset is the offset coded for this block.
  • the final prediction value at pixel position is Pi + Offset.
  • InterSDC mode no residues are coded. Thus the final prediction value will be output as the reconstructed value.
  • DLT Depth lookup table
  • SDC intra simplified depth coding
  • DDM depth map modeling
  • Fig.2 demonstrates an example of DLT approach. DLT is signaled in picture parameter set (PPS). And it is left to be an encoder issue how to get the DLT when encoding.
  • the prediction block is processed by a segmental process before it is used to get residues at an encoder or get reconstruction at a decoder.
  • the segmental prediction method comprises classifying pixels in the prediction block into different categories, named as 'segment', wherein the pixels in the segment can be adjacent or not, treating the pixels in different segments in different ways, and forming a new prediction block after the treatment.
  • the new prediction block is used to get the residues at the encoder or get the reconstruction at the decoder.
  • FIG. 1 is a diagram illustrating intra block copying
  • Fig. 2 is a diagram illustrating is a diagram illustrating an example of DLT where five valid values appear in depth samples.
  • Fig. 3 is a diagram illustrating a block with two segments with sharp sample value changes
  • Fig. 4 is a diagram illustrating a block with three segments with sharp sample value changes
  • Fig. 5 is a diagram illustrating an exemplary segmental prediction architecture at the decoder
  • FIG. 6 is a diagram illustrating an exemplary segmental process architecture
  • Fig. 7 is a diagram illustrating an exemplary treatment for a segment. After the treatment, pixels in the segment can hold different values;
  • Fig. 8 is a diagram illustrating an exemplary treatment for a segment. After the treatment, pixels in the segment hold only one value.
  • the prediction block is processed by a segmental process before it is used to get the residues at encoder or get the reconstruction at decoder.
  • the prediction block is processed by a segmental process, and then the modified prediction block is output as the reconstruction without adding to residues.
  • Fig. 5 demonstrates segmental prediction architecture at the decoder.
  • Fig.6 In the segmental process, there are two steps generally as depicted in Fig.6.
  • the first step is called 'classification', in which the pixels in a prediction block are classified into different categories, named as 'segment'. Pixels in a segment can be adjacent or not.
  • the second step is called 'treatment', in which pixels in different segments are treated in different ways.
  • all the pixels after the treatment form a new prediction block which is output then.
  • the number of segments can be any positive integers such as 1, 2, 3, etc.
  • the prediction values can be classified according to their values.
  • the prediction values can be classified according to their positions.
  • the prediction values can be classified according to their gradients. Gradients can be got by applying operators such as Sobel, Roberts, and Prewitt.
  • classification is not applied if the number of segments is one.
  • the prediction block is classified into two segments in the classification step.
  • a pixel is classified according to its relationship with a threshold number T. In one example, a pixel is classified into segment 0 if its value is larger than T, Otherwise it is classified into segment 1. In another example, a pixel is classified into segment 0 if its value is larger than or equal to T, Otherwise it is classified into segment 1.
  • T is calculated as a function of all the pixel values in the prediction block.
  • T ftP
  • P represents all the pixel values in the prediction block.
  • T is calculated as the average value of all the pixel values in the prediction block.
  • T is calculated as the middle value of all the pixel values in the prediction block.
  • the prediction block is classified into M (M>2) segments in the classification step.
  • a pixel is classified into segment 0 if its value is smaller than T ⁇ , it is classified into segment 2 if its value is larger than T 2 ; otherwise it is classified into segment 1.
  • a pixel is classified into segment 0 if its value is smaller than or equal to T ⁇ , it is classified into segment 2 if its value is larger than ⁇ 2 ; otherwise it is classified into segment 1.
  • Tk is calculated as a function of all the pixel values in the prediction block, where k if from 1 to -l .
  • 7* f (P), where P represents all the pixel values in the prediction block.
  • T k is calculated as
  • T 2 (Vmax+T)/2
  • T is the average value of all the pixel values in the prediction block.
  • Vmax and Vmin are the minimum and the maximum pixel value in the prediction block respectively.
  • an offset Offu is added to a pixel in a segment, denoted as segment U, in the treatment process to get the new prediction value.
  • Vnew Vold+ Offu, where Void is the pixel value before the treatment and Vnew is the pixel value after the treatment respectively.
  • Eu is calculated as a function of all the pixel values in segment U.
  • Eu fPu
  • Pu represents all the pixel values in segment U.
  • Eu is calculated as the average value of all the pixel prediction values in segment U.
  • Eu is calculated as the middle value of all the pixel prediction values in segment U.
  • Eu is calculated as the average value of the minimum and the maximum pixel prediction value in segment U.
  • Eu are the minimum and the maximum pixel prediction value in segment [/ respectively.
  • the offset Offu for segment U can be signaled explicitly by the encoder to decoder, or it can be derived implicitly by the decoder.
  • the offset Offu for segment U is calculated by the encoder according to the pixel original values in segment U and pixel prediction values in segment U. For example, the offset Offu for segment U is calculated by the encoder by subtracting the average value of all the pixel original values in segment U and the average value of all the pixel prediction values in segment U. [0046] In another embodiment, the offset Offu for segment U is calculated by the encoder by subtracting the average value of all the pixel original values in segment U
  • Offldxu instead of Offu is coded.
  • Offldxj is the DLT index offset.
  • f(Eu) + Offldxu should be clipped to a valid DLT index.
  • the DLT index offset Offldxu for segment U is calculated by the encoder by subtracting DLT index of the average value of all the pixel original values in segment U and the DLT index of E JJ .
  • Offldxu f(A-u)- f(Eu), where A j is the average value of all the original pixel values in segment U and / represents a function mapping a depth value to a DLT index.
  • the segmental prediction method can be used or not adaptively.
  • the encoder can send the information of whether to use the segmental prediction method to the decoder explicitly. Or the decoder can derive whether to use the segmental prediction method in the same way as the encoder implicitly.
  • the segmental prediction method can be applied on coding tree unit (CTU), coding unit (CU), prediction unit (PU) or transform unit (TU).
  • CTU coding tree unit
  • CU coding unit
  • PU prediction unit
  • TU transform unit
  • the encoder can send the information of whether to use the segmental prediction method to the decoder in video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), slice header (SH), CTU, CU, PU, or TU.
  • VPS video parameter set
  • SPS sequence parameter set
  • PPS picture parameter set
  • SH slice header
  • CTU CTU
  • CU CU
  • PU PU
  • the segmental prediction method can only be applied for CU with some particular sizes. For example, it can only be applied to a CU with size larger than 8x8. In another example, it can only be applied to a CU with size smaller than 64x64.
  • the segmental prediction method can only be applied for CU with some particular PU partition. For example, it can only be applied to a CU with 2Nx2N partition.
  • the segmental prediction method can only be applied for CU with some particular coding mode. For example, it can only be applied to a CU with IBC mode.
  • the segmental prediction method can only be applied for CU with InterSDC mode.
  • the number of segments in the segmental prediction method is adaptively.
  • the encoder can send the information of how many segments to the decoder explicitly when the segmental prediction method is used. Or the decoder can derive the number in the same way as the encoder implicitly.
  • the encoder can send the information of how many segments to the decoder in video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), slice header (SH), CTU, CU, PU, or TU where the segmental prediction method is used.
  • VPS video parameter set
  • SPS sequence parameter set
  • PPS picture parameter set
  • SH slice header
  • CTU CTU
  • CU CU
  • PU PU
  • TU segmental prediction method
  • the encoder can send the information of the offsets for each segment to the decoder in video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), slice header (SH), CTU, CU, PU, or TU where the segmental prediction method is used.
  • VPS video parameter set
  • SPS sequence parameter set
  • PPS picture parameter set
  • SH slice header
  • CTU CTU
  • CU CU
  • PU PU
  • TU segmental prediction method
  • the encoder can send the information of whether to use the segmental prediction method to the decoder in a CU coded with InterSDC mode.
  • the encoder can send the information of how many segments to the decoder in a CU coded with InterSDC mode.
  • the encoder can send the information of the offsets or DLT index offsets for each segment to the decoder in a CU coded with InterSDC and in which the segmental prediction method is used.
  • the segmental prediction method can be applied to the texture component. It can also be applied to depth components in 3D video coding.
  • the segmental prediction method can be applied to the luma component. It can also be applied to chroma components.
  • the decision of whether to use segmental prediction method can be made separately for each component, with information signaled separately. Or, the decision of whether to use segmental prediction method can be made together for all components, with a single piece of information signaled.
  • the number of segments when segmental prediction method is used can be controlled separately for each component separately, with information signaled separately.
  • the number of segments when segmental prediction method is used can be controlled together for all components, with a single piece of information signaled.
  • the offsets for each segment when segmental prediction method is used can be decided separately for each component separately, with information signaled separately.
  • the offsets for each segment when segmental prediction method is used can be decided together for all components, with a single piece of information signaled.
  • an embodiment of the present invention can be a circuit integrated into a video compression chip or program codes integrated into video compression software to perform the processing described herein.
  • An embodiment of the present invention may also be program codes to be executed on a Digital Signal Processor (DSP) to perform the processing described herein.
  • DSP Digital Signal Processor
  • the invention may also involve a number of functions to be performed by a computer processor, a digital signal processor, a microprocessor, or field programmable gate array (FPGA).
  • processors can be configured to perform particular tasks according to the invention, by executing machine-readable software code or firmware code that defines the particular methods embodied by the invention.
  • the software code or firmware codes may be developed in different programming languages and different format or style.
  • the software code may also be compiled for different target platform.
  • different code formats, styles and languages of software codes and other means of configuring code to perform the tasks in accordance with the invention will not depart from the spirit and scope of the invention.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

A segmental prediction coding method is proposed.By segment prediction pixels into different segments with different treatments,prediction coding can be more efficient.

Description

SEGMENTAL PREDICTION FOR VIDEO CODING
TECHNICAL FIELD
The invention relates generally to video/image processing.
BACKGROUND
[0002] Prediction takes a critical role in video coding. When coding or decoding a block, a prediction block, which is generated by intra-prediction or inter-prediction, is obtained first before producing residues at encoder or reconstructing the reconstruction samples at decoder.
[0003] Besides inter-prediction and intra-prediction, a new prediction mode named intra-block copy (IBC) has been adopted in the screen content coding (SCC) profile for high efficiency video coding (HEVC) standard. IBC was adopted to take advantage of reduplicated content in a picture. As depicted in Fig. l, a reference block in the current picture is copied to the current block as the prediction if IBC is applied. The reference block is located by applying a block-copying vector (BV). The samples in the reference block must have been reconstructed already before the current block is coded or decoded.
[0004] Inter simplified depth coding (InterSDC) is adopted into 3D-HEVC as a special prediction mode for depth coding. When InterSDC is used, a normal inter- prediction is performed for the current block first. Then each pixel in the prediction block is added by a coded offset. Suppose PLj represents the prediction value at pixel position ( i, j) after performing the normal inter-prediction, Offset is the offset coded for this block. Then the final prediction value at pixel position is Pi + Offset. With InterSDC mode, no residues are coded. Thus the final prediction value will be output as the reconstructed value.
[0005] Depth lookup table (DLT) is adopted into 3D-HEVC. Since there are often only several values appearing in the depth component, DLT signals those valid values from the encoder to the decoder. When a CU is coded in intra simplified depth coding (SDC) mode or depth map modeling (DMM) mode, DLT is used to map the valid depth value to a DLT index which is much easier to compress. Fig.2 demonstrates an example of DLT approach. DLT is signaled in picture parameter set (PPS). And it is left to be an encoder issue how to get the DLT when encoding.
[0006] Since prediction values come from reconstructed pixels, there are distortions between the prediction values and the original values, even if the original pixels in the current block and the original pixels in the reference block are exactly the same. Because the reconstructed signal loses high-frequency information generally, the quality of prediction is deteriorated more when there are sharp pixel value changes in the reference block. Fig. 3 and Fig. 4 demonstrate two examples where there are two and three segments with sharp edges in a block.
SUMMARY
[0007] In light of the previously described problems, a segmental prediction method is proposed. The prediction block is processed by a segmental process before it is used to get residues at an encoder or get reconstruction at a decoder. The segmental prediction method comprises classifying pixels in the prediction block into different categories, named as 'segment', wherein the pixels in the segment can be adjacent or not, treating the pixels in different segments in different ways, and forming a new prediction block after the treatment. The new prediction block is used to get the residues at the encoder or get the reconstruction at the decoder.
[0008] Other aspects and features of the invention will become apparent to those with ordinary skill in the art upon review of the following descriptions of specific embodiments.
BRIEF DESCRIPTION OF DRAWINGS
[0009] The invention can be more fully understood by reading the subsequent detailed description and examples with references made to the accompanying drawings, wherein:
[0010] Fig. 1 is a diagram illustrating intra block copying;
[0011] Fig. 2 is a diagram illustrating is a diagram illustrating an example of DLT where five valid values appear in depth samples.
[0012] Fig. 3 is a diagram illustrating a block with two segments with sharp sample value changes;
[0013] Fig. 4 is a diagram illustrating a block with three segments with sharp sample value changes;
[0014] Fig. 5 is a diagram illustrating an exemplary segmental prediction architecture at the decoder;
[0015] Fig. 6 is a diagram illustrating an exemplary segmental process architecture;
[0016] Fig. 7 is a diagram illustrating an exemplary treatment for a segment. After the treatment, pixels in the segment can hold different values;
[0017] Fig. 8 is a diagram illustrating an exemplary treatment for a segment. After the treatment, pixels in the segment hold only one value.
DETAILED DESCRIPTION [0018] The following description is of the best-contemplated mode of carrying out the invention. This description is made for the purpose of illustrating the general principles of the invention and should not be taken in a limiting sense. The scope of the invention is best determined by reference to the appended claims.
[0019] A segmental prediction method is proposed.
[0020] In one embodiment, the prediction block is processed by a segmental process before it is used to get the residues at encoder or get the reconstruction at decoder. In another embodiment, the prediction block is processed by a segmental process, and then the modified prediction block is output as the reconstruction without adding to residues. Fig. 5 demonstrates segmental prediction architecture at the decoder.
[0021] In the segmental process, there are two steps generally as depicted in Fig.6. The first step is called 'classification', in which the pixels in a prediction block are classified into different categories, named as 'segment'. Pixels in a segment can be adjacent or not. The second step is called 'treatment', in which pixels in different segments are treated in different ways. Finally, all the pixels after the treatment form a new prediction block which is output then. The number of segments can be any positive integers such as 1, 2, 3, etc.
[0022] In one embodiment, the prediction values can be classified according to their values.
[0023] In another embodiment, the prediction values can be classified according to their positions.
[0024] In still another embodiment, the prediction values can be classified according to their gradients. Gradients can be got by applying operators such as Sobel, Roberts, and Prewitt.
[0025] In still another embodiment, classification is not applied if the number of segments is one.
[0026] In still another embodiment, the prediction block is classified into two segments in the classification step. A pixel is classified according to its relationship with a threshold number T. In one example, a pixel is classified into segment 0 if its value is larger than T, Otherwise it is classified into segment 1. In another example, a pixel is classified into segment 0 if its value is larger than or equal to T, Otherwise it is classified into segment 1.
[0027] In still another embodiment, T is calculated as a function of all the pixel values in the prediction block. In a formula way, T= ftP), where P represents all the pixel values in the prediction block.
[0028] In still another embodiment, T is calculated as the average value of all the pixel values in the prediction block.
[0029] In still another embodiment, T is calculated as the middle value of all the pixel values in the prediction block.
[0030] In still another embodiment, T is calculated as the average value of the minimum and the maximum pixel value in the prediction block. In a formula way, T= (Vmax+ Vmin)/2, where Vmax and Vmin are the minimum and the maximum pixel value in the prediction block respectively.
[0031] In still another embodiment, the prediction block is classified into M (M>2) segments in the classification step. A pixel is classified according to its relationship with M- 1 threshold numbers T\<= T2<=... <= TM-i - In one example where Mis equal to 3, a pixel is classified into segment 0 if its value is smaller than T\, it is classified into segment 2 if its value is larger than T2; otherwise it is classified into segment 1. In another example, a pixel is classified into segment 0 if its value is smaller than or equal to T\, it is classified into segment 2 if its value is larger than Γ2; otherwise it is classified into segment 1.
[0032] In still another embodiment, Tk is calculated as a function of all the pixel values in the prediction block, where k if from 1 to -l . In a formula way, 7* =f (P), where P represents all the pixel values in the prediction block.
[0033] In still another embodiment where Mis equal to 3, Tk is calculated as
[0034] Ti= (T+ Vmin)/2
[0035] and
[0036] T2= (Vmax+T)/2,
[0037] where T is the average value of all the pixel values in the prediction block. Vmax and Vmin are the minimum and the maximum pixel value in the prediction block respectively.
[0038] In one embodiment as depicted in Fig. 7, an offset Offu is added to a pixel in a segment, denoted as segment U, in the treatment process to get the new prediction value. In a formula way, Vnew=Vold+ Offu, where Void is the pixel value before the treatment and Vnew is the pixel value after the treatment respectively.
[0039] In another embodiment as depicted in Fig. 8, all pixels possess the same value Vu in a segment, denoted as segment U, after the treatment process, an offset Offu is added to an estimated value Eu to obtain Vu. In a formula way, Vu = Eu + Offu.
[0040] In another embodiment, Eu is calculated as a function of all the pixel values in segment U. In a formula way, Eu = fPu), where Pu represents all the pixel values in segment U.
[0041] In still another embodiment, Eu is calculated as the average value of all the pixel prediction values in segment U.
[0042] In still another embodiment, Eu is calculated as the middle value of all the pixel prediction values in segment U.
[0043] In still another embodiment, Eu is calculated as the average value of the minimum and the maximum pixel prediction value in segment U. In a formula way, Eu=
Figure imgf000006_0001
are the minimum and the maximum pixel prediction value in segment [/ respectively.
[0044] In one embodiment, the offset Offu for segment U can be signaled explicitly by the encoder to decoder, or it can be derived implicitly by the decoder.
[0045] In one embodiment, the offset Offu for segment U is calculated by the encoder according to the pixel original values in segment U and pixel prediction values in segment U. For example, the offset Offu for segment U is calculated by the encoder by subtracting the average value of all the pixel original values in segment U and the average value of all the pixel prediction values in segment U. [0046] In another embodiment, the offset Offu for segment U is calculated by the encoder by subtracting the average value of all the pixel original values in segment U
[0047] In still another embodiment, Offldxu instead of Offu is coded. Offldxj is the DLT index offset.
[0048] In still another embodiment, Vu is calculated as Vu =g( f(Eu) + Offldxu), where / represents a function mapping a depth value to a DLT index, and g represents a function mapping a DLT index to a depth value. f(Eu) + Offldxu should be clipped to a valid DLT index.
[0049] In another embodiment, the DLT index offset Offldxu for segment U is calculated by the encoder by subtracting DLT index of the average value of all the pixel original values in segment U and the DLT index of EJJ. In a formula way, Offldxu= f(A-u)- f(Eu), where A j is the average value of all the original pixel values in segment U and / represents a function mapping a depth value to a DLT index.
[0050] In one embodiment, the segmental prediction method can be used or not adaptively. The encoder can send the information of whether to use the segmental prediction method to the decoder explicitly. Or the decoder can derive whether to use the segmental prediction method in the same way as the encoder implicitly.
[0051] In still another embodiment, the segmental prediction method can be applied on coding tree unit (CTU), coding unit (CU), prediction unit (PU) or transform unit (TU).
[0052] In still another embodiment, the encoder can send the information of whether to use the segmental prediction method to the decoder in video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), slice header (SH), CTU, CU, PU, or TU.
[0053] In still another embodiment, the segmental prediction method can only be applied for CU with some particular sizes. For example, it can only be applied to a CU with size larger than 8x8. In another example, it can only be applied to a CU with size smaller than 64x64.
[0054] In still another embodiment, the segmental prediction method can only be applied for CU with some particular PU partition. For example, it can only be applied to a CU with 2Nx2N partition.
[0055] In still another embodiment, the segmental prediction method can only be applied for CU with some particular coding mode. For example, it can only be applied to a CU with IBC mode.
[0056] In still another embodiment, the segmental prediction method can only be applied for CU with InterSDC mode.
[0057] In one embodiment, the number of segments in the segmental prediction method is adaptively. The encoder can send the information of how many segments to the decoder explicitly when the segmental prediction method is used. Or the decoder can derive the number in the same way as the encoder implicitly.
[0058] In still another embodiment, the encoder can send the information of how many segments to the decoder in video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), slice header (SH), CTU, CU, PU, or TU where the segmental prediction method is used.
[0059] In still another embodiment, the encoder can send the information of the offsets for each segment to the decoder in video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), slice header (SH), CTU, CU, PU, or TU where the segmental prediction method is used.
[0060] In still another embodiment, the encoder can send the information of whether to use the segmental prediction method to the decoder in a CU coded with InterSDC mode.
[0061] In still another embodiment, the encoder can send the information of how many segments to the decoder in a CU coded with InterSDC mode.
[0062] In still another embodiment, the encoder can send the information of the offsets or DLT index offsets for each segment to the decoder in a CU coded with InterSDC and in which the segmental prediction method is used.
[0063] In still another embodiment, the segmental prediction method can be applied to the texture component. It can also be applied to depth components in 3D video coding.
[0064] In still another embodiment, the segmental prediction method can be applied to the luma component. It can also be applied to chroma components.
[0065] In still another embodiment, the decision of whether to use segmental prediction method can be made separately for each component, with information signaled separately. Or, the decision of whether to use segmental prediction method can be made together for all components, with a single piece of information signaled.
[0066] In still another embodiment, the number of segments when segmental prediction method is used can be controlled separately for each component separately, with information signaled separately. Or, the number of segments when segmental prediction method is used can be controlled together for all components, with a single piece of information signaled.
[0067] In still another embodiment, the offsets for each segment when segmental prediction method is used can be decided separately for each component separately, with information signaled separately. Or, the offsets for each segment when segmental prediction method is used can be decided together for all components, with a single piece of information signaled.
[0068] The methods described above can be used in a video encoder as well as in a video decoder. Embodiments of disparity vector derivation methods according to the present invention as described above may be implemented in various hardware, software codes, or a combination of both. For example, an embodiment of the present invention can be a circuit integrated into a video compression chip or program codes integrated into video compression software to perform the processing described herein. An embodiment of the present invention may also be program codes to be executed on a Digital Signal Processor (DSP) to perform the processing described herein. The invention may also involve a number of functions to be performed by a computer processor, a digital signal processor, a microprocessor, or field programmable gate array (FPGA). These processors can be configured to perform particular tasks according to the invention, by executing machine-readable software code or firmware code that defines the particular methods embodied by the invention. The software code or firmware codes may be developed in different programming languages and different format or style. The software code may also be compiled for different target platform. However, different code formats, styles and languages of software codes and other means of configuring code to perform the tasks in accordance with the invention will not depart from the spirit and scope of the invention.
[0069] The invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described examples are to be considered in all respects only as illustrative and not restrictive. To the contrary, it is intended to cover various modifications and similar arrangements (as would be apparent to those skilled in the art). Therefore, the scope of the appended claims should be accorded the broadest interpretation so as to encompass all such modifications and similar arrangements.

Claims

1. A method of segmental prediction coding, in which a prediction block is processed by a segmental process before used to get residues at an encoder or get reconstruction at a decoder, comprising,
classifying pixels in the prediction block into different categories, named as 'segment', wherein the pixels in a segment is either adjacent or not;
treating the pixels in different segments in different ways; and
forming a new prediction block after treatment for all the pixels, wherein the new prediction block is used to get the residues at the encoder or get the reconstruction at the decoder.
2. The method as claimed in claim 1 , wherein prediction values are classified according to values, positions, or gradients.
3. The method as claimed in claim 2, wherein the prediction block is classified into two segments, and a pixel is classified according to its relationship with a threshold number T.
4. The method as claimed in claim 3, wherein T is calculated as a function of all the pixel values in the prediction block, and the function comprises one of the following calculations:
T is calculated as an average value of all the pixel values in the prediction block;
T is calculated as a middle value of all the pixel values in the prediction block; and
T is calculated as an average value of a minimum and a maximum pixel value in the prediction block, T= (Vmax+ Vmin)/2, where Vmax and Vmin are the minimum and the maximum pixel value in the prediction block respectively.
5. The method as claimed in claim 2, wherein the prediction block is classified into M (M>2) segments, and a pixel is classified according to its relationship with M- 1 threshold numbers
Figure imgf000010_0001
<= TMA -
6. The method as claimed in claim 5, wherein Tk is calculated as a function of all the pixel values in the prediction block, where k if from 1 to -l, Tk =fk (P), where P represents all the pixel values in the prediction block.
7. The method as claimed in claim 5, wherein Mis equal to 3, a pixel is classified into segment 0 if its value is smaller than T\, it is classified into segment 2 if its value is larger than T2; otherwise it is classified into segment 1, T\= (T+Vmin)/2 and T2= (Vmax+T)/2, where T is an average value of all the pixel values in the prediction block, Vmax and Vmin are a minimum and a maximum pixel value in the prediction block respectively.
8. The method as claimed in claim 1, wherein an offset Offu is added to a pixel in a segment, denoted as segment U, in the treatment process to get the new prediction value, Vnew=Vold+ Offu, where Void is the pixel value before treatment and Vnew is the pixel value after treatment respectively.
9. The method as claimed in claim 1, wherein all pixels possess a same value Vu in a segment, denoted as segment U, after treatment, and an offset Offu is added to an estimated value Eu to obtain Vu, Vu = Eu + Offu.
10. The method as claimed in claim 9, Eu is calculated as a function of all the pixel values in segment U, Eu = f[Pu), where Pu represents all the pixel values in segment U, and functions include but not limited to:
Eu is calculated as an average value of all the pixel prediction values in segment U;
Eu is calculated as a middle value of all the pixel prediction values in segment U; Eu is calculated as an average value of a minimum and a maximum pixel prediction value in segment U, Eu=
Figure imgf000011_0001
are a minimum and the maximum pixel prediction value in segment U respectively.
11. The method as claimed in claim 8, wherein the offset Offu for segment U is signaled explicitly by the encoder to decoder, or the offset Offu is derived implicitly by the decoder.
12. The method as claimed in claim 11, wherein Offldxu instead of Offu is coded, and Offldxu is depth lookup table (DLT) index offset.
13. The method as claimed in claim 9, wherein Vu is calculated as Vu =g( f(Eu) + Offldxu), where / represents a function mapping a depth value to a DLT index, and g represents a function mapping a DLT index to a depth value, f(Eu) + Offldxu is clipped to a valid DLT index.
14. The method as claimed in claim 8, wherein the offset Offu for segment U is calculated by the encoder according to pixel original values in segment U and pixel prediction values in segment U, wherein the offset Offu for segment U is calculated by the encoder by subtracting an average value of all the pixel original values in segment U and Eu.
15. The method as claimed in claim 12, wherein the DLT index offset Offldxjj for segment U is calculated by the encoder by subtracting DLT index of the average value of all the pixel original values in segment U and the DLT index of Eu,
Figure imgf000012_0001
where Au is the average value of all the original pixel values in segment U and / represents a function mapping a depth value to a DLT index.
16. The method as claimed in claim 1 , wherein the segmental prediction method is used adaptively, the encoder sends information of whether to use the segmental prediction method to the decoder explicitly, or the decoder derives whether to use the segmental prediction method in the same way as the encoder implicitly.
17. The method as claimed in claim 1 , wherein the segmental prediction method is applied on coding tree unit (CTU), coding unit (CU), prediction unit (PU) or transform unit (TU), and the encoder sends information of whether to use the segmental prediction method to the decoder in video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), slice header (SH), CTU, CU, PU, or TU.
18. The method as claimed in claim 1 , wherein the segmental prediction method only applies for CU with a predetermined set of sizes.
19. The method as claimed in claim 1 , wherein the segmental prediction method only applies for CU with a predetermined set of PU partitions.
20. The method as claimed in claim 1 , wherein the segmental prediction method only applies for CU with a predetermined set of coding modes.
21. The method as claimed in claim 1 , wherein a number of segments in the segmental prediction method is adaptively, and the encoder sends information of how many segments to the decoder explicitly when the segmental prediction method is used, or the decoder derives the number in the same way as the encoder implicitly.
22. The method as claimed in claim 1 , wherein the encoder sends information of how many segments to the decoder in video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), slice header (SH), CTU, CU, PU, or TU where the segmental prediction method is used.
23. The method as claimed in claim 1 , wherein the encoder sends information of offsets or DLT index offsets for each segment to the decoder in video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), slice header (SH), CTU, CU, PU, or TU where the segmental prediction method is used.
24. The method as claimed in claim 20, wherein the segmental prediction method is only applied for CU with InterSDC mode.
25. The method as claimed in claim 1, wherein the segmental prediction method is applied to luma component, chroma components, or both luma and chroma components.
26. The method as claimed in claim 1, wherein the segmental prediction method is applied to a texture component or depth components in 3D video coding.
27. The method as claimed in claim 1, wherein a decision of whether to use segmental prediction method is made separately for each component, with information signaled separately, or the decision of whether to use segmental prediction method is made together for all components, with a single piece of information signaled.
28. The method as claimed in claim 1, wherein a number of segments when segmental prediction method is used is controlled separately for each component separately, with information signaled separately, or the number of segments when segmental prediction method is used is controlled together for all components, with a single piece of information signaled.
29. The method as claimed in claim 1, wherein the offsets for each segment when segmental prediction method is used is decided separately for each component separately, with information signaled separately, or the offsets for each segment when segmental prediction method is used is decided together for all components, with a single piece of information signaled.
30. The method as claimed in claim 16, wherein the encoder sends information of whether to use the segmental prediction method to the decoder in a CU coded with InterSDC mode, for a CU coded with InterSDC and in which the segmental prediction method is used, the encoder sends information of how many segments to the decoder and information of the offsets or DLT index offsets for each segment.
PCT/CN2014/080515 2014-06-23 2014-06-23 Segmental prediction for video coding Ceased WO2015196333A1 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
PCT/CN2014/080515 WO2015196333A1 (en) 2014-06-23 2014-06-23 Segmental prediction for video coding
US15/032,205 US10244258B2 (en) 2014-06-23 2015-06-23 Method of segmental prediction for depth and texture data in 3D and multi-view coding systems
PCT/CN2015/082074 WO2015196966A1 (en) 2014-06-23 2015-06-23 Method of segmental prediction for depth and texture data in 3d and multi-view coding systems
CN201580001847.4A CN105556968B (en) 2014-06-23 2015-06-23 Device and method for predictive coding in 3D or multi-view video coding system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2014/080515 WO2015196333A1 (en) 2014-06-23 2014-06-23 Segmental prediction for video coding

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2014/087094 Continuation-In-Part WO2016044979A1 (en) 2014-06-23 2014-09-22 Segmental prediction for video coding

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US15/032,205 Continuation-In-Part US10244258B2 (en) 2014-06-23 2015-06-23 Method of segmental prediction for depth and texture data in 3D and multi-view coding systems

Publications (1)

Publication Number Publication Date
WO2015196333A1 true WO2015196333A1 (en) 2015-12-30

Family

ID=54936410

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2014/080515 Ceased WO2015196333A1 (en) 2014-06-23 2014-06-23 Segmental prediction for video coding

Country Status (1)

Country Link
WO (1) WO2015196333A1 (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6798834B1 (en) * 1996-08-15 2004-09-28 Mitsubishi Denki Kabushiki Kaisha Image coding apparatus with segment classification and segmentation-type motion prediction circuit
CN103004206A (en) * 2010-07-20 2013-03-27 株式会社Ntt都科摩 Image predictive encoding device, image predictive encoding method, image predictive encoding program, image predictive decoding device, image predictive decoding method, and image predictive decoding program
CN103108181A (en) * 2007-03-23 2013-05-15 三星电子株式会社 Method and apparatus for image encoding and image decoding
CN103650498A (en) * 2012-06-04 2014-03-19 松下电器产业株式会社 Moving image encoding method, moving image encoding device, moving image decoding method, and moving image decoding device

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6798834B1 (en) * 1996-08-15 2004-09-28 Mitsubishi Denki Kabushiki Kaisha Image coding apparatus with segment classification and segmentation-type motion prediction circuit
CN103108181A (en) * 2007-03-23 2013-05-15 三星电子株式会社 Method and apparatus for image encoding and image decoding
CN103004206A (en) * 2010-07-20 2013-03-27 株式会社Ntt都科摩 Image predictive encoding device, image predictive encoding method, image predictive encoding program, image predictive decoding device, image predictive decoding method, and image predictive decoding program
CN103650498A (en) * 2012-06-04 2014-03-19 松下电器产业株式会社 Moving image encoding method, moving image encoding device, moving image decoding method, and moving image decoding device

Similar Documents

Publication Publication Date Title
US11234002B2 (en) Method and apparatus for encoding and decoding a texture block using depth based block partitioning
KR101885633B1 (en) Video encoding using block-based mixed-resolution data pruning
WO2017045580A1 (en) Method and apparatus of advanced de-blocking filter in video coding
CN105981385B (en) Intra-frame prediction coding method and device thereof
WO2017041676A1 (en) Method and apparatus of context modelling for syntax elements in image and video coding
US20190068977A1 (en) Method and apparatus of localized luma prediction mode inheritance for chroma prediction in video coding
CN113950837A (en) Image decoding device, image decoding method, and program
CN110036637B (en) Method and device for denoising and vocalizing reconstructed image
WO2015090217A1 (en) Method and apparatus for palette table prediction
US11006147B2 (en) Devices and methods for 3D video coding
JP6857053B2 (en) A method and device for encoding at least one image unit, and a method and device for decoding a stream representing at least one image unit.
WO2024006167A1 (en) Inter coding using deep learning in video compression
US10244258B2 (en) Method of segmental prediction for depth and texture data in 3D and multi-view coding systems
KR20150113524A (en) Device for decoding image using prediction mode based on improved intra block copy and method thereof
CN110063059B (en) Video coding and decoding method, related coder and decoder and computer readable storage medium
WO2020007747A1 (en) Deblocking of intra-reference samples
WO2016044979A1 (en) Segmental prediction for video coding
WO2015196333A1 (en) Segmental prediction for video coding
WO2016061743A1 (en) Segmental prediction for video coding
CN114175659A (en) Apparatus and method for bit width control of bi-directional optical flow
WO2024235244A1 (en) Methods and apparatus for selecting transform type in a video coding system
WO2016115733A1 (en) Improvements for inter-component residual prediction
WO2026007109A1 (en) Encoding method, decoding method, bitstream, encoder, decoder, and storage medium
Yu et al. Sample edge offset compensation for HEVC based 3D Video Coding
WO2015100712A1 (en) The method to perform the deblocking on sub-pu edge

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14895698

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 14895698

Country of ref document: EP

Kind code of ref document: A1