WO2015139175A1

WO2015139175A1 - Improved block copying

Info

Publication number: WO2015139175A1
Application number: PCT/CN2014/073526
Authority: WO
Inventors: Kai Zhang; Jicheng An; Xianguo Zhang; Han HUANG
Original assignee: Mediatek Singapore Pte. Ltd.
Priority date: 2014-03-17
Filing date: 2014-03-17
Publication date: 2015-09-24

Abstract

Reformed copying methods are proposed. By utilizing reformed copying, the prediction module can manipulate symmetric patterns in images more efficiently.

Description

IMPROVED BLOCK COPYING

TECHNICAL FIELD

[0001] The invention relates generally to video/image processing. In particular, the presented invention relates to block copying.

BACKGROUND

[0002] In the current HEVC range extensions draft specification [1], Intra block copying (IBC) was adopted to take advantage of reduplicated content in a picture. As depicted in Fig.1, a reference block in the current picture is copied to the current block as the prediction if IBC is applied. The reference block is located by applying a block- copying vector (BV). The samples in the reference block must have been reconstructed already before the current block is coded or decoded.

[0003] IBC can reflect reduplicated pattern in an image. Besides reduplication, symmetry is another usual phenomenon in natural or screen-content images. Fig. 2~Fig.4 demonstrates examples for horizontal symmetry, vertical symmetry, and centro symmetry in screen-content images respectively. Fig.5 shows an example for horizontal symmetry in a natural image.

[0004] IBC cannot deal with symmetric patterns efficiently.

SUMMARY

[0005] In light of the previously described problems, reformed copying ( C) methods are proposed.

[0006] Other aspects and features of the invention will become apparent to those with ordinary skill in the art upon review of the following descriptions of specific embodiments.

BRIEF DESCRIPTION OF DRAWINGS

The invention can be more fully understood by reading the subsequent detailed description and examples with references made to the accompanying drawings, wherein:

[0007] Fig. 1 is a diagram illustrating Intra block copying;

[0008] Fig. 2 is a diagram illustrating a horizontal symmetric example in a screen content image: character Ά';

[0009] Fig. 3 is a diagram illustrating a vertical symmetric example in a screen content image: character 'C;

[0010] Fig.4 is a diagram illustrating a centrosymmetric example in a screen content image: character 'S';

[0011] Fig.5 is a diagram illustrating an approximate horizontal symmetric example in a nature image: human face;

[0012] Fig.6 is a diagram illustrating reformed copying method;

[0013] Fig.7 is a diagram illustrating horizontal intra reformed copying;

[0014] Fig.8 is a diagram illustrating vertical intra reformed copying;

[0015] Fig.9 is a diagram illustrating horizontal + vertical reformed copying;

[0016] Fig.10 is a diagram illustrating clockwise rotation reformed copying;

[0017] Fig.11 is a diagram illustrating counterclockwise rotation reformed copying;

[0018] Fig.12 is a diagram illustrating transposition reformed copying;

[0019] Fig.13 is a diagram illustrating cascade reformed copying.

DETAILED DESCRIPTION

[0020] The following description is of the best-contemplated mode of carrying out the invention. This description is made for the purpose of illustrating the general principles of the invention and should not be taken in a limiting sense. The scope of the invention is best determined by reference to the appended claims.

[0021] Several reformed copying ( C) methods are proposed. In RC methods, a reference block is reformed first, and then the reformed block is used to predict the current block, as depicted in Fig. 6.

[0022] In one embodiment, a reference block is horizontally flipped before it is used to predict the current block, as depicted in Fig. 7. The reference block can be in the same picture of the current block, or it can be in a different picture of the current block. An exemplary flipping process is described as follows. Suppose the width of the reference block is W, R(x, y) is a sample in the reference block with x from 0 to W-l, then a sample R'(x,y) in the flipped reference block can be derived as R'(x,y)=R(W-l-x, y).

[0023] In another embodiment, a reference block is vertically flipped before it is used to predict the current block, as depicted in Fig. 8. The reference block can be in the same picture of the current block, or it can be in a different picture of the current block. An exemplary flipping process is described as follows. Suppose the height of the reference block is H, R(x, y) is a sample in the reference block with y from 0 to H- 1, then a sample R'(x,y) in the flipped reference block can be derived as R'(x,y)=R(x, H-l-y).

[0024] In still another embodiment, a reference block is vertically flipped then horizontally flipped or horizontally flipped then vertically flipped before it is used to predict the current block, as depicted in Fig. 9. The reference block can be in the same picture of the current block, or it can be in a different picture of the current block. An exemplary flipping process is described as follows. Suppose the width of the reference block is W, the height of the reference block is H, and R(x, y) is a sample in the reference block with x from 0 to W-l and y from 0 to H-1, then a sample R'(x,y) in the flipped reference block can be derived as R'(x,y)=R(W-l-x, H-l-y).

[0025] In still another embodiment, a reference block is rotated clock wisely before it is used to predict the current block, as depicted in Fig. 10. The reference block can be in the same picture of the current block, or it can be in a different picture of the current block. An exemplary rotation process is described as follows. Suppose the height of the reference block is H, and R(x, y) is a sample in the reference block with y from 0 to H-1, then a sample R'(x,y) in the rotated reference block can be derived as R'(x,y)=R(H-l-y, x).

[0026] In still another embodiment, a reference block is rotated counter clock wisely before it is used to predict the current block, as depicted in Fig. 11. The reference block can be in the same picture of the current block, or it can be in a different picture of the current block. An exemplary rotation process is described as follows. Suppose the width of the reference block is W, and R(x, y) is a sample in the reference block with x from 0 to W-l, then a sample R'(x,y) in the rotated reference block can be derived as R'(x,y)=R(y, W-l-x).

[0027] In still another embodiment, a reference block is transposed before it is used to predict the current block, as depicted in Fig. 12. The reference block can be in the same picture of the current block, or it can be in a different picture of the current block. An exemplary transposition process is described as follows. Suppose R(x, y) is a sample in the reference block, then a sample R'(x,y) in the transposed reference block can be derived as R'(x,y)=R(y, x).

[0028] In still another embodiment, a reference block is reformed arbitrarily before it is used to predict the current block. The reference block can be in the same picture of the current block, or it can be in a different picture of the current block. An exemplary transposition process is described as follows. Suppose R(x, y) is a sample in the reference block, then a sample R'(x,y) in the reformed reference block can be derived as R'(x,y)=R( ( x, y ), g( x, y )), where / and g are any functions. Function / can depend on x only, on y only, or on both. Function g can depend on x only, on y only, or on both. Function / can depend on the size, width, height, shape or coding mode of the current block or the reference block. Function g can depend on the size, width, height, shape or coding mode of the current block or the reference block. Function / can depend on the type of the current or reference picture or slice. Function g can depend on the type of the current or reference picture or slice. Function / can be a linear or non-linear function. Function g can be a linear or non-linear function. / and g can be the same, or they can be different.

[0029] In still another embodiment, a reference block is reformed by a combination of two or more reforming methods, such as horizontal flipping, vertical flipping, clock wise rotation, counter clock wise rotation, transposition and any arbitrary reforms. The reforms can be done in a cascade way, in which a reference block is reformed sequentially before it is used predict the current block as depicted in Fig. 13. The output from a former reform is treated as the input to a subsequent reform.

[0030] In still another embodiment, an encoder can decide to apply reformed copying or not for a block and signal the information to the decoder. The information can be signaled for a macro-block (MB), a coding tree unit (CTU) a coding unit (CU), a transform unit (TU) or a prediction unit (PU). The encoder can make the decision based on the rate-distortion optimization (RDO) criterion or other criterions.

[0031] In still another embodiment, an encoder can decide to apply which kind of reformed copying for a block and signal the information to the decoder. The information can be signaled for a macro-block (MB), a coding tree unit (CTU) a coding unit (CU), a transform unit (TU) or a prediction unit (PU). The encoder can make the decision based on the rate-distortion optimization (RDO) criterion or other criterions.

[0032] In still another embodiment, a decoder can infer whether to apply reformed copying or not for a block implicitly. No information is signaled to indicate whether to apply reformed copying or not from the encoder explicitly.

[0033] In still another embodiment, a decoder can infer to apply which kind of reformed copying for a block implicitly. No information is signaled to indicate which kind of reformed copying is applied from the encoder explicitly.

[0034] In still another embodiment, the reformed copying methods can be applied to inter block copying, or motion compensation.

[0035] In still another embodiment, the reformed copying methods can be applied to intra block copying.

[0036] In still another embodiment, the reformed copying methods can be applied to a block with size MxN, where M and N are arbitrary positive integers. M can be equal to N or M can be not equal to N. For example, M can be equal to a*N where a is a positive integer equal or larger than 2. N can be equal to b*M, where b is a positive integer equal or larger than 2. Specially, M or N can be 1.

[0037] In still another embodiment, the reformed copying methods can be applied to all components in an image. For example, it can be applied to Y, U, or V components. It can also be applied to Q R, or B components. It can also be applied to depth component for 3D video/image coding.

[0038] In still another embodiment, it can be done separately for each component to decide whether to apply reformed copying or not and signal this information for a block. Or all components can be treated in the same manner and only single information is signaled for a block.

[0039] In still another embodiment, it can be done separately for each component to decide which kind of reformed copying is applied and signal this information for a block. Or all components can be treated in the same manner and only single information is signaled for a block.

[0040] In still another embodiment, it can be done separately for each component to infer whether to apply reformed copying or not for a block at decoder implicitly. Or all components can be treated in the same manner for a block.

[0041] In still another embodiment, it can be done separately for each component to infer which kind of reformed copying is applied for a block at decoder implicitly. Or all components can be treated in the same manner for a block.

[0042] In still another embodiment, it depends on the size, shape or coding mode of the current block or the current CU\TU\PU to decide or infer whether to apply reformed copying or not.

[0043] In still another embodiment, it depends on the size, shape or coding mode of the current block or the current CU\TU\PU to decide or infer which kind of reformed copying is applied.

[0044] In still another embodiment, it depends on BVs of the current block in intra block copying to decide or infer whether to apply reformed copying or not.

[0045] In still another embodiment, it depends on BVs of the current block to decide or infer which kind of reformed copying is applied.

[0046] In still another embodiment, it depends on motion vectors (MVs) of the current block in inter block copying to decide or infer whether to apply reformed copying or not.

[0047] In still another embodiment, it depends on MVs of the current block in inter block copying to decide or infer which kind of reformed copying is applied.

[0048] In still another embodiment, the reference block is horizontally flipped before it is used for prediction if reformed copying is used in the current block and the reference block and the current block possess the same vertical position. In other words, the y component of the BV for the current block is 0.

[0049] In still another embodiment, the reference block is vertically flipped before it is used for prediction if reformed copying is used in the current block and the reference block and the current block possess the same horizontal position. In other words, the x component of the BV for the current block is 0.

[0050] In still another embodiment, reformed copying is only allowed for CU with intra block copying mode.

[0051] In still another embodiment, reformed copying is only allowed for CU with partition size 2N><2N.

[0052] In still another embodiment, a flag is coded by context adaptive binarized arithmetic coding (CAB AC) for a CU, TU or PU to indicate whether reformed copying is used for the current block. The flags in its neighboring blocks are used to determine the context model to code the flag for the current block. The flag is treated as 0 if it is not signaled. Or it is treated as 1 if it is not signaled.

[0053] In still another embodiment, the BVs or MVs for the current block are coded in the same way as the case where reformed copying is not used when reformed copying is used for the current block. Or the BVs or MVs for the current block are coded a different way to the case where reformed copying is not used when reformed copying is used for the current block.

[0054] The methods described above can be used in a video encoder as well as in a video decoder. Embodiments of disparity vector derivation methods according to the present invention as described above may be implemented in various hardware, software codes, or a combination of both. For example, an embodiment of the present invention can be a circuit integrated into a video compression chip or program codes integrated into video compression software to perform the processing described herein. An embodiment of the present invention may also be program codes to be executed on a Digital Signal Processor (DSP) to perform the processing described herein. The invention may also involve a number of functions to be performed by a computer processor, a digital signal processor, a microprocessor, or field programmable gate array (FPGA). These processors can be configured to perform particular tasks according to the invention, by executing machine-readable software code or firmware code that defines the particular methods embodied by the invention. The software code or firmware codes may be developed in different programming languages and different format or style. The software code may also be compiled for different target platform. However, different code formats, styles and languages of software codes and other means of configuring code to perform the tasks in accordance with the invention will not depart from the spirit and scope of the invention.

[0055] The invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described examples are to be considered in all respects only as illustrative and not restrictive. To the contrary, it is intended to cover various modifications and similar arrangements (as would be apparent to those skilled in the art). Therefore, the scope of the appended claims should be accorded the broadest interpretation so as to encompass all such modifications and similar arrangements.

Claims

1. A method of reformed coding, wherein a reference block is reformed first, and then the reformed block is used to predict the current block.

2. The method as claimed in claim 1, wherein the reference block can be in the same picture of the current block, or it can be in a different picture of the current block.

3. The method as claimed in claim 1, wherein a reference block is reformed arbitrarily, suppose R(x, y) is a sample in the reference block, then a sample R'(x,y) in the reformed reference block can be derived as R'(x,y)=R( ( x, y ), g( x, y )), where / and g are any functions.

4. The method as claimed in claim 3, wherein Function / can depend on x only, on y only, or on both; function g can depend on x only, on y only, or on both; function / can depend on the size, width, height, shape or coding mode of the current block or the reference block; function g can depend on the size, width, height, shape or coding mode of the current block or the reference block; function / can depend on the type of the current or reference picture or slice; function g can depend on the type of the current or reference picture or slic; function / can be a linear or non-linear function; function g can be a linear or non-linear function. / and g can be the same, or they can be different.

5. The method as claimed in claim 1, wherein a reference block can be reformed by but not limited to: horizontal flipping, vertical flipping, clock wise rotation, counter clock wise rotation, transposition.

6. The method as claimed in claim 1, wherein a reference block is reformed by a combination of two or more reforming methods, the reforms can be done in a cascade way, in which a reference block is reformed sequentially before it is used predict the current block, the output from a former reform is treated as the input to a subsequent reform.

7. The method as claimed in claim 1, wherein an encoder can decide to apply reformed copying or not for a block and signal the information to the decoder, the information can be signaled for a macro-block (MB), a coding tree unit (CTU) a coding unit (CU), a transform unit (TU) or a prediction unit (PU), the encoder can make the decision based on the rate-distortion optimization (RDO) criterion or other criterions.

8. The method as claimed in claim 1, wherein an encoder can decide to apply which kind of reformed copying for a block and signal the information to the decoder, the information can be signaled for a macro-block (MB), a coding tree unit (CTU) a coding unit (CU), a transform unit (TU) or a prediction unit (PU), the encoder can make the decision based on the rate-distortion optimization (RDO) criterion or other criterions.

9. The method as claimed in claim 1, wherein a decoder can infer whether to apply reformed copying or not for a block implicitly; no information is signaled to indicate whether to apply reformed copying or not from the encoder explicitly.

10. The method as claimed in claim 1, wherein a decoder can infer to apply which kind of reformed copying for a block implicitly; no information is signaled to indicate which kind of reformed copying is applied from the encoder explicitly.

11. The method as claimed in claim 1, wherein the reformed copying methods can be applied to inter block copying, or motion compensation.

12. The method as claimed in claim 1, wherein the reformed copying methods can be applied to intra block copying.

13. The method as claimed in claim 1, wherein the reformed copying methods can be applied to a block with size M X N, where M and N are arbitrary positive integers. M can be equal to N or M can be not equal to N. For example, M can be equal to a*N where a is a positive integer equal or larger than 2. N can be equal to b*M, where b is a positive integer equal or larger than 2. Specially, M or N can be 1.

14. The method as claimed in claim 1, wherein the reformed copying methods can be applied to all components in an image. For example, it can be applied to Y, U, or V components. It can also be applied to G, R, or B components. It can also be applied to depth component for 3D video/image coding.

15. The method as claimed in claim 1, wherein it depends on the size, shape, coding mode, MV information or BV information of the current block, the current CU, the current PU or the current TU to decide or infer whether to apply reformed copying or not.

16. The method as claimed in claim 1, wherein it depends on the size, shape, coding mode, MV information or BV information of the current block, the current CU, the current PU or the current TU to decide or infer which kind of reformed copying is applied.

17. The method as claimed in claim 1, wherein the reference block is horizontally flipped before it is used for prediction if reformed copying is used in the current block and the reference block and the current block possess the same vertical position. In other words, the y component of the BV for the current block is 0.

18. The method as claimed in claim 1, wherein the reference block is vertically flipped before it is used for prediction if reformed copying is used in the current block and the reference block and the current block possess the same horizontal position. In other words, the x component of the BV for the current block is 0.

19. The method as claimed in claim 1, wherein reformed copying is allowed only when one or any logical combination of the conditions below are true; reformed copying can be used only if it is allowed; the information about reformed copying for a block is only signaled if reformed copying is allowed;

the current CU is coded with intra block copying mode;

the current CU is coded with inter mode;

the current CU is coded with partition mode 2N X 2N;

the current CU is coded with a size smaller than MxM, M can be any integer such as 64, 32, 16, 8, and 4;

the current PU is coded with a size smaller than MxM, M can be any integer such as 64, 32, 16, 8, and 4;

the current TU is coded with a size smaller than MxM, M can be any integer such as 64, 32, 16, 8, and 4;

the current CU is coded with a size larger than MxM, M can be any integer such as 64, 32, 16, 8, and 4;

the current PU is coded with a size larger than MxM, M can be any integer such as 64, 32, 16, 8, and 4;

the current TU is coded with a size larger than MxM, M can be any integer such as 64, 32, 16, 8, and 4;

the current PU size is 2Nx2N;

the current PU size is NxN;

the current PU size is 2NxN;

the current PU size is Nx2N;

the current BV possess the form of (x, 0);

the current BV possess the form of (0, y).

20. The method as claimed in claim 1, wherein a flag is coded by context adaptive binarized arithmetic coding (CABAC) for a CU, TU or PU to indicate whether reformed copying is used for the current block.

21. The method as claimed in claim 20, wherein the flags in its neighboring blocks are used to determine the context model to code the flag for the current block.

22. The method as claimed in claim 20, wherein the flag is treated as 0 if it is not signaled, or it is treated as 1 if it is not signaled.

23. The method as claimed in claim 1, wherein the BVs or MVs for the current block are coded in the same way as the case where reformed copying is not used when reformed copying is used for the current block.

24. The method as claimed in claim 1, wherein the BVs or MVs for the current block are coded a different way to the case where reformed copying is not used when reformed copying is used for the current block.