WO2015154222A1 - Improved methods for background residual prediction - Google Patents
Improved methods for background residual prediction Download PDFInfo
- Publication number
- WO2015154222A1 WO2015154222A1 PCT/CN2014/074900 CN2014074900W WO2015154222A1 WO 2015154222 A1 WO2015154222 A1 WO 2015154222A1 CN 2014074900 W CN2014074900 W CN 2014074900W WO 2015154222 A1 WO2015154222 A1 WO 2015154222A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- brp
- weight factor
- utilize
- cus
- predictor
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/103—Selection of coding mode or of prediction mode
- H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/20—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding
- H04N19/23—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/46—Embedding additional information in the video signal during the compression process
Definitions
- the invention relates to generally video processing.
- the present invention relates to optimized methods of background residual prediction used for high-efficiency coding of general, surveillance, teleconferencing and screen content videos.
- Background residual prediction is developed for encoding or decoding data of video pictures which have similar background data. It can be summarized as one kind of prediction which utilizes static background to achieve more efficient prediction.
- BRP background residual prediction
- BRP mainly works for blocks which have both foreground and background pixels, where CurrentMarix is different from BgcMatrix but (CurrentMarix - BgcMatrix) is similar to (RefMarix - BgrMatrix).
- the current BRP is signalled in CU level for all non-skip inter CUs which are larger than 8x8, no whether what sizes of prediction units (PUs) the current CU is predicted by.
- PUs prediction units
- the smaller PU sizes there is low probability for the blocks where foreground and background pixels co-exist (namely hybrid blocks), but BRP is actually suitable for these hybrid blocks.
- weight factor is proposed to apply to the BRP procedure, and also, BRP is only available for CUs with limited types or sizes.
- Fig. 1 is a diagram illustrating the principle concepts of background residual prediction.
- the flag utilized to signalize BRP is transmitted for each PU, when the PU size or CU type is in one available set.
- An example set of available PUs for BRP are ⁇ 2Nx2N, 2NxN, Nx2N ⁇ .
- the flag utilized to signalize BRP is transmitted in CU level when the PU size or CU type is in one available set. In such case, all the PUs of the current CU enables or disables BRP at the same time.
- An example set of available PUs or CUs for BRP are with PU types of ⁇ 2Nx2N, 2NxN, Nx2N ⁇ .
- a weight factor is utilized to multiply the (RefMarix - BgrMatrix), and the result is utilized as the predictor for (CurrentMarix - BgcMatrix).
- the weight factor is binarized into a certain range for an efficient signalization at sequence, picture, slice, CU or PU level.
- the weight factors include 0.5 and 1, which are signalled by an additional flag after the BRP mode flag in CU level.
- the flag utilized to signalize BRP is transmitted for each CU, when there are coefficients for the current CU to decode. For example, the cbp or ctp flag of the current CU is not zero.
- the weight factor can also be utilized to only multiply the chroma component of (RefMarix - BgrMatrix) to generate the modified predictor of chroma components, when the luma components are equal to that of (RefMarix - BgrMatrix).
- an embodiment of the present invention can be a circuit integrated into a video compression chip or program codes integrated into video compression software to perform the processing described herein.
- An embodiment of the present invention may also be program codes to be executed on a Digital Signal Processor (DSP) to perform the processing described herein.
- DSP Digital Signal Processor
- the invention may also involve a number of functions to be performed by a computer processor, a digital signal processor, a microprocessor, or field programmable gate array (FPGA).
- processors can be configured to perform particular tasks according to the invention, by executing machine-readable software code or firmware code that defines the particular methods embodied by the invention.
- the software code or firmware codes may be developed in different programming languages and different format or style.
- the software code may also be compiled for different target platform.
- different code formats, styles and languages of software codes and other means of configuring code to perform the tasks in accordance with the invention will not depart from the spirit and scope of the invention.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
A method of background residual prediction for general, surveillance, conferencing and screen content video coding are disclosed. The background residual prediction is optimized by utilizing a weight factor to produce more accurate predictors for the current block. Besides, the mode flag for BRP and the weight factor are efficiently transmitted.
Description
IMPROVED METHODS FOR BACKGROUND RESIDUAL
PREDICTION
TECHNICAL FIELD
[0001] The invention relates to generally video processing. In particular, the present invention relates to optimized methods of background residual prediction used for high-efficiency coding of general, surveillance, teleconferencing and screen content videos.
BACKGROUND [0002] Background residual prediction is developed for encoding or decoding data of video pictures which have similar background data. It can be summarized as one kind of prediction which utilizes static background to achieve more efficient prediction.
[0003] For the videos which are captured by relative static cameras or cameras having seldom shot cuts (e.g., surveillance, teleconferencing and screen content videos), background residual prediction (BRP) is a suitable prediction method. This is because it further removes the redundancy of similar background residual between the current block and its reference block. In detail, as shown in Fig. 1, the current block CurrentMarix and its reference block RefMatrix may have similar residuals after subtracting their corresponding background data BgcMatrix and BgrMatrix. One typical video coding standard adopting BRP is AVS2. BRP mainly works for blocks which have both foreground and background pixels, where CurrentMarix is different from BgcMatrix but (CurrentMarix - BgcMatrix) is similar to (RefMarix - BgrMatrix).
[0004] In the current BRP, (CurrentMarix - BgcMatrix) is directly predicted by (RefMarix - BgrMatrix), which means CurrentMatrix is predicted by BgcMatrix plus (RefMarix - BgrMatrix). However for some blocks, (RefMarix - BgrMatrix) multiplied by a weight factor will be more sufficient than (RefMarix - BgrMatrix) itself to predict (CurrentMarix - BgcMatrix).
[0005] Besides, the current BRP is signalled in CU level for all non-skip inter CUs
which are larger than 8x8, no whether what sizes of prediction units (PUs) the current CU is predicted by. However, for the smaller PU sizes, there is low probability for the blocks where foreground and background pixels co-exist (namely hybrid blocks), but BRP is actually suitable for these hybrid blocks.
SUMMARY
[0006] It is proposed to improve the prediction efficiency of the "hybrid-foreground- and-background blocks" by optimizing background residual prediction.
[0007] In this invention, weight factor is proposed to apply to the BRP procedure, and also, BRP is only available for CUs with limited types or sizes.
[0008] Other aspects and features of the invention will become apparent to those with ordinary skill in the art upon review of the following descriptions of specific embodiments.
BRIEF DESCRIPTION OF DRAWINGS
[0009] The invention can be more fully understood by reading the subsequent detailed description and examples with references made to the accompanying drawings, wherein:
[0010] Fig. 1 is a diagram illustrating the principle concepts of background residual prediction.
DETAILED DESCRIPTION
[0011] The following description is of the best-contemplated mode of carrying out the invention. This description is made for the purpose of illustrating the general principles of the invention and should not be taken in a limiting sense. The scope of the invention is best determined by reference to the appended claims.
[0012] Several methods for BRP are proposed to optimize its signalization or improve its prediction efficiency.
[0013] In embodiment, the flag utilized to signalize BRP is transmitted for each PU, when the PU size or CU type is in one available set. An example set of available PUs for BRP are {2Nx2N, 2NxN, Nx2N}.
[0014] In 2nd embodiment, the flag utilized to signalize BRP is transmitted in CU level when the PU size or CU type is in one available set. In such case, all the PUs of the current CU enables or disables BRP at the same time. An example set of available PUs or CUs for BRP are with PU types of {2Nx2N, 2NxN, Nx2N}.
[0015] In 3rd embodiment, a weight factor is utilized to multiply the (RefMarix - BgrMatrix), and the result is utilized as the predictor for (CurrentMarix - BgcMatrix). The weight factor is binarized into a certain range for an efficient signalization at sequence, picture, slice, CU or PU level. For example, the weight factors include 0.5 and 1, which are signalled by an additional flag after the BRP mode flag in CU level.
[0016] In 4th embodiment, the flag utilized to signalize BRP is transmitted for each CU, when there are coefficients for the current CU to decode. For example, the cbp or ctp flag of the current CU is not zero.
[0017] In other embodiments, arbitrary combinations of 1st, 2nd, 3rd and 4th embodiments are included. Note that, the weight factor can also be utilized to only multiply the chroma component of (RefMarix - BgrMatrix) to generate the modified predictor of chroma components, when the luma components are equal to that of (RefMarix - BgrMatrix).
[0018] The proposed method described above can be used in a video encoder as well as in a video decoder. Embodiments of the method according to the present invention as described above may be implemented in various hardware, software codes, or a combination of both. For example, an embodiment of the present invention can be a circuit integrated into a video compression chip or program codes integrated into video compression software to perform the processing described herein. An embodiment of the present invention may also be program codes to be executed on a Digital Signal Processor (DSP) to perform the processing described herein. The invention may also involve a number of functions to be performed by a computer processor, a digital signal processor, a microprocessor, or field programmable gate array (FPGA). These processors can be configured to perform particular tasks according to the invention, by executing machine-readable software code or firmware code that defines the particular methods embodied by the invention. The software code or firmware codes may be developed in different programming languages and
different format or style. The software code may also be compiled for different target platform. However, different code formats, styles and languages of software codes and other means of configuring code to perform the tasks in accordance with the invention will not depart from the spirit and scope of the invention.
[0019] The invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described examples are to be considered in all respects only as illustrative and not restrictive. To the contrary, it is intended to cover various modifications and similar arrangements (as would be apparent to those skilled in the art). Therefore, the scope of the appended claims should be accorded the broadest interpretation so as to encompass all such modifications and similar arrangements.
Claims
1. A method of background residual prediction (BRP) for general video, surveillance video and conferencing video coding, wherein the background residual prediction is selectively applied according to a constraint.
2. The method as claimed in claim 1, wherein the background residual prediction is only applied to a limited set of prediction unit (PU) sizes.
3. The method as claimed in claim 1, wherein the background residual prediction is only applied to limited types of coding units (CUs).
4. The method as claimed in claim 1, wherein the background residual prediction is only applied to limited types of pictures.
5. The method as claimed in claim 1, wherein weight factors is utilized to generate an improved predictor for (CurrentMarix - BgcMatrix).
6. The method as claimed in claim 2, wherein the set of PUs which utilize BRP does not include 2NxnU, 2NxnD, nLx2N or nRx2N.
7. The method as claimed in claim 2, wherein the set of PUs which utilize BRP does not include NxN PU.
8. The method as claimed in claim 2, wherein the set of PUs which utilize BRP does not include 2NxN or Nx2N PUs.
9. The method as claimed in claim 2, wherein the set of PUs which utilize BRP does not include PUs whose shorter edge length is smaller than W, and W is 8, 16 or 32.
10. The method as claimed in claim 3, wherein the set of CUs which utilize BRP does not include intra CUs.
11. The method as claimed in claim 3, wherein the set of CUs which utilize BRP does not include the CUs which have no coefficients to decode.
12. The method as claimed in claim 3, wherein the set of CUs which utilize BRP does not include CUs smaller than MxM size, and M is 8, 16 or 32.
13. The method as claimed in claim 3, wherein the set of CUs which utilize BRP does not include CUs coded by direct, skip or merge modes.
14. The method as claimed in claim 4, wherein the set of pictures which utilize
BRP does not include picture with less than K reference pictures, and K can be 2, 3 or 4.
15. The method as claimed in claim 4, wherein the set of pictures which utilize BRP does not include picture without using background picture as reference.
16. The method as claimed in claim 5, wherein the weight factor is only utilized only for better luma-component predictor of (RefMarix - BgrMatrix).
17. The method as claimed in claim 5, wherein the weight factor is only utilized only for better chroma-component predictor of (RefMarix - BgrMatrix).
18. The method as claimed in claim 5, wherein the improved predictor is produced by adding or multiplying the weight factor to the corresponding component of the original predictor (RefMarix - BgrMatrix).
19. The method as claimed in claim 5, wherein the improved predictor is produced by dividing or subtracting the weight factor from the corresponding component of the original predictor (RefMarix - BgrMatrix).
20. The method as claimed in claim 5, wherein the improved predictor is guaranteed in the range of [0, (l«bitDepth)-l], where bitDepth is the internal bit- depth of the current encoder or decoder.
21. The method as claimed in claim 5, wherein the weight factor is binaried after mapping into minV to maxV, where minV and maxV are the minimum and largest parameter of a new symbol indentifying the weight factor.
22. The method as claimed in claim 5, wherein the weight factor values include 0.5 and 1, and the new predictor is produced by multiplying the weight factor to the corresponding component of the original predictor (RefMarix - BgrMatrix).
23. The method as claimed in claim 5, wherein the weight factor is binaried after mapping into 0 to 1.
24. The method as claimed in claim 5, wherein the weight factor is signalized in sequence, picture, slice, coding tree unit, CU or PU level.
25. The method as claimed in claim 5, wherein the weight factor is signalized rightly after the BRP mode flag.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2014/074900 WO2015154222A1 (en) | 2014-04-08 | 2014-04-08 | Improved methods for background residual prediction |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2014/074900 WO2015154222A1 (en) | 2014-04-08 | 2014-04-08 | Improved methods for background residual prediction |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2015154222A1 true WO2015154222A1 (en) | 2015-10-15 |
Family
ID=54287083
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2014/074900 Ceased WO2015154222A1 (en) | 2014-04-08 | 2014-04-08 | Improved methods for background residual prediction |
Country Status (1)
| Country | Link |
|---|---|
| WO (1) | WO2015154222A1 (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2006129184A1 (en) * | 2005-06-03 | 2006-12-07 | Nokia Corporation | Residual prediction mode in scalable video coding |
| EP2373049A1 (en) * | 2010-03-31 | 2011-10-05 | British Telecommunications Public Limited Company | Video quality measurement |
-
2014
- 2014-04-08 WO PCT/CN2014/074900 patent/WO2015154222A1/en not_active Ceased
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2006129184A1 (en) * | 2005-06-03 | 2006-12-07 | Nokia Corporation | Residual prediction mode in scalable video coding |
| EP2373049A1 (en) * | 2010-03-31 | 2011-10-05 | British Telecommunications Public Limited Company | Video quality measurement |
Non-Patent Citations (2)
| Title |
|---|
| ZHANG, XIANGUO ET AL.: "Background-Modeling-Based Adaptive Prediction for Surveillance Video Coding", IEEE TRANSACTIONS ON IMAGE PROCESSING, vol. 23, no. 2, 28 February 2014 (2014-02-28), pages 769 - 782, XP011536818 * |
| ZHANG, XIANGUO.: "Surveillance Video Coding with Background Modeling", CHINESE DO CTORAL DISSERTATIONS FULL-TEXT DATABASE INFORMATION SCIENCE AND TECHNOLOGY, 15 October 2013 (2013-10-15), pages 81 - 85 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| AU2012286432B2 (en) | Method and apparatus for compressing coding unit in high efficiency video coding | |
| US10531122B2 (en) | Method and apparatus for coded block flag coding in high efficiency video coding | |
| AU2011334448B2 (en) | Apparatus and method of constrained partition size for high efficiency video coding | |
| US10812806B2 (en) | Method and apparatus of localized luma prediction mode inheritance for chroma prediction in video coding | |
| US10390050B2 (en) | Method of video coding using separate coding tree for luma and chroma | |
| US20180352226A1 (en) | Method and apparatus of block partition with smallest block size in video coding | |
| WO2015100522A1 (en) | Methods for inter-component residual prediction | |
| CN107836117A (en) | Method for Conditional Binary Tree Block Partitioning Structures for Video and Image Codecs | |
| WO2015180014A1 (en) | An improved merge candidate list construction method for intra block copy | |
| WO2015085575A1 (en) | Methods for background residual prediction | |
| EP2727363A1 (en) | Method and apparatus of intra mode coding | |
| WO2016003819A1 (en) | Method for motion vector difference (mvd) coding of screen content video data | |
| WO2015062002A1 (en) | Methods for sub-pu level prediction | |
| EP3881534B1 (en) | Method and apparatus of luma-chroma separated coding tree coding with constraints | |
| WO2013067687A1 (en) | Residue quad tree depth for chroma components | |
| WO2015100731A1 (en) | Methods for determining the prediction partitions | |
| WO2015135175A1 (en) | Simplified depth based block partitioning method | |
| WO2013102299A1 (en) | Residue quad tree depth for chroma components | |
| WO2016115736A1 (en) | Additional intra prediction modes using cross-chroma-component prediction | |
| WO2017035833A1 (en) | Neighboring-derived prediction offset (npo) | |
| WO2015154222A1 (en) | Improved methods for background residual prediction | |
| WO2016123801A1 (en) | Methods for partition mode coding | |
| AU2015255238B2 (en) | Method and apparatus for compressing coding unit in high efficiency video coding | |
| WO2016165122A1 (en) | Inter prediction offset | |
| WO2015196424A1 (en) | Methods for the transmission of flags related to dbbp |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 14888647 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 14888647 Country of ref document: EP Kind code of ref document: A1 |