CN109889829B - Fast sample adaptive compensation for 360 degree video - Google Patents

Abstract

A method for SAO for 360 degree video in High Efficiency Video Coding (HEVC), comprising: performing ERP projection on the 360-degree video to obtain an ERP projection video; performing intra-frame prediction or inter-frame prediction on a CTU of a current frame in the ERP projection video to determine an optimal RD-cost; comparing the RD-cost of the CTU to a threshold to determine whether to perform SAO, wherein the threshold is determined based at least in part on a quantization parameter for the ERP projected video and an ERP projection weight, and wherein the ERP projection weight is determined based at least in part on a number of CTUs in an altitude of the ERP projected video and a location of the CTUs in a current frame of the ERP projected video.

Description

Fast sample adaptive compensation for 360 degree video

Technical Field

The present invention relates to the field of image and video processing, and more particularly to fast sample adaptive compensation (SAO) for 360 degree video in High Efficiency Video Coding (HEVC).

Background

In 4 months 2010, two international Video coding standards organizations VCEG and MPEG established Video compression joint group JCT-vc (joint Video coding), which together develop a high efficiency Video coding hevc (high efficiency Video coding) standard, also known as h.265. The main objective of the HEVC standard is to achieve a large increase in coding efficiency with the previous generation standard h.264/AVC, especially for high resolution video sequences. The goal is to reduce the code rate to 50% of the h.264 standard at the same video quality (PSNR).

At this stage, HEVC still continues to use the hybrid coding framework that h.264 just started to adopt. Inter and intra prediction coding: the correlation between the time domain and the spatial domain is eliminated. Transform coding: the residual is transform coded to remove spatial correlation. Entropy coding: eliminating statistical redundancy. HEVC will focus on research of new coding tools or techniques within the framework of hybrid coding to improve video compression efficiency.

The new characteristics of many of the encodings that have been proposed in the discussion of the JCT-VC organization at present may be added to the HEVC standard, and the specific literature in each discussion may be found in http: // wftp3.itu. int.

The first edition of HEVC standard [4] has been completed in january of 2013. And 3 versions released in succession at months 4 in 2013, 10 in 2014 and 4 in 2015, which can be easily obtained from the network, and the present application incorporates the three versions of the HEVC standard described above in the present specification as background for the present invention.

In HEVC, since the block-based hybrid coding framework is still used, there is still a need to deal with blocking effects, ringing effects, and so on. To reduce the impact of such distortion on video quality, HEVC employs In-loop filtering techniques, which include Deblocking filtering (Deblocking filtering) and pixel sample Adaptive compensation (SAO). SAO is one of many new technologies for HEVC [5 ]. As shown in fig. 1, the SAO is located after the deblocking filter. The SAO classifies and counts each pixel of each Coding Tree Unit (CTU), calculates a compensation value, selects an optimal SAO type, and writes the SAO type and the compensation value into a code stream. An offset value is then added to each pixel of the reconstructed frame to reduce distortion between the reconstructed frame and the original frame. SAO can significantly improve subjective and objective video quality [5 ]. SAO is mainly composed of three parts: statistics collection, SAO type decision and SAO filtering, as shown in fig. 2.

And (3) statistics collection: there are two main types of offsets for SAO that require a statistical collection process: boundary compensation (EO) and sideband compensation (BO). For the EO type, there are four EO subtypes (EO 0, EO 90, EO 135 and EO 45). Each EO subtype is classified according to classification rules and the number of pixels in each class and the distortion sum are calculated. For the BO type, the pixel intensity is equally divided into 32 sidebands, and the 32 sidebands are classified according to the classification rule, and the number of pixels in each class and the distortion sum are calculated. The classification rules for EO and BO are shown in fig. 2.

SAO type decision: four SAO types can be selected: EO, BO, OFF and MERGE, where OFF indicates that SAO is not applied, and is implemented by a switch parameter in a video stream, and MERGE indicates that for a block, the SAO parameter directly uses the SAO of the block above or on the left, and it only needs to identify which neighboring block SAO parameter is used. Based on the statistically collected information, the SAO type decision calculates an optimal compensation value for each SAO type through a fast Rate Distortion Optimization (RDO) process [6], and selects an optimal SAO type.

SAO filtering: and classifying and compensating each pixel of the CTU according to the obtained optimal SAO type and offset value.

Fig. 2 shows that the SAO process consists of three parts: statistics collection, SAO type decision and SAO filtering. [16] The computational complexity of each part was studied. The results show that statistical collection accounts for about 82% of the total SAO processing time, with the SAO type decision and SAO filtering being 11% and 7%, respectively. The complex statistical collection process is a major factor that limits the speed of SAO processing.

In a virtual reality system, multiple cameras are used to capture 360 degree scenes, and the subsequently captured scenes are stitched into a 360 degree video in a spherical format. The user can freely view any scene display (HMD) in a 360 degree scene through the head mounted device and get an immersive experience [1 ]. 360 degree video is a new type of video coding content. Although 360-degree video is popular after the HEVC standard is proposed and 360-degree video is spherical video, [2] a 360-degree video coding framework under the HEVC standard has been proposed. In a typical 360-degree video compression framework, spherical video needs to be converted to flat video before encoding, and flat video needs to be converted to spherical video after encoding [3 ]. The conversion method is called projection. Various projection formats have been proposed, such as Equal Rectangular Projection (ERP), adjusted equal area projection (AEP), cube projection (CMP), equiangular cube projection (EAC), truncated square pyramid projection (TSP), compact octahedral projection (COHP), compact icosahedral projection (CISP), and the like. When ERP is selected as the projection format, the encoding process of the 360-degree video includes: and projecting the original video into an ERP projection format, performing coding and decoding on the ERP projection video, and re-back-projecting the reconstructed video in the ERP projection format into a reconstructed video. The projection process is essential for 360 degree video coding. The projection format, which is an intermediate format, affects the encoding performance of 360-degree video. In fact, it has not been determined which projection format has the best coding performance. However, ERP is widely used and is the default format for 360 degree video. Therefore, the characteristics of the ERP projection format are mainly studied herein.

Compared with a flat video, a 360-degree video has different characteristics, and the optimal parameters and processes of the existing SAO fast algorithm are not suitable for the 360-degree video. In the application, a fast SAO algorithm for a 360-degree video is provided based on the characteristics of the 360-degree video.

This application is an improvement over the existing HEVC protocol and, in order to enable those skilled in the art to fully understand the present invention, references to various concepts mentioned in this application are attached below and are incorporated herein in their entirety and as part of the specification of this application.

1.B.Luo，F.Xu，C.Richardt and J.Yong，″Parallax360：Stereoscopic 360°Scene Representation for Head-Motion Parallax，″in IEEE Transactions on Visualization and Computer Graphics，vol.24，no.4，pp.1545-1553，April 2018.

2.Y.Y，E.Alshina，J.Boyce，“Algorithm descriptions of projection format conversion and video quality metrics in 360Lib”，Joint Video Exploration Team of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11，JVET-H1004，7th Meeting，July 2017.

3.W.Zou，F.Yang and S.Wan，″Perceptual video quality metric for compression artefacts：from two-dimensional to omnidirectional，″in IET Inage Processing，vol.12，no.3，pp.374-381，3 2018.

4.Sullivan，Gary J.，et al.″Overview of the high efficiency video coding(HEVC)standard.″Circuits and Systems for Video Technology，IEEE Transactions on 22.12(2012)：1649-1668.

5.C.-M.Fu，E.Alshina，A.Alshin，Y.-W.Huang，C.-Y.Chen，C.-Y.Tsai，C.-W.Hsu，S.-M.Lei，J.-H.Park，and W.-J.Han，”Sample adaptive offset in the HEVC standard，”Circuits and Systems for Video Technology，IEEE Transactions on，vol.22，no.12，pp.1755-1764，2012.

6.Zhang M，Bai H，Lin C，et al.Texture Characteristics Based Fast Coding Unit Partition in HEVC Intra Coding，Data Compression Conference.IEEE，2015：477-477.

7.Z.Zhengyong，C.Zhiyun and P.Peng，″A fast SAO algorithm based on coding unit partition for HEVC，″2015 6th IEEE International Conference on Software Engineering and Service Science(ICSESS)，Beijing，2015，pp.392-395.

8.J.Joo，Y.Choi and K.Lee，″Fast sample adaptive offset encoding algorithm for HEVC based on intra prediction mode，″2013 IEEE Third International Conference on Consumer Electronics Berlin(ICCE-Berlin)，Berlin，2013，pp.50-53.

9.T.Y.Kuo，H.Chiu and F.Amirul，″Fast sample adaptive offset encoding for HEVC，″2016 IEEE International Conference on Consumer Electronics-Taiwan(ICCE-TW)，Nantou，2016，pp.1-2.

10.S.Yin，X.Zhang and Z.Gao，″Efficient SAO coding algorithm for x265 encoder，″2015 Visual Communications and Image Processing(VCIP)，Singapore，2015，pp.1-4.

11.S.E.Gendy，A.Shalaby and M.S.Sayed，″Fast parameter estimation algorithm for sample adaptive offset in HEVC encoder，″2015 Visual Communications and Image Processing(VCIP)，Singapore，2015，pp.1-4.

12.K.Yang，S.Wan，Y.Gong，Y.Yang and Y.Feng，″Fast sample adaptive offset for H.265/HEVC based on temporal dependency，″2016 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference(APSIPA)，Jeju，2016，pp.1-4.

13.Sungjei Kim，Jinwoo Jeong，Jeong-Mee Moon and Yong-HWan Kim，″Fast sample adaptive offset parameter estimation algorithm based on early termination for HEVC encoder，″2017 IEEE International Conference on Consumer Electronics(ICCE)，Las Vegas，NV，2017，pp.241-242.

14.W.Zhang and C.Guo，″Design and implementation of parallel algorithms for sample adaptive offset in HEVC based on GPU，″2016 Sixth International Conference on Information Science and Technology(ICIST)，Dalian，2016，pp.181-187.

15.Y.Wang，X.Guo，Y.Lu，X.Fan and D.Zhao，″GPU-based optimization for sample adaptive offset in HEVC，″2016 IEEE International Conference on Image Processing(ICIP)，Phoenix，AZ，2016，pp.829-833.

16.Y.Choi and J.Joo，″Exploration of Practical HEVC/H.265 Sample Adaptive offset Encoding Policies，″in IEEE Signal Processing Letters，vol.22，no.4，pp.465-468，April 2015.

17.Y.Li，J.Xu and Z.Chen，″Spherical domain rate-distortion optimization for 360-degree video coding，″2017 IEEE International Conference on Multimedia and Expo(ICME)，Hong Kong，2017，pp.709-714.

18.Y.Sun and L.Yu，″Coding optimization based on weighted-to-spherically-uniform quality metric for 360 video，″2017 IEEE Visual Communications and Image Processing(VCIP)，St.Petersburg，FL，2017，pp.1-4.

19.Jill Boyce，Elena Alshina，Adeel Abbas，“JVET common test conditions and evaluation procedures for 360°video”，Joint Video Exploration Team(JVET)of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11，JVET-H1030，8th Meeting，Oct.2017.

20.X.Xiu，Y.He，Y.Ye and B.Vishwanath，″An evaluation framework for 360-degtee video compression，″2017 IEEE Visual Communications and Image Processing(VCIP)，St.Petersburg，FL，2017，pp.1-4.

21.H.Bai，C.Zhu and Y.Zhao，″Optimized Multiple Description Lattice Vector Quantization for Wavelet Image Coding，″in IEEE Transactions on Circuits and Systems for Video Technology，vol.17，no.7，pp.912-917，July 2007.

22.C.Yeh，Z.Zhang，M.Chen and C.Lin，″HEVC Intra Frame Coding Based on Convolutional Neural Network，″in IEEE Access，vol.6，pp.50087-50095，2018.

23.L.Chang，Z.Liu，L.Wang and X.Li，″Enhance the HEVC Fast Intra CU Mode Decision Based on Convolutional Neural Network by Corner Power Estimation，″2018 Data Compression Conference，Snowbird，UT，2018，pp.400-400。

24.T.Katayama，K.Kuroda，W.Shi，T.Song and T.Shimamoto，″Low-complexity intra coding algorithm based on convolutional neural network for HEVC，″2018 International Conference on Information and Computer Technologies (ICICT)，DeKalb，IL，2018，pp.115-118.

25.G.Bjontegaard，“Calculation of average PSNR differences between RD-curves，”ITU-T VCEG-M33，2001.

Disclosure of Invention

The invention provides a fast SAO method for a 360-degree video aiming at the characteristics of the 360-degree video. The proposed algorithm improves the SAO process, adding a simplified SAO process on the basis of preserving the entire SAO process. After performing a decision by the SAO based on the threshold, a simplified SAO procedure can be used, which will greatly reduce the time for statistical data collection, thereby reducing the computational complexity of the SAO.

According to an aspect of the present invention, a method for adaptive sample offset (SAO) for 360 degree video in High Efficiency Video Coding (HEVC) is proposed, the method comprising:

performing projection on the 360-degree video to obtain a projected video;

performing intra-frame prediction or inter-frame prediction on a Coding Tree Unit (CTU) of a current frame in the ERP projection video to determine an optimal RD-cost;

comparing the RD-cost of the CTU with a threshold to determine whether to perform SAO,

wherein the threshold is determined based at least in part on a quantization parameter and a projection weight for the projection video, and wherein the projection weight is determined based at least in part on a number of CTUs in an altitude of the projection video and a location of the CTUs in a current frame of the projection video.

According to a further aspect of the invention, the method further comprises: performing at least no boundary compensation (EO) and sideband compensation (BO) on the CTU if it is determined not to perform SAO; if it is determined to perform SAO, one of OFF or MERGE operations are performed for the CTU.

According to further aspects of the invention, wherein the threshold is based at least in part on at least one of: a base-2 logarithm of the projection weight, or a power of e of the quantization parameter, or a combination thereof.

According to a further aspect of the present invention, wherein the RD-cost of the CTU is compared with a threshold to determine whether to perform SAO only for heights above 1/4 and below 1/4 in the projected video.

According to another aspect, a High Efficiency Video Coding (HEVC) hardware encoder adapted for adaptive sample offset (SAO) for 360 degree video is configured to perform the above method.

According to another aspect, the invention proposes a decoder for decoding a 360 video stream encoded using a method as described or an encoder as described.

According to another aspect, the invention proposes a computer program product for carrying out the above-mentioned method.

According to another aspect, the present invention provides an apparatus usable for video encoding and decoding, the apparatus comprising: one or more processors; a memory having stored therein computer code which, when executed by the processor, implements the above method.

According to another aspect, the projection is an Equal Rectangular Projection (ERP).

Drawings

Fig. 1 illustrates one embodiment of an encoder block diagram of HEVC.

Fig. 2 shows a brief block diagram of SAO in HEVC.

Fig. 3 shows a weight distribution of an ERP projection.

Fig. 4 illustrates a flow diagram of a method in accordance with various aspects of the present disclosure.

Fig. 5 illustrates a schematic diagram of an apparatus for video codec according to various aspects of the present disclosure.

Detailed Description

Various aspects are now described with reference to the drawings. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of one or more aspects. It may be evident, however, that such aspect(s) may be practiced without these specific details.

As used in this application, the terms "component," "module," "system," and the like are intended to refer to a computer-related entity, such as but not limited to hardware, firmware, a combination of hardware and software, or software in execution. For example, a component may be, but is not limited to: a process running on a processor, an object, an executable, a thread of execution, a program, and/or a computer. By way of illustration, both an application running on a computing device and the computing device can be a component. One or more components can reside within a process and/or thread of execution and a component may be localized on one computer and/or distributed between two or more computers. In addition, these components can execute from various computer readable media having various data structures stored thereon. The components may communicate by way of local and/or remote processes such as in accordance with a signal having one or more data packets, e.g., data from one component interacting with another component in a local system, distributed system, and/or across a network such as the internet with other systems by way of the signal.

The invention provides a fast SAO method for a 360-degree video aiming at the characteristics of the 360-degree video. The proposed algorithm improves the SAO process, adding a simplified SAO process on the basis of preserving the entire SAO process. After the decision is performed by the SAO based on the threshold, the conventional SAO process specified by HEVC may be selected to be performed, or the SAO may be selected not to be performed or only the MERGE process may be performed, thereby implementing a simplified SAO process, which may greatly reduce the time for collecting statistical data, thereby reducing the computational complexity of the SAO.

I. Algorithm overview

(1) Weight in 360 degree video (weight)

The 360-degree video is a spherical video and is the biggest difference between the 360-degree video and the traditional video. In order to encode 360-degree video under the HEVC standard, 360-degree video must be projected into planar video. Although both projection video and conventional video are flat video, projection video has the distortion and stretching of spherical video. Therefore, the PSNR, which is an objective quality evaluation index of the conventional video, is not suitable for the projection video. WS-PSNR was proposed as an objective quality assessment indicator for projection video [18 ]. And designing weights for the projection video, wherein the weights of the projection video in the distortion and stretching areas are smaller, and vice versa, and then calculating the WS-PSNR by a weighted average method. WS-PSNR is recognized by the Joint Video Exploration Team (JVET) as an objective quality assessment indicator for 360 degrees video quality. Thus, the weight is the largest difference between the projected video and the conventional video.

Where (i, j) represents the pixel location and height represents the height of the video. Fig. 3 shows ERP weight distribution. The darker the color, the closer to 0. The lighter the color, the closer to 1. Region0 is defined as the area near the two poles, and the weight is small; region1 is defined as the Region near the equator and is heavily weighted. As shown in fig. 3, Region0 includes the upper 1/4 and lower 1/4 regions of the video; region1 represents the middle 1/2 Region of the video.

(2)RD-cost

In HEVC, the best prediction modes and the best CU partitions for intra prediction and inter prediction in the HEVC standard are recursively calculated by Rate Distortion Optimization (RDO) [17 ].

J＝D+λ·R (2)

Where D represents the distortion in the current prediction mode, R represents the number of bits required to encode all information in the current prediction mode, λ is the lagrangian factor, and J represents the lagrangian cost (RD-cost). The smaller the RD-cost, the higher the coding efficiency of the prediction mode, and the larger the RD-cost, the lower the coding efficiency of the prediction mode.

(3) Proposed algorithm

In one embodiment of the present invention, a Threshold (Threshold) is set to determine in advance whether to perform the SAO process. The SAO process is performed when RD-cost > Threshold, otherwise, the SAO process is not performed. By skipping SAO processing based on the threshold value, the amount of encoding computation can be greatly reduced.

In one embodiment of the invention, the threshold for determining whether to perform the SAO process is determined based at least in part on the quantization parameter and the ERP projection weight for the 360 degree video.

In one embodiment of the invention, the quantized coefficients have an impact on the decision whether to perform SAO. According to experimental statistics, with the increase of the quantization coefficient, the probability that the CTU does not apply the SAO is increased; as the quantization coefficients increase, the RD-cost of the same CTU also increases. Therefore, considering the quantization coefficients in determining the threshold, the determination of whether to perform SAO can be more accurate with the set threshold.

In one exemplary but non-limiting embodiment, the power of e of the quantization parameter may be considered.

In one embodiment of the invention, the weights of the projected video have an impact on the decision whether to perform SAO. Since ERP projection is used herein, the weights are also referred to as ERP projection weights. In the study, the inventors noted that for the weights of ERP projection, the weights of CTUs in Region0 are close to 0, so the distortion of CTUs in Region0 has little impact on the final video quality. Therefore, the determination as to whether or not SAO is performed takes into account the characteristics of the projection video in which the weight of the projection video conforms to ERP projection, so that the determination does not cause degradation in the quality of the encoded video.

In one exemplary, but non-limiting embodiment of the invention, the inventors propose the following weight setting:

where (x, y) denotes the position of the CTU, m is the number of CTUs in the video height, and weight (x, y) denotes the average weight of all pixels of the CTU. Therefore, the weights of different positions are different, and the influence of the distortion of the CTU in Region1 on the video quality is much smaller than that of the distortion of the CTU in Region 0. Therefore, making a detailed decision on the CTUs in Region1 and a coarse decision on the CTUs in Region0 is a way to adapt to the characteristics of the projected video.

In one embodiment of the invention, the inventors modify the threshold according to the weight of different latitudes (corresponding to the y-value). The inventor uses

To represent scale factors for different latitudes. The larger the latitude, the larger the factor.

In one exemplary but non-limiting embodiment, threshold settings are given as shown in table 1 below.

TABLE 1 improved thresholding for ERP projection video at different QPs

Where α and 1- α represent the percentage of the fixed threshold and the variable threshold, respectively, and QP is the quantization parameter.

In one embodiment of the present invention, the inventors consider the power of e of the quantization parameter. For example, in another exemplary but non-limiting embodiment, the threshold may be determined as follows:

in one embodiment of the present invention, as described above, the inventors noted that the effect of distortion of the CTUs in Region0 on video quality is much smaller than the effect of distortion of the CTUs in Region1 on video quality. Therefore, in order to ensure video quality while reducing the amount of computation caused by SAO, the above operation may be performed only for regions 0 (i.e., the height of regions above 1/4 and below 1/4 of the current frame) in the ERP projection video, while SAO processing specified by the HEVC protocol is still performed for regions 1 (regions of the height of the middle 1/2 of the current frame).

In one embodiment of the present invention, since the calculation amount of the MERGE operation is small, when the above operation is performed, it may be considered that the MERGE operation is performed or the SAO operation (OFF operation) is not performed when a threshold condition (for example, less than a threshold) is satisfied.

Fig. 4 illustrates a flow diagram of a method in accordance with various aspects of the present disclosure. The method is used for sample adaptive compensation (SAO) for 360 degree video in High Efficiency Video Coding (HEVC).

According to one embodiment, the method comprises: performing equal-rectangular projection (ERP) on the 360-degree video to obtain an ERP projection video. Those skilled in the art will readily appreciate that other projection methods besides ERP may be performed, and that the particular projection method is not a concern of the present invention.

According to another embodiment, the method further comprises: performing intra-frame prediction or inter-frame prediction on a Coding Tree Unit (CTU) of a current frame in the ERP projection video to determine an optimal RD-cost.

According to another embodiment, the method further comprises: comparing the RD-cost of the CTU to a threshold to determine whether to perform SAO, wherein the threshold is determined based at least in part on a quantization parameter for an ERP projected video and an ERP projection weight, and wherein the ERP projection weight is determined based at least in part on a number of CTUs in an altitude of the ERP projected video and a location of the CTUs in a current frame of the ERP projected video.

According to another embodiment, if it is decided not to perform SAO, at least boundary compensation (EO) and sideband compensation (BO) are not performed on the CTU; and if it is determined to perform SAO, one of OFF or MERGE operations is performed for the CTU.

According to another embodiment, the threshold is based at least in part on at least one of: a base-2 logarithm of the ERP projection weight, or a power of e of the quantization parameter, or a combination thereof.

According to another embodiment, the RD-cost of the CTU is compared to a threshold to determine whether to perform SAO only for the heights of the top 1/4 and the bottom 1/4 in the ERP projection video.

Fig. 5 illustrates a schematic diagram of an apparatus for video codec according to various aspects of the present disclosure. As shown in fig. 5, the apparatus may include one or more processors and memory having stored therein computer code that, when executed by the processors, implements a method of adaptive sample offset (SAO) for 360 degree video in High Efficiency Video Coding (HEVC) as described herein.

According to another aspect, the present disclosure may also relate to an encoder for implementing the above-described encoding method. The encoder may be dedicated hardware.

According to another aspect, the present disclosure may also relate to a corresponding decoder that decodes the encoded 360 video stream.

According to another aspect, the present disclosure may also relate to a computer program product for performing the method described herein.

When implemented in hardware, the video encoder may be implemented or performed with a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. Additionally, at least one processor may include one or more modules operable to perform one or more of the steps and/or operations described above.

When the video encoder is implemented in hardware circuitry, such as an ASIC, FPGA, or the like, it may include various circuit blocks configured to perform various functions. Those skilled in the art can design and implement these circuits in various ways to achieve the various functions disclosed herein, depending on various constraints imposed on the overall system.

While the foregoing disclosure discusses illustrative aspects and/or embodiments, it should be noted that many changes and modifications could be made herein without departing from the scope of the described aspects and/or embodiments as defined by the appended claims. Furthermore, although elements of the described aspects and/or embodiments may be described or claimed in the singular, the plural is contemplated unless limitation to the singular is explicitly stated. Additionally, all or a portion of any aspect and/or embodiment may be utilized with all or a portion of any other aspect and/or embodiment, unless stated to the contrary.

Claims (6)

1. A method of adaptive sample compensation (SAO) for 360 degree video in High Efficiency Video Coding (HEVC), comprising:

performing projection on the 360-degree video to obtain a projected video;

performing intra-prediction or inter-prediction on a Coding Tree Unit (CTU) of a current frame in the projected video to determine an optimal RD-cost;

comparing the RD-cost of the CTU with a threshold to determine whether to perform SAO,

wherein the threshold is determined based at least in part on a quantization parameter and a projection weight for the projection video, and wherein the projection weight is determined based at least in part on a number of CTUs in an altitude of the projection video and a location of the CTUs in a current frame of the projection video.

2. The method of claim 1, further comprising:

performing at least no boundary compensation (EO) and sideband compensation (BO) on the CTU if it is determined not to perform SAO;

if it is determined to perform SAO, one of OFF or MERGE operations are performed for the CTU.

3. The method of claim 1 or 2, wherein the threshold is based at least in part on at least one of: a base-2 logarithm of the projection weight, or a power of e of the quantization parameter, or a combination thereof.

4. The method of claim 1 or 2, wherein the RD-cost of the CTU is compared to a threshold to determine whether to perform SAO only for heights above 1/4 and below 1/4 in the projected video.

5. The method of claim 1 or 2, wherein the projections are Equal Rectangular Projections (ERP).

6. An apparatus usable for video coding, the apparatus comprising:

one or more processors;

a memory having stored therein computer code that, when executed by the processor, implements a method of adaptive sample offset (SAO) for 360 degree video in High Efficiency Video Coding (HEVC), the method comprising:

performing projection on the 360-degree video to obtain a projected video;

performing intra-prediction or inter-prediction on a Coding Tree Unit (CTU) of a current frame in the projected video to determine an optimal RD-cost;

comparing the RD-cost of the CTU with a threshold to determine whether to perform SAO,

wherein the threshold is determined based at least in part on a quantization parameter and a projection weight for the projection video, and wherein the projection weight is determined based at least in part on a number of CTUs in an altitude of the projection video and a location of the CTUs in a current frame of the projection video.

Images