CN116866559A

CN116866559A - Method for simplifying adaptive loop filter in video coding

Info

Publication number: CN116866559A
Application number: CN202310786104.9A
Authority: CN
Inventors: 拉胡尔·瓦纳姆; 贺玉文; 叶艳
Original assignee: Vid Scale Inc
Current assignee: Vid Scale Inc
Priority date: 2017-11-01
Filing date: 2018-10-31
Publication date: 2023-10-10
Also published as: ZA202002084B; SG11202003927TA; JP7390300B2; US20230388553A1; KR20200095464A; EP3704858A1; WO2019089695A1; JP2024012656A; JP2021503862A; CN111512632A; US20200267381A1; CA3081252A1; CN111512632B; US11641488B2; CN117041536A

Abstract

Systems, methods, and instrumentalities are disclosed for adaptively selecting an Adaptive Loop Filter (ALF) procedure for a frame based on at which temporal layer the frame is. The ALF procedure may vary in computational complexity. One or more frames including the current frame may be in a temporal layer of the coding scheme. A decoder may determine a temporal level of the current frame within the coding scheme. The decoder may select an ALF procedure based on a temporal level of the current frame. If the temporal level of the current frame is higher than some other temporal levels within the coding scheme, a lower computational complexity ALF procedure may be selected for the current frame. The decoder may then perform the selected ALF procedure on the current frame.

Description

Method for simplifying adaptive loop filter in video coding

The present application is a divisional application of chinese patent application 201880081527.8 filed on 2018, 10, 31 entitled "method for simplifying adaptive loop filter in video coding".

Cross Reference to Related Applications

The application claims the benefit of U.S. provisional patent application 62/579,977 filed on 1 month 11 2017, U.S. provisional patent application 62/607,033 filed on 18 months 12 2017, the contents of which are incorporated herein by reference.

Background

Video coding systems are widely used to compress digital video signals to reduce the storage requirements and/or transmission bandwidth of such signals. Among various types of video coding systems, such as block-based, wavelet-based, and object-based systems, block-based hybrid video coding systems are probably the most widely used and deployed. Examples of block-based video coding systems include international video coding standards such as H.261, MPEG-1, MPEG-2, H.263, H.264/AVC, and H.265/HEVC.

Disclosure of Invention

Systems, methods, and instrumentalities are disclosed for adaptively selecting an Adaptive Loop Filter (ALF) procedure for a frame based on at which temporal layer the frame is. The computational complexity of the ALF procedure may vary.

The decoder may receive an indication of whether ALF is enabled. The decoder may apply the ALF to the current frame based on the indication that the ALF is enabled. The current frame may be in a temporal layer of the coding scheme. The coding scheme may include multiple temporal layers from a low temporal level to a high temporal level. The decoder may determine a temporal level of the current frame within the coding scheme.

The decoder may select an ALF procedure based on the temporal level of the current frame. If the temporal level of the current frame is higher than some other temporal levels within the coding scheme, a first ALF program having lower computational complexity than a second ALF program may be selected for the current frame. If the temporal level is lowest within the coding scheme, a second ALF procedure may be selected for the current frame. The decoder may then perform the selected ALF procedure on the current frame.

The current frame may include a current block having a plurality of pixels. The less computationally complex ALF process may classify blocks based on a subset of pixels. If the temporal level of the current frame is highest within the coding scheme, a subset of pixels may be selected from the pixels in the current block. One or more gradients may be calculated for the current block using the selected subset of pixels, and the current block may be classified for ALF based on the calculated gradients. A subset of pixels may be selected from the pixels in the current block by skipping at least one pixel in the current block.

For example, a subset of pixels may be selected from among pixels in the current block by skipping at least one pixel in the vertical direction and/or at least one pixel in the horizontal direction. If the temporal level of the current frame is highest within the coding scheme, a subset of pixels may be selected from the pixels in the current block by skipping one or more pixels in the vertical direction and one or more pixels in the horizontal direction. If the temporal level of the current frame is above the lowest level and below the highest level within the coding scheme, a subset of pixels may be selected from the pixels in the current block by skipping one or more pixels in the vertical direction, by skipping one or more pixels in the horizontal direction, or by skipping one or more pixels in the diagonal direction.

The ALF procedure may include a block classification based on each pixel of the current block. The ALF procedure may be more computationally complex than an ALF procedure that includes block classification based on a subset of pixels. The gradient of the current block may be calculated using each pixel in the current block, and the current block may be classified for ALF based on the calculated gradient.

The decoder may receive an indication that the subset of pixels is to be selected for calculating the gradient and/or an indication of how the subset of pixels is to be selected.

The less computationally complex ALF procedure may include block classifications that are calculated based on fewer gradients than the block classifications that are based on each pixel in the block. For example, the ALF program may include block classifications based on gradient calculations for a subset of pixels (e.g., based on a lesser number of gradient calculations than gradient calculations for all pixels). If the temporal level of the current frame is high within the coding scheme, the calculation of gradients in at least one of the vertical, horizontal, or diagonal directions of the pixels of the current block may be skipped. If the temporal level of the current frame is lowest within the coding scheme, a gradient in each of the vertical, horizontal, or diagonal directions of the pixels of the current block may be calculated.

Less computationally complex ALF procedures may include selectively skipping block classifications. If the temporal level of the current frame is higher than some temporal levels within the coding scheme, then the block classification may be skipped for the current frame. If the temporal level of the current frame is lowest within the coding scheme, block classification may be performed for the current frame.

The less computationally complex ALF procedure may conditionally skip ALF filtering. One or more gradients for the current block of the current frame may be calculated using the pixels of the current block. The sum of the gradients for the current block may be determined and then compared to a threshold to determine whether to disable ALF for the current block. If the sum of the plurality of gradients is less than the threshold, an ALF for the current block may be disabled, and the current block may be reconstructed if the ALF is disabled.

Drawings

Fig. 1 is an example block diagram of a block-based hybrid video encoder.

Fig. 2 is an example block diagram of a block-based hybrid video decoder.

Fig. 3 shows an example of ALF shape: a 5 x 5 diamond shape, (b) a 7 x 7 diamond shape, and (c) a 9 x 9 diamond shape.

Fig. 4 shows an example of an ALF procedure at the encoder.

Fig. 5 shows an example of gradient computation: (a) block classification, wherein for each 2 x 2 block, a gradient may be calculated for each pixel in a 6 x 6 window, (b) a window sub-sampled twice in the horizontal direction, (c) a window sub-sampled twice in the vertical direction, and (d) a window sub-sampled twice in the horizontal direction and sub-sampled twice in the vertical direction.

Fig. 6 shows an example of prefix code establishment for different sub-sampling (subsampling) schemes.

Fig. 7 shows an example of an ALF procedure of reduced computational complexity using frame adaptive ALF skipping.

FIG. 8 shows the sum g of the gradients used _sum And threshold T _G And an example of block classification for a 2 x 2 block using ALF on/off decision is employed.

Fig. 9 shows an example of applying ALF for a 2×2 block.

FIG. 10 shows training of threshold T using a histogram of (sum of) gradient values for a frame _G Is an example of (a).

Fig. 11 illustrates an exemplary rate-distortion (RD) based method for selecting between applying ALF and using pixel-adaptive ALF skipping with a skip percentage of "a".

FIG. 12 illustrates an exemplary RD cost based selection process when two skip percentages ("A" and "B") are specified for pixel adaptive ALF skipping.

Fig. 13 shows an example of resolving parameters associated with pixel adaptive ALF skipping.

Fig. 14A is a system diagram illustrating an example communication system in which one or more disclosed embodiments may be implemented.

Fig. 14B is a system diagram illustrating an example wireless transmit/receive unit (WTRU) that may be used within the communication system shown in fig. 14A.

Fig. 14C is a system diagram illustrating an exemplary Radio Access Network (RAN) and an exemplary Core Network (CN) that may be used within the communication system shown in fig. 14A.

Fig. 14D is a system diagram illustrating additional example RANs and additional example CNs that may be used within the communication system shown in fig. 14A.

Detailed Description

A detailed description of illustrative embodiments will now be described with reference to the accompanying drawings. Although the present description provides detailed examples of possible embodiments, it should be noted that these details are intended to be exemplary and in no way limit the scope of the application.

Fig. 1 shows an example of a block-based hybrid video coding system. The input video signal 302 may be processed block by block. For example, extended block sizes (e.g., coding Units (CUs)) may be used (e.g., in HEVC) to efficiently compress high resolution (e.g., 1080p and above) video signals. A CU may be, for example, 64 x 64 pixels. A CU may be partitioned (e.g., into Prediction Units (PUs)). Separate (e.g., the same or different) prediction processes may be applied to the PUs. For example, spatial prediction 360 and/or temporal prediction 362 may be performed on (e.g., each) input video block (e.g., a Macroblock (MB) or CU)).

Spatial prediction (e.g., intra prediction) may predict a current video block, for example, by using pixels from samples (e.g., reference samples) of coded neighboring blocks in the same video picture/slice. Spatial prediction may reduce the spatial redundancy inherent in video signals. Temporal prediction (e.g., inter prediction or motion compensated prediction) may predict a current video block, for example, by using reconstructed pixels from coded video pictures. Temporal prediction may reduce temporal redundancy that may be inherent in video signals. The temporal prediction signal for a given video block may be signaled, for example, by one or more Motion Vectors (MVs), which may indicate the amount and direction of motion between the current block and the reference block. For example, when multiple reference pictures may be supported (e.g., for h.264/AVC or HEVC), a reference picture index may be sent (e.g., for each video block). The reference index may be used to identify a reference picture (e.g., in reference picture store 364) from which the temporal prediction signal originated.

Mode decision block 380 in the encoder may select the (e.g., best) prediction mode (e.g., after spatial and/or temporal prediction) based on, for example, a rate-distortion optimization program. The prediction block may be subtracted from the current video block 316. The prediction residual (residual) may be decorrelated (e.g., using transform 304) and quantized 306. The quantized residual coefficients may be inverse quantized 310 and inverse transformed 312, for example, to form a reconstructed residual. The reconstructed residual may be added back to the prediction block 326, e.g., to form a reconstructed video block. In-loop filtering 366 (e.g., deblocking filter and/or adaptive loop filter) may be applied to reconstructed video blocks, e.g., before they are placed in reference picture store 364 and used to code future video blocks. Entropy coding unit 308 may be used to output video bitstream 320, e.g., by compressing and encapsulating coding modes (e.g., inter or intra), prediction mode information, motion information, and/or quantized residual coefficients. The encoder may include a WTRU (e.g., as described herein), a processor of the WTRU, and/or the like.

Fig. 2 shows an example of a block-based video decoder. The video bitstream 202 may be unpacked and entropy decoded at an entropy decoding unit 208. For example, coding mode and prediction information may be sent to spatial prediction unit 260 (e.g., if intra coded) or temporal prediction unit 262 (e.g., if inter coded) to form a prediction block. The residual transform coefficients may be provided to an inverse quantization unit 210 and an inverse transform unit 212, for example, to reconstruct the residual block. The prediction block and the residual block may be added together, for example, at summation 226. For example, in-loop filtering may be applied to reconstructed blocks before storing the reconstructed blocks in reference picture store 264. For example, the reconstructed video in reference picture store 264 may be sent out to drive a display device and/or predict future video blocks. The decoder may include a WTRU (e.g., as described herein), a processor of the WTRU, and/or the like.

Motion information (e.g., MV and reference picture index) may be determined by the encoder and (e.g., explicitly) transmitted to the decoder. A significant amount of overhead may be expended in encoding the motion parameters for the inter-coded block. For example, overhead of signaling motion information may be reduced by coding modes (e.g., FRUC). For example, signaling of MV and/or reference picture indices may be skipped when FRUC coding mode is enabled for a CU. For example, the information may be derived (e.g., on the decoder side) by template matching or bilateral matching techniques.

An in-loop filter may be employed at the encoder and/or decoder. The in-loop filter may include one or more of a deblocking filter, a sampling adaptive offset, or an ALF. The ALF may be a Wiener (Wiener) filter that is applicable to the reconstructed block (e.g., to minimize the mean square error between the original block and the reconstructed block). The ALF may be trained at the encoder. The ALF parameters may be signaled in the bitstream. ALF may be applied to luminance and/or chrominance components. ALF may be applied to luma components (e.g., luma ALF) and/or chroma components (e.g., chroma ALF). Luminance ALF and chrominance ALF may be trained separately. The luma and chroma components may not share a common ALF.

Chroma ALF may have fewer options when compared to luma ALF. For example, the luminance ALF may be selected among three different filter shapes: 5×5 diamond, 7×7 diamond, 9×9 diamond. Fig. 3 shows an example of ALF shape: a 5 x 5 diamond shape, (b) a 7 x 7 diamond shape, and (c) a 9 x 9 diamond shape. The chroma ALF may (e.g., may always) use a 5 x 5 diamond shaped filter. For luma ALF, ALF may be applied over the entire frame using frame-level ALF or on a block-by-block basis using block-level ALF. Frame-level ALF may be used for chroma ALF. For luma ALF, block classification may be performed on each 2 x 2 block to classify them into one of 25 categories. Each class may use a different ALF filter. Block classification may not be performed on the chrominance components. Chroma samples in a frame may be considered to belong to a class (e.g., class zero).

Block classification may be performed on reconstructed luma samples. Fig. 4 shows an example of an ALF procedure at the encoder. Block classification may be performed on the luma component of the reconstructed frame to classify one or more (e.g., each) 2 x 2 blocks. The encoder may train the 9 x 9 frame level luminance ALF for (e.g., each) class, for example, using the corresponding reconstructed and original pixels in the frame. The encoder may detect some (e.g., all possible) ALF block depths by testingTo train a 9 x 9 block level ALF. The 9 x 9 frame-level and/or block-level luminance ALF may be diamond-shaped. For ALF block depth, the encoder may run one or more (e.g., two) iterations.

For example, in the first iteration, a frame filtered by a 9×9 frame-level ALF may be used as a starting point. The encoder may determine a block partition of a Coding Tree Unit (CTU) to use a block level ALF. The encoder may start at the CTU and, for example, at a current block depth that is less than the associated CU depth and the associated CU depthThe CTU is recursively split into four equal sub-blocks. If the current block depth is not less than the associated CU depth and +.>Then for a given block, two Sum of Squares (SSD) values may be calculated: an SSD (Filt SSD) between the ALF filtered block and the original uncompressed block, and an SSD (unfiltssd) between the unfiltered reconstructed block and the original block. If Filt_SSD is smaller than unfilt_SSD, ALF may be enabled for the block. If Filt_SSD is not smaller than unfilt_SSD, ALF may be disabled for the block. Once the CTUs in the frame are processed, the Rate Distortion (RD) cost of the block filtered frame may be calculated and the associated ALF parameters of the block filtered frame may be saved.

In the second iteration, a different set of ALFs may be trained using the blocks that selected the ALFs in the first iteration. Different sets of ALFs may be applied to the reconstructed frame (e.g., the entire reconstructed frame) as frame-level ALFs. The encoder may determine a block partition for one or more (e.g., each) CTUs to use a different set of ALFs. The encoder may start at the CTU. For example, if the current block depth is less than the associated CU depth and the associated CU depthThe encoder may recursively split the CTU into four equal sub-blocks. If the current block depth is not less than the associated CU depth and +.>Then for a given block, two Sum of Squares (SSD) values may be calculated: an SSD (Filt SSD) between the ALF filtered block and the original uncompressed block, and an SSD (unfiltssd) between the unfiltered reconstructed block and the original block. If Filt_SSD is smaller than unfilt_SSD, ALF may be enabled for the block. If Filt_SSD is not less than unfilt_SSD, ALF may be disabled for that block. Once the CTUs in the frame are processed, the Rate Distortion (RD) cost of the block filtered frame may be calculated and the associated ALF parameters of the block filtered frame may be saved.

The candidate block level ALF that yields the smallest RD cost may be selected as the block level ALF. The RD cost at the frame level and the RD cost ALF at the block level may be compared. Frame-level and/or block-level ALFs (e.g., filter-shaped) that produce lower RD costs may be selected. The encoder may test other filter shapes (e.g., 7 x 7 and 5 x 5 filters) and/or select one with a lower RD cost. If ALF is not used, the RD cost of the selected frame-level or block-level ALF at the selected filter shape may be compared to the RD cost. A scheme that yields lower RD costs may be determined. Based on the determination, a picture level ALF flag (e.g., alf_flag) may have a value of 1 (e.g., indicating that ALF is on) or 0 (e.g., indicating that ALF is off). When luma ALF is enabled, chroma ALF may be calculated. A decision as to whether to use chroma ALF for a frame or a picture may be made based on RD cost.

A test (e.g., as performed for the current frame) may be performed to check the ALF generated for the previous frame. The previous frame may belong to the same temporal layer as the temporal layer of the current frame or to a temporal layer lower than the temporal layer of the current frame. If the use of an ALF generated for a previous frame yields the lowest RD cost, then the temporal prediction flag 1 and the index to the previous ALF may be signaled.

Block classification may be performed on the luma component of a (e.g., each) 2 x 2 reconstructed block to classify the reconstructed block into one of 25 categories. The classification index C may be derived, for example, according to equation 1.

D may be directional andmay be an active quantized value. To calculate D and +.>For example, according to equations 2-5, the 1-D Laplacian may be used to calculate horizontal, vertical, and two diagonal directional gradients. The horizontal, vertical and two diagonal directional gradients may be referred to as directional gradients.

i and j may refer to the coordinates of the up-left sample in the 2 x 2 reconstructed block, and R (i, j) may indicate the reconstructed sample at coordinates (i, j). The maximum and minimum values of the horizontal and vertical gradients may be set, for example, in the manner of equation 6.

The maximum and minimum of the two diagonal gradients may be set, for example, in the manner of equation 7.

The value of the directivity D can be derived by comparing the above values with each other and with two thresholds, and is as follows:

step 1. IfAnd->Both are true, D may be set to 0.

Step 2, ifContinuing from step 3; otherwise, continuing from step 4.

Step 3, ifD may be set to 2; otherwise D may be set to 1. Step 4. If->D may be set to 4; otherwise D may be set to 3.

For example, the activity value a may be calculated according to equation 8.

A may be further quantized to a range of 0 to 4 (inclusive), and the quantized value may be represented as

LIC may be used for inter-prediction of content with luminance variations over time. LIC may involve deriving the scale factor 'a' and/or the offset 'b' by a least squares method of the CU for inter coding. For example, adjacent reconstructed samples indicated by motion information and corresponding temporal reference samples may be used in the least squares method. The encoder may check whether LIC is enabled for the picture, e.g., based on whether there is a change in illumination between the current picture and a reference picture of the current picture. The histogram of the current picture and the reference picture(s) of the current picture may be calculated at the encoder. If the histogram difference between the current picture and the reference picture(s) is less than a given threshold, the LIC may be disabled for the current picture. If the histogram difference between the current picture and the reference picture(s) is greater than a given threshold, LIC may be enabled for the current picture.

As an in-loop process, ALF may be applied to reconstructed luma and/or chroma samples (e.g., at the encoder and/or decoder). The ALF procedure may include one or more of block classification or filtering. The block classification may be based on granularity of 2 x 2 blocks. The filtering may be based on one or more filter sizes from the set of 9 x 9,7 x 7,5 x 5.

The computational complexity of the encoder and/or decoder may be different for different ALF programs. Some ALF programs may be associated with lower computational complexity than other ALF programs. As an example, for block classification, the computational complexity of an encoder and/or decoder may be reduced by selecting a subset of pixels of a block. By selectively disabling filtering, the computational complexity of the encoder and/or decoder may be reduced.

Performing the ALF procedure may include, for example, applying block classification and/or filtering to the reconstructed block at the encoder and/or decoder. Classifying a 2 x 2 block may include calculating four directional gradients of the luminance component of the block. Each 2 x 2 block may be classified. The filtering may include applying a 5×5 diamond filter (e.g., 7 taps), a 7×7 diamond filter (e.g., 13 taps), or a 9×9 diamond filter (e.g., 21 taps) to the luma component, and/or applying a 5×5 diamond filter to the chroma component. The minimum filter tap may be seven. Filter taps greater than seven may be used for the luminance component. The symmetric nature of ALF may be considered.

ALF programs of varying computational complexity may be used based on temporal layer and/or gradient calculations.

The ALF procedure may be applied to a frame based on the temporal layer in which the frame is located in a coding scheme. The coding scheme may include a layered coding structure. The coding scheme may include multiple temporal layers. The temporal layer may contain one or more frames. Each temporal layer may be associated with a temporal hierarchy. In a non-limiting example, the coding scheme may include 4 temporal layers at temporal levels 0, 1, 2, and 3, respectively. The frames in temporal levels 3 and 2 may be in higher temporal layers within the coding scheme. The frames in temporal level 3 may be in the highest temporal layer within the coding scheme. The frames in temporal levels 0 and 1 may be in lower temporal layers within the coding scheme. The frames in temporal level 0 may be in the lowest temporal layer within the coding scheme. In another non-limiting example, the coding scheme may include 5 temporal layers at temporal levels 0, 1, 2, 3, and 4, respectively. Frames in temporal level 2 may be in an intermediate temporal layer within a coding scheme that includes 5 temporal layers. Frames in a higher temporal hierarchy may refer to frames in a lower temporal hierarchy. There may be multiple time tiers with different quality settings. For example, frames at lower temporal levels may use more coded bits with better quality than frames at higher temporal levels.

In one example, block classifications at one or more of the encoder or decoder may be skipped for frames in higher temporal layers in the coding scheme. Samples in the higher layer time frame may be mapped to one category. The encoder may quantize one or more higher temporal layers using a larger QP. Frames (e.g., reconstructed frames) in one or more higher temporal layers may be smooth, e.g., due to re-quantization. Frames in one or more higher temporal layers may contain fewer gradients than frames at lower or intermediate temporal layers. For frames in one or more higher temporal layers, the block classifications may yield fewer categories than the block classifications of the lower or intermediate temporal layers.

In an example, different ALF programs may include different block classification techniques with different computational complexity. The reduced computational complexity block classifications may be used for frames in one or more higher temporal layers. For example, for frames in one or more higher temporal layers, blocks of frames may be classified into two categories. For frames in the middle or lower temporal layers, blocks of frames may be classified into 25 categories.

Fewer gradients may be calculated for blocks in a frame in the one or more higher temporal layers for block classification. For frames in one or more higher temporal layers, gradient calculations in one or more of the vertical, horizontal, or diagonal directions of pixels of blocks in the frame may be skipped. For example, gradient computation in the diagonal direction of pixels of a block may be skipped. Gradients in the horizontal and vertical directions may be calculated (e.g., only).

Gradient calculations may be performed for blocks in a frame for block classification, e.g., based on a subset of pixels in the block. Fig. 5 shows an example of gradient computation: (a) a block classification, wherein for each 2 x 2 block, a gradient may be calculated for each pixel in a 6 x 6 window, (b) a window of two sub-samples in the horizontal direction, (c) a window of two sub-samples in the vertical direction, and (d) a window of two sub-samples in the horizontal direction and two sub-samples in the vertical direction.

In one example, for each 2×2 block, a 6×6 pixel window may be used (e.g., as shown in fig. 5 (a)). For each pixel in the window, four gradients may be calculated. The four gradients may include gradients in the horizontal, vertical, and two diagonal directions.

Gradients may be calculated for a subset of pixels of a block in a frame for block classification. In one example, gradients of a subset of pixels in a 6 x 6 pixel window may be calculated for one or more 2 x 2 blocks (e.g., as shown in fig. 5 (b) - (d)). Calculating gradients using a subset of pixels may reduce computational complexity (e.g., at an encoder or decoder). For example, a subset of pixels may be selected from the pixels in the (e.g., current) block by skipping at least one pixel in the horizontal direction of the pixels in the block. Fig. 5 (b) may show an example of calculating gradients of a subset of pixels in a 6 x 6 pixel window for a 2 x 2 block. As shown in fig. 5 (b), every other column of pixels may be skipped. The number of pixels used to calculate the gradient in the 6 x 6 pixel window may be reduced (e.g., by a factor of two). Using a subset of pixels (e.g., sub-sampling), a 6 x 6 pixel window may become a 3 x 6 pixel window. Gradients for one or more (e.g., each) pixel in the 3 x 6 window may be calculated. Gradient calculations (e.g., the number of gradients calculated or to be calculated) may be reduced by a factor of two for one or more 2 x 2 blocks. The gradient calculations (e.g., those of equations 2-5) may be updated based on the subset of pixels. For example, the sub-sampling operation may be incorporated into the gradient calculations of equations 2-5. An updated set of gradient equations for a twice reduced number of pixels (e.g., subset) in the horizontal direction may be given by Eq 9-12.

The gradient calculated as in equations 9-12 may be multiplied by a sub-sampling factor (e.g., two) to hold a range of values for the gradient prior to sub-sampling.

Gradients may be calculated for a subset of pixels that are vertically sub-sampled. Fig. 5 (c) may show an example of calculating gradients of a subset of pixels in a 6 x 6 pixel window for a 2 x 2 block. As shown in fig. 5 (c), pixels in every other row in the vertical direction may be skipped. Using a subset of pixels (e.g., sub-sampling), the 6 x 6 pixel window may become a 6 x 3 window, as shown in fig. 5 (c).

Gradients may be calculated for a subset of pixels that are vertically and horizontally sub-sampled. Fig. 5 (d) may show an example of calculating gradients of a subset of pixels in a 6 x 6 pixel window for a 2 x 2 block. As shown in fig. 5 (d), pixels in every other row in the vertical direction and pixels in every other column in the horizontal direction may be skipped. Using a subset of pixels (e.g., sub-sampling), the 6 x 6 pixel window may be changed to a 3 x 3 window, as shown in fig. 5 (d).

The subset of pixels may be used for gradient computation in different block classification schemes. For example, a subset of pixels may be selected to calculate a gradient for block classification performed on a 4 x 4 block with an 8 x 8 window of pixels.

Gradients may be calculated for a subset of pixels in a block of frames belonging to a higher temporal layer (e.g., highest temporal layer or next highest temporal layer) and/or an intermediate temporal layer. Gradients may be calculated for a subset of pixels in a block of a frame belonging to a lower temporal layer. Blocks in frames belonging to higher temporal layers may be smoother (e.g., have weak edges). A sub-sampling operation may be used to calculate gradients of blocks in frames belonging to higher, intermediate, and/or lower temporal layers.

In an example, a frame in the highest temporal layer may use a subset of vertically and horizontally sub-sampled pixels for gradient computation. At least one pixel in the vertical direction and at least one pixel in the horizontal direction of the pixels in the block may be skipped. For example, if the temporal level of the current frame is the highest level within the coding scheme, the ALF procedure for the current frame may be selected. In the ALF procedure of the current frame in the highest temporal layer, a subset of pixels may be selected from the pixels in the block of the current frame by skipping every other row of pixels in the vertical direction and every other column of pixels in the horizontal direction. The gradient of the block may be calculated using the selected subset of pixels. The blocks may be classified for ALF based on the calculated gradients. The block may be a current block.

Frames in the lower temporal layer, the intermediate temporal layer, or the second highest temporal layer may use sub-sampling of pixels in the block in one direction (e.g., only one of a vertical, only horizontal, or only diagonal direction). Sub-sampling may reduce pixels in a 6 x 6 pixel window by a factor of two.

For example, if the temporal level of the current frame is above the lowest level and below the highest level within the coding scheme, an ALF procedure for the current frame may be selected. The ALF procedure selected may be different from the ALF procedure for frames in the highest temporal layer. The ALF procedure selected may be different from the ALF procedure for the frames in the lowest temporal layer. In the ALF procedure, a subset of pixels may be selected from the pixels in the block of the current frame by skipping at least one pixel in one of the vertical, horizontal, or diagonal directions of the pixels in the block. The gradient of the block may be calculated using the selected subset of pixels. The blocks may be classified for ALF based on the calculated gradients.

The ALF procedure for the current frame may be selected if the temporal level of the current frame is the lowest level within the coding scheme. In the ALF procedure of the current frame in the lowest temporal layer, the gradient of the block may be calculated using each pixel in the block. The blocks may be classified for ALF based on the calculated gradients.

The encoder or decoder may determine or select an ALF procedure for a frame (e.g., a current frame) based on, for example, a temporal level of the frame. The encoder or decoder may determine whether a subset of pixels is to be selected to calculate the gradient of the block classification. The encoder or decoder may determine how the subset of pixels is to be selected.

The encoder may signal to the decoder whether sub-sampling is to be used for the frame. The encoder can signal which sub-samples to use for the frame. In an example, an indication of whether sub-sampling is to be used and/or an indication of which sub-sampling scheme is to be used may be signaled for each frame. For example, the indication may be a syntax element signaled in the bitstream.

The encoder may determine or select the ALF procedure by comparing RD costs. The encoder may perform ALF (e.g., as described herein) without sub-sampling for the frame. If ALF is selected or enabled for the frame, the encoder may execute a different ALF program and/or calculate RD costs associated with the ALF program. For example, the encoder may repeat the ALF process using a different subset of pixels in each sub-sampling scheme (e.g., as described herein), and/or compare the RD costs associated with each sub-sampling scheme.

The encoder may signal whether an indication of certain conditions is met. Equation 13 shows one example of a condition.

In equation 13, RD _ref And RD (RD) _minSub The RD cost of a scheme (e.g., a reference scheme) and the minimum RD cost in different sub-sampling schemes may be represented separately. ω may represent a scale factor greater than one. If one of the conditions in equation 13 is satisfied, the encoder may signal one or more of the following: a sub-sample flag with a value of 1 or an index to the sub-sampling scheme. The signaling may be performed at the frame level in a slice parameter set or in a slice header. In an example, the prefix code may be constructed as an index for (e.g., each) sub-sampling scheme. Fig. 6 shows an example of the establishment of a prefix code for different sub-sampling schemes. As shown in fig. 6, a determination may be made as to whether to use vertical sub-sampling. If vertical sub-sampling is used, the index or the digit of the index may be set to 0. If vertical sub-sampling is not used, the index or the digit of the index may be set to 1. A determination may be made as to whether to use horizontal sub-sampling. If horizontal sub-sampling is used, the index or the digit of the index may be set to 0. If horizontal sub-sampling is not used, the index or the bit of the index may be set to 1. Table 1 may show the resulting index map. A sub-sample flag with a value of 0 may be signaled in the bitstream if none of the conditions in equation 13 are satisfied.

Table 1 may describe examples of indexes for different sub-sampling schemes.

TABLE 1

Sub-sampling	Index (binary representation provided in brackets)
		Vertical and vertical	0(0)
Horizontal level	2(10)
		Vertical and horizontal	3(11)

The decoder may receive an indication of whether sub-sampling is to be used for the frame and/or an indication of which sub-sampling is to be used for the frame. The decoder may determine, based on the indication(s), which sub-samples to use for the frame and/or which sub-samples to use for the frame. The decoder may perform ALF based on the indicated sub-sampling scheme. The decoder may skip sub-samples based on the indication that sub-samples are to be skipped for the frame.

In one example, the ω value in equation 13 may be set to 1. For example, an ω value equal to 1 may indicate that no bias is provided to the reference ALF scheme. Sub-sampling may be applied (e.g., only) to the ALF block classifications and/or may not necessarily increase the RD cost of the filtered frame. In one example, the ω value in equation 13 may be set to be less than 1. Omega values less than 1 may provide bias to the reference ALF scheme.

The ALF programs of different computational complexity may include various filtering operations. For example, an ALF program of reduced computational complexity may skip one or more filtering operations.

Frame adaptive ALF skipping can be used to reduce computational complexity. For example, as shown in fig. 4, the encoder may determine and/or select an ALF (e.g., luma ALF) for the luma component of a given frame. The luminance ALF may be one that generates a minimum RD cost (e.g., an optimal luminance ALF) among the plurality of luminance ALFs. The encoder may compare the RD cost of frames filtered using the determined luma ALF with the RD cost of unfiltered frames. The encoder may make a picture level determination as to whether to use luma ALF for the frame.

If luma ALF is not used (e.g., selected) for the luma component of the frame, the encoder may not test chroma ALF for the chroma component of the frame. The encoder may test the temporal predicted ALF. The encoder may determine (e.g., select or pick) the temporally predicted ALF based on the test. The selected temporally predicted ALF may result in a minimum RD cost in the temporally predicted ALF and/or the ALF from the current frame.

Fig. 7 shows an example of an ALF procedure of reduced computational complexity using frame adaptive ALF skipping. For frames in higher temporal layers, ALF may be skipped or disabled. For frames in higher temporal layers, the QP may be larger. In an example, the ALF for frames in the highest temporal layer may be skipped or disabled (e.g., only).

As shown in fig. 7, block classification may be performed on luma samples in an ALF program. Frame-level luminance ALF may be performed (e.g., starting from a 9x9 diamond-shaped filter). Block level luminance ALF may be performed (e.g., starting from a 9x9 diamond shaped filter). The brightness filter shape may be determined. An appropriate luminance method (e.g., a luminance method with minimum RD cost) may be determined. Block classification may be performed on the luma component of the reconstructed frame to classify one or more (e.g., each) 2 x 2 blocks 662. The encoder may train 9x9 frame-level luma ALF for one (e.g., each) class, e.g., using the corresponding reconstructed and original pixels in the frame. The encoder may detect some (e.g., all possible) ALF block depths by testingTo train a 9x9 block level ALF. The 9x9 frame-level and/or block-level luminance ALF may be diamond-shaped. The encoder may test other filter shapes (e.g., 7 x 7 and 5 x 5 filters) and/or select one filter shape with a lower RD cost. If ALF is not used, the RD cost of the selected frame-level or block-level ALF at the selected filter shape may be compared to the RD cost. A scheme that yields lower RD costs may be determined.

The square difference and SSE (alf_sse) of the associated filtered frame may be calculated relative to the original frame (e.g., by the encoder). The square error and SSE (ALF_SSE) of the relevant filtered frame can be compared to the SSE (origin_SSE) of the unfiltered frame as shown in equation 14.

origin_SSE.ltoreq.T.ALF_SSE equation 14

T may be a deviation factor. In one example, T may be set to 1.003. If origin_SSE is less than or equal to ALF_SSE multiplied by the deviation factor T, then ALF may be disabled for the luma and chroma components of a given frame. If origin_SSE is greater than ALF_SSE multiplied by the deviation factor T, the encoder may enable the use of ALF for the luma component of the frame. A chrominance filter may be derived and it may be determined whether to apply chrominance ALF to the chrominance components of the frame. The time predicted ALF may be checked.

The deviation factor may allow the encoder to deactivate ALF if the frame distortion (origin_sse) of the unfiltered frame is within a given tolerance of the ALF filtered frame distortion (alf_sse). The bias factor may enable more frames to be processed (e.g., encoded or decoded) with the ALF disabled than if the bias factor was not used. The bias factor may be set to a value that facilitates skipping ALF. For example, if the deviation factor is set to a larger value, it may be advantageous to skip ALF. For example, the decoder may skip or disable the ALF upon receiving an indication from the encoder.

The bias factor may be set differently for frames in different temporal layers. The bias factor may be set to a value that facilitates skipping ALF for frames in higher temporal layers. For example, the bias factor for frames in a higher temporal layer within a coding scheme may be set to be greater than the bias factor for frames in a lower temporal layer.

The bias factor may be set differently for frames related to different QPs for picture coding. The bias factor may be set to a value that facilitates skipping ALF for frames related to larger coding QPs. For example, the bias factor for frames associated with larger coded QPs may be set to be greater than the bias factor for frames associated with smaller coded QPs.

The deviation factor may be set differently for frames associated with different frame resolutions. The deviation factor may be set to a value that is advantageous for skipping ALF for frames with smaller resolutions. For example, the deviation factor for frames with a larger resolution may be set to be relatively smaller than frames with a smaller resolution.

The deviation factor may be set differently for frames associated with different illumination changes. The bias factor may be set based on whether there is a change in illumination between the current frame and some (e.g., all) of the reference frames of the current frame. If the current frame has no illumination change from all reference frames of the current frame, the deviation factor may be set to a value that facilitates skipping of ALF. For example, if no illuminance change is detected between the current frame and all reference frames of the current frame, the deviation factor may be set to a larger value. If the deviation factor is set to a larger value, it may be advantageous to skip the ALF. If a change in illumination between the current frame and some (e.g., all) of the reference frames of the current frame is detected, the deviation factor may be set to a smaller value. The illumination change may be detected by a histogram-based scheme used by Local Illumination Compensation (LIC). If the LIC is enabled by the codec, a bias factor may be determined based on whether the LIC is enabled for the current frame. For example, if LIC is enabled for the current frame, the deviation factor of ALF may be set to a smaller value. The bias factor for ALF may be set to a higher value if LIC is not enabled for the current frame.

Pixel adaptive ALF skips ALF procedures that can be used for reduced computational complexity. The ALF may include a wiener filter that minimizes the mean square error between the reconstructed samples and the original samples. The ALF may be disabled in the ALF program with reduced computational complexity if the reconstructed block is smooth and/or lacks gradients.

During block classification, gradient g, as described herein _h 、g _v 、g _d0 And g _d1 May be calculated for (e.g., each) 2 x 2 block. In one example, the gradient information may be used to decide whether to skip ALF filtering for a 2 x 2 block.

FIG. 8 shows the use of gradient sums g _sum And threshold T _G And an example of block classification for a 2 x 2 block with ALF on/off decision is employed. For example, the encoder and/or decoder may make the determination at the block classification stage. The sum of gradients (e.g., gradient sum) may be calculated for a 2 x 2 block as shown in equation 15.

g _sum ＝g _v +g _h +g _d0 +g _d1 Equation 15

H and W may represent the height and width of the video, respectively. May be maintained at the encoder and/or decoder with dimensionsFor example, alf_mask). The buffer may be used to store information whether the luminance reconstructed pixels (e.g., reconstructed luminance components) in a 2 x 2 block are ALF filtered. The gradient sum can be combined with a threshold T _G A comparison is made. If the sum of the gradients is less than the threshold T _G An ALF on (on) indication (e.g., alf_mask flag) associated with a 2x2 block may be set to 0. If the sum of gradients is not less than the threshold t _G The ALF on indication associated with a 2x2 block may be set to 1. If the ALF on indication associated with a 2x2 block is set to 1, ALF may be performed on the 2x2 block (e.g., by a decoder).

A determination may be made as to whether an ALF on indication associated with a 2x2 block is equal to 1. For example, the determination may be made prior to performing the ALF filtering step. Fig. 9 shows an example of applying ALF for a 2×2 block. As shown in fig. 9, it may be checked whether the ALF on indication associated with a given 2x2 block has a value of 1. The filtering may be performed when a particular condition is satisfied (e.g., only when the particular condition is satisfied). If the condition is not met, filtering may be skipped for the block (e.g., the current block). As shown in fig. 9, a determination may be made as to whether alf_mask is equal to 1. If the ALF_mask is not equal to 1, then the ALF may not be applied (e.g., skipped). If ALF_mask is equal to 1, ALF may be applied to a 2×2 block.

Threshold T _G May be predetermined and/or fixed. In an example, threshold T _G May be fixed and/or may be derived based on offline training. For coding schemes using temporal layers, thresholds may be trained offline for the corresponding temporal layers. For example, the threshold may be trained offline for each temporal layer. The threshold may or may not be signaled (e.g., for each temporal layer). The threshold may be predetermined and fixed at the encoder and/or decoder.

Threshold T _G May be derived based on, for example, a desired percentage of ALF skipping. In one example, the threshold may be trained during the encoding process. In coding schemes using temporal layers, thresholds may be trained for the respective temporal layers. For example, the first frame of each temporal layer may be used to train a threshold used by subsequent frames in the same temporal layer. The threshold may be determined based on a percentage of ALF skipping desired. The percentage of ALF skipping desired may be specified as an input parameter to the encoder. Fig. 10 shows the use of the sum of gradients for frames (g _sum ) Training the threshold T by histogram _G Is an example of (a).

As shown in fig. 10, parameters of the histogram may be selected. The parameters of the histogram may include one or more of bin(s) size, bin total number, and maximum limit of the histogram. During block classification of a frame (e.g., the frame under consideration), a histogram may be constructed using the sum of gradients of 2 x 2 blocks in the frame. The resulting histogram may represent a count of occurrences of (e.g., each) interval of the gradient sum. The count associated with the ALF skip percentage may be determined using the skip percentage (ALF skip_) and the total count of the histogram (total_count), e.g., as expressed in equation 16.

Alf_skip_count=alf_skip_permanent_count equation 16

The histogram may be examined to determine at which of the min-interval indices the cumulative count is greater than or equal to the alf_skip_. The check may start with interval index zero. The value of the selected interval can be used as a threshold T _G . The value of the selected interval may include a central value or a maximum value of the selected interval. The value of the selected interval (e.g., threshold T _G ) Can be usedThe decision is skipped at ALF for subsequent frames in the same temporal layer. Can signal a threshold T _G The threshold may not be signaled. The threshold T may be derived at the encoder and/or decoder _G 。

The threshold T may be calculated for (e.g., each) frame in the video sequence _G . The encoder may signal a threshold T for each frame in the bitstream _G Once. The decoder may receive the threshold T _G And/or determining whether an ALF on indication associated with a 2 x 2 block has a value of 1.

The encoder may signal the skip percentage (e.g., only once per sequence). In examples where the encoder may signal the skip percentage only once for each sequence, histogram generation may be performed on a block or group of blocks at the encoder and/or decoder. Based on the signaled skip percentage, a threshold may be calculated at the decoder.

Whether to skip ALF may be determined based on RD cost. The encoder may use RD cost to select between applying ALF to the frame and using pixel adaptive ALF skipping. Fig. 11 illustrates an exemplary rate-distortion (RD) based method for selecting between applying ALF and pixel-adaptive ALF skipping using a skip percentage "a". For the skip method, the threshold may be signaled per frame. If the threshold is to be derived at the decoder, no signaling may be performed.

The bias factor β may be used to enable more frames to be processed (e.g., encoded or decoded) with pixel-adaptive ALF skipping than if the bias factor was not used. The deviation factor may be set to a value that facilitates pixel adaptive ALF skipping. For example, if the deviation factor is set to a larger value, pixel adaptive ALF skipping is advantageous.

As shown in fig. 11, the encoder may execute the ALF and/or may calculate an associated RD cost (RD _orig ). The encoder may apply ALF (e.g., perform an ALF process) to the specified skip percentage 'a'. Applying ALF to the specified skip percentage 'a' may include calculating a threshold T _G And/or performing pixel adaptive ALF skipping. The encoder may calculate the associated RD cost (RD) _skipA )。RD _orig The deviation factor beta may be determined (e.g., selected) with a value greater than or equal to 1. May be multiplied by a deviation factor beta. The deviation factor beta can be combined with RD _orig The multiplied product is multiplied by an associated RD cost RD _skipA A comparison is made. If the product is greater than the associated RD cost RD _skipA The flag (e.g., alf_skip_flag) may have a value of 1 in the bitstream. An alf_skip_flag having a value of 1 may indicate that pixel adaptive ALF skipping may be applied to the current frame. A threshold T for specifying the skip percentage 'a' may be signaled _G . If the product is equal to or less than the associated RD cost RD _skipA The alf_skip_flag may have a value of 0 in the bitstream. An alf_skip_flag having a value of 0 may indicate that ALF may be applied to the current frame.

The skip percentages 'a' and 'B' may be used (e.g., by an encoder) to select between applying ALF and using pixel adaptive ALF skipping. In one example, the method depicted in FIG. 11 may be extended to select from more than one skip percentage. FIG. 12 illustrates an exemplary RD cost based selection process when two skip percentages ("A" and "B") are specified for pixel adaptive ALF skipping. The skip percentages may be tested in an ascending order, e.g., when "A" is less than "B," the "A" is tested first. As shown in fig. 11, the encoder may execute the ALF and/or may calculate an associated RD cost (RD _orig ). The encoder may apply ALF (e.g., perform an ALF process) to the specified skip percentage 'a'. Applying ALF to the specified skip percentage 'a' may include calculating a threshold T _G And/or performing pixel adaptive ALF skipping. The encoder may calculate an associated RD cost (RD _origA ). The deviation factor beta may be determined (e.g., selected) with a value greater than or equal to 1. The bias factor β may be used to enable more frames to be processed (e.g., encoded or decoded) with pixel-adaptive ALF skipping than if the bias factor was not used.

RD _orig May be multiplied by a deviation factor beta. The deviation factor beta can be combined with RD _orig The multiplied product is multiplied by an associated RD cost RD _origA A comparison is made. If the product is equal to or less than the associatedRD cost RD _origA The alf_skip_flag may have a value of 0 in the bitstream indicating that ALF is to be applied to the current frame. If the product beta x RD _orig Greater than RD _origA The encoder may test the skip percentage 'B' and/or calculate the associated RD cost (RD _skipB ). If the product beta x RD _orig Greater than RD _skipB ，β×RD _orig May have a value of 1 indicating that pixel adaptive ALF skipping may be performed using a threshold associated with skip percentage B. The threshold associated with 'B' may be signaled. If the product beta x RD _orig Equal to RD _skipB Or with RD _skipB Similarly, alf_skip_flag may have a value of 1, indicating that pixel adaptive ALF skipping may be performed using a threshold associated with skip percentage a. The threshold associated with 'a' may be signaled.

For example, at the decoder, parameters associated with pixel adaptive ALF skipping may be parsed. Fig. 13 shows an example of parsing parameters associated with pixel adaptive ALF skipping at a decoder. As shown in fig. 13, the decoder may parse the bitstream to read parameters associated with the ALF. If ALF is enabled for a given frame (e.g., when alf_flag has a value of 1), then the bitstream may be analyzed to read alf_skip_flag. An alf_skip_flag having a value of 1 may indicate that pixel adaptive ALF skipping may be applied to the current frame. If alf_skip_flag has a value of 1, then the threshold T _G Parsing may be derived from the bitstream and/or used for ALF skip decisions. If this flag is not equal to 1, ALF may be applied to the current frame and/or threshold T may be applied _G Set to a default value (e.g., zero).

The decoder may derive the threshold(s) if and/or when multiple skip percentages are specified at the encoder (e.g., using the methods herein). The encoder may signal some (e.g., all) of the candidate skip percentages, e.g., once per video sequence. The encoder may signal an index corresponding to the selected skip percentage if and/or when selecting a pixel adaptive ALF skip for a frame. The decoder may derive the threshold(s) based on the candidate skip percentages and the index corresponding to the selected skip percentages.

The ALF filter tap selection may be based on the temporal layer. 5×5, 7×7, and 9×9 ALFs may be used for the luminance component of the frame. A 5 x 5ALF may be used for both chrominance components. In coding schemes using temporal layers, frames at higher temporal layers may be assigned a greater QP than frames at lower temporal layers. Frames at higher temporal layers may be smoother than frames at lower temporal layers. For higher temporal layers, small sized ALFs may be used. In an example, the size of the ALF for a frame may be limited based on which temporal layer in the coding scheme the frame is in. For example, frames in higher temporal layers may be limited to using small-sized filters. In an example, in a random access configuration using five temporal layers, frames in a fourth temporal layer may be limited to using 5×5ALF (e.g., only 5×5×665 ALF). The frames in the second and third layers may be limited to use of 5 x 5 and/or 7 x 7ALF. Frames in the remaining two lower layers may use any of three filter sizes.

A block level ALF may be provided that reduces computational complexity. A block level ALF may be generated. The encoder may compare SSDs for the filtered and unfiltered blocks. The encoder may decide whether to enable or disable ALFs for a given block. An SSD between the reconstructed block and the original block may be calculated. Deviations in SSD comparisons can be used to support disabling ALF for blocks. Using a bias in SSD comparison may result in fewer blocks to be filtered at the decoder. The deviation factor y may be multiplied by the SSD (Filt SSD) of the filtered block. The deviation factor y may have a value greater than 1. If the product of the deviation factor gamma times the SSD of the filtered block (Filt_SSD) is greater than the SSD of the unfiltered block, the ALF for that block may be disabled.

The computational complexity of ALF may be reduced by combining one or more of the methods described herein. In one example, block classification to reduce computational complexity and/or filtering to reduce computational complexity may be combined. For example, for frames in the highest temporal layer, vertical sub-sampling may be used during gradient computation and pixel adaptive ALF skipping may be used during filtering.

Fig. 14A is a diagram illustrating an example communication system 100 in which one or more disclosed embodiments may be implemented. Communication system 100 may be a multiple-access system that provides content, such as voice, data, video, messages, broadcasts, etc., to a plurality of wireless users. Communication system 100 may enable multiple wireless users to access such content by sharing system resources including wireless bandwidth. For example, communication system 100 may employ one or more channel access methods such as Code Division Multiple Access (CDMA), time Division Multiple Access (TDMA), frequency Division Multiple Access (FDMA), orthogonal FDMA (OFDMA), single carrier FDMA (SC-FDMA), zero-tail unique word DFT-spread OFDM (ZT-UW DTS-s OFDM), unique word OFDM (UW-OFDM), resource block filtered OFDM, filter Bank Multicarrier (FBMC), and the like.

As shown in fig. 14A, the communication system 100 may include wireless transmit/receive units (WTRUs) 102a, 102b, 102c, 102d, RANs 104/113, CNs 106/115, public Switched Telephone Networks (PSTN) 108, the internet 110, and other networks 112, although it should be understood that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and/or network elements. Each WTRU102a, 102b, 102c, 102d may be any type of device configured to operate and/or communicate in a wireless environment. For example, the WTRUs 102a, 102b, 102c, 102d (any of which may be referred to as a "station" and/or a "STA") may be configured to transmit and/or receive wireless signals and may include User Equipment (UE), mobile stations, fixed or mobile subscriber units, subscription-based units, pagers, cellular telephones, personal Digital Assistants (PDAs), smartphones, laptop computers, netbooks, personal computers, wireless sensors, hotspots or MiFi devices, internet of things (IoT) devices, watches or other wearable devices, head Mounted Displays (HMDs), vehicles, drones, medical devices and applications (e.g., tele-surgery), industrial devices and applications (e.g., robots and/or other wireless devices operating in an industrial and/or automated processing chain environment), consumer electronics devices, devices operating on commercial and/or industrial wireless networks, and the like. Any of the WTRUs 102a, 102b, 102c, and 102d may be interchangeably referred to as a UE.

Communication system 100 may also include base station 114a and/or base station 114b. Each of the base stations 114a, 114b may be any type of device configured to wirelessly interface with at least one of the WTRUs 102a, 102b, 102c, 102d to facilitate access to one or more communication networks (e.g., the CN106/115, the internet 110, and/or other networks 112). By way of example, the base stations 114a, 114B may be Base Transceiver Stations (BTSs), nodes B, e node bs, home enode bs, gnbs, NR node bs, site controllers, access Points (APs), wireless routers, and the like. Although the base stations 114a, 114b are each depicted as a single element, it will be appreciated that the base stations 114a, 114b may include any number of interconnected base stations and/or network elements.

Base station 114a may be part of RAN104/113 and may also include other base stations and/or network elements (not shown), such as Base Station Controllers (BSCs), radio Network Controllers (RNCs), relay nodes, and the like. Base station 114a and/or base station 114b may be configured to transmit and/or receive wireless signals on one or more carrier frequencies, which may be referred to as cells (not shown). These frequencies may be in licensed spectrum, unlicensed spectrum, or a combination of licensed and unlicensed spectrum. A cell may provide coverage of a wireless service to a particular geographic area, which may be relatively fixed or may change over time. The cell may be further divided into cell sectors. For example, a cell associated with base station 114a may be divided into three sectors. Thus, in one embodiment, the base station 114a may include three transceivers, i.e., one for each sector of the cell. In an embodiment, base station 114a may employ multiple-input multiple-output (MIMO) technology and may use multiple transceivers for each sector of a cell. For example, beamforming may be used to transmit and/or receive signals in desired spatial directions.

The base stations 114a, 114b may communicate with one or more of the WTRUs 102a, 102b, 102c, 102d via an air interface 116, which air interface 116 may be any suitable wireless communication link (e.g., radio Frequency (RF), microwave, centimeter wave, millimeter wave, infrared (IR), ultraviolet (UV), visible light, etc.). The air interface 116 may be established using any suitable Radio Access Technology (RAT).

More specifically, as described above, communication system 100 may be a multiple access system and may employ one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, and the like. For example, a base station 114a in the RAN104/113 and the WTRUs 102a, 102b, 102c may implement a radio technology such as Universal Mobile Telecommunications System (UMTS) terrestrial radio access (UTRA), which may use Wideband CDMA (WCDMA) to establish the air interfaces 115/116/117.WCDMA may include communication protocols such as High Speed Packet Access (HSPA) and/or evolved HSPA (hspa+). HSPA may include high speed Downlink (DL) packet access (HSDPA) and/or High Speed UL Packet Access (HSUPA).

In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as evolved UMTS terrestrial radio access (E-UTRA), which may use Long Term Evolution (LTE) and/or LTE-advanced (LTE-a) and/or LTE-advanced Pro (LTE-a Pro) to establish the air interface 116.

In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as NR radio access, which may use a New Radio (NR) to establish the air interface 116.

In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement multiple radio access technologies. For example, the base station 114a and the WTRUs 102a, 102b, 102c may implement LTE radio access and NR radio access together, e.g., using a Dual Connectivity (DC) principle. Thus, the air interface used by the WTRUs 102a, 102b, 102c may be characterized by multiple types of radio access technologies and/or transmissions to/from multiple types of base stations (e.g., enbs and gnbs).

In other embodiments, the base station 114a and the WTRUs 102a, 102b, 102c may implement wireless technologies such as IEEE802.11 (i.e., wireless fidelity (WiFi), IEEE802.16 (i.e., worldwide Interoperability for Microwave Access (WiMAX)), CDMA2000, CDMA20001X, CDMA EV-DO, internet standard 2000 (IS-2000), internet standard 95 (IS-95), internet standard 856 (IS-856), global system for mobile communications (GSM), enhanced data rates for GSM evolution (EDGE), GSM EDGE (GERAN), and the like.

The base station 114B in fig. 14A may be, for example, a wireless router, home node B, home enode B, or access point, and may utilize any suitable RAT to facilitate wireless connections in a local area, such as a business, home, vehicle, campus, industrial facility, air corridor (e.g., for use by a drone), road, etc. In one embodiment, the base station 114b and the WTRUs102 c, 102d may implement a radio technology such as IEEE802.11 to establish a Wireless Local Area Network (WLAN). In one embodiment, the base station 114b and the WTRUs102 c, 102d may implement a radio technology, such as IEEE802.15, to establish a Wireless Personal Area Network (WPAN). In yet another embodiment, the base station 114b and the WTRUs102 c, 102d may utilize a cellular-based RAT (e.g., WCDMA, CDMA2000, GSM, LTE-A, LTE-A, LTE-a Pro, NR, etc.) to establish a pico cell or femto cell. As shown in fig. 14A, the base station 114b may have a direct connection to the internet 110. Thus, the base station 114b may not need to access the Internet 110 via the CN 106/115.

The RANs 104/113 may communicate with the CNs 106/115, and the CNs 106/115 may be any type of network configured to provide voice, data, application, and/or voice over internet protocol (VoIP) services to one or more of the WTRUs102a, 102b, 102c, 102 d. The data may have varying quality of service (QoS) requirements, such as different throughput requirements, latency requirements, error tolerance requirements, reliability requirements, data throughput requirements, mobility requirements, and the like. The CN106/115 may provide call control, billing services, mobile location based services, prepaid calls, internet connections, video distribution, etc., and/or perform advanced security functions (e.g., user authentication). Although not shown in fig. 14A, it should be appreciated that the RANs 104/113 and/or CNs 106/115 may communicate directly or indirectly with other RANs that employ the same RAT as the RANs 104/113 or a different RAT. For example, in addition to being connected to a RAN104/113 that may utilize NR radio technology, the CN106/115 may also communicate with another RAN (not shown) employing GSM, UMTS, CDMA, wiMAX, E-UTRA or WiFi radio technology.

The CN106/115 may also act as a gateway for the WTRUs 102a, 102b, 102c, 102d to access the PSTN108, the Internet 110, and/or other networks 112.PSTN108 may include circuit-switched telephone networks that provide Plain Old Telephone Service (POTS). The internet 110 may include a global system of interconnected computer networks and devices that use common communication protocols, such as Transmission Control Protocol (TCP), user Datagram Protocol (UDP), and/or Internet Protocol (IP) in the TCP/IP internet protocol suite. Network 112 may include wired and/or wireless communication networks owned and/or operated by other service providers. For example, the network 112 may include another CN connected to one or more RANs, which may use the same RAT as the RANs 104/113 or a different RAT.

Some or all of the WTRUs 102a, 102b, 102c, 102d in the communication system 100 may include multi-mode capabilities (e.g., the WTRUs 102a, 102b, 102c, 102d may include multiple transceivers to communicate with different wireless networks over different wireless links). For example, the WTRU102c shown in fig. 14A may be configured to communicate with the base station 114A, which may use a cellular-based radio technology, and with the base station 114b, which may use an IEEE802 radio technology.

Fig. 14B is a system diagram illustrating an example WTRU 102. As shown in fig. 14B, the WTRU102 may include a processor 118, a transceiver 120, a transmit/receive element 122, a speaker/microphone 124, a keypad 126, a display/touchpad 128, a non-removable memory 130, a removable memory 132, a power source 134, a Global Positioning System (GPS) chipset 136, and/or other peripherals 138, and the like. It is understood that the WTRU102 may include any sub-combination of the foregoing elements while remaining consistent with an embodiment.

The processor 118 may be a general purpose processor, a special purpose processor, a conventional processor, a Digital Signal Processor (DSP), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, application Specific Integrated Circuits (ASICs), field Programmable Gate Arrays (FPGAs) circuits, any other type of Integrated Circuit (IC), a state machine, or the like. The processor 118 may perform signal decoding, data processing, power control, input/output processing, and/or any other function that enables the WTRU102 to operate in a wireless environment. The processor 118 may be coupled to a transceiver 120, and the transceiver 120 may be coupled to a transmit/receive element 122. Although fig. 14B depicts the processor 118 and the transceiver 120 as separate components, it will be appreciated that the processor 118 and the transceiver 120 may be integrated together in an electronic package or chip.

The transmit/receive element 122 may be configured to transmit signals to and receive signals from a base station (e.g., base station 114 a) over the air interface 116. For example, in one embodiment, the transmit/receive element 122 may be an antenna configured to transmit and/or receive RF signals. In one embodiment, the transmit/receive element 122 may be a transmitter/detector configured to transmit and/or receive, for example, IR, UV, or visible light signals. In yet another embodiment, the transmit/receive element 122 may be configured to transmit and/or receive both RF and optical signals. It should be appreciated that the transmit/receive element 122 may be configured to transmit and/or receive any combination of wireless signals.

Although the transmit/receive element 122 is depicted as a single element in fig. 14B, the WTRU102 may include any number of transmit/receive elements 122. More specifically, the WTRU102 may use MIMO technology. Thus, in one embodiment, the WTRU102 may include two or more transmit/receive elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals over the air interface 116.

The transceiver 120 may be configured to modulate signals to be transmitted by the transmit/receive element 122 and demodulate signals received by the transmit/receive element 122. As described above, the WTRU102 may have multi-mode capabilities. Thus, the transceiver 120 may include multiple transceivers for enabling the WTRU102 to communicate via multiple RATs (e.g., NR and IEEE 802.11).

The processor 118 of the WTRU102 may be connected to and may receive user input data from a speaker/microphone 124, a keypad 126, and/or a display/touchpad 128, such as a Liquid Crystal Display (LCD) display unit or an Organic Light Emitting Diode (OLED) display unit. The processor 118 may also output user data to the speaker/microphone 124, the keyboard 126, and/or the display/touchpad 128. Additionally, the processor 118 may access information from any type of suitable memory and store data in the memory, such as the non-removable memory 130 and/or the removable memory 132. The non-removable memory 130 may include Random Access Memory (RAM), read Only Memory (ROM), a hard disk, or any other type of memory storage device. Removable memory 132 may include a Subscriber Identity Module (SIM) card, a memory stick, a Secure Digital (SD) memory card, and the like. In other embodiments, the processor 118 may access information from and store data in memory that is not physically located on the WTRU102, such as on a server or home computer (not shown).

The processor 118 may receive power from the power supply 134 and may be configured to allocate and/or control power to other components in the WTRU 102. The power source 134 may be any suitable device for powering the WTRU 102. For example, the power source 134 may include one or more dry cells (e.g., nickel cadmium, nickel zinc, nickel metal hydride (NiMH), lithium ion (Li-ion), etc.), solar cells, fuel cells, and the like.

The processor 118 may also be coupled to a GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) regarding the current location of the WTRU 102. In addition to, or in lieu of, the information from the GPS chipset 136, the WTRU102 may receive location information from base stations (e.g., base stations 114a, 114 b) over the air interface 116 and/or determine its location based on the timing of signals received from two or more neighboring base stations. It should be appreciated that the WTRU102 may obtain location information by any suitable location determination method while remaining consistent with an embodiment.

The processor 118 may also be coupled to other peripheral devices 138, and the removal may include one or more software and/or hardware modules that provide additional features, functionality, and/or wired or wireless connections. For example, the number of the cells to be processed, peripheral devices 138 may include an accelerometer, an electronic compass, a camera satellite transceiver, digital camera (for photo and/or video), camera Universal Serial Bus (USB) port, vibration device, television transceiver, hands-free headset,Module, frequency Modulation (FM) radio unit,Digital music players, media players, video game player modules, internet browsers, virtual reality and/or augmented reality (VR/AR) devices, activity trackers, and the like. The peripheral devices 138 may include one or more sensors, which may be one or more of gyroscopes, accelerometers, hall effect sensors, magnetometers, orientation sensors, proximity sensors, temperature sensors, time sensors; a geolocation sensor; altimeters, light sensors, touch sensors, magnetometers, barometers, attitude sensors, biometric sensors, and/or humidity sensors.

WTRU102 may include a full duplex radio for which transmission and reception of some or all signals (e.g., signals associated with a particular subframe for UL (e.g., for transmission) and downlink (e.g., for reception)) may be concurrent and/or simultaneous. The full duplex radio may comprise an interference management unit that reduces and/or substantially eliminates self-interference by means of hardware, such as a choke coil, or by means of signal processing by a processor, such as a separate processor (not shown) or by means of the processor 118. In embodiments, WTRU102 may include a half-duplex radio that transmits and receives some or all signals (e.g., signals associated with a particular subframe for UL (e.g., for transmission) or downlink (e.g., for reception)).

Fig. 14C is a system diagram illustrating RAN104 and CN106 in accordance with an embodiment. As described above, the RAN104 may employ an E-UTRA radio technology to communicate with WTRUs 102a, 102b, 102c over an air interface 116. RAN104 may also communicate with CN 106.

RAN104 may include enode bs 160a, 160B, 160c, although it should be appreciated that RAN104 may include any number of enode bs while remaining consistent with embodiments. The enode bs 160a, 160B, 160c may each include one or more transceivers to communicate with the WTRUs 102a, 102B, 102c over the air interface 116. In one embodiment, the enode bs 160a, 160B, 160c may implement MIMO technology. Thus, for example, the enode B160a may use multiple antennas to transmit wireless signals to and/or receive wireless signals from the WTRU102 a.

each of the enode bs 160a, 160B, 160c may be associated with a particular cell (not shown) and may be configured to process radio resource management decisions, handover decisions, user scheduling in UL and/or DL, and the like. As shown in fig. 14C, the enode bs 160a, 160B, 160C may communicate with each other via an X2 interface.

The CN106 shown in fig. 14C may include a Mobility Management Entity (MME) 162, a Serving Gateway (SGW) 164, and a Packet Data Network (PDN) gateway (or PGW) 166. While each of the foregoing elements are depicted as part of the CN106, it will be appreciated that any of these elements may be owned and/or operated by entities other than the CN operator.

The MME162 may be connected to each of the enode bs 162a, 162B, 162c in the RAN104 via an S1 interface and may function as a control node. For example, the MME162 may be responsible for authenticating users of the WTRUs 102a, 102b, 102c, bearer activation/deactivation, selecting a particular serving gateway during initial attachment of the WTRUs 102a, 102b, 102c, and so forth. MME162 may provide control plane functionality for switching between RAN104 and other RANs (not shown) employing other radio technologies, such as GSM and/or WCDMA.

SGW164 may be connected to each of the enode bs 160a, 160B, 160c in RAN104 via an S1 interface. The SGW164 may generally route and forward user data packets to/from the WTRUs 102a, 102b, 102 c. The SGW164 may perform other functions such as anchoring the user plane during inter-enodeb handovers, triggering paging when DL data is available to the WTRUs 102a, 102B, 102c, managing and storing the contexts of the WTRUs 102a, 102B, 102c, and so forth.

The SGW164 may be connected to the PGW166, and the PGW166 may provide the WTRUs 102a, 102b, 102c with access to a packet switched network, such as the internet 110, to facilitate communications between the WTRUs 102a, 102b, 102c and IP enabled devices.

The CN106 may facilitate communications with other networks. For example, the CN106 may provide the WTRUs 102a, 102b, 102c with access to a circuit switched network (e.g., the PSTN 108) to facilitate communications between the WTRUs 102a, 102b, 102c and legacy landline communication devices. For example, the CN106 may include an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server), or may communicate with an IP gateway that serves as an interface between the CN106 and the PSTN 108. In addition, the CN106 may provide the WTRUs 102a, 102b, 102c with access to other networks 112, and the other networks 112 may include other wired and/or wireless networks owned and/or operated by other service providers.

Although the WTRU is depicted in fig. 14A-14D as a wireless terminal, it is contemplated that in some representative embodiments such a terminal may use (e.g., temporarily or permanently) a wired communication interface with the communication network.

In representative embodiments, the other network 112 may be a WLAN.

A WLAN in infrastructure Basic Service Set (BSS) mode may have an Access Point (AP) for the BSS and one or more Stations (STAs) associated with the AP. An AP may access or interface to a Distribution System (DS) or another type of wired/wireless network that carries traffic into and/or out of the BSS. Traffic originating outside the BSS and destined for the STA may arrive through the AP and may be delivered to the STA. Traffic originating from the STA and destined for a destination outside the BSS may be sent to the AP to be delivered to the respective destination. Traffic between STAs within a BSS may be transmitted by the AP, for example, where a source STA may transmit traffic to the AP and the AP may deliver the traffic to a destination STA. Traffic between STAs within a BSS may be considered and/or referred to as peer-to-peer traffic. Peer-to-peer traffic may be sent between the source and destination STAs (e.g., directly between the source and destination STAs) using Direct Link Setup (DLS). In certain representative embodiments, the DLS may use 802.11e DLS or 802.11z Tunnel DLS (TDLS). A WLAN using an Independent BSS (IBSS) mode may not have an AP, and STAs (e.g., all STAs) within or using the IBSS may communicate directly with each other. The IBSS communication mode may sometimes be referred to herein as an "ad-hoc" communication mode.

When using the 802.11ac infrastructure mode of operation or similar modes of operation, the AP may transmit beacons on a fixed channel (e.g., a primary channel). The primary channel may be a fixed width (e.g., 20MHz wide bandwidth) or a dynamically set width via signaling. The primary channel may be an operating channel of the BSS and may be used by the STA to establish a connection with the AP. In some representative embodiments, carrier sense multiple access with collision avoidance (CSMA/CA) may be implemented, for example, in an 802.11 system. For CSMA/CA, STAs (e.g., each STA) that include an AP may sense the primary channel. A particular STA may backoff if the primary channel is sensed/detected and/or determined to be busy. One STA (e.g., only one station) may transmit at any given time in a given BSS.

High Throughput (HT) STAs may communicate using 40MHz wide channels, for example, by combining a primary 20MHz channel with an adjacent or non-adjacent 20MHz channel to form a 40MHz wide channel.

Very High Throughput (VHT) STAs may support channels that are 20MHz, 40MHz, 80MHz and/or 160MHz wide. 40MHz and/or 80MHz channels may be formed by combining adjacent 20MHz channels. The 160MHz channel may be formed by combining 8 transmitted 20MHz channels or by combining two discontinuous 80MHz channels, which may be referred to as an 80+80 configuration. For the 80+80 configuration, after channel coding, the data may be passed through a segment parser, which may divide the data into two streams. An Inverse Fast Fourier Transform (IFFT) process and a time domain process may be performed on each stream, respectively. The streams may be mapped onto two 80MHz channels and data may be transmitted by a transmitting STA. At the receiver of the receiving STA, the operation of the 80+80 configuration described above may be reversed and the combined data may be sent to a media control (MAC).

Modes of operation below 1GHz are supported by 802.11af and 802.11 ah. The channel operating bandwidth and carrier are reduced in 802.11af and 802.11ah relative to the channel operating bandwidth and carrier used in 802.11n and 802.11 ac. The 802.11af supports 5MHz, 10MHz, and 20MHz bandwidths in the TV white space (TVWS) spectrum, while the 802.11ah supports 1MHz, 2MHz, 4MHz, 8MHz, and 16MHz bandwidths using non-TVWS spectrum. According to representative embodiments, 802.11ah may support instrumented phenotype control/machine type communications, such as MTC devices in macro coverage areas. MTC devices may have certain capabilities, e.g., limited capabilities including support for certain and/or limited bandwidth (e.g., support only). MTC devices may include batteries having battery lives above a threshold (e.g., to maintain very long battery lives).

WLAN systems, such as 802.11n, 802.11ac, 802.11af, and 802.11ah, which may support multiple channels and channel bandwidths, include channels that may be designated as primary channels. The primary channel may have a bandwidth equal to the maximum common operating bandwidth supported by all STAs in the BSS. The bandwidth of the primary channel may be set and/or limited by an STA among all STAs operating in the BSS (which supports a minimum bandwidth operation mode). In the example of 802.11ah, for STAs (e.g., MTC-type devices) that support (e.g., support only) 1MHz modes, the primary channel may be 1MHz wide, even though the AP and other STAs in the BSS support 2MHz, 4MHz, 8MHz, 16MHz, and/or other channel bandwidth modes of operation. Carrier sensing and/or Network Allocation Vector (NAV) settings may depend on the state of the primary channel. If the primary channel is busy (e.g., because the STA (which only supports 1MHz mode of operation) transmits to the AP), the entire available frequency band may be considered busy even though most of the frequency band remains idle and available.

In the united states, the available frequency band that 802.11ah can use is from 902MHz to 928MHz. In korea, the available frequency band is from 917.5MHz to 923.5MHz. In Japan, the available frequency band is from 916.5MHz to 927.5MHz. The total bandwidth available for 802.11ah is 6MHz to 26MHz, depending on the country code.

Fig. 14D is a system diagram illustrating RAN113 and CN115 according to an embodiment. As described above, RAN113 may communicate with WTRUs 102a, 102b, 102c over an air interface 116 using NR wireless technology. RAN113 may also communicate with CN 115.

RAN113 may include gnbs 180a, 180b, 180c, but it should be understood that RAN113 may include any number of gnbs while remaining consistent with embodiments. Each of the gnbs 180a, 180b, 180c includes one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 116. In one embodiment, gnbs 180a, 180b, 180c may implement MIMO technology. For example, gnbs 180a, 108b may utilize beamforming to transmit signals to and/or receive signals from gnbs 180a, 180b, 180 c. Thus, the gNB180a may use multiple antennas to transmit wireless signals to and/or receive wireless signals from the WTRU102a, for example. In one embodiment, the gnbs 180a, 180b, 180c may implement carrier aggregation techniques. For example, the gNB180a may transmit a plurality of component carriers (not shown) to the WTRU102 a. A subset of the component carriers may be on an unlicensed spectrum while the remaining component carriers may be on a licensed spectrum. In one embodiment, the gnbs 180a, 180b, 180c may implement coordinated multipoint (CoMP) techniques. For example, WTRU102a may receive coordinated transmissions from gNB180a and gNB180b (and/or gNB180 c).

The WTRUs 102a, 102b, 102c may communicate with the gnbs 180a, 180b, 180c using transmissions associated with a scalable parameter configuration (numerology). For example, the OFDM symbol interval and/or OFDM subcarrier interval may vary for different transmissions, different cells, and/or different portions of the wireless transmission spectrum. The WTRUs 102a, 102b, 102c may communicate with the gnbs 180a, 180b, 180c using subframes or Transmission Time Intervals (TTIs) having various or scalable lengths (e.g., including different numbers of OFDM symbols and/or continuously varying absolute time lengths).

The gnbs 180a, 180b, 180c may be configured to communicate with the WTRUs 102a, 102b, 102c in an independent configuration and/or in a non-independent configuration. In a standalone configuration, the WTRUs 102a, 102B, 102c may communicate with the gnbs 180a, 180B, 180c without also having access to other RANs (e.g., enode bs 160a, 160B, 160 c). In an independent configuration, the WTRUs 102a, 102b, 102c may utilize one or more of the gnbs 180a, 180b, 180c as mobility anchors. In a stand-alone configuration, the WTRUs 102a, 102b, 102c may communicate with the gnbs 180a, 180b, 180c using signals in an unlicensed frequency band. In a non-standalone configuration, the WTRUs 102a, 102B, 102c may communicate/connect with the gnbs 180a, 180B, 180c, while also communicating/connecting with another RAN, such as the enode bs 160a, 160B, 160 c. For example, the WTRUs 102a, 102B, 102c may implement DC principles to communicate substantially simultaneously with one or more of the gnbs 180a, 180B, 180c and one or more of the enodebs 160a, 160B, 160 c. In a non-standalone configuration, the enode bs 160a, 160B, 160c may serve as mobility anchors for the WTRUs 102a, 102B, 102c and the gnbs 180a, 180B, 180c may provide additional coverage and/or throughput for serving the WTRUs 102a, 102B, 102 c.

Each of the gnbs 180a, 180b, 180c may be associated with a particular cell (not shown) and may be configured to process radio resource management decisions, handover decisions, user scheduling in UL and/or DL, support network fragmentation, dual connectivity, interworking between NR and E-UTRA, routing user plane data to User Plane Functions (UPFs) 184a, 184b, routing control plane information to access and mobility management functions (AMFs) 182a, 182b, and the like. As shown in fig. 14D, gnbs 180a, 180b, 180c may communicate with each other through an Xn interface.

The CN115 shown in fig. 14D may include at least one of the AMFs 182a, 182b, at least one of the UPFs 184a, 184b, at least one of the Session Management Functions (SMFs) 183a, 183b, and possibly Data Networks (DNs) 185a, 185b. While each of the foregoing elements are depicted as part of the CN115, it will be appreciated that any of these elements may be owned and/or operated by entities other than the CN operator.

AMFs 182a, 182b may be connected to one or more of gNB180a, 180b, 180c in RAN113 via an N2 interface and may act as a control node. For example, the AMFs 182a, 182b may be responsible for authenticating users of the WTRUs 102a, 102b, 102c, supporting network slicing (e.g., handling different PDU sessions with different requirements), selecting a particular SMF183a, 183b, managing registration areas, terminating NAS signaling, mobility management, and so on. The AMFs 182a, 182b may use network slices to customize the support of the WTRUs 102a, 102b, 102c by the CN based on the type of service used by the WTRUs 102a, 102b, 102 c. For example, different network slices may be established for different use cases, such as services that rely on ultra-reliable low latency (URLLC) access, services that rely on enhanced large-scale mobile broadband (eMBB) access, services for Machine Type Communication (MTC) access, and so on. AMF162 may provide control plane functionality for switching between RAN113 and other RANs (not shown) employing other radio technologies (e.g., LTE-A, LTE-a Pro, and/or non-3 GPP access technologies such as WiFi).

The SMFs 183a, 183b may be connected to AMFs 182a, 182b in the CN115 via an N11 interface. The SMFs 183a, 183b may also be connected to UPFs 184a, 184b in CN115 via N4 interfaces. SMFs 183a, 183b may select and control UPFs 184a, 184b and configure traffic routing through UPFs 184a, 184b. The SMFs 183a, 183b may perform other functions such as managing and assigning UE IP addresses, managing PDU sessions, controlling policy enforcement and QoS, providing downlink data notifications, etc. The PDU session type may be IP-based, non-IP-based, ethernet-based, and so forth.

UPFs 184a, 184b may connect to one or more of the gnbs 180a, 180b, 180c in the RAN113 via an N3 interface, which may provide the WTRUs 102a, 102b, 102c with access to a packet-switched network, such as the internet 110, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices. UPFs 184, 184b may perform other functions such as routing and forwarding packets, enforcing user plane policies, supporting multi-homing PDU sessions, handling user plane QoS, buffering downlink packets, providing mobility anchoring, and so forth.

The CN115 may facilitate communications with other networks. For example, the CN115 may include or may communicate with an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that serves as an interface between the CN115 and the PSTN 108. In addition, the CN115 may provide the WTRUs 102a, 102b, 102c with access to other networks 112, and the other networks 112 may include other wired and/or wireless networks owned and/or operated by other service providers. In one embodiment, the WTRUs 102a, 102b, 102c may connect to the local Data Networks (DNs) 185a, 185b through the UPFs 184a, 184b via an N3 interface to the UPFs 184a, 184b and an N6 interface between the UPFs 184a, 184b and the DNs 185a, 185b.

14A-14D and their respective descriptions, one or more or all of the functions described herein with respect to one or more of the following may be performed by one or more emulation devices (not shown): the WTRUs 102a-d, the base stations 114a-B, the enode bs 160a-c, the MME162, the SGW164, the PGW166, the gNB180a-c, the AMFs 182a-B, the UPFs 184a-B, the SMFs 183a-B, the DNs 185a-B, and/or any other device(s) described herein. These simulation devices may be one or more devices configured to simulate one or more or all of the functions described herein. For example, the emulating device may be used to test other devices and/or analog network and/or WTRU functions.

The simulated device may be designed to enable one or more tests of other devices in a laboratory environment and/or an operator network environment. For example, one or more emulated devices may perform one or more or all functions while being fully or partially implemented and/or deployed as part of a wired and/or wireless communication network in order to test other devices within the communication network. One or more emulation devices can perform one or more or all of the functions while being temporarily implemented/deployed as part of a wired and/or wireless communication network. The emulated device may be directly coupled to another device for testing purposes and/or may perform the test using over-the-air wireless communications.

One or more of the emulated devices may perform one or more functions (including all functions) rather than being implemented/deployed as part of a wired and/or wireless communication network. For example, the simulation device may be used in a test laboratory and/or a test scenario in an undeployed (e.g., testing) wired and/or wireless communication network in order to enable testing of one or more components. The one or more simulation devices may be test devices. The emulating device may transmit and/or receive data using direct RF coupling and/or via RF circuitry (e.g., which may include one or more antennas) and/or wireless communication.

The processes described above may be implemented in a computer program, software, and/or firmware incorporated in a computer readable medium for execution by a computer and/or processor. Examples of computer readable media include, but are not limited to, electronic signals (transmitted over wired and/or wireless connections) and/or computer readable storage media. Examples of computer readable storage media include, but are not limited to, read Only Memory (ROM), random Access Memory (RAM), registers, caches, semiconductor memory devices, magnetic media, such as, but not limited to, internal hard disks and removable disks, magneto-optical media, and/or optical media, such as CD-ROM disks and/or Digital Versatile Disks (DVDs). A processor associated with the software may be used to implement a radio frequency transceiver for use in a WTRU, terminal, base station, RNC, and/or any host computer.

Claims

1. A video decoding method, comprising:

selecting a subset of pixels for classifying a current video block comprising a plurality of pixels from the current video block, wherein the subset of pixels is selected by skipping alternating rows of pixels in the current video block in a diagonal direction;

obtaining a plurality of diagonal changes using the subset of pixels selected by skipping alternating rows of pixels in the current video block in the diagonal direction;

determining a diagonal gradient based on a sum of the plurality of diagonal changes;

classifying the current video block for Adaptive Loop Filtering (ALF) based on the diagonal gradient;

performing ALF on the current video block based on the classification of the current video block; and

a frame including the current video block is decoded.

2. A video decoding apparatus comprising:

a processor configured to:

a frame including the current video block is decoded.

3. The video decoding method of claim 1 or the video decoding apparatus of claim 2, wherein the subset of pixels is selected from the plurality of pixels in the current video block by skipping at least one pixel in a vertical direction and at least one pixel in a horizontal direction of the plurality of pixels in the current video block.

4. The video decoding method of claim 1, further comprising: at least one of a first indication that the subset of pixels is to be selected for block classification or a second indication of how the subset of pixels is to be selected is received.

5. The video decoding method of claim 1, further comprising:

Determining a temporal level of a current frame within a coding scheme, wherein the current frame comprises the current video block; and

based on the temporal level of the current frame, determining whether to select the subset of pixels for classifying the current video block, wherein the subset of pixels is determined to be selected based on a condition that the temporal level is highest within the coding scheme.

6. The video decoding device of claim 2, wherein the processor is further configured to: at least one of a first indication that the subset of pixels is to be selected for block classification or a second indication of how the subset of pixels is to be selected is received.

7. A video encoding method, comprising:

a frame including the current video block is encoded.

8. The video decoding method of claim 1 or the video encoding method of claim 7, wherein the sum of the plurality of diagonal changes is associated with a first diagonal direction, the determined diagonal gradient is a first diagonal gradient associated with the first diagonal direction, and the method further comprises:

obtaining a sum of diagonal changes associated with the second diagonal direction; and

a second diagonal gradient associated with the second diagonal direction is determined based on the sum of diagonal changes associated with the second diagonal direction, wherein the current video block is also classified based on the second diagonal gradient.

9. The video decoding method of claim 1 or the video encoding method of claim 7, wherein the diagonal gradient is a first diagonal gradient, and the method further comprises: the vertical gradient, horizontal gradient, and second diagonal gradient are acquired using the subset of pixels selected by skipping alternating rows of pixels in the current video block in the diagonal direction, wherein the current video block is further classified based on the acquired vertical gradient, the acquired horizontal gradient, and the acquired second diagonal gradient.

10. A video encoding apparatus comprising:

a processor configured to:

a frame including the current video block is encoded.