RU2786015C2 - Obtainment of sample for encoding of 360-degree video - Google Patents

Abstract

FIELD: video encoding.

SUBSTANCE: method is proposed for implementation of horizontal geometrical filling on the current sample based on reception of an indication with wrapping permission, which indicates , whether compensation of horizontal wrapping motion is allowed. If compensation of horizontal wrapping motion is allowed based on the indication with wrapping permission, a device for video signal encoding can determine displacement of wrapping of a reference sample for the current sample in an image. Displacement of wrapping of the reference sample can indicate a panorama width. The device for video signal encoding can determine a location of the reference sample relatively to the current sample based on displacement of wrapping of the reference sample, an image width, and a location of the current sample, as well as it can predict the current sample based on the location of the reference sample in a horizontal direction. Refilling or clipping can be used in a vertical direction.

EFFECT: provision of natural interaction of a user with objects in a virtual world with tactile feedback for the user.

27 cl, 24 dwg, 5 tbl

Description

This application claims priority over U.S. Provisional Application No. 62/737,303, filed September 27, 2018, the contents of which are incorporated herein by reference in their entirety.

Prerequisites for the creation of the invention

Virtual Reality (VR) is becoming more and more part of our daily lives. VR is used in many areas, including healthcare, education, social media, industrial design/education, gaming, cinema, shopping, entertainment, and more. VR is of interest to both businesses and consumers because it can provide a fully immersive viewing experience. VR creates a virtual environment surrounding the viewer and generates a true sense of presence for the viewer. The way in which the experience of full reality is provided in the VR environment affects the user experience. For example, interaction with a VR system can be achieved through posture, gesture, gaze, voice, and so on. For natural user interaction with objects in the VR world, haptic feedback for the user can be provided in VR.

Statement of the Invention

Systems, methods, and means are described for performing horizontal geometric filling on the current sample based on receiving a wrapping permission indication that indicates that horizontal wrapping motion compensation is enabled.

The video encoder may receive an image associated with the video content in a bitstream. The video encoding apparatus described herein may include an encoder and/or a decoder. The image may be or may include a packed frame image. The video encoder may receive an indication with permission to wrap. For example, the video encoder may receive an indication to permit wrapping at the sequence level. The tip with enable screwdriving can indicate whether compensation for the horizontal screwdriving movement is enabled or disabled. The wrap-enable indication may be or may include a wrap-enable flag that indicates whether horizontal wrap motion compensation is enabled or disabled.

Based on the wrapping permission indication indicating that the horizontal wrapping motion compensation is enabled, the video encoder may determine the reference sample wrapping offset for the current sample in the picture. In the examples, the video encoder may receive in the bitstream an indication of the wrapping offset of the reference sample. The wrapping offset of the reference sample may be determined based on the offset specification. The wrapping offset of the reference sample can indicate the width of the pan. In examples, the video encoder may receive a size indication indicating the width of an image panorama. For example, the size indication may be or may include the width of the guard band in luma samples. The video encoder may calculate the image panorama width based on the size indication, and may use the calculated width as a reference sample wrapping offset. In the examples, the guard strip may have the same width. In the examples, the guard strip may have different widths.

The video encoder may determine the location of the reference sample with respect to the current sample based on the wrapping offset of the reference sample, the image width, and the location of the current sample. The video encoder may receive a wrapping offset of the reference sample at the sequence level.

In the examples, the video encoder may determine if the original reference sample location (eg, the primary reference sample location) associated with the current sample location is outside the picture. If the original location of the reference sample is outside the picture, the video encoder can calculate the location of the reference sample for the horizontal geometric fill. For example, the video encoder may calculate the location of the reference sample by applying the wrap offset of the reference sample to the horizontal coordinate of the sample associated with the original location of the reference sample.

In examples, a video encoder may calculate an image pan width based on a reference sample wrap offset and image width. The video encoder can identify the location of the current sample in the image. The video encoder can determine if the reference sample's original location is outside the picture. If the reference sample home location is outside the leftmost horizontal border of the picture, the video encoder may determine the reference sample location by adding the reference sample wrap offset to the horizontal sample coordinate associated with the reference sample home location. If the reference sample home location is outside the rightmost horizontal edge of the picture, the video encoder may determine the reference sample location by subtracting the reference sample wrap offset from the horizontal sample coordinate associated with the reference sample home location.

The video encoder may predict the current sample based on the location of the reference sample.

The video encoder may determine whether the compensation of the horizontal screw-in movement is allowed based on the indication with the screw-in permission. If the wrap-enable indication indicates that horizontal wrap motion compensation is disabled, the video encoder can identify the location of the current sample in the image. The video encoder can determine if the location of the current sample is on the horizontal edge of the image. The video encoder can determine if the location of the reference sample is outside the horizontal boundary of the image. If the video encoder determines that the location of the reference sample is outside the horizontal boundary of the picture, the video encoder may identify the location of the reference sample at the horizontal boundary of the picture. The video encoder may predict the current sample based on the location of the reference sample.

The image may be or may include Equidistant Projection (ERP), Equal Area Projection (EAP), or Adjusted Equal Area Projection (AEP) format.

The video encoder may determine whether to receive an offset indication for the wrapping offset of the reference sample based on the wrapping-enabled indication. If the video encoder determines that a wrapping permission indication has been issued, the video encoding device may parse the bitstream to receive an offset indication for the wrapping offset of the reference sample. If the video encoder determines that the wrapping permission indication is not issued, the video encoding device may skip parsing the bitstream and skip receiving the offset indication for the wrapping offset of the reference sample.

An indication may be signaled (eg, in a bitstream) to enable geometric filling. The correct position of one or more neighbors in a spherical neighborhood can be determined. The geometric filling may take into account the position and/or size of the guardbands to calculate the correct position of one or more neighbors in a spherical neighborhood. The position and/or size of the guard bands may be added when one or more panoramas are packaged in an image (eg, before encoding). One or more indications relating to geometric infill may be signaled to indicate whether geometric infill has been applied and/or to indicate guard bands.

For projection geometries that include a panorama (eg, a single panorama), such as ERP, EAP, AEP, and/or similar projection formats, geometric filling can be performed in the horizontal direction. When performing geometric infill in the horizontal direction, you can use repeated infill in the vertical direction. The sample position can be defined as the horizontal coordinate of the sample, which can be wrapped in the encoded image. The vertical coordinate of the sample may be clipped at one or more encoded image boundaries, such as in a repeat fill. An integer sample derivation process and/or a fractional sample interpolation process for external prediction (eg, using horizontal geometric filling) associated with a projection geometry based on a single panorama can be defined.

Brief description of graphic materials

In FIG. 1A shows an example of spherical sampling in longitude and latitude.

In FIG. 1B shows an example of a two-dimensional (2D) plane with an equirectangular projection.

In FIG. 2A shows an example of a three-dimensional (3D) geometry for Cube Mapped Projection (CMP).

In FIG. 2B shows an example of a 2D glider for six CMP panoramas.

In FIG. 3A is an example of an image with filled borders using Equidistant Projection (ERP).

In FIG. 3B shows an example of an image with filled borders using CMP.

In FIG. 4A shows an example infill geometry for ERP.

In FIG. 4B shows an example of a completed ERP image.

In FIG. 5A shows an example infill geometry for CMP.

In FIG. 5B shows an example of filled CMP panoramas.

In FIG. Figure 6 shows an example workflow for getting a 360-degree video.

In FIG. 7 shows an example of a video encoder.

In FIG. 8 shows an example of a video decoder.

In FIG. 9 shows an example of a reference sample used in intra prediction.

In FIG. 10 shows an example of specifying the directions of internal prediction.

In FIG. 11 shows an example of external motion vector prediction (eg, uniprediction).

In FIG. 12 shows an example of filling reference samples outside the image boundary.

In FIG. 13 shows an example of spatial neighbors used in determining candidates for a spatial union.

In FIG. 14 shows an example of caching recovered samples for external prediction using horizontal geometric filling.

In FIG. 15 shows an example of the positions of an integer sample (eg, a shaded block with uppercase letters) and a fractional sample (eg, unshaded blocks with lowercase letters) to interpolate sample brightness with quarter-pixel precision.

In FIG. 16A is a system diagram of an exemplary communications system in which one or more of the described embodiments may be implemented.

In FIG. 16B is a system diagram of an example of a wireless transmit/receive unit (WTRU) that may be used in the communication system shown in FIG. 16A.

Fig. 16C is a system diagram of an example radio access network (RAN) and an example core network (CN) that may be used in the communication system shown in FIG. 16A.

In FIG. 16D is a system diagram of another example of a RAN and an additional example of a CN network that may be used in the communication system shown in FIG. 16A.

Detailed description

The following is a detailed description of illustrative embodiments with reference to various figures. Although the present description provides detailed examples of possible implementation options, it should be noted that this detailed description is given as an example and in no way limits the scope of the application.

The 360-degree videos described herein may include or may be spherical videos, omnidirectional videos, virtual reality (VR) videos, panoramic videos, immersive videos (e.g., light field videos, which may include into itself 6 degrees of freedom), video in a cloud of points and / or the like.

In VR systems, 360-degree videos can be used to provide the user with the ability to view the scene at angles up to 360 degrees in the horizontal direction and at angles up to 180 degrees in the vertical direction. VR and 360-degree videos can be seen as a direction for using multimedia beyond the capabilities of an ultra-high-definition (UHD) display service. To improve the quality of 360-degree videos in VR systems and/or to standardize the information processing sequence for customer interaction, requirements and potential technologies for an omnidirectional multimedia application format can be developed. According to Free view TV (FTV) technology, one or more of the following can be tested: (1) a 360-degree video system (eg, omnidirectional video); (2) multi-species system.

The VR system may include a processing chain. The processing chain may be or include capturing, processing, displaying and/or applying. In terms of capture, the VR system may use one or more cameras to capture scenes from different opposite angles (eg, 6 to 12 angles). Views can be stitched together to form 360-degree video in high definition (eg 4K or 8K). On the client side and/or on the user side, a virtual reality (VR) system may include a computing platform, a head-mounted display (HMD), and/or one or more sensors for tracking the position of the user's head. The computing platform may receive and/or decode the 360-degree video and/or generate a viewport for display. Two images can be displayed for the viewport, one for each eye. The HMD can display two images (eg for stereo viewing). You can use a lens to magnify the image displayed on the HMD for easier viewing. The user's head tracking sensor may store (eg, permanently store) the viewer's head orientation tracking data and/or may send orientation information to the system to display a viewport image for that orientation.

The VR system may include a sensory device for the viewer, for example, to interact with objects in the virtual world. System Control VR can be controlled by a graphics processing unit (GPU)-enabled workstation. The VR system may use a smartphone as a computing platform, an HMD display and/or a head-tracking sensor. The spatial resolution of the HMD may be 2160 x 1200, the refresh rate may be 90 Hz, and/or the field of view (FOV) may be 110 degrees. The sampling density for the user's head tracking sensor can be as high as 1000 Hz, which can capture an image while moving quickly. The VR system may include a lens and/or a board and/or may be controlled by a smartphone.

360-degree video can be compressed and/or delivered, for example, by dynamic adaptive streaming via HTTP-based video streaming (DASH) video streaming techniques. Delivery of 360-degree video can be implemented, for example, using a spherical geometric structure to represent 360-degree information. For example, a plurality of synchronized views captured by multiple cameras may be stitched onto a sphere (eg, as a single structure). Spherical information can be projected onto a flat 2D surface by a geometric shape transformation (eg, in the form of an equidistant projection and/or a cube textured projection).

Equidistant projection can be performed. In FIG. 1A shows an example of a spherical sample in longitudes (ϕ) and latitudes (θ). In FIG. 1B is an example of a sphere projected onto a 2D plane using Equidistant Projection (ERP). Longitude ϕ in the range [-π, π] may be yaw, and latitude θ in the range [-π/2, π/2] may be yaw in aviation. The value of π can be the ratio of the circumference of a circle to its diameter. The (x, y, z) coordinates can represent the coordinates of a point in 3D space. The coordinates (ue, ve) may represent the coordinates of a point in the 2D plane after the ERP. The ERP projection can be represented mathematically, for example, as shown in equations (1) and (2).

ue = (ϕ / (2 * π) + 0.5) * W (one) ve = (0.5 - θ / π) * H (2)

where W and H can be the width and height of the 2D flat image. As shown in FIG. 1A, point P, the intersection point between longitude L4 and latitude A1 on the sphere can be mapped to a unique point q in FIG. 1B on a 2D plane using equations (1) and/or (2). Point q on the 2D plane shown in FIG. 1B can be projected back to point P on the sphere shown in FIG. 1A, for example, by transverse projection. The field of view (FOV) in FIG. 1B is an example of FOV mapping on a sphere to a 2D plane, for example, at an X-axis view angle of about 110 degrees.

One or more 360-degree videos can be mapped to 2D videos. The associated video may be encoded with a video codec (eg, H.264, HEVC, and/or the like) and/or delivered to the client. On the client side, the equidistant video may be decoded and/or displayed based on the user's viewport (eg, by projecting and/or displaying a portion belonging to the FOV onto the equidistant image on the HMD). The spherical video can be converted to a 2D flat image for encoding with ERP. The performance of an equidistant 2D image may differ from that of an unequally spaced 2D image (eg, straight-line video). The top of the image, which may correspond to the north pole, and the bottom of the image, which may correspond to the south pole, can be stretched (for example, when compared with the middle part of the image, which may correspond to the equator). Stretching may indicate that equidistant sampling in the 2D spatial domain may be uneven. The field of motion in a 2D equidistant image can be complex and occur along the time direction.

The left and/or right borders of the ERP image may be encoded (eg independently encoded). Unwanted visual artifacts in the form of panorama seams on the reconstructed video may be created, for example, when the reconstructed video is used to display a viewport, which is then displayed to the user via an HMD or via a 2D screen. Filling N (eg, 8) brightness samples may be applied to the left and/or right sides of the image. An ERP filled image including fill samples may be encoded. Refurbished ERP with infill can be converted back. For example, the recovered ERP with padding can be converted back by mixing duplicated samples and/or trimming the padded areas (eg, after decoding).

The Lambert Cylindrical Equal Area Projection (EAP) can use the same longitude sampling as ERP. Cylindrical Lambert EAP can compensate for the increase in sample density near the poles by decreasing the vertical sample density. In EAP, the vertical sample density can be set equal to cos(φ) and the combined sample density can be constant over the entire sphere. Adjusted Equal Area Projection (AEP) may be a generalization of EAP. AEP can introduce a parameter that can control the resampling rate in the projection.

A projection with a cubic texture can be performed. The top and/or bottom portions of the ERP image, such as those that may correspond to the north pole and south pole, respectively, may be stretched (compared to, for example, the middle portion of the image). This may indicate that the spherical density of the image sample may be non-uniform. The motion field, which can describe the temporal correlation between adjacent ERP images, can be complex. Some video codecs (eg, MPEG-2, H.264 and/or HEVC) may use a translational model to describe the motion field and may not be able to represent reshaping motions in ERP flat images.

Geometric projection formats can be used to map a 360-degree video to multiple panoramas. For example, you can use Cube Mapped Projection (CMP). In FIG. 2A shows an example of a CMP geometric projection. As shown in FIG. 2A, the CMP projection may include six square panoramas, which may be labeled PX, PY, PZ, NX, NY, and/or NZ. P can mean positive and N can mean negative. X, Y and Z can mean axes. Panoramas can be labeled with numbers 0–5. For example, panoramas can be labeled as PX (0), NX (1), PY (2), NY (3), PZ (4), NZ (5). If the radius of the tangent sphere is 1, the side length of each panorama can be 2. Six CMP format panoramas can be packed together (eg, as packed together frames) into an image (eg, one image). The panorama can be rotated by some angle (eg, by a certain angle), which can influence (eg, maximally influence) the continuity between adjacent panoramas. In FIG. 2B shows an example of a packaging in which six panoramas are placed in an equidistant image. The pan index can be placed in the same direction as the corresponding pan rotation. For example, Panorama #3 and Panorama #1 can be rotated counterclockwise by 180 and 270 degrees, respectively. Other panoramas may not be rotated. In examples, the frame packing method may include a 3x2 layout (eg, as shown in FIG. 2B). As shown in FIG. 2B, the top row of 3 panoramas may be spatially adjacent panoramas in a 3D geometry and may have a continuous texture. As shown in FIG. 2B, the bottom row of 3 panoramas may be spatially adjacent panoramas in a 3D geometric structure and may have a continuous texture. The top row of panoramas and the bottom row of panoramas may not be spatially continuous in the 3D geometry, and there may be a seam, such as a broken border, between the two rows of panoramas.

In the CMP projection, if the sample density is 1 in the center of the panorama, the sample density may increase towards the edges. The texture around the edges may be stretched compared to the texture in the center. Cube textured projections (e.g., Conformal Cube Texture Projection (EAC), Corrected Cube Texture Projection (ACP), and/or the like) can correct a panorama (e.g., each panorama) using a non-linear warp function in the vertical and/or or horizontal directions. For example, in an EAC projection, adjustments can be made using the tangent function. In the ACP projection, the adjustment can be made using a second order polynomial function.

A hybrid cube texture projection (CMP) can be performed. In the HCP, the correction function and its parameters can be configured for the panorama as a whole and/or for a specific direction. The cube texture projection format may include a hybrid conformal cube texture (HEC) format. Cubic texture projections can be packed. For example, projections with a cubic texture can be packaged similarly to CMP. Panorama tearing in a packed frame image can occur in a cube textured projection.

Geometric filling can be performed for 360-degree video encoding.

Video codecs can be designed with 2D video captured on the plane. When motion-compensated prediction uses a sample outside the boundaries of the reference image, padding can be performed by copying sample values from the boundaries of the image. This type of padding may be known as repetitive padding. In FIG. 3A and 3B illustrate examples of original image enhancement (eg, bounded by a dotted box) using repeated padding for ERP and CMP, respectively.

A 360-degree video may cover video information on a sphere (eg, the entire sphere) and/or may have a cyclic property. The reference image of a 360-degree video may not have borders. For example, the reference image of a 360-degree video may be wrapped around a sphere (eg, and may not have borders). A cyclic property may exist when presenting 360-degree video on a 2D plane. The cyclic property can exist regardless of which projection format and/or which frame packing implementation is used. Geometric filling can be performed for 360-degree video encoding by filling a sample with 3D geometry.

Geometric filling for ERP can be performed. ERP can be defined on a sphere with some longitude and latitude. If the point (u, v) is to be filled (for example, outside the ERP image), the point (u', v') can be used to output the fill sample. This can be defined as follows:

where W and H can be the width and height of the ERP image.

In FIG. 4A shows an example of a geometric filling process for ERP. Padding can be done outside the left border of the image. For example, as shown in FIG. 4A, samples A, B, and C that are outside the left edge of the image may be filled with samples A', B', and C', which may be within the right edge of the image. Padding can be done outside the right border of the image. For example, as shown in FIG. 4A, samples D, E, and F may be filled with samples D', E', and F', which may be within the left border of the image. Padding can be done beyond the top border of the image. For example, as shown in FIG. 4A, samples G, H, I, and J may be filled with samples G', H', I', and J', which may be within the upper border of the image offset by half the width. Padding can be done beyond the bottom of the image. For example, as shown in FIG. 4A, samples K, L, M, and N may be filled with samples K', L', M', and N', which may be within the lower border of the image offset by half the width. In FIG. 4B shows an example of an enhanced ERP display using geometric filling. As shown in FIG. 4B, the geometric filling may provide continuity between adjacent samples for areas outside the boundaries of the ERP image. Geometric filling for ERP can be simplified. For example, the left and/or right borders of an image may be filled using geometric fill, and the top and bottom borders of an image may be filled using repeat fill. Geometric padding of the left and/or right borders of the image and repeated padding of the top and bottom borders of the image may be desirable, for example, since areas around the equator, such as the left and/or right borders, may include more interesting video content and/or may be viewed by users more often compared to areas near the poles, such as the upper and lower boundaries.

If the encoded image is in CMP format, one or more CMP panoramas may be filled using geometric filling. In FIG. 5A shows an example of a geometric infill performed on a given panorama in a 3D geometric structure. In FIG. 5A, point P may be on the F1 panorama, but may also be outside the F1 panorama. Point P can be filled. As shown in FIG. 5A, the point O may be at the center of the sphere. As shown in FIG. 5A, point R may be a point on the left border that may be closest to P and within panorama F1. As shown in FIG. 5A, point Q may be a projection point of point P on panorama F2 from center point O. Geometric filling may be performed using the sample value at point Q to fill the sample value at point P. In FIG. 5B shows an example of enhanced panoramas using geometric fill for a CMP 3x2 image. As shown in FIG. 5B, padding can be performed on each panorama individually. Geometric fill can provide samples for areas outside the CMP panorama boundary.

Hybrid video coding may be performed. An example implementation of 360 video delivery is shown in FIG. 6. As shown in FIG. 6, an example implementation of 360 video delivery may include capturing 360 video, which can use a plurality of cameras to capture video covering a spherical space (eg, the entire spherical space). Individual videos can be stitched together (for example, stitched together in their own geometric structure). For example, individual videos can be stitched together in ERP format. The native geometry may be converted to one or more projection formats for encoding, eg based on video codecs. At the receiver, the video may be decoded and/or the decompressed video may be converted to geometry for display. The video can be used for display by projecting into the viewport according to the user's viewing angle.

In FIG. 7 shows an example of a hybrid block-based video coding system 600. The input video signal 602 may be processed block by block. Extended block sizes (eg, referred to as a coding unit (CU)) can be used (eg, used in HEVC) to compress high definition video signals (eg, 1080 pixels and/or more). A CU unit can have up to 64x64 pixels (eg in HEVC). The CU may be divided into Prediction Elements (PU) for which separate predictions may be applied. For an input video block (eg, a macroblock (MB) or CU), spatial prediction 660 or motion prediction 662 may be performed. Spatial prediction (eg, or inter-prediction) may use pixels from already encoded adjacent blocks in the same video image and/or tile to predict the current video block. Spatial prediction can reduce the spatial redundancy inherent in the video signal. Motion prediction (eg, called extrinsic prediction or temporal prediction) may use pixels from already encoded video images to predict the current video block. Motion prediction can reduce the temporal redundancy inherent in the video signal. The motion prediction signal for a given video block may be signaled with a motion vector that indicates the amount and/or direction of motion between the current block and its corresponding reference block. If multiple reference pictures are supported (eg, in H.264/AVC, HEVC, and/or the like), the video block reference picture index may be signaled to a decoder. The reference index may be used to identify which reference image in the reference image store 664 may receive a temporal prediction signal.

After spatial prediction and/or motion prediction, a mode decision means 680 in the encoder can select a prediction mode, for example, based on the optimization of the distortion versus bit rate. The prediction block may be subtracted from the current video block at 616. The residual prediction values may be de-correlated using transform module 604 and quantizer 606 to achieve the target bit rate. The quantized residual coefficients may be inversely quantized at 610 and inversely transformed at 612 to form reconstructed residuals. The reconstructed residuals may be added back to the prediction block at 626 to form a reconstructed video block. An intra-loop filter, such as a deblocking filter and/or an adaptive loop filter, may be applied to the reconstructed video block at 666 before it is placed in the reference image store 664. The reference pictures in the reference picture store 664 may be used to encode future video blocks. An output video bitstream 620 may be generated. The coding mode (eg, inter- or intra-coding mode), prediction mode information, motion information, and/or quantized residual coefficients may be sent to entropy coding unit 608 for compression and packing to form a bitstream. 620.

In FIG. 8 shows an example of a block-based hybrid video decoder. The video bit stream 202 may be received, decompressed, and/or entropy decoded at entropy decoding node 208 . The encoding and/or prediction mode information may be sent to the spatial predictor 260 (eg, if intra coding has been performed) and/or to the temporal predictor 262 (eg, if inter coding has been performed). A prediction block may be generated using a spatial predictor 260 and/or a temporal predictor 262 . The residual transform coefficients may be sent to inverse quantizer 210 and to inverse transform 212 to recover the residual block. A prediction block and a residual block may be added at 226. The reconstructed block may pass through the in-loop filter means 266 and may be stored in the reference image store 264. The reconstructed video in the reference image store 264 may be used to drive a display device and/or to predict future video blocks.

A video codec such as H.264 and/or HEVC can be used to encode 2D flat lay video. Video coding may use spatial and/or temporal correlation, for example, to remove redundant information. During encoding of video signals, one or more prediction methods such as intra prediction and/or inter prediction may be applied. Intra-prediction can predict the value of a sample using reconstructed samples adjacent to it. In FIG. 9 shows an example of reference samples that can be used for intra-prediction of the current transform unit (TU). The reference samples may be or include recovered samples located above and/or to the left of the current TU. The reference samples may be or include the left and/or top adjacent recovered samples.

In FIG. 10 shows an example of specifying directions for intra-prediction in HEVC. For example, HEVC may indicate 35 intra prediction modes, which include flat (0), DC (1), and/or angle prediction (2 ~ 34), as shown in FIG. 10. You can select the appropriate intra prediction mode. For example, on the encoder side, an appropriate intra prediction mode can be selected. It is possible to compare the predictions generated by multiple potential intra prediction modes. It is possible to select a potential intra-prediction mode that creates the least distortion between the prediction samples and the original samples. The selected intra prediction mode may be encoded in the bitstream.

Angular predictions can be used to predict directional textures. In FIG. 11 shows an example of motion vector (MV) inter prediction. Blocks B0' and B1' in the reference image may be the corresponding reference blocks for blocks B0 and B1 of the current image. The reference block B0' may be partly outside the boundary of the reference image. Padding can be used to pad unknown samples outside of the image boundaries. In FIG. 12 shows an example of filling reference samples outside the image boundary. For example, padding examples for block B0‘ might contain four parts: P0, P1, P2, and P3. Parts P0, P1 and P2 may be outside the border of the image and may be filled, for example, by padding. Part P0 may be filled with the upper left sample of the reference image. Part P1 may be filled by vertical padding using the topmost row of the reference image. Part P2 can be filled by horizontal filling using the leftmost column of the image.

In the merge mode, MV information from spatial and/or temporal neighbor PUs can be used (eg, reused). Motion vectors for a PU (eg, the current PU) may not be encoded. The encoder and/or decoder may generate a list of candidates for combining motion vectors. For example, a list may be created using MV information from spatial and/or temporal neighbor PUs. In FIG. 13 shows an example of spatial neighbors (eg, bottom left, left, top right, top, and/or top left) used to derive a merge candidate. The index of the selected merge candidate may be encoded and/or signaled. The merge candidate list may be generated by the decoder. The creation of the list by the decoder may be similar (eg, similar) to the creation of the list by the encoder. The merge candidate index entry signaled can be used as the MV for the PU (eg, the current PU).

It is possible to implement geometric filling for 360-degree video encoding. For example, geometric filling for 360-degree video coding can be implemented based on 2D to 3D and 3D to 2D mapping functions. The 2D to 3D and 3D to 2D mapping functions may use division and/or trigonometric functions such as sine, cosine, tangent, and/or the like. When implementing geometric filling, computing resources such as a CPU and/or a storage device may be used. Geometric filling can be implemented in hardware. It is possible to implement division operations with a constant denominator. For example, constant denominator division operations can be implemented using bit shift operations. Variable denominator division operations can be tricky to implement. One or more functions may be implemented using lookup tables (LUTs). LUTs may not be available for the decoding platform. The encoder and/or decoder may store the LUTs in a storage device.

Geometric filling can be implemented by creating a reference image. The number of samples in the reference image may be pre-filled around the edges of the image. One or more prefilled samples may not be used. For example, one or more pre-filled samples may be missing and may not be used if no motion vector in the bitstream belongs to the filled samples. The reference image and/or pre-filled samples may be saved. For example, the reference image and/or pre-populated samples may be stored in a storage device.

In the examples, geometric filling may be implemented based on the derivation of one or more samples when obtaining spherical neighbors, for example, as described herein. For example, one or more samples may be derived from a projection geometry based on a single panorama, such as ERP, EAP, AEP, and/or the like. formats, when using horizontal geometric filling. The cropping operation can be used to limit a sample in a decoded image.

For example, the current block may be divided into one or more subblocks. For example, a block may be divided into one or more sub-blocks based on its motion vectors. It is possible to obtain prediction for a sub-block from different parts of the projected video. For example, if the motion vector takes the left side of the block outside the left edge of the ERP image, this block can be divided into multiple parts, such as two parts. These parts may be inside and/or outside the edge of the ERP image. The part outside the left border can be obtained on the right side of the ERP image, and the part inside the left border can be obtained on the left side of the ERP image.

The video encoder may perform geometric filling. The video encoding apparatus described herein may include an encoder and/or a decoder. For example, a video encoder may perform geometric filling based on deriving one or more samples upon receiving one or more spherical neighbors.

An indication to allow geometric infill, such as horizontal geometric infill, may be signaled. For example, a wrapping permission indication may be signaled to indicate that geometric filling is permitted. The indication with screw-in permission can indicate whether the compensation of the horizontal screw-in movement is enabled. The wrap-enable indication may be or may include a wrap-enable flag that may indicate whether the horizontal wrap motion compensation is enabled. An indication, such as a wrapper permission indication, may be conveyed by bitstream signaling.

The correct position of one or more neighbors in a spherical neighborhood can be determined. For example, the video encoder may determine the correct position of one or more spherical neighbors associated with the current sample based on the reference sample's wrapping offset relative to the current sample. The video encoder may account for an offset, such as a wrapping offset of a reference sample, through geometric filling, and may calculate the correct position of one or more spherical neighbors. For example, the reference sample wrap offset may be or may include information that indicates the panorama width of an image, such as a frame packed image, associated with video content. The video encoder may calculate the correct position of one or more spherical neighbors based on a size indication indicating the panorama width of the packed frame image. The size indication may be or may include the size of the guard band, such as the width of the guard band in luminance samples. Based on the indication of the size, which may be or may include the guard band width, the video encoder may calculate the correct position of one or more spherical neighbors associated with the current sample. The position and/or size of the guard bands may be (eg, could be) added by packaging one or more panoramas in an image. When packing one or more panoramas in an image, such as a packed image, guard bands may be added. For example, when packaging one or more panoramas in an image, guards may be added before encoding. As described herein, one or more indications relating to geometric infill may be signaled to indicate whether horizontal infill has been applied and/or to indicate guard bands.

For projection geometries that include a panorama, such as a projection format based on a single panorama, such as ERP, EAP, AEP, and/or similar projection formats, geometric filling can be performed in the horizontal direction. When performing geometric infill in the horizontal direction, you can use repeated infill in the vertical direction. The sample position may be defined as the horizontal coordinate of the sample, which may be wrapped in the encoded image (eg, see eq. (3)). The vertical coordinate of the sample may be clipped at one or more encoded image boundaries, such as in a repeat fill. An integer sample derivation process and/or a fractional sample interpolation process for external prediction (eg, using horizontal geometric fill) for projection geometries based on a panorama (eg, a single panorama) may be defined.

One or more indications relating to geometric filling may be signaled. As described herein, a wrapping permission instruction may be signaled. The indication with screwdriving permission can indicate whether the compensation of the horizontal screwdriving movement is enabled. The wrapping permission indication may be signaled at the sequence and/or image level. For example, a wrap-enabled indication that indicates geometric fill may be conveyed by sequence-level signaling, such as a sequence parameter set (SPS).

One or more frame packing parameters may be signaled in the bitstream. For example, one or more frame packing parameters may be signaled at the sequence and/or picture level using High Level Syntax (HLS) elements. The position and/or orientation of the panorama of the packed frame image may be signaled in the bitstream. An indication indicating the width of a panorama that is associated with a packed frame image may be transmitted by bitstream signaling. An indication for one or more offsets, where such a size indication indicates the width of the guard band and/or the presence of one or more guard bands, may be signaled in the bitstream. For example, an indication of the size and/or presence of a guard band for one or more pixels (eg, additional pixels) at the boundaries of a panorama may be signaled in the bitstream. The position of the guard bands, such as offsets, may be based on the projection format. For example, for projection geometries based on a single panorama, guardbands can be located on the left and/or right borders of the panoramas. In the examples, the guard strips located on the left and right borders of the panoramas may have the same size. In the examples, the guard bars located on the left and right borders of the panoramas may have different sizes. The dimension indication indicating the guard band width may include guard band widths such as left and right guard bands. For a projection format based on multiple panoramas, such as the CMP format, guardbands may be arranged around a group of continuous panoramas and/or a series of panoramas. If the sizes of the guard strips differ in the projection format based on multiple panoramas, the size indication may include the width of each guard strip.

One or more guard bands and/or geometric padding syntax elements described herein may be signaled. For example, a wrapping permission indication indicating whether compensation for the horizontal wrapping movement is allowed is signaled, for example. The indication may be or may include a wrapping permission flag. The indication may include a size indication indicating the panorama width of the packed frame image, and may be signaled. As described herein, the size indication (eg, indicating the width of the panorama) may be or may include some parameter, such as guard_bands_param_present_flag, indicating the width of the guard band. Table 1 shows an example of geometric padding and guardband syntax elements. Geometric padding and/or guardband syntax elements may be signaled at the sequence level and/or picture level.

Table 1. Geometric filling and guardband syntax

In the examples, table 1 parameters may have the following semantics.

The projection_geometry syntax element may be, or may include, a mapping index (eg, as shown in Table 2) of the projection geometry used.

The guard_bands_param_present_flag syntax element may indicate the presence of the guard_bands_size_in_luma_samples syntax element. If the guard_bands_size_in_luma_samples syntax element is not present, the value of the guard_bands_param_present_flag syntax element may be output as 0.

An indication of the presence of an offset, such as an indication of whether an offset is present, may include information about the offset. For example, an indication of the presence of an offset, such as guard_bands_param_present_flag, may be used to determine if an offset is present in the bitstream. If the indication of the presence of an offset indicates the presence of an offset in the bitstream, a video encoder such as a decoder may parse the bitstream to receive the offset. If the indication of the presence of an offset indicates that there is no offset in the bitstream, the video encoder may skip parsing the bitstream to receive the offset. For example, if an indication of the presence of an offset, such as guard_bands_param_presnt_flag, indicates that there is no offset, the video encoder may skip using the guard band. The leftmost and rightmost edges of the projected panorama may be connected in a sphere, while the edges may not be connected in a packed frame image. The padded/spread samples in the guardband can be encoded in the bitstream. For example, the left guard band information may be obtained from the rightmost region within the panorama, and the right guardband information may be obtained from the leftmost region within the panorama. Recovered samples at the leftmost and rightmost boundaries may be contiguous (eg, recovered samples at the leftmost and rightmost boundaries are treated as adjacent samples when encoding). Samples in the guardband may be discarded when displayed (for example, because the samples do not physically exist).

A size indication indicating the width of an image panorama (eg, images with packed frames) may be received. For example, the size indication may include a syntax element such as guard_bands_size_in_luma_samples. The panorama width of a packed frame image can be calculated based on the specified size. The calculated pan width can be used as a wrapping offset for the reference sample. For example, the syntax element guard_bands_size_in_luma_samples may represent the size in one or more luminance samples of the guard bands used in a packed frame image. The location of the guard bands (eg offsets) can be defined for the projection format. For example, for projection geometry based on a single panorama, such as ERP, EAP, AEP, and/or the like. formats, one or more guard bands can be defined on the left and/or right borders of the panorama. In the examples, the guard bars on the left and right borders of the panorama may have the same width. In the examples, the guard bars on the left and right borders of the panorama may have different widths. For projection geometry based on multiple panoramas, such as the CMP format, one or more guardbands may be defined around a group of panoramas (eg, each group of continuous panoramas or each row of panoramas). The guard_bands_size_in_luma_samples syntax element may not be 0. The guard_bands_size_in_luma_samples syntax element MAY be an integer multiple of MinCbSizeY.

If the geometry_padding_enabled_flag syntax element is equal to 1, the geometry_padding_enabled_flag syntax element may indicate that reference image geometry padding is applied to one or more samples located outside the image boundary. If the geometry_padding_enabled_flag syntax element is 0, the geometry_padding_enabled_flag syntax element may indicate that geometry padding is omitted (eg, not applied). If the geometry_padding_enabled_flag syntax element is absent, the value of the geometry_padding_enabled_flag syntax element may be output as 0.

Table 2 provides examples of projection geometry indexes.

Table 2. Indices of projection geometry

Index Projection geometry 0 Equidistant projection one Projection with cube texture 2 Equal area projection 3 Corrected Equal Area Projection

The actual size of the panorama, such as the panorama width without guardbands, may be calculated based on the signaling information in Table 1 and/or the size of the encoded picture. For example, for projection geometry based on a single panorama, such as ERP, EAP, AEP, and/or similar formats, with guard bands on the left and/or right sides of the encoded image, the actual panorama width can be calculated, for example, using Eq. (6). A parameter, such as specifying a size that specifies the panorama width of a packed frame image, may be or may include a pic_width_in_luma_samples element as described herein. A size indication or a parameter such as pic_width_in_luma_samples may represent the width of the decoded image in units of luma samples. The pic_width_in_luma_samples parameter may be passed through signaling at the sequence level. The height of the panorama can be derived from the encoded image height, for example, in the absence of guard bands on the top and/or bottom sides.

face_width_in_luma_samples = pic_width_in_luma_samples - 2 * guard_bands_size_in_luma_samples (6)

In the examples, the left and right guard strips may be the same size. In the examples, the left and right guard strips may have different sizes. If the left and right guardbands have different sizes, the size (eg, individual size) of the guardband may be signaled, eg, according to Table 3. Table 3 shows an example of the geometric padding and syntax of the guardbands.

Table 3. Geometric filling and guardband syntax

The guard_bands_param_present_flag syntax element may indicate whether at least one of the left_guard_band_width_in_luma_samples, right_guard_band_width_in_luma_samples, or guard_bands_size_in_luma_samples syntax elements is present. If at least one of the syntax elements, such as left_guard_band_width_in_luma_samples, right_guard_band_width_in_luma_samples, or guard_bands_size_in_luma_samples, is missing, the value of the guard_bands_param_present_flag syntax element may be output as 0.

The left_guard_band_width_in_luma_samples syntax element may represent the width in luma samples of a guard band used in a frame packed image at the left panorama border of an image associated with a single panorama projection (eg, ERP, EAP and/or AEP images). The left_guard_band_width_in_luma_samples syntax element can be an integer multiple of MinCbSizeY.

The right_guard_band_width_in_luma_samples syntax element may represent the width in luminance samples of a guard band used in a frame packed image at the right panorama border of an image associated with a single panorama projection (eg, ERP, EAP and/or AEP images). The right_guard_band_width_in_luma_samples syntax element can be an integer multiple of MinCbSizeY.

As described herein, a panorama width of a packed frame image may be calculated based on a size indication that indicates the panorama width of a packed frame image, such as left_guard_band_width_in_luma_samples and/or right_guard_band_width_in_luma_samples. For example, using the representation described herein, the width of a panorama can be calculated, for example, based on Eq. (7).

face_width_in_luma_samples = pic_width_in_luma_samples - (left_guard_band_width_in_luma_samples + right_guard_band_width_in_luma_samples) (7)

In the examples, the guardband width values may be multiples of MinCbSizeY. If guardband widths are multiples of MinCbSizeY, the syntax elements may be specified in units of MinCbSizeY, for example, instead of or in addition to being specified in units of luma samples.

The projection geometry based on one panorama can be filled with geometry based on one or more samples. In inter prediction, the current block in the current picture may be predicted from a reference block in the reference picture. For example, the current block in the current picture may be predicted from the reference block in the reference picture using the MV corresponding to the movement from the current block position to the reference block position. For the position of the sample (x, y) in the current block, the prediction signal I ^pred (x, y) can be obtained from the reference image I ^ref and MV (∆x, ∆y), for example, using eq. (eight).

… (8)

… (eight)

,

и

могут представлять собой арифметические сдвиги вправо и влево соответственно. & может представлять собой побитовый оператор «и».MV can use fractional error p. For example, an MV might use 1/2, 1/4, 1/8, or 1/16 pixel tolerance. The fractional error may use interpolation from one or more available integer sample positions. The position of the sample in the reference image can be represented as an integer part and a fractional part, for example, using eq. (9)–(12).

,

and

can be arithmetic shifts to the right and left, respectively. & can be a bitwise "and" operator.

…(9)

…(10)

…(nine)

…(10)

…(11)

…(12)

…(eleven)

…(12)

находится за пределами границы эталонного изображения, например за пределами изображения с упакованными кадрами, можно применять операцию обрезки для ограничения положения сэмпла в пределах границы кодированного изображения. Это может быть аналогично (например, эквивалентно) выполнению повторяющегося заполнения. Для выборок яркости координаты x_Int и y_Int могут быть обрезаны, например, с использованием ур. (13) и (14) соответственно.The prediction signal may be obtained by interpolating one or more adjacent samples to integer positions. A prediction signal may be obtained to determine a value in a fractional position. If the sample is in integer position

is outside the boundary of the reference image, such as outside the packed frame image, a cropping operation can be applied to constrain the position of the sample within the encoded image boundary. This can be analogous (eg, equivalent) to performing repeated padding. For brightness samples, the x _Int and y _Int coordinates can be clipped, for example, using eq. (13) and (14) respectively.

(13)

(13)

(14)

(fourteen)

For example, if an indication with wrapping permission indicates that horizontal wrapping motion compensation is not allowed, a cropping operation and/or repeated filling may be performed. The location of the current sample in the packed frame image can be identified. The video encoder may determine if the location of the current sample is on the horizontal boundary of the packed frame image. The video encoder may determine if the location of the reference sample is outside the horizontal boundary of the packed frame image. The location of the reference sample on the horizontal boundary of the packed frame image can be identified. For example, as described herein, the location of a reference sample at the horizontal boundary of a packed frame image can be identified based on cropping. The current sample can be predicted based on the location of the reference sample.

в кодированном изображении может быть завернута, например, с использованием ур. (15), где

,

и

, чтобы учесть циклическое свойство 360-градусного видео. Как описано в настоящем документе, указание, такое как указание с разрешением заворачивания, может предоставлять информацию о том, что использовано горизонтальное геометрическое заполнение.In the case of the luminance component, when using horizontal geometric filling, the coordinate

in the encoded image can be wrapped, for example, using eq. (15), where

,

and

to account for the cyclic property of 360-degree video. As described herein, an indication, such as a wrapping permission indication, may provide information that horizontal geometric infill has been used.

(15)

(fifteen)

As described herein, a video encoder can determine if a reference sample's original location (eg, the reference sample's primary location) is outside of a packed frame image. If the reference sample home location is outside the packed frame image, the reference sample location can be calculated by applying the reference sample wrap offset to the horizontal sample coordinate associated with the reference sample home location.

), связанной с исходным местоположением эталонного сэмпла (например, как показано в уравнении 15). Если исходное местоположение эталонного сэмпла находится за пределами крайней правой горизонтальной границы изображения с упакованными кадрами, местоположение эталонного сэмпла может быть определено путем вычитания смещения заворачивания эталонного сэмпла из горизонтальной координаты сэмпла, связанной с исходным местоположением эталонного сэмпла (например, как показано в уравнении 15).In the examples, the width of the panorama of a packed frame image may be determined based on the wrapping offset of the reference sample. The location of the current sample in the packed frame image can be identified. The video encoder may determine if the original location of the reference sample (eg, the primary location of the reference sample) is outside the packed frame image. If the reference sample's original location is outside the leftmost horizontal boundary of the packed frame image, the location of the reference sample can be determined by adding the reference sample's wrap offset to the sample's horizontal coordinate (e.g., coordinate

) associated with the original location of the reference sample (for example, as shown in equation 15). If the reference sample origin location is outside the rightmost horizontal boundary of the packed frame image, the reference sample location can be determined by subtracting the reference sample wrap offset from the horizontal sample coordinate associated with the reference sample origin location (e.g., as shown in Equation 15).

For projection geometry, based on a single panorama (eg, ERP, EAP, AEP, and/or the like formats), the luminance sample position can be obtained, for example, as shown in Table 4. Table 4 provides an example approach for obtaining the luminance sample position . For example, Table 4 provides an example of an integer derivation process and/or a fractional sample interpolation process for external prediction.

Table 4. An example approach to deriving the position of the luminance sample

и

могут быть обрезаны, например с помощью ур. (16) и (17) соответственно.For one or more chrominance samples when using repeated coordinate padding

and

can be cut, for example with the help of ur. (16) and (17) respectively.

(16)

(16)

… (17)

… (17)

в кодированном изображении может быть завернута, например с помощью ур. (18), где

и In the case of the chrominance component, when using horizontal geometric filling, the coordinate

in the encoded image can be wrapped, for example using ur. (18), where

and

, для учета циклического свойства 360-градусного видео.

, to account for the cyclic property of 360-degree video.

(18)

(eighteen)

For projection geometry, based on a single panorama (eg, ERP, EAP, AEP, and/or the like formats), a chroma sample position can be obtained, for example, as shown in Table 5. Table 5 provides an example approach for obtaining a chrominance sample position . For example, Table 5 provides an example of an integer derivation process and/or a fractional sample interpolation process for external prediction.

Table 5. An example approach to deriving the position of a chrominance sample

The sample location wrapping described herein using a modulo operation can be applied to horizontal geometric fill if the projection format used to represent the 360-degree video is a projection format based on a single panorama (e.g., ERP, EAP, AEP, and /or similar projection format). For other projection formats, such as the multi-panoramic projection format (eg, CMP and/or CMP-based projection format), 2D to 3D as well as 3D to 2D matching functions can be implemented. The modulo operation described herein can be applied to a projection format based on multiple panoramas (eg, CMP and/or projection format based on CMP) as an approximation.

The video encoder may determine the position of the sample in the current CU and/or may perform motion compensation at the sample level. For standard dynamic range (SDR) video and/or the like. motion compensation may be performed at the block level (eg, the block may be a CU or a sub-CU). The CU may be divided into multiple parts (eg, two parts) and/or may perform motion compensation using geometric filling for each of the multiple parts (eg, two parts).

In FIG. 14 shows an example of caching recovered samples for external prediction using horizontal geometric filling. In inter-prediction, a sample in a reference image may be accessed more than once because the same area may be referenced by multiple blocks, interpolation processes, and/or refinement processes. When accessing the decoded picture buffer, a portion of the reference picture may be stored in a storage device (eg, local storage) for quick access when performing external prediction, as shown in FIG. 14. The saved area may be a neighboring area centered on the position of the current block and/or the position of the current CTU. The size of the saved area can be limited. If geometric filling is used and the current block is located on the border of the first panorama and/or near it, the saved area can be divided into two sub-regions: the first sub-region, located around the position of the current block in the first panorama; and a second subregion located on the other side of the border of the first panorama, for example on the second panorama adjacent to the first panorama. For example, considering the ERP image shown in FIG. 12, in the reference image, the stored data may include two sub-regions: a first region located around the position of the current block (for example, to predict regions of the two sub-portions P1 and P3 in FIG. 12); and a second region (eg, for predicting the regions of the two sub-parts P0 and P2 in FIG. 12) corresponding to the wrapping data, eg taking into account the cyclic properties of 360-degree video.

In order to predict the part that corresponds to the regions P1 and P3 shown in FIG. 12, one or more prediction samples may be on the left side of the image. If the motion vector contains a fractional sample error, interpolation can be performed and/or neighbors on the right side of the image (eg, regions corresponding to regions P0 and P2 shown in FIG. 12) can be changed by interpolation. Repeated filling may be applied to obtain one or more sample values used in interpolation to predict the regions P1 and P3 shown in FIG. 12.

If interpolation is needed to predict the portion that corresponds to the regions P0 and P2, one or more neighboring samples can be obtained by repeating filling (eg, by repeating one or more samples at the right edge of the image using the 360-degree video looping property). Getting one or more samples from the left border of the image can be skipped. By obtaining two motion vectors for each of the two parts for the first part, which corresponds to the regions P1 and P3, the motion vector may remain unchanged. For the second part, which corresponds to the regions P0 and P2, a modulo operation can be applied to the horizontal component of the motion vector, for example, to achieve a wrapping effect, as described herein.

Data can be accessed from outside the cache. For horizontal geometric filling, the wrapping operation can be limited to skipping getting one or more samples located outside of the stored data. For example, horizontal geometric filling may be performed within a given range S _L (S _L ≤ face_width_in_luma_samples, S _{C =} S _L / SubWidthC), which may be related to the cache size. For one or more samples located outside the saved data, you can apply repeated filling that goes beyond this range. Lv. (15) and (18) can be replaced by eq. (19) and (20) respectively.

(19)

(nineteen)

(20)

(twenty)

When interpolating one or more samples at the fractional sample locations, one or more samples at the full sample locations used by the interpolation filter may be output, for example, as shown in Table 4 and/or Table 5 for the luma and chrominance components, respectively. For example, considering the situation shown in FIG. 15, the brightness samples a _{0, 0} -r _{0, 0} at the fractional sample positions may be interpolated using the brightness samples A _{i, j} at the full sample locations (xA _{i, j} , yA _{i, j} ). For the luma sample A _{i, j} at the full sample locations, xA _{i, j} and yA _{i, j} coordinates can be derived as shown in Table 4 and/or Table 5 for the luma and chrominance components, respectively.

In FIG. 16A is a diagram illustrating an example of a communications system 100 in which one or more of the described embodiments may be implemented. Communication system 100 may be a multiple access system from which multiple wireless users receive content such as voice, data, video, messaging, broadcast, and so on. Communication system 100 may be configured to provide multiple wireless users with access to such content by sharing system resources, including wireless connection bandwidth. For example, communication systems 100 may use one or more channel access techniques such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), multiple Orthogonal Frequency Division Access (OFDMA), Single Carrier FDMA (SC-FDMA), Non-Redundant Extended Discrete Fourier Transform (DFT) Extended OFDM with Sync Burst (ZT UW DTS-s OFDM), Sync Burst OFDM (UW-OFDM) , OFDM with resource block filtering, multi-carrier filter bank (FBMC), and the like.

As shown in FIG. 16A, communication system 100 may include wireless transmit/receive (WTRU) modules 102a, 102b, 102c, 102d, RAN 104/113, CN 106/115, public switched telephone network (PSTN) 108, Internet 110, and others. network 112, although it should be understood that any number of WTRUs, base stations, networks, and/or network elements are contemplated in the described embodiments. Each of the WTRUs 102a, 102b, 102c, 102d may be any type of device capable of operating and/or interacting in a wireless communication environment. For example, WTRUs 102a, 102b, 102c, 102d, any of which may be referred to as a station and/or STA, may be configured to transmit and/or receive radio signals and may include a user equipment (UE), a mobile station, a fixed or mobile indoor unit, indoor unit, pager, cell phone, personal digital assistant (PDA), smartphone, laptop, netbook, personal computer, wireless sensor, Mi-Fi hotspot or device, Internet of Things (IoT) device, watch, or other wearable devices, head-mounted display (HMD), vehicle, radio controlled unmanned aerial vehicle, medical device and applications (e.g. used in remote surgery), industrial device and applications (e.g. robot and/or other wireless devices operating in industrial and/or or automated technological chain), a device related to consumer electronics, a device operating in commercial and/or industrial wireless network, etc. Any of the WTRUs 102a, 102b, 102c, and 102d may be referred to interchangeably as a UE.

Communication systems 100 may also include base station 114a and/or base station 114b. Each of the base stations 114a, 114b may be any type of device configured to wirelessly communicate with at least one of the WTRUs 102a, 102b, 102c, 102d to facilitate access to one or more communication networks, such as CN 106/115, Internet 110 and/or other networks 112. By way of example, base stations 114a, 114b may be base transceiver stations (BTS), Node-Bs, eNode Bs, Home Node Bs, Home eNode Bs, base stations of the next generation (gNB), NodeB based on NR, site controller, access point (AP), wireless router, etc. Although each of the base stations 114a, 114b is shown as a separate element, it should be understood that the base stations 114a, 114b may include any number of interconnected base stations and/or network elements.

Base station 114a may be part of RAN 104/113, which may also include other base stations and/or network elements (not shown), such as a base station controller (BSC), a radio network controller (RNC), relay nodes, etc. P. Base station 114a and/or base station 114b may be configured to transmit and/or receive radio signals on one or more carrier frequencies, which may be referred to as a cell (not shown). These frequencies may be licensed spectrum, unlicensed spectrum, or a combination of licensed and unlicensed spectrum. A cell may provide coverage for a wireless service in a particular geographic area, which may be relatively fixed or which may change over time. The cell may be further divided into cell sectors. For example, a cell associated with base station 114a may be divided into three sectors. Thus, in one embodiment, base station 114a may include three transceivers, such as one for each cell sector. In an embodiment, multiple input multiple output (MIMO) technology may be used at base station 114a and multiple transceivers may be used for each cell sector. For example, beamforming can be used to transmit and/or receive signals in the desired spatial directions.

Base stations 114a, 114b may communicate with one or more of the WTRUs 102a, 102b, 102c, 102d over an air interface 116, which may be any suitable wireless communication system (e.g., for signaling in the radio frequency (RF), microwave spectrum, spectrum centimeter wave, micrometer wave spectrum, infrared (IR), ultraviolet (UV) spectrum, visible light spectrum, etc.). The air interface 116 may be established using any suitable radio access technology (RAT).

More specifically, as noted above, communication system 100 may be a multiple access system and may use one or more channel access schemes such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, and the like. For example, base station 114a in RANs 104/113 and WTRUs 102a, 102b, 102c may implement a radio technology such as Universal Terrestrial Access (UTRA) for Universal Mobile Telecommunications System (UMTS) in which a radio interface 115/116 may be established. /117 using wideband CDMA (WCDMA). WCDMA may include communication protocols such as High Speed Packet Access (HSPA) and/or Enhanced HSPA (HSPA+). The HSPA protocol may include High Speed Downlink (DL) Packet Access (HSDPA) and/or High Speed Uplink (UL) Packet Access (HSUPA).

In an embodiment, base station 114a and WTRUs 102a, 102b, 102c may implement a radio technology such as UMTS Evolved Terrestrial Radio Access (E-UTRA), which may establish a radio interface 116 using a Long Term Evolution (LTE) standard, and /or LTE-Advanced (LTE-A), and/or LTE-Advanced Pro (LTE-A Pro).

In one embodiment, base station 114a and WTRUs 102a, 102b, 102c may implement a radio technology such as New Radio Access Technology (NR Radio Access), which may establish a radio interface 116 using New Radio (NR) technology.

In an embodiment, multiple radio access technologies may be implemented in base station 114a and WTRUs 102a, 102b, 102c. For example, together, base station 114a and WTRUs 102a, 102b, 102c can implement LTE radio access and NR radio access, eg, using dual connectivity (DC) principles. Thus, the air interface used by WTRUs 102a, 102b, 102c may be characterized by the use of multiple types of radio access technologies and/or transmissions directed to/from base stations of multiple types (eg, eNBs and gNBs).

In other embodiments, base station 114a and WTRUs 102a, 102b, 102c may implement radio technologies such as IEEE 802.11 (eg, Wireless Fidelity (WiFi)), IEEE 802.16 (eg, global interoperability for microwave access (WiMAX)), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Interim Standard 2000 (IS-2000), Interim Standard 95 (IS-95), Interim Standard 856 (IS-856), Global System for Mobile Communications (GSM), Enhanced Data Rates for GSM Networks Evolution (EDGE), GSM EDGE (GERAN), etc.

Base station 114b shown in FIG. 16A may be, for example, a wireless router, Home Node B, Home eNode B, or access point, and any suitable RAT may be used to facilitate wireless communication in a localized area such as a business, residential, vehicle, educational institution, industrial facility, air corridor (for example, for use by RC unmanned aerial vehicles), roadway, etc. In one embodiment, a radio technology such as IEEE 802.11 may be implemented in base station 114b and WTRU 102c, 102d to create a wireless local area network (WLAN). In an embodiment, a radio technology such as IEEE 802.15 may be implemented in base station 114b and WTRU 102c, 102d to create a wireless personal area network (WPAN). In yet another embodiment, base station 114b and WTRUs 102c, 102d may use cell-based RATs (eg, WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR, etc.) to create a pico cell, or femtocells. As shown in FIG. 16A, base station 114b may have a direct connection to Internet 110. Thus, base station 114b may not need to access Internet 110 via CN 106/115.

The RAN 104/113 may communicate with the CN 106/115, which may be any type of network configured to provide voice, data, applications, and/or Voice over Internet Protocol (VoIP) services to one or more of the WTRUs 102a , 102b, 102c, 102d. Data may have different quality of service (QoS) requirements, such as different performance requirements, latency requirements, fault tolerance requirements, reliability requirements, data rate requirements, mobility requirements, and the like. The CN 106/115 may provide call control, billing services, location-based mobile communication services, prepaid calls, Internet connectivity, video distribution, and the like. and/or perform high-level security functions such as user authentication. Although in FIG. 16A is not shown, it should be understood that RAN 104/113 and/or CN 106/115 may directly or indirectly communicate with other RANs that use the same RAT as RAN 104/113 or a different RAT. For example, in addition to communicating with RAN 104/113 that may use NR radio technology, CN 106/115 may also communicate with another RAN (not shown) using GSM, UMTS, CDMA 2000, WiMAX, E radio technology. -UTRA or WiFi.

CN 106/115 may also act as a gateway for WTRUs 102a, 102b, 102c, 102d to provide access to PSTN 108, Internet 110, and/or other networks 112. PSTN 108 may include circuit switched telephone networks, that provide traditional telephone services (POTS). Internet 110 may include a global system of interconnected computer networks and devices that use common communication protocols such as Transmission Control Protocol (TCP), User Datagram Protocol (UDP), and/or Internet Protocol (IP) in the TCP/Internet protocol suite. IP. Networks 112 may include wired and/or wireless communications networks that are owned and/or provided for use by other service providers. For example, networks 112 may include another CN connected to one or more RANs that may use the same RAT as RAN 104/113 or a different RAT.

Some or all of the WTRUs 102a, 102b, 102c, 102d in communication system 100 may include multi-mode capabilities (e.g., WTRUs 102a, 102b, 102c, 102d may include multiple transceivers for communicating with different wireless networks over different wireless links ). For example, WTRU 102c shown in FIG. 16A may be configured to communicate with base station 114a, which may use cell-based radio technology, as well as with base station 114b, which may use IEEE 802 radio technology.

In FIG. 16B is a system diagram illustrating an example of WTRU 102. As shown in FIG. 16B, WTRU 102 may include, but is not limited to, processor 118, transceiver 120, transmitter/receiver 122, speaker/microphone 124, keyboard 126, display/touchpad 128, non-removable storage 130, removable storage 132, source 134 power supply, global positioning system (GPS) chipset 136, and/or other peripheral devices 138. It should be understood that WTRU 102 may include any subcombination of the above elements and still conform to the embodiment.

Processor 118 may be a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, application specific integrated circuits (ASICs), user programmable circuits gate arrays (FPGA), integrated circuit (IC) of any other type, state machine, etc. Processor 118 may perform signal coding, data processing, power control, input/output processing, and/or any other functionality by which WTRU 102 operates in a wireless communication environment. Processor 118 may be coupled to transceiver 120, which may be coupled to transmitter/receiver element 122. Although FIG. 16B, processor 118 and transceiver 120 are shown as separate components, it should be understood that processor 118 and transceiver 120 may be integral and incorporated into an electronic unit or chip.

Transmitting/receiving element 122 may be configured to transmit signals to or receive signals from a base station (eg, base station 114a) over air interface 116. For example, in one embodiment, transmitter/receiver element 122 may be an antenna configured to transmit and/or receive RF signals. In an embodiment, transmitter/receiver element 122 may be an emitter/detector capable of transmitting and/or receiving, for example, IR, UV, or visible light signals. In yet another embodiment, transmitter/receiver element 122 may be configured to transmit and/or receive signals in both the RF spectrum and the visible light spectrum. It should be understood that transmitter/receiver element 122 may be configured to transmit and/or receive any combination of wireless signals.

Although in FIG. 16B, transmit/receive element 122 is shown as a single element, WTRU 102 may include any number of transmit/receive elements 122. More specifically, MIMO technology may be used in WTRU 102. Thus, in one embodiment, WTRU 102 may include two or more transmit/receive elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals over air interface 116.

Transceiver 120 may be configured to modulate signals to be transmitted by transmitter/receiver element 122 as well as demodulate signals received by transmitter/receiver element 122. As noted above, WTRU 102 may have multi-mode capabilities. Thus, the transceiver 120 may include a plurality of transceivers by which the WTRU 102 is enabled to interact via a plurality of RATs, such as NR and IEEE 802.11, for example.

The processor 118 of the WTRU 102 may be coupled to a speaker/microphone 124, a keyboard 126, and/or a display/touch panel 128 (eg, a liquid crystal display (LCD) or an organic light emitting diode (OLED) display) and may receive user input from them. Processor 118 may also output user data to speaker/microphone 124, keyboard 126, and/or display/touchpad 128. In addition, processor 118 may access information from any type of suitable storage device, such as non-removable storage device 130 and/or removable storage device 132 and store data thereon. The non-removable storage device 130 may include random access memory (RAM), read only memory (ROM), a hard drive, or any other type of storage device. The removable storage device 132 may include a Subscriber Identity Module (SIM) card, a memory card, a secure digital memory (SD) card, and the like. In other embodiments, the processor 118 may access and store information from a storage device that is not physically located in the WTRU 102 but, for example, on a server or home computer (not shown).

Processor 118 may receive power from power supply 134 and may be configured to manage power and/or distribute power to other components in WTRU 102. Power supply 134 may be any suitable device for supplying power to WTRU 102. For example, power supply 134 may include one or more dry batteries (e.g. nickel cadmium (NiCd), nickel zinc (NiZn), nickel metal hydride (NiMH), lithium ion (Li-ion), etc.), solar cells , fuel cells, etc.

Processor 118 may also be coupled to GPS chipset 136, which may be configured to provide location information (eg, longitude and latitude) relative to WTRU 102's current location. location over the air interface 116 from a base station (eg, from base stations 114a, 114b) and/or determine the location based on the timing of signals received from two or more adjacent base stations. It should be understood that the WTRU 102 may acquire location information by any suitable location method and still comply with the embodiment.

Processor 118 may be further coupled to other peripheral devices 138, which may include one or more software and/or hardware modules that provide additional features, functionality, and/or wired or wireless communication capabilities. For example, peripheral devices 138 may include an accelerometer, an electronic compass, a satellite transceiver, a digital camera (for photography and video), a universal serial bus (USB) port, a vibration device, a television transceiver, a wireless headset, a Bluetooth® module, a radio module with frequency modulation (FM), digital music player, multimedia player, video game playback device module, Internet browser, virtual reality and/or augmented reality (VR/AR) device, activity tracker, etc. Peripheral devices 138 may include one or more sensors, and the sensors may be one or more of a gyroscope, an accelerometer, a Hall sensor, a magnetometer, an attitude sensor, a proximity sensor, a temperature sensor, a time sensor; geographic location sensor; altimeter, ambient light sensor, touch sensor, magnetometer, barometer, gesture sensor, biometric sensor and/or humidity sensor.

The WTRU 102 may include a full duplex radio for which the transmission and reception of some or all of the signals, e.g., associated with particular subframes, for both UL (e.g., for transmitting) and downlink (e.g., for receiving), may be carried out jointly and/or simultaneously. A full duplex radio may include an interference control unit to reduce and/or substantially eliminate its own interference using either hardware (e.g., a choke) or signal processing by a processor (e.g., a separate processor (not shown) or processor 118) . In one embodiment, WRTU 102 may include a half-duplex radio for transmitting and receiving some or all of the signals (eg, associated with specific subframes for either UL (eg, transmit) or downlink (eg, receive)).

In FIG. 16C is a system diagram illustrating RAN 104 and CN 106 in accordance with one embodiment. As noted above, RAN 104 may use E-UTRA radio technology to communicate with WTRUs 102a, 102b, 102c over air interface 116. RAN 104 may also communicate with CN 106.

RAN 104 may include eNode-Bs 160a, 160b, 160c, although it should be understood that RAN 104 may include any number of eNode-Bs and still conform to the embodiment. Each eNode-B 160a, 160b, 160c may include one or more transceivers for communicating with WTRU 102a, 102b, 102c over air interface 116. In one embodiment, MIMO technology may be implemented in eNode B 160a, 160b, 160c. Thus, eNode-B 160a may, for example, use a plurality of antennas to transmit wirelessly to and/or receive wireless signals from WTRU 102a.

Each eNode-B 160a, 160b, 160c may be associated with a specific cell (not shown) and may be configured to make radio resource management decisions, handover decisions, UL and/or DL user scheduling, and the like. As shown in FIG. 16C, eNode-Bs 160a, 160b, 160c can communicate with each other over the X2 interface.

CN 106 shown in FIG. 16C may include a mobility management entity (MME) 162, a serving gateway 164 (SGW), and a packet data network (PDN) gateway 166 (or PGW). While each of the above elements is shown as part of CN 106, it should be understood that any of these elements may be owned by and/or provided to an entity other than the CN operator for use.

MME 162 may be connected to each eNode-B 162a, 162b, 162c in RAN 104 over an S1 interface and may act as a control node. For example, MME 162 may be responsible for user authentication of WTRUs 102a, 102b, 102c, channel activation/deactivation, selection of a particular serving gateway during initial WTRU connection 102a, 102b, 102c, and the like. MME 162 may provide a control plane function for switching between RAN 104 and other RANs (not shown) that use other radio technologies such as GSM and/or WCDMA.

SGW 164 may be connected to each eNode B 160a, 160b, 160c in RAN 104 over an S1 interface. SGW 164 may essentially direct and forward user data packets to and from WTRUs 102a, 102b, 102c. SGW 164 may perform other functions such as user plane binding during handover between eNode Bs, initiating paging when DL data is available to WTRUs 102a, 102b, 102c, managing and storing WTRU context 102a, 102b, 102c, etc. .

SGW 164 may be connected to PGW 166, which may provide WTRUs 102a, 102b, 102c with access to packet-switched networks such as the Internet 110 to facilitate data exchange between WTRUs 102a, 102b, 102c and IP-enabled devices.

CN 106 may facilitate data exchange with other networks. For example, CN 106 may provide WTRUs 102a, 102b, 102c access to circuit switched networks such as PSTN 108 to facilitate data exchange between WTRUs 102a, 102b, 102c and traditional landline communication devices. For example, CN 106 may include an IP gateway (eg, an IP Multimedia Subsystem (IMS) server) that acts as an interface between CN 106 and PSTN 108 or may communicate with it. In addition, CN 106 may provide WTRU 102a, 102b, 102c with access to other networks 112, which may include other wired and/or wireless networks that are owned and/or provided for use by other service providers.

Although the WTRU is described in FIG. 16A-16D as a wireless terminal, it is contemplated that, in certain exemplary embodiments, such a terminal may use (eg, temporarily or permanently) wired communication interfaces with a communications network.

In exemplary embodiments, the other network 112 may be a WLAN.

A WLAN in infrastructure basic service set (BSS) mode may have an access point (AP) for the BSS and one or more stations (STA) associated with the AP. The AP may access or interface with a Distribution System (DS) or communicate over another type of wired/wireless network that carries traffic into and/or out of the BSS. Traffic to the STA generated outside the BSS may arrive via the AP and may be delivered to the STA. Traffic originating from the STA to recipients outside the BSS may be sent to the AP for delivery to the appropriate recipients. Traffic between STAs within a BSS can be sent via an AP, for example, if a source STA can send traffic to an AP and an AP can deliver traffic to a destination STA. Traffic between STAs within a BSS may be considered and/or referred to as peer-to-peer traffic. Peer-to-peer traffic may be transferred between (eg, directly between) a source STA and a sink STA in a forward link (DLS) setup. In certain exemplary embodiments, the DLS may use 802.11e DLS or 802.11z Tunneled DLS (TDLS). A WLAN using independent BSS (IBSS) mode may not have an AP, and STAs (eg, all of STAs) within or using IBSS may communicate directly with each other. In this document, the IBSS mode may sometimes be referred to as the direct connection communication mode.

When using the 802.11ac infrastructure mode of operation or a similar mode of operation, the AP may transmit a beacon on a fixed channel, such as a primary channel. The primary channel may have a fixed width (eg, 20 MHz bandwidth) or a dynamically set signaling width. The primary channel may be a working channel of the BSS and may be used by the STA to establish a connection with the AP. In certain exemplary embodiments, Carrier Sense Multiple Access with Collision Avoidance (CSMA/CA) may be implemented, such as in 802.11 systems. STAs (eg, each STA), including the AP, can discover the primary channel for CSMA/CA. Upon recognizing/detecting and/or determining whether a particular STA's primary channel is busy, that particular STA may turn off. One STA (eg, only one station) can transmit at any given time in a given BSS.

For high throughput (HT) STA communication, a 40 MHz channel can be used, for example, by combining a 20 MHz primary channel with an adjacent or non-adjacent 20 MHz channel to form a 40 MHz channel.

Very High Throughput (VHT) STAs may support 20 MHz, 40 MHz, 80 MHz, and/or 160 MHz channels. The 40 MHz and/or 80 MHz channels may be formed by combining continuous 20 MHz channels. A 160 MHz channel can be formed by combining 8 continuous 20 MHz channels, or by combining two non-continuous 80 MHz channels, which may be referred to as an 80+80 configuration. two streams. Inverse Fast Fourier Transform (IFFT) processing and time domain processing may be performed separately for each stream. The streams may be mapped to two 80 MHz channels and the data may be transmitted by the transmitting STA. At the receiver of the receiving STA, the above operation for the 80 + 80 configuration may be inverted, and the combined data may be sent to a media access control (MAC) device.

802.11af and 802.11ah support 1GHz sub-modes of operation. Channel bandwidths and carriers are reduced in 802.11af and 802.11ah from those used in 802.11n and 802.11ac. 802.11af supports 5 MHz, 10 MHz, and 20 MHz bandwidths in the unused television frequency spectrum (TVWS), while 802.11ah supports 1 MHz, 2 MHz, 4 MHz, 8 MHz, and 16 MHz bandwidths using spectrum other than TVWS. According to the exemplary embodiment, 802.11ah can support metering/machine-to-machine communications, such as MTC devices in a macro coverage area. MTC devices may have certain capabilities, such as limited capabilities, including support (eg, support only) certain and/or limited bandwidths. The MTC devices may include a battery having a battery life exceeding a threshold value (eg, to provide a very long battery life).

WLAN systems that can support multiple channels and channel bandwidths, such as 802.11n, 802.11ac, 802.11af, and 802.11ah, include a channel that can be designated as a primary channel. The primary channel may have a bandwidth equal to the largest total operating bandwidth supported by all STAs in the BSS. The primary channel bandwidth may be set and/or limited by the STA of all STAs operating in the BSS that supports the lowest bandwidth mode of operation. In the 802.11ah example, the primary channel may be 1 MHz wide for STAs (e.g., MTC type devices) that support (e.g., only support) 1 MHz mode, even if the APs and other STAs in the BSS support 2 MHz, 4 MHz, 8 MHz , 16 MHz and/or modes of operation with other channel bandwidths. The carrier detection and/or network allocation vector (NAV) parameters may depend on the state of the primary link. If the primary channel is busy, for example, due to a STA (which only supports 1 MHz mode of operation) transmitting to an AP, all available bands may be considered busy even though most bands are still unoccupied and may be available.

In the United States of America, the available frequency bands that can be used by 802.11ah are between 902 MHz and 928 MHz. The available frequency bands in Korea are from 917.5 MHz to 923.5 MHz. The available frequency bands in Japan are from 916.5 MHz to 927.5 MHz. The total bandwidth available for 802.11ah is between 6 MHz and 26 MHz depending on the country code.

In FIG. 16D is a system diagram illustrating RAN 113 and CN 115 according to an embodiment. As noted above, RAN 113 may use NR radio technology to communicate with WTRUs 102a, 102b, 102c over air interface 116. RAN 113 may also communicate with CN 115.

RAN 113 may include gNBs 180a, 180b, 180c, although it should be understood that RAN 113 may include any number of gNBs and still comply with the embodiment. Each gNB 180a, 180b, 180c may include one or more transceivers for communicating with WTRUs 102a, 102b, 102c over air interface 116. In one embodiment, gNBs 180a, 180b, 180c may implement MIMO technology. For example, gNBs 180a, 108b may use beamforming to transmit signals to and/or receive signals from gNBs 180a, 180b, 180c. Thus, gNB 180a, for example, can use a plurality of antennas to transmit radio signals to and/or receive radio signals from WTRU 102a. In an embodiment, gNBs 180a, 180b, 180c may implement carrier aggregation technology. For example, gNB 180a may transmit multiple carrier components (not shown) to WTRU 102a. A subset of these carriers may be in unlicensed spectrum, while the remaining carriers may be in licensed spectrum. In an embodiment, gNBs 180a, 180b, 180c may implement Point to Point Negotiation (CoMP) technology. For example, WTRU 102a may receive negotiated transmissions from gNB 180a and gNB 180b (and/or gNB 180c).

WTRUs 102a, 102b, 102c may communicate with gNBs 180a, 180b, 180c using scaled-value transmissions. For example, the OFDM symbol spacing and/or OFDM subcarrier spacing may differ for different transmissions, different cells, and/or different portions of the wireless transmission spectrum. WTRUs 102a, 102b, 102c may communicate with gNBs 180a, 180b, 180c using subframes or transmission time intervals (TTIs) of varying or scalable duration (e.g., containing varying numbers of OFDM symbols and/or having constant different absolute time durations) .

gNBs 180a, 180b, 180c may be configured to communicate with WTRUs 102a, 102b, 102c in a standalone configuration and/or a non-standalone configuration. In a standalone configuration, WTRUs 102a, 102b, 102c can communicate with gNBs 180a, 180b, 180c without concurrent access to other RANs (eg, such as eNode-Bs 160a, 160b, 160c). In a standalone configuration, WTRUs 102a, 102b, 102c may use one or more gNBs 180a, 180b, 180c as an anchor point for mobility. In a standalone configuration, WTRUs 102a, 102b, 102c can communicate with gNBs 180a, 180b, 180c using unlicensed band signals. In a non-autonomous configuration, WTRUs 102a, 102b, 102c can communicate/connect with gNBs 180a, 180b, 180c while simultaneously communicating/connecting with another RAN such as eNode-B 160a, 160b, 160c. For example, WTRUs 102a, 102b, 102c may implement DC principles to communicate substantially simultaneously with one or more gNBs 180a, 180b, 180c and one or more eNode-Bs 160a, 160b, 160c. In a non-standalone configuration, eNode-Bs 160a, 160b, 160c may act as a mobility anchor point for WTRUs 102a, 102b, 102c, and gNBs 180a, 180b, 180c may provide additional coverage and/or capacity to serve WTRUs 102a, 102b, 102s.

Each of the gNBs 180a, 180b, 180c may be associated with a particular cell (not shown) and may be configured to make radio resource management decisions, handover decisions, UL and/or DL user scheduling, network segmentation support, dual connectivity , interaction between NR and E-UTRA, routing of user plane data to the user plane functional block (UPF) 184a, 184b, routing of control plane information to the access and mobility management function (AMF) 182a, 182b, and the like. As shown in FIG. 16D, gNBs 180a, 180b, 180c can communicate with each other over an Xn interface.

CN 115 shown in FIG. 16D may include at least one AMF 182a, 182b, at least one UPF 184a, 184b, at least one session management function (SMF) 183a, 183b, and possibly a data communication network (DN) 185a, 185b. ). While each of the above elements is shown as part of CN 115, it should be understood that any of these elements may be owned by and/or made available to an entity other than the CN operator.

AMFs 182a, 182b may be connected to one or more of the gNBs 180a, 180b, 180c in RAN 113 over the N2 interface and may act as a control node. For example, AMF 182a, 182b may be responsible for user authentication of WTRUs 102a, 102b, 102c, network segmentation support (e.g., handling different PDU sessions with different requirements), specific SMF selection 183a, 183b, registration area management, NAS signaling termination, mobility management etc. Network segmentation may be used in AMFs 182a, 182b to configure CN support for WTRUs 102a, 102b, 102c based on the service types used by WTRUs 102a, 102b, 102c. For example, different network slices can be set up for different use cases, such as services based on high reliability low latency communication (URLLC), services based on access to Enhanced Mobile Broadband (eMBB), services for access to machine-to-machine communications ( MTC) and/or the like. AMF 162 may provide a control plane function for switching between RAN 113 and other RANs (not shown) that use other radio technologies such as LTE, LTE-A, LTE-A Pro, and/or non-3GPP access technologies, e.g. WiFi.

SMF 183a, 183b can be connected to AMF 182a, 182b in CN 115 via N11 interface. SMF 183a, 183b can also be connected to UPF 184a, 184b in CN 115 via N4 interface. SMF 183a, 183b can select and control UPF 184a, 184b, and configure traffic routing with UPF 184a, 184b. The SMF 183a, 183b may perform other functions such as UE IP address management and allocation, PDU session management, policy enforcement and QoS management, downlink data notification provision, and the like. The session type of the PDU may be IP based, non-IP based, Ethernet based, and so on.

UPFs 184a, 184b may be connected to one or more gNBs 180a, 180b, 180c in RAN 113 over an N3 interface, which may provide WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, to facilitate data exchange. between WTRUs 102a, 102b, 102c and IP devices. The UPFs 184, 184b may perform other functions such as packet routing and forwarding, user plane policing, support for multipath PDU sessions, user plane QoS processing, downlink packet buffering, anchoring to support mobility, and the like.

CN 115 can facilitate data exchange with other networks. For example, CN 115 may include an IP gateway (eg, an IP Multimedia Subsystem (IMS) server) that acts as an interface between CN 115 and PSTN 108 or may communicate with it. In addition, CN 115 may provide WTRU 102a, 102b, 102c access to other networks 112, which may include other wired and/or wireless networks that are owned and/or provided for use by other service providers. In one embodiment, WTRUs 102a, 102b, 102c may be connected to a local area network (DN) 185a, 185b via UPF 184a, 184b over an N3 interface to UPF 184a, 184b and an N6 interface between UPF 184a, 184b and DN 185a, 185b .

Taking into account FIG. 16A-16D and the corresponding descriptions of FIG. 16A-16D one or more or all of the functions described herein in connection with one or more of: WTRU 102a-d, base station 114a-b, eNode-B 160a-c, MME 162, SGW 164, PGW 166, gNB 180a–c, AMF 182a–b, UPF 184a–b, SMF 183a–b, DN 185a–b and/or any other device(s) described herein , may be implemented by one or more emulation devices (not shown). Emulation devices may be one or more devices configured to emulate one or more or all of the functions described herein. For example, emulation devices can be used to test other devices and/or to simulate network and/or WTRU functions.

The emulation devices may be configured to perform one or more tests on other devices in a lab environment and/or in an operator's network environment. For example, one or more emulation devices may perform one or more functions, or all of the functions, while they are fully or partially implemented and / or deployed as part of a wired and / or wireless communication network, to test other devices in the communication network. One or more emulation devices may perform one or more functions, or all functions, while they are temporarily implemented/deployed as part of a wired and/or wireless communication network. The emulation device may be directly connected to another device for testing and/or performing testing using wireless communication via a wireless communication channel.

One or more emulation devices may perform one or more functions, including all functions, and yet not be implemented/deployed as part of a wired and/or wireless communication network. For example, emulation devices can be used in a test lab scenario and/or in a non-deployed (eg, test) wired and/or wireless communications network to test one or more components. One or more emulation devices may be test equipment. Emulation devices can use a direct RF connection and/or wireless communications via an RF circuit (which may, for example, include one or more antennas) to transmit and/or receive data.

The processes and/or methods described herein may be implemented in a computer program, software, and/or firmware recorded on a computer-readable medium for execution by a computer and/or processor. Examples of computer-readable media include, without limitation, electronic signals (transmitted over wired and/or wireless connections) and/or computer-readable storage media. Examples of computer-readable storage media include, without limitation, read-only memory (ROM), random access memory (RAM), registry, high-speed buffer memory, semiconductor storage devices, magnetic media such as, but not limited to, internal hard drives, and removable discs, magneto-optical media and/or optical media such as CD-ROMs and/or digital versatile discs (DVDs). The processor, in combination with software, may be used to implement an RF transceiver for use in a WTRU, terminal, base station, RNC, and/or any host computer.

Claims (81)

1. Method for decoding video content, including: obtaining an image associated with the video content in the video data; obtaining a wrapping permission indication indicating whether compensation for the horizontal wrapping movement is allowed; based on the indication with the screw-in permission, determining that the compensation of the horizontal screw-in movement is allowed; based on determining that compensation for the horizontal motion of the wrapping is enabled, analyzing the video data to obtain an indication of the offset of the wrapping offset of the reference sample; determining a wrapping offset for the current sample in the image; determining a location of the reference sample with respect to the current sample based on the wrapping offset of the reference sample, the image width, and the location of the current sample; and predicting the current sample based on the location of the reference sample. 2. The method of claim 1, further comprising: based on the determination that compensation for the horizontal movement of the wrapping is not allowed, skipping the analysis of the video data to obtain an offset indication for the wrapping offset of the reference sample. 3. The method of claim 1, wherein the wrapping enable indication is a wrapping enabled flag that indicates whether compensation for the horizontal wrapping motion is enabled. 4. The method of claim 1, wherein the reference sample's wrapping offset is determined based on the indication of the offset, and wherein the reference sample's wrapping offset indicates the image pan width. 5. The method of claim 1, wherein determining the reference sample wrap offset for the current sample in the image includes: obtaining a size indication that is configured to indicate the width of the guard band in the image; and obtaining an image panorama width based on the size indication, wherein the size indication is configured to indicate the guard band width in the brightness samples. 6. The method of claim 5, wherein the guard band width is a first guard band width in luminance samples, wherein the size indication is configured to indicate a second guard band width in luminance samples, the image pan width being obtained based on the first guard band width in luminance samples. luminance samples and a second guardband width in luminance samples. 7. The method of claim 1, wherein the wrapping resolution indication and the wrapping offset of the reference sample are obtained at the sequence level. 8. The method of claim. 1, in which the location of the reference sample includes: obtaining an image panorama width based on a reference sample wrap offset and an image width; identifying the location of the current sample in the image; determining that the original location of the reference sample is outside the image; after determining that the original location of the reference sample is outside the leftmost horizontal border of the image, determining the location of the reference sample by adding a wrapping offset of the reference sample to the horizontal coordinate of the sample associated with the original location of the reference sample; and after determining that the reference sample's home location is outside the rightmost horizontal border of the image, determining the reference sample's location by subtracting the reference sample's wrap offset from the horizontal sample coordinate associated with the reference sample's home location. 9. The method according to p. 1, including: determining that the original location of the reference sample is outside the image; and obtaining the location of the reference sample by applying the wrapping offset of the reference sample to the horizontal coordinate of the sample associated with the original location of the reference sample. 10. The method according to p. 1, including: identifying the location of the current sample in the image; determining that the location of the current sample is on the horizontal border of the image; determining that the location of the reference sample is outside the horizontal boundary of the image; based on the determination that the horizontal wrapping motion compensation is not allowed, identifying the location of the reference sample at the horizontal boundary of the image; and predicting the current sample based on the location of the reference sample. 11. The method of claim 1, wherein the image is associated with an equal area projection (ERP) format, an equal area projection (EAP) format, or an adjusted equal area projection (AEP) format. 12. A device for decoding video content, comprising: a processor configured to: obtaining an image associated with the video content in the video data; obtaining a wrapping permission indication indicating whether the compensation of the horizontal wrapping movement is allowed; based on the indication with the screw-in permission, determining whether the compensation of the horizontal screw-in movement is allowed; based on determining that compensation for the horizontal movement of the wrapping is enabled, analyzing the video data to obtain an indication of the wrapping offset of the reference sample; determining a wrapping offset of the reference sample for the current sample in the image; locating the reference sample with respect to the current sample based on the wrapping offset of the reference sample, the image width, and the location of the current sample; and predicting the current sample based on the location of the reference sample. 13. The apparatus of claim 12, wherein the roll-on indication is a roll-on enable flag that indicates whether horizontal roll-on motion compensation is enabled, the roll-off offset of the reference sample is determined based on the offset indication, and the roll-off offset of the reference sample indicates a pan width. Images. 14. The apparatus of claim 12, wherein in order to determine the reference sample wrap offset for the current sample in the image, the processor is configured to: obtaining a size indication that is configured to indicate the width of the guard band in the image; and obtaining an image panorama width based on the size indication, wherein the size indication is configured to indicate the guard band width in the luminance samples. 15. The apparatus of claim 12, wherein, in order to determine the location of the reference sample, the processor is configured to: obtaining an image panorama width based on the reference sample's wrapping offset and the image width; identifying the location of the current sample in the image; determining that the original location of the reference sample is outside the image; after determining that the original location of the reference sample is outside the leftmost horizontal border of the image, determining the location of the reference sample by adding the wrapping offset of the reference sample to the horizontal coordinate of the sample associated with the original location of the reference sample; and after determining that the reference sample's home location is outside the rightmost horizontal border of the image, determining the reference sample's location by subtracting the reference sample's wrap offset from the sample's horizontal coordinate associated with the reference sample's home location. 16. The apparatus of claim 12, wherein the processor is configured to: based on the determination that compensation for the horizontal movement of the wrapping is not allowed, skip analyzing the video data to obtain an offset indication for the wrapping offset of the reference sample. 17. The apparatus of claim 14, wherein the guard band width is a first guard band width in luminance samples, wherein the size indication is configured to indicate a second guard band width in luminance samples, the image pan width being obtained based on the first guard band width in luminance samples. luminance samples and a second guardband width in luminance samples. 18. Method for encoding video content, including: obtaining an image associated with the video content in the video data; obtaining the location of the current sample, and the location of the current sample is associated with the current sample in the received image; determining the location of the reference sample, which is associated with the current sample; determining that the reference sample is outside the image at a specific location of the reference sample, the width of the image, and the location of the current sample, and the reference sample is associated with the location of the reference sample; and based on the determination that the reference sample is outside the image, including a wrapping resolution indication and an offset indication in the video data, wherein the wrapping resolution indication indicates whether compensation for the horizontal movement of the wrapping is allowed, and an offset indication is provided for the wrapping offset of the reference sample. 19. The method of claim 18, wherein the wrapping permission indications is a wrapping permission flag, and the method includes: including at least one of the indication with wrapping permission, or indication of an offset at the sequence level. 20. The method according to claim 18, including: including an indication of the size of the video data, wherein the size indication is configured to indicate the first guard band width in the image and the second guard band width in the image, the first guard band width in luminance samples and the second guard band width in luminance samples. 21. The method according to claim 18, according to which the method includes: enable wrapping-enabled picking or sequence-level offset picking. 22. A device for encoding video content, comprising: a processor configured to: obtaining an image associated with the video content in the video data; identifying the location of the current sample, and the location of the current sample is associated with the current sample in the received image; determining the location of the reference sample, and the location of the reference sample is associated with the current sample; determining that the reference sample is outside the image, based on the determined location of the reference sample, the width of the image and the location of the current sample, and the reference sample is associated with the location of the reference sample; and based on determining that the reference sample is outside the image, including a wrapping permission indication and an offset indication in the video data, where the wrapping permission indication indicates whether compensation for the horizontal movement of the wrapping is allowed, wherein the offset indication is provided for the wrapping offset of the reference sample. 23. The apparatus of claim 22, wherein the processor is configured to: including at least one indication with wrapping permission or indication of an offset at the sequence level. 24. The apparatus of claim 22, wherein the wrap enable indication is a wrap enable flag, wherein the processor is configured to include at least one wrap enable indication or sequence level offset indication. 25. The apparatus of claim 22, wherein the processor is configured to: include in the video data a size indication that is configured to indicate the first guard band width in the image and the first guard band width in the image, the first guard band width in luminance samples and the second guard band width in luminance samples. 26. Computer-readable storage medium, configured to store instructions for decoding video content, prompting the processor to perform the method according to any one of paragraphs. 1-11. 27. Computer-readable storage medium, configured to store instructions for encoding video content, prompting the processor to perform the method according to any one of paragraphs. 18-21.

Images