EP4674130A1

EP4674130A1 - Payload protocol for holographic communication

Info

Publication number: EP4674130A1
Application number: EP24708149.0A
Authority: EP
Inventors: Johan Kuijpers
Original assignee: Telefonaktiebolaget LM Ericsson AB
Current assignee: Telefonaktiebolaget LM Ericsson AB
Priority date: 2023-02-27
Filing date: 2024-02-26
Publication date: 2026-01-07
Also published as: WO2024179971A1; CN120917756A

Abstract

The techniques described herein use a Holographic Communication Transport Protocol (HCTP) to communicate the color and depth frames (204, 202) associated with a captured image, as well as specific frame-related information that is used to process the color and depth frames in a communication network (10). The techniques also support the transmission of compressed point cloud data (212).

Description

PAYLOAD PROTOCOL FOR HOLOGRAPHIC COMMUNICATION

TECHNICAL FIELD

[001] The present disclosure relates generally to processing data in communications networks, and more particularly to communicating and processing data color and depth frame data in communications networks.

BACKGROUND

[002] Virtual/remote meetings have recently become a crucial communication method in modern-day personal interactions. Services that facilitate such personal interactions, therefore, need to provide a very high quality of experience (QoE) to end users. Ideally, the QoE for end users will mimic the degree of person-to-person interaction in real-life communication as closely as possible.

[003] Currently available 3-Dimensional (3D) extended Reality (XR) technology can be leveraged to provide a higher level of immersion for users involved in virtual/remote interactions. To accomplish this goal, 3D XR technology uses transferred point cloud data to enable the construction of 3D representations of shapes and objects. However, there are certain requirements for implementing conventional 3D communication solutions. These requirements define the encoding/decoding, processing, and transferring of 3D data from peer-to-peer, and necessarily demand higher resource consumption and bandwidth when compared to more conventional 2D communication solutions. Therefore, how to provide a cost-effective and lightweight 3D communication solution that can be widely implemented and adopted by users remains a challenge.

SUMMARY

[004] Aspects of the present disclosure configure apparatuses to utilize a Holographic Communication Transport Protocol (HCTP) to communicate color and depth frames associated with a captured image, as well as specific frame-related control information that is used to process the color and depth frames in a communication network.

[005] A first aspect of the present disclosure comprises a method, implemented by a network node in a communications network, for enabling holographic communications. In this aspect, the method calls for the network node receiving a plurality of color frames and a plurality of depth frames for an image. Each frame is received in a corresponding data packet of a data stream and comprises a payload and a header. The payload comprises color information for the color frames or depth information for the depth frames. The header comprises frame control information for processing the payload. The method then calls for pairing the color frames with the depth frames according to the frame control information to generate corresponding color/depth frame pairs, generating point cloud data from the color/depth frame pairs, and sending the point cloud data and the frame control information to a receiving terminal. [006] A second aspect of the present disclosure provides a method, implemented by a user terminal configured as a content producer, for enabling holographic communications across a communications network. In this aspect, the method calls for the user terminal obtaining a plurality of color frames and a plurality of depth frames for an image and generating a plurality of data packets. Each data packet is generated to comprise a payload and a header. The payload comprises color information for the color frames or depth information for the depth frames. The header comprises frame control information for processing the payload at the network node. The method also calls for the user terminal sending the plurality of data packets to the network node in the communications network.

[007] A third aspect of the present disclosure provides a method, implemented by a user terminal configured as a content receiver, for enabling holographic communications across a communications network. In this aspect, the method calls for the user terminal receiving a stream of data packets from a network node in the communications network. Each data packet comprises a payload and a header. The payload comprises point cloud data representing a color/depth frame pair associated with an image. The header comprises frame control information for processing the point cloud data. So received, the method calls for the user terminal generating a 3-dimensional (3D) representation of the image from the point cloud data according to the frame control information, and rendering the 3D representation of the image to a display device.

[008] A fourth aspect of the present disclosure provides a network node in a communications network for enabling holographic communications. The network node in this aspect is configured to receive a plurality of color frames and a plurality of depth frames for an image. Each frame is received in a corresponding data packet of a data stream and comprises a payload comprising color information for the color frames or depth information for the depth frames, and a header comprising frame control information for processing the payload. The network node is also configured to pair the color frames with the depth frames according to the frame control information to generate corresponding color/depth frame pairs, generate point cloud data from the color/depth frame pairs, and send the point cloud data and the frame control information to a receiving terminal.

[009] An fifth aspect of the present disclosure provides a network node in a communications network for enabling holographic communications. In this aspect, the network node comprises processing circuitry and memory configured to store instructions executable by the processing circuitry. The instructions, when executed, configured the processing circuitry to receive a plurality of color frames and a plurality of depth frames for an image. Each frame is received in a corresponding data packet of a data stream and comprises a payload comprising color information for the color frames or depth information for the depth frames, and a header comprising frame control information for processing the payload. The instructions also configure the processing circuitry to pair the color frames with the depth frames according to the frame control information to generate corresponding color/depth frame pairs, generate point cloud data from the color/depth frame pairs, and send the point cloud data and the frame control information to a receiving terminal.

[010] A sixth aspect of the present disclosure provides a computer program comprising executable instructions that, when executed by a processing circuit in a network node in a wireless communication network, causes the network node to perform the method of the first aspect.

[011] A seventh aspect of the present disclosure provides a carrier containing a computer program of the fifth aspect. The carrier is one of an electronic signal, optical signal, radio signal, or computer readable storage medium.

[012] A eighth aspect of the present disclosure provides a non-transitory computer-readable storage medium containing a computer program comprising executable instructions that, when executed by a processing circuit in network node in a wireless communication network causes the network node to perform the method of the first aspect.

[013] A ninth aspect of the present disclosure provides a user terminal for enabling holographic communications across a communications network. In this aspect, the user terminal functions as a content producer and is configured to obtain a plurality of color frames and a plurality of depth frames for an image and generate a plurality of data packets. Each generated data packet comprises a payload comprising color information for the color frames or depth information for the depth frames and a header comprising frame control information for processing the payload at the network node. The user terminal is also configured in this aspect to send the plurality of data packets to the network node in the communications network.

[014] A tenth aspect of the present disclosure provides a user terminal for enabling holographic communications across a communications network. The user terminal in this aspect is configured to function as a content producer and comprises processing circuitry and memory configured to store instructions executable by the processing circuitry. The instructions, when executed by the processing circuitry, configures the processing circuitry to obtain a plurality of color frames and a plurality of depth frames for an image and generate a plurality of data packets. Each data packet comprises a payload comprising color information for the color frames or depth information for the depth frames, and a header comprising frame control information for processing the payload at the network node. Additionally, the instructions, when executed by the processing circuitry, configures the user terminal to send the plurality of data packets to the network node in the communications network.

[015] An eleventh aspect of the present disclosure provides a computer program comprising executable instructions that, when executed by a processing circuit in a user terminal in a wireless communication network, causes the user terminal to perform the method of the second aspect. [016] A twelfth aspect of the present disclosure provides a carrier containing a computer program of the eleventh aspect. The carrier is one of an electronic signal, optical signal, radio signal, or computer readable storage medium.

[017] A thirteenth aspect of the present disclosure provides a non-transitory computer- readable storage medium containing a computer program comprising executable instructions that, when executed by a processing circuit in a user terminal in a wireless communication network causes the user terminal to perform the method of the second aspect.

[018] In a thirteenth aspect, the present disclosure provides a user terminal for enabling holographic communications across a communications network. In this aspect, the user terminal is configured to function as a content receiver and is configured to receive a stream of data packets from a network node in the communications network. Each data packet comprises a payload comprising point cloud data representing a color/depth frame pair associated with an image and a header comprising frame control information for processing the point cloud data. Additionally, the user terminal is configured to generate a 3-dimensional (3D) representation of the image from the point cloud data according to the frame control information, and render the 3D representation of the image to a display device.

[019] In a fourteenth aspect, the present disclosure provides a user terminal for enabling holographic communications across a communications network. The user terminal in this aspect is configured to function as a content receiver and comprises processing circuitry and memory configured to store instructions executable by the processing circuitry. When executed by the processing circuitry, the instructions configure the processing circuitry to receive a stream of data packets from a network node in the communications network. Each data packet comprises a payload comprising point cloud data representing a color/depth frame pair associated with an image and a header comprising frame control information for processing the point cloud data. Additionally, when executed by the processing circuitry, the instructions configure the processing circuitry to generate a 3-dimensional (3D) representation of the image from the point cloud data according to the frame control information and render the 3D representation of the image to a display device.

[020] A fifteenth aspect of the present disclosure provides a computer program comprising executable instructions that, when executed by a processing circuit in a user terminal in a wireless communication network, causes the user terminal to perform the method of the third aspect.

[021] A sixteenth aspect of the present disclosure provides a carrier containing a computer program of the fifteenth aspect. The carrier is one of an electronic signal, optical signal, radio signal, or computer readable storage medium.

[022] A seventeenth aspect of the present disclosure provides a non-transitory computer- readable storage medium containing a computer program comprising executable instructions that, when executed by a processing circuit in a user terminal in a wireless communication network causes the user terminal to perform the method of the third aspect.

BRIEF DESCRIPTION OF THE DRAWINGS

[023] Figure 1 illustrates a call flow for cloud-based 3-Diemensional (3D) extended Reality (XR) communications.

[024] Figure 2 illustrates a Web Real Time Communications (RTC) Framework Architecture.

[025] Figure 3 illustrates a format for a Stream Control Transmission Protocol (SCTP) packet.

[026] Figures 4A-4C illustrates architecture stacks for SCTP.

[027] Figure 5 illustrates a mesh element defining an object in 3D space.

[028] Figure 6 illustrates a UV mapping projecting a 2D image onto the surface of a 3D model for texture mapping.

[029] Figure 7 illustrates a texture of a Red Green Blue (RGB) image of an object.

[030] Figure 8 illustrates different angles of a hologram generated from a combination of UV, mesh, and texture data.

[031] Figure 9 illustrates an exemplary Hologram Communication Transport Protocol (HCTP) stack according to one embodiment of the present disclosure.

[032] Figure 10 illustrates an exemplary structure for a HCTP Protocol Data Unit (PDU) according to one embodiment of the present disclosure.

[033] Figure 11 illustrates an exemplary communications system for supporting XR communications according to one embodiment of the present disclosure.

[034] Figure 12 illustrates the contents of exemplary headers in a color frame and a depth frame according to one embodiment of the present disclosure.

[035] Figure 13 is a signaling diagram illustrating exemplary signaling in a cloud-based 3D XR communications session according to one embodiment of the present disclosure.

[036] Figure 14 illustrates exemplary processing of HCTP data packets according to one embodiment of the present disclosure.

[037] Figure 15 is a flow diagram illustrating an exemplary method, implemented at a network node, for enabling holographic communications in a communications network according to one embodiment of the present disclosure.

[038] Figure 16 is a flow diagram illustrating an exemplary method, implemented at a user terminal functioning as a content producer, for enabling holographic communications in a communications network according to one embodiment of the present disclosure.

[039] Figure 17 is a flow diagram illustrating an exemplary method, implemented at a user terminal functioning as a content receiver, for enabling holographic communications in a communications network according to one embodiment of the present disclosure.

[040] Figure 18 is a block diagram illustrating some components of a user terminal configured as a content producer or a content receiver according to one embodiment of the present disclosure. [041] Figure 19 is a block diagram illustrating some components of a network node configured according to one embodiment of the present disclosure.

[042] Figure 20 illustrates an example of a communication system in accordance with some embodiments.

[043] Figure 21 is a block diagram of a host in accordance with various aspects described herein.

[044] Figure 22 shows a communication diagram of a host communicating via a network node with a UE over a partially wireless connection in accordance with some embodiments.

DETAILED DESCRIPTION

[045] Figure 1 illustrates a call flow for cloud-based 3-Dimensional (3D) extended Reality (XR) communications. As seen in Figure 1, a cloud-based system 10 comprises a 3D camera 12, a first user terminal 14 configured to function as a content producer, a data communications network 16 communicatively connecting the first user terminal 14 to the cloud network 18, and a second user terminal 20 equipped with, or communicating with, a display device, such as a pair of XR glasses 22.

[046] As stated previously, the requirements for implementing 3D communication solutions demand higher resource consumption and bandwidth when compared to more conventional 2D communication solutions. Therefore, heavy processing tasks, such as those related to the generation of point cloud data, for example, can be offloaded to the cloud. This helps system 10 to cope with the high processing demands of a 3D communication solution, as well as to reduce the required bandwidth between peer devices, such as the first and second user terminals 14, 16.

[047] Particularly, 3D camera captures 3D images and sends live data frames (i.e. , RGB frames and depth frames) associated with those images to the first user terminal 14. As seen in Figure 1 , 3D camera 12 may also use a microphone (not shown) to capture audio data associated with the live data frames to send to the first user terminal 14. Upon receipt, the first user terminal 14 processes and encodes the RGB frames, the depth frames, and the audio data, and communicates both a compressed stream of RGB and depth frames and the audio data to a network node in the cloud 18.

[048] The network node then decodes and processes the RGB frames, depth frames, and the audio data before encoding those frames for delivery to the second user terminal 20.

Particularly, the network node generates point cloud data from the RGB and depth frames and sends that point cloud data with the audio data to the second user terminal 20. The second user terminal 20, upon receipt, decodes the point cloud data and the audio data for rendering on XR glasses 22.

[049] The document WebRTC 1.0 entitled “Real-Time Communication Between Browsers,” W3C Recommendation 26 January 2021, which is expressly incorporated herein in its entirety, defines a framework for real-time communications (RTC). The WebRTC technology described in this document enables the transmission of media (i.e. , audio and video data) and generic data between peer devices. Generally, the WebRTC technology is implemented as an open web standard and is available on modern browsers as well as on native clients for platforms such as ANDROID and iOS.

[050] Figure 2 illustrates an exemplary framework architecture 30 for WebRTC. As seen in this figure, the framework architecture comprises mainly signaling and peer-to-peer connections. Generally, the WebRTC Application Programming Interface (API), which enables applications executing on peer devices to communicate data, does not provide any specific implementation for signaling. However, for peer-to-peer connections, the framework architecture provides a Media API implementation to facilitate the transmission of audio data and video data based on the Stream Control Transmission Protocol (SCTP), as well as a Data API for the communication of arbitrary data (i.e., non-media data) based on SCTP.

[051] The SCTP is defined in IETF Standard RFC 9260 entitled “Stream Control Transmission Protocol,” June 2022, which is incorporated herein by reference in its entirety. In this standard, SCTP is defined as a reliable message-oriented transport protocol that operates on top of a connectionless packet network (e.g., an IP network). In general, SCTP provides congestion and flow control for the transmission of user messages between peer devices, as well as configurable reliability and delivery. The SCTP protocol also provides additional features, such as multi-streaming and multi-homing. In this sense, SCTP is considered as a transport mechanism tailored for specific use cases that cannot be implemented over other transport protocols such as TCP or UPD. Moreover, SCTP is also commonly used in real-time communication cases where Real-Time Transport Protocol (RTP)/Secure Real-time Transport Protocol (SRTP) does not support the required data formats to be transmitted.

[052] Figure 3 illustrates an exemplary SCTP packet format 40 comprising a common header and payload chunks. Each chunk further comprises either SCTP control data or user data along with a specific, respective chunk header that defines a chunk type, a chunk flag, and a chunk length. Additionally, fragmentation and reassembly are supported to fit a user data message into SCTP packets. This ensures that the size of each SCTP packet will not exceed that of a Path Maximum Transmission Unit (PMTU), thereby avoiding the need for fragmentation at the IP level.

[053] There are various approaches for supporting the implementation of SCTP, such as the protocol stacks 50, 60, 70 illustrated in Figures 4A-4C. A first approach (i.e., first protocol stack 50 in Figure 4A) implements SCTP directly over the IP network layer. A second approach (i.e., second protocol stack 60 in Figure 4B) encapsulates SCTP over UDP to mitigate network native support issues. A third approach (i.e., third protocol stack 60 in Figure 4C) implements SCTP over Datagram Transport Layer Security (DTLS) over User Datagram Protocol (UDP) to add support for secure communications (e.g., as implemented on WebRTC Data API). [054] Figure 5 illustrates a mesh topology 80 captured by a depth camera, such as camera 12, for example. As is known in the art, a mesh is an element that defines the shape of an object in 3D space. There are a variety of known mesh topologies with which to build a mesh. For example, a polygon mesh uses polygons to build a model of a 3D object. With a polygon mesh, reference points in the X, Y, and Z axes define shapes with height, width, and depth. A tringle mesh, which is a type of polygon mesh, builds a model of a 3D object using triangles connected at their common vertices and edges. A line mesh uses lines to build a model of a 3D object in which each end of a line is connected to an end of another line. A point mesh builds a model of a 3D object using point clouds (i.e. , 3D coordinates that define the shape of the 3D object). A type indicates the way the surface of the mesh is created - i.e., using triangulation, points, or lines, where each line is composed of two vertex indices and so on.

[055] Figure 6 illustrates the result of a UV mapping 90, which is the 3D modeling process for projecting a 2D image onto the surface of a 3D model to create a texture mapping. UV texturing permits polygons that make up a 3D object to be painted with color from an image. With a UV mapping process, pixels in the image are assigned to surface mappings on the polygon and the rendering computation uses the UV texture coordinates to determine how to paint the three- dimensional surface.

[056] The image, such as that shown in Figure 7, is called a texture 100. Particularly, texture 100 seen in Figure 7 is a RGB image of an object correlated with a mesh angle. The combination of mesh, texture, and UV mappings creates a 3D representation or a hologram. Figure 8, for example, illustrates a hologram 110 of the person seen in the image of Figure 7 captured from different angles.

[057] In existing real-time communication frameworks, it is possible to send both RGB/depth and point cloud data over the SCTP protocol through many different types of implementations. However, challenges still remain. For example, conventional techniques do not currently provide a means for describing specific information regarding transmitted packets containing RGB/depth frames and point cloud data. Such information is required to enable cloud-based 3D XR communications, and therefore, conventional techniques do not provide the ability for a destination device to correctly re-group, pair, and/or process the RGB/depth frames and point cloud data.

[058] The present disclosure, however, defines a new application-level protocol (i.e., a Holographic Communication Transport Protocol (HCTP)) to address such issues. One example of such a protocol is the Ericsson Holographic Communication Transport Protocol (EHCTP) developed by ERICCSON. The HCTP not only enables the transmission of RGB/depth frames along with specific frame-related information to a destination device, but also supports the transmission of compressed point cloud data. According to the present disclosure, the HCTP is implemented over the standard SCTP transport protocol, represented in this case over one of its supported implementations such as the WebRTC data channel, as communication media. [059] Figure 9 illustrates a HCTP protocol stack 120 according to one embodiment of the present disclosure. As seen in Figure 9, the HCTP layer 122 is defined at the application layer and is implemented over the SCTP transport layer. In this specific case SCTP payload 124 is encapsulated into the DTLS and UPD protocols 126, 128, respectively. The full UDP data payload is then further encapsulated into the IP network layer 130 for its transmission over the network. As described previously described with respect to Figures 4A-4C, other SCTP implementations are also possible.

[060] Figure 10 illustrates an HCTP packet structure 140 at the application layer according to one embodiment of the present disclosure. As seen in Figure 10, packet structure 140 comprises a version field 142, a type field 144, a flags field 146, a FramelD field 148, a timestamp field 150, a FrameLength field 152, a PacketLength field 154, and a payload data field 156.

[061] The version field 142 is a 1 byte field that contains a value indicating the version of the HCTP (e.g., version 0x11).

[062] The type field 144 is also 1 byte and contains a value indicating the type of packet for structure 140. The packet type is part of the PDU header describing the contents of the payload. Table 1 lists some possible values and their meanings according to one embodiment.

Table 1

Of course, other values are also possible, as needed or desired.

[063] The flags field 146 is 2 bytes and contains flags for frame reconstruction. Table 2 lists some possible values and their meanings according to one embodiment.

Table 2

As with Table 1, other values not specifically seen in Table 2 are also possible as needed or desired.

[064] The FramelD field 148 is 4 bytes and contains a value indicating the original frame identifier of the camera frame. As described in more detail later, the FramelD field 148 is utilized in the present embodiments for pairing the RGB and depth frames.

[065] The Timestamp field 150 is 8 bytes and contains the camera timestamp data (e.g., converted into UNIX epoch format).

[066] The FrameLengthfield 152 is 4 bytes and contains a value that indicates the total length of the frame data.

[067] The PacketLength field 154 is 4 bytes and contains a value indicating the total length of the encoded data packet.

[068] The Packet Data field 156 is less than 65512 bytes and carries the HCTP payload, which comprises the color/depth/mesh data. According to the present disclosure, additional headers may also be included in structure 140 depending on the type of SCTP implementation. Such headers may include, but are not limited to, an SCTP header (e.g., 16 bytes), a UDP header (e.g., 8 bytes), and an IP header (e.g., 20 bytes).

[069] Additionally, as described herein, the HCTP header is 24 bytes long and is attached to the HCTP payload in order to construct the HCTP data packet. As an example, HCTP version 0x11 implies usage of the H264 codec for color data Packet Data units (PDUs) and the WebP codec for depth data PDUs.

[070] According to the present embodiments, a user terminal configured to function as a content provider or content producer implements the HCTP to send both RGB frames and depth frames associated with an image independently of each other in data packets with frame data being encapsulated together with frame-specific information. Upon receiving the data packets in the cloud, a network node configured according to the present embodiments implements the HCTP to reconstruct and assemble the RGB frames and depth frames (i.e. , depth-color pairing) based on the frame-specific information embedded in the packet headers. Point cloud data can then be generated and compressed and transmitted in data packets to a receiving user terminal using the same HCTP implementation. The receiving user terminal can then decompress and reconstruct the point cloud data from received data packets using the HCTP. As a result, the embodiments described herein enable the implementation of cloud-based 3D XR communications.

[071] The embodiments described herein provide benefits and advantages that conventional methods either cannot or do not provide. By way of example only, the HCTP described herein enables the transmission of both RGB frame data and depth frame data, as well as the reconstruction of these frames upon receipt. Additionally, the information embedded in HCTP data packets facilitates frame assembly (i.e. , the pairing of RGB frames and depth frames) on a network node in the cloud. The HCTP protocol also supports the transmission of compressed point cloud data, which can then be reconstructed at a receiver, and enables performing 3D data processing in the cloud, which is a requirement for cloud-based 3D XR communications. [072] Figure 11 illustrates an exemplary cloud-based XR communications system 160 configured according to one embodiment. As seen in Figure 11, user-A on the production side (i.e., location A) is equipped with a 3D camera 12 and a first user terminal 16 (e.g., a computer), which allows the user to set up an XR virtual meeting and start streaming a 3D representation of user-A. On the receiving side (i.e., location B), user-B is equipped with XR glasses 22 and second user terminal 20 (e.g., a mobile phone), which enables user-B to connect to the XR virtual meeting and display user-A’s received 3D representation. According to the present disclosure, the 3D representation of user-A is considered to arrive at the second user terminal 20 of user-B in point cloud format. In at least one embodiment, first user terminal 16 and second user terminal 20 are each connected to network node 162 in cloud 18 via one or more data networks 164 and a Radio Access Network (RAN) 166, respectively.

[073] Underneath the described user experience, the processing of the transferred 3D representation from user-A towards user-B is handled by a network node 162 in the cloud 18. According to the present embodiments, the application-level protocol defined herein (i.e., the HCTP) functions as the enabling transmission method that allows user-A to communicate holographic data with user-B via cloud 18 communication channel(s) (i.e., user-A <- cloud user-B).

[074] Figure 12 illustrates an exemplary data flow comprising a data stream of HCTP packets. In this embodiment, application-level packets arrive at the network node 162 in the cloud 18. Each packet contains either a RGB frame 172a or depth frame 172b, 172c. Each packet also includes respective frame-specific data 164a, 164b, 164c embedded in the HCTP header. This frame-specific data 164a, 164b, 164c, as described in more detail below, comprises “frame control information.”

[075] As seen in this figure, the value in the TYPE field of each header is either 0x01 indicating a RGB frame 172a or 0x02 indicating a depth frame 172b, 172c. The value in the FLAGS field is either 0x01 indicating that the data packet is an unchunked message, 0x02 indicating that the data packet is the start of chunked message, or 0x04 indicating that the data packet is an end of a chunked message. Based on the “frame control information” defined in the TYPE and FLAGS fields, the network node in the cloud 18 is able to properly reconstruct the RGB and depth frames 172a, 172b, 172c. So reconstructed, the value in the FRAME ID field can be used to pair the RGB and depth frames 172a, 172b, 172c. In this embodiment, the FRAME ID field in each of the data packet headers is 01. Therefore, the RGB and depth frames172a, 172b, 172c in Figure 11 all belong to Frame 01.

[076] The “frame control information” defined in the TYPE and FLAGS fields embedded in the HCTP packet headers allows for the reconstruction and pairing of the RGB and depth frames 172a, 172b, 172c. In some embodiments, as described in more detail below, the FRAME ID field is also in the frame control information. Therefore, both the RGB and Depth data are packaged and transmitted using the HCTP described herein.

[077] After processing of paired frames is performed in the cloud 18, the resultant point cloud data 176 is further encapsulated in an HCTP data packet. The value in the TYPE field of 0x03 of frame-specific data 178 in the header indicates compressed point cloud data. Additionally, according to the present disclosure, the TYPE field is also used to send data in either chunked or unchunked format so the receiving user terminate. g., second user terminal 20) can reconstruct the point cloud data 176 from the received HCTP packets.

[078] Figure 13 is a signaling diagram 180 illustrating the use of a HCTP protocol for communicating RGB/depth frames 172a, 172b, 172c (collectively, 172) and point cloud data 176 according to one embodiment of the present disclosure.

[079] RGB and depth frames 172 are captured on the 3D camera 12 and streamed towards the first user terminal 14 (line 180-1). Upon receipt, the first user terminal 14, configured as a content producer, may perform some pre-processing tasks (box 180-2). Such tasks may include, but are not limited to, rate reduction and filtering. First user terminal 14 then compresses and encodes the RGB and depth frames 162 (box 180-3) and packages the compressed RGB and depth frames 162 into one or more HCTP packets (box 180-4). The compressed RGB/depth frames 172 are then streamed towards the cloud 18 using the HCTP protocol (line 180-5).

[080] Upon receiving the HCTP packets, a network node 162 in cloud 18 decodes and reconstructs the RGB/depth frames 172 using data (i.e. , the frame control information) embedded in the HCTP headers of the corresponding data packets. This frame control information is also used to assemble the RGB and depth frames 172 into RGB/depth frame pairs (box 180-6). With the RGB/depth pairs as input, network node 164 creates the point cloud data 166 (box 180-7). Network node 164 then compresses and encodes the point cloud data 166 (box 180-8), packages the compressed point cloud data 166 into HCTP packets (box 180- 9), and streams those packets to the receiving second user terminal 20 using the HCTP protocol (line 180-10).

[081] Responsive to receiving the HCTP packets, second user terminal 20 decodes the point cloud data 166 from the payload of the HCTP packets before decompressing and rendering the decompressed data into its final format for 3D representation (box 180-11). The second user terminal 20 then streams the 3D representation of the data towards the XR glasses 22 (line ISO- 12) for display to the user (box 180-13).

[082] Figure 14 illustrates the use of the HCTP protocol in a cloud implementation for 3D XR communications according to one embodiment of the present disclosure. As seen in Figure 14, a stream of data packets 192, formatted according to the HCTP described herein, arrives for processing at network node 164 in cloud 18. Each data packet 192 has a HCTP header 192a and a payload 164 encapsulating the RGB/depth data. Using the frame control information (i.e. , the data in the Type and Flags fields in the HCTP header 192a), the arriving packets 192 are decoded 200 and the resultant RBG and depth frames 204, 202 frames reconstructed. Next, frame assembly 206 is performed using the frame control information (i.e., the data in the FramelD field) so that RGB/depth frame pairs can be created and used as input for the creation of the point cloud 210. The resultant point cloud data 212 is then encoded and packaged 214 into an HCTP payload 222b and streamed towards in one or more packets 220 to a receiving terminal, such as second user terminal 20. As seen in Figure 14, each packet 220 streamed to the receiving terminal comprises a HCTP header 222a and the HCTP payload 222b comprising the generated point cloud data 212. Upon receipt, the receiving terminal uses data in the HCTP header 222a (i.e., the Type and Flags fields) to decode and reconstruct the received point cloud data 212 (i.e., the 3D data).

[083] Figure 15 is a flow diagram illustrating a method 230, implemented by a network node in a communications network, for enabling holographic communications according to one embodiment of the present disclosure. As seen in Figure 15, the network node implementing method 230 receives a plurality of color frames (e.g., RGB frames) and a plurality of depth frames for an image. Each frame is received in a corresponding data packet of a data stream and comprises a payload comprising color information for the color frames, or depth information for the depth frames, and a header comprising frame control information for processing the payload (box 132). So received, the network node then pairs the color frames with the depth frames according to the frame control information to generate corresponding color/depth frame pairs (box 234). The network node then generates point cloud data from the color/depth frame pairs (box 236) and sends the point cloud data and the frame control information to a receiving terminal (box 238).

[084] In one exemplary embodiment, the plurality of color frames and the plurality of depth frames are received in a data stream from a user terminal that produces the image. The plurality of color frames may be, for example, Red Green Blue (RGB) frames.

[085] In one embodiment, the frame control information comprises a type indicator indicating whether the payload comprises a color frame or a depth frame.

[086] Additionally, in one embodiment, the frame control information further comprises a flag indicating whether the color frame or the depth frame is the start or end of a message, whether the color frame or the depth frame is chunked, or whether the color frame or the depth frame is unchunked.

[087] Further, in at least one embodiment, the frame control information also comprises a frame identifier. In such cases, the color frames and the depth frames are paired according to the frame identifier. The frame identifier comprises, in one embodiment, an original frame identifier of the color frame for the image.

[088] In one embodiment, the plurality of color frames and the plurality of depth frames are decoded according to the frame control information.

[089] Additionally, in one embodiment, the plurality of color frames and the plurality of depth frames are reconstructed according to the frame control information.

[090] In one embodiment, the point cloud data is compressed and encoded.

[091] In one embodiment, each of the plurality of color frames and each of the plurality of depth frames are received in corresponding data packets, with each data packet being formatted according to a Holographic Communication Transport Protocol (HCTP).

[092] Additionally, in one embodiment, the point cloud data and the frame control information are sent to the receiving terminal in one or more data packets, with each data packet being formatted according to the HCTP.

[093] In one embodiment, each of the corresponding data packets received at the network node, and each of the one or more data packets sent to the receiving terminal, are encapsulated in messages formatted according to a Stream Control Transmission Protocol (SCTP).

[094] Figure 16 is a flow diagram illustrating a method 240, implemented by a user terminal in a communications network, for enabling holographic communications according to one embodiment of the present disclosure. In this embodiment, the user terminal implementing method 240 is configured to function as a content producer.

[095] As seen in Figure 16, the user terminal implementing method 240 obtains a plurality of color frames and a plurality of depth frames for an image (box 242). So received, the user terminal generates a plurality of data packets (box 244). Each generated data packet comprises a payload comprising color information for the color frames, or depth information for the depth frames, and a header comprising frame control information for processing the payload at the network node. The user terminal then sends the plurality of data packets to a network node in the communications network (box 246).

[096] In one embodiment, the plurality of color frames and the plurality of depth frames are received from a camera, such as a 3D camera.

[097] Additionally, in one embodiment, the plurality of color frames are Red Green Blue (RGB) frames.

[098] In one embodiment, the frame control information comprises a type indicator indicating whether the payload comprises a color frame or a depth frame. [099] In one embodiment, the frame control information further comprises a flag indicating whether the color frame or the depth frame is the start or end of a message or is chunked or unchunked.

[0100] Additionally, in one embodiment, the frame control information further comprises a frame identifier indicating an original frame identifier of the color frame for the image.

[0101] In at least one embodiment, the frame control information further comprises a frame identifier indicating an original frame identifier of the depth frame for the image.

[0102] In one embodiment, the color frames and the depth frames are compressed and encoded.

[0103] In one embodiment, each of the plurality of data packets sent to the network node are formatted according to a Holographic Communication Transport Protocol (HCTP).

[0104] In one embodiment, each of the plurality of data packets sent to the network node are encapsulated in corresponding messages formatted according to a Stream Control Transmission Protocol (SCTP).

[0105] Figure 17 is a flow diagram illustrating a method 250, implemented by a user terminal in a communications network, for enabling holographic communications according to one embodiment of the present disclosure. In this embodiment, the user terminal implementing method 250 is configured to function as a content receiver that receives the content produced by the user terminal implementing method 240.

[0106] As seen in Figure 17, the user terminal of this embodiment receives a stream of data packets from a network node in the communications network (box 252). Each received data packet comprises a payload comprising point cloud data representing a color/depth frame pair associated with an image and a header comprising frame control information for processing the point cloud data. The user terminal then generates a 3-dimensional (3D) representation of the image from the point cloud data according to the frame control information (box 254) and renders the 3D representation of the image to a display device (box 256).

[0107] In one embodiment, each color frame in a color/depth frame pair is a Red Green Blue (RGB) frame.

[0108] In one embodiment, the point cloud data in the payload is decoded according to the frame control information to obtain the color/depth frame pair.

[0109] In one embodiment, the color/depth pair is decompressed to obtain a color frame and a depth frame.

[0110] In one embodiment, the frame control information comprises a type indicator indicating whether the color/depth frame pair comprises a color frame or a depth frame.

[0111] In one embodiment, the frame control information further comprises a flag indicating whether the color frame or the depth frame is the start or end a message, or is chunked, or is unchunked. [0112] In one embodiment, the frame control information further comprises a frame identifier indicating an original frame identifier of the color frame for the image.

[0113] In one embodiment, the frame control information further comprises a frame identifier indicating an original frame identifier of the depth frame for the image.

[0114] In one embodiment, each data packet in the stream of data packets is formatted according to a Holographic Communication Transport Protocol (HCTP).

[0115] An apparatus can perform any of the methods herein described by implementing any functional means, modules, units, or circuitry. In one embodiment, for example, the apparatuses comprise respective circuits or circuitry configured to perform the steps shown in the method figures. The circuits or circuitry in this regard may comprise circuits dedicated to performing certain functional processing and/or one or more microprocessors in conjunction with memory. For instance, the circuitry may include one or more microprocessors or microcontrollers, as well as other digital hardware, which may include Digital Signal Processors (DSPs), special-purpose digital logic, and the like. The processing circuitry may be configured to execute program code stored in memory, which may include one or several types of memory such as read-only memory (ROM), random-access memory, cache memory, flash memory devices, optical storage devices, etc. Program code stored in memory may include program instructions for executing one or more telecommunications and/or data communications protocols as well as instructions for carrying out one or more of the techniques described herein, in several embodiments. In embodiments that employ memory, the memory stores program code that, when executed by the one or more processors, carries out the techniques described herein. [0116] Figure 18 illustrates the main functional components of a user terminal 400. The user terminal 400 may be either the first user terminal 14 functioning as the producer of the 3D content, or the second user terminal 20 functioning as the receiver of the 3D content produced by the first user terminal 14.

[0117] As seen in Figure 18, user terminal 400 includes an antenna panel or antenna array comprising a plurality of antennas 410, communication circuitry 420, processing circuitry 430, and memory 440.

[0118] The communication circuitry 420 connects to the antennas 410 and comprises radio frequency (RF) circuitry 422 for communicating over a wireless communication link with multiple TRPs in a wireless communication system. The RF circuitry may comprise, for example, a transmitter and receiver configured to operate according to the 5G standards or other wireless communication standard. In exemplary embodiments, the RF circuitry includes two or more receiver chains for receiving signals transmitted from spatially separated TRPs.

[0119] The processing circuitry 430 comprises one or more microprocessors, hardware, firmware, or a combination thereof that control the overall operation of the user terminal 400. The processing circuitry 430 can be configured by software to perform the methods herein described including the methods 240, 250 shown in Figures 16 and 17, respectively. [0120] Memory 440 comprises both volatile and non-volatile memory for storing computer program code and data needed by the processing circuitry 430 for operation. Memory 440 may comprise any tangible, non-transitory computer-readable storage medium for storing data including electronic, magnetic, optical, electromagnetic, or semiconductor data storage.

Memory 440 stores a computer program 450 comprising executable instructions that configure the processing circuit 430 in the user terminal 400 to perform the methods herein described including the methods 240, 250 shown in Figures 16 and 17, respectively. A computer program 450 in this regard may comprise one or more code modules corresponding to the means or units described above. In general, computer program instructions and configuration information are stored in a non-volatile memory, such as a ROM, erasable programmable read only memory (EPROM) or flash memory. Temporary data generated during operation may be stored in a volatile memory, such as a random access memory (RAM). In some embodiments, computer program 450 for configuring the processing circuitry 430 as herein described may be stored in a removable memory, such as a portable compact disc, portable digital video disc, or other removable media. The computer program 450 may also be embodied in a carrier such as an electronic signal, optical signal, radio signal, or computer readable storage medium.

[0121] Figure 19 illustrates the main functional components of a network node 500, which may comprise, for example, a node in the cloud 18. As seen in Figure 19, network node 500 comprises communication circuitry 520, processing circuitry 530, and memory 540.

[0122] In some embodiments, the communication circuitry 520 comprises both radio frequency (RF) circuitry 522 and network interface circuitry (NIC) 524. In other embodiments, the network node 500 may comprise only NIC 424. The RF circuitry 422 can be located at one or more TRPs and comprises the RF components necessary for communicating with UEs over a wireless communication link. The RF circuitry may comprise, for example, a transmitter and receiver configured to operate according to the 5G standards or other wireless communication standard. The interface circuitry 520 comprises network interface circuitry for communication with other RAN nodes, core network nodes, and or external systems. The network interface circuitry may, for example, comprise an Ethernet interface, optical network interface, or a wireless interface.

[0123] The processing circuitry 530 comprises one or more microprocessors, hardware, firmware, or a combination thereof that control the overall operation of the network node 500. The processing circuitry 530 can be configured by software to perform one or more of the methods herein described including method 230 as shown in Figure 15.

[0124] Memory 540 comprises both volatile and non-volatile memory for storing computer program code and data needed by the processing circuitry 530 for operation. Memory 540 may comprise any tangible, non-transitory computer-readable storage medium for storing data including electronic, magnetic, optical, electromagnetic, or semiconductor data storage.

Memory 540 stores a computer program 550 comprising executable instructions that configure the processing circuit 530 in the network node 500 to perform one or more of the methods herein described including the method 230 as shown in Figure 15. A computer program 550 in this regard may comprise one or more code modules corresponding to the means or units described above. In general, computer program instructions and configuration information are stored in a non-volatile memory, such as a ROM, erasable programmable read only memory (EPROM) or flash memory. Temporary data generated during operation may be stored in a volatile memory, such as a random access memory (RAM). In some embodiments, computer program 550 for configuring the processing circuitry 530 as herein described may be stored in a removable memory, such as a portable compact disc, portable digital video disc, or other removable media. The computer program 550 may also be embodied in a carrier such as an electronic signal, optical signal, radio signal, or computer readable storage medium.

[0125] Those skilled in the art will also appreciate that embodiments herein further include corresponding computer programs. A computer program comprises instructions which, when executed on at least one processor of an apparatus, cause the apparatus to carry out any of the respective processing described above. A computer program in this regard may comprise one or more code modules corresponding to the means or units described above.

[0126] Embodiments further include a carrier containing such a computer program. This carrier may comprise one of an electronic signal, optical signal, radio signal, or computer readable storage medium.

[0127] In this regard, embodiments herein also include a computer program product stored on a non-transitory computer readable (storage or recording) medium and comprising instructions that, when executed by a processor of an apparatus, cause the apparatus to perform as described above.

[0128] Embodiments further include a computer program product comprising program code portions for performing the steps of any of the embodiments herein when the computer program product is executed by a computing device. This computer program product may be stored on a computer readable recording medium.

[0129] Additional embodiments will now be described. At least some of these embodiments may be described as applicable in certain contexts and/or wireless network types for illustrative purposes, but the embodiments are similarly applicable in other contexts and/or wireless network types not explicitly described.

[0130] Figure 20 shows an example of a communication system 1100 in accordance with some embodiments.

[0131] In the example, the communication system 1100 includes a telecommunication network 1102 that includes an access network 1104, such as a radio access network (RAN), and a core network 1106, which includes one or more core network nodes 1108. The access network 1104 includes one or more access network nodes, such as network nodes 1110a and 1110b (one or more of which may be generally referred to as network nodes 1110), or any other similar 3rd Generation Partnership Project (3GPP) access node or non-3GPP access point. The network nodes 1110 facilitate direct or indirect connection of user equipment (UE), such as by connecting UEs 1112a, 1112b, 1112c, and 1112d (one or more of which may be generally referred to as UEs 1112) to the core network 1106 over one or more wireless connections. [0132] Example wireless communications over a wireless connection include transmitting and/or receiving wireless signals using electromagnetic waves, radio waves, infrared waves, and/or other types of signals suitable for conveying information without the use of wires, cables, or other material conductors. Moreover, in different embodiments, the communication system 1100 may include any number of wired or wireless networks, network nodes, UEs, and/or any other components or systems that may facilitate or participate in the communication of data and/or signals whether via wired or wireless connections. The communication system 1100 may include and/or interface with any type of communication, telecommunication, data, cellular, radio network, and/or other similar type of system.

[0133] The UEs 1112 may be any of a wide variety of communication devices, including wireless devices arranged, configured, and/or operable to communicate wirelessly with the network nodes 1110 and other communication devices. Similarly, the network nodes 1110 are arranged, capable, configured, and/or operable to communicate directly or indirectly with the UEs 1112 and/or with other network nodes or equipment in the telecommunication network 1102 to enable and/or provide network access, such as wireless network access, and/or to perform other functions, such as administration in the telecommunication network 1102.

[0134] In the depicted example, the core network 1106 connects the network nodes 1110 to one or more hosts, such as host 1116. These connections may be direct or indirect via one or more intermediary networks or devices. In other examples, network nodes may be directly coupled to hosts. The core network 1106 includes one more core network nodes (e.g., core network node 1108) that are structured with hardware and software components. Features of these components may be substantially similar to those described with respect to the UEs, network nodes, and/or hosts, such that the descriptions thereof are generally applicable to the corresponding components of the core network node 1108. Example core network nodes include functions of one or more of a Mobile Switching Center (MSC), Mobility Management Entity (MME), Home Subscriber Server (HSS), Access and Mobility Management Function (AMF), Session Management Function (SMF), Authentication Server Function (AUSF), Subscription Identifier De-concealing function (SIDF), Unified Data Management (UDM), Security Edge Protection Proxy (SEPP), Network Exposure Function (NEF), and/or a User Plane Function (UPF).

[0135] The host 1116 may be under the ownership or control of a service provider other than an operator or provider of the access network 1104 and/or the telecommunication network 1102, and may be operated by the service provider or on behalf of the service provider. The host 1116 may host a variety of applications to provide one or more services. Examples of such applications include live and pre-recorded audio/video content, data collection services such as retrieving and compiling data on various ambient conditions detected by a plurality of UEs, analytics functionality, social media, functions for controlling or otherwise interacting with remote devices, functions for an alarm and surveillance center, or any other such function performed by a server.

[0136] As a whole, the communication system 1100 of Figure 20 enables connectivity between the UEs, network nodes, and hosts. In that sense, the communication system may be configured to operate according to predefined rules or procedures, such as specific standards that include, but are not limited to: Global System for Mobile Communications (GSM); Universal Mobile Telecommunications System (UMTS); Long Term Evolution (LTE), and/or other suitable 2G, 3G, 4G, 5G standards, or any applicable future generation standard (e.g., 6G); wireless local area network (WLAN) standards, such as the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standards (WiFi); and/or any other appropriate wireless communication standard, such as the Worldwide Interoperability for Microwave Access (WiMax), Bluetooth, Z-Wave, Near Field Communication (NFC) ZigBee, LiFi, and/or any low- power wide-area network (LPWAN) standards such as LoRa and Sigfox.

[0137] In some examples, the telecommunication network 1102 is a cellular network that implements 3GPP standardized features. Accordingly, the telecommunications network 1102 may support network slicing to provide different logical networks to different devices that are connected to the telecommunication network 1102. For example, the telecommunications network 1102 may provide Ultra Reliable Low Latency Communication (URLLC) services to some UEs, while providing Enhanced Mobile Broadband (eMBB) services to other UEs, and/or Massive Machine Type Communication (mMTC)/Massive loT services to yet further UEs.

[0138] In some examples, the UEs 1112 are configured to transmit and/or receive information without direct human interaction. For instance, a UE may be designed to transmit information to the access network 1104 on a predetermined schedule, when triggered by an internal or external event, or in response to requests from the access network 1104. Additionally, a UE may be configured for operating in single- or multi-RAT or multi-standard mode. For example, a UE may operate with any one or combination of Wi-Fi, NR (New Radio) and LTE, i.e. being configured for multi-radio dual connectivity (MR-DC), such as E-UTRAN (Evolved-UMTS Terrestrial Radio Access Network) New Radio - Dual Connectivity (EN-DC).

[0139] In the example, the hub 1114 communicates with the access network 1104 to facilitate indirect communication between one or more UEs (e.g., UE 1112c and/or 1112d) and network nodes (e.g., network node 1110b). In some examples, the hub 1114 may be a controller, router, content source and analytics, or any of the other communication devices described herein regarding UEs. For example, the hub 1114 may be a broadband router enabling access to the core network 1106 for the UEs. As another example, the hub 1114 may be a controller that sends commands or instructions to one or more actuators in the UEs. Commands or instructions may be received from the UEs, network nodes 1110, or by executable code, script, process, or other instructions in the hub 1114. As another example, the hub 1114 may be a data collector that acts as temporary storage for UE data and, in some embodiments, may perform analysis or other processing of the data. As another example, the hub 1114 may be a content source. For example, for a UE that is a VR headset, display, loudspeaker or other media delivery device, the hub 1114 may retrieve VR assets, video, audio, or other media or data related to sensory information via a network node, which the hub 1114 then provides to the UE either directly, after performing local processing, and/or after adding additional local content. In still another example, the hub 1114 acts as a proxy server or orchestrator for the UEs, in particular if one or more of the UEs are low energy loT devices.

[0140] The hub 1114 may have a constant/persistent or intermittent connection to the network node 1110b. The hub 1114 may also allow for a different communication scheme and/or schedule between the hub 1114 and UEs (e.g., UE 1112c and/or 1112d), and between the hub 1114 and the core network 1106. In other examples, the hub 1114 is connected to the core network 1106 and/or one or more UEs via a wired connection. Moreover, the hub 1114 may be configured to connect to an M2M service provider over the access network 1104 and/or to another UE over a direct connection. In some scenarios, UEs may establish a wireless connection with the network nodes 1110 while still connected via the hub 1114 via a wired or wireless connection. In some embodiments, the hub 1114 may be a dedicated hub - that is, a hub whose primary function is to route communications to/from the UEs from/to the network node 1110b. In other embodiments, the hub 1114 may be a non-dedicated hub - that is, a device which is capable of operating to route communications between the UEs and network node 1110b, but which is additionally capable of operating as a communication start and/or end point for certain data channels.

[0141] Figure 21 is a block diagram of a host 1400, which may be an embodiment of the host 1116 of Figure 20, in accordance with various aspects described herein. As used herein, the host 1400 may be or comprise various combinations hardware and/or software, including a standalone server, a blade server, a cloud-implemented server, a distributed server, a virtual machine, container, or processing resources in a server farm. The host 1400 may provide one or more services to one or more UEs.

[0142] The host 1400 includes processing circuitry 1402 that is operatively coupled via a bus 1404 to an input/output interface 1406, a network interface 1408, a power source 1410, and a memory 1412. Other components may be included in other embodiments. Features of these components may be substantially similar to those described with respect to the devices of previous figures, such that the descriptions thereof are generally applicable to the corresponding components of host 1400.

[0143] The memory 1412 may include one or more computer programs including one or more host application programs 1414 and data 1416, which may include user data, e.g., data generated by a UE for the host 1400 or data generated by the host 1400 for a UE. Embodiments of the host 1400 may utilize only a subset or all of the components shown. The host application programs 1414 may be implemented in a container-based architecture and may provide support for video codecs (e.g., Versatile Video Coding (VVC), High Efficiency Video Coding (HEVC), Advanced Video Coding (AVC), MPEG, VP9) and audio codecs (e.g., FLAG, Advanced Audio Coding (AAC), MPEG, G.711), including transcoding for multiple different classes, types, or implementations of UEs (e.g., handsets, desktop computers, wearable display systems, heads-up display systems). The host application programs 1414 may also provide for user authentication and licensing checks and may periodically report health, routes, and content availability to a central node, such as a device in or on the edge of a core network. Accordingly, the host 1400 may select and/or indicate a different host for over-the-top services for a UE. The host application programs 1414 may support various protocols, such as the HTTP Live Streaming (HLS) protocol, Real-Time Messaging Protocol (RTMP), Real-Time Streaming Protocol (RTSP), Dynamic Adaptive Streaming over HTTP (MPEG-DASH), etc.

[0144] Figure 22 shows a communication diagram of a host 1602 communicating via a network node 1604 with a UE 1606 over a partially wireless connection in accordance with some embodiments. Example implementations, in accordance with various embodiments, of the UE (such as a UE 1112a of Figure 20), network node (such as network node 1110a of Figure 20), and host (such as host 1116 of Figure 20) discussed in the preceding paragraphs will now be described with reference to Figure 22.

[0145] Like host 1400, embodiments of host 1602 include hardware, such as a communication interface, processing circuitry, and memory. The host 1602 also includes software, which is stored in or accessible by the host 1602 and executable by the processing circuitry. The software includes a host application that may be operable to provide a service to a remote user, such as the UE 1606 connecting via an over-the-top (OTT) connection 1650 extending between the UE 1606 and host 1602. In providing the service to the remote user, a host application may provide user data which is transmitted using the OTT connection 1650.

[0146] The network node 1604 includes hardware enabling it to communicate with the host 1602 and UE 1606. The connection 1660 may be direct or pass through a core network (like core network 1106 of Figure 20) and/or one or more other intermediate networks, such as one or more public, private, or hosted networks. For example, an intermediate network may be a backbone network or the Internet.

[0147] The UE 1606 includes hardware and software, which is stored in or accessible by UE 1606 and executable by the UE’s processing circuitry. The software includes a client application, such as a web browser or operator-specific “app” that may be operable to provide a service to a human or non-human user via UE 1606 with the support of the host 1602. In the host 1602, an executing host application may communicate with the executing client application via the OTT connection 1650 terminating at the UE 1606 and host 1602. In providing the service to the user, the UE's client application may receive request data from the host's host application and provide user data in response to the request data. The OTT connection 1650 may transfer both the request data and the user data. The UE's client application may interact with the user to generate the user data that it provides to the host application through the OTT connection 1650.

[0148] The OTT connection 1650 may extend via a connection 1660 between the host 1602 and the network node 1604 and via a wireless connection 1670 between the network node 1604 and the UE 1606 to provide the connection between the host 1602 and the UE 1606. The connection 1660 and wireless connection 1670, over which the OTT connection 1650 may be provided, have been drawn abstractly to illustrate the communication between the host 1602 and the UE 1606 via the network node 1604, without explicit reference to any intermediary devices and the precise routing of messages via these devices.

[0149] As an example of transmitting data via the OTT connection 1650, in step 1608, the host 1602 provides user data, which may be performed by executing a host application. In some embodiments, the user data is associated with a particular human user interacting with the UE 1606. In other embodiments, the user data is associated with a UE 1606 that shares data with the host 1602 without explicit human interaction. In step 1610, the host 1602 initiates a transmission carrying the user data towards the UE 1606. The host 1602 may initiate the transmission responsive to a request transmitted by the UE 1606. The request may be caused by human interaction with the UE 1606 or by operation of the client application executing on the UE 1606. The transmission may pass via the network node 1604, in accordance with the teachings of the embodiments described throughout this disclosure. Accordingly, in step 1612, the network node 1604 transmits to the UE 1606 the user data that was carried in the transmission that the host 1602 initiated, in accordance with the teachings of the embodiments described throughout this disclosure. In step 1614, the UE 1606 receives the user data carried in the transmission, which may be performed by a client application executed on the UE 1606 associated with the host application executed by the host 1602.

[0150] In some examples, the UE 1606 executes a client application which provides user data to the host 1602. The user data may be provided in reaction or response to the data received from the host 1602. Accordingly, in step 1616, the UE 1606 may provide user data, which may be performed by executing the client application. In providing the user data, the client application may further consider user input received from the user via an input/output interface of the UE 1606. Regardless of the specific manner in which the user data was provided, the UE 1606 initiates, in step 1618, transmission of the user data towards the host 1602 via the network node 1604. In step 1620, in accordance with the teachings of the embodiments described throughout this disclosure, the network node 1604 receives user data from the UE 1606 and initiates transmission of the received user data towards the host 1602. In step 1622, the host 1602 receives the user data carried in the transmission initiated by the UE 1606. [0151] One or more of the various embodiments improve the performance of OTT services provided to the UE 1606 using the OTT connection 1650, in which the wireless connection 1670 forms the last segment. More precisely, the teachings of these embodiments may enable the UE to adapt faster to radio conditions and realize higher data throughput. In an example scenario, factory status information may be collected and analyzed by the host 1602. As another example, the host 1602 may process audio and video data which may have been retrieved from a UE for use in creating maps. As another example, the host 1602 may collect and analyze real-time data to assist in controlling vehicle congestion (e.g., controlling traffic lights). As another example, the host 1602 may store surveillance video uploaded by a UE. As another example, the host 1602 may store or control access to media content such as video, audio, VR or AR which it can broadcast, multicast or unicast to UEs. As other examples, the host 1602 may be used for energy pricing, remote control of non-time critical electrical load to balance power generation needs, location services, presentation services (such as compiling diagrams etc. from data collected from remote devices), or any other function of collecting, retrieving, storing, analyzing and/or transmitting data.

[0152] In some examples, a measurement procedure may be provided for the purpose of monitoring data rate, latency and other factors on which the one or more embodiments improve. There may further be an optional network functionality for reconfiguring the OTT connection 1650 between the host 1602 and UE 1606, in response to variations in the measurement results. The measurement procedure and/or the network functionality for reconfiguring the OTT connection may be implemented in software and hardware of the host 1602 and/or UE 1606. In some embodiments, sensors (not shown) may be deployed in or in association with other devices through which the OTT connection 1650 passes; the sensors may participate in the measurement procedure by supplying values of the monitored quantities exemplified above, or supplying values of other physical quantities from which software may compute or estimate the monitored quantities. The reconfiguring of the OTT connection 1650 may include message format, retransmission settings, preferred routing etc.; the reconfiguring need not directly alter the operation of the network node 1604. Such procedures and functionalities may be known and practiced in the art. In certain embodiments, measurements may involve proprietary UE signaling that facilitates measurements of throughput, propagation times, latency and the like, by the host 1602. The measurements may be implemented in that software causes messages to be transmitted, in particular empty or ‘dummy’ messages, using the OTT connection 1650 while monitoring propagation times, errors, etc.

[0153] Additionally, it should be noted here that, according to the present embodiments, host 1116 of Figure 20, host 1400 of Figure 21 , and host 1602 of Figure 22 are all examples of network node 500 illustrated in Figure 19. That is, each host 1116, 1400, and 1602 comprises respective processing circuitry that can be configured to perform one or more of the methods herein described, including method 230 as shown in Figure 15. This includes, for example, providing the cloud processing capabilities of network node 162 in cloud 18, as seen in Figure

11. In this context, any of hosts 1116, 1400, and 1602 could be part of a service network that is provided by, or on behalf of, a service provider that offers cloud processing capabilities or VR communication services. [0154] The present embodiments may, of course, be carried out in other ways than those specifically set forth herein without departing from characteristics described herein. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, and all changes coming within the meaning and equivalency range of the appended claims are intended to be embraced therein.

Claims

CLAIMS What Is Claimed Is:

1. A method (230), implemented by a network node (500) in a communications network (10), for enabling holographic communications, the method comprising: receiving (232) a plurality of color frames and a plurality of depth frames for an image, wherein each frame is received in a corresponding data packet (192) of a data stream and comprises: a payload(192b) comprising color information for the color frames or depth information for the depth frames; and a header (192a) comprising frame control information for processing the payload; pairing (234) the color frames with the depth frames according to the frame control information to generate corresponding color/depth frame pairs (208); generating (236) point cloud data (212) from the color/depth frame pairs; and sending (238) the point cloud data and the frame control information to a receiving terminal (20).

2. The method of embodiment 1 , wherein the plurality of color frames and the plurality of depth frames are received in a data stream from a user terminal (400) that produces the image.

3. The method of embodiments 1-2, wherein the plurality of color frames are Red Green Blue (RGB) frames.

4. The method of embodiments 1-3, wherein the frame control information comprises a type indicator (144) indicating whether the payload comprises a color frame or a depth frame.

5. The method of embodiment 4 wherein the frame control information further comprises a flag (146) indicating whether the color frame or the depth frame is: a start of a message; or an end of the message; or chunked; or unchunked.

6. The method of embodiment 5, wherein the frame control information further comprises a frame identifier (148), and wherein the color frames and the depth frames are paired further according to the frame identifier.

7. The method of embodiment 6, wherein the frame identifier comprises an original frame identifier of the color frame for the image.

8. The method of embodiment 6, wherein the frame identifier comprises an original frame identifier of the depth frame for the image.

9. The method of embodiments 1-8, wherein the plurality of color frames and the plurality of depth frames are decoded according to the frame control information.

10. The method of embodiments 1-9, wherein the plurality of color frames and the plurality of depth frames are reconstructed according to the frame control information.

11. The method of embodiments 1-10, wherein the point cloud data is compressed and encoded.

12. The method of any of the preceding embodiments, wherein each of the plurality of color frames and each of the plurality of depth frames are received in corresponding data packets, with each data packet being formatted according to a Holographic Communication Transport Protocol (HCTP).

13. The method of any of the preceding embodiments, wherein the point cloud data and the frame control information are sent to the receiving terminal in one or more data packets (220), with each data packet being formatted according to the HCTP.

14. The method of embodiments 12-13, wherein each of the corresponding data packets received at the network node, and each of the one or more data packets sent to the receiving terminal, are encapsulated in messages formatted according to a Stream Control Transmission Protocol (SCTP).

15. A method (240), implemented by a user terminal (400) configured as a content producer, for enabling holographic communications across a communications network (10), the method comprising: obtaining (242) a plurality of color frames and a plurality of depth frames for an image; generating (244) a plurality of data packets (192), each data packet comprising: a payload (192b) comprising color information for the color frames or depth information for the depth frames; and a header (192a) comprising frame control information for processing the payload at a network node (500); and sending (246) the plurality of data packets to the network node in the communications network.

16. The method of embodiment 15 wherein the plurality of color frames and the plurality of depth frames are received from a camera.

17. The method of embodiments 15-16, wherein the plurality of color frames are Red Green Blue (RGB) frames.

18. The method of embodiments 15-17, wherein the frame control information comprises a type indicator (144) indicating whether the payload comprises a color frame or a depth frame.

19. The method of embodiment 18, wherein the frame control information further comprises a flag (146) indicating whether the color frame or the depth frame is: a start of a message; or an end of the message; or chunked; or unchunked.

20. The method of embodiment 19, wherein the frame control information further comprises a frame identifier (148) indicating an original frame identifier of the color frame for the image.

21. The method of embodiment 19, wherein the frame control information further comprises a frame identifier indicating an original frame identifier of the depth frame for the image.

22. The method of embodiments 15-21 , wherein the color frames and the depth frames are compressed and encoded.

23. The method of any of the preceding embodiments, wherein each of the plurality of data packets (220) sent to the network node are formatted according to a Holographic Communication Transport Protocol (HCTP).

24. The method of any of the preceding embodiments, wherein each of the plurality of data packets sent to the network node are encapsulated in corresponding messages formatted according to a Stream Control Transmission Protocol (SCTP).

25. A method, implemented by a user terminal (400) configured as a content receiver, for enabling holographic communications across a communications network (10), the method comprising: receiving (252) a stream of data packets (220) from a network node (500) in the communications network, each data packet comprising: a payload (192b) comprising point cloud data (212) representing a color/depth frame pair (208) associated with an image; and a header (192a) comprising frame control information for processing the point cloud data; generating (254) a 3-dimensional (3D) representation of the image from the point cloud data according to the frame control information; and rendering (256) the 3D representation of the image to a display device.

26. The method of embodiment 25, wherein each color frame in a color/depth frame pair is a Red Green Blue (RGB) frame.

27. The method of embodiments 25-26, wherein the point cloud data in the payload is decoded according to the frame control information to obtain the color/depth frame pair.

28. The method of embodiments 25-27, wherein the color/depth pair is decompressed to obtain a color frame (204) and a depth frame (202).

29. The method of embodiments 25-28, wherein the frame control information comprises a type indicator (144) indicating whether the color/depth frame pair comprises a color frame or a depth frame.

30. The method of embodiment 29 wherein the frame control information further comprises a flag (146) indicating whether the color frame or the depth frame is: a start of a message; or an end of the message; or chunked; or unchunked.

31. The method of embodiment 29, wherein the frame control information further comprises a frame identifier (148) indicating an original frame identifier of the color frame for the image.

32. The method of embodiment 29, wherein the frame control information further comprises a frame identifier indicating an original frame identifier of the depth frame for the image.

33. The method of any of the preceding embodiments, wherein each data packet in the stream of data packets is formatted according to a Holographic Communication Transport Protocol (HCTP).

34. A network node (500) in a communications network (10) for enabling holographic communications, the network node being configured to: receive (232) a plurality of color frames and a plurality of depth frames for an image, wherein each frame is received in a corresponding data packet (192) of a data stream and comprises: a payload (192b) comprising color information for the color frames or depth information for the depth frames; and a header (192a) comprising frame control information for processing the payload; pair (234) the color frames with the depth frames according to the frame control information to generate corresponding color/depth frame pairs (208); generate (236) point cloud data (212) from the color/depth frame pairs; and send (238) the point cloud data and the frame control information to a receiving terminal (20).

35. The network node of embodiment 34, further configured to perform the method of any one of embodiments 2-14.

36. A network node (500) for enabling holographic communications in a communications network (10), the network node comprising: processing circuitry (530); and memory (540) configured to store instructions (550) executable by the processing circuitry, whereby the processing circuitry is configured to: receive (232) a plurality of color frames and a plurality of depth frames for an image, wherein each frame is received in a corresponding data packet (192) of a data stream and comprises: a payload (192b) comprising color information for the color frames or depth information for the depth frames; and a header (192a) comprising frame control information for processing the payload; pair (234) the color frames with the depth frames according to the frame control information to generate corresponding color/depth frame pairs (208); generate (236) point cloud data (212) from the color/depth frame pairs; and send (238) the point cloud data and the frame control information to a receiving terminal (20).

37. The network node of embodiment 36, wherein the processing circuitry is further configured to perform the method of any one of embodiments 2-14.

38. A computer program (550) comprising executable instructions that, when executed by processing circuitry (530) in a network node (500) in a wireless communication network (10), causes the network node to perform the method of any one of embodiments 2-14.

39. A carrier containing a computer program of embodiment 38, wherein the carrier is one of an electronic signal, optical signal, radio signal, or computer readable storage medium.

40. A non-transitory computer-readable storage medium (540) comprising a computer program (550) having executable instructions that, when executed by processing circuitry (530) in network node (500) in a wireless communication network (10) causes the network node to perform the methods of any one of embodiments 1-14.

41. A user terminal (400) for enabling holographic communications across a communications network (10), the user terminal configured to function as a content producer and configured to: obtain (242) a plurality of color frames and a plurality of depth frames for an image; generate (244) a plurality of data packets (192), each data packet comprising: a payload (192b) comprising color information for the color frames or depth information for the depth frames; and a header (192a) comprising frame control information for processing the payload at q network node (500); and send (246) the plurality of data packets to the network node in the communications network.

42. The user terminal of embodiment 41 , further configured to perform the method of any one of embodiments 16-24.

43. A user terminal (400) for enabling holographic communications across a communications network (10), the user terminal configured to function as a content producer and comprising: processing circuitry (430); and memory (440) configured to store instructions (450) executable by the processing circuitry, whereby the processing circuitry is configured to: obtain (242) a plurality of color frames and a plurality of depth frames for an image; generate (244) a plurality of data packets (192), each data packet comprising: a payload (192b) comprising color information for the color frames or depth information for the depth frames; and a header (192a) comprising frame control information for processing the payload at a network node (500); and send (246) the plurality of data packets to the network node in the communications network.

44. The user terminal of embodiment 43, wherein the processing circuitry is further configured to perform the method of any one of embodiments 16-24.

45. A computer program (450) comprising executable instructions that, when executed by processing circuitry (430) in a user terminal (400) in a communication network (10), causes the user terminal to perform the method of any one of embodiments 15-24.

46. A carrier containing a computer program of embodiment 45, wherein the carrier is one of an electronic signal, optical signal, radio signal, or computer readable storage medium.

47. A non-transitory computer-readable storage medium (450) comprising a computer program (450) having executable instructions stored thereon that, when executed by processing circuitry (430) in a user terminal (400) in a wireless communication network (10), causes the user terminal to perform the methods of any one of embodiments 15-24.

48. A user terminal (400) for enabling holographic communications across a communications network (10), the user terminal configured to function as a content receiver and configured to: receive (252) a stream of data packets (220) from a network node (500) in the communications network, each data packet comprising: a payload (192b) comprising point cloud data (212) representing a color/depth frame pair (208) associated with an image; and a header (192a) comprising frame control information for processing the point cloud data; generate (254) a 3-dimensional (3D) representation of the image from the point cloud data according to the frame control information; and render (256) the 3D representation of the image to a display device.

49. The user terminal of embodiment 48, further configured to perform the method of any one of embodiments 26-33.

50. A user terminal (400) for enabling holographic communications across a communications network (10), the user terminal configured to function as a content receiver and comprising: processing circuitry (430); and memory (440) configured to store instructions (450) executable by the processing circuitry, whereby the processing circuitry is configured to: receive (252) a stream of data packets (220) from a network node (500) in the communications network, each data packet comprising: a payload (192b) comprising point cloud data (212) representing a color/depth frame pair (208) associated with an image; and a header (192a) comprising frame control information for processing the point cloud data; generate (254) a 3-dimensional (3D) representation of the image from the point cloud data according to the frame control information; and render (256) the 3D representation of the image to a display device.

51. The user terminal of embodiment 50, wherein the processing circuitry is further configured to perform the method of any one of embodiments 26-33.

52. A computer program (450) comprising executable instructions that, when executed by processing circuitry (430) in a user terminal (400) in a communication network (10), causes the user terminal to perform the method of any one of embodiments 25-33.

53. A carrier containing a computer program of embodiment 52, wherein the carrier is one of an electronic signal, optical signal, radio signal, or computer readable storage medium.

54. A non-transitory computer-readable storage medium (440) comprising a computer program (450) that includes executable instructions that, when executed by processing circuitry (430) in a user terminal (400) in a communication network (10), causes the user terminal to perform the methods of any one of embodiments 25-33.