WO2022206016A1 - 一种数据分层传输方法、装置及系统 - Google Patents

一种数据分层传输方法、装置及系统 Download PDF

Info

Publication number
WO2022206016A1
WO2022206016A1 PCT/CN2021/137079 CN2021137079W WO2022206016A1 WO 2022206016 A1 WO2022206016 A1 WO 2022206016A1 CN 2021137079 W CN2021137079 W CN 2021137079W WO 2022206016 A1 WO2022206016 A1 WO 2022206016A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
layer data
server
enhancement layer
mobile communication
Prior art date
Application number
PCT/CN2021/137079
Other languages
English (en)
French (fr)
Inventor
张冠华
杨军
苏敏
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2022206016A1 publication Critical patent/WO2022206016A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/65—Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/80—Responding to QoS
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
    • H04L69/165—Combined use of TCP and UDP protocols; selection criteria therefor
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
    • H04N19/37—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability with arrangements for assigning different transmission priorities to video input data or to video coded data
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W28/00—Network traffic management; Network resource management
    • H04W28/16—Central resource management; Negotiation of resources or communication parameters, e.g. negotiating bandwidth or QoS [Quality of Service]
    • H04W28/24—Negotiating SLA [Service Level Agreement]; Negotiating QoS [Quality of Service]

Definitions

  • the present application relates to the technical field of data transmission, and in particular, to a method, device and system for layered data transmission.
  • Virtual reality (VR) technology is a new practical technology developed in the 20th century.
  • Virtual reality technology includes computer, electronic information, and simulation technology.
  • VR technology With the continuous development of social productivity and science and technology, the demand for VR technology in all walks of life is growing, VR technology has also made great progress, and gradually become a new field of science and technology.
  • 8K VR live broadcast is a combination of virtual reality and 8K video live broadcast. Compared with traditional TV viewing, the biggest difference of VR live broadcast is that the audience is immersed in the scene and has an all-round experience in real time.
  • the presentation time of the high-definition field of view (FOV) picture should be as short as possible, for example, the picture transition time should be within 200ms , so that users will not feel stuck.
  • the presentation of high-definition FOV pictures goes through multiple steps such as network transmission, decapsulation, data packet (tile) merging, decoding, rendering, and display.
  • the present application provides a method, device and system for layered data transmission, which are used to reduce the situation of jamming when users browse data and increase the number of users carried by a mobile communication network.
  • an embodiment of the present application provides a method for layered data transmission, including: acquiring base layer data and enhancement layer data obtained by encoding media data; sending the base layer data to the terminal; and sending the enhancement layer data to the terminal through the mobile communication network based on an unreliable transmission protocol in the transport layer. Since the basic layer data is transmitted by a reliable transmission protocol, and the enhancement layer data is transmitted by an unreliable transmission protocol, when the burst code rate is high and the transmission resources of the mobile communication network are insufficient, the basic layer can be guaranteed from the perspective of the transmission layer.
  • Data transmission from the perspective of the transmission layer, adopts a best-effort transmission method for the enhancement layer data, and does not require successful transmission, so as to avoid the user from being stuck when viewing media data, reduce the load of the mobile communication network, and make the mobile communication network. Can serve more users.
  • the method further includes: setting a first quality of service for the base layer data, and setting a second quality of service for the enhancement layer data, where the first quality of service is higher than the first quality of service 2.
  • Quality of service sending the information of the first quality of service and the information of the second quality of service to a device in the mobile communication network, instructing the device to convert the basic The layer data is sent to the terminal, and the enhancement layer data is sent to the terminal according to the second quality of service information.
  • different quality of service can also be set for the base layer data and the enhancement layer data, so that the device in the mobile communication network sends the base layer data and the enhancement layer data to the terminal according to different service qualities, wherein the base layer data
  • the service quality of the data is higher than that of the enhancement layer data; compared with the method of sending the base layer data and the enhancement layer data according to the same service quality, this method helps to reduce the resource consumption of the mobile communication network, thus ensuring the normal operation of the user. While in use, the number of users that can be carried by the mobile communication network is increased.
  • the reliable transmission protocol includes the transmission control protocol TCP or the fast UDP network connection QUIC protocol.
  • the unreliable transmission protocol includes User Datagram Protocol UDP or Real Time Transport Protocol RTP.
  • the method further includes: setting a transmission time window corresponding to target frame data in the enhancement layer data, where the target frame data is any frame data in the enhancement layer data;
  • the sending the enhancement layer data to the terminal through the mobile communication network based on the unreliable transmission protocol in the transport layer includes: within the transmission time window, based on the unreliable transmission protocol in the transport layer, through the The mobile communication network sends the target frame data to the terminal.
  • the method further includes: if one or more data packets of the target frame data cannot be sent within the transmission time window; sending the data to the terminal through the mobile communication network Sending indication information, where the indication information is used to indicate that one or more data packets in the target frame data are no longer sent.
  • the indication information is used to indicate that one or more data packets in the target frame data are no longer sent.
  • the obtaining the base layer data and the enhancement layer data obtained by encoding the media data includes: obtaining mixed data of the media data, where the mixed data includes multiple data packets; Decapsulate each of the data packets to determine whether the data packets belong to the base layer data or to the enhancement layer data.
  • the server may obtain a data packet of media data from a media data source station based on a data stream, and only after decapsulating the data packet can determine whether the data packet belongs to the base layer data or the enhancement layer data.
  • the obtaining the base layer data and the enhancement layer data obtained by encoding the media data includes: obtaining data packets of the media data, according to the quintuple information of each of the data packets , and determine that the data packet belongs to the base layer data or to the enhancement layer data.
  • the server can obtain the base layer data and the enhancement layer data from the media data source station based on different data streams, and the data streams are defined by the quintuple information, that is, the quintuple information of the two data streams is not exactly the same .
  • the obtaining the base layer data and the enhancement layer data obtained by encoding the media data includes: obtaining source data of the media data, and encoding the source data to obtain the base layer data and the enhancement layer data.
  • the server may also obtain the source data from the media source station, and then the server encodes the source data to obtain the base layer data and the enhancement layer data.
  • an embodiment of the present application provides a method for layered data transmission, including: based on a reliable transmission protocol in the transmission layer, receiving basic layer data sent by a server through a mobile communication network; based on an unreliable transmission protocol in the transmission layer, The enhancement layer data sent by the server is received through the mobile communication network; wherein, the base layer data and the enhancement layer data are data obtained by encoding media data.
  • the reliable transmission protocol includes the transmission control protocol TCP or the fast UDP network connection QUIC protocol.
  • the unreliable transmission protocol includes User Datagram Protocol UDP or Real Time Transport Protocol RTP.
  • the method further includes: receiving, through the mobile communication network, indication information sent by a server, where the indication information is used to instruct the server to respond to one or more of the target frame data.
  • the data packets are no longer sent; the indication information is sent by the server when one or more data packets in the target frame data cannot be sent within the transmission time window set for the target frame.
  • an embodiment of the present application provides a server, including a processor, a memory and a communication interface respectively coupled to the processor; the communication interface is used to communicate with other devices; the processor is used to run all The instruction or program in the memory, and the following steps are executed through the communication interface: obtaining the base layer data and the enhancement layer data obtained after encoding the media data; layer data to the terminal; based on the unreliable transmission protocol in the transport layer, the enhancement layer data is sent to the terminal through the mobile communication network.
  • the processing is further configured to: set a first quality of service for the base layer data, and set a second quality of service for the enhancement layer data, where the first quality of service is higher than the second quality of service; sending the information of the first quality of service and the information of the second quality of service to a device in the mobile communication network, instructing the device to send the information to the device according to the information of the first quality of service
  • the base layer data is sent to the terminal, and the enhancement layer data is sent to the terminal according to the second quality of service information.
  • the reliable transmission protocol includes the transmission control protocol TCP or the fast UDP network connection QUIC protocol.
  • the unreliable transmission protocol includes User Datagram Protocol UDP or Real Time Transport Protocol RTP.
  • the processor is further configured to: set a transmission time window corresponding to target frame data in the enhancement layer data, where the target frame data is any frame data in the enhancement layer data
  • the processor when sending the enhancement layer data to the terminal through the mobile communication network based on the unreliable transmission protocol in the transport layer, is specifically used for: within the transmission time window, based on the transport layer
  • the unreliable transmission protocol in the mobile communication network sends the target frame data to the terminal.
  • the processor is further configured to: if one or more data packets of the target frame data cannot be sent within the transmission time window; The terminal sends indication information, where the indication information is used to indicate that one or more data packets in the target frame data are no longer sent.
  • the processor when acquiring the base layer data and the enhancement layer data obtained by encoding the media data, is specifically configured to: acquire mixed data of the media data, where the mixed data includes a plurality of data packets; decapsulate each of the data packets to determine whether the data packets belong to the base layer data or the enhancement layer data.
  • an embodiment of the present application provides a terminal, including: a processor, a memory and a communication interface respectively coupled to the processor; the communication interface is used to communicate with other devices; the processor is used to run The instruction or program in the memory executes the following steps through the communication interface: based on the reliable transmission protocol in the transmission layer, receiving the basic layer data sent by the server through the mobile communication network; based on the unreliable transmission protocol in the transmission layer, through the The mobile communication network receives the enhancement layer data sent by the server; wherein the base layer data and the enhancement layer data are data obtained by encoding media data.
  • the reliable transmission protocol includes the transmission control protocol TCP or the fast UDP network connection QUIC protocol.
  • the unreliable transmission protocol includes User Datagram Protocol UDP or Real Time Transport Protocol RTP.
  • the processor is further configured to: receive, through the mobile communication network, indication information sent by the server, where the indication information is used to instruct the server to respond to one of the target frame data One or more data packets of the target frame are no longer sent; the indication information is when the server fails to send one or more data packets in the target frame data within the transmission time window set for the target frame sent.
  • an embodiment of the present application provides a layered data transmission system, including: the server described in the third aspect and any implementation manner of the third aspect, and the server described in any implementation manner of the fourth aspect and the fourth aspect. described terminal.
  • the system further includes a source station, configured to encode source data to obtain the base layer data and the enhancement layer data, and convert the base layer data and the enhancement layer data sent to the server.
  • a source station configured to encode source data to obtain the base layer data and the enhancement layer data, and convert the base layer data and the enhancement layer data sent to the server.
  • an embodiment of the present application provides a computer-readable storage medium, where instructions are stored in the computer-readable storage medium, and when the instructions are run on a computer, the computer is made to perform the operations described in the first aspect and the first aspect.
  • the method described in any implementation manner, or, the computer is caused to execute the method described in the second aspect and any implementation manner of the second aspect.
  • FIG. 1 is a schematic diagram of an architecture to which the method for layered data transmission provided by an embodiment of the present application is applicable;
  • FIG. 2 is a schematic flowchart of a data layered transmission method provided by an embodiment of the present application
  • FIG. 3 is a schematic diagram of a data layered transmission architecture provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of an internal implementation of a server and a terminal according to an embodiment of the present application
  • FIG. 5 is a schematic diagram of another internal implementation of a server and a terminal provided by an embodiment of the present application.
  • FIG. 6 is a schematic diagram of another internal implementation of a server and a terminal provided by an embodiment of the present application.
  • FIG. 7 is a schematic diagram of another internal implementation of a server and a terminal provided by an embodiment of the present application.
  • FIG. 8 is a schematic structural diagram of a server according to an embodiment of the present application.
  • FIG. 9 is a schematic structural diagram of a terminal according to an embodiment of the present application.
  • FIG. 10 is a schematic structural diagram of another server provided by an embodiment of the present application.
  • FIG. 11 is a schematic structural diagram of another terminal according to an embodiment of the present application.
  • VR terminal devices such as VR helmets, VR glasses, etc.
  • receive VR data from the VR server, and then perform operations such as decapsulation, decoding, and rendering on the received data, so as to display it to the user. Bring users an immersive experience.
  • VR helmets and VR glasses on the market can support WiFi and mobile communication networks (such as 4G and 5G).
  • WiFi and mobile communication networks such as 4G and 5G.
  • data transmission is performed through mobile networks, due to the large amount of VR data and the higher requirements for transmission delay , especially in the field of VR live broadcast, it needs to occupy a higher bit rate and increase the load of the mobile communication network, so that the number of VR users that can be carried by the mobile communication network is less, and it may also cause a large delay in data transmission and reduce the user experience. .
  • the embodiments of the present application provide a data layered transmission method, which is used to solve the above problems from the perspective of the transmission layer, reduce the situation of jamming when users watch, and increase the number of users that can be carried by the mobile communication network.
  • the data layered transmission method provided by the embodiment of the present application can be applied to the architecture shown in FIG. 1 .
  • the server sends data to the terminal through the mobile communication network.
  • the layered data transmission method provided by the embodiment of the present application is suitable for transmitting media data with a large quantity and high real-time requirements through a mobile communication network, for example, a VR scenario in which data transmission is performed by using a mobile communication network, VR games, VR live broadcast, etc.
  • the terminal in this embodiment of the present application may include, but is not limited to, user equipment such as VR helmets, VR glasses, and projectors.
  • the base layer data in the embodiment of the present application is data used to provide users with the most basic video quality, frame rate and resolution;
  • the enhancement layer data is data used to improve the video quality, and provides users with higher quality data.
  • one channel of base layer data and one channel of enhancement layer data may be obtained, or one channel of base layer data and multiple channels of enhancement layer data may also be obtained.
  • FIG. 2 is a schematic flowchart of a data layered transmission method provided by an embodiment of the present application, as shown in the figure, the method may include the following steps:
  • Step 201 The server obtains the base layer data and the enhancement layer data obtained by encoding the media data.
  • the server may obtain the base layer data and enhancement layer data of the media data from the content provider (hereinafter referred to as the source station), that is, the source station encodes the source media data to obtain the base layer data and the enhancement layer data.
  • the server may also receive the source data of the media data from the source station, and the server encodes the source data, thereby obtaining the base layer data and the enhancement layer data.
  • the source station when the source station encodes the source data to obtain the base layer data and the enhancement layer data, the source station may send the base layer data and the enhancement layer data to the server through one data stream, that is, the server sends the base layer data and the enhancement layer data through one data stream Accept mixed data of media data, mixed data includes several data packets of base layer data and several data packets of enhancement layer data, and the server cannot determine whether the data packet belongs to the base layer data or the quintuple information of the received data packet. belongs to the enhancement layer.
  • the server may decapsulate each received data, and determine that the data packet belongs to the base layer data or the enhancement layer data according to the packet header information obtained after decapsulation.
  • the source station when the source station encodes the source data to obtain the base layer data and the enhancement layer data, the source station may send the base layer data and the enhancement layer data to the server through different data streams.
  • the data stream may be defined by quintuple, that is, one data stream corresponds to one quintuple information, and different data streams correspond to different quintuple information.
  • a quintuple is a collection of five types of information: source IP address, source port, destination IP address, destination port, and transport layer protocol.
  • the base layer data and the number of enhancement layers are sent to the server through different data streams, indicating that the quintuple corresponding to the data packet of the base layer data is different from the quintuple corresponding to the data packet of the enhancement layer.
  • the transport layer protocol in the quintuple corresponding to the base layer data packet is different from the transport layer protocol in the quintuple corresponding to the enhancement layer data packet.
  • the server determines the received The data packet belongs to the data packet of the base layer data or the data packet of the enhancement layer data; alternatively, the destination port in the quintuple corresponding to the base layer data packet can be different from the destination port in the quintuple corresponding to the enhancement layer data packet Alternatively, the source port in the quintuple corresponding to the base layer data packet may be different from the source port in the quintuple corresponding to the enhancement layer data packet.
  • the step of encoding the source data to obtain the base layer data and the enhancement layer data may also be performed by the server.
  • the source station sends the source data of the media data to the server, and the server encodes the source data according to a preset encoding method, thereby obtaining the base layer data and the enhancement layer data.
  • Step 202 the server sends the base layer data to the terminal through the mobile communication network based on the reliable transmission protocol in the transport layer; correspondingly, the terminal receives the base layer data sent by the server through the mobile communication network based on the reliable transmission protocol in the transport layer.
  • the transport layer is a particularly critical layer in the open system interconnection (OSI).
  • OSI open system interconnection
  • the transport layer provides an end-to-end mechanism for exchanging data, checking packet numbering and sequence.
  • the transport layer provides reliable transmission services to its upper layers, such as the application layer, and provides reliable destination site information to the network layer.
  • the transmission protocols in the transport layer are classified according to the reliability of the protocol transmission, and can be divided into reliable transmission protocols and unreliable transmission protocols.
  • the reliable transmission protocol can guarantee the reliable attribute of data transmission;
  • the unreliable transmission protocol refers to an unreliable connectionless datagram protocol. for" delivery services.
  • the base layer data is used to provide users with the most basic video quality, frame rate and resolution data. Therefore, in this embodiment of the present application, the server sends the base layer data through a mobile communication network based on a reliable transmission protocol. To the terminal to ensure that the terminal receives the base layer data.
  • Reliable transmission protocol can have confirmation, retransmission mechanism, verification mechanism, and byte numbering mechanism.
  • the common reliable transmission protocol can establish a connection before data transmission, and then perform data transmission after "both handshake".
  • the reliable transmission protocol may include, but is not limited to, a transmission control protocol (transmission control protocol, TCP), a quick UDP internet connection (quick UDP internet connections, QUIC) protocol, and the like.
  • TCP transmission control protocol
  • QUIC quick UDP internet connections
  • the server sends the base layer data based on the reliable transmission protocol, and accordingly, the terminal receives the base layer data based on the reliable transmission protocol, that is, decapsulates the received data packets based on the reliable transmission protocol.
  • Step 203 the server sends the enhancement layer data to the terminal through the mobile communication network based on the unreliable transmission protocol in the transport layer; correspondingly, the terminal receives the enhancement layer data sent by the server through the mobile communication network based on the unreliable transmission protocol in the transport layer .
  • the enhancement layer data is data used to improve the video quality, and provide users with higher video quality, frame rate, and resolution, so as to improve the user's viewing effect. Therefore, if the terminal can receive the enhancement layer data, the viewing experience of the user can be improved; if the terminal cannot receive the enhancement layer data, the display can also be performed based on the base layer data. Based on this, in the embodiments of the present application, a "best effort" delivery service may be provided to the terminal from the perspective of the transport layer, but it is not necessary to ensure that the terminal can receive the service successfully. Specifically, the server may send the enhancement layer data to the terminal through the mobile communication network based on the unreliable transmission protocol.
  • Unreliable transport protocols may not have confirmation, retransmission mechanism, verification mechanism, byte numbering mechanism, etc.
  • unreliable transport protocols can also transfer data without a connection.
  • Unreliable transport protocols may include, but are not limited to, user datagram protocol (UDP).
  • UDP user datagram protocol
  • the server sends the enhancement layer data based on the unreliable transmission protocol, and accordingly, the terminal receives the enhancement layer data based on the unreliable transmission protocol, that is, decapsulates the received data packet based on the unreliable transmission protocol.
  • a reliable transmission protocol is used for transmission of the base layer data
  • an unreliable transmission protocol is used for transmission of the enhancement layer data.
  • the burst bit rate is high and the transmission resources of the mobile communication network are insufficient
  • the transmission of data at the base layer can be guaranteed from the perspective of the transmission layer, and the terminal can receive the most basic video quality, frame rate and resolution.
  • the basic layer data will not be sent to the situation of stalling; from the perspective of the transmission layer, the enhancement layer data is transmitted in a best-effort manner, and the transmission is not forced to be successful, thereby reducing the load of the mobile communication network, thereby enabling the mobile communication network to be more efficient. serve many users.
  • the above-mentioned base layer data and enhancement layer data may be obtained based on scalable video coding (scalable video coding, SVC).
  • the source station may encode the source data based on SVC to obtain the base layer data and the enhancement layer data
  • the server may encode the source data based on the SVC to obtain the base layer data and the enhancement layer data.
  • SVC technology is designed to solve the problem that the encoder needs multiple encodings or the server needs secondary encoding and decoding in the face of different network conditions, different terminal processing capabilities and different user quality requirements.
  • temporal scalability temporal scalability
  • spatial scalability spatial scalability
  • quality scalability quality scalability
  • the above-mentioned base layer data and enhancement layer data may also be obtained based on low complexity enhancement coding (low complexity enhancement video coding, LCEVC).
  • LCEVC low complexity enhancement video coding
  • the source station may encode the source data based on LCEVC to obtain the base layer data and the enhancement layer data
  • the server may encode the source data based on the LCEVC to obtain the base layer data and the enhancement layer data.
  • the working principle of LCEVC is to use a lower resolution to obtain the base codec, which can use any standard codec to obtain base layer data with basic quality, and then add a residual layer for upsampling, After the enhancement layer data is obtained, a higher quality video can be reconstructed by superimposing the base layer and the enhancement layer.
  • the server can also set the corresponding quality of service for the base layer data and the enhancement layer data respectively.
  • the QoS is higher than the second QoS set for the enhancement layer data.
  • the server sends the parameter information of the first QoS and the parameter information of the second QoS to the device of the mobile communication network, so as to instruct the device in the mobile communication network to send the base layer data to the terminal according to the first QoS, and to send the enhancement layer data according to the second QoS to the terminal. data is sent to the terminal.
  • Setting different QoS for base layer data and enhancement layer data helps to reduce resource consumption of mobile communication network, so as to ensure the normal use of users at the same time. , to increase the number of users that the mobile communication network can carry.
  • the server will send the information of the first QoS set for the base layer data and the information of the second QoS set for the enhancement layer data to the policy control function (policy control function) in the core network.
  • policy control function policy control function
  • SMF session management function
  • the server can send the base layer data and the enhancement layer data to a user plane function (UPF) in the core network, and the UPF sends the base layer data and the enhancement layer data to the access network device.
  • the access network device sends the base layer data to the terminal according to the information of the first QoS, and sends the enhancement layer data to the terminal according to the information of the second QoS.
  • UPF user plane function
  • the purpose of setting QoS is to "customize on demand" to provide network services with differentiated quality of service for services under the condition of limited resources.
  • QoS can usually include two meanings, one is how the service quality is, that is, the specific indicators (parameters) that characterize QoS; the other is how to ensure these indicators, that is, the mechanism for realizing QoS.
  • the first QoS set by the server for the base layer data may include parameters involving the above two meanings, or only one of them; the second QoS set by the server for the enhancement layer data may also include parameters involving the above two meanings. meaning of the parameter, or, it can also include only one of them.
  • QoS can support two types of models, namely QoS flows that require guaranteed bit rate (GBR) and QoS flows that do not require guaranteed bit rate (non-GBR QoS flows).
  • the server can set the first QoS set for the base layer data as a GBR QoS stream, and the second QoS set for the enhancement layer data as a non-GBR QoS stream, so as to preferentially guarantee the bits of the base layer data transmission.
  • the bit rate of data transmission in the enhancement layer so as to reduce the load of the mobile communication network, thereby enabling the mobile communication network to provide services for more users.
  • the server can also set a corresponding transmission time window for the target frame data in the enhancement layer data, and then the server sends the data packet of the target frame data to the mobile communication network based on the unreliable transmission protocol within the transmission time window.
  • the target frame is any frame data in the enhancement layer data.
  • the server may set a transmission time window for each frame of data in the enhancement layer data, or the server may also set the same transmission time window for each frame of data in the enhancement layer data.
  • the server can first determine whether the data packet of the enhancement layer data to be sent exceeds the transmission time window corresponding to the target frame, If it does not exceed, the server will send it; if it has exceeded the corresponding transmission time window, the server can no longer send the unsent data packets in the target frame data. Because, after the corresponding transmission time window is exceeded, even if the server sends it to the terminal, the terminal may not have enough time to perform subsequent operations such as decapsulation, decoding, rendering and display, and the user's real-time synchronous viewing requirements cannot be achieved. In the case of transmission, enhancement layer data may be discarded.
  • the server sets the corresponding transmission time window for the target frame data in the enhancement layer data, which not only helps to meet the user's real-time requirements, but also helps when the mobile communication network is under heavy load and cannot fully meet the transmission of the enhancement layer data. Reduce the transmission of enhancement layer data.
  • the server may also send indication information to the terminal through the mobile communication network to indicate that one or more data packets in the target frame are no longer sent.
  • the terminal determines that the data of the target frame has not been completely sent and received, and the terminal can delete the received data packets belonging to the target frame, and then process the received data later.
  • the base layer data corresponding to the target frame can be decapsulated, decoded, rendered and displayed, that is, the target frame is displayed and played only according to the base layer data.
  • the server when the server sends multiple data packets of the enhancement layer data of the Nth frame, before sending each data packet, it first determines whether the transmission time window corresponding to the Nth frame is exceeded. If it does not exceed, the server continues to send; if it exceeds, the server Sending indication information to the terminal through the mobile communication network, indicating that the enhancement layer data of the Nth frame is no longer sent, and continues to send the data packets of the N+1th frame enhancement layer data.
  • the VR source station can send the base layer data and the enhancement layer data to the VR server.
  • the VR server sets the first QoS for the base layer data and the second QoS for the enhancement layer data, and combines the information of the first QoS with the data of the enhancement layer.
  • the information of the second QoS is sent to the PCF or the SMF in the mobile communication network, and then the PCF or the SMF sends the first QoS information and the second QoS information to the 5G radio access network (RAN).
  • RAN radio access network
  • the server sends the base layer data to the UPF in the mobile communication network based on the reliable transmission protocol, so that the mobile communication network forwards the base layer data to the terminal; it sends the enhancement layer data to the UPF based on the unreliable transmission protocol, so that the mobile communication network Forward the enhancement layer data to the terminal.
  • Figures 4 to 7 exemplarily provide the internal implementation of the VR server and the VR terminal based on the data transmission method shown in Figure 3 .
  • the VR source station sends the base layer data and the enhancement layer data to the VR server through a mixed data stream.
  • a VR server can include an application layer and a transport layer.
  • the application layer can include the following modules:
  • the splitting module is used to parse the video stream obtained from the VR source station and separate the base layer data and the enhancement layer data.
  • the QoS management module for setting high QoS for the base layer and low QoS for the enhancement layer, and sending the set QoS information to the PCF.
  • the QoS management module can also be used to calculate a corresponding transmission time window for each frame of data in the enhancement layer data, and send the information of the transmission time window to the transmission layer.
  • -Multi-stream transmission management module which is used to interface with the transport layer to manage the distribution and transmission of the data of the base layer and the data of the enhancement layer.
  • the transport layer in the VR server may include a reliable transport layer and an unreliable transport layer, which are respectively used to transmit base layer data and enhancement layer data.
  • the reliable transport layer in the VR server can support reliable transport protocols such as TCP and/or QUIC.
  • the unreliable transport layer in the VR server can indicate unreliable transport protocols such as UDP and/or RTP.
  • the unreliable transport layer may also include sub-modules such as delay queue, retransmission, and ACK/NACK.
  • the retransmission submodule is used to retransmit if the packet is still in the delay queue after packet loss occurs.
  • -ACK/NACK sub-module for sending acknowledgment, packet loss detection, including but not limited to ACK or NACK mechanisms.
  • the VR terminal may also include an application layer and a transport layer.
  • the transport layer of the VR terminal also includes a reliable transport layer and an unreliable transport layer, which are used to receive base layer data and enhancement layer data respectively.
  • the unreliable transport layer in the VR terminal may further include sub-modules such as order-preserving group and ACK/NACK. specific:
  • the sequence-preserving assembly sub-module is used to sort and assemble the data packets received from the VR server, and notify the application layer to obtain them after assembly.
  • ACK/NACK sub-module used for sending acknowledgment, packet loss detection, including but not limited to ACK or NACK mechanism.
  • the application layer of the VR terminal can include the following modules:
  • the decapsulation module is used to decapsulate the received data.
  • Decapsulation is the reverse process of video encapsulation, which separates audio and video.
  • the encapsulation format is FLV, MKV, MP4, AVI and other encapsulated files, and the audio data and video data are parsed.
  • the decapsulation module may include two sub-modules, respectively configured to decapsulate the base layer data received based on the reliable transport layer and decapsulate the number of enhancement layers received based on the unreliable transport layer.
  • the decoding module is used to jointly decode the decapsulated base layer data and the enhancement layer data.
  • Decoding is the inverse process of video encoding, parsing the original image data and original sound data.
  • the rendering and display module is used to render and display the decoded data.
  • Rendering refers to the process of generating images from models in software; models are descriptions of three-dimensional objects or virtual scenes that are strictly defined in language or data structures, and can include information such as geometry, viewpoints, textures, lighting, and shadows; images are digital images or Bitmap image.
  • Display is to transmit the image information generated by rendering to the screen for display.
  • the VR source station sends the base layer data and the enhancement layer data to the VR server through different data streams.
  • the enhancement layer includes several enhancement layers, the several enhancement layer data can pass through one channel.
  • the data stream is sent to the VR server, or it can be sent to the server through a different data stream. Among them, one data stream corresponds to a quintuple.
  • the VR server in the embodiment shown in FIG. 5 may also include an application layer and a transport layer, and the modules and functions included in each of them may refer to the embodiment shown in FIG. 4 , which will not be repeated here.
  • the VR terminal in the embodiment shown in FIG. 5 may also include an application layer and a transport layer, and the modules and functions included in each of them may refer to the embodiment shown in FIG. 4 , which will not be repeated here.
  • the VR source station sends the base layer data and the enhancement layer data to the VR server through a mixed data stream.
  • the VR server may include an application layer and a transport layer.
  • the modules and their functions protected by the application layer and the transport layer of the VR server may refer to the VR server in the embodiment shown in FIG. 4 , which will not be repeated here.
  • the VR terminal in the embodiment shown in FIG. 6 may include a transport layer and an application layer.
  • transport layer reference may be made to the transport layer of the VR terminal in the embodiment shown in FIG. 4 , which will not be repeated here.
  • the application layer of the VR terminal may also include modules such as decapsulation, decoding, confluence rendering, and display.
  • the decapsulation module includes a first decapsulation submodule and a second decapsulation submodule.
  • the first decapsulation submodule is used to decapsulate the base layer data received based on the reliable transport layer
  • the second decapsulation submodule is used to decapsulate the base layer data received based on the reliable transport layer. Decapsulates the number of enhancement layers received based on the unreliable transport layer.
  • the decoding module includes a first decoding module and a second decoding module.
  • the first decoding module is used for decoding the decapsulated base layer data
  • the second decoding module is used for decoding the decapsulated enhancement layer data.
  • the confluent rendering and display module is used for confluent rendering and display of the decoded base layer data and the decoded enhancement layer data.
  • the VR source station sends the base layer data and the enhancement layer data to the VR server through different data streams.
  • the enhancement layer includes several enhancement layers, the several enhancement layer data can pass through one channel.
  • the data stream is sent to the VR server, or it can be sent to the server through a different data stream. Among them, one data stream corresponds to a quintuple.
  • the VR server in the embodiment shown in FIG. 7 may also include an application layer and a transport layer.
  • the VR terminal in the embodiment shown in FIG. 7 may also include an application layer and a transport layer, and the modules and functions included in each may refer to the VR terminal in the embodiment shown in FIG. 6 , which will not be repeated here.
  • FIG. 8 is a schematic structural diagram of a server provided by an embodiment of the present application. As shown in the figure, the server includes an obtaining module 801 and a sending module 802 , and further, the server may further include a setting module 803 .
  • the obtaining module 801 is configured to obtain the base layer data and the enhancement layer data obtained by encoding the media data.
  • the sending module 802 is configured to send the base layer data to the terminal through the mobile communication network based on the reliable transmission protocol in the transport layer; based on the unreliable transmission protocol in the transport layer, send the enhancement layer data to the terminal through the mobile communication network. the terminal.
  • the server may further include a setting module 803, configured to set a first quality of service for the base layer data and a second quality of service for the enhancement layer data, where the first quality of service is higher than the second quality of service.
  • the sending module 802 is further configured to send the information of the first quality of service and the information of the second quality of service to the device in the mobile communication network, instructing the device according to the information of the first quality of service. information to send the base layer data to the terminal, and send the enhancement layer data to the terminal according to the second quality of service information.
  • the setting module 803 may be configured to set a transmission time window corresponding to target frame data in the enhancement layer data, where the target frame data is any frame data in the enhancement layer data.
  • the sending module 802 when sending the enhancement layer data to the terminal through the mobile communication network based on the unreliable transmission protocol in the transmission layer, the sending module 802 is specifically configured to, within the transmission time window, send the data based on the transmission layer to the terminal.
  • the unreliable transmission protocol in the mobile communication network sends the target frame data to the terminal.
  • modules may also be used to support other processes performed by the server in the embodiments shown in FIG. 2 to FIG. 7 .
  • the above-mentioned modules may also be used to support other processes performed by the server in the embodiments shown in FIG. 2 to FIG. 7 .
  • FIG. 9 is a schematic structural diagram of a terminal according to an embodiment of the present application. As shown in the figure, the terminal includes a processing module 901 and a receiving module 902 .
  • the processing module 901 is configured to, based on the reliable transmission protocol in the transport layer, receive the basic layer data sent by the server from the mobile communication network through the receiving module 902; Module 902 receives the enhancement layer data sent by the server from the mobile communication network; wherein, the base layer data and the enhancement layer data are data obtained by encoding the media data.
  • the receiving module 902 is further configured to receive, from the mobile communication network, indication information sent by the server, where the indication information is used to indicate to the server that one or more data packets in the target frame data are no longer available. send.
  • the foregoing modules may also be used to support other processes performed by the terminal in the embodiments shown in FIG. 2 to FIG. 7 .
  • the foregoing description may be made to the foregoing description, which will not be repeated here.
  • each module can be integrated together or can be implemented independently.
  • the processing element described here may be an integrated circuit with signal processing capability.
  • each step of the above-mentioned method or each of the above-mentioned modules can be completed by an integrated logic circuit of hardware in the processor element or an instruction in the form of software.
  • the above sending module is a module for controlling sending, and can send information through a sending device, such as an antenna and a radio frequency device.
  • the receiving module can also receive information through a receiving device, such as an antenna and a radio frequency device.
  • the above modules may be one or more integrated circuits configured to implement the above methods, such as: one or more application specific integrated circuits (ASIC), or, one or more microprocessors, or, a Or multiple field programmable gate arrays (field programmable gate array, FPGA), etc.
  • ASIC application specific integrated circuit
  • microprocessors or, a Or multiple field programmable gate arrays (field programmable gate array, FPGA), etc.
  • FPGA field programmable gate array
  • the processing element may be a general-purpose processor, such as a central processing unit (central processing unit, CPU) or other processors that can call programs.
  • CPU central processing unit
  • these modules can be integrated together and implemented in the form of a system-on-a-chip (SOC).
  • SOC system-on-a-chip
  • FIG. 10 is a schematic structural diagram of a server according to an embodiment of the present application.
  • the server includes a processor 1001 and a communication interface 1002 . Further, the server may further include a memory 1003 and a communication bus 1004 .
  • the processor 1001 may be a general-purpose CPU, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the programs of the present application.
  • Communication interface 1002 using any transceiver-like device, is used to communicate with other devices or a communication network, such as a RAN.
  • Communication bus 1004 may include a path to communicate information between the above-described components.
  • the communication bus 1004 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, or the like.
  • PCI peripheral component interconnect
  • EISA extended industry standard architecture
  • Memory 1003 may be read-only memory (ROM) or other types of static storage devices that can store static information and instructions, random access memory (RAM), or other types of information and instructions
  • the dynamic storage device can also be an electrically erasable programmable read-only memory (EEPROM) or can be used to carry or store the desired program code in the form of instructions or data structures and can be stored by a computer. any other medium taken, but not limited to this.
  • the memory 1003 can exist independently, such as an off-chip memory, and is connected to the processor 1001 through a communication bus 1004 .
  • the memory 1003 may also be integrated with the processor 1001 .
  • the communication interface 1002 is responsible for communicating with other devices or communication networks, and the processor 1001 is configured to implement other functions in the communication methods provided by the foregoing embodiments of the present application.
  • the processor 1001 may include one or more CPUs.
  • the processor 1001 can run the instructions or programs in the memory 1003, and perform the following steps through the communication interface 1002: obtain the base layer data and the enhancement layer data obtained by encoding the media data;
  • the mobile communication network sends the base layer data to the terminal; based on the unreliable transmission protocol in the transport layer, the enhancement layer data is sent to the terminal through the mobile communication network.
  • each of the above-mentioned devices can also be used to support other processes performed by the server in the embodiments shown in FIG. 2 to FIG. 7 .
  • the server in the embodiments shown in FIG. 2 to FIG. 7 .
  • FIG. 11 is a schematic structural diagram of a terminal provided by an embodiment of the application.
  • the terminal includes: a processor 1101 and a communication interface 1102 .
  • the server may also include a memory 1103 and a communication bus 1104 .
  • the processor 1101 may be a general-purpose CPU, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the programs of the present application.
  • Communication interface 1102 using any transceiver-like device, is used to communicate with other devices or a communication network, such as a wireless access network.
  • the communication bus 1104 may include a path to communicate information between the aforementioned components.
  • the communication bus 1104 may be a PCI bus, an EISA bus, or the like.
  • the memory 1103 may be a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, or an EEPROM or can be used to carry or store instructions or data with A desired program code in a structured form and any other medium that can be accessed by a computer, but not limited thereto.
  • the memory 1103 may exist independently, such as an off-chip memory, and is connected to the processor 1101 through the communication bus 1104 .
  • the memory 1103 may also be integrated with the processor 1101 .
  • the communication interface 1102 is responsible for communicating with other devices or communication networks, and the processor 1101 is configured to implement other functions in the communication methods provided by the above embodiments of the present application.
  • the processor 1101 may include one or more CPUs.
  • the processor 1101 can run the instructions or programs in the memory 1103, and perform the following steps through the communication interface 1102: based on the reliable transmission protocol in the transport layer, receive the base layer data sent by the server through the mobile communication network;
  • the unreliable transmission protocol is used to receive enhancement layer data sent by the server through the mobile communication network; wherein the base layer data and the enhancement layer data are data obtained by encoding media data.
  • the above-mentioned devices may also be used to support other processes performed by the terminal in the embodiments shown in FIG. 2 to FIG. 7 .
  • the foregoing description which will not be repeated here.
  • Embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium.
  • the computer program When the computer program is executed by a computer, the computer can implement FIG. 2 to FIG. 2 provided by the foregoing method embodiments.
  • the process related to the server in any one of the embodiments shown, or the computer can implement the process related to the terminal in any one of the embodiments shown in FIG. 2 to FIG. 7 provided by the above method embodiments.
  • Embodiments of the present application further provide a computer program product, where the computer program product is used to store a computer program, and when the computer program is executed by a computer, the computer can implement any one of FIG. 2 to FIG. 7 provided by the foregoing method embodiments.
  • the process related to the server in the shown embodiment, or the computer can implement the process related to the terminal in any of the embodiments shown in FIG. 2 to FIG. 7 provided by the above method embodiment.
  • the present application also provides a chip including a processor.
  • the processor is used to read and run the computer program stored in the memory, so as to execute the corresponding operations and/or processes of the server or the terminal in the data layered transmission method provided in this application.
  • the chip further includes a memory, the memory and the processor are connected to the memory through a circuit or a wire, and the processor is used for reading and executing the computer program in the memory.
  • the chip further includes a communication interface, and the processor is connected to the communication interface.
  • the communication interface is used to receive processed data and/or information, and the processor acquires the data and/or information from the communication interface and processes the data and/or information.
  • the communication interface may be an input/output interface, an interface circuit, an output circuit, an input circuit, a pin or a related circuit on the chip, and the like.
  • the processor may also be embodied as a processing circuit or a logic circuit.
  • the above-mentioned chip can also be replaced by a chip system, which will not be repeated here.
  • references in this specification to "one embodiment” or “some embodiments” and the like mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application.
  • appearances of the phrases “in one embodiment,” “in some embodiments,” “in other embodiments,” “in other embodiments,” etc. in various places in this specification are not necessarily All refer to the same embodiment, but mean “one or more but not all embodiments” unless specifically emphasized otherwise.
  • the terms “including”, “including”, “having” and their variants mean “including but not limited to” unless specifically emphasized otherwise.
  • An embodiment of the present application provides a computer-readable storage medium storing a computer program, where the computer program includes instructions for executing the foregoing method embodiments.
  • the embodiments of the present application provide a computer program product containing instructions, which, when executed on a computer, cause the computer to execute the above method embodiments.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Quality & Reliability (AREA)
  • Computer Security & Cryptography (AREA)
  • Mobile Radio Communication Systems (AREA)

Abstract

提出了一种数据分层传输方法、装置及系统。在该方法中，获取对媒体数据编码后得到的基础层数据和增强层数据（201）；基于传输层中的可靠传输协议，通过移动通信网络发送基础层数据给终端；基于传输层中的不可靠传输协议，通过移动通信网络发送增强层数据给终端。由于对基础层数据采用可靠的传输协议进行传输，对增强层数据采用不可靠传输协议进行传输，使得在突发码率较高、移动通信网络传输资源不足时，从传输层的角度保障基础层数据的传输，从传输层的角度对增强层数据采用尽力而为的传输方式，不强求传输成功，从而避免用户在观看媒体数据时发生卡顿，降低移动通信网络的负载，进而使得移动通信网络能够为更多的用户提供服务。

Description

一种数据分层传输方法、装置及系统
相关申请的交叉引用
本申请要求在2021年03月31日提交中国专利局、申请号为202110348713.7、申请名称为“一种数据分层传输方法、装置及系统”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及数据传输技术领域，尤其涉及一种数据分层传输方法、装置及系统。
背景技术
虚拟现实(virtual reality，VR)技术，是20世纪发展起来的一项全新的实用技术。虚拟现实技术囊括计算机、电子信息、仿真技术于一体，其基本实现方式是计算机模拟虚拟环境，从而给用户以环境沉浸感。随着社会生产力和科学技术的不断发展，各行各业对VR技术的需求日益旺盛，VR技术也取得了巨大进步，并逐步成为一个新的科学技术领域。
8K VR直播，是虚拟现实与8K视频直播的结合。与传统电视观看相比，VR直播最大区别是让观众身临其境来到现场，实时全方位体验。为了保证8K VR直播观看的真实感、交互感和愉悦感，在用户转头时，高清视场角(filed of view，FOV)画面呈现的时间要尽可能的小，例如画面转换时间在200ms以内，可以使用户不至于感受到卡顿。高清FOV画面的呈现历经网络传输、解封装、数据包(tile)合并、解码、渲染、显示等多个步骤，其中解码、渲染、显示需要消耗大部分时间，也就是说，可能需要在20～30ms内完成高清FOV的12～18个Tile的同步传输。这一情况可能导致高清FOV突发码率高达200Mbps以上。而传统的无差别的数据传输方式，使得移动通信网络面对如此高的突发码率，难以大规模承载VR用户，进而限制了VR技术的大规模商用。
发明内容
本申请提供一种数据分层传输方法、装置及系统，用于减少用户浏览数据时发生卡顿的情况，提高移动通信网络承载的用户数量。
第一方面，本申请实施例提供一种数据分层传输方法，包括：获取对媒体数据编码后得到的基础层数据和增强层数据；基于传输层中的可靠传输协议，通过移动通信网络发送所述基础层数据给终端；基于传输层中的不可靠传输协议，通过所述移动通信网络发送所述增强层数据给所述终端。由于对基础层数据采用可靠的传输协议进行传输，对增强层数据采用不可靠传输协议进行传输，使得在突发码率较高、移动通信网络传输资源不足时，从传输层的角度保障基础层数据的传输，从传输层的角度对增强层数据采用尽力而为的传输方式，不强求传输成功，从而避免用户在观看媒体数据时发生卡顿，降低移动通信网络的负载，进而使得移动通信网络能够为更多的用户提供服务。
在一种可能的实现方式中，所述方法还包括：对所述基础层数据设置第一服务质量，对所述增强层数据设置第二服务质量，所述第一服务质量高于所述第二服务质量；将所述 第一服务质量的信息和所述第二服务质量的信息发送至所述移动通信网络中的设备，指示所述设备根据所述第一服务质量的信息将所述基础层数据发送给所述终端，根据所述第二服务质量的信息将所述增强层数据发送给所述终端。在该实现方式中，还可以为基础层数据和增强层数据设置不同的服务质量，以使移动通信网络中的设备按照不同的服务质量向终端发送基础层数据和增强层数据，其中，基础层数据的服务质量高于增强层数据的服务质量；该方式相较于按照相同的服务质量发送基础层数据和增强层数据的方式，有助于降低移动通信网络的资源消耗，从而在保证用户正常使用的同时，提升移动通信网络能够承载的用户数量。
在一种可能的实现方式中，所述可靠传输协议包括传输控制协议TCP或快速UDP网络连接QUIC协议。
在一种可能的实现方式中，所述不可靠传输协议包括用户数据报协议UDP或实时传输协议RTP。
在一种可能的实现方式中，所述方法还包括：设置所述增强层数据中目标帧数据对应的传输时间窗口，所述目标帧数据为所述增强层数据中的任一帧数据；所述基于传输层中的不可靠传输协议，通过所述移动通信网络发送所述增强层数据给所述终端，包括：在所述传输时间窗口内，基于传输层中的不可靠传输协议，通过所述移动通信网络发送所述目标帧数据给所述终端。
在一种可能的实现方式中，所述方法还包括：若所述目标帧数据的一个或多个数据包未能在所述传输时间窗口内完成发送；通过所述移动通信网络向所述终端发送指示信息，所述指示信息用于指示所述目标帧数据中的一个或多个数据包不再发送。对于实时性要求较高的场景，例如VR直播的场景，若在某一帧数据未能在预设的时间窗口内发送完成，则没有再发送该帧数据的必要，因为即使发送给终端，终端也来不及对其进行后续解封装、解码、渲染显示等操作，无法实现用户的实时同步观看。
在一种可能的实现方式中，所述获取对媒体数据编码后得到的基础层数据和增强层数据，包括：获取所述媒体数据的混合数据，所述混合数据包括多个数据包；对每个所述数据包进行解封装，确定所述数据包属于所述基础层数据或属于所述增强层数据。在该实现方式中，服务器可以基于一条数据流从媒体数据源站获取媒体数据的数据包，在对数据包进行解封装后，才能确定数据包属于基础层数据还是增强层数据。
在一种可能的实现方式中，所述获取对媒体数据编码后得到的基础层数据和增强层数据，包括：获取所述媒体数据的数据包，根据每个所述数据包的五元组信息，确定所述数据包属于所述基础层数据或属于所述增强层数据。在该实现方式中，服务器可以基于不同的数据流从媒体数据源站获取基础层数据和增强层数据，数据流通过五元组信息定义，即，两条数据流的五元组信息不完全相同。
在一种可能的实现方式中，所述获取对媒体数据编码后得到的基础层数据和增强层数据，包括：获取所述媒体数据的源数据，对所述源数据进行编码得到所述基础层数据和所述增强层数据。在该实现方式中，服务器也可以从媒体源站获取源数据，然后由服务器对源数据进行编码从而得到基础层数据和增强层数据。
第二方面，本申请实施例提供一种数据分层传输方法，包括：基于传输层中的可靠传输协议，通过移动通信网络接收服务器发送的基础层数据；基于传输层中的不可靠传输协议，通过所述移动通信网络接收所述服务器发送的增强层数据；其中，所述基础层数据和 所述增强层数据为对媒体数据编码后得到的数据。
在一种可能的实现方式中，所述可靠传输协议包括传输控制协议TCP或快速UDP网络连接QUIC协议。
在一种可能的实现方式中，所述不可靠传输协议包括用户数据报协议UDP或实时传输协议RTP。
在一种可能的实现方式中，所述方法还包括：通过所述移动通信网络接收服务器发送的指示信息，所述指示信息用于指示所述服务器对一个所述目标帧数据中的一个或多个数据包不再发送；所述指示信息为所述服务器在所述目标帧数据中的一个或多个数据包，未能在为所述目标帧设置的传输时间窗口内发送时发送的。
第三方面，本申请实施例提供一种服务器，包括处理器，以及分别与所述处理器耦合的存储器和通信接口；所述通信接口用于与其他设备进行通信；所述处理器用于运行所述存储器内的指令或程序，通过所述通信接口执行以下步骤：获取对媒体数据编码后得到的基础层数据和增强层数据；基于传输层中的可靠传输协议，通过移动通信网络发送所述基础层数据给终端；基于传输层中的不可靠传输协议，通过所述移动通信网络发送所述增强层数据给所述终端。
在一种可能的实现方式中，所述处理还用于：对所述基础层数据设置第一服务质量，对所述增强层数据设置第二服务质量，所述第一服务质量高于所述第二服务质量；将所述第一服务质量的信息和所述第二服务质量的信息发送至所述移动通信网络中的设备，指示所述设备根据所述第一服务质量的信息将所述基础层数据发送给所述终端，根据所述第二服务质量的信息将所述增强层数据发送给所述终端。
在一种可能的实现方式中，所述可靠传输协议包括传输控制协议TCP或快速UDP网络连接QUIC协议。
在一种可能的实现方式中，所述不可靠传输协议包括用户数据报协议UDP或实时传输协议RTP。
在一种可能的实现方式中，所述处理器还用于：设置所述增强层数据中目标帧数据对应的传输时间窗口，所述目标帧数据为所述增强层数据中的任一帧数据；所述处理器，在基于传输层中的不可靠传输协议，通过所述移动通信网络发送所述增强层数据给所述终端时，具体用于：在所述传输时间窗口内，基于传输层中的不可靠传输协议，通过所述移动通信网络发送所述目标帧数据给所述终端。
在一种可能的实现方式中，所述处理器还用于：若所述目标帧数据的一个或多个数据包未能在所述传输时间窗口内完成发送；通过所述移动通信网络向所述终端发送指示信息，所述指示信息用于指示所述目标帧数据中的一个或多个数据包不再发送。
在一种可能的实现方式中，所述处理器，在获取对媒体数据编码后得到的基础层数据和增强层数据时，具体用于：获取所述媒体数据的混合数据，所述混合数据包括多个数据包；对每个所述数据包进行解封装，确定所述数据包属于所述基础层数据或属于所述增强层数据。
第四方面，本申请实施例提供一种终端，包括：处理器，以及分别与所述处理器耦合的存储器和通信接口；所述通信接口用于与其他设备进行通信；所述处理器用于运行所述存储器内的指令或程序，通过所述通信接口执行以下步骤：基于传输层中的可靠传输协议，通过移动通信网络接收服务器发送的基础层数据；基于传输层中的不可靠传输协议，通过 所述移动通信网络接收所述服务器发送的增强层数据；其中，所述基础层数据和所述增强层数据为对媒体数据编码后得到的数据。
在一种可能的实现方式中，所述可靠传输协议包括传输控制协议TCP或快速UDP网络连接QUIC协议。
在一种可能的实现方式中，所述不可靠传输协议包括用户数据报协议UDP或实时传输协议RTP。
在一种可能的实现方式中，所述处理器还用于：通过所述移动通信网络接收所述服务器发送的指示信息，所述指示信息用于指示所述服务器对一个所述目标帧数据中的一个或多个数据包不再发送；所述指示信息为所述服务器在所述目标帧数据中的一个或多个数据包，未能在为所述目标帧设置的传输时间窗口内发送时发送的。
第五方面，本申请实施例提供一种数据分层传输系统，包括：第三方面及第三方面任一实现方式中所述的服务器，和第四方面及第四方面任一实现方式中所述的终端。
在一种可能的实现方式中，所述系统还包括源站，用于对源数据进行编码，得到所述基础层数据和所述增强层数据，将所述基础层数据和所述增强层数据发送给所述服务器。
第六方面，本申请实施例提供一种计算机可读存储介质，所述计算机可读存储介质中存储有指令，当所述指令在计算机上运行时，使得所述计算机执行如第一方面及第一方面任一实现方式所述的方法，或者，使得所述计算机执行如第二方面及第二方面任一实现方式所述的方法。
附图说明
图1为本申请实施例提供的数据分层传输方法所适用的架构示意图；
图2为本申请实施例提供的数据分层传输方法的流程示意图；
图3为本申请实施例提供的数据分层传输架构示意图；
图4为本申请实施例提供的一种服务器和终端内部实现示意图；
图5为本申请实施例提供的另一种服务器和终端内部实现示意图；
图6为本申请实施例提供的另一种服务器和终端内部实现示意图；
图7为本申请实施例提供的另一种服务器和终端内部实现示意图；
图8为本申请实施例提供的一种服务器结构示意图；
图9为本申请实施例提供的一种终端结构示意图；
图10为本申请实施例提供的另一种服务器结构示意图；
图11为本申请实施例提供的另一种终端结构示意图。
具体实施方式
在VR的应用场景中，VR终端设备，如VR头盔、VR眼镜等，从VR服务器接收VR数据，然后对接收到的数据进行解封装、解码、渲染等操作，从而将其展示给用户，以带给用户身临其境的体验感。目前市场上的VR头盔、VR眼镜等能够支持WiFi、移动通信网络(如4G、5G)，然而，当通过移动网络进行数据传输时，由于VR数据量大，且对传输时延的要求较高，尤其是在VR直播领域，需要占用较高的码率，增加移动通信网络的负载，使得移动通信网络能够承载的VR用户数量较少，还可能会导致数据传输延时较大， 降低用户体验。
因此，本申请实施例提供一种数据分层传输方法，用于从传输层的角度解决上述问题，减少用户观看时发生卡顿的情况，提高移动通信网络能够承载的用户数量。
本申请实施例提供的数据分层传输方法，可以应用于如图1所述的架构中，如图所示，服务器通过移动通信网络将数据发送给终端。本申请实施例提供的数据分层传输方法，适用于通过移动通信网络传输数量较大的、实时性要求较高的媒体数据，例如，通过使用移动通信网络进行数据传输的VR场景，VR游戏、VR直播等。
本申请实施例中的终端，可以包括但不限于VR头盔、VR眼镜、投影等用户设备。
本申请实施例中的基础层数据，是用于为用户提供最基本的视频质量、帧率和分辨率的数据；增强层数据是用于对视频质量进行完善的数据，为用户提供较高的视频质量、帧率、分辨率，以提高用户观看效果。此外，根据不同的编码方式，可以得到一路基础层数据和一路增强层数据，或者，也可以得到一路基础层数据和多路增强层数据。
参见图2，为本申请实施例提供的数据分层传输方法的流程示意图，如图所示，该方法可以包括以下步骤：
步骤201、服务器获取对媒体数据编码后得到的基础层数据和增强层数据。
具体的，服务器可以从内容提供方(以下简称源站)获取媒体数据的基础层数据和增强层数据，即，由源站对源媒体数据进行编码得到基础层数据和增强层数据。或者，服务器也可以从源站接收媒体数据的源数据，由服务器对源数据进行编码，从而获得基础层数据和增强层数据。
在一些实施例中，当源站对源数据进行编码得到基础层数据和增强层数据时，源站可以将基础层数据和增强层数据通过一个数据流发送给服务器，即，服务器通过一个数据流接受媒体数据的混合数据，混合数据包括基础层数据的若干数据包和增强层数据的若干数据包，而服务器无法通过接受到的数据包的五元组信息确定出该数据包属于基础层数据还是属于增强层。此时，服务器可以对接收到的每个数据进行解封装，根据解封装后获取到的包头信息确定该数据包属于基础层数据或属于增强层数据。
在另外一些实施例中，当源站对源数据进行编码得到基础层数据和增强层数据时，源站可以将基础层数据和增强层数据通过不同的数据流发送给服务器。其中，数据流可以通过五元组定义，即，一条数据流对应一个五元组信息，不同的数据流对应不同的五元组信息。五元组，为源IP地址、源端口、目的IP地址、目的端口和传输层协议这五种信息组成的一个集合。基础层数据和增强层数通过不同的数据流发送给服务器，表示基础层数据的数据包对应的五元组，与增强层数据的数据包对应的五元组不同。例如，基础层数据包对应的五元组中传输层协议与增强层数据包对应的五元组中传输层协议不同，服务器根据数据包对应的五元组中的传输层协议，确定接收到的数据包属于基础层数据的数据包，还是属于增强层数据的数据包；或者，也可以令基础层数据包对应的五元组中目的端口与增强层数据包对应的五元组中目的端口不同；或者，还可以令基础层数据包对应的五元组中源端口与增强层数据包对应的五元组中源端口不同。
此外，对源数据进行编码得到基础层数据和增强层数据的步骤，也可以由服务器执行。源站将媒体数据的源数据发送给服务器，服务器根据预设的编码方式，对源数据进行编码，从而得到基础层数据和增强层数据。
步骤202、服务器基于传输层中的可靠传输协议，通过移动通信网络发送基础层数据 给终端；相应地，终端基于传输层中的可靠传输协议，通过移动通信网络接收服务器发送的基础层数据。
传输层是开放式系统互联(open system interconnection，OSI)中尤为关键的一层。传输层提供端到端的交换数据的机制，检查分组编号与次序。传输层对其上层，如应用层等，提供可靠的传输服务，对网络层提供可靠的目的地站点信息。
传输层中的传输协议，按照协议传输的可靠性来分类，可将分为可靠传输协议和不可靠传输协议。可靠传输协议能够保证数据传输的可靠属性；不可靠传输协议是指一种不可靠的无连接数据报协议，不可靠协议不保证数据被无损传输，甚至不保证传输成功与否，提供“尽力而为”的交付服务。
如前所述，基础层数据是用于为用户提供最基本的视频质量、帧率和分辨率的数据，因此，本申请实施例中，服务器基于可靠传输协议，通过移动通信网络发送基础层数据给终端，以保证终端对基础层数据的接收。可靠传输协议，可以具有确认、重传机制，还可以具有校验机制，还可以具有字节编号机制。此外，常见的可靠传输协议，可以在数据传输前先建立连接，在“双方握手”后再进行数据传输。
具体的，可靠传输协议可以包括但不限于传输控制协议(transmission control protocol，TCP)、快速UDP网络连接(quick UDP internet connections，QUIC)协议等。
服务器基于可靠传输协议发送基础层数据，相应的，终端则基于可靠传输协议接收基础层数据，即，基于该可靠传输协议对接收到的数据包进行解封装。
步骤203、服务器基于传输层中的不可靠传输协议，通过移动通信网络发送增强层数据给终端；相应地，终端基于传输层中的不可靠传输协议，通过移动通信网络接收服务器发送的增强层数据。
如前所述，增强层数据是用于对视频质量进行完善的数据，为用户提供较高的视频质量、帧率、分辨率，以提高用户观看效果。故，若终端能够接收到增强层数据，则可以提高用户的观看体验；若终端不能接收到增强层数据，也可以基于基础层数据进行展示。基于此，在本申请实施例中，可以从传输层的角度，向终端提供“尽力而为”的交付服务，但不必保障终端一定能够接收成功。具体的，服务器可以基于不可靠传输协议，通过移动通信网络发送增强层数据给终端。
常见的不可靠传输协议，可以不具备确认、重传机制，校验机制，字节编号机制等。此外，不可靠传输协议也可以在无连接的情况下进行数据传输。不可靠传输协议可以包括但不限于用户数据报协议(user datagram protocol，UDP)。
服务器基于不可靠传输协议发送增强层数据，相应的，终端则基于不可靠传输协议接收增强层数据，即，基于该不可靠传输协议对接收到的数据包进行解封装。
在本申请上述实施例中，对基础层数据采用可靠的传输协议进行传输，对增强层数据采用不可靠传输协议进行传输。使得在突发码率较高、移动通信网络传输资源不足时，从传输层的角度保障基础层数据的传输，保障了终端能够接到用于提供最基本的视频质量、帧率和分辨率的基础层数据，不至于发送卡顿的情况；从传输层的角度对增强层数据采用尽力而为的传输方式，不强求传输成功，从而降低移动通信网络的负载，进而使得移动通信网络能够为更多的用户提供服务。
在一种可能的实现方式中，上述基础层数据和增强层数据可以是基于可伸缩视频编码(scalable video coding，SVC)得到的。具体的，可以是源站基于SVC对源数据编码得到 基础层数据和增强层数据，也可以是服务器基于SVC对源数据编码得到基础层数据和增强层数据。SVC技术旨在解决面对不同网络条件、不同终端处理能力以及不同用户质量需求，编码端需要多次编码或服务器需要二次编解码的问题，通过SVC技术的使用，可实现时域可分级(temporal scalability)、空间可分级(spatial scalability)以及质量可分级(quality scalability)，即一次编码可产生不同帧率、分辨率、图像质量的视频压缩码流解码端进行自适应调整，减轻了编码端与服务器额外的运算负担。
或者，上述基础层数据和增强层数据也可以是基于低复杂度增强编码(low complexity enhancement video coding，LCEVC)得到的。具体的，可以是源站基于LCEVC对源数据编码得到基础层数据和增强层数据，也可以是服务器基于LCEVC对源数据编码得到基础层数据和增强层数据。LCEVC的工作原理是采用较低的分辨率获取基本编解码器，该基本编解码器可以用任一标准编解码器，得到具有基本质量的基础层数据，然后可以添加残差层进行上采样，得到增强层数据，将基础层与增强层叠加后便可以重建更高质量的视频。
此外，也可以采用其他编码方式得到上述基础层数据和增强层数据，本申请实施例不再一一举例。
为了进一步保障基础层数据的传输、降低移动通信网络的负载，服务器还可以为基础层数据和增强层数据分别设置各自对应的服务质量(quality of service)，其中，为基础层数据设置的第一QoS高于为增强层数据设置的第二QoS。服务器将第一QoS的参数信息和第二QoS的参数信息发送给移动通信网络的设备，以指示移动通信网络中的设备根据第一QoS将基础层数据发送给终端，根据第二QoS将增强层数据发送给终端。对基础层数据和增强层数据设置不同的QoS，相较于按照相同的QoS发送基础层数据和增强层数据的方式，有助于降低移动通信网络的资源消耗，从而在保证用户正常使用的同时，提升移动通信网络能够承载的用户数量。
以第五代移动通信系统(5G)为例，服务器将为基础层数据设置的第一QoS的信息和为增强层数据设置的第二QoS的信息发送给核心网中的策略控制功能(policy control function，PCF)、会话管理功能(session management function，SMF)等网元，以使上述网元再将第一QoS的信息和第二QoS的信息发送给接入网设备。服务器可以将基础层数据和增强层数据发送给核心网中的用户面功能(user plane function，UPF)，UPF将基础层数据和增强层数据发送给接入网设备。接入网设备则根据第一QoS的信息将基础层数据发送给终端，根据第二QoS的信息将增强层数据发送给终端。
在移动通信网络中，设置QoS的目的，是在资源有限的情况下，“按需定制”为业务提供差异化服务质量的网络服务。QoS通常可以包含有两种含义，一是服务质量怎么样，即表征QoS的具体指标(参数)；二是如何保证这些指标，即实现QoS的机制。相应的，服务器为基础层数据设置的第一QoS，可以包括涉及上述两种含义的参数，也可以仅包括其中一种；服务器为增强层数据设置的第二QoS，也可以包括涉及上述两种含义的参数，或者，也可以仅包括其中一种。
此外，在5G中，QoS可以支持两种类型的模型，即需要保证比特速率(guaranteed bit rate，GBR)的QoS流和不需要保证比特速率的QoS流(非GBR QoS流)。可选的，服务器可以将为基础层数据设置的第一QoS，设置为GBR QoS流，将为增强层数据设置的第二QoS，设置为非GBR QoS流，从而优先保证基础层数据传输的比特速率，而对于增强层数据传输的比特速率不做较高要求，以降低移动通信网络的负载，进而使得移动通信网 络能够为更多的用户提供服务。
在一些实时性要求较高的场景下，例如VR直播的场景，为了满足用户实时同步观看的需要，对终端接收数据的时延有着较高的要求，若实时数据未能在较短的时延内，则不能满足用户对实时性的需要，除了降低用户体验，还可能给用户带来损失。因此，服务器还可以为增强层数据中的目标帧数据设置对应的传输时间窗口，然后服务器在该传输时间窗口内，将该目标帧数据的数据包，基于不可靠传输协议通过移动通信网络发送给终端。其中，目标帧为增强层数据中的任一帧数据。在设置传输时间窗口时，可以将不影响用户观看的最大时延减去编解码、渲染等处理的耗时，剩下时间的即为传输的时间，即，传输时间窗口的大小。
服务器可以为增强层数据中的每一帧数据设置一个传输时间窗口，或者，服务器也可以为增强层数据中的每一帧数据设置相同的传输时间窗口。
当服务器基于不可靠传输协议通过移动通信网络向终端发送增强层数据中的目标帧数据时，服务器可以先对待发送的增强层数据的数据包，确定是否超出了该目标帧对应的传输时间窗口，若未超出，服务器对其进行发送；若已超出对应的传输时间窗口，则服务器可以不再发送该目标帧数据中未发送完成的数据包。因为，在超出相应传输时间窗口后，即使服务器发送给终端，终端也可能来不及对其进行后续解封装、解码、渲染显示等操作，无法实现用户的实时同步观看的需求，而在保证基础层数据传输的情况下，增强层数据可以被放弃。
服务器为增强层数据中的目标帧数据设置对应的传输时间窗口，不仅有助于满足用户对实时性的要求，还有助于在移动通信网络负载较重无法完全满足增强层数据的传输时，减少增强层数据的传输。
此时，服务器还可以通过移动通信网络向终端发送指示信息，以指示目标帧中的一个或多个数据包不再发送。相应的，终端在接收到该指示信息后，确定目标帧的数据没有被完全发送、接收，则终端可以删除已接收到的属于该目标帧的数据包，在后续对接收到的数据进行处理时，可以对该目标帧对应的基础层数据进行解封装、解码、渲染显示等操作，即，该目标帧仅根据基础层数据进行显示播放。
例如，服务器在发送第N帧增强层数据的多个数据包时，在发送每个数据包之前先判断是否超出第N帧对应的传输时间窗口，若未超出，服务器继续发送；若超出，服务器通过移动通信网络向终端发送指示信息，指示第N帧的增强层数据不再被发送，并继续发送第N+1帧增强层数据的数据包。
为了更加清楚理解本申请上述实施例，下面结合图3至图7，以VR应用场景为例进行举例说明。
如图3所示，VR源站可以向VR服务器发送基础层数据和增强层数据，VR服务器为基础层数据设置第一QoS，为增强层数据设置第二QoS，并将第一QoS的信息和第二QoS的信息发送给移动通信网络中的PCF或SMF，然后PCF或SMF将第一QoS信息和第二QoS信息发送给5G无线接入网(radio access network，RAN)。服务器将基于可靠传输协议将基础层数据发送给移动通信网络中UPF，以使移动通信网络将基础层数据转发给终端；将基于不可靠传输协议将增强层数据发送给UPF，以使移动通信网络将增强层数据转发给终端。
图4至图7示例性的提供了基于图3所示数据传输方式时，VR服务器和VR终端的 内部实现。
在图4所示的实施例中，VR源站通过一路混合数据流将基础层数据和增强层数据发送给VR服务器。VR服务器可以包括应用层和传输层。其中，应用层可以包括以下模块：
-拆分模块，用于解析从VR源站获取的视频流，将基础层数据和增强层数据分离开来。
-QoS管理模块，用于为基础层设置高QoS，为增强层设置低QoS，并将设置的QoS信息发送给PCF。此外，QoS管理模块还可以用于实现为增强层数据中的每帧数据计算相应的传输时间窗口，并将传输时间窗口的信息发送至传输层。
-多流传输管理模块，用于与传输层对接，为基础层数据、增强层数据进行分流传输管理。
VR服务器中的传输层可以包括可靠传输层和不可靠传输层，分别用于发送基础层数据和增强层数据。
VR服务器中的可靠传输层，可以支持TCP和/或QUIC等可靠传输协议。
VR服务器中的不可靠传输层可以指示UDP和/或RTP等不可靠传输协议。此外，不可靠传输层还可以包括延迟队列、重传、ACK/NACK等子模块。
-延迟队列子模块，用于保存尚未成功发送到对端且尚未超出传输时间窗口的数据包。
-重传子模块，用于在发生丢包后，如果该数据包还在延迟队列中，则进行重传。
-ACK/NACK子模块，用于发送确认、丢包检测，包括但不限于ACK或NACK机制。
VR终端也可以包括应用层和传输层。VR终端的传输层也包括可靠传输层和不可靠传输层，分别用于接收基础层数据和增强层数据。
其中，VR终端中的不可靠传输层又可以包括保序组、ACK/NACK等子模块。具体的：
保序组装子模块，用于对从VR服务器接收到的数据包进行排序组装，组装后通知应用层获取。
ACK/NACK子模块，用于发送确认、丢包检测，包括但不限于ACK或NACK机制。
VR终端的应用层可以包括以下模块：
解封装模块，用于对接收到的数据进行解封装。解封装即为视频封装的逆过程，将音视频分离，例如将封装格式为FLV、MKV、MP4、AVI等封装格式的文件，将音频数据、视频数据解析出来。进一步的，解封装模块可以包括两个子模块，分别用于对基于可靠传输层接收的基础层数据进行解封装以及对基于不可靠传输层接收到的增强层数进行解封装。
解码模块，用于对解封装后的基础层数据和增强层数据进行联合解码。解码即为视频编码的逆过程，将原始图像数据、原始声音数据解析出来。
渲染、显示模块，用于对解码后的数据进行渲染、显示。渲染是指以软件由模型生成图像的过程；模型是用语言或者数据结构进行严格定义的三维物体或虚拟场景的描述，可以包括几何、视点、纹理、照明和阴影等信息；图像是数字图像或者位图图像。显示则是将渲染生成的图像信息输送到屏幕上显示呈现出来。
在图5所示的实施例中，VR源站将基础层数据和增强层数据通过不同的数据流发送给VR服务器，若增强层包括若干个增强层，则该若干个增强层数据可以通过一路数据流发送给VR服务器，或者也可以通过不同的数据流发送给服务器。其中，一路数据流对应一个五元组。
图5所述实施例中的VR服务器，也可以包括应用层和传输层，其各自包含的模块及 其功能可以参见图4所示的实施例，此处不再赘述。图5所述实施例中的VR终端也可以包括应用层和传输层，其各自包含的模块及其功能可以参见图4所示的实施例，此处不再赘述。
在图6所述的实施例中，VR源站通过一路混合数据流将基础层数据和增强层数据发送给VR服务器。VR服务器可以包括应用层和传输层，VR服务器的应用层和传输层所波爱护的模块及其功能，可以参见图4所示实施例VR服务器，此处不再赘述。
图6所述实施例中的VR终端，可以包括传输层和应用层，其中，传输层可以参见图4所示实施例中VR终端的传输层，此处不再赘述。VR终端的应用层，也可以包括解封装、解码及合流渲染、显示等模块。
其中，解封装模块包括第一解封装子模块和第二解封装子模块，第一解封装子模块用于对基于可靠传输层接收的基础层数据进行解封装，第二解封装子模块用于对基于不可靠传输层接收到的增强层数进行解封装。
解码模块包括第一解码模块和第二解码模块，第一解码模块用于对解封装后的基础层数据进行解码，第二解码模块用于对解封装后的增强层数据进行解码。
合流渲染、显示模块，用于对解码后的基础层数据和解码后的增强层数据进行合流渲染并显示。
在图7所示的实施例中，VR源站将基础层数据和增强层数据通过不同的数据流发送给VR服务器，若增强层包括若干个增强层，则该若干个增强层数据可以通过一路数据流发送给VR服务器，或者也可以通过不同的数据流发送给服务器。其中，一路数据流对应一个五元组。
图7所述实施例中的VR服务器，也可以包括应用层和传输层，其各自包含的模块及其功能可以参见图4所示实施例中的VR服务器，此处不再赘述。图7所述实施例中的VR终端，也可以包括应用层和传输层，其各自包含的模块及其功能可以参见图6所示实施例中的VR终端，此处不再赘述。
基于相同的技术构思，本申请实施例还提供一种服务器，用于实现上述数据分层传输方法中服务器所执行的方法步骤。图8为本申请实施例提供的一种服务器的结构示意图，如图所示，该服务器包括获取模块801和发送模块802，进一步的，该服务器还可以包括设置模块803。
具体的，获取模块801用于获取对媒体数据编码后得到的基础层数据和增强层数据。
发送模块802用于基于传输层中的可靠传输协议，通过移动通信网络发送所述基础层数据给终端；基于传输层中的不可靠传输协议，通过所述移动通信网络发送所述增强层数据给所述终端。
可选的，该服务器还可以包括设置模块803，用于对所述基础层数据设置第一服务质量，对所述增强层数据设置第二服务质量，所述第一服务质量高于所述第二服务质量。相应的，发送模块802还用于将所述第一服务质量的信息和所述第二服务质量的信息发送至所述移动通信网络中的设备，指示所述设备根据所述第一服务质量的信息将所述基础层数据发送给所述终端，根据所述第二服务质量的信息将所述增强层数据发送给所述终端。
可选的，设置模块803，可以用于设置所述增强层数据中目标帧数据对应的传输时间窗口，所述目标帧数据为所述增强层数据中的任一帧数据。相应的，发送模块802，在基于传输层中的不可靠传输协议，通过所述移动通信网络发送所述增强层数据给所述终端时， 具体用于在所述传输时间窗口内，基于传输层中的不可靠传输协议，通过所述移动通信网络发送所述目标帧数据给所述终端。
此外，上述各个模块还可以用于支持图2至图7所示实施例中服务器所执行的其它过程。有益效果可参考前面的描述，此处不再赘述。
基于相同的技术构思，本申请实施例还提供一种终端，用于实现上述数据分层传输方法中终端所执行的方法步骤。图9为本申请实施例提供的一种终端的结构示意图，如图所示，该终端包括处理模块901和接收模块902。
具体的，处理模块901用于，基于传输层中的可靠传输协议，通过所述接收模块902从移动通信网络接收服务器发送的基础层数据；基于传输层中的不可靠传输协议，通过所述接收模块902从移动通信网络接收服务器发送的增强层数据；其中，所述基础层数据和所述增强层数据为对媒体数据编码后得到的数据。
可选的，接收模块902还用于从所述移动通信网络接收所述服务器发送的指示信息，所述指示信息用于指示所述服务器所述目标帧数据中的一个或多个数据包不再发送。
此外，上述各个模块还可以用于支持图2至图7所示实施例中终端所执行的其它过程。有益效果可参考前面的描述，此处不再赘述。
需要说明的是，以上各个模块的划分仅仅是一种逻辑功能的划分，实际实现时可以全部或部分集成到一个物理实体上，也可以物理上分开。且这些模块可以全部以软件通过处理元件调用的形式实现；也可以全部以硬件的形式实现；还可以部分模块通过软件通过处理元件调用的形式实现，部分模块通过硬件的形式实现。各个模块可以集成在一起，也可以独立实现。这里所述的处理元件可以是一种集成电路，具有信号的处理能力。在实现过程中，上述方法的各步骤或以上各个模块可以通过处理器元件中的硬件的集成逻辑电路或者软件形式的指令完成。此外，以上发送模块是一种控制发送的模块，可以通过发送装置，例如天线和射频装置发送信息。同理，接收模块也可以通过接收装置，例如天线和射频装置接收信息。
以上这些模块可以是被配置成实施以上方法的一个或多个集成电路，例如：一个或多个特定集成电路(application specific integrated circuit，ASIC)，或，一个或多个微处理器，或，一个或者多个现场可编程门阵列(field programmable gate array，FPGA)等。再如，当以上某个模块通过处理元件调度程序的形式实现时，该处理元件可以是通用处理器，例如中央处理器(central processing unit，CPU)或其它可以调用程序的处理器。再如，这些模块可以集成在一起，以片上系统(system-on-a-chip，SOC)的形式实现。
基于相同的技术构思，本申请实施例还提供一种服务器，用于实现上述数据分层传输方法中服务器所执行的方法步骤。图10为本申请实施例提供的一种服务器的结构示意图，如图所示，该服务器包括处理器1001和通信接口1002。进一步地，该服务器还可以包括存储器1003、通信总线1004。
具体地，处理器1001可以是一个通用CPU，微处理器，ASIC，或一个或多个用于控制本申请方案程序执行的集成电路。
通信接口1002，使用任何收发器一类的装置，用于与其他设备或通信网络通信，如RAN。
通信总线1004可包括一通路，在上述组件之间传送信息。通信总线1004可以是外设部件互连标准(peripheral component interconnect，PCI)总线或扩展工业标准结构(extended  industry standard architecture，EISA)总线等。
存储器1003可以是只读存储器(read-only memory，ROM)或可存储静态信息和指令的其他类型的静态存储设备，随机存取存储器(random access memory，RAM)或者可存储信息和指令的其他类型的动态存储设备，也可以是电可擦可编程只读存储器(electrically erasable programmable read-only memory，EEPROM)或者能够用于携带或存储具有指令或数据结构形式的期望的程序代码并能够由计算机存取的任何其他介质，但不限于此。存储器1003可以是独立存在，例如片外存储器，通过通信总线1004与处理器1001相连接。存储器1003也可以和处理器1001集成在一起。
通信接口1002负责与其他设备或通信网络通信，处理器1001用于实现本申请上述实施例提供的通信方法中的其他功能。
在具体实现中，作为一种实施例，处理器1001可以包括一个或多个CPU。
具体的，处理器1001可以运行存储器1003内的指令或程序，通过通信接口1002执行以下步骤：获取对媒体数据编码后得到的基础层数据和增强层数据；基于传输层中的可靠传输协议，通过移动通信网络发送基础层数据给终端；基于传输层中的不可靠传输协议，通过移动通信网络发送增强层数据给终端。
此外，上述各个器件还可以用于支持图2至图7所示实施例中服务器所执行的其它过程。有益效果可参考前面的描述，此处不再赘述。
基于相同的技术构思，本申请实施例还提供一种终端，用于实现上述数据分层传输方法中终端所执行的方法步骤。图11为本申请实施例提供的一种终端的结构示意图，如图所示，该终端包括：处理器1101和通信接口1102。进一步地，该服务器还可以包括存储器1103、通信总线1104。
具体地，处理器1101可以是一个通用CPU，微处理器，ASIC，或一个或多个用于控制本申请方案程序执行的集成电路。
通信接口1102，使用任何收发器一类的装置，用于与其他设备或通信网络通信，如无线接入网。
通信总线1104可包括一通路，在上述组件之间传送信息。通信总线1104可以是PCI总线或EISA总线等。
存储器1103可以是ROM或可存储静态信息和指令的其他类型的静态存储设备，RAM或者可存储信息和指令的其他类型的动态存储设备，也可以是EEPROM或者能够用于携带或存储具有指令或数据结构形式的期望的程序代码并能够由计算机存取的任何其他介质，但不限于此。存储器1103可以是独立存在，例如片外存储器，通过通信总线1104与处理器1101相连接。存储器1103也可以和处理器1101集成在一起。
通信接口1102负责与其他设备或通信网络通信，处理器1101用于实现本申请上述实施例提供的通信方法中的其他功能。
在具体实现中，作为一种实施例，处理器1101可以包括一个或多个CPU。
具体的，处理器1101可以运行存储器1103内的指令或程序，通过通信接口1102执行以下步骤：基于传输层中的可靠传输协议，通过移动通信网络接收服务器发送的基础层数据；基于传输层中的不可靠传输协议，通过所述移动通信网络接收所述服务器发送的增强层数据；其中，所述基础层数据和所述增强层数据为对媒体数据编码后得到的数据。
此外，上述各个器件还可以用于支持图2至图7所示实施例中终端所执行的其它过程。 有益效果可参考前面的描述，此处不再赘述。
本申请实施例还提供一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，该计算机程序被计算机执行时，所述计算机可以实现上述方法实施例提供的图2至图2中任一所示的实施例中与服务器相关的流程，或者，所述计算机可以实现上述方法实施例提供的图2至图7中任一所示的实施例中与终端相关的流程。
本申请实施例还提供一种计算机程序产品，所述计算机程序产品用于存储计算机程序，该计算机程序被计算机执行时，所述计算机可以实现上述方法实施例提供的图2至图7中任一所示的实施例中与服务器相关的流程，或者，所述计算机可以实现上述方法实施例提供的图2至图7中任一所示的实施例中与终端相关的流程。
本申请还提供一种芯片，包括处理器。该处理器用于读取并运行存储器中存储的计算机程序，以执行本申请提供的数据分层传输方法中由服务器或终端的相应操作和/或流程。可选地，该芯片还包括存储器，该存储器与该处理器通过电路或电线与存储器连接，处理器用于读取并执行该存储器中的计算机程序。进一步可选地，该芯片还包括通信接口，处理器与该通信接口连接。通信接口用于接收处理的数据和/或信息，处理器从该通信接口获取该数据和/或信息，并对该数据和/或信息进行处理。该通信接口可以是该芯片上的输入/输出接口、接口电路、输出电路、输入电路、管脚或相关电路等。所述处理器也可以体现为处理电路或逻辑电路。
上述的芯片也可以替换为芯片系统，这里不再赘述。
另外，需要理解的是，在本申请的描述中，“第一”、“第二”等词汇，仅用于区分描述的目的，而不能理解为指示或暗示相对重要性，也不能理解为指示或暗示顺序。
在本说明书中描述的参考“一个实施例”或“一些实施例”等意味着在本申请的一个或多个实施例中包括结合该实施例描述的特定特征、结构或特点。由此，在本说明书中的不同之处出现的语句“在一个实施例中”、“在一些实施例中”、“在其他一些实施例中”、“在另外一些实施例中”等不是必然都参考相同的实施例，而是意味着“一个或多个但不是所有的实施例”，除非是以其他方式另外特别强调。术语“包括”、“包含”、“具有”及它们的变形都意味着“包括但不限于”，除非是以其他方式另外特别强调。
本申请实施例提供了一种计算机可读存储介质，存储有计算机程序，该计算机程序包括用于执行上述方法实施例的指令。
本申请实施例提供了一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行上述方法实施例。
尽管已描述了本申请的优选实施例，但本领域内的技术人员一旦得知了基本创造性概念，则可对这些实施例作出另外的变更和修改。所以，所附权利要求意欲解释为包括优选实施例以及落入本申请范围的所有变更和修改。
显然，本领域的技术人员可以对本申请实施例进行各种改动和变型而不脱离本申请实施例的精神和范围。这样，倘若本申请实施例的这些修改和变型属于本申请权利要求及其等同技术的范围之内，则本申请也意图包含这些改动和变型在内。

Claims (24)

  1. 一种数据分层传输方法，其特征在于，包括：
    获取对媒体数据编码后得到的基础层数据和增强层数据；
    基于传输层中的可靠传输协议，通过移动通信网络发送所述基础层数据给终端；
    基于传输层中的不可靠传输协议，通过所述移动通信网络发送所述增强层数据给所述终端。
  2. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    对所述基础层数据设置第一服务质量，对所述增强层数据设置第二服务质量，所述第一服务质量高于所述第二服务质量；
    将所述第一服务质量的信息和所述第二服务质量的信息发送至所述移动通信网络中的设备，指示所述设备根据所述第一服务质量的信息将所述基础层数据发送给所述终端，根据所述第二服务质量的信息将所述增强层数据发送给所述终端。
  3. 根据权利要求1或2所述的方法，其特征在于，所述可靠传输协议包括传输控制协议TCP或快速UDP网络连接QUIC协议。
  4. 根据权利要求1-3任一项所述的方法，其特征在于，所述不可靠传输协议包括用户数据报协议UDP或实时传输协议RTP。
  5. 根据权利要求1-4任一项所述的方法，其特征在于，所述方法还包括：
    设置所述增强层数据中目标帧数据对应的传输时间窗口，所述目标帧数据为所述增强层数据中的任一帧数据；
    所述基于传输层中的不可靠传输协议，通过所述移动通信网络发送所述增强层数据给所述终端，包括：
    在所述传输时间窗口内，基于传输层中的不可靠传输协议，通过所述移动通信网络发送所述目标帧数据给所述终端。
  6. 根据权利要求5所述的方法，其特征在于，所述方法还包括：
    若所述目标帧数据的一个或多个数据包未能在所述传输时间窗口内完成发送；
    通过所述移动通信网络向所述终端发送指示信息，所述指示信息用于指示所述目标帧数据中的一个或多个数据包不再发送。
  7. 根据权利要求1-6任一项所述的方法，其特征在于，所述获取对媒体数据编码后得到的基础层数据和增强层数据，包括：
    获取所述媒体数据的混合数据，所述混合数据包括多个数据包；
    对每个所述数据包进行解封装，确定所述数据包属于所述基础层数据或属于所述增强层数据。
  8. 一种数据分层传输方法，其特征在于，包括：
    基于传输层中的可靠传输协议，通过移动通信网络接收服务器发送的基础层数据；
    基于传输层中的不可靠传输协议，通过所述移动通信网络接收所述服务器发送的增强层数据；
    其中，所述基础层数据和所述增强层数据为对媒体数据编码后得到的数据。
  9. 根据权利要求8所述的方法，其特征在于，所述可靠传输协议包括传输控制协议TCP或快速UDP网络连接QUIC协议。
  10. 根据权利要求8或9所述的方法，其特征在于，所述不可靠传输协议包括用户数据报协议UDP或实时传输协议RTP。
  11. 根据权利要求8-10任一项所述的方法，其特征在于，所述方法还包括：
    通过所述移动通信网络接收所述服务器发送的指示信息，所述指示信息用于指示所述服务器对一个目标帧数据中的一个或多个数据包不再发送；
    所述指示信息为所述服务器在所述目标帧数据中的一个或多个数据包，未能在为所述目标帧设置的传输时间窗口内发送时发送的。
  12. 一种服务器，其特征在于，包括：处理器，以及分别与所述处理器耦合的存储器和通信接口；
    所述通信接口，用于与其他设备进行通信；
    所述处理器，用于运行所述存储器内的指令或程序，通过所述通信接口执行以下步骤：
    获取对媒体数据编码后得到的基础层数据和增强层数据；
    基于传输层中的可靠传输协议，通过移动通信网络发送所述基础层数据给终端；
    基于传输层中的不可靠传输协议，通过所述移动通信网络发送所述增强层数据给所述终端。
  13. 根据权利要求12所述的服务器，其特征在于，所述处理还用于：
    对所述基础层数据设置第一服务质量，对所述增强层数据设置第二服务质量，所述第一服务质量高于所述第二服务质量；
    将所述第一服务质量的信息和所述第二服务质量的信息发送至所述移动通信网络中的设备，指示所述设备根据所述第一服务质量的信息将所述基础层数据发送给所述终端，根据所述第二服务质量的信息将所述增强层数据发送给所述终端。
  14. 根据权利要求12或13所述的服务器，其特征在于，所述可靠传输协议包括传输控制协议TCP或快速UDP网络连接QUIC协议。
  15. 根据权利要求12-14任一项所述的服务器，其特征在于，所述不可靠传输协议包括用户数据报协议UDP或实时传输协议RTP。
  16. 根据权利要求12-15任一项所述的服务器，其特征在于，所述处理器还用于：
    设置所述增强层数据中目标帧数据对应的传输时间窗口，所述目标帧数据为所述增强层数据中的任一帧数据；
    所述处理器，在基于传输层中的不可靠传输协议，通过所述移动通信网络发送所述增强层数据给所述终端时，具体用于：
    在所述传输时间窗口内，基于传输层中的不可靠传输协议，通过所述移动通信网络发送所述目标帧数据给所述终端。
  17. 根据权利要求16所述的服务器，其特征在于，所述处理器还用于：
    若所述目标帧数据的一个或多个数据包未能在所述传输时间窗口内完成发送；
    通过所述移动通信网络向所述终端发送指示信息，所述指示信息用于指示所述目标帧数据中的一个或多个数据包不再发送。
  18. 根据权利要求12-17任一项所述的服务器，其特征在于，所述处理器，在获取对媒体数据编码后得到的基础层数据和增强层数据时，具体用于：
    获取所述媒体数据的混合数据，所述混合数据包括多个数据包；
    对每个所述数据包进行解封装，确定所述数据包属于所述基础层数据或属于所述增强 层数据。
  19. 一种终端，其特征在于，包括：处理器，以及分别与所述处理器耦合的存储器和通信接口；
    所述通信接口，用于与其他设备进行通信；
    所述处理器，用于运行所述存储器内的指令或程序，通过所述通信接口执行以下步骤：
    基于传输层中的可靠传输协议，通过移动通信网络接收服务器发送的基础层数据；基于传输层中的不可靠传输协议，通过所述移动通信网络接收所述服务器发送的增强层数据；
    其中，所述基础层数据和所述增强层数据为对媒体数据编码后得到的数据。
  20. 根据权利要求19所述的终端，其特征在于，所述可靠传输协议包括传输控制协议TCP或快速UDP网络连接QUIC协议。
  21. 根据权利要求19或20所述的终端，其特征在于，所述不可靠传输协议包括用户数据报协议UDP或实时传输协议RTP。
  22. 根据权利要求19-21任一项所述的终端，其特征在于，所述处理器还用于：
    通过所述移动通信网络接收所述服务器发送的指示信息，所述指示信息用于指示所述服务器对一个所述目标帧数据中的一个或多个数据包不再发送；
    所述指示信息为所述服务器在所述目标帧数据中的一个或多个数据包，未能在为所述目标帧设置的传输时间窗口内发送时发送的。
  23. 一种数据分层传输系统，其特征在于，包括：如权利要求12至18任一所述的服务器和如权利要求19至22任一所述的终端。
  24. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质中存储有指令，当所述指令在计算机上运行时，使得所述计算机执行如权利要求1-11任一项所述的方法。
PCT/CN2021/137079 2021-03-31 2021-12-10 一种数据分层传输方法、装置及系统 WO2022206016A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110348713.7 2021-03-31
CN202110348713.7A CN115150367A (zh) 2021-03-31 2021-03-31 一种数据分层传输方法、装置及系统

Publications (1)

Publication Number Publication Date
WO2022206016A1 true WO2022206016A1 (zh) 2022-10-06

Family

ID=83403447

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/137079 WO2022206016A1 (zh) 2021-03-31 2021-12-10 一种数据分层传输方法、装置及系统

Country Status (2)

Country Link
CN (1) CN115150367A (zh)
WO (1) WO2022206016A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116634177B (zh) * 2023-06-16 2024-02-20 北京行者无疆科技有限公司 一种基于hdmi通信设备的视频通信解码处理方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1742476A1 (en) * 2005-07-06 2007-01-10 Thomson Licensing Scalable video coding streaming system and transmission mechanism of the same system
CN101563929A (zh) * 2006-12-22 2009-10-21 高通股份有限公司 基础层与增强层之间的多媒体数据重组
CN104618337A (zh) * 2014-12-31 2015-05-13 浙江理工大学 一种基于tcp/udp混合协议的流媒体无线自适应传输方法
CN109150876A (zh) * 2018-08-20 2019-01-04 深圳市昊源科技有限公司 一种视频无线传输的qos方法、装置及系统
CN111147606A (zh) * 2020-01-06 2020-05-12 北京字节跳动网络技术有限公司 数据传输的方法、装置、终端及存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1742476A1 (en) * 2005-07-06 2007-01-10 Thomson Licensing Scalable video coding streaming system and transmission mechanism of the same system
CN101563929A (zh) * 2006-12-22 2009-10-21 高通股份有限公司 基础层与增强层之间的多媒体数据重组
CN104618337A (zh) * 2014-12-31 2015-05-13 浙江理工大学 一种基于tcp/udp混合协议的流媒体无线自适应传输方法
CN109150876A (zh) * 2018-08-20 2019-01-04 深圳市昊源科技有限公司 一种视频无线传输的qos方法、装置及系统
CN111147606A (zh) * 2020-01-06 2020-05-12 北京字节跳动网络技术有限公司 数据传输的方法、装置、终端及存储介质

Also Published As

Publication number Publication date
CN115150367A (zh) 2022-10-04

Similar Documents

Publication Publication Date Title
WO2021047419A1 (zh) 直播交互方法、直播系统、电子设备及存储介质
US20230275698A1 (en) Method and communications apparatus for transmitting data packets of a media stream
WO2013113281A1 (zh) 传输多媒体数据的方法、装置及系统
WO2014183478A1 (en) Video encoding method and apparatus
US20220369000A1 (en) Split rendering of extended reality data over 5g networks
WO2022206016A1 (zh) 一种数据分层传输方法、装置及系统
CN114221909B (zh) 数据传输方法、装置、终端及存储介质
US9936266B2 (en) Video encoding method and apparatus
WO2022022471A1 (zh) 一种媒体流切换方法及装置
WO2022052102A1 (zh) 一种通信方法及装置
WO2024041239A1 (zh) 一种沉浸媒体的数据处理方法、装置、设备、存储介质及程序产品
WO2010108416A1 (zh) 转发可伸缩视频编码数据报文的方法、设备和通信系统
CN114710568B (zh) 音视频数据通信方法、设备及存储介质
CN111641878A (zh) 三维模型展示方法、装置和电子设备
KR101710011B1 (ko) 영상 데이터 전송 및 수신 방법 및 장치
CN1996813B (zh) 用于区域/广域网络环境中连续媒体流的自适应媒体传输管理
WO2022116822A1 (zh) 沉浸式媒体的数据处理方法、装置和计算机可读存储介质
EP4338424A1 (en) Split rendering of extended reality data over 5g networks
KR20220033445A (ko) 몰입형 미디어를 처리하기 위한 방법 및 장치
WO2021134346A1 (zh) 一种反馈方法及装置
WO2022126437A1 (zh) 通信方法及装置
WO2024060719A1 (zh) 一种数据传输的方法、装置、电子设备及存储介质
WO2023246752A1 (zh) 通信方法和通信装置
WO2024060991A1 (zh) 一种多路径的数据流引流方法及装置
WO2023155633A1 (zh) 一种数据传输方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21934652

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21934652

Country of ref document: EP

Kind code of ref document: A1