WO2023071605A1 - 投屏数据传输方法、装置、电子设备及存储介质 - Google Patents

投屏数据传输方法、装置、电子设备及存储介质 Download PDF

Info

Publication number
WO2023071605A1
WO2023071605A1 PCT/CN2022/119533 CN2022119533W WO2023071605A1 WO 2023071605 A1 WO2023071605 A1 WO 2023071605A1 CN 2022119533 W CN2022119533 W CN 2022119533W WO 2023071605 A1 WO2023071605 A1 WO 2023071605A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
projection data
screen projection
data transmission
encoding
Prior art date
Application number
PCT/CN2022/119533
Other languages
English (en)
French (fr)
Inventor
余振华
沈显超
Original Assignee
北京字节跳动网络技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京字节跳动网络技术有限公司 filed Critical 北京字节跳动网络技术有限公司
Priority to EP22885492.3A priority Critical patent/EP4284004A1/en
Publication of WO2023071605A1 publication Critical patent/WO2023071605A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H04N21/4363—Adapting the video or multiplex stream to a specific local network, e.g. a IEEE 1394 or Bluetooth® network
    • H04N21/43637—Adapting the video or multiplex stream to a specific local network, e.g. a IEEE 1394 or Bluetooth® network involving a wireless protocol, e.g. Bluetooth, RF or wireless LAN [IEEE 802.11]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440263—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the spatial resolution, e.g. for displaying on a connected PDA
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440281—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the temporal resolution, e.g. by frame skipping
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H04N21/44227—Monitoring of local network, e.g. connection or bandwidth variations; Detecting new devices in the local network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/462—Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
    • H04N21/4621—Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/643—Communication protocols
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2350/00—Solving problems of bandwidth in display systems

Definitions

  • the present disclosure relates to the technical field of computers, for example, to a screen projection data transmission method, device, electronic equipment, and storage medium.
  • the screen projection data transmission between the screen projection data source end and the video playback end is all data transmission under the wireless local area network.
  • the process of data transmission there will be delays, jitters, packet loss and other instability caused by the network, which will cause video playback freezes, and even screen projection interruptions, giving users a very bad experience.
  • a data transmission method based on User Datagram Protocol is usually used.
  • UDP User Datagram Protocol
  • application developers will give priority to data transmission based on Transmission Control Protocol (TCP), which can perform data transmission more stably and obtain greater investment in equipment operating expenses. output ratio.
  • TCP-based data transmission cannot perceive the network status based on the traditional packet loss rate and retransmission rate, and control the data transmission process of screen projection to improve the video viewing experience of screen projection users.
  • the present disclosure provides a screen projection data transmission method, device, electronic equipment, and storage medium capable of improving a user's screen projection video viewing experience.
  • the present disclosure provides a projection data transmission method, including:
  • Adjust the encoding parameters of the projection data according to the adjustment strategy and encode the projection data to be sent according to the adjusted encoding parameters, and complete the data transmission.
  • the present disclosure also provides a screen projection data transmission device, including:
  • the data transmission monitoring module is configured to receive the feedback information sent by the target data receiving end according to the received projection data, and determine the projection according to the information content of the preset data transmission field in the TCP header information in the feedback information.
  • Data round-trip time of screen data
  • the data encoding control module is configured to estimate the network bandwidth according to the numerical change of the round-trip time of the data, and determine the adjustment strategy of the projection data encoding parameters based on the network bandwidth estimation result and the current data transmission bandwidth;
  • the data encoding and transmission module is configured to adjust the screen projection data encoding parameters according to the adjustment strategy, encode the screen projection data to be sent according to the adjusted encoding parameters, and complete the data transmission.
  • the present disclosure also provides an electronic device, the electronic device comprising:
  • processors one or more processors
  • a storage device configured to store one or more programs
  • the one or more processors are made to implement the above screen projection data transmission method.
  • the present disclosure also provides a storage medium containing computer-executable instructions, and the computer-executable instructions are used to execute the above-mentioned screen projection data transmission method when executed by a computer processor.
  • FIG. 1 is a schematic flowchart of a screen projection data transmission method provided in Embodiment 1 of the present disclosure
  • FIG. 2 is a schematic flow diagram of a projection data transmission method provided in Embodiment 2 of the present disclosure
  • FIG. 3 is a schematic diagram of a projection data transmission process provided by Embodiment 2 of the present disclosure.
  • FIG. 4 is a schematic structural diagram of a screen projection data transmission device provided by Embodiment 3 of the present disclosure.
  • FIG. 5 is a schematic structural diagram of an electronic device provided by Embodiment 4 of the present disclosure.
  • the term “comprise” and its variations are open-ended, ie “including but not limited to”.
  • the term “based on” is “based at least in part on”.
  • the term “one embodiment” means “at least one embodiment”; the term “another embodiment” means “at least one further embodiment”; the term “some embodiments” means “at least some embodiments.” Relevant definitions of other terms will be given in the description below.
  • FIG. 1 is a schematic flowchart of a screen projection data transmission method provided by Embodiment 1 of the present disclosure.
  • the embodiment of the present disclosure is applicable to the situation of monitoring the transmission process of screen projection data.
  • the method can be executed by a screen projection data transmission device, which can be implemented in the form of software and/or hardware, and the device can be configured in an electronic device, such as a mobile terminal or a server device.
  • the screen projection data transmission method provided in this embodiment includes:
  • the target data receiving end is the projection data display device.
  • the target data receiving end and the screen projection source end will establish a communication channel based on a preset communication protocol, and receive the screen projection data sent by the screen projection source end through the communication channel.
  • the communication protocol between the screen projection source and the target data receiver is TCP.
  • the target data receiver After receiving the screen projection data, the target data receiver will also send feedback information to inform that the screen projection source has successfully received the corresponding projection data. screen data.
  • the data packet is composed of a header and data.
  • the header contains communication control information such as the data sending destination, and the data is the data content that the data sender wants to send to the data receiver.
  • a preset data transmission field is set in the TCP header information, and this field can be named as a payload (PAYLOAD) field, and more information of the screen projection data to be sent can be displayed in this field.
  • the 0th byte represents the video data
  • the 1st byte represents the audio data
  • the 2nd byte represents the first count value sent by the projection data
  • the 3rd byte represents the projection Screen data receives the second count value fed back.
  • the first count value and the second count value are used to determine the data round-trip time (that is, data round-trip time delay (Round-Trip Time, RTT)) of the sent screen projection data.
  • the data packet count value When sending data at the projection source, the data packet count value will be written at the position of the first count value of the projection data sent in the preset data transmission field.
  • the count value usually starts from 1, and the second count value field is null.
  • the target data receiving end After receiving the data packet from the projection source, the target data receiving end will write the same value as the first count value in the second count value position of the preset data transmission field when sending the confirmation feedback information.
  • the projection source end When the projection source end receives the feedback information sent by the target data receiving end based on the received projection data, it can analyze the information in the preset data transmission field in the TCP header of the feedback information, and identify the information in the preset data transmission field Indicates the first count value of screen projection data transmission and the second count value of screen projection data reception feedback; when the first count value and the second count value are the same, according to the transmission time of screen projection data and the reception of feedback information Time, to determine the data round-trip time of the projection data.
  • Network bandwidth refers to the maximum data rate that can pass from the projection source to the target data receiver within a unit of time. Forecasting the network bandwidth may be to determine the change trend of the network bandwidth in the next time. Network bandwidth and data round-trip time have an inverse trend. When data round-trip times become longer, network bandwidth decreases. The more the data round-trip time becomes longer, the more the network bandwidth is reduced. Therefore, the projection data encoding parameters can be adjusted according to the estimated change trend of the network bandwidth to reduce the occupation of the network bandwidth by the data to be sent, thereby reducing the lag in the display of the projection data.
  • the current data transmission bandwidth refers to the data transmission bandwidth when sending the projection data corresponding to the received feedback information.
  • Screen data transmission duration then, determine the current data transmission bandwidth according to the screen projection data transmission duration and the data volume of the projection data, for example, the data volume of the projection data can be divided by the projection data transmission duration to determine the current Data sending bandwidth.
  • the screen projection data encoding parameters that can be controlled include data encoding bit rate, frame rate, and encoding resolution, etc.
  • the code rate refers to the size of data encoded by the encoder per second, and the unit is kbps.
  • 800kbps means that the encoder generates 800kb (or 100KB) of data per second.
  • Frame rate refers to the number of image frames played per second.
  • Coding resolution refers to the number of pixels contained in an image per inch.
  • the adjustment strategy of the projection data encoding parameters is a different parameter adjustment method formulated according to the estimated change degree of the network bandwidth.
  • the priority of parameter adjustment can be set, when the estimated reduction in network bandwidth is small , give priority to adjusting the data encoding bit rate; when the estimated decrease in network bandwidth is large, the encoding bit rate and resolution can be adjusted at the same time, so as to achieve the effect of reducing bandwidth occupation and reduce the stuttering of the screen-casting video.
  • a correlation function relationship between the parameter value and the change value of the network bandwidth may be established in advance, and the adjusted value of the parameter is determined according to the functional relationship.
  • the premise of adopting the above adjustment strategy is that the current data transmission bandwidth is greater than or equal to the unadjusted current data coding rate.
  • the encoding of the projection data to be sent will be suspended directly until all the data in the sending data buffer queue is sent and the network bandwidth returns to normal. This is because In this network state, even after adjusting the data encoding parameters, the network bandwidth requirement cannot be met.
  • the adjusted new data encoding parameters are used to encode the unencoded screen projection data, and the encoded data is sent to the target data receiving end.
  • corresponding information will be written in the preset data transmission field in the TCP header information, including the first count value used to calculate the RTT indicating data sending.
  • the screen projection source In order for the screen projection source to continuously observe the RTT value based on the received feedback information, and then estimate the network bandwidth, adjust the screen projection data encoding parameters in a timely manner, so that the screen projection caused by uncontrollable sudden instability of the network The problem can be solved quickly, reducing the situation of screen projection stuttering or even interruption of projection screen, and improving user experience.
  • the information content of the preset data transmission field determines the data round-trip time of each projection data packet; then, the network bandwidth is estimated according to the numerical change of the data round-trip time, and the projection is determined based on the network bandwidth estimation result and the current data transmission bandwidth Adjustment strategy for data encoding parameters; adjust the projection data encoding parameters according to the determined adjustment strategy, and encode the projection data to be sent according to the adjusted encoding parameters, and complete the data transmission.
  • the technical solution of the embodiment of the present disclosure solves the problem that the network status cannot be directly monitored in the process of data transmission based on TCP to control the transmission process of screen projection data, and realizes the monitoring of the network status between the two parties performing screen projection data interaction based on TCP.
  • the network status adjusts the projection data encoding strategy and transmits the projection data, reducing video freezes and pauses caused by network problems during the projection process, and improving the user's projection video viewing experience.
  • Embodiments of the present disclosure may be combined with multiple solutions in the screen projection data transmission method provided in the above embodiments.
  • the screen projection data transmission method provided in this embodiment describes the complete process of screen projection data transmission and a strategy for adjusting screen projection parameters.
  • FIG. 2 is a schematic flowchart of a screen projection data transmission method provided by Embodiment 2 of the present disclosure. As shown in Figure 2, the screen projection data transmission method provided by this embodiment includes:
  • the display content in Figure 3 is the content to be displayed by the projection screen display device. After the display content is encoded by the encoder, it will be packaged by the sending unit, and the communication control information such as the sending destination will be added to the data packet as the TCP header. information to form the screen projection data sent by the screen projection source.
  • the sending unit of the screen projection source establishes a network connection with the target data receiving end through the network and TCP, and sends the screen projection data to the target data receiving end.
  • a preset data transmission field is set in the TCP header information, and the information in the field is used for reference to determine the RTT of the projection data.
  • the preset data transmission field there are two bytes, one byte is used to record the first count value sent by the projection data, and one byte is used to indicate the second count value of the feedback received by the projection data .
  • the first count value will be written into certain count information when the sending unit performs data packing, and the second count value is empty.
  • S220 Receive the feedback information sent by the receiving end of the target data according to the received screen projection data, and determine the information content of the screen projection data according to the information content of the preset data transmission field in the TCP header information in the feedback information Data round trip time.
  • the target data receiving end After receiving the screen projection data, the target data receiving end will also send feedback information based on the network channel established with the screen projection source, informing the screen projection source that the corresponding screen projection data has been successfully received.
  • the target data receiving end There is the same preset data transmission field in the TCP header information of the feedback information, and the second count value information will be written in according to the first count value of the TCP header information of the projection data.
  • the first count value and the second count value are generally the same value, counting from 1.
  • the screen projection source can detect the network status of the screen projection source and the target data receiving end according to the received feedback information.
  • the screen projection source when the screen projection source receives the feedback information sent by the target data receiver based on the received screen projection data, it can analyze the information in the preset data transmission field in the TCP header of the feedback information, and identify the preset data transmission field. Assume that the first count value in the data transmission field represents the transmission of the projection data and the second count value represents the feedback of the reception of the projection data; when the first count value and the second count value are the same, according to the transmission time of the projection data and The receiving time of receiving the feedback information determines the data round-trip time of the projection data.
  • the screen projection source detects the network status, it adopts the corresponding adjustment strategy of the screen projection data encoding parameters to perform encoder control on the encoder.
  • Network bandwidth and data round-trip time have an inverse trend.
  • network bandwidth decreases.
  • the RTT value analyzed by the source end of the screen projection is a relatively fixed and stable value.
  • the RTT value is within 5ms.
  • the adjustment strategy of the projection data encoding parameters can be different parameter adjustment schemes specified according to the estimated change trend of the network bandwidth, that is, the change degree of the corresponding RTT value.
  • the settings of multiple adjustment strategies are set on the premise of not affecting the subjective clarity experience and stuttering experience of the user's limit.
  • the change of the RTT value is directly used to represent the change of the network bandwidth.
  • the data encoding rate adjustment is prioritized, and the resolution and frame rate are not adjusted temporarily. value. For example, if the projection data resolution is 1920x1080, and the upstream bandwidth (current data transmission bandwidth) is 12Mbps, when the RTT value increases to twice the stable value during the projection process, the data encoding rate of the control encoder will drop to Half of the original transmission bandwidth, that is, 6Mbps, the resolution and frame rate will not be adjusted.
  • the resolution is reduced to 1280x720, the frame rate remains unchanged, and the bit rate is adjusted to 5Mbps.
  • the coding of the display content is suspended until the sending buffer queue of the sending unit is finished sending.
  • the actual sending bit rate is the current unadjusted data encoding bit rate
  • the preset bit rate is a target value to be adjusted for the data encoding bit rate.
  • the data encoding parameters will not be adjusted.
  • each parameter value of the encoding parameter will be adjusted to the preset value. The parameter value corresponding to the code rate.
  • the technical solutions of the embodiments of the present disclosure describe the complete screen projection data transmission process on the basis of the above embodiments.
  • the data round-trip time of each projection data packet is determined according to the information content of the preset data transmission field in the TCP header information in the feedback information; furthermore, the network bandwidth is pre-predicted according to the numerical change of the data round-trip time
  • the adjustment strategy of the screen projection data encoding parameters is determined, and the adjustment strategy is given as an example; the projection data encoding parameters are adjusted according to the determined adjustment strategy, and according to the adjusted
  • the encoding parameter encodes the projection data to be sent and completes the data transmission.
  • the technical solution of the embodiment of the present disclosure solves the problem that the network status cannot be directly monitored in the process of data transmission based on TCP to control the transmission process of screen projection data, and realizes the monitoring of the network status between the two parties performing screen projection data interaction based on TCP.
  • the network status adjusts the projection data encoding strategy and transmits the projection data, reducing video freezes and pauses caused by network problems during the projection process, and improving the user's projection video viewing experience.
  • FIG. 4 is a schematic structural diagram of a screen projection data transmission device provided by Embodiment 3 of the present disclosure.
  • the screen projection data transmission device provided in this embodiment is suitable for monitoring the transmission process of screen projection data.
  • the screen projection data transmission device includes: a data transmission monitoring module 310 , a data encoding control module 320 and a data encoding and transmission module 330 .
  • the data transmission monitoring module 310 is configured to receive the feedback information sent by the target data receiving end according to the received projection data, and determine the information content of the preset data transmission field in the TCP header information in the feedback information.
  • the data round-trip time of the screen projection data; the data encoding control module 320 is configured to estimate the network bandwidth according to the numerical change of the data round-trip time, and determine the encoding parameters of the screen projection data based on the network bandwidth estimation result and the current data transmission bandwidth Adjustment strategy: the data encoding and transmission module 330 is configured to adjust the encoding parameters of the projection data according to the adjustment strategy, encode the projection data to be sent according to the adjusted encoding parameters, and complete the data transmission.
  • the technical solution of the embodiment of the present disclosure solves the problem that the network status cannot be directly monitored in the process of data transmission based on TCP to control the transmission process of screen projection data, and realizes the monitoring of the network status between the two parties performing screen projection data interaction based on TCP.
  • the network status adjusts the projection data encoding strategy and transmits the projection data, reducing video freezes and pauses caused by network problems during the projection process, and improving the user's projection video viewing experience.
  • the data encoding and transmission module 330 is further configured to add information corresponding to the preset data transmission field in the TCP header information of the projection data before receiving the feedback information content, and send the screen projection data to the target data receiving end according to TCP.
  • the data transmission monitoring module 310 is set to:
  • the receiving time of the feedback information is determined to determine the data round-trip time of the projection data.
  • the data transmission monitoring module 310 can also be set to:
  • timing the time for calling the data sending interface and sending the screen projection data and determining the transmission duration of the projection data; according to the transmission duration of the projection data and the time of the projection data
  • the amount of data determines the current data sending bandwidth.
  • the data encoding control module 320 is set to:
  • the screen projection data encoding parameters include data encoding code rate, resolution, and frame rate, and in the preset screen projection data encoding parameter adjustment strategy, the priority of adjusting the data encoding code rate is high prior to adjusting the resolution and/or the frame rate.
  • the data encoding and transmission module 330 is also configured to:
  • the encoding of the screen projection data to be sent is suspended until the encoded projection screen data in the screen projection data transmission buffer queue is completely sent.
  • the screen projection data transmission device provided in the embodiments of the present disclosure can execute the screen projection data transmission method provided in any embodiment of the present disclosure, and has corresponding functional modules and effects for executing the method.
  • the multiple units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be realized; in addition, the names of multiple functional units are only for the convenience of distinguishing each other , and are not intended to limit the protection scope of the embodiments of the present disclosure.
  • FIG. 5 it shows a schematic structural diagram of an electronic device (such as the terminal device or server in FIG. 5 ) 400 suitable for implementing the embodiments of the present disclosure.
  • the terminal device 400 in the embodiment of the present disclosure may include but not limited to mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (Personal Digital Assistant, PDA), tablet computers (Portable Android Device, PAD), portable multimedia player Mobile terminals such as Portable Media Player (PMP), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital television (Television, TV), desktop computers, etc.
  • PDA Personal Digital Assistant
  • PAD Portable Multimedia Player
  • vehicle-mounted terminals such as vehicle-mounted navigation terminals
  • fixed terminals such as digital television (Television, TV), desktop computers, etc.
  • the electronic device 400 shown in FIG. 5 is only an example, and should not limit the functions and scope of use of the embodiments of the present disclosure.
  • an electronic device 400 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 408 is loaded into a program in a random access memory (Random Access Memory, RAM) 403 to execute various appropriate actions and processes.
  • a processing device such as a central processing unit, a graphics processing unit, etc.
  • RAM Random Access Memory
  • various programs and data necessary for the operation of the electronic device 400 are also stored.
  • the processing device 401, the ROM 402, and the RAM 403 are connected to each other through a bus 404.
  • An input/output (Input/Output, I/O) interface 405 is also connected to the bus 404 .
  • an input device 406 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; including, for example, a liquid crystal display (Liquid Crystal Display, LCD) , an output device 407 such as a speaker, a vibrator, etc.; a storage device 408 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 409.
  • the communication means 409 may allow the electronic device 400 to perform wireless or wired communication with other devices to exchange data.
  • FIG. 5 shows electronic device 400 having various means, it is not a requirement to implement or possess all of the means shown. More or fewer means may alternatively be implemented or provided.
  • embodiments of the present disclosure include a computer program product, which includes a computer program carried on a non-transitory computer readable medium, where the computer program includes program code for executing the method shown in the flowchart.
  • the computer program may be downloaded and installed from a network via communication means 409, or from storage means 408, or from ROM 402.
  • the processing device 401 When the computer program is executed by the processing device 401, the above-mentioned functions defined in the projection data transmission method of the embodiment of the present disclosure are executed.
  • the electronic device provided by the embodiment of the present disclosure belongs to the same concept as the screen projection data transmission method provided by the above embodiment.
  • An embodiment of the present disclosure provides a computer storage medium, on which a computer program is stored, and when the program is executed by a processor, the screen projection data transmission method provided in the foregoing embodiments is implemented.
  • the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two.
  • a computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof.
  • Examples of computer readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, RAM, ROM, Erasable Programmable Read-Only Memory (EPROM) ) or flash memory (FLASH), optical fiber, portable compact disk read-only memory (Compact Disc Read-Only Memory, CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
  • a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave carrying computer-readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • a computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can transmit, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device .
  • the program code contained on the computer readable medium can be transmitted by any appropriate medium, including but not limited to: electric wire, optical cable, radio frequency (Radio Frequency, RF), etc., or any suitable combination of the above.
  • the client and the server can communicate using any currently known or future-developed network protocols such as Hyper Text Transfer Protocol (Hyper Text Transfer Protocol, HTTP), and can communicate with any form or medium of digital Data communication (eg, communication network) interconnections.
  • Examples of communication networks include local area networks (Local Area Network, LAN), wide area networks (Wide Area Network, WAN), internetworks (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently existing networks that are known or developed in the future.
  • the above-mentioned computer-readable medium may be included in the above-mentioned electronic device, or may exist independently without being incorporated into the electronic device.
  • the above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the electronic device, the electronic device: receives the feedback information sent by the target data receiving end according to the received projection data, and According to the information content of the preset data transmission field in the TCP header information in the feedback information, the data round-trip time of the projection data is determined; the network bandwidth is estimated according to the numerical change of the data round-trip time, and based on the network The bandwidth estimation result and the current data transmission bandwidth determine the adjustment strategy of the screen projection data coding parameters; adjust the screen projection data coding parameters according to the adjustment strategy, and encode the screen projection data to be sent according to the adjusted coding parameters, and complete data transmission.
  • Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, or combinations thereof, including but not limited to object-oriented programming languages—such as Java, Smalltalk, C++, and Includes conventional procedural programming languages - such as the "C" language or similar programming languages.
  • the program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server.
  • the remote computer can be connected to the user computer through any kind of network, including a LAN or WAN, or it can be connected to an external computer (eg via the Internet using an Internet Service Provider).
  • each block in a flowchart or block diagram may represent a module, program segment, or portion of code that contains one or more logical functions for implementing specified executable instructions.
  • the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved.
  • each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations can be implemented by a dedicated hardware-based system that performs the specified functions or operations , or may be implemented by a combination of dedicated hardware and computer instructions.
  • the units involved in the embodiments described in the present disclosure may be implemented by software or by hardware. Among them, the names of units and modules do not constitute limitations on the units and modules themselves in one case.
  • the data encoding and transmission module can also be described as "adjusting the projection data encoding parameters according to the adjustment strategy.” , and encode the projection data to be sent according to the adjusted encoding parameters, and complete the module of data transmission".
  • exemplary types of hardware logic components include: Field Programmable Gate Arrays (Field Programmable Gate Arrays, FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (Application Specific Standard Parts, ASSP), System on Chip (System on Chip, SOC), Complex Programmable Logic Device (CPLD), etc.
  • a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device.
  • a machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.
  • a machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Examples of machine-readable storage media would include one or more wire-based electrical connections, portable computer disks, hard drives, RAM, ROM, EPROM or flash memory, optical fibers, CD-ROMs, optical storage devices, magnetic storage devices, or Any suitable combination of the above.
  • Example 1 provides a screen projection data transmission method, the method including:
  • Adjust the encoding parameters of the projection data according to the adjustment strategy and encode the projection data to be sent according to the adjusted encoding parameters, and complete the data transmission.
  • Example 2 provides a screen projection data transmission method, which further includes:
  • the method before receiving the feedback information, the method further includes:
  • TCP header information of the screen projection data add the information content corresponding to the preset data transmission field, and send the screen projection data to the target data receiving end according to TCP.
  • Example 3 provides a screen projection data transmission method, which further includes:
  • the determining the data round-trip time of the projection data according to the information content of the preset data transmission field in the TCP header information in the feedback information includes:
  • Example 4 provides a screen projection data transmission method, which further includes:
  • the method before determining the adjustment strategy for the encoding parameters of the projection data, the method further includes:
  • the current data transmission bandwidth is determined according to the screen projection data transmission duration and the data volume of the screen projection data.
  • Example 5 provides a screen projection data transmission method, which further includes:
  • the network bandwidth estimation is performed according to the numerical change of the data round-trip time, and the adjustment strategy of the projection data encoding parameters is determined based on the network bandwidth estimation result and the current data transmission bandwidth, including:
  • Example 6 provides a screen projection data transmission method, which further includes:
  • the screen projection data encoding parameters include data encoding code rate, resolution, and frame rate, and in the preset screen projection data encoding parameter adjustment strategy, the priority of adjusting the data encoding code rate is high prior to adjusting the resolution and/or the frame rate.
  • Example 7 provides a screen projection data transmission method, which further includes:
  • the encoding of the screen projection data to be sent is suspended until the encoded screen projection data in the screen projection data sending buffer queue is completely sent.
  • Example 8 provides a screen projection data transmission device, including:
  • the data transmission monitoring module is configured to receive the feedback information sent by the target data receiving end according to the received projection data, and determine the projection according to the information content of the preset data transmission field in the TCP header information in the feedback information.
  • Data round-trip time of screen data
  • the data encoding control module is configured to estimate the network bandwidth according to the numerical change of the round-trip time of the data, and determine the adjustment strategy of the projection data encoding parameters based on the network bandwidth estimation result and the current data transmission bandwidth;
  • the data encoding and transmission module is configured to adjust the screen projection data encoding parameters according to the adjustment strategy, encode the screen projection data to be sent according to the adjusted encoding parameters, and complete the data transmission.
  • Example 9 provides a screen projection data transmission device, which further includes:
  • the data encoding and transmission module is further configured to add the information content corresponding to the preset data transmission field in the TCP header information of the projection data before receiving the feedback information , and send the screen projection data to the target data receiving end according to TCP.
  • Example 10 provides a screen projection data transmission device, which further includes:
  • the data transmission monitoring module is set to:
  • the data round trip time of the screen projection data is determined according to the sending time of the screen projection data and the receiving time of receiving the feedback information.
  • Example Eleven provides a screen projection data transmission device, which further includes:
  • the data transmission monitoring module can also be set to:
  • the current data transmission bandwidth is determined according to the screen projection data transmission duration and the data volume of the screen projection data.
  • Example 12 provides a screen projection data transmission device, further comprising:
  • the data encoding control module is set to:
  • Example 13 provides a screen projection data transmission device, further comprising:
  • the screen projection data encoding parameters include data encoding code rate, resolution, and frame rate, and in the preset screen projection data encoding parameter adjustment strategy, the priority of adjusting the data encoding code rate is high prior to adjusting the resolution and/or the frame rate.
  • Example Fourteen provides a screen projection data transmission device, further comprising:
  • the data encoding and transmission module is also set to:
  • the encoding of the screen projection data to be sent is suspended until the encoded projection screen data in the screen projection data transmission buffer queue is completely sent.

Abstract

本文公开了一种投屏数据传输方法、装置、电子设备及存储介质。投屏数据传输方法包括：接收目标数据接收端根据接收到的投屏数据发出的反馈信息，根据反馈信息中的TCP头部信息中的预设数据传输字段的信息内容确定投屏数据的数据往返时间；根据数据往返时间的数值变化进行网络带宽预估，基于网络带宽预估结果和当前数据发送带宽确定投屏数据编码参数的调整策略；调整投屏数据编码参数，并根据调整后的编码参数对待发送投屏数据进行编码，并传输。

Description

投屏数据传输方法、装置、电子设备及存储介质
本申请要求在2021年10月25日提交中国专利局、申请号为202111242981.7的中国专利申请的优先权，该申请的全部内容通过引用结合在本申请中。
技术领域
本公开涉及计算机技术领域，例如涉及一种投屏数据传输方法、装置、电子设备及存储介质。
背景技术
通常，在投屏数据源端和视频播放端之间投屏数据的传输均是在无线局域网下的数据传输。在数据传输的过程中会碰到网络带来的延迟、抖动、丢包等不稳定情况，会造成视频播放卡顿，甚至投屏中断，给用户的体验是非常糟糕的。
相关技术中，为了能够动态监控网络状态，以控制投屏数据传输的过程，通常会采用基于用户数据报协议(User Datagram Protocol，UDP)的数据传输方式。但是，从应用开发者的角度来说，应用开发者会优先选择基于传输控制协议(Transmission Control Protocol，TCP)的数据传输，能够更加稳定的进行数据传输，在设备运行开销上获得更大的投入产出比。然而，基于TCP的数据传输无法根据传统的丢包率、重传率来感知网络状态，控制投屏数据传输过程，以改善投屏用户的视频观看体验。
发明内容
本公开提供了一种投屏数据传输方法、装置、电子设备及存储介质能够提升用户的投屏视频观看体验。
本公开提供了一种投屏数据传输方法，包括：
接收目标数据接收端根据接收到的投屏数据发出的反馈信息，并根据所述反馈信息中的TCP头部信息中的预设数据传输字段的信息内容确定所述投屏数据的数据往返时间；
根据所述数据往返时间的数值变化进行网络带宽预估，并基于网络带宽预估结果和当前数据发送带宽确定投屏数据编码参数的调整策略；
按照所述调整策略调整所述投屏数据编码参数，并根据调整后的编码参数对待发送投屏数据进行编码，并完成数据传输。
本公开还提供了一种投屏数据传输装置，包括：
数据传输监测模块，设置为接收目标数据接收端根据接收到的投屏数据发出的反馈信息，并根据所述反馈信息中的TCP头部信息中的预设数据传输字段的信息内容确定所述投屏数据的数据往返时间；
数据编码控制模块，设置为根据所述数据往返时间的数值变化进行网络带宽预估，并基于网络带宽预估结果和当前数据发送带宽确定投屏数据编码参数的调整策略；
数据编码和传输模块，设置为按照所述调整策略调整所述投屏数据编码参数，并根据调整后的编码参数对待发送投屏数据进行编码，并完成数据传输。
本公开还提供了一种电子设备，所述电子设备包括：
一个或多个处理器；
存储装置，设置为存储一个或多个程序；
当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现上述的投屏数据传输方法。
本公开还提供了一种包含计算机可执行指令的存储介质，所述计算机可执行指令在由计算机处理器执行时用于执行上述的投屏数据传输方法。
附图说明
图1为本公开实施例一所提供的一种投屏数据传输方法的流程示意图；
图2为本公开实施例二所提供的一种投屏数据传输方法的流程示意图；
图3为本公开实施例二所提供的一种投屏数据传输过程示意图；
图4为本公开实施例三所提供的一种投屏数据传输装置结构示意图；
图5为本公开实施例四所提供的一种电子设备结构示意图。
具体实施方式
下面将参照附图描述本公开的实施例。虽然附图中显示了本公开的一些实施例，然而本公开可以通过多种形式来实现，提供这些实施例是为了理解本公开。本公开的附图及实施例仅用于示例性作用。
本公开的方法实施方式中记载的多个步骤可以按照不同的顺序执行，和/或并行执行。此外，方法实施方式可以包括附加的步骤和/或省略执行示出的步骤。本公开的范围在此方面不受限制。
本文使用的术语“包括”及其变形是开放性包括，即“包括但不限于”。术语“基于”是“至少部分地基于”。术语“一个实施例”表示“至少一个实施例”；术语“另一实施例”表示“至少一个另外的实施例”；术语“一些实施例”表示“至少一些实施例”。其他术语的相关定义将在下文描述中给出。
本公开中提及的“第一”、“第二”等概念仅用于对不同的装置、模块或单元进行区分，并非用于限定这些装置、模块或单元所执行的功能的顺序或者相互依存关系。
本公开中提及的“一个”、“多个”的修饰是示意性而非限制性的，本领域技术人员应当理解，除非在上下文另有指出，否则应该理解为“一个或多个”。
实施例一
图1为本公开实施例一所提供的一种投屏数据传输方法流程示意图，本公开实施例适用于对投屏数据的传输过程进行监控的情形。该方法可以由投屏数据传输装置来执行，该装置可以通过软件和/或硬件的形式实现，该装置可配置于电子设备中，例如配置于移动终端或服务器设备中。
如图1所示，本实施例提供的投屏数据传输方法，包括：
S110、接收目标数据接收端根据接收到的投屏数据发出的反馈信息，并根据所述反馈信息中的TCP头部信息中的预设数据传输字段的信息内容确定所述投屏数据的数据往返时间。
在投屏数据传输的场景下，目标数据接收端即投屏数据显示装置。目标数据接收端与投屏源端会基于预设的通信协议建立通信通道，并通过通信通道接收投屏源端发送的投屏数据。在本实施例中，投屏源端与目标数据接收端的通信协议为TCP，目标数据接收端在接收到投屏数据之后，还会发出反馈信息，告知投屏源端已经成功接受到相应的投屏数据。
投屏源端与目标数据接收端在执行连接、收发、断开等多个阶段的操作中，双方交互的数据均是被封装成数据包进行传输的。数据包由头部和数据两部分组成，其中，头部包含数据发送目的地等通信控制信息，数据即数据发送方要发给数据接收方的数据内容。在本实施例中，TCP头部信息中设置有一个预设数据传输字段，该字段可命名为载荷(PAYLOAD)字段，在该字段中可展示被发送的投屏数据的更多的信息。示例性的，在PAYLOAD字段中，第0个字节表示视频数据，第1个字节表示音频数据，第2个字节表示投屏数据发送的第一计数值，第3个字节表示投屏数据接收反馈的第二计数值。其中，第一计数值和第二计数值用于确定被发送的投屏数据的数据往返时间(也即数据往返时延(Round-Trip Time，RTT))。
在投屏源端发送数据时，会在预设数据传输字段的表示投屏数据发送的第一计数值位置处，写入数据包计数值，计数值通常从1开始，第二计数值字段为空。目标数据接收端接收到投屏源端的数据包后，发送接收确认反馈信息时，会在预设数据传输字段的第二计数值位置处写入与第一计数值相同的数值。当投屏源端接收到目标数据接收端根据接收到的投屏数据发出的反馈信息时，便可以解析反馈信息的TCP头部中预设数据传输字段的信息，识别预设数据传输字段中的表示投屏数据发送的第一计数值和表示投屏数据接收反馈的第二计数值；当第一计数值和第二计数值相同时，根据投屏数据的发送时间和接收到反馈信息的接收时间，确定投屏数据的数据往返时间。
S120、根据所述数据往返时间的数值变化进行网络带宽预估，并基于网络带宽预估结果和当前数据发送带宽确定投屏数据编码参数的调整策略。
网络带宽是指在单位时间内，从投屏源端到目标数据接收端所能够通过的最大数据率。对网络带宽预估可以为确定网络带宽在接下来的时间中的变化趋势。网络带宽与数据往返时间有着相反的变化趋势。当数据往返时间变长时，网络带宽便会减小。数据往返时间变长的程度越高，网络带宽减小的程度也越大。因此，便可以根据预估的网络带宽变化趋势调整投屏数据编码参数，减少待发送数据对网络带宽的占用，从而减少投屏数据显示的卡顿。
当前数据发送带宽是指发送接收到的反馈信息对应的投屏数据时的数据发送带宽，在发送投屏数据的过程中，可以对调用数据发送接口并发送投屏数据的时间进行计时，确定投屏数据发送时长；然后，根据所述投屏数据发送时长和所述投屏数据的数据量确定当前数据发送带宽，例如可以是用投屏数据的数据量除以投屏数据发送时长，确定当前数据发送带宽。
通常来说，可以控制的投屏数据编码参数包括数据编码码率、帧率和编码分辨率等。其中，码率是指编码器每秒编出的数据大小，单位是kbps，比如800kbps代表编码器每秒产生800kb(或100KB)的数据。帧率是指每秒钟播放的图像帧数。编码分辨率是指单位英寸图像中所包含的像素点数。投屏数据编码参数的调整策略是根据网络带宽预估的变化程度而制定的不同的参数调整方式。例如，在不影响用户极限主观清晰度体验和卡顿体验情况的前提下，如帧率不小于30帧每秒，可以设定参数调整的优先级，当预估网络带宽减小的程度较小时，优先调整数据编码码率；当预估网络带宽减小的程度较大时，便可以同时调整编码码率和分辨率，从而达到降低带宽占用的效果，以降低投屏视频的卡顿。针对每个参数调整的数值，可以预先建立一个参数值与网络带宽变化值的相关函数关系，依据该函数关系确定参数调整的数值。采用上述调整策略的前提是，当前数据发送带宽是大于或等于未调整的当前数据编码码率。当当 前数据发送带宽是小于未调整的当前数据编码码率，则会直接暂停对待发送的投屏数据进行编码，直到发送数据缓冲队列中的数据全部发送完毕，网络带宽恢复正常状态，这是因为在该网络状态下，调整数据编码参数后，也不能满足网络带宽的需求。
S130、按照所述调整策略调整所述投屏数据编码参数，并根据调整后的编码参数对待发送投屏数据进行编码，并完成数据传输。
在确定了数据编码参数的调整策略之后，便采用调整后的新的数据编码参数对还未进行编码的投屏数据进行编码，并在编码后发送到目标数据接收端。在每一个发送数据的数据包中，其TCP头部信息中的预设数据传输字段中都会写入相应的信息，包括用于计算RTT的表示数据发送的第一计数值。以便投屏源端持续的根据接收到的反馈信息进行RTT数值观测，进而对网络带宽进行预估，及时的调整投屏数据编码参数，让因为不可控的网络突发不稳定情况导致的投屏问题得以快速解决，减少投屏卡顿、甚至投屏中断的情况，提升用户体验。
本公开实施例的技术方案，可以在进行投屏数据传输的过程中，在接收到目标数据接收端根据接收到的投屏数据发出的反馈信息时，根据反馈信息中的TCP头部信息中的预设数据传输字段的信息内容确定每个投屏数据包的数据往返时间；进而，根据数据往返时间的数值变化进行网络带宽预估，并基于网络带宽预估结果和当前数据发送带宽确定投屏数据编码参数的调整策略；按照确定的调整策略调整投屏数据编码参数，并根据调整后的编码参数对待发送投屏数据进行编码，并完成数据传输。本公开实施例的技术方案解决了基于TCP的数据传输过程中无法直接监测网络状态以控制投屏数据传输过程的问题，实现了监测基于TCP进行投屏数据交互的双方之间的网络状态，根据网络状态调整投屏数据编码的策略并进行投屏数据传输，减少在投屏过程中由于网络问题带来的视频卡顿和暂停，提升用户的投屏视频观看体验。
实施例二
本公开实施例与上述实施例中所提供的投屏数据传输方法中多个方案可以结合。本实施例所提供的投屏数据传输方法，描述了投屏数据传输的完整过程，以及投屏参数调整的策略。
图2为本公开实施例二所提供的一种投屏数据传输方法的流程示意图。如图2所示，本实施例提供的投屏数据传输方法，包括：
S210、在投屏数据的TCP头部信息中，添加预设数据传输字段对应的信息内容，并依据TCP将所述投屏数据发送到目标数据接收端。
投屏实现的完整过程可参考图3所示投屏数据传输过程。在图3中的显示内容是要通过投屏显示装置显示的内容，显示内容经过编码器编码后会被发送单元进行数据打包，并在数据包上增加发送目的地等通信控制信息作为TCP头部信息，从而形成投屏源端发送的投屏数据。投屏源端的发送单元通过网络和TCP与目标数据接收端建立网络连接，并将投屏数据发送到目标数据接收端。
在本实施例中，TCP头部信息中设置有一个预设数据传输字段，字段中的信息用于参考确定投屏数据的RTT。示例性的，在预设数据传输字段中，有两个字节，一个字节用于记录投屏数据发送的第一计数值，一个字节用于表示投屏数据接收反馈的第二计数值。其中，第一计数值在发送单元进行数据打包时，会写入确定的计数信息，第二计数值为空。
S220、接收所述目标数据接收端根据接收到的投屏数据发出的反馈信息，并根据所述反馈信息中的TCP头部信息中的预设数据传输字段的信息内容确定所述投屏数据的数据往返时间。
目标数据接收端在接收到投屏数据之后，还会基于与投屏源端建立连接的网络通道发出反馈信息，告知投屏源端已经成功接受到相应的投屏数据。同时，反馈信息的TCP头部信息中有相同的预设数据传输字段，会按照投屏数据的TCP的头部信息的第一计数值，将第二计数值信息写入。对于一个投屏数据的数据包来说，第一计数值和第二计数值通常是相同的数值，从1开始计数。
从而，投屏源端可以根据接收到的反馈信息对投屏源端和目标数据接收端的网络状态进行检测。在一种实施方式中，投屏源端接收到目标数据接收端根据接收到的投屏数据发出的反馈信息时，便可以解析反馈信息的TCP头部中预设数据传输字段的信息，识别预设数据传输字段中的表示投屏数据发送的第一计数值和表示投屏数据接收反馈的第二计数值；当第一计数值和第二计数值相同时，根据投屏数据的发送时间和接收到反馈信息的接收时间，确定投屏数据的数据往返时间。
S230、根据所述数据往返时间的数值变化进行网络带宽预估，并基于网络带宽预估结果和当前数据发送带宽确定投屏数据编码参数的调整策略。
参照图3，即投屏源端在进行网络状态检测后，采用对应的投屏数据编码参数的调整策略，对编码器进行编码器控制的过程。
网络带宽与数据往返时间有着相反的变化趋势。当数据往返时间变长时，网络带宽便会减小。数据往返时间变长的程度越高，网络带宽减小的程度也越大。因此，可以通过RTT数值的变化程度反映网络带宽的变化趋势。
一般情况下，投屏源端解析出的RTT值是较为固定的稳定值，比如常见的 局域网wifi下，RTT值为5ms以内。当RTT值稳定不变化，同时，计算出的当前发送带宽和当前数据编码码率一致，则暂时先不对编码器的编码参数进行调整。
投屏数据编码参数的调整策略可以是根据网络带宽预估变化趋势，也即响应的RTT数值变化程度，指定的不同的参数调整方案。多个调整策略的设定均是在不影响用户极限的主观清晰度体验和卡顿体验的前提下进行设置的。
为了方便描述，直接采用RTT数值的变化来表示网络带宽的变化。示例性的，在一些实现方式中，当RTT值开始逐渐增大，而当前数据发送带宽和当前数据编码码率仍然一致时，优先进行数据编码码率调整，暂时不调整分辨率和帧率的数值。比如，投屏数据分辨率为1920x1080，上行带宽(当前数据发送带宽)为12Mbps，在投屏过程中，RTT数值增大到稳定值的两倍时，则调控编码器的数据编码码率降至原发送带宽的一半，也就是6Mbps，分辨率和帧率不进行调整。当RTT数值变化的程度增加，持续增大至稳定值的三倍时，此时则降低分辨率至1280x720，帧率保持不变，码率调整至5Mbps。而当计算出的当前数据发送带宽小于当前数据编码码率时，则暂停对显示内容进行编码，直至发送单元的发送缓冲队列发送完毕。
当RTT值开始逐渐减小时，表明网络带宽正在恢复或增大，则增大编码码率、分辨率和帧率的数值。每个参数调整的数值可参考表1所示的编码参数参考表。表1中的每个数值是根据经验或测试结果确定的参数配置结果。
表1
Figure PCTCN2022119533-appb-000001
在表1所示的数据表中，包括实际发送码率、RTT、预设码率、分辨率和帧率共计5个数据项。其中，实际发送码率即为当前的未调整的数据编码码率，预设码率为数据编码码率要调整的目标数值。当实际发送的码率和预设码率是一样的，数据编码参数则不做调整，当实际发送的码率和预设码率不一致时， 则将编码参数的每个参数值调整到预设码率对应的参数值。
在确定编码参数调整策略时，还可以参考不同的编码压缩方式下的参数关系，例如数据编码码率、分辨率和帧率的关系，可表示为数据编码码率＝分辨率*帧率*A，其中，参数A表示一个压缩率，比如采用h264压缩格式或者mpeg2压缩格式时，压缩比是不一样的。A不是固定的参数，可以调整，但是调整是有个范围的，比如5～20；因此，在一定情况下需要通过减小分辨率、帧率来实现码率的降低
S240、按照所述调整策略调整所述投屏数据编码参数，并根据调整后的编码参数对待发送投屏数据进行编码，并完成数据传输。
本公开实施例的技术方案，在上述实施例的基础上，描述了完整的投屏数据传输过程，在进行投屏数据传输的过程中，在接收到目标数据接收端根据接收到的投屏数据发出的反馈信息时，根据反馈信息中的TCP头部信息中的预设数据传输字段的信息内容确定每个投屏数据包的数据往返时间；进而，根据数据往返时间的数值变化进行网络带宽预估，并基于网络带宽预估结果和当前数据发送带宽确定投屏数据编码参数的调整策略，并对调整策略进行了举例说明；按照确定的调整策略调整投屏数据编码参数，并根据调整后的编码参数对待发送投屏数据进行编码，并完成数据传输。本公开实施例的技术方案解决了基于TCP的数据传输过程中无法直接监测网络状态以控制投屏数据传输过程的问题，实现了监测基于TCP进行投屏数据交互的双方之间的网络状态，根据网络状态调整投屏数据编码的策略并进行投屏数据传输，减少在投屏过程中由于网络问题带来的视频卡顿和暂停，提升用户的投屏视频观看体验。
实施例三
图4为本公开实施例三所提供的一种投屏数据传输装置结构示意图。本实施例提供的投屏数据传输装置适用于对投屏数据的传输过程进行监控的情形。
如图4所示，投屏数据传输装置包括：数据传输监测模块310、数据编码控制模块320和数据编码和传输模块330。
数据传输监测模块310，设置为接收目标数据接收端根据接收到的投屏数据发出的反馈信息，并根据所述反馈信息中的TCP头部信息中的预设数据传输字段的信息内容确定所述投屏数据的数据往返时间；数据编码控制模块320，设置为根据所述数据往返时间的数值变化进行网络带宽预估，并基于网络带宽预估结果和当前数据发送带宽确定投屏数据编码参数的调整策略；数据编码和传输模块330，设置为按照所述调整策略调整所述投屏数据编码参数，并根据调整后的编码参数对待发送投屏数据进行编码，并完成数据传输。
本公开实施例的技术方案，在进行投屏数据传输的过程中，在接收到目标数据接收端根据接收到的投屏数据发出的反馈信息时，根据反馈信息中的TCP头部信息中的预设数据传输字段的信息内容确定每个投屏数据包的数据往返时间；进而，根据数据往返时间的数值变化进行网络带宽预估，并基于网络带宽预估结果和当前数据发送带宽确定投屏数据编码参数的调整策略；按照确定的调整策略调整投屏数据编码参数，并根据调整后的编码参数对待发送投屏数据进行编码，并完成数据传输。本公开实施例的技术方案解决了基于TCP的数据传输过程中无法直接监测网络状态以控制投屏数据传输过程的问题，实现了监测基于TCP进行投屏数据交互的双方之间的网络状态，根据网络状态调整投屏数据编码的策略并进行投屏数据传输，减少在投屏过程中由于网络问题带来的视频卡顿和暂停，提升用户的投屏视频观看体验。
在一些实现方式中，所述数据编码和传输模块330还设置为在接收到所述反馈信息之前，在所述投屏数据的TCP头部信息中，添加所述预设数据传输字段对应的信息内容，并依据TCP将所述投屏数据发送到所述目标数据接收端。
在一些实现方式中，所述数据传输监测模块310设置为：
识别所述预设数据传输字段中的第一计数值和第二计数值；当所述第一计数值和所述第二计数值相同时，根据所述投屏数据的发送时间和接收到所述反馈信息的接收时间，确定所述投屏数据的数据往返时间。
在一些实现方式中，所述数据传输监测模块310还可设置为：
在发送所述投屏数据的过程中，对调用数据发送接口并发送所述投屏数据的时间进行计时，确定投屏数据发送时长；根据所述投屏数据发送时长和所述投屏数据的数据量确定当前数据发送带宽。
在一些实现方式中，所述数据编码控制模块320设置为：
根据所述数据往返时间的变化程度预估所述网络带宽的变化程度；当所述当前数据发送带宽大于或等于当前数据编码码率时，根据所述网络带宽的变化程度，匹配对应的预设投屏数据编码参数调整策略。
在一些实现方式中，所述投屏数据编码参数包括数据编码码率、分辨率和帧率，所述预设投屏数据编码参数调整策略中，对所述数据编码码率调整的优先级高于对所述分辨率和/或所述帧率进行调整的优先级。
在一些实现方式中，所述数据编码和传输模块330还设置为：
当所述当前数据发送带宽小于当前数据编码码率时，暂停对所述待发送投屏数据进行编码，直到投屏数据发送缓冲队列中的已编码投屏数据发送完毕。
本公开实施例所提供的投屏数据传输装置，可执行本公开任意实施例所提供的投屏数据传输方法，具备执行方法相应的功能模块和效果。
上述装置所包括的多个单元和模块只是按照功能逻辑进行划分的，但并不局限于上述的划分，只要能够实现相应的功能即可；另外，多个功能单元的名称也只是为了便于相互区分，并不用于限制本公开实施例的保护范围。
实施例四
下面参考图5，其示出了适于用来实现本公开实施例的电子设备(例如图5中的终端设备或服务器)400的结构示意图。本公开实施例中的终端设备400可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、个人数字助理(Personal Digital Assistant，PDA)、平板电脑(Portable Android Device，PAD)、便携式多媒体播放器(Portable Media Player，PMP)、车载终端(例如车载导航终端)等等的移动终端以及诸如数字电视(Television，TV)、台式计算机等等的固定终端。图5示出的电子设备400仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
如图5所示，电子设备400可以包括处理装置(例如中央处理器、图形处理器等)401，其可以根据存储在只读存储器(Read-Only Memory，ROM)402中的程序或者从存储装置408加载到随机访问存储器(Random Access Memory，RAM)403中的程序而执行多种适当的动作和处理。在RAM 403中，还存储有电子设备400操作所需的多种程序和数据。处理装置401、ROM 402以及RAM 403通过总线404彼此相连。输入/输出(Input/Output，I/O)接口405也连接至总线404。
通常，以下装置可以连接至I/O接口405：包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置406；包括例如液晶显示器(Liquid Crystal Display，LCD)、扬声器、振动器等的输出装置407；包括例如磁带、硬盘等的存储装置408；以及通信装置409。通信装置409可以允许电子设备400与其他设备进行无线或有线通信以交换数据。虽然图5示出了具有多种装置的电子设备400，并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装置。
根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在非暂态计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信装置409从网络上被下载和安装，或者从存储装置408被安装，或者从ROM 402被安装。在该计算机程序被处理装置401执行时，执行本公开实施例的投屏数据传输方 法中限定的上述功能。
本公开实施例提供的电子设备与上述实施例提供的投屏数据传输方法属于同一构思，未在本实施例中详尽描述的技术细节可参见上述实施例，并且本实施例与上述实施例具有相同的效果。
实施例五
本公开实施例提供了一种计算机存储介质，其上存储有计算机程序，该程序被处理器执行时实现上述实施例所提供的投屏数据传输方法。
本公开上述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、RAM、ROM、可擦式可编程只读存储器(Erasable Programmable Read-Only Memory，EPROM)或闪存(FLASH)、光纤、便携式紧凑磁盘只读存储器(Compact Disc Read-Only Memory，CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开中，计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读信号介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：电线、光缆、射频(Radio Frequency，RF)等等，或者上述的任意合适的组合。
在一些实施方式中，客户端、服务器可以利用诸如超文本传输协议(Hyper Text Transfer Protocol，HTTP)之类的任何当前已知或未来研发的网络协议进行通信，并且可以与任意形式或介质的数字数据通信(例如，通信网络)互连。通信网络的示例包括局域网(Local Area Network，LAN)，广域网(Wide Area Network，WAN)，网际网(例如，互联网)以及端对端网络(例如，ad hoc端对端网络)，以及任何当前已知或未来研发的网络。
上述计算机可读介质可以是上述电子设备中所包含的；也可以是单独存在，而未装配入该电子设备中。
上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被该电子设备执行时，使得该电子设备：接收目标数据接收端根据接收到的投屏数据发出的反馈信息，并根据所述反馈信息中的TCP头部信息中的预设数据传输字段的信息内容确定所述投屏数据的数据往返时间；根据所述数据往返时间的数值变化进行网络带宽预估，并基于网络带宽预估结果和当前数据发送带宽确定投屏数据编码参数的调整策略；按照所述调整策略调整所述投屏数据编码参数，并根据调整后的编码参数对待发送投屏数据进行编码，并完成数据传输。
可以以一种或多种程序设计语言或其组合来编写用于执行本公开的操作的计算机程序代码，上述程序设计语言包括但不限于面向对象的程序设计语言—诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括LAN或WAN—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
附图中的流程图和框图，图示了按照本公开多种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的单元可以通过软件的方式实现，也可以通过硬件的方式来实现。其中，单元、模块的名称在一种情况下并不构成对该单元、模块本身的限定，例如，数据编码和传输模块还可以被描述为“按照所述调整策略调整所述投屏数据编码参数，并根据调整后的编码参数对待发送投屏数据进行编码，并完成数据传输的模块”。
本文中以上描述的功能可以至少部分地由一个或多个硬件逻辑部件来执行。例如，非限制性地，可以使用的示范类型的硬件逻辑部件包括：现场可编程门 阵列(Field Programmable Gate Array，FPGA)、专用集成电路(Application Specific Integrated Circuit，ASIC)、专用标准产品(Application Specific Standard Parts，ASSP)、片上系统(System on Chip，SOC)、复杂可编程逻辑设备(CPLD)等等。
在本公开的上下文中，机器可读介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。机器可读存储介质的示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、RAM、ROM、EPROM或快闪存储器、光纤、CD-ROM、光学储存设备、磁储存设备、或上述内容的任何合适组合。
根据本公开的一个或多个实施例，【示例一】提供了一种投屏数据传输方法，该方法包括：
接收目标数据接收端根据接收到的投屏数据发出的反馈信息，并根据所述反馈信息中的TCP头部信息中的预设数据传输字段的信息内容确定所述投屏数据的数据往返时间；
根据所述数据往返时间的数值变化进行网络带宽预估，并基于网络带宽预估结果和当前数据发送带宽确定投屏数据编码参数的调整策略；
按照所述调整策略调整所述投屏数据编码参数，并根据调整后的编码参数对待发送投屏数据进行编码，并完成数据传输。
根据本公开的一个或多个实施例，【示例二】提供了一种投屏数据传输方法，还包括：
在一些实现方式中，在接收到所述反馈信息之前，所述方法还包括：
在所述投屏数据的TCP头部信息中，添加所述预设数据传输字段对应的信息内容，并依据TCP将所述投屏数据发送到所述目标数据接收端。
根据本公开的一个或多个实施例，【示例三】提供了一种投屏数据传输方法，还包括：
在一些实现方式中，所述根据所述反馈信息中的TCP头部信息中的预设数据传输字段的信息内容确定所述投屏数据的数据往返时间，包括：
识别所述预设数据传输字段中的第一计数值和第二计数值；
当所述第一计数值和所述第二计数值相同时，根据所述投屏数据的发送时 间和接收到所述反馈信息的接收时间，确定所述投屏数据的数据往返时间。
根据本公开的一个或多个实施例，【示例四】提供了一种投屏数据传输方法，还包括：
在一些实现方式中，在确定投屏数据编码参数的调整策略之前，所述方法还包括：
在发送所述投屏数据的过程中，对调用数据发送接口并发送所述投屏数据的时间进行计时，确定投屏数据发送时长；
根据所述投屏数据发送时长和所述投屏数据的数据量确定当前数据发送带宽。
根据本公开的一个或多个实施例，【示例五】提供了一种投屏数据传输方法，还包括：
在一些实现方式中，所述根据所述数据往返时间的数值变化进行网络带宽预估，并基于网络带宽预估结果和当前数据发送带宽确定投屏数据编码参数的调整策略，包括：
根据所述数据往返时间的变化程度预估所述网络带宽的变化程度；
当所述当前数据发送带宽大于或等于当前数据编码码率时，根据所述网络带宽的变化程度，匹配对应的预设投屏数据编码参数调整策略。
根据本公开的一个或多个实施例，【示例六】提供了一种投屏数据传输方法，还包括：
在一些实现方式中，所述投屏数据编码参数包括数据编码码率、分辨率和帧率，所述预设投屏数据编码参数调整策略中，对所述数据编码码率调整的优先级高于对所述分辨率和/或所述帧率进行调整的优先级。
根据本公开的一个或多个实施例，【示例七】提供了一种投屏数据传输方法，还包括：
在一些实现方式中，当所述当前数据发送带宽小于当前数据编码码率时，暂停对所述待发送投屏数据进行编码，直到投屏数据发送缓冲队列中的已编码投屏数据发送完毕。
根据本公开的一个或多个实施例，【示例八】提供了一种投屏数据传输装置，包括：
数据传输监测模块，设置为接收目标数据接收端根据接收到的投屏数据发出的反馈信息，并根据所述反馈信息中的TCP头部信息中的预设数据传输字段的信息内容确定所述投屏数据的数据往返时间；
数据编码控制模块，设置为根据所述数据往返时间的数值变化进行网络带宽预估，并基于网络带宽预估结果和当前数据发送带宽确定投屏数据编码参数的调整策略；
数据编码和传输模块，设置为按照所述调整策略调整所述投屏数据编码参数，并根据调整后的编码参数对待发送投屏数据进行编码，并完成数据传输。
根据本公开的一个或多个实施例，【示例九】提供了一种投屏数据传输装置，还包括：
在一些实现方式中，所述数据编码和传输模块还设置为在接收到所述反馈信息之前，在所述投屏数据的TCP头部信息中，添加所述预设数据传输字段对应的信息内容，并依据TCP将所述投屏数据发送到所述目标数据接收端。
根据本公开的一个或多个实施例，【示例十】提供了一种投屏数据传输装置，还包括：
在一些实现方式中，所述数据传输监测模块设置为：
识别所述预设数据传输字段中的第一计数值和第二计数值；
当所述第一计数值和所述第二计数值相同时，根据所述投屏数据的发送时间和接收到所述反馈信息的接收时间，确定所述投屏数据的数据往返时间。
根据本公开的一个或多个实施例，【示例十一】提供了一种投屏数据传输装置，还包括：
在一些实现方式中，所述数据传输监测模块还可设置为：
在发送所述投屏数据的过程中，对调用数据发送接口并发送所述投屏数据的时间进行计时，确定投屏数据发送时长；
根据所述投屏数据发送时长和所述投屏数据的数据量确定当前数据发送带宽。
根据本公开的一个或多个实施例，【示例十二】提供了一种投屏数据传输装置，还包括：
在一些实现方式中，所述数据编码控制模块设置为：
根据所述数据往返时间的变化程度预估所述网络带宽的变化程度；
当所述当前数据发送带宽大于或等于当前数据编码码率时，根据所述网络带宽的变化程度，匹配对应的预设投屏数据编码参数调整策略。
根据本公开的一个或多个实施例，【示例十三】提供了一种投屏数据传输装置，还包括：
在一些实现方式中，所述投屏数据编码参数包括数据编码码率、分辨率和帧率，所述预设投屏数据编码参数调整策略中，对所述数据编码码率调整的优先级高于对所述分辨率和/或所述帧率进行调整的优先级。
根据本公开的一个或多个实施例，【示例十四】提供了一种投屏数据传输装置，还包括：
在一些实现方式中，所述数据编码和传输模块还设置为：
当所述当前数据发送带宽小于当前数据编码码率时，暂停对所述待发送投屏数据进行编码，直到投屏数据发送缓冲队列中的已编码投屏数据发送完毕。
此外，虽然采用特定次序描绘了多个操作，但是这不应当理解为要求这些操作以所示出的特定次序或以顺序次序执行来执行。在一定环境下，多任务和并行处理可能是有利的。同样地，虽然在上面论述中包含了多个实现细节，但是这些不应当被解释为对本公开的范围的限制。在单独的实施例的上下文中描述的一些特征还可以组合地实现在单个实施例中。相反地，在单个实施例的上下文中描述的多种特征也可以单独地或以任何合适的子组合的方式实现在多个实施例中。

Claims (10)

  1. 一种投屏数据传输方法，包括：
    接收目标数据接收端根据接收到的投屏数据发出的反馈信息，并根据所述反馈信息中的传输控制协议TCP头部信息中的预设数据传输字段的信息内容确定所述投屏数据的数据往返时间；
    根据所述数据往返时间的数值变化进行网络带宽预估，并基于网络带宽预估结果和当前数据发送带宽确定投屏数据编码参数的调整策略；
    按照所述调整策略调整所述投屏数据编码参数，并根据调整后的编码参数对待发送投屏数据进行编码，并完成数据传输。
  2. 根据权利要求1所述的方法，在接收到所述反馈信息之前，还包括：
    在所述投屏数据的TCP头部信息中，添加所述预设数据传输字段对应的信息内容，并依据TCP将所述投屏数据发送到所述目标数据接收端。
  3. 根据权利要求1或2所述的方法，其中，所述根据所述反馈信息中的TCP头部信息中的预设数据传输字段的信息内容确定所述投屏数据的数据往返时间，包括：
    识别所述预设数据传输字段中的第一计数值和第二计数值；
    在所述第一计数值和所述第二计数值相同的情况下，根据所述投屏数据的发送时间和接收到所述反馈信息的接收时间，确定所述投屏数据的数据往返时间。
  4. 根据权利要求1所述的方法，在所述确定投屏数据编码参数的调整策略之前，还包括：
    在发送所述投屏数据的过程中，对调用数据发送接口并发送所述投屏数据的时间进行计时，确定投屏数据发送时长；
    根据所述投屏数据发送时长和所述投屏数据的数据量确定所述当前数据发送带宽。
  5. 根据权利要求1所述的方法，其中，所述根据所述数据往返时间的数值变化进行网络带宽预估，并基于网络带宽预估结果和当前数据发送带宽确定投屏数据编码参数的调整策略，包括：
    根据所述数据往返时间的变化程度预估所述网络带宽的变化程度；
    在所述当前数据发送带宽大于或等于当前数据编码码率的情况下，根据所述网络带宽的变化程度，匹配对应的预设投屏数据编码参数调整策略。
  6. 根据权利要求5所述的方法，其中，所述投屏数据编码参数包括数据编码 码率、分辨率和帧率，所述预设投屏数据编码参数调整策略中，对所述数据编码码率调整的优先级高于对所述分辨率和所述帧率中的至少之一进行调整的优先级。
  7. 根据权利要求5所述的方法，还包括：
    在所述当前数据发送带宽小于当前数据编码码率的情况下，暂停对所述待发送投屏数据进行编码，直到投屏数据发送缓冲队列中的已编码投屏数据发送完毕。
  8. 一种投屏数据传输装置，包括：
    数据传输监测模块，设置为接收目标数据接收端根据接收到的投屏数据发出的反馈信息，并根据所述反馈信息中的传输控制协议TCP头部信息中的预设数据传输字段的信息内容确定所述投屏数据的数据往返时间；
    数据编码控制模块，设置为根据所述数据往返时间的数值变化进行网络带宽预估，并基于网络带宽预估结果和当前数据发送带宽确定投屏数据编码参数的调整策略；
    数据编码和传输模块，设置为按照所述调整策略调整所述投屏数据编码参数，并根据调整后的编码参数对待发送投屏数据进行编码，并完成数据传输。
  9. 一种电子设备，包括：
    至少一个处理器；
    存储装置，设置为存储至少一个程序；
    当所述至少一个程序被所述至少一个处理器执行，使得所述至少一个处理器实现如权利要求1-7中任一项所述的投屏数据传输方法。
  10. 一种包含计算机可执行指令的存储介质，所述计算机可执行指令在由计算机处理器执行时用于执行如权利要求1-7中任一项所述的投屏数据传输方法。
PCT/CN2022/119533 2021-10-25 2022-09-19 投屏数据传输方法、装置、电子设备及存储介质 WO2023071605A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP22885492.3A EP4284004A1 (en) 2021-10-25 2022-09-19 Screen projection data transmission method and apparatus, electronic device, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111242981.7A CN113992967B (zh) 2021-10-25 2021-10-25 一种投屏数据传输方法、装置、电子设备及存储介质
CN202111242981.7 2021-10-25

Publications (1)

Publication Number Publication Date
WO2023071605A1 true WO2023071605A1 (zh) 2023-05-04

Family

ID=79741193

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/119533 WO2023071605A1 (zh) 2021-10-25 2022-09-19 投屏数据传输方法、装置、电子设备及存储介质

Country Status (3)

Country Link
EP (1) EP4284004A1 (zh)
CN (1) CN113992967B (zh)
WO (1) WO2023071605A1 (zh)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114501122B (zh) * 2022-01-30 2023-04-07 深圳创维－Rgb电子有限公司 音视频数据的投屏方法、装置、存储介质和投屏器
CN114389975B (zh) * 2022-02-08 2024-03-08 北京字节跳动网络技术有限公司 网络带宽预估方法、装置、系统、电子设备及存储介质
CN114786060A (zh) * 2022-02-28 2022-07-22 长沙朗源电子科技有限公司 基于安卓平台无线投屏实现帧率控制的方法及其装置
CN115134641B (zh) * 2022-07-05 2023-10-27 北京字跳网络技术有限公司 投屏方法、装置和电子设备
CN115278319A (zh) * 2022-07-13 2022-11-01 海信视像科技股份有限公司 一种投屏数据传输方法、装置、电子设备和存储介质
CN117278538B (zh) * 2023-11-22 2024-04-16 荣耀终端有限公司 调整编码器的参数的方法及电子设备

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170094294A1 (en) * 2015-09-28 2017-03-30 Cybrook Inc. Video encoding and decoding with back channel message management
CN109729439A (zh) * 2019-01-11 2019-05-07 北京三体云联科技有限公司 实时视频传输方法
CN109819322A (zh) * 2019-03-15 2019-05-28 网易（杭州）网络有限公司 视频传输方法、装置、计算机可读存储介质及电子设备
CN111741248A (zh) * 2020-05-26 2020-10-02 视联动力信息技术股份有限公司 一种数据传输方法、装置、终端设备和存储介质
CN111836079A (zh) * 2019-04-18 2020-10-27 杭州海康威视数字技术股份有限公司 一种视频码流的传输方法及装置
CN112203138A (zh) * 2020-10-16 2021-01-08 深圳乐播科技有限公司 基于udp协议的投屏数据传输方法、装置、设备及存储介质

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170094296A1 (en) * 2015-09-28 2017-03-30 Cybrook Inc. Bandwidth Adjustment For Real-time Video Transmission
CN108769716A (zh) * 2018-06-08 2018-11-06 四川乐望崇智科技有限公司 一种高效智能手机投屏方法
CN110602548B (zh) * 2019-09-20 2021-11-02 北京市博汇科技股份有限公司 一种超高清视频高质量无线传输的方法和系统
CN112312137A (zh) * 2020-10-27 2021-02-02 唐桥科技(杭州)有限公司 一种视频传输方法、装置、电子设备及存储介质
CN113038180A (zh) * 2021-02-26 2021-06-25 深圳创维－Rgb电子有限公司 智能镜像投屏方法、装置、设备及存储介质
CN113259760B (zh) * 2021-07-06 2021-11-30 深圳乐播科技有限公司 局域网投屏的通信协议方法、系统、设备及存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170094294A1 (en) * 2015-09-28 2017-03-30 Cybrook Inc. Video encoding and decoding with back channel message management
CN109729439A (zh) * 2019-01-11 2019-05-07 北京三体云联科技有限公司 实时视频传输方法
CN109819322A (zh) * 2019-03-15 2019-05-28 网易（杭州）网络有限公司 视频传输方法、装置、计算机可读存储介质及电子设备
CN111836079A (zh) * 2019-04-18 2020-10-27 杭州海康威视数字技术股份有限公司 一种视频码流的传输方法及装置
CN111741248A (zh) * 2020-05-26 2020-10-02 视联动力信息技术股份有限公司 一种数据传输方法、装置、终端设备和存储介质
CN112203138A (zh) * 2020-10-16 2021-01-08 深圳乐播科技有限公司 基于udp协议的投屏数据传输方法、装置、设备及存储介质

Also Published As

Publication number Publication date
CN113992967A (zh) 2022-01-28
EP4284004A1 (en) 2023-11-29
CN113992967B (zh) 2022-11-01

Similar Documents

Publication Publication Date Title
WO2023071605A1 (zh) 投屏数据传输方法、装置、电子设备及存储介质
WO2020034758A1 (zh) 多通道数据传输方法及装置
WO2020140729A1 (zh) 数据传输方法、装置、计算机可读介质及电子设备
WO2022194140A1 (zh) 远程视频传输方法、传输装置、存储介质及电子设备
CN112312137A (zh) 一种视频传输方法、装置、电子设备及存储介质
WO2015126741A1 (en) Efficiently mixing voip data
US11792130B2 (en) Audio/video communication method, terminal, server, computer device, and storage medium
WO2022001175A1 (zh) 数据包发送的方法、装置
WO2023051243A1 (zh) 视频码率切换方法、装置、电子设备及存储介质
WO2023071598A1 (zh) 音视频同步监控方法、装置、电子设备及存储介质
CN112583529B (zh) 一种数据处理方法、装置、设备及存储介质
CN112312229A (zh) 一种视频传输方法、装置、电子设备及存储介质
CN114640886A (zh) 自适应带宽的音视频传输方法、装置、计算机设备及介质
WO2014204180A1 (en) Method and apparatus for rate adaptation in motion picture experts group media transport
EP3281317A1 (en) Multi-layer timing synchronization framework
WO2016008131A1 (en) Techniques for separately playing audio and video data in local networks
CN113438180B (zh) Udp数据包的传输控制方法、装置、设备及可读存储介质
CN114710568B (zh) 音视频数据通信方法、设备及存储介质
CN112994848B (zh) Mcs的调整方法、装置、存储介质及无线节点
CN113242446B (zh) 视频帧的缓存方法、转发方法、通信服务器及程序产品
CN112153322A (zh) 数据分发方法、装置、设备及存储介质
WO2023119512A1 (ja) 中継装置、通信システム、及び中継方法
WO2024011962A1 (zh) 视频流传输的控制方法及装置、设备、介质
WO2022193182A1 (zh) 一种传输参数调整方法及通信装置
WO2023119511A1 (ja) 通信装置、通信システム、及び通信方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22885492

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022885492

Country of ref document: EP

Effective date: 20230822

WWE Wipo information: entry into national phase

Ref document number: 18549068

Country of ref document: US