CN114390322A - Video coding method based on remote desktop protocol and application thereof - Google Patents

Video coding method based on remote desktop protocol and application thereof Download PDF

Info

Publication number
CN114390322A
CN114390322A CN202210056778.9A CN202210056778A CN114390322A CN 114390322 A CN114390322 A CN 114390322A CN 202210056778 A CN202210056778 A CN 202210056778A CN 114390322 A CN114390322 A CN 114390322A
Authority
CN
China
Prior art keywords
video stream
coding mode
display channel
monitoring period
data change
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210056778.9A
Other languages
Chinese (zh)
Inventor
张文靖
高健
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Anchaoyun Software Co ltd
Original Assignee
Shanghai Anchaoyun Software Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Anchaoyun Software Co ltd filed Critical Shanghai Anchaoyun Software Co ltd
Priority to CN202210056778.9A priority Critical patent/CN114390322A/en
Publication of CN114390322A publication Critical patent/CN114390322A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/24—Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
    • H04N21/2402—Monitoring of the downstream path of the transmission network, e.g. bandwidth available
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display

Abstract

The invention discloses a video coding method based on a remote desktop protocol and application thereof. The method comprises the following steps: the protocol server side and the client side establish connection of a display channel; and the protocol server monitors the network condition in the display channel and the total data change amount of the video stream in the display channel, and dynamically selects an encoding mode to encode the video stream according to the network condition and the total data change amount of the video stream in the display channel. The video coding method based on the remote desktop protocol can dynamically select the coding mode of the video stream according to the current network condition and the total data change amount in the display channel, so that a user can obtain good viewing experience under different network conditions.

Description

Video coding method based on remote desktop protocol and application thereof
Technical Field
The present invention relates to the field of computers, and more particularly, to a video encoding method based on a remote desktop protocol and an application thereof.
Background
The existing remote desktop protocol comprises a client, a server, corresponding display equipment, a display driver and the like, wherein the client runs on user terminal equipment to provide a desktop environment for a user. After the remote desktop protocol is connected, a plurality of channels are established, wherein the channels comprise a display channel, and the display channel controls the display of the remote desktop.
The video compression technology in the existing remote desktop protocol supports lossy coding and lossless coding, high-quality image quality experience can be obtained when the lossless coding is selected, but the video compression technology is blocked when the network quality is poor; when lossy coding is selected, smooth use experience can be obtained when the network quality fluctuates, the influence of network fluctuation is low, but the picture quality is low, and the watching effect is influenced.
Therefore, in view of the above technical problems, it is necessary to provide a video encoding method based on a remote desktop protocol.
Disclosure of Invention
The invention aims to provide a video coding method based on a remote desktop protocol, which can automatically select a corresponding coding mode according to different network conditions and improve the watching experience of a user.
In order to achieve the purpose, the technical scheme provided by the invention is as follows:
in a first aspect, the present invention provides a video encoding method based on a remote desktop protocol, which includes:
the protocol server side and the client side establish connection of a display channel; and the protocol server monitors the network condition in the display channel and the total data change amount of the video stream in the display channel, and dynamically selects an encoding mode to encode the video stream according to the network condition and the total data change amount.
In one or more embodiments, the protocol server monitors a network condition in the display channel and a total amount of data change of a video stream in the display channel, and dynamically selects an encoding mode to encode the video stream according to the network condition and the total amount of data change, including:
the protocol server side periodically monitors the network condition in the display channel and the total data change amount of the video stream in the display channel, wherein each monitoring period is defined as Tn(n is not less than 1) during the monitoring period T1In the method, the protocol server side optionally selects a lossless coding mode, a high-quality coding mode and a low-quality coding modeEncoding the video stream; monitoring period TmAnd (m is more than or equal to 2) dynamically selecting an encoding mode to encode the video stream according to the network condition in the previous monitoring period and the data change total amount of the video stream in the display channel.
In one or more embodiments, the period T is monitored whilenWhen the internal monitoring result is that the network condition is good, the protocol server side monitors the total amount of data change of the video stream in the display channel in the next monitoring period Tn+1Intra-selection lossless coding mode or high quality coding mode for encoding the video stream; when monitoring period TnWhen the internal monitoring result is that the network condition is general, the protocol server side monitors the period T according to the total data change amount of the video stream in the display channeln+1Intra-selecting either a high quality encoding mode or a low quality encoding mode to encode the video stream; when monitoring period TnWhen the internal monitoring result is that the network condition is poor, the protocol server side is in the next monitoring period Tn+1An intra-direct selection low quality coding mode encodes the video stream.
In one or more embodiments, the total amount of data change in the display channel is a single monitoring period TnThe frame data in the inner display channel change the sum of the rectangular area lengths.
In one or more embodiments, the period T is monitorednThe length of the rectangular area of each frame data change is defined as s, and s is expressed by the following formula:
s＝(right-left)×(bottom-top)×d
wherein left and top are point coordinates of the upper left corner of each frame of data; right and bottom are the coordinates of the points in the lower right corner, and d is the color depth; then the period T is monitorednDisplaying the total amount of data change S in the channeln＝s1+s2+s3+......+sp(p≥1)。
In one or more embodiments, the current monitoring period TnWhen the internal monitoring result is that the network condition is good, the protocol server side is in the total data change amount of the video stream in the display channelNext monitoring period Tn+1Intra-selection lossless coding mode or high quality coding mode for encoding the video stream, comprising:
if monitoring the period TnThe internal monitoring result is that the network condition is good and the monitoring period TnTotal amount of data change S of video stream in said display channelnGreater than a first threshold Q1Then in the next monitoring period Tn+1Internally selecting a high-quality coding mode; if monitoring the period TnThe internal monitoring result is that the network condition is good and the monitoring period TnTotal amount of data change S of video stream in said display channelnLess than a second threshold Q2Then in the next monitoring period Tn+1Intra-selection of a lossless coding mode; if monitoring the period TnThe internal monitoring result is that the network condition is good and the monitoring period TnUsing low-quality coding mode or high-quality coding mode, and monitoring period TnTotal amount of data change S of video stream in said display channelnNot less than a second threshold Q2Then in the next monitoring period Tn+1Intra-switching or maintaining high quality coding modes; wherein Q is1＝P1×R，Q2＝P2X R, R ═ width × height/resolution, and P × resolution × d > P1＞P2Resolution x d; p1Is a preset threshold value, P, for switching from a lossless coding mode to a high-quality coding mode2The threshold value for switching from the high-quality coding mode to the lossless coding mode is R, the threshold value factor is R, the resolution is the resolution of the video stream in the display channel, and width and height are the highest resolution which can be supported by the display screen.
In one or more embodiments, the current monitoring period TnWhen the internal monitoring result is that the network condition is general, the protocol server side monitors the period T according to the total data change amount of the video stream in the display channeln+1Intra-selecting either a low quality coding mode or a high quality coding mode for encoding the video stream, comprising:
if monitoring the period TnThe internal monitoring result is that the network condition is general, and the monitoring period TnThe display unitTotal amount of data change S of in-track video streamnGreater than a third threshold Q3Then in the next monitoring period Tn+1Internally selecting a low quality coding mode; if monitoring the period TnThe internal monitoring result is that the network condition is general, and the monitoring period TnTotal amount of data change S of video stream in said display channelnLess than a fourth threshold Q4Then in the next monitoring period Tn+1Internally selecting a high-quality coding mode; if monitoring the period TnThe internal monitoring result is that the network condition is general, and the monitoring period TnUsing lossless coding mode or high-quality coding mode, and monitoring period TnTotal amount of data change S of video stream in said display channelnNot less than a fourth threshold Q4Then in the next monitoring period Tn+1Intra-switching to a low quality encoding mode; wherein Q is3＝P3×R，Q4＝P4X R, R ═ width × height/resolution, P × resolution × d > P3＞P4Resolution x d; p3For a predetermined threshold for switching from a high-quality coding mode to a low-quality coding mode, P4The threshold value for switching from the low-quality coding mode to the high-quality coding mode is set, the resolution is the resolution of the video stream in the display channel, R is the threshold value factor, and width and height are the highest resolution supported by the display screen.
In one or more embodiments, the evaluation indicators of the network condition include a network packet loss rate, a round trip delay, and a network jitter.
In one or more embodiments, upper and lower limits of network packet loss rate, round trip delay, and network jitter are preset; when monitoring period TnWhen the internally monitored network packet loss rate, the round trip delay and the network jitter are not higher than the lower limit, judging that the network condition is good; when monitoring period TnIf the network packet loss rate, the round-trip delay and the network jitter which are monitored in the network are not lower than the upper limit, judging that the network condition is poor; when monitoring period TnWhen any evaluation index of the internally monitored network packet loss rate, the round-trip delay and the network jitter is between the lower limit and the upper limit, judging that the network condition is oneAnd (4) the steps are as follows.
In a second aspect, the present invention provides a remote desktop protocol-based video encoding apparatus, comprising: the device comprises a communication module, a monitoring module, a selection module and an encoding module; the communication module is used for establishing the connection of a display channel between the protocol server and the client; the monitoring module is used for monitoring the network condition in the display channel and the total data change amount of the video stream in the display channel by the protocol server; the selection module is used for dynamically selecting a coding mode by the protocol server according to the network condition and the total data change amount of the video stream in the display channel; the encoding module is used for encoding the video stream according to the encoding mode selected by the selection module.
In a third aspect, the present invention provides an electronic device, comprising: at least one processor, and a memory storing instructions that, when executed by the at least one processor, cause the at least one processor to perform a remote desktop protocol based video encoding method as previously described.
In a fourth aspect, the present invention provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a remote desktop protocol-based video encoding method as described above.
Compared with the prior art, the video coding method based on the remote desktop protocol can dynamically select the coding mode of the video stream according to the current network condition and the total data change amount in the display channel, and automatically select the low-quality coding mode of the high-compression image when the network condition is poor so as to reduce the occupation of network resources as much as possible; when the network condition is general, the low-quality coding mode and the high-quality coding mode can be dynamically switched according to the total data change amount in the display channel so as to ensure the smooth watching experience of a user; when the network condition is good, the data in the display channel can be dynamically switched between the lossless coding mode and the high-quality coding mode according to the total amount of data change, so that the user can obtain the viewing experience of higher image quality under the condition that the network is not limited.
Drawings
FIG. 1 is a flow chart of a video encoding method based on a remote desktop protocol according to an embodiment of the present invention;
FIG. 2 is a block diagram of a video encoding apparatus based on remote desktop protocol according to an embodiment of the present invention;
FIG. 3 is a block diagram of a computing device according to an embodiment of the invention.
Detailed Description
The following detailed description of the present invention is provided in conjunction with the accompanying drawings, but it should be understood that the scope of the present invention is not limited to the specific embodiments.
Throughout the specification and claims, unless explicitly stated otherwise, the word "comprise", or variations such as "comprises" or "comprising", will be understood to imply the inclusion of a stated element or component but not the exclusion of any other element or component.
Fig. 1 shows a flowchart of an embodiment of a video encoding method based on a remote desktop protocol, where the method includes the following steps:
s101: and the protocol server establishes connection of the display channel with the client.
Before video coding based on a remote desktop protocol is carried out, a remote desktop protocol environment needs to be deployed and prepared so as to establish connection between a protocol server and a client display channel. The display channel is one of the channels after the remote desktop protocol is connected, and is mainly responsible for processing graphic commands and transmitting image and video data. The remote desktop protocol processes different events by establishing different channels, and when the protocol server is connected with the protocol client, the display channel is successfully established, so that details are not repeated herein.
The remote desktop protocol may be selected from existing remote desktop protocols according to actual needs, and is not particularly limited herein.
S102: the protocol server monitors the network condition in the display channel and the total data change amount of the video stream in the display channel, and dynamically selects an encoding mode to encode the video stream according to the network condition and the total data change amount in the display channel.
Step S102 specifically includes that the protocol server periodically monitors and displays the network status in the channel and displays the total data change amount of the video stream in the channel. Wherein each monitoring period is defined as Tn(n is not less than 1) during the monitoring period T1And the protocol server selects one of a lossless coding mode, a high-quality coding mode and a low-quality coding mode to code the video stream. Wherein, the high-quality coding mode and the low-quality coding mode are both lossy coding, and the high-quality coding has higher image quality compared with the low-quality coding.
Monitoring period TmAnd (m is more than or equal to 2) dynamically selecting an encoding mode to encode the video stream according to the network condition in the previous monitoring period and the data change total amount of the video stream in the display channel.
In an exemplary embodiment, the evaluation indicators of the network condition mainly include a network packet loss rate, a round trip delay, and a network jitter, and therefore the protocol server needs to monitor the evaluation indicators of the network condition. The current network condition can be rated according to the evaluation index of the network condition monitored by the protocol server. In practical application, other indexes for judging the network state can be added, and specific index setting can be selected according to actual needs.
Specifically, the network conditions can be divided into good, general and poor levels according to the network packet loss rate, round trip delay and network jitter monitored by the protocol server, and corresponding different coding modes can be formulated according to the network conditions of different levels.
Specifically, the upper limit and the lower limit of the network packet loss rate, the round-trip delay and the network jitter can be preset; when monitoring period TnWhen the internally monitored network packet loss rate, the round trip delay and the network jitter are not higher than the lower limit, judging that the network condition is good; when monitoring period TnIf the network packet loss rate, the round-trip delay and the network jitter which are monitored in the network are not lower than the upper limit, judging that the network condition is poor; when monitoring period TnInternally monitored network packet loss rateAnd when any evaluation index in the round trip delay and the network jitter is between the lower limit and the upper limit, judging that the network condition is general.
For example, when the network bandwidth is 50-100Gbp/s, the upper limit and the lower limit of the network packet loss rate may be set to 0.1% and 0.01%, the upper limit and the lower limit of the round trip delay may be set to 100ms and 30ms, and the upper limit and the lower limit of the network jitter may be set to 40ms and 10ms, respectively. Different upper and lower limits may be preset based on different network bandwidths.
And when the network packet loss rate monitored by the protocol server is less than or equal to 0.01%, the round-trip delay is less than or equal to 30ms, and the network jitter is less than or equal to 10ms, judging that the network condition is good. And when the network packet loss rate monitored by the protocol server is greater than or equal to 0.1%, the round-trip delay is greater than or equal to 100ms, and the network jitter is greater than or equal to 40ms, determining that the network condition is poor. When the network packet loss rate monitored by the protocol server is between 0.01 and 0.1 percent, and/or the round-trip delay is between 30 and 100ms, and/or the network jitter is between 10 and 40ms, the network condition is judged to be poor.
In the invention, the threshold of the evaluation index of the network condition can be preset according to different actual use environments (such as the size of the whole bandwidth), and can also be changed according to the change of the actual use environments. Therefore, it can be understood that the threshold values of the evaluation indexes used for evaluating different levels of the network conditions may be changed according to actual use conditions.
In other embodiments, the network conditions may be classified into more or less levels, and may be selected according to actual needs. For example, network conditions may be classified into more levels when more choices of coding modes based on network conditions are involved. The threshold value of each index for classifying the network condition may be adjusted according to the actual situation, and is not particularly limited herein. In this embodiment, the total amount of data change in the display channel means a single monitoring period TnThe frame data in the inner display channel change the sum of the rectangular area lengths.
Suppose a monitoring period TnIn the period (the period duration can be set according to actual needs, and can be any value in 1-3 s, for example), p frame data are shared in the display channel, the length of the rectangular area of the change of each frame data is defined as s, and s can be expressed by the following formula:
s＝(right-left)×(bottom-top)×d
wherein left and top are point coordinates of the upper left corner; right and bottom are the coordinates of the points in the lower right corner; d is the color depth.
Then the period T is monitorednIn the display channel, the length of the rectangular area of the first frame data change is s1The second frame data change rectangle has an area length of s2The third frame data change rectangle has an area length of s3.... p frame data change rectangle area length is sp. Then, during the monitoring period TnIn the display channel, the total data change S of the p frame datan＝s1+s2+s3+......+sp。
In an exemplary embodiment, when monitoring period TnWhen the internal monitoring result is that the network condition is good, the protocol server side displays the total data change amount of the video stream in the channel in the next monitoring period Tn+1Intra-selection lossless coding mode or high quality coding mode encodes the video stream. Under the condition of good network conditions, two coding modes, namely a lossless coding mode and a high-quality coding mode, can be selected so that a user can obtain a high-quality video experience.
Specifically, if the period T is monitorednThe internal monitoring result is that the network condition is good and the monitoring period TnTotal amount of data change S of video stream in said display channelnGreater than a first threshold Q1Then in the next monitoring period Tn+1The high quality coding mode is selected.
If monitoring the period TnThe internal monitoring result is that the network condition is good and the monitoring period TnTotal amount of data change S of video stream in said display channelnLess than a second threshold Q2Then in the next monitoring period Tn+1Internal selection lossless codingA code pattern.
If monitoring the period TnThe internal monitoring result is that the network condition is good and the monitoring period TnUsing low-quality coding mode or high-quality coding mode, and monitoring period TnTotal amount of data change S of video stream in said display channelnNot less than a second threshold Q2Then in the next monitoring period Tn+1Inter-switching or maintaining high quality coding modes.
Wherein Q is1＝P1×R，Q2＝P2X R, R ═ width × height/resolution, and P × resolution × d > P1＞P2Resolution x d; p1Is a preset threshold value, P, for switching from a lossless coding mode to a high-quality coding mode2The threshold value for switching from the high-quality coding mode to the lossless coding mode is R, the threshold value factor is R, the resolution is the resolution of the video stream in the display channel, and width and height are the highest resolution which can be supported by the display screen. First threshold value Q1And a second threshold value Q2The method can be set according to actual conditions, and is set by combining actual network quality factors such as monitoring time, resolution, network downlink bandwidth and the like. For example, the longer the monitoring period duration corresponding to the total amount of data change in the display channel is set, the greater the number of frames corresponding to the longer the monitoring period duration is set, and the first threshold Q is set1And a second threshold value Q2The height needs to be correspondingly increased; otherwise, the first threshold Q1And a second threshold value Q2And correspondingly reduced. As another example, the greater the resolution, the first threshold Q1And a second threshold value Q2The height needs to be correspondingly increased; otherwise, the first threshold Q1And a second threshold value Q2And correspondingly reduced. The better the network condition, the better the first threshold Q1And a second threshold value Q2The height needs to be correspondingly increased; otherwise, the first threshold Q1And a second threshold value Q2And correspondingly reduced.
In an exemplary embodiment, when monitoring period TnWhen the internal monitoring result is that the network condition is general, the protocol server side is in the next monitoring period T according to the total data change amount of the video stream in the display channeln+1Internally selecting either a high quality coding mode or a low quality coding modeEncoding the video stream. Under the general condition of network conditions, two coding modes, namely a high-quality coding mode and a low-quality coding mode, can be selected.
Specifically, if the period T is monitorednThe internal monitoring result is that the network condition is general, and the monitoring period TnTotal amount of data change S of video stream in said display channelnGreater than a third threshold Q3Then in the next monitoring period Tn+1The low quality coding mode is selected internally.
If monitoring the period TnThe internal monitoring result is that the network condition is general, and the monitoring period TnTotal amount of data change S of video stream in said display channelnLess than a fourth threshold Q4Then in the next monitoring period Tn+1The high quality coding mode is selected.
If monitoring the period TnThe internal monitoring result is that the network condition is general, and the monitoring period TnUsing lossless coding mode or high-quality coding mode, and monitoring period TnTotal amount of data change S of video stream in said display channelnNot less than a fourth threshold Q4Then in the next monitoring period Tn+1Intra-switching to a low quality coding mode.
Wherein Q is3＝P3×R，Q4＝P4X R, R ═ width × height/resolution, P × resolution × d > P3＞P4Resolution x d; p3For a predetermined threshold for switching from a high-quality coding mode to a low-quality coding mode, P4The threshold value for switching from the low-quality coding mode to the high-quality coding mode is set, the resolution is the resolution of the video stream in the display channel, R is the threshold value factor, and width and height are the highest resolution supported by the display screen.
Third threshold value Q3And a fourth threshold value Q3The method can be set according to actual conditions, and is set by combining actual network quality factors such as monitoring time, resolution, network downlink bandwidth and the like. For example, the longer the time T corresponding to the total amount of data change in the display channel is set, the greater the number of frames corresponding to the time T, and the third threshold Q3And a fourth thresholdValue Q4The height needs to be correspondingly increased; otherwise, the third threshold Q3And a fourth threshold value Q4And correspondingly reduced. As another example, the greater the resolution, the third threshold Q3And a fourth threshold value Q4The height needs to be correspondingly increased; otherwise, the third threshold Q3And a fourth threshold value Q4And correspondingly reduced. The better the network condition, the better the third threshold Q3And a fourth threshold value Q4The height needs to be correspondingly increased; otherwise, the third threshold Q3And a fourth threshold value Q4And correspondingly reduced.
In an exemplary embodiment, when monitoring period TnWhen the internal monitoring result is that the network condition is poor, the protocol server side is in the next monitoring period Tn+1An intra-direct selection low quality coding mode encodes the video stream. In the case of poor network conditions, to ensure that the user obtains a smooth viewing experience, a low-quality encoding mode may be selected to encode the video stream to adapt to the poor network conditions.
Specifically, the encoding mode may be based on encoding modes of h.263, h.263+ +, h.264/AVC, h.265/HEVC, h.266/VVC, MPEG4, and the like.
Among them, MPEG (Moving Picture Experts Group) is a working Group established by ISO (International Organization for Standardization) and specifically responsible for establishing a standard related to Moving Picture compression coding, and the established MPEG standard is an International general standard which is composed of video, audio and system. MPEG4 is one of the network video image compression standards and is characterized by high compression ratio and clear imaging. The MPEG4 video compression algorithm can provide extremely high compression ratios, up to 200:1, and more importantly, MPEG4 provides high compression ratios with little loss of data.
H.263 is a draft standard of ITU (International telecommunications Union), designed for low-stream communication. But in practice this standard can be used over a wide code stream range rather than only for low code streams, and the standard has higher performance and error correction capability. The motion compensation of h.263 uses half-pixel precision, some parts of the data stream hierarchy are optional in h.263 so that the codec can be configured for lower data rates or better error correction capability, h.263 contains four negotiable options to improve performance, h.263 uses unlimited motion vectors and syntax-based arithmetic coding, and h.263 supports 5 resolutions.
The video encoding method based on the remote desktop protocol according to the present invention is further described below with reference to specific embodiments.
Example 1
The method comprises the steps of deploying and preparing a remote desktop protocol environment, and setting a first threshold value Q according to factors such as a network environment and resolution1A second threshold value Q2A third threshold value Q3And a fourth threshold value Q4. Wherein the first threshold value Q1Greater than a second threshold Q2Third threshold value Q3Greater than a fourth threshold Q4. The upper limit and the lower limit of the network packet loss rate are respectively set to be 0.1% and 0.01%, the upper limit and the lower limit of the round trip delay are respectively set to be 100ms and 30ms, and the upper limit and the lower limit of the network jitter are respectively set to be 40ms and 10 ms.
And starting the virtual machine, and establishing connection (including connection of a display channel) between the protocol server and the client. The protocol server side periodically monitors and displays the network condition in the channel and displays the total data change amount of the video stream in the channel.
Suppose a monitoring period TnAnd if the monitored network packet loss rate is less than or equal to 0.01%, the round-trip delay is less than or equal to 30ms, and the network jitter is less than or equal to 10ms, the network condition is judged to be good.
Protocol server side is in monitoring period TnIn the method, the total quantity of data change in the display channel is Sn. The total amount of data change S in the display channelnAnd a first threshold value Q1And a second threshold value Q2A comparison is made. There are several situations:
if the total amount of data change S of the video stream in the display channelnGreater than a first threshold Q1I.e. representing the monitoring period TnTotal amount of internal data change SnBig watchDisplay monitoring period TnIf the transmission of the internal video stream is unstable and does not satisfy the lossless coding mode, the next monitoring period T is neededn+1The high quality coding mode is selected.
If the total amount of data change S of the video stream in the display channelnLess than a second threshold Q2I.e. representing the monitoring period TnTotal amount of internal data change SnSmaller, representing a monitoring period TnThe transmission of the inner video stream is stable, and the lossless coding mode is satisfied, so that the next monitoring period T can be usedn+1Intra-select lossless coding mode.
If the total amount of data change S of the video stream in the display channelnNot less than a second threshold Q2And monitoring the period TnWhen the coding mode in the inner layer is a low quality coding mode or a high quality coding mode, then in the next monitoring period Tn+1Inter-switching or maintaining high quality coding modes.
In the present embodiment, description will be given mainly on dynamic selection of a video encoding mode when a network condition is good.
Example 2
The method comprises the steps of deploying and preparing a remote desktop protocol environment, and setting a first threshold value Q according to factors such as a network environment and resolution1A second threshold value Q2A third threshold value Q3And a fourth threshold value Q4. Wherein the first threshold value Q1Greater than a second threshold Q2Third threshold value Q3Greater than a fourth threshold Q4. The upper limit and the lower limit of the network packet loss rate are respectively set to be 0.1% and 0.01%, the upper limit and the lower limit of the round trip delay are respectively set to be 100ms and 30ms, and the upper limit and the lower limit of the network jitter are respectively set to be 40ms and 10 ms.
And starting the virtual machine, and establishing connection (including connection of a display channel) between the protocol server and the client. The protocol server side periodically monitors and displays the network condition in the channel and displays the total data change amount of the video stream in the channel.
Suppose a monitoring period TnIn the method, the monitored network packet loss rate is between 0.01 and 0.1 percent, the round-trip delay is between 30 and 100ms, and the networkIf the jitter is between 10-40 ms, the network status is determined to be normal.
Protocol server side is in monitoring period TnIn the method, the total quantity of data change in the display channel is Sn. The total amount of data change S in the display channelnAnd a third threshold value Q3And a fourth threshold value Q4A comparison is made. There are several situations:
if the total amount of data change S of the video stream in the display channelnGreater than a third threshold Q3I.e. representing the monitoring period TnTotal amount of internal data change SnLarger, indicating a monitoring period TnIf the transmission of the inner video stream is unstable and does not meet the high-quality coding mode, the next monitoring period T is providedn+1The low quality coding mode is selected internally.
If the total amount of data change S of the video stream in the display channelnLess than a fourth threshold Q4I.e. representing the monitoring period TnTotal amount of internal data change SnSmaller, representing a monitoring period TnThe transmission of the inner video stream is stable, and meets the high-coding mode, so that the next monitoring period T can be usedn+1The inner selection is a high coding mode.
If the total amount of data change S of the video stream in the display channelnNot less than a second threshold Q2And monitoring the period TnWhen the coding mode in the inner layer is a low quality coding mode or a high quality coding mode, then in the next monitoring period Tn+1Inter-switching or maintaining high quality coding modes.
In the present embodiment, description will be mainly given of dynamic selection of a video encoding mode in a case where a network condition is general.
Example 3
The method comprises the steps of deploying and preparing a remote desktop protocol environment, and setting a first threshold value Q according to factors such as a network environment and resolution1A second threshold value Q2A third threshold value Q3And a fourth threshold value Q4. Wherein the first threshold value Q1Greater than a second threshold Q2Third threshold value Q3Greater than a fourth threshold Q4. Upper and lower limits of network packet loss rateThe limits are set to 0.1% and 0.01%, respectively, the upper and lower limits of the round trip delay are set to 100ms and 30ms, respectively, and the upper and lower limits of the network jitter are set to 40ms and 10ms, respectively.
And starting the virtual machine, and establishing connection (including connection of a display channel) between the protocol server and the client. The protocol server side periodically monitors and displays the network condition in the channel and displays the total data change amount of the video stream in the channel.
Suppose a monitoring period TnAnd if the monitored network packet loss rate is greater than 0.1%, the round-trip delay is greater than 100ms, and the network jitter is greater than 40ms, determining that the network condition is poor. At this time, the network is relatively stuck, and the protocol server is in the next monitoring period Tn+1An intra-direct selection low quality coding mode encodes the video stream. By adopting the low-quality coding mode, the user can be ensured to obtain smooth watching experience under the condition of poor network condition.
Based on the same inventive concept, a structural frame of a remote desktop protocol-based video encoding device 2 according to an embodiment of the present invention is shown in fig. 2, and the remote desktop protocol-based video encoding device includes: a communication module 21, a monitoring module 22, a selection module 23 and an encoding module 24.
The communication module 21 is used for the protocol server to establish a connection of the display channel with the client. The monitoring module 22 is used for the protocol server to monitor the network condition in the display channel and the total data change amount of the video stream in the display channel. The selection module 23 is used for the protocol server to dynamically select the coding mode according to the network condition and the total data change amount of the video stream in the display channel. The encoding module 24 is configured to encode the video stream according to the encoding mode selected by the selection module.
The communication module 21 may implement a communication connection between the protocol server and the client, and the communication module 21 may be a wireless communication module or a wired communication module.
As shown in fig. 3, an embodiment of the present invention further provides a computing device 3, where the computing device 3 includes at least one processor 31, a storage 32 (e.g., a non-volatile storage), a memory 33, and a communication interface 34, and the at least one processor 31, the storage 32, the memory 33, and the communication interface 34 are connected together via a bus 35. The at least one processor 31 is configured to invoke at least one program instruction stored or encoded in the memory 32 to cause the at least one processor 31 to perform various operations and functions of the remote desktop protocol based video encoding method described in various embodiments of the present specification.
In embodiments of the present description, computing device 3 may include, but is not limited to: personal computers, server computers, workstations, desktop computers, laptop computers, notebook computers, mobile computing devices, smart phones, tablet computers, cellular phones, Personal Digital Assistants (PDAs), handheld devices, messaging devices, wearable computing devices, consumer electronics, and so forth.
An embodiment of the present invention further provides a computer-readable storage medium, on which computer program instructions are stored, the computer program instructions being used for causing a computer to execute the above video encoding method based on the remote desktop protocol.
The computer-readable storage medium can be any available medium or data storage device that can be accessed by a computer, including but not limited to magnetic memory (e.g., floppy disks, hard disks, magnetic tape, magneto-optical disks (MOs), etc.), optical memory (e.g., CDs, DVDs, BDs, HVDs, etc.), and semiconductor memory (e.g., ROMs, EPROMs, EEPROMs, non-volatile memory (NAND FLASH), Solid State Disks (SSDs), etc.).
In summary, the video encoding method based on the remote desktop protocol provided by the present invention can dynamically select the encoding mode for the video stream according to the current network status and the total amount of data change in the display channel, and automatically select the low quality encoding mode for the high compression image when the network status is poor, so as to reduce the occupation of network resources as much as possible; when the network condition is general, the low-quality coding mode and the high-quality coding mode can be dynamically switched according to the total data change amount in the display channel so as to ensure the smooth watching experience of a user; when the network condition is good, the data in the display channel can be dynamically switched between the lossless coding mode and the high-quality coding mode according to the total amount of data change, so that the user can obtain the viewing experience of higher image quality under the condition that the network is not limited.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The foregoing descriptions of specific exemplary embodiments of the present invention have been presented for purposes of illustration and description. It is not intended to limit the invention to the precise form disclosed, and obviously many modifications and variations are possible in light of the above teaching. The exemplary embodiments were chosen and described in order to explain certain principles of the invention and its practical application to enable one skilled in the art to make and use various exemplary embodiments of the invention and various alternatives and modifications as are suited to the particular use contemplated. It is intended that the scope of the invention be defined by the claims and their equivalents.

Claims (12)

1. A video coding method based on a remote desktop protocol is characterized by comprising the following steps:
the protocol server side and the client side establish connection of a display channel;
and the protocol server monitors the network condition in the display channel and the total data change amount of the video stream in the display channel, and dynamically selects an encoding mode to encode the video stream according to the network condition and the total data change amount.
2. The method as claimed in claim 1, wherein the protocol server monitors network conditions in the display channel and a total amount of data change of the video stream in the display channel, and dynamically selects an encoding mode to encode the video stream according to the network conditions and the total amount of data change, comprising:
the protocol server side periodically monitors the network condition in the display channel and the total data change amount of the video stream in the display channel, wherein each monitoring period is defined as Tn(n is not less than 1) during the monitoring period T1In the protocol service end, one pair of lossless coding mode, high-quality coding mode and low-quality coding mode is selected optionallyEncoding the video stream;
monitoring period TmAnd (m is more than or equal to 2) dynamically selecting an encoding mode to encode the video stream according to the network condition in the previous monitoring period and the data change total amount of the video stream in the display channel.
3. The remote desktop protocol-based video encoding method of claim 2,
when monitoring period TnWhen the internal monitoring result is that the network condition is good, the protocol server side monitors the total amount of data change of the video stream in the display channel in the next monitoring period Tn+1Intra-selection lossless coding mode or high quality coding mode for encoding the video stream;
when monitoring period TnWhen the internal monitoring result is that the network condition is general, the protocol server side monitors the period T according to the total data change amount of the video stream in the display channeln+1Intra-selecting either a high quality encoding mode or a low quality encoding mode to encode the video stream;
when monitoring period TnWhen the internal monitoring result is that the network condition is poor, the protocol server side is in the next monitoring period Tn+1An intra-direct selection low quality coding mode encodes the video stream.
4. The method of claim 3, wherein the total amount of data change in the display channel is a single monitoring period TnThe frame data in the inner display channel change the sum of the rectangular area lengths.
5. The remote desktop protocol-based video encoding method of claim 4, wherein the monitoring period T isnThe length of the rectangular area of each frame data change is defined as s, and s is expressed by the following formula:
s＝(right-left)×(bottom-top)×d
wherein left and top are point coordinates of the upper left corner of each frame of data; right and bottom are the coordinates of the points in the lower right corner, and d is the color depth;
then the period T is monitorednDisplaying the total amount of data change S in the channeln＝s1+s2+s3+......+sp(p≥1)。
6. The method of claim 5, wherein the current monitoring period T is equal to or greater than the monitoring period TnWhen the internal monitoring result is that the network condition is good, the protocol server side monitors the total amount of data change of the video stream in the display channel in the next monitoring period Tn+1Intra-selection lossless coding mode or high quality coding mode for encoding the video stream, comprising:
if monitoring the period TnThe internal monitoring result is that the network condition is good and the monitoring period TnTotal amount of data change S of video stream in said display channelnGreater than a first threshold Q1Then in the next monitoring period Tn+1Internally selecting a high-quality coding mode;
if monitoring the period TnThe internal monitoring result is that the network condition is good and the monitoring period TnTotal amount of data change S of video stream in said display channelnLess than a second threshold Q2Then in the next monitoring period Tn+1Intra-selection of a lossless coding mode;
if monitoring the period TnThe internal monitoring result is that the network condition is good and the monitoring period TnUsing low-quality coding mode or high-quality coding mode, and monitoring period TnTotal amount of data change S of video stream in said display channelnNot less than a second threshold Q2Then in the next monitoring period Tn+1Intra-switching or maintaining high quality coding modes;
wherein Q is1＝P1×R，Q2＝P2X R, R ═ width × height/resolution, and P × resolution × d > P1＞P2Resolution x d;
P1for a preset threshold value for switching from lossless coding mode to high-quality coding mode,P2the threshold value for switching from the high-quality coding mode to the lossless coding mode is R, the threshold value factor is R, the resolution is the resolution of the video stream in the display channel, and width and height are the highest resolution which can be supported by the display screen.
7. The method of claim 6, wherein the current monitoring period T is equal to or greater than the monitoring period TnWhen the internal monitoring result is that the network condition is general, the protocol server side monitors the period T according to the total data change amount of the video stream in the display channeln+1Intra-selecting either a low quality coding mode or a high quality coding mode for encoding the video stream, comprising:
if monitoring the period TnThe internal monitoring result is that the network condition is general, and the monitoring period TnTotal amount of data change S of video stream in said display channelnGreater than a third threshold Q3Then in the next monitoring period Tn+1Internally selecting a low quality coding mode;
if monitoring the period TnThe internal monitoring result is that the network condition is general, and the monitoring period TnTotal amount of data change S of video stream in said display channelnLess than a fourth threshold Q4Then in the next monitoring period Tn+1Internally selecting a high-quality coding mode;
if monitoring the period TnThe internal monitoring result is that the network condition is general, and the monitoring period TnUsing lossless coding mode or high-quality coding mode, and monitoring period TnTotal amount of data change S of video stream in said display channelnNot less than a fourth threshold Q4Then in the next monitoring period Tn+1Intra-switching to a low quality encoding mode;
wherein Q is3＝P3×R，Q4＝P4X R, R ═ width × height/resolution, P × resolution × d > P3＞P4Resolution x d;
P3for a predetermined threshold for switching from a high-quality coding mode to a low-quality coding mode, P4For coding the mode by low qualityAnd switching to a threshold value of a high-quality coding mode, wherein the resolution is the resolution of the video stream in the display channel, R is a threshold value factor, and width and height are the highest resolution supported by the display screen.
8. The method according to any one of claims 1 to 7, wherein the evaluation indicators of the network condition include a network packet loss rate, a round trip delay and a network jitter.
9. The remote desktop protocol-based video encoding method of claim 8,
presetting the upper limit and the lower limit of network packet loss rate, round-trip delay and network jitter;
when monitoring period TnWhen the internally monitored network packet loss rate, the round trip delay and the network jitter are not higher than the lower limit, judging that the network condition is good;
when monitoring period TnIf the network packet loss rate, the round-trip delay and the network jitter which are monitored in the network are not lower than the upper limit, judging that the network condition is poor;
when monitoring period TnAnd when any judgment index of the network packet loss rate, the round-trip delay and the network jitter which are monitored in the network is between the lower limit and the upper limit, judging that the network condition is general.
10. A video encoding apparatus based on a remote desktop protocol, comprising:
the communication module is used for establishing the connection of a display channel between the protocol server and the client;
the monitoring module is used for monitoring the network condition in the display channel and the total data change amount of the video stream in the display channel by the protocol server;
the selection module is used for dynamically selecting a coding mode by the protocol server according to the network condition and the total data change amount of the video stream in the display channel;
and the coding module is used for coding the video stream according to the coding mode selected by the selection module.
11. An electronic device, comprising:
at least one processor; and
a memory storing instructions that, when executed by the at least one processor, cause the at least one processor to perform the remote desktop protocol based video encoding method of any of claims 1-9.
12. A computer-readable storage medium, having stored thereon a computer program which, when executed by a processor, implements a remote desktop protocol-based video encoding method according to any one of claims 1 to 9.
CN202210056778.9A 2022-01-18 2022-01-18 Video coding method based on remote desktop protocol and application thereof Pending CN114390322A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210056778.9A CN114390322A (en) 2022-01-18 2022-01-18 Video coding method based on remote desktop protocol and application thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210056778.9A CN114390322A (en) 2022-01-18 2022-01-18 Video coding method based on remote desktop protocol and application thereof

Publications (1)

Publication Number Publication Date
CN114390322A true CN114390322A (en) 2022-04-22

Family

ID=81203229

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210056778.9A Pending CN114390322A (en) 2022-01-18 2022-01-18 Video coding method based on remote desktop protocol and application thereof

Country Status (1)

Country Link
CN (1) CN114390322A (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101340575A (en) * 2007-07-03 2009-01-07 英华达(上海)电子有限公司 Method and terminal for dynamically regulating video code
US20110090949A1 (en) * 2008-09-27 2011-04-21 Tencent Technology (Shenzhen) Company Limited Multi-Channel Video Communication Method And System
CN103139609A (en) * 2013-02-01 2013-06-05 深圳市深信服电子科技有限公司 Method and device for optimizing remote application video play
CN108495142A (en) * 2018-04-11 2018-09-04 腾讯科技（深圳）有限公司 Method for video coding and device
CN110401875A (en) * 2018-04-24 2019-11-01 深信服科技股份有限公司 A kind of virtual desktop video pictures method of quality control, system and equipment

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101340575A (en) * 2007-07-03 2009-01-07 英华达(上海)电子有限公司 Method and terminal for dynamically regulating video code
US20110090949A1 (en) * 2008-09-27 2011-04-21 Tencent Technology (Shenzhen) Company Limited Multi-Channel Video Communication Method And System
CN103139609A (en) * 2013-02-01 2013-06-05 深圳市深信服电子科技有限公司 Method and device for optimizing remote application video play
CN108495142A (en) * 2018-04-11 2018-09-04 腾讯科技（深圳）有限公司 Method for video coding and device
CN110401875A (en) * 2018-04-24 2019-11-01 深信服科技股份有限公司 A kind of virtual desktop video pictures method of quality control, system and equipment

Similar Documents

Publication Publication Date Title
JP5404339B2 (en) Optimal power usage in data stream encoding
KR101557504B1 (en) Method for transmitting adapted channel condition apparatus using the method and providing system
EP3866474B1 (en) Video self-adaptation method, related device and storage medium
CN105163134A (en) Video coding parameter setting method, device and video coding device for live video
EP3016395B1 (en) Video encoding device and video encoding method
CN113747194B (en) Remote video transmission method, transmission device, storage medium and electronic equipment
US8842159B2 (en) Encoding processing for conferencing systems
ITTO20090486A1 (en) DYNAMIC CONTROLLER OF INDEPENDENT TRANSMISSION SPEED FROM THE GROUP OF IMAGES
MX2013004481A (en) Dynamic encode setting adjustment.
US8867611B2 (en) Video transmitting method and system with image quality adjusting function
CN111031389B (en) Video processing method, electronic device and storage medium
EP3595304A1 (en) Code rate allocation method, device, and storage medium
CN111225240A (en) Method and device for automatically adjusting occupied bandwidth, storage medium and electronic equipment
WO2021057705A1 (en) Video encoding and decoding methods, and related apparatuses
KR102171227B1 (en) A method of controlling bit rate and an apparatus therefor
US11641447B2 (en) System and method for automatically adjusting key frame quantization parameter and frame rate
JP2005020731A (en) Image data processing method of mobile communication terminal
CN107105317B (en) Video playing method and device
WO2021057479A1 (en) Video coding method, video decoding method, and related devices
JP5428702B2 (en) Stream communication system, server device, and client device
CN104967917A (en) Instant video coding method and electronic device
CN111294546B (en) Resolution adjustment method and device for video call
CN114390322A (en) Video coding method based on remote desktop protocol and application thereof
WO2021057478A1 (en) Video encoding and decoding method and related apparatus
WO2021057480A1 (en) Video coding method and video decoding method, and related apparatuses

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination