WO2015188678A1

WO2015188678A1 - Transmission control method and system for video stream of mobile device

Info

Publication number: WO2015188678A1
Application number: PCT/CN2015/079029
Authority: WO
Inventors: 魏子恒
Original assignee: 珠海全志科技股份有限公司
Priority date: 2014-06-13
Filing date: 2015-05-15
Publication date: 2015-12-17
Also published as: CN105263066A

Abstract

Provided are a transmission control method and system for a video stream of a mobile device. The method comprises the steps of: in the transmission process of a video stream, detecting network detection parameters of a transmission network in real time; and according to the network detection parameters, readjusting a video and network transmission parameter, the adjustment of the video and network transmission parameter comprising the adjustment of parameters related to a video coding end and the adjustment of parameters of a wireless chip network driver. By means of the present invention, according to the weight ratio of the network detection parameters, a preset threshold level satisfied thereby in a threshold table is calculated, and according to the preset threshold level, the frame rate, code rate and display resolution are adjusted, thereby ensuring that the video playing has a low time delay and fluency, and can be recovered quickly without influencing or slightly reducing the picture quality; and a flag bit which is output by a video encoder for distinguishing video stream data into a B frame or a P frame is detected, so that the transmission of a video key frame can be ensured, thereby ensuring a reliable picture quality and providing a good user experience.

Description

Mobile device video stream transmission control method and system

Related application

The present application claims priority to Chinese Patent Application No. 2014-10263758.4, entitled "Mobile Device Video Streaming Control Method and System", which is incorporated herein by reference in its entirety.

Technical field

The invention relates to the field of digital video data transmission of a wireless network, and in particular to a method and system for controlling video stream transmission of a mobile device.

Background technique

With the rise of smart systems such as iOS (Iphone Operating System) and Android (Android operating system), mobile devices (smart phones, tablets) are becoming more and more powerful, and due to the limitations of mobile devices, many entertainments Content (such as video, real-time video or games) needs to be transferred to a large-screen device to play the display. The content is usually directly transferred to the streaming media file content, or the buffer (buffer) displayed by the system is captured in real time and transmitted through the video codec. Displaying to the decoding device side presents new challenges to the performance of mobile devices and the performance of real-time streaming media delivery frameworks.

When a mobile device transmits a high-resolution, high-rate real-time video stream in a wireless local area network, if the wireless network environment is slightly worse, the throughput of the WIFI (Wireless Fidelity, wireless network transmission technology) is seriously reduced, resulting in a delay in video playback. More and more big, Caton and Huaping are serious.

Summary of the invention

The invention provides a video stream transmission control method and system for a mobile device with low delay and good image quality for the problem of playing the Karton and the flower screen in the conventional video stream.

To achieve the technical purpose, an embodiment of the present invention provides a mobile device video stream transmission control method, including the following steps:

During the video stream transmission process, the network detection parameters of the transmission network are detected in real time;

Re-adjusting the video and network transmission parameters according to the network detection parameters;

The adjustment processing of the video and network transmission parameters includes parameter adjustment related to the video encoding end and parameter adjustment of the wireless chip network driver.

As an implementation manner, the real-time detecting network detection parameters of the transmission network includes the following steps:

Real-time detection of internal transmission statistical parameters of the wireless chip network driver as a first input;

Real-time detection of the total data buffer usage size of the current streaming media package as a second input;

Real-time detection of available memory of the current system as a third input;

The threshold value table stores a parameter adjustment of the decision video encoder and a preset threshold level of the display resolution adjustment as a fifth input;

The weight ratio of the first input > the weight ratio of the second input > the weight ratio of the third input.

As an implementation manner, the internal transmission statistics parameters of the wireless chip network driver include an 802.11 frame retransmission times per second, a packet loss number per second, a received signal strength indication, and a transmission buffer margin.

As an implementation manner, the performing parameter adjustment related to the video encoding end according to the network detection parameter includes the following steps:

Determining, by the first input, the second input, and the third input, a preset threshold level that is satisfied in a threshold table of the fifth input;

The parameters and display resolution of the video encoder are adjusted according to the predetermined threshold level that is satisfied.

As an implementable manner, the method further includes the following steps:

Real-time detecting that the differentiated video stream data output by the video encoder is a flag bit of a P frame or a B frame, as a fourth input;

Performing parameter adjustment of the wireless chip network driver according to the network detection parameter includes the following steps:

According to the flag bit, the scanning mode of the wireless chip network driving is passive scanning, and the number of retransmissions of the P frame is set to be three times of the number of retransmissions of the B frame.

The embodiment of the present invention further provides a mobile device video stream transmission control system, including a real-time transmission processing module, configured to detect a network detection parameter of a transmission network in real time during a video stream transmission process, and re-pair the video according to the network detection parameter. And network transmission parameters for adjustment processing;

As an implementation manner, the real-time transmission processing module includes a first input unit, a second input unit, a third input unit, and a fifth input unit, where:

The first input unit is configured to detect an internal transmission statistical parameter of the wireless chip network driver in real time;

The second input unit is configured to detect a total data buffer usage size after the current streaming media package in real time;

The third input unit is configured to detect available memory of the current system in real time;

The fifth input unit is configured to store a preset threshold level of parameter adjustment and display resolution adjustment of the decision video encoder;

The weight ratio of the first input unit > the weight ratio of the second input unit > the weight ratio of the third input unit.

As an implementation manner, the internal transmission statistical parameters of the wireless chip network driver include 802.11 frame retransmission The number of times per second, the number of packet loss per second, the received signal strength indication, and the send buffer margin.

As an implementation manner, the real-time transmission processing module further includes a determining unit and a first adjusting unit, where:

The determining unit is configured to determine the internal transmission statistical parameter, the total data buffer usage size, and a preset threshold level that is satisfied by the available memory of the current system in the threshold table;

The first adjusting unit is configured to adjust a parameter and a display resolution of the video encoder according to the preset threshold level that is satisfied.

As an implementation manner, the fourth input unit and the second adjustment unit are further included, where:

The fourth input unit is configured to detect, in real time, the differentiated video stream data output by the video encoder as a flag of a P frame or a B frame;

The second adjusting unit is configured to set the scanning mode of the wireless chip network driving to passive scanning according to the flag bit, and set the number of retransmissions of the P frame to be three times of the number of retransmissions of the B frame.

Advantageous effects of the present invention include:

The mobile device video stream transmission control method and system of the invention adds a feedback link in each link of the transmission network, detects the network detection parameters of the transmission network in real time, and re-adjusts the video and network transmission parameters according to the network detection parameters. , realizes the information interaction control with the transmission network. The network detection parameter of the present invention includes an internal transmission statistical parameter driven by the wireless chip network, a total data buffer usage size, and an available memory of the current system, and the preset threshold value satisfied in the threshold table is calculated according to the weight ratio of the network detection parameters. Level, adjust the frame rate, code rate and display resolution according to the preset threshold level, so as to ensure low delay and smoothness of video playback and fast recovery without detecting or slightly degrading the image quality, detecting video coding The distinguishing video stream data outputted by the device is the flag of the B frame or the P frame, and is fed back to the wireless chip network driver. The wireless chip network driver recognizes the B frame and the P frame to ensure the transmission of the video key frame, thereby ensuring reliable picture quality. Provide a good user experience.

DRAWINGS

1 is a flowchart of an embodiment of a video stream transmission control method for a mobile device according to the present invention;

2 is a schematic structural diagram of a frame of an embodiment of a transmission network of a video stream transmission control method for a mobile device according to the present invention;

FIG. 3 is a schematic structural diagram of an embodiment of a video stream transmission control system for a mobile device according to the present invention.

detailed description

In order to make the objectives, technical solutions and advantages of the present invention more clear, the video stream transmission control method and system of the mobile device of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

As shown in FIG. 1 and FIG. 2, an embodiment of the present invention provides a video stream transmission control method for a mobile device, including the following steps:

S100: In the process of video stream transmission, detecting network detection parameters of the transmission network in real time.

S200, re-adjusting the video and network transmission parameters according to the network detection parameters. The adjustment processing of the video and network transmission parameters includes parameter adjustment related to the video encoding end and parameter adjustment of the wireless chip network driver.

The conventional mobile device video stream transmission method, the data source of the video image is a system display buffer (buffer), which is scaled and output to the video encoder, and the video encoder performs a fixed code rate according to the previously set code rate and frame rate. And frame rate output, and then streamed and packaged (such as packaged into mpeg4ts format), streamed and output to the data cache list waiting for output to UDP socket buffer (user data packet protocol socket cache), and finally through the wireless chip The network driver transmits it out. The traditional RTSP (Real Time Streaming Protocol)/RTP (Real Time Transport Protocol) has control of transmission bandwidth and data packet loss retransmission control, but there is a large delay due to implementation at the protocol level. When waiting for a retransmission of a lost packet within a certain period of time, the timeout is discarded, and a certain amount of system memory needs to be used to save the video stream data that has not been acknowledged, and the real-time performance cannot be guaranteed. The conventional technology also has the following disadvantages: WIFI will actively discard data due to too many retransmissions, regardless of whether the transmission data is important, or the local buffer data is increasingly discarded due to the delay of transmission, and the buffer data is actively discarded. Proactively discarding video data in the link can cause serious screen and jamming problems and cannot be recovered.

The object of the present invention is to overcome the above problems, and to effectively ensure the quality of video playback while low delay. The embodiment of the invention detects the network detection parameters of the transmission network in real time, realizes the information interaction control with the transmission network, and closely combines the memory usage on the data link of the wireless chip network driver and the transmission network, according to the detected network detection parameters. Adjust the video and network transmission parameters of the transmission network, such as adjusting the frame rate and code rate of the video encoder, and adjusting the display resolution of the scaling, so that the transmission network guarantees the video without affecting or slightly degrading the image quality. Low latency and smoothness of playback and fast recovery, to achieve the purpose of reducing network traffic and quality assurance of video key frames, thus providing users with a good user experience.

As an implementable manner, detecting network detection parameters of the transmission network in real time includes the following steps:

The internal transmission statistical parameters of the wireless chip network driver are detected in real time as the first input.

The total data buffer usage size of the current streaming media package is detected in real time as a second input.

Real-time detection of the available memory of the current system as a third input.

The threshold value table stores the parameter adjustment of the decision video encoder and the preset threshold level of the display resolution adjustment as the fifth input.

Obtain internal transmission statistics parameters of the wireless chip network driver, obtain the total data buffer usage size after streaming media encapsulation, obtain available memory of the current system, and calculate a preset threshold that is satisfied in the threshold table according to their weight ratio Value level. Wherein, the weight ratio of the internal transmission statistical parameter of the wireless chip network driver can be set to 70%, the weight ratio of the total data buffer usage size is 20%, and the weight ratio of the available memory of the current system is 10%. Of course, in other embodiments, adjustments may also be made to adjust the ratio of each parameter to other values. The preset threshold levels are defined according to specific product specifications.

As an implementation manner, the internal transmission statistics parameters of the wireless chip network driver include the number of 802.11 frame retransmissions per second, the number of packet loss per second, the received signal strength indication, and the transmission buffer margin.

The number of 802.11 frame retransmissions represents the number of retransmissions of 802.11 frames in 1s. The smaller the value, the better the network environment. It can estimate the busyness of the current wireless transmission network and can be directly obtained from the wireless chip network driver. The number of packet loss per second indicates the number of times the device loses packets within 1 s. The smaller the value, the better the network environment. It can measure the number of retransmission failures and estimate the current wireless transmission network. Received Signal Strength Indication (RSSI) indicates the strength of the current transmitted data signal. The general home environment or conference room environment range is [-100dBm, -35dBm], and its range is related to the wireless receiving antenna and receiving environment. Affect the quality of the wireless transmission network, which can be directly obtained from the wireless chip network driver. The send buffer margin represents the speed of the transmission and the amount of data transmitted by the upper layer. The data range is expressed as a percentage, usually [0.1, 1], and can also be directly obtained from the wireless chip network driver. Their weights are: 802.11 frame retransmissions per second > packet loss per second > received signal strength indication > send buffer margin. Judging the quality of the wireless transmission network according to their weights, and obtaining the parameters that best represent the wireless chip network driver.

As an implementation manner, performing parameter adjustment related to the video encoding end according to the network detection parameter includes the following steps:

A predetermined threshold level that is satisfied in the threshold table of the fifth input is determined by the first input, the second input, and the third input.

The parameters and display resolution of the video encoder are adjusted according to a predetermined preset threshold level.

The first input represents the internal transmission statistical parameter of the wireless chip network driver, which can effectively measure the quality of the current wireless transmission network; the second input represents the total data buffer usage size after the streaming media package; the third input represents The available memory of the current system; calculating a parameter value indicating the status of the transmission network according to the weight ratio of the first input, the second input, and the third input, and comparing it with a preset threshold range stored in the threshold table, and determining the threshold value table The preset threshold level is met, and the strategy that affects the minimum user experience is selected, such as reducing the frame rate, adjusting the code rate, and dynamically adjusting the display resolution. Among them, the parameters of the video encoder include a frame rate and a code rate. The adjustment priority of the frame rate, code rate, and display resolution is: frame rate > code rate > display resolution.

The specific process of adjusting the parameters related to the video coding end according to the network detection parameters is described below in conjunction with a specific embodiment:

Assume that the optional frame rate includes 60fps, 30fps and 24fps. The optional bit rate includes 5Mbps, 3Mbps and 1Mbps. The supported display resolutions include 1080p, 720p and 480p, and the preset threshold level can be divided into up to 27 levels. The division of the level is also defined according to the specific product specifications. For example, some products can only support 60fps/5Mbps/1080p at most, and the preset threshold level corresponding to the product can be divided into nine, wherein: the first preset threshold level corresponds to The output is 60fps/5Mbps/1080p, the output corresponding to the second preset threshold level is 30fps/5Mbps/1080p, and the output corresponding to the third preset threshold level is 24fps/5Mbps/1080p, and the output corresponding to the fourth preset threshold level is 60fps/3Mbps/1080p, the output corresponding to the fifth preset threshold level is 30fps/3Mbps/1080p, the output corresponding to the sixth preset threshold level is 24fps/3Mbps/1080p, and the output corresponding to the seventh preset threshold level is 60fps/ 1Mbps/1080p, the output corresponding to the eighth preset threshold level is 30fps/1Mbps/1080p, and the output corresponding to the ninth preset threshold level is 24fps/1Mbps/1080p. Assuming that the preset threshold value stored in the threshold table is (0, 1), the range of the corresponding first preset threshold level may be (0, 0.1), and the range of the second preset threshold level is (0.1, 0.2], and so on, the range of the ninth preset threshold level is (0.9, 1).

In one embodiment, the first input has a value of 30, the weight ratio is 70%, the second input has a value of 20, the weight ratio is 20%, the third input has a value of 10, and the weight ratio is 10%. The input parameter value indicating the status of the transmission network is normalized to 0.26, which satisfies the third preset threshold level, and the output corresponding to the third preset threshold level is 24fps/5Mbps/1080p, thereby adjusting the frame of the video encoder. Rate, bit rate and display resolution.

As an implementable manner, the method further includes the following steps:

Real-time detection of the video stream data output by the video encoder is a flag of a P frame or a B frame as a fourth input.

The parameter adjustment of the wireless chip network driver according to the network detection parameter includes the following steps:

According to the flag bit, the scanning mode of the wireless chip network driver is passive scanning, and the number of retransmissions of the P frame is set to be three times the number of retransmissions of the B frame.

Real-time detection to obtain the differentiated video stream data output by the video encoder as the flag of the P frame or the B frame, and after obtaining the flag bit, feedback to the wireless chip network driver, and the wireless chip network driver identifies which video stream data is P according to the flag bit. Frame, which video stream data is B frame, and set the number of retransmissions of the P frame to be 3 times of the number of retransmissions of the B frame, to ensure the transmission quality of the key frame, thereby ensuring the quality of the video. When transmitting the real-time video stream, the scanning mode of the wireless chip network driver is set to passive scanning, and the passive scanning time is three times of the normal scanning time, which reduces the resource consumption caused by the scanning.

Based on the same inventive concept, the present invention further provides a mobile device video stream transmission control system. Since the principle of solving the problem of the system is similar to the foregoing video stream transmission control method of the mobile device, the implementation of the system can be referred to the foregoing method. Implementation, repetition will not be repeated.

The video stream transmission control system of the mobile device provided by the embodiment of the present invention, as shown in FIG. 3, includes a real-time transmission processing module 100 configured to detect network detection parameters of the transmission network in real time during the video stream transmission process, and detect parameters according to the network. The video and network transmission transmission parameters are re-processed; wherein the adjustment processing of the video and network transmission parameters includes parameter adjustment related to the video coding end and parameter adjustment of the wireless chip network driver.

The mobile device video stream transmission control system of the present invention adds a feedback link at each link of the transmission network. Real-time decision making provides a good user experience, reduces latency, and guarantees reliable picture quality.

As an implementation manner, the real-time transmission processing module 100 includes a first input unit 101, a second input unit 102, a third input unit 103, and a fifth input unit 105, where:

The first input unit 101 is configured to detect an internal transmission statistical parameter of the wireless chip network driver in real time.

The second input unit 102 is configured to detect the total data buffer usage size after the current streaming media package in real time.

The third input unit 103 is configured to detect the available memory of the current system in real time.

The fifth input unit 105 is configured to store a parameter adjustment of the decision video encoder and a preset threshold level of the display resolution adjustment.

As an implementation manner, the real-time transmission processing module 100 further includes a determining unit 110 and a first adjusting unit 120, where:

The determining unit 110 is configured to determine an internal transmission statistical parameter, a total data buffer usage size, and a preset threshold level that is satisfied in the threshold table by the available memory of the current system.

The first adjusting unit 120 is configured to adjust parameters and display resolution of the video encoder according to the preset preset threshold level.

As an implementation manner, the fourth input unit 104 and the second adjustment unit 130 are further included, wherein:

The fourth input unit 104 is configured to detect in real time that the differentiated video stream data output by the video encoder is a flag bit of a P frame or a B frame.

The second adjusting unit 130 is configured to set the scanning mode of the wireless chip network driving to passive scanning according to the flag bit, and set the number of retransmissions of the P frame to be three times the number of retransmissions of the B frame.

The above-mentioned embodiments are merely illustrative of several embodiments of the present invention, and the description thereof is more specific and detailed, but is not to be construed as limiting the scope of the invention. It should be noted that a number of variations and modifications may be made by those skilled in the art without departing from the spirit and scope of the invention. Therefore, the scope of the invention should be determined by the appended claims.

Claims

A mobile device video stream transmission control method, comprising the following steps:

During the video stream transmission process, the network detection parameters of the transmission network are detected in real time;

Re-adjusting the video and network transmission parameters according to the network detection parameters;

The adjustment processing of the video and network transmission parameters includes parameter adjustment related to the video encoding end and parameter adjustment of the wireless chip network driver.
The mobile device video stream transmission control method according to claim 1, wherein the real-time detecting a network detection parameter of the transmission network comprises the following steps:

Real-time detection of internal transmission statistical parameters of the wireless chip network driver as a first input;

Real-time detection of the total data buffer usage size of the current streaming media package as a second input;

Real-time detection of available memory of the current system as a third input;

The threshold value table stores a parameter adjustment of the decision video encoder and a preset threshold level of the display resolution adjustment as a fifth input;

The weight ratio of the first input > the weight ratio of the second input > the weight ratio of the third input.
The mobile device video stream transmission control method according to claim 2, wherein the internal transmission statistical parameters of the wireless chip network driver include 802.11 frame retransmission times per second, data packet loss per second, and received signals. Intensity indication and send buffer margin.
The mobile device video stream transmission control method according to claim 2, wherein the parameter adjustment related to the video encoding end is performed according to the network detection parameter, comprising the following steps:

Determining, by the first input, the second input, and the third input, a preset threshold level that is satisfied in a threshold table of the fifth input;

The parameters and display resolution of the video encoder are adjusted according to the predetermined threshold level that is satisfied.
The mobile device video stream transmission control method according to claim 4, further comprising the following steps:

Real-time detecting that the differentiated video stream data output by the video encoder is a flag bit of a P frame or a B frame, as a fourth input;

Performing parameter adjustment of the wireless chip network driver according to the network detection parameter includes the following steps:

According to the flag bit, the scanning mode of the wireless chip network driving is passive scanning, and the number of retransmissions of the P frame is set to be three times of the number of retransmissions of the B frame.
A mobile device video stream transmission control system, comprising: a real-time transmission processing module configured to detect a network detection parameter of a transmission network in real time during a video stream transmission process, and re-pair the video and the network according to the network detection parameter Transfer parameters for adjustment processing;

The adjustment processing of the video and network transmission parameters includes parameter adjustment related to the video encoding end and parameter adjustment of the wireless chip network driver.
The mobile device video stream transmission control system according to claim 6, wherein the real-time transmission processing module comprises a first input unit, a second input unit, a third input unit and a fifth input unit, wherein:

The first input unit is configured to detect an internal transmission statistical parameter of the wireless chip network driver in real time;

The second input unit is configured to detect a total data buffer usage size after the current streaming media package in real time;

The third input unit is configured to detect available memory of the current system in real time;

The fifth input unit is configured to store a preset threshold level of parameter adjustment and display resolution adjustment of the decision video encoder;

The weight ratio of the first input unit > the weight ratio of the second input unit > the weight ratio of the third input unit.
The mobile device video stream transmission control system according to claim 7, wherein the internal transmission statistical parameters of the wireless chip network driver include 802.11 frame retransmission times per second, data packet loss per second, and received signals. Intensity indication and send buffer margin.
The mobile device video stream transmission control system according to claim 7, wherein the real-time transmission processing module further comprises a determining unit and a first adjusting unit, wherein:

The determining unit is configured to determine the internal transmission statistical parameter, the total data buffer usage size, and a preset threshold level that is satisfied by the available memory of the current system in the threshold table;

The first adjusting unit is configured to adjust a parameter and a display resolution of the video encoder according to the preset threshold level that is satisfied.
The mobile device video stream transmission control system according to claim 9, further comprising a fourth input unit and a second adjustment unit, wherein:

The fourth input unit is configured to detect, in real time, the differentiated video stream data output by the video encoder as a flag of a P frame or a B frame;

The second adjusting unit is configured to set the scanning mode of the wireless chip network driving to passive scanning according to the flag bit, and set the number of retransmissions of the P frame to be three times of the number of retransmissions of the B frame.