CN112565833A - HLS-based multimedia adaptive play control method - Google Patents
HLS-based multimedia adaptive play control method Download PDFInfo
- Publication number
- CN112565833A CN112565833A CN202011346190.4A CN202011346190A CN112565833A CN 112565833 A CN112565833 A CN 112565833A CN 202011346190 A CN202011346190 A CN 202011346190A CN 112565833 A CN112565833 A CN 112565833A
- Authority
- CN
- China
- Prior art keywords
- speed
- network
- network speed
- data packets
- control method
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/266—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
- H04N21/2662—Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
- H04N21/47202—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for requesting content on demand, e.g. video on demand
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/643—Communication protocols
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/647—Control signaling between network components and server or clients; Network processes for video distribution between server and clients, e.g. controlling the quality of the video stream, by dropping packets, protecting content from unauthorised alteration within the network, monitoring of network load, bridging between two different networks, e.g. between IP and wireless
- H04N21/64723—Monitoring of network processes or resources, e.g. monitoring of network load
- H04N21/64738—Monitoring network characteristics, e.g. bandwidth, congestion level
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
- H04N21/845—Structuring of content, e.g. decomposing content into time segments
- H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments
Landscapes
- Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Computer Networks & Wireless Communication (AREA)
- Databases & Information Systems (AREA)
- Computer Security & Cryptography (AREA)
- Human Computer Interaction (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Communication Control (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a multimedia self-adaptive play control method based on HLS, which can effectively distinguish the type of network speed change by comparing instant speed information, average speed information and m3u8 configuration information, and make a selection according to the type of network speed change.
Description
Technical Field
The invention relates to a multimedia self-adaptive play control method based on HLS, belonging to the field of multimedia self-adaptive play control.
Background
The HLS is a streaming media transmission protocol based on HTTP and realized by Apple, can realize live broadcast and on-demand broadcast of streaming media, and provides an audio/video live broadcast and on-demand broadcast scheme for mobile equipment.
The HLS protocol converts the media data stream into continuous media files (MPEG-TS format, generally each small file can be played for 8 seconds, called fragments) of different quality and very short duration at the server, and the client continuously downloads and plays the fragments.
Generally, for the same program, a server provides multiple audio/video streams with different qualities (each quality audio/video stream includes a group of fragments corresponding to a code rate), and gives information of the multiple audio/video streams (including the code rate of each audio/video stream and the acquisition mode of the included fragments) through a configuration file in an m3u8 format. The 7 audio and video stream information in the configuration file are as follows:
each audio and video stream consists of a plurality of fragments, and the fragments are in a TS format and contain 8 seconds of audio and video;
audio and video stream 00: 40Kbps has audio stream, no video stream and 1 JPEG picture;
audio-video stream 01: 120Kbps has audio stream, has video stream (video stream resolution 176x 144);
audio and video stream 02: 300Kbps has an audio stream and a video stream (video stream resolution 320x 180);
audio and video stream 03: 500Kbps has an audio stream and a video stream (video stream resolution 400x 224);
audio and video stream 04: 800Kbps has audio stream and video stream (video stream resolution 480x 270);
audio and video stream 05: 1300Kbps has audio stream and video stream (video stream resolution 640x 360);
audio and video stream 06: 2200Kbps has audio stream and video stream (video stream resolution 1024x 576).
Because the data is transmitted by the HTTP protocol, the problems of firewalls or proxies are not considered at all, the playable time of the fragmented files is short, and the client can select and switch to the corresponding code rate at any time so as to adapt to the playing under different bandwidth conditions.
When downloading audio and video stream, we use the fragment as the most basic playing unit, and for one fragment, there are only two options: either completely downloaded and played or not downloaded or discarded after a portion is downloaded.
For a fragment, we generally divide a fragment into several (≧ 5) data packets for downloading. Each time the data packet is downloaded, we can obtain the downloaded information (the number of bytes downloaded and the download time), and we can calculate the network speed by the downloaded information.
At the mobile phone terminal, the HLS client can be used for watching programs on the server. The HLS client side firstly receives the configuration file in the m3u8 format, then calculates the current network speed, and downloads the audio and video stream with the corresponding bandwidth according to the network speed in real time so as to achieve the best user experience.
The method for calculating the current network speed comprises the following steps: and obtaining the current network average speed according to the download time of 50 data packet byte sizes/50 data packets which have been downloaded recently. Usually, one fragment is less than 50 packets, so that at least one fragment needs to be downloaded before the current average network speed can be measured.
When the network speed fluctuates greatly, the original calculation method obtains the average speed of 50 packet networks, so that the switching to the corresponding code stream cannot be selected in time according to the changed network speed, which results in a longer pause time (when the network speed is decreased rapidly) or a longer time for entering the optimal playing state (when the network speed is increased rapidly).
Disclosure of Invention
The invention provides a multimedia self-adaptive playing control method based on HLS, aiming at solving the problem that the playing time of HLS media stream is longer than pause time or the time of playing in the best playing state is longer when the network speed fluctuates greatly in the existing algorithm, and the method can effectively distinguish the type of network speed change and make selection according to the type of the network speed change by comparing instant speed information, average speed information and m3u8 configuration information.
The invention adopts the following technical scheme for solving the technical problems:
a multimedia self-adaptive playing control method based on HLS comprises the following specific steps:
step one, acquiring the real-time speed and the average speed of a network;
step two, judging whether the current network speed is in a steep rise or a steep fall according to the instant speed and the average speed;
and step three, if the current network speed is in a steep rise or a steep fall, taking the average speed of the instant speed, and selecting the corresponding flow according to the average speed of the instant speed and the configuration of m3u 8.
Further, in the first step, according to the download information of the M data packets downloaded recently, the instant speed of the network is determined:
instantaneous velocity VmThe downloading speed of the mth data packet is equal to the byte number of the mth data packet/the downloading time of the mth data packet;
according to the download information of the N data packets downloaded recently, calculating the average speed:
average velocity VNThe total byte number of N data packets/the total downloading time of N data packets;
the download information includes the number of bytes downloaded and the time required for downloading, and M is 1,2, …, and M, M and N are set as required.
Further, in the step two:
if V1、V2、…、VMAre all greater than b Va or b VNIf so, the network speed is increased suddenly at present;
if V1、V2、…、VMAre all less than a Va or a VNIf so, the network speed is decreased suddenly;
wherein, a and b are respectively the threshold values of network speed dropping and rising, and the bandwidth Va of the current audio/video stream is the code rate corresponding to the currently downloaded audio/video stream.
Further, the value of the network speed steep drop threshold a is 0.6, and the value of the network speed steep rise threshold b is 2.0.
Further, the average speed of the instant speed is the total byte number of the M data packets/the total downloading time of the M data packets
Compared with the prior art, the invention adopting the technical scheme has the following technical effects:
1. the fluctuations in wire speed are summarized into the following 5 types: the network speed is jittered, the network speed is slowly increased, the network speed is slowly decreased, the network speed is suddenly increased, and the network speed is suddenly decreased;
2. distinguishing 5 types of network speed variation by comparing the instant speed information, the average speed information and the m3u8 configuration information with each other, and selecting according to the type of the network speed variation;
3. through 2 types of optimization processing of the great and sudden change of the network speed, the user experience is obviously improved.
Drawings
FIG. 1 is a flow chart of the present invention;
FIGS. 2 to 6 are schematic diagrams of a network speed flutter, a network speed slow-up, a network speed slow-down, a network speed steep-up and a network speed steep-down, respectively;
FIG. 7 shows the test data of HLS network streaming audio and video program on demand under different network speed variation conditions.
Detailed Description
According to the multimedia adaptive play control method based on HLS, disclosed by the invention, as shown in fig. 1, by comparing the instant speed information, the average speed information and the m3u8 configuration information, the type of network speed change can be effectively distinguished, and selection is made according to the type of network speed change.
First step, obtaining the instantaneous speed and average speed of network
Specifically, the download information of M data packets which are downloaded most recently (the download information includes the number of bytes downloaded and the time required for downloading) is stored, and the download speed of each packet of the M data packets is calculated. The download information for the last period of the data packet (N) is stored to calculate the average speed.
For example, the value of m is 3, the value of n is 50, that is, the instant speed is calculated by the download information of the last 3 data packets, and the average speed is calculated by the download information of the last 50 data packets. Wherein, the instant speed can reflect the current download speed, and the average speed reflects the download speed in a period of time. To achieve the instantaneous and average speed of the computing network, the system needs to maintain the downloaded information of several data packets at all times.
And secondly, judging whether the current network speed is in a steep rise or a steep fall according to the instant speed and the average speed.
Instantaneous velocity V21 st data packet byte number which is downloaded most recently/corresponding data packet downloading time;
instantaneous velocity V2The number of bytes of the 2 nd data packet downloaded most recently/the downloading time of the corresponding data packet;
instantaneous velocity V3The number of bytes of the 3 rd data packet downloaded most recently/the downloading time of the corresponding data packet;
average velocity V50The total byte number of the 50 data packets which are downloaded recently/the total downloading time of the corresponding data packets;
and the current audio and video stream bandwidth Va is the code rate corresponding to the currently downloaded audio and video stream.
If V1、V2、V3Are all greater than b Va or b VNIf so, the network speed is increased suddenly at present;
if V1、V2、V3Are all less than a Va or a VNThen the network speed is decreased suddenly.
Wherein, a is a network speed steep drop threshold (value is 0.6), and b is a network speed steep rise threshold (value is 2.0).
And thirdly, if the current network speed is steep rising or steep falling, taking the average speed of the instant speed, and selecting the corresponding flow according to the average speed of the instant speed and the configuration of m3u 8.
The average speed of the instant speed is the latest downloaded 3 number of bytes of data packets/corresponding 3 data packet download time instances: the configuration of the m3u8 configuration is as the 7 audio and video stream information in the previous configuration file
The currently played audio and video stream is 02300 Kbps
VN＝350KBps
V1＝150KBps；V2＝160KBps；V3＝140KBps；
Discovery V1、V2、V3Are all less than a VNThis is considered a sharp drop in the network speed, and then the average speed of 150KBps is calculated to get the instantaneous speed, switching to stream 01 depending on the network speed required by the different streams in the m3u8 configuration.
The effect after the network speed steep rise condition is improved is as follows:
through the test comparison of 3 groups, the time required by self-adaptation is reduced by 50-80%, and the average reduced self-adaptation time is 70% of the original total time.
The effect after the network speed is steeply reduced and the condition is improved is as follows:
for the original algorithm, the original user experience is not acceptable at all, and in some extreme tests (the network speed is reduced in a large proportion), the audio and video can be captured for more than 1 minute.
Through the test comparison of 3 groups, the audio and video can be averaged for more than 20 seconds by the original algorithm; after the improved method is adopted, the situation of no card basically occurs, and the card is clamped for 1-3 seconds in a few cases, and is within an acceptable range.
The present invention generalizes the fluctuations in wire speed into 5 types as shown in fig. 2 to 6: the network speed is jittered, the network speed is slowly increased, the network speed is slowly decreased, the network speed is suddenly increased, and the network speed is suddenly decreased; distinguishing 5 types of network speed variation by comparing the instant speed information, the average speed information and the m3u8 configuration information with each other, and selecting according to the type of the network speed variation; through 2 types of optimization processing of the great and sudden change of the network speed, the user experience is improved.
Fig. 7 shows the specific test data of the HLS network streaming audio/video program on demand under different network speed changes.
It should be noted that the above description of the embodiments is only for the purpose of assisting understanding of the method of the present application and the core idea thereof, and that those skilled in the art can make several improvements and modifications to the present application without departing from the principle of the present application, and these improvements and modifications are also within the protection scope of the claims of the present application.
Claims (5)
1. A multimedia adaptive playing control method based on HLS is characterized by comprising the following specific steps:
step one, acquiring the real-time speed and the average speed of a network;
step two, judging whether the current network speed is in a steep rise or a steep fall according to the instant speed and the average speed;
and step three, if the current network speed is in a steep rise or a steep fall, taking the average speed of the instant speed, and selecting the corresponding flow according to the average speed of the instant speed and the configuration of m3u 8.
2. The adaptive multimedia playback control method according to claim 1, wherein in step one, the instantaneous speed of the network is determined according to the download information of the most recently downloaded M data packets:
instantaneous velocity VmThe downloading speed of the mth data packet is equal to the byte number of the mth data packet/the downloading time of the mth data packet;
according to the download information of the N data packets downloaded recently, calculating the average speed:
average velocity VNThe total byte number of N data packets/the total downloading time of N data packets;
the download information includes the number of bytes downloaded and the time required for downloading, and M is 1,2, …, and M, M and N are set as required.
3. The adaptive multimedia playback control method according to claim 1, wherein in step two:
if V1、V2、…、VMAre all greater than b Va or b VNIf so, the network speed is increased suddenly at present;
if V1、V2、…、VMAre all less than a Va or a VNIf so, the network speed is decreased suddenly;
wherein, a and b are respectively the threshold values of network speed dropping and rising, and the bandwidth Va of the current audio/video stream is the code rate corresponding to the currently downloaded audio/video stream.
4. The adaptive multimedia play control method according to claim 3, wherein the value of the network speed steepness decreasing threshold a is 0.6, and the value of the network speed steepness increasing threshold b is 2.0.
5. The adaptive multimedia playback control method based on HLS as claimed in claim 1, wherein the average speed of the instant speed is equal to the total number of bytes of M data packets/total time of downloading M data packets.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011346190.4A CN112565833A (en) | 2020-11-26 | 2020-11-26 | HLS-based multimedia adaptive play control method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011346190.4A CN112565833A (en) | 2020-11-26 | 2020-11-26 | HLS-based multimedia adaptive play control method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112565833A true CN112565833A (en) | 2021-03-26 |
Family
ID=75045663
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011346190.4A Pending CN112565833A (en) | 2020-11-26 | 2020-11-26 | HLS-based multimedia adaptive play control method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112565833A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103152651A (en) * | 2013-01-31 | 2013-06-12 | 广东欧珀移动通信有限公司 | Method and system of automatically adjusting play threshold of streaming media buffering area |
| CN106210929A (en) * | 2016-07-13 | 2016-12-07 | 诚迈科技（南京）股份有限公司 | Network flow download terminal based on HLS client and optimization method |
| CN108124192A (en) * | 2016-11-30 | 2018-06-05 | 亦非云互联网技术（上海）有限公司 | The playback method and system of HLS Streaming Medias |
| CN109348265A (en) * | 2018-11-27 | 2019-02-15 | 湖南国科微电子股份有限公司 | Based on the live broadcasting method and system under HLS multi-code stream mode |
| CN110121100A (en) * | 2019-05-28 | 2019-08-13 | 平安科技（深圳）有限公司 | Method for down loading, device, terminal and the computer readable storage medium of media file |
-
2020
- 2020-11-26 CN CN202011346190.4A patent/CN112565833A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103152651A (en) * | 2013-01-31 | 2013-06-12 | 广东欧珀移动通信有限公司 | Method and system of automatically adjusting play threshold of streaming media buffering area |
| CN106210929A (en) * | 2016-07-13 | 2016-12-07 | 诚迈科技（南京）股份有限公司 | Network flow download terminal based on HLS client and optimization method |
| CN108124192A (en) * | 2016-11-30 | 2018-06-05 | 亦非云互联网技术（上海）有限公司 | The playback method and system of HLS Streaming Medias |
| CN109348265A (en) * | 2018-11-27 | 2019-02-15 | 湖南国科微电子股份有限公司 | Based on the live broadcasting method and system under HLS multi-code stream mode |
| CN110121100A (en) * | 2019-05-28 | 2019-08-13 | 平安科技（深圳）有限公司 | Method for down loading, device, terminal and the computer readable storage medium of media file |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11076187B2 (en) | Systems and methods for performing quality based streaming | |
| US10298985B2 (en) | Systems and methods for performing quality based streaming | |
| CN107147919B (en) | Live broadcast quick starting method and system | |
| US7779146B2 (en) | Methods and systems for HTTP streaming using server-side pacing | |
| KR100486713B1 (en) | Apparatus and method for streaming multimedia data | |
| US20080133766A1 (en) | Method and apparatus for streaming media to a plurality of adaptive client devices | |
| CN106803968B (en) | Method for adaptively adjusting video resolution based on bandwidth of HLS protocol | |
| CN110121100B (en) | Method, device, terminal and computer readable storage medium for downloading media file | |
| JP5147950B2 (en) | Apparatus and method for performing simulcast over variable bandwidth channel | |
| EP2437458A1 (en) | Content delivery | |
| MX2013013373A (en) | Method for dynamic adaptation of the reception bitrate and associated receiver. | |
| CN108124192B (en) | HLS streaming media playing method and system | |
| CN109561347B (en) | Internet video playing quality judgment method and system | |
| KR20170018333A (en) | Method for operating a network equipment arranged along a transmission path between a client terminal and at least one server, and corresponding network equipment | |
| KR101702426B1 (en) | Video transmission method based on multi HTTP threads for reducing the viewpoint change delay in multi-view video service | |
| CN112565833A (en) | HLS-based multimedia adaptive play control method | |
| WO2009053595A1 (en) | Device for the continuous reception of audio and/or video data packets | |
| CN110881018B (en) | Real-time receiving method and client of media stream | |
| EP3664456A1 (en) | Apparatus and method for playing streamed media | |
| CN115336283B (en) | Client device and method for streaming media content to client device | |
| KR20190048186A (en) | Multipath-based split transmission system and streaming method for adaptive streaming service | |
| WO2017084772A1 (en) | Delivery of media content segments in a content delivery network | |
| WO2021167531A1 (en) | Methods and systems for bandwidth estimation | |
| JP2022542767A (en) | Latency management for content delivery | |
| CN106559684B (en) | Method, terminal and system for reducing live broadcast delay |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |