TWI736180B - System and method for stabilizing network video delivery delay - Google Patents
System and method for stabilizing network video delivery delay Download PDFInfo
- Publication number
- TWI736180B TWI736180B TW109108969A TW109108969A TWI736180B TW I736180 B TWI736180 B TW I736180B TW 109108969 A TW109108969 A TW 109108969A TW 109108969 A TW109108969 A TW 109108969A TW I736180 B TWI736180 B TW I736180B
- Authority
- TW
- Taiwan
- Prior art keywords
- frame
- client
- delay
- server
- quality
- Prior art date
Links
Images
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
Description
本發明係關於一種網路影音遞送技術,特別是指一種穩定網路影音遞送時延之系統及方法。 The present invention relates to a network video and audio delivery technology, in particular to a system and method for stabilizing network video and audio delivery delay.
隨著第五代(5G)通訊或更先進網路之快速發展,智慧手機、智慧手錶或平板電腦等亦日益普及,使得用戶端對於穩定網路影音遞送時延之要求也越來越高。 With the rapid development of fifth-generation (5G) communications or more advanced networks, smart phones, smart watches, or tablets have become increasingly popular, making the client's requirements for stable network audio and video delivery delays higher and higher.
在一現有技術中,提出一種視頻分發場景下的低延時取流秒開方法,包括由視頻直播場景處理模組於實現視頻直播場景下之低延時取流秒開功能,並由視頻轉發場景處理模組於實現視頻轉發下之低延時取流秒開功能,且對顯示時間戳(Presentation Timestamp;PTS)與解碼時間戳(Decode Timestamp;DTS)進行修正處理,再由視頻點播場景處理模組於實現視頻點播下之低延時取流秒開功能。惟,此現有技術未考量到有關影片群組(Group Of Pictures;GOP)之遞送效率、用戶端之解碼效能、網路狀態大幅度切換等問題,故不易達成用戶端對於穩定網路影音遞送時延之要求。 In an existing technology, a low-latency streaming access method in a video distribution scene is proposed, which includes a video live scene processing module to realize the low-latency streaming access function in a live video scene, and is processed by the video forwarding scene. The module realizes the low-latency streaming function in seconds under video forwarding, and corrects the presentation timestamp (PTS) and decoding timestamp (DTS), and then the video-on-demand scene processing module Realize the low-latency streaming function in seconds under video-on-demand. However, this prior art does not take into account the delivery efficiency of the group of pictures (GOP), the decoding performance of the client, and the large switching of the network status. Therefore, it is not easy to achieve the stability of the client’s ability to deliver online video and audio. Extension of the request.
因此,如何提供一種新穎且創新之穩定網路影音遞送時延之技術,以達成用戶端對於穩定網路影音遞送時延之要求,實已成為本領域技術人員之一大研究課題。 Therefore, how to provide a novel and innovative technology for stabilizing the network audio and video delivery delay to meet the requirements of the client for stable network audio and video delivery delay has become a major research topic for those skilled in the art.
本發明提供一種新穎且創新之穩定網路影音遞送時延之系統及方法,例如能達成用戶端對於穩定網路影音遞送時延之要求,或者穩定服務端與用戶端之間的播放時延,抑或者確保用戶端在即時影音之觀看過程中處於穩定時延之狀態。 The present invention provides a novel and innovative system and method for stabilizing the network video and audio delivery delay, for example, it can meet the requirements of the client to stabilize the network video and audio delivery delay, or to stabilize the playback delay between the server and the client. Or to ensure that the user terminal is in a stable state of delay during the viewing of real-time video and audio.
本發明中穩定網路影音遞送時延之系統包括:一服務端,係透過網路遞送複數影格封包與一多品質編碼列表;以及一用戶端,係透過網路接收來自服務端之影格封包與多品質編碼列表。同時,用戶端具有:一探勘器,係探勘用戶端與服務端之間的網路狀態;一溝通器,係基於探勘器所探勘之用戶端與服務端之間的網路狀態,向服務端發出依據多品質編碼列表來動態調整影格封包之影格品質之請求;一時延累加器,係在用以判別用戶端與服務端之間的時延的最大時延影片群組索引超過門檻值時,增加時延累加器所計算之次數,以在時延累加器所計算之次數超過最大時延次數時,由用戶端啟動或執行有關品質異動事件中之延遲事件而降低影格封包之影格品質;以及一良好累加器,係當用戶端於週期時間之檢測未發生有關品質異動事件中之延遲事件時,視為發生良好判別,且每當發生良好判別,增加良好累加器所計算之次數,以在良好累加器所計算之次數超過最大良好次數時,由用戶端啟動或執行有關品質異動事件中之良好事 件而提高影格封包之影格品質。 The system for stabilizing the delay of network video and audio delivery in the present invention includes: a server, which delivers multiple frame packets and a multi-quality code list through the network; and a client, which receives frame packets and packets from the server through the network Multi-quality encoding list. At the same time, the client has: a prospecting device to survey the network status between the client and the server; a communicator, based on the network status between the client and the server surveyed by the prospecting device, to the server Send a request to dynamically adjust the frame quality of the frame package according to the multi-quality code list; a delay accumulator is used to determine the delay between the client and the server when the maximum delay video group index exceeds the threshold, Increase the number of times calculated by the delay accumulator so that when the number of times calculated by the delay accumulator exceeds the maximum number of delays, the client initiates or executes the delay event in the quality change event to reduce the frame quality of the frame packet; and A good accumulator means that when a delay event in the quality change event does not occur during the cycle time detection of the user terminal, it is deemed that a good judgment has occurred, and every time a good judgment occurs, the number of times the good accumulator is calculated is increased. When the number of times calculated by the good accumulator exceeds the maximum number of good times, the client initiates or executes the good things in the quality change event To improve the frame quality of the frame package.
本發明中穩定網路影音遞送時延之方法包括:由一服務端透過網路遞送複數影格封包與一多品質編碼列表;由一用戶端透過網路接收來自服務端之影格封包與多品質編碼列表;由一探勘器探勘用戶端與服務端之間的網路狀態;由一溝通器基於探勘器所探勘之用戶端與服務端之間的網路狀態,向服務端發出依據多品質編碼列表來動態調整影格封包之影格品質之請求;當用以判別用戶端與服務端之間的時延的最大時延影片群組索引超過門檻值時,增加一時延累加器所計算之次數,以在時延累加器所計算之次數超過最大時延次數時,由用戶端啟動或執行有關品質異動事件中之延遲事件而降低影格封包之影格品質;以及當用戶端於週期時間之檢測未發生有關品質異動事件中之延遲事件時,視為發生良好判別,且每當發生良好判別,增加一良好累加器所計算之次數,以在良好累加器所計算之次數超過最大良好次數時,由用戶端啟動或執行有關品質異動事件中之良好事件而提高影格封包之影格品質。 The method for stabilizing the delay of network video and audio delivery in the present invention includes: a server sends multiple frame packets and a multi-quality code list through the network; and a client receives a frame packet and multi-quality code from the server through the network List; a surveyor surveys the network status between the client and the server; a communicator sends a multi-quality coding list to the server based on the network status between the client and the server surveyed by the surveyor To dynamically adjust the frame quality of the frame package; when the maximum delay video group index used to determine the delay between the client and the server exceeds the threshold, add a delay accumulator to calculate the number of times When the number of times calculated by the delay accumulator exceeds the maximum number of delays, the client initiates or executes the delay event in the quality change event to reduce the frame quality of the frame packet; and when the client detects the cycle time without the relevant quality When a delay event in an abnormal event occurs, it is considered that a good judgment has occurred, and every time a good judgment occurs, the number of times calculated by a good accumulator is increased, so that when the number of times calculated by the good accumulator exceeds the maximum number of good times, the client starts Or implement good events in the quality change events to improve the frame quality of the frame package.
為讓本發明之上述特徵和優點能更明顯易懂,下文特舉實施例,並配合所附圖式作詳細說明。在以下描述內容中將部分闡述本發明之額外特徵及優點,且此等特徵及優點將部分自所述描述內容可得而知,或可藉由對本發明之實踐習得。應理解,前文一般描述與以下詳細描述二者均僅為例示性及解釋性的,且不欲約束本發明所欲主張之範圍。 In order to make the above-mentioned features and advantages of the present invention more comprehensible, embodiments are specifically described below in conjunction with the accompanying drawings. In the following description, the additional features and advantages of the present invention will be partially explained, and these features and advantages will be partly known from the description, or can be learned by practicing the present invention. It should be understood that both the foregoing general description and the following detailed description are only illustrative and explanatory, and are not intended to limit the scope of the present invention.
1:穩定網路影音遞送時延之系統 1: A system that stabilizes the delay of network audio and video delivery
10:服務端 10: Server
11:來源器 11: Source device
12:編碼器 12: encoder
13:發送器 13: transmitter
14:溝通器 14: Communicator
20:用戶端 20: client
21:呈現器 21: renderer
22:解碼器 22: Decoder
23:接收器 23: receiver
24:溝通器 24: Communicator
25:探勘器 25: Prospector
26:播放器 26: Player
A、B、C、D、D1、D2、P:程序 A, B, C, D, D1, D2, P: program
A1至A3、B1至B3、P1至P6:步驟 A1 to A3, B1 to B3, P1 to P6: steps
DC:時延累加器 DC: Delay accumulator
DE1:第一延遲事件標註器 DE1: The first delayed event tagger
DE2:第二延遲事件標註器 DE2: The second delayed event tagger
DGIdx:用戶端與服務端之間的時延、時延影片群組索引 DGIdx: Latency between the client and the server, and the index of the latency video group
F1至F4:已編號之影格封包 F1 to F4: Numbered Frame Pack
FC:良好累加器 FC: good accumulator
GIdx:影片群組索引 GIdx: Video group index
MDC:最大時延累加器 MDC: Maximum delay accumulator
MDGIdx:最大時延影片群組索引 MDGIdx: Maximum Delay Video Group Index
MFC:最大良好累加器 MFC: Maximum good accumulator
MNBC:最大無緩累加器 MNBC: The largest accumulator without slowing
MPC:最大拉取累加器 MPC: Maximum pull accumulator
NBC:無緩累加器 NBC: No slow accumulator
NGIdx:最新影片群組索引 NGIdx: latest video group index
PC:拉取累加器 PC: pull accumulator
PGIdx:正在進行播放的影片群組索引 PGIdx: Index of the video group being played
Q:影格品質 Q: Frame quality
QMax:最高影格品質 QMax: The highest frame quality
RGIdx:已經完整接收的影片群組索引 RGIdx: Index of video groups that have been completely received
S11至S13、S21至S23、S31至S41:步驟 S11 to S13, S21 to S23, S31 to S41: steps
第1圖為本發明中穩定網路影音遞送時延之系統及其運作方式的示意圖; Figure 1 is a schematic diagram of the system for stabilizing network video and audio delivery delay and its operation mode in the present invention;
第2圖為本發明第1圖之程序A中有關服務端建立服務的細部流程圖; Figure 2 is a detailed flow chart of the server establishment service in the procedure A of Figure 1 of the present invention;
第3圖為本發明第1圖之程序B中有關用戶端連入服務的細部流程圖; Figure 3 is a detailed flow chart of the client connected to the service in Procedure B of Figure 1 of the present invention;
第4圖至第5圖為本發明第1圖之程序C中有關封包遺失而主動拉取的概念示意圖,其中,第4圖為將多個已編號之影格封包按照編號依序填入用戶端之緩衝內,第5圖為封包遺失判別及主動由用戶端拉取影格封包之遺失者; Figures 4 to 5 are schematic diagrams of the concept of actively pulling the lost packets in the program C of Figure 1 of the present invention. Figure 4 shows that multiple numbered frame packets are filled into the user terminal according to their numbers. In the buffer, Figure 5 shows the packet loss judgment and the lost frame of the frame that is actively pulled by the client;
第6圖為本發明中品質動態調整之方法的流程圖,用以說明透過主動拉取封包(影格封包)所導致的品質異動; Figure 6 is a flow chart of the method for dynamic quality adjustment in the present invention to illustrate the quality change caused by actively pulling packets (frame packets);
第7圖為本發明中重設拉取累加器之機制的流程圖; Figure 7 is a flowchart of the mechanism of resetting the pull accumulator in the present invention;
第8圖為本發明中用戶端之緩衝內有關影片群組(GOP)索引之定義說明的示意圖; Figure 8 is a schematic diagram of the definition of the group of movies (GOP) index in the buffer of the client in the present invention;
第9圖為本發明中用戶端與服務端之間的時延的示意圖; Figure 9 is a schematic diagram of the time delay between the client and the server in the present invention;
第10圖與第11圖分別為本發明之第一延遲判別與第二延遲判別的示意圖; Figures 10 and 11 are schematic diagrams of the first delay determination and the second delay determination of the present invention, respectively;
第12圖為本發明第1圖之程序D1中有關延遲事件而降低品質的示意圖; Fig. 12 is a schematic diagram of quality degradation related to delay events in the procedure D1 of Fig. 1 of the present invention;
第13圖為本發明第1圖之程序D2中有關良好事件而提高品質的示意圖; Figure 13 is a schematic diagram of the improvement of quality related to good events in the procedure D2 of Figure 1 of the present invention;
第14圖為本發明第1圖之程序D中有關品質異動而調整品質的細部 流程圖;以及 Figure 14 is the details of adjusting quality for quality changes in procedure D of Figure 1 of the present invention Flow chart; and
第15圖為本發明中穩定網路影音遞送時延之方法的流程示意圖。 Figure 15 is a schematic flow chart of the method for stabilizing network video and audio delivery delay in the present invention.
以下藉由特定的具體實施形態說明本發明之實施方式,熟悉此技術之人士可由本說明書所揭示之內容了解本發明之其他優點與功效,亦可因而藉由其他不同的具體等同實施形態加以施行或應用。 The following describes the implementation of the present invention with specific specific embodiments. Those familiar with this technology can understand the other advantages and effects of the present invention from the contents disclosed in this specification, and can also implement other different specific equivalent embodiments. Or apply.
本發明中穩定網路影音遞送時延之系統及方法,可支援影片群組(GOP)之影格內容的影音壓縮內容,並建立基於用戶資料報協定(User Datagram Protocol;UDP)的遞送方式,亦可額外建立基於傳輸控制協定(Transmission Control Protocol;TCP)的溝通通道、或基於網際網路控制訊息協定(Internet Control Message Protocol;ICMP)的探勘通道;而且,上述彼此之間能依據本發明妥善交流,以穩定服務端(Server)與用戶端(Client)之間的播放時延。 The system and method for stabilizing the delay of network video and audio delivery in the present invention can support video and audio compression content of group of videos (GOP), and establish a delivery method based on User Datagram Protocol (UDP). A communication channel based on Transmission Control Protocol (TCP) or a survey channel based on Internet Control Message Protocol (ICMP) can be additionally established; moreover, the above-mentioned one can communicate with each other properly according to the present invention , In order to stabilize the playback delay between the server and the client.
本發明能透過主動查知遞送時延情形、判別網路封包遺失與否、動態調整影格品質及用戶端之緩衝(buffer)內容,以確保用戶端在即時影音之觀看過程中處於穩定時延之狀態。同時,本發明之影格品質調整能於同一播放階段中完成,使得用戶端之播放器可不需額外重新連接而中斷影音服務。因此,本發明能提升影音網路即時服務之品質。 The present invention can actively check the delivery delay situation, judge whether the network packet is lost or not, dynamically adjust the frame quality and the buffer content of the client to ensure that the client is in a stable delay during the viewing of real-time video and audio. state. At the same time, the frame quality adjustment of the present invention can be completed in the same playback stage, so that the player on the user side can interrupt the audio-visual service without additional reconnection. Therefore, the present invention can improve the quality of the real-time service of the audio-visual network.
本發明能依據網路狀態清空用戶端之緩衝,使得用戶端之播放器不再呈現時延過久之緩衝影格。又,本發明能依據用戶端之緩衝充盈與否,由用戶端動態與服務端溝通以調整影格品質(如解析度、位元速率 (Bitrate)等),以達成穩定影音遞送時延。 The present invention can clear the buffer of the client terminal according to the network status, so that the player of the client terminal no longer presents the buffered frame with a long delay. In addition, the present invention can dynamically communicate with the server to adjust the frame quality (such as resolution, bit rate, (Bitrate), etc.) in order to achieve a stable video and audio delivery delay.
本發明對網路影音遞送時延之定義,可為服務端取得一影格且送出該影格後,至用戶端呈現該影格之時間差距,其中穩定時延之需求來自用戶端之播放器於播放過程中,時延會有逐步增長之趨勢,此常肇因於用戶端之緩衝空間因網路品質抖動或延遲變動(Jitter)而自適應的調整所導致。 The present invention defines the delay of network video and audio delivery, which can be the time gap between the server obtaining a frame and sending the frame, and then presenting the frame at the user end. The demand for stable delay comes from the player of the user end during the playback process. In this case, the delay will gradually increase, which is often caused by the adaptive adjustment of the buffer space on the client side due to network quality jitter or delay variation (Jitter).
第1圖為本發明中穩定網路影音遞送時延之系統1及其運作方式的示意圖。如圖所示,穩定網路影音遞送時延之系統1包括一服務端10及至少一(如一或複數)用戶端20,服務端10可透過網路遞送複數影格封包與一多品質編碼列表(見表1),且用戶端20可透過網路接收來自服務端10之影格封包(影音封包)與多品質編碼列表。例如,服務端10為可提供服務之伺服器(如雲端伺服器/網路伺服器/遠端伺服器)、伺服主機、電腦等,用戶端20為可使用服務之用戶設備(UE)、個人電腦(如桌上型電腦/筆記型電腦)、可攜式裝置(如智慧手機/平板電腦)、穿戴式裝置(如智慧手錶)等。網路可為無線網路、有線網路、5G網路、更先進網路(如6G網路)或各種的通訊網路等。但是,本發明並不以此為限。
Fig. 1 is a schematic diagram of the
服務端10可包括一來源器11、一編碼器12、一發送器13與一溝通器14。例如,來源器11可為硬體之攝影機或照相機等,亦可為軟體之攝影程式、照相程式或影格(影音)資料擷取程式等,用以提供或取得原始(Raw)之影格資料。編碼器12可為硬體之編碼晶片(電路)、軟體之編碼程式或其組合所構成。發送器13可為硬體之發送晶片(電路)、軟體之發送程式或其組合所構成。溝通器14可為硬體之溝通晶片(電路)、軟體之溝
通程式或其組合所構成。
The
用戶端20可包括一呈現器21、一解碼器22、一接收器23、一溝通器24、一探勘器25與一播放器26。例如,呈現器21可為硬體之顯示器或投影機等,用以呈現連續影像。解碼器22可為硬體之解碼晶片(電路)、軟體之解碼程式或其組合所構成。接收器23可為硬體之接收晶片(電路)、軟體之接收程式或其組合所構成。溝通器24可為硬體之溝通晶片(電路)、軟體之溝通程式或其組合所構成。探勘器25可為硬體之探勘晶片(電路)、軟體之探勘程式或其組合所構成。播放器26可為硬體之播放機、播放晶片(電路)、軟體之播放程式、或其任意組合所構成。
The
用戶端20亦可進一步包括一拉取累加器(PullCounter)PC、一最大拉取累加器(MaxPullCounter)MPC、一時延累加器(DelayCounter)DC、一最大時延累加器(MaxDelayCounter)MDC、一無緩累加器(NoBuffCounter)NBC、一最大無緩累加器(MaxNoBufferCounter)MNBC、一良好累加器(FineCounter)FC、一最大良好累加器(MaxFineCounter)MFC、一第一延遲(Delay)事件標註器DE1與一第二延遲(Delay)事件標註器DE2。
The
例如,拉取累加器PC可為硬體之拉取累加晶片(電路)、軟體之拉取累加程式或其組合所構成。最大拉取累加器MPC可為硬體之最大拉取累加晶片(電路)、軟體之最大拉取累加程式或其組合所構成。時延累加器DC可為硬體之時延累加晶片(電路)、軟體之時延累加程式或其組合所構成。最大時延累加器MDC可為硬體之最大時延累加晶片(電路)、軟體之最大時延累加程式或其組合所構成。無緩累加器NBC可為硬體之無緩累 加晶片(電路)、軟體之無緩累加程式或其組合所構成。最大無緩累加器MNBC為硬體之最大無緩累加晶片(電路)、軟體之最大無緩累加程式或其組合所構成。良好累加器FC可為硬體之良好累加晶片(電路)、軟體之良好累加程式或其組合所構成。最大良好累加器MFC可為硬體之最大良好累加晶片(電路)、軟體之最大良好累加程式或其組合所構成。第一延遲事件標註器DE1或第二延遲事件標註器DE2可為硬體之延遲事件標註晶片(電路)、軟體之延遲事件標註程式或其組合所構成。 For example, the pull accumulator PC can be a hardware pull accumulator chip (circuit), a software pull accumulator program, or a combination thereof. The maximum pull accumulator MPC can be composed of the maximum pull accumulation chip (circuit) of hardware, the maximum pull accumulation program of software, or a combination thereof. The delay accumulator DC can be composed of a hardware delay accumulation chip (circuit), a software delay accumulation program, or a combination thereof. The maximum delay accumulator MDC can be composed of a hardware maximum delay accumulation chip (circuit), a software maximum delay accumulation program, or a combination thereof. The non-slow accumulator NBC can be the non-slow accumulation of the hardware It is composed of chip (circuit), software's non-slow accumulation program or a combination thereof. The largest non-deferred accumulator MNBC is composed of the largest non-deferred accumulation chip (circuit) of hardware, the largest non-deferred accumulation program of software, or a combination thereof. The good accumulator FC can be a good accumulation chip (circuit) of hardware, a good accumulation program of software, or a combination thereof. The maximum good accumulator MFC can be composed of the maximum good accumulation chip (circuit) of hardware, the maximum good accumulation program of software, or a combination thereof. The first delayed event tagger DE1 or the second delayed event tagger DE2 can be a hardware delayed event tagging chip (circuit), a software delayed event tagging program, or a combination thereof.
服務端10之來源器11提供或取得原始之影格資料後,服務端10之編碼器12可將影格資料編碼成影格封包,並針對影格資料之同一影格內容壓縮成多種影格品質,例如影格品質最低為Q0且遞送時需要最低之網路頻寬,而影格品質最高為QN且遞送時需要最高之網路頻寬。當服務端10服務用戶端20時,服務端10之發送器13會透過網路遞送來自編碼器12之已編碼之影格封包至用戶端20之接收器23。服務端10之溝通器14可與用戶端20之溝通器14互相溝通,以允許用戶端20動態調整影格封包之影格品質、或向服務端10主動拉取複數影格封包之遺失者。
After the
用戶端20之接收器23從服務端10之發送器13接收或取得已編碼之影格封包後,用戶端20之解碼器22可對已編碼之影格封包進行解碼以取得已解碼之影格封包(如連續影格畫面),再由用戶端20之呈現器21顯示已解碼之影格封包(如連續影格畫面)。用戶端20之探勘器25可透過網際網路控制訊息協定(ICMP)定期探勘用戶端20與服務端10之間的網路狀態,以取得有關網路狀態之往返時間(Round Trip Time;RTT)或延遲變動(Jitter)之資訊。用戶端20之溝通器24可配合服務端10之溝通器
14,並透過傳輸控制協定(TCP)以基於探勘器25所探勘之網路狀態向服務端10發出依據多品質編碼列表來動態調整影格封包之影格品質之請求,且溝通器24亦可主動向服務端10拉取複數影格封包之遺失者。
After the
如第1圖所示,穩定網路影音遞送時延之系統1的運作方式主要包括下列程序A至程序D。
As shown in Figure 1, the operation mode of the
在第1圖之程序A中,服務端10建立服務,包括連接來源器11以取得原始(Raw)之影格資料,並建立編碼器12以取得多品質編碼影格及提供(建立)多品質編碼列表(如影格品質Q0-QN),且建立溝通器14以允許用戶端20調整影格品質(影音品質)或拉取封包(影格封包)。接著,等待用戶端20連入服務端10之服務(即時影音服務),在用戶端20連入服務端10之服務後,服務端10建立發送器13以遞送影格封包至用戶端20。
In the procedure A in Figure 1, the
在第1圖之程序B中,建立用戶端20之各種功能元件,包括建立接收器23以從服務端10取得已編碼之影格封包,建立解碼器22對已編碼之影格封包進行解碼以取得已解碼之影格封包(如連續影格畫面),建立呈現器21以顯示已解碼之影格封包(如連續影格畫面),並建立溝通器24以發送調整影格封包之影格品質(影音品質)之請求、或向服務端10主動拉取封包(影格封包),且建立探勘器25以透過網際網路控制訊息協定(ICMP)之回音檢查(Ping)(即ICMP-Ping)功能取得用戶端20與服務端10之間的網路狀態(如RTT或Jitter)。
In the procedure B of Figure 1, various functional elements of the
在用戶端20連入服務端10之服務後,用戶端20可進行下列程序C至程序D。在第1圖之程序C中,判別封包遺失之事件,並主動拉取遺失封包。在第1圖之程序D中,判別品質異動事件,並主動重設用
戶端20之緩衝或調整所需之影格品質(影音品質)。
After the
同時,第1圖之程序D可進一步包括或判別下列程序D1所述延遲事件而降低品質(影格封包之影格品質)與程序D2所述良好事件而提高品質(影格封包之影格品質)。在第1圖之程序D1中,發生延遲事件,降低影格品質(影音品質)或增加緩衝大小。又,在第1圖之程序D2中,緩衝情形充盈(良好事件),提高影格品質(影音品質)。 At the same time, the program D in Figure 1 may further include or determine the delay events described in the following program D1 to reduce the quality (frame quality of the frame packet) and the good events described in the program D2 to improve the quality (frame quality of the frame packet). In the procedure D1 in Figure 1, a delay event occurs, which reduces the frame quality (video quality) or increases the buffer size. In addition, in the procedure D2 in Figure 1, the buffering situation is full (good events), and the frame quality (video quality) is improved.
第2圖為本發明第1圖之程序A中有關服務端10建立服務的細部流程圖,且第1圖之程序A可包括下列第2圖之步驟A1至步驟A3。
Figure 2 is a detailed flow chart of the service establishment of the
在第2圖之步驟A1中,服務端10取得來源器11之內容,例如原始之影格資料(Video Frames)。
In step A1 in Figure 2, the
在第2圖之步驟A2中,服務端10建立多品質編碼列表,例如以Q0、Q1...QN表示,其中Q0為最低影格品質且所需頻寬最低,而QN為最高影格品質且所需頻寬最高。
In step A2 of Figure 2, the
在第2圖之步驟A3中,服務端10選用具有影片群組(GOP)特性之方式(如H.264)對影格資料進行編碼以產生影格封包,並對影格資料僅進行I影格(I-Frame)及P影格(P-Frame)之壓縮,且對影格資料選用或設定低(較低)的影片群組(GOP)數(如4),以加快讀入用戶端20之緩衝之效率。
In step A3 of Figure 2, the
第3圖為本發明第1圖之程序B中有關用戶端20連入服務的細部流程圖,且第1圖之程序B可包括下列第3圖之步驟B1至步驟B3。
Fig. 3 is a detailed flow chart of the connection service of the
在第3圖之步驟B1中,建立用戶端20之溝通器24,以由溝通器24向服務端10取得多品質編碼列表。
In step B1 of FIG. 3, the
在第3圖之步驟B2中,建立用戶端20之探勘器25,以由探勘器25於週期時間取得用戶端20與服務端10之間的網路狀態,例如下列公式1所示往返時間(RTT)及公式2所示延遲變動(Jitter)之資訊。
In step B2 of Figure 3, the
公式1:RTT=[new_RTT+RTT x(n-1)]/n。每個週期之RTT(往返時間)數值都會進行更新,new_RTT為新一輪經量測取得的RTT(往返時間)數值,n為正整數且可自行決定;當n值越大時,表示越將過往的數值列入考慮,以穩定RTT(往返時間)數值的平均性。 Formula 1: RTT=[new_RTT+RTT x(n-1)]/n. The RTT (round trip time) value of each cycle will be updated, new_RTT is the RTT (round trip time) value obtained by the new round of measurement, n is a positive integer and can be determined by yourself; when the value of n is larger, it means the past The value of is taken into consideration to stabilize the average of RTT (round trip time) values.
公式2:Jitter=[new_Jitter+Jitter x(m-1)]/m。每個週期之延遲變動(Jitter)數值都會進行更新,new_Jitter為新一輪經量測取得的延遲變動(Jitter)數值,m為正整數且可自行決定;當m值越大時,表示越將過往的數值列入考慮,以穩定延遲變動(Jitter)的平均性。 Formula 2: Jitter=[new_Jitter+Jitter x(m-1)]/m. The jitter value of each cycle will be updated, new_Jitter is the jitter value obtained by the new round of measurement, m is a positive integer and can be determined by oneself; when the value of m is larger, it means that the past The value of is taken into consideration in order to stabilize the average of the delay variation (Jitter).
在第3圖之步驟B3中,建立用戶端20之接收器23與播放器26之緩衝,接收器23預設取得如影格品質Q0(最低影格品質且所需頻寬最低)之影片內容,且播放器26於接收器23將第一個完整的影片群組(GOP)接收完整後才進行播放。
In step B3 of Figure 3, the buffers between the
第4圖至第5圖為本發明第1圖之程序C中有關封包遺失而主動拉取的概念示意圖,其中,第4圖為將多個已編號之影格封包F1至F4按照編號依序填入用戶端20之緩衝內,第5圖為封包遺失判別及主動由用戶端20拉取影格封包之遺失者。
Figures 4 to 5 are schematic diagrams of the concept of actively pulling the lost packets in the program C of Figure 1 of the present invention. Figure 4 shows the numbered frame packets F1 to F4 being filled in sequence according to the numbers. Entered into the buffer of the
如第4圖所示,本發明以基於用戶資料報協定(UDP)傳送影格封包來提高遞送效率,此方法可能存在封包遺失或亂序的問題,必須進行自適應調整。具體方法可將每個影格封包進行編號,以供用戶端20取得
各別影格封包時,將多個已編號之影格封包F1至F4按照編號依序填入用戶端20之播放器26之緩衝內。
As shown in Figure 4, the present invention uses User Datagram Protocol (UDP) to transmit frame packets to improve delivery efficiency. This method may have packet loss or disorder problems, which must be adaptively adjusted. The specific method can be to number each frame packet for the
在封包遺失的判別方式中,可限定用戶端20於可計算之限定時間內進行判別,且可依據網路狀態進行動態調整。例如,用戶端20之接收器23在接收到一影格封包後(時間點標註為T1),等待後續封包之時間點超過指定之時間點,則此事件可被判別為封包遺失之事件,其中等待後續封包之時間限制為T1+(RTT+Jitter)/2。
In the judgment method of packet loss, the
當封包遺失之事件發生後,如第5圖之右側所示,由第1圖之用戶端20主動向服務端10拉取指定之影格封包以快速填充欠缺之影格,並將指定之影格封包填入用戶端20之緩衝內。
When the packet loss event occurs, as shown on the right side of Figure 5, the
第6圖為本發明中品質動態調整之方法的流程圖,用以說明透過主動拉取封包(影格封包)所導致的品質異動。如第6圖所示,在第1圖所示之用戶端20主動拉取影格封包之遺失者可視為網路品質不佳之依據,故可設計一品質動態調整之方法。例如,本發明可設計用戶端20之拉取累加器PC以拉取封包或影格封包(見第6圖之步驟S11),若拉取累加器PC在拉取累加器時間(PullCounterSec;PCS)內觸發主動向服務端10拉取複數影格封包之遺失者之次數超過最大拉取累加器MPC之次數(見第6圖之步驟S12),則用戶端20會啟動或執行有關第1圖之程序D1所述品質異動事件中之延遲事件而降低品質(影格封包之影格品質),否則不做任何事(見第6圖之步驟S13)。
Figure 6 is a flow chart of the method for dynamic quality adjustment in the present invention, to illustrate the quality change caused by actively pulling packets (frame packets). As shown in FIG. 6, the lost frame packet that is actively pulled by the
第7圖為本發明中重設拉取累加器PC之機制的流程圖。如圖所示,本發明可開啟一執行緒以定期檢測第7圖之步驟S21所示規則: 是否(最後一次拉取時間-現在時間)>拉取累加器時間(PCS)?若是,則進行步驟S22以重設拉取累加器PC為零(0);反之,若否,則返回步驟S21以持續檢測。 Figure 7 is a flowchart of the mechanism of resetting the pull accumulator PC in the present invention. As shown in the figure, the present invention can open a thread to periodically check the rule shown in step S21 in Figure 7: Is it (last pull time-current time)> pull accumulator time (PCS)? If yes, proceed to step S22 to reset the pull accumulator PC to zero (0); otherwise, if not, then return to step S21 to continue the detection.
另外,上述第1圖之程序D主要在於判別程序D1所述延遲事件而降低品質(影格封包之影格品質)與程序D2所述良好事件而提高品質(影格封包之影格品質),故以下說明如何判別延遲事件及良好事件。 In addition, the procedure D in Figure 1 above is mainly to determine the delayed events described in the procedure D1 to reduce the quality (frame quality of the frame packet) and the good events described in the procedure D2 to improve the quality (frame quality of the frame packet), so the following explains how Identify delayed events and good events.
第8圖為本發明中用戶端20之緩衝內有關影片群組索引(GOPIndex;GIdx)之定義說明的示意圖,並參閱第1圖予以說明。如第8圖所示,最左側之X為正在播放的影格,且用戶端20之緩衝以完整的影片群組索引(GIdx)進行區隔,例如影片群組(GOP)大小為4。PlayingGOPIndex(簡寫成PGIdx)表示正在進行播放的影片群組索引;ReadyGOPIndex(簡寫成RGIdx)表示已經完整接收的影片群組索引;以及NewestGOPIndex(簡寫成NGIdx)表示最新影片群組索引,在本發明中作為服務端10之最新影片群組索引。
FIG. 8 is a schematic diagram of the definition of the GOPIndex (GIdx) in the buffer of the
第9圖為本發明中用戶端20與服務端10之間的時延的示意圖,並參閱第1圖予以說明。
FIG. 9 is a schematic diagram of the time delay between the
如第9圖所示,當用戶端20之播放器26建立用戶端20之緩衝且開始進行有關影格封包之影片播放時建立一執行緒,以透過執行緒於週期時間計算DiffGOPIndex=NGIdx-PGIdx。DiffGOPIndex(簡寫成DGIdx)之意義表示用戶端20與服務端10之間的時延(時延影片群組索引),並等於最新影片群組索引(NGIdx)減掉正在進行播放的影片群組索引(PGIdx)。
As shown in FIG. 9, when the
同時,用戶端20(播放器26)之執行緒於週期時間檢測一次,且初始設定下列三個累加器及其次數均為零(0),包括:(1)時延累加器DC、最大時延次數(MaxDelayTimes;MDT);(2)無緩累加器NBC、最大無緩次數(MaxNoBuffTimes;MNBT);以及(3)良好累加器FC、最大良好次數(MaxFineTimes;MFT)。 At the same time, the thread of the client 20 (player 26) is checked once in the cycle time, and the following three accumulators and their times are initially set to zero (0), including: (1) delay accumulator DC, maximum time Delay times (MaxDelayTimes; MDT); (2) No-buffer accumulator NBC, maximum no-buffer times (MaxNoBuffTimes; MNBT); and (3) good accumulator FC, maximum good times (MaxFineTimes; MFT).
第10圖與第11圖分別為本發明之第一延遲判別與第二延遲判別的示意圖,並參閱第1圖予以說明。亦即,用戶端20在判別是否為延遲事件時,有下列第一延遲判別與第二延遲判別兩種情形。
FIG. 10 and FIG. 11 are schematic diagrams of the first delay determination and the second delay determination of the present invention, respectively, and refer to FIG. 1 for description. That is, when the
第一延遲判別:如第10圖所示,用戶端20設定一最大時延影片群組索引(MaxDelayGOPIndex;MDGIdx)與一初始之最大時延影片群組索引(MaxDelayGOPIndexINIT;MDGIdxINIT),且最大時延影片群組索引(MDGIdx)之值與初始之最大時延影片群組索引(MDGIdxINIT)之值相同。最大時延影片群組索引(MDGIdx)用以判別用戶端20與服務端10之間的時延(時延影片群組索引DGIdx)是否超過門檻值,若超過門檻值,則將用戶端20之時延累加器DC加一(即DC++),且加一表示「加一或一的倍數」。當用戶端20之時延累加器DC所計算之次數超過最大時延次數(MDT)時視為延遲事件,用戶端20會啟動或執行第1圖之程序D1中有關品質異動事件中之延遲事件而降低影格封包之品質(影格品質)。
First delay judgment: as shown in Figure 10, the
第二延遲判別:如第11圖所示,用戶端20檢測正在進行播放的影片群組索引(PGIdx)與已經完整接收的影片群組索引(RGIdx)兩者是否相等?若PGIdx與RGIdx兩者相等,則將用戶端20之無緩累加器NBC加一(即NBC++),且加一表示「加一或一的倍數」。當用戶端20之
無緩累加器NBC所計算之次數超過最大無緩次數(MNBT)時視為延遲事件,用戶端20會啟動或執行第1圖之程序D1所示有關品質異動事件中之延遲事件而降低品質(影格封包之影格品質)。
Second delay judgment: as shown in Figure 11, the
當用戶端20於週期時間之檢測未發生上述第一延遲判別的事件與第二延遲判別的事件,則可視為發生良好判別。每當發生良好判別,就將用戶端20之良好累加器FC加一(即FC++),且加一表示「加一或一的倍數」。當用戶端20之良好累加器FC所計算之次數超過最大良好次數(MFT)時,則為良好事件,用戶端20會第1圖之程序D2中會啟動有關品質異動事件中之良好事件而提高影格封包之品質(影格品質)。
When the
第12圖為本發明第1圖之程序D1中有關延遲事件而降低品質(影格封包之影格品質)的示意圖,並參閱第1圖予以說明。如第12圖所示,考量用戶端20(見符號C)目前選定之影格品質分為左側所示Q>0與右側所示Q<=0兩種情形。 FIG. 12 is a schematic diagram of the quality reduction (frame quality of the frame packet) related to the delay event in the procedure D1 of FIG. 1 of the present invention, and refer to FIG. 1 for description. As shown in Figure 12, considering the frame quality currently selected by the user terminal 20 (see symbol C), there are two cases: Q>0 on the left and Q<=0 on the right.
在第12圖左側所示Q>0(即影格封包之影格品質Q大於0)時,在用戶端20(見符號C)與服務端10(見符號S)之間的溝通通道上,用戶端20透過溝通器24向服務端10發起降低品質(影格封包之影格品質)之請求,以影格封包之目前影格品質Q降一(即Q--)的方式傳送予服務端10(見符號S),且降一表示「降一或一的倍數」。繼之,服務端10接收到影格品質Q降一(即Q--)後,會回覆可以或完成(OK)予用戶端20。接著,用戶端20接收到服務端10之回覆後,會重設時延累加器DC、無緩累加器NBC與良好累加器FC三者均為零(0)。然後,在用戶端20(見符號C)與服務端10(見符號S)之間的串流通道上,服務端10傳送至用戶端20之第
一個影格內容可以是即時解碼刷新(Instantaneous Decoding Refresh;IDR)影格(IDR-Frame),會迫使用戶端20之緩衝清空緩衝內容(如全部緩衝之內容),且用戶端20之解碼器22會採用新的影格品質對後續的影格內容進行解碼。
When Q>0 (that is, the frame quality Q of the frame packet is greater than 0) as shown on the left side of Figure 12, on the communication channel between the user end 20 (see symbol C) and the server 10 (see symbol S), the
在第12圖右側所示Q<=0(即影格封包之影格品質Q小於或等於0)時,由於用戶端20(見符號C)選定的品質已經是最低影格品質而無法再繼續降低品質(影格封包之影格品質),故用戶端20僅將最大時延影片群組索引(MDGIdx)加一(即MDGIdx++),且加一表示「加一或一的倍數」,並重設時延累加器DC、無緩累加器NBC與良好累加器FC三者均為零(0)。
When Q<=0 (that is, the frame quality Q of the frame package is less than or equal to 0) as shown on the right side of Figure 12, the quality selected by the user terminal 20 (see symbol C) is already the lowest frame quality and cannot continue to reduce the quality ( The frame quality of the frame packet), so the
第13圖為本發明第1圖之程序D2中有關良好事件而提高品質(影格封包之影格品質)的示意圖,並參閱第1圖予以說明。如第13圖所示,考量用戶端20(見符號C)目前選定之影格分為左側所示Q<QMax與右側所示Q>=QMax兩種情形。 FIG. 13 is a schematic diagram of improving the quality (frame quality of the frame package) related to good events in the procedure D2 in FIG. 1 of the present invention, and refer to FIG. 1 for description. As shown in Figure 13, considering that the currently selected frame of the user terminal 20 (see symbol C) is divided into two situations: Q<QMax shown on the left and Q>=QMax shown on the right.
在第13圖左側所示Q<QMax(即影格封包之影格品質Q小於最高影格品質QMax)時,在用戶端20(見符號C)與服務端10(見符號S)之間的溝通通道上,用戶端20透過溝通器24向服務端10發起提高品質(影格封包之影格品質)之請求,以目前影格品質Q加一(即Q++)的方式傳送予服務端10,且加一表示「加一或一的倍數」。繼之,服務端10接收到影格品質Q加一(即Q++)後,會回覆可以或完成(OK)予用戶端20。接著,用戶端20接收到服務端10之回覆後,會將最大時延影片群組索引(MDGIdx)設定為初始之最大時延影片群組索引(MDGIdxINIT),並重設時
延累加器DC、無緩累加器NBC與良好累加器FC三者均為零(0)。然後,在用戶端20(見符號C)與服務端10(見符號S)之間的串流通道上,服務端10傳送至用戶端20之第一個影格內容可以是IDR(即時解碼刷新)影格(IDR-Frame),會迫使用戶端20之緩衝清空緩衝內容(如全部緩衝之內容),且用戶端20之解碼器22會採用新的影格品質對後續的影格內容進行解碼。
When Q<QMax (that is, the frame quality Q of the frame packet is less than the highest frame quality QMax) as shown on the left side of Figure 13, on the communication channel between the client 20 (see symbol C) and the server 10 (see symbol S) , The
在第13圖右側所示Q>=QMax(即影格封包之影格品質Q大於或等於最高影格品質QMax)時,由於用戶端20(見符號C)選定的品質已經是最高影格品質而無法再繼續提高品質(影格封包之影格品質),故用戶端20僅將最大時延影片群組索引(MDGIdx)設定為初始之最大時延影片群組索引(MDGIdxINIT),並重設時延累加器DC、無緩累加器NBC與良好累加器FC三者均為零(0)。
When Q>=QMax (that is, the frame quality Q of the frame package is greater than or equal to the highest frame quality QMax) as shown on the right side of Figure 13, the quality selected by the client 20 (see symbol C) is already the highest frame quality and cannot continue Improve the quality (frame quality of the frame packet), so the
舉例而言,本發明中穩定網路影音遞送時延之方法包括:第1圖之程序A所示服務端10建立服務,程序B所示用戶端20連入服務,程序C所示封包遺失與主動拉取,程序D所示品質異動而調整品質。
For example, the method of stabilizing the delay of network video and audio delivery in the present invention includes: the
同時,第1圖之程序A所示服務端10建立服務包括第2圖之步驟A1至步驟A3。在第2圖之步驟A1中,服務端10取得來源器11之內容(原始之影格資料),例如以直播作為實施例,並透過全高清Full HD(如1080p-1920 x 1080 yuv格式)之攝影機,30 FPS(Frame per Second;每秒顯示影格)取得原始之影格資料,亦即每33ms(毫秒)取得一個影格。然後,在第2圖之步驟A2中,服務端10建立多品質編碼列表(如影格品質Q0-QN),如下列表1所示。
At the same time, the service establishment of the
表1:多品質編碼列表,包括影格品質Q0-QN(如Q0-Q5)。
在第2圖之步驟A3中,服務端10之編碼器12對影格資料以具有影片群組(GOP)特性之方式進行編碼以產生影格封包,但對影格資料不進行B影格(B-Frame)之壓縮,且對影格資料選用或設定低(較低)的影片群組(GOP)數。例如,服務端10對影格資料之影片格式選用H.264進行編碼,但不進行B影格(B-Frame)之壓縮,且選擇影片群組(GOP)數為4。
In step A3 of Figure 2, the
然後,等待第1圖所示用戶端20連入服務端10中,且用戶端20連入服務包括下列第3圖之步驟B1至步驟B3。
Then, waiting for the
在第3圖之步驟B1中,建立用戶端20之溝通器24,以由溝通器24向服務端10取得多品質編碼列表。用戶端20接收服務端10給予的多品質編碼列表(見表1),亦即取得Q0-Q5共計6種影格品質列表。
In step B1 of FIG. 3, the
在第3圖之步驟B2中,建立用戶端20之探勘器25,以由探勘器25於週期時間取得有關網路狀態之往返時間(RTT)及延遲變動(Jitter)之資訊。例如,用戶端20建立一執行緒,且週期時間(如每1秒)執行一次執行緒,對服務端10發送網際網路控制訊息協定(ICMP)之回音檢
查(Ping)(即ICMP-Ping)指令以取得有關網路狀態之往返時間(RTT)及延遲變動(Jitter)之資訊,且設定上述公式1之n值及公式2之m值均為3。例如,下列表2所示往返時間(RTT)之取得資訊,以及表3所示延遲變動(Jitter)之取得資訊。
In step B2 of Fig. 3, the
表2:往返時間(RTT)之取得資訊,RTT=[new_RTT+RTT x(n-1)]/n。
表3:延遲變動(Jitter)之取得資訊,Jitter=[new_Jitter+Jitter x(m-1)]/m)。
從上列表2及表3之第10週期的檢測結果,可得結算之往返時間(RTT)約為100.17ms,且結算之延遲變動(Jitter)約為16.16ms。 From the test results of the 10th cycle in Table 2 and Table 3 above, the round-trip time (RTT) of settlement is about 100.17ms, and the jitter of settlement is about 16.16ms.
在第3圖之步驟B3中,建立用戶端20之接收器23與播放器26之緩衝,並預設取得如影格品質Q0(最低影格品質且所需頻寬最低)之影片內容。例如,透過第3圖之步驟B1取得表1之多品質編碼列表,並從多品質編碼列表中選定影格品質Q0之定義的解析度內容為[360p、H.264、AAC、位元速率1.5Mbps],以針對用戶端20之緩衝內的影格內容進行解碼。
In step B3 of Figure 3, the buffers of the
在第1圖之程序C中,用戶端20可於封包(影格封包)遺失時主動拉取封包(影格封包)。例如,透過第3圖之步驟B2所取得有關網路狀態之往返時間(RTT)及延遲變動(Jitter)之資訊,可以判別等待下一封包多久時間後是否為封包遺失。具體而言,某一時間點T1,假定為AM 11:00:01.105(早上11點、0分、1秒、105毫秒),收到前一封包後(IDX=1),待到超過以下時間(如58.165ms)沒有收到下一個封包(IDX=2)則為封包遺
失。例如,T1+(RTT+Jitter)/2=T1+(100.17+16.16)/2=T1+58.165ms=AM 11:00:01.105+59.165ms=AM 11:00:01.163.165。
In the procedure C of Figure 1, the
若用戶端20之接收器23沒有在AM 11:00:01.163165(早上11點、0分、1秒、163.165毫秒)前收到下一影格封包則為封包遺失,用戶端20需要主動向服務端10拉取該封包(IDX=2)。拉取封包(影格封包)送出後,會累加拉取累加器PC一次,並檢查拉取累加器PC是否超過最大拉取累加器MPC。若設定最大拉取累加器MPC為5,並假定下列表4之情形。例如,表4所示編號10之影格封包會導致網路品質不佳事件,故用戶端20會進行第1圖之程序D1以降低品質(影格封包之影格品質)。
If the
表4:封包遺失導致網路品質不佳。
本發明會考量用戶端20之拉取累加器PC的重設。例如,用戶端20在第一次遺失封包的主動拉取事件發生後產生一執行緒,假定週期時間為1秒,透過執行緒執行一次重設拉取累加器PC之掃描,並設定拉取累加器時間(PCS)之門檻值(例如10秒)。如下列表5之方法所示,觀察或判定是否拉取累加器PC之(最後一次拉取時間-現在時間)>拉取累加器時間(取絕對值),以供進一步重設拉取累加器PC。由表5可觀察或判定現在時間到了[11:00:10]的當下,用戶端20判定拉取累加器之最後一次拉取時間與現在時間兩者之間隔時間已抵達拉取累加器時間(PCS)所設定之門檻值(10秒),故重設拉取累加器PC為零(0)。
The present invention considers the resetting of the pull accumulator PC of the
表5:重設拉取累加器PC之事件。
第14圖為本發明第1圖之程序D中有關品質異動而調整品
質的細部流程圖,並包括下列步驟S31至步驟S41,且參閱第1圖予以說明。如第14圖所示,用戶端20開始接收影格內容後開啟一執行緒,此處假定週期時間(如每100ms)以執行緒執行一次確認。
Figure 14 shows the product adjusted for quality change in procedure D of Figure 1 of the present invention
The qualitative detailed flow chart includes the following steps S31 to S41, and is described with reference to Figure 1. As shown in FIG. 14, the
在第14圖之步驟S31中,用戶端20之執行緒等待週期時間,並設定第一延遲事件標註器DE1與第二延遲事件標註器DE2均為零(0),亦即DE1=0且DE2=0。另外,當DE1=1(第一延遲事件標註器為1)時,表示第一延遲事件(如距離服務端10的最新影格過遠)發生;而當DE2=1(第二延遲事件標註器為1)時,表示第二延遲事件(如無緩衝時間過長)發生。
In step S31 in Figure 14, the thread of the
在第14圖之步驟S32中,從用戶端20之播放器26之緩衝中,取得正在進行播放的影片群組索引(PGIdx)、已經完整接收的影片群組索引(RGIdx)、最新影片群組索引(NGIdx)。
In step S32 in Fig. 14, from the buffer of the
在第14圖之步驟S33中,計算用戶端20與服務端10之間的時延(時延影片群組索引DGIdx),亦即計算DGIdx=NGIdx-PGIDx,表示用戶端20與服務端10之間的時延(時延影片群組索引DGIdx)等於最新影片群組索引(NGIdx)減掉正在進行播放的影片群組索引(PGIdx)。
In step S33 in Figure 14, the time delay between the
在第14圖之步驟S34至步驟S35中,判別是否第一延遲事件(DE1=1)發生(如距離服務端10的最新影格過遠),亦即DGIdx>=MDGIdx(時延影片群組索引大於或等於最大時延影片群組索引)?例如,預先設定最大時延影片群組索引(MDGIdx)為5。如下列表6所示之實施例,當到達1000毫秒(ms)時,DGIdx=NGIdx-PGIDx=9-4=5,亦即用戶端20與服務端10之間的時延(時延影片群組索引DGIdx)已經等於或大
於最大時延影片群組索引(MDGIdx=5),故時延累加器DC會被累加(DC++)一次,且表示第一延遲事件(DE1=1)發生。
In step S34 to step S35 in Figure 14, it is determined whether the first delay event (DE1=1) occurs (for example, the latest frame of the
表6:第一延遲事件(DE1=1),例如距離服務端10的最新影格過遠之實施例。PGIdx為正在進行播放的影片群組索引,RGIdx為已經完整接收的影片群組索引,NGIdx為最新影片群組索引,DC為時延累加器。
在第14圖之步驟S36至步驟S37中,判別是否第二延遲事件(DE2=1)發生(如無緩衝時間過長),亦即PGIdx==RGIdx(正在進行播放的影片群組索引等於已經完整接收的影片群組索引)?如下列表7所示之實施例,在500至1000毫秒(ms)中,共計6次發生無緩衝時間過長,故無緩累加器NBC總計會被累加(NBC++)六次成為6,且表示第二延遲事件 (DE1=2)發生。 In step S36 to step S37 in Figure 14, it is judged whether the second delay event (DE2=1) occurs (if no buffering time is too long), that is, PGIdx==RGIdx (the group index of the movie being played is equal to the Group index of videos received in full)? In the example shown in Table 7 below, in 500 to 1000 milliseconds (ms), a total of 6 occurrences of unbuffered time is too long, so the total NBC of the non-slow accumulator will be accumulated (NBC++) six times to become 6, and it means the first Two delayed events (DE1=2) occurs.
表7:第二延遲事件,例如無緩衝時間過長之實施例。PGIdx為正在進行播放的影片群組索引,RGIdx為已經完整接收的影片群組索引,NGIdx為最新影片群組索引,NBC為無緩累加器。
在第14圖之步驟S38至步驟S39中,判別是否為良好事件發生,亦即DE1==0 && DE2==0(第一延遲事件標註器等於0且第二延遲事件標註器等於0)?若是,則良好累加器FC會被累加(FC++)一次。如下列表8所示之實施例,當DE1==0 && DE2==0時,良好累加器FC會被累加一次(即FC++),且表示良好事件發生。 In step S38 to step S39 in Figure 14, it is determined whether a good event has occurred, that is, DE1==0 && DE2==0 (the first delayed event tagger is equal to 0 and the second delayed event tagger is equal to 0)? If it is, the good accumulator FC will be accumulated (FC++) once. In the embodiment shown in Table 8 below, when DE1==0 && DE2==0, the good accumulator FC will be accumulated once (that is, FC++), and it indicates that a good event has occurred.
表8:良好事件。DE1為第一延遲事件標註器,DE2為第二延遲事件標註器,FC為良好累加器。
於週期時間完成第14圖之步驟S34至步驟S39後,依據用戶端20之時延累加器DC、無緩累加器NBC與良好累加器FC,就可以進行是否調整影格品質之判別,如下列第14圖之步驟S40至步驟S41所示。
After completing steps S34 to S39 in Figure 14 in the cycle time, according to the delay accumulator DC, the non-slow accumulator NBC and the good accumulator FC of the
在第14圖之步驟S40中,用戶端20判別是否該進行影格品質降低?如下列表9所示之實施例,時延累加器DC呈現因延遲(Delay)導致的影格品質降低之延遲事件,假定最大時延累加器MDC為5。
In step S40 in Fig. 14, the
表9:延遲導致的影格品質降低。DC為時延累加器,NBC為無緩累加器,FC為良好累加器。
如下列表10所示之實施例,無緩累加器NBC呈現因無緩衝導致的影格品質降低之延遲事件,假定最大無緩累加器MNBC為5。 In the embodiment shown in Table 10 below, the non-buffering accumulator NBC presents a delayed event that causes the frame quality to be reduced due to no buffering. It is assumed that the maximum non-buffering accumulator MNBC is 5.
表10:無緩衝導致的影格品質降低。DC為時延累加器,NBC為無緩累加器,FC為良好累加器。
上列表9及表10所導致的延遲事件,後續皆透過第1圖之程序D1,由用戶端20與服務端10主動溝通,以進行降低影格品質之動作。
The delay events caused by the above Table 9 and Table 10 are followed by the process D1 in Figure 1, and the
在第14圖之步驟S41中,用戶端20判別是否該進行影格品質提高?如下列表11所示之實施例,良好累加器FC呈現未發生第一延遲事件(DE1)或第二延遲事件(DE2),而給予播放器26向上調整影格品質
之良好事件,假定最大良好累加器MFC為5。表11所導致的良好事件,後續皆透過第1圖之程序D2,由用戶端20與服務端10主動溝通,以進行提高影格品質之動作。
In step S41 in Figure 14, the
表11:良好事件判定。DC為時延累加器,NBC為無緩累加器,FC為良好累加器。
上述第1圖之程序D1為延遲事件發生而降低影格品質,第12圖為降低影格品質的示意圖。在表1所示之多品質編碼列表中,假定影格品質Q0-QN為Q0-Q5共計6種影格品質,最大時延影片群組索引(MDGIdx)及初始之最大時延影片群組索引(MDGIdxINIT)均為5,此時分為[Q<=0]及[Q>0]兩個情境。 The above-mentioned procedure D1 in Figure 1 is a delay event that reduces the frame quality, and Figure 12 is a schematic diagram of reducing the frame quality. In the multi-quality coding list shown in Table 1, it is assumed that the frame quality Q0-QN is Q0-Q5, a total of 6 frame qualities, the maximum delay video group index (MDGIdx) and the initial maximum delay video group index (MDGIdxINIT) ) Are all 5, and are divided into two scenarios [Q<=0] and [Q>0].
如第12圖右側所示,若Q<=0(即影格品質Q小於或等於0)時,假定目前之影格品質Q為零(0),則不能再進行影格品質降低,此時將最大時延影片群組索引(MDGIdx)加一(即MDGIdx++),也就成為6。此目的在於網路品質持續不佳的狀況下,用戶端20(見編號C)可減少再次發 動延遲事件的次數,直到網路品質回到正常,再將最大時延影片群組索引(MDGIdx)設定回初始之最大時延影片群組索引(MDGIdxINIT),並重設時延累加器DC、無緩累加器NBC、良好累加器FC三者均為零(0)。 As shown on the right side of Figure 12, if Q<=0 (that is, the frame quality Q is less than or equal to 0), assuming that the current frame quality Q is zero (0), the frame quality can no longer be reduced. The extended movie group index (MDGIdx) plus one (ie MDGIdx++), it becomes 6. The purpose of this is to reduce the number of retransmissions by the client 20 (see No. C) when the network quality continues to be poor. Set the maximum delay video group index (MDGIdx) back to the initial maximum delay video group index (MDGIdxINIT) and reset the delay accumulator DC until the network quality returns to normal. The slow accumulator NBC and the good accumulator FC are all zero (0).
如第12圖左側所示,若Q>0(即影格品質Q大於0)時,假定目前之影格品質Q為3(即Q=3),則用戶端20(見編號C)向服務端10(見編號S)請求降低影格品質Q(即Q--)等於2(即Q=2);待到服務端10回應可以或完成(OK),再重設時延累加器DC、無緩累加器NBC、良好累加器FC三者均為零(0),並在用戶端20(見符號C)與服務端10(見符號S)之間的串流通道於指定之影片群組索引(GIdx)開始時,由解碼器22使用新的(New)影格品質(即Q=N=2)對指定之影片群組索引(GIdx)進行解碼。
As shown on the left side of Figure 12, if Q>0 (i.e. frame quality Q is greater than 0), assuming that the current frame quality Q is 3 (i.e. Q=3), then the client 20 (see number C) will send to the server 10 (See No. S) Request to reduce frame quality Q (ie Q--) is equal to 2 (ie Q=2); when the
上述第1圖之程序D2為良好事件發生而提高影格品質,第13圖為影格品質提高的示意圖。在表1所示之多品質編碼列表中,假定影格品質Q0-QN為Q0-Q5共計6種影格品質,最大時延影片群組索引(MDGIdx)及初始之最大時延影片群組索引(MDGIdxINIT)均為5,此時分為[Q>=QMax]及[Q<QMax]兩個情境,且設定最高影格品質QMax為5。 The above-mentioned procedure D2 in Fig. 1 is for the occurrence of a good event to improve the frame quality, and Fig. 13 is a schematic diagram of the improvement of the frame quality. In the multi-quality coding list shown in Table 1, it is assumed that the frame quality Q0-QN is Q0-Q5, a total of 6 frame qualities, the maximum delay video group index (MDGIdx) and the initial maximum delay video group index (MDGIdxINIT) ) Are both 5. At this time, it is divided into two scenarios, [Q>=QMax] and [Q<QMax], and set the highest frame quality QMax to 5.
如第13圖右側所示,若Q>=QMax(即影格品質Q大於或等於最高影格品質QMax)時,假定目前之影格品質Q為5(等於最高影格品質QMax為5),則用戶端20(見編號C)不能再進行影格品質提高,此時僅將最大時延影片群組索引(MDGIdx)設定回初始之最大時延影片群組索引(MDGIdxINIT),並重設時延累加器DC、無緩累加器NBC、良好累加器FC三者均為零(0)。 As shown on the right side of Figure 13, if Q>=QMax (that is, the frame quality Q is greater than or equal to the highest frame quality QMax), assuming the current frame quality Q is 5 (equal to the highest frame quality QMax being 5), then the client is 20 (See No. C) Frame quality can no longer be improved. At this time, only the maximum delay video group index (MDGIdx) is set back to the initial maximum delay video group index (MDGIdxINIT), and the delay accumulator DC is reset. The slow accumulator NBC and the good accumulator FC are all zero (0).
如第13圖左側所示,若Q<QMax(即影格品質Q小於最高
影格品質QMax)時,假定目前之影格品質Q為3,則用戶端10(見編號C)向服務端10(見編號S)請求提高影格品質Q等於4(即Q=N=4);待到服務端10回應可以或完成(OK),再將最大時延影片群組索引(MDGIdx)設定回初始之最大時延影片群組索引(MDGIdxINIT),且重設時延累加器DC、無緩累加器NBC、良好累加器FC三者均為零(0),並在用戶端20(見符號C)與服務端10(見符號S)之間的串流通道於指定之影片群組索引(GIdx)開始時,由解碼器22使用新的(New)影格品質(即Q=N=4)對指定之影片群組索引(GIdx)進行解碼。
As shown on the left side of Figure 13, if Q<QMax (that is, the frame quality Q is less than the highest
When frame quality QMax), assuming that the current frame quality Q is 3, the client 10 (see number C) requests the server 10 (see number S) to increase the frame quality Q equals 4 (ie Q=N=4); Respond to the
第15圖為本發明中穩定網路影音遞送時延之方法的流程示意圖,並主要包括下列程序A所述服務端10建立服務、程序B所述用戶端20連入服務、程序C所述封包遺失及主動拉取、程序P所述延遲或良好事件檢測、程序D所述品質異動,且參閱第1圖至第3圖予以說明。
Figure 15 is a schematic flow chart of the method for stabilizing network audio and video delivery delays in the present invention, and mainly includes the following procedures A, the
第15圖之程序A所述服務端10建立服務可包括下列步驟A1至步驟A3(見第1圖及第2圖)。
The establishment of the service by the
在第15圖之步驟A1中,服務端10取得來源器11之內容(影格資料),例如透過可執行之方案(如攝影機、系統外之網路影音串流等)取得原始之影格資料。
In step A1 of Figure 15, the
在第15圖之步驟A2中,服務端10建立多品質編碼列表(Q0-QN),例如以影格品質Q0、Q1...QN表示,其中Q0為最低影格品質且所需頻寬最低,而QN為最高影格品質且所需頻寬最高。
In step A2 of Figure 15, the
在第15圖之步驟A3中,服務端10之編碼器12以具有影片群組(GOP)特性之方式對影格資料進行編碼以產生影格封包,但對影格
資料不進行B影格(B-Frame)之壓縮,且對影格資料選用或設定低(較低)的影片群組(GOP)數。例如,服務端10選用具有影片群組(GOP)特性之方式(如H.264)對影格資料進行編碼以產生影格封包,並僅對影格資料進行I影格(I-Frame)及P影格(P-Frame)之壓縮,且對影格資料選用或設定低(較低)的影片群組(GOP)數(如4),以加快讀入用戶端20之緩衝之效率。
In step A3 of Figure 15, the
第15圖之程序B所述用戶端20連入服務可包括下列步驟B1至步驟B3(見第1圖及第3圖)。
The connection of the
在第15圖之步驟B1中,建立用戶端20之溝通器24,以由溝通器24向服務端10取得多品質編碼列表。
In step B1 of FIG. 15, the
在第15圖之步驟B2中,建立用戶端20之探勘器25,以由探勘器25於週期時間取得有關網路狀態之往返時間(RTT)及延遲變動(Jitter)之資訊。
In step B2 in Figure 15, the
在第15圖之步驟B3中,建立用戶端20之接收器23與播放器26之緩衝。例如,接收器23預設取得影格品質Q0(最低影格品質且所需頻寬最低)之影片內容,且播放器26於接收器23將第一個完整的影片群組(GOP)接收完整後才進行播放。
In step B3 of FIG. 15, the buffer between the
第15圖之程序C所述封包遺失及主動拉取可包括下列步驟C1至步驟C3。 The packet loss and active pulling of the program C in FIG. 15 may include the following steps C1 to C3.
在第15圖之步驟C1中,用戶端20之封包遺失判別及主動拉取指定封包。例如,在封包遺失的判別方式中,可限定用戶端20於可計算之限定時間內進行判別,且可依據網路狀態進行動態調整。在用戶端20接收到一影格封包後(時間點標註為T1),等待後續封包之時間點超過指定
之時間點,則此事件可被判別為封包遺失之事件,其中等待後續封包之時間限制為T1+(RTT+Jitter)/2。當封包遺失之事件發生後,用戶端20可主動向服務端10拉取指定之影格封包以快速填充欠缺之影格,並將指定之影格封包填入用戶端20之緩衝內。
In step C1 in Figure 15, the
在第15圖之步驟C2中,用戶端20透過主動拉取封包(影格封包)導致品質異動。例如,用戶端20對品質動態調整之方法可設計一拉取累加器PC以拉取封包(影格封包),若拉取累加器PC在拉取累加器時間(PCS)內觸發主動拉取遺失封包之次數超過最大拉取累加器MPC之次數,則用戶端啟動或執行有關品質異動事件中之延遲事件而降低品質(影格封包之影格品質)的流程(見第15圖或第1圖之程序D1)。
In step C2 in Figure 15, the
在第15圖之步驟C3中,重設用戶端20之拉取累加器PC。例如,用戶端20可檢測是否拉取累加器PC於(最後一次拉取時間-現在時間)>拉取累加器時間(PCS)?若是,則用戶端20重設拉取累加器PC為零(0);反之,若否,則用戶端20可持續檢測。
In step C3 in Figure 15, the pull accumulator PC of the
第15圖之程序P所述延遲或良好事件檢測可包括下列步驟P1至步驟P6。 The delayed or good event detection described in the procedure P in FIG. 15 may include the following steps P1 to P6.
在第15圖之步驟P1中,用戶端20之執行緒週期(即週期時間)執行檢測。例如,用戶端20之執行緒每週期時間進行第一延遲事件(如與服務端10之延遲過大或距離服務端10的最新影格過遠)、第二延遲事件(如用戶端20之緩衝無資料)之判別。
In step P1 in Figure 15, the thread cycle (ie cycle time) of the
在第15圖之步驟P2中,檢測第一延遲事件,如用戶端20與服務端10之延遲過大或距離服務端10的最新影格過遠。例如,檢測最
新接收到的服務端10之封包之影片群組索引(GIdx)與用戶端20之影片群組索引(GIdx)是否超過門檻值?若是,則於有關第一延遲事件之時延累加器DC加一(即DC++),並標註第一延遲事件於該週期(週期時間)觸發。
In step P2 of FIG. 15, the first delay event is detected, such as the delay between the
在第15圖之步驟P3中,檢測第二延遲事件,如用戶端20之緩衝之資料匱乏或無緩衝時間過長。例如,檢測用戶端20正在播放之影片群組索引(GIdx)與用戶端20準備好之影片群組索引(GIdx)是否相同?若是,則於有關第二延遲事件之無緩累加器NBC加一(即NBC++),並標註第二延遲事件於該週期(週期時間)觸發。
In step P3 of FIG. 15, a second delay event is detected, such as a lack of buffered data of the
在第15圖之步驟P4中,檢測良好事件。例如,若第一延遲事件與第二延遲事件皆無觸發,則於用戶端20之良好累加器FC加一(即FC++)。
In step P4 in Figure 15, a good event is detected. For example, if neither the first delay event nor the second delay event is triggered, the good accumulator FC at the
在第15圖之步驟P5中,檢測有關第一延遲事件之時延累加器DC或有關第二延遲事件之無緩累加器NBC是否超過門檻值?若是,則跳往第15圖(或第1圖)之程序D1所述降低品質(影格封包之影格品質)之程序。 In step P5 of Figure 15, check whether the delay accumulator DC related to the first delay event or the non-delay accumulator NBC related to the second delay event exceeds the threshold? If yes, skip to the process of reducing the quality (frame quality of the frame package) described in the process D1 of Fig. 15 (or Fig. 1).
在第15圖之步驟P6中,檢測有關良好事件之良好累加器FC是否超過門檻值?若是,則跳往第15圖(或第1圖)之程序D2所述提高品質(影格封包之影格品質)之程序。 In step P6 in Figure 15, check whether the good accumulator FC of the good event exceeds the threshold? If yes, skip to the process of improving the quality (frame quality of the frame package) described in the process D2 of Fig. 15 (or Fig. 1).
第15圖之程序D所述品質異動可包括下列程序D1至程序D2(見第1圖)。 The quality change described in the procedure D in Fig. 15 may include the following procedures D1 to D2 (see Fig. 1).
在第15圖之程序D1中,降低影格品質。如第12圖左側所示,若能降低影格品質,則用戶端20與服務端10進行溝通以降低影格品
質(Q--),並重設時延累加器DC、無緩累加器NBC、良好累加器FC三者均為零(0),且用戶端20之解碼器22採用指定之影格品質(新的影格品質)對指定之影片群組索引(GIdx)進行解碼。反之,如第12圖右側所示,若不能降低影格品質,則用戶端20僅將最大時延影片群組索引(MDGIdx)加一(即MDGIdx++),並重設時延累加器DC、無緩累加器NBC、良好累加器FC三者均為零(0)。
In the procedure D1 in Figure 15, the frame quality is reduced. As shown on the left side of Figure 12, if the frame quality can be reduced, the
在第15圖之程序D2中,提高影格品質。如第13圖左側所示,若能提高影格品質,則用戶端20與服務端10進行溝通以提高影格品質(Q++),將最大時延影片群組索引(MDGIdx)設定為初始之最大時延影片群組索引(MDGIdxINIT),並重設時延累加器DC、無緩累加器NBC、良好累加器FC三者均為零(0),且用戶端20之解碼器22採用指定之影格品質(新的影格品質)對指定之影片群組索引(GIdx)進行解碼。反之,如第13圖右側所示,若不能提高影格品質,則僅將最大時延影片群組索引(MDGIdx)設定為初始之最大時延影片群組索引(MDGIdxINIT),並重設時延累加器DC、無緩累加器NBC、良好累加器FC三者均為零(0)。
In the procedure D2 of Figure 15, improve the frame quality. As shown on the left side of Figure 13, if the frame quality can be improved, the
綜上,本發明中穩定網路影音遞送時延之系統及方法係至少具有下列特色、優點或技術功效。 In summary, the system and method for stabilizing network video and audio delivery delay in the present invention has at least the following features, advantages, or technical effects.
一、本發明能達成用戶端對於穩定網路影音遞送時延之要求,或者穩定服務端與用戶端之間的播放時延,抑或者確保用戶端在即時影音之觀看過程中處於穩定時延之狀態。 1. The present invention can meet the requirements of the client for stable network video delivery delay, or stabilize the playback delay between the server and the client, or ensure that the client is in a stable delay during the real-time video viewing process state.
二、本發明可建構在基於用戶資料報協定(UDP)之影音遞送服務方案(如RTSP),且遞送前能選擇可支援影片群組(GOP)之壓縮方案(如 H.264)。 2. The present invention can be constructed on a video and audio delivery service solution (such as RTSP) based on User Datagram Protocol (UDP), and can select a compression solution that can support Group of Videos (GOP) (such as H.264).
三、本發明能建構用戶端之探勘器(探勘通道),以用戶端之角度主動察知往來服務端之網路狀態,例如有關網路狀態之往返時間(RTT)及延遲變動(Jitter),作為判別封包遺失進而調整影音壓縮解析度之依據。 3. The present invention can construct a prospecting device (exploring channel) on the client side to actively detect the network status of the server from the perspective of the client side, such as the round trip time (RTT) and delay change (Jitter) related to the network status, as The basis for judging the loss of packets and then adjusting the video and audio compression resolution.
四、本發明為達成低時延影音服務之需求,可採用基於用戶資料報協定(UDP)之傳輸機制,故需考量其特性具有封包遺失或封包亂序之可能性,所以建構用戶端主動察知封包遺失之機制,能於設定時間內判別網路封包遺失或網路狀態不佳之事件。 4. In order to meet the needs of low-latency audio and video services, the present invention can adopt a transmission mechanism based on User Datagram Protocol (UDP). Therefore, it needs to consider its characteristics and the possibility of packet loss or packet disorder, so the client is constructed to actively detect The packet loss mechanism can identify the event of network packet loss or poor network status within a set time.
五、本發明建立服務端與用戶端兩者之溝通器(溝通通道),使用戶端能主動與服務端溝通以動態調整影格品質,同時確保用戶端於網路狀態不佳或降低時可選用較低的影格品質,而於網路狀態較佳或提高時可選用較佳或較高之影格品質。 5. The present invention establishes a communicator (communication channel) between the server and the client, so that the client can actively communicate with the server to dynamically adjust the frame quality, while ensuring that the client can be selected when the network status is poor or degraded Lower frame quality, and better or higher frame quality can be selected when the network status is better or improved.
六、本發明透過溝通器(溝通通道)允許用戶端在封包遺失後,能立即主動從服務端拉取所需之封包(影格封包),以降低時延發生後的緩衝重讀時間。 6. Through the communicator (communication channel), the present invention allows the client to actively pull the required packet (frame packet) from the server immediately after the packet is lost, so as to reduce the buffer reread time after the delay occurs.
七、本發明能依據網路狀態主動清空過久之緩衝內容,以降低用戶端與服務端之時延,進而提升影音遞送的服務品質。 7. The present invention can actively clear the buffered content that is too long according to the network status, so as to reduce the time delay between the client and the server, and thereby improve the service quality of video and audio delivery.
八、本發明於影格品質調動時,會逐步移除舊有之緩衝內容,同時也就剔除了時延較久之影格內容,能在不重新連接服務端的情形下,達成穩定時延之需求。 8. When the frame quality is adjusted, the present invention gradually removes the old buffer content, and also eliminates the frame content with a longer delay, so that the demand for stable delay can be achieved without reconnecting to the server.
九、本發明可能應用於例如電信業、網路服務提供業等,亦可能應用於例如網路影音服務平台、網路遞送維運服務產品等。 9. The present invention may be applied to, for example, the telecommunications industry, network service provision industry, etc., and may also be applied to, for example, network audio-visual service platforms, network delivery and maintenance service products, etc.
上述實施形態僅例示性說明本發明之原理、特點及其功效,並非用以限制本發明之可實施範疇,任何熟習此項技藝之人士均能在不違背本發明之精神及範疇下,對上述實施形態進行修飾與改變。任何使用本發明所揭示內容而完成之等效改變及修飾,均仍應為申請專利範圍所涵蓋。因此,本發明之權利保護範圍,應如申請專利範圍所列。 The above embodiments are only illustrative of the principles, features and effects of the present invention, and are not intended to limit the scope of implementation of the present invention. Anyone familiar with the art can comment on the above without departing from the spirit and scope of the present invention. Modifications and changes to the implementation form. Any equivalent changes and modifications made using the content disclosed in the present invention should still be covered by the scope of the patent application. Therefore, the protection scope of the present invention should be as listed in the scope of the patent application.
1:穩定網路影音遞送時延之系統 1: A system that stabilizes the delay of network audio and video delivery
10:服務端 10: Server
11:來源器 11: Source device
12:編碼器 12: encoder
13:發送器 13: transmitter
14:溝通器 14: Communicator
20:用戶端 20: client
21:呈現器 21: renderer
22:解碼器 22: Decoder
23:接收器 23: receiver
24:溝通器 24: Communicator
25:探勘器 25: Prospector
26:播放器 26: Player
DC:時延累加器 DC: Delay accumulator
DE1:第一延遲事件標註器 DE1: The first delayed event tagger
DE2:第二延遲事件標註器 DE2: The second delayed event tagger
FC:良好累加器 FC: good accumulator
MDC:最大時延累加器 MDC: Maximum delay accumulator
MFC:最大良好累加器 MFC: Maximum good accumulator
MNBC:最大無緩累加器 MNBC: The largest accumulator without slowing
MPC:最大拉取累加器 MPC: Maximum pull accumulator
NBC:無緩累加器 NBC: No slow accumulator
PC:拉取累加器 PC: pull accumulator
A、B、C、D、D1、D2:程序 A, B, C, D, D1, D2: program
Claims (19)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
TW109108969A TWI736180B (en) | 2020-03-18 | 2020-03-18 | System and method for stabilizing network video delivery delay |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
TW109108969A TWI736180B (en) | 2020-03-18 | 2020-03-18 | System and method for stabilizing network video delivery delay |
Publications (2)
Publication Number | Publication Date |
---|---|
TWI736180B true TWI736180B (en) | 2021-08-11 |
TW202137772A TW202137772A (en) | 2021-10-01 |
Family
ID=78283106
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
TW109108969A TWI736180B (en) | 2020-03-18 | 2020-03-18 | System and method for stabilizing network video delivery delay |
Country Status (1)
Country | Link |
---|---|
TW (1) | TWI736180B (en) |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
TW200805965A (en) * | 2006-07-13 | 2008-01-16 | Sunplus Technology Co Ltd | Method and system for controlling multimedia QoS in a handheld device |
TW201110617A (en) * | 2009-09-11 | 2011-03-16 | Hon Hai Prec Ind Co Ltd | Network detecting device and method of network quality detecting thereof |
-
2020
- 2020-03-18 TW TW109108969A patent/TWI736180B/en active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
TW200805965A (en) * | 2006-07-13 | 2008-01-16 | Sunplus Technology Co Ltd | Method and system for controlling multimedia QoS in a handheld device |
TW201110617A (en) * | 2009-09-11 | 2011-03-16 | Hon Hai Prec Ind Co Ltd | Network detecting device and method of network quality detecting thereof |
Also Published As
Publication number | Publication date |
---|---|
TW202137772A (en) | 2021-10-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN106686438B (en) | method, device and system for synchronously playing audio images across equipment | |
US9204203B2 (en) | Reduction of latency in video distribution networks using adaptive bit rates | |
US7984179B1 (en) | Adaptive media transport management for continuous media stream over LAN/WAN environment | |
AU2010345675B2 (en) | Method for synchronized content playback | |
US7881335B2 (en) | Client-side bandwidth allocation for continuous and discrete media | |
US10757481B2 (en) | Class-based intelligent multiplexing over unmanaged networks | |
US10791366B2 (en) | Fast channel change in a video delivery network | |
US20120290739A1 (en) | Adaptive bitrate management for streaming media over packet networks | |
JP2016021763A (en) | Feedback channel for wireless display devices | |
US20090178087A1 (en) | Intelligent retransmission of data stream segments | |
US10862940B1 (en) | Low latency live video on a communication session | |
US20080148327A1 (en) | Method and Apparatus for Providing Adaptive Trick Play Control of Streaming Digital Video | |
CN113271316A (en) | Multimedia data transmission control method and device, storage medium and electronic equipment | |
WO2021208321A1 (en) | Multi-view video view switching method and system | |
WO2022194140A1 (en) | Remote video transmitting method and transmitting apparatus, storage medium, and electronic device | |
CN104918133A (en) | Method and device for playing video streams in articulated naturality web | |
JP2006050604A (en) | Method and apparatus for flexibly adjusting buffer amount when receiving av data depending on content attribute | |
US20080133744A1 (en) | Multimedia data streaming server and method for dynamically changing amount of transmitting data in response to network bandwidth | |
KR20130005873A (en) | Method and apparatus for receiving contents in broadcast system | |
JP2009512265A (en) | Video data transmission control system and method on network | |
JP2015520964A (en) | Integrated controller-based pacing for HTTP pseudo-streaming | |
US20060200577A1 (en) | Method for transmitting moving picture data to mobile terminal using pseudo-streaming technology | |
CN114554277A (en) | Multimedia processing method, device, server and computer readable storage medium | |
JP2022545623A (en) | Prediction-Based Drop Frame Handling Logic in Video Playback | |
US20080104659A1 (en) | Prioritized real-time data transmission |