TWI475842B - Real-time control method of servo-to-client data stream transfer rate - Google Patents

Real-time control method of servo-to-client data stream transfer rate Download PDF

Info

Publication number
TWI475842B
TWI475842B TW095100616A TW95100616A TWI475842B TW I475842 B TWI475842 B TW I475842B TW 095100616 A TW095100616 A TW 095100616A TW 95100616 A TW95100616 A TW 95100616A TW I475842 B TWI475842 B TW I475842B
Authority
TW
Taiwan
Prior art keywords
client
data
video
server
packet
Prior art date
Application number
TW095100616A
Other languages
Chinese (zh)
Other versions
TW200727618A (en
Inventor
Huang Jeffrey
Original Assignee
Sextant Navigation Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sextant Navigation Inc filed Critical Sextant Navigation Inc
Priority to TW095100616A priority Critical patent/TWI475842B/en
Publication of TW200727618A publication Critical patent/TW200727618A/en
Application granted granted Critical
Publication of TWI475842B publication Critical patent/TWI475842B/en

Links

Landscapes

  • Data Exchanges In Wide-Area Networks (AREA)
  • Small-Scale Networks (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Description

伺服端至客戶端資料流傳輸率之即時控制方法Instant control method for server-to-client data stream transmission rate

本發明有關多媒體資料之傳輸,尤其有關區域網路/廣域網路環境中連續資料即時傳輸之方法與系統。The invention relates to the transmission of multimedia materials, in particular to a method and system for real-time transmission of continuous data in a regional network/wide area network environment.

本申請案為美國申請案第11/172,489號,發明名稱「用於網路/廣域網路環境中連續媒體流之自適應媒體傳輸管理」之延續申請案,該案係於2005年6月29日由Huang提出申請,其中請求2004年6月29日提出之美國暫時申請案第60/583,360號之權益,並全案併入本案。This application is a continuation application of U.S. Application No. 11/172,489, entitled "Adaptive Media Transmission Management for Continuous Media Streaming in a Network/WAN Environment", which was filed on June 29, 2005. An application was filed by Huang, which requested the right of US Provisional Application No. 60/583,360, filed on June 29, 2004, and the case was incorporated into the case.

隨著種種新應用方法與新使用模式的出現,區域網路/廣域網路環境下流動的視訊已獲得顯著的重要性。消費者可以從桌上型或膝上型電腦享受觀賞視訊流及/或聆聽音訊流的樂趣,從傳統媒體的耗用中創造一種全新的典型。不但在個人數位視訊娛樂的環境中,視訊流也已隨著視訊會議及遠距離教學的日漸普及而在企業環境中獲得接受。With the emergence of new application methods and new usage models, video streaming in the regional/wide area network environment has gained significant importance. Consumers can enjoy watching video streams and/or listening to audio streams from desktop or laptop computers, creating a new paradigm from the consumption of traditional media. Not only in the personal digital video entertainment environment, video streaming has been accepted in the corporate environment with the increasing popularity of video conferencing and distance learning.

不幸的是,視訊流的普及化對於現有網路基礎建設的可用帶寬,已造成極大的負擔。在已往以網路為基礎的各種應用中,具有意義的資料可以用一種爆發模式(burst mode)傳送或接收。例如,使用網路瀏覽器的使用者可以在一URL(單一資源定址器)鏈結上按滑鼠鍵來呼叫喚用(invoking)一網址,以更新(update)使用者之網頁。網路瀏覽器與所要的網址間的網路通信,可能持續到網頁更新完成或使用者選取另一URL更新(refresh)其瀏覽器為止。然而,與已往以網路為基礎的各種應用不同的是,流動的影音資料在客戶端與伺服端之間的連線建立後,就必須以連續流的方式送達一客戶(觀看者)。若資料流的數量未維持在某一位準以上時(或未等於一最低輸貫量時),視訊可能會因為解碼器沒有足夠的資料可處理(解碼)而出現停止狀況。但等資料再度可用時,視訊會立即回復。Unfortunately, the popularity of video streaming has placed a significant burden on the available bandwidth of existing network infrastructure. In various web-based applications, meaningful data can be transmitted or received in a burst mode. For example, a user using a web browser can invoke a URL on a URL (single resource locator) link to invoking a web address to update the user's web page. Network communication between the web browser and the desired web address may continue until the web page update is complete or the user selects another URL to refresh their browser. However, unlike previous web-based applications, streaming video and audio data must be delivered to a client (viewer) in a continuous stream after the connection between the client and the server is established. If the number of streams is not maintained above a certain level (or not equal to a minimum amount of transmission), the video may be stopped because the decoder does not have enough data to process (decode). However, the video will respond immediately when the data is available again.

已往的視訊廣播是在類比網域中傳送視頻資訊,流動的視訊資訊也與已往的視訊廣播不同,它必須先轉換成數位格式,因此成為一IP資料封包(IP packets),才能在一網際網路通信協定(IP)網路[諸如家庭網路(區域網路)或網際網路(廣域網路)]上傳送。然而,視訊從類比網域簡單轉換成數位網域,並無法立即讓使用者可以在IP網路上傳輸視訊,因為所需的視訊帶寬大於多數現有網路允許的帶寬。例如,(根據國家電視標準委員會)以標準的每秒30幀的速率放映,且像素達720x480的標準家用電視而言,其中每一像素需要2位元組作彩色放映,若要傳輸一全螢幕視訊而不需任何資料壓縮時,其最小輸貫量(throughput)將在19.78MBps或158Mbps左右。此一條件顯然超過一典型區域網路之容量(10/100Mbps)所能支援的範圍。因此,透過習用網路架構傳送原始視訊總長並不實際;或者,至少很難用現有技術在網路上作現場視訊傳輸。因此,在IP網路上傳輸資料前壓縮視訊資料,成為一種必要條件。In the past, video broadcasting was to transmit video information in the analog domain. The streaming video information was different from the previous video broadcasting. It had to be converted into a digital format, so it became an IP packet. A communication protocol (IP) network [such as a home network (regional network) or an internet (wide area network)]. However, the simple conversion of video from an analog domain to a digital domain does not immediately allow users to transmit video over an IP network because the required video bandwidth is greater than the bandwidth allowed by most existing networks. For example, (according to the National Television Standards Committee) at a standard rate of 30 frames per second, and a standard home TV with 720x480 pixels, each pixel requires 2 bytes for color projection, to transmit a full screen. For video without any data compression, the minimum throughput will be around 19.78MBps or 158Mbps. This condition obviously exceeds the range supported by the capacity of a typical local area network (10/100 Mbps). Therefore, it is not practical to transmit the original video length through the conventional network architecture; or, at least, it is difficult to use the existing technology for live video transmission on the network. Therefore, compressing video data before transmitting data on an IP network becomes a necessary condition.

目前已有多種視訊壓縮技術推出,並不斷在改進中。其中最令人注意的幾種壓縮技術是MPEG、H.261、H.263、及H.264。然而,即使有這些壓縮引擎,仍需要恆定的輸貫量,才能將流動視訊從伺服端傳送到一顯示客戶端。不幸的是,區域/廣域網路是設計成一種「盡力型」媒介,意即它雖然試圖在最低可能之時間範圍內使資料從來源端(伺服端)抵達接收端(客戶端),但是並不能保證一定如此。A variety of video compression technologies have been introduced and are constantly being improved. The most notable compression techniques are MPEG, H.261, H.263, and H.264. However, even with these compression engines, a constant amount of throughput is required to transfer mobile video from the server to a display client. Unfortunately, the WAN/WAN is designed as a “best effort” medium, meaning that although it attempts to get data from the source (server) to the receiver (client) in the lowest possible time range, it does not. This must be guaranteed.

若未滿足最低輸貫量,在播放時可能會出現視訊跳動。為了找出流動視訊中既有的視訊跳動,各種線上視訊(VOD)應用中廣泛使用不同方法,諸如在客戶端及伺服端都配置緩衝器。典型線上視訊應用中的內容提供者(等於伺服端)可於顯示客戶端因傳送中產生封包漏失或轉訛而要求時,重新傳輸視訊資料。此種技術可除去視訊跳動並增強觀賞感受。然而,此種方法雖然達到線上視訊的目的,但在需要即時觀賞的應用中,可能引發顯著的延遲。某些即時應用中,諸如現場視訊轉播、保全監看、及視訊會議,具有時間敏感性,在此情況下,伺服端只能傳送視訊資料一次。If the minimum amount of transmission is not met, video jitter may occur during playback. In order to find out the existing video jitter in mobile video, different methods are widely used in various online video (VOD) applications, such as configuring buffers on both the client and the server. The content provider (equivalent to the server) in a typical online video application can retransmit the video data when the display client requests that the packet is lost or transferred during transmission. This technology removes video jitter and enhances viewing experience. However, while this approach achieves the goal of online video, it can cause significant delays in applications that require instant viewing. In some instant applications, such as live video broadcast, security monitoring, and video conferencing, it is time sensitive. In this case, the server can only transmit video data once.

為了能克服網路的「盡力型」特性,並同時以更有效更具可預測性的方式管理網路上的通信量,現今已發展出多種不同的服務品質(QoS)處理方法。例如,其中一種方法是,可將一優先旗標放在資訊流的標頭中,以允許一流動應用從路由器及其他應用接收較高的服務水準。此種方法係用DiffServ表示之。另一種方法是,允許流動應用經由一保留請求將其服務要件廣播給網路。此種方法係用IntServ表示之。通常,根據服務品質知悉型(QoS-awared)路由器的流動應用,可以使用資源保留設置協定(RSVP)作為其端對端信號協定。不幸的是,這些方法都不能保證伺服端與客戶端之間的每一轉送點都有支援或理解這些協定的設備,因此,有可能無法達成那些任務重大型應用所需的連續資料流要件。In order to overcome the "best effort" nature of the network while managing traffic on the network in a more efficient and predictable manner, a number of different quality of service (QoS) processing methods have been developed. For example, one method is to place a priority flag in the header of the information stream to allow a mobile application to receive a higher level of service from the router and other applications. This method is represented by DiffServ. Another approach is to allow the mobile application to broadcast its service requirements to the network via a reservation request. This method is represented by IntServ. In general, according to the mobile application of a QoS-awared router, the Resource Reservation Setting Protocol (RSVP) can be used as its end-to-end signal protocol. Unfortunately, none of these methods guarantees that each transfer point between the server and the client has a device that supports or understands these protocols, so it may not be possible to achieve the continuous stream elements required for mission-critical applications.

因此,吾人亟需一種系統,既可跨越傳統網路以即時方式傳輸資料的系統,並可同時在資料傳輸期間滿足最低閾要件,以維持人類視覺可接受的視訊品質。Therefore, we need a system that can transmit data in real time across traditional networks, and at the same time meet minimum thresholds during data transmission to maintain human visually acceptable video quality.

本發明提供多種可跨越傳統網路或通道以即時傳輸已壓縮影音資料流的方法及系統。每一系統均使用自適應速率控制(Adaptive Rate Control,ARC)技術。此種技術係以偵測瑕疵封包及其他諸如整體系統效能、使用模式、及通道特性等影響性因素為基礎。為了控制各通道中的資料流壅塞,並將目視顯示品質維持在某一水準以上,本發明進而有效利用人類的視覺及人類對變動的目視狀態的適應力。The present invention provides a variety of methods and systems for instantaneously transmitting compressed video data streams across conventional networks or channels. Each system uses Adaptive Rate Control (ARC) technology. This technology is based on the detection of 瑕疵 packets and other influencing factors such as overall system performance, usage patterns, and channel characteristics. In order to control the flow choking in each channel and maintain the visual display quality above a certain level, the present invention further utilizes human vision and human adaptability to changing visual conditions.

前述自適應速率控制(ARC)技術有賴於客戶端對伺服端所傳送視訊封包品質的計算能力,以及對伺服端傳送所需傳輸率資訊的能力。此種方法簡化了已往伺服端或內容提供端所承擔的硬體與軟體實施工具複雜性,並將監看帶寬與傳輸控制的負擔從伺服端移轉到客戶端,藉此減少整體設計成本。The aforementioned adaptive rate control (ARC) technology relies on the client's ability to calculate the video packet quality transmitted by the server and the ability to transmit the required transmission rate information to the server. This method simplifies the hardware and software implementation complexity of the previous server or content provider, and shifts the monitoring bandwidth and transmission control burden from the server to the client, thereby reducing the overall design cost.

本發明目的之一係提供一種伺服端至客戶端資料流傳輸率之即時控制方法,其中,該客戶端具有使用其一組特徵產生的比對檢查表。此方法包括以下步驟:以一傳輸率從伺服端接收一資料流的第一部份,其中該第一部份包括多數資料封包;檢查該等資料封包之完整性;根據其完整性決定一封包漏失比;使用該封包漏失比及該比對檢查表產生一調整該傳輸率之要求;將該要求傳送至該伺服端;以及,以調整後的傳輸率從該伺服端接收該資料流的第二部份。One of the objects of the present invention is to provide an instant control method for a server-to-client data stream transmission rate, wherein the client has a comparison check table generated using a set of characteristics thereof. The method comprises the steps of: receiving a first portion of a data stream from a server at a transmission rate, wherein the first portion comprises a majority data packet; checking the integrity of the data packet; determining a packet based on its integrity a leakage ratio; using the packet leakage ratio and the comparison checklist to generate a request for adjusting the transmission rate; transmitting the request to the server; and receiving the data stream from the server at the adjusted transmission rate Two parts.

本發明另一目的係提供一種伺服端至客戶端資料流傳輸率之即時控制方法,其中,該客戶端具有使用其一組特徵產生的比對檢查表。此方法包括以下步驟:接收客戶端對調整傳輸率的要求,其中該客戶端係使用一封包漏失比及該比對檢查表產生該項要求;接收至少一筆音頻輸入及至少一筆視訊輸入;根據該項要求更新一資料率及該傳輸率;以更新後的資料率編碼該音頻輸入及該視訊輸入;將編碼後音頻及視訊輸入複合成該資料流內;以及,將該資料流傳輸至該客戶端。Another object of the present invention is to provide an instant control method for a server-to-client data stream transmission rate, wherein the client has a comparison check table generated using a set of features thereof. The method includes the steps of: receiving a request from a client to adjust a transmission rate, wherein the client generates the request by using a packet loss ratio and the comparison checklist; receiving at least one audio input and at least one video input; Requires updating a data rate and the transmission rate; encoding the audio input and the video input at an updated data rate; composing the encoded audio and video input into the data stream; and transmitting the data stream to the client end.

本發明再一目的係提供一種伺服端至客戶端資料流傳輸率之即時控制方法,其中,該客戶端具有使用其一組特徵產生的比對檢查表。此方法包括以下步驟:使該伺服端以一傳輸率將一資料流之第一部份傳輸至該客戶端;使該客戶端接收該第一部份,其中該第一部份包括多數資料封包;使該客戶端檢查該等資料封包之完整性,並根據其完整性決定一封包漏失比;使該客戶端使用該封包漏失比及該比對檢查表產生一調整該傳輸率之要求,並將該要求傳送至該伺服端;使該伺服端接收該項要求並根據該項要求調整傳輸率;以及,使該伺服端以調整後的傳輸率傳輸該資料流的第二部份。Still another object of the present invention is to provide an instant control method for a server-to-client data stream transmission rate, wherein the client has a comparison check table generated using a set of features thereof. The method includes the steps of: causing the server to transmit a first portion of a data stream to the client at a transmission rate; causing the client to receive the first portion, wherein the first portion includes a majority data packet Causing the client to check the integrity of the data packet and determine a packet loss ratio based on its integrity; causing the client to use the packet loss ratio and the comparison checklist to generate a request to adjust the transmission rate, and Transmitting the request to the server; causing the server to receive the request and adjusting the transmission rate according to the requirement; and causing the server to transmit the second portion of the data stream at the adjusted transmission rate.

藉由參照以下圖式、說明及申請專利範圍,可以更加了解本發明上述及其他特點、目的及優點。The above and other features, objects and advantages of the present invention will become more apparent from the <RTIgt;

以下詳細說明係本發明目前最佳實施模式,其目的並非用以限制本發明,僅係說明本發明之一般原理,因為本發明範圍係由所附申請專利範圍充分定義之。The present invention is described in the following detailed description of the preferred embodiments of the invention.

首先必須指出,本說明書及所附申請專利範圍中使用的單數形式「一」、「及」、「該」,除非上下文義中另有明確表示,否則係包括多數。因此,例如,提及「一客戶端」時,係包括一或多個客戶端及熟悉此類技術之人士所知的同等意義。It must be noted that the singular forms "a", "and" and "the" are used in the <RTIgt; Thus, for example, reference to "a client" includes the equivalent meaning of one or more clients and those skilled in the art.

廣義而言,本發明提供多種在即時環境中跨越多數傳統網路或通道而傳輸壓縮影音資料流的方法與系統。每一系統可使用自適應速率控制(Adaptive Rate Control,ARC)技術。此種技術係以偵測瑕疵封包及其他諸如整體系統效能、使用模式、及通道特性等影響性因素為基礎。本發明與現行僅根據通道特徵或多層流研究法的壅塞控制機制不同,它可有效利用人類的視覺及人類對變動的目視狀態的適應力,將目視顯示品質維持在一可接受的水準之上。Broadly speaking, the present invention provides a variety of methods and systems for transmitting compressed video data streams across most conventional networks or channels in an instant environment. Adaptive Rate Control (ARC) technology can be used for each system. This technology is based on the detection of 瑕疵 packets and other influencing factors such as overall system performance, usage patterns, and channel characteristics. The present invention is different from the current occlusion control mechanism based only on the channel characteristics or the multi-layer flow research method, and can effectively utilize the human visual and human adaptability to the changing visual state to maintain the visual display quality at an acceptable level. .

前述自適應速率控制(ARC)技術可依賴客戶端對伺服端所傳送視訊封包品質的計算能力,以及傳送所需傳輸率資訊給伺服端的能力。此種方法簡化了已往伺服端或內容提供端所承擔的硬體/軟體實施工具複雜性,並將監看帶寬與傳輸控制的負擔從伺服端移轉到客戶端,藉此減少整體設計成本。The aforementioned adaptive rate control (ARC) technology can rely on the computing power of the client to transmit the video packet quality of the server, and the ability to transmit the required transmission rate information to the server. This method simplifies the hardware/software implementation tool complexity that has been undertaken by the previous server or content provider, and shifts the monitoring bandwidth and transmission control burden from the server to the client, thereby reducing the overall design cost.

請參照圖1。圖1係根據本發明之即時影音資料傳輸系統100之示意圖,該系統包括一區域網路(LAN)系統101及一廣域網路(WAN)系統121。如圖所示,區域網路101係指空中120左側的網路系統,它可包括一網路纜線或WiFi通道130(以下簡稱纜線),多數與纜線130連接之個人電腦114、116,一與纜線130連接之伺服端106,一與纜線130連接之路由器112,及一架構為可與路由器112通信之膝上型電腦110。區域網路系統101亦可包括多數影音顯示裝置,諸如網路攝影機108、放音機104、及電視機102。其中,該等影音顯示裝置可與伺服端106耦接,並與伺服端(有線或無線)106即時交換影音資料。區域網路系統101可透過空中120而與其他區域網路或廣域網路系統通信。所謂空中,較佳是指網際網路(Internet)。區域網路系統101中可包括一防火牆118防止諸如電腦病毒及間諜軟體等惡意的資料/程式進入系統。Please refer to Figure 1. 1 is a schematic diagram of a real-time video and audio data transmission system 100 in accordance with the present invention, the system including a local area network (LAN) system 101 and a wide area network (WAN) system 121. As shown, the local area network 101 refers to the network system on the left side of the air 120, which may include a network cable or WiFi channel 130 (hereinafter referred to as a cable), and most of the personal computers 114, 116 connected to the cable 130. A server 106 coupled to the cable 130, a router 112 coupled to the cable 130, and a laptop 110 configured to communicate with the router 112. The local area network system 101 may also include most video display devices such as a webcam 108, a player 104, and a television set 102. The audio-visual display device can be coupled to the server 106 and exchange audio and video data with the server (wired or wireless) 106 in real time. The regional network system 101 can communicate with other regional or wide area network systems over the air 120. The so-called air, preferably refers to the Internet. A firewall 118 can be included in the local area network system 101 to prevent malicious data/programs such as computer viruses and spyware from entering the system.

廣域網路係指空中120右側的系統,它可包括一組客戶端或客戶端裝置及一通常稱為熱點(Hot Spot)的存取(通達)點122,用以提供對客戶端的無線通信。其中,客戶端包括一個人電腦128及其他手持裝置,諸如個人數位助理(PDA)124及手機126。客戶端124、126、128可透過空中120而與伺服端通信。其中,伺服端可為一媒體伺服器或一內容提供者,用以即時提供流動資料給客戶端。以下提及客戶端或客戶端裝置一詞時,係指架構為可與伺服端106通信並可與伺服端106即時交換流動資料的裝置。一客戶端可具有一或多個影音顯示系統用以顯示流動資料。以下參照圖2-4進一步說明伺服端與多數客戶端之一間的資料流細節。圖1中,為方便之故,僅顯示九個客戶端102、104、108、110、114、116、124、126與128。然而,具備一般技能之人士應能了解,系統100中還可以使用其他適合類型的客戶端裝置。Wide area network refers to the system on the right side of the air 120, which may include a set of client or client devices and an access (access point) 122, commonly referred to as a hot spot, for providing wireless communication to the client. Among them, the client includes a personal computer 128 and other handheld devices, such as a personal digital assistant (PDA) 124 and a mobile phone 126. Clients 124, 126, 128 can communicate with the server over the air 120. The server can be a media server or a content provider for providing mobile data to the client immediately. References below to the term client or client device refer to a device that is configured to communicate with the server 106 and to exchange mobile data with the server 106 in real time. A client may have one or more audiovisual display systems for displaying streaming data. The details of the data flow between the server and one of the clients are further described below with reference to Figures 2-4. In FIG. 1, only nine clients 102, 104, 108, 110, 114, 116, 124, 126, and 128 are shown for convenience. However, those with general skills should be able to understand that other suitable types of client devices can be used in system 100.

圖2之方塊圖200中,顯示圖1整體所述於伺服端與多數客戶端裝置之一間的即時模式資料流。如圖所示,一伺服端202可經由一網路(區域或廣域網路)210將複合的影音輸出資料傳送給一客戶端212。以下將交互使用「複合」(multiplex)及「多元插配」(interleave)二詞。複合的影音輸出資料可包括以預定資料率(壓縮率)壓縮以減少網路210中的流動壅塞、並根據一使用者資料包協定(user datagram protocol,UDP)格式化的視頻資料。稍後會詳細說明決定資料率的方法。In block diagram 200 of FIG. 2, an instant mode data stream between the server and one of the plurality of client devices is shown in FIG. As shown, a server 202 can transmit composite audio and video output data to a client 212 via a network (area or wide area network) 210. The following will use the words "multiplex" and "interleave" interactively. The composite video output data may include video data compressed at a predetermined data rate (compression rate) to reduce flow congestion in the network 210 and formatted according to a user datagram protocol (UDP). The method of determining the data rate will be described in detail later.

當客戶端212收到複合的影音資料時,可將資料暫時儲存在一收件緩衝器214內。然後,收件緩衝器214可將資料轉傳給一影音複合分解系統218。影音複合分解系統218可將複合資料分解成二組個別的資料,其中每一組資料包含一視訊信號或一音頻信號;檢查二組資料的完整性,並將二組資料解碼。稍後會解釋,伺服端202會在傳送資料給客戶端212前,先將音頻流與視訊流多元插配成為單一資料流。如前所述,每一視訊幀可能需要相當的資料量,因此,每一幀資料在透過網路210傳送前,可先分成許多資料封包。每一封包可有一標頭,記載幀內封包總數、幀內封包編號、幀(的同位)總和檢查、及封包長度等資訊。When the client 212 receives the composite video material, the data can be temporarily stored in the recipient buffer 214. The recipient buffer 214 can then forward the data to a video composite decomposition system 218. The audio-visual composite decomposition system 218 can decompose the composite data into two sets of individual data, wherein each set of data includes a video signal or an audio signal; check the integrity of the two sets of data, and decode the two sets of data. As will be explained later, the server 202 will multiply the audio stream and the video stream into a single data stream before transmitting the data to the client 212. As mentioned above, each video frame may require a considerable amount of data. Therefore, each frame of data may be divided into a plurality of data packets before being transmitted through the network 210. Each packet can have a header, which records information such as the total number of packets in the frame, the number of intra-packets, the sum of the frames (the parity), and the length of the packet.

影音複合分解系統218有一視訊環狀緩衝器244及一音頻環狀緩衝器242(見圖4)可分別用來驗證視訊資料封包及音頻資料封包的完整性。以下會參照圖4進一步詳細說明影音複合分解系統218。當視訊環狀緩衝器244遭遇轉訛的或漏失的封包(總稱為瑕疵封包)時,它會拋棄含有瑕疵封包的幀,並對一封包錯誤累加器222傳送通知。在另一實施例中,若有轉訛的封包,可使用諸如前向錯誤更正(Forward Error Corrections)之錯誤更正法來重建該幀。在另一實施例中,若有漏失的資料(遺失的封包),可使用諸如可逆變數長度(Reversible Variable Length)碼之復原法來處理該幀。影音複合分解系統218可將解碼後的音頻及視訊資料分別傳送給一放音器/揚聲器220a與一視訊顯示器220b。The video composite decomposition system 218 has a video ring buffer 244 and an audio ring buffer 242 (see FIG. 4) for verifying the integrity of the video data packet and the audio data packet, respectively. The video composite decomposition system 218 will be described in further detail below with reference to FIG. When the video ring buffer 244 encounters a switched or lost packet (collectively referred to as a packet), it discards the frame containing the packet and transmits a notification to the packet error accumulator 222. In another embodiment, if there is a packet that is switched, the error correction method such as Forward Error Corrections can be used to reconstruct the frame. In another embodiment, if there is missing data (missing packets), the frame can be processed using a restoration method such as a Reversible Variable Length code. The audio/video composite decomposition system 218 can transmit the decoded audio and video data to a speaker/speaker 220a and a video display 220b, respectively.

封包錯誤累加器222可定期決定與傳送一封包漏失比給一傳輸比對檢查表(Transmission LUT)224。其中,該封包漏失比之定義是:瑕疵封包計數與一時程內收到的封包總數之比。在該時程開始時,封包錯誤累加器222可重設瑕疵封包計數。封包錯誤累加器222於收到影音複合分解系統218的瑕疵封包通知時,可將瑕疵封包計數增加1。同時,它可計數於該時程內收到的封包總數。在該時程結束時,封包錯誤累加器222可計算封包漏失比,並將該比率傳送給傳輸比對檢查表224。The packet error accumulator 222 can periodically determine and transmit a packet miss ratio to a transmission LUT 224. The packet leakage ratio is defined as: the ratio of the packet count to the total number of packets received in a time interval. At the beginning of the time period, the packet error accumulator 222 can reset the packet count. The packet error accumulator 222 may increment the packet count by one when it receives the packet notification from the video composite decomposition system 218. At the same time, it counts the total number of packets received during that time period. At the end of the time period, the packet error accumulator 222 can calculate the packet leakage ratio and transmit the ratio to the transmission comparison check table 224.

該客戶端212亦可在伺服端202與客戶端212間的通信開始時,將其特徵資訊傳送給智慧型視訊傳輸管理器208。客戶端特徵資訊可包括顯示器尺寸、最高解析度、可支援色彩位元數(或色彩深度)、顯示器類型(諸如LCD、CRT、或電漿型)、CPU類型、指示可支援格式類型(諸如MPEG1、MPEG2、MPEG4、H.263、H.264等等)之解碼器能力、可用記憶體、連線類型(諸如WiFi、LAN、藍芽等等)、說明傳輸資訊的先前連線記述(諸如幀速率、資料率、量化級別、解析度等等)、及視訊顯示器220b之線路結構(續行式/交錯式)。然後,根據這些特徵資訊,伺服端202可決定初始傳輸率,如此可使資料傳輸中的等待時間最小化,並使收到的資料品質最大化。The client 212 can also transmit its feature information to the smart video transmission manager 208 when communication between the server 202 and the client 212 begins. Client feature information may include display size, highest resolution, number of supportable color bits (or color depth), display type (such as LCD, CRT, or plasma type), CPU type, indication supportable format type (such as MPEG1) , MPEG2, MPEG4, H.263, H.264, etc.) decoder capabilities, available memory, connection type (such as WiFi, LAN, Bluetooth, etc.), description of previously connected information (such as frames) Rate, data rate, quantization level, resolution, etc.), and the line structure of the video display 220b (continuous/interleaved). Then, based on these feature information, the server 202 can determine the initial transmission rate, which minimizes latency in data transmission and maximizes the quality of received data.

傳輸比對檢查表224可使用該封包漏失比讀取一比對檢查表,以決定目前的傳輸率是否需要調整。該比對檢查表可為客戶專用的,並使用該客戶端212的特徵來產生。然後,傳輸比對檢查表224可經由網路210將調整目前傳輸率的要求(如圖2中標示的「控制命令」)傳送給智慧型視訊傳輸管理器208。以下會參照圖4進一步說明調整的細節。The transmission alignment check table 224 can use the packet leakage ratio to read a comparison checklist to determine whether the current transmission rate needs to be adjusted. The alignment checklist can be customer specific and generated using the characteristics of the client 212. Then, the transmission comparison check table 224 can transmit the current transmission rate requirement (such as the "control command" indicated in FIG. 2) to the smart video transmission manager 208 via the network 210. Details of the adjustment will be further explained below with reference to FIG.

利用客戶端212傳送的要求及特徵資訊,智慧型視訊傳輸管理器208可以調整一組參數,包括目標幀速率、目標顯示位元率、目標解析度、目標音頻抽樣率、量化值,調整亮度色度(YUV)轉換法(4:2:2至4:2:0)、比例尺大小。管理器208亦可調整資料壓縮率(簡稱資料率),並將調整後資料率資訊傳送給一影音複合系統204。影音複合系統204可接收一或多筆影音輸入流205,編碼接收的輸入流,使用收自智慧型視訊傳輸管理器208的資料率資訊來壓縮輸入流,將每一幀資料分成多數封包,並將此等封包傳送至一媒體存取控制(MAC)緩衝器206。如前所述,每一視訊幀可包括相當的資料量,並可分成多數資料封包,以易於透過網路210傳輸。以下會參照圖3提供有關影音複合系統204更詳細的資訊。媒體存取控制(MAC)緩衝器206可暫時包含欲透過網路210傳輸的封包,並視客戶端的數目而定,以單聲道調頻廣播(unicast)、立體聲雙聲道調頻廣播(multicast)、或廣播(broadcast)模式傳送此等封包。Using the requirements and feature information transmitted by the client 212, the smart video transmission manager 208 can adjust a set of parameters including target frame rate, target display bit rate, target resolution, target audio sampling rate, quantized value, and adjust the brightness color. Degree (YUV) conversion method (4:2:2 to 4:2:0), scale size. The manager 208 can also adjust the data compression rate (referred to as the data rate) and transmit the adjusted data rate information to an audio/video composite system 204. The audio/video composite system 204 can receive one or more audio and video input streams 205, encode the received input stream, compress the input stream using the data rate information received from the smart video transmission manager 208, and divide each frame of data into a plurality of packets, and These packets are passed to a Media Access Control (MAC) buffer 206. As previously mentioned, each video frame can include a substantial amount of data and can be divided into a plurality of data packets for easy transmission over the network 210. More detailed information about the audio/video composite system 204 will be provided below with reference to FIG. The Media Access Control (MAC) buffer 206 may temporarily contain packets to be transmitted over the network 210, depending on the number of clients, in mono unicast, stereo two-channel FM, and Or broadcast (broadcast) mode to transmit these packets.

圖3為圖2所示伺服端202內實施的影音複合系統204之方塊圖。如圖所示,影音複合系統204可包括一音頻編碼器230,用以編碼一音頻輸入流;一音頻環狀緩衝器232,用以轉播編碼後音頻輸入流給一影音複合器238;一視訊編碼器234,可依收自智慧型視訊傳輸管理器208的資料率編碼(或壓縮)一視訊輸入流;一視訊環狀緩衝器236,可將編碼後視訊流轉播給影音複合器238;及該影音複合器238,可將編碼後音頻流與視訊流複合成單一的複合資料流。為了維持音頻/視訊同步化及抽樣時脈完整性,影音複合器238可交織編碼後音頻流與編碼後視訊流。FIG. 3 is a block diagram of the AV composite system 204 implemented in the servo end 202 of FIG. As shown, the audio/video composite system 204 can include an audio encoder 230 for encoding an audio input stream, and an audio ring buffer 232 for relaying the encoded audio input stream to an audio/video combiner 238; The encoder 234 can encode (or compress) a video input stream according to the data rate received from the smart video transmission manager 208; a video ring buffer 236 can relay the encoded video stream to the audio/video combiner 238; The video and audio synthesizer 238 can combine the encoded audio stream and the video stream into a single composite data stream. To maintain audio/video synchronization and sample clock integrity, the video and audio synthesizer 238 can interleave the encoded audio stream with the encoded video stream.

如前所述,由一伺服端傳送的複合流封包,可能會因為網路壅塞、解碼器負載飽和、或系統負載飽和而於透過網路傳輸期間漏失或轉訛。因此,必須在分解複合流封包後,檢查透過網路接收的複合流封包之完整性。例如,圖4顯示圖2所示影音複合分解系統218之方塊圖,其中,該影音複合分解系統218可將接收的複合流分解成一音頻及一視訊流;檢查該視訊流之完整性;並將此等音頻流及視訊流解碼,以供放音器/揚聲器220a及視訊顯示器220b播放。如圖所示,影音複合分解器240可將接收的資料流解耦成一音頻流及一視訊流。然後,它可將視訊流放入視訊環狀緩衝器244,將音頻流放入一音頻環狀緩衝器242。音頻環狀緩衝器242與視訊環狀緩衝器244可分別檢查影音同步與幀同步,並於解碼前執行必要的更正。視訊環狀緩衝器244亦可檢查是否有任何轉訛及/或漏失的封包,以驗證視訊流(或封包)的完整性。當視訊環狀緩衝器244遇到瑕疵封包時,它可傳送一通知給封包錯誤累加器222。As mentioned earlier, a composite stream packet transmitted by a server may be lost or transferred during transmission through the network due to network congestion, decoder load saturation, or system load saturation. Therefore, the integrity of the composite stream packet received through the network must be checked after the composite stream packet is decomposed. For example, FIG. 4 shows a block diagram of the video composite decomposition system 218 of FIG. 2, wherein the video composite decomposition system 218 can decompose the received composite stream into an audio and a video stream; check the integrity of the video stream; These audio streams and video streams are decoded for playback by the speaker/speaker 220a and the video display 220b. As shown, the video composite resolver 240 can decouple the received data stream into an audio stream and a video stream. It then places the video stream into video ring buffer 244 and the audio stream into an audio ring buffer 242. The audio ring buffer 242 and the video ring buffer 244 can check the video sync and frame sync, respectively, and perform the necessary corrections before decoding. The video ring buffer 244 can also check for any transitions and/or missed packets to verify the integrity of the video stream (or packet). When the video ring buffer 244 encounters a packet, it can transmit a notification to the packet error accumulator 222.

如前解說,封包錯誤累加器222可決定並傳送一封包漏失比給傳輸比對檢查表224。傳輸比對檢查表224可根據該封包漏失比產生調整傳輸率的要求,並將此項要求(或控制命令)透過一使用者資料包協定(UDP)通道傳送給伺服端202。UDP命令埠/通道是一種任意埠(arbitrary port),由伺服端202特別指定與保留,以供與客戶端212通信之用。隨後,伺服端202可藉由控制壓縮率、幀速率、圖像群組(GOP)大小、量化級別、顯示解析度、或以上各項任何組合而調整傳輸率。As previously explained, the packet error accumulator 222 can determine and transmit a packet miss ratio to the transmission alignment check table 224. The transmission comparison check table 224 can generate a transmission rate requirement according to the packet loss ratio, and transmits the request (or control command) to the server 202 through a User Datagram Protocol (UDP) channel. The UDP command port/channel is an arbitrary port that is specifically designated and reserved by the server 202 for communication with the client 212. Subsequently, the server 202 can adjust the transmission rate by controlling the compression ratio, frame rate, group of pictures (GOP) size, quantization level, display resolution, or any combination of the above.

封包漏失比可作為網路210中傳輸狀態的指示器。傳輸比對檢查表224可使用該封包漏失比讀取一比對檢查表。其中,該比對檢查表可為客戶專用的,並使用一組特徵來產生。這些特徵包括顯示器尺寸、最高解析度、可支援色彩位元數(或色彩深度)、顯示器類型(諸如LCD、CRT、或電漿型)、CPU類型、指示可支援格式類型(諸如MPEG1、MPEG2、MPEG4、H.263、H.264等等)之解碼器能力、可用記憶體、連線類型(諸如WiFi、LAN、藍芽等等)、說明傳輸資訊的先前連線記述(諸如幀速率、資料率、量化級別、解析度等等)、及視訊顯示器220b之線路結構(續行式/交錯式)。The packet loss ratio can be used as an indicator of the state of transmission in the network 210. The transmission comparison check table 224 can use the packet missing ratio to read a comparison check list. Wherein, the comparison checklist can be customer-specific and generated using a set of features. These features include display size, highest resolution, number of supportable color bits (or color depth), display type (such as LCD, CRT, or plasma type), CPU type, indication supportable format types (such as MPEG1, MPEG2) Decoder capabilities of MPEG4, H.263, H.264, etc., available memory, connection type (such as WiFi, LAN, Bluetooth, etc.), description of previously connected information (such as frame rate, data) Rate, quantization level, resolution, etc.), and the line structure of the video display 220b (continuous/interleaved).

傳輸狀態可細分成三種子狀態:(1)無負載狀態(2)負載狀態(3)壅塞狀態。每一子狀態可有一預定範圍的封包漏失比。例如,假設目前的傳輸狀態是在無負載狀態,其中之封包漏失比小於5%(表示傳輸速度在每秒30幀(30fps)時的最大漏失為1.5fps)。當封包漏失比小於5%時,傳輸比對檢查表224可判定目前傳輸狀態係在無負載狀態,並且可以從客戶端特徵側錄達成較高的資料率。傳輸比對檢查表224可要求伺服端202以增量方式增加傳輸率,直到封包漏失比超過30%為止,亦即,直到傳輸狀態抵達壅塞狀態為止。然後,傳輸比對檢查表224可要求伺服端202降低傳輸率,直到達到負載狀態為止(封包漏失比達到一穩定狀態)。若封包漏失比指示出,傳輸狀態已達穩定狀態,則傳輸比對檢查表224可對智慧型視訊傳輸管理器208傳送一要求,其中包括一NOP信號(無操作信號或「不動作」信號,亦即,「勿改變資料率」信號)。因為網路上的通信量概況可隨時間而變化,所以客戶端212會對伺服端202隨機傳送增加資料率的要求。The transmission state can be subdivided into three sub-states: (1) no load state (2) load state (3) congestion state. Each substate can have a predetermined range of packet leakage ratios. For example, suppose the current transmission state is in a no-load state where the packet leakage ratio is less than 5% (indicating that the maximum leakage loss at a transmission speed of 30 frames per second (30 fps) is 1.5 fps). When the packet leakage ratio is less than 5%, the transmission comparison check table 224 can determine that the current transmission state is in an unloaded state, and can achieve a higher data rate from the client feature side. The transmission alignment check table 224 may require the servo terminal 202 to incrementally increase the transmission rate until the packet leakage ratio exceeds 30%, that is, until the transmission state reaches the congestion state. Then, the transmission comparison check table 224 may require the servo terminal 202 to lower the transmission rate until the load state is reached (the packet leakage ratio reaches a steady state). If the packet loss ratio indicates that the transmission status has reached a steady state, the transmission comparison check table 224 can transmit a request to the smart video transmission manager 208, including a NOP signal (no operation signal or "no action" signal, That is, the "Do not change the data rate" signal). Since the traffic profile on the network can change over time, the client 212 will randomly transmit the request for increased data rate to the server 202.

封包漏失比的閾位準可以謹慎調整,以使客戶端212之用戶所感知的顯示品質,可隨著網路狀況變更而適度下降或改進。由於客戶端212除了其本身目前的傳輸率外,並無其他可用的帶寬通信量資訊,所以客戶端212可能會定期要求伺服端202增加傳輸率,以便可以達成較高的視訊品質。除了監看進入的影音封包流品質外,客戶端212亦可通知伺服端202顯示器尺寸是否已被顧客縮小,藉此降低傳輸資料率以匹配新的顯示器大小,並降低帶寬要件。The threshold level of the packet loss ratio can be carefully adjusted so that the display quality perceived by the user of the client 212 can be moderately reduced or improved as the network conditions change. Since the client 212 has no other available bandwidth traffic information except its current transmission rate, the client 212 may periodically request the server 202 to increase the transmission rate so that higher video quality can be achieved. In addition to monitoring the incoming video packet stream quality, the client 212 can also notify the server 202 whether the display size has been reduced by the customer, thereby reducing the transmission data rate to match the new display size and reducing the bandwidth requirements.

值得注意的是,目前用於控制傳輸率的方法,稱為自適應速率控制(ARC)技術,它可用漏失封包偵測為基礎,而非已往託即時傳輸協定(RTP)之助而以緩衝器為基礎的方法。因為自適應速率控制(ARC)技術主要可以依靠有能力計算視訊流封包品質且能對伺服端傳送所需傳輸率資訊的客戶端,所以監看帶寬與資料率控制的負擔,可以從伺服端移轉到客戶端。此外,自適應速率控制(ARC)技術亦可允許客戶端要求伺服端以最佳化的傳輸率傳送資料流,減少網路中傳輸壅塞的可能性。It is worth noting that the current method for controlling the transmission rate, called adaptive rate control (ARC) technology, can be based on lossy packet detection instead of buffering with the help of the Instant Transfer Protocol (RTP). The based approach. Because the adaptive rate control (ARC) technology can mainly rely on the client that has the ability to calculate the quality of the video stream packet and can transmit the required transmission rate information to the server, the burden of monitoring the bandwidth and data rate control can be shifted from the servo end. Go to the client. In addition, Adaptive Rate Control (ARC) technology allows the client to require the server to transmit data streams at an optimized transmission rate, reducing the possibility of traffic congestion in the network.

圖5係本發明資料傳輸率控制之典型步驟流程圖500。具有一般技能之人士可以理解,圖中所示方法可用多種方式修改而不脫離本發明精神與範圍。例如,在本發明另一實施例中,圖示方法中可結合各種不同部份、以另一種順序重新安排此等不同部份、或移除此等不同部份。此外應予注意的是,此方法可用多種不同方式實施之,諸如在一通用電腦中執行軟體、使用微處理器執行韌體及/或電腦可讀媒體、使用專門用途之硬體,等等。Figure 5 is a flow chart 500 of a typical procedure for data rate control of the present invention. It will be appreciated by those skilled in the art that the methods illustrated in the figures can be modified in various ways without departing from the spirit and scope of the invention. For example, in another embodiment of the present invention, various different portions may be combined, the different portions may be rearranged in another order, or the different portions may be removed. It should also be noted that this method can be implemented in a number of different ways, such as executing software in a general purpose computer, executing firmware and/or computer readable media using a microprocessor, using specialized hardware, and the like.

本方法可從一狀態502開始。在狀態502中,一客戶端可經由一網路(區域/廣域網路)建立與一伺服端間的連線或通道,以供資料交換。然後,在狀態504中,客戶端可在通道建立開始時,對伺服端傳送一組客戶端特徵資訊。客戶端特徵資訊可包括顯示器尺寸、最高解析度、可支援色彩位元數(或色彩深度)、顯示器類型(諸如LCD、CRT、或電漿型)、CPU類型、指示可支援格式類型(諸如MPEG1、MPEG2、MPEG4、H.263、H.264等等)之解碼器能力、可用記憶體、連線類型(諸如WiFi、LAN、藍芽等等)、說明傳輸資訊的先前連線記述(諸如幀速率、資料率、量化級別、解析度等等)、及視訊顯示器220b之線路結構(續行式/交錯式)。然後,在狀態506中,伺服端可根據客戶端特徵資訊決定一傳輸率,並以即時模式使用該傳輸率對客戶端傳送一複合資料流。該複合資料可包括呈資料封包形式之音頻與視訊資料。然後,程序可進入狀態508。The method can begin with a state 502. In state 502, a client can establish a connection or channel with a server via a network (area/wide area network) for data exchange. Then, in state 504, the client can transmit a set of client feature information to the server at the beginning of channel establishment. Client feature information may include display size, highest resolution, number of supportable color bits (or color depth), display type (such as LCD, CRT, or plasma type), CPU type, indication supportable format type (such as MPEG1) , MPEG2, MPEG4, H.263, H.264, etc.) decoder capabilities, available memory, connection type (such as WiFi, LAN, Bluetooth, etc.), description of previously connected information (such as frames) Rate, data rate, quantization level, resolution, etc.), and the line structure of the video display 220b (continuous/interleaved). Then, in state 506, the server can determine a transmission rate based on the client characteristic information, and transmit the composite data stream to the client using the transmission rate in an immediate mode. The composite material may include audio and video data in the form of data packets. The program can then enter state 508.

在狀態508中,客戶端可從伺服端接收該複合資料流。其次,在狀態510中,客戶端可將接收的複合資料流分解成音頻及視訊資料封包。隨後,在狀態512中,客戶端可根據瑕疵資料封包數決定視訊資料封包之封包漏失比。然後,程序可進到一決定菱形514。In state 508, the client can receive the composite stream from the server. Second, in state 510, the client can decompose the received composite data stream into audio and video data packets. Then, in state 512, the client can determine the packet loss ratio of the video data packet according to the number of data packets. The program can then proceed to a decision diamond 514.

在狀態514中,客戶端可決定該封包漏失比是否指示網路中的壅塞狀態。狀態514的答覆為肯定時,在狀態516中,客戶端可產生降低傳輸率的要求。然後,程序可進到狀態522。否則,程序可進到另一決定菱形518。In state 514, the client may determine whether the packet miss rate ratio indicates a congestion status in the network. When the answer to state 514 is affirmative, in state 516, the client may generate a request to reduce the transmission rate. The program can then proceed to state 522. Otherwise, the program can proceed to another decision diamond 518.

在狀態518中,可以決定封包漏失比是否指示網路中的無負載狀態。狀態518的回覆若為肯定時,則程序可進到狀態519。在狀態519中,可決定傳輸率是否已達最大傳輸率。狀態519的回覆若為否定時,在狀態520中,客戶端可產生增加傳輸率的要求。然後,程序可進到狀態522。In state 518, it may be determined whether the packet miss rate ratio indicates an unloaded state in the network. If the reply to state 518 is affirmative, the program can proceed to state 519. In state 519, it may be determined whether the transmission rate has reached the maximum transmission rate. If the reply to state 519 is negative, in state 520, the client may generate a request to increase the transmission rate. The program can then proceed to state 522.

狀態518的回覆為否定時,或狀態519的回覆為肯定時,則程序進到狀態521。在狀態521中,客戶端可產生一包含NOP(勿改變資料率)信號。然後,程序可進到狀態522。在狀態522中,客戶端可對伺服端傳送該項要求。隨後,在狀態524中,伺服端可用調整後的傳輸率傳輸該複合資料流;其中,調整後傳輸率是依據要求決定的。然後,程序可等到傳輸完成後進到狀態508。值得注意的是,步驟508至524應定期實施,以維持伺服端與客戶端之間建立的通道。When the reply to state 518 is negative, or if the reply to state 519 is positive, then the program proceeds to state 521. In state 521, the client can generate a signal containing a NOP (Do Not Change Data Rate). The program can then proceed to state 522. In state 522, the client can transmit the request to the server. Subsequently, in state 524, the server can transmit the composite data stream with the adjusted transmission rate; wherein the adjusted transmission rate is determined according to requirements. The program can then wait until the transfer is complete to proceed to state 508. It is worth noting that steps 508 through 524 should be implemented periodically to maintain the channel established between the server and the client.

吾人應了解前述係有關本發明之典型實施例,其中可作多種修改而不脫離本方法如下述申請專利範圍所述精神及範圍。It is to be understood that the foregoing is a description of the exemplary embodiments of the invention, and various modifications may be made without departing from the spirit and scope of the present invention.

100...即時影音資料傳輸系統100. . . Instant video data transmission system

101...區域網路系統101. . . Regional network system

102...電視機102. . . TV set

104...放音器104. . . Sounder

106...伺服端106. . . Servo end

108...網路攝影機108. . . Webcam

110...膝上型電腦110. . . Laptop

112...路由器112. . . router

114...個人電腦114. . . personal computer

116...個人電腦116. . . personal computer

118...防火牆118. . . Firewall

120...空中(網際網路)120. . . Air (internet)

121...廣域網路系統121. . . WAN system

122...通達點(熱點)122. . . Access point (hot spot)

124...個人數位助理124. . . Personal digital assistant

126...手機126. . . Mobile phone

128...個人電腦128. . . personal computer

130...網路纜線或WiFi通道130. . . Network cable or WiFi channel

200...伺服端與客戶端間即時模式資料流方塊圖200. . . Instant mode data flow block diagram between server and client

202...伺服端202. . . Servo end

204...影音複合系統204. . . Audio-visual composite system

205...影音輸入流205. . . Video input stream

206...媒體存取控制緩衝器206. . . Media access control buffer

208...智慧型視訊傳輸管理器208. . . Smart Video Transfer Manager

210...網路(區域/廣域)210. . . Network (region/wide area)

212...客戶端212. . . Client

214...收件緩衝器214. . . Receiving buffer

218...影音複合分解系統218. . . Audio-visual composite decomposition system

220a...放音器/揚聲器220a. . . Speaker/speaker

220b...視訊顯示器220b. . . Video display

222...封包錯誤累加器222. . . Packet error accumulator

224...傳輸比對檢查表224. . . Transmission comparison checklist

230...音頻編碼器230. . . Audio encoder

232...音頻環狀緩衝器232. . . Audio ring buffer

234...視訊編碼器234. . . Video encoder

236...視訊環狀緩衝器236. . . Video ring buffer

238...影音複合器238. . . Audio and video composite

240...影音複合分解器240. . . Video composite resolver

242...音頻環狀緩衝器242. . . Audio ring buffer

244...視訊環狀緩衝器244. . . Video ring buffer

500...資料傳輸率控制之典型步驟流程圖500. . . Flow chart of typical steps of data transmission rate control

502-524...資料傳輸率控制各步驟(狀態)502-524. . . Data transfer rate control steps (status)

圖1係根據本發明之即時影音資料流傳輸系統示意圖;圖2係方塊圖,說明圖1所述伺服端與客戶端之一間的資料流;圖3係圖2所示伺服端內實施之影音複合系統方塊圖;圖4係圖2所示客戶端內實施之影音複合分解系統方塊圖;及圖5係本發明資料傳輸率控制之典型步驟流程圖。1 is a schematic diagram of a real-time video and audio data stream transmission system according to the present invention; FIG. 2 is a block diagram showing the data flow between the server and the client of FIG. 1; FIG. 3 is implemented in the server shown in FIG. Block diagram of video and audio composite system; FIG. 4 is a block diagram of a video/audio composite decomposition system implemented in the client shown in FIG. 2; and FIG. 5 is a flow chart of typical steps of data transmission rate control according to the present invention.

100...即時影音資料傳輸系統100. . . Instant video data transmission system

101...區域網路系統101. . . Regional network system

102...電視機102. . . TV set

104...放音器104. . . Sounder

106...伺服端106. . . Servo end

108...網路攝影機108. . . Webcam

110...膝上型電腦110. . . Laptop

112...路由器112. . . router

114...個人電腦114. . . personal computer

116...個人電腦116. . . personal computer

118...防火牆118. . . Firewall

120...空中(網際網路)120. . . Air (internet)

121...廣域網路系統121. . . WAN system

122...通達點(熱點)122. . . Access point (hot spot)

124...個人數位助理124. . . Personal digital assistant

126...手機126. . . Mobile phone

128...個人電腦128. . . personal computer

130...網路纜線或WiFi通道130. . . Network cable or WiFi channel

Claims (20)

一種伺服端至客戶端資料流傳輸率之即時控制方法,其中該客戶端以動態的客戶端特徵變數產生一組比對檢查表,並用一動態更新資料的封包漏失比作為一比對檢查表的引索來存取該比對檢查表,此方法包括:該客戶端以一第一傳輸率從該伺服端接收一資料流之第一部份,該第一部份包括多數資料封包;該客戶端檢查該等資料封包之完整性;該客戶端根據該完整性推導出上述動態更新資料的封包漏失比;該客戶端產生一要求給伺服器,以調整該第一傳輸率,其中該客戶端調整該第一傳輸率係依據該動態更新資料的封包漏失比以及該客戶端的比對檢查表中的一個以上動態變數配合該組動態的客戶端特徵變數來決定,該組動態的客戶端特徵變數會在資料由該伺服器傳輸至該客戶端的過程中不斷改變;該客戶端將該項要求傳送給該伺服端;該伺服端以一智能型視訊傳輸管理器處理該項要求,該智能型視訊傳輸管理器依據該項要求調整該第一傳輸率或一亮度色度(YUV)轉換法,將較高的彩度視頻內容轉換至較低的視頻彩度內容,反之亦然;以及該客戶端以一調整後傳輸率從該伺服端接收該資料流之第二部份。 An instant control method for data rate of a server-to-client data stream, wherein the client generates a set of comparison checklists with dynamic client characteristic variables, and uses a packet loss ratio of a dynamically updated data as a comparison checklist. Soliciting access to the comparison checklist, the method comprising: the client receiving a first portion of a data stream from the server at a first transmission rate, the first portion comprising a majority data packet; the client Checking the integrity of the data packet; the client derives a packet loss ratio of the dynamic update data according to the integrity; the client generates a request to the server to adjust the first transmission rate, wherein the client Adjusting the first transmission rate is determined according to the packet loss ratio of the dynamic update data and one or more dynamic variables in the comparison check table of the client, which are matched with the dynamic client feature variables, and the dynamic client feature variables of the group are determined. Will change continuously while the data is transmitted by the server to the client; the client transmits the request to the server; the server is The capable video transmission manager processes the request, and the intelligent video transmission manager adjusts the first transmission rate or a luminance chrominance (YUV) conversion method according to the requirement to convert the higher chroma video content to the comparison. Low video chroma content, and vice versa; and the client receives the second portion of the data stream from the server at an adjusted transmission rate. 如申請專利範圍第1項之方法,在該接收一資料流第一 部份之步驟前,進而包括:在該伺服端與該客戶端之間建立一連線;使該客戶端傳送該一個以上的客戶端特徵變數中的值給該伺服端;以及使該伺服端使用該一個以上的客戶端特徵變數中的值來決定該第一傳輸率,並以該第一傳輸率傳輸該第一部份給客戶端。 For example, in the method of claim 1, the first data stream is received Before the step, the method further includes: establishing a connection between the server and the client; causing the client to transmit the value in the one or more client feature variables to the server; and causing the server The first transmission rate is determined using the value of the one or more client feature variables, and the first portion is transmitted to the client at the first transmission rate. 如申請專利範圍第2項之方法,其中該組客戶端特徵變數包括該亮度色度(YUV)轉換法,根據動態更新資料的封包漏失比,使較高的彩度視頻內容資料轉換成較低的彩度視頻內容資料,反之亦然。 The method of claim 2, wherein the set of client feature variables comprises the luminance chrominance (YUV) conversion method, and the higher chroma video content data is converted to lower according to the packet loss ratio of the dynamically updated data. The chroma video content and vice versa. 如申請專利範圍第1項之方法,其中該資料流為包括一音頻流與一視訊流之複合資料流;本方法在該檢查完整性之步驟前,進而包括:將該複合資料流之第一部份分解為該音頻流與視訊流。 The method of claim 1, wherein the data stream is a composite data stream comprising an audio stream and a video stream; the method, before the step of checking integrity, further comprises: first Partially decomposed into the audio stream and the video stream. 如申請專利範圍第4項之方法,在該檢查完整性之步驟前,進而包括:將該等音頻流與視訊流解碼。 For example, the method of claim 4, before the step of checking integrity, further comprises: decoding the audio stream and the video stream. 如申請專利範圍第1項之方法,其中決定該封包漏失比之步驟包括以下步驟:量取該等多數資料封包內之瑕疵資料封包百分比;該等瑕疵封包或為轉訛資料封包,或為在該接收資料流第一部份之步驟中遺失之資料封包。 For example, in the method of claim 1, wherein the step of determining the packet loss ratio includes the steps of: measuring the percentage of the data packet in the majority of the data packets; the packet is either a data packet, or The data packet lost in the step of receiving the first part of the data stream. 如申請專利範圍第1項之方法,其中該產生調整要求之 步驟包括以下步驟:決定該封包漏失比是否指示一壅塞狀態;若該動態更新資料的封包漏失比指示一壅塞狀態,則產生一降低該傳輸率之要求;決定該封包漏失比是否指示一無負載狀態、一最大傳輸速率以及;若該動態更新資料的封包漏失比並非指示一無負載狀態,則產生一包括NOP(無操作)信號之要求;若該動態更新資料的封包漏失比指示一無負載狀態,則決定該第一傳輸率是否已達一最高傳輸率;若該第一傳輸率尚未達到一最高傳輸率,則產生一增加該第一傳輸率之要求;以及若該第一傳輸率已達到該最高傳輸率,則產生一包括NOP(無操作)信號之要求。 For example, the method of claim 1 of the patent scope, wherein the adjustment request is generated The step includes the following steps: determining whether the packet loss ratio indicates a congestion state; if the packet loss ratio of the dynamic update data indicates a congestion state, generating a requirement for lowering the transmission rate; determining whether the packet loss ratio indicates a no load a state, a maximum transmission rate, and if the packet loss ratio of the dynamic update data does not indicate an unloaded state, generating a request including a NOP (no operation) signal; if the packet loss ratio of the dynamic update data indicates a no load a state, determining whether the first transmission rate has reached a highest transmission rate; if the first transmission rate has not reached a maximum transmission rate, generating a request for increasing the first transmission rate; and if the first transmission rate has been Achieving this highest transmission rate results in a requirement to include a NOP (no operation) signal. 如申請專利範圍第1項之方法,其中該資料流係根據一使用者資料包協定(UDP)格式化。 The method of claim 1, wherein the data stream is formatted according to a User Data Package Agreement (UDP). 如申請專利範圍第1項之方法,其中該資料流係透過一網路傳輸,該網路至少包括一區域網路(LAN)與一廣域網路(WAN)。 The method of claim 1, wherein the data stream is transmitted through a network, the network comprising at least a local area network (LAN) and a wide area network (WAN). 如申請專利範圍第1項之方法,其中該客戶端包括一目視裝置。 The method of claim 1, wherein the client comprises a visual device. 如申請專利範圍第1項之方法,進而包括:重覆該完整性檢查之步驟至該接收第二部份之步驟,直到該客戶端收到該資料流的整體部份。 The method of claim 1, further comprising: repeating the step of the integrity check to the step of receiving the second portion until the client receives the entire portion of the data stream. 如申請專利範圍第1項之方法,該傳輸率具有一組參數,至少包括一目標幀速率、一目標顯示位元率、一目標解析度、一目標音頻抽樣率、一量化值、一亮度色度(YUV)轉換法、一比例尺大小、及一說明以往連線狀態的連線記述。 For example, in the method of claim 1, the transmission rate has a set of parameters including at least a target frame rate, a target display bit rate, a target resolution, a target audio sampling rate, a quantized value, and a gray color. Degree (YUV) conversion method, a scale size, and a description of the connection state of the previous connection state. 一種即時控制資料流傳輸率的方法,其中該客戶端結合客戶特徵變數產生一比對檢查表,並且利用動態更新資料的封包漏失比作為比對檢查表的引索,此方法包括:接收一客戶端要求,以調整該傳輸率或調整該亮度色度(YUV)轉換法,將客戶端中較高的彩色視頻內容資料轉換成較低的彩色視頻內容資料,反之亦然,且其中該客戶端將該動態更新資料的封包漏失比累積以形成該比對檢查表的引索,以存取一個以上比對檢查表中的值,以產生該客戶端的要求,且其中該客戶端調整該傳輸率係依據動態資料的封包漏失比以及該客戶端的比對檢查表中的一個以上動態變數配合該組動態的客戶端特徵變數來決定,該組動態的客戶端特徵變數會在資料由該伺服器傳輸至該客戶端的過程中不斷改變;藉由一音訊/視訊多工器系統組,來接收至少一音訊輸入以及至少一視訊輸入;以及從該客戶端接受該客戶端的要求,以調整該傳輸率或調整該亮度色度(YUV)轉換法,用以將較高的彩色視頻內容轉換至較低的視頻彩度內容,反之亦然,且其中接收該客戶端要求的步驟,最後係根據一智能型視訊傳 輸管理器,而不是該客戶端的要求做調整;在該智能型視訊傳輸管理器中處理該客戶端的要求,調整該傳輸率或調整該亮度色度(YUV)轉換法,用以將較高的彩色視訊內容資料轉換成較低的彩色視訊內容資料,反之亦然,且該智能型視頻傳輸管理器對於處理客戶端要求的程序包含:根據客戶端要求解碼該音訊輸入以及視訊輸入;根據客戶端要求多工處理該解碼後的音訊和視訊輸入,形成資料流;根據客戶端要求,傳輸該資料流。 A method for instantly controlling a data stream transmission rate, wherein the client generates a comparison check table in combination with a customer feature variable, and utilizes a packet loss ratio of the dynamic update data as a reference for the comparison checklist, the method comprising: receiving a client End request to adjust the transmission rate or adjust the luminance chrominance (YUV) conversion method to convert the higher color video content data in the client into lower color video content data, and vice versa, and wherein the client The packet loss ratio of the dynamic update data is accumulated to form a reference of the comparison check table to access values in one or more comparison check tables to generate a request of the client, and wherein the client adjusts the transmission rate Based on the packet loss ratio of the dynamic data and one or more dynamic variables in the comparison check table of the client, the dynamic client characteristic variables are determined by the server. Changing to the client; receiving at least one audio input and at least one by an audio/video multiplexer system group Inputting; and accepting the client's request from the client to adjust the transmission rate or adjust the luminance chrominance (YUV) conversion method to convert higher color video content to lower video chroma content, And vice versa, and the steps of receiving the client request are finally based on a smart video transmission. The change manager, rather than the client's request to make adjustments; handle the client's requirements in the smart video transfer manager, adjust the transfer rate or adjust the luminance chromaticity (YUV) conversion method to use the higher The color video content data is converted into lower color video content data, and vice versa, and the intelligent video transmission manager processes the client requirements: decoding the audio input and video input according to the client request; according to the client The multiplexed processing of the decoded audio and video input is required to form a data stream; the data stream is transmitted according to the requirements of the client. 如申請專利範圍第13項之方法,其中該比對檢查表係由客戶端產生,且至少包括顯示器尺寸、最高解析度、色彩深度、顯示器類型、CPU類型、一或多種解碼器能力、可用記憶體、連線類型、具有傳輸資訊的先前連線記述、及視訊顯示器線路結構等其中之一。 The method of claim 13, wherein the comparison checklist is generated by the client and includes at least a display size, a maximum resolution, a color depth, a display type, a CPU type, one or more decoder capabilities, and an available memory. One of the body, the type of connection, the previous connection description with the transmission information, and the line structure of the video display. 一種即時的自適性媒體資料封包連續資料流傳輸管理的方法,此方法包括:在該伺服端與該客戶端之間建立一連線;該伺服器藉由LAN或WAN傳送多媒體資料封包的資料流第一部分給客戶端,其中該伺服器包含一智能型視訊傳輸管理器,根據客戶端提供的資料的封包漏失比來調整資料速率或調整亮度色度(YUV)轉換法,用以將較高的彩色視頻內容資料轉換成較低的彩色視頻內容資料; 該客戶端藉由該LAN或WAN由該伺服器接收多媒體資料封包的資料流第一部分,且其中該客戶端具有一比對檢查表,該比對檢查表包含偵測一多媒體資料封包第二部分調整成該亮度色度(YUV)轉換法的需求,該第二部分係即將傳輸給客戶端,且該客戶端基於一動態更新資料的封包漏失比以及一個以上比對檢查表中的動態變數值結合一組在資料封包持續傳輸過程中不斷變動的動態的客戶端特徵變數,獨立地要求該伺服器調整速度及亮度色度(YUV)轉換法;以及該LAN或WAN有效連接至該伺服器以及客戶端,其中該LAN或WAN提供資料封包傳輸的路徑。 A method for real-time adaptive media data packet continuous data stream transmission management, the method comprising: establishing a connection between the server and the client; the server transmitting the data stream of the multimedia data packet by using a LAN or a WAN The first part is given to the client, wherein the server includes an intelligent video transmission manager, which adjusts the data rate or adjusts the luminance and chrominance (YUV) conversion method according to the packet loss ratio of the data provided by the client, so as to be higher. Converting color video content data into lower color video content data; The client receives the first part of the data stream of the multimedia data packet by the server by using the LAN or the WAN, and wherein the client has a comparison check table, where the comparison check table includes detecting the second part of the multimedia data packet. Adjusted to the requirement of the luminance chrominance (YUV) conversion method, the second part is to be transmitted to the client, and the client is based on the packet loss ratio of a dynamically updated data and the dynamic variable value in one or more comparison checklists Combining a set of dynamic client feature variables that are constantly changing during the continuous transmission of the data packet, the server is independently required to adjust the speed and luminance chrominance (YUV) conversion method; and the LAN or WAN is operatively connected to the server and Client, where the LAN or WAN provides a path for data packet transmission. 如申請專利範圍第15項之方法,在該傳輸第二部份之步驟前,進而包括:一音訊/視訊多工器系統以接收及解碼一音訊/視訊接收資料流,以及以一MAC緩衝器來暫存一多媒體資料封包的解碼資料流。 For example, in the method of claim 15, before the step of transmitting the second part, the method further comprises: an audio/video multiplexer system for receiving and decoding an audio/video receiving data stream, and a MAC buffer To temporarily store the decoded data stream of a multimedia data packet. 如申請專利範圍第15項之方法,進而包括:一收件緩衝器來累積該客戶端所傳輸的多媒體資料封包解碼資料流,以及一音訊/視訊的解多工器系統,來解碼至少一部分多媒體資料解碼資料流。 The method of claim 15, further comprising: a receiving buffer to accumulate the multimedia data packet decoded data stream transmitted by the client, and an audio/video demultiplexer system to decode at least a portion of the multimedia Data decoding data stream. 如申請專利範圍第17項之方法,其中該客戶端進一步包含一封包錯誤累加器,用以即時計算該資料的封包漏失比,且該資料的封包漏失比係用來作為客戶端比對檢查表中的比對檢查表引索。 The method of claim 17, wherein the client further comprises a packet error accumulator for calculating the packet loss ratio of the data in real time, and the packet loss ratio of the data is used as a client comparison checklist. The comparison checklist in the reference. 如申請專利範圍第15項之方法,其中該客戶端發出一要求給伺服器,以調整該多媒體資料封包第二部分的亮度色度(YUV)轉換法,該第二部分係由伺服端傳輸至客戶端的。 The method of claim 15, wherein the client sends a request to the server to adjust a luminance chrominance (YUV) conversion method of the second portion of the multimedia data packet, the second portion being transmitted by the server to Client's. 如申請專利範圍第19項之方法,其中該伺服器調整該多媒體資料封包第二部分的亮度色度(YUV)轉換法。 The method of claim 19, wherein the server adjusts a luminance chrominance (YUV) conversion method of the second portion of the multimedia data packet.
TW095100616A 2006-01-06 2006-01-06 Real-time control method of servo-to-client data stream transfer rate TWI475842B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
TW095100616A TWI475842B (en) 2006-01-06 2006-01-06 Real-time control method of servo-to-client data stream transfer rate

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
TW095100616A TWI475842B (en) 2006-01-06 2006-01-06 Real-time control method of servo-to-client data stream transfer rate

Publications (2)

Publication Number Publication Date
TW200727618A TW200727618A (en) 2007-07-16
TWI475842B true TWI475842B (en) 2015-03-01

Family

ID=53185879

Family Applications (1)

Application Number Title Priority Date Filing Date
TW095100616A TWI475842B (en) 2006-01-06 2006-01-06 Real-time control method of servo-to-client data stream transfer rate

Country Status (1)

Country Link
TW (1) TWI475842B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8532139B2 (en) * 2007-09-21 2013-09-10 Broadcom Corporation Method and system for indicating a transition in rate and/or power consumption utilizing a distinct physical pattern on one or more idle channel(s)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5928330A (en) * 1996-09-06 1999-07-27 Motorola, Inc. System, device, and method for streaming a multimedia file
US20030016673A1 (en) * 2001-06-29 2003-01-23 Ramesh Pendakur Correcting for data losses with feedback and response
US20030039398A1 (en) * 2001-08-21 2003-02-27 Mcintyre Kristen A. Dynamic bandwidth adaptive image compression/decompression scheme
TWI243593B (en) * 2004-03-05 2005-11-11 Chunghwa Telecom Co Ltd Independent MPEG-2 hardware coding system with a VGA interface

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5928330A (en) * 1996-09-06 1999-07-27 Motorola, Inc. System, device, and method for streaming a multimedia file
US20030016673A1 (en) * 2001-06-29 2003-01-23 Ramesh Pendakur Correcting for data losses with feedback and response
US20030039398A1 (en) * 2001-08-21 2003-02-27 Mcintyre Kristen A. Dynamic bandwidth adaptive image compression/decompression scheme
TWI243593B (en) * 2004-03-05 2005-11-11 Chunghwa Telecom Co Ltd Independent MPEG-2 hardware coding system with a VGA interface

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Jean-Chrysostome Bolot, etc., "Scalable Feedback Control for Multicast Video Distribution in the Internet", SIGCOMM 94 -8/94 London England UK, 1994 ACM. *

Also Published As

Publication number Publication date
TW200727618A (en) 2007-07-16

Similar Documents

Publication Publication Date Title
US7984179B1 (en) Adaptive media transport management for continuous media stream over LAN/WAN environment
TWI383684B (en) System and method for dynamic video encoding in multimedia streaming
US8446452B2 (en) Video rate adaptation for congestion control
JP4195030B2 (en) Video data transmission method and reception method for continuous video display
US7333439B2 (en) Sending/receiving method, and device for the same
US8471890B1 (en) Adaptive video communication channel
US9578179B2 (en) Method, apparatus and system for transmitting multimedia data
US7881335B2 (en) Client-side bandwidth allocation for continuous and discrete media
WO2009128528A1 (en) Server device, content distribution method, and program
US20080025414A1 (en) Apparatus and method for transmitting moving picture stream using bluetooth
US11197051B2 (en) Systems and methods for achieving optimal network bitrate
EP3016395B1 (en) Video encoding device and video encoding method
WO2020086452A1 (en) Low-latency video internet streaming for management and transmission of multiple data streams
CN114339445B (en) Real-time video stream self-adaptive coordination system and method
JP3708950B2 (en) Transmission / reception method and apparatus
US20110088069A1 (en) Network device, information processing apparatus, stream switching method, information processing method, program, and content distribution system
JP2011501478A (en) Traffic control method for IPTV broadcast service
KR100792025B1 (en) Method of transmitting video data in case of channel change in iptv system
CN1996813B (en) Self-adapted media transfer management of the continuous media stream used for LAN/WAN environment
Fitzek et al. Video streaming in wireless internet
TWI475842B (en) Real-time control method of servo-to-client data stream transfer rate
KR101625663B1 (en) Method and Apparatus for Receiving Content
KR20050093438A (en) Frame rate control method according to packet loss rate
Zhu et al. Research on adaptive transmission of H. 264 video stream and QoS guarantee based on SIP
Sarni et al. A novel scheme for a fast channel change in multicast IPTV system

Legal Events

Date Code Title Description
MM4A Annulment or lapse of patent due to non-payment of fees