WO2010117644A1 - Procédé et appareil pour une transmission vidéo asynchrone sur un réseau de communication - Google Patents

Procédé et appareil pour une transmission vidéo asynchrone sur un réseau de communication Download PDF

Info

Publication number
WO2010117644A1
WO2010117644A1 PCT/US2010/028564 US2010028564W WO2010117644A1 WO 2010117644 A1 WO2010117644 A1 WO 2010117644A1 US 2010028564 W US2010028564 W US 2010028564W WO 2010117644 A1 WO2010117644 A1 WO 2010117644A1
Authority
WO
WIPO (PCT)
Prior art keywords
frame
video
video frame
encoded
request
Prior art date
Application number
PCT/US2010/028564
Other languages
English (en)
Inventor
Mark Edwards
Dean S. Dyson
Original Assignee
Motorola, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Motorola, Inc. filed Critical Motorola, Inc.
Priority to EP10727998A priority Critical patent/EP2417766A1/fr
Publication of WO2010117644A1 publication Critical patent/WO2010117644A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/61—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio
    • H04L65/613—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio for the control of the source by the destination
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/16—Analogue secrecy systems; Analogue subscription systems
    • H04N7/173—Analogue secrecy systems; Analogue subscription systems with two-way working, e.g. subscriber sending a programme selection signal
    • H04N7/17309—Transmission or handling of upstream communications
    • H04N7/17318—Direct or substantially direct transmission and handling of requests
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/162—User input
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/164—Feedback from the receiver or from the transmission channel
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/2187—Live feed
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H04N21/44209—Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/637—Control signals issued by the client directed to the server or network components
    • H04N21/6375—Control signals issued by the client directed to the server or network components for requesting retransmission, e.g. of data packets lost or corrupted during transmission from server
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/637—Control signals issued by the client directed to the server or network components
    • H04N21/6377—Control signals issued by the client directed to the server or network components directed to server
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/637—Control signals issued by the client directed to the server or network components
    • H04N21/6377—Control signals issued by the client directed to the server or network components directed to server
    • H04N21/6379—Control signals issued by the client directed to the server or network components directed to server directed to encoder, e.g. for requesting a lower encoding rate
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/65—Transmission of management data between client and server
    • H04N21/658—Transmission by the client directed to the server
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/65—Transmission of management data between client and server
    • H04N21/658—Transmission by the client directed to the server
    • H04N21/6587—Control parameters, e.g. trick play commands, viewpoint selection
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/183—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a single remote source

Definitions

  • the technical field relates generally to communication systems, and in particular, it relates to a method and apparatus for asynchronously transmitting video over wired or wireless networks.
  • video in addition to or in place of audio, is transmitted over wired and wireless networks.
  • Various protocols and network requirements and usages impose limitations, however, on the transmission of video.
  • meeting public safety requirements for live video transmission in a wireless network is difficult today.
  • Content passes through a number of nodes, routers, and/or computers on its way to the end user or client.
  • bandwidth is the data transmission rate capacity or maximum amount of information, in bits/second, that can be transmitted along a channel
  • end user clients buffer streamed video content such that a certain amount of the video data is downloaded before it begins to play so that the end client is working through previously received video data while more video data is being downloaded.
  • Video is more susceptible than audio to such interruptions.
  • a video encoder for example one included in a video codec that can both encode and decode a digital data stream or signal
  • available data bandwidth in a wireless network can vary significantly during a video session due to client mobility (e.g., due to signal strength variations, cell handovers, sharing with other clients, etc.).
  • the network For a mobile video client in a wireless network, frequently the network is not able to provide enough bandwidth to transport the video encoder's total bit stream for at least a portion of the video session due to, for example, traffic congestion. This can lead to pauses and breaks in the video transmission and reception and a steady increase in the end-to-end delay making it difficult to watch the video with any degree of continuity. Thus, a client's operational requirements for video will not be reliably met, even in a broadband wireless network.
  • bandwidth variations mean that video servers have to employ "workarounds", such as manually reconfiguring video encoder bit rates during the video session in an attempt to match prevailing wireless conditions or configuring the encoder to work continuously at a minimum acceptable bandwidth and hope that the chosen bandwidth is available through the entire video session.
  • workarounds such as manually reconfiguring video encoder bit rates during the video session in an attempt to match prevailing wireless conditions or configuring the encoder to work continuously at a minimum acceptable bandwidth and hope that the chosen bandwidth is available through the entire video session.
  • an end-to-end delay builds up, such as at nodes and routers, in the network infrastructure, and the video sequence is interrupted as bandwidth drops below the minimum level needed.
  • FIG. 1 is a block diagram illustrating a video streaming system in accordance with some embodiments.
  • FIG. 2 is a high level flow diagram of a method for asynchronous live video transmission in accordance with some embodiments.
  • FIG. 3 is a detailed flow diagram of a method for asynchronous live video transmission in accordance with some embodiments.
  • Skilled artisans will appreciate that elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. For example, the dimensions of some of the elements in the figures may be exaggerated relative to other elements to help improve understanding of various elements. In addition, the description and drawings do not necessarily require the order illustrated. It will be further appreciated that certain actions and/or steps may be described or depicted in a particular order of occurrence while those skilled in the art will understand that such specificity with respect to sequence is not actually required.
  • a video server implements a method for asynchronous video transmission over one or more communication networks.
  • the method includes: receiving, from a video client, a frame request for a single video frame, wherein the frame request includes a first set of encoding parameters determined by the video client; capturing a video frame that has not been previously sent to the video client, wherein the video frame comprises a portion of live video captured by a video camera; encoding the video frame using the first set of encoding parameters to produce one encoded video frame; and transmitting the one encoded video frame such that only the one encoded video frame is in transit over the communication network in response to the frame request.
  • the video server uses a motion compensation algorithm to encode the video frames as predicted (P) frames and intra (I) frames, wherein I-frame periodicity is determined by the video client.
  • the video client can request retransmission of lost P frames and I frames from the video server, wherein the video server is able to adapt P frame encoding if necessary depending on information from the video client regarding which frame has been lost.
  • the video server can generate P frames asynchronously, and the video client accordingly decodes P frames asynchronously.
  • the video server receives a frame request
  • the video server updates its codec encoding parameters using the second set of encoding parameters; retrieves and encodes a next video frame (or a previously sent video frame needing retransmission) using the second encoding parameters; and transmits (retransmits) the encoded video frame in response to the frame request (or retransmission request).
  • the second set of encoding parameters are determined by the video client in response to detecting an increased delay in receiving an encoded video frame or in response to a failure to receive the encoded video frame altogether, wherein the video client adjusts the encoding parameters for the video server to decrease the size of the encoded video frame so that an acceptable frame rate is achieved at the video client given the current network bandwidth conditions and/or video client connectivity to the network.
  • the video client changes the encoding parameters to balance a tradeoff between quality of displayed video versus frame rate at the video client.
  • the video client changes the encoding parameters in response to improved network bandwidth and/or video client network connectivity.
  • Such a dynamic methodology enables the video client to minimize breaks and delays in the video stream being displayed on the client side when the network is experiencing bandwidth limitations or the video client is experiencing network connectivity limitations.
  • the video client can adjust the encoding parameters to enable higher resolution and/or frame rate when network bandwidth and/or connectivity improves.
  • FIG. 1 there is shown a block diagram illustrating an asynchronous live video transmission system 100 (i.e., a video transmission system that asynchronously transmits live video frames) in accordance with principles of the present disclosure.
  • asynchronous i.e., a video transmission system that asynchronously transmits live video frames
  • the term "asynchronous”, “asynchronously”, or other variations of the word is defined as of or pertaining to operations without the use of fixed time intervals, or may also be defined as having a operation start only after a previous operation has been completed.
  • live video is defined as an electronic representation of live scenes that is captured by a video camera in real time as the scenes are occurring, wherein the live video comprises a sequence of video frames (also referred to herein as live video frames), wherein a video frame represents the smallest media unit of live video that may be processed and transmitted by a video server over a communication network.
  • Video transmission system 100 uses a client-pull technology, wherein a video server transmits encoded video frames only in response to a frame request (or retransmission request) from a video client.
  • System 100 comprises a video server 104 and a video client 132 communicatively coupled together over a communication network 130.
  • Communication network 130 may represent one or more wired or wireless communication networks, or combinations thereof, and includes one or more infrastructure devices used to facilitate communications (including live video communications) over the network 130.
  • An infrastructure device is a device that is part of a fixed network infrastructure and can receive information (either control or media, e.g., voice (audio), video, etc.) in a signal and transmit information in signals to one or more end user devices via a communication link.
  • Examples of infrastructure devices include, but are not limited to, equipment commonly referred to as controllers, base stations, base transceiver stations, access points, routers, and the like.
  • System 100 further comprises a video camera 102 operatively coupled to the video server 104, wherein the video camera 102 provides live video that is processed by the video server and transmitted as a sequence of encoded video frames over communication network 130 to the video client 104.
  • system 100 includes a video display 134 operatively to the video client 132 to display video frames that are received and decoded by the video client.
  • Video server 104 comprises an analog video input 106 that receives live video from the video camera 102 in an analog signal format such as: an RGB (red, green, blue) component analog signal carried as three separate signals, wherein the video input 106 includes three BNC or RCA electrical jacks coupled to the video camera 102 via one or more suitable cables having corresponding plugs or connectors at both ends of the cable; or a composite analog signal such as NTSC, PAL, or SECAM, wherein the video input 106 includes an RCA jack coupled to the video camera 102 via a suitable cable having corresponding plugs or connectors at both ends of the cable.
  • an RGB red, green, blue
  • analog video input 106 comprises known elements (e.g., processing and transceiver (i.e., transmitter and receiver) elements, wherein some of the functionality of the processing and transceiver may be performed in processing device(s) 110) that enable short range wireless transmission techniques such as Bluetooth technology.
  • Video server 104 further comprises: one or more processing devices
  • I/O input/output
  • I/O interface 108 may comprise a serial port interface (e.g., RS-232-C, EIA-232-D, or EIA-232-E) or known elements (e.g., processing and transceiver elements, wherein some of the functionality of the processing and transceiver may be performed in processing device(s) 110) that enable short range wireless transmission techniques such as Bluetooth technology.
  • serial port interface e.g., RS-232-C, EIA-232-D, or EIA-232-E
  • known elements e.g., processing and transceiver elements, wherein some of the functionality of the processing and transceiver may be performed in processing device(s) 110 that enable short range wireless transmission techniques such as Bluetooth technology.
  • I/O interface 108 comprises known elements including processing, modulating, and transceiver elements that are operable in accordance with any one or more standard or proprietary wireless interfaces, wherein some of the functionality of the processing, modulating, and transceiver elements may be performed in processing device(s) 110.
  • Processing device(s) 110 comprise logic including, but not limited to, a frame capturer 112, a codec 114, and a frame request processor 116.
  • the frame capturer 112 "captures" or pulls analog video frames from the video camera 102 via the analog video input 106 and converts the analog frames into a raw digital format, also known as a raw image file or a digital negative.
  • a raw image file is defined as minimally processed data from the image sensor of a video camera that is not usable as an image having a viewable format but has all of the information needed to create an image that that has a viewable format. If the analog frames are provided by the video camera 102 at a faster rate than is requested by the video client 132, the video server 104 is configured to "drop" some of the analog frames or raw frames. CODEC 114 encodes the raw image file using programmed encoding parameters to generate an encoded video frame having a viewable format that may or may not use motion compensation (i.e., the use of I, P, and/or B frames) such as, for instance, JPEG, MPEG, H.264, etc.
  • motion compensation i.e., the use of I, P, and/or B frames
  • the encoding algorithm used generally compresses the amount of data that needs to be transmitted relative to the data included in the corresponding raw video frame.
  • the frame request processor 116 receives and processes frame requests and retransmission requests received at the I/O interface 108 from the video client 132.
  • Memory unit(s) 120 include suitable data storage capability to implement: a transmit buffer 122 that stores one or more encoded video frames that have not yet been transmitted to the video client; and a frame storage 124 that stores one or more previously transmitted frames, such as a last transmitted (Tx) frame 126 that may be used in response to a retransmission request and/or motion compensation processing.
  • a transmit buffer 122 that stores one or more encoded video frames that have not yet been transmitted to the video client
  • a frame storage 124 that stores one or more previously transmitted frames, such as a last transmitted (Tx) frame 126 that may be used in response to a retransmission request and/or motion compensation processing.
  • the video server 104 stores a most recently transmitted video frame in frame storage 124 to produce a recorded video frame, and uses the recorded video frame as a reference frame to derive a next frame for encoding and transmitting as a predicted frame.
  • the transmit buffer is implemented as a rolling buffer that stores a plurality of sequential encoded video frames.
  • each encoded video frame is transmitted such that only one encoded video frame is transmitted in response to any given frame (or retransmission) request processed by the frame request processor 116 and such that only one encoded video frame at a time is in transit over the communication network 130 in response to a given frame (or retransmission) request.
  • Video client 132 comprises an input/output (I/O) interface 136 suitable for receiving the encoded video frames over the communication network 130 depending on the type of network being used.
  • I/O interface 136 may comprise a serial port interface (e.g., RS-232-C, EIA-232-D, or EIA-232-E) or known elements (e.g., processing and transceiver elements, wherein some of the functionality of the processing and transceiver may be performed in processing device(s) 140) that enable short range wireless transmission techniques such as Bluetooth technology.
  • I/O interface 136 comprises known elements including processing, modulating, and transceiver elements that are operable in accordance with any one or more standard or proprietary wireless interfaces (as corresponds to those implemented in the video server), wherein some of the functionality of the processing, modulating and transceiver elements may be performed in processing device(s) 140.
  • Video client 132 further comprises: one or more memory storage units
  • a display driver 138 that includes the necessary processing and hardware for presenting the decoded video frames to the video display 134 so that the live video can be seen by a viewer of the video display 134.
  • Memory unit(s) 150 include at least a receive buffer 152 and frame storage 154 and may also include further storage capability for storing application software used to program processing device(s) 110, thereby enabling the video client functionality in accordance with the teachings herein.
  • Receive buffer 152 stores a single encoded video frame that is currently being received via interface 136 (in response to a frame or retransmission request) and that will next be processed by the video client.
  • Frame storage 154 stores a last encoded frame 156 that was received and processed by the video client. The last received (Rx) video frame 156 is stored, for example, for use in motion compensation processing.
  • Processing device(s) 140 comprise at least a frame request processor
  • Frame request processor 142 generates frame requests and if needed retransmission requests, each for a single encoded video frame from the video server 104. In one embodiment, the frame request processor 142 generates a next frame request only after receiving an encoded video frame into receive buffer 152. In another embodiment, the frame request processor 142 generates a next frame request only after receiving an encoded video frame into receive buffer 152 and after the CODEC 144 properly decodes the frame that is stored in the receive buffer 152.
  • the frame request processor 142 can change encoding parameters such as: resolution (which means a number of pixels included in a video frame); quantization (which means a number of bits used to encode each pixel of the video frame); whether a next frame to be encoded is an infra (I) frame or a predicted (P) frame; an infra frame period or I frame periodicity (which means a ratio of P frames generated for every I frame generated), etc.
  • an I frame is an encoded video frame generated using a motion compensation algorithm or process, which contains all necessary rendering information within itself for the video client to decode and display the video frame.
  • a P frame is an encoded video frame generated using a motion compensation algorithm or process, which contains only information on the differences from a previous frame.
  • the frame request processor 142 can monitor its current network connectivity and current network conditions such as available bandwidth in the network. If the network connectivity falls below a threshold or if the level of congestion in the network 130 rises about a threshold, then the frame request processor 142 can adjust the encoding parameters to decrease the size of the encoded video frame, for example, by decreasing the resolution and/or the quantization of the encoded video frame, in order to maintain a certain frame rate at the video client.
  • the frame request processor 142 can adjust the encoding parameters to increase the size of the encoded video frame, for example, by increasing the resolution and/or the quantization of the encoded video frame to maintain the frame rate at the video client.
  • the frame request processor 142 includes the newly determined encoding parameters in a frame request or retransmission request to the video client.
  • the frame request processor 142 is connected to the I/O interface 136 wherein it can measure one or more parameters that indicate signal connectivity such as received signal strength, signal-to-noise ratio, signal-to-interference ratio, and the like.
  • the frame request processor 142 also contains logic used to determine the existence and perhaps the level of network congestion or network loading within communication network 130.
  • each encoded video frame may include a time stamp provided by the video server that indicates the time that the video frame was encoded. The frame request processor 142 can then compare the time stamp to the time at which the encoded video frame was received to determine a time delay of receipt of the encoded video frame.
  • the frame request processor can determine that there is a decrease in available bandwidth within network 130 and determine new encoding parameters to decrease the size of the encoded video frame. Also, some communication protocols enable the video client to receive information regarding available network bandwidth directly from infrastructure devices within the network, and the video client can, thereby, adjust the encoding parameters responsive to receiving such data from the network. [0034] In a worst case scenario, the video client 132 never receives a transmitted encoded video frame or receives it "late.” In a simplest implementation, the frame request processor 142 sets a timer upon sending a frame request and then determines that a frame is lost if an encoded video frame is not received in response to the frame request before expiration of the timer.
  • the frame request processor 142 thereupon requests a new I frame from the video server 104.
  • Video sequence numbers also referred to herein as frame numbers
  • frame numbers can be used to facilitate this implementation so that the video client can request (within the frame request) that the next I frame have a certain sequence (frame) number and so that both the video server and video client are aware of a next encoded video frame in the sequence. Moreover, if the "lost" frame eventually arrives at the video client (as indicated by the sequence number), the video client simply discards the late frame.
  • the frame request processor 142 may send a retransmission request to the video server to retransmit the frame and may further include new encoding parameters that change (in this case decrease) the size of the next one or more encoded video frames.
  • the frame request processor 142 could issue the retransmission request without changing the encoding parameters.
  • the frame request processor 142 sends a frame request for encoded video frame (N+ 1) to the server with an indication that encoded video frame N did not arrive. Note that the default in the frame request is to indicate to the server that the last frame did arrive and was successfully decoded.
  • the video server 104 If the video server 104 is configured to send P frames, and a P frame is now due, the video server encodes the next P frame based on the last frame that it knows the client successfully received, which requires that frame storage 124 needs to store at least the last transmitted frame and the frame before that, which was confirmed as received by the video client 132. Where the video server is next due to send an I frame, then an I frame is sent.
  • the encoded video frame might be late at the video client because network bandwidth has suddenly reduced (due to, for example, wireless fading, handover to a busy cell, handover from a broadband network to a narrowband network, etc.).
  • the video client 132 upon determining that that the cause of the late or lost frame is due to bandwidth limitations, requests that encoded frame N be retransmitted with a smaller size, using a change in encoding parameters.
  • the video server 104 learns that the last transmitted encoded video frame has not arrived (for instance upon receiving a second request from the video client for frame N), it clears out its own transmit buffer in case the congestion is on its own wireless uplink and sends a Null frame to the video client 132 to indicate that the missing frame was actually attempted and to indicate its size.
  • the Null frame serves the purpose of indicating to the video client that the encoded video frame was lost and not the frame request and further indicates the size of the encoded video frame to assist the video client in determining how the respond.
  • the video client 132 can respond to this information by: sending a retransmission request without changing the encoding parameters, so the video server re-sends the encoded video frame at the same size; sending a retransmission request with adjusted encoding parameters that specifies, for instance, a lower resolution or quantization, which results in the retransmitted frame having a smaller size, thereby, making it less likely to be held up in the network; requesting a next new encoded video frame using the current encoding parameters; or requesting a next new encoded video frame using new encoding parameters.
  • the video client's decision of how to proceed is based on its own estimate of network latency, which it can derive by examining the delay between sending its last frame request and receiving the video server's Null frame. If the delay is larger than a configured threshold, the video client assumes congestion and requests a smaller frame size (lower resolution and/or quantization). If the delay is smaller than the threshold, the video client assumes that the problem is just a temporary loss of coverage and requests the same size frame with unchanged resolution and quantization.
  • the frame request processor 142 may decide to change the encoding parameters at the video server for other reasons. For example, it may be important for a user viewing video on video display 134 to see high resolution video (perhaps to see some greater detail in the video); so the frame request processor would change the encoding parameters to increase the resolution at the expense of having a decreased frame rate (i.e., rate at which successive encoded video frames are received into the receive buffer). Alternatively, the user may determine that a high frame rate is more desirable at the expense of resolution, and the frame request processor 142 accordingly adjusts the encoding parameters. The user (via a graphical user interface) may adjust the encoding parameters in response to user preference, or the frame request processor 142 may automatically adjust the encoding parameters in response to network connectivity and/or available network bandwidth.
  • CODEC 144 decodes the encoded video frame sitting in the receive buffer 152 using programmed decoding parameters that correspond to the encoding parameters programmed into CODEC 114 to generate a video frame having a format that is viewable on video display 134.
  • CODEC 144 may use codec standards such as JPEG, MPEG, H.264, etc., that may or may not use a motion compensation algorithm. Where motion compensation is used, the CODEC 144 retrieves the last Rx frame 156 from frame storage 154 to properly decode the current frame where the current frame is a P frame.
  • a flow diagram is shown illustrating a high level method 200 for asynchronous live video transmission in accordance with an embodiment of the teachings herein.
  • the steps of method 200 do not necessarily have to be performed in the order indicated, and more or fewer of the steps may be performed at any given point in time in order to implement the teachings herein.
  • the frame request processor 116 in the video server 104 receives, from the video client 132 via the I/O interface 108, a frame request for a single video frame, wherein the frame request includes a first set of encoding parameters determined by the video client.
  • the frame capturer 112 captures a video frame that has not been previously sent to the video client 132, wherein the video frame comprises a portion of live video captured by the video camera 102 and received into analog video input 106.
  • CODEC 114 encodes the video frame using the first set of encoding parameters to produce one encoded video frame that is provided to the transmit buffer 122 and transmitted via I/O interface 108 to the video client 132 such that only the one encoded video frame is in transit over the communication network in response to the frame request.
  • the video server need not transmit encoded frames at a known synchronous frame rate.
  • the rate at which frames are transmitted is asynchronous and is determined by the video client since each encoded video frame transmitted by the video server is transmitted only in response to a frame (or retransmission) request and only one encoded video frame is in transit over the communication network 130 in response to a given frame (or retransmission) request.
  • Such asynchronous video transmission enables a control loop mechanism that allows the video client to adjust the rate of video frame transmission in response to changing network conditions and network connectivity and user preference and to match the video server CODEC configurations to available network bandwidth. This enables a customer's maximum latency requirements to be met over changing network conditions.
  • FIG. 3 a flow diagram is shown illustrating a detailed method 300 for asynchronous live video transmission in accordance with an embodiment of the teachings herein.
  • the steps of method 300 do not necessarily have to be performed in the order indicated, and more or fewer of the steps may be performed at any given point in time in order to implement the teachings herein.
  • the frame request processor 116 in the video server 104 receives (302) a frame request.
  • the frame request processor 116 determines (304) that there are new encoding parameters (i.e., a second set of encoding parameters that are different from a first (current) set of encoding parameters) included in the frame request to change the size of a next encoded video frame (by, for instance, changing one or more of the frame rate at the encoder, the resolution of the encoded video frames, the quantization of the encoded video frames, etc.), the frame request processor 116 updates (306) the encoding parameters for the CODEC 114 with the new (second set of) encoding parameters so that the next one or more video frames are encoded using the new (second set of) encoding parameters. If not, the next one or more video frames are encoded using the current (first) encoding parameters.
  • new encoding parameters i.e., a second set of encoding parameters that are different from a first (current) set of encoding parameters
  • the frame request may further include a frame number for a next encoded frame.
  • the frame request processor 116 determines that the frame request is a retransmission request, the last transmitted frame 126 is retrieved (312) from frame storage 124. If the frame request contains a particular frame number, the encoded video frame having that frame number is retrieved from frame storage 124.
  • the retransmission request contained updated encoding parameters (for instance to change the size of the encoded video frame): the last transmitted frame undergoes transcoding (320) in the codec 114 using the updated encoding parameters in the retransmission request; as appropriate, a frame number and/or time stamp to indicate when the video frame was encoded may be inserted with the encoded video frame during the codec processing 320; the encoded video frame having a different frame size is provided to the transmit buffer 122 for temporary storage before retransmission (322) via the I/O interface 108 to the video client 132 in response to the retransmission request; and frame storage 124 is updated (324) with the last transmitted encoded video frame.
  • the retransmission request does not contain updated encoding parameters
  • the last transmitted frame can simply be pulled from frame storage 124 into the transmit buffer 122 for retransmission to the video client.
  • the frame request processor 116 determines that the frame request is not a request for retransmission but is a request for a new encoded video frame, the subsequent video server processing depends on whether (310) the video server implements the transmit buffer 122 as a rolling buffer.
  • the video server implements a rolling buffer, and it is determined (310) that the next encoded frame is in the rolling buffer, and the frame request contained (314) updated encoding parameters (for instance to change the size of the encoded video frame): the next encoded frame is retrieved (312) from the rolling buffer to undergo transcoding (320) in the codec 114 using the updated encoding parameters in the frame request; as appropriate, a frame number and/or time stamp to indicate when the video frame was encoded may be inserted with the encoded video frame; the encoded video frame having a different frame size from the previous encoded video frame is provided to the transmit buffer 122 for temporary storage before transmission (322) via the I/O interface 108 to the video client 132 in response to the frame request; and frame storage 124 is updated (324) with the last transmitted encoded video frame.
  • the frame capturer 112 captures (316) the next analog frame via the analog video input 106, said capturing including converting (318) the analog frame to a raw digital format.
  • the raw digital frame undergoes codec processing (320) using the encoding parameters (i.e., first encoding parameters) used to encode the last video frame or using updated (i.e., second) encoding parameters if updated encoding parameters were included in the frame request, and as appropriate, a frame number and/or time stamp to indicate when the video frame was encoded may be inserted with the encoded video frame.
  • the encoded video frame is provided to the transmit buffer 122 for temporary storage before transmission (322) via the I/O interface 108 to the video client 132 in response to the frame request; and frame storage 124 is updated (324) with the last transmitted encoded video frame.
  • Coupled as used herein is defined as connected, although not necessarily directly and not necessarily mechanically.
  • a device or structure that is “configured” in a certain way is configured in at least that way, but may also be configured in ways that are not listed.
  • sequence of steps in a flow diagram or elements in the claims, even when preceded by a letter does not imply or require that sequence.
  • processors such as microprocessors, digital signal processors, customized processors and field programmable gate arrays (FPGAs) and unique stored program instructions (including both software and firmware) that control the one or more processors to implement, in conjunction with certain non-processor circuits, some, most, or all of the functions of the method and apparatus for indicating status of channels assigned to a talkgroup described herein.
  • the non-processor circuits may include, but are not limited to, a radio receiver, a radio transmitter, signal drivers, clock circuits, power source circuits, and client input devices.
  • these functions may be interpreted as steps of a method to perform the indicating of status of channels assigned to a talkgroup described herein.
  • some or all functions could be implemented by a state machine that has no stored program instructions, or in one or more application specific integrated circuits (ASICs), in which each function or some combinations of certain of the functions are implemented as custom logic.
  • ASICs application specific integrated circuits
  • Both the state machine and ASIC are considered herein as a "processing device" for purposes of the foregoing discussion and claim language.
  • an embodiment can be implemented as a computer-readable storage element or medium having computer readable code stored thereon for programming a computer (e.g., comprising a processing device) to perform a method as described and claimed herein.
  • Examples of such computer-readable storage elements include, but are not limited to, a hard disk, a CD-ROM (Compact Digital Read Only Memory), an optical storage device, a magnetic storage device, a ROM (Read Only Memory), a PROM (Programmable Read Only Memory), an EPROM (Erasable Programmable Read Only Memory), an EEPROM (Electrically Erasable Programmable Read Only Memory) and a Flash memory.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Databases & Information Systems (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

L'invention porte sur un procédé (200) pour une transmission vidéo asynchrone en direct sur un réseau de communication, qui comprend : la réception (202) à partir d'un client vidéo, une requête de trame pour une trame vidéo unique, la requête de trame comprenant un premier ensemble de paramètres de codage déterminés par le client vidéo; la capture (204) d'une trame vidéo qui n'a pas été précédemment envoyée au client vidéo, la trame vidéo comprenant une partie de vidéo en direct capturée par une caméra vidéo; le codage (206) de la trame vidéo à l'aide du premier ensemble de paramètres de codage pour produire une trame vidéo codée; et la transmission (208) de la trame vidéo codée de telle sorte que seule la trame vidéo codée est en transit sur le réseau de communication en réponse à la requête de trame.
PCT/US2010/028564 2009-04-06 2010-03-25 Procédé et appareil pour une transmission vidéo asynchrone sur un réseau de communication WO2010117644A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP10727998A EP2417766A1 (fr) 2009-04-06 2010-03-25 Procédé et appareil pour une transmission vidéo asynchrone sur un réseau de communication

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
GB0905977.5 2009-04-06
GB0905977A GB2469281B (en) 2009-04-06 2009-04-06 Method and apparatus for asynchronous video transmission over a communication network

Publications (1)

Publication Number Publication Date
WO2010117644A1 true WO2010117644A1 (fr) 2010-10-14

Family

ID=40750211

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2010/028564 WO2010117644A1 (fr) 2009-04-06 2010-03-25 Procédé et appareil pour une transmission vidéo asynchrone sur un réseau de communication

Country Status (3)

Country Link
EP (1) EP2417766A1 (fr)
GB (1) GB2469281B (fr)
WO (1) WO2010117644A1 (fr)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10085050B2 (en) 2014-08-15 2018-09-25 Xiaomi Inc. Method and apparatus for adjusting video quality based on network environment
CN104202614B (zh) * 2014-08-15 2016-03-09 小米科技有限责任公司 一种基于网络环境调整视频画质的方法及装置
CN114827662B (zh) * 2022-03-18 2024-06-25 百果园技术(新加坡)有限公司 视频分辨率自适应调节方法、装置、设备和存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0798929A2 (fr) * 1996-03-28 1997-10-01 Nec Corporation Appareil pour le décodage d'images codées en MPEG
WO2002021844A1 (fr) * 2000-09-06 2002-03-14 M-Wave Limited Remise adaptative de videos
US20030074674A1 (en) * 2001-10-17 2003-04-17 Magliaro Maximilian Matthew Method and system for dynamically adjusting video bit rates
US20040071106A1 (en) * 2000-03-06 2004-04-15 Koichi Ito Data transmission system and communication devices

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2001080558A2 (fr) * 2000-04-14 2001-10-25 Solidstreaming, Inc. Systeme et procede destine au multimedia en continu
NO315887B1 (no) * 2001-01-04 2003-11-03 Fast Search & Transfer As Fremgangsmater ved overforing og soking av videoinformasjon

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0798929A2 (fr) * 1996-03-28 1997-10-01 Nec Corporation Appareil pour le décodage d'images codées en MPEG
US20040071106A1 (en) * 2000-03-06 2004-04-15 Koichi Ito Data transmission system and communication devices
WO2002021844A1 (fr) * 2000-09-06 2002-03-14 M-Wave Limited Remise adaptative de videos
US20030074674A1 (en) * 2001-10-17 2003-04-17 Magliaro Maximilian Matthew Method and system for dynamically adjusting video bit rates

Also Published As

Publication number Publication date
GB2469281B (en) 2011-08-10
GB0905977D0 (en) 2009-05-20
GB2469281A (en) 2010-10-13
EP2417766A1 (fr) 2012-02-15

Similar Documents

Publication Publication Date Title
US9716737B2 (en) Video streaming in a wireless communication system
US8711929B2 (en) Network-based dynamic encoding
JP3931595B2 (ja) データ修正装置及びデータ修正方法
US20150373075A1 (en) Multiple network transport sessions to provide context adaptive video streaming
US11197051B2 (en) Systems and methods for achieving optimal network bitrate
US9585062B2 (en) System and method for implementation of dynamic encoding rates for mobile devices
US9014277B2 (en) Adaptation of encoding and transmission parameters in pictures that follow scene changes
EP3016395B1 (fr) Dispositif de codage vidéo et procédé de codage vidéo
KR20090124993A (ko) 송신 장치, 송신 방법 및 수신 장치
US20120239999A1 (en) Video transmitting apparatus, video receiving apparatus, and video transmission system
AU2021200428B2 (en) System and method for automatic encoder adjustment based on transport data
CN110012363B (zh) 一种基于sip协议的视频聊天系统
US20120236927A1 (en) Transmission apparatus, transmission method, and recording medium
US10085029B2 (en) Switching display devices in video telephony
US9313508B1 (en) Feeding intra-coded video frame after port reconfiguration in video telephony
EP2417766A1 (fr) Procédé et appareil pour une transmission vidéo asynchrone sur un réseau de communication
US9118803B2 (en) Video conferencing system
US8290063B2 (en) Moving image data conversion method, device, and program
WO2021140768A1 (fr) Dispositif de transmission et procédé de transmission
CN104702970A (zh) 一种同步视频数据的方法、设备及系统
WO2020214048A1 (fr) Procédé d'adaptation de transmission d'image vidéo
KR20190067386A (ko) 배터리 소모 절감 방법 및 시스템

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 10727998

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 2010727998

Country of ref document: EP