Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
  • H04L65/60—Network streaming of media packets
  • H04L65/70—Media network packetisation
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
  • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
  • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
  • H04N21/23406—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving management of server-side video buffer
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
  • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
  • H04N21/24—Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
  • H04N21/2401—Monitoring of the client buffer
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
  • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
  • H04N21/24—Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
  • H04N21/2402—Monitoring of the downstream path of the transmission network, e.g. bandwidth available
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
  • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
  • H04N21/242—Synchronization processes, e.g. processing of PCR [Program Clock References]
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
  • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
  • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
  • H04N21/44004—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving video buffer management, e.g. video decoder buffer or video display buffer
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
  • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
  • H04N21/238—Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams
  • H04N21/2389—Multiplex stream processing, e.g. multiplex stream encrypting

Definitions

• Wired and wireless communications systems are widely deployed to communicate various types of content such as voice, video, packet data, messaging, broadcast, and so on.
• Wired communication systems include packet-based communication systems (e.g., Ethernet, and the like) and non-packet based communication systems.
• Wireless communication systems include wireless local area network (WLAN), and cellular multiple-access systems. Generally, these wireless communication systems are capable of supporting communication with multiple users by sharing the available system resources (e.g., time, frequency, and power).
• Wireless communication systems may use radio technologies for multiple-access including code- division multiple access (CDMA), time-division multiple access (TDMA), frequency- division multiple access (FDMA), and orthogonal frequency-division multiple access (OFDMA).
• CDMA code- division multiple access
• TDMA time-division multiple access
• FDMA frequency- division multiple access
• OFDMA orthogonal frequency-division multiple access
• a source device e.g., smartphone, tablet, and the like
• a sink device e.g., TV, etc.
• the source device may transmit the media stream over a wireless link operating according to one of the 802.1 1 family of standards ("Wi-Fi").
• Wi-Fi is often jittery and error-prone
• some amount of buffering is provided at the sink device to smooth the jitter and packet latency caused by errors in the channel (e.g., retransmissions of data, etc.) to maintain good quality of video rendered at the sink device. It may also be desirable to reduce latency between capture or display of the video at the source device and display of the video at the sink device.
• the described features generally relate to one or more improved systems, methods, and/or apparatuses for dynamic control of latency or jitter buffer size at the sink device for audio and/or video streaming over an error-prone channel.
• the sink buffer size may be dynamically controlled by the source device based on a type of application for a media stream being transmitted from the source device to the sink device for presentation. For example, the techniques may select a buffer size that is smaller for gaming applications, larger for interactive media applications (e.g., interactive computing, presentations, bi- directional communication, etc.), and even larger for non-interactive media types (e.g., streaming video, static images, etc.).
• the techniques adjust the time delta between a shared clock reference and time reference values of a transport stream that are used by the sink device to determine decoding or presenting of media frames of the transport stream relative to a shared clock reference.
• the source device encapsulates content using an MPEG2 Transport Stream (MPEG-TS) for transmission over a medium.
• MPEG-TS MPEG2 Transport Stream
• the source device may adjust the time delta between the program clock reference (PCR) and presentation time stamp (PTS) and/or decode time stamp (DTS) values in the MPEG-TS to control the amount of latency or jitter buffering at the sink device before decoding and rendering of the content stream.
• the techniques may account for transceiver latency at the source device due to scanning or multiple concurrent connections.
• application-based sink buffer size control is implemented via an internal application programming interface (API) of the source device.
• the API may determine a use case or application type by program calls of the application or by monitoring a task manager and may determine an amount of buffering based on the application type.
• the API may determine concurrency and/or scanning operation of the source device based on parameters associated with communication drivers and may increase the amount of buffering based on the scanning or concurrency operation.
• the API may notify the encoder and transport stream multiplexor of the buffer time delta and the encoder and multiplexor may associate DTS and/or PTS values with audio and/or video frames of the encoded content in the transport stream according to the buffer time delta and PCR.
• a user override may be provided to allow explicit configuration of the buffer time delta.
• Some embodiments are directed to a method performed by a source device, including determining an application type of a media stream for transmission to a sink device, determining, based at least in part on the application type, a buffer size used by the sink device for buffering of a transport stream used for encapsulation of the media stream, encoding frames of the media stream with time reference values relative to a shared clock reference based on the determined buffer size, and encapsulating the encoded frames in the transport stream.
• Determining the application type may be based on an application associated with the media stream.
• the application type of the media stream may be a gaming application type, an interactive computing application type, or a media viewing application type.
• the method may include receiving, by an application programming interface of the source device, a call from an application running on the source device for establishing a streaming display connection with the sink device.
• the application type of the media stream may be determined based on the received call.
• the method includes determining an off-channel concurrency latency based on a concurrent connection configuration of the source device. Determining the buffer size may be further based on the off-channel concurrency latency. In some embodiments, the method includes determining a scanning latency is based on a channel scanning configuration of the source device. Determining the buffer size may be further based on the scanning latency. [0010] In some embodiments, the method includes transmitting the transport stream to the sink device over a wireless local area network connection. The method may include displaying the media stream at the source device concurrently with encoding of the media stream for transmission to the sink device. The method may include transmitting time values of the shared clock reference at periodic intervals in the transport stream.
• the transport stream is a Moving Picture Experts Group (MPEG) transport stream (MPEG-TS).
• MPEG-TS Moving Picture Experts Group
• the time reference values may include a presentation time stamp (PTS) or a decode time stamp (DTS), or both PTS and DTS values.
• the shared clock reference may be a program clock reference (PCR) synchronized at the sink device.
• Some embodiments are directed to an apparatus for dynamic control of sink device buffering by a source device, including means for determining an application type of a media stream for transmission to a sink device, means for determining, based at least in part on the application type, a buffer size used by the sink device for buffering of a transport stream used for encapsulation of the media stream, means for encoding frames of the media stream with time reference values relative to a shared clock reference based on the determined buffer size, and means for encapsulating the encoded frames in the transport stream.
• the means for determining the application type may determine the application type based on an application associated with the media stream.
• the application type of the media stream may be, for example, a gaming application type, an interactive computing application type, or a media viewing application type.
• the apparatus may include means for receiving, by an application programming interface of the source device, a call from an application running on the source device for establishing a streaming display connection with the sink device.
• the means for determining the application type of the media stream may determine the application type based on the received call.
• the apparatus for dynamic control of sink device buffering includes means for determining an off-channel concurrency latency based on a concurrent connection configuration of the source device.
• the means for determining the buffer size may further determine the buffer size based on the off-channel concurrency latency.
• the apparatus includes means for determining a scanning latency based on a channel scanning configuration of the source device.
• the means for determining the buffer size may further determine the buffer size based on the determined scanning latency.
• the apparatus for dynamic control of sink device buffering includes means for transmitting the transport stream to the sink device over a wireless local area network connection.
• the apparatus may include means for displaying the media stream at the source device concurrently with encoding of the media stream for transmission to the sink device.
• the apparatus may include means for transmitting time values of the shared clock reference at periodic intervals in the transport stream.
• the transport stream is a Moving Picture Experts Group (MPEG) transport stream (MPEG-TS).
• MPEG-TS Moving Picture Experts Group
• the time reference values may be a presentation time stamp (PTS) or a decode time stamp (DTS), or both PTS and DTS values.
• the shared clock reference may be a program clock reference (PCR) synchronized at the sink device.
• Some embodiments are directed to a device for dynamic control of sink device buffering by a source device, including a processor, and a memory in electronic
• the memory may include instructions being executable by the processor to determine an application type of a media stream for transmission to a sink device, determine, based at least in part on the application type, a buffer size used by the sink device for buffering of a transport stream used for encapsulation of the media stream, encode frames of the media stream with time reference values relative to a shared clock reference based on the determined buffer size, and encapsulate the encoded frames in the transport stream.
• the memory may include instructions being executable by the processor to determine the application type based on an application associated with the media stream.
• the application type of the media stream may be, for example, a gaming application type, an interactive computing application type, or a media viewing application type.
• the memory may include instructions being executable by the processor to receive, by an application programming interface of the source device, a call from an application running on the source device for establishing a streaming display connection with the sink device and determine the application type of the media stream based on the received call.
• the memory includes instructions being executable by the processor to determine an off-channel concurrency latency based on a concurrent connection configuration of the source device, and determine the buffer size further based on the determined off-channel concurrency latency. In some embodiments, the memory includes instructions being executable by the processor to determine a scanning latency based on a channel scanning configuration of the source device, and determine the buffer size further based on the determined scanning latency.
• the memory includes instructions being executable by the processor to transmit the transport stream to the sink device over a wireless local area network connection.
• the memory may include instructions being executable by the processor to display the media stream at the source device concurrently with encoding of the media stream for transmission to the sink device.
• the memory may include instructions being executable by the processor to transmit time values of the shared clock reference at periodic intervals in the transport stream.
• the transport stream is a Moving Picture Experts Group (MPEG) transport stream (MPEG-TS).
• MPEG-TS Moving Picture Experts Group
• the time reference values may include a presentation time stamp (PTS) or a decode time stamp (DTS), or both PTS and DTS values.
• the shared clock may be a program clock reference (PCR) synchronized at the sink device.
• Some embodiments are directed to a computer program product for dynamic control of sink device buffering by a source device.
• the computer program product includes a computer-readable medium, including code for determining an application type of a media stream for transmission to a sink device, determining, based at least in part on the application type, a buffer size used by the sink device for buffering of a transport stream used for encapsulation of the media stream, encoding frames of the media stream with time reference values relative to a shared clock reference based on the determined buffer size, and encapsulating the encoded frames in the transport stream.
• the computer-readable medium may include code for determining the application type based on an application associated with the media stream.
• the application type of the media stream may be, for example, a gaming application type, an interactive computing application type, or a media viewing application type.
• the computer-readable medium may include code for receiving, by an application programming interface of the source device, a call from an application running on the source device for establishing a streaming display connection with the sink device and determining the application type of the media stream based on the received call.
• the computer-readable medium includes code for
• the computer-readable medium includes code for determining a scanning latency based on a channel scanning configuration of the source device, and determining the buffer size further based on the scanning latency.
• the computer-readable medium includes code for
• the computer-readable medium may include code for displaying the media stream at the source device concurrently with encoding of the media stream for transmission to the sink device.
• the computer-readable medium may include code for transmitting time values of the shared clock reference at periodic intervals in the transport stream.
• the transport stream is a Moving Picture Experts Group (MPEG) transport stream (MPEG-TS).
• MPEG-TS Moving Picture Experts Group
• the time reference values may include a presentation time stamp (PTS) or a decode time stamp (DTS), or both PTS and DTS values.
• the shared clock reference may be a program clock reference (PCR) synchronized at the sink device.
• FIG. 1 shows a diagram of a system for display of content from one device onto the display of another device by video and/or audio content streaming;
• FIG. 2 shows an example system for display of content from a source device to a sink device using application-based control of sink buffering using frame reference timing in accordance with various embodiments
• FIG. 3 shows a diagram illustrating an example transmission of a media stream in the architecture of FIG. 2 in accordance with various embodiments
• FIG. 4 shows a timing diagram that illustrates latency based on concurrency and scanning performed by source device in accordance with various embodiments
• FIG. 5 shows a method for application-based control of sink buffer size using frame reference timing in accordance with various embodiments
• FIG. 6 shows a system architecture for application-based control of sink buffer size using frame reference timing in accordance with various embodiments
• FIG. 7 shows a block diagram illustrating a device for source device control of sink buffer size in accordance with various embodiments
• FIG. 8 shows a block diagram illustrating a device for source device control of sink buffer size in accordance with various embodiments
• FIG. 9 shows a block diagram of a source device configured for dynamic control of sink buffer size in accordance with various embodiments.
• FIG. 10 shows a flow chart illustrating an embodiment of a method for dynamic control of sink buffer size in accordance with various embodiments.
• FIG. 11 shows a flow chart illustrating an embodiment of a method 1100 for dynamic control of sink buffer size in accordance with various embodiments.
• Described embodiments are directed to systems and methods for dynamic control of latency or jitter buffer size at the sink device for audio and/or video streaming over an error- prone channel.
• the sink buffer size may be dynamically controlled by the source device based on a type of application for a media stream being transmitted from the source device to the sink device for presentation. For example, the techniques may select a buffer size that is smaller for gaming applications, larger for interactive media applications (e.g., interactive computing, presentations, bi-directional communication, etc.), and even larger for non- interactive media types (e.g., streaming video, static images, etc.).
• the techniques adjust the time delta between a shared clock reference and time reference values of a transport stream that are used by the sink device to determine decoding or presenting of media frames of the transport stream relative to a shared clock reference.
• the source device encapsulates content using an MPEG-TS for transmission over a medium.
• the source device may adjust the time delta between the PCR and PTS/DTS values in the MPEG-TS to control the amount of latency or jitter buffering at the sink device before decoding and rendering of the content stream.
• the techniques may account for transceiver latency at the source device due to scanning or multiple concurrent connections. While application-based sink buffer size control is described with reference to use of a transport stream over Wi-Fi, these techniques may be applied to encoded content transmitted in a transport stream over any wired or wireless transmission medium for which latency buffering may be applied.
• application-based sink buffer size control is implemented via an internal API of the source device.
• the API may determine a use case or application type by program calls of the application or by monitoring a task manager and may determine an amount of buffering based on the application type.
• the API may determine concurrency and/or scanning operation of the source device based on parameters associated with communication drivers and may increase the amount of buffering based on the scanning or concurrency operation.
• the API may notify the encoder and transport stream multiplexor of the buffer time delta and the encoder and multiplexor may associate DTS and/or PTS values with audio and/or video frames of the encoded content in the transport stream according to the buffer time delta and PCR.
• a user override may be provided to allow explicit configuration of the buffer time delta.
• a system 100 includes a source device 1 15 and a sink device 135 and may include one or more access points 105.
• Examples of the source device 1 15 may include, but are not limited to, smartphones, cell phones, wireless headphones, wearable computing devices, tablets, personal digital assistants (PDAs), laptops, or any other device capable of communicating with a sink device via a connection (e.g., wired, cellular wireless, Wi-Fi, etc).
• Examples of the sink devices 135 may include, but are not limited to, in-vehicle infotainment devices, TVs, computers, laptops, projectors, cameras, smartphones, wearable computing devices, or any other device capable of communicating with a source device 1 15 and displaying content received from the source device 1 15.
• the sink device 135 may be a combination of devices.
• the sink device 135 may include a display device and a separate device for receiving, buffering, and decoding content for display on the display device.
• Source device 1 15 may be connected to sink device 135 via link 125.
• Link 125 is illustrated in FIG. 1 as a wireless link but may be a wired or wireless link, in embodiments.
• Some wired or wireless links use networking protocols that may have non-deterministic packet transfer timing.
• some communication links utilize protocols in which a device verifies the absence of other traffic before transmitting on a shared transmission medium, such as an electrical bus, or a band of the electromagnetic spectrum.
• Arbitration for use of the medium by several devices can cause variation in transmission time from packet to packet.
• interference may cause packet loss and re-try, which may result in packet latency or packets being received out of order. While the following techniques are described using the wireless networking architecture illustrated in FIG. 1 , the described techniques are applicable to any suitable wired or wireless communication technology.
• source device 1 15 is connected to sink device 135 via a Wi-Fi Display connection.
• Wi-Fi Display which may be known as Miracast, allows a portable device or computer to transmit video and audio to a compatible display wirelessly. It enables delivery of compressed standard or high-definition video over a wireless link 125.
• Wireless link 125 may be a direct wireless link (e.g., peer-to-peer link 125-a), or an indirect wireless link (e.g., indirect link 125-b). Examples of direct wireless links include Wi-Fi Direct connections and connections established by using a Wi-Fi Tunneled Direct Link Setup (TDLS) link.
• TDLS Wi-Fi Tunneled Direct Link Setup
• WLAN radio and baseband protocol including physical and MAC layers from IEEE 802.1 1 , and its various versions including, but not limited to, 802-1 lb, 802.1 lg, 802.1 la, 802.1 In, 802.1 lac, 802.1 lad, 802.1 1 ah, etc.
• Miracast allows users to echo the display from one device onto the display of another device by video and/or audio content streaming.
• the link 125 between the source device 1 15 and sink device 135 may be bi-directional.
• the connection between the source device 1 15 and a sink device 135 may also allow users to launch applications stored on the source device 1 15 via the sink device 135.
• the sink device 135 may include various input controls (e.g., mouse, keyboard, knobs, keys, user interface buttons). These controls may be used at the sink device 135 to initialize and interact with applications stored on the source device 1 15.
• Miracast may use a transport stream such as an MPEG2 Transport Stream (MPEG- TS).
• the content may be encoded according to a media encoding format (e.g., h.264, MPEG- 4, etc.) and may be multiplexed into the transport stream with other information (e.g., error correction, stream synchronization, etc.) for transmission to the sink device 135.
• the source device 1 15 may maintain a shared clock reference and periodically transmit the reference clock time in the transport stream.
• the sink device 135 may synchronize a local shared clock reference to the clock reference of the source device 1 15 using the periodically transmitted reference clock time values.
• the source device 1 15 may encode frames of the transport stream with reference values used by the sink device 135 to re-order the frames for decoding and to synchronize output of the media stream relative to the shared reference clock.
• some communication links may have non-deterministic transmission latency or jitter between transmitted packets.
• latency in shared- medium communication systems may be caused by collisions, retries, packet-loss, and/or arbitration between users for the medium.
• the transport stream may be buffered at the sink device 135 to reduce rendering issues such as frame skipping or stutter due to packet latency and jitter over the link 125.
• the source device 1 15 may scan other channels for other connections or support multiple concurrent connections over multiple channels.
• the source device 1 15 may have a Wi-Fi Direct connection for transmitting the content stream to the sink device 135 and may support other concurrent connections to other devices, Wi-Fi access points 105, or cellular access points.
• the source device 1 15 may spend a certain amount of time scanning or operating on channels associated with the other connections.
• the buffer size at the sink device 135 should be large enough to account for time periods where the source device 1 15 is operating on other channels.
• the transport stream contains a clock reference known as the program clock reference (PCR).
• PCR program clock reference
• the PCR is periodically transmitted (e.g., once per 100 ms, etc.) and the sink device synchronizes a local PCR timing clock to the transmitted PCR values.
• the encoded video and audio frames in the transport stream are associated with a decode time known as the decode time stamp (DTS) and a presentation time known as the presentation time stamp (PTS).
• DTS decode time stamp
• PTS presentation time stamp
• the sink device decodes and presents audio and video elements based on the DTS and PTS associated with the audio and video frames relative to the PCR.
• the sink device 135 may display decoded audio and/or video frames when PTS values associated with the frames correspond to the PCR.
• DTS values may be used at the sink device 135 to determine when the frames are decoded.
• the system 100 including the source device 1 15 may be configured to dynamically control latency or jitter buffer size at the sink device 135 for audio and/or video streaming over an error-prone channel.
• the sink buffer size may be dynamically controlled by the source device 1 15 based on a type of application for a media stream being transmitted from the source device 1 15 to the sink device 135 for presentation.
• the techniques may select a buffer size that is smaller for gaming applications, larger for interactive media applications (e.g., interactive computing, presentations, bi-directional communication, etc.), and even larger for non-interactive media types (e.g., streaming video, static images, etc.).
• the techniques adjust the time delta between a shared clock reference and time reference values of a transport stream that are used by the sink device to determine decoding or presenting of media frames of the transport stream relative to a shared clock reference.
• the source device encapsulates content using an MPEG-TS for transmission over a medium.
• the source device may adjust the time delta between the PCR and PTS/DTS values in the MPEG-TS to control the amount of latency or jitter buffering at the sink device before decoding and rendering of the content stream.
• the techniques may account for transceiver latency at the source device due to scanning or multiple concurrent connections. While application-based sink buffer size control is described with reference to use of a transport stream over Wi-Fi, these techniques may be applied to encoded content transmitted in a transport stream over any wired or wireless transmission medium for which latency buffering may be applied.
• application-based sink buffer size control is implemented via an internal API of the source device.
• the API may determine a use case or application type by program calls of the application or by monitoring a task manager and may determine an amount of buffering based on the application type.
• the API may determine concurrency and/or scanning operation of the source device based on parameters associated with communication drivers and may increase the amount of buffering based on the scanning or concurrency operation.
• the API may notify the encoder and transport stream multiplexor of the buffer time delta and the encoder and multiplexor may associate DTS and/or PTS values with audio and/or video frames of the encoded content in the transport stream according to the buffer time delta and PCR.
• a user override may be provided to allow explicit configuration of the buffer time delta.
• FIG. 2 illustrates an example system 200 for display of content from a source device 1 15-a to a sink device 135-a using application-based control of sink buffer size using encapsulated frame reference timing in accordance with various embodiments.
• FIG. 2 may illustrate streaming of content from a source device 1 15-a to a sink device 135-a over a Wi-Fi connection 125-c using a transport stream such as an MPEG2 Transport Stream (MPEG-TS).
• the Wi-Fi connection 125-c may be direct (e.g., Wi-Fi Direct, etc.) or indirect (e.g., via a Wi-Fi access point 105 or other device, etc.).
• the content may be encoded according to a media encoding format (e.g., h.264, MPEG-4, etc.) and may be multiplexed into a transport stream (e.g., MPEG-TS, etc.) for transmission.
• the source device 1 15-a may include a system clock reference 205 (e.g., PCR), one or more applications 215, sink buffer size module 210, audio/video encoder 225, transport stream multiplexor 235, transceiver 240, output device 280 (e.g., display device, speaker, etc.), and antenna(s) 245.
• a system clock reference 205 e.g., PCR
• applications 215 e.g., sink buffer size module 210
• audio/video encoder 225 e.g., transport stream multiplexor 235
• transceiver 240 e.g., display device, speaker, etc.
• the sink device 135-a may include antenna(s) 295, transceiver 290, transport stream de-multiplexor 255, clock reference 260, buffer 265, audio/video decoder 270, and output device 285 (e.g., display device, speaker, etc.).
• source device 1 15-a may encode a content stream 220 from an application 215 according to an encoding format and may encapsulate the encoded video into transport stream 230.
• Clock reference values from the clock reference 205 may be transmitted periodically in transport stream 230.
• the sink device 135-a may synchronize the local clock reference 260 to the clock reference 205 of the source device 1 15-a.
• the source device 1 15-a may also present the content from application 215 on output device 280 (e.g., display device, speaker, etc.).
• FIG. 3 is a diagram 300 illustrating an example transmission of a media stream in the architecture of system 200 in accordance with various embodiments.
• application 215 may generate media stream 220 which may be, for example, a stream of video frames 305.
• the media stream may have a frame rate of 1/tp (e.g., 30 Hz, 60 Hz, etc.).
• Media stream 220 may be input into encoder 225, which may generate an encoded media stream 320.
• Encoder 225 may use an encoding format that may compress media stream 220 and may use different types of encoded frames which may have different levels of compression.
• diagram 300 illustrates encoded media stream 320 having Intra (I) frames which contain information for a full image of a video stream, Predicted (P) frames which are predicted from past I or P frames, and Bi-directional Predicted (B) frames which use past and future I and P frames for motion compensation and offer the greatest
• the encoder 225 may re-order the encoded frames for transmission based on the order in which the frames are used for decoding. For example, timing diagram 300 illustrates that I and/or P frames may be re-ordered before B frames in the encoded media stream 320.
• the encoded media stream 320 may be encapsulated into a transport stream format (e.g., MPEG-TS, etc.) by transport stream multiplexor 235.
• the transport stream 230 is then sent to transceiver 240 for transmission to the sink device 135-a via transmission link 125-c.
• the transport stream 230 may include time reference values associated with the encapsulated frames for decoding and display of the frames based on a shared clock reference (e.g. , PCR 205, etc.).
• the transport stream 230 may also include periodic transmission of reference clock values for synchronization of the local shared clock reference at the sink device 135-a.
• the MPEG-TS transport stream 230 may include PTS and/or DTS values 330 associated with each of the frames in the transport stream.
• the PTS and/or DTS values 330 may be encoded in packets of the transport stream 230.
• the transport stream 230 may include PCR values 310-a and 310-b, which may be used by the sink device 135-a to synchronize a local PCR 260 to the PCR 205 at the source device 1 15-a.
• the received transport stream 250 at the sink device 135-a may have variable latency and jitter in packet transmission over the transmission link 125-c.
• diagram 300 shows variable latency in receipt of packets in the received transport stream 250.
• the received transport stream 250 may be buffered and decoded.
• the sink device 135-a may buffer packets of the received transport stream 250 prior to decoding. Additionally or alternatively, it may decode the received packets and buffer the decoded media stream 275 prior to output of the decoded media stream 275 on the output device 285. Frames of the decoded media stream 275 in the sink device 135-a may be displayed on the display 285 according to their associated PTS values.
• Source device 1 15-a may determine a sink buffer size t B u F 340 based on the use case of the media stream 220 (e.g., via sink buffer size module 210). The use case of the media stream may be determined based on the type of application 215. For example, source device 1 15-a may select a higher sink buffer size t B u F 340 for more buffering at the sink device 135- a for non-interactive multimedia display and other latency tolerant applications and media streams. Source device 1 15-a may select a lower sink buffer size t B u F 340 for less buffering at the sink device 135-a for applications and media streams such as interactive computer or gaming that are less latency tolerant.
• source device 1 15-a may select a sink buffer size t B u F 340 of 200-500 milliseconds (ms) for multimedia viewing (e.g., video and/or audio playback or streaming, etc.), 80-200 ms for interactive computing (e.g., power-point, etc.), and 40-80 ms for gaming. These are merely illustrative examples and source device 1 15-a may have more or fewer categories for selecting sink buffer size t B up 340 and may select other values for various categories. [0062] Source device 1 15-a may also select sink buffer size t B u F 340 using other factors including the selected transmission medium (e.g., Wi-Fi vs.
• the selected transmission medium e.g., Wi-Fi vs.
• the capacity of the communication link e.g., in bps, etc.
• the data rate of the media stream e.g., in bps, etc.
• the encoding format e.g., measured, etc.
• Source device 1 15-a may determine decode time reference values (e.g., DTS, etc.) and presentation time reference values (e.g., PTS, etc.) for frames of the transport stream 230 based on the determined sink buffer size t B u F 340 and a decode order of the frames.
• the decode time reference values and presentation time reference values may be multiplexed with the frames of the transport stream as illustrated by time reference values 330 for each transport stream frame 325.
• the source device 1 15-a does not have to communicate the sink buffer size tjjuF 340 to the sink device 135-a prior to transmitting the transport stream.
• sink devices do not need to support additional features for source device control of sink buffer size.
• the sink buffer size is controlled through the presentation time reference values and/or decode time reference values already used by the transport stream protocol for synchronization of output of media content.
• the presentation time reference values for transmission in the transport stream may be determined by the value of the reference clock (e.g., PCR, etc.) when the frame is transmitted and the sink buffer size tjjuF 340.
• the decode time reference values may be determined by the value of the reference clock (e.g., PCR, etc.), the sink buffer size t B u F 340, and may indicate the order for decoding.
• the decode time reference values may account for the decode time (to E c), which may vary based on the type of encoding (e.g., number of B frames, etc.).
• Table 1 shows calculated values for presentation time reference values (e.g., PTS, etc.) and decode time reference values (e.g., DTS, etc.) of transport stream frames 325, according to one example.
• presentation time reference values e.g., PTS, etc.
• decode time reference values e.g., DTS, etc.
• Source device 1 15-a may include user settings that allow the user to select preferences related to the amount of buffering at the sink device 135-a. For example, the user may be able to choose a preference between lower latency or higher reliability display of the media stream and the source device 1 15-a may adjust the sink buffer size t B uF 340
• the user may be able to set sink buffer size preferences based on the use categories, for individual applications, or for individual media streams. Additionally or Alternatively, the user may be able to associate particular applications with application types. In some examples, the user may be able to set the sink buffer size t B uF 340 directly (e.g., in ms, etc.).
• Source device 1 15-a may also vary sink buffer size t B uF 340 based on concurrency and scanning configuration.
• FIG. 4 shows a timing diagram 400 that illustrates latency based on concurrency and scanning performed by a source device 1 15 (e.g., source device 1 15-a illustrated in FIG. 2, etc.).
• source device 1 15 may be transmitting a transport stream on channel 410-a to a sink device 135 while also maintaining a concurrent connection with another device or access point over channel 410-b. After transmission of packets of the transport stream in data transmission 415-a over channel 410-a, the source device 1 15 may switch to transmit or receive data 420 on channel 410-b.
• Source device 1 15 may determine an off-channel concurrency time tcoN 430 for which the source device 1 15 will switch from the connection link for transmitting the transport stream over channel 410-a to transmit or receive data in service of the concurrent connection link.
• the off-channel concurrency time tco N 430 may represent, for example, a time period for switching channels, handshaking with the other devices or access points, and transfer of one or more data packets.
• source device 115 may increase the offset 340 by approximately 60 ms for supporting off-channel concurrency.
• the off-channel concurrency time tco N 430 may be added to the sink buffer size t B u F 340 for each concurrent connection supported.
• the total off-channel concurrency time may be given by:
• one off-channel concurrency time tco N 430 may be added to the sink buffer size tjjuF 340 regardless of the number of connections, or the off-channel concurrency time tco N 430 added for each connection may be reduced by a scaling factor F.
• the total off-channel concurrency time may be given by:
• Source device 115 may also be associated with one or more access points 105 and may perform active or passive scanning 425 of channels of the access points 105.
• timing diagram 400 shows that, after performing data transfer 415-b, source device 115 performs scanning 425-a and 425-b on channels 410-c and 410-d, respectively.
• Source device 115 may perform data transfer 415-c over channel 410-a.
• Source device 115 may determine a scanning time tscAN 440 for which the source device 115 will switch from the connection link for transmitting the transport stream over channel 410-a to scanning one or more channels associated with the access points 105.
• the scanning time tscAN 440 may increase channel latency by, for example, approximately 110 ms for passive scanning and approximately 50 ms for active scanning.
• the source device 115 may determine the additional latency to be added to sink buffer size t B u F 340 in a variety of ways.
• additional latency for sink buffer size t B u F 340 may be determined according to the sum of off-channel concurrency time tco N 430 and scanning time tscAN 440.
• the additional latency may be determined according to the higher of the off-channel concurrency time tco N 430 and scanning time tscAN 440. This technique may be selected, for example where the capacity of the link is high relative to the data rate of the transport stream.
• FIG. 5 illustrates a method 500 for application-based control of sink buffer size using frame reference timing in accordance with various embodiments.
• Method 500 may be employed, for example, by the source devices 115 of FIG. 1 or FIG. 2.
• source devices 115 of FIG. 1 or FIG. 2 may be employed, for example, by the source devices 115 of FIG. 1 or FIG. 2.
• the sink buffer size module 210 or the devices 700 or 800 described with reference to FIG. 2, FIG. 7 or FIG. 8 may execute one or more sets of codes to control the functional elements of a source device 115 to perform the functions described below.
• the source device 115 determines a use case for a media stream.
• the source device 115 may determine the use case based on a type of application associated with the media stream. For example, the source device 115 may determine whether the associated application is used for multimedia playback, interactive computing, gaming, or other use case.
• the source device 115 determines the sink buffer size based on the use case of the media stream. For example, the source device 115 may select a higher sink buffer size for non-interactive multimedia display and other latency tolerant applications and media streams and lower sink buffer size for applications and media streams such as interactive computer or gaming that are less latency tolerant. The source device may receive user input associated with the user's preferences of lower latency or higher reliability media transfer and may adjust the sink buffer size accordingly.
• the source device 115 may determine if other connections are concurrently supported that will result in latency to the media stream. For example, where the source device 115 is using a Wi-Fi Display connection to transmit the media stream to the sink device 135, the source device 115 may determine if other Wi-Fi connections are concurrently supported and the effect of the concurrently supported connections on latency of the Wi-Fi Display connection. If other connections are concurrently supported, the source device 115 adds a time period tcoN to the sink buffer size at block 520. The time period tcoN may be added for each additionally supported connection, or may be scaled for multiple additional concurrent connections.
• the source device 1 15 may determine if scanning is to be performed based on associations with wireless access points. For example, the source device 1 15 may be currently connected to an access point 105 and may determine that active or passive scanning is to be performed for channels associated with the access point 105. If scanning is to be performed, the source device 1 15 may add the scanning time t S cAN to the sink buffer size at block 530.
• the source device 1 15 transmits the media stream in a transport stream format with frames of the transport stream associated with presentation time reference values calculated based on the determined sink buffer size from blocks 510, 515, 520, 525, and/or 530.
• the source device 1 15 may associate the frames of the transport stream with decode time reference values calculated based on the determined sink buffer size, frame decode order, and decode time.
• FIG. 6 illustrates a system architecture 600 for application-based control of sink buffer size using frame reference timing in accordance with various embodiments.
• System architecture 600 may include application layer 610, latency buffer API 620, and operating system (OS) subsystems and hardware layer 630.
• System architecture 600 may illustrate, for example, a layer stack for implementation of application-based control of sink buffer size using frame reference timing in the source devices 1 15 of FIG. 1 or FIG. 2.
• latency buffer API 620 may determine the use case of the media stream (e.g., by application type, etc.). For example, the application 615 may call the latency buffer API 620 as part of setting up the transport stream for transmission of the media stream, or the latency buffer API 620 may query the task manager to determine when application 615 is establishing a transport stream for transmitting a media stream to a sink device 135 for output. Delay manager 625 may determine a sink buffer size t B u F for the transport stream based on the use case of the media stream.
• Delay manager 625 may determine a presentation offset to apply to packets of a transport stream used to transmit the media stream to the sink device 135 based on the sink buffer size tjjuF- Latency buffer API 620 may be, for example, a component of a mobile device at the mobile OS service layer. [0078] Latency buffer API 620 may query the wireless driver manager 635 to determine the concurrency and scanning configuration of the device. Delay manager 625 may determine an off-channel concurrency time tco N and/or scanning time tscAN based on the concurrency and scanning configuration. Delay manager 625 may increase the presentation offset to account for the off-channel concurrency time tco N and/or scanning time tscAN-
• Delay manager 625 may notify the encoder 225 and transport stream multiplexor 235 of the presentation offset for the frames of the media stream.
• the encoder 225 and transport stream multiplexor 235 may generate decode time reference values (e.g., DTS, etc.) and presentation time reference values (e.g., PTS, etc.) for frames of the transport stream based on the determined presentation offset and the clock reference (e.g., PCR, etc.).
• decode time reference values e.g., DTS, etc.
• presentation time reference values e.g., PTS, etc.
• FIG. 7 is a block diagram illustrating a device 700 for source device control of sink buffer size in accordance with various embodiments.
• the device 700 may be an example of one or more aspects of one of the source devices 1 15 described with reference to FIG. 1 or FIG. 2.
• the device 700 may also be a processor.
• the device 700 may include a content use case module 705, a latency buffer size module 710, an encoder module 715, and an encapsulation module 720. Each of these components may be in communication with each other.
• Content use case module 705 may determine a use case for a media stream to be transmitted to a sink device for output (e.g., display, etc.) at the sink device.
• Content use case module 705 may determine the use case based on a type of application associated with the media stream. For example, content use case module 705 may determine whether the associated application is used for multimedia playback, interactive computing, gaming, or other use case.
• Latency buffer size module 710 may determine a sink buffer size based on the use case of the media stream. For example, the source device 1 15 may select a higher presentation offset for non-interactive multimedia display and other latency tolerant applications and media streams and lower presentation offset for applications and media streams such as interactive computer or gaming that are less latency tolerant. Latency buffer size module 710 may provide the presentation offset to the encoder module 715 and encapsulation module 720. [0084] Encoder module 715 may encode frames of the media stream and encapsulation module 720 may encapsulate the encoded frames into a transport stream for transmission to the sink device.
• FIG. 8 is a block diagram illustrating a device 800 in accordance with various embodiments.
• the device 800 may be an example of one or more aspects of one of the source devices 1 15 described with reference to FIG. 1 or FIG. 2.
• the device 800 may also be a processor.
• the device 800 may include a content use case module 705-a, a latency buffer size module 710-a, an encoder module 715-a, an encapsulation module 720-a, and a scan/concurrency latency module 805. Each of these components may be in communication with each other.
• the device 800 may be configured to implement aspects discussed above with respect to device 700 of FIG. 7 and may not be repeated here for the sake of brevity.
• the content use case module 705-a, latency buffer size module 710-a, encoder module 715-a, and encapsulation module 720-a may be examples of the content use case module 705, latency buffer size module 710, encoder module 715, and encapsulation module 720 of FIG. 7.
• Scan/concurrency latency module 805 may determine an off-channel concurrency time tco N and/or scanning time t S cAN based on the concurrency and scanning configuration. Scan/concurrency latency module 805 may communicate the off-channel concurrency time tco N and/or scanning time tscAN to the latency buffer size module 710-a. The latency buffer size module 710-a may determine the sink buffer size based on the use case of the media stream received from the content use case module 705-a and further based on the off-channel concurrency time tco N and/or scanning time tscAN as described above with reference to FIG. 4.
• the components of the devices 700 and 800 may, individually or collectively, be implemented with one or more application-specific integrated circuits (ASICs) adapted to perform some or all of the applicable functions in hardware.
• ASICs application-specific integrated circuits
• the functions may be performed by one or more other processing units (or cores), on one or more integrated circuits.
• other types of integrated circuits may be used (e.g.,
• FIG. 9 is a block diagram 900 of a source device 1 15-b configured for dynamic control of sink buffer size in accordance with various embodiments.
• Source device 1 15-b may be, for example, the source device 1 15 of FIG. 1 or FIG. 2.
• the source device 1 15-b may have any of various configurations, such as personal computers (e.g., laptop computers, netbook computers, tablet computers, etc.), smartphones, cellular telephones, PDAs, wearable computing devices, digital video recorders (DVRs), internet appliances, gaming consoles, e-readers, etc.
• the source device 1 15-b may have an internal power supply (not shown), such as a small battery, to facilitate mobile operation.
• the source device 1 15-b includes antenna(s) 245-a, a transceiver module 240-a, memory 920, a processor module 970, and I/O devices 980, which each may be in
• the transceiver module 240-a is configured to communicate bi-directionally, via the antennas 245-a and/or one or more wired or wireless links, with one or more networks, as described above.
• the transceiver module 240-a may be configured to communicate bi- directionally with sink devices 135 of FIG. 1 or FIG. 2.
• the transceiver module 240-a may include a modem configured to modulate the packets and provide the modulated packets to the antennas 245-a for transmission, and to demodulate packets received from the antennas 245-a.
• the transceiver module 240-a may be configured to maintain multiple concurrent communication links using the same or different radio interfaces (e.g., Wi-Fi, cellular, etc.).
• the source device 1 15-b may include a single antenna 245-a, or the source device 1 15-b may include multiple antennas 245-a.
• the source device 1 15-b may be capable of employing multiple antennas 245 -a for transmitting and receiving communications in a MIMO communication system.
• the memory 920 may include random access memory (RAM) and read-only memory (ROM).
• the memory 920 may store computer-readable, computer-executable software code 925 containing instructions that are configured to, when executed, cause the processor module 970 to perform various functions described herein (e.g., Wi-Fi Display, dynamic configuration of sink buffer size, etc.).
• the software 925 may not be directly executable by the processor module 970 but be configured to cause the computer (e.g., when compiled and executed) to perform functions described herein.
• the processor module 970 may include an intelligent hardware device, e.g., a central processing unit (CPU), a microcontroller, an application specific integrated circuit (ASIC), etc.
• CPU central processing unit
• ASIC application specific integrated circuit
• the processor module 970 may include a speech encoder (not shown) configured to receive audio via a microphone, convert the audio into packets (e.g., 30 ms in length) representative of the received audio, provide the audio packets to the transceiver module 240-a, and provide indications of whether a user is speaking.
• a speech encoder (not shown) configured to receive audio via a microphone, convert the audio into packets (e.g., 30 ms in length) representative of the received audio, provide the audio packets to the transceiver module 240-a, and provide indications of whether a user is speaking.
• an encoder may only provide packets to the transceiver module 240-a, with the provision or withholding/suppression of the packet itself providing the indication of whether a user is speaking.
• the source device 115-b further includes content use case module 705-b, latency buffer size module 710-b, scan/concurrency latency module 805-a, encoder module 715-b, and encapsulation module 720-b.
• these modules may be components of the source device 115-b in communication with some or all of the other components of the source device 115-b via bus 975. Additionally or Alternatively, functionality of these modules may be implemented as a component of the transceiver module 240-a, as a computer program product, and/or as one or more controller elements of the processor module 970.
• Content use case module 705-b may determine a use case for a media stream to be transmitted to a sink device 135 for output (e.g., display, etc.) at the sink device 135.
• Content use case module 705-b may determine the use case based on a type of application associated with the media stream. For example, content use case module 705-b may determine whether the associated application is used for multimedia playback, interactive computing, gaming, or other use case.
• the media stream may be concurrently output at the source device via I/O devices 980.
• Latency buffer size module 710-b may determine a sink buffer size based on the use case of the media stream. For example, the latency buffer size module 710-b may select a higher sink buffer size for non-interactive multimedia display and other latency tolerant applications and media streams and lower sink buffer size for applications and media streams such as interactive computer or gaming that are less latency tolerant. Latency buffer size module 710-b may provide the sink buffer size to the encoder module 715-b and
• Encoder module 715-b may encode frames of the media stream and encapsulation module 720-b may encapsulate the encoded frames into a transport stream for transmission to the sink device.
• the encoded frames may be associated with time reference values (e.g., PTS, DTS, etc.) in the transport stream, where the associated time reference values are offset from a shared clock reference by values determined from the sink buffer size.
• Scan/concurrency latency module 805-a may determine an off-channel concurrency time tco N and/or scanning time tscAN based on the concurrency and scanning configuration.
• Scan/concurrency latency module 805-a may communicate the off-channel concurrency time tco N and/or scanning time tscAN to the latency buffer size module 710-b.
• the latency buffer size module 710-b may determine the sink buffer size based on the use case of the media stream received from the content use case module 705 -b and further based on the off-channel concurrency time tcoN and/or scanning time tscAN-
• the components of the source device 1 15-b may, individually or collectively, be implemented with one or more application-specific integrated circuits (ASICs) adapted to perform some or all of the applicable functions in hardware.
• ASICs application-specific integrated circuits
• the functions may be performed by one or more other processing units (or cores), on one or more integrated circuits.
• other types of integrated circuits may be used (e.g.,
• each unit may also be implemented, in whole or in part, with instructions embodied in a memory, formatted to be executed by one or more general or application-specific processors.
• Each of the noted modules may be a means for performing one or more functions related to operation of the source device 1 15-b.
• FIG. 10 is a flow chart illustrating an embodiment of a method 1000 for dynamic control of sink buffer size in accordance with various embodiments.
• the method 1000 is described below with reference to the source devices 1 15 of FIG. 1 , FIG. 2, or FIG. 9, and/or the devices 700 or 800 of FIG. 7 or FIG. 8.
• the devices 700 or 800 described with reference to FIG. 7 or FIG. 8 may execute one or more sets of codes to control the functional elements of a source device 1 15 to perform the functions described below.
• an application type of a media stream for transmission to a sink device 135 may be determined.
• a source device 1 15 may determine whether an application associated with the media stream is used for multimedia playback, interactive computing, gaming, or other use case.
• the application type may be determined by an API of the source device based on a call from the application or from monitoring the task manager.
• a buffer size for buffering the transport stream at the sink device 135 is determined based on the application type. For example, a source device 1 15 may select a higher sink buffer size for non-interactive multimedia display and other latency tolerant applications and media streams and lower sink buffer size for applications and media streams such as interactive computer or gaming that are less latency tolerant.
• frames of the media stream are encoded and may be associated with time reference values relative to a shared clock reference based on the determined sink buffer size.
• decode time reference values e.g., DTS, etc.
• presentation time reference values e.g., PTS, etc.
• the clock reference e.g., PCR, etc.
• the encoded frames may be encapsulated in a transport stream (e.g., MPEG-TS, etc.) for transmission to the sink device 135.
• the source device 1 15 may transmit the transport stream to the sink device 135 over a connection link (e.g., Wi-Fi Display connection, etc.) supporting the transport stream for output at the sink device 135.
• FIG. 11 is a flow chart illustrating an embodiment of a method 1100 for dynamic control of sink buffer size in accordance with various embodiments. For clarity, the method 1100 is described below with reference to the source devices 115 of FIG. 1, FIG. 2, or FIG. 9, and/or the devices 700 or 800 of FIG. 7 or FIG. 8. In one implementation, the devices 700 or 800 described with reference to FIG. 7 or FIG. 8 may execute one or more sets of codes to control the functional elements of a source device 115 to perform the functions described below.
• an application type of a media stream for transmission to a sink device may be determined.
• a source device may determine whether an application associated with the media stream is used for multimedia playback, interactive computing, gaming, or other use case.
• the application type may be determined by an API of the source device based on a call from the application or from monitoring the task manager.
• the source device 115 may determine an off-channel concurrency latency based on a concurrent connection configuration of the source device. For example, where the source device 115 is using a Wi-Fi Display connection to transmit the media stream to the sink device 135, the source device 1 15 may determine if other Wi-Fi connections or cellular network connections are concurrently supported and the effect of the concurrently supported connections on latency of the Wi-Fi Display connection. [0107] At block 1115, the source device 115 may determine a scanning latency based on a scanning configuration of the source device. For example, the source device 115 may be currently connected to an access point 105 and may determine that active or passive scanning is to be performed for channels associated with the access point 105.
• the source device 115 determines a buffer size for buffering the transport stream at the sink device 135 based on the application type, the off-channel concurrency latency, and/or the scanning latency. For example, the source device 115 may determine an application-based sink buffer size based on the determined application type of the media stream and may add the off-channel concurrency latency and the scanning latency to the application-based sink buffer size to determine the sink buffer size to be used for encapsulation of the media stream into a transport stream. [0109] At block 1 125, frames of the media stream are encoded and may be associated with time reference values relative to a shared clock reference based on the determined sink buffer size.
• decode time reference values e.g., DTS, etc.
• presentation time reference values e.g., PTS, etc.
• the sink buffer size e.g., PCR, etc.
• the clock reference e.g., PCR, etc.
• the encoded frames may be encapsulated in a transport stream (e.g., MPEG-TS, etc.) for transmission to the sink device 135.
• the source device 1 15 may transmit the transport stream to a sink device 135 over a connection link (e.g., Wi-Fi Direct connection, etc.) supporting the transport stream for output at the sink device 135.
• a connection link e.g., Wi-Fi Direct connection, etc.
• Information and signals may be represented using any of a variety of different technologies and techniques.
• data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.
• DSP digital signal processor
• ASIC application specific integrated circuit
• FPGA field programmable gate array
• a general-purpose processor may be a
• processor may be any conventional processor, controller, microcontroller, or state machine.
• a processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.
• the functions described herein may be implemented in hardware, software executed by a processor, firmware, or any combination thereof. If implemented in software executed by a processor, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Other examples and implementations are within the scope and spirit of the disclosure and appended claims. For example, due to the nature of software, functions described above can be implemented using software executed by a processor, hardware, firmware, hardwiring, or combinations of any of these. Features implementing functions may also be physically located at various positions, including being distributed such that portions of functions are implemented at different physical locations.
• Computer-readable media includes both computer storage media and
• a storage medium may be any available medium that can be accessed by a general purpose or special purpose computer.
• computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code means in the form of instructions or data structures and that can be accessed by a general-purpose or special- purpose computer, or a general-purpose or special-purpose processor.
• any connection is properly termed a computer-readable medium.
• Disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above are also included within the scope of computer-readable media.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Computer Networks & Wireless Communication (AREA)
• Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
• Synchronisation In Digital Transmission Systems (AREA)

Priority Applications (3)

Applications Claiming Priority (2)

Publications (1)

Family

ID=51539334

Family Applications (1)

Country Status (5)

Cited By (2)

Families Citing this family (27)

Citations (5)

Family Cites Families (43)

• 2013
  • 2013-09-04 US US14/018,019 patent/US9826015B2/en not_active Expired - Fee Related
• 2014
  • 2014-08-25 JP JP2016540908A patent/JP6382319B2/ja not_active Expired - Fee Related
  • 2014-08-25 CN CN201480049037.1A patent/CN105556977A/zh active Pending
  • 2014-08-25 WO PCT/US2014/052476 patent/WO2015034698A1/en not_active Ceased
  • 2014-08-25 EP EP14766033.6A patent/EP3042502A1/en not_active Withdrawn

Patent Citations (5)

Also Published As

Similar Documents

Legal Events