CA3050636C - Reducing latency for streaming video - Google Patents
Reducing latency for streaming video Download PDFInfo
- Publication number
- CA3050636C CA3050636C CA3050636A CA3050636A CA3050636C CA 3050636 C CA3050636 C CA 3050636C CA 3050636 A CA3050636 A CA 3050636A CA 3050636 A CA3050636 A CA 3050636A CA 3050636 C CA3050636 C CA 3050636C
- Authority
- CA
- Canada
- Prior art keywords
- media
- encoded
- media program
- program
- media player
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 claims description 21
- 230000008569 process Effects 0.000 claims description 18
- 230000004044 response Effects 0.000 claims description 8
- 230000006870 function Effects 0.000 description 16
- 230000015654 memory Effects 0.000 description 9
- 238000004891 communication Methods 0.000 description 7
- 238000012545 processing Methods 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000012546 transfer Methods 0.000 description 3
- 238000013500 data storage Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 230000002123 temporal effect Effects 0.000 description 2
- 241000183290 Scleropages leichardti Species 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000010267 cellular communication Effects 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000005236 sound signal Effects 0.000 description 1
Landscapes
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
Before receiving an instruction from a media player to encode a media program, a placeshifting device or other video encoder pre-encodes at least a portion of the media program to a low-bandwidth format that can be readily transmitted to the media player upon request. When the media player does instruct the placeshifting device to encode the media program, the placeshifting device initially provides the pre-encoded portion of the media program to thereby allow the media player to begin immediate playback of the pre-encoded portion. After providing the preencoded portion of the media program, the placeshifting device live encodes the remaining portions of the media program according to then-current network conditions to thereby provide the placeshifted video stream to the media player.
Description
REDUCING LATENCY FOR STREAMING VIDEO
TECHNICAL FIELD
[0001] The following discussion generally relates to placeshifted media streaming, and more particularly relates to systems, devices and methods for reducing latency while during placeshifting of live encoded video content.
BACKGROUND
TECHNICAL FIELD
[0001] The following discussion generally relates to placeshifted media streaming, and more particularly relates to systems, devices and methods for reducing latency while during placeshifting of live encoded video content.
BACKGROUND
[0002] Recently, consumers have expressed significant interest in "place shifting"
devices that allow viewing of television or other media content at locations other than their primary television set. Place shifting devices typically packetize media content that can be transmitted over a local or wide area network to a portable computer, mobile phone, personal digital assistant, remote television or other remote device capable of playing back the packetized media stream for the viewer.
Placeshifting therefore allows consumers to view their media content from remote locations such as other rooms, hotels, offices, and/or any other locations where portable media player devices can gain access to a wireless or other communications network.
devices that allow viewing of television or other media content at locations other than their primary television set. Place shifting devices typically packetize media content that can be transmitted over a local or wide area network to a portable computer, mobile phone, personal digital assistant, remote television or other remote device capable of playing back the packetized media stream for the viewer.
Placeshifting therefore allows consumers to view their media content from remote locations such as other rooms, hotels, offices, and/or any other locations where portable media player devices can gain access to a wireless or other communications network.
[0003] Typically, placeshifted media streams are encoded in real time "on the fly"
to adapt to then-current network bandwidth, processing resources, and the like.
Generally speaking, the placeshifted stream is encoded in real-time (or near real time) at the best possible quality that can be transmitted over the then-current network conditions for presentation to the viewer without stalling during playback.
to adapt to then-current network bandwidth, processing resources, and the like.
Generally speaking, the placeshifted stream is encoded in real-time (or near real time) at the best possible quality that can be transmitted over the then-current network conditions for presentation to the viewer without stalling during playback.
[0004] One drawback of real-time encoding, however, is that some latency is inherent, especially when the video is first started or restarted after a "skip", "seek"
or other trick play operation. Generally speaking, it is desired that users have the same experiences with placeshifted content that they would have with other media viewing experiences (e.g., watching television at home, or viewing a VOD
stream).
In practice, however, this is difficult to achieve. The realities of live video encoding, network communications and other factors can often make starting up and/or skipping around in the video stream much more difficult. This is because it can take some time after the user commands the change in the video stream to encode the content from the new start point, to transfer the newly-encoded content to the player over the network, and to buffer enough newly-encoded video at the player to support reliable playback.
or other trick play operation. Generally speaking, it is desired that users have the same experiences with placeshifted content that they would have with other media viewing experiences (e.g., watching television at home, or viewing a VOD
stream).
In practice, however, this is difficult to achieve. The realities of live video encoding, network communications and other factors can often make starting up and/or skipping around in the video stream much more difficult. This is because it can take some time after the user commands the change in the video stream to encode the content from the new start point, to transfer the newly-encoded content to the player over the network, and to buffer enough newly-encoded video at the player to support reliable playback.
[0005] It is therefore desirable to create systems and methods to efficiently and effectively provide placeshifted video content to a remote media player. These and other desirable features and characteristics will become apparent from the subsequent detailed description and the appended claims, taken in conjunction with the accompanying drawings and this background section.
BRIEF SUMMARY
BRIEF SUMMARY
[0006] Various embodiments relate to systems, devices and automated processes to reduce latency in placeshifted media streams through the use of pre-encoded video.
[0007] Before receiving an instruction from a media player to encode a media program, a placeshifting device or other video encoder pre-encodes at least a portion of the media program to a low-bandwidth format that can be readily transmitted to the media player upon request. When the media player does instruct the placeshifting device to encode the media program, the placeshifting device initially provides the pre-encoded portion of the media program to thereby allow the media player to begin immediate playback of the pre-encoded portion, which typically represents a few seconds or so of content. After providing the pre-encoded portion of the media program, the placeshifting device live encodes the remaining portions of the media program according to then-current network conditions to thereby provide the placeshifted video stream to the media player.
[0008]In one example embodiment, an automated process performed by a video encoder device provides a placeshifted video stream to a media player via a network.
The process suitably comprises receiving a media program by the video encoder device before receiving an instruction from the media player via the network to encode the media program, the video encoder device pre-encoding at least a portion of the media program to a placeshifting format; in response to the video encoder device receiving the instruction from the media player to encode the media program, the video encoder device initially providing the pre-encoded portion of the media program to the media player to thereby allow the media player to begin immediate playback of the pre-encoded portion; and after providing the pre-encoded portion of the media program in response to the instruction, the video encoder device live encoding remaining portions of the media program according to then-current network conditions to thereby provide the placeshifted video stream to the media player.
The process suitably comprises receiving a media program by the video encoder device before receiving an instruction from the media player via the network to encode the media program, the video encoder device pre-encoding at least a portion of the media program to a placeshifting format; in response to the video encoder device receiving the instruction from the media player to encode the media program, the video encoder device initially providing the pre-encoded portion of the media program to the media player to thereby allow the media player to begin immediate playback of the pre-encoded portion; and after providing the pre-encoded portion of the media program in response to the instruction, the video encoder device live encoding remaining portions of the media program according to then-current network conditions to thereby provide the placeshifted video stream to the media player.
[0009] In another example embodiment, a media encoder device is provided. The media encoder device suitably comprises: a television receiver interface configured to receive broadcasts of television programming; a network interface to a digital network; a digital storage configured to record and store at least some of the television programming received via the television receiver interface in a storage format; and a processor. The processor is appropriately configured to execute instructions to perform an automated process; the instructions may be stored in a digital memory and executed by the processor as desired. In one example, the processor is configured: prior to receiving an instruction from a remote media player to encode at least some of the television programming for placeshifting, to automatically pre-encode portions of at least some of the television programming stored on the digital storage in a placeshifting format that is different from the storage format; in response to subsequently receiving the instruction from the remote media player to encode the media program, initially providing the pre-encoded portion of the media program to the media player to thereby allow the media player to begin immediate playback of the pre-encoded portion; and after providing the pre-encoded portion of the media program in response to the instruction, live encoding remaining portions of the media program according to then-current conditions of the digital network to thereby provide the placeshifted video stream to the remotely-located media player via the digital network.
[oolo] Additional details about these and other example embodiments are provided herein.
BRIEF DESCRIPTION OF THE DRAWING FIGURES
[coon] Various example embodiments will hereinafter be described in conjunction with the following drawing figures, wherein like numerals denote like elements, and [0012] FIG. 1 is a block diagram of an example media encoding system;
[0013] FIG. 2 is a block diagram of an example media live encoding device; and [0014] FIG. 3 illustrates an example process to deliver encoded content with reduced latency.
DETAILED DESCRIPTION
[0015] The following detailed description of the invention is merely exemplary in nature and is not intended to limit the invention or the application and uses of the invention. Furthermore, there is no intention to be bound by any theory presented in the preceding background or the following detailed description.
[0016] Real-time video encoding used in the placeshifting setting can be enhanced with pre-encoded video to reduce latency. According to various embodiments, the placeshifting device pre-encodes at least a portion of the media program prior to placeshifting so that the expected start points (e.g., the beginning of the program) can be immediately (or at least very quickly) provided to the media player without real-time encoding. While the viewer is watching the first few seconds of the media program from the pre-encoded content, the placeshifting device can live encode the remainder of the stream in real-time (or even faster than real time) so that the video quality can be improved and the adaptive benefits of traditional placeshifting can be enjoyed through the remainder of the stream.
[0017] The amount of content that will be pre-encoded will vary from embodiment to embodiment according to the amount of content, the amount of storage that is available, and the amount of time that is available to perform the pre-encoding. If the source program is recorded on a DVR or is otherwise readily available in its entirety, for example, then there may be sufficient time and data storage to pre-encode the entire program (or at least substantial portions of the program) in the fast start format so that any seeking or trick play within the program can be met with fast start content at the newly-commanded start point. If the source program is a live broadcast (e.g., a sports program), however, there may not be sufficient cached source content to pre-encode more than a few seconds or so without creating additional delay for the viewer. Even with live broadcasts, however, it may be possible to quickly pre-encode a few seconds of startup content without creating substantial lag between the live broadcast and the placeshifted stream.
[0018] Various embodiments can pre-encode startup content at expected start points in the video content. Such start points may include the beginning of the program, the first few seconds after the viewer previously paused or switched away from the program, or the like.
[0019] Pre-encoded content may be delivered to the media player in any manner.
In various embodiments, pre-encoded portions are only available for the start point of the video, and/or the resume point of a previously-watched/paused video. In other embodiments wherein multiple pre-encoded portions are available, however, all of the pre-encoded portions could be transferred to the media player as soon as possible (as determined by factors such as storage, processing capabilities and bandwidth) during the streaming session. If the user were to skip to a different point in the video that has been pre-encoded, then the player could immediately begin playback of the cached pre-encoded content while the remote encoder catches up with live encoding. The pre-encoded content may be delivered out-of-band, if desired. Various embodiments could pre-cache the pre-encoded contents of certain programs on the player even before streaming begins if sufficient time, bandwidth and storage are available.
[0020] A "few seconds" of pre-encoded content as used herein is intended to convey that the specific amount of pre-encoded content will vary from embodiment to embodiment. Ideally, the "few seconds" of content represents sufficient lead time to allow the encoder to produce and deliver live-encoded content without substantial lag to the viewer. Currently, this lag time can be on the order of three to five seconds, although that time may change with evolutions in encoder speed, network bandwidth, program delivery and other technologies.
[0021] Different embodiments may produce different amounts of pre-encoded content to support placeshifting to different types of devices or over different types of networks, as appropriate. Streaming to a personal computer via a LAN, for example, may incur less transmit time lag than streaming to a mobile phone over a mobile network. While encoder delay for both of these scenarios will typically be consistent, the additional delays may nevertheless dictate different amounts of preencoded data in some embodiments.
[0022] Turning now to the drawing figures and with initial reference to FIG.
1, an example placeshifting system 100 suitably includes a placeshifting encoder system 102 that receives media content 122 from a DVR or other content source 106 and pre-encodes at least some of the received content 122 prior to streaming over network 105. The pre-encoded content 109 may be stored in a data storage 107 until placeshifting begins, as desired. When the media player 104 requests a stream of the received content 122, the placeshifting device 102 initially retrieves the preencoded content 109 from storage 107 and delivers the pre-encoded content 109 to support fast startup by media player 104. Placeshifting device 102 then begins live encoding the content 122, as would be expected in traditional placeshifting. The live encoded content can then be transmitted as the encoded media stream 120 to a media player 104 over network 110 while the placeshifting device 102 continues to live encode additional content.
[0023] The media player 104 initially receives the pre-encoded content 109, which can be rendered for immediate playback on display 108 until the player 104 receives the live-encoded stream 120. In various embodiments, a server 112 may also be provided to communicate with encoder system 102 and/or player 104 via network no for assisting these devices in locating each other, maintaining security, providing or receiving content or information, and/or performing any other features. This feature is not required in all embodiments, however, and the concepts described herein may be deployed in any data streaming application or environment, including placeshifting but also any other media or other data streaming situations as desired.
[0024] Placeshifting encoder system 102 is any component, hardware, software logic and/or the like capable of transmitting a packetized stream of media content over network no. In various embodiments, placeshifting device 102 is a set top box (STB) or other integrated device with television receiver, DVR and/or other content sources 106 built in. Placeshifting device 102 incorporates suitable encoder and/or transcoder (collectively "encoder") logic to convert audio/video or other media data 122 into a packetized format that can be transmitted over network no. The media data 122 may be received in any format, and may be received from any internal or external source 106 such as any sort of broadcast, cable or satellite television programming source, a "video-on-demand" or similar source, a digital video disk (DVD) or other removable media, a video camera, a video-on-demand (VOD) stream, and/or the like. Encoder system 102 encodes media data 122 to create media stream 120 in any manner. As noted above, some or all of the available program content may be pre-encoded 109 and stored in storage 107 prior to placeshifting. These pre-encoded contents 109 can be immediately delivered to the media player 104 for fast start up, as described in more detail herein.
[0025] In various embodiments, encoder system 102 contains a transmit buffer 105 that temporarily stores live encoded data prior to transmission on network no.
As buffer 105 fills or empties, one or more parameters of the live encoding (e.g., the bit rate of media stream 120) may be adjusted to maintain desirable picture quality and data throughput in view of the then-current network performance. Changes in the network transfer rate may be identified from, for example, changes in the utilization of the outgoing buffer 105. Other embodiments may additionally or alternatively consider feedback from the remote media player 104, or any other data as appropriate. As noted herein, one or more parameters such as bit rate, frame rate, resolution, etc. of the live encoded stream 120 may be set so that the content is delivered to the remote media player 104 at an appropriate quality level so that video quality is as good as possible in view of then-current network conditions.
[0026] Several examples of placeshifting systems 102 may be implemented using any of the various SLINGBOXC) products available from Sling Media, Inc. of Foster City, California, although other products could be used in other embodiments.
Many different types of encoder systems 102 are generally capable of receiving media content 122 from an external source 106 such as any sort of digital video recorder (DVR), set top box (STB), cable or satellite programming source, DVD
player, and/or the like. In such embodiments, encoder system 102 may additionally provide commands 124 to the source 106 to produce desired signals 122. Such commands 124 may be provided over any sort of wired or wireless interface, such as an infrared or other wireless transmitter that emulates remote control commands receivable by the source 106.
[0027] In other embodiments, encoder system 102 may be integrated with any sort of content receiving or other capabilities typically affiliated with source 106.
Encoder system 102 may be a hybrid STB, receiver, media player, DVR, game console or other device, for example, that also provides transcoding and placeshifting features. Such a device may receive satellite, cable, broadcast and/or other signals that encode television programming or other content received from an antenna, modem, server and/or other source. The receiver may further demodulate or otherwise decode the received signals to extract programming that can be locally viewed, stored in DVR or other storage, and/or place shifted to a remote player 104 as appropriate. Such devices 102 may also include a content database stored on a hard disk drive, memory, or other storage medium to support a personal or digital video recorder (DVR) feature or other content library as appropriate. Hence, in many embodiments, source 106 and encoder system 102 may be physically and/or logically contained within a common box, component, housing or chassis.
[0028] Media player 104 is any device, component, module, hardware, software and/or the like capable of receiving a media stream 120 from one or more encoder systems 102. In various embodiments, remote player 104 is personal computer (e.g., a "laptop" or similarly portable computer, although desktop-type computers could also be used), a mobile phone, a personal digital assistant, a personal media player (such as the media player products available from Roku Inc. of Saratoga, California) or the like. In many embodiments, remote player 104 is a general purpose computing device (e.g., a phone, tablet, personal computer, etc.) that includes a media player application 157 in software or firmware that is capable of securely connecting to placeshifting encoder system 102, as described more fully below, and of receiving and presenting media content to the user of the device as appropriate.
In other embodiments, however, media player 104 is a standalone or other separate hardware device capable of receiving the media stream 120 via any portion of network 110 and decoding the media stream 120 to provide an output signal 126 that is presented on a television or other display 108. Media players 104 typically include any conventional processors, memory and input/output features found in typical computing systems. Media player 104 may include any sort of storage 117 for caching pre-encoded video 109, as desired. Storage 117 may physically reside in memory or other mass storage present within player device 104.
[0029] Network no is any digital or other communications network capable of transmitting messages between senders (e.g., encoder system 102) and receivers (e.g., receiver 104). In various embodiments, network no includes any number of public or private data connections, links or networks supporting any number of communications protocols. Network no may include the Internet, for example, or any other network based upon TCP/IP or other conventional protocols. In various embodiments, network 110 also incorporates a wireless and/or wired telephone network, such as a cellular communications network for communicating with mobile phones, personal digital assistants, and/or the like. Network no may also incorporate any sort of wireless or wired local area networks, such as one or more IEEE 802.3 and/or IEEE 802.11 networks.
[0030] Encoder system 102 is therefore able to communicate with player 104 in any manner (e.g., using any sort of data connections 128 and/or 125, respectively).
Such communication may take place over a wide area link that includes the Internet and/or a telephone network, for example; in other embodiments, communications between devices 102 and 104 may take place over one or more wired or wireless local area links that are conceptually incorporated within network no. In various equivalent embodiments, encoder system 102 and receiver 104 may be directly connected via any sort of wired or wireless link (e.g., an Ethernet cable or the like) with little or no other network functionality provided.
[0031] Many different placeshifting scenarios could be formulated based upon available computing and communications resources, consumer demand and/or any other factors. In various embodiments, consumers may wish to placeshift content within a home, office or other structure, such as from a placeshifting encoder system 102 to a desktop or portable computer located in another room. In such embodiments, the content stream will typically be provided over a wired or wireless local area network operating within the structure. In other embodiments, consumers may wish to placeshift content over a broadband or similar network connection from a primary location to a computer or other remote player 104 located in a second home, office, hotel or other remote location. In still other embodiments, consumers may wish to placeshift content to a mobile phone, personal digital assistant, media player, video game player, automotive or other vehicle media player, and/or other device via a mobile link (e.g., a GSM/EDGE
or CDMA/EVDO connection, any sort of 3G, 4G and/or subsequent telephone link, an IEEE 802.11 "Wi-fl" link, and/or the like). Several examples of placeshifting applications available for various platforms are provided by Sling Media, Inc.
of Foster City, California, although the concepts described herein could be used in conjunction with products and services available from any source.
[0032] Encoder system 102, then, generally creates a media stream 120 that is routable on network no based upon content 122 received from media source 106.
Expected start locations (e.g., the beginning of the program, the resume point after previous viewing, etc.) of the program can be pre-encoded 109 and cached in storage 107 for immediate use when placeshifting begins (or continues, as appropriate). These pre-encoded portions 109 may be shared with the media player 104 for caching in storage 117 on any temporal basis. In some implementations, the live encoding of media stream 120 may occur simultaneous with transmission of the stream on network no at a rate that is faster than the rate at which content is received and/or played back to the viewer, thereby allowing excess content to be cached at media player 104 (e.g., in storage 117 or the like). Storage 117 may therefore be able to store both pre-encoded content 109 and excess live-encoded content as well. In the example shown in FIG. 1, media player 104 suitably provides an interface 151 that includes the rendered media stream, along with user controls 152 to initiate play, pause, fast forward, rewind or other trick play functions.
[0033] Interface 151 also includes a buffer indicator 150 that shows the contents of the media stream for the program currently being played back for the user. In this example, a portion 153 of buffer 150 is shaded, indicating that that portion of the encoded media program has been received at the media player 104. The current playback location is indicated by feature 154; in various embodiments, the user may manipulate feature 154 to jump forward or backward in buffer 150 and thereby change the playback location in the rendered program, as desired. If pre-encoded content 109 is cached in storage 117 for the new playback location, then playback can begin immediately from the pre-encoded content 109 without waiting for placeshifting device 102. If pre-encoded content 109 is not cached on the player storage 117, such content may be available from placeshifting device 102 (e.g., from storage 107). If no pre-encoded content 109 is available for the new start point, then encoder 102 will begin live encoding the requested content, albeit with some latency.
[oolo] Additional details about these and other example embodiments are provided herein.
BRIEF DESCRIPTION OF THE DRAWING FIGURES
[coon] Various example embodiments will hereinafter be described in conjunction with the following drawing figures, wherein like numerals denote like elements, and [0012] FIG. 1 is a block diagram of an example media encoding system;
[0013] FIG. 2 is a block diagram of an example media live encoding device; and [0014] FIG. 3 illustrates an example process to deliver encoded content with reduced latency.
DETAILED DESCRIPTION
[0015] The following detailed description of the invention is merely exemplary in nature and is not intended to limit the invention or the application and uses of the invention. Furthermore, there is no intention to be bound by any theory presented in the preceding background or the following detailed description.
[0016] Real-time video encoding used in the placeshifting setting can be enhanced with pre-encoded video to reduce latency. According to various embodiments, the placeshifting device pre-encodes at least a portion of the media program prior to placeshifting so that the expected start points (e.g., the beginning of the program) can be immediately (or at least very quickly) provided to the media player without real-time encoding. While the viewer is watching the first few seconds of the media program from the pre-encoded content, the placeshifting device can live encode the remainder of the stream in real-time (or even faster than real time) so that the video quality can be improved and the adaptive benefits of traditional placeshifting can be enjoyed through the remainder of the stream.
[0017] The amount of content that will be pre-encoded will vary from embodiment to embodiment according to the amount of content, the amount of storage that is available, and the amount of time that is available to perform the pre-encoding. If the source program is recorded on a DVR or is otherwise readily available in its entirety, for example, then there may be sufficient time and data storage to pre-encode the entire program (or at least substantial portions of the program) in the fast start format so that any seeking or trick play within the program can be met with fast start content at the newly-commanded start point. If the source program is a live broadcast (e.g., a sports program), however, there may not be sufficient cached source content to pre-encode more than a few seconds or so without creating additional delay for the viewer. Even with live broadcasts, however, it may be possible to quickly pre-encode a few seconds of startup content without creating substantial lag between the live broadcast and the placeshifted stream.
[0018] Various embodiments can pre-encode startup content at expected start points in the video content. Such start points may include the beginning of the program, the first few seconds after the viewer previously paused or switched away from the program, or the like.
[0019] Pre-encoded content may be delivered to the media player in any manner.
In various embodiments, pre-encoded portions are only available for the start point of the video, and/or the resume point of a previously-watched/paused video. In other embodiments wherein multiple pre-encoded portions are available, however, all of the pre-encoded portions could be transferred to the media player as soon as possible (as determined by factors such as storage, processing capabilities and bandwidth) during the streaming session. If the user were to skip to a different point in the video that has been pre-encoded, then the player could immediately begin playback of the cached pre-encoded content while the remote encoder catches up with live encoding. The pre-encoded content may be delivered out-of-band, if desired. Various embodiments could pre-cache the pre-encoded contents of certain programs on the player even before streaming begins if sufficient time, bandwidth and storage are available.
[0020] A "few seconds" of pre-encoded content as used herein is intended to convey that the specific amount of pre-encoded content will vary from embodiment to embodiment. Ideally, the "few seconds" of content represents sufficient lead time to allow the encoder to produce and deliver live-encoded content without substantial lag to the viewer. Currently, this lag time can be on the order of three to five seconds, although that time may change with evolutions in encoder speed, network bandwidth, program delivery and other technologies.
[0021] Different embodiments may produce different amounts of pre-encoded content to support placeshifting to different types of devices or over different types of networks, as appropriate. Streaming to a personal computer via a LAN, for example, may incur less transmit time lag than streaming to a mobile phone over a mobile network. While encoder delay for both of these scenarios will typically be consistent, the additional delays may nevertheless dictate different amounts of preencoded data in some embodiments.
[0022] Turning now to the drawing figures and with initial reference to FIG.
1, an example placeshifting system 100 suitably includes a placeshifting encoder system 102 that receives media content 122 from a DVR or other content source 106 and pre-encodes at least some of the received content 122 prior to streaming over network 105. The pre-encoded content 109 may be stored in a data storage 107 until placeshifting begins, as desired. When the media player 104 requests a stream of the received content 122, the placeshifting device 102 initially retrieves the preencoded content 109 from storage 107 and delivers the pre-encoded content 109 to support fast startup by media player 104. Placeshifting device 102 then begins live encoding the content 122, as would be expected in traditional placeshifting. The live encoded content can then be transmitted as the encoded media stream 120 to a media player 104 over network 110 while the placeshifting device 102 continues to live encode additional content.
[0023] The media player 104 initially receives the pre-encoded content 109, which can be rendered for immediate playback on display 108 until the player 104 receives the live-encoded stream 120. In various embodiments, a server 112 may also be provided to communicate with encoder system 102 and/or player 104 via network no for assisting these devices in locating each other, maintaining security, providing or receiving content or information, and/or performing any other features. This feature is not required in all embodiments, however, and the concepts described herein may be deployed in any data streaming application or environment, including placeshifting but also any other media or other data streaming situations as desired.
[0024] Placeshifting encoder system 102 is any component, hardware, software logic and/or the like capable of transmitting a packetized stream of media content over network no. In various embodiments, placeshifting device 102 is a set top box (STB) or other integrated device with television receiver, DVR and/or other content sources 106 built in. Placeshifting device 102 incorporates suitable encoder and/or transcoder (collectively "encoder") logic to convert audio/video or other media data 122 into a packetized format that can be transmitted over network no. The media data 122 may be received in any format, and may be received from any internal or external source 106 such as any sort of broadcast, cable or satellite television programming source, a "video-on-demand" or similar source, a digital video disk (DVD) or other removable media, a video camera, a video-on-demand (VOD) stream, and/or the like. Encoder system 102 encodes media data 122 to create media stream 120 in any manner. As noted above, some or all of the available program content may be pre-encoded 109 and stored in storage 107 prior to placeshifting. These pre-encoded contents 109 can be immediately delivered to the media player 104 for fast start up, as described in more detail herein.
[0025] In various embodiments, encoder system 102 contains a transmit buffer 105 that temporarily stores live encoded data prior to transmission on network no.
As buffer 105 fills or empties, one or more parameters of the live encoding (e.g., the bit rate of media stream 120) may be adjusted to maintain desirable picture quality and data throughput in view of the then-current network performance. Changes in the network transfer rate may be identified from, for example, changes in the utilization of the outgoing buffer 105. Other embodiments may additionally or alternatively consider feedback from the remote media player 104, or any other data as appropriate. As noted herein, one or more parameters such as bit rate, frame rate, resolution, etc. of the live encoded stream 120 may be set so that the content is delivered to the remote media player 104 at an appropriate quality level so that video quality is as good as possible in view of then-current network conditions.
[0026] Several examples of placeshifting systems 102 may be implemented using any of the various SLINGBOXC) products available from Sling Media, Inc. of Foster City, California, although other products could be used in other embodiments.
Many different types of encoder systems 102 are generally capable of receiving media content 122 from an external source 106 such as any sort of digital video recorder (DVR), set top box (STB), cable or satellite programming source, DVD
player, and/or the like. In such embodiments, encoder system 102 may additionally provide commands 124 to the source 106 to produce desired signals 122. Such commands 124 may be provided over any sort of wired or wireless interface, such as an infrared or other wireless transmitter that emulates remote control commands receivable by the source 106.
[0027] In other embodiments, encoder system 102 may be integrated with any sort of content receiving or other capabilities typically affiliated with source 106.
Encoder system 102 may be a hybrid STB, receiver, media player, DVR, game console or other device, for example, that also provides transcoding and placeshifting features. Such a device may receive satellite, cable, broadcast and/or other signals that encode television programming or other content received from an antenna, modem, server and/or other source. The receiver may further demodulate or otherwise decode the received signals to extract programming that can be locally viewed, stored in DVR or other storage, and/or place shifted to a remote player 104 as appropriate. Such devices 102 may also include a content database stored on a hard disk drive, memory, or other storage medium to support a personal or digital video recorder (DVR) feature or other content library as appropriate. Hence, in many embodiments, source 106 and encoder system 102 may be physically and/or logically contained within a common box, component, housing or chassis.
[0028] Media player 104 is any device, component, module, hardware, software and/or the like capable of receiving a media stream 120 from one or more encoder systems 102. In various embodiments, remote player 104 is personal computer (e.g., a "laptop" or similarly portable computer, although desktop-type computers could also be used), a mobile phone, a personal digital assistant, a personal media player (such as the media player products available from Roku Inc. of Saratoga, California) or the like. In many embodiments, remote player 104 is a general purpose computing device (e.g., a phone, tablet, personal computer, etc.) that includes a media player application 157 in software or firmware that is capable of securely connecting to placeshifting encoder system 102, as described more fully below, and of receiving and presenting media content to the user of the device as appropriate.
In other embodiments, however, media player 104 is a standalone or other separate hardware device capable of receiving the media stream 120 via any portion of network 110 and decoding the media stream 120 to provide an output signal 126 that is presented on a television or other display 108. Media players 104 typically include any conventional processors, memory and input/output features found in typical computing systems. Media player 104 may include any sort of storage 117 for caching pre-encoded video 109, as desired. Storage 117 may physically reside in memory or other mass storage present within player device 104.
[0029] Network no is any digital or other communications network capable of transmitting messages between senders (e.g., encoder system 102) and receivers (e.g., receiver 104). In various embodiments, network no includes any number of public or private data connections, links or networks supporting any number of communications protocols. Network no may include the Internet, for example, or any other network based upon TCP/IP or other conventional protocols. In various embodiments, network 110 also incorporates a wireless and/or wired telephone network, such as a cellular communications network for communicating with mobile phones, personal digital assistants, and/or the like. Network no may also incorporate any sort of wireless or wired local area networks, such as one or more IEEE 802.3 and/or IEEE 802.11 networks.
[0030] Encoder system 102 is therefore able to communicate with player 104 in any manner (e.g., using any sort of data connections 128 and/or 125, respectively).
Such communication may take place over a wide area link that includes the Internet and/or a telephone network, for example; in other embodiments, communications between devices 102 and 104 may take place over one or more wired or wireless local area links that are conceptually incorporated within network no. In various equivalent embodiments, encoder system 102 and receiver 104 may be directly connected via any sort of wired or wireless link (e.g., an Ethernet cable or the like) with little or no other network functionality provided.
[0031] Many different placeshifting scenarios could be formulated based upon available computing and communications resources, consumer demand and/or any other factors. In various embodiments, consumers may wish to placeshift content within a home, office or other structure, such as from a placeshifting encoder system 102 to a desktop or portable computer located in another room. In such embodiments, the content stream will typically be provided over a wired or wireless local area network operating within the structure. In other embodiments, consumers may wish to placeshift content over a broadband or similar network connection from a primary location to a computer or other remote player 104 located in a second home, office, hotel or other remote location. In still other embodiments, consumers may wish to placeshift content to a mobile phone, personal digital assistant, media player, video game player, automotive or other vehicle media player, and/or other device via a mobile link (e.g., a GSM/EDGE
or CDMA/EVDO connection, any sort of 3G, 4G and/or subsequent telephone link, an IEEE 802.11 "Wi-fl" link, and/or the like). Several examples of placeshifting applications available for various platforms are provided by Sling Media, Inc.
of Foster City, California, although the concepts described herein could be used in conjunction with products and services available from any source.
[0032] Encoder system 102, then, generally creates a media stream 120 that is routable on network no based upon content 122 received from media source 106.
Expected start locations (e.g., the beginning of the program, the resume point after previous viewing, etc.) of the program can be pre-encoded 109 and cached in storage 107 for immediate use when placeshifting begins (or continues, as appropriate). These pre-encoded portions 109 may be shared with the media player 104 for caching in storage 117 on any temporal basis. In some implementations, the live encoding of media stream 120 may occur simultaneous with transmission of the stream on network no at a rate that is faster than the rate at which content is received and/or played back to the viewer, thereby allowing excess content to be cached at media player 104 (e.g., in storage 117 or the like). Storage 117 may therefore be able to store both pre-encoded content 109 and excess live-encoded content as well. In the example shown in FIG. 1, media player 104 suitably provides an interface 151 that includes the rendered media stream, along with user controls 152 to initiate play, pause, fast forward, rewind or other trick play functions.
[0033] Interface 151 also includes a buffer indicator 150 that shows the contents of the media stream for the program currently being played back for the user. In this example, a portion 153 of buffer 150 is shaded, indicating that that portion of the encoded media program has been received at the media player 104. The current playback location is indicated by feature 154; in various embodiments, the user may manipulate feature 154 to jump forward or backward in buffer 150 and thereby change the playback location in the rendered program, as desired. If pre-encoded content 109 is cached in storage 117 for the new playback location, then playback can begin immediately from the pre-encoded content 109 without waiting for placeshifting device 102. If pre-encoded content 109 is not cached on the player storage 117, such content may be available from placeshifting device 102 (e.g., from storage 107). If no pre-encoded content 109 is available for the new start point, then encoder 102 will begin live encoding the requested content, albeit with some latency.
-10-[0034] With reference now to FIG. 2, encoder system 102 typically includes an encoder module 202, a buffer 105 and a network interface 206 in conjunction with appropriate control logic 205. In operation, encoder module 202 typically receives media content 122 from an internal or external source 106, encodes the data into the desired format for media stream 120, and stores the encoded data in buffer 105.
Network interface 206 then retrieves the formatted data from buffer 105 for transmission on network no. Control module 205 suitably monitors and controls the encoding and network transmit processes carried out by encoding module 202 and network interface 206, respectively, and may perform other features as well. In the example of FIG. 2, an encoder control application 217 suitably resides as software or firmware in memory or other storage and is executed by control module 205 to control and adapt the encoding of media stream 120 using the techniques described herein. Encoder system 102 may also have a module 208 or other feature capable of generating and providing commands 124 to an external media source 106, as described above.
[0035] In the exemplary embodiment shown in FIG. 2, modules 202, 105, 205, 206 and 208 may be implemented in software or firmware residing in any memory, mass storage or other storage medium within encoder system 102 in source code, object code and/or any other format. Such features may be executed on any sort of processor or microcontroller executing within encoder system 102. In various embodiments, encoder system 102 is implemented as a system on a chip (SoC) type system with integrated processing, storage and input/output features. Various SoC
hardware implementations are available from Texas Instruments, Conexant Systems, Broadcom Inc., VIXS Inc., NXP/Qualcomm and/or other suppliers as appropriate. Other embodiments may use any number of discrete and/or integrated processing components, memories, input/output features and/or other features as desired.
[0036] As noted above, creating a media stream 120 typically involves encoding and/or transcoding an input media stream 122 received from an internal or external media source 106 into a suitable digital format that can be transmitted on network no. Generally, the media stream 120 is placed into a standard or other known format (e.g., the WINDOWS MEDIA format available from the Microsoft Corporation of Redmond, Washington although other formats such as the HTTP
LIVE STREAMING format, REALPLAYER format, MPEG format, and/or the like
Network interface 206 then retrieves the formatted data from buffer 105 for transmission on network no. Control module 205 suitably monitors and controls the encoding and network transmit processes carried out by encoding module 202 and network interface 206, respectively, and may perform other features as well. In the example of FIG. 2, an encoder control application 217 suitably resides as software or firmware in memory or other storage and is executed by control module 205 to control and adapt the encoding of media stream 120 using the techniques described herein. Encoder system 102 may also have a module 208 or other feature capable of generating and providing commands 124 to an external media source 106, as described above.
[0035] In the exemplary embodiment shown in FIG. 2, modules 202, 105, 205, 206 and 208 may be implemented in software or firmware residing in any memory, mass storage or other storage medium within encoder system 102 in source code, object code and/or any other format. Such features may be executed on any sort of processor or microcontroller executing within encoder system 102. In various embodiments, encoder system 102 is implemented as a system on a chip (SoC) type system with integrated processing, storage and input/output features. Various SoC
hardware implementations are available from Texas Instruments, Conexant Systems, Broadcom Inc., VIXS Inc., NXP/Qualcomm and/or other suppliers as appropriate. Other embodiments may use any number of discrete and/or integrated processing components, memories, input/output features and/or other features as desired.
[0036] As noted above, creating a media stream 120 typically involves encoding and/or transcoding an input media stream 122 received from an internal or external media source 106 into a suitable digital format that can be transmitted on network no. Generally, the media stream 120 is placed into a standard or other known format (e.g., the WINDOWS MEDIA format available from the Microsoft Corporation of Redmond, Washington although other formats such as the HTTP
LIVE STREAMING format, REALPLAYER format, MPEG format, and/or the like
- 11 -could be used in any other embodiments) that can be transmitted on network no.
This encoding may take place, for example, in any sort of encoding module 202 as appropriate. Encoding module 202 may be any sort of hardware (e.g., a digital signal processor or other integrated circuit used for media encoding), software (e.g., software or firmware programming used for media encoding that executes on the SoC or other processor described above), or the like. Encoding module 202 is therefore any feature that receives media data 122 from the internal or external source 106 (e.g., via any sort of hardware and/or software interface) and encodes or transcodes the received data into the desired format for transmission on network no. Although FIG. 2 shows a single encoding module 202, in practice system 102 may include any number of encoding modules 202. Different encoding modules 202 may be selected based upon preference of player 104, network conditions, and/or the like.
[0037] In various embodiments, encoder 202 may also apply other modifications, transforms and/or filters to the received content before or during the transcoding process. Video signals, for example, may be resized, cropped and/or skewed.
Similarly, the color, hue and/or saturation of the signal may be altered, and/or noise reduction or other filtering may be applied. Audio signals may be modified by adjusting volume, sampling rate, mono/stereo parameters, noise reduction, multichannel sound parameters and/or the like. Digital rights management encoding and/or decoding may also be applied in some embodiments, and/or other features may be applied as desired.
[0038] As initially noted above, system 102 may also include a hard disk drive, memory, or other storage medium 107 to support storage of pre-encoded video 109.
Storage 107 may be implemented with any sort of solid state, magnetic, optical or other storage such as a memory, disk or the like. Storage 107 may be the same storage used to support a digital video recorder (DVR) feature or other content source 106 as appropriate.
[0039] FIG. 3 shows an example process 300 to pre-encode portions of the placeshifted media stream, as desired. The various functions shown in FIG. 3 may be implemented in software, firmware or other programmable logic executing in media player 104 and/or encoder 102, as appropriate. Application 217 of encoder 102, for example, may control the various actions of encode module 202, while
This encoding may take place, for example, in any sort of encoding module 202 as appropriate. Encoding module 202 may be any sort of hardware (e.g., a digital signal processor or other integrated circuit used for media encoding), software (e.g., software or firmware programming used for media encoding that executes on the SoC or other processor described above), or the like. Encoding module 202 is therefore any feature that receives media data 122 from the internal or external source 106 (e.g., via any sort of hardware and/or software interface) and encodes or transcodes the received data into the desired format for transmission on network no. Although FIG. 2 shows a single encoding module 202, in practice system 102 may include any number of encoding modules 202. Different encoding modules 202 may be selected based upon preference of player 104, network conditions, and/or the like.
[0037] In various embodiments, encoder 202 may also apply other modifications, transforms and/or filters to the received content before or during the transcoding process. Video signals, for example, may be resized, cropped and/or skewed.
Similarly, the color, hue and/or saturation of the signal may be altered, and/or noise reduction or other filtering may be applied. Audio signals may be modified by adjusting volume, sampling rate, mono/stereo parameters, noise reduction, multichannel sound parameters and/or the like. Digital rights management encoding and/or decoding may also be applied in some embodiments, and/or other features may be applied as desired.
[0038] As initially noted above, system 102 may also include a hard disk drive, memory, or other storage medium 107 to support storage of pre-encoded video 109.
Storage 107 may be implemented with any sort of solid state, magnetic, optical or other storage such as a memory, disk or the like. Storage 107 may be the same storage used to support a digital video recorder (DVR) feature or other content source 106 as appropriate.
[0039] FIG. 3 shows an example process 300 to pre-encode portions of the placeshifted media stream, as desired. The various functions shown in FIG. 3 may be implemented in software, firmware or other programmable logic executing in media player 104 and/or encoder 102, as appropriate. Application 217 of encoder 102, for example, may control the various actions of encode module 202, while
- 12 -application 157 may control the various actions of media player 104. Other embodiments may carry out the various functions of process 300 using different logic or differently-organized logic, as desired.
[0040]As shown in FIG. 3, process 300 begins with placeshifting encoder 102 receiving content that may be of future placeshifting interest to a viewer (function 302). Such content may include programs stored in a DVR, for example, or other programs that are available from any media source 105. Programs may be manually selected for future placeshifting by the user in some embodiments, whereas other embodiments will automatically attempt to create pre-encoded start points for some or all of the programs available from a content source 106 on an a priori basis as time and resources allow. If an encoder is otherwise idle, for example, the placeshifting device 102 may make use of the idle time to pre-encode as much available content as possible.
[0041] Pre-encoding may be performed in any suitable manner (function 304). In various embodiments, pre-encoding involves converting at least some portions of available programs to a streaming format for quick playback. To that end, it may be desirable to pre-encode at least the startup content at a relatively low quality to facilitate rapid transit of network no, even via limited bandwidth. Lower quality encodings will also conserve space on storage 107, thereby allowing more space for additional pre-encoded content 109, if desired.
[0042] Pre-encoding 304 could encode some or all of the program, as appropriate.
Starting points are likely to be the highest priority for pre-encoding, since this is the content most likely to be needed. Start points could include the beginning of the program, the next few seconds immediately following a previous viewing that was paused or otherwise interrupted, as well as any likely "skip points" for future trick play. Convenient skip points may include the program points just after commercial breaks, the portions of the program that are just before the most interesting or exciting portions of the program (as determined by supplied excitement data or the like), or any other skip points that are determined in any other manner. Some implementations may further be able to recognize the first few seconds after a commercial break, scene changes, exciting or interesting moments, favorite or most-watched moments, or other portions of the program that are likely to become start points for viewers. Start or skip points may be automatically predicted by
[0040]As shown in FIG. 3, process 300 begins with placeshifting encoder 102 receiving content that may be of future placeshifting interest to a viewer (function 302). Such content may include programs stored in a DVR, for example, or other programs that are available from any media source 105. Programs may be manually selected for future placeshifting by the user in some embodiments, whereas other embodiments will automatically attempt to create pre-encoded start points for some or all of the programs available from a content source 106 on an a priori basis as time and resources allow. If an encoder is otherwise idle, for example, the placeshifting device 102 may make use of the idle time to pre-encode as much available content as possible.
[0041] Pre-encoding may be performed in any suitable manner (function 304). In various embodiments, pre-encoding involves converting at least some portions of available programs to a streaming format for quick playback. To that end, it may be desirable to pre-encode at least the startup content at a relatively low quality to facilitate rapid transit of network no, even via limited bandwidth. Lower quality encodings will also conserve space on storage 107, thereby allowing more space for additional pre-encoded content 109, if desired.
[0042] Pre-encoding 304 could encode some or all of the program, as appropriate.
Starting points are likely to be the highest priority for pre-encoding, since this is the content most likely to be needed. Start points could include the beginning of the program, the next few seconds immediately following a previous viewing that was paused or otherwise interrupted, as well as any likely "skip points" for future trick play. Convenient skip points may include the program points just after commercial breaks, the portions of the program that are just before the most interesting or exciting portions of the program (as determined by supplied excitement data or the like), or any other skip points that are determined in any other manner. Some implementations may further be able to recognize the first few seconds after a commercial break, scene changes, exciting or interesting moments, favorite or most-watched moments, or other portions of the program that are likely to become start points for viewers. Start or skip points may be automatically predicted by
- 13 -encoder 102, in some embodiments, based upon excitement data, viewer information, collected information about particular programs (e.g., viewer behavior during past broadcasts of the same program, crowd-sourced data, data received from other television receivers, etc.), and/or data from any other source.
Other embodiments could receive out-of-band data (e.g., data from an electronic program guide or from a separate service 112) that instructs the device 102 as to which points of the particular program should be pre-encoded for best efficiency and effectiveness.
[0043] As noted above, some embodiments may pre-encode the entire program content if sufficient time, processing capability and storage capacity is available.
This would allow maximum flexibility in providing needed startup content regardless of the user's skipping, scanning or other trick play actions. Pre-encoding 304 may take place prior to placeshifting at any time. After a program is recorded in a DVR, for example, various embodiments could pre-encode some or all of the program content so that it is ready for subsequent placeshifting. Pre-encoding may be scheduled according to resource availability, and may be limited by available storage or other resources.
[0044] If sufficient resources are available, pre-encoding 304 of the future stream of content could take place even after a placeshifting session has begun. That is, if a viewer begins placeshifting a media stream 122, a separate encoder 202 could preencode the later portions of the program (e.g., portions that have not yet been live encoded) while a primary encoder 202 handles the live encoding of the current stream. Since the pre-encoding could be at a lower quality, encoding may occur faster than real time (assuming sufficient source content is available), thereby allowing pre-encoded content to be made available ahead of time even if the viewer skips ahead in the program stream. The primary encoder could catch up as needed to provide better quality through live encoding based upon the current conditions of the network no.
[0045] When the media player 104 requests content for placeshifting (function 306), the placeshifting device 102 suitably provides the pre-encoded content 109 for immediate startup of the media stream (function 308). The media player 103 is able to immediately initiate playback (function 310) of the pre-encoded content 109 without waiting for the encoder 202 to produce a live encoded stream 314. When
Other embodiments could receive out-of-band data (e.g., data from an electronic program guide or from a separate service 112) that instructs the device 102 as to which points of the particular program should be pre-encoded for best efficiency and effectiveness.
[0043] As noted above, some embodiments may pre-encode the entire program content if sufficient time, processing capability and storage capacity is available.
This would allow maximum flexibility in providing needed startup content regardless of the user's skipping, scanning or other trick play actions. Pre-encoding 304 may take place prior to placeshifting at any time. After a program is recorded in a DVR, for example, various embodiments could pre-encode some or all of the program content so that it is ready for subsequent placeshifting. Pre-encoding may be scheduled according to resource availability, and may be limited by available storage or other resources.
[0044] If sufficient resources are available, pre-encoding 304 of the future stream of content could take place even after a placeshifting session has begun. That is, if a viewer begins placeshifting a media stream 122, a separate encoder 202 could preencode the later portions of the program (e.g., portions that have not yet been live encoded) while a primary encoder 202 handles the live encoding of the current stream. Since the pre-encoding could be at a lower quality, encoding may occur faster than real time (assuming sufficient source content is available), thereby allowing pre-encoded content to be made available ahead of time even if the viewer skips ahead in the program stream. The primary encoder could catch up as needed to provide better quality through live encoding based upon the current conditions of the network no.
[0045] When the media player 104 requests content for placeshifting (function 306), the placeshifting device 102 suitably provides the pre-encoded content 109 for immediate startup of the media stream (function 308). The media player 103 is able to immediately initiate playback (function 310) of the pre-encoded content 109 without waiting for the encoder 202 to produce a live encoded stream 314. When
- 14-the live encoded stream 314 is available, the media player 104 is able to switch from the lower-quality pre-encoded stream to the higher quality live-encoded stream, thereby improving the viewer experience.
[0046] Any additional pre-encoded content 109 cached by the placeshifting device in storage 107 or elsewhere may be delivered as time and resources are available. If sufficient bandwidth is present, pre-encoded content 109 from device 102 may be transferred to player 104 for caching in storage 117 until such time as the content is needed. This transfer can take place in parallel with the live stream, or the content could be transferred out-of-band even prior to the placeshifting session (e.g., during a LAN or cable based synch with the placeshifting device), as desired. Again, content representing some or all of the program can be pre-encoded and cached at the placeshifting device 102 and/or at the media player 104 at any time after the program is made available to the placeshifting device until the relevant content is requested for live encoding.
[0047] Live encoding and playback of the live stream can continue (function 316) until the user performs a trick play function such as a fast forward, rewind, skip, seek or the like (function 318). When the user performs a trick play function that results in a new playback point for the media stream, the media player 104 may check to see if it already has pre-encoded content 109 for that playback point (function 330). If cached content 109 is available, then such content can be retrieved from storage 117 and rendered for immediate playback, without waiting for the live-encoded stream.
[0048] If the new playback point is not available to the media player 104, then the player 104 will update the encoder 102 (function 319) so that live encoding can continue at the new playback point (function 322). Playback points in the media program may be referenced based upon MPEG presentation time stamps (PTS), or based upon any other temporal reference shared between the player 104 and placeshifting device 102 with respect to the streamed programming.
Placeshifting device 102 will then update the encoding point, and provide live encoding for the new program time in an updated stream 324.
[0049] In various embodiments, encoder 102 may be able to perform the live encoding of stored programs (e.g., programs from a DVR) at a rate that is faster than real time. Even though this encoding is still "live" in the sense that it is
[0046] Any additional pre-encoded content 109 cached by the placeshifting device in storage 107 or elsewhere may be delivered as time and resources are available. If sufficient bandwidth is present, pre-encoded content 109 from device 102 may be transferred to player 104 for caching in storage 117 until such time as the content is needed. This transfer can take place in parallel with the live stream, or the content could be transferred out-of-band even prior to the placeshifting session (e.g., during a LAN or cable based synch with the placeshifting device), as desired. Again, content representing some or all of the program can be pre-encoded and cached at the placeshifting device 102 and/or at the media player 104 at any time after the program is made available to the placeshifting device until the relevant content is requested for live encoding.
[0047] Live encoding and playback of the live stream can continue (function 316) until the user performs a trick play function such as a fast forward, rewind, skip, seek or the like (function 318). When the user performs a trick play function that results in a new playback point for the media stream, the media player 104 may check to see if it already has pre-encoded content 109 for that playback point (function 330). If cached content 109 is available, then such content can be retrieved from storage 117 and rendered for immediate playback, without waiting for the live-encoded stream.
[0048] If the new playback point is not available to the media player 104, then the player 104 will update the encoder 102 (function 319) so that live encoding can continue at the new playback point (function 322). Playback points in the media program may be referenced based upon MPEG presentation time stamps (PTS), or based upon any other temporal reference shared between the player 104 and placeshifting device 102 with respect to the streamed programming.
Placeshifting device 102 will then update the encoding point, and provide live encoding for the new program time in an updated stream 324.
[0049] In various embodiments, encoder 102 may be able to perform the live encoding of stored programs (e.g., programs from a DVR) at a rate that is faster than real time. Even though this encoding is still "live" in the sense that it is
- 15 -encoded based upon current network conditions, the faster encoding capabilities could allow some future program content to be live encoded and transmitted to the player 104 prior to the playback time. "Live" encoding in this context therefore refers to the time that the content is encoded, and not necessarily the playback time.
[0050] Player 104 will typically cache this live encoded content in storage 117 or the like for future playback. In such embodiments, player 104 may be able to select between live encoded future content and pre-encoded future content 109 even if the user skips ahead in a program. Typically the player 104 will choose the live encoded content over the pre-encoded content 109 due to the higher quality of most live encoded content, but other embodiments could be programmed to select between live and pre-encoded content in any manner desired.
[0051] According to various embodiments, then, a placeshifting device is able to reduce encoder latency by pre-encoding certain portions of the media stream on a priori basis, prior to the beginning of the placeshifting session. By providing pre-encoded content until live encoded content is available, the latency previously experienced during many placeshifting sessions can be significantly reduced, thereby greatly improving the user experience.
[0052] The term "exemplary" is used herein to represent one example, instance or illustration that may have any number of alternates. Any implementation described herein as exemplary is not necessarily to be construed as preferred or advantageous over other implementations. While several exemplary embodiments have been presented in the foregoing detailed description, it should be appreciated that a vast number of alternate but equivalent variations exist, and the examples presented herein are not intended to limit the scope, applicability, or configuration of the invention in any way. To the contrary, various changes may be made in the function and arrangement of elements described without departing from the scope of the claims and their legal equivalents.
[0050] Player 104 will typically cache this live encoded content in storage 117 or the like for future playback. In such embodiments, player 104 may be able to select between live encoded future content and pre-encoded future content 109 even if the user skips ahead in a program. Typically the player 104 will choose the live encoded content over the pre-encoded content 109 due to the higher quality of most live encoded content, but other embodiments could be programmed to select between live and pre-encoded content in any manner desired.
[0051] According to various embodiments, then, a placeshifting device is able to reduce encoder latency by pre-encoding certain portions of the media stream on a priori basis, prior to the beginning of the placeshifting session. By providing pre-encoded content until live encoded content is available, the latency previously experienced during many placeshifting sessions can be significantly reduced, thereby greatly improving the user experience.
[0052] The term "exemplary" is used herein to represent one example, instance or illustration that may have any number of alternates. Any implementation described herein as exemplary is not necessarily to be construed as preferred or advantageous over other implementations. While several exemplary embodiments have been presented in the foregoing detailed description, it should be appreciated that a vast number of alternate but equivalent variations exist, and the examples presented herein are not intended to limit the scope, applicability, or configuration of the invention in any way. To the contrary, various changes may be made in the function and arrangement of elements described without departing from the scope of the claims and their legal equivalents.
- 16 -
Claims (15)
1. An automated process performed by a video encoder device to provide a placeshifted video stream to a media player via a network, the process comprising:
receiving a media program by the video encoder device before receiving an instruction from the media player via the network to encode the media program, the video encoder device pre-encoding at least a portion of the media program to a placeshifting format;
in response to the video encoder device receiving the instruction from the media player to encode the media program, the video encoder device initially providing the pre-encoded portion of the media program to the media player to thereby allow the media player to begin immediate playback of the pre-encoded portion; and after providing the pre-encoded portion of the media program in response to the instruction, the video encoder device live encoding remaining portions of the media program according to then-current network conditions to thereby provide the placeshifted video stream to the media player.
receiving a media program by the video encoder device before receiving an instruction from the media player via the network to encode the media program, the video encoder device pre-encoding at least a portion of the media program to a placeshifting format;
in response to the video encoder device receiving the instruction from the media player to encode the media program, the video encoder device initially providing the pre-encoded portion of the media program to the media player to thereby allow the media player to begin immediate playback of the pre-encoded portion; and after providing the pre-encoded portion of the media program in response to the instruction, the video encoder device live encoding remaining portions of the media program according to then-current network conditions to thereby provide the placeshifted video stream to the media player.
2. The process of claim 1 wherein the pre-encoded portion comprises the first few seconds of the media program.
3. The process of claim 1 wherein the pre-encoded portion comprises the first few seconds of the media program following a commercial advertisement in the media program.
4. The process of claim 1 wherein the pre-encoded portion comprises the first few seconds of the media program following a stop point from previous viewing of the media program.
5. The process of claim 1 wherein the pre-encoding comprises encoding the entire media program in a low bandwidth format so that the first few seconds of the media program following any start point commanded by the media player are pre-encoded before the start point is commanded by the media player.
6. The process of claim 1 wherein the pre-encoded portion is encoded in a low bandwidth format in comparison to the remaining portions of the media stream.
7. The process of claim 1 wherein the video encoder device initially pre-encodes a plurality of portions of the media program, wherein each of the pre-encoded portions corresponds to a likely start point for playback of the media program.
8. A media encoder device comprising:
a television receiver interface configured to receive broadcasts of television programming;
a network interface to a digital network;
a digital storage configured to record and store at least some of the television programming received via the television receiver interface in a storage format; and a processor configured:
prior to receiving an instruction from a remote media player to encode at least some of the television programming for placeshifting, to automatically pre-encode portions of at least some of the television programming stored on the digital storage in a placeshifting format that is different from the storage format;
in response to subsequently receiving the instruction from the remote media player to encode the media program, initially providing the pre-encoded portion of the media program to the media player to thereby allow the media player to begin immediate playback of the pre-encoded portion; and after providing the pre-encoded portion of the media program in response to the instruction, live encoding remaining portions of the media program according to then-current conditions of the digital network to thereby provide the placeshifted video stream to the remotely-located media player via the digital network.
a television receiver interface configured to receive broadcasts of television programming;
a network interface to a digital network;
a digital storage configured to record and store at least some of the television programming received via the television receiver interface in a storage format; and a processor configured:
prior to receiving an instruction from a remote media player to encode at least some of the television programming for placeshifting, to automatically pre-encode portions of at least some of the television programming stored on the digital storage in a placeshifting format that is different from the storage format;
in response to subsequently receiving the instruction from the remote media player to encode the media program, initially providing the pre-encoded portion of the media program to the media player to thereby allow the media player to begin immediate playback of the pre-encoded portion; and after providing the pre-encoded portion of the media program in response to the instruction, live encoding remaining portions of the media program according to then-current conditions of the digital network to thereby provide the placeshifted video stream to the remotely-located media player via the digital network.
9. The media encoder device of claim 8 wherein the pre-encoded portion comprises the first few seconds of the media program.
10. The media encoder device of claim 8 wherein the pre-encoded portion comprises the first few seconds of the media program following a commercial advertisement in the media program.
11. The media encoder device of claim 8 wherein the pre-encoded portion comprises the first few seconds of the media program following a stop point from previous viewing of the media program.
12. The media encoder device of claim 8 wherein the entire media program is initially encoded in a low bandwidth format so that the first few seconds of the media program following any start point commanded by the media player are pre-encoded before the start point is commanded by the media player.
13. The media encoder device of claim 8 wherein the pre-encoded portion is encoded in a low bandwidth format in comparison to the remaining portions of the media stream.
14. The media encoder device of claim 8 wherein multiple portions of the media program are pre-encoded, wherein each of the pre-encoded portions corresponds to a likely start point for playback of the media program.
15. The media encoder device of claim 14 wherein the pre-encoded portions of the media program comprise the beginning of the program and the portions of the program immediately following commercial breaks in the media program.
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US201662441246P | 2016-12-31 | 2016-12-31 | |
| US62/441,246 | 2016-12-31 | ||
| PCT/US2017/068796 WO2019132938A1 (en) | 2017-12-28 | 2017-12-28 | Reducing latency for streaming video |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CA3050636A1 CA3050636A1 (en) | 2019-07-04 |
| CA3050636C true CA3050636C (en) | 2021-02-23 |
Family
ID=67477326
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA3050636A Active CA3050636C (en) | 2016-12-31 | 2017-12-28 | Reducing latency for streaming video |
Country Status (1)
| Country | Link |
|---|---|
| CA (1) | CA3050636C (en) |
-
2017
- 2017-12-28 CA CA3050636A patent/CA3050636C/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CA3050636A1 (en) | 2019-07-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10250664B2 (en) | Placeshifting live encoded video faster than real time | |
| US10826963B2 (en) | Reducing latency for streaming video | |
| US11166058B2 (en) | Systems and methods to provide trick play during streaming playback | |
| US11089373B2 (en) | Seek with thumbnail generation and display during placeshifting session | |
| KR101064762B1 (en) | Fast start-up for digital video streams | |
| US8837586B2 (en) | Bandwidth-friendly representation switching in adaptive streaming | |
| US8532472B2 (en) | Methods and apparatus for fast seeking within a media stream buffer | |
| US11128897B2 (en) | Method for initiating a transmission of a streaming content delivered to a client device and access point for implementing this method | |
| JP2008523688A (en) | Hierarchical digital video encoding system and method in a digital video recorder | |
| US20170127101A1 (en) | Reducing startup latency in a video place-shifting system | |
| WO2018005835A1 (en) | Systems and methods for fast channel change | |
| US9705727B2 (en) | Remote viewing of media content using layered video encoding | |
| CA3050636C (en) | Reducing latency for streaming video | |
| EP3732888B1 (en) | Reducing latency for streaming video | |
| US11778008B2 (en) | Method for managing adaptive progressive downloading of digital content which is broadcast in real time | |
| US11714850B2 (en) | Method and apparatus for thumbnail generation for a video device | |
| US11647063B2 (en) | Method and apparatus for presentation of video content | |
| US20140189766A1 (en) | Methods and apparatus for seeking within recorded media stored on a set-top box | |
| KR100796777B1 (en) | Method for the delivery of multimedia file compressed by differential pulse code modulation using ftp file transter |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request |
Effective date: 20190717 |