EP4256789A1 - Systems and methods for dynamically syncing from time-shifted frame to live stream of content - Google Patents
Systems and methods for dynamically syncing from time-shifted frame to live stream of contentInfo
- Publication number
- EP4256789A1 EP4256789A1 EP21839738.8A EP21839738A EP4256789A1 EP 4256789 A1 EP4256789 A1 EP 4256789A1 EP 21839738 A EP21839738 A EP 21839738A EP 4256789 A1 EP4256789 A1 EP 4256789A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- content item
- segments
- segment
- manifest
- subset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 131
- 230000004044 response Effects 0.000 claims abstract description 72
- 230000008569 process Effects 0.000 description 39
- 238000004891 communication Methods 0.000 description 28
- 238000013459 approach Methods 0.000 description 17
- 238000012545 processing Methods 0.000 description 17
- 238000003860 storage Methods 0.000 description 16
- 230000009471 action Effects 0.000 description 10
- 238000010801 machine learning Methods 0.000 description 8
- 230000002950 deficient Effects 0.000 description 5
- 238000009826 distribution Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000002411 adverse Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 230000003111 delayed effect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- PGLIUCLTXOYQMV-UHFFFAOYSA-N Cetirizine hydrochloride Chemical compound Cl.Cl.C1CN(CCOCC(=O)O)CCN1C(C=1C=CC(Cl)=CC=1)C1=CC=CC=C1 PGLIUCLTXOYQMV-UHFFFAOYSA-N 0.000 description 1
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000003139 buffering effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000005266 casting Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000002716 delivery method Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 229920001296 polysiloxane Polymers 0.000 description 1
- 238000003825 pressing Methods 0.000 description 1
- 238000010079 rubber tapping Methods 0.000 description 1
- 229910052710 silicon Inorganic materials 0.000 description 1
- 239000010703 silicon Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/262—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists
- H04N21/26258—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists for generating a list of items to be played back in a given order, e.g. playlist, or scheduling item distribution according to such list
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
- H04N21/4331—Caching operations, e.g. of an advertisement for later insertion during playback
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
- H04N21/47217—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
- H04N21/845—Structuring of content, e.g. decomposing content into time segments
- H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/85—Assembly of content; Generation of multimedia applications
- H04N21/854—Content authoring
- H04N21/8549—Creating video summaries, e.g. movie trailer
Definitions
- This disclosure is generally directed to dynamically synchronizing a presentation of a time-shifted frame of a content item to a live stream of the content item.
- methods and systems are provided for transitioning to the latest frame of the live stream of the content item by intelligently skipping a subset of time-shifted frames.
- a presentation of the content item on a user device may lag behind the live streaming of the content item, causing a noticeable latency of the presentation between what should have been presented and what is currently presented.
- the lag may occur due to an unstable or slow network connection or low computer performance.
- the lag may occur due to a trick play (e.g., rewind or pause) requested by a user.
- the system will simply keep playing the content item behind the live streaming without ever catching up.
- This approach is deficient because the user may be exposed to a spoiler via a different content delivery method.
- the user may receive a notification showing an up-to-date soccer game score on a mobile phone while watching a delayed soccer game on a laptop.
- the system will allow the user to instantly catch up to live (e.g., by pressing a “live” button).
- This approach is also deficient because the instant catch-up will entail simply not showing a large number of frames (e.g., if the lag was 2 minutes, then 2 minutes of the content may be not shown), which will cause the user to miss important scenes that are critical to the content item and end up not having enough context to fully understand the content.
- a streaming application determines a skippable bit value for a missed segment of the content item.
- the streaming application may determine skippable bit value from a manifest of the content item, which includes segment information and skippable bit value information for each segment.
- the skippable bit value indicates whether a presentation of a respective segment or a frame can be skipped to catch up to live.
- a skippable bit value includes an enabled bit value
- the streaming application skips a presentation of a respective segment. If a skippable bit value includes a disabled bit value, then the streaming application does not skip a presentation of a respective segment and plays the respective segment because it is considered an important segment to understand the context of the content item. If a respective segment includes a disabled skippable bit value, then the data of the respective segment may be retrieved from a respective network address, and the respective segment may be played based on the data retrieved from the respective network address.
- the present disclosure addresses the problems of both a) never catching up or b) instantly catching up. It may do so by, for example, identifying important portions of the missed segments (e.g., by disabling the skippable bit for those portions) and determining which segments to skip to eventually catch up to the live stream, thereby enabling the user to consume live video yet still presenting enough content of the missed segments needed to understand the context of the content item.
- the present disclosure further addresses the problems described above by, for example, saving the network bandwidth and reducing network traffic by not sending less important segments, for the purpose of catching up to live.
- the streaming application may display an option for the user to skip all the missed segments to go live instantly.
- the streaming application may not present any options.
- the present disclosure addresses the problems of both a) never catching up to live or b) catching up instantly, by generating for display a user-selectable option to choose a time period during which to view the missed portion of the content item or to specify a desired amount of content to be presented to the user.
- a streaming application may present user- selectable options.
- the example selectable options include an option for a user to select a particular time period, which specifies how long the user wants the missed segments to be presented before catching up to live. Selecting the particular time period (e.g., two minutes) allows the user to skip only a certain portion of the missed segments and play a user- designated amount of the missed segments, such that the playing of the content item catches up to live in the selected particular time period.
- the system when playing of the content item lags behind the live streaming, the system takes various approaches to address the lagging problem.
- the system caches all the remaining segments of the content item from a server of a content item source to store the segments in a cache. This approach may be deficient, because it may take up too much memory to store all the cached segments, some of which may never even be played.
- the streaming application may request segments of the content item from the server of the content item source directly in real time for data transmission without caching any of the segments.
- This approach may also be deficient, because it may strain the network in real time. Requesting segments of the content item in real time may lead to a more severe lag due to the network bandwidth consumption for transmitting a large amount of data during the period of the lag.
- the present disclosure addresses the problems described above by storing an appropriate number of time-shifted segments (e.g., missed segments due to a delay) in a way that does not overuse the network resources.
- methods and systems are described herein for storing a portion of segments of a content item in the cache and playing the portion of the segments from the cache.
- the streaming application identifies a duration of the lag (e.g., two minutes) from a manifest of the content.
- the manifest may include segment information of the content item and priority level value information for each segment (or a frame).
- a priority level value may establish a relative position of a respective segment or frame with other segments or frames in determining whether the respective segment or frame needs to be stored in the cache for later presentation.
- the streaming application determines a priority threshold (e.g., five).
- a priority threshold may be used to determine the number of segments to store in the cache to provide a seamless viewing experience with the content item and still to catch up to live. For example, based on the duration of the lag, the streaming application determines a set of segments of the content item that fits within the lag (e.g., three segments within the two-minute lag).
- a subset of the segments with a priority level above the priority threshold is determined (e.g., segment 2 with a priority level of seven is identified because its priority level is higher than the priority threshold of five).
- the streaming application stores a respective identified segment with a higher priority level than the priority threshold in the cache for later viewing.
- FIG. 1 depicts an exemplary user interface for providing a content item that has fallen behind live streaming of the content item, in accordance with some embodiments of the disclosure
- FIG. 2 depicts an exemplary user interface for providing user-selectable catching-up options, in accordance with some embodiments of the disclosure
- FIG. 3 depicts an exemplary user interface for catching up to live stream of the content item, in accordance with some embodiments of the disclosure
- FIG. 4 depicts an exemplary user interface for presenting a live stream of the content item, in accordance with some embodiments of the disclosure
- FIG. 5 shows an exemplary embodiment for determining, by a client device, whether to skip a presentation of a missed segment based on a manifest of the content item, in accordance with some embodiments of the disclosure
- FIG. 6 shows an exemplary embodiment for determining, by the client device, whether to skip a presentation of the missed segment based on metadata of the missed segment that is received upon requesting the content item, in accordance with some embodiments of the disclosure
- FIG. 7 shows an exemplary embodiment for determining, by the client device, whether to skip a presentation of the missed segment based on metadata of each missed segment that is received upon requesting a respective missed segment, in accordance with some embodiments of the disclosure
- FIG. 8 shows an exemplary embodiment for determining, by a server, whether to skip a presentation of the missed segment based on metadata of the missed segment, in accordance with some embodiments of the disclosure
- FIG. 9 shows an exemplary embodiment for storing the missed segment in the cache based on the manifest, in accordance with some embodiments of the disclosure.
- FIG. 10 shows an exemplary embodiment for storing the missed segment in the cache based on the received segment metadata, in accordance with some embodiments of the disclosure
- FIG. 11 shows an illustrative process for catching up to live based on a user selection, in accordance with some embodiments of the disclosure
- FIG. 12 shows an illustrative process for catching up to live based on a skippable bit value, in accordance with some embodiments of the disclosure
- FIG. 13 shows an illustrative process for caching a time-shifted segment to catch up to live, in accordance with some embodiments of the disclosure
- FIG. 14 shows a block diagram of an illustrative user device, in accordance with some embodiments of the disclosure.
- FIG. 15 shows a block diagram of an illustrative media system, in accordance with some embodiments of the disclosure.
- FIG. 1 shows an example content instantiation 100 that shows a content item that has fallen behind live streaming of the content item, in accordance with some embodiments of the disclosure.
- FIGS. 1-4 depict user device 102, 202, 302, 402 as a television, it will be understood that any suitable device for displaying a live stream may be used, such as user television or gaming equipment 1502, user computer equipment 1504, or wireless user communications device 1506 discussed below in relation to FIG. 15.
- the user device 102, 202, 302, 402 may have control circuitry 1404 of FIG. 14 configured to request the live stream of the content item from a server for display (e.g., on a display such as display 1412 discussed later in more detail with reference to FIG. 14).
- FIG. 1 depict improved user interface that includes exemplary content instantiation, which illustrate catching-up operations.
- user device 102 requests an audio-visual media asset that includes a group of segments. It will be understood that, consistent with the present disclosure, any content item may be requested for streaming from the media server.
- the term “content item” should be understood to mean an electronically consumable user asset, such as an electronic version of a printed book, electronic television programming, as well as pay-per-view programs, on-demand programs (as in video-on- demand (VOD) systems), Internet content (e.g., streaming content, downloadable content, Webcasts, etc.), video clips, audio, content information, pictures, rotating images, documents, playlists, websites, articles, books, articles, newspapers, blogs, advertisements, chat sessions, social media, applications, games, and/or any other media or multimedia and/or combination of the same.
- VOD video-on- demand
- a segment 104 of a content item is streamed on user device 102 via the streaming application.
- the streaming application may be a standalone application implemented on user equipment device 1400.
- the streaming application may generate a live status indicator 101 for display, which indicates whether a segment that is currently presented on user device 102 is a real time stream (i.e., live segment).
- live status indicator 101 indicates that segment 104 is not a live stream (by showing “Not Live”).
- live status indicator 101 may be a word “live” that appears grayed out or otherwise altered.
- the streaming application may generate for display a progress bar 108, which depicts the progress of the viewing history of the content item.
- Progress bar 108 may include a current position bar 110, which indicates a current segment location in the duration of the content item and a live position bar 112 which indicates a live segment location in the duration of the content item.
- segment 104 currently lags behind the live stream of the content item as the current position bar 110 and the live position bar 112 are not aligned together (current position bar 110 is located on the left side of live position bar 112).
- FIG. 2 depicts an exemplary user interface 200 for providing user-selectable catching-up options on a user device 202, in accordance with some embodiments of the disclosure.
- the streaming application may display the user selectable options 204, 206, 210, which can be selected via user input.
- the streaming application displays user selectable options 204, 206, 210 only when the lag is enough (e.g., when it exceeds a certain threshold, for example, when it’s longer than 1 minute or longer than 5 minutes).
- Selecting of each user-selectable option triggers the streaming application to perform different catching-up actions.
- User selection may be made by any suitable type of user interface input method, such as clicking, tapping, or voice input as described in 1410 of FIG. 14.
- the streaming application skips all of the missed segments of the content item and catches up to live instantly. As a result, the user will view the most up-to-date segment of the content item in real time.
- the streaming application may keep playing the content item without ever caching up to live. As a result, the user will permanently remain behind the live streaming unless a further action is taken.
- the streaming application may display a drop-down menu that offers a list of sub-options.
- the drop-down menu may be pre-populated with a list of values.
- the drop-down menu can be optional (e.g., it may not be displayed unless requested by the user).
- the sub-options may include a list of numbers (e.g., time periods) that the user can choose from.
- the user can enter any value that is not displayed in the list.
- the particular time period may be a time basis for catching up on the missed segments. For example, if the user selects “1 min,” then the streaming application will catch up the presentation of the content item to the live stream within 1 minute, regardless of the duration of the lag.
- the drop-down menu includes a system-configured default value.
- the streaming application determines the default value based on the user’s past history or patterns for catching up to live. For example, the default value may be determined based on a user profile or a type of a content item (e.g., action v. romance). The default value may be determined based on a time of day that content item is played. The default value may be determined based on network connectivity and hardware condition. These are exemplary factors that may be used to determine the default value, but any suitable factors may be used to determine the default value.
- the streaming application catches up to live in the time period corresponding to the default value.
- FIG. 3 depicts an exemplary user interface 300 for catching up to the live stream of the content item on a user device 302, in accordance with some embodiments of the disclosure.
- the content item Upon receiving user input specifying a value for the particular time period of option 206, the content item will be caught up to live in the particular time period (e.g., 2 minutes).
- the streaming application displays a live status bar 301 which indicates “Catching up to live” is in progress.
- the “Catching up to live” progress can also be shown by a progress bar 308.
- a current position bar 310 is closer to a live position bar 312 in progress bar 308 compared to current position bar 110 in content instantiation 100.
- FIG. 4 depicts an exemplary user interface 400 for presenting a live frame 404 of the content item on a user device 402, in accordance with some embodiments of the disclosure.
- Content instantiation 400 may be a resulting view of the user selecting “Catch up in 2 min” option 208 at 3 :50pm in FIG.2.
- content item is catching up to live at 3 :51 pm in FIG. 3 (showing a frame 304 where a soccer player is shooting a goal) and is finally caught up to live at 3:52 pm in FIG. 4 (showing live frame 404 where the soccer player is scoring a goal).
- the streaming application changes live status bar 401 from “Catching Up to Live” to “Live.”
- the live status bar 401 includes a count-up or count-down countdown timer.
- the countdown timer indicates how much time is left or passed to catch up to live (e.g., by showing “in 3, 2, 1 seconds, live!)
- the completion of catching up is also shown by a progress bar 408.
- progress bar 408 of FIG. 4 a current position bar 410 and a live position bar 412 are aligned, indicating that content item is caught up to live. Catching-up operations will be explained in connection with FIGS. 5-8.
- FIG. 5 shows an exemplary embodiment 500 for determining, by a client device, whether to skip a presentation of a missed segment based on a manifest of the content item, in accordance with some embodiments of the disclosure.
- the client device can be any user television or gaming equipment 1502, user computer equipment 1504, or wireless user communications device 1506 discussed below in relation to FIG. 15.
- embodiments 500-1000 may be performed by control circuitry 1404 of FIG. 14 as instructed by the streaming application, which may be performed on any client device.
- one or more steps of embodiments 500-1000 may be incorporated into or combined with one or more steps of any other processes or embodiments described in the present disclosure.
- the user may request a presentation of a content item (e.g., an episode of “Game of Thrones”) via the streaming application on the client device.
- the content item consists of one or more segments.
- Each segment consists of one or more frames, where the length of each segment ranges approximately from zero to a few seconds (e.g., 0-6 seconds).
- a frame is one of the many still images that compose the content item.
- the manifest may be an MPEG-DASH manifest.
- the streaming application requests a live streaming of a content item to a media server.
- the media server may reside in a content item data source 1518 of FIG. 15.
- the media server sends a manifest of the content item.
- the manifest contains information about sequential segments of the content item, for example, timing of each segment, location on the network such as a network address from where each segment can be retrieved, or a skippable bit value for each segment.
- a time period for segment 1 is from 0:00 to 0:03
- a network address for retrieving segment 1 is address 1
- a skippable bit value for segment 1 is “False.”
- the skippable bit value indicates whether a presentation of a respective segment may be skipped to catch up to live. For the delayed content item to catch up to live stream, some segments of the content item need to be skipped due to the time gap.
- the skippable bit value indicates which of the missed segments may be skipped to catch up the time gap. If a skippable bit value includes an enabled bit value, then the streaming application skips a presentation of a respective segment (only when the content item lags behind the live). If a skippable bit value includes a disabled bit value, then the streaming application does not skip a presentation of a respective segment, because it is considered as an important segment to understand the context of the content item.
- the data of the respective segment may be retrieved from a respective network address, and the respective segment may be played based on the data retrieved from the respective network address.
- the media server may determine a skippable bit value based on an importance level of a segment.
- the media server determines an importance level of the segment based on various factors, including a number of scheduled replays of the segment in a predetermined time period (e.g., sports highlights which are scheduled to be played three times this week), a type of content item (e.g., movie v. live sports game), or a genre of the segment (e.g., action v. romance).
- a machine learning algorithm is used to identify skippable segments. For example, a training set including segments that were commonly skipped by a large number of users may be compiled. The machine learning algorithm may then take a new segment as an input and compare it to known skipped segments. When the similarity is high, the machine learning algorithm may identify the new segment as a skippable segment and a skippable bit value may be enabled. The machine learning algorithm may train a set of data based on the user’s preferences or past history. This is an example method of determining a skippable bit value, and any suitable methods may be used to determine the skippable bit value for the segment. [0047] The skippable bit information for each segment may be inserted into the manifest or metadata of the segment by the media server either enabling or disabling the skippable bit value. The manifest or the metadata of the segment may be stored in a database of the media server.
- the skippable bit includes one of the two values, such as an enabled bit value (e.g., 1) or disabled bit value (e.g., 0).
- An enabled skippable bit indicates that a presentation of a respective segment may be skipped.
- a disabled skippable bit indicates that a presentation of a respective segment may not be skipped.
- the skippable bit value includes a range of numbers. If a respective segment has a skippable bit value that is higher than an importance threshold, then the respective segment is determined to have a disabled bit value. On the other hand, if a respective segment has a skippable bit value that is lower than the importance threshold, then the respective segment is determined to have an enabled bit value.
- the importance threshold may be determined by the media server or the client device.
- the importance threshold may be determined based on the user profile or the content item in a similar technique and manner described above. For example, if the content item is an action movie, which the user indicated to be the favorite genre, then the importance threshold for the content item is determined to be higher than a romance movie about which the user cares less.
- the streaming application upon receiving the manifest, which includes network address information for segment 1, retrieves segment 1 of the content item from network address 1 and plays the retrieved segment 1 on the client device.
- the streaming application determines whether playing of the content item lags behind the live streaming.
- One method of detecting the lag is described with respect to FIG. 11.
- one or more steps of process 1100 described in FIG. 11 or embodiments 100-400 of FIGS. 1- 4 may be incorporated into or combined with step 510. For example, upon the user selecting “catch up to live in X min” option 206 in FIG. 2, the streaming application performs the steps described in FIGS. 5-10.
- the streaming application upon detecting the lag between the currently displayed segment and the live segment, the streaming application evaluates the next segment that follows segment 1 (segment 2) and reads a skippable bit value from the manifest. As shown in the transmitted manifest at step 506, segment 2 has an enabled skippable bit, indicating that a presentation of segment 2 can be skipped. Therefore, the streaming application skips retrieving of segment 2 from address 2 because the presentation of segment 2 is not necessary during the catching up to live operation.
- the streaming application reads a skippable bit value for a segment that follows segment 2: segment 3, at step 514.
- the skippable bit value for segment 3 is “false,” a disabled bit value, indicating that a presentation of segment 3 cannot be skipped. Therefore, the streaming application retrieves data for the segment 3 from network address 3 and plays segment 3 from the network address 3.
- the streaming application determines whether the content item is caught up to live. If the streaming application determines that the content item is caught up to live, then the streaming application plays segment 4 regardless of the skippable bit value. Since there is no latency anymore, the streaming application does not skip any segment of the content item. If the streaming application determines that the content item is not caught up to live, then the streaming application repeats the steps in FIG. 5 until the content item is caught up to live.
- FIG. 6 shows an exemplary embodiment 600 for determining, by the client device, whether to skip a presentation of the missed frame based on metadata of missed frame that is received upon requesting the content item, in accordance with some embodiments of the disclosure.
- the content item consists of one or more segments. Each segment consists of one or more frames, where the length of a frame ranges approximately from zero to 16.33 milliseconds. A frame is one of the many still images that compose the content item.
- Embodiment 600 is different from embodiment 500 in that the streaming application does not receive the manifest of the content item. Instead, the streaming application receives frame data and metadata upon requesting the live streaming of the content item.
- the frame data may include content data for the frame, such as digital pixel values.
- the metadata contains information about the skippable bit for each metadata, which indicates whether a presentation of a respective metadata may be skipped and a sequential location data of a frame in the content item.
- the user may request a presentation of the content item.
- the streaming application requests live streaming of the content item to the media server.
- the media server sends frame data and metadata of the content item.
- the frame data and metadata may be a complete packet with all the content data necessary to play the content item from the beginning to the end.
- the frame data and the metadata may be stored in a database of the media server, which may reside in content item data source 1518.
- the streaming application plays frame 1 using the received frame 1 data.
- the streaming application determines whether a lag is detected. In some embodiments, upon detecting the lag, the streaming application may generate the user- selectable “catching-up” options for display, as described in FIG. 2.
- the streaming application determines that a presentation of frame 2 may be skipped because frame 2 has an enabled skippable bit (e.g., or when the user selects “catch up to live in X min” option 206 of FIG. 2). Thus, the streaming application skips the presentation of frame 2.
- the streaming application identifies a next frame of the content item that follows frame 2 and performs similar steps. For frame 3, the streaming application determines that a presentation of frame 3 cannot be skipped because frame 3 has a disabled skippable bit value. Thus, the streaming application plays frame 3 using the received frame 3 data.
- the streaming application determines whether the content item is caught up to live. If the content item is caught up to live, then the streaming application plays frame 4 regardless of a skippable bit value for frame 4. If the content item is not caught up to live, then the streaming application repeats similar steps until the content item is live.
- FIG. 7 shows an exemplary embodiment 700 for determining, by the client device, whether to skip a presentation of the missed segments based on metadata that is received upon requesting a respective missed segment, in accordance with some embodiments of the disclosure.
- Embodiment 700 is different from embodiment 600 in that the streaming application does not receive the complete packet including all of the frame data and the metadata at once but receives individual frame data and metadata each time when the streaming application requests a respective frame from the media server.
- the user may request a presentation of the content item.
- the streaming application requests live streaming of the content item from the media server.
- the streaming application sends only frame 1 data and metadata, instead of sending all the frame data and metadata of the content item.
- the metadata contains information about a skippable bit value for each frame.
- the streaming application obtains frame 1 data from the media server and plays frame 1 using the received frame 1 data.
- the streaming application determines whether a lag exists between the live frame and the currently displayed frame on the client device. In one embodiment, step 710 may be optional, and the streaming application may perform step 712 without performing step 710. [0063] At step 712, the streaming application requests frame 2 data and metadata from the media server. In one embodiment, the streaming application may only request frame 2 data without requesting metadata of frame 2 if a lag is not detected at step 710. Upon receiving the request for frame 2 from the streaming application, the media server sends frame 2 data and metadata to the streaming application.
- the streaming application receives frame 2 data and metadata from the media server. Based on the metadata, the streaming application determines that frame 2 has an enabled skippable bit (“true” skippable bit). The streaming application may skip a presentation of segment 2 based on the enabled skippable bit value. In one embodiment, the streaming application performs step 710 after receiving frame 2 data from the media server. Step 710 may be performed each time frame data is received at the streaming application to determine whether a respective received frame may be skipped to catch up to live.
- the streaming application identifies the next frame (frame 3) and requests frame 3 data and metadata from the media server.
- the streaming application receives requested data from the media server and reads the skippable bit value for frame 3.
- frame 3 has a disabled skippable bit (“false” skippable bit), and based on this, the streaming application determines that a presentation of frame 3 cannot be skipped.
- the streaming application plays frame 3 using the received frame data. The steps in FIG. 7 may be repeated until the content item is caught up to live.
- FIG. 8 shows an exemplary embodiment 800 for determining, by a server, whether to skip a presentation of the missed segment based on metadata, in accordance with some embodiments of the disclosure.
- Embodiment 800 is different from embodiment 700 in that embodiment 800 is performed by the media server that resides in the content item data source 1518 of FIG. 15, compared to embodiment 700, which was performed by the client device.
- the media server determines which of the missed frames to be sent to the streaming application.
- the user may request a presentation of the content item.
- the streaming application requests live streaming of the content item to the media server.
- the media server sends frame 1 data and metadata.
- the respective metadata contains skippable bit information for corresponding frames.
- the streaming application receives frame 1 data and metadata from the media server and streams frame 1 with the received frame 1 data.
- the streaming application detects a lag between the currently displayed frame and the live frame.
- the streaming application sends an indication to the media server that there is a lag between the currently displayed frame and the live frame.
- the media server queries the database of the media server and determines that frame 2 has an enabled skippable bit (“true” skippable bit value).
- the media server determines that segment 2 can be skipped for presentation and skips sending frame 2 data to the streaming application.
- the media server identifies a subsequent frame to frame 2 and determines that frame 3 has a disabled skippable bit (“false” skippable bit). Upon determining that frame 3 has the disabled skippable bit, the media server sends frame 3 data and metadata to the streaming application at step 820. In some embodiments, the media server only sends frame 3 data without the metadata for frame 3. Sending the metadata may be optional.
- the streaming application receives frame 3 data and plays frame 3 using the received frame 3 data.
- the process of FIG. 8 may continue until the content item is caught up to live.
- FIG. 9 shows an exemplary embodiment 900 for storing missed segments in the cache based on the manifest, in accordance with some embodiments of the disclosure.
- the user may request a presentation of the content item.
- the streaming application requests live streaming of the content item to the media server.
- the media server sends the manifest of the content item.
- the manifest contains information about the content item, such as a time duration of each segment, a priority level value for each segment, or a network address for each segment.
- the time duration of each segment may be associated with a respective timestamp of the segment, indicating the timing of the segment in the duration of the content item.
- a priority level value may establish a relative position of a respective segment with other segments in determining whether the respective segment needs to be stored in the cache for a smooth presentation of the content item.
- the media server assigns a priority level to each segment based on the following factors.
- the media server assigns a priority level to each segment of the content item based on a respective relevance level of the segment. For example, the media server determines a respective relevance level of a segment to the user based on the user profile data, such that a segment of a higher relevance (e.g., matching user’s preference) is determined to be more relevant to the user than other scenes.
- the media server assigns a higher priority level value to the particular segment than a segment that is determined to be of lower relevance to the user (e.g., romance scene). In some embodiments, the media server assigns a priority level to each frame based on the similar factors.
- the media server assigns a priority level to each segment of the content item based on a respective importance level of the segment in the context of the content item. For example, the media server determines a respective importance level of a segment, such that the segment includes scenes that are more important to the content itself than other scenes of the content (e.g., climax scene v. ending credits). The media server assigns a higher priority level value to a particular segment that has a higher importance level (e.g., climax) than a segment that has a lower importance level (e.g., ending credits).
- a higher importance level e.g., climax
- ending credits e.g., ending credits
- the media server assigns a priority level to each segment of the content items based on a number of replays of the segment in a predetermined time period. For example, the media server calculates the number of times the segment is streamed or scheduled to be streamed in a particular time period. As an example, replays of a previously streamed segment affect the priority level of a respective segment adversely because the user is likely to have been exposed to the particular segment already, and the particular segment may not need to be presented to the user again.
- a machine learning algorithm is used to identify a priority level for a segment. For example, a training set including segments with predetermined priority levels may be compiled. The machine learning algorithm may then take a new segment as an input and compare it to known segments. When the similarity is high, the machine learning algorithm may assign to a new segment a priority level similar to that of the known segments. The machine learning algorithm may train a set of data based on the user’s preferences or past history. This is an exemplary way of determining a priority level, and any suitable methods may be used to determine the priority level for the segment.
- the client device assigns a priority level to each segment or a frame of the content item using the similar techniques described above.
- the client device stores the priority level for each segment or frame in a database or storage of the client device.
- the client device may determine which segment or a frame needs to be cached and make a request to the media server data for a corresponding segment or a frame.
- the streaming application plays segment 1 from network address 1.
- the streaming application detects a lag between the currently played segment and the live segment.
- one or more steps of process 1100 described in FIG. 11 or embodiments 100-400 of FIGS. 1-4 may be incorporated into or combined with step 910.
- the streaming application may generate the user-selectable “catching-up” options for display, as described in FIG. 2.
- the streaming application identifies the duration of the lag. For example, the streaming application determines the duration of the lag based on the timestamp associated with each segment in the manifest. In embodiment 900, the duration of the lag is determined to be from 0:03 to 0: 12, as shown in FIG. 9.
- the streaming application selects a priority threshold based on the duration of the lag.
- the priority threshold may be proportional to the duration of the lag.
- the priority threshold may be dynamically adjusted based on the user profile data and content item, based on the techniques similar to determining the importance threshold.
- the streaming application identifies a set of segments that is within the duration of the lag.
- the streaming application identifies, in this example, three segments that match the length of the lag, segment 2 (0:03-0:06), segment 3 (0:06-0:09), segment 4 (0:09-0:012).
- the streaming application identifies a segment that has a priority level that is higher than a priority threshold.
- the streaming application determines that segment 3 has a priority level of seven, which is higher than the threshold of five.
- the streaming application retrieves segment 3 from network address 3 based on the manifest.
- the streaming application may not retrieve segment 2 data and segment 4 data as a presentation of segments 2 and 4 may be skipped based on the priority threshold.
- the streaming application stores the retrieved segment 3 in a cache of the client device for later presentation.
- a cache device may be implemented on non- transitory or transitory hardware of the client device.
- the streaming application plays segment 3 from the cache. The steps in FIG. 9 may be repeated until the content item catches up to live.
- FIG. 10 shows an exemplary embodiment 1000 for storing the missed segments in the cache based on the received segment metadata, in accordance with some embodiments of the disclosure.
- Embodiment 1000 is different from embodiment 900 in that the media server does not send the manifest to the streaming application, but the media server sends segment data to the streaming application when a request for a segment is received. Furthermore, the media server determines which of the missed segments of the content item need be sent to the streaming application and sends the determined segment data to the streaming application. Even though embodiment 1000 is described in the form of a segment, a frame may be used.
- the user may request a presentation of the content item.
- the streaming application requests live streaming of the content item to the media server.
- the media server sends segment 1 of the content item to the streaming application.
- the streaming application plays segment 1 from network address 1.
- the streaming application determines that playing of the content item lags behind the live streaming.
- the streaming application identifies the duration of the lag.
- the streaming application sends an indication to the media server that playing of the content item lags behind the live streaming.
- the media server determines a threshold based on the duration of the lag.
- the media server queries the database of the media server and determines that segment 2 has a priority level (e.g., 4) that is below a threshold.
- the media server upon determining that segment 2 has the priority level that is below the priority threshold, the media server skips sending segment 2 data to the streaming application.
- the media server identifies a segment of the content item that follows segment 2 and queries the database of the media server for the skippable bit value for segment 3.
- the media server determines that segment 3 has a priority level (e.g., 7) that is higher than the priority threshold (e.g., 5).
- the media server upon determining that segment 3 has the priority level higher than the priority threshold, the media server sends segment 3 data to the streaming application.
- the streaming application receives segment 3 data from the media server and stores segment 3 data in the cache.
- the streaming application plays segment 3 from the cache. The process 1000 may continue until the content item is caught up to live.
- process 1100 for catching up to live based on a user selection, in accordance with some embodiments of the disclosure. It should be noted that process 1100 may be performed by control circuitry 1404 of FIG. 14 as instructed by a streaming application that maybe performed on any devices 1502, 1504, 1506 of FIG. 15. In addition, one or more steps of process 1100 may be incorporated into or combined with one or more steps of any other process or embodiment (e.g., process 1200 of FIG. 12 and process 1300 of FIG. 13). The process 1100 may be performed in accordance with the embodiments discussed in connection with FIGS. 1-4.
- the user requests the live stream of the content item via the streaming application on the user device.
- the streaming application requests the live streaming of the content item from the media server.
- the control circuitry generates the content item for live streaming.
- the content item includes one or more segments where each segment may have one or more frames.
- the control circuitry determines whether playing of the content item lags behind the live streaming of the content item.
- determining the lag is by calculating a hash of a segment of the content item.
- the streaming application receives a manifest of the content item that contains information about (a) Uniform Resource Locator (URL) of the content item, (b) hash code, and (c) timestamp for each segment.
- the streaming application calculates a hash code of a live segment using a timestamp of the live segment.
- the streaming application also determines a hash code for a segment that is currently displayed based on the manifest.
- the streaming application compares the calculated hash code of the live segment to the determined hash code of the currently displayed segment.
- the streaming application determines that there is a lag between the live segment and the currently displayed segment.
- the streaming application determines that there is a lag between the live segment and the currently displayed segment.
- the streaming application determines that there is a lag between the live segment and the currently displayed segment.
- the streaming application generates one or more selectable “catching-up” options for display. The options include (a) catch up to live in a particular time period, (b) catch up to live immediately, and (c) stop the live streaming.
- the control circuitry determines that playing of the content item does not lag behind the live streaming of the content item, then the streaming application does not present the user-selectable “catching-up” options for display.
- the user-selectable options may include a graphical user interface element that prompts the user to select one of the presented options.
- the streaming application receives user input selecting one of the presented options.
- the streaming application performs an action without receiving user input selecting the presented options. For example, based on the user profile data, the streaming application may perform one of the catching-up operations without receiving user input selecting any of the options (a), (b) or (c).
- the user profile data may include the user’s preferred catching-up options and user history data on previously selected options.
- the streaming application presents a sub-option for further user input.
- the streaming application displays a dropdown menu or pop-up window that displays a list of values for the particular time period in the graphical user interface.
- the graphical interface element may include a field where the user can provide a random value for the particular time period. The particular time period may be adjusted based on the user input.
- the sub-option may be optional, and the list of values may be presented with the option (a) interface.
- the streaming application determines a value for the particular time period based on user profile data and presents the system-configured values for display for the user to choose from.
- the user profile data includes user preference information for a preferred catching-up time period or past history of catching-up patterns.
- the streaming application may pre-populate the user’s preferred particular time period in the sub-option.
- the streaming application Based on the selected particular time period, the streaming application skips a presentation of a portion of the missed segments, such that the playing of the content item eventually catches up to live in the particular time period.
- the streaming application may skip a portion of the missed segment based on the user profile data without receiving user input specifying the particular time period.
- the streaming application presents a frame corresponding to the live streaming of the content item (e.g., live frame), such that the playing of the content item catches up to live immediately. The missed frames due to the lag will not be displayed.
- a frame corresponding to the live streaming of the content item e.g., live frame
- FIG. 12 shows an illustrative process for catching up to live based on a skippable bit value, in accordance with some embodiments of the disclosure. It should be noted that process 1200 may be performed by control circuitry 1404 of FIG.
- process 1200 may be incorporated into or combined with one or more steps of any other process or embodiment (e.g., process 1100 of FIG. 11 and process 1300 of FIG. 13).
- the process 1200 may be performed in accordance with the embodiments discussed in connection with FIGS. 5-8.
- the user requests the live stream of the content item via the streaming application on the user device.
- the streaming application requests the live streaming of the content item from the media server.
- the control circuitry receives a manifest of the content item.
- the manifest identifies a plurality of sequential segments of the content item. For each segment of the content item, the manifest contains information about a respective skippable bit value and a respective network address.
- the control circuitry based on the manifest, identifies a first segment as a last-played segment of the content item on the user device.
- the control circuitry also identifies a second segment of the content item as a segment that follows the first segment of the content item in the plurality of sequential segments of the content item.
- the control circuitry determines that playing of the content item lags behind the live streaming of the content item.
- unstable network connectivity causes the lag in the live streaming of the content item.
- the control circuitry determines that the playing of the content item lags behind the live streaming of the content item.
- the control circuitry determines whether a respective skippable bit value of the second segment comprises a first skippable bit value based on the manifest of the content item.
- the first skippable bit value may indicate that the respective segment may be skipped for presentation (e.g., enabled skippable bit).
- the control circuitry skips requesting data from a respective network address of the second segment. Skipping a presentation of a segment may cause the content item to catch up to the live streaming.
- the control circuitry upon determining that the respective skippable bit value of the second segment does not comprise the first skippable bit value, requests data from a respective network address of the second segment.
- the control circuitry plays the second segment from the respective network address of the second segment on the user device.
- the control circuitry identifies a third segment as a subsequent segment that follows the second segment of the content item in the plurality of sequential segments of the content item.
- the control circuitry determines whether a respective skippable bit value of the third segment comprises a second skippable bit value based on the manifest.
- the second skippable bit value may indicate that a presentation of a respective segment cannot be skipped (e.g., disabled skippable bit).
- the control circuitry upon determining that the respective skippable bit value of the third segment comprises the second skippable bit value, the control circuitry requests data from a respective network address of the third segment.
- the control circuitry plays the third segment from the respective network address of the third segment on the user device.
- the control circuitry skips requesting data from the respective network of the third segment.
- the third segment comprises the first skippable bit value different from the second skippable bit value.
- the third segment comprises a skippable bit value that is different from both the first skippable bit value and the second skippable bit value.
- the control circuitry identifies a fourth segment of the content item as a segment that follows the third segment of the content item in the plurality of sequential segments of the content item.
- the control circuitry requests data from a respective network address of the fourth segment of the content item regardless of a respective skippable bit value of the fourth segment.
- the control circuitry nevertheless plays the fourth segment of the content item based on the data requested from the respective network address of the fourth segment of the content item, because the content item is no longer lagging.
- FIG. 13 shows an illustrative process for caching a time-shifted frame to catch up to live, in accordance with some embodiments of the disclosure.
- process 1300 may be performed by control circuitry 1404 of FIG. 14 as instructed by the streaming application that maybe performed on any devices 1502, 1504, 1506 of FIG. 15.
- one or more steps of process 1300 may be incorporated into or combined with one or more steps of any other process or embodiment (e.g., process 1100 of FIG. 11 and process 1200 of FIG. 12).
- the process 1300 may be performed in accordance with the embodiments discussed in connection with FIGS. 9-10.
- the user requests the live streaming of the content item via the streaming application on the user device.
- the streaming application requests the live streaming of the content item to the media server.
- the control circuitry receives a manifest of the content item.
- the manifest identifies a plurality of sequential segments of the content item. Each segment is associated with a timestamp, which identifies the location of each segment in the content item.
- the manifest contains information about a network address and a priority level value for each segment.
- the manifest may be stored in a database of the media server.
- sending the manifest to the streaming application is optional and may only be performed when the lag is identified by the streaming application.
- the media server assigns a priority level to each segment of the content item based on a number of replays of the segment in a predetermined time period. For example, the media server calculates the number of times the segment is streamed or scheduled to be streamed in a particular time period. As an example, replays of a previously streamed segment affect the priority level of a respective segment adversely because the user is likely to have been exposed to the particular segment previously, and the particular segment may not need to be presented to the user again.
- the media server assigns a priority level to each segment of the content item based on a respective relevance level of the segment. For example, the media server determines a respective relevance level of a segment to the user based on the user profile data, such that a segment of a higher relevance (e.g., matching user’s preference) is determined to be more relevant to the user than other scenes. If a particular segment is of high importance to the user (e.g., action scene), then the media server assigns a higher priority level value to the particular segment than a segment that is determined to be of lower relevance to the user (e.g., romance scene).
- a particular segment is of high importance to the user (e.g., action scene)
- the media server assigns a higher priority level value to the particular segment than a segment that is determined to be of lower relevance to the user (e.g., romance scene).
- the media server assigns a priority level to each segment of the content item based on a respective importance level of the segment in the context of the content item. For example, the media server determines a respective importance level of a segment, such that the segment includes scenes that are more important to the content itself than other scenes of the content (e.g., climax scene vs. ending credits). The media server assigns a higher priority level value to a particular segment that has a higher importance level than a segment with a lower importance level.
- the control circuitry determines whether the playing of the content item lags behind the live streaming of the content item. If the playing of the content item does not lag behind the live streaming of the content item, then the control circuitry streams the content item in real time at block 1320. If the playing of the content item lags behind the live streaming of the content item, process 1300 continues to block 1306.
- the control circuitry identifies a duration of the lag from the timestamp of the segment included in the manifest.
- the control circuitry calculates the duration of the lag based on a time difference between when the content item is paused and when the content item is resumed. If there is more than one lag, the control circuitry calculates the total length of the lags.
- the control circuitry selects a priority threshold based on the duration of the lag.
- the priority threshold may be proportional to the duration of the lag. For example, if the duration of the lag is extended, then the priority threshold becomes high as the streaming application may present only a few of the highly important segments to catch up within a limited amount of time.
- the control circuitry identifies a set of segments of the plurality of segments where a total length of the set of segments matches the duration of the lag. For example, when there is more than one lag, the control circuitry identifies, based on the manifest, all of the missed segments to match the total length of the delay.
- the control circuitry identifies a segment that has a priority level value that is higher than the priority threshold based on the received manifest from the media server.
- the media server sends the priority level information of the identified segment upon receiving an indication of the lag from the streaming application at block 1312. For example, upon receiving an indication of the lag, the media server calculates priority level information for the segments that fall within the total length of the lag to minimize the pre-calculation process of the priority level values and the network data transmission. This way, the media server does not have to calculate priority level values for all of the segments of the content item. Instead, the media server only calculates the priority level values for a limited number of segments that matches the length of the delay.
- the control circuitry retrieves a respective segment of the subset using a respective network address.
- the control circuitry skips retrieving a respective segment of the subset with a priority level lower than the priority threshold.
- the control circuitry stores each of the retrieved subset of the segments in a cache of the user device. The control circuitry may store the identified segments until the content item is caught up to live. The segments with priority level values lower than the priority threshold may not be stored in the cache as the presentation of those segments will be skipped.
- control circuitry plays each of the stored subsets of segments from the cache.
- the control circuitry fetches the cached segments and causes the content item to catch up to live.
- the control circuitry upon determining that the playing of the content item no longer lags behind the live streaming, requests data from a respective network address of the set of segments of the plurality of content items regardless of a respective priority level value of the set of segments that follow the stored subset of the plurality of sequential segments in the content item, and stores the retrieved set of segments in the cache of the user device. For example, even though the retrieved set of segments comprises a priority level value that is lower than the priority threshold, the control circuitry nevertheless stores the retrieved set of segments in the cache and plays the set of segments from the cache. In another embodiment, upon determining that the playing of the content item no longer lags behind the live streaming, the control circuitry does not cache any of the segments.
- FIG. 14 depicts a generalized embodiment of an illustrative device (e.g., user device 102, 202, 302, 402) that displays a live video stream.
- User equipment device 1400 may be any of a plurality of user devices such as a smartphone, a tablet, personal computer, set-top box, etc. (discussed further below with respect to FIG. 15).
- User equipment device 1400 may receive the video stream and data via input/output (hereinafter “VO”) path 1402.
- I/O path 1402 may provide the video stream (e.g., on-demand programming, Internet content, content available over LAN or WAN, and the like) and data to control circuitry 1404, which includes processing circuitry 1406 and storage 1408.
- Control circuitry 1404 may be used to send and receive commands, requests, and other suitable data using I/O path 1402.
- I/O path 1402 may connect control circuitry 1404 (and specifically processing circuitry 1406) to one or more communications paths (described below in relation to FIG. 15). I/O functions may be provided by one or more of these communications paths but are shown as a single path in FIG. 14 to avoid overcomplicating the drawing.
- Control circuitry 1404 may be based on any suitable processing circuitry such as processing circuitry 1406.
- Processing circuitry should be understood to mean circuitry based on one or more microprocessors, microcontrollers, digital signal processors, programmable logic devices, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), etc., and may include a multi-core processor (e.g., quad-core).
- processing circuitry may be distributed across multiple separate processor or processing units, for example, multiple of the same type of processing units (e.g., two Intel Core i7 processors) or multiple different processors (e.g., Ryzen processor with integrated CPU and GPU processing cores) or may include a multi-core processor (e.g., dual-core, quad-core, hexa-core, or any suitable number of cores) or supercomputer.
- control circuitry 1404 executes instructions for an application stored in memory (e.g., memory 1408). Specifically, control circuitry 1404 may be instructed by a media application to perform the functions discussed above and below. For example, the media application may provide instructions to control circuitry 1404 to request a video stream from server.
- control circuitry 1404 may include tuning circuitry, such as one or more analog tuners, one or more MP3 decoders or other digital decoding circuitry, or any other suitable tuning or audio circuits or combinations of such circuits. Encoding circuitry (e.g., for converting analog or digital signals to signals for storage in storage 1408) may also be provided. Control circuitry 1404 may also include scaler circuitry for upconverting and downconverting content into the preferred output format of user equipment device 1400, and converter circuitry for converting between digital and analog signals.
- tuning circuitry such as one or more analog tuners, one or more MP3 decoders or other digital decoding circuitry, or any other suitable tuning or audio circuits or combinations of such circuits.
- Encoding circuitry e.g., for converting analog or digital signals to signals for storage in storage 1408
- Control circuitry 1404 may also include scaler circuitry for upconverting and downconverting content into the preferred output format of user equipment device 1400, and converter circuitry for converting between digital and analog signals.
- the tuning and encoding circuitry may be used by user equipment device 1400 to receive, play, and buffer content.
- the circuitry described herein, including, for example, the tuning, audio generating, encoding, decoding, encrypting, decrypting, scaler, and analog/digital circuitry, may be implemented using software running on one or more general purpose or specialized processors. If storage 1408 is provided as a separate device from user equipment device 1400, the tuning and encoding circuitry may be associated with storage 1408.
- Storage 1408 may be any device for storing electronic data, such as random-access memory, solid state devices, quantum storage devices, hard disk drives, non-volatile memory or any other suitable fixed or removable storage devices, and/or any combination of the same.
- Control circuitry 1404 may allocate portions of storage 1408 for various purposes such as caching application instructions, recording media assets, storing portions of a media asset, buffering segments of media, etc.
- storage 1408 may be used to store one or more LUTs storing a number of MAC addresses associated with a plurality of user equipment devices and their corresponding profile information.
- a user may send instructions to control circuitry 1404 using user input interface 1410.
- User input interface 1410 may be any suitable user input interface, such as a remote control, mouse, trackball, keypad, keyboard, touchscreen, touchpad, stylus inputjoystick, voice recognition interface, or other user input interfaces.
- Instructions to control circuitry 1404 may be transmitted through I/O path 1402, which could consist of a video tracking and detection mechanism, Internet of Things (loT) and home automation triggers, emergency alert systems, and software or hardware communication pipelines and/or notification centers.
- Display 1412 may be provided as a stand-alone device or integrated with other elements of each one of user equipment device 1400.
- display 1412 may be a touchscreen or touch-sensitive display, a projector, or a casting device.
- Display 1412 may be one or more of a monitor, a television, a liquid-crystal display (LCD) for a mobile device, silicon display, e-ink display, light-emitting diode (LED) display, or any other suitable equipment for displaying visual images.
- Graphics processing circuitry may generate the output to the display 1412.
- the graphics processing circuitry may be external to processing circuitry 1406 (e.g., as a graphics processing card that communicates with processing circuitry 1406 via I/O path 1402) or may be internal to processing circuitry 1406 or control circuitry 1404 (e.g., on a same silicone die as control circuitry 1404 or processing circuitry 1406).
- the graphics processing circuitry may be used to receive, display, and play content.
- Speakers 1414 may be provided as integrated with other elements of user equipment device 1400 or may be stand-alone units.
- the audio component of videos and other content displayed on display 1412 may be played through speakers 1414.
- the audio may be distributed to a receiver (not shown), which processes and outputs the audio via speakers 1414.
- the speakers 1414 may be part of, but not limited to, a home automation system.
- Streaming applications may be, for example, stand-alone applications implemented on user devices.
- the streaming application may be implemented as software or a set of executable instructions, which may be stored in non-transitory storage 1408 and executed by control circuitry 1404 of a user device 1502, 1504, 1506.
- instructions of the streaming application are stored locally (e.g., in storage 1408), and data for use by the application is downloaded on a periodic basis (e.g., from an out-of-band feed, from an Internet resource, or using another suitable approach).
- Control circuitry 1404 may retrieve instructions of the streaming application from storage 1408 and process the instructions to generate any of the displays discussed herein. Based on the processed instructions, control circuitry 1404 may determine what action to perform when user input is received from input interface 1410.
- FIG. 15 depicts an exemplary media system in accordance with some embodiments of the disclosure, in which user equipment device 102, 202, 302, 402, and user equipment device 1400 can be implemented in the media system 1500 of FIG. 15 as user television equipment 1502, user computer equipment 1504, wireless user communications device 1506, or any other type of user equipment suitable for accessing media.
- these devices may be referred to herein collectively as user equipment or a user device.
- User equipment, on which the media application or the streaming application is implemented, may function as a stand-alone device or may be part of a network of devices.
- Various network configurations of devices may be implemented and are discussed in more detail below.
- User television equipment 1502 may include a set-top box, an integrated receiver decoder (IRD) for handling satellite television, a television set, a digital storage device, or other user television equipment. One or more of these devices may be integrated to be a single device, if desired.
- User computer equipment 1504 may include a PC, a laptop, a streaming content aggregator, a PC media center, or other user computer equipment. It may include devices like digital assistance, smart speakers, and/or home automation.
- Wireless user communications device 1506 may include a smartphone, a portable video player, a portable music player, a portable gaming machine, a tablet, a wireless streaming device or other wireless device. It should be noted that the lines are blurred when trying to classify a device as one of the above devices and one device may be categorized into one or more of the categories listed above.
- each user may utilize more than one type of user equipment (e.g., a user may have a computer and a tablet) and also more than one of each type of user equipment device (e.g., a user may have multiple television sets).
- each type of user equipment e.g., a user may have a computer and a tablet
- each type of user equipment device e.g., a user may have multiple television sets.
- the user equipment may be coupled to communications network 1514.
- user television equipment 1502, user computer equipment 1504, and wireless user communications device 1506 are coupled to communications network 1514 via communications paths 1508, 1510, and 1512, respectively.
- Communications network 1514 is used by the user equipment to obtain the video stream.
- Communications network 1514 may be one or more networks including the Internet, a mobile phone network, ad-hoc network, or other types of communications network or combination of communications networks.
- Paths 1508, 1510, and 1512 may separately or together include one or more communications paths, including any suitable wireless communications path.
- Path 1512 is drawn as a dotted line to indicate it is a wireless path. Communications with the user equipment may be provided by one or more of these communications paths but are shown as a single path in FIG. 15 to avoid overcomplicating the drawing.
- the user equipment devices may communicate with each other directly through an indirect path via communications network 1514.
- System 1500 includes content item source 1516 and content item data source 1518 coupled to communications network 1514 via communications paths 1520 and 1522, respectively.
- Paths 1520 and 1522 may include any of the communications paths described above in connection with paths 1508, 1510, and 1512. Communications with the content item source 1516 and content item data source 1518 may be exchanged over one or more communications paths but are shown as a single path in FIG. 15 to avoid overcomplicating the drawing.
- content item source 1516 and content item data source 1518 are integrated as one source device.
- sources 1516 and 1518 may communicate directly with user equipment devices 1502, 1504, and 1506 via communications paths (not shown) such as those described above in connection with paths 1508, 1510, and 1512.
- Content item source 1516 or content item data source 1518 may include one or more types of media distribution equipment such as a media server, cable system headend, satellite distribution facility, intermediate distribution facilities and/or servers, Internet providers, on- demand media servers, and other media providers.
- Content item source 1516 or content item data source 1518 may be the originator of media content or may not be the originator of media content.
- Content item source 1516 or content item data source 1518 may also include a remote media server used to store different types of media content (including a media asset selected by a user), in a location remote from any of the user equipment.
- Systems and methods for providing remotely stored media to user equipment are discussed in greater detail in connection with Ellis et al., U.S. Patent No. 7,761,892, issued July 20, 2010, which is hereby incorporated by reference herein in its entirety.
- System 1500 is intended to illustrate a number of approaches, or network configurations, by which user equipment devices and sources of media content and guidance data may communicate with each other for the purpose of accessing media and data related to the media.
- the configuration of the devices and paths in system 1500 may change without departing from the scope of the present disclosure.
- processes 1100, 1200 and 1300 can be performed on any of the devices shown in FIGS. 14-15. Additionally, any of the steps in processes 1100, 1200 and 1300 can be performed in any order, can be omitted, and/or can be combined with any of the steps from any other process.
- a method comprising: generating a content item for live streaming, the content item comprising a plurality of frames; in response to determining that playing of the content item lags behind the live streaming of the content item, presenting a selectable option for a user input, the option comprising: (a) catch up to live in a particular time period; and in response to receiving the user input via a corresponding option to (a) catch up to live in the particular time period: skipping presentation of a portion of the plurality of frames, such that the playing of the content item catches up to live in the particular time period.
- the method of item 1 further comprising: receiving a user input selecting a particular value for the particular time period, wherein the particular time period is adjusted based on the user input.
- the option comprises: (c) stop the live streaming, wherein the method further comprises: in response to receiving a user input via a corresponding option to (c) stop the live streaming: presenting a corresponding frame of when determining that the playing of the content item lags behind the live streaming, such that the playing of the content item does not match the live streaming of the content item.
- the method of item 1 further comprising: receiving a manifest of the content item, wherein the manifest identifies the plurality of frames of the content item; wherein for each respective frame of the content item, the manifest comprises:
- the method of item 5 further comprising: identifying a first frame of the content item as a last played frame of the content item; identifying a second frame of the content item that follows the first frame of the plurality of frames; and identifying a third frame of the content item that is currently being presented that is different from the second frame.
- determining that the playing of the content item lags behind the live streaming of the content item comprises: determining that the third frame does not match the second frame.
- determining that the third frame does not match the second frame comprises: determining a respective hash code for the second frame based on the manifest; calculating a respective hash code based on a respective timestamp of the third frame; and determining that the respective hash code for the second frame does not correspond to the calculated hash code of the third frame.
- the method of item 1 further comprising: determining, based on user profile data, user preference information for skipping presentation of the portion of the plurality of frames; and presenting a corresponding frame based on the user preference information.
- a system comprising: control circuitry configured to: generate a content item for live streaming, the content item comprising a plurality of frames; user interface circuitry configured to: in response to determining that playing of the content item lags behind the live streaming of the content item, present a selectable option for a user input, the option comprising: (a) catch up to live in a particular time period; and wherein the control circuitry is further configured to: in response to receiving the user input via a corresponding option to (a) catch up to live in the particular time period: skip presentation of a portion of the plurality of frames, such that the playing of the content item catches up to live in the particular time period.
- the control circuitry is further configured to: receive a user input selecting a particular value for the particular time period, wherein the particular time period is adjusted based on the user input.
- the option comprises: (b) catch up to live immediately
- the user interface circuitry is further configured to: in response to receiving a user input via a corresponding option to (b) catch up to live immediately: present a frame corresponding to the live streaming of the content item, such that the playing of the content item catches up to live immediately.
- the option comprises: (c) stop the live streaming, wherein the user interface circuitry is further configured to: in response to receiving a user input via a corresponding option to (c) stop the live streaming: present a corresponding frame of when determining that the playing of the content item lags behind the live streaming, such that the playing of the content item does not match the live streaming of the content item.
- control circuitry is further configured to: receive a manifest of the content item, wherein the manifest identifies the plurality of frames of the content item; wherein for each respective frame of the content item, the manifest comprises:
- control circuitry is further configured to: identify a first frame of the content item as a last played frame of the content item; identify a second frame of the content item that follows the first frame of the plurality of frames; and identify a third frame of the content item that is currently being presented that is different from the second frame. 17.
- control circuitry wherein determining that the playing of the content item lags behind the live streaming of the content item, is further configured to: determine that the third frame does not match the second frame.
- control circuitry wherein determining that the third frame does not match the second frame, is further configured to: determine a respective hash code for the second frame based on the manifest; calculate a respective hash code based on a respective timestamp of the third frame; and determine that the respective hash code for the second frame does not correspond to the calculated hash code of the third frame.
- control circuitry is further configured to: determine, based on user profile data, user preference information for skipping presentation of the portion of the plurality of frames; and present a corresponding frame based on the user preference information.
- a system comprising: means for generating a content item for live streaming, the content item comprising a plurality of frames; in response to determining that playing of the content item lags behind the live streaming of the content item, means for presenting a selectable option for a user input, the option comprising: (a) catch up to live in a particular time period; and in response to receiving the user input via a corresponding option to (a) catch up to live in the particular time period: means for skipping presentation of a portion of the plurality of frames, such that the playing of the content item catches up to live in the particular time period.
- the option comprises: (b) catch up to live immediately, wherein the system further comprises: in response to receiving a user input via a corresponding option to (b) catch up to live immediately: means for presenting a frame corresponding to the live streaming of the content item, such that the playing of the content item catches up to live immediately.
- the option comprises: (c) stop the live streaming, wherein the system further comprises: in response to receiving a user input via a corresponding option to (c) stop the live streaming: means for presenting a corresponding frame of when determining that the playing of the content item lags behind the live streaming, such that the playing of the content item does not match the live streaming of the content item.
- the system of item 25 further comprising: means for identifying a first frame of the content item as a last played frame of the content item; means for identifying a second frame of the content item that follows the first frame of the plurality of frames; and means for identifying a third frame of the content item that is currently being presented that is different from the second frame.
- means for determining that the playing of the content item lags behind the live streaming of the content item comprises: means for determining that the third frame does not match the second frame.
- means for determining that the third frame does not match the second frame comprises: means for determining a respective hash code for the second frame based on the manifest; means for calculating a respective hash code based on a respective timestamp of the third frame; and means for determining that the respective hash code for the second frame does not correspond to the calculated hash code of the third frame.
- the non-transitory computer readable medium of item 31 further comprising: instructions for receiving a user input selecting a particular value for the particular time period, wherein the particular time period is adjusted based on the user input.
- non-transitory computer readable medium of item 31 wherein the option comprises: (b) catch up to live immediately, and wherein the non-transitory computer readable medium further comprises: in response to receiving a user input via a corresponding option to (b) catch up to live immediately: instructions for presenting a frame corresponding to the live streaming of the content item, such that the playing of the content item catches up to live immediately.
- non-transitory computer readable medium of item 31 wherein the option comprises: (c) stop the live streaming, and wherein the non-transitory computer readable medium further comprises: in response to receiving a user input via a corresponding option to (c) stop the live streaming: instructions for presenting a corresponding frame of when determining that the playing of the content item lags behind the live streaming, such that the playing of the content item does not match the live streaming of the content item.
- non-transitory computer readable medium of item 31 further comprising: instructions for receiving a manifest of the content item, wherein the manifest identifies the plurality of frames of the content item; and wherein for each respective frame of the content item, the manifest comprises:
- the non-transitory computer readable medium of item 35 further comprising: instructions for identifying a first frame of the content item as a last played frame of the content item; instructions for identifying a second frame of the content item that follows the first frame of the plurality of frames; and instructions for identifying a third frame of the content item that is currently being presented that is different from the second frame.
- instructions for determining that the playing of the content item lags behind the live streaming of the content item comprises: instructions for determining that the third frame does not match the second frame.
- instructions for determining that the third frame does not match the second frame comprises: instructions for determining a respective hash code for the second frame based on the manifest; instructions for calculating a respective hash code based on a respective timestamp of the third frame; and instructions for determining that the respective hash code for the second frame does not correspond to the calculated hash code of the third frame.
- non-transitory computer readable medium of item 31 further comprising: instructions for determining, based on user profile data, user preference information for skipping presentation of the portion of the plurality of frames; and instructions for presenting a corresponding frame based on the user preference information.
- a method comprising: generating a content item for live streaming, the content item comprising a plurality of frames; in response to determining that playing of the content item lags behind the live streaming of the content item, presenting a selectable option for a user input, the option comprising: (a) catch up to live in a particular time period; and in response to receiving the user input via a corresponding option to (a) catch up to live in the particular time period: skipping presentation of a portion of the plurality of frames, such that the playing of the content item catches up to live in the particular time period.
- the method of item 41 further comprising: receiving a user input selecting a particular value for the particular time period, wherein the particular time period is adjusted based on the user input.
- any one of items 41-43 wherein the option comprises: (c) stop the live streaming, wherein the method further comprises: in response to receiving a user input via a corresponding option to (c) stop the live streaming: presenting a corresponding frame of when determining that the playing of the content item lags behind the live streaming, such that the playing of the content item does not match the live streaming of the content item.
- any one of items 41-44 further comprising: receiving a manifest of the content item, wherein the manifest identifies the plurality of frames of the content item; wherein for each respective frame of the content item, the manifest comprises:
- any one of items 41-45 further comprising: identifying a first frame of the content item as a last played frame of the content item; identifying a second frame of the content item that follows the first frame of the plurality of frames; and identifying a third frame of the content item that is currently being presented that is different from the second frame.
- determining that the playing of the content item lags behind the live streaming of the content item comprises: determining that the third frame does not match the second frame.
- determining that the third frame does not match the second frame comprises: determining a respective hash code for the second frame based on the manifest; calculating a respective hash code based on a respective timestamp of the third frame; and determining that the respective hash code for the second frame does not correspond to the calculated hash code of the third frame.
- any one of items 41-48 further comprising: determining, based on user profile data, user preference information for skipping presentation of the portion of the plurality of frames; and presenting a corresponding frame based on the user preference information.
- a method compri sing receiving a manifest of a content item, wherein the manifest identifies a plurality of sequential segments of the content item, wherein for each respective segment of the content item, the manifest comprises:
- a respective skippable bit value identifying a first segment of the content item as a last played segment of the content item; identifying a second segment of the content item that follows the first segment of the content item in the plurality of sequential segments of the content item; determining that playing of the content item lags behind live streaming of the content item: in response to determining that a respective skippable bit value for the second segment of the content item comprises a first skippable bit value, skipping requesting data from a respective network address of the second segment of the content item; in response to determining that a respective skippable bit value for a third segment of the content item comprises a second skippable bit value, requesting data from a respective network address of the third segment of the content item, wherein the third segment of the content item follows the second segment of the content item in the plurality of sequential segments of the content item; and playing the third segment of the content item based on the data requested from the respective network address of the third segment of the content item.
- the method of item 51 further comprising: identifying a fourth segment of the content item that follows the third segment of the content item in the plurality of sequential segments of the content item; in response to determining that playing of the content item no longer lags behind the live streaming: requesting data from a respective network address of the fourth segment of the content item, wherein a respective skippable bit value for the fourth segment of the content item comprises the first skippable bit value; and playing the fourth segment of the content item based on the data requested from the respective network address of the fourth segment of the content item.
- the respective importance level of the respective segment is determined based on at least one of a number of replays of the respective segment in a predetermined time period, a type of the content item, or a genre of the respective segment in the content item.
- the first skippable bit value is below an importance threshold and the second skippable bit value is above the importance threshold, and wherein the second skippable bit value is different from the first skippable bit value.
- a method comprising: transmitting, by a server, a manifest of a content item to a client device, wherein the manifest identifies a plurality of sequential segments of the content item, wherein for each respective segment of the content item, the manifest comprises:
- a respective skippable bit value identifying, by the server, a first segment of the content item as a last played segment of the content item on the client device; identifying, by the server, a second segment of the content item that follows the first segment of the content item in the plurality of sequential segments of the content item; receiving, from the client device, an indication that playing of the content item on the client device lags behind live streaming of the content item: in response to determining that a respective skippable bit value for the second segment of the content item comprises a first skippable bit value, causing the client device to skip requesting data from a respective network address of the second segment of the content item; in response to determining that a respective skippable bit value for a third segment of the content item comprises a second skippable bit value, causing the client device to send a request of data from a respective network address for the third segment of the content item, wherein the third segment of the content item follows the second segment of the content item in the plurality of sequential segments of the content item; and
- a system comprising: control circuity configured to: receive a manifest of a content item, wherein the manifest identifies a plurality of sequential segments of the content item, wherein for each respective segment of the content item, the manifest comprises:
- a respective skippable bit value identifies a first segment of the content item as a last played segment of the content item; identify a second segment of the content item that follows the first segment of the content item in the plurality of sequential segments of the content item; determine that playing of the content item lags behind live streaming of the content item: in response to determining that a respective skippable bit value for the second segment of the content item comprises a first skippable bit value, skip requesting data from a respective network address of the second segment of the content item; in response to determining that a respective skippable bit value for a third segment of the content item comprises a second skippable bit value, request data from a respective network address of the third segment of the content item, wherein the third segment of the content item follows the second segment of the content item in the plurality of sequential segments of the content item; and user interface circuitry configured to: play the third segment of the content item based on the data requested from the respective network address of the third segment of the content item.
- control circuitry is further configured to: identify a fourth segment of the content item that follows the third segment of the content item in the plurality of sequential segments of the content item; in response to determining that playing of the content item no longer lags behind the live streaming: request data from a respective network address of the fourth segment of the content item, wherein a respective skippable bit value for the fourth segment of the content item comprises the first skippable bit value; and play the fourth segment of the content item based on the data requested from the respective network address of the fourth segment of the content item.
- a lag of the live streaming of the content item is caused based on a network connectivity, and wherein playing the content item for display is paused, such that playing the content item falls behind the live streaming of the content item.
- the control circuitry is further configured to receive metadata of the content item, wherein the metadata comprises (a) a respective network address; and (b) a respective skippable bit value.
- a system comprising: means for receiving a manifest of a content item, wherein the manifest identifies a plurality of sequential segments of the content item, wherein for each respective segment of the content item, the manifest comprises:
- a respective skippable bit value means for identifying a first segment of the content item as a last played segment of the content item; means for identifying a second segment of the content item that follows the first segment of the content item in the plurality of sequential segments of the content item; means for determining that playing of the content item lags behind live streaming of the content item: in response to determining that a respective skippable bit value for the second segment of the content item comprises a first skippable bit value, means for skipping requesting data from a respective network address of the second segment of the content item; in response to determining that a respective skippable bit value for a third segment of the content item comprises a second skippable bit value, means for requesting data from a respective network address of the third segment of the content item, wherein the third segment of the content item follows the second segment of the content item in the plurality of sequential segments of the content item; and means for playing the third segment of the content item based on the data requested from the respective network address of the third segment of the content item
- the system of item 71 further comprising: means for identifying a fourth segment of the content item that follows the third segment of the content item in the plurality of sequential segments of the content item; in response to determining that playing of the content item no longer lags behind the live streaming: means for requesting data from a respective network address of the fourth segment of the content item, wherein a respective skippable bit value for the fourth segment of the content item comprises the first skippable bit value; and means for playing the fourth segment of the content item based on the data requested from the respective network address of the fourth segment of the content item.
- the non-transitory computer readable medium of item 81 further comprising: instructions for identifying a fourth segment of the content item that follows the third segment of the content item in the plurality of sequential segments of the content item; in response to determining that playing of the content item no longer lags behind the live streaming: instructions for requesting data from a respective network address of the fourth segment of the content item, wherein a respective skippable bit value for the fourth segment of the content item comprises the first skippable bit value; and instructions for playing the fourth segment of the content item based on the data requested from the respective network address of the fourth segment of the content item.
- the respective skippable bit value is determined based on a respective importance level of a respective segment.
- the non-transitory computer readable medium of item 83 wherein the respective importance level of the respective segment is determined based on at least one of a number of replays of the respective segment in a predetermined time period, a type of the content item, or a genre of the respective segment in the content item.
- the non-transitory computer readable medium of item 81 further comprising: instructions for receiving metadata of the content item, wherein the metadata comprises (a) a respective network address; and (b) a respective skippable bit value.
- a method compri sing receiving a manifest of a content item, wherein the manifest identifies a plurality of sequential segments of the content item, wherein for each respective segment of the content item, the manifest comprises:
- a respective skippable bit value identifying a first segment of the content item as a last played segment of the content item; identifying a second segment of the content item that follows the first segment of the content item in the plurality of sequential segments of the content item; determining that playing of the content item lags behind live streaming of the content item: in response to determining that a respective skippable bit value for the second segment of the content item comprises a first skippable bit value, skipping requesting data from a respective network address of the second segment of the content item; in response to determining that a respective skippable bit value for a third segment of the content item comprises a second skippable bit value, requesting data from a respective network address of the third segment of the content item, wherein the third segment of the content item follows the second segment of the content item in the plurality of sequential segments of the content item; and playing the third segment of the content item based on the data requested from the respective network address of the third segment of the content item.
- a method compri sing receiving a manifest of a content item, wherein the manifest identifies a plurality of sequential segments of the content item, wherein for each segment of the plurality of sequential segments, the manifest comprises:
- the method of item 101 further compri sing : identifying, based on the manifest, another subset of segments from the set of segments, wherein each respective segment in another subset comprises a respective priority level value lower than the priority threshold; and in response to identifying another subset; skipping retrieving of the respective segment of another subset from a respective network address.
- the method of item 101 further comprising: in response to determining that playing of the content item no longer lags behind the live streaming: retrieving data for a set of segments of the content item of the plurality of segments, wherein a respective priority level value for the set of segments of the content item is at least one of: (a) higher than the priority threshold; and (b) lower than the priority threshold; storing the retrieved set of segments of the content item; and playing the stored set of segments of the content item.
- the method of item 101 further comprising: determining, based on user profile data, user preference information for storing the retrieved subset of segments in the cache; and presenting a corresponding frame based on the user preference information.
- priority level value is determined based on a respective relevance level of the frame based on the user profile data.
- a method compri sing transmitting, by a server, a manifest of a content item to a client device, wherein the manifest identifies a plurality of sequential segments of the content item, wherein for each segment of the plurality of sequential segments, the manifest comprises:
- a respective priority level value in response to receiving an indication from the client device that playing of the content item lags behind live streaming of the content item: identifying, by the server, a duration of a lag from the manifest of the content item; based on the duration of the lag, selecting, by the server, a priority threshold; identifying, based on the manifest, a set of segments of the plurality of segments of the content item, wherein a total time length of the set of segments matches the duration of the lag; identifying a subset of segments from the set of segments, wherein each respective segment in the subset comprises a priority level value higher than the priority threshold; retrieving each respective segment of the subset of segments using a respective network address; storing each of the retrieved subset of segments in a cache; and causing the client device to play each of the stored subset of segments from the cache.
- a system comprising: control circuitry configured to: receive a manifest of a content item, wherein the manifest identifies a plurality of sequential segments of the content item, wherein for each segment of the plurality of sequential segments, the manifest comprises:
- control circuitry is further configured to: identify, based on the manifest, another subset of segments from the set of segments, wherein each respective segment in another subset comprises a respective priority level value lower than the priority threshold; and in response to identifying another subset; skip retrieving of the respective segment of another subset from a respective network address.
- control circuitry is further configured to: in response to determining that playing of the content item no longer lags behind the live streaming: retrieve data for a set of segments of the content item of the plurality of segments, wherein a respective priority level value for the set of segments of the content item is at least one of: (a) higher than the priority threshold; and (b) lower than the priority threshold; store the retrieved set of segments of the content item; and play the stored set of segments of the content item.
- control circuitry is further configured to: determine, based on user profile data, user preference information for storing the retrieved subset of segments in the cache; and present a corresponding frame based on the user preference information.
- priority level value is determined based on a respective relevance level of the frame based on the user profile data.
- control circuitry is further configured to: in response to determining that playing of the content item no longer lags behind the live streaming: skip retrieving data for a set of segments of the content item of the plurality of segments, wherein a respective priority level value for the set of segments of the content item is at least one of: (a) higher than the priority threshold; and (b) lower than the priority threshold.
- a system comprising: means for receiving a manifest of a content item, wherein the manifest identifies a plurality of sequential segments of the content item, wherein for each segment of the plurality of sequential segments, the manifest comprises:
- the system of item 121 further comprising: means for identifying, based on the manifest, another subset of segments from the set of segments, wherein each respective segment in another subset comprises a respective priority level value lower than the priority threshold; and in response to identifying another subset; means for skipping retrieving of the respective segment of another subset from a respective network address.
- skipping a presentation of the respective segment of another subset causes the content item to catch up to the live streaming.
- the system of item 121 further comprising: in response to determining that playing of the content item no longer lags behind the live streaming: means for retrieving data for a set of segments of the content item of the plurality of segments, wherein a respective priority level value for the set of segments of the content item is at least one of: (a) higher than the priority threshold; and (b) lower than the priority threshold; means for storing the retrieved set of segments of the content item; and means for playing the stored set of segments of the content item.
- the system of item 121 further comprising: means for determining, based on user profile data, user preference information for storing the retrieved subset of segments in the cache; and means for presenting a corresponding frame based on the user preference information.
- priority level value is determined based on a respective relevance level of the frame based on the user profile data.
- the respective priority level value is determined based on a respective importance level of the segment in context of the content item. 128. The system of item 121, wherein the respective priority level value is determined based on a number of replays of the respective frame in a predetermined time period.
- the system of item 121 further comprising: in response to determining that playing of the content item no longer lags behind the live streaming: means for skip retrieving data for a set of segments of the content item of the plurality of segments, wherein a respective priority level value for the set of segments of the content item is at least one of: (a) higher than the priority threshold; and (b) lower than the priority threshold.
- non-transitory computer readable medium of item 131 further comprising: instructions for identifying, based on the manifest, another subset of segments from the set of segments, wherein each respective segment in another subset comprises a respective priority level value lower than the priority threshold; and in response to identifying another subset; instructions for skipping retrieving of the respective segment of another subset from a respective network address.
- non-transitory computer readable medium of item 132 wherein skipping a presentation of the respective segment of another subset causes the content item to catch up to the live streaming.
- the non-transitory computer readable medium of item 131 further comprising: in response to determining that playing of the content item no longer lags behind the live streaming: instructions for retrieving data for a set of segments of the content item of the plurality of segments, wherein a respective priority level value for the set of segments of the content item is at least one of: (a) higher than the priority threshold; and (b) lower than the priority threshold; instructions for storing the retrieved set of segments of the content item; and instructions for playing the stored set of segments of the content item.
- the non-transitory computer readable medium of item 131 further comprising: instructions for determining, based on user profile data, user preference information for storing the retrieved subset of segments in the cache; and instructions for presenting a corresponding frame based on the user preference information.
- priority level value is determined based on a respective relevance level of the frame based on the user profile data.
- the non-transitory computer readable medium of item 131 further comprising: in response to determining that playing of the content item no longer lags behind the live streaming: instructions for skip retrieving data for a set of segments of the content item of the plurality of segments, wherein a respective priority level value for the set of segments of the content item is at least one of: (a) higher than the priority threshold; and (b) lower than the priority threshold.
- a method compri sing receiving a manifest of a content item, wherein the manifest identifies a plurality of sequential segments of the content item, wherein for each segment of the plurality of sequential segments, the manifest comprises:
- the method of item 141 further comprising: identifying, based on the manifest, another subset of segments from the set of segments, wherein each respective segment in another subset comprises a respective priority level value lower than the priority threshold; and in response to identifying another subset; skipping retrieving of the respective segment of another subset from a respective network address.
- any of items 141-143 further comprising: in response to determining that playing of the content item no longer lags behind the live streaming: retrieving data for a set of segments of the content item of the plurality of segments, wherein a respective priority level value for the set of segments of the content item is at least one of: (a) higher than the priority threshold; and (b) lower than the priority threshold; storing the retrieved set of segments of the content item; and playing the stored set of segments of the content item.
- any of items 141-144 further comprising: determining, based on user profile data, user preference information for storing the retrieved subset of segments in the cache; and presenting a corresponding frame based on the user preference information.
- any of items 141-149 further comprising: in response to determining that playing of the content item no longer lags behind the live streaming: skipping retrieving data for a set of segments of the content item of the plurality of segments, wherein a respective priority level value for the set of segments of the content item is at least one of: (a) higher than the priority threshold; and (b) lower than the priority threshold.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Human Computer Interaction (AREA)
- Computer Security & Cryptography (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Information Transfer Between Computers (AREA)
Abstract
Description
Claims
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US17/114,045 US11490153B2 (en) | 2020-12-07 | 2020-12-07 | Systems and methods for dynamically syncing from time-shifted frame to live stream of content |
US17/114,038 US11490167B2 (en) | 2020-12-07 | 2020-12-07 | Systems and methods for dynamically syncing from time-shifted frame to live stream of content |
US17/114,027 US11770588B2 (en) | 2020-12-07 | 2020-12-07 | Systems and methods for dynamically syncing from time-shifted frame to live stream of content |
PCT/US2021/062226 WO2022125559A1 (en) | 2020-12-07 | 2021-12-07 | Systems and methods for dynamically syncing from time-shifted frame to live stream of content |
Publications (1)
Publication Number | Publication Date |
---|---|
EP4256789A1 true EP4256789A1 (en) | 2023-10-11 |
Family
ID=79283264
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP21839738.8A Pending EP4256789A1 (en) | 2020-12-07 | 2021-12-07 | Systems and methods for dynamically syncing from time-shifted frame to live stream of content |
Country Status (3)
Country | Link |
---|---|
EP (1) | EP4256789A1 (en) |
CA (1) | CA3204498A1 (en) |
WO (1) | WO2022125559A1 (en) |
Family Cites Families (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1867068A (en) | 1998-07-14 | 2006-11-22 | 联合视频制品公司 | Client-server based interactive television program guide system with remote server recording |
WO2011075016A1 (en) * | 2009-12-15 | 2011-06-23 | Telefonaktiebolaget L M Ericsson (Publ) | Pausing of a live media stream |
US8825790B2 (en) * | 2012-03-06 | 2014-09-02 | Adobe Systems Incorporated | Caching of fragmented streaming media |
EP2775731A1 (en) * | 2013-03-05 | 2014-09-10 | British Telecommunications public limited company | Provision of video data |
US10536758B2 (en) * | 2014-10-09 | 2020-01-14 | Thuuz, Inc. | Customized generation of highlight show with narrative component |
US10708662B1 (en) * | 2018-04-04 | 2020-07-07 | Amazon Technologies, Inc. | Customized streaming of digital content |
-
2021
- 2021-12-07 WO PCT/US2021/062226 patent/WO2022125559A1/en unknown
- 2021-12-07 EP EP21839738.8A patent/EP4256789A1/en active Pending
- 2021-12-07 CA CA3204498A patent/CA3204498A1/en active Pending
Also Published As
Publication number | Publication date |
---|---|
CA3204498A1 (en) | 2022-06-16 |
WO2022125559A1 (en) | 2022-06-16 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11770588B2 (en) | Systems and methods for dynamically syncing from time-shifted frame to live stream of content | |
US11627364B1 (en) | Systems and methods for dynamically syncing from time-shifted frame to live stream of content | |
US11595721B2 (en) | Methods and systems for predictive buffering of related content segments | |
US20140325556A1 (en) | Alerts and web content over linear tv broadcast | |
US8886765B2 (en) | System and method for predicitive trick play using adaptive video streaming | |
US11909988B2 (en) | Systems and methods for multiple bit rate content encoding | |
US11122325B1 (en) | Multiscreen experience for parallel playback of time shifted live stream content | |
US20220321972A1 (en) | Transmitting content based on genre information | |
US20240340493A1 (en) | Systems and methods for increasing first user subscription | |
US11490167B2 (en) | Systems and methods for dynamically syncing from time-shifted frame to live stream of content | |
US20220264170A1 (en) | Systems and methods for dynamically adjusting quality levels for transmitting content based on context | |
US9215267B2 (en) | Adaptive streaming for content playback | |
US20210345003A1 (en) | Systems and methods for providing timeline of content items on a user interface | |
EP4256789A1 (en) | Systems and methods for dynamically syncing from time-shifted frame to live stream of content | |
US20240259650A1 (en) | Systems and methods for pausing a digital media stream in response to determining that an associated display device is turned off | |
WO2023069853A1 (en) | Systems and methods for predictive selection of pause points |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: UNKNOWN |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20230705 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
DAV | Request for validation of the european patent (deleted) | ||
DAX | Request for extension of the european patent (deleted) | ||
RAP3 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: ADEIA GUIDES INC. |