US20150350622A1 - Packed i-frames - Google Patents

Packed i-frames Download PDF

Info

Publication number
US20150350622A1
US20150350622A1 US14/501,941 US201414501941A US2015350622A1 US 20150350622 A1 US20150350622 A1 US 20150350622A1 US 201414501941 A US201414501941 A US 201414501941A US 2015350622 A1 US2015350622 A1 US 2015350622A1
Authority
US
United States
Prior art keywords
playback mode
media
frames
video data
segments
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
US14/501,941
Inventor
Roger N. Pantos
Zhenheng Li
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Apple Inc
Original Assignee
Apple Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to US201462005803P priority Critical
Application filed by Apple Inc filed Critical Apple Inc
Priority to US14/501,941 priority patent/US20150350622A1/en
Assigned to APPLE INC. reassignment APPLE INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: LI, ZHENHENG, PANTOS, ROGER N.
Publication of US20150350622A1 publication Critical patent/US20150350622A1/en
Application status is Pending legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00Details of colour television systems
    • H04N9/79Processing of colour television signals in connection with recording
    • H04N9/80Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
    • H04N9/804Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback involving pulse code modulation of the colour picture signal components
    • H04N9/8042Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback involving pulse code modulation of the colour picture signal components involving data reduction
    • GPHYSICS
    • G11INFORMATION STORAGE
    • G11BINFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/005Reproducing at a different information rate from the information rate of recording
    • GPHYSICS
    • G11INFORMATION STORAGE
    • G11BINFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G11B27/34Indicating arrangements
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00Network arrangements or protocols for real-time communications
    • H04L65/40Services or applications
    • H04L65/4069Services related to one way streaming
    • H04L65/4084Content on demand
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00Network arrangements or protocols for real-time communications
    • H04L65/40Services or applications
    • H04L65/4069Services related to one way streaming
    • H04L65/4092Control of source by destination, e.g. user controlling streaming rate of server
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00Network arrangements or protocols for real-time communications
    • H04L65/60Media handling, encoding, streaming or conversion
    • H04L65/601Media manipulation, adaptation or conversion
    • H04L65/604Media manipulation, adaptation or conversion at the destination
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00Network arrangements or protocols for real-time communications
    • H04L65/60Media handling, encoding, streaming or conversion
    • H04L65/607Stream encoding details
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/23439Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements for generating different versions
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/238Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams
    • H04N21/2387Stream processing in response to a playback request from an end-user, e.g. for trick-play
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/65Transmission of management data between client and server
    • H04N21/658Transmission by the client directed to the server
    • H04N21/6587Control parameters, e.g. trick play commands, viewpoint selection
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845Structuring of content, e.g. decomposing content into time segments
    • H04N21/8455Structuring of content, e.g. decomposing content into time segments involving pointers to the content, e.g. pointers to the I-frames of the video stream
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845Structuring of content, e.g. decomposing content into time segments
    • H04N21/8456Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85Assembly of content; Generation of multimedia applications
    • H04N21/854Content authoring
    • H04N21/8543Content authoring using a description language, e.g. Multimedia and Hypermedia information coding Expert Group [MHEG], eXtensible Markup Language [XML]

Abstract

Systems and methods for enabling playback control functions of a media player are disclosed. For example, a user of a client device receiving streaming playback of a video stream may perform rewind and fast forward control functions. The client device may implement these playback control functions by retrieving an enhanced playback segment. Using the enhanced playback segment, the media player may display selected frames at a predetermined interval while maintaining a visual cadence that is pleasing to a viewer. In the described embodiments, a client device may render a video stream, receive a command to control a fast forward or rewind playback mode for the video stream, and retrieve, from a distribution server or associated edge cache, one or more enhanced playback segments adapted to implement the user command.

Description

    CROSS-REFERENCE TO RELATED APPLICATION
  • This application is a non-provisional application of U.S. Provisional Application Ser. No. 62/005,803, filed May 30, 2014, the contents of which are referenced herein in its entirety.
  • BACKGROUND
  • The present invention generally relates to streaming digital media and, in particular, to the organization of coded video data among transmission segments to support enhanced playback functions for streaming media.
  • Streaming digital media from a remote server to a client device over a communication network is an increasingly popular way for retrieving and viewing various digital media, which may include audio and video streams. Within a media player, users are accustomed to traditional playback control functions, such as rewind and fast forward. To implement these playback control functions for digital media, I-frames are selected at predetermined intervals depending on the fast forward or rewind speed.
  • However, implementing playback control functions for streaming digital media presents difficulties that are not encountered for their locally-stored counterparts. For example, when implementing fast forward functions, upcoming frames may not yet be stored at a client device. As a result, the client device must retrieve frames from the remote server as part of the streaming operation. The I-frames used for fast forward and rewind functions are intermixed with too many frames of other types to be downloaded efficiently. In addition, characteristics of typical network connections can present further difficulties in retrieving these frames in a manner compatible with achieving desirable qualities of the control functions. For example, it is typically desirable to perform rewind and fast forward functions by displaying frames selected at a predetermined interval in order to achieve a visual cadence that is pleasing to a viewer (e.g., not too choppy). However, retrieving frames over a communications network often involves variable and unpredictable transmission. Such variability and unpredictability has, thus far, caused processes dependent upon those retrieved frames, such as the implementation of playback control functions, to be unreliable.
  • In light of the drawbacks discussed above, the inventors have developed improved systems and methods for implementing playback control functions for streaming media.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • FIG. 1 is a simplified block diagram of an example media distribution system suitable for use with the present invention.
  • FIG. 2 illustrates example media player application hosted on a client device according to an example embodiment.
  • FIG. 3 illustrates a simplified user interface of a media player that implements media stream playback control functions according to an example embodiment.
  • FIG. 4 illustrates a video stream suitable for use with the present invention.
  • FIG. 5 illustrates the composition of an enhanced playback segment according to an example embodiment.
  • FIGS. 6A-6E illustrate various video frame display timelines during normal speed forward and other control playback modes of a media stream according to example embodiments.
  • FIG. 7 illustrates a method for implementing a playback control function according to an example embodiment.
  • FIG. 8 illustrates a simplified schematic view of an example client device according to an example embodiment.
  • DETAILED DESCRIPTION
  • Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be apparent to one of ordinary skill in the art that the present invention may be practiced without these specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to unnecessarily obscure aspects of the embodiments. Wherever possible, like reference numbers will be used for like elements.
  • Systems and methods for enabling playback control functions of a media player are disclosed. For example, a user of a client device receiving a video stream may perform rewind and fast forward control functions. The client device may implement these playback control functions by retrieving a dedicated type of segment to support these functions. Using the packed I-frame segment, the media player may display frames selected at a predetermined interval while maintaining a visual cadence that is pleasing to a viewer. In addition, the use of a packed I-frame segment alleviates transmission problems associated with existing communication systems.
  • In the described embodiments, a client device may, during a normal playback mode, retrieve from a media source a plurality of transmission segments that contain coded video data of the media asset, the coded video data representing frames of asset content at a first temporal spacing, during an enhanced playback mode, retrieve from the media source at least one transmission segment that contains coded video data of the media asset, the coded video data representing frames of asset content at a second temporal spacing larger than the first, wherein all frames of the transmission segment are coded by intra-coding.
  • FIG. 1 is a simplified block diagram of an example media distribution system 100 suitable for use with the present invention.
  • The system 100 may include a distribution server 110 and a client device 120 connected via a communication network 130. The distribution server 110 may include a storage system 115 that may store a variety of media streams str1-str3 (e.g., music, movies, television shows, advertisements, etc.) for download by the client device 120. The distribution server 115 may transmit media streams str1-str3 to the client device 120 via the network 130 in response to client requests. For example, the streaming media may be pre-stored at distribution server 115, as media assets 117. In another example, “live streamed” data may be stored at distribution server 115 on a real-time basis.
  • One or more media assets 117 may be stored, within the storage system 115. Media assets 117 may be transmitted as a plurality of streams (e.g., str1-str3), each stream having a plurality of coded media segments (e.g., segments 1.1-1.n, 2.1-2.n, 3.1-3.n). Each media stream (e.g., str1) may be coded at a respective bitrate, frame rate, and/or frame size. In the illustrated embodiment, each of the plurality of segments 1.1-1.n, 2.1-2.n, 3.1-3.n may include media content of a predetermined duration (e.g., six seconds). In some instances, the distribution server 110 may store multiple copies of a particular media stream, especially video media streams. The example illustrated in FIG. 1 depicts three coded video streams str1-str3 that are coded at varying bitrates (e.g., 4 Mb/s, 2 Mb/s and 500 Kb/s).
  • In addition, each of the plurality of segments 1.1-1.n, 2.1-2.n, 3.1-3.n may contain frames coded by a predetermined protocol. For example, video data in each segment 1.1-1.n, 2.1-2.n, 3.1-3.n may be coded according to ITU-T H.265 (commonly “HEVC”), H.264, H.263 or other standard or proprietary protocols. Coded video data typically is coded according to predictive coding techniques that exploit spatial and/or temporal redundancy in a source video sequence. Accordingly, frames of a source video sequence may be coded according to intra-prediction techniques (I-coding) or inter-prediction techniques (often, P- and/or B-coding). I-coded frames may be decoded without reference to other frames in the coded video data but P-coded and B-coded frames cannot be decoded without reference to other frames. P- and B-coding often achieves lower bit rates for individual frames and, therefore, coded video data often has a higher percentage of P- and B-coded frames than I-coded frames. For convenience, these segments are called “normal playback segments” herein.
  • Embodiments of the present invention may provide, within a distribution server 110, a type of segment to support enhanced playback control functions, called enhanced transmission or “packed I-frame” segments, for convenience. An enhanced transmission segment may store I-coded frames of a media stream spaced at temporal intervals that correspond to the enhanced playback mode being supported. Thus, in an example where segments 1.1-1.n of a media stream str1 are coded at 30 frames per second with various distributions of I-, P- and B-coded frames, a packed I frame segment IS1 may support fast forward and rewind playback modes. In this example, packed I-frame segment IS1 consists of sampled I-frames originating from stream str1.
  • In some implementations, the normal playback segments, by convention, may include I-frames as the first coded frame in each segment and include other I-coded frames at regular intervals. For example, the distribution server 110 may operate according to a segmentation policy that requires two I-coded frames to appear in every segment and that those I-coded frames appear at regularly-spaced time intervals in display order.
  • Packed I-frame segments IS1-IS3 may be generated in a variety of ways. For example, I-frames may be copied from normal playback segments 1.1-1.n, 2.1-2.n, 3.1-3.n to the packed I-frame segments IS1-IS3, respectively. In this example, playback of I-frames is used to implement playback control functions. As a result, cache utilization inside communication network is improved and unpredictable latencies are reduced. In another example, normal playback segments 1.1-1.n, 2.1-2.n, 3.1-3.n may be re-encoded in order to produce I-frames more frequently. In this example, only frames being re-encoded as I-frames may be processed. By rendering I-frames at shorter intervals, playback control functions may be provided with a more pleasing cadence to the user. In addition, this approach avoids copying of I-frames from normal playback segments segment 1.1-1.n, 2.1-2.n, 3.1-3.n.
  • The distribution server 110 also may store a manifest file 116 that, for each media asset 117, provides configuration information for associated streams str1-str3 (e.g., bitrate, size, etc.) and, for each stream, identifies coded video segments and packed I-frame segments that correspond to the stream. The manifest file 116 may also correlate segments of coded video with corresponding segments having varying bitrates. Manifest file 116 may identify segments by a network location resource such as a URL.
  • The client device 120 may be any electronic device. The client device may include a media player adapted to download streaming media from the distribution server 110. The distribution server 110 may transmit media to the client device 120 via channel 140 and communication network 130. The client device 120 decodes the downloaded segments such that they may be rendered for playback.
  • Although the client device 120 is illustrated as a tablet computer in FIG. 1, client devices may be provided as a variety of computing platforms, including smartphones, personal computers, laptop computers, media players, other servers, and/or dedicated video conferencing equipment. The network 130 represents any number of networks that convey coded video data among the distribution server 110 and the client device 120, including, for example, wireline and/or wireless communication networks. A communication network 130 may exchange data in circuit-switched and/or packet-switched channels. Representative networks include telecommunications networks, local area networks, wide area networks and/or the Internet. For the purposes of the present discussion, the architecture and topology of the network 130 is immaterial to the operation of the present invention unless discussed hereinbelow.
  • FIG. 1 illustrates a simplified implementation of media distribution system 100. The example architecture depicted in FIG. 1 may be expanded to accommodate multiple distribution servers, edge caches associated with the distributions servers, content servers, coding servers, client devices, communications networks, etc. In some implementations, it is permissible to provide some servers as dedicated media coding servers and other servers as dedicated media transmission servers.
  • FIG. 2 illustrates example media player application hosted on a client device according to an example embodiment.
  • The media player application 200 may include program instructions that are executable by the processor to provide a user interface and media stream control functions to a user of the client device. The media player application 200 may include a media playback component 201, configured to provide the user interface and associated functionality of the media player that enables the user to select and perform playback control functions of media streams. The media player application 200 may also include and a media input/output (I/O) component 202, configured to provide media stream requests to and receive media streams from the media distribution server 110.
  • FIG. 3 illustrates a simplified user interface 300 of a media player that implements media stream playback control functions according to an example embodiment.
  • The user interface 300 can include a video display portion 301 to display a video or other visual components of a media stream, and playback control buttons 302 to receive media stream playback control inputs from the user, including one or more of a normal speed forward playback control, a stop playback control, a pause playback control, a rewind playback control, and a fast forward playback control.
  • FIG. 4 illustrates a video stream 400 suitable for use with the present invention. As shown in FIG. 4, the example video stream 400 may be comprised of a plurality of normal playback segments 410.1-410.n. Each of these segments 410.1-410.n may further include a plurality of frames 420.1-420.n. A variety of frame types are typically assembled to construct a segment, including I-frames (intra-coded), P-frames (predicted) and/or B-frames (bi-predictive).
  • During coding, a coding server (not shown) may assign to each frame a certain frame type, which can affect the coding techniques that are applied to the respective frame. For example, frames often are assigned as one of the following frame types. An I-frame generally operates as a standalone frame that is coded and decoded without using any other frame in the sequence as a source of prediction. As an I-Frame may contain information for rendering an entire image, an I-frame is typically larger than other frame types. By contrast, P-frames and B-frames generally include references to other frames. A P-frame is one that is coded and decoded using a single previously-coded frame in the sequence as a candidate source of prediction, and a B-frame is one that is coded and decoded using a pair of earlier-coded frames in the sequence as candidate sources of prediction.
  • The example media stream 400 may be utilized to perform a normal speed forward playback of video media. In addition, the example media stream 400 may also be utilized to perform playback control functions according to conventional techniques or according to embodiments of the present invention.
  • FIG. 5 illustrates the composition of an enhanced playback segment according to an example embodiment. As shown in FIG. 5, the example video stream 500 may be comprised of a plurality of segments 510.1-510.n. Each of these individual segments 510.1-510.n may further include a plurality of frames 520.1-520.n including I-, P-, and/or B-frames.
  • In addition, an enhanced playback segment 530 is composed of I-frames from normal playback segments 510.1-510.n. The enhanced playback segment 530 may be a standalone media segment that includes I-frames from normal playback segments 510.1-510.n. For example, an enhanced playback segment may be generated for a predetermined number of media segments. Alternatively, an enhanced playback segment may be generated to store a predetermined number of I-frames. In either example, multiple enhanced playback segments may be used to separately store I-frames of a stream 500.
  • Consider an example where, each of normal playback segments 510.1-510.n may include approximately 180 frames and include a few I-frames (e.g., 2 or 3) such that an I-frame is rendered every 2-4 seconds. As I-frames may be substantially larger than P-, and B-frames, each enhanced playback segment 530 may include fewer frames than normal playback segments 510.1-510.n. For example, an enhanced playback segment may include 45, 60, or 90 frames.
  • The use of an enhanced playback segment 530 may cause the rendering of an I-frame every half second to every eighth of a second or faster. For example, when implementing fast forward and rewind control functions at 16×, the rendering of eight I-frames per second produces a visual cadence that is pleasing to a viewer. In another example, when implementing fast forward and rewind control functions at 4×, two I-frames per second may be visually too fragmented for some viewers.
  • FIGS. 6A-6E illustrate various video frame display timelines during normal speed forward and other control playback modes of a media stream according to example embodiments.
  • FIG. 6A depicts an embodiment of a timeline of display of video frames by the media player program during a normal speed forward playback of a video component of a media stream that can be produced utilizing media segment files (e.g., 410.1-410.n or 510.1-510.n). In the depicted timeline, the normal speed playback includes displaying each frame I1, P1, B1, I2, P2, B2, I3, P3, B3, . . . of the video stream in order and at a predetermined rate of display. For example, each frame I1, P1, B1, I2, P2, B2, I3, P3, B3, . . . may be displayed for 1/30th of a second during normal playback mode.
  • FIGS. 6B-6E depict embodiments of a timeline of display of video frames by the media player program during other control playback modes that can be produced by utilizing enhanced playback segments (e.g., 530). During other control playback modes, frames may still be displayed for 1/30th of a second, but the interval between selected frames may be varied. For example, every I-frame or every 10th I-frame may be displayed.
  • FIG. 6B depicts an embodiment of a relatively slower speed fast forward playback mode that includes displaying each I-frame I1, I2, I3, I4 . . . of the video in a forward order. FIG. 6C depicts an embodiment of a relatively higher speed fast forward playback mode that includes displaying a subset of I-frames I10, I20, I30, I40 . . . of the video in a forward order. In the example of FIG. 6C, every 10th I-frame may be displayed. FIG. 6D depicts an embodiment of a relatively slower speed rewind playback mode that includes displaying only each I-frame I80, I79, I78, I77 . . . of the video stream in a reverse order. FIG. 6E depicts an embodiment of a relatively higher speed rewind playback mode that includes displaying a subset of the I-frames I80, I70, I60, I50 . . . of the video in a reverse order. In the example of FIG. 6E, every 10th I-frame may be displayed.
  • Using packed I-frames, the media streaming system can provide improved techniques for requesting video frames for control playback functions of a media player. The embodiments reduce or eliminate problems associated with conventional systems, such as delays associated variable or unpredictable transmission of numerous media segments in the communication network.
  • FIG. 7 illustrates a method 700 for implementing a playback control function according to an example embodiment.
  • At 701, a media asset may be rendered by a client device. Rendering generally includes buffering and decoding the coded segments of the media asset so that the resulting decoded segments may be rendered by a media player.
  • At 702, the client device may receive a user command indicating a playback control function, including one or more of a normal speed forward playback control, a stop playback control, a pause playback control, a rewind playback control, and a fast forward playback control.
  • Next, at 703, the media player determines whether the user-selected command includes one of a fast forward or rewind playback control. If so, the media player may request one or more packed I-frames that may be utilized to implement the user selected control command, at 704. Otherwise, the media player may request additional media segments if the user command denotes a normal speed forward playback control. Additional segments may not be needed if the user selects a stop or pause commands. Lastly, at 706, the media player implements the user-select control.
  • FIG. 8 is a simplified schematic view of a client device 800 according to an example embodiment.
  • Client device 800 may include a processing system and control circuitry 802, memory 804, communications circuitry 806, input/output components 808, and display assembly 810. The client device 800 may also include a bus 803 that may provide a data transfer path for transferring data and/or power, to, from, or between various other components of client device 800.
  • Processing system 802 may control the operation of components within client device 800. For example, processor 802 may receive input signals from an input component 808 and/or drive output signals to display assembly 810. In another example, processor 802 may execute instructions for one or more applications APP0-APPN, including media streaming applications, stored in memory 804.
  • Memory 804 stores the operating system OS of the client device 800 as well as one or more applications (APP0, APP1 . . . APPN). Included among the applications may be a streaming application service. Memory 804 may include one or more storage mediums, including for example, a hard-drive, flash memory, permanent memory such as read-only memory (“ROM”), semi-permanent memory such as random access memory (“RAM”), any other suitable type of storage component, or any combination thereof. Memory 804 may include cache memory, which may be one or more different types of memory used for temporarily storing data for electronic device applications. Memory 804 may store media streams (e.g., music, image, and video files), software, firmware, preference information (e.g., media playback preferences), wireless connection information, subscription information (e.g., information that tracks podcasts, television shows, or other media a user subscribes to), etc.
  • Communications circuitry 806 may be provided to enable the client device 800 to communicate with one or more other electronic devices or servers using any suitable communications protocol. For example, communications circuitry 806 may support Wi-Fi (e.g., an 802.11 protocol), Ethernet, Bluetooth, high frequency systems (e.g., 900 MHz, 2.4 GHz, and 5.6 GHz communication systems), infrared, transmission control protocol/internet protocol (“TCP/IP”), hypertext transfer protocol (“HTTP”), real-time transport protocol (“RTP”), real-time streaming protocol (“RTSP”), and other standardized or propriety communications protocols, or combinations thereof.
  • In some embodiments, one or more components of electronic device 800 may be combined or omitted. Moreover, electronic device 800 may include additional components not depicted in FIG. 8. Client device 800 may be any stationary or portable electronic device, including tablet computers, smartphones, laptop computers, personal computers, set-top boxes, wearable electronic devices, and other consumer electronic products designed to decode and render coded media streams.
  • Electronic device 800 may also include one or more output components 808 that may render information (e.g., audio and video) to a user of device 800. An output component of client device 800 may take various forms, including, but not limited, to audio speakers, headphones, visual displays via display assembly 810, etc.
  • Display assembly 810 may include any suitable type of display or interface for presenting visible information to a user of client device 800. In some embodiments, display assembly 810 may include an embedded or coupled display. Display assembly 810 may include, for example, a touch screen, a liquid crystal display (“LCD”), a light emitting diode (“LED”) display, an organic light-emitting diode (“OLED”) display, or any other suitable type of display.
  • The foregoing discussion has described operation of the embodiments of the present invention in the context of distribution servers, client devices, content servers, and coding servers. Commonly, these components are provided as electronic devices. They can be embodied in integrated circuits, such as application specific integrated circuits, field programmable gate arrays and/or digital signal processors. Alternatively, they can be embodied in computer programs that execute on personal computers, notebook computers or computer servers. Similarly, decoders can be embodied in integrated circuits, such as application specific integrated circuits, field programmable gate arrays and/or digital signal processors, or they can be embodied in computer programs that execute on personal computers, notebook computers or computer servers. Decoders commonly are packaged in consumer electronics devices, such as gaming systems, DVD players, portable media players and the like and they also can be packaged in consumer software applications such as video games, browser-based media players and the like. And, of course, these components may be provided as hybrid systems that distribute functionality across dedicated hardware components and programmed general purpose processors as desired.
  • It will be apparent to those skilled in the art that various modifications and variations can be made in the system and method for item to item transitions of the present invention without departing form the spirit or scope of the invention. Thus, it is intended that the present invention cover the modifications and variations of this invention provided they come within the scope of the appended claims and their equivalents.

Claims (31)

1. A method of rendering a media asset at a media player, comprising:
during a normal playback mode, retrieving from a media source a plurality of transmission segments that contain coded video data of the media asset, the coded video data representing frames of asset content at a first temporal spacing;
during an enhanced playback mode, retrieving from the media source at least one transmission segment that contains coded video data of the media asset, the coded video data representing frames of asset content at a second temporal spacing larger than the first, wherein all frames of the transmission segment are coded by intra-coding;
decoding the coded video data of the retrieved segments; and
rendering the decoded video data on a display of the media player.
2. (canceled)
3. (canceled)
4. The method of claim 1, wherein the first transmission segment downloaded during the enhanced playback mode is identified based on a portion of the media asset that was being rendered in normal playback mode when the enhanced playback mode was engaged.
5. The method of claim 1, further comprising
toggling between the normal playback mode and the enhanced playback mode in response to operator commands,
wherein, on each toggling between playback modes, a next transmission segment to be retrieved is identified based on a portion of the media asset that was being rendered when the toggling occurred.
6. The method of claim 1, wherein the second temporal spacing is at least double the first temporal spacing.
7. (canceled)
8. (canceled)
9. The method of claim 1, wherein all frames of the transmission segment are selected from the plurality of transmission segments.
10. The method of claim 1, wherein the transmission segment is generated by re-encoding the plurality of transmission segments using only intra-coding at a reduced frame rate.
11. A non-transitory computer readable medium storing a media streaming application having playback control functions, the media streaming application executable by at least one processing system, the media streaming application comprising instructions for:
during a normal playback mode, retrieving from a media source a plurality of transmission segments that contain coded video data of the media asset, the coded video data representing frames of asset content at a first temporal spacing;
during an enhanced playback mode, retrieving from the media source at least one transmission segment that contains coded video data of the media asset, the coded video data representing frames of asset content at a second temporal spacing larger than the first, wherein all frames of the transmission segment are coded by intra-coding;
decoding the coded video data of the retrieved segments; and
rendering the decoded video data on a display of the media player.
12. The non-transitory computer readable medium of claim 11, wherein the enhanced playback mode is initiated in response to a user command to enter a fast forward playback mode.
13. The non-transitory computer readable medium of claim 11, wherein the enhanced playback mode is initiated in response to a user command to enter a rewind playback mode.
14. The non-transitory computer readable medium of claim 11, wherein the first transmission segment downloaded during the enhanced playback mode is identified based on a portion of the media asset that was being rendered in normal playback mode when the enhanced playback mode was engaged.
15. The non-transitory computer readable medium of claim 11, further comprising
toggling between the normal playback mode and the enhanced playback mode in response to operator commands,
wherein, on each toggling between playback modes, a next transmission segment to be retrieved is identified based on a portion of the media asset that was being rendered when the toggling occurred.
16. (canceled)
17. (canceled)
18. (canceled)
19. The non-transitory computer readable medium of claim 11, wherein all frames of the transmission segment are selected from the plurality of transmission segments.
20. The non-transitory computer readable medium of claim 11, wherein the transmission segment is generated by re-encoding the plurality of transmission segments using only intra-coding at a reduced frame rate.
21. A electronic device comprising:
a processing system;
memory storing one or more programs for execution by the processing system, the one or more programs including instructions for:
during a normal playback mode, retrieving from a media source a plurality of transmission segments that contain coded video data of the media asset, the coded video data representing frames of asset content at a first temporal spacing;
during an enhanced playback mode, retrieving from the media source at least one transmission segment that contains coded video data of the media asset, the coded video data representing frames of asset content at a second temporal spacing larger than the first, wherein all frames of the transmission segment are coded by intra-coding;
decoding the coded video data of the retrieved segments; and
rendering the decoded video data on a display of the media player.
22. (canceled)
23. (canceled)
24. The electronic device of claim 21, wherein the first transmission segment downloaded during the enhanced playback mode is identified based on a portion of the media asset that was being rendered in normal playback mode when the enhanced playback mode was engaged.
25. The electronic device of claim 21, further comprising
toggling between the normal playback mode and the enhanced playback mode in response to operator commands,
wherein, on each toggling between playback modes, a next transmission segment to be retrieved is identified based on a portion of the media asset that was being rendered when the toggling occurred.
26. (canceled)
27. The electronic device of claim 21, wherein the second temporal spacing is at least four times the first temporal spacing.
28. The electronic device of claim 21, wherein the second temporal spacing is at least eight times the first temporal spacing.
29. The electronic device of claim 21, wherein all frames of the transmission segment are selected from the plurality of transmission segments.
30. The electronic device of claim 21, wherein the transmission segment is generated by re-encoding the plurality of transmission segments using only intra-coding at a reduced frame rate.
31. (canceled)
US14/501,941 2014-05-30 2014-09-30 Packed i-frames Pending US20150350622A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US201462005803P true 2014-05-30 2014-05-30
US14/501,941 US20150350622A1 (en) 2014-05-30 2014-09-30 Packed i-frames

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US14/501,941 US20150350622A1 (en) 2014-05-30 2014-09-30 Packed i-frames
PCT/US2015/032039 WO2015183695A1 (en) 2014-05-30 2015-05-21 Packed l-frames for playback control functions
US15/628,073 US20170289514A1 (en) 2014-05-30 2017-06-20 Packed I-Frames

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US15/628,073 Continuation US20170289514A1 (en) 2014-05-30 2017-06-20 Packed I-Frames

Publications (1)

Publication Number Publication Date
US20150350622A1 true US20150350622A1 (en) 2015-12-03

Family

ID=53404860

Family Applications (2)

Application Number Title Priority Date Filing Date
US14/501,941 Pending US20150350622A1 (en) 2014-05-30 2014-09-30 Packed i-frames
US15/628,073 Pending US20170289514A1 (en) 2014-05-30 2017-06-20 Packed I-Frames

Family Applications After (1)

Application Number Title Priority Date Filing Date
US15/628,073 Pending US20170289514A1 (en) 2014-05-30 2017-06-20 Packed I-Frames

Country Status (2)

Country Link
US (2) US20150350622A1 (en)
WO (1) WO2015183695A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170111666A1 (en) * 2015-10-20 2017-04-20 International Business Machines Corporation Video storage and video playing

Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6057832A (en) * 1997-12-02 2000-05-02 V Soft Ltd. Method and apparatus for video-on-demand with fast play capability
US6122433A (en) * 1994-10-20 2000-09-19 Thomson Licensing S.A. HDTV trick play stream derivation for VCR
US20020112247A1 (en) * 2001-02-09 2002-08-15 Horner David R. Method and system for creation, delivery, and presentation of time-synchronized multimedia presentations
US20030099457A1 (en) * 2000-02-10 2003-05-29 Toshiaki Takahashi Receiving terminal device and control method therefor
US20050155072A1 (en) * 2003-10-07 2005-07-14 Ucentric Holdings, Inc. Digital video recording and playback system with quality of service playback from multiple locations via a home area network
US20050185923A1 (en) * 2004-02-25 2005-08-25 Taisuke Tsurui Video/audio playback apparatus and video/audio playback method
US20060078294A1 (en) * 2004-09-28 2006-04-13 Kabushiki Kaisha Toshiba Image-reproducing apparatus and image-reproducing method
US7106798B1 (en) * 2002-05-23 2006-09-12 Microsoft Corporation Smooth scanning presenter
US20070058926A1 (en) * 2005-09-09 2007-03-15 Microsoft Corporation Optimizing trick modes for streaming media content
US20080317246A1 (en) * 2005-12-23 2008-12-25 Koninklijke Philips Electronics N.V. Device for and a Method of Processing Data Stream
US20090100496A1 (en) * 2006-04-24 2009-04-16 Andreas Bechtolsheim Media server system
US20120042090A1 (en) * 2010-08-10 2012-02-16 Qualcomm Incorporated Manifest file updates for network streaming of coded multimedia data
US20120251086A1 (en) * 2009-12-18 2012-10-04 Gregory William Cook Trick mode technique for a bandwidth limited channel
US20130185756A1 (en) * 2010-10-05 2013-07-18 Telefonaktiebolaget L M Ericsson (Publ) Client, a Content Creator Entity and Methods Thereof for Media Streaming
US8826346B1 (en) * 2013-03-15 2014-09-02 General Instrument Corporation Methods of implementing trickplay

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB0104785D0 (en) * 2001-02-27 2001-04-18 Pace Micro Tech Plc MPEG decoder video in the form of cue and/or review streams of data
JP2007511948A (en) * 2003-11-18 2007-05-10 コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ Play of the trick play signal
US8055783B2 (en) * 2005-08-22 2011-11-08 Utc Fire & Security Americas Corporation, Inc. Systems and methods for media stream processing
KR101750048B1 (en) * 2009-11-13 2017-07-03 삼성전자주식회사 Method and apparatus for providing trick play service
JP5709900B2 (en) * 2010-01-18 2015-04-30 テレフオンアクチーボラゲット エル エム エリクソン(パブル) A method and apparatus to support the playout of content
EP2534845B1 (en) * 2010-02-11 2019-05-29 EchoStar Technologies L.L.C. Systems and methods to provide trick play during streaming playback
US9510029B2 (en) * 2010-02-11 2016-11-29 Echostar Advanced Technologies L.L.C. Systems and methods to provide trick play during streaming playback
US20120030723A1 (en) * 2010-07-27 2012-02-02 Motorola, Inc. Method and apparatus for streaming video
WO2012167558A1 (en) * 2011-11-16 2012-12-13 华为技术有限公司 Method, terminal, server and system for playing media file
JP5752585B2 (en) * 2011-12-16 2015-07-22 株式会社東芝 Image processing apparatus, method and program
WO2014142716A1 (en) * 2013-03-13 2014-09-18 Telefonaktiebolaget L M Ericsson (Publ) Arrangements and method thereof for channel change during streaming
US20140280688A1 (en) * 2013-03-14 2014-09-18 Comcast Cable Communications, Llc Methods And Systems For Dynamic Data Management
US9094737B2 (en) * 2013-05-30 2015-07-28 Sonic Ip, Inc. Network video streaming with trick play based on separate trick play files
US9756369B2 (en) * 2013-06-19 2017-09-05 Telefonaktiebolaget Lm Ericsson (Publ) Method and apparatus for streaming media data segments of different lengths wherein the segment of different length comprising data not belonging to the actual segment and beginning with key frames or containing key frames only

Patent Citations (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6122433A (en) * 1994-10-20 2000-09-19 Thomson Licensing S.A. HDTV trick play stream derivation for VCR
US6057832A (en) * 1997-12-02 2000-05-02 V Soft Ltd. Method and apparatus for video-on-demand with fast play capability
US20030099457A1 (en) * 2000-02-10 2003-05-29 Toshiaki Takahashi Receiving terminal device and control method therefor
US20020112247A1 (en) * 2001-02-09 2002-08-15 Horner David R. Method and system for creation, delivery, and presentation of time-synchronized multimedia presentations
US7106798B1 (en) * 2002-05-23 2006-09-12 Microsoft Corporation Smooth scanning presenter
US20050155072A1 (en) * 2003-10-07 2005-07-14 Ucentric Holdings, Inc. Digital video recording and playback system with quality of service playback from multiple locations via a home area network
US20050185923A1 (en) * 2004-02-25 2005-08-25 Taisuke Tsurui Video/audio playback apparatus and video/audio playback method
US20060078294A1 (en) * 2004-09-28 2006-04-13 Kabushiki Kaisha Toshiba Image-reproducing apparatus and image-reproducing method
US20070058926A1 (en) * 2005-09-09 2007-03-15 Microsoft Corporation Optimizing trick modes for streaming media content
US20080317246A1 (en) * 2005-12-23 2008-12-25 Koninklijke Philips Electronics N.V. Device for and a Method of Processing Data Stream
US20090100496A1 (en) * 2006-04-24 2009-04-16 Andreas Bechtolsheim Media server system
US20120251086A1 (en) * 2009-12-18 2012-10-04 Gregory William Cook Trick mode technique for a bandwidth limited channel
US20120042090A1 (en) * 2010-08-10 2012-02-16 Qualcomm Incorporated Manifest file updates for network streaming of coded multimedia data
US20130185756A1 (en) * 2010-10-05 2013-07-18 Telefonaktiebolaget L M Ericsson (Publ) Client, a Content Creator Entity and Methods Thereof for Media Streaming
US8826346B1 (en) * 2013-03-15 2014-09-02 General Instrument Corporation Methods of implementing trickplay
US20140282760A1 (en) * 2013-03-15 2014-09-18 General Instrument Corporation Methods of implementing trickplay

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170111666A1 (en) * 2015-10-20 2017-04-20 International Business Machines Corporation Video storage and video playing

Also Published As

Publication number Publication date
WO2015183695A1 (en) 2015-12-03
US20170289514A1 (en) 2017-10-05

Similar Documents

Publication Publication Date Title
US20130290444A1 (en) Connected multi-screen social media application
US8677428B2 (en) System and method for rule based dynamic server side streaming manifest files
JP5201512B2 (en) Hybrid method for delivering "streaming" media in the home
CN104871551B (en) Adaptive bit rate in the trick play video content delivery embodiment disabled
US7802006B2 (en) Multi-location buffering of streaming media data
US8862763B2 (en) Downloading video using excess bandwidth
KR20070061768A (en) Apparatus, system, and method for adaptive-rate shifting of streaming content
US9203816B2 (en) Controlling access to copies of media content by a client device
US20100303440A1 (en) Method and apparatus for simultaneously playing a media program and an arbitrarily chosen seek preview frame
US10225299B2 (en) Systems, methods, and media for controlling delivery of content
US10075744B2 (en) Systems and methods to provide trick play during streaming playback
US9721254B2 (en) Method and apparatus for providing streaming media programs and targeted advertisements using multiple advertisement version segments
US9083640B2 (en) Adaptive bit rate switching
US9571827B2 (en) Techniques for adaptive video streaming
US20180041787A1 (en) Method and apparatus for streaming multimedia data with access point positioning information
EP2517121B1 (en) Audio splitting with codec-enforced frame sizes
US8438484B2 (en) Video preview module to enhance online video experience
CN102187667B (en) A method of switching from a first encoded video stream to a second encoded video stream
US20140123162A1 (en) Eye tracking based defocusing
US20170155912A1 (en) Hevc-tiled video streaming
JP5179508B2 (en) Data stream reproducing method and a data stream reproducing apparatus for distributing and switching content between the data streams to each other
US9317188B2 (en) Devices and methods for providing navigation images associated with adaptive bit rate video content
EP2952007B1 (en) User control of replacement television advertisements inserted by a smart television
US9191725B2 (en) Method and apparatus for streaming video
US8918821B2 (en) Client device with video playlist translation via client-side proxy and methods for use therewith

Legal Events

Date Code Title Description
AS Assignment

Owner name: APPLE INC., CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:PANTOS, ROGER N.;LI, ZHENHENG;REEL/FRAME:033852/0200

Effective date: 20140924

STCV Information on status: appeal procedure

Free format text: ON APPEAL -- AWAITING DECISION BY THE BOARD OF APPEALS