Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L12/00—Data switching networks
  • H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
  • H04L12/2801—Broadband local area networks
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L12/00—Data switching networks
  • H04L12/54—Store-and-forward switching systems 
  • H04L12/56—Packet switching systems
  • H04L12/5691—Access to open networks; Ingress point selection, e.g. ISP selection
  • H04L12/5692—Selection among different networks
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
  • H04L65/60—Network streaming of media packets
  • H04L65/61—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio
  • H04L65/612—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio for unicast
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
  • H04L65/60—Network streaming of media packets
  • H04L65/61—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio
  • H04L65/613—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio for the control of the source by the destination
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
  • H04L65/60—Network streaming of media packets
  • H04L65/65—Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
  • H04L65/60—Network streaming of media packets
  • H04L65/70—Media network packetisation
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
  • H04L65/60—Network streaming of media packets
  • H04L65/75—Media network packet handling
  • H04L65/752—Media network packet handling adapting media to network capabilities
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L67/00—Network arrangements or protocols for supporting network services or applications
  • H04L67/50—Network services
  • H04L67/56—Provisioning of proxy services
  • H04L67/561—Adding application-functional data or data for application control, e.g. adding metadata
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L67/00—Network arrangements or protocols for supporting network services or applications
  • H04L67/50—Network services
  • H04L67/56—Provisioning of proxy services
  • H04L67/565—Conversion or adaptation of application format or content
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
  • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
  • H04N21/231—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
  • H04N21/23106—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion involving caching operations
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
  • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
  • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
  • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
  • H04N21/234309—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 or from Quicktime to Realvideo
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
  • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
  • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
  • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
  • H04N21/234381—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
  • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
  • H04N21/238—Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams
  • H04N21/2381—Adapting the multiplex stream to a specific network, e.g. an Internet Protocol [IP] network
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
  • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
  • H04N21/258—Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data
  • H04N21/25866—Management of end-user data
  • H04N21/25875—Management of end-user data involving end-user authentication
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
  • H04N21/27—Server based end-user applications
  • H04N21/274—Storing end-user multimedia data in response to end-user request, e.g. network recorder
  • H04N21/2743—Video hosting of uploaded data from client
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
  • H04N21/47—End-user applications
  • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
  • H04N21/47202—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for requesting content on demand, e.g. video on demand
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
  • H04N21/47—End-user applications
  • H04N21/475—End-user interface for inputting end-user data, e.g. personal identification number [PIN], preference data
  • H04N21/4753—End-user interface for inputting end-user data, e.g. personal identification number [PIN], preference data for user identification, e.g. by entering a PIN or password
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
  • H04N21/61—Network physical structure; Signal processing
  • H04N21/6106—Network physical structure; Signal processing specially adapted to the downstream path of the transmission network
  • H04N21/6125—Network physical structure; Signal processing specially adapted to the downstream path of the transmission network involving transmission via Internet
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
  • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
  • H04N21/835—Generation of protective data, e.g. certificates
  • H04N21/8355—Generation of protective data, e.g. certificates involving usage data, e.g. number of copies or viewings allowed
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
  • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
  • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
  • H04N21/845—Structuring of content, e.g. decomposing content into time segments
  • H04N21/8455—Structuring of content, e.g. decomposing content into time segments involving pointers to the content, e.g. pointers to the I-frames of the video stream
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N7/00—Television systems
  • H04N7/16—Analogue secrecy systems; Analogue subscription systems
  • H04N7/173—Analogue secrecy systems; Analogue subscription systems with two-way working, e.g. subscriber sending a programme selection signal
  • H04N7/17309—Transmission or handling of upstream communications
  • H04N7/17336—Handling of requests in head-ends

Definitions

• the invention relates to electronic storage and transmission of content. More particularly, the invention relates to a method and apparatus tor preprocessing and postprocessing content in an interactive information distribution system.
• VOD video on demand
• Information systems such as video on demand (VOD) systems are capable of streaming program content to a great number of users or subscribers.
• VOD video on demand
• the VOD system retrieves the requested program content from a video server, streams the content ovei a stream distribution network, and converts the content to an access network that is coupled to a particular neighborhood of subscriber terminals. The user then views the requested program content at the subscriber terminal.
• the different types of access networks have different limitations with respect to transmission latency, bandwidth, and the like.
• the VOD systems currently implement different solutions for each type of access network.
• VOD systems that provide web-based video content must account for a public and private wide area networks that support content of a particular quality of service (QoS), typically medium latency, low bandwidth and poor quality video, e.g high jitter.
• QoS quality of service
• VOD systems that provide cable-based video must account for cable networks that support low latency, high bandwidth and high quality video.
• the invention piovides a method and system for preprocessing content for a stream caching server in an interactive information distribution system.
• the method initially retrieves of content in a subscriber terminal.
• the retrieved content is encapsulated in accordance to an Internet Protocol (IP) format used for streaming content to various access netwoi ks
• IP Internet Protocol
• the encapsulated content is then uploaded for storage in a stream caching server and for future streaming of content to different types of access networks.
• the present invention preprocesses content into a common format suitable for a stream caching server capable of transmitting content to different types of access networks.
• a user executes an applet program to prepiocess content stored in a computer terminal
• an applet program to prepiocess content stored in a computer terminal
• the invention also postprocesses content into a format supported by a particular type of player and access network used to receive the content from the sti earn caching server.
• FIG. 1 depicts a high level block diagram of a first portion of an interactive information distribution system embodied in the present invention
• FIG. 2A depicts one embodiment of an Internet Protocol (IP) packet used in the information distribution system of FIG. 1;
• FIG. 2B depicts one embodiment of a Realtime Transport Packet (RTP) contained in a payload section of the IP packet of FIG. 2A;
• IP Internet Protocol
• RTP Realtime Transport Packet
• FIG. 3 depicts a data structure useful in understanding an embodiment of the present invention
• FIG. 4 depicts a flow diagram of a method for implementing the preprocessing ol program content in accordance to one embodiment of the present invention.
• FIG. 5 depicts a flow diagram of a method for and postprocessing content in accordance to another embodiment of the present invention.
• FIG 1 depicts a high level block diagram of an interactive information distribution system 100.
• One application of the distribution system 100 is as a video of demand (VOD) system, as described in U.S. Patent Application No. 08/984,710, filed December 3, 1997 and incorporated herein by reference.
• VOD video of demand
• a user may request and receive a particular content selection, e.g., video, movie or programming content, Irom a service provider without any time restrictions associated with normal cable progiamming
• the information distribution system 100 comprises a stream caching server 102, a stream distribution network 104, at least one access network and at least one subscriber terminal.
• the stream caching server 102 receives, stores and streams content in accordance to an Internet Protocol (IP).
• IP Internet Protocol
• One example of such a stream caching server 102 is disclosed in simultaneously filed U.S. Apphcation . attorney docket DIVA 253, entitled
• the content is configured within a payload portion of each IP packet received, stored and streamed by the stream caching server 102.
• the use of this IP formatted content enables a single stream caching server 102 to stream content via an integrated stream distribution network 104 to different types of access networks.
• the system 100 is capable of streaming the same content to any cable service subscriber or any person using the Internet.
• the stream caching server 102 may receive content that is preprocessed at, for example, a remotely located subscriber terminal.
• One such subscriber terminal is a computer terminal 116 comprising a processor 150, a storage medium 152, a random access memory (RAM) 154 and support circuits 156
• the RAM 154 stores an applet 154 that is downloaded from, for example, a HTTP server 148 coupled to an access network, for example., a private local area netwoi k (LAN) oi wide area network (WAN) 106
• the processor 150 executes the applet 154 to initiate the preprocessing in the present invention.
• the storage medium 152 stores the content to be preprocessed.
• the support circuits 156 provide an interface for receiving the applet 154 from the http server 148, receiving content from the streaming cache server 102, or uploading preprocessed content to the http server 148.
• Possible configurations ot the applet 154 include a JAVA Applet Plug In for an Internet biowser, or a sof tware program written in a particular programming language, e.g., C++.
• the processor 150 executes the applet 154, the content is retrieved from the storage medium 152.
• the content may include standard multimedia files in a variety of formats, e.g., AVI (Audio Video Interleaved), Moving JPEG. MPEG- 1 , MPEG-2, MPEG- 4, MP3, Quicktime, and the like.
• the retrieved content may be transcoded into a format supported by a viewer oi subscriber terminal that eventually receives the downstream content
• the tianscoding ol content changes the format and rate of the retrieved content.
• One example of such transcoding is the conversion of MPEG-2 content into MPEG-4 content that can be played on a graphic processor in set top terminals or personal computer (PC) terminals.
• Other types of content may be transcoded into MPEG-2 content playable on conventional set top terminals
• Some transcoding requires decoding to baseband followed by encoding according to the desired format Some transcoding may be performed without baseband decoding
• the content, whether transcoded or not, is then encapsulated into a format that is optimal for the stream caching server 102.
• the content is encapsulated in a payload of each Realtime Transfer Protocol (RTP) packet contained in the payload of an IP packet.
• RTP Realtime Transfer Protocol
• the format of such encapsulated content is shown in FIGS. 2A-2B.
• the use of the RTP packet also supports non real time applications.
• FIG. 2A depicts one embodiment of an Internet Protocol (IP) packet 200 used in the present invention.
• the IP packet comprises an IP header 210 and an IP payload 320.
• the IP payload comprises a UDP (User Datagram Protocol) packet 221 comprising a UDP header 222 and a UDP payload 224.
• a RTP packet 230, a stream integrity check 226 and a cyclic redundancy check (CRC) 228 is illustratively contained in the UDP payload 224.
• the IP header 210 is 20 bytes
• the UDP header 222 is 8 bytes
• the stream integrity check field 226 is 4 bytes
• the CRC field 228 is 4 bytes.
• RTP Realtime Transport Packet
• the RTP packet 230 comprises a RTP header 240 and a RTP payload 250.
• Five MPEG-2 packets 352, 354, 356, 358 and 360 are illustratively contained in each RTP payload 250.
• the number of MPEG-2 packets in the RPT payload 250 corresponds to the buffer space in the Fibre Channel controller in the packet processor 144.
• the transcoding may also include compression of the IP header 310, the UDP header 322 and the RTP header 340.
• the compression of these headers 310, 322 and 340 optimizes the storage of content on the storage medium 146 of the stream caching server 102. Additionally, the transcoding may include encryption of the content. Compression of the IP header 310 may include compression ol non-address fields, and deletion of source and destination IP addresses. The IP source address is subsequently decompressed (at the packet processor 144) as the IP address at the output interface of the stream caching server 102 The IP destination address is assigned prior to streaming and is based upon the data link converter 1 12. 1 18 or 126 or edge device for the target access network 106, 108 or 1 10 The compression of the UDP headei 322 may delete source and destination port numbers in the storage medium 146.
• New values are then assigned to the source and destination port numbers prior lo breaming of content over the stream distribution network 104.
• the source port number is assigned a unique stream number with IP addresses.
• the destination port number is assigned a unique target stream number within the data link converter 112, 118 or 126 or the target edge device.
• the content is then uploaded to the http server 148 via the modem 1 14 and the LAN/WAN 106.
• the HTTP server 148 provides a user interface, e.g., a HTML (HyperText Markup Language) page, for the user to upload the content to the stream caching server 102.
• the http server 148 also transmits the li anscoded content upstream to the data link converter 1 12 via the LAN/WAN 106.
• the data link converter 112 modulates the encapsulated content for upstream transmission over the distribution network 104 for storage in the stream caching server 102.
• the preprocessing of the present invention is not limited to a computer terminal 1 16 uploading content over a LAN/WAN 106.
• the encapsulation may also occur in the computer terminal 1 16 prior to uploading to content to the http server 148.
• the preprocessing may be initiated from a computer terminal 122 or digital video recorder 124 over a carrier network 108, e.g., a T-l or T-3 line.
• a user of any computer terminal 1 16 and 122 author multimedia content over a network, e.g., Internet, and store the content in a virtual video shelf at the stream caching server 102 for playback by other users.
• the preprocessing is also applicable to content from a content provider such as a movie manufacturer.
• the preprocessing may also include the creation ot metadata once the processor 150 executes the applet 154.
• the metadata generally contains a variety of information about the content to be stored on the stream caching server 102 and streamed to a viewer or subscriber terminal.
• One embodiment of the metadata is in the form of a data structure that is prepended prior to a file associated with the content.
• the metadata may comprise many diffeient types of information used by the stream caching server 102 in steaming the content to a viewer device or subscriber terminal
• Metadata One type ot metadata that identify the content include title, author, screenwriter, actors, length of play, timing information, play rate, e.g., constant bit rate (CBR) or variable bit rate (VBR), genre of content, size of content, and the like.
• Other forms of metadata indicate the type of content and the type of player used to play the content.
• Illustrative types of content include AVI, MPEG-1, MPEG-2, MPEG-4, MP3, Quicktime, and Moving JPEG.
• the metadata may also include MPEG7 structure including scene descriptions and indices.
• Exemplary types of player devices include MPEG- 1 player, MPEG-2 player, MPEG-4 player, Microsoft Media Player, Real Video / Real Audio Player, and QuickTime Playei
• the metadata may include pricing information and restrictions to view the content from the server 102.
• a user or service provider may preset the price and access restrictions that are required to view the preprocessed content from the server 102.
• Pricing information include a price to view the content, applicable discounts associated with viewing the content, and applicable package deals when ordering the preprocessed content with other content selections.
• Access restrictions include rating of content, viewing window information and sales window information.
• the viewing window is a graphical interface that requires a subscriber to enter a correct password to receive the content from the servei 102.
• the sales window is a graphical interface that requires a subscriber to pay for viewing a particular content selection.
• the metadata comprises indexing at IP packet boundaries, e.g., Group of Pictures (GOP) boundaries and frame boundaries.
• This indexing enables the stream caching server 102 in responding to an interactive VCR like commands, e.g., fast forward (FF), rewind (REW), pause, stop, bookmark, and return to place
• FF fast forward
• REW rewind
• pause stop
• bookmark return to place
• the indexing lnloi malion supports random frame access at a content stream granularity, separate FF and REW tracks, random frame access of FF and REW tracks, pause/play, bookmarks lor each active subscriber, and DVD scene selection.
• the metadata may also include markers for changes in variable bit rate (VBR) and statistical multiplexing.
• VBR variable bit rate
• the indexing enables the use of MPEG-7 based descriptors for indexed access of content.
• the indexing of content can be on a per fia e basis oi a pei GOP basis
• MPEG-7 based descriptors include a length of descriptor, i.e., from a start frame to an end frame, and a schema for a database of indices
• the database is a hierarchical based database that allows for hierarchical scene description. For example, a root index may represent a scene of Pans, while a branch index may represent a scene of the Eiffel Tower.
• the indexing is also used for stream creation.
• the stream is created in realtime from a single MPEG-2 or MPEG-4 content stream, e.g., a start GOP to an end GOP, or a start frame to an end frame.
• the stream is created in non real time and the modified stream file is stored.
• the stream is transcoded or re-encoded such that a reference frame or I-frame is forced to be the frame with information desired in an index, even if the frame arrived in the packet processor 144 as a predictive frame, e.g., B-frame or P-frame Stream indexing will also be discussed below with respect to FIG. 3
• the stream caching server 102 may store and stream the preprocessed content, or alternatively stream the content in real time
• the streaming of content encapsulated in IP format enables the stream caching server 102 to stream content to subscribers via different types of access netwoi ks 106, 108 and 1 10.
• only one stream caching server 102 and one distribution network 104 is required to piovide scalable streaming
• one stream caching server 102 may stream content to the LAN/WAN 106, the carrier network 108, the cable network 1 10 and any other access network that supports IP. This greatly reduces the hardware cost at the head end 138, as the prior art requires a streaming caching server 102 and a distribution network 104 for each type of access network 106, 108 and 1 10
• the server 102 may stream content to a private Local Area Netwoi k (LAN) or Wide Area Netwoi k (WAN) 106.
• the system 100 may stream content through the stream distribution network 104 to a digital link converter 1 12, the LAN or WAN 106, a modem 1 14 and a display device coupled to a computer terminal 1 16 If a user decides to request content from the server 102 or uplink content, e.g., a home movie, to the server 102, that request or content would travel upstream in a path reverse to that of the downstreamed content.
• uplink content e.g., a home movie
• the digital link converter 1 12 modulates the program content for transmission via the private LAN/WAN 106 Additionally, the data link converter 1 12 extracts a MPEG formatted program content from the RTP formatted stream fiom the stream distribution network 104 and transcodes the piogram content into a format that is supported by the LAN/WAN 106.
• One example of the data link converter 112 is a DIVA Digital Link (DDL 500) that performs quadrature amplitude modulation (QAM) on the downstream program content.
• the LAN or WAN 106 is a private network provided by a private party or an Internet Service Provider (ISP).
• the modem 1 14 demodulates video content for viewing on the computer terminal 116.
• the server 102 may also stream content via the stream distribution network 104 to a data link converter 118, a carrier network 108, a digital subscriber line access multiplexer (DSLAM) 119, an x-DSL modem 120 to either a computer terminal 122 or a digital ⁇ ideo recorder (DVR) 124.
• a request for content or upload of content would travel in the reverse path taken by the downstream content.
• the carrier network 108 may include a T- l or T-3 transmission link.
• the data link converter 118 multiplexes the downstream content for transmission via the carrier network 108.
• the data link converter 1 18 may extract MPEG packets from the IP formatted stream from the stream distribution network 104.
• the DSLAM 1 19 demultiplexes the downstream content to a particular xDSL modem 120.
• the xDSL modem 120 demodulates the content for viewing on a computer terminal 122 or a display device (not shown) coupled to the DVR 124.
• the xDSL modem 120 may comprise a ADSL (asynchronous digital subscriber line) modem, a VDSL (very high data rate digital subscriber line), and the like.
• the server 100 may also stream content (or program content selection) via the stream distribution network 104 to a data hnk converter 126, the cable network 1 10 to either a set top terminal 128 or a cable modem that is coupled to a computer terminal 1 0 or a DVR 132.
• the data link converter 126 operates in a similar mannei to the digital link converter 112 except to format the content for transmission in the cable network 1 10.
• the content is transmitted from the cable network 110 to a set top terminal 128 or a cable modem 130 that demodulates the program content for viewing on a computer terminal 132 or a display device coupled to the DVR 134.
• a request from a cable subscriber or user is processed via the cable network 1 10, the OOB (out of band) router 136 and the modulator 126 that modulates the request back to the stream distribution network 104.
• the system 100 may also stream content to other types of access networks.
• the system 100 may also stream program content to satellite anc terrestrial networks.
• each system 100 actually streams content over rr.any more access networks and subscriber terminals than the example shown in FIG. 1.
• the stream caching server 102 is located at the local head end 138 with an infrastructure system manager 140, a switch 142 and a packet processor 144.
• the stream caching server 102 comprises a storage medium 146 to store the content preprocessed in accordance to the present invention.
• the infrastructure system managei 140 coordinates a (user) request from the subscriber terminal by passing the request to the stream caching server 102 and establishing a session between the subscriber terminal and the stream caching server 102.
• An exemplary infrastructure system manager 140 is the DIVA System Manager (DSM). As disclosed in U.S. Application , Attorney docket DIVA 256, entitled “Method and Apparatus for Managing an Integrated Information Distribution System", which is fully incorporated by reference in its entirety.
• the switch 142 routes the user request from the stream distribution network 104 to the system manager 140.
• the switch 142 routes the retrieved content from the stream caching server 102 to the packet processor 144.
• the storage medium 148 stores the preprocessed content in an IP format.
• the content is configured as a plurality of MPEG, e.g , MPEG-2 or MPEG-4, packets contained in a payload of a RTP packet within an IP packet
• the payload of each RTP packet may contain five MPEG-2 packets.
• the structure of the IP packet is shown to FIG 3B.
• the RTP format (RFC 1889) minimizes the latency in streaming content from the server, by supporting the streaming of content in real time.
• the content in the IP packet can be configured to have a minimal Quality of Service (QoS), e.g., data latency.
• QoS Quality of Service
• the packet processor 144 postprocesses the content into a format supported by a particular type of player and access network 106, 108 and 1 10 used to receive the content from the stream caching server 102
• a playei is either a software module downloaded from a HTTP server 1 16 to a computer terminal 122, a hardware module coupled to a subscriber terminal, or a card inserted into a subscriber terminal.
• Exemplary players include a MPEG-1 player, a MPEG-2 player, a MPEG-4 player, a Microsoft Media Player, a Real Video/Real Audio Player, a QuickTime Player, a Wireless Device Video or Audio Player, and the like.
• the packet processor 144 transcodes the content is performed without disturbing the
• the packet processor 144 separates the content, e.g., MPEG-2 packets, and header information in the IP packet, transcodes the content packets into a desired format supported by the access network and downstream player, and combines the transcoded packets with the header information to recreate the IP packet.
• Such transcoding is performed at an elementary packet level for transmitting at the transport packet level. Additional functions performed by the packet processor 144 include jitter correction, creating of a PES (packet elementary stream), stream splicing, and statistical multiplexing
• the transcoding includes the conversion content in the RTP payload into a format suitable lor the access network 106, 108 and 1 10, but the transcoded content is still encapsulated in the IP packet stream.
• Such transcoding may change the format and rate of the content.
• the transcoding may include the conversion of MPEG-2 formatted content into MPEG-1 , MPEG-4, AVI, Moving JPEG, MP3, Quic time, Wireless Applications Protocol content, and the like.
• the transcoding is performed in accordance to an extended Real Time Streaming Protocol (RTSP - RFC 2326) such that stream manipulations conform to Internet standards and are applicable to any access networks that support IP.
• RTSP - RFC 2326 Real Time Streaming Protocol
• the packet processor 144 may perform statistical multiplexing to dynamically allocate the amount of available bandwidth for streaming content to a particular viewer To perform such statistical multiplexing, the packet processoi 144 may stream content at either a constant bit rates or variable bit rates.
• the transcoding is also ad * ustable to bandwidth degradations
• the transcoding may include lossy filtering within frames ol content, dropping frames of content, e.g, resulting in a playback rate of 30 frames per second to 15 frames per second, and delivering still frames that contain important information.
• the transcoding may include dropping MPEG null packets, and transcoding or re-encoding content to an acceptable quality.
• the packet processor 144 may automatically perform such transcoding, or perform transcoding in accordance to user configured preferences. These preferences may include choices for a particular player, e.g., formatting, play rates, and type of conversion or transcoding For example, if the player is embodied m software in a PC, then the content is transcoded into MPEG-2 format However, it the player is a hand held device, then the content is transcoded into JPEG or MPEG-4 format. Additionally, the transcoding may be dynamically performed based on a user preference profile. Such a profile is based either on history or a default preference. For example, if the player is in the PC, the packet processor 144 transcodes the content into MPEG-2 at 4 Mbps and a constant bit rate.
• the content in the payload of each RTP packet is sized to minimize the latencies in streaming content from the stream caching server 102 to the distribution netwoi k 104.
• the read block for the packet processor 144 is sized to the MPEG packets in the payload of each RTP packet.
• the number of MPEG packets in each RTP packet is constrained by an available buff er space in a Fibre Channel controller that is used to read the content.
• the content streamed by the stream caching server 102 is not limited to content previously stored in the storage medium 148
• the stream caching servei 102 streams content from anothei lemotely located server, i.e., a server located at a remote headend Such a configuration is further described with respect to FIG. 2.
• the manager 140 provides session management tor streaming content in accordance to the RTP Control Protocol (RTCP). Such management is particularly important in the case of content streamed to the local stream caching server 102 from the remote server. If any errors occurred during the streaming from the remote server, these errors are multiplied when the cached or stored content is then streamed to the many subscribers RTCP enables the detection and transmission of only the read blocks affected by the streaming errors.
• RTCP RTP Control Protocol
• FIG 2 depicts another portion of the interactive information distribution system 100 of FIG. 1.
• This portion of the system 100 comprises the stream caching server 102 and the infrastructure system manager 140 at the local head end 138, a stream caching server 202 and an infrastructure system manager 204 at a remote head end 206, and a backbone streaming network 210.
• the stream caching server 202 and the infrastructure system manager 204 at the remote head end 206 operates in a similar manner to the respective stream caching server 102 and the infrastructure system manager 140 at the local head end 138 that were previously described.
• the local infrastructure system manager 140 receives a request for a particular content selection and determines whether a user requested content selection is stored in the storage medium 148. If the request content is not in the storage medium 148, the local infrastructure system manager 140 identifies a remote stream caching server 202 that stores the requested program content and provides a (server) request to the remote system manager 204. For example, a local system manager 140 in San Francisco may request content from another remote remotely located server 202 in Boston.
• FIG. 4 depicts a flow diagram of a method 400 for implementing the preprocessing of content e.g., a video program, in accordance to one embodiment of the present invention.
• the method 400 assumes that a user or subscriber has already downloaded an applet 154 from a http server 1 15.
• the method 400 starts at step 402 and proceeds to step 404 where preprocessing is initiated when the applet 154 is executed by the processor 150 in the computer terminal 1 16.
• a query determines whether a user has purchased shelf space. Namely, step 406 determines whether the user has purchased storage space or use of a portion of the storage medium 146 at the stream caching server 102. Step 406 may also cover situations where a user has access to shelf space.
• the method 400 proceeds to end at step 424. If the user has already purchased shelf space on the storage medium 148, the method 400 proceeds to step 408, where content is loaded from a memory 152 of the computer erminal 1 16
• the content may comprise any multimedia presentation, e.g., a home mac e move, created by the user.
• the method creates metadata for the loaded content.
• the metadata contains indexing information that enables the stream caching server 102 to respond to user interactivity commands within a group of pictures or approximately one half a second. Examples of user interactivity commands include fast forward (FF), rewind (REW), pause, stop, bookmark and return to place. Additionally, the metadata includes information that enables the stream caching server 102 to determine the type of file and the resolution of the content.
• the method 400 proceeds to step 412 where a query determines whether to transcode content. If no transcoding is required, the method 400 proceeds to step 416
• Step 414 may convert both the f ormat and bit late of the program content.
• the content may be transcoded (at a elementary stream level) from MPEG-1 , MP3, AVI, QuickTime or Moving formed into MPEG-2 formatted packets.
• the method 400 encapsulates the transcoded content, e.g.. MPEG-2 format, into an IP packet format (at the transport stream level). As previously described with respect to FIGS. 2A and 2B, storage of content in this IP packet format minimizes the retrieval time when the stored content is retrieved from the storage medium 146 and stream to the distribution network 104.
• the method 400 proceeds to step 418, where the transcoded content and metadata is uploaded from the computer terminal 1 16 and into the storage medium 148 of the sti earn caching server 102.
• the method 400 enables the user (sender ot the content) to establish or set permissions at step 420. This step establishes a subset of users who may access the content from the stream caching server 102.
• the method 400 proceeds to step 422, where a HTML page is established at the http server 1 15 for access by other users.
• the method 400 ends a step 424.
• FIGs. 5A and 5B depict a flow diagram of a method for accessing program content that was preprocessed and stored at the stream caching server 102.
• a user may access the full version of the program content only after correctly entering the password and paying to view the content.
• the method 500 starts at step 502 and proceeds to step 504, where a user access a html page and selects a particular program content to be played from the stream caching server.
• the method 500 queries whether the user has entered the correct password as previously established by the owner of the program content. Step 506 may alternatively query for a particular subscriber name or keyword. If the correct password is not entered, the method 500 ends at step 540. If the correct password is entered, the method 500 proceeds to step 508, where a query determines whether the user has a viewer or player to view the selected program content.
• step 508 determines whether a correct type of viewer or player is detected at the subscriber terminal, e.g., a computer terminal 1 16, 122 or 132. If the correct player is not detected, the method 500 proceeds to download the player at 510 and then proceeds to step 512. If the correct player is detected, the method 500 proceeds directly to step 512.
• a correct type of viewer or player e.g., a computer terminal 1 16, 122 or 132. If the correct player is not detected, the method 500 proceeds to download the player at 510 and then proceeds to step 512. If the correct player is detected, the method 500 proceeds directly to step 512.
• a query determines whether the downloaded player supports playback (of content) at full quality. If the player does not support playing of content at full quality, the method 500 proceeds to step 526. If the player supports playing of content at full quality, the method 400 proceeds to step 514, where a query determines whether the user has paid to view a full quality version of the content, e.g., a program content file . If the user has paid to view the full quality version of the program content, the method 500 proceeds to retrieve the full IP formatted content from the stream caching server 102 at step 516 and streams the retrieved content over the distribution network 104 at step 518.
• the method 500 proceeds to step 520, where the data link converter 1 12, 1 18 or 126 extracts the content (at the transport level) from the IP packets, content or MPEG-2 packets from the IP formatted content.
• a query determines whether to transcode the extracted content. Such transcoding is required to satisfy constraints in (downstream) transmitting the content over the access network 106, 108 and 1 10, and for playing the content on the viewer. If transcoding is not required, the method 400 proceeds to step 536. If transcoding is required, the method 400 transcodes the content at normal quality at step 524 and proceeds to step 536.
• the method 500 proceeds to step 526.
• the method 500 proceeds to retrieve a sample or predefined portion of the IP formatted content (file) from the stream caching server 102 at step 526.
• the method 500 then proceeds to stream the retrieved portion over the distribution network 104 at step 528 and extract the content, e.g., MPEG-2 packets, from the IP formatted content at step 530 .
• the method 500 proceeds to step 532, where a query determines whether to transcode the extract content for transmission over the access network 106, 108 or 110 and for playback on the viewer or playei . If no transcoding is requned, the method 500 proceeds to step 536. If transcoding is required, the method 550 proceeds to transcode the partial content at low quality at step 534 and then to step 536.
• the method 500 sends the transcoded content the subscriber terminal, e.g. a computer terminal 1 16, 122 and 132 via the access network 106, 108 and 1 10.
• the method 500 proceeds to play either the full or partial quality content at step 538 and ends at step 540.
• FIG. 3 depicts a data structure useful in understanding an embodiment of the present invention.
• FIG. 3 depicts a content stream 310 including a plurality of index points denoted as I,, I 2 and so on up to I (collectively index points I).
• index points I index points
• I N index points
• Each index point represents an appropriate stream entiy point beginning with, for example, an I-frame in the case of an MPEG content stream.
• Each indexed portion may be long or short, may comprise for example an entire scene oi a plurality of scenes, other mdex divisions may be used
• FIG 3 also depicts a sub-stream 320 comprising a plurality of index points denoted as 1 1 , ⁇ 2 and so on up to ⁇ 8 .
• the sub-stream comprises portions of content between index point L and index point I , yet not portions of the content traversing those index boundaries.
• the shaded portion of content stream 310 depicts the content within the sub-stream.
• the shaded portions of the sub-stream 320 depict content that is not included within the sub-stream.
• the sub-stream may be stored as a separate entity along with the content stream 310.
• the sub-stream may comprise those image frames associated with a portion ol content indexed according to the MPEG-7 standards.
• a particular object e.g., a sunset, classic automobile or actor
• the sub-stream 320 stores content specifically associated with a desired object so that such content may be retrieved.
• the first f rame foi example an MPEG-2 siteam
• the sub-stream 320 when created, includes transcoding of at least the first frame within the sub-stream into an intra-frame coding format.
• each ot the index points within the sub-stream may be reencoded to insure that these points also comprise I-tiames.
• many sub-streams may be genciated and stored and associated with the main content stream.
• local content is loaded onto a client, such as a set-top terminal or computer, from a local content source such as a video cassette lecorder (VCR), digital video recorder (DVR), personal video recorder (PVR), computer or othei storage and/or content source.
• a local content source such as a video cassette lecorder (VCR), digital video recorder (DVR), personal video recorder (PVR), computer or othei storage and/or content source.
• VCR video cassette lecorder
• DVR digital video recorder
• PVR personal video recorder
• computer othei storage and/or content source.
• local content may be loaded onto the set top terminal or streamed to the set top tei minal horn a camei a, live video and/oi live audit) feed.
• the content loaded onto the set top terminal is transcoded using a transcoding application. If the client does not have an appropriate transcoding application, such transcoding application is downloaded from a server within the system The transcoding application is used
• the loaded content comprises sitesam g video and audio at baseband
• such streaming video and audio may be encoded according to any one of the formats pieviously discussed (e.g., AVI, MPEG, etc.)
• the content so encoded is transcoded using the transcoding application to produce content in the desired player format.
• the transcoded local content is ther encapsulated in a desired transport format.
• the desired transport format is a transport format adapted to a particular access network. The particular formats associated with various access networks are described above.
• the transport encapsulated content is then encapsulated in a realtime protocol packet, such as desc ⁇ bed above.
• the RTP packet stream is then uploaded to the server for subsequent distribution to clients (i.e., set-top terminals or computers) utilizing the desired player format via access networks utilizing the desired access network transport format.
• clients i.e., set-top terminals or computers
• the client may also provide access right data to the server for subsequent use in determining who may view the content, who may use the content, how long the content may be viewed, how long the content may be used, which geographic region the viewer or user is in, which set or sub-set of clients within a particular system are to have access, which passwords, if any, are to be used and so on.
• the video imagery and associated audio information may be input to a set-top terminal from the video camera.
• the set-top terminal uses a coding or transcoding application to encode the video and audio information according to a desired format, such as the above-mentioned AVI format.
• a desired format such as the above-mentioned AVI format.
• the subscribers within a system who are to receive this imagery e.g., friends and neighbors
• the set-top terminal or client will then transport encode the AVI encoded content to pioduce an MPEG-2 transport stream.
• the MPEG-2 transport stream comprising the AVI encoded content will be further transport encoded according to the realtime protocol techniques described above.
• the RTP encoded content and associated access information e.g., passwords for family members and the like
• the server is then uploaded to the server for subsequent distribution on demand to the appropriate viewers or users.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Computer Networks & Wireless Communication (AREA)
• Databases & Information Systems (AREA)
• Computer Security & Cryptography (AREA)
• Human Computer Interaction (AREA)
• Library & Information Science (AREA)
• Computer Graphics (AREA)
• Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
• Information Transfer Between Computers (AREA)
• Data Exchanges In Wide-Area Networks (AREA)

Applications Claiming Priority (13)

Publications (2)

Family

ID=27558688

Family Applications (1)

Country Status (4)

Cited By (1)

Families Citing this family (15)

Citations (2)

Family Cites Families (5)

• 2001
  • 2001-01-29 WO PCT/US2001/002802 patent/WO2001055877A1/en active Application Filing
  • 2001-01-29 AU AU2001237978A patent/AU2001237978A1/en not_active Abandoned
  • 2001-01-29 EP EP01910365A patent/EP1252576A4/de not_active Withdrawn
  • 2001-01-29 CA CA002398071A patent/CA2398071A1/en not_active Abandoned
  • 2001-01-29 WO PCT/US2001/002602 patent/WO2001055860A1/en active Application Filing
  • 2001-01-29 CA CA2397975A patent/CA2397975C/en not_active Expired - Lifetime
  • 2001-01-29 AU AU2001234579A patent/AU2001234579A1/en not_active Abandoned

Patent Citations (2)

Non-Patent Citations (2)

Also Published As

Similar Documents

Legal Events