WO2009116024A2 - A controlled rate vod server - Google Patents

A controlled rate vod server Download PDF

Info

Publication number
WO2009116024A2
WO2009116024A2 PCT/IL2008/000377 IL2008000377W WO2009116024A2 WO 2009116024 A2 WO2009116024 A2 WO 2009116024A2 IL 2008000377 W IL2008000377 W IL 2008000377W WO 2009116024 A2 WO2009116024 A2 WO 2009116024A2
Authority
WO
WIPO (PCT)
Prior art keywords
stream
bit rate
data
segment
video
Prior art date
Application number
PCT/IL2008/000377
Other languages
French (fr)
Other versions
WO2009116024A3 (en
Inventor
Nir Drang
Tomer Amiaz
Ram Ben-Yakir
Original Assignee
Fabrix Tv Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fabrix Tv Ltd. filed Critical Fabrix Tv Ltd.
Priority to ES08720002T priority Critical patent/ES2816639T3/en
Priority to CA2717882A priority patent/CA2717882A1/en
Priority to EP08720002.8A priority patent/EP2269143B1/en
Priority to CN2008801293695A priority patent/CN102047236A/en
Priority to US12/933,435 priority patent/US8732786B2/en
Priority to PCT/IL2008/000377 priority patent/WO2009116024A2/en
Priority to EP20180944.9A priority patent/EP3748514B1/en
Priority to PT201809449T priority patent/PT3748514T/en
Publication of WO2009116024A2 publication Critical patent/WO2009116024A2/en
Publication of WO2009116024A3 publication Critical patent/WO2009116024A3/en
Priority to US14/280,736 priority patent/US9282347B2/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/115Selection of the code volume for a coding unit prior to coding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146Data rate or code amount at the encoder output
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/164Feedback from the receiver or from the transmission channel
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/177Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a group of pictures [GOP]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/40Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21Server components or server architectures
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21Server components or server architectures
    • H04N21/222Secondary servers, e.g. proxy server, cable television Head-end
    • H04N21/2225Local VOD servers
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/231Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
    • H04N21/23106Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion involving caching operations
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/23424Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234309Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 or from Quicktime to Realvideo
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/23439Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements for generating different versions
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2347Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving video stream encryption
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/236Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
    • H04N21/23611Insertion of stuffing data into a multiplex stream, e.g. to obtain a constant bitrate
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/236Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
    • H04N21/2365Multiplexing of several video streams
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/236Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
    • H04N21/2365Multiplexing of several video streams
    • H04N21/23655Statistical multiplexing, e.g. by controlling the encoder to alter its bitrate to optimize the bandwidth utilization
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/238Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams
    • H04N21/2383Channel coding or modulation of digital bit-stream, e.g. QPSK modulation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/262Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists
    • H04N21/26208Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists the scheduling operation being performed under constraints
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/434Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
    • H04N21/4347Demultiplexing of several video streams
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47End-user applications
    • H04N21/472End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/47202End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for requesting content on demand, e.g. video on demand
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/835Generation of protective data, e.g. certificates
    • H04N21/8358Generation of protective data, e.g. certificates involving watermark
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845Structuring of content, e.g. decomposing content into time segments
    • H04N21/8456Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments

Definitions

  • the present invention in some embodiments thereof, relates to a digital video server and, more particularly, but not exclusively, to a digital video server for video- on-demand services.
  • Video streams such as MPEG streams, typically have a Variable Bit Rate
  • VBR Video streaming
  • the video streams are typically compressed, and the amount of compression depends on complexity of the scene, and on the amount of changes between scenes.
  • the MPEG video compression protocols compress video by reducing pixel data redundancy within an image, and also redundancy between neighboring images.
  • MPEG video compression groups several consecutive images into a Group Of Pictures
  • GOP defines most of the pictures, also termed video frames, by their difference from a neighboring picture, which is also in the GOP.
  • a GOP contains X amount of data for N video frames, which need to be played out at a rate of R frames per second, thereby having a bit rate proportional to X*R/N. If there is need to lower the bit rate of a video stream, pictures in a GOP need to be changed together, since the pictures depend on each other for reconstruction.
  • Transport Stream (MPTS). In some cases a transport stream contains only one program. Such transport streams are termed Single Program Transport Streams
  • VOD Video On Demand
  • Customers request to receive specific videos, and VOD suppliers transmit each video requested by the customers.
  • the VOD supplier transmits several MPEG streams, some of different videos, and some of the same videos started at different times.
  • a transport stream often has a Constant Bit Rate (CBR).
  • CBR Constant Bit Rate
  • the bit rate of the transport stream is often a physical upper limit of some device in the transmission path.
  • the transport stream has a bit rate assigned to it by a service provider which performs the transmission.
  • a VOD supplier may purchase one or more units of such and such bandwidth, to use for supplying video on demand.
  • Combining VBR streams into a CBR transport stream can be done. Care must be taken that the total bit rate of the combination of VBR streams never exceeds the CBR limit, otherwise the CBR transport stream may well crash, or be cut off by the service provider which performs the transmission. Sometimes, the combining combines the streams by taking each of the constituent VBR streams into account as if it constantly has a maximum bit rate, and whenever the total bit rate of the combined VBR streams falls below the CBR of the transport stream, padding is added.
  • MPEG streams are compressed for ease of decompression, having a relatively computationally simple decompression, and a relatively computation intensive compression. Therefore bit rate lowering of MPEG streams, which involves decompression (easy) and subsequent compression (hard) is computationally intensive.
  • An aspect of some embodiments of the invention relates to using a distributed data storage system for distributed processing of data stored on the distributed storage nodes.
  • Some types of processing are computation intensive, such as, by way of a non- limiting example, video stream processing.
  • One or more computers may control the processing, instructing data storage nodes, which also have some processing capability, to perform some or all of the processing.
  • An aspect of some embodiments of the invention relates to controlling a bit rate of a transport stream by controlling the bit rate of components of the transport stream.
  • Changing bit rate can be computation intensive, and changing the bit rate of components of the transport stream is performed in a distributed fashion, so as to spread the computation load away from just one computer.
  • the changing of bit rate may be done in real time, that is, just prior to transmitting the transport stream, or may be done ahead of real time, and changed bit rate stream components can be saved until needed.
  • a distributed stream processing system including a plurality of storage devices, at least one of which is configured to process at least one segment of a data stream, thereby producing a changed segment of the data stream, and provide output of at least the changed segment of the data stream, and a server configured to accept the output of the storage devices and combine the output into a transport stream.
  • the process includes changing the bit rate.
  • the at least one storage device is further configured to store at least the changed segment of the data stream.
  • the server is configured to provide bit rate control instructions to the plurality of storage devices, and the plurality of storage devices are configured to accept the bit rate control instructions from the server and to change the bit rate of the segment of a data stream according to the instructions.
  • At least one of the data streams is a video stream.
  • the process includes transcoding the video stream.
  • the process includes inserting content into the video stream.
  • the process includes producing video effects in the video stream.
  • the process includes content protection processing with respect to the video stream.
  • the process includes searching in the video stream.
  • the storage devices use a data protection scheme to store the changed segment of the data stream.
  • a method for combining a plurality of data streams including using one or more storage devices for processing at least one segment of a data stream, thereby producing a changed segment of the data stream, and providing output of at least the changed segment of the data stream, and combining the output of the storage devices into a combined data stream.
  • the processing includes changing a bit rate.
  • bit rate control instructions to the plurality of storage devices, and the storage devices accepting the bit rate control instructions from the server and changing the bit rate of the segment of a data stream according to the instructions.
  • At least one of the data streams is a video data stream.
  • the processing includes video stream processing.
  • a system configured to receive a transport stream including a plurality of data streams including a server configured to split the transport stream into a plurality of data streams, a plurality of storage devices configured to receive and store one or more of the data streams from the server, at least one of the storage devices being further configured to process at least one segment of a data stream, thereby producing a changed segment of the data stream, and store at least the changed segment of the data stream.
  • the process includes changing the bit rate.
  • At least one of the data streams includes a video stream, and the process includes video stream processing.
  • a method for receiving a transport stream including a plurality of data streams including splitting the transport stream into a plurality of data streams, storing the data streams on a plurality of storage devices, using at least one of the storage devices for processing at least one segment of a data stream, thereby producing a changed segment of the data stream, and storing the changed segment of the data stream.
  • the processing includes changing a bit rate.
  • At least one of the data streams is a video stream
  • the processing includes video stream processing.
  • Implementation of the method and/or system of embodiments of the invention can involve performing or completing selected tasks manually, automatically, or a combination thereof. Moreover, according to actual instrumentation and equipment of embodiments of the method and/or system of the invention, several selected tasks could be implemented by hardware, by software or by firmware or by a combination thereof using an operating system.
  • a data processor such as a computing platform for executing a plurality of instructions.
  • the data processor includes a volatile memory for storing instructions and/or data and/or a non- volatile storage, for example, a magnetic hard-disk and/or removable media, for storing instructions and/or data.
  • a network connection is provided as well.
  • a display and/or a user input device such as a keyboard or mouse are optionally provided as well.
  • FIG. 1 is a simplified illustration of a system constructed and operative according to an exemplary embodiment of the invention
  • FIG. 2 is a simplified flow chart illustration of a method operative according to an exemplary embodiment of the invention
  • Fig. 3 is a more detailed illustration of an example embodiment of the system of Fig. 1;
  • Fig. 4 is a simplified graphical illustration of variable bit rates in a video stream.
  • Fig. 5 is a simplified graphical illustration of how chopping off bit rate peaks reduces a maximal bit rate of a video stream in an embodiment of the invention.
  • the present invention in some embodiments thereof, relates to a digital video server and, more particularly, but not exclusively, to a digital video server for video- on-demand services.
  • An aspect of some embodiments of the invention relates to using a distributed data storage system for distributed processing of data stored on the distributed storage nodes.
  • One or more computers may control the processing, and data storage nodes, which also have some processing capability, perform at least some of the processing.
  • Some types of processing are computation intensive, such as, by way of a non- limiting example, video stream processing.
  • Non-limiting examples of computation intensive video stream processing include:
  • Rate shaping video streams that is, changing the bit rate of video streams. Often the changing is a lowering of the bit rate of a video stream in order to fir the video stream into a narrower bandwidth.
  • Transcoding video streams from one format to another, such as, by way of a non-limiting example, MPEG2 to MPEG4 or vice versa, High Definition (HD) to Standard Definition (SD) or vice versa.
  • MPEG2 to MPEG4 or vice versa
  • HD High Definition
  • SD Standard Definition
  • Content processing such, by way of a non-limiting example, inserting additional content such as a banner into a video stream, a logo into a video stream, producing a Picture-In-Picture (PIP) video stream, and adding a voice overlay or a video overlay.
  • additional content such as a banner into a video stream, a logo into a video stream, producing a Picture-In-Picture (PIP) video stream, and adding a voice overlay or a video overlay.
  • PIP Picture-In-Picture
  • Video effect processing such as, by way of a non-limiting example, fade-in, fade-out, alpha blending, changing brightness, and a wipe effect.
  • Content protection processing such as, by way of a non-limiting example, encryption, decryption, inserting a watermark, deleting a watermark.
  • Searching within a video stream such as, by way of a non-limiting example, searching for a caption, and/or other searchable attributes.
  • Some embodiments contemplated include combining data streams stored on data storage nodes into a transport stream, while processing at least some of the data streams.
  • Some embodiments contemplated include receiving a transport stream containing data streams, splitting the transport stream into data streams for storage in storage devices, while having some of the storage devices process at least part of the data streams, and storing the processed part.
  • VBR Variable Bit Rate
  • CBR Constant Bit Rate
  • Other specific embodiments contemplated include receiving a transport stream containing video streams, splitting the transport stream into video streams for storage in storage devices, and instructing some of the storage devices to process at least part of the video streams and store the changed, processed part.
  • An aspect of some embodiments of the invention relates to controlling a bit rate of a transport stream by controlling the bit rate of video stream components of the transport stream.
  • Changing a bit rate of a video stream can be computation intensive, and changing the bit rate of the video stream components is performed in a distributed fashion, so as to spread the computation load.
  • video streams are stored on distributed data disks, and combined into a transport stream by a streamer, which is a dedicated computer.
  • the data disks and the streamer are likely to be expensive pieces of equipment, dedicated to their intended tasks.
  • An alternative approach to the above is using a lower cost general purpose computer, also termed a server, as a streamer, and using low cost storage on low cost general purpose computers as the distributed storage.
  • the alternative approach produces an additional advantage, which is that the distributed storage also has processing power, opening the door to distributed processing.
  • An aspect of some embodiments of the invention relates to other types of processing of the transport stream, being performed in a distributed fashion.
  • the invention is suitable for distributing processing of stream components.
  • video stream can refer to a stream which is limited in time, such as, by way of a non-limiting example, a movie, a television program, and a 30 second commercial, and the term can refer to a video stream which is not limited in time, such as a 24 hour a day program.
  • FIG. 1 is a simplified illustration of a system constructed and operative according to an exemplary embodiment of the invention.
  • the system comprises a streamer 100, connected to one or more storage clusters 105.
  • the streamer 100 is configured to provide an output 120 or outputs 120 of one or more transport streams.
  • the streamer 100 requests the storage clusters 105 to send it video streams (not shown), and combines the video streams into the one or more transport streams.
  • the streamer 100 instructs one or more of the storage clusters 105 to perform the processing and provide a changed, processed, video stream for the combining.
  • the streamer 100 tracks the bit rate of each of the transport streams. If a bit rate of a transport stream is about to exceed a pre-defined limit, the streamer 100 ensures, as will be further described below, that the bit rate of one or more of the video streams combined into the transport stream is lowered, thereby ensuring that the bit rate of the transport stream does not exceed the limit.
  • the streamer 100 instructs one or more of the storage clusters 105 to provide a version of one or more of video streams having a lower than nominal bit rate.
  • the storage clusters 105 provide the streamer 100 with a lower bit rate version of the one or more video stream which the streamer instructed to be at a lower bit rate.
  • Fig. 2 is a simplified flow chart illustration of a method operative according to an exemplary embodiment of the invention.
  • Fig. 2 depicts a flow chart of the method used in the system of Fig. 1, which uses one or more storage devices (150) for processing at least one segment of one of the data streams (160).
  • An optional method of processing used by an embodiment of the invention is described further below with reference to Fig. 5.
  • Output of at least the changed data stream is provided (165), as described above with reference to Fig. 1, and will be described further below with reference to Fig. 3.
  • a streamer is used for combining the output of the storage devices into a combined data stream (170), as described above with reference to Fig. 1, and will be described further below with reference to Fig. 3.
  • Fig. 3 is a more detailed illustration of an example embodiment of the system of Fig. 1.
  • the system comprises a streamer 100, connected to one or more storage clusters 105.
  • the storage clusters 105 each contain one or more storage machines 110, each of which is connected to one or more storage devices 115.
  • the streamer 100 optionally instructs one or more of the storage clusters 105 to provide a version of one or more of video streams having a lower than nominal bit rate.
  • the storage clusters 105 optionally provide the streamer 100 with a lower bit rate version of the one or more video streams which the streamer instructed to be at a lower bit rate.
  • the streamer 100 The streamer 100
  • the streamer 100 combines several video streams into a transport stream.
  • the streamer optionally ensures that a sum of the bit rates of the video streams does not exceed a bit rate limit associated with the transport stream.
  • Combining VBR video streams into a CBR transport stream, when the total bit rate of the video streams is less than the bit rate limit, is known in the art. For example, one way of doing so, is tracking the combined bit rate, and adding null data to the transport stream as needed to bring the bit rate of the combined-transport stream up to a constant value.
  • a more efficient use of a CBR transport stream is to combine as many video streams as possible into the transport stream. Since the video streams show a variable bit rate, it is possible to combine the video streams such that their combined bit rate will usually be below the limit of the transport stream.
  • the streamer 100 When the combined bit rate of the video streams is expected to exceed the limit, the streamer 100 optionally replaces some or all of the video streams with lower bit rate versions of the video streams, and prevents the exceeding.
  • the average bit rate of each VBR video file or video file segment participating in a transport stream is optionally derived based on the following parameters: a modulation type for the MPTS, such as, by way of a non-limiting example, Quadrature
  • Amplitude Modulation (QAM) type such as 64/128/256 Symbol Rate; and the number of streams participating in the MPTS .
  • QAM Amplitude Modulation
  • Fig. 4 is a simplified graphical illustration of variable bit rates in a video stream.
  • Fig. 4 depicts a graph 200 of bit rate of a segment of a video stream vs. time.
  • the graph is of a segment of an original video stream with an original, unmodified, bit rate.
  • the horizontal axis 205 of the graph 200 depicts time, in units of video frames. Since video is typically displayed at a constant frame rate, frames are directly proportional to time.
  • the vertical axis 210 of the graph 200 depicts bit rate, in units of
  • the segment contains several GOPs 215, 216, 217, 218.
  • 215, 216, 217, 218, comprises individual pictures, one bar of the graph 200 standing for each picture.
  • Each of the pictures has an inherent bit rate associated, based on the amount of data required to portray the picture, and a frame rate of the video stream.
  • the inherent bit rate is depicted by the relative height of the bar which stands for the picture.
  • each GOP is also associated with a bit rate.
  • the bit rate of a GOP is based on the amount of data, after compression, which the GOP contains, a frame rate of the video stream, and the number of pictures within the GOP.
  • GOP bit rates 220, 221, 222, 223 are depicted by dotted lines corresponding to the bit rates of the GOPs 215, 216, 217, 218 respectively.
  • the GOP bit rates 220, 221, 222, 223 are not equal, depending as they do on contents of the pictures of the video stream.
  • a maximal bit rate of the segment in the illustration of Fig. 4 is the GOP bit rate 221, associated with the GOP 2 216.
  • FIG. 5 5 is a simplified graphical illustration of how chopping off bit rate peaks reduces a maximal bit rate of a video stream in an embodiment of the invention.
  • Fig. 5 depicts the segment of Fig. 4, in which bit rate peaks of individual pictures have been modified by "chopping off, that is reducing, just the bit rate peaks.
  • the values of the unmodified picture bit rates and the unmodified GOP bit rates 220, 221, 222, 223 are still depicted in Fig. 5, for purpose of comparison and to aid understanding.
  • GOP 2 was reduced in bit rate by a first amount 225.
  • a GOP bit rate 231 of GOP 2 216 is lower than the GOP bit rate 221 of GOP 2 216 before bit rate modification.
  • a picture in GOP 3 217 which had a highest inherent bit rate of the pictures of GOP 3 217, was reduced in bit rate by a second amount 226, different from the first amount 225.
  • a GOP bit rate 232 of GOP 3 217 is lower than the GOP bit rate 222 of GOP 3 217 before bit rate modification.
  • a GOP bit rate 233 of GOP 4 218 is lower than the GOP bit rate 223 of GOP 4 218 before bit rate modification.
  • the files which together make up a transport stream change over time.
  • the streamer is a VOD server
  • sessions are allocated by end users, and the streamer 100 is required to read files from the storage cluster 105.
  • the streamer instructs each processing unit 110 in the storage cluster 105 to reduce the rate of certain GOPs according to a statistical multiplexing algorithm.
  • the streamer 100 optionally causes rate shaping of video streams to be performed by computing power available in the storage clusters 105 (Fig. 1). While storage clusters 105 typically contain a large amount of storage, which is needed for storing the video streams, the storage clusters are nevertheless capable of computation. Having several storage clusters 105 enables using the computation power built into a distributed video storage system for also rate shaping the video. It is noted that, similarly to the example of Fig. 5, rate shaping is optionally not performed on an entire file, but rather on a file segment. The size of the file segment ranges from an entire file, to a portion of a file, down to a GOP.
  • the streamer 100 Upon having an MPTS or a group of SPTS streams directed at a specific output with a specified maximum allowed bit rate, the streamer 100 (Fig. 1) starts reading one or more streams from one of the storage clusters 105,
  • the streamer 100 reads metadata associated with each stream. More detail about the metadata is provided below, with reference to "Ingesting a Video File”.
  • the streamer 100 can predict what will be the aggregated bit rate of all streams in the MPTS, or group of SPTS streams going into a specific output.
  • the metadata also contains information about complexity of each stream, and enables the streamer 100 to predict a level of deterioration in video stream quality due to rate reduction.
  • the streamer compiles a set of commands to the storage clusters 105, requesting the storage clusters 105 for relevant data segments with a specific bit rate per GOP.
  • the streamer 100 is a component combining the streams into the data stream, and the component which considers the metadata and issues commands to the storage clusters 105 is a stream management server (not shown).
  • the streamer 100 and the stream management server may or may not be separately housed and/or separately located.
  • the statistical multiplexing method described above may be distributed among several
  • Each streamer 100 manages one or more transport streams, and instructs relevant storage machines 110, via one or more storage clusters
  • Each VBR program stream is padded with null stuffing into a CBR SPTS at the maximum allowed bit rate, and sent to a suitable output.
  • a distributed statmuxing method optionally manages an aggregated bandwidth of all streams going into a specific output transport stream, even when some video streams originate from different streamers machine,
  • an end user may choose to use trick play modes, such as Fast Forward (FF), rewind, and pause.
  • FF Fast Forward
  • rewind rewind
  • pause pause
  • FF and rewind modes only a first I-frame from a GOP is requested and received by the streamer 100, and not all pictures of the GOP.
  • Managing the bit rate is optionally done by the streamer 100, by adding a suitable number of P- frames to I-frames and forming newly generated GOPs. If the bit rate of a GOP comprising I-frames only is still too high, bit rate reduction is optionally requested by the streamer 100 from the storage machines, in order to reduce the bit rate of the I-frame. Alternatively, the streamer 100 performs rate reduction for the I-frames.
  • the streamer 100 stops transmitting the video stream, so there is no need to use rate reduction for a paused video stream,
  • the streamer 100 takes pause mode into consideration, and for example, in case of a paused video stream, other streams in the transport stream may use more bit rate.
  • rate shaping is done in real time, or approximately in real time, that is, just prior to transmitting the transport stream.
  • the rate shaping is done ahead of real time, and changed bit rate video streams are saved until needed.
  • each of the storage machines 110 which performs rate shaping optionally receives instructions for rate shaping from the streamer 100, which include a bit rate parameter which specifies a maximum bit rate, or by how much to reduce the bit rate of a file segment.
  • each storage machine 110 is configured for one or more configurable levels of rate shaping.
  • rate reduction reduces quality of images in frames of each GOP, so that a maximum bit rate is not exceeded.
  • rate reduction reduces the bit rate by a percentage of the bit rate, or by subtracting a bit rate from an unmodified bit rate.
  • each storage cluster 105 handles an internal reallocation of a bit rate budget within each GOP, for example, allocating more or less bit rate to different frames within a GOP based on the frame's complexity, optionally providing some areas of the frames with a specific treatment.
  • the streamer When rate shaping is performed in real time or approximately real time, the streamer combines several video streams into the transport stream based, at least in part, on a total bit rate of the video streams and optionally on a bit rate limit for the transport stream. It is noted that transport streams sometimes carry more than just video streams.
  • the data carried by the transport stream beyond the video streams is hereby considered a data stream, and the description of data streams is taken to be included in the video stream examples.
  • the streamer 100 estimates the bit rate of the transport stream, and optionally estimates the bit rate of each of the video streams.
  • the estimation is done by the streamer actually measuring the bit rate of the transport stream and optionally measuring the bit rate of each of the video streams.
  • the streamer 100 instructs the storage machines 110 which video streams to rate shape, and how to rate shape the video streams. In some embodiments of the invention the streamer 100 instructs the storage machines 110 which video stream and at what bit rate to provide the streamer. In other embodiments of the invention the streamer 100 instructs the storage machines 110 which video stream to provide the streamer 100, and by what amount, or by what percentage, to lower the bit rate of the video stream.
  • the storage machines 110 which rate shape a video stream, also cache the rate shaped video stream, so that of the video stream needs to be output again as a rate shaped video stream, the cached rate shaped video stream is recovered from the cache.
  • the streamer 100 When rate shaping is done ahead of real time, the streamer 100 optionally instructs the storage clusters 105 which video stream to rate shape, to what bit rate to shape the video stream, and where to store the rate shaped video stream.
  • the streamer 100 instructs the storage clusters 105 which video stream to rate shape, by what amount, or by what percentage, to lower the bit rate of the video stream, and where to store the rate shaped video stream.
  • Metadata is optionally produced, associating some or all of the saved video streams with an estimated or measured bit rate.
  • the metadata includes the bit rate of segments of the video streams, rather than a bit rate for an entire video stream.
  • the metadata includes the bit rate for each individual GOP.
  • the metadata for a video stream or for a segment of a video stream includes a maximum bit rate as well as an average bit rate.
  • a video stream is stored in more than one version, with different versions having been rate shaped and comprising different bit rates.
  • different versions having been rate shaped and comprising different bit rates.
  • only some segments of the video stream are rate shaped, the rate shaped segments preferably being the higher bit rate segments.
  • the segments of the video stream which are rate shaped are as small as one GOP.
  • rate shaping is performed ahead of real time, there may not be certainty that rate shaped versions or segments of a video stream will actually be used.
  • deciding which video streams or segments thereof will be rate shaped is performed according to projected statistics of video demand. Ia some embodiments of the invention the deciding is performed according to actual video demand.
  • Various schemes for saving computation are contemplated, such as: rate shaping only video streams which are most in demand; rate shaping only high bit rate segments of video streams; rate shaping most-in-demand video streams in advance of real time, and rate shaping other video streams in real time or approximate real time; and rate shaping a video stream in real time when first combined into a transport stream, and saving the rate shaped video stream or rate shaped segments of the video stream, and their associated metadata, for later re-use.
  • Various schemes for saving storage space are contemplated, such as: saving rate shaped versions only of video streams which are most in demand; saving rate shaped versions of only high bit rate segments of video streams; and saving rate shaped versions of a video stream only after a first combining into a transport stream.
  • An exemplary embodiment of the invention includes a storage cluster 105, which comprises several storage machines, also termed data servers. Each storage machine has one or more storage devices, and some processing power.
  • the storage cluster 105 protects data by storing redundant data.
  • the protection is performed using schemes such a RAID. It is to be noted that the protection is configured per file segment.
  • the file segment corresponds to a video stream, segment of a video stream, and even a GOP of a video stream.
  • the 100 requests rate reduced file segments from a second one of the storage clusters 105.
  • the second storage cluster 105 gathers the required file segments from the second storage cluster 105.
  • the second storage cluster 105 performs rate reduction on the recovered file segments.
  • each file is stored in a predefined rate reduced version on the storage cluster.
  • the version is optionally either a CBR version, or a capped VBR with a maximum bit rate which is not higher then the average bit rate of the original stream.
  • the predefined rate reduced version is termed a fallback copy.
  • the fallback copy is optionally used.
  • the fallback copy is also optionally used if the bit rate of the fallback copy falls inside the limits of the bit rate required by the streamer 100.
  • Each storage machine 110 optionally has a cache containing already rate reduced GOPs.
  • the cache is optionally based upon an LRU method.
  • Some embodiments of the invention use a cached version of a rate reduced GOP rather than utilize rate reduction.
  • the preference for using the cache, and a size of the cache, are determined based at least partly on the available processing power of the storage machine 110.
  • GOP at a bit rate of 2.5Mbps, and if there is a cached version of the same GOP reduced to 2.4Mbps, the system optionally chooses to use the cached 2.4 Mbps version. This frees the storage machine 110 to rate shape other GOPs that are not already cached .
  • the storage machine 110 optionally has a dedicated amount of storage and storage IO for caching purposes.
  • a data server may optionally get multiple requests for rate shaping a file segment, from multiple streamers, or optionally multiple requests from a single streamer but intended to be combined in an MPTS the single streamer processes.
  • Such a scenario can be common when many users would like to watch a specific file at substantially the same moment, for instance a newly issued popular event.
  • the data server which stores the requested video segment receives many requests for the same segment, more than it can process.
  • the data node reaches a point where it is not able to serve all the requests and maintain an approximately real-time response.
  • the data server aggregates requests and chooses to process only several rate shaping commands, such that will not exceed its processing power and its read-from-storage capabilities.
  • the data server optionally provides a copy of a closest rate shaped version, or a closest rate shaped version with less than the required bit rate, available of the requested rate shaped segment.
  • An ingested video file is typically encoded to a VBR format.
  • the streamer 100 parses the video file, and breaks the video file into file segments.
  • the streamer 100 optionally produces and stores, optionally on one or more storage clusters 105, metadata about the stream behavior and complexity.
  • the metadata is optionally used later for statistical multiplexing decision making.
  • the metadata for the video file is generated, and kept in a separate file.
  • the information saved in the metadata file contains information about the stream bit rate, frame structure and complexity.
  • Video streams are optionally striped across storage devices 115 in the storage machines 110 and optionally also across storage machines 110 in a storage cluster 105.
  • Each file is optionally cut into segments which are optionally distributed among the storage devices.
  • Each segment contains one or more GOPs belonging to the file.
  • each storage device may have a segment from each file, and vice versa, segments from each file may appear on each storage device.
  • the number of GOPs per segment is determined in order to optimize throughput of a clustered file system.
  • the number of GOPs per segment is determined based, at least partly, on storage parameters such as block sizes in the storage device, on computation parameters such as computation speed on the storage machine, on the number of computation machines, and on a bit rate of the segment. For example, when a segment size is determined to be 1 MB, the segment can contain 10 GOPs of a Standard Definition (SD) video stream, or 5 GOPs of a High Definition (HD) stream.
  • SD Standard Definition
  • HD High Definition
  • a segment is smaller than an MPEG
  • a segment can be, by way of a non-limiting example, a frame; a number of frames, with no relation to the underlying MPEG GOP; and even part of a frame, even as small as a slice of an MPEG macroblock.
  • the rate shaped segment can optionally be stored on one data server, and can optionally be stored on more than one data server.
  • a segment is minimally chosen to be one or more GOPs, that is all frames from one I-frame to a next I-frame, which fit into and are aligned to a 1MB data block.
  • Each segment of data written into storage is optionally protected by a data protection scheme, by way of a non-limiting example a data protection scheme using data redundancy, such as RAID.
  • the level of RAID protection is any level of RAID protection, such as RAID level 0, 1, 2, 3, 4, 5, and so on, and may optionally be different per file.
  • Each segment is thus protected by redundant data, which is written into the storage cluster, to different storage devices, on different storage machines.
  • protection management is provided by the storage cluster.
  • each of the storage machines In some embodiments of the invention protection management is provided by the streamer.
  • a streamer attempts to fetch a file segment from a first storage device on a first storage machine. Due to failure of the first storage device or the first storage machine, the data segment is unavailable.
  • Recovery option 1 when the streamer manages the protection scheme, the streamer fetches redundant data written by the protection algorithm on a second storage device, and reconstructs the unavailable data. The streamer does not necessarily rate shape the reconstructed data. If rate shaping is necessary, the streamer optionally instructs the storage clusters providing other streams to provide the other streams at lower bit rate.
  • Recovery option 2 when each of the storage machines manages the protection scheme, the streamer requests a second storage machine to fetch the unavailable data.
  • the second storage machine retrieves the redundant data, and reconstructs the previously unavailable data using the protection scheme.
  • the storage machine then performs rate shaping on the reconstructed segment, and deliver the rate shaped and recovered data to the streamer.
  • Recovery option 3 when the streamer manages the protection scheme, the streamer fetches redundant rate shaped data from the storage cluster, and reconstructs the unavailable data according to the protection scheme.
  • the storage machine 110 optionally overlays text, video, and/or sound over file segments, according to instructions received from the streamer 100.
  • the overlay is optionally performed in the compressed domain.
  • Each storage machine 110 can optionally convert video segments from one video format to another, using a transcoding algorithm.
  • the transcoding is optionally per data segment, so that each storage machine 110 is only required to transcode segments which are stored thereon. It is expected that during the life of a patent maturing from this application many relevant rate shaping methods will be developed and the scope of the term rate shaping is intended to include all such new technologies a priori.
  • compositions, method or structure may include additional ingredients, steps and/or parts, but only if the additional ingredients, steps and/or parts do not materially alter the basic and novel characteristics of the claimed composition, method or structure.
  • a video stream or “at least one video stream” may include a plurality of video streams, including combinations thereof.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Business, Economics & Management (AREA)
  • Marketing (AREA)
  • Computer Security & Cryptography (AREA)
  • Human Computer Interaction (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Television Signal Processing For Recording (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

A distributed stream processing system including a plurality of storage devices, at least one of which is configured to process at least one segment of a data stream, thereby producing a changed segment of the data stream, and provide output of at least the changed segment of the data stream, and a server configured to accept the output of the storage devices and combine the output into a transport stream. A method for combining a plurality of data streams including using one or more storage devices for processing at least one segment of a data stream, thereby producing a changed segment of the data stream, and providing output of at least the changed segment of the data stream, and combining the output of the storage devices into a combined data stream. Related apparatus and methods are also described.

Description

A CONTROLLED RATE VOD SERVER
FIELD AND BACKGROUND OF THE INVENTION The present invention, in some embodiments thereof, relates to a digital video server and, more particularly, but not exclusively, to a digital video server for video- on-demand services.
Digital video is often distributed as streams of digital data. One well known protocol for video data is MPEG. Video streams, such as MPEG streams, typically have a Variable Bit Rate
(VBR). The video streams are typically compressed, and the amount of compression depends on complexity of the scene, and on the amount of changes between scenes.
The MPEG video compression protocols compress video by reducing pixel data redundancy within an image, and also redundancy between neighboring images. MPEG video compression groups several consecutive images into a Group Of Pictures
(GOP), and defines most of the pictures, also termed video frames, by their difference from a neighboring picture, which is also in the GOP.
A GOP contains X amount of data for N video frames, which need to be played out at a rate of R frames per second, thereby having a bit rate proportional to X*R/N. If there is need to lower the bit rate of a video stream, pictures in a GOP need to be changed together, since the pictures depend on each other for reconstruction.
In cases where video streams are distributed to customers, several video streams are sometimes combined in one transport stream, termed a Multiple Program
Transport Stream (MPTS). In some cases a transport stream contains only one program. Such transport streams are termed Single Program Transport Streams
(SPTS).
One example of video distribution, which is becoming more and more common nowadays, is distribution of Video On Demand (VOD). Customers request to receive specific videos, and VOD suppliers transmit each video requested by the customers. For example, the VOD supplier transmits several MPEG streams, some of different videos, and some of the same videos started at different times.
A transport stream often has a Constant Bit Rate (CBR). The bit rate of the transport stream is often a physical upper limit of some device in the transmission path. In other cases the transport stream has a bit rate assigned to it by a service provider which performs the transmission. A VOD supplier may purchase one or more units of such and such bandwidth, to use for supplying video on demand. Combining VBR streams into a CBR transport stream can be done. Care must be taken that the total bit rate of the combination of VBR streams never exceeds the CBR limit, otherwise the CBR transport stream may well crash, or be cut off by the service provider which performs the transmission. Sometimes, the combining combines the streams by taking each of the constituent VBR streams into account as if it constantly has a maximum bit rate, and whenever the total bit rate of the combined VBR streams falls below the CBR of the transport stream, padding is added.
Lowering a bit rate of a video stream can be done. It is a computation intensive task. MPEG streams, for example, are compressed for ease of decompression, having a relatively computationally simple decompression, and a relatively computation intensive compression. Therefore bit rate lowering of MPEG streams, which involves decompression (easy) and subsequent compression (hard) is computationally intensive.
SUMMARY OF THE INVENTION An aspect of some embodiments of the invention relates to using a distributed data storage system for distributed processing of data stored on the distributed storage nodes. Some types of processing are computation intensive, such as, by way of a non- limiting example, video stream processing. One or more computers may control the processing, instructing data storage nodes, which also have some processing capability, to perform some or all of the processing.
An aspect of some embodiments of the invention relates to controlling a bit rate of a transport stream by controlling the bit rate of components of the transport stream. Changing bit rate can be computation intensive, and changing the bit rate of components of the transport stream is performed in a distributed fashion, so as to spread the computation load away from just one computer.
The changing of bit rate may be done in real time, that is, just prior to transmitting the transport stream, or may be done ahead of real time, and changed bit rate stream components can be saved until needed.
According to an aspect of some embodiments of the present invention there is provided a distributed stream processing system including a plurality of storage devices, at least one of which is configured to process at least one segment of a data stream, thereby producing a changed segment of the data stream, and provide output of at least the changed segment of the data stream, and a server configured to accept the output of the storage devices and combine the output into a transport stream.
According to some embodiments of the invention, the process includes changing the bit rate. According to some embodiments of the invention, the at least one storage device is further configured to store at least the changed segment of the data stream.
According to some embodiments of the invention, the server is configured to provide bit rate control instructions to the plurality of storage devices, and the plurality of storage devices are configured to accept the bit rate control instructions from the server and to change the bit rate of the segment of a data stream according to the instructions.
According to some embodiments of the invention, at least one of the data streams is a video stream. According to some embodiments of the invention, the process includes transcoding the video stream. According to some embodiments of the invention, the process includes inserting content into the video stream. According to some embodiments of the invention, the process includes producing video effects in the video stream. According to some embodiments of the invention, the process includes content protection processing with respect to the video stream. According to some embodiments of the invention, the process includes searching in the video stream.
According to some embodiments of the invention, the storage devices use a data protection scheme to store the changed segment of the data stream.
According to some embodiments of the invention, further including producing and storing metadata including information about a bit rate of the changed segment of the data stream.
According to an aspect of some embodiments of the present invention there is provided a method for combining a plurality of data streams including using one or more storage devices for processing at least one segment of a data stream, thereby producing a changed segment of the data stream, and providing output of at least the changed segment of the data stream, and combining the output of the storage devices into a combined data stream. According to some embodiments of the invention, the processing includes changing a bit rate.
According to some embodiments of the invention, further including storing the at least the changed segment of the data stream.
According to some embodiments of the invention, further including providing bit rate control instructions to the plurality of storage devices, and the storage devices accepting the bit rate control instructions from the server and changing the bit rate of the segment of a data stream according to the instructions.
According to some embodiments of the invention, at least one of the data streams is a video data stream. According to some embodiments of the invention, the processing includes video stream processing.
According to some embodiments of the invention, further including, after the changing, storing the changed segment of the data stream, and performing the providing upon a request from the server.
According to some embodiments of the invention, further including producing and storing metadata including information about a bit rate of the changed segment of the data stream.
According to an aspect of some embodiments of the present invention there is provided a system configured to receive a transport stream including a plurality of data streams including a server configured to split the transport stream into a plurality of data streams, a plurality of storage devices configured to receive and store one or more of the data streams from the server, at least one of the storage devices being further configured to process at least one segment of a data stream, thereby producing a changed segment of the data stream, and store at least the changed segment of the data stream. According to some embodiments of the invention, the process includes changing the bit rate.
According to some embodiments of the invention, at least one of the data streams includes a video stream, and the process includes video stream processing.
According to an aspect of some embodiments of the present invention there is provided a method for receiving a transport stream including a plurality of data streams including splitting the transport stream into a plurality of data streams, storing the data streams on a plurality of storage devices, using at least one of the storage devices for processing at least one segment of a data stream, thereby producing a changed segment of the data stream, and storing the changed segment of the data stream.
According to some embodiments of the invention, the processing includes changing a bit rate.
According to some embodiments of the invention, at least one of the data streams is a video stream, and the processing includes video stream processing.
Unless otherwise defined, all technical and/or scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the invention pertains. Although methods and materials similar or equivalent to those described herein can be used in the practice or testing of embodiments of the invention, exemplary methods and/or materials are described below. In case of conflict, the patent specification, including definitions, will control. In addition, the materials, methods, and examples are illustrative only and are not intended to be necessarily limiting.
Implementation of the method and/or system of embodiments of the invention can involve performing or completing selected tasks manually, automatically, or a combination thereof. Moreover, according to actual instrumentation and equipment of embodiments of the method and/or system of the invention, several selected tasks could be implemented by hardware, by software or by firmware or by a combination thereof using an operating system.
For example, hardware for performing selected tasks according to embodiments of the invention could be implemented as a chip or a circuit. As software, selected tasks according to embodiments of the invention could be implemented as a plurality of software instructions being executed by a computer using any suitable operating system. In an exemplary embodiment of the invention, one or more tasks according to exemplary embodiments of method and/or system as described herein are performed by a data processor, such as a computing platform for executing a plurality of instructions. Optionally, the data processor includes a volatile memory for storing instructions and/or data and/or a non- volatile storage, for example, a magnetic hard-disk and/or removable media, for storing instructions and/or data. Optionally, a network connection is provided as well. A display and/or a user input device such as a keyboard or mouse are optionally provided as well.
BRIEF DESCRIPTION OF THE DRAWINGS
Some embodiments of the invention are herein described, by way of example only, with reference to the accompanying drawings. With specific reference now to the drawings in detail, it is stressed that the particulars shown are by way of example and for purposes of illustrative discussion of embodiments of the invention. In this regard, the description taken with the drawings makes apparent to those skilled in the art how embodiments of the invention may be practiced. In the drawings:
Fig. 1 is a simplified illustration of a system constructed and operative according to an exemplary embodiment of the invention;
Fig. 2 is a simplified flow chart illustration of a method operative according to an exemplary embodiment of the invention; Fig. 3 is a more detailed illustration of an example embodiment of the system of Fig. 1;
Fig. 4 is a simplified graphical illustration of variable bit rates in a video stream; and
Fig. 5 is a simplified graphical illustration of how chopping off bit rate peaks reduces a maximal bit rate of a video stream in an embodiment of the invention.
DESCRIPTION OF SPECIFIC EMBODIMENTS OF THE INVENTION
The present invention, in some embodiments thereof, relates to a digital video server and, more particularly, but not exclusively, to a digital video server for video- on-demand services.
Before explaining at least one embodiment of the invention in detail, it is to be understood that the invention is not necessarily limited in its application to the details of construction and the arrangement of the components and/or methods set forth in the following description and/or illustrated in the drawings and/or the examples. The invention is capable of other embodiments or of being practiced or carried out in various ways.
An aspect of some embodiments of the invention relates to using a distributed data storage system for distributed processing of data stored on the distributed storage nodes. One or more computers may control the processing, and data storage nodes, which also have some processing capability, perform at least some of the processing. Some types of processing are computation intensive, such as, by way of a non- limiting example, video stream processing. Non-limiting examples of computation intensive video stream processing include:
1. Rate shaping video streams, that is, changing the bit rate of video streams. Often the changing is a lowering of the bit rate of a video stream in order to fir the video stream into a narrower bandwidth.
2. Transcoding video streams, from one format to another, such as, by way of a non-limiting example, MPEG2 to MPEG4 or vice versa, High Definition (HD) to Standard Definition (SD) or vice versa.
3. Content processing, such, by way of a non-limiting example, inserting additional content such as a banner into a video stream, a logo into a video stream, producing a Picture-In-Picture (PIP) video stream, and adding a voice overlay or a video overlay.
4. Video effect processing, such as, by way of a non-limiting example, fade-in, fade-out, alpha blending, changing brightness, and a wipe effect. 5. Content protection processing, such as, by way of a non-limiting example, encryption, decryption, inserting a watermark, deleting a watermark.
6. Searching within a video stream, such as, by way of a non-limiting example, searching for a caption, and/or other searchable attributes.
Some embodiments contemplated include combining data streams stored on data storage nodes into a transport stream, while processing at least some of the data streams.
Some embodiments contemplated include receiving a transport stream containing data streams, splitting the transport stream into data streams for storage in storage devices, while having some of the storage devices process at least part of the data streams, and storing the processed part.
Examples of specific embodiments are provided herein with reference to Variable Bit Rate (VBR) video streams, such as MPEG video streams, being combined into a Constant Bit Rate (CBR) transport stream. The examples are to be understood in a broader context of VBR data streams being combined into a CBR transport stream.
Other specific embodiments contemplated include receiving a transport stream containing video streams, splitting the transport stream into video streams for storage in storage devices, and instructing some of the storage devices to process at least part of the video streams and store the changed, processed part.
An aspect of some embodiments of the invention relates to controlling a bit rate of a transport stream by controlling the bit rate of video stream components of the transport stream. Changing a bit rate of a video stream can be computation intensive, and changing the bit rate of the video stream components is performed in a distributed fashion, so as to spread the computation load.
It is noted that often video streams are stored on distributed data disks, and combined into a transport stream by a streamer, which is a dedicated computer. The data disks and the streamer are likely to be expensive pieces of equipment, dedicated to their intended tasks. An alternative approach to the above is using a lower cost general purpose computer, also termed a server, as a streamer, and using low cost storage on low cost general purpose computers as the distributed storage. The alternative approach produces an additional advantage, which is that the distributed storage also has processing power, opening the door to distributed processing.
An aspect of some embodiments of the invention relates to other types of processing of the transport stream, being performed in a distributed fashion. The invention is suitable for distributing processing of stream components.
The term "changing of bit rate" in all its grammatical forms is used throughout the present specification and claims interchangeably with the term "rate shaping" and the term "rate reducing" and their grammatical forms.
The term "video stream" in all its grammatical forms is used throughout the present specification and claims interchangeably with the terms "stream",
"data stream", "data file", "file", "MPEG stream", and "MPEG video stream" and their grammatical forms.
It is noted that the term video stream can refer to a stream which is limited in time, such as, by way of a non-limiting example, a movie, a television program, and a 30 second commercial, and the term can refer to a video stream which is not limited in time, such as a 24 hour a day program.
Apparatus which combines streams into a transport stream is termed herein a streamer. Reference is now made to Fig. 1, which is a simplified illustration of a system constructed and operative according to an exemplary embodiment of the invention.
The system comprises a streamer 100, connected to one or more storage clusters 105.
The streamer 100 is configured to provide an output 120 or outputs 120 of one or more transport streams.
The streamer 100 requests the storage clusters 105 to send it video streams (not shown), and combines the video streams into the one or more transport streams.
If there is a need for processing to be performed on one or more of the video streams, the streamer 100 instructs one or more of the storage clusters 105 to perform the processing and provide a changed, processed, video stream for the combining. By way of a non-limiting example, the streamer 100 tracks the bit rate of each of the transport streams. If a bit rate of a transport stream is about to exceed a pre-defined limit, the streamer 100 ensures, as will be further described below, that the bit rate of one or more of the video streams combined into the transport stream is lowered, thereby ensuring that the bit rate of the transport stream does not exceed the limit.
The streamer 100 instructs one or more of the storage clusters 105 to provide a version of one or more of video streams having a lower than nominal bit rate.
The storage clusters 105 provide the streamer 100 with a lower bit rate version of the one or more video stream which the streamer instructed to be at a lower bit rate. Reference is now made to Fig. 2, which is a simplified flow chart illustration of a method operative according to an exemplary embodiment of the invention.
Fig. 2 depicts a flow chart of the method used in the system of Fig. 1, which uses one or more storage devices (150) for processing at least one segment of one of the data streams (160). An optional method of processing used by an embodiment of the invention is described further below with reference to Fig. 5.
Output of at least the changed data stream is provided (165), as described above with reference to Fig. 1, and will be described further below with reference to Fig. 3.
A streamer is used for combining the output of the storage devices into a combined data stream (170), as described above with reference to Fig. 1, and will be described further below with reference to Fig. 3. Reference is now made to Fig. 3, which is a more detailed illustration of an example embodiment of the system of Fig. 1.
The system comprises a streamer 100, connected to one or more storage clusters 105. The storage clusters 105 each contain one or more storage machines 110, each of which is connected to one or more storage devices 115. The streamer 100 optionally instructs one or more of the storage clusters 105 to provide a version of one or more of video streams having a lower than nominal bit rate.
The storage clusters 105 optionally provide the streamer 100 with a lower bit rate version of the one or more video streams which the streamer instructed to be at a lower bit rate.
The streamer 100
The streamer 100 combines several video streams into a transport stream. The streamer optionally ensures that a sum of the bit rates of the video streams does not exceed a bit rate limit associated with the transport stream. Combining VBR video streams into a CBR transport stream, when the total bit rate of the video streams is less than the bit rate limit, is known in the art. For example, one way of doing so, is tracking the combined bit rate, and adding null data to the transport stream as needed to bring the bit rate of the combined-transport stream up to a constant value. A more efficient use of a CBR transport stream is to combine as many video streams as possible into the transport stream. Since the video streams show a variable bit rate, it is possible to combine the video streams such that their combined bit rate will usually be below the limit of the transport stream. When the combined bit rate of the video streams is expected to exceed the limit, the streamer 100 optionally replaces some or all of the video streams with lower bit rate versions of the video streams, and prevents the exceeding. The average bit rate of each VBR video file or video file segment participating in a transport stream is optionally derived based on the following parameters: a modulation type for the MPTS, such as, by way of a non-limiting example, Quadrature
Amplitude Modulation (QAM) type such as 64/128/256 Symbol Rate; and the number of streams participating in the MPTS . Reference is now made to Fig. 4, which is a simplified graphical illustration of variable bit rates in a video stream.
Fig. 4 depicts a graph 200 of bit rate of a segment of a video stream vs. time.
The graph is of a segment of an original video stream with an original, unmodified, bit rate. The horizontal axis 205 of the graph 200 depicts time, in units of video frames. Since video is typically displayed at a constant frame rate, frames are directly proportional to time. The vertical axis 210 of the graph 200 depicts bit rate, in units of
Mbps.
The segment contains several GOPs 215, 216, 217, 218. Each of the GOPs
215, 216, 217, 218, comprises individual pictures, one bar of the graph 200 standing for each picture. Each of the pictures has an inherent bit rate associated, based on the amount of data required to portray the picture, and a frame rate of the video stream.
The inherent bit rate is depicted by the relative height of the bar which stands for the picture.
Since each of the GOPs 215, 216, 217, 218 is compressed together, each GOP is also associated with a bit rate. The bit rate of a GOP is based on the amount of data, after compression, which the GOP contains, a frame rate of the video stream, and the number of pictures within the GOP.
GOP bit rates 220, 221, 222, 223 are depicted by dotted lines corresponding to the bit rates of the GOPs 215, 216, 217, 218 respectively. The GOP bit rates 220, 221, 222, 223 are not equal, depending as they do on contents of the pictures of the video stream.
A maximal bit rate of the segment in the illustration of Fig. 4 is the GOP bit rate 221, associated with the GOP 2 216.
Reference is now made to Fig. 55 which is a simplified graphical illustration of how chopping off bit rate peaks reduces a maximal bit rate of a video stream in an embodiment of the invention. Fig. 5 depicts the segment of Fig. 4, in which bit rate peaks of individual pictures have been modified by "chopping off, that is reducing, just the bit rate peaks. The values of the unmodified picture bit rates and the unmodified GOP bit rates 220, 221, 222, 223 are still depicted in Fig. 5, for purpose of comparison and to aid understanding. A picture in GOP 2 216, which had a highest inherent bit rate of the pictures of
GOP 2, was reduced in bit rate by a first amount 225. A GOP bit rate 231 of GOP 2 216 is lower than the GOP bit rate 221 of GOP 2 216 before bit rate modification.
A picture in GOP 3 217, which had a highest inherent bit rate of the pictures of GOP 3 217, was reduced in bit rate by a second amount 226, different from the first amount 225. A GOP bit rate 232 of GOP 3 217 is lower than the GOP bit rate 222 of GOP 3 217 before bit rate modification.
Two pictures in GOP 4 218, which had the two highest inherent bit rate of the pictures of GOP 4 218, were reduced in bit rate by a third amount 227 and a fourth amount 228. A GOP bit rate 233 of GOP 4 218 is lower than the GOP bit rate 223 of GOP 4 218 before bit rate modification.
The files which together make up a transport stream change over time. For example, when the streamer is a VOD server, sessions are allocated by end users, and the streamer 100 is required to read files from the storage cluster 105. The streamer instructs each processing unit 110 in the storage cluster 105 to reduce the rate of certain GOPs according to a statistical multiplexing algorithm.
Since rate shaping can be computationally intensive, the streamer 100 optionally causes rate shaping of video streams to be performed by computing power available in the storage clusters 105 (Fig. 1). While storage clusters 105 typically contain a large amount of storage, which is needed for storing the video streams, the storage clusters are nevertheless capable of computation. Having several storage clusters 105 enables using the computation power built into a distributed video storage system for also rate shaping the video. It is noted that, similarly to the example of Fig. 5, rate shaping is optionally not performed on an entire file, but rather on a file segment. The size of the file segment ranges from an entire file, to a portion of a file, down to a GOP.
Statistical Multiplexing (Statmux) decision making
Upon having an MPTS or a group of SPTS streams directed at a specific output with a specified maximum allowed bit rate, the streamer 100 (Fig. 1) starts reading one or more streams from one of the storage clusters 105,
Along with the streams being read, the streamer 100 reads metadata associated with each stream. More detail about the metadata is provided below, with reference to "Ingesting a Video File".
Based at least partly on the metadata, the streamer 100 can predict what will be the aggregated bit rate of all streams in the MPTS, or group of SPTS streams going into a specific output. The metadata also contains information about complexity of each stream, and enables the streamer 100 to predict a level of deterioration in video stream quality due to rate reduction.
By talcing the metadata parameters into consideration, the streamer compiles a set of commands to the storage clusters 105, requesting the storage clusters 105 for relevant data segments with a specific bit rate per GOP. In some embodiments of the invention, the streamer 100 is a component combining the streams into the data stream, and the component which considers the metadata and issues commands to the storage clusters 105 is a stream management server (not shown). The streamer 100 and the stream management server may or may not be separately housed and/or separately located.
SPTS
For legacy transport streams which do not support MPTS as their input, the statistical multiplexing method described above may be distributed among several
SPTS and several streamers 100. Each streamer 100 manages one or more transport streams, and instructs relevant storage machines 110, via one or more storage clusters
105, to reduce the rate of video streams when necessary.
Each VBR program stream is padded with null stuffing into a CBR SPTS at the maximum allowed bit rate, and sent to a suitable output.
A distributed statmuxing method optionally manages an aggregated bandwidth of all streams going into a specific output transport stream, even when some video streams originate from different streamers machine,
Trick play
During play out of a video stream, an end user may choose to use trick play modes, such as Fast Forward (FF), rewind, and pause. This section describes performance of the invention in case at least one of the video streams is played out using a trick mode.
Optionally, In FF and rewind modes, only a first I-frame from a GOP is requested and received by the streamer 100, and not all pictures of the GOP. Managing the bit rate is optionally done by the streamer 100, by adding a suitable number of P- frames to I-frames and forming newly generated GOPs. If the bit rate of a GOP comprising I-frames only is still too high, bit rate reduction is optionally requested by the streamer 100 from the storage machines, in order to reduce the bit rate of the I-frame. Alternatively, the streamer 100 performs rate reduction for the I-frames.
In pause mode, the streamer 100 stops transmitting the video stream, so there is no need to use rate reduction for a paused video stream,
The streamer 100 takes pause mode into consideration, and for example, in case of a paused video stream, other streams in the transport stream may use more bit rate.
Rate shaping
Optionally, rate shaping is done in real time, or approximately in real time, that is, just prior to transmitting the transport stream. Alternatively, the rate shaping is done ahead of real time, and changed bit rate video streams are saved until needed.
Whether the rate shaping is performed in real time or ahead of time, each of the storage machines 110 which performs rate shaping optionally receives instructions for rate shaping from the streamer 100, which include a bit rate parameter which specifies a maximum bit rate, or by how much to reduce the bit rate of a file segment. Alternatively, each storage machine 110 is configured for one or more configurable levels of rate shaping.
Optionally, rate reduction reduces quality of images in frames of each GOP, so that a maximum bit rate is not exceeded. Alternatively the rate reduction reduces the bit rate by a percentage of the bit rate, or by subtracting a bit rate from an unmodified bit rate.
Reducing the bit rate is optionally performed over a period of time, corresponding to a certain segment of the video stream, and the amount of reduction may change after the period of time. In some embodiments of the invention, each storage cluster 105 handles an internal reallocation of a bit rate budget within each GOP, for example, allocating more or less bit rate to different frames within a GOP based on the frame's complexity, optionally providing some areas of the frames with a specific treatment.
Rate shaping in approximately real time
When rate shaping is performed in real time or approximately real time, the streamer combines several video streams into the transport stream based, at least in part, on a total bit rate of the video streams and optionally on a bit rate limit for the transport stream. It is noted that transport streams sometimes carry more than just video streams.
The data carried by the transport stream beyond the video streams is hereby considered a data stream, and the description of data streams is taken to be included in the video stream examples.
Optionally, the streamer 100 estimates the bit rate of the transport stream, and optionally estimates the bit rate of each of the video streams. Optionally, the estimation is done by the streamer actually measuring the bit rate of the transport stream and optionally measuring the bit rate of each of the video streams.
The streamer 100 instructs the storage machines 110 which video streams to rate shape, and how to rate shape the video streams. In some embodiments of the invention the streamer 100 instructs the storage machines 110 which video stream and at what bit rate to provide the streamer. In other embodiments of the invention the streamer 100 instructs the storage machines 110 which video stream to provide the streamer 100, and by what amount, or by what percentage, to lower the bit rate of the video stream.
Optionally the storage machines 110 which rate shape a video stream, also cache the rate shaped video stream, so that of the video stream needs to be output again as a rate shaped video stream, the cached rate shaped video stream is recovered from the cache.
Rate shaping ahead of real time
When rate shaping is done ahead of real time, the streamer 100 optionally instructs the storage clusters 105 which video stream to rate shape, to what bit rate to shape the video stream, and where to store the rate shaped video stream.
In other embodiments of the invention the streamer 100 instructs the storage clusters 105 which video stream to rate shape, by what amount, or by what percentage, to lower the bit rate of the video stream, and where to store the rate shaped video stream.
When rate shaping is done ahead of real time, metadata is optionally produced, associating some or all of the saved video streams with an estimated or measured bit rate. Optionally the metadata includes the bit rate of segments of the video streams, rather than a bit rate for an entire video stream. Optionally, the metadata includes the bit rate for each individual GOP.
Optionally, the metadata for a video stream or for a segment of a video stream includes a maximum bit rate as well as an average bit rate.
Optionally, a video stream is stored in more than one version, with different versions having been rate shaped and comprising different bit rates. Optionally, only some segments of the video stream are rate shaped, the rate shaped segments preferably being the higher bit rate segments.
Optionally, the segments of the video stream which are rate shaped are as small as one GOP.
Since in the example described above rate shaping is performed ahead of real time, there may not be certainty that rate shaped versions or segments of a video stream will actually be used. In some embodiments of the invention deciding which video streams or segments thereof will be rate shaped is performed according to projected statistics of video demand. Ia some embodiments of the invention the deciding is performed according to actual video demand.
Various schemes for saving computation are contemplated, such as: rate shaping only video streams which are most in demand; rate shaping only high bit rate segments of video streams; rate shaping most-in-demand video streams in advance of real time, and rate shaping other video streams in real time or approximate real time; and rate shaping a video stream in real time when first combined into a transport stream, and saving the rate shaped video stream or rate shaped segments of the video stream, and their associated metadata, for later re-use. Various schemes for saving storage space are contemplated, such as: saving rate shaped versions only of video streams which are most in demand; saving rate shaped versions of only high bit rate segments of video streams; and saving rate shaped versions of a video stream only after a first combining into a transport stream.
Storage
An exemplary embodiment of the invention includes a storage cluster 105, which comprises several storage machines, also termed data servers. Each storage machine has one or more storage devices, and some processing power.
Optionally, the storage cluster 105 protects data by storing redundant data. The protection is performed using schemes such a RAID. It is to be noted that the protection is configured per file segment. The file segment corresponds to a video stream, segment of a video stream, and even a GOP of a video stream.
Rate Reduction with Data Protection In case there is data loss on a first one of the storage clusters 105, the streamer
100 requests rate reduced file segments from a second one of the storage clusters 105. The second storage cluster 105 gathers the required file segments from the second storage cluster 105. The second storage cluster 105 performs rate reduction on the recovered file segments.
Fallback Copy
Optionally, each file is stored in a predefined rate reduced version on the storage cluster. The version is optionally either a CBR version, or a capped VBR with a maximum bit rate which is not higher then the average bit rate of the original stream. The predefined rate reduced version is termed a fallback copy.
In case there is data loss of an original version, or if there is no available processing power to reduce the rate of the original version, the fallback copy is optionally used. The fallback copy is also optionally used if the bit rate of the fallback copy falls inside the limits of the bit rate required by the streamer 100.
Caching of Processed GOPs Each storage machine 110 optionally has a cache containing already rate reduced GOPs. The cache is optionally based upon an LRU method. Some embodiments of the invention use a cached version of a rate reduced GOP rather than utilize rate reduction. The preference for using the cache, and a size of the cache, are determined based at least partly on the available processing power of the storage machine 110.
By way of a non-limiting example, when the streamer 100 requests a certain
GOP at a bit rate of 2.5Mbps, and if there is a cached version of the same GOP reduced to 2.4Mbps, the system optionally chooses to use the cached 2.4 Mbps version. This frees the storage machine 110 to rate shape other GOPs that are not already cached .
The storage machine 110 optionally has a dedicated amount of storage and storage IO for caching purposes.
Rate Reduction and "flash-crowd" A data server may optionally get multiple requests for rate shaping a file segment, from multiple streamers, or optionally multiple requests from a single streamer but intended to be combined in an MPTS the single streamer processes.
Such a scenario can be common when many users would like to watch a specific file at substantially the same moment, for instance a newly issued popular event. The data server which stores the requested video segment receives many requests for the same segment, more than it can process. The data node reaches a point where it is not able to serve all the requests and maintain an approximately real-time response.
In some embodiments of the invention, the data server aggregates requests and chooses to process only several rate shaping commands, such that will not exceed its processing power and its read-from-storage capabilities. In case a specific rate reduction request wasn't selected for real-time processing, the data server optionally provides a copy of a closest rate shaped version, or a closest rate shaped version with less than the required bit rate, available of the requested rate shaped segment.
Ingesting a Video File
An ingested video file is typically encoded to a VBR format. The streamer 100 parses the video file, and breaks the video file into file segments.
In some embodiments of the invention the streamer 100 optionally produces and stores, optionally on one or more storage clusters 105, metadata about the stream behavior and complexity. The metadata is optionally used later for statistical multiplexing decision making.
During the ingestion process, and optionally during real time rate shaping, the metadata for the video file is generated, and kept in a separate file. The information saved in the metadata file contains information about the stream bit rate, frame structure and complexity.
Data segments
Video streams are optionally striped across storage devices 115 in the storage machines 110 and optionally also across storage machines 110 in a storage cluster 105. Each file is optionally cut into segments which are optionally distributed among the storage devices. Each segment contains one or more GOPs belonging to the file.
It is to be noted that each storage device may have a segment from each file, and vice versa, segments from each file may appear on each storage device. The striping and distribution of the files among the storage machines, coupled with the rate shaping requirements made of the storage machines, cause a distribution of the computation load among the storage machines. The number of GOPs per segment is determined in order to optimize throughput of a clustered file system. The number of GOPs per segment is determined based, at least partly, on storage parameters such as block sizes in the storage device, on computation parameters such as computation speed on the storage machine, on the number of computation machines, and on a bit rate of the segment. For example, when a segment size is determined to be 1 MB, the segment can contain 10 GOPs of a Standard Definition (SD) video stream, or 5 GOPs of a High Definition (HD) stream.
It is noted that when an optimal size of a segment is not completely filled by GOPs, padding is optionally applied. In some embodiments of the invention, a segment is smaller than an MPEG
GOP. A segment can be, by way of a non-limiting example, a frame; a number of frames, with no relation to the underlying MPEG GOP; and even part of a frame, even as small as a slice of an MPEG macroblock.
It is noted that storing am MPEG GOP together can save time in assembling data for rate shaping.
It is noted that once a segment has been rate shaped, the rate shaped segment can optionally be stored on one data server, and can optionally be stored on more than one data server.
In some embodiments of the invention, especially where MPEG2 stream are concerned, a segment is minimally chosen to be one or more GOPs, that is all frames from one I-frame to a next I-frame, which fit into and are aligned to a 1MB data block.
Data protection
Each segment of data written into storage is optionally protected by a data protection scheme, by way of a non-limiting example a data protection scheme using data redundancy, such as RAID. The level of RAID protection is any level of RAID protection, such as RAID level 0, 1, 2, 3, 4, 5, and so on, and may optionally be different per file. Each segment is thus protected by redundant data, which is written into the storage cluster, to different storage devices, on different storage machines. In some embodiments of the invention protection management is provided by the storage cluster. In some embodiments of the invention protection management is provided by each of the storage machines. In some embodiments of the invention protection management is provided by the streamer.
A non-limiting example of protection functionality is now provided. A streamer attempts to fetch a file segment from a first storage device on a first storage machine. Due to failure of the first storage device or the first storage machine, the data segment is unavailable.
Recovery option 1: when the streamer manages the protection scheme, the streamer fetches redundant data written by the protection algorithm on a second storage device, and reconstructs the unavailable data. The streamer does not necessarily rate shape the reconstructed data. If rate shaping is necessary, the streamer optionally instructs the storage clusters providing other streams to provide the other streams at lower bit rate.
Recovery option 2: when each of the storage machines manages the protection scheme, the streamer requests a second storage machine to fetch the unavailable data. The second storage machine retrieves the redundant data, and reconstructs the previously unavailable data using the protection scheme. The storage machine then performs rate shaping on the reconstructed segment, and deliver the rate shaped and recovered data to the streamer.
Recovery option 3: when the streamer manages the protection scheme, the streamer fetches redundant rate shaped data from the storage cluster, and reconstructs the unavailable data according to the protection scheme.
Additional features
The storage machine 110 optionally overlays text, video, and/or sound over file segments, according to instructions received from the streamer 100. The overlay is optionally performed in the compressed domain.
Each storage machine 110 can optionally convert video segments from one video format to another, using a transcoding algorithm. The transcoding is optionally per data segment, so that each storage machine 110 is only required to transcode segments which are stored thereon. It is expected that during the life of a patent maturing from this application many relevant rate shaping methods will be developed and the scope of the term rate shaping is intended to include all such new technologies a priori. The terms "comprises", "comprising", "includes", "including",
"having", and their conjugates, mean "including but not limited to".
The term "consisting of means "including and limited to".
The term "consisting essentially of means that the composition, method or structure may include additional ingredients, steps and/or parts, but only if the additional ingredients, steps and/or parts do not materially alter the basic and novel characteristics of the claimed composition, method or structure.
As used herein, the singular form "a", "an" and "the" include plural references unless the context clearly dictates otherwise. For example, the term "a video stream" or "at least one video stream" may include a plurality of video streams, including combinations thereof.
It is appreciated that certain features of the invention, which are, for clarity, described in the context of separate embodiments, may also be provided in combination in a single embodiment. Conversely, various features of the invention, which are, for brevity, described in the context of a single embodiment, may also be provided separately or in any suitable sub-combination or as suitable in any other described embodiment of the invention. Certain features described in the context of various embodiments are not to be considered essential features of those embodiments, unless the embodiment is inoperative without those elements. Although the invention has been described in conjunction with specific embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art. Accordingly, it is intended to embrace all such alternatives, modifications and variations that fall within the spirit and broad scope of the appended claims. All publications, patents and patent applications mentioned in this specification are herein incorporated in their entirety by reference into the specification, to the same extent as if each individual publication, patent or patent application was specifically and individually indicated to be incorporated herein by reference. In addition, citation or identification of any reference in this application shall not be construed as an admission that such reference is available as prior ail to the present invention. To the extent that section headings are used, they should not be construed as necessarily limiting.

Claims

WHAT IS CLAIMED IS:
1. A distributed stream processing system comprising: a plurality of storage devices, at least one of which is configured to: process at least one segment of a data stream, thereby producing a changed segment of the data stream; and provide output of at least said changed segment of the data stream, and a server configured to accept the output of the storage devices and combine the output into a transport stream.
2. The system of claim 1 in which the process comprises changing the bit rate.
3. The system of any one of claims 1 and 2 in which the at least one storage device is further configured to store at least the changed segment of the data stream.
4. The system of any one of claims 1-3 in which the server is configured to provide bit rate control instructions to the plurality of storage devices, and the plurality of storage devices are configured to accept the bit rate control instructions from the server and to change the bit rate of said segment of a data stream according to the instructions.
5. The system of any one of claims 1-4 in which at least one of the data streams is a video stream.
6. The system of claim 5 in which the process comprises transcoding the video stream.
7. The system of any one of claims 5 and 6 in which the process comprises inserting content into the video stream.
8. The system of any one of claims 5-7 in which the process comprises producing video effects in the video stream.
9. The system of any one of claims 5-8 in which the process comprises content protection processing with respect to the video stream.
10. The system of any one of claims 5-9 in which the process comprises searching in the video stream.
11. The system of any one of claims 1 -10 in which the storage devices use a data protection scheme to store said changed segment of the data stream.
12. The system of any one of claims 1-11 and further comprising producing and storing metadata comprising information about a bit rate of said changed segment of the data stream.
13. A method for combining a plurality of data streams comprising: using one or more storage devices for: processing at least one segment of a data stream, thereby producing a changed segment of the data stream; and providing output of at least said changed segment of the data stream, and combining the output of the storage devices into a combined data stream.
14. The method of claim 13 in which the processing comprises changing a bit rate.
15. The method of any one of claims 13 and 14 and further comprising storing the at least the changed segment of the data stream.
16. The method of any one of claims 13-15 and further comprising providing bit rate control instructions to the plurality of storage devices, and the storage devices accepting the bit rate control instructions from the server and changing the bit rate of said segment of a data stream according to the instructions.
17. The method of any one of claims 13 - 16 in which at least one of the data streams is a video data stream.
18. The method of claim 13 in which the processing comprises video stream processing.
19. The method of any one of claims 13-18 and further comprising, after the changing, storing said changed segment of the data stream, and performing the providing upon a request from the server.
20. The method of any one of claims 13-19 and further comprising producing and storing metadata comprising information about a bit rate of said changed segment of the data stream.
21. A system configured to receive a transport stream comprising a plurality of data streams comprising: a server configured to split the transport stream into a plurality of data streams; a plurality of storage devices configured to receive and store one or more of the data streams from the server, at least one of the storage devices being further configured to: process at least one segment of a data stream, thereby producing a changed segment of the data stream; and store at least the changed segment of the data stream.
22. The system of claim 21 in which the process comprises changing the bit rate.
23. The system of any one of claims 21 and 22 in which at least one of the data streams comprises a video stream, and the process comprises video stream processing.
24. A method for receiving a transport stream comprising a plurality of data streams comprising: splitting the transport stream into a plurality of data streams; storing the data streams on a plurality of storage devices; using at least one of the storage devices for processing at least one segment of a data stream, thereby producing a changed segment of the data stream; and storing the changed segment of the data stream.
25. The method of claim 24 in which the processing comprises changing a bit rate.
26. The method of any one of claims 24 and 25 in which at least one of the data streams is a video stream, and the processing comprises video stream processing.
PCT/IL2008/000377 2008-03-18 2008-03-18 A controlled rate vod server WO2009116024A2 (en)

Priority Applications (9)

Application Number Priority Date Filing Date Title
ES08720002T ES2816639T3 (en) 2008-03-18 2008-03-18 A speed controlled VOD server
CA2717882A CA2717882A1 (en) 2008-03-18 2008-03-18 A controlled rate vod server
EP08720002.8A EP2269143B1 (en) 2008-03-18 2008-03-18 A controlled rate vod server
CN2008801293695A CN102047236A (en) 2008-03-18 2008-03-18 A controlled rate vod server
US12/933,435 US8732786B2 (en) 2008-03-18 2008-03-18 Controlled rate VOD server
PCT/IL2008/000377 WO2009116024A2 (en) 2008-03-18 2008-03-18 A controlled rate vod server
EP20180944.9A EP3748514B1 (en) 2008-03-18 2008-03-18 A controlled rate vod server
PT201809449T PT3748514T (en) 2008-03-18 2008-03-18 A controlled rate vod server
US14/280,736 US9282347B2 (en) 2008-03-18 2014-05-19 Controlled rate VOD server

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/IL2008/000377 WO2009116024A2 (en) 2008-03-18 2008-03-18 A controlled rate vod server

Related Child Applications (2)

Application Number Title Priority Date Filing Date
US12/933,435 A-371-Of-International US8732786B2 (en) 2008-03-18 2008-03-18 Controlled rate VOD server
US14/280,736 Continuation US9282347B2 (en) 2008-03-18 2014-05-19 Controlled rate VOD server

Publications (2)

Publication Number Publication Date
WO2009116024A2 true WO2009116024A2 (en) 2009-09-24
WO2009116024A3 WO2009116024A3 (en) 2010-02-25

Family

ID=41091305

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IL2008/000377 WO2009116024A2 (en) 2008-03-18 2008-03-18 A controlled rate vod server

Country Status (7)

Country Link
US (2) US8732786B2 (en)
EP (2) EP3748514B1 (en)
CN (1) CN102047236A (en)
CA (1) CA2717882A1 (en)
ES (1) ES2816639T3 (en)
PT (1) PT3748514T (en)
WO (1) WO2009116024A2 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2586201A1 (en) * 2010-06-22 2013-05-01 Verizon Patent and Licensing Inc. Enhanced media content transport stream for media content delivery systems and methods
US8732786B2 (en) 2008-03-18 2014-05-20 Fabrix Tv Ltd. Controlled rate VOD server
US11541010B2 (en) 2009-07-09 2023-01-03 Biontech Delivery Technologies Gmbh Amphoteric liposomes comprising imino lipids

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10003851B2 (en) * 2009-11-24 2018-06-19 Imagine Communications Corp. Managed multiplexing of video in an adaptive bit rate environment
JP5625997B2 (en) * 2011-02-23 2014-11-19 富士通株式会社 Communication system and transmission apparatus
US8661479B2 (en) * 2011-09-19 2014-02-25 International Business Machines Corporation Caching large objects with multiple, unknown, and varying anchor points at an intermediary proxy device
HK1157132A2 (en) * 2012-01-06 2012-06-15 Sengital Ltd A method and system of playing the streaming content and smart buffering management on network television
WO2013173721A1 (en) 2012-05-18 2013-11-21 Home Box Office, Inc. Audio-visual content delivery
KR101991214B1 (en) * 2013-03-06 2019-06-19 인터디지탈 패튼 홀딩스, 인크 Power aware adaptation for video streaming
US9438652B2 (en) 2013-04-15 2016-09-06 Opentv, Inc. Tiered content streaming
US20150312601A1 (en) * 2014-04-28 2015-10-29 Magnum Semiconductor, Inc. Methods and apparatuses including a statistical multiplexer with multiple channel rate control
US9654804B2 (en) 2014-09-03 2017-05-16 Vigor Systems Inc. Replacing video frames in a transport stream
DE102014220423A1 (en) * 2014-10-08 2016-04-14 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Apparatus for generating a video output data stream, video source, video system and method for generating a video output data stream or a video source data stream, respectively
JP6642016B2 (en) * 2016-01-12 2020-02-05 船井電機株式会社 Distribution equipment and information equipment

Family Cites Families (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5732217A (en) * 1995-12-01 1998-03-24 Matsushita Electric Industrial Co., Ltd. Video-on-demand system capable of performing a high-speed playback at a correct speed
US5978843A (en) * 1995-12-06 1999-11-02 Industrial Technology Research Institute Scalable architecture for media-on-demand servers
US20030040962A1 (en) * 1997-06-12 2003-02-27 Lewis William H. System and data management and on-demand rental and purchase of digital data products
US6285661B1 (en) 1998-01-28 2001-09-04 Picturetel Corporation Low delay real time digital video mixing for multipoint video conferencing
US6477706B1 (en) * 1998-05-01 2002-11-05 Cogent Technology, Inc. Cable television system using transcoding method
EP0971542A3 (en) * 1998-07-10 2001-08-01 Tektronix, Inc. Readjustment of bit rates when switching between compressed video streams
KR20010010109A (en) * 1999-07-16 2001-02-05 구자홍 Broadcasting Service System Based On Mobile Network
US6970939B2 (en) * 2000-10-26 2005-11-29 Intel Corporation Method and apparatus for large payload distribution in a network
US7242324B2 (en) * 2000-12-22 2007-07-10 Sony Corporation Distributed on-demand media transcoding system and method
US6947996B2 (en) * 2001-01-29 2005-09-20 Seabridge, Ltd. Method and system for traffic control
US7356079B2 (en) 2001-11-21 2008-04-08 Vixs Systems Inc. Method and system for rate control during video transcoding
US7403564B2 (en) * 2001-11-21 2008-07-22 Vixs Systems, Inc. System and method for multiple channel video transcoding
WO2006047722A2 (en) * 2004-10-27 2006-05-04 Eg Technology, Inc. Decentralized method for generating an mpeg-2 multiprogram transport stream
GB0428160D0 (en) * 2004-12-22 2005-01-26 British Telecomm Variable bit rate processing
US20060227813A1 (en) * 2005-04-11 2006-10-12 Mavrogeanes Richard A Method and system for synchronized video recording/delivery
US8077707B2 (en) * 2005-11-18 2011-12-13 Sri International Systems and methods for digital stream denting
US20070200949A1 (en) * 2006-02-21 2007-08-30 Qualcomm Incorporated Rapid tuning in multimedia applications
US20080235200A1 (en) * 2007-03-21 2008-09-25 Ripcode, Inc. System and Method for Identifying Content
US8543667B2 (en) * 2008-01-14 2013-09-24 Akamai Technologies, Inc. Policy-based content insertion
CA2717882A1 (en) 2008-03-18 2009-09-24 Fabrix Tv Ltd. A controlled rate vod server

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See references of EP2269143A4 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8732786B2 (en) 2008-03-18 2014-05-20 Fabrix Tv Ltd. Controlled rate VOD server
US9282347B2 (en) 2008-03-18 2016-03-08 Fabrix Tv Ltd. Controlled rate VOD server
US11541010B2 (en) 2009-07-09 2023-01-03 Biontech Delivery Technologies Gmbh Amphoteric liposomes comprising imino lipids
EP2586201A1 (en) * 2010-06-22 2013-05-01 Verizon Patent and Licensing Inc. Enhanced media content transport stream for media content delivery systems and methods
EP2586201A4 (en) * 2010-06-22 2014-12-24 Verizon Patent & Licensing Inc Enhanced media content transport stream for media content delivery systems and methods

Also Published As

Publication number Publication date
US9282347B2 (en) 2016-03-08
ES2816639T3 (en) 2021-04-05
EP2269143B1 (en) 2020-06-24
EP2269143A4 (en) 2012-02-01
US8732786B2 (en) 2014-05-20
EP2269143A2 (en) 2011-01-05
CA2717882A1 (en) 2009-09-24
US20110023080A1 (en) 2011-01-27
EP3748514A1 (en) 2020-12-09
US20140250476A1 (en) 2014-09-04
PT3748514T (en) 2024-07-18
WO2009116024A3 (en) 2010-02-25
EP3748514B1 (en) 2024-05-01
CN102047236A (en) 2011-05-04

Similar Documents

Publication Publication Date Title
US9282347B2 (en) Controlled rate VOD server
US11758103B2 (en) Video client optimization during pause
US8621543B2 (en) Distributed statistical multiplexing of multi-media
TWI511544B (en) Techniques for adaptive video streaming
US10602240B2 (en) Decoding method switching for preview image processing using a bundle of preview images
US20160219286A1 (en) Parallel video transcoding
US8789119B2 (en) Statistical remultiplexer performance for video on demand applications by use of metadata
US20130117418A1 (en) Hybrid platform for content delivery and transcoding
US9338204B2 (en) Prioritized side channel delivery for download and store media
US20240205467A1 (en) Delivery and playback of content
US9060184B2 (en) Systems and methods for adaptive streaming with augmented video stream transitions using a media server
KR20140007893A (en) A method for optimizing a video stream
US12126782B2 (en) Video client optimization during pause
WO2013163221A1 (en) Systems and methods for adaptive streaming with augmented video stream transitions
US10135896B1 (en) Systems and methods providing metadata for media streaming
Phillips et al. System and method for encoding 360 immersive video
JPH10108200A (en) Image coding method and its device
Ducloux et al. Improving streaming quality and bitrate efficiency with dynamic resolution selection
US20130287092A1 (en) Systems and Methods for Adaptive Streaming with Augmented Video Stream Transitions
EP2464105A1 (en) Method and system for data management

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 200880129369.5

Country of ref document: CN

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 08720002

Country of ref document: EP

Kind code of ref document: A2

WWE Wipo information: entry into national phase

Ref document number: 2717882

Country of ref document: CA

WWE Wipo information: entry into national phase

Ref document number: 12933435

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 2175/MUMNP/2010

Country of ref document: IN

WWE Wipo information: entry into national phase

Ref document number: 2008720002

Country of ref document: EP