EP2064884A2 - A method and an apparatus for data streaming - Google Patents

A method and an apparatus for data streaming

Info

Publication number
EP2064884A2
EP2064884A2 EP07802919A EP07802919A EP2064884A2 EP 2064884 A2 EP2064884 A2 EP 2064884A2 EP 07802919 A EP07802919 A EP 07802919A EP 07802919 A EP07802919 A EP 07802919A EP 2064884 A2 EP2064884 A2 EP 2064884A2
Authority
EP
European Patent Office
Prior art keywords
data
streaming
sub
sets
payload data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP07802919A
Other languages
German (de)
French (fr)
Inventor
Joachim Roos
Karl Henriksson
Lukas Holm
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Edgeware AB
Original Assignee
Edgeware AB
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Edgeware AB filed Critical Edgeware AB
Publication of EP2064884A2 publication Critical patent/EP2064884A2/en
Withdrawn legal-status Critical Current

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/16Analogue secrecy systems; Analogue subscription systems
    • H04N7/173Analogue secrecy systems; Analogue subscription systems with two-way working, e.g. subscriber sending a programme selection signal
    • H04N7/17309Transmission or handling of upstream communications
    • H04N7/17336Handling of requests in head-ends
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845Structuring of content, e.g. decomposing content into time segments
    • H04N21/8455Structuring of content, e.g. decomposing content into time segments involving pointers to the content, e.g. pointers to the I-frames of the video stream
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/40Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21Server components or server architectures
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21Server components or server architectures
    • H04N21/222Secondary servers, e.g. proxy server, cable television Head-end
    • H04N21/2225Local VOD servers
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/231Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
    • H04N21/23106Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion involving caching operations
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2347Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving video stream encryption
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/238Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams
    • H04N21/2387Stream processing in response to a playback request from an end-user, e.g. for trick-play
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/239Interfacing the upstream path of the transmission network, e.g. prioritizing client content requests
    • H04N21/2393Interfacing the upstream path of the transmission network, e.g. prioritizing client content requests involving handling client requests
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/266Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
    • H04N21/2665Gathering content from different sources, e.g. Internet and satellite
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41Structure of client; Structure of client peripherals
    • H04N21/414Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance
    • H04N21/4147PVR [Personal Video Recorder]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/432Content retrieval operation from a local storage medium, e.g. hard-disk
    • H04N21/4325Content retrieval operation from a local storage medium, e.g. hard-disk by playing back content from the storage medium
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4405Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving video stream decryption
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/462Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
    • H04N21/4623Processing of entitlement messages, e.g. ECM [Entitlement Control Message] or EMM [Entitlement Management Message]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47End-user applications
    • H04N21/472End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/47202End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for requesting content on demand, e.g. video on demand
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/65Transmission of management data between client and server
    • H04N21/658Transmission by the client directed to the server
    • H04N21/6581Reference data, e.g. a movie identifier for ordering a movie or a product identifier in a home shopping application
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/76Television signal recording
    • H04N5/765Interface circuits between an apparatus for recording and another apparatus
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/16Analogue secrecy systems; Analogue subscription systems
    • H04N7/167Systems rendering the television signal unintelligible and subsequently intelligible
    • H04N7/1675Providing digital key or authorisation information for generation or regeneration of the scrambling sequence
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/76Television signal recording
    • H04N5/765Interface circuits between an apparatus for recording and another apparatus
    • H04N5/775Interface circuits between an apparatus for recording and another apparatus between a recording apparatus and a television receiver
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/76Television signal recording
    • H04N5/78Television signal recording using magnetic recording
    • H04N5/782Television signal recording using magnetic recording on tape
    • H04N5/783Adaptations for reproducing at a rate different from the recording rate
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/76Television signal recording
    • H04N5/907Television signal recording using static stores, e.g. storage tubes or semiconductor memories
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00Details of colour television systems
    • H04N9/79Processing of colour television signals in connection with recording
    • H04N9/80Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
    • H04N9/804Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback involving pulse code modulation of the colour picture signal components
    • H04N9/8042Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback involving pulse code modulation of the colour picture signal components involving data reduction

Definitions

  • the present invention relates to an apparatus and a method for data streaming, as well as a combination, of an apparatus for data streaming and a plurality of user devices.
  • Streaming is conventionally performed on data with video, sound or multimedia content, and enables the playback of such data without the need to download the entire resource file in advance of playback. This means that such data has to be sent at a pace corresponding, at least on average, to the pace at which it is to be presented.
  • users e.g. video viewers
  • display devices e.g. television sets or personal computers, that are connectable, e.g. via an IP-network, to a server in which media data is stored.
  • each user can request individually a video stored in the server to be displayed, so that the video is displayed to the user immediately upon streaming from the server, i.e. without storage at the display device of the entire video.
  • the stream is normally compressed or coded, for example by MPEG-2, and the display device is adapted to decode it.
  • EP0698982A2 discloses describes a method in which video/multimedia content is stored in switches or routers of a network in the format of network packets, linked with links fields. This means that the video/multimedia content has to be formatted before being stored, and before formatting the video/multimedia content, an amount of network specific information has to be given. For example, a network protocol has to be chosen before formatting and storing of the video/multimedia content. If the such network specific information, e.g. the network protocol, can vary in dependence on users and requests, which is in practice a common case, several versions of a specific video/multimedia content have to be stored, each with individual network specific information. This requires a large amount of storage space, which makes the system complicated and expensive.
  • US2003/0095783A1 discloses a system in which protocol stacks are generated in hardware as streams are sent to networks. Among other things, the system presented does not support trick-play modes, such as fast forward or reverse video playback modes, in a capacity efficient manner.
  • US6067108 discloses a streaming device in which an address table is used during streaming to access a memory for media data.
  • the need to repeatedly read, during data streaming, in the address table adds time to the streaming process, which limits the overall capacity of the device.
  • Embodiments of the invention provide an apparatus and method that increase the capacity of data streaming, e.g., for user numbers, request numbers, and/or both.
  • An embodiment of the invention provides an apparatus for data streaming, adapted to receive a plurality of streaming requests from a plurality of user devices, the apparatus comprising - a control unit adapted to receive media data in the form of a plurality of payload data sets from at least one data source,
  • a streaming unit adapted to stream the payload data sets according to the streaming requests
  • control unit being adapted to pre-process the payload data sets before being stored, whereby control data sets are created
  • the streaming unit being adapted to stream the payload data sets using the control data sets.
  • control unit may be adapted to pre-process the payload data sets before being stored, whereby control data sets are created, and the streaming unit may be adapted to stream the payload data sets using the control data sets, then demanding and complicated, but less frequent tasks involved when receiving the media data, and those that require flexibility for a service provider, are efficiently separated from pure streaming operations.
  • Such an embodiment provides a good balance of functions between the units, which enables a very efficient use of processing resources, which allows a very high capacity.
  • Embodiments of the invention provides for said demanding and complicated, but less frequent tasks to be handled by a software controlled processor, which allows flexibility for a service provider, while more simple but very frequent streaming operations can be handled by a hard- coded programmed logic device, such as a field programmable gate array (FPGA), which provides extremely high stream rates.
  • a software controlled processor which allows flexibility for a service provider, while more simple but very frequent streaming operations can be handled by a hard- coded programmed logic device, such as a field programmable gate array (FPGA), which provides extremely high stream rates.
  • FPGA field programmable gate array
  • the invention provides an apparatus with a low level of complexity, which allows flexibility for less frequent tasks, and the cost of the apparatus can be allocated to a large amount of users. This reduces overall costs for an administrator of streaming services.
  • control unit is adapted to, when pre-processing a payload data set, divide it into a plurality of payload data sub-sets, and create for each of at least some of the pay load data sub-sets a control data sub-set, each comprising data corresponding to a location in the memory unit of another payload data sub-set.
  • the streaming unit is adapted to receive a first sub-set of a first set of payload data from the memory unit, and to receive from the memory unit a first sub-set of a first set of control data, which comprises data corresponding to a location in the memory unit of a second sub-set of the first set of payload data. Since the streaming unit receives from the memory unit, along with the sub-sets of payload data, sub-sets control data, each giving a location in the memory unit of a further sub-set of payload data, there is no necessity for repeatedly read, during data streaming, in a separate information source, such as an address table.
  • a separate information source such as an address table
  • the first sub-set of the payload data set and the first sub-set of the control data set are stored at the same location in the memory unit.
  • the memory unit comprises a plurality of memory sectors, and wherein the payload data set and the control data set is arranged in pairs of payload data sub-sets and control data sub-sets, each pair being stored in a respective of the memory sectors.
  • each of at least some of the control data sub-sets in the pairs of payload data sub-sets and control data sub-sets comprises data corresponding to a location of a memory sector of another pair.
  • the streaming unit is adapted to store for each of at least some of the streaming requests a request data set related to the respective streaming request, to create data packets, each based at least partly on a portion of a payload data set and a request data set, and to send the data packets to the user devices. Since data packets are created "on the fly", i.e. during streaming of the payload data, no network specific formatting is needed before storing the payload data. Therefore, the payload data does not have to be stored in multiple versions for different network formats, and this will in turn increase storage capabilities for data streaming applications.
  • the streaming unit is adapted to determine, based at least partly on a plurality of time codes in each of at least some of the payload data sets, bit-rate information, and to stream portions of the payload data sets based on the bit-rate information.
  • the control unit comprises a software controlled processor
  • the streaming unit comprises a hard-coded programmed logic device.
  • the memory unit is a solid state memory.
  • the streaming unit is adapted to encrypt a payload data set to at least one of the user devices.
  • the streaming unit is adapted to encrypt the payload data set when streamed.
  • the streaming unit is adapted to encrypt the payload data set according to at least one encryption key, each corresponding to at least one of the user devices.
  • each encryption key corresponds to a group of user devices.
  • the apparatus is adapted to store the at least one encryption key.
  • the streaming requests include mode requests, the apparatus comprising a state device adapted to set a mode in accordance with each one of the mode requests, the streaming unit being adapted to stream the payload data sets in accordance with the set modes, the streaming unit being adapted to receive from the memory unit a plurality of control data sub-sets, each comprising at least two edit portions, each comprising data corresponding to an address in the memory unit.
  • each edit portion may comprise data corresponding to a location in the memory unit of a payload data sub-set.
  • the streaming unit receives from the memory unit, along with the sub-sets of pay load data, sub-sets control data, each giving a location in the memory unit of a further sub-set of payload data. This means that there is no need to repeatedly read, during data streaming, in a separate information source, such as an address table.
  • the edit portions of the control data subsets each gives, in dependence on a respective playback mode, an address to a further sub-set of payload data.
  • each of the edit portions provides a separate linking structure for the payload data sub-sets in dependence on a respective playback mode, according to an embodiment of the invention.
  • the payload is a motion picture video
  • the edit portions of the control data can provide for a sequential streaming of all payload data sub-sets, whereas in a trick-play mode such as a fast forward mode the edit portions can provide for some of the payload data sub-sets to be skipped in the memory retrieval process.
  • An embodiment of the invention provides for a very effective way of controlling trick-play modes by providing pre-stored linking structures in the memory unit, which structures can be followed selectively, without changing the work process of the streaming unit.
  • the streaming unit can therefore be kept relatively simple in its construction, and streaming can be carried out at a high rate regardless of playback modes requested by the user devices.
  • the streaming unit is adapted to receive from the memory unit a first sub-set of a first payload data set, and a first sub-set of the control data set, which first control data sub-set comprises a first edit portion comprising data corresponding to a location in the memory unit of a second sub-set of the first payload data set, and a second edit portion comprising data corresponding to a location in the memory unit of a third sub-set of the first payload data set.
  • the control unit is adapted to locate a plurality of identification data groups within a payload data set, and provide in each of a plurality of the edit portions of the control data sub-sets data corresponding to the location of an identification data group.
  • each identification data group corresponds to a full image data frame.
  • Embodiments of the invention may also comprise a combination of an apparatus according to any of the claims 1-19, and a plurality of user devices, from which the apparatus is adapted to receive the plurality of streaming requests.
  • the user devices are audio/video devices.
  • the audio/video devices are at least one of set-top boxes, personal video recorders, or personal computers.
  • Embodiments of the invention may also comprise a method for data streaming, comprising receiving media data in the form of a plurality of payload data sets from at least one data source, storing the payload data sets in a memory unit, receiving a plurality of streaming requests from a plurality of user devices, and streaming the payload data sets according to the streaming requests, the method comprising pre- processing the payload data sets before being stored, whereby control data sets are created, and streaming the payload data sets using the control data sets.
  • the method comprises dividing, when pre-processing a payload data set, it into a plurality of payload data sub-sets, and creating for each of at least some of the payload data sub-sets a control data sub-set, each comprising data corresponding to a location in the memory unit of another payload data sub-set.
  • the method comprises receiving a first sub-set of a first set of payload data from the memory unit, and receiving from the memory unit a first sub-set of a first set of control data, which comprises data corresponding to a location in the memory unit of a second sub-set of the first set of payload data.
  • the first sub-set of the payload data set and the first sub-set of the control data set are stored at the same location in the memory unit.
  • the memory unit comprises a plurality of memory sectors, and wherein the payload data set and the control data set is arranged in pairs of payload data sub-sets and control data sub-sets, each pair being stored in a respective of the memory sectors.
  • each of at least some of the control data sub-sets in the pairs of payload data sub-sets and control data sub-sets comprises data corresponding to a location of a memory sector of another pair.
  • the method comprises storing for each of at least some of the streaming requests a request data set related to the respective streaming request, creating data packets, each based at least partly on a portion of a payload data set and a request data set, and sending the data packets to the user devices.
  • the method comprises determining, based at least partly on a plurality of time codes in each of at least some of the payload data sets, bit-rate information, and streaming portions of the payload data sets based on the bit-rate information.
  • the method comprises encrypting a payload data set to at least one of the user devices.
  • the method comprises encrypting the payload data set when streamed.
  • the method comprises encrypting the payload data set according to at least one encryption key, each corresponding to at least one of the user devices.
  • each encryption key corresponds to a group of user devices.
  • the method comprises storing the at least one encryption key.
  • the streaming requests include mode requests, the method comprising setting a mode in accordance with each one of the mode requests, streaming the payload data sets in accordance with the set modes, and receiving from the memory unit a plurality of control data sub-sets, each comprising at least two edit portions, each comprising data corresponding to an address in the memory unit.
  • the method comprises receiving from the memory unit a first sub-set of a first payload data set, and a first sub-set of the control data set, which first control data sub-set comprises a first edit portion comprising data corresponding to a location in the memory unit of a second sub-set of the first payload data set, and a second edit portion comprising data corresponding to a location in the memory unit of a third sub-set of the first payload data set.
  • the method comprises locating a plurality of identification data groups within a payload data set, and providing in each of a plurality of the edit portions of the control data sub-sets data corresponding to the location of an identification data group.
  • each identification data group corresponds to the location of a full image data frame.
  • Embodiments of the invention may also comprise an apparatus, comprising a control unit configured to accept media data from a data source and a plurality of streaming requests from a plurality of user devices, a memory unit configured to store the media data under control of the control unit, and a streaming unit configured to stream portions of the media data corresponding to the streaming requests to the plurality of user devices under the control of the control unit.
  • the streaming unit is further configured to stream the portions of the media data using control data.
  • the apparatus further comprises a network interface configured to send the portions of the media data over a network to the plurality of user devices.
  • the apparatus further comprises a preprocessor configured to accept the media data and determine the control data prior to storage of the media data in the memory unit.
  • the apparatus further comprises a user device request unit configured to receive the plurality of streaming requests from the plurality of user devices and transmit the plurality of streaming requests to the streaming unit.
  • the apparatus further comprises an encryption device configured to encrypt the portions of the media data using an encryption key corresponding to each user device of the plurality of user devices.
  • the apparatus further comprises an encryption key storage unit configured to store a plurality of encryption keys.
  • the encryption key storage unit is operably coupled to the control unit.
  • the plurality of user devices are audio/video devices.
  • the audio/video devices are at least one of set-top boxes, personal video recorders, or personal computers.
  • at least one of the media data is an audio/visual data file and one of the plurality of streaming requests is a request for the streaming of the audio/visual data file.
  • Embodiments of the invention may also comprise a method, comprising accepting media data from a data source and a plurality of streaming requests from a plurality of user devices, generating control data from the media data, storing the media data, and streaming at least a portion of the media data to the plurality of user devices under control of the control data.
  • generation of the control data from the media data comprises reviewing the media data to locate identification data groups within the media data; and adding data representing locations of the identification data groups to the control data.
  • the identification data groups are complete image data.
  • the method further comprises receiving a streaming request from a user device of the plurality of user devices, and streaming a portion of the media data to the user device of the plurality of user devices.
  • the method further comprises encrypting the portions of the media data using an encryption key corresponding to each user device of the plurality of user devices.
  • the method further comprises storing the plurality of encryption keys in an encryption key store.
  • one of the plurality of streaming requests is a request for the streaming of an audio/visual data file.
  • - fig. 1 is a block diagram of a data network and devices connected thereto including a data streaming device according to one embodiment of the invention
  • - fig. 3 is an illustration of data sets in the data streaming device in fig. 2
  • - fig. 4 is a block diagram of a part of the data streaming device in fig. 2
  • - fig. 5 is an illustration of data sub-sets handled by the part depicted in fig. 4,
  • - fig. 6 is a block diagram of memory sectors in a memory unit in the data streaming device in fig. 2,
  • - fig. 7 is a block diagram of a data network and devices connected thereto including a data streaming device according to an alternative embodiment of the invention
  • - fig. 8 is a block diagram depicting an arrangement in a network system according to a special aspect of the invention
  • - fig. 9 is a diagram showing correlation between data stream request frequency and band width requirements
  • - fig. 10 is a diagram showing correlation between payload data sets and data stream request frequency.
  • Fig. 1 depicts schematically a system with an apparatus for data streaming, herein also referred to as a data streaming device 1 for data streaming, connectable to a data network 2, e.g., in the form of an IP-network.
  • the network can alternatively be any kind of network, global or local, suitable for transporting data.
  • data refers to information in a form that can be used by a computer, router or any other component of a communications system, and it can include text, numbers, sounds, pictures, and combinations thereof.
  • a plurality of user devices 11a, 1 Ib are also connectable to the network 2.
  • the user devices 11a, 1 Ib can be any kind of remote devices adapted to receive media data, such as set-top boxes, personal video recorders, personal computers, or any combination thereof.
  • An embodiment of the data streaming device 1 comprises a streaming unit 3, a control unit 6, and memory unit 7, being connected to each other in a manner described closer below.
  • an embodiment of the control unit 6 comprises a processor (CPU) and a solid state memory, and is adapted to process data according to software programs, including pre-processing of media data, here also referred to as media payload data, or payload data.
  • the memory unit 7 may be a solid state memory unit adapted to store media payload data.
  • the streaming unit 3 is adapted to read in the memory unit 7, edit and schedule data to be streamed, and send data streams to the user devices l la, 1 Ib via the network 2.
  • the logic functions of the streaming unit may be carried out by one or more programmed logic devices, i.e. hardware units, as described closer below.
  • the streaming unit 3 of the data streaming device 1 comprises a networking unit 4, adapted to send and receive data to and from, respectively, the network 2, via a network interface 4a.
  • the streaming unit 3 also comprises an editing unit 5 connected to the networking unit 4 by a suitable bus B54.
  • the editing unit 5 is connected to the memory unit 7 by a bus B75.
  • the editing unit 5 and the networking unit 4 are connected to the control unit by a respective bus B65, B64, such as a PCI (Peripheral Component Interconnect) bus.
  • PCI Peripheral Component Interconnect
  • the networking unit 4 and the editing unit 5 may be hard wired (i.e. hard-coded) with each of them comprising a programmed logic device in the form of a field programmable gate array (FPGA).
  • FPGA field programmable gate array
  • any other suitable type of programmed logic device, or customized hardware can be used, such as CPLD (Complex Programmable Logic Device), ASIC (Application Specific Integrated Circuit ) or gate array.
  • the memory unit 7 is adapted to store a plurality of sets of payload data, e.g., in the form of multimedia sequences, for example movies, music files, and/or advertisements. More generally, the payload data can be any form of media data, whether audio, visual, text or code, provided separately or in any combination. Also, a set of payload data, herein also referred to as a payload data set, can provide data for a time limited media display, or be received from a continuous flow of data, for example in the form of a transmission from a TV station.
  • the memory unit 7 comprises a solid state memory in the form of a plurality of interconnected so-called flash memory units, i.e.
  • any other suitable type of memory can be used, for example a DRAM or an SRAM.
  • a solid state memory creates several advantages over conventional hard- drives in streaming applications. An important limiting factor with conventional hard drives is that they provide a speed of information handling (i.e., extraction from storage) that is too low for large scale streaming application, due to, among other things, physical constraints of moving parts. Also, hard drives have a dependability which is less than desired for a good quality of service to users of data streaming services. In addition, hard drives have a high power consumption. In contrast to hard drives, certain solid state memories, specially flash memories, provide a speed of information handling which is more suitable for large scale streaming applications, due to the absence of moving parts, a significantly improved dependability, and a much lower power consumption.
  • the sets of payload data can be received by the data streaming device 1 from a data source in the form of a remote server 8 via the network 2 and the network interface 4a.
  • the networking unit 4 is adapted to receive payload data sets 9a, 9b, for example motion picture videos, via the network interface 4a.
  • the streaming device 1 can be adapted to receive the payload data sets from any type of data storing and/or transmitting device.
  • the same network 2 is utilized for all network transmissions involving the streaming device 1.
  • separate networks can be used for receiving data from the remote server 8 and for sending data to the user devices l la, 1 Ib.
  • the streaming device 1 can alternatively be provided with more than one network interface 4a.
  • Embodiments of the networking unit 4 may be hard coded to detect the incoming traffic of payload data sets 9a, and, in response to such detection, forward it to the control unit 6, which is adapted to store it temporarily in a control unit storage means 6a, which can be provided in the form of a RAM-memory, for example.
  • the network 2 does not necessarily need to connect both the remote server 8 and the user devices 1 Ia, 1 Ib, e.g., the connections may be private and/or use different protocols.
  • the control unit 6 is adapted to pre-process sets of pay load data, according to an embodiment of the invention. Referring to fig.
  • the control unit 6 is adapted to divide, in the pre-processing, each set of pay load data 9a, 9b into a plurality of payload sub-sets 91a, 91b.
  • the payload data sub-sets 91a, 91b may be adapted to the size of the memory sectors of the flash memories of the memory unit 7. Therefore, apart from the sequence of data within the payload, the division of the payload data into sub-sets need not depend upon the arrangement of payload data in view of the playback thereof.
  • each sub-set 91a of payload data can include data for a sequence of video frames, or only a part of such a frame.
  • the control unit 6 may be adapted to create, in the pre-processing of the payload data 9a, 9b, control data, here also referred to as side information. More specifically, the control unit 6 is adapted to create, in the pre-processing of each payload data set 9a, 9b, a set of control data.
  • Each control data set is related to the respective payload data set and includes data for linking the payload data sub-sets 91a, 91b, or parts thereof, when streaming the respective payload data set 9a, 9b to users.
  • the linking data is such that it allows the streaming to be carried out in accordance with any of a number of playback modes, herein also simply referred to as modes, for example normal viewing mode, four fast forwarding modes with respective individual speeds, and four reverse modes with respective individual speeds.
  • modes for example normal viewing mode, four fast forwarding modes with respective individual speeds, and four reverse modes with respective individual speeds.
  • each set of control data is typically provided in the form of control data sub-sets 101a, 101b, each associated with a respective payload data sub-set 91a, 91b.
  • the size of each pair of payload data sub-set 91a, 91b and control data sub-set 101a, 101b is adapted for occupying a respective memory sector of the flash memory units of the memory unit 7.
  • the control unit 6 is adapted to locate, in the preprocessing, a plurality of identification data groups within a payload data set 9a, 9b, and provide in each of a plurality of edit portions, further described below, of the control data sub- sets 101a, 101b, data corresponding to the location of an identification data group.
  • Each identification data group corresponds to a full data frame, such as a full image data frame, e.g., a frame that does not include any reference to previous or other frames for their composition.
  • videos can be compressed or coded, for example by MPEG-2, and the user devices 11a, 1 Ib can be adapted to decode them.
  • full image data frames are referred to as I-frames (intra coded frames).
  • I-frames intra coded frames.
  • the identification of locations of full image data frames makes it possible to provide, as described closer below, for only such full image data frames to be streamed in compressed videos and at higher playback speeds.
  • software is used to identify the I-frames to obtain the linking structures of the control data set, according to an embodiment of the invention.
  • a hardware mechanism may be employed that performs a similar function.
  • identification data groups can be performed on any media data, whether it is video, audio or of some other type, being in a format such that it contains such data groups, or data sequences, that have a function corresponding to that of the I-frames in the MPEG-2 format, i.e., serving to identify a portion of the media content without referring to other media data.
  • the control unit 6 is adapted to send, upon pre-processing, the pairs of payload data sub-sets 91a and control data sub-sets 101a to the editing unit 5, which is adapted to forward them for storage in respective memory sectors of the memory unit 7. More specifically, referring to fig. 2 and 4, (fig. 4 being a block diagram depicting functions of the editing unit 5), the control unit 6 is adapted to send the payload data and the control data via the bus B65 to the editing unit 5, which is adapted to forward it to the memory unit 7 via the bus B75, as indicated by the arrow 14 in fig. 4.
  • control unit 6 can be adapted to send the payload data and control data directly to the memory unit 7, for example via a bus (not shown) between the two units 6, 7.
  • the data streaming device 1 is adapted to receive streaming requests, herein also referred to simply as requests, via the network 2 from a plurality of user devices. It should be noted that the data streaming device according to a preferred embodiment of the invention is, when used in practice, able to store, e.g., with a storage capacity of 1.5 TBytes, a large amount of payload data sets, e.g. corresponding to 460-1000 hours of video content, and to work simultaneously with requests from up to 10000 user devices.
  • a first and a second user device l la, 1 Ib are shown in fig. 1.
  • the networking unit 4 is adapted to receive, via the network interface 4a, from user devices l la, l ib requests, including playback mode requests, for sets of payload data 9a, 9b.
  • the networking unit is adapted to forward the user device requests to the control unit 6, to be treated by a user request unit 6b, which is adapted to read the requests according to a protocol for media streaming requests, such as RTSP (Real Time Streaming Protocol).
  • RTSP Real Time Streaming Protocol
  • the user request unit 6b is adapted to process the requests, each including the identity of the respective user device l la, l ib, the respective set of payload data 9a, 9b and respective the playback mode request, so as to be adapted to a streaming device internal language format, and send them to the editing unit 5.
  • the user request unit 6b is also adapted to extract from the requests sets of request data, each including the identity of the network protocol used for the respective request, and a network address of the respective user device 11a, l ib.
  • the user request unit 6b is further adapted to send the extracted sets of request data to the networking unit 4 to be stored in networking unit storage means in the form of a RAM memory 401.
  • the streaming unit 3 can be hard-coded to process requests, so as for the requests to be adapted to a streaming device internal language format.
  • the editing unit 5 comprises a user status memory 506, herein also referred to as a state device 506, adapted to receive, as indicated by the arrow A3, from the control unit 6 information corresponding to user requests, received by the streaming device 1 as described above.
  • the user status memory 506 is adapted to receive the requests including the identities of the user devices l la, 1 Ib and the playback mode requests Pl Ia, Pl Ib of the user devices l la, l ib, according to an embodiment of the invention.
  • the information about the identity of the user devices l la, 1 Ib, the requested payload data sets 9a, 9b and the playback mode requests may be stored in the user status memory 506.
  • the user status memory 506 can be provided as a unit, for example including a data table, separate from the streaming unit 3 and the control unit 6.
  • the editing unit 5 comprises a first and a second multiplexer 501, 502, each adapted to receive from the memory unit 7 control data sub-sets 101a, 101b (see fig. 3), as indicated by the arrows C3 in fig. 4, according to an embodiment of the invention.
  • the editing unit 5 may also comprise a trimming device 505, adapted to receive from the memory unit 7 payload data sub-sets 91a, 91b (see fig. 3), as indicated by the arrow B3 in fig. 4, as well as a pacing device 504 and a scheduling device 507, described further below.
  • the sets of payload data 9a, 9b, requested by the user devices l la, 1 Ib can be videos. Fig.
  • the control data set may provide a linking structure for each playback mode that can be requested, e.g normal viewing speed IX, and "trick-play modes".
  • the trick-play modes can include fast forward modes and fast reverse modes.
  • Each payload data sub-set 91a, 92a, 93 a contains a number of picture frames Fl, F2, F3 .. FN.
  • the linking structure provides for all frames within a payload data sub-set 91a, 92a, 93a to be streamed, and for the first frame Fl in a payload data sub-set to be streamed after the last frame FN of a another payload data sub-set, as indicated in fig. 5 by the arrows A4.
  • the linking structure indicates which of the frames are to be streamed, according to an embodiment of the invention. For example, after a frame which is not the last in a payload data sub-set, a frame in another payload data sub-set can be streamed, as indicated in fig. 5 by the arrows B4. Also, the subsequent frame can be included in a payload data sub-set which does not follow immediately in a normal viewing playback mode, as indicated in fig. 5 by the arrow C4.
  • the streaming device 1 can be adapted to receive a playback request for a pause mode, i.e. the presentation of a video being frozen so that a non-moving image is shown.
  • the streaming unit 3 can be adapted to stream repeatedly the same I-frame to the user device in question.
  • the user device 11a, l ib can be adapted to store an I-frame at a pause request from a user, so that the I- frame is shown to the user for an extended period of time.
  • the size of the payload data sub-sets 91a, 92a, 93 a are adapted to the size of the memory sectors in the memory unit 7, according to an embodiment of the invention. If the size of each of these memory sectors is relatively small, e.g. 16 kB, the data composing a frame in a video, for example an I-frame according to the MPEG-2 format, can be larger than the memory sector size. (Typically the data for an I-frame can be 64 kB or more.) In such a case, the frame data will simply use two or more memory sectors, and the linking structure of the control data set will provide a proper sequence of the memory sectors storing parts of the same frame. An advantage with providing relatively small memory sectors is that in trick-play modes, compared to a case with relatively large memory sectors, the amount of data cut away by the editing unit 5, as described closer below, will be less.
  • control data sub-sets also provides addresses to memory sectors in the memory unit 7 storing subsequent sub-sets of payload data, depending on the playback mode requested.
  • the editing unit 5 is adapted to, upon receiving the data corresponding to the identity of a user device 1 Ia, 1 Ib, the identity of a set of payload data 9a, 9b, and a playback mode request Pl Ia, Pl Ib, retrieve a first sub-set 91a, 91b of the requested payload data set 9a, 9b, as well as a first sub-set 101a, 101b of the control data set.
  • the control unit 6 is adapted to store the address in the memory unit 7 of the first subset 91a, 91b of each payload data set 9a, 9b, i.e.
  • the control unit 6 is further adapted to, upon a user request for a payload data set 9a, 9b to retrieve the memory unit 7 address of the first subset 91a, 91b of the payload data set 9a, 9b, and to send this memory unit address to the editing unit with other data regarding the user request.
  • the user status memory 506 is adapted to receive the first memory unit address and to forward it to the pacing device 504, which is adapted to send a request to the memory unit 7 for retrieving the first sub-set 91a, 91b of the requested payload data set 9a, 9b, as well as a first sub-set 101a, 101b of the control data set.
  • the trimming device 505 is adapted to receive from the memory unit 7 payload data subsets 91a, 91b upon requests from the pacing device 504.
  • the first and second multiplexers 501, 502 are adapted to receive control data sub-sets 101a, 101b, upon requests from the pacing device 504, according to an embodiment of the invention.
  • the user status memory 506 is adapted to receive periodically from the pacing device 504 data, as indicated by the arrow J4 in fig. 4, corresponding to information regarding the user device, i.e. stream, for which a read request is currently being sent to the memory unit 7.
  • the user status memory 506 may comprise a plurality of rows, each corresponding to one stream according to a user request Pl Ia, Pl Ib, and controls the multiplexers 501, 502 for filtering the control data in accordance with the playback modes of the respective user requests.
  • the user status memory 506 is adapted to send, in response to the stream identity information J4 from the pacing device 504 control signals, indicated by the arrows K4, L4 in fig. 4, so as to adjust the setting of the multiplexers to the playback mode (IX, FFl, ...) of the stream currently being served.
  • the first multiplexer 501 is adapted to receive control data sub-sets 101a, 101b, and, in dependence on the control of the user status memory 506 concerning the requested playback mode Pl Ia, Pl Ib, extract an edit portion of each sub-set 101a of control data, which edit portion gives linking information for the requested playback mode Pl Ia, Pl Ib.
  • the edit portions of the control data sub-sets 101a, 101b are used to read in the trimming device 505, so that a data sequence according to the linking structure of the respective edit portion of the control data sub-sets 101a, 101b is sent to an output buffer 503a, 503b of an output memory 503.
  • the output memory 503 is preferably a large solid state memory, which in practice can be a 512 MB DRAM, or alternatively some other kind of random access memory, e.g. SRAM, according to an embodiment of the invention.
  • the output memory 503 is arranged so as to provide a plurality of output buffers 503a, 503b, in the form of FIFO (first-in, first-out) queues, corresponding to the plurality of user devices 11a, 1 Ib to which data streaming is carried out.
  • the output memory 503 is adapted to receive periodically from the pacing device 504 data, as indicated by the arrow M4 in fig. 4, corresponding to information regarding the user device, i.e. stream, for which a read request is currently being sent to the memory unit 7. Based on this information, an output buffer 503a, 503b is identified for receiving the edited contents of the trimming device 505.
  • the second multiplexer 502 is also adapted to receive the control data sub-sets 101a, 101b, and to extract, in dependence of the user status memory control, a edit portion of each control data sub-set 101a, 101b, according to an embodiment of the invention. As described further below, each extracted edit portion gives the address in the memory unit 7 of a subsequent payload data sub-set 92a, 92b. As indicated by the arrow E4 in fig. 4, each edit portion of the control data sub-sets 101a, 101b is received by the pacing device 504.
  • first and second multiplexer 501, 502 with other devices, such as only one multiplexer, the output of which is sent to the trimming device 505 as well as the pacing device 504.
  • several parallel data streams could be provided from the memory unit 7 to the editing unit 5.
  • the scheduling device 507 is adapted to determine the timing of data outputs from the output memory 503. As indicated by the line F4 in fig. 4, the scheduling device 507 is adapted to read from data traffic from the output buffers 503a, 503b of the output memory 503 information based on which a bit-rate or a pace, at which the respective payload data sets 9a, 9b are to be displayed, can be determined. More specifically, the bit-rate information is determined from known time codes, for example PCR (programme clock reference), used in multimedia display methods, and included in the sets of payload data. The bit-rates corresponding to the respective time codes can differ from one payload data set 9a to another 9b. Thus, the bit-rate of data streamed can differ from one output buffer 503 a to another output buffer 503b.
  • PCR programe clock reference
  • the scheduling device 507 is adapted to control the data traffic from the output buffers 503a, 503b to the networking unit 4, as indicated by the arrow G4 in fig. 4, in dependence on the bit-rate information for each stream.
  • the data from each output buffer are sent in stream portions, the size of which is adapted to network requirements (e.g. in the order of 1,5 kB). More specifically, the scheduling device 507 is adapted to determine a bite-rate based on time codes in the payload data stream, as mentioned above.
  • the stream in question is sent based on this determined bit-rate, and when a further time code is detected, the determined bit-rate is compared to time information from a clock, and the scheduling device 507 is adapted to adjust, based on this comparison, the timing of data sent in the stream. If it is determined that a stream portion of data from one output buffer 503 a, 503b should not be sent, the scheduling device 507 makes a similar determination concerning the another output buffer 503a, 503b, and works in this respect as a scheduler, performing a cyclic sequence of determinations whether to send stream portions of data from the respective output buffets 503a, 503b to the networking unit 4.
  • such determination of the bit-rate information or display pace information can be done at the above-described pre-processing of the payload data set 9a, 9b, in the control unit 6, and included in the control data set corresponding to the payload data set 9a, 9b.
  • the network unit 4 is adapted to receive stream portions from the respective output buffers 503a, 503b, as well as information identifying the respective output buffer 503a, 503b, and to create data packets based on the stream portions from the output buffers 503a, 503b and the request data sets stored in the networking unit storage means 401. Thereby, the respective output buffer identification provides, or corresponds to an address in the networking unit storage means 401 of the corresponding request data set.
  • the network unit 4 is further adapted to send the data packets to the user devices 11a, 1 Ib via the network interface 4a. Thus, network data packets are created as the data is streamed by the streaming device 1.
  • each output buffer 503a, 503b is provided with a respective threshold level WMa, WMb in the respective FIFO queue.
  • the pacing device 504 When determining whether to request more data for a specific output buffer 503a, 503b, the pacing device 504 is adapted to compare the amount of data temporarily stored in the output buffer to the respective threshold level WMa, WMb, and to request more data from the memory unit 7 if the amount of data stored is below the respective threshold level WMa, WMb. Thereby, as indicated by the arrow H4 in fig. 4, based on the memory sector addresses in the control data sub-sets 101a, 101b, the pacing device 504 sends to the memory unit 7 read requests for further payload data sub-sets 92a, 92b and a further control data sub-sets 102a, 102b.
  • Subsequent sub-sets 92a, 92b, 102a, 102b are received by the trimming device 505 and the multiplexers 501, 502, and data is forwarded to the respective output buffer 503a, 503b, in a manner corresponding to what has been described above.
  • threshold levels WMa, WMb can be set individually and differently for the output buffers 503a, 503b, or to be the same for a group of output buffers or for all output buffers.
  • the networking unit 4 can receive a request from a user device for an alternative playback mode.
  • the streaming device 1 is adapted to change the playback mode setting Pl Ia for a user device l la, 1 Ib if the user device l la, 1 Ib requests an alternative playback mode, for example a trick-play mode, e.g., fast forward or fast reverse.
  • the networking unit 4 if the networking unit 4 receives, via the network interface 4a, from a user device l la, 1 Ib a request for an alternative playback mode, the networking unit 4 is adapted to forward the request to the control unit 6, which is adapted to treat the request in a manner corresponding to what has been described above.
  • the control unit 6 is adapted to send the request to the editing unit 5.
  • the user status memory 506 receives the request including the identity of the user device l la, 1 Ib in question, and stores the data corresponding to the requested alternative playback mode.
  • the editing unit 5 is adapted to retrieve, upon read instructions (H4) from the pacing device 504, payload data sub-sets 91a, 91b, as well as control data sub-sets 101a, 101b.
  • the trimming device 505 is adapted to receive and temporarily store the payload data sub-sets 91a, 91b.
  • the first multiplexer 501 is adapted to receive control data sub-sets 101a, 101b and extract an edit portion of each of them in accordance with the requested trick-play mode Pl Ia, Pl Ib, which edit portions
  • the linking structure provided by an edit portion of the control data sub-set 101a, 101b, which edit portion corresponds to the requested 15 trick-play mode Pl Ia, Pl Ib, is used to extract a trimmed portion of the payload data sub-set 92a, 92b to be sent to the output buffer 503a, 503b for the stream in question.
  • Parts of the payload data sub-set 92a, 92b excluded in the trimming device by the linking structure are simply deleted from the stream.
  • the linking structure of the control data sub-set is such that only full image data frames, i.e. frames that do not include references to previous frames for their composition, e.g. I-frames in MPEG-2, are sent to the output buffers 503a, 503b.
  • I-frames intra coded frames
  • a software is used to identify the I-frames to obtain the linking structures of the control data set, as described closer below.
  • a trick-play linking structure of a control data set provides for a separate addressing sequence in the memory unit 7, followed upon read instructions by the pacing unit 504, so that memory sectors that would have been included in a normal viewing speed play mode might be "skipped".
  • the scheduling device 507 is adapted to determine the timing of the outputs from the respective output buffer 503a, 503b, based on respective bit- rates at normal viewing speeds IX, in turn determined from time codes as described above.
  • a payload data set will be streamed to a user device at a bit-rate being the same as that at which it is streamed at a normal viewing speed mode IX.
  • the bit-rate at trick-play modes can be predetermined individually for the payload data sets, or for groups of payload data sets, or collectively for all payload data sets.
  • FIG. 6 showing an example of how memory sector locations in the memory unit 7 are accessed, according to an embodiment of the invention.
  • Three pairs of payload data sub-sets 91a, 92a, 93a and control data sub-sets 101a, 102a, 103 a are in this example stored in consecutive first, second and third memory sectors 71, 72, 73 of the memory unit 7.
  • the memory space available for payload data and control data is the same, in this example 16000 bytes and 1000 bytes, respectively.
  • the editing unit 5 determines from the first edit portion (Ix) of the first control data subset 101a the memory address (17001) of the start of the next memory sector 72. The editing unit 5 also determines from the first edit portion (Ix) of the first control data sub-set 101a that the offset from the start of the second payload data subset 92a is 0 bytes and the length within the subset 92a of the data to be streamed is 16000 bytes, which means that the output buffer 503a (fig. 4) for the stream in question will receive all data in the second payload data subset 92a.
  • the editing unit 5 determines from a second edit portion (FFl) of the first control data sub-set 101a the memory address (17001) of the start of the next memory sector 72.
  • the editing unit 5 also determines from the second edit portion (FFl) of the first control data sub-set 101a that the offset from the start of the second payload data subset 92a is 6000 bytes and the length within the subset 92a of the data to be streamed is 5000 bytes, which means that the output buffer 503a (fig. 4) for the stream in question will receive all data in the interval [A, B] in fig. 6.
  • the editing unit 5 determines from the third edit portion (FF2) of the first control data sub-set 101a the memory address (34001) of the start of the next memory sector 73, which means that a memory sector 72 read in normal viewing speed playback mode (IX) is skipped.
  • the editing unit 5 also determines from the third edit portion (FF2) of the first control data sub-set 101a that the offset from the start of the third payload data subset 93 a is 8000 bytes and the length within the subset 93 a of the data to be streamed is 4000 bytes, which means that the output buffer 503a (fig. 4) for the stream in question will receive all data in the interval [C, D] in fig. 6.
  • the editing unit 5 and the memory unit 7 are arranged so that at each data retrieval step, the respective control data subsets 101a, 102a, 103 a are read in their entirety. Thus, all data between a predetermined offset location, in this example 16001 bytes, from the start location of the respective memory sector 71, 72, 73 to the end of the memory sector, is retrieved.
  • fig. 6 shows an edit portion of a third control data subset 103 a giving local linking information for the first fast reverse playback mode RWl.
  • the editing unit 5 determines from the edit portion (RWl) of the third control data sub-set 103 a the memory address (17001) of the start of the next memory sector 72, which would have preceded the third sector 73 in normal viewing playback mode streaming.
  • the offset value (6000) and the length value (5000) are used in a manner corresponding to what has been described above with reference to fig. 6 concerning forward playback modes.
  • the edit portions of the control data sub-sets 101a, 102a, 103a are described as pointing to the start of the next memory sector 71, 72, 73 to be read for the stream.
  • different ways of identifying the next memory sector 71, 72, 73 can be used. For example, a position in the middle of the next memory sector 71, 72, 73 can be identified, possibly combined with a local pointer within the next memory sector 71, 72, 73.
  • a first set of payload data 9a for example a first motion picture video, is received by the networking unit 4 via the network interface 4a.
  • the networking unit 4 detects the incoming traffic of the first set of payload data 9a and forwards it to the control unit 6.
  • the first set of payload data 9a is divided into a plurality of sub-sets 91a, and a first set of control data, in the form of sub-sets 101a of control data, is created, including data for linking the sub-sets 91a of payload data, or parts thereof.
  • the pairs of payload data subsets 91a and control data sub-sets 101a are sent to the editing unit 5, and then stored in respective memory sectors of the memory unit 7.
  • a second set of payload data 9b is received by the networking unit 4, and forwarded to the control unit 6 to be pre-processed as described above.
  • the second set of payload data 9b is divided into a plurality of sub-sets 91b, and a second set of control data in the form of sub-sets 101b of control data is created, and the pairs of payload data sub-sets 91a and control data sub-sets 101a, 101b are stored in respective memory sectors of the memory unit 7.
  • the networking unit 4 receives, via the network interface 4a, from a first user device 1 Ia a first request, including a playback mode request, for the first set of payload data 9a.
  • the networking unit 4 forwards the first request to the control unit 6, to be adapted to a streaming device internal language format, and the first request send to the editing unit 5.
  • the control unit 6 extracts from the first request a first request data set, including the identity of the network protocol used for the first request, and a network address of the user device 11a, and sends the first request data set to the networking unit 4 to be stored in the RAM memory 401.
  • the networking unit 4 also receives from the second user device l ib a first request, including a playback mode request, for the second set of payload data 9b, whereupon the first request, including the identity of the second user device 1 Ib, the second set of payload data 9b and the playback mode request, is forwarded to the control unit 6, which extracts a first set of request data to be stored in the RAM memory 401, in the same manner as outlined above.
  • the user status memory 506 receives, as indicated by the arrow A3 in fig. 4, from the control unit 6 data corresponding to the identity of the first user device 11a, the playback mode request Pl Ia, as well as a memory unit address of the first subset 91a of the first payload data set 9a, provided as described above, which data is stored in the user status memory 506.
  • the user status memory 506 forwards the memory unit address to the pacing device 504, which sends a request to the memory unit 7 for retrieving the first sub-set 91a of the first payload data set 9a, as well as the first sub-set 101a of the first control data set.
  • the user status memory 506 receives data corresponding to the identity of the second user device 1 Ib, the playback mode request Pl Ib, as well as a memory unit address of the first subset 91b of the second payload data set 9b, provided as described above, which data is stored in the user status memory 506.
  • the user status memory 506 forwards the memory unit address to the pacing device 504, which sends a request to the memory unit 7 for retrieving the first sub-set 91b of the second payload data set 9b, as well as the first sub-set 101b of the second control data set.
  • the playback mode request Pl Ia of the first user device 1 Ia is a request for a normal viewing speed (IX) of the video.
  • the first sub-set 91a of payload data is sent to the trimming device 505, where it is temporarily stored.
  • the first multiplexer 501 receives a first control data sub-set 101a, and extracts, in dependence on the control of the user status memory 506 concerning the requested playback mode Pl Ia, a first edit portion of the first control data sub-set 101a, which first edit portion gives linking information for the requested normal viewing speed (IX).
  • the first edit portion of the first control data sub-set 101a is used to read in the trimming device 505, so that the full sequence of data of the first payload data subset 91a is stored in a first output buffer 503 a of the output memory 503.
  • the second multiplexer 502 also receives the first control data sub-set 101a, and extracts, in dependence of the user status memory control, a first edit portion of the first control data sub-set 101a, which first edit portion gives the address in the memory unit 7 of a second sub-set 92a of the first payload data set 9a.
  • the first edit portion of the first control data sub-set 101a is received by the pacing device 504. Based on the playback mode request Pl Ib of the second user device 1 Ib, and a first edit portion of a first sub-set 101b of a second set of control data, a first subset 91b of payload data is stored in a second output buffer 503b.
  • the scheduling device 507 determines first bit-rate information, or first display pace information, based on information in the first set of pay load data 9a, and corresponding to a pace at which the first set of pay load data 9a is to be displayed in the first user device 9a.
  • Data from the first output buffer 503 a is sent based on the first bit-rate information, and when a further time code in the first payload data set 91a is detected, the first bit-rate information is compared to time information from the clock, and based on this comparison the scheduling device 507 adjusts the timing of data sent from the first output buffer 503a.
  • the scheduling device 507 makes a similar determination concerning the second output buffer 503b, performing a cyclic sequence of determinations whether to send stream portions of data from the respective output buffets 503a, 503b to the networking unit 4.
  • a data packet is created based on the stream portion of data and the first set of request data stored in the networking unit storage means 401, and the data packet is sent to the to the first user device 1 Ia via the network interface 4a.
  • the networking unit 4 a data packet is created based on the stream portion of data and the second set of request data stored in the networking unit storage means 401, and the data packet is sent to the to the second user device 1 Ib via the network interface 4a.
  • the pacing device 504 in dependence, as described above, on the data level in the respective output buffer 503a, 503b, and based on the memory sector address in the first sub-set 101a of the first set of control data, the pacing device 504 sends to the memory unit 7 a read request for a second sub-set 92a of the first set 9a of payload data and a second sub-set 102a of the first set of control data.
  • These sub-sets 92a, 102a are received by the trimming device 505 and the multiplexers 501, 502, and data is forwarded to the output memory 503, in a manner corresponding to what has been described above.
  • the networking unit 4 receives, via the network interface 4a, from the first user device 1 Ia a second request, including a playback mode request for the second fast forward mode FF2.
  • the networking unit 4 forwards the second request to the control unit 6, to be treated by the user request unit 6b, and send to the editing unit 5.
  • the user status memory 506 receives the second request including the identity of the first user device l la, which is stored in the user status memory 506.
  • the editing unit 5 retrieves a first sub-set 91a of the first set of payload data 9a, as well as a first sub-set 101a of the first set of control data.
  • the first sub-set 91a of payload data is sent to the trimming device 505, where it is temporarily stored.
  • the first control data sub-set 101a is received by the first multiplexer 501, which in dependence of the requested playback mode Pl Ia, extracts a third edit portion of the first sub-set 101a of control data, which third edit portion gives linking information for the requested fast forward viewing speed (FF2), (see also below with reference to fig. 6).
  • the third edit portion (FF2) of the first sub-set 101a of control data is used to read in the trimming device 505, so that a data sequence according to the linking structure of the third edit portion (FF2) is sent to the first output buffer 503 a of the output memory 503.
  • the linking structure provided by the third edit portion (FF2), indicated by the arrow B4 in fig. 5, is used to extract a trimmed portion of the a second sub-set 92a of the first set of payload data 9a to be sent to the first output buffer 503 a. Parts of the first set of payload data 9a excluded in the trimming device by the linking structure are simply deleted from the stream.
  • the linking structure can be made so that commercials are shown at normal viewing speed, although fast speed has been requested by the user.
  • the fast speed linking structure e.g. for FF3
  • IX normal speed linking structure
  • a slower fast speed linking structure e.g. for FF2
  • the data streaming device can in practice be provided in a large number of embodiments, varying from what has been described in the example above.
  • any of the functional units of the streaming device 1, such as the streaming unit 3, the editing unit 5, the networking unit 4, the memory unit 7 or the control unit 6, can be provided as a plurality of logically interconnected devices.
  • the editing unit 5 and the networking unit 4 can be provided as a physically integrated unit.
  • the memory unit 7 can be provided as two or more separate units. One such memory sub-unit can be adapted to store payload data sets, and another memory unit can be adapted to store control data sets, both sub-units being accessed by the streaming unit 3 during data streaming.
  • control unit 6 can be provided in a separate apparatus, for example a computer, which could be located remotely from the streaming unit 3 and the memory unit 7, and connectable thereto via a network.
  • control unit comprises two sub-units, one of which, here referred to as a request device 602 is provided at the streaming unit 3 and is adapted to receive and process user device requests as described above.
  • the other sub-unit, herein referred to as a pre-processing unit 601 could be located remotely from other streaming device components and be adapted to perform the above described pre-processing of media data.
  • the streaming unit 3 is adapted to encrypt a payload data set 9a, 9b to at least one of the user devices 1 Ia, 1 Ib.
  • the control unit comprises an encryption key storage unit 6c adapted to store the at least one encryption key KlOl, K 102.
  • the encryption key storage unit 6c could be in any suitable form, for example a RAM, and alternatively provided as a unit separate from the control unit 6, but accessible to the latter.
  • the encryption key storage unit 6c stores encryption keys KlOl, K 102, and data mapping each encryption key KlOl, Kl 02 to a payload data set 9a, 9b and at least one user device 1 Ia, 1 Ib.
  • the user devices 1 Ia, 1 Ib that are mapped to an encryption key stores corresponding encryption keys, in order to decrypt the respective payload data set 9a, 9b when received.
  • the control unit is adapted to determine, when receiving a streaming request, whether the payload data set 9a, 9b requested is to be encrypted when streamed.
  • the control unit is also adapted to determine, if it is determined that the requested payload data set is to be encrypted, whether the user device l la, 1 Ib is mapped to an encryption key KlOl, Kl 02 for the payload data set 9a, 9b. If the user device l la, 1 Ib does not correspond to an encryption key KlOl, Kl 02, the requested payload data set will not be streamed.
  • the control unit 6 is adapted to send the encryption key KlOl, Kl 02 to the networking unit 4, which is adapted to store it, for example in the request data set corresponding to the streaming request, (see above).
  • the networking unit 4 is further adapted to, when receiving from the editing unit 5 the stream portions, (see above), of the payload data set 9a, 9b to be encrypted, to encrypt the stream portions using the encryption key KlOl, K 102, before creating the respective network packets.
  • Each encryption key KlOl, K 102 can be mapped to one user device KlOl, K 102 or a group of user devices.
  • this embodiment provides for session-based encryption during streaming, which can be very useful to operators of streaming services.
  • Fig. 8 is a simplified depiction of a network system comprising one network 2.
  • the system could comprise any number of networks and any type, such as IP (Internet Protocol), ATM (Asynchronous Transfer Mode), or Ethernet based networks, or a network guided by any other protocol, or a combination of any of different network types.
  • Each memory unit 701, 702 is accessible to a streaming unit 3 in the respective streaming device 1001, 1002.
  • the streaming units 3 are adapted to stream the payload data sets 9a, 9b according a plurality of streaming requests from a plurality of user devices 11, also connectable to the network 2.
  • the network comprises a plurality of nodes Nl, N2, in the form of first and second nodes, the second nodes N2 being positioned more centrally in the network 2 than the first nodes Nl .
  • Fig. 8 shows three first nodes Nl and one second node N2.
  • First streaming devices 1001 with respective first memory units 701 are connected to a respective of the first nodes Nl, and a second streaming device 1002 with a second memory unit 702 is connected to a second node N2.
  • the second memory unit 702 is positioned more centrally in the network 2 than the first memory units 701.
  • Fig. 9 is a diagram showing the bandwidth BW requirements for streaming a payload data set in dependence on the data stream request frequency RF for the payload data set, i.e. the popularity of the payload data set. It can be seen that the bandwidth BW requirements increases with the data stream request frequency RF.
  • an administration device ADM in the form of a server, adapted to periodically determine the stream request frequency RF for a plurality of payload data sets stored in the memory units 701, 702. Based on the stream request frequencies RF for the payload data sets, the administration device ADM can relocate payload data sets. More particularly, based at least partly on the stream request frequency RF for a payload data set, the administration device ADM determines whether to store the payload data set in each of the three first memory units 701 or in the second memory unit 702. If the stream request frequency RF for the payload data set is above a first request frequency threshold value RFTl, see fig. 9, the payload data set is stored in each of the three first memory units 701. If the stream request frequency RF for the payload data set is below the first request frequency threshold value RFTl, the payload data set is stored in the second memory unit 702.
  • RFTl first request frequency threshold value
  • a degree of distribution in the network system i.e. more or less centralised, of each memory unit is based at least partly on the stream request frequency RF of the payloads in the respective memory unit.
  • the administration device ADM can be adapted to distribute the payload data sets between the first memory units 701 and the second memory unit 702 such that the bandwidth utilisation in the network 2 for streaming payload data sets stored in the first memory units 701 is higher than the bandwidth utilisation in the network system for streaming payload data sets stored in the second memory unit 702.
  • the payload data sets stored in the memory units can be divided into two groups.
  • a first group comprises the payload data sets (in fig. 10 payload data sets 1-10) having the highest request frequency.
  • a second group comprises payload data sets (in fig. 10 payload data sets 11-25) having request frequencies all lower than the request frequency of any payload data set in the first group.
  • all payload data sets in the first group will be stored in the first memory units 701, and all payload data sets in the second group will be stored in the second memory unit 702. If the administration device ADM determines that the request frequency pattern changes, so that the request frequency of a first payload data set in the first group is lower than the request frequency of a second payload data set in the second group, these two payload data sets will be relocated so that second payload data set is stored in the first memory units 701 and the first payload data set is stored in the second memory unit 702.
  • the second streaming device 1002 is connected to a third memory unit 703, which can be located remotely from the second streaming device 1002.
  • the second streaming device comprises a streaming unit 3, and a control unit 6, as described above.
  • the streaming unit 3 is adapted to stream according to user requests, under the control of the control unit 6, payload data sets stored in the third memory unit 703, as well as from the second memory unit 702.
  • the third memory unit 703 can be located anywhere in the network in relationship to the second streaming device 1002. To illustrate this, in the simplified example shown in fig. 8, the third memory unit 703 can be located in an alternative location indicated by the arrow M.
  • the data streaming from the third memory unit 703 is performed similarly to what has been described above, with the following essential difference:
  • the second streaming device 1002 communicates with the third memory unit 703 via a network interface.
  • the streaming unit 3 sends read requests to the third memory unit 703 in the form of network packets.
  • the retrieved pay load data subsets 91a, 91b and control data subsets 101a, 101b are received from the third memory unit 703 in network packets.
  • the received data packets can contain one or more pairs of payload data subsets 91a, 91b and control data subsets 101a, 101b.
  • the received data packets can contain one or more payload data subsets 91a, 91b or one or more control data subsets 101a, 101b.
  • the received data packets can contain one or more portions of payload data subsets 91a, 91b or one or more portions of control data subsets 101a, 101b.
  • the third memory unit 703 can be included in the memory allocation scheme of the administration device ADM. Thus, if the stream request frequency RF for any payload data set is below a second request frequency threshold value RFT2, see fig. 9, the payload data set is stored in the third memory unit 703.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Astronomy & Astrophysics (AREA)
  • Computer Security & Cryptography (AREA)
  • Human Computer Interaction (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

An apparatus and a method for data streaming, as well as a combination, of an apparatus (1) for data streaming and a plurality of user devices, are presented. The apparatus is adapted to receive a plurality of streaming requests from a plurality of user devices (11a, 11b), the apparatus (1) comprising a control unit (6) adapted to receive media data in the form of a plurality of pay load data sets (9a, 9b) from at least one data source (8), a memory unit (7) adapted to store the payload data sets (9a, 9b), and a streaming unit (3) adapted to stream the payload data sets (9a, 9b) according to the streaming requests. The control unit (6) is adapted to pre-process the payload data sets (9a, 9b) before being stored, whereby control data sets are created, and the streaming unit (3) is adapted to stream the payload data sets (9a, 9b) using the control data sets.

Description

A METHOD AND AN APPARATUS FOR DATA STREAMING
TECHNICAL FIELD
The present invention relates to an apparatus and a method for data streaming, as well as a combination, of an apparatus for data streaming and a plurality of user devices.
BACKGROUND
Streaming is conventionally performed on data with video, sound or multimedia content, and enables the playback of such data without the need to download the entire resource file in advance of playback. This means that such data has to be sent at a pace corresponding, at least on average, to the pace at which it is to be presented. In many streaming applications, users, e.g. video viewers, have display devices, e.g. television sets or personal computers, that are connectable, e.g. via an IP-network, to a server in which media data is stored.
In the case of videos, each user can request individually a video stored in the server to be displayed, so that the video is displayed to the user immediately upon streaming from the server, i.e. without storage at the display device of the entire video. The stream is normally compressed or coded, for example by MPEG-2, and the display device is adapted to decode it.
Where streaming is to be carried out among a large number of users, with individual data in each stream, conventional storage and computation units, including hard drives, provides a limited capacity, and therefore the number of users that can be served with the use of such equipment is very low compared to the total amount of media consumers. EP0698982A2 discloses describes a method in which video/multimedia content is stored in switches or routers of a network in the format of network packets, linked with links fields. This means that the video/multimedia content has to be formatted before being stored, and before formatting the video/multimedia content, an amount of network specific information has to be given. For example, a network protocol has to be chosen before formatting and storing of the video/multimedia content. If the such network specific information, e.g. the network protocol, can vary in dependence on users and requests, which is in practice a common case, several versions of a specific video/multimedia content have to be stored, each with individual network specific information. This requires a large amount of storage space, which makes the system complicated and expensive.
US2003/0095783A1 discloses a system in which protocol stacks are generated in hardware as streams are sent to networks. Among other things, the system presented does not support trick-play modes, such as fast forward or reverse video playback modes, in a capacity efficient manner.
US6067108 discloses a streaming device in which an address table is used during streaming to access a memory for media data. The need to repeatedly read, during data streaming, in the address table adds time to the streaming process, which limits the overall capacity of the device.
SUMMARY
Embodiments of the invention provide an apparatus and method that increase the capacity of data streaming, e.g., for user numbers, request numbers, and/or both.
An embodiment of the invention provides an apparatus for data streaming, adapted to receive a plurality of streaming requests from a plurality of user devices, the apparatus comprising - a control unit adapted to receive media data in the form of a plurality of payload data sets from at least one data source,
- a memory unit adapted to store the payload data sets, and
- a streaming unit adapted to stream the payload data sets according to the streaming requests,
- the control unit being adapted to pre-process the payload data sets before being stored, whereby control data sets are created,
- and the streaming unit being adapted to stream the payload data sets using the control data sets.
Since the control unit may be adapted to pre-process the payload data sets before being stored, whereby control data sets are created, and the streaming unit may be adapted to stream the payload data sets using the control data sets, then demanding and complicated, but less frequent tasks involved when receiving the media data, and those that require flexibility for a service provider, are efficiently separated from pure streaming operations. Such an embodiment provides a good balance of functions between the units, which enables a very efficient use of processing resources, which allows a very high capacity. Embodiments of the invention provides for said demanding and complicated, but less frequent tasks to be handled by a software controlled processor, which allows flexibility for a service provider, while more simple but very frequent streaming operations can be handled by a hard- coded programmed logic device, such as a field programmable gate array (FPGA), which provides extremely high stream rates.
The invention provides an apparatus with a low level of complexity, which allows flexibility for less frequent tasks, and the cost of the apparatus can be allocated to a large amount of users. This reduces overall costs for an administrator of streaming services.
Preferably, the control unit is adapted to, when pre-processing a payload data set, divide it into a plurality of payload data sub-sets, and create for each of at least some of the pay load data sub-sets a control data sub-set, each comprising data corresponding to a location in the memory unit of another payload data sub-set.
Preferably, the streaming unit is adapted to receive a first sub-set of a first set of payload data from the memory unit, and to receive from the memory unit a first sub-set of a first set of control data, which comprises data corresponding to a location in the memory unit of a second sub-set of the first set of payload data. Since the streaming unit receives from the memory unit, along with the sub-sets of payload data, sub-sets control data, each giving a location in the memory unit of a further sub-set of payload data, there is no necessity for repeatedly read, during data streaming, in a separate information source, such as an address table. Thus, since a memory location of further media data is given from the memory unit along with media data portions, memory location information retrieval from a separate source is eliminated. This reduces the number of steps performed in the cyclic streaming operations, which in turn reduced time spent on each stream, so that more streams can be provided, i.e. more users can be served by the apparatus, according to an embodiment of the invention.
Preferably, the first sub-set of the payload data set and the first sub-set of the control data set are stored at the same location in the memory unit. Preferably, the memory unit comprises a plurality of memory sectors, and wherein the payload data set and the control data set is arranged in pairs of payload data sub-sets and control data sub-sets, each pair being stored in a respective of the memory sectors. Preferably, each of at least some of the control data sub-sets in the pairs of payload data sub-sets and control data sub-sets comprises data corresponding to a location of a memory sector of another pair.
Preferably, the streaming unit is adapted to store for each of at least some of the streaming requests a request data set related to the respective streaming request, to create data packets, each based at least partly on a portion of a payload data set and a request data set, and to send the data packets to the user devices. Since data packets are created "on the fly", i.e. during streaming of the payload data, no network specific formatting is needed before storing the payload data. Therefore, the payload data does not have to be stored in multiple versions for different network formats, and this will in turn increase storage capabilities for data streaming applications.
Preferably, the streaming unit is adapted to determine, based at least partly on a plurality of time codes in each of at least some of the payload data sets, bit-rate information, and to stream portions of the payload data sets based on the bit-rate information. Preferably, the control unit comprises a software controlled processor, and the streaming unit comprises a hard-coded programmed logic device. Preferably, the memory unit is a solid state memory. Preferably, the streaming unit is adapted to encrypt a payload data set to at least one of the user devices. Preferably, the streaming unit is adapted to encrypt the payload data set when streamed. Preferably, the streaming unit is adapted to encrypt the payload data set according to at least one encryption key, each corresponding to at least one of the user devices. Preferably, each encryption key corresponds to a group of user devices. Preferably, the apparatus is adapted to store the at least one encryption key.
Preferably, the streaming requests include mode requests, the apparatus comprising a state device adapted to set a mode in accordance with each one of the mode requests, the streaming unit being adapted to stream the payload data sets in accordance with the set modes, the streaming unit being adapted to receive from the memory unit a plurality of control data sub-sets, each comprising at least two edit portions, each comprising data corresponding to an address in the memory unit.
If the streaming unit is adapted to receive from the memory unit a plurality of subsets of the payload data sets, each edit portion may comprise data corresponding to a location in the memory unit of a payload data sub-set. In an embodiment of the invention, the streaming unit receives from the memory unit, along with the sub-sets of pay load data, sub-sets control data, each giving a location in the memory unit of a further sub-set of payload data. This means that there is no need to repeatedly read, during data streaming, in a separate information source, such as an address table. Further, the edit portions of the control data subsets each gives, in dependence on a respective playback mode, an address to a further sub-set of payload data. Thus, each of the edit portions provides a separate linking structure for the payload data sub-sets in dependence on a respective playback mode, according to an embodiment of the invention. Where the payload is a motion picture video, for a mode of normal viewing speed, the edit portions of the control data can provide for a sequential streaming of all payload data sub-sets, whereas in a trick-play mode such as a fast forward mode the edit portions can provide for some of the payload data sub-sets to be skipped in the memory retrieval process. An embodiment of the invention provides for a very effective way of controlling trick-play modes by providing pre-stored linking structures in the memory unit, which structures can be followed selectively, without changing the work process of the streaming unit. The streaming unit can therefore be kept relatively simple in its construction, and streaming can be carried out at a high rate regardless of playback modes requested by the user devices.
The fact that there is no need to repeatedly read, during data streaming, in a separate information source, such as an address table, for further memory addresses, memory location information retrieval from a separate source is eliminated, according to an embodiment of the invention. This reduces the number of steps performed in the cyclic streaming operations, which in turn reduced time spent on each stream, so that more streams can be provided, i.e. more users can be served by the apparatus.
Preferably, the streaming unit is adapted to receive from the memory unit a first sub-set of a first payload data set, and a first sub-set of the control data set, which first control data sub-set comprises a first edit portion comprising data corresponding to a location in the memory unit of a second sub-set of the first payload data set, and a second edit portion comprising data corresponding to a location in the memory unit of a third sub-set of the first payload data set. Preferably, the control unit is adapted to locate a plurality of identification data groups within a payload data set, and provide in each of a plurality of the edit portions of the control data sub-sets data corresponding to the location of an identification data group. Preferably, each identification data group corresponds to a full image data frame.
Embodiments of the invention may also comprise a combination of an apparatus according to any of the claims 1-19, and a plurality of user devices, from which the apparatus is adapted to receive the plurality of streaming requests. Preferably, the user devices are audio/video devices. Preferably, the audio/video devices are at least one of set-top boxes, personal video recorders, or personal computers.
Embodiments of the invention may also comprise a method for data streaming, comprising receiving media data in the form of a plurality of payload data sets from at least one data source, storing the payload data sets in a memory unit, receiving a plurality of streaming requests from a plurality of user devices, and streaming the payload data sets according to the streaming requests, the method comprising pre- processing the payload data sets before being stored, whereby control data sets are created, and streaming the payload data sets using the control data sets. Preferably, the method comprises dividing, when pre-processing a payload data set, it into a plurality of payload data sub-sets, and creating for each of at least some of the payload data sub-sets a control data sub-set, each comprising data corresponding to a location in the memory unit of another payload data sub-set. Preferably, the method comprises receiving a first sub-set of a first set of payload data from the memory unit, and receiving from the memory unit a first sub-set of a first set of control data, which comprises data corresponding to a location in the memory unit of a second sub-set of the first set of payload data. Preferably, the first sub-set of the payload data set and the first sub-set of the control data set are stored at the same location in the memory unit. Preferably, the memory unit comprises a plurality of memory sectors, and wherein the payload data set and the control data set is arranged in pairs of payload data sub-sets and control data sub-sets, each pair being stored in a respective of the memory sectors. Preferably, each of at least some of the control data sub-sets in the pairs of payload data sub-sets and control data sub-sets comprises data corresponding to a location of a memory sector of another pair. Preferably, the method comprises storing for each of at least some of the streaming requests a request data set related to the respective streaming request, creating data packets, each based at least partly on a portion of a payload data set and a request data set, and sending the data packets to the user devices. Preferably, the method comprises determining, based at least partly on a plurality of time codes in each of at least some of the payload data sets, bit-rate information, and streaming portions of the payload data sets based on the bit-rate information. Preferably, the method comprises encrypting a payload data set to at least one of the user devices. Preferably, the method comprises encrypting the payload data set when streamed. Preferably, the method comprises encrypting the payload data set according to at least one encryption key, each corresponding to at least one of the user devices. Preferably, each encryption key corresponds to a group of user devices. Preferably, the method comprises storing the at least one encryption key.
Preferably, the streaming requests include mode requests, the method comprising setting a mode in accordance with each one of the mode requests, streaming the payload data sets in accordance with the set modes, and receiving from the memory unit a plurality of control data sub-sets, each comprising at least two edit portions, each comprising data corresponding to an address in the memory unit. Preferably, the method comprises receiving from the memory unit a first sub-set of a first payload data set, and a first sub-set of the control data set, which first control data sub-set comprises a first edit portion comprising data corresponding to a location in the memory unit of a second sub-set of the first payload data set, and a second edit portion comprising data corresponding to a location in the memory unit of a third sub-set of the first payload data set. Preferably, the method comprises locating a plurality of identification data groups within a payload data set, and providing in each of a plurality of the edit portions of the control data sub-sets data corresponding to the location of an identification data group. Preferably, each identification data group corresponds to the location of a full image data frame.
Embodiments of the invention may also comprise an apparatus, comprising a control unit configured to accept media data from a data source and a plurality of streaming requests from a plurality of user devices, a memory unit configured to store the media data under control of the control unit, and a streaming unit configured to stream portions of the media data corresponding to the streaming requests to the plurality of user devices under the control of the control unit. Preferably, the streaming unit is further configured to stream the portions of the media data using control data. Preferably, the apparatus further comprises a network interface configured to send the portions of the media data over a network to the plurality of user devices. Preferably, the apparatus further comprises a preprocessor configured to accept the media data and determine the control data prior to storage of the media data in the memory unit. Preferably, the apparatus further comprises a user device request unit configured to receive the plurality of streaming requests from the plurality of user devices and transmit the plurality of streaming requests to the streaming unit. Preferably, the apparatus further comprises an encryption device configured to encrypt the portions of the media data using an encryption key corresponding to each user device of the plurality of user devices. Preferably, the apparatus further comprises an encryption key storage unit configured to store a plurality of encryption keys. Preferably, the encryption key storage unit is operably coupled to the control unit. Preferably, the plurality of user devices are audio/video devices. Preferably, the audio/video devices are at least one of set-top boxes, personal video recorders, or personal computers. Preferably, at least one of the media data is an audio/visual data file and one of the plurality of streaming requests is a request for the streaming of the audio/visual data file.
Embodiments of the invention may also comprise a method, comprising accepting media data from a data source and a plurality of streaming requests from a plurality of user devices, generating control data from the media data, storing the media data, and streaming at least a portion of the media data to the plurality of user devices under control of the control data. Preferably, generation of the control data from the media data comprises reviewing the media data to locate identification data groups within the media data; and adding data representing locations of the identification data groups to the control data. Preferably, the identification data groups are complete image data. Preferably, the method further comprises receiving a streaming request from a user device of the plurality of user devices, and streaming a portion of the media data to the user device of the plurality of user devices. Preferably, the method further comprises encrypting the portions of the media data using an encryption key corresponding to each user device of the plurality of user devices. Preferably, the method further comprises storing the plurality of encryption keys in an encryption key store. Preferably, one of the plurality of streaming requests is a request for the streaming of an audio/visual data file.
DESCRIPTION OF THE FIGURES
Below, embodiments of the invention will be described closer with reference to the drawing, in which - fig. 1 is a block diagram of a data network and devices connected thereto including a data streaming device according to one embodiment of the invention,
- fig. 2 block diagram of the data streaming device in fig. 1,
- fig. 3 is an illustration of data sets in the data streaming device in fig. 2, - fig. 4 is a block diagram of a part of the data streaming device in fig. 2,
- fig. 5 is an illustration of data sub-sets handled by the part depicted in fig. 4,
- fig. 6 is a block diagram of memory sectors in a memory unit in the data streaming device in fig. 2,
- fig. 7 is a block diagram of a data network and devices connected thereto including a data streaming device according to an alternative embodiment of the invention, - fig. 8 is a block diagram depicting an arrangement in a network system according to a special aspect of the invention,
- fig. 9 is a diagram showing correlation between data stream request frequency and band width requirements, and - fig. 10 is a diagram showing correlation between payload data sets and data stream request frequency.
DETAILED DESCRIPTION
Fig. 1 depicts schematically a system with an apparatus for data streaming, herein also referred to as a data streaming device 1 for data streaming, connectable to a data network 2, e.g., in the form of an IP-network. The network can alternatively be any kind of network, global or local, suitable for transporting data. Here data refers to information in a form that can be used by a computer, router or any other component of a communications system, and it can include text, numbers, sounds, pictures, and combinations thereof.
A plurality of user devices 11a, 1 Ib are also connectable to the network 2. The user devices 11a, 1 Ib can be any kind of remote devices adapted to receive media data, such as set-top boxes, personal video recorders, personal computers, or any combination thereof.
An embodiment of the data streaming device 1 comprises a streaming unit 3, a control unit 6, and memory unit 7, being connected to each other in a manner described closer below. As also described closer below, an embodiment of the control unit 6 comprises a processor (CPU) and a solid state memory, and is adapted to process data according to software programs, including pre-processing of media data, here also referred to as media payload data, or payload data. The memory unit 7 may be a solid state memory unit adapted to store media payload data. The streaming unit 3 is adapted to read in the memory unit 7, edit and schedule data to be streamed, and send data streams to the user devices l la, 1 Ib via the network 2. The logic functions of the streaming unit may be carried out by one or more programmed logic devices, i.e. hardware units, as described closer below.
Reference is made to fig. 2. In this embodiment, the streaming unit 3 of the data streaming device 1 comprises a networking unit 4, adapted to send and receive data to and from, respectively, the network 2, via a network interface 4a. The streaming unit 3 also comprises an editing unit 5 connected to the networking unit 4 by a suitable bus B54. The editing unit 5 is connected to the memory unit 7 by a bus B75. The editing unit 5 and the networking unit 4 are connected to the control unit by a respective bus B65, B64, such as a PCI (Peripheral Component Interconnect) bus.
In an embodiment of the invention, the networking unit 4 and the editing unit 5 may be hard wired (i.e. hard-coded) with each of them comprising a programmed logic device in the form of a field programmable gate array (FPGA). Alternatively, any other suitable type of programmed logic device, or customized hardware can be used, such as CPLD (Complex Programmable Logic Device), ASIC (Application Specific Integrated Circuit ) or gate array.
The memory unit 7 is adapted to store a plurality of sets of payload data, e.g., in the form of multimedia sequences, for example movies, music files, and/or advertisements. More generally, the payload data can be any form of media data, whether audio, visual, text or code, provided separately or in any combination. Also, a set of payload data, herein also referred to as a payload data set, can provide data for a time limited media display, or be received from a continuous flow of data, for example in the form of a transmission from a TV station. In an embodiment of the invention, the memory unit 7 comprises a solid state memory in the form of a plurality of interconnected so-called flash memory units, i.e. in-circuit programmable non- volatile memories segmented into memory sectors, and is described closer below. Instead of flash memory, any other suitable type of memory can be used, for example a DRAM or an SRAM. Using a solid state memory creates several advantages over conventional hard- drives in streaming applications. An important limiting factor with conventional hard drives is that they provide a speed of information handling (i.e., extraction from storage) that is too low for large scale streaming application, due to, among other things, physical constraints of moving parts. Also, hard drives have a dependability which is less than desired for a good quality of service to users of data streaming services. In addition, hard drives have a high power consumption. In contrast to hard drives, certain solid state memories, specially flash memories, provide a speed of information handling which is more suitable for large scale streaming applications, due to the absence of moving parts, a significantly improved dependability, and a much lower power consumption.
The sets of payload data can be received by the data streaming device 1 from a data source in the form of a remote server 8 via the network 2 and the network interface 4a. Referring to fig. 2 and 3, more specifically, the networking unit 4 is adapted to receive payload data sets 9a, 9b, for example motion picture videos, via the network interface 4a. It should be noted that, besides a server 8, the streaming device 1 can be adapted to receive the payload data sets from any type of data storing and/or transmitting device. Also, in the embodiment presented in fig. 1, the same network 2 is utilized for all network transmissions involving the streaming device 1. Alternatively, separate networks can be used for receiving data from the remote server 8 and for sending data to the user devices l la, 1 Ib. Also, the streaming device 1 can alternatively be provided with more than one network interface 4a. Embodiments of the networking unit 4 may be hard coded to detect the incoming traffic of payload data sets 9a, and, in response to such detection, forward it to the control unit 6, which is adapted to store it temporarily in a control unit storage means 6a, which can be provided in the form of a RAM-memory, for example. It should be noted that the network 2 does not necessarily need to connect both the remote server 8 and the user devices 1 Ia, 1 Ib, e.g., the connections may be private and/or use different protocols. The control unit 6 is adapted to pre-process sets of pay load data, according to an embodiment of the invention. Referring to fig. 3, the control unit 6 is adapted to divide, in the pre-processing, each set of pay load data 9a, 9b into a plurality of payload sub-sets 91a, 91b. The payload data sub-sets 91a, 91b may be adapted to the size of the memory sectors of the flash memories of the memory unit 7. Therefore, apart from the sequence of data within the payload, the division of the payload data into sub-sets need not depend upon the arrangement of payload data in view of the playback thereof. For example, in the case of the payload data being a video, depending on the size of the memory sectors, each sub-set 91a of payload data can include data for a sequence of video frames, or only a part of such a frame.
The control unit 6 may be adapted to create, in the pre-processing of the payload data 9a, 9b, control data, here also referred to as side information. More specifically, the control unit 6 is adapted to create, in the pre-processing of each payload data set 9a, 9b, a set of control data. Each control data set is related to the respective payload data set and includes data for linking the payload data sub-sets 91a, 91b, or parts thereof, when streaming the respective payload data set 9a, 9b to users. As will be described further below, in the case of a video, the linking data is such that it allows the streaming to be carried out in accordance with any of a number of playback modes, herein also simply referred to as modes, for example normal viewing mode, four fast forwarding modes with respective individual speeds, and four reverse modes with respective individual speeds.
Referring to fig. 3, each set of control data is typically provided in the form of control data sub-sets 101a, 101b, each associated with a respective payload data sub-set 91a, 91b. The size of each pair of payload data sub-set 91a, 91b and control data sub-set 101a, 101b is adapted for occupying a respective memory sector of the flash memory units of the memory unit 7. In an embodiment of the invention, the control unit 6 is adapted to locate, in the preprocessing, a plurality of identification data groups within a payload data set 9a, 9b, and provide in each of a plurality of edit portions, further described below, of the control data sub- sets 101a, 101b, data corresponding to the location of an identification data group. Each identification data group corresponds to a full data frame, such as a full image data frame, e.g., a frame that does not include any reference to previous or other frames for their composition. More specifically, videos can be compressed or coded, for example by MPEG-2, and the user devices 11a, 1 Ib can be adapted to decode them. In MPEG-2, full image data frames are referred to as I-frames (intra coded frames). The identification of locations of full image data frames makes it possible to provide, as described closer below, for only such full image data frames to be streamed in compressed videos and at higher playback speeds. Thus, at the pre-processing of the video in the control unit 6, software is used to identify the I-frames to obtain the linking structures of the control data set, according to an embodiment of the invention. Alternatively, a hardware mechanism may be employed that performs a similar function.
It should be noted that the determination of locations of identification data groups can be performed on any media data, whether it is video, audio or of some other type, being in a format such that it contains such data groups, or data sequences, that have a function corresponding to that of the I-frames in the MPEG-2 format, i.e., serving to identify a portion of the media content without referring to other media data.
The control unit 6 is adapted to send, upon pre-processing, the pairs of payload data sub-sets 91a and control data sub-sets 101a to the editing unit 5, which is adapted to forward them for storage in respective memory sectors of the memory unit 7. More specifically, referring to fig. 2 and 4, (fig. 4 being a block diagram depicting functions of the editing unit 5), the control unit 6 is adapted to send the payload data and the control data via the bus B65 to the editing unit 5, which is adapted to forward it to the memory unit 7 via the bus B75, as indicated by the arrow 14 in fig. 4.
Alternatively, the control unit 6 can be adapted to send the payload data and control data directly to the memory unit 7, for example via a bus (not shown) between the two units 6, 7.
In this manner, a large number of payload data sets can be received by the data streaming device 1, pre-processed by the control unit 6, and stored in the memory unit 7.
Reference is made to fig. 1. The data streaming device 1 is adapted to receive streaming requests, herein also referred to simply as requests, via the network 2 from a plurality of user devices. It should be noted that the data streaming device according to a preferred embodiment of the invention is, when used in practice, able to store, e.g., with a storage capacity of 1.5 TBytes, a large amount of payload data sets, e.g. corresponding to 460-1000 hours of video content, and to work simultaneously with requests from up to 10000 user devices. Here, for reasons of simplicity of the presentation, only a first and a second user device l la, 1 Ib are shown in fig. 1.
Referring to fig. 2, the networking unit 4 is adapted to receive, via the network interface 4a, from user devices l la, l ib requests, including playback mode requests, for sets of payload data 9a, 9b. The networking unit is adapted to forward the user device requests to the control unit 6, to be treated by a user request unit 6b, which is adapted to read the requests according to a protocol for media streaming requests, such as RTSP (Real Time Streaming Protocol). The user request unit 6b is adapted to process the requests, each including the identity of the respective user device l la, l ib, the respective set of payload data 9a, 9b and respective the playback mode request, so as to be adapted to a streaming device internal language format, and send them to the editing unit 5. The user request unit 6b is also adapted to extract from the requests sets of request data, each including the identity of the network protocol used for the respective request, and a network address of the respective user device 11a, l ib. The user request unit 6b is further adapted to send the extracted sets of request data to the networking unit 4 to be stored in networking unit storage means in the form of a RAM memory 401.
It should be mentioned that alternatively, the streaming unit 3 can be hard-coded to process requests, so as for the requests to be adapted to a streaming device internal language format.
Referring to an embodiment of the invention shown in fig. 4, the editing unit 5 comprises a user status memory 506, herein also referred to as a state device 506, adapted to receive, as indicated by the arrow A3, from the control unit 6 information corresponding to user requests, received by the streaming device 1 as described above. Specifically, the user status memory 506 is adapted to receive the requests including the identities of the user devices l la, 1 Ib and the playback mode requests Pl Ia, Pl Ib of the user devices l la, l ib, according to an embodiment of the invention. The information about the identity of the user devices l la, 1 Ib, the requested payload data sets 9a, 9b and the playback mode requests may be stored in the user status memory 506.
Alternatively, the user status memory 506 can be provided as a unit, for example including a data table, separate from the streaming unit 3 and the control unit 6.
Further, the editing unit 5 comprises a first and a second multiplexer 501, 502, each adapted to receive from the memory unit 7 control data sub-sets 101a, 101b (see fig. 3), as indicated by the arrows C3 in fig. 4, according to an embodiment of the invention. The editing unit 5 may also comprise a trimming device 505, adapted to receive from the memory unit 7 payload data sub-sets 91a, 91b (see fig. 3), as indicated by the arrow B3 in fig. 4, as well as a pacing device 504 and a scheduling device 507, described further below. As stated, the sets of payload data 9a, 9b, requested by the user devices l la, 1 Ib can be videos. Fig. 5 shows an embodiment of the invention with three payload data sub-sets 91a, 92a, 93 a, one following another in a normal viewing playback mode of the video. The control data set may provide a linking structure for each playback mode that can be requested, e.g normal viewing speed IX, and "trick-play modes". The trick-play modes can include fast forward modes and fast reverse modes. In one embodiment there is a first, second, third and fourth fast forward mode FFl, FF2, FF3, FF4, each providing an individual playback speed, and a first, second, third and fourth fast reverse mode RWl, RW2, RW3, RW4, each providing an individual playback speed. Each payload data sub-set 91a, 92a, 93 a contains a number of picture frames Fl, F2, F3 .. FN. At normal playback speed of the video IX, the linking structure provides for all frames within a payload data sub-set 91a, 92a, 93a to be streamed, and for the first frame Fl in a payload data sub-set to be streamed after the last frame FN of a another payload data sub-set, as indicated in fig. 5 by the arrows A4.
At a higher playback speeds FFl, FF2, FF3, FF4, RWl, RW2, RW3, RW4, the linking structure indicates which of the frames are to be streamed, according to an embodiment of the invention. For example, after a frame which is not the last in a payload data sub-set, a frame in another payload data sub-set can be streamed, as indicated in fig. 5 by the arrows B4. Also, the subsequent frame can be included in a payload data sub-set which does not follow immediately in a normal viewing playback mode, as indicated in fig. 5 by the arrow C4.
As stated above, according to an embodiment of the invention, in compressed videos and at higher playback speeds only frames that do not include references to previous frames for their composition (I-frames) are streamed. In one embodiment, at the first fast forward mode FFl and the first fast reverse mode RWl every I- frame is streamed to the user devices, and at the second fast forward mode FF2 and the second fast reverse mode RW2 every I-frame is streamed to the user devices, and so on.
In addition, the streaming device 1 can be adapted to receive a playback request for a pause mode, i.e. the presentation of a video being frozen so that a non-moving image is shown. Thereby, the streaming unit 3 can be adapted to stream repeatedly the same I-frame to the user device in question. Alternatively, the user device 11a, l ib can be adapted to store an I-frame at a pause request from a user, so that the I- frame is shown to the user for an extended period of time.
The size of the payload data sub-sets 91a, 92a, 93 a are adapted to the size of the memory sectors in the memory unit 7, according to an embodiment of the invention. If the size of each of these memory sectors is relatively small, e.g. 16 kB, the data composing a frame in a video, for example an I-frame according to the MPEG-2 format, can be larger than the memory sector size. (Typically the data for an I-frame can be 64 kB or more.) In such a case, the frame data will simply use two or more memory sectors, and the linking structure of the control data set will provide a proper sequence of the memory sectors storing parts of the same frame. An advantage with providing relatively small memory sectors is that in trick-play modes, compared to a case with relatively large memory sectors, the amount of data cut away by the editing unit 5, as described closer below, will be less.
As described further below, the control data sub-sets also provides addresses to memory sectors in the memory unit 7 storing subsequent sub-sets of payload data, depending on the playback mode requested.
Referring to fig. 4 and 5, in an embodiment of the invention, the editing unit 5 is adapted to, upon receiving the data corresponding to the identity of a user device 1 Ia, 1 Ib, the identity of a set of payload data 9a, 9b, and a playback mode request Pl Ia, Pl Ib, retrieve a first sub-set 91a, 91b of the requested payload data set 9a, 9b, as well as a first sub-set 101a, 101b of the control data set. In the pre-processing described above, the control unit 6 is adapted to store the address in the memory unit 7 of the first subset 91a, 91b of each payload data set 9a, 9b, i.e. in the case of a video, the data for the beginning of the video. The control unit 6 is further adapted to, upon a user request for a payload data set 9a, 9b to retrieve the memory unit 7 address of the first subset 91a, 91b of the payload data set 9a, 9b, and to send this memory unit address to the editing unit with other data regarding the user request. The user status memory 506 is adapted to receive the first memory unit address and to forward it to the pacing device 504, which is adapted to send a request to the memory unit 7 for retrieving the first sub-set 91a, 91b of the requested payload data set 9a, 9b, as well as a first sub-set 101a, 101b of the control data set.
The trimming device 505 is adapted to receive from the memory unit 7 payload data subsets 91a, 91b upon requests from the pacing device 504. The first and second multiplexers 501, 502 are adapted to receive control data sub-sets 101a, 101b, upon requests from the pacing device 504, according to an embodiment of the invention.
The user status memory 506 is adapted to receive periodically from the pacing device 504 data, as indicated by the arrow J4 in fig. 4, corresponding to information regarding the user device, i.e. stream, for which a read request is currently being sent to the memory unit 7. The user status memory 506 may comprise a plurality of rows, each corresponding to one stream according to a user request Pl Ia, Pl Ib, and controls the multiplexers 501, 502 for filtering the control data in accordance with the playback modes of the respective user requests. Thus, the user status memory 506 is adapted to send, in response to the stream identity information J4 from the pacing device 504 control signals, indicated by the arrows K4, L4 in fig. 4, so as to adjust the setting of the multiplexers to the playback mode (IX, FFl, ...) of the stream currently being served.
Thus, according to an embodiment of the invention, the first multiplexer 501 is adapted to receive control data sub-sets 101a, 101b, and, in dependence on the control of the user status memory 506 concerning the requested playback mode Pl Ia, Pl Ib, extract an edit portion of each sub-set 101a of control data, which edit portion gives linking information for the requested playback mode Pl Ia, Pl Ib. As indicated by the arrow D4 in fig. 4, the edit portions of the control data sub-sets 101a, 101b are used to read in the trimming device 505, so that a data sequence according to the linking structure of the respective edit portion of the control data sub-sets 101a, 101b is sent to an output buffer 503a, 503b of an output memory 503.
The output memory 503 is preferably a large solid state memory, which in practice can be a 512 MB DRAM, or alternatively some other kind of random access memory, e.g. SRAM, according to an embodiment of the invention. The output memory 503 is arranged so as to provide a plurality of output buffers 503a, 503b, in the form of FIFO (first-in, first-out) queues, corresponding to the plurality of user devices 11a, 1 Ib to which data streaming is carried out. Similarly to the user status memory 506, the output memory 503 is adapted to receive periodically from the pacing device 504 data, as indicated by the arrow M4 in fig. 4, corresponding to information regarding the user device, i.e. stream, for which a read request is currently being sent to the memory unit 7. Based on this information, an output buffer 503a, 503b is identified for receiving the edited contents of the trimming device 505.
The second multiplexer 502 is also adapted to receive the control data sub-sets 101a, 101b, and to extract, in dependence of the user status memory control, a edit portion of each control data sub-set 101a, 101b, according to an embodiment of the invention. As described further below, each extracted edit portion gives the address in the memory unit 7 of a subsequent payload data sub-set 92a, 92b. As indicated by the arrow E4 in fig. 4, each edit portion of the control data sub-sets 101a, 101b is received by the pacing device 504.
Alternatively, it is possible to replace the first and second multiplexer 501, 502 with other devices, such as only one multiplexer, the output of which is sent to the trimming device 505 as well as the pacing device 504. In the example here, there is one data path from the memory unit 7 to the editing unit 5, so that sub-sets of data are read for one of the individual user device streams at the time in a manner serving the individual user device streams in a cyclic manner, (see further below). However, in practice, several parallel data streams could be provided from the memory unit 7 to the editing unit 5.
The scheduling device 507 is adapted to determine the timing of data outputs from the output memory 503. As indicated by the line F4 in fig. 4, the scheduling device 507 is adapted to read from data traffic from the output buffers 503a, 503b of the output memory 503 information based on which a bit-rate or a pace, at which the respective payload data sets 9a, 9b are to be displayed, can be determined. More specifically, the bit-rate information is determined from known time codes, for example PCR (programme clock reference), used in multimedia display methods, and included in the sets of payload data. The bit-rates corresponding to the respective time codes can differ from one payload data set 9a to another 9b. Thus, the bit-rate of data streamed can differ from one output buffer 503 a to another output buffer 503b.
The scheduling device 507 is adapted to control the data traffic from the output buffers 503a, 503b to the networking unit 4, as indicated by the arrow G4 in fig. 4, in dependence on the bit-rate information for each stream. The data from each output buffer are sent in stream portions, the size of which is adapted to network requirements (e.g. in the order of 1,5 kB). More specifically, the scheduling device 507 is adapted to determine a bite-rate based on time codes in the payload data stream, as mentioned above. The stream in question is sent based on this determined bit-rate, and when a further time code is detected, the determined bit-rate is compared to time information from a clock, and the scheduling device 507 is adapted to adjust, based on this comparison, the timing of data sent in the stream. If it is determined that a stream portion of data from one output buffer 503 a, 503b should not be sent, the scheduling device 507 makes a similar determination concerning the another output buffer 503a, 503b, and works in this respect as a scheduler, performing a cyclic sequence of determinations whether to send stream portions of data from the respective output buffets 503a, 503b to the networking unit 4.
In addition, or alternatively, such determination of the bit-rate information or display pace information can be done at the above-described pre-processing of the payload data set 9a, 9b, in the control unit 6, and included in the control data set corresponding to the payload data set 9a, 9b.
The network unit 4 is adapted to receive stream portions from the respective output buffers 503a, 503b, as well as information identifying the respective output buffer 503a, 503b, and to create data packets based on the stream portions from the output buffers 503a, 503b and the request data sets stored in the networking unit storage means 401. Thereby, the respective output buffer identification provides, or corresponds to an address in the networking unit storage means 401 of the corresponding request data set. The network unit 4 is further adapted to send the data packets to the user devices 11a, 1 Ib via the network interface 4a. Thus, network data packets are created as the data is streamed by the streaming device 1.
Reference is made to fig. 4 and 5. According to an embodiment of the invention, in a similar manner to the cyclic sequence of determinations performed by the scheduling device 507 whether to send stream portions of data from the respective output buffets 503a, 503b to the networking unit 4, the pacing device 504 is adapted to make a cyclic sequence of determinations regarding the data streams whether to send further read requests to the memory unit 7, and works in this respect as a scheduler for read instructions to the memory unit 7. As indicated in fig. 4, each output buffer 503a, 503b is provided with a respective threshold level WMa, WMb in the respective FIFO queue. When determining whether to request more data for a specific output buffer 503a, 503b, the pacing device 504 is adapted to compare the amount of data temporarily stored in the output buffer to the respective threshold level WMa, WMb, and to request more data from the memory unit 7 if the amount of data stored is below the respective threshold level WMa, WMb. Thereby, as indicated by the arrow H4 in fig. 4, based on the memory sector addresses in the control data sub-sets 101a, 101b, the pacing device 504 sends to the memory unit 7 read requests for further payload data sub-sets 92a, 92b and a further control data sub-sets 102a, 102b. Subsequent sub-sets 92a, 92b, 102a, 102b are received by the trimming device 505 and the multiplexers 501, 502, and data is forwarded to the respective output buffer 503a, 503b, in a manner corresponding to what has been described above.
It should be noted that the threshold levels WMa, WMb can be set individually and differently for the output buffers 503a, 503b, or to be the same for a group of output buffers or for all output buffers.
During steaming of payload data, the networking unit 4 can receive a request from a user device for an alternative playback mode. The streaming device 1 is adapted to change the playback mode setting Pl Ia for a user device l la, 1 Ib if the user device l la, 1 Ib requests an alternative playback mode, for example a trick-play mode, e.g., fast forward or fast reverse. More specifically, according to an embodiment of the invention, if the networking unit 4 receives, via the network interface 4a, from a user device l la, 1 Ib a request for an alternative playback mode, the networking unit 4 is adapted to forward the request to the control unit 6, which is adapted to treat the request in a manner corresponding to what has been described above. The control unit 6 is adapted to send the request to the editing unit 5. In the editing unit 5, the user status memory 506 receives the request including the identity of the user device l la, 1 Ib in question, and stores the data corresponding to the requested alternative playback mode.
Referring to fig. 4 and 5, in trick-play modes, similarly to what has been described above, the editing unit 5 is adapted to retrieve, upon read instructions (H4) from the pacing device 504, payload data sub-sets 91a, 91b, as well as control data sub-sets 101a, 101b. Thereby, the trimming device 505 is adapted to receive and temporarily store the payload data sub-sets 91a, 91b. The first multiplexer 501 is adapted to receive control data sub-sets 101a, 101b and extract an edit portion of each of them in accordance with the requested trick-play mode Pl Ia, Pl Ib, which edit portions
5 gives linking information for the requested trick-play mode Pl Ia, Pl Ib, (see also below with reference to fig. 6). As indicated by the arrow D4 in fig. 4, the edit portions of the control data sub-sets 101a, 101b is used to read in the trimming device 505, so that a data sequence according to the linking structure of the respective edit portions is sent to the respective output buffers 503a, 503b of the l o output memory 503.
Thus, referring to fig. 4 and 5, upon reception in the trimming device 505 of a payload data sub-set 92a, 92b, the linking structure provided by an edit portion of the control data sub-set 101a, 101b, which edit portion corresponds to the requested 15 trick-play mode Pl Ia, Pl Ib, is used to extract a trimmed portion of the payload data sub-set 92a, 92b to be sent to the output buffer 503a, 503b for the stream in question. Parts of the payload data sub-set 92a, 92b excluded in the trimming device by the linking structure are simply deleted from the stream.
20 In fast trick-play modes of videos, preferably the linking structure of the control data sub-set is such that only full image data frames, i.e. frames that do not include references to previous frames for their composition, e.g. I-frames in MPEG-2, are sent to the output buffers 503a, 503b. In compressed videos and at higher playback speeds only so called I-frames (intra coded frames), i.e., are streamed, according to
25 an embodiment of the invention. At the pre-processing of the video in the control unit 6, a software is used to identify the I-frames to obtain the linking structures of the control data set, as described closer below.
As understood from the presentation above, according to an embodiment of the 30 invention, a trick-play linking structure of a control data set provides for a separate addressing sequence in the memory unit 7, followed upon read instructions by the pacing unit 504, so that memory sectors that would have been included in a normal viewing speed play mode might be "skipped".
At trick-play modes, the scheduling device 507 is adapted to determine the timing of the outputs from the respective output buffer 503a, 503b, based on respective bit- rates at normal viewing speeds IX, in turn determined from time codes as described above. Thus, at a trick-play mode, a payload data set will be streamed to a user device at a bit-rate being the same as that at which it is streamed at a normal viewing speed mode IX. Alternatively, the bit-rate at trick-play modes can be predetermined individually for the payload data sets, or for groups of payload data sets, or collectively for all payload data sets.
Reference is made to fig. 6, showing an example of how memory sector locations in the memory unit 7 are accessed, according to an embodiment of the invention. Three pairs of payload data sub-sets 91a, 92a, 93a and control data sub-sets 101a, 102a, 103 a are in this example stored in consecutive first, second and third memory sectors 71, 72, 73 of the memory unit 7. In all memory sectors, the memory space available for payload data and control data is the same, in this example 16000 bytes and 1000 bytes, respectively.
Assuming that the playback mode of the current user device request is normal viewing speed (IX), after having retrieved data from the first memory sector 71, the editing unit 5 determines from the first edit portion (Ix) of the first control data subset 101a the memory address (17001) of the start of the next memory sector 72. The editing unit 5 also determines from the first edit portion (Ix) of the first control data sub-set 101a that the offset from the start of the second payload data subset 92a is 0 bytes and the length within the subset 92a of the data to be streamed is 16000 bytes, which means that the output buffer 503a (fig. 4) for the stream in question will receive all data in the second payload data subset 92a. Assuming differently that the playback mode of the current user device request is the first fast forward mode FFl, after having retrieved data from the first memory sector 71, the editing unit 5 determines from a second edit portion (FFl) of the first control data sub-set 101a the memory address (17001) of the start of the next memory sector 72. The editing unit 5 also determines from the second edit portion (FFl) of the first control data sub-set 101a that the offset from the start of the second payload data subset 92a is 6000 bytes and the length within the subset 92a of the data to be streamed is 5000 bytes, which means that the output buffer 503a (fig. 4) for the stream in question will receive all data in the interval [A, B] in fig. 6.
Alternatively, assuming that the playback mode of the current user device request is the second fast forward mode FF2, after having retrieved data from the first memory sector 71, the editing unit 5 determines from the third edit portion (FF2) of the first control data sub-set 101a the memory address (34001) of the start of the next memory sector 73, which means that a memory sector 72 read in normal viewing speed playback mode (IX) is skipped. The editing unit 5 also determines from the third edit portion (FF2) of the first control data sub-set 101a that the offset from the start of the third payload data subset 93 a is 8000 bytes and the length within the subset 93 a of the data to be streamed is 4000 bytes, which means that the output buffer 503a (fig. 4) for the stream in question will receive all data in the interval [C, D] in fig. 6.
The editing unit 5 and the memory unit 7 are arranged so that at each data retrieval step, the respective control data subsets 101a, 102a, 103 a are read in their entirety. Thus, all data between a predetermined offset location, in this example 16001 bytes, from the start location of the respective memory sector 71, 72, 73 to the end of the memory sector, is retrieved.
To illustrate the memory access sequence in a reverse playback mode, according to an embodiment of the invention, fig. 6 shows an edit portion of a third control data subset 103 a giving local linking information for the first fast reverse playback mode RWl. After having retrieved data from the third memory sector 73, the editing unit 5 determines from the edit portion (RWl) of the third control data sub-set 103 a the memory address (17001) of the start of the next memory sector 72, which would have preceded the third sector 73 in normal viewing playback mode streaming. The offset value (6000) and the length value (5000) are used in a manner corresponding to what has been described above with reference to fig. 6 concerning forward playback modes.
In the example presented with reference to fig. 6, the edit portions of the control data sub-sets 101a, 102a, 103a are described as pointing to the start of the next memory sector 71, 72, 73 to be read for the stream. However, different ways of identifying the next memory sector 71, 72, 73 can be used. For example, a position in the middle of the next memory sector 71, 72, 73 can be identified, possibly combined with a local pointer within the next memory sector 71, 72, 73.
Having described the streaming device above, an example of a method in the streaming device 1 will be described, according to an embodiment of the invention.
Referring to fig. 2 and 3, a first set of payload data 9a, for example a first motion picture video, is received by the networking unit 4 via the network interface 4a. The networking unit 4 detects the incoming traffic of the first set of payload data 9a and forwards it to the control unit 6.
Referring to fig. 3, in the pre-processing, the first set of payload data 9a is divided into a plurality of sub-sets 91a, and a first set of control data, in the form of sub-sets 101a of control data, is created, including data for linking the sub-sets 91a of payload data, or parts thereof. Upon pre-processing, the pairs of payload data subsets 91a and control data sub-sets 101a are sent to the editing unit 5, and then stored in respective memory sectors of the memory unit 7. Further, a second set of payload data 9b is received by the networking unit 4, and forwarded to the control unit 6 to be pre-processed as described above. Thus, referring to fig. 3, the second set of payload data 9b is divided into a plurality of sub-sets 91b, and a second set of control data in the form of sub-sets 101b of control data is created, and the pairs of payload data sub-sets 91a and control data sub-sets 101a, 101b are stored in respective memory sectors of the memory unit 7.
The networking unit 4 receives, via the network interface 4a, from a first user device 1 Ia a first request, including a playback mode request, for the first set of payload data 9a. The networking unit 4 forwards the first request to the control unit 6, to be adapted to a streaming device internal language format, and the first request send to the editing unit 5. The control unit 6 extracts from the first request a first request data set, including the identity of the network protocol used for the first request, and a network address of the user device 11a, and sends the first request data set to the networking unit 4 to be stored in the RAM memory 401.
In this example the networking unit 4 also receives from the second user device l ib a first request, including a playback mode request, for the second set of payload data 9b, whereupon the first request, including the identity of the second user device 1 Ib, the second set of payload data 9b and the playback mode request, is forwarded to the control unit 6, which extracts a first set of request data to be stored in the RAM memory 401, in the same manner as outlined above.
The user status memory 506 receives, as indicated by the arrow A3 in fig. 4, from the control unit 6 data corresponding to the identity of the first user device 11a, the playback mode request Pl Ia, as well as a memory unit address of the first subset 91a of the first payload data set 9a, provided as described above, which data is stored in the user status memory 506. The user status memory 506 forwards the memory unit address to the pacing device 504, which sends a request to the memory unit 7 for retrieving the first sub-set 91a of the first payload data set 9a, as well as the first sub-set 101a of the first control data set. Similarly, the user status memory 506 receives data corresponding to the identity of the second user device 1 Ib, the playback mode request Pl Ib, as well as a memory unit address of the first subset 91b of the second payload data set 9b, provided as described above, which data is stored in the user status memory 506. The user status memory 506 forwards the memory unit address to the pacing device 504, which sends a request to the memory unit 7 for retrieving the first sub-set 91b of the second payload data set 9b, as well as the first sub-set 101b of the second control data set.
In this example, initially, the playback mode request Pl Ia of the first user device 1 Ia is a request for a normal viewing speed (IX) of the video. The first sub-set 91a of payload data is sent to the trimming device 505, where it is temporarily stored.
The first multiplexer 501 receives a first control data sub-set 101a, and extracts, in dependence on the control of the user status memory 506 concerning the requested playback mode Pl Ia, a first edit portion of the first control data sub-set 101a, which first edit portion gives linking information for the requested normal viewing speed (IX). As indicated by the arrow D4 in fig. 4, the first edit portion of the first control data sub-set 101a is used to read in the trimming device 505, so that the full sequence of data of the first payload data subset 91a is stored in a first output buffer 503 a of the output memory 503.
The second multiplexer 502 also receives the first control data sub-set 101a, and extracts, in dependence of the user status memory control, a first edit portion of the first control data sub-set 101a, which first edit portion gives the address in the memory unit 7 of a second sub-set 92a of the first payload data set 9a. The first edit portion of the first control data sub-set 101a is received by the pacing device 504. Based on the playback mode request Pl Ib of the second user device 1 Ib, and a first edit portion of a first sub-set 101b of a second set of control data, a first subset 91b of payload data is stored in a second output buffer 503b. In a manner described above, the scheduling device 507 determines first bit-rate information, or first display pace information, based on information in the first set of pay load data 9a, and corresponding to a pace at which the first set of pay load data 9a is to be displayed in the first user device 9a. Data from the first output buffer 503 a is sent based on the first bit-rate information, and when a further time code in the first payload data set 91a is detected, the first bit-rate information is compared to time information from the clock, and based on this comparison the scheduling device 507 adjusts the timing of data sent from the first output buffer 503a. If it is determined that a stream portion of data from the first output buffer 503 a should not be sent, the scheduling device 507 makes a similar determination concerning the second output buffer 503b, performing a cyclic sequence of determinations whether to send stream portions of data from the respective output buffets 503a, 503b to the networking unit 4.
After reception of a stream portion of data from the first output buffer 503 a, in the networking unit 4 a data packet is created based on the stream portion of data and the first set of request data stored in the networking unit storage means 401, and the data packet is sent to the to the first user device 1 Ia via the network interface 4a. Similarly, after reception of a stream portion of data from the second output buffer 503b, in the networking unit 4 a data packet is created based on the stream portion of data and the second set of request data stored in the networking unit storage means 401, and the data packet is sent to the to the second user device 1 Ib via the network interface 4a.
As indicated by the arrow H4 in fig. 4, in dependence, as described above, on the data level in the respective output buffer 503a, 503b, and based on the memory sector address in the first sub-set 101a of the first set of control data, the pacing device 504 sends to the memory unit 7 a read request for a second sub-set 92a of the first set 9a of payload data and a second sub-set 102a of the first set of control data. These sub-sets 92a, 102a are received by the trimming device 505 and the multiplexers 501, 502, and data is forwarded to the output memory 503, in a manner corresponding to what has been described above.
After starting of the data streaming to the first user device l la, the first user device l la requests the second fast forwarding speed of the video FF2, and the playback mode setting Pl Ia for the first user 1 Ia in the user status memory 506 will be changed. More specifically, the networking unit 4 receives, via the network interface 4a, from the first user device 1 Ia a second request, including a playback mode request for the second fast forward mode FF2. The networking unit 4 forwards the second request to the control unit 6, to be treated by the user request unit 6b, and send to the editing unit 5. In the editing unit 5, the user status memory 506 receives the second request including the identity of the first user device l la, which is stored in the user status memory 506.
Referring also to fig. 5, using the same denomination for the data sub-sets as in the example above, upon a read instruction H4 from the pacing device 504, the editing unit 5 retrieves a first sub-set 91a of the first set of payload data 9a, as well as a first sub-set 101a of the first set of control data. The first sub-set 91a of payload data is sent to the trimming device 505, where it is temporarily stored.
The first control data sub-set 101a is received by the first multiplexer 501, which in dependence of the requested playback mode Pl Ia, extracts a third edit portion of the first sub-set 101a of control data, which third edit portion gives linking information for the requested fast forward viewing speed (FF2), (see also below with reference to fig. 6). As indicated by the arrow D4 in fig. 4, the third edit portion (FF2) of the first sub-set 101a of control data is used to read in the trimming device 505, so that a data sequence according to the linking structure of the third edit portion (FF2) is sent to the first output buffer 503 a of the output memory 503. Thus, referring to fig. 4 and 5, upon reception in the trimming device 505 of a second sub-set 92a of the first set of payload data 9a, the linking structure provided by the third edit portion (FF2), indicated by the arrow B4 in fig. 5, is used to extract a trimmed portion of the a second sub-set 92a of the first set of payload data 9a to be sent to the first output buffer 503 a. Parts of the first set of payload data 9a excluded in the trimming device by the linking structure are simply deleted from the stream.
It should be mentioned that embodiments of the invention allow a stream at fast speed playback modes to be "tailored" according to desires of a service operator. For example, the linking structure can be made so that commercials are shown at normal viewing speed, although fast speed has been requested by the user. More specifically, the fast speed linking structure (e.g. for FF3) can be made to coincide with a normal speed linking structure (IX), or a slower fast speed linking structure (e.g. for FF2).
The data streaming device according to the invention can in practice be provided in a large number of embodiments, varying from what has been described in the example above. For example, any of the functional units of the streaming device 1, such as the streaming unit 3, the editing unit 5, the networking unit 4, the memory unit 7 or the control unit 6, can be provided as a plurality of logically interconnected devices. Further, the editing unit 5 and the networking unit 4 can be provided as a physically integrated unit. In a further embodiment, the memory unit 7 can be provided as two or more separate units. One such memory sub-unit can be adapted to store payload data sets, and another memory unit can be adapted to store control data sets, both sub-units being accessed by the streaming unit 3 during data streaming.
Further, the control unit 6 can be provided in a separate apparatus, for example a computer, which could be located remotely from the streaming unit 3 and the memory unit 7, and connectable thereto via a network. Specially, referring to fig. 7, in an alternative embodiment, the control unit comprises two sub-units, one of which, here referred to as a request device 602 is provided at the streaming unit 3 and is adapted to receive and process user device requests as described above. The other sub-unit, herein referred to as a pre-processing unit 601, could be located remotely from other streaming device components and be adapted to perform the above described pre-processing of media data.
According to a special embodiment, the streaming unit 3 is adapted to encrypt a payload data set 9a, 9b to at least one of the user devices 1 Ia, 1 Ib. As shown in fig. 2, the control unit comprises an encryption key storage unit 6c adapted to store the at least one encryption key KlOl, K 102. The encryption key storage unit 6c could be in any suitable form, for example a RAM, and alternatively provided as a unit separate from the control unit 6, but accessible to the latter. The encryption key storage unit 6c stores encryption keys KlOl, K 102, and data mapping each encryption key KlOl, Kl 02 to a payload data set 9a, 9b and at least one user device 1 Ia, 1 Ib. The user devices 1 Ia, 1 Ib that are mapped to an encryption key stores corresponding encryption keys, in order to decrypt the respective payload data set 9a, 9b when received.
The control unit is adapted to determine, when receiving a streaming request, whether the payload data set 9a, 9b requested is to be encrypted when streamed. The control unit is also adapted to determine, if it is determined that the requested payload data set is to be encrypted, whether the user device l la, 1 Ib is mapped to an encryption key KlOl, Kl 02 for the payload data set 9a, 9b. If the user device l la, 1 Ib does not correspond to an encryption key KlOl, Kl 02, the requested payload data set will not be streamed.
If it is determined that the user device l la, 1 Ib is mapped to an encryption key
KlOl, K 102, the control unit 6 is adapted to send the encryption key KlOl, Kl 02 to the networking unit 4, which is adapted to store it, for example in the request data set corresponding to the streaming request, (see above). The networking unit 4 is further adapted to, when receiving from the editing unit 5 the stream portions, (see above), of the payload data set 9a, 9b to be encrypted, to encrypt the stream portions using the encryption key KlOl, K 102, before creating the respective network packets.
Each encryption key KlOl, K 102 can be mapped to one user device KlOl, K 102 or a group of user devices. Thus, this embodiment provides for session-based encryption during streaming, which can be very useful to operators of streaming services.
In data streaming applications, it is desirable to obtain a maximum efficiency in the use of the data transport networks. In particular, the use of bandwidth available should be as efficient as possible.
In order to increase, in data streaming applications, the efficiency of network utilisation, one aspect of the invention provides an arrangement, one embodiment of which is described here with reference to fig. 8 and 9. Fig. 8 is a simplified depiction of a network system comprising one network 2. In practice, the system could comprise any number of networks and any type, such as IP (Internet Protocol), ATM (Asynchronous Transfer Mode), or Ethernet based networks, or a network guided by any other protocol, or a combination of any of different network types.
Connectable to the network are a number of data streaming devices 1001, 1002, each with a memory unit 701, 702 for storing media data, for example as described above, in the form of payload data sets 9a, 9b. Each memory unit 701, 702 is accessible to a streaming unit 3 in the respective streaming device 1001, 1002. The streaming units 3 are adapted to stream the payload data sets 9a, 9b according a plurality of streaming requests from a plurality of user devices 11, also connectable to the network 2.
The network comprises a plurality of nodes Nl, N2, in the form of first and second nodes, the second nodes N2 being positioned more centrally in the network 2 than the first nodes Nl . Fig. 8 shows three first nodes Nl and one second node N2. First streaming devices 1001 with respective first memory units 701 are connected to a respective of the first nodes Nl, and a second streaming device 1002 with a second memory unit 702 is connected to a second node N2. Thus, the second memory unit 702 is positioned more centrally in the network 2 than the first memory units 701.
Fig. 9 is a diagram showing the bandwidth BW requirements for streaming a payload data set in dependence on the data stream request frequency RF for the payload data set, i.e. the popularity of the payload data set. It can be seen that the bandwidth BW requirements increases with the data stream request frequency RF.
Connectable to the network 2 is an administration device ADM, in the form of a server, adapted to periodically determine the stream request frequency RF for a plurality of payload data sets stored in the memory units 701, 702. Based on the stream request frequencies RF for the payload data sets, the administration device ADM can relocate payload data sets. More particularly, based at least partly on the stream request frequency RF for a payload data set, the administration device ADM determines whether to store the payload data set in each of the three first memory units 701 or in the second memory unit 702. If the stream request frequency RF for the payload data set is above a first request frequency threshold value RFTl, see fig. 9, the payload data set is stored in each of the three first memory units 701. If the stream request frequency RF for the payload data set is below the first request frequency threshold value RFTl, the payload data set is stored in the second memory unit 702.
Thus, a degree of distribution in the network system, i.e. more or less centralised, of each memory unit is based at least partly on the stream request frequency RF of the payloads in the respective memory unit. Thereby, a very good balance between bandwidth usage and storage usage is accomplished resulting in a high efficiency in the usage of components in the network system. Referring to fig. 10, alternatively, or in addition, the administration device ADM can be adapted to distribute the payload data sets between the first memory units 701 and the second memory unit 702 such that the bandwidth utilisation in the network 2 for streaming payload data sets stored in the first memory units 701 is higher than the bandwidth utilisation in the network system for streaming payload data sets stored in the second memory unit 702. Thereby, the payload data sets stored in the memory units can be divided into two groups. (In alternative embodiments more than two payload data set groups can be introduced.) A first group comprises the payload data sets (in fig. 10 payload data sets 1-10) having the highest request frequency. A second group comprises payload data sets (in fig. 10 payload data sets 11-25) having request frequencies all lower than the request frequency of any payload data set in the first group.
Referring to fig. 8 and 10, according to this embodiment, all payload data sets in the first group will be stored in the first memory units 701, and all payload data sets in the second group will be stored in the second memory unit 702. If the administration device ADM determines that the request frequency pattern changes, so that the request frequency of a first payload data set in the first group is lower than the request frequency of a second payload data set in the second group, these two payload data sets will be relocated so that second payload data set is stored in the first memory units 701 and the first payload data set is stored in the second memory unit 702.
Referring to fig. 8, the second streaming device 1002 is connected to a third memory unit 703, which can be located remotely from the second streaming device 1002. The second streaming device comprises a streaming unit 3, and a control unit 6, as described above. The streaming unit 3 is adapted to stream according to user requests, under the control of the control unit 6, payload data sets stored in the third memory unit 703, as well as from the second memory unit 702. It should be noted that the third memory unit 703 can be located anywhere in the network in relationship to the second streaming device 1002. To illustrate this, in the simplified example shown in fig. 8, the third memory unit 703 can be located in an alternative location indicated by the arrow M.
The data streaming from the third memory unit 703 is performed similarly to what has been described above, with the following essential difference: The second streaming device 1002 communicates with the third memory unit 703 via a network interface. The streaming unit 3 sends read requests to the third memory unit 703 in the form of network packets. The retrieved pay load data subsets 91a, 91b and control data subsets 101a, 101b (see above) are received from the third memory unit 703 in network packets. The received data packets can contain one or more pairs of payload data subsets 91a, 91b and control data subsets 101a, 101b. Alternatively, the received data packets can contain one or more payload data subsets 91a, 91b or one or more control data subsets 101a, 101b. As a further alternative, the received data packets can contain one or more portions of payload data subsets 91a, 91b or one or more portions of control data subsets 101a, 101b.
The third memory unit 703 can be included in the memory allocation scheme of the administration device ADM. Thus, if the stream request frequency RF for any payload data set is below a second request frequency threshold value RFT2, see fig. 9, the payload data set is stored in the third memory unit 703.
The above description of illustrated embodiments of the invention is not intended to be exhaustive or to limit the invention to the precise form disclosed. While specific embodiments of, and examples for, the invention are described herein for illustrative purposes, various equivalent modifications are possible within the scope of the invention, as those skilled in the relevant art will recognize. The teachings provided herein of the invention can be applied to other streaming systems, not necessarily the exemplary device described above.
The various embodiments described above can be combined to provide further embodiments. All of the above patents and applications are incorporated by reference. Aspects of the invention can be modified, if necessary, to employ the systems, circuits and concepts of the various patents and applications described above to provide yet further embodiments of the invention.
These and other changes can be made to the invention in light of the above detailed description. In general, in the following claims, the terms used should not be construed to limit the invention to the specific embodiments disclosed in the specification and the claims, but should be construed to include all streaming media systems that operate under the claims to provide a method for separating control from streaming. Accordingly, the invention is not limited by the disclosure, but instead the scope of the invention is to be determined by the following claims.

Claims

1. An apparatus for data streaming, adapted to receive a plurality of streaming requests from a plurality of user devices (Ha, 1 Ib), the apparatus (1) comprising
- a control unit (6) adapted to receive media data in the form of a plurality of payload data sets (9a, 9b) from at least one data source (8),
- a memory unit (7) adapted to store the payload data sets (9a, 9b), and
- a streaming unit (3) adapted to stream the payload data sets (9a, 9b) according to the streaming requests,
- the control unit (6) being adapted to pre-process the payload data sets (9a, 9b) before being stored, whereby control data sets are created,
- and the streaming unit (3) being adapted to stream the payload data sets (9a, 9b) using the control data sets.
2. An apparatus according to claim 1, wherein the control unit (6) is adapted to, when pre-processing a payload data set (9a, 9b), divide it into a plurality of payload data sub-sets (91a, 91b), and create for each of at least some of the payload data sub-sets (91a, 91b) a control data sub-set (101a, 101b), each comprising data corresponding to a location in the memory unit (7) of another payload data sub-set (92a, 92b).
3. An apparatus according to any of the preceding claims, wherein the streaming unit (3) is adapted to receive a first sub-set (91a) of a first set of payload data (9a) from the memory unit (7), and to receive from the memory unit (7) a first sub-set (101a) of a first set of control data, which comprises data corresponding to a location in the memory unit of a second sub-set (92a) of the first set of payload data (9a).
4. An apparatus according to claim 3, wherein the first sub-set (91a, 91b) of the payload data set (9a, 9b) and the first sub-set (101a, 101b) of the control data set are stored at the same location in the memory unit (7).
5. An apparatus according to any of the claims 3-4, wherein the memory unit (7) comprises a plurality of memory sectors (71, 72, 73), and wherein the payload data set (9a, 9b) and the control data set is arranged in pairs of payload data sub-sets (91a, 91b) and control data sub-sets (101a, 101b), each pair being stored in a respective of the memory sectors (71, 72, 73).
6. An apparatus according to claim 5, wherein each of at least some of the control data sub-sets (101a, 101b) in the pairs of payload data sub-sets (91a, 91b) and control data sub-sets (101a, 101b) comprises data corresponding to a location of a memory sector of another pair.
7. An apparatus according to any of the preceding claims, wherein the streaming unit (3) is adapted to store for each of at least some of the streaming requests a request data set related to the respective streaming request, to create data packets, each based at least partly on a portion of a payload data set (9a, 9b) and a request data set, and to send the data packets to the user devices (Ha, 1 Ib).
8. An apparatus according to any of the preceding claims, wherein the streaming unit (3) is adapted to determine, based at least partly on a plurality of time codes in each of at least some of the payload data sets, bit-rate information, and to stream portions of the payload data sets (9a, 9b) based on the bit-rate information.
9. An apparatus according to any of the preceding claims, wherein the control unit (6) comprises a software controlled processor, and the streaming unit (3) comprises a hard-coded programmed logic device.
10. An apparatus according to any of the preceding claims, wherein the memory unit (7) is a solid state memory.
11. An apparatus according to any of the preceding claims, wherein the streaming unit (3) is adapted to encrypt a payload data set (9a, 9b) to at least one of the user devices (Ha, 1 Ib).
12. An apparatus according to claim 11, wherein the streaming unit (3) is adapted to encrypt the payload data set (9a, 9b) when streamed.
13. An apparatus according to any of the claims 11-12, wherein the streaming unit (3) is adapted to encrypt the payload data set (9a, 9b) according to at least one encryption key (KlOl, K 102), each corresponding to at least one of the user devices (1 Ia, 1 Ib).
14. An apparatus according to claim 13, wherein each encryption key (KlOl, K102) corresponds to a group of user devices (1 Ia, 1 Ib).
15. An apparatus according to any of the claims 13-14, adapted to store the at least one encryption key (KlOl, K102).
16. An apparatus according to any of the preceding claims, wherein the streaming requests include mode requests (Pl Ia, Pl Ib), the apparatus comprising a state device (506) adapted to set a mode in accordance with each one of the mode requests (Pl Ia, Pl Ib), the streaming unit (3) being adapted to stream the payload data sets (9a, 9b) in accordance with the set modes, the streaming unit (3) being adapted to receive from the memory unit (7) a plurality of control data sub-sets (101a), each comprising at least two edit portions (IX, FFl, FF2 ... ), each comprising data corresponding to an address in the memory unit (7).
17. An apparatus according to claim 16, wherein the streaming unit (3) is adapted to receive from the memory unit (7) a first sub-set (91a) of a first payload data set (9a), and a first sub-set (101a) of the control data set, which first control data sub-set (101a) comprises a first edit portion comprising data corresponding to a location in the memory unit (7) of a second sub-set (92a) of the first payload data set (9a), and a second edit portion comprising data corresponding to a location in the memory unit (7) of a third sub-set (93a) of the first payload data set (9a).
18. An apparatus according to any of the claims 16-17, wherein the control unit (6) is adapted to locate a plurality of identification data groups within a payload data set (9a), and provide in each of a plurality of the edit portions (IX, FFl, FF2 ... ) of the control data sub-sets (101a) data corresponding to the location of an identification data group.
19. An apparatus according to claim 18, wherein each identification data group corresponds to a full image data frame.
20. A combination of an apparatus ( 1 ) according to any of the claims 1-19, and a plurality of user devices (Ha, 1 Ib), from which the apparatus (1) is adapted to receive the plurality of streaming requests.
21. A combination according to claim 20, wherein the user devices (Ha, 1 Ib) are audio/video devices.
22. A combination according to claim 21, wherein the audio/video devices are at least one of set-top boxes, personal video recorders, or personal computers.
23. A method for data streaming, comprising - receiving media data in the form of a plurality of payload data sets (9a, 9b) from at least one data source (8),
- storing the payload data sets (9a, 9b) in a memory unit (7),
- receiving a plurality of streaming requests from a plurality of user devices (Ha, l ib), and
- streaming the payload data sets (9a, 9b) according to the streaming requests,
- the method comprising pre-processing the payload data sets (9a, 9b) before being stored, whereby control data sets are created,
- and streaming the payload data sets (9a, 9b) using the control data sets.
24. A method according to claim 23, comprising dividing, when pre-processing a payload data set (9a, 9b), it into a plurality of payload data sub-sets (91a, 91b), and creating for each of at least some of the payload data sub-sets (91a, 91b) a control data sub-set (101a, 101b), each comprising data corresponding to a location in the memory unit (7) of another payload data sub-set (92a,
92b).
25. A method according to any of the claims 23-24, comprising receiving a first sub-set (91a) of a first set of payload data (9a) from the memory unit (7), and receiving from the memory unit (7) a first sub-set (101a) of a first set of control data, which comprises data corresponding to a location in the memory unit of a second sub-set (92a) of the first set of payload data (9a).
26. A method according to claim 25, wherein the first sub-set (91a, 91b) of the payload data set (9a, 9b) and the first sub-set (101a, 101b) of the control data set are stored at the same location in the memory unit (7).
27. A method according to any of the claims 25-26, wherein the memory unit (7) comprises a plurality of memory sectors (71, 72, 73), and wherein the payload data set (9a, 9b) and the control data set is arranged in pairs of payload data sub-sets (91a, 91b) and control data sub-sets (101a, 101b), each pair being stored in a respective of the memory sectors (71, 72, 73).
28. A method according to claim 27, wherein each of at least some of the control data sub-sets (101a, 101 b) in the pairs of payload data sub-sets (91 a, 91 b) and control data sub-sets (101a, 101b) comprises data corresponding to a location of a memory sector of another pair.
29. A method according to any of the claims 23-28, comprising storing for each of at least some of the streaming requests a request data set related to the respective streaming request, creating data packets, each based at least partly on a portion of a payload data set (9a, 9b) and a request data set, and sending the data packets to the user devices (Ha, 1 Ib).
30. A method according to any of the claims 23-29, comprising determining, based at least partly on a plurality of time codes in each of at least some of the payload data sets, bit-rate information, and streaming portions of the payload data sets (9a, 9b) based on the bit-rate information.
31. A method according to any of the claims 23-30, comprising encrypting a payload data set (9a, 9b) to at least one of the user devices (11a, 1 Ib).
32. A method according to claim 31, comprising encrypting the payload data set (9a, 9b) when streamed.
33. A method according to any of the claims 31-32, comprising encrypting the payload data set (9a, 9b) according to at least one encryption key (KlOl, K102), each corresponding to at least one of the user devices (1 Ia, 1 Ib).
34. A method according to claim 33, wherein each encryption key (KlOl, K102) corresponds to a group of user devices (Ha, 1 Ib).
35. A method according to any of the claims 33-34, comprising storing the at least one encryption key (KlOl, K102).
36. A method according to any of the claims 23-35, wherein the streaming requests include mode requests (Pl Ia, Pl Ib), the method comprising setting a mode in accordance with each one of the mode requests (Pl Ia, Pl Ib), streaming the payload data sets (9a, 9b) in accordance with the set modes, and receiving from the memory unit (7) a plurality of control data sub-sets (101a), each comprising at least two edit portions (IX, FFl, FF2 ... ), each comprising data corresponding to an address in the memory unit (7).
37. A method according to claim 36, comprising receiving from the memory unit (7) a first sub-set (91a) of a first payload data set (9a), and a first sub-set (101a) of the control data set, which first control data sub-set (101a) comprises a first edit portion comprising data corresponding to a location in the memory unit (7) of a second sub-set (92a) of the first payload data set (9a), and a second edit portion comprising data corresponding to a location in the memory unit (7) of a third sub-set (93a) of the first payload data set (9a).
38. A method according to any of the claims 36-37, comprising locating a plurality of identification data groups within a payload data set (9a), and providing in each of a plurality of the edit portions (IX, FFl, FF2 ... ) of the control data sub-sets (101a) data corresponding to the location of an identification data group.
39. A method according to claim 38, wherein each identification data group corresponds to a full image data frame.
40. An apparatus, comprising a control unit configured to accept media data from a data source and a plurality of streaming requests from a plurality of user devices, a memory unit configured to store the media data under control of the control unit, and a streaming unit configured to stream portions of the media data corresponding to the streaming requests to the plurality of user devices under the control of the control unit.
41. The apparatus of claim 40, wherein the streaming unit is further configured to stream the portions of the media data using control data.
42. The apparatus of any of the claims 40-41, further comprising a network interface configured to send the portions of the media data over a network to the plurality of user devices.
43. The apparatus of any of the claims 40-42, further comprising a preprocessor configured to accept the media data and determine the control data prior to storage of the media data in the memory unit.
44. The apparatus of any of the claims 40-43, further comprising a user device request unit configured to receive the plurality of streaming requests from the plurality of user devices and transmit the plurality of streaming requests to the streaming unit.
45. The apparatus of any of the claims 40-44, further comprising an encryption device configured to encrypt the portions of the media data using an encryption key corresponding to each user device of the plurality of user devices.
46. The apparatus of claims 45, further comprising an encryption key storage unit configured to store a plurality of encryption keys.
47. The apparatus of claims 46, wherein the encryption key storage unit is operably coupled to the control unit.
48. The apparatus of any of the claims 40-47, wherein the plurality of user devices are audio/video devices.
49. The apparatus of claims 48, wherein the audio/video devices are at least one of set-top boxes, personal video recorders, or personal computers.
50. The apparatus of any of the claims 40-49, wherein at least one of the media data is an audio/visual data file and one of the plurality of streaming requests is a request for the streaming of the audio/visual data file.
51. A method, comprising accepting media data from a data source and a plurality of streaming requests from a plurality of user devices, generating control data from the media data, storing the media data, and streaming at least a portion of the media data to the plurality of user devices under control of the control data.
52. The method of claim 51, wherein generation of the control data from the media data comprises reviewing the media data to locate identification data groups within the media data; and adding data representing locations of the identification data groups to the control data.
53. The method of claim 52, wherein the identification data groups are complete image data.
54. The method of any of the claims 51-53, further comprising receiving a streaming request from a user device of the plurality of user devices, and streaming a portion of the media data to the user device of the plurality of user devices.
55. The method of any of the claims 51-53, further comprising encrypting the portions of the media data using an encryption key corresponding to each user device of the plurality of user devices.
56. The method of claim 55, further comprising storing the plurality of encryption keys in an encryption key store.
57. The method of any of the claims 51-56, wherein one of the plurality of streaming requests is a request for the streaming of an audio/visual data file.
EP07802919A 2006-09-08 2007-08-27 A method and an apparatus for data streaming Withdrawn EP2064884A2 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US84293906P 2006-09-08 2006-09-08
SE0601864 2006-09-08
GB0700311A GB2441575A (en) 2006-09-08 2007-01-09 Video server using FPGA streamers with control GPU and memory wherein video data segments are chained with play, FF and rewind pointers
PCT/EP2007/058881 WO2008028835A2 (en) 2006-09-08 2007-08-27 A method and an apparatus for data streaming

Publications (1)

Publication Number Publication Date
EP2064884A2 true EP2064884A2 (en) 2009-06-03

Family

ID=56290898

Family Applications (1)

Application Number Title Priority Date Filing Date
EP07802919A Withdrawn EP2064884A2 (en) 2006-09-08 2007-08-27 A method and an apparatus for data streaming

Country Status (3)

Country Link
EP (1) EP2064884A2 (en)
GB (1) GB2441575A (en)
WO (1) WO2008028835A2 (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8233478B2 (en) 2008-09-12 2012-07-31 Edgeware Ab Method and an apparatus for data storage and communications
US8782746B2 (en) 2008-10-17 2014-07-15 Comcast Cable Communications, Llc System and method for supporting multiple identities for a secure identity device
WO2010049440A1 (en) 2008-10-29 2010-05-06 Edgeware Ab A method and an apparatus for data recording and streaming

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5535008A (en) * 1993-03-16 1996-07-09 Victor Company Of Japan, Ltd. Method for jump-reproducing video data of moving picture coded with high efficiency
TW252248B (en) * 1994-08-23 1995-07-21 Ibm A semiconductor memory based server for providing multimedia information on demand over wide area networks
US6057832A (en) * 1997-12-02 2000-05-02 V Soft Ltd. Method and apparatus for video-on-demand with fast play capability
US20030095783A1 (en) * 2001-11-21 2003-05-22 Broadbus Technologies, Inc. Methods and apparatus for generating multiple network streams from a large scale memory buffer

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See references of WO2008028835A2 *

Also Published As

Publication number Publication date
GB0700311D0 (en) 2007-02-14
WO2008028835A3 (en) 2008-08-21
GB2441575A (en) 2008-03-12
WO2008028835A2 (en) 2008-03-13

Similar Documents

Publication Publication Date Title
KR101082642B1 (en) Apparatus system and method for adaptive-rate shifting of streaming content
US8826345B2 (en) Method and an apparatus for data streaming
EP1699240B1 (en) Video data processing method and video data processing device
EP3096526B1 (en) Communication apparatus, communication data generation method, and communication data processing method
US7035278B2 (en) Method and apparatus for forming and utilizing a slotted MPEG transport stream
US20100115121A1 (en) Method and an apparatus for data recording and streaming
US11284135B2 (en) Communication apparatus, communication data generation method, and communication data processing method
EP3096524B1 (en) Communication apparatus, communication data generation method, and communication data processing method
US20080068993A1 (en) Method and an apparatus for data streaming
WO2008028836A2 (en) A method and an apparatus for data streaming
CN105900437B (en) Communication apparatus, communication data generating method, and communication data processing method
EP2064884A2 (en) A method and an apparatus for data streaming
WO2008028834A2 (en) A method and an apparatus for data streaming
US20100121970A1 (en) method and an apparatus for streaming data containing trick play streaming modes
US20080062869A1 (en) Method and an apparatus for data streaming

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20090408

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC MT NL PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA HR MK RS

DAX Request for extension of the european patent (deleted)
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN

18D Application deemed to be withdrawn

Effective date: 20170301