WO2000072599A1 - Serveur de moyens de communication a compression evolutive et multidimensionnelle de donnees - Google Patents

Serveur de moyens de communication a compression evolutive et multidimensionnelle de donnees Download PDF

Info

Publication number
WO2000072599A1
WO2000072599A1 PCT/CA2000/000131 CA0000131W WO0072599A1 WO 2000072599 A1 WO2000072599 A1 WO 2000072599A1 CA 0000131 W CA0000131 W CA 0000131W WO 0072599 A1 WO0072599 A1 WO 0072599A1
Authority
WO
WIPO (PCT)
Prior art keywords
client
compression
data
video
accomplished
Prior art date
Application number
PCT/CA2000/000131
Other languages
English (en)
Inventor
Joe Toth
James Schellenberg
David Graves
Original Assignee
Edge Networks Corporation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CA 2272590 external-priority patent/CA2272590A1/fr
Priority claimed from CA 2277373 external-priority patent/CA2277373A1/fr
Application filed by Edge Networks Corporation filed Critical Edge Networks Corporation
Priority to AU26528/00A priority Critical patent/AU2652800A/en
Publication of WO2000072599A1 publication Critical patent/WO2000072599A1/fr

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21Server components or server architectures
    • H04N21/222Secondary servers, e.g. proxy server, cable television Head-end
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60Network streaming of media packets
    • H04L65/61Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio
    • H04L65/612Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio for unicast
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60Network streaming of media packets
    • H04L65/75Media network packet handling
    • H04L65/756Media network packet handling adapting media to device capabilities
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60Network streaming of media packets
    • H04L65/75Media network packet handling
    • H04L65/765Media network packet handling intermediate
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/80Responding to QoS
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/04Protocols for data compression, e.g. ROHC
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/186Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
    • H04N19/36Scalability techniques involving formatting the layers as a function of picture distortion after decoding, e.g. signal-to-noise [SNR] scalability
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/62Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding by frequency transforming in three dimensions
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/63Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using sub-band based transform, e.g. wavelets
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234327Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by decomposing into layers, e.g. base layer and one or more enhancement layers
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/238Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/258Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data
    • H04N21/25808Management of client data
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/266Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
    • H04N21/2662Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/4508Management of client data or end-user data
    • H04N21/4516Management of client data or end-user data involving client characteristics, e.g. Set-Top-Box type, software version or amount of memory available
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/454Content or additional data filtering, e.g. blocking advertisements
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/462Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
    • H04N21/4621Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/61Network physical structure; Signal processing
    • H04N21/6106Network physical structure; Signal processing specially adapted to the downstream path of the transmission network
    • H04N21/6125Network physical structure; Signal processing specially adapted to the downstream path of the transmission network involving transmission via Internet
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/637Control signals issued by the client directed to the server or network components
    • H04N21/6377Control signals issued by the client directed to the server or network components directed to server
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/647Control signaling between network components and server or clients; Network processes for video distribution between server and clients, e.g. controlling the quality of the video stream, by dropping packets, protecting content from unauthorised alteration within the network, monitoring of network load, bridging between two different networks, e.g. between IP and wireless
    • H04N21/64784Data processing by the network
    • H04N21/64792Controlling the complexity of the content stream, e.g. by dropping packets
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/65Transmission of management data between client and server
    • H04N21/658Transmission by the client directed to the server

Definitions

  • the present invention relates to a system and method for providing scalable data compression in a caching server and more particularly to a system and method for optimizing and managing such compression for streaming media edge servers.
  • the Internet today is defined as the interconnection of Internet protocol (IP) - based networks.
  • IP Internet protocol
  • the Internet protocol stack diagram is represented in terms of the ISO - 7-layer model.
  • Various equipment types and products may be associated with the layer functionality that they service.
  • the Internet may be viewed as a single integrated network in which various access types are interconnected to various backbone types through edge servers and edge equipment (also called remote access servers or network access servers). There are approximately twenty or more different variations of access paths that can be used to connect the backbone services to a customer (interchangeably referred to as a client). There are six basic access types of connections, namely wireless terrestrial, wireless, satellite, copper, coaxial cable and fiber. In the future, additional access types may be created.
  • the Internet has been a resounding success. Ironically, it is this very success and more specifically, the success of the graphical World Wide Web (the web) that may be its undoing.
  • the number of web subscribers, content providers, and requests by those subscribers for content grows exponentially faster than the capability of the network to meet the demand. The majority of current data transfers involve text and graphics. However, the future of the Internet appears to be evolving towards the transfer of full motion video and audio.
  • Streaming media is different from the typical transfer of multimedia data. Normally, hyperlinks point to multimedia files that are downloaded in their entirety to a user's local disk before being viewed or played.
  • streaming media allows users to watch live video and audio as the file is downloading.
  • Streaming media often requires a continual transfer of large volumes of data. If many people request the same data at the same time, it will lead to bandwidth restrictions or bottlenecks. Some of the reasons for these bandwidth restrictions or bottlenecks are highlighted in the following description. Since the Internet is IP based, all packets must be evaluated by routers to determine the destination delivery paths, creating traffic congestion, particularly with the increased demand in real-time media, such as video and audio.
  • Oracle Corporation has proposed a solution to the above problem.
  • the multimedia data repository is placed closer to the consumer of the multimedia.
  • servers are deployed at the edge of the web and multimedia data is replicated on these edge servers where the user connection terminates at the POP.
  • Hyperlinks on the web pages become pointers to streaming media servers that are physically closest to the consumer.
  • the philosophy behind this implementation is that the POP is the logical termination of the user's access point, and thus packets flowing into or out of the POP are only limited by the access speed of the user's connection. Any data packets that flow behind or through the ISP back channel, for example, router, are affected by bottlenecks.
  • the media repository at the POP and behind the router, the user is insulated from traffic conditions that exist on the Internet at any given time. It is envisioned that content providers and web publishers use a combination of mirroring or caching techniques to replicate data to the edge servers.
  • a disadvantage of the above scheme is that in the mirroring scheme it requires the content providers and web publishers themselves to stage, propagate, and update the multimedia data to be replicated.
  • a dialogue box would inform the user with the approximate time the media would be available and might suggest that they visit other sites in the interim. In general, both situations are unacceptable to most users since most users require instant access to the requested data.
  • bandwidth negotiation This has been referred to as "bandwidth negotiation". This process is not dynamic, so if a user's actual throughput changes due to congestion or packet loss, the server cannot adjust. Another difficulty is the increased labor required for coding and then managing the media clip for different bandwidths.
  • the Real Networks solution to these problems in its most recent incarnation is to provide an encoding framework for combining multiple data streams, each at different bit rates into a single file.
  • a sophisticated client server mechanism is provided for detecting changes in bandwidth and translating those changes into combinations of different streams.
  • image compression methods may be classified as those methods, which reproduce the original data exactly, that is, “lossless compression” and those, which trade a tolerable divergence from the original data for greater compression, that is, “lossy compression”.
  • lossless methods have a problem that they are unable to achieve a compression of much more than 70%. Therefore, where higher compression ratios are needed, lossy techniques have been developed. In general, the amount by which the original media source is reduced is referred to as the compression ratio.
  • Compression technologies have evolved over time to adapt to the various user requirements. Historically, compression technology focused on telephony, where sound wave compression algorithms were developed and optimized.
  • ID one-dimensional
  • 2D two-dimensional
  • the motion compensated predictive coding scheme processes the video data in groups of frames in order to achieve relatively high levels of compression without allowing the performance of the system to be degraded by excessive error propagation.
  • image frames are classified into one of three types: the intraframe (I-Frame), the predicted frame (P -Frame) and the bi-directional frame (B-Frame).
  • I-Frame intraframe
  • P -Frame predicted frame
  • B-Frame bi-directional frame
  • a 2D DCT is applied to small regions such as blocks of 8 x 8 pixels to encode each of the I-Frames.
  • the resulting data stream is quantized and encoded using a variable length code such as amplitude run length Huffman code to produce the compressed output signal.
  • this quantization technique still focuses on compressing single frames or images which may not be the most effective means of compression for current multimedia requirements.
  • MPEG suffers from 8 x 8 blocking artifacts known as tiling.
  • these second-generation compression approaches as described above have reduced the media of data requirements for video by as much as 100:1.
  • wavelet algorithms are implemented with efficient significance map coding such as EZW and line detection with gradient vectors depending on the application's final reconstructed resolution.
  • the wavelet algorithms operate on the entire image and have efficient implementation due to finite impulse response (FIR) filter realizations.
  • FIR finite impulse response
  • All wavelet algorithms decompose an image into coarser, smooth approximations with low pass digital filtering (convolution) on the image.
  • the wavelet algorithms generate detailed approximations (error signals) with high pass digital filtering or convolution on the image.
  • This decomposition process can be continued as far down the pyramid as a designer requires where each step in the pyramid has a sample rate reduction of two.
  • This technique is also known as spatial sample rate decimation or down sampling of the image where the resolution is one half in the next sub-band of the pyramid.
  • NQ vector quantization
  • the technique as described in this application may be applied to video data signals.
  • the method comprises the steps of selecting a sequence of image frames in a video stream, applying a three dimensional transform to the selected sequence to produce a transformed output, and then encoding the transformed output to produce a compressed stream output.
  • MMIP multicast media over Internet protocol
  • a method for managing scalable compression of multicast or unicast media over an Internet protocol, between a media server and a client comprising the steps of:
  • Figure 1 is a schematic diagram of an Internet architecture
  • Figure 2 is a schematic system diagram of an edge or gateway server located at an ISP according to an embodiment of the invention
  • Figure 3 is a schematic system diagram of a client according to an embodiment of the mvention
  • Figure 4 is a schematic functional block diagram of a caching bandwidth manager of figure 2;
  • Figure 5 is a schematic functional block diagram of a client player of figure 3;
  • Figure 6 is a schematic flow diagram of the bandwidth manager operation
  • Figure 7 is a schematic flow diagram of a bandwidth optimizer according to an embodiment of the present invention.
  • Figure 8 is a schematic flow diagram of a quantizer according to an embodiment of the present invention.
  • Figure 9 is a schematic diagram of a real-time frame by frame quantizer for a 2D wavelet case
  • Figure 10 is a schematic diagram of a real-time frame by frame quantizer for a
  • the architecture comprises a backbone network 22 which is defined as the interconnection equipment concerned with connecting local web sites to local POPs and an access network 24 that is defined as the interconnection between the local POPs and the consumers.
  • the Web sites 26 host both digital and analog content from various Content providers 28, which are in-turn connected via a global and national Internet infrastructure 30 to the local access Internet infrastructure 32.
  • the consumer or viewer 34 connects to the national Internet infrastructure 30 i.e. at the POP, by one or more access links 33.
  • Cache 36 sites are provided between the global and national Internet infrastructure 30 and the local access Internet infrastructure 32.
  • the cache sites 36 are normally the demarcation between the backbone network 22 and the access network 24.
  • Backbone web sites 26 typically do not consider the needs of various types of access 33 employed by clients 34 and various qualities of access links in their consideration of web content. It is normally the responsibility of the web content provider 28 to customizing the web site content for different access links.
  • the present invention leverages off the existing architecture, but implements a content compression architecture that uses technology in the data link level to application level of the ISO model to optimize the access from the local POP to the customer 34.
  • FIG 2 a system block diagram of local access server 37 is shown, while in figure 3, a client or consumer 34-system block diagram according to an embodiment of the present invention is shown.
  • the local access link 32 is a 56k modem.
  • MMIP internet protocol
  • the manager 100 functions as a network edge server/gateway and is compliant with the IETF working groups protocol recommendations for streaming media.
  • the manager 100 includes an m-dimension wavelet or spectral video translator module 102, an audio translator module 104, an access protocol optimization module 106, a cache database manager module 108, and a server management module 110.
  • the function performed by the manager 100 is to stream media over IP to a client player, shown in the schematic system diagram of figure 3 and the corresponding functional diagram of figure 5, by implementing a controlled compression or extended client filtering.
  • controlled compression the manager 100 receives compressed video from standard COTS video servers and then translates the video stream using an m-dimensional wavelet or spectral codec.
  • the manager 100 is also capable of receiving m-dimensional compressed video streams over IP from a originating media server in accordance with an embodiment of the present invention.
  • the caching bandwidth manager 100 receives streamed MPEG data compliant with RTP and sends translated streamed media data in wavelet format with header compression that is optimized for the access link 33. In addition the manager receives end user access configuration data in the form of client video/audio capabilities, access link capabilities, and media requests. The caching bandwidth manager utilizes the configuration data to send MPEG server requests to indicate when a user requests media from the WWW. The caching bandwidth manager 100 performs the functions of an edge server or traditional gateway for converting protocols between the backbone network and the access network.
  • the m-dimensional wavelet video translator module 102 receives streamed MPEG video data 120 from an MPEG media server (not shown) that is preferably compliant with RFC2250.
  • the MPEG data is decompressed back into the luminance and chromanence frame data values. This uncompressed frame data is then re-compressed using controlled compression according to an embodiment of the present invention, as described below.
  • the wavelet video translator module 102 is enabled by a video translation control signal 126 generated by the edge server when the access configuration data indicates that there is an MPEG media request being made by the end user.
  • the wavelet stream is stored in the media database 124 as N level deep pyramidal multiresolution sub-bands coded with the embedded zerotrees of wavelet coefficients (EZW) compression algorithm.
  • the highly correlated lowest resolution significance map or sub-band in the tree is processed with an algorithm such as the Discrete Cosine Transform (DCT), and then entropy coded with Huffman coding or arithmetic coding.
  • DCT Discrete Cosine Transform
  • Huffman coding or arithmetic coding.
  • the EZW algorithm is used to code all the other subbands or children of the lowest resolution subband that is coded by the DCT. This technique will result in efficient compression of the principle components of the video stream by a method, which closely approximates the optimal Karhunen-Loeve transformation.
  • the Audio Translator module 104 receives streamed MPEG audio data from the Media Server that is compliant with RFC2250.
  • the compressed audio stream may be uncompressed back to sample data values and this uncompressed stream efficiently re-compressed with timestamps using the audio wavelet codec.
  • the audio translator module 104 sends compressed audio data to the media database for efficient streaming to the client player over the access link.
  • An efficient application of the EZW transformation algorithm is in providing progressive video over various access link bandwidths.
  • the Access Protocol Optimization module 106 uses access optimization protocols, such as the controlled compression and extended client filtering to read Compressed Media Data from the Media Database and Stream Media Data in the form of time synchronized media wavelet coefficients to the Client Player based on the available bandwidth and the MMIP Client Player configuration.
  • the PPP (RFC 1661) and the Serial Line Internet Protocol (SLIP) shall be supported with 10:1 IPv4 header compression compliant to RFC 1144 and RFC2508/RFC2509 respectively.
  • RFC2507 for non serial links for header compression in mobile IP, etc will result in approximately 15:1 IPv6 compression of the header information.
  • the UDP/IPv4 headers consume 11.2 kbits/s and UDP/IPv6 headers consume 19.2 kbits/s when uncompressed.
  • the overhead can be reduced to approximately 1.7 KBPS.
  • up to 2:1 lossless compression can be achieved for the packet payload if the web data is not already in an encrypted or compressed format.
  • An algorithm is used here to detect if the compression of the web data results in data expansion. Data expansion is typical when a compression algorithm is applied to encrypted data.
  • the Access Protocol Optimization module supports RTP according to RFC1889 and RTSP according to RFC2326 to stream media to the Client Player.
  • the Access Protocol Optimization module 106 implements algorithms to efficiently perform Controlled Compression and Extended Client Filtering in media streaming to the MMIP Client Player over the access link bandwidth.
  • the algorithms implemented include the following functions:
  • M Stream only M of a total of N multiresolution subbands starting from the lowest resolution, where M ⁇ N.
  • the algorithm for which M subbands to stream to the Client Player is based on access infrastructure and bandwidth available (i.e.: twisted pair, wireless, satellite) to maintain the best subjective image quality according to the human visual systems logarithmic sensitivity to light intensity and sensitivity to abrupt spatial changes.
  • M Selectively stream only an area of each of the M subbands in 3).
  • the algorithm will determine the shape of the area to stream of the M subbands and vary it from all the lowest subband coefficients to a small geometrical area in the center of the highest resolution subband.
  • the algorithm to control the rate of change of the geometrical area from low to high resolution subbands in the center of the image area is optimized based on the access infrastructure, bandwidth available, and client capability.
  • Congestion algorithms optimized for the access infrastructure such as graceful degradation of video while maintaining the audio quality and audio degradation only after the frame rate is zero (RFC2001/RFC2581 identifies TCP congestion control algorithms).
  • Congestion algorithms optimized for the access infrastructure such as graceful degradation of video while maintaining the audio quality and audio degradation only after the frame rate is zero (RFC2001/RFC2581 identifies TCP congestion control algorithms).
  • a flow chart showing the operation of the caching bandwidth manager 100 is shown generally by numeral 200.
  • the caching bandwidth manager runs two loops, a non-real-time loop 204 and a real-time loop 206.
  • the bandwidth manager determines the bandwidth characteristics of the client 208. The characteristics are used to generate a quantization mask 210 to be applied by the real time loop 206 on a frame by frame basis, or group of frames in 3D cases, for creating a bitstream to the client.
  • the bandwidth manager receives the transformed pixel data and performs coding thereon using the quantization mask 210.
  • the bitstream generated by this process is then sent 212 to the client.
  • the client performs the inverse operation 214 on the bitstream to generate an appropriate display.
  • an edge server receives a client request.
  • the client request is buffered and the server determines the access bandwidth statistics of the client.
  • the client buffer status is determined, as to whether the client buffer is approaching an underflow condition or approaching an overflow condition. If the client buffer status indicates that the buffer is approaching an overflow condition, the bandwidth optimizer reduces video quality by changing the co-efficient array quantization. Spatial decimation and temporal decimation follow this.
  • the bandwidth optimizer may perform colorspace reduction and decrease subjective quality based on the human visual system sensitivity.
  • the values are updated in the array for the N x N quantization mask and the coded bit rate decreased to the client.
  • the next step is then to update the new N x N quantization in the real time quantization process. If the client buffer state is determined to be approaching an underflow condition the bandwidth optimizer improves video quality by controlling the coefficient array quantization. Next spatial interpolation and temporal interpolation are performed as above. Also color space expansion is performed followed by an increase in subject subjective quality using a human visual system sensitivity profile. This is similar to the sensitivity profile described above for the overflow condition however in this case the optimizer moves in the direction to increase the quality of the image. Next the N x N quantization mask is updated to increase the coded bit rate to the client. It may be noted, that the quantization mask is not restricted to N x N but my be a N x M or N x M x Z.
  • the outputs are used to update the new N x N quantization mask in the real-time quantization process.
  • a flow chart showing the real time quantization process is shown generally by numeral 500. It may be shown that the quantization process is repeated for each frame, or group of frames in 3D cases, transformed.
  • the quantization process begins by inputting a transformed pixel data of size N x N. It is next determined whether a new real-time quantization mask is available. If so the system updates the quantizer co-efficient mask from the bandwidth optimizer as described with reference to Figure 7. This quantization mask is then applied to the N x N array of transformed frame data as shown schematically in Figure 9.
  • the quantizer performs entropy (for example arithmetic) coding on the quantized data, the resulting bit stream is then sent to the client.
  • entropy for example arithmetic
  • a schematic representation of a real-time frame by frame quantization with independent co-efficient quantization is shown for a 2D wavelet case.
  • a quantization mask of size N x N 604 is created and then applied to the data 602 on a frame by frame basis.
  • the purpose of the quantization mask is for real-time selective wavelet sub-band quantization to reject certain sub- bands or portions of sub-bands for NxN, N x M or N x M x P cases.
  • an AND is performed on a corresponding element in the transformed coefficient array 602 with the quantization mask 604.
  • the independent value for each quantization mask array element will truncate the corresponding transformed coefficient value in bit positions in the quantization mask array elements that are zero.
  • a real time group of frames quantization with independent co-efficient quantization for a 3D spectral case is shown generally by numeral 700.
  • a 3D quantization mask block is created and applied to an entire sequence of frames.
  • the cache database manager module 108 receives Access Configuration Data from the user.
  • the Cache Database Manager sends an MPEG Server request to the MPEG Media Server to enable Streamed MPEG Data to be sent over IP on the WWW.
  • the Cache Database Manager is also be capable of sending Media Requests to enable other Compressed Media Data comprised of audio, video, or data for transmission over IP on the WWW based on the Access Configuration Data received from the user.
  • an embodiment of the present invention provides a method and system for optimizing the quality of media displayed at a client that is connected to an IP network.
  • the invention has only been described in detail relating to media transmissions over an IP network, it may be extended to other forms of data transmission.
  • cable television companies typically transmit over 6MHz channels. Using today's MPEG compression technology, they are generally only able to accommodate from four to six television stations per channel. Using the controlled compression described herein, different types of television shows can be compressed with different compression rates. Therefore, in the cable television industry each 6 MHz channel statistically will be able to accommodate much more television shows.
  • the subject of the present invention may be applied to other industries, such as broadcast television, jukeboxes, personal electronic devices and the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Databases & Information Systems (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Graphics (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

L'invention concerne un procédé et un système de gestion de compression évolutive de multimédia de diffusion sur un protocole de l'Internet, entre un serveur de multimédia et un client. Ce procédé comprend les étapes consistant à déterminer les statistiques de largeur de bande d'accès client, à déterminer un état de mémoire tampon de données du client, à produire un masque de quantification en réponse à l'état de la mémoire tampon, à appliquer ce masque à un groupe de données de trame transformées pour chaque trame d'une séquence ou groupe de trames dans le cas tridimensionnel, à produire des données quantifiées, et à exécuter un codage arithmétique sur les données quantifiées afin de produire un flux binaire destiné au client.
PCT/CA2000/000131 1999-05-21 2000-02-15 Serveur de moyens de communication a compression evolutive et multidimensionnelle de donnees WO2000072599A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU26528/00A AU2652800A (en) 1999-05-21 2000-02-15 Media server with multi-dimensional scalable data compression

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
CA 2272590 CA2272590A1 (fr) 1999-05-21 1999-05-21 Systeme et methode de transmission en continu selon le protocole internet
CA2,272,590 1999-05-21
CA2,277,373 1999-07-09
CA 2277373 CA2277373A1 (fr) 1999-05-21 1999-07-09 Compression multidimensionnelle des donnees
CA 2280662 CA2280662A1 (fr) 1999-05-21 1999-09-02 Serveur de media a compression evolutive multidimensionnelle des donnees
CA2,280,662 1999-09-02

Publications (1)

Publication Number Publication Date
WO2000072599A1 true WO2000072599A1 (fr) 2000-11-30

Family

ID=27170970

Family Applications (3)

Application Number Title Priority Date Filing Date
PCT/CA2000/000132 WO2000072602A1 (fr) 1999-05-21 2000-02-15 Compression multidimensionnelle de donnees
PCT/CA2000/000133 WO2000072517A1 (fr) 1999-05-21 2000-02-15 Utilisation en continu de supports via un systeme en protocole internet et systeme a cet effet
PCT/CA2000/000131 WO2000072599A1 (fr) 1999-05-21 2000-02-15 Serveur de moyens de communication a compression evolutive et multidimensionnelle de donnees

Family Applications Before (2)

Application Number Title Priority Date Filing Date
PCT/CA2000/000132 WO2000072602A1 (fr) 1999-05-21 2000-02-15 Compression multidimensionnelle de donnees
PCT/CA2000/000133 WO2000072517A1 (fr) 1999-05-21 2000-02-15 Utilisation en continu de supports via un systeme en protocole internet et systeme a cet effet

Country Status (3)

Country Link
AU (3) AU2652800A (fr)
CA (1) CA2280662A1 (fr)
WO (3) WO2000072602A1 (fr)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8139487B2 (en) 2007-02-28 2012-03-20 Microsoft Corporation Strategies for selecting a format for data transmission based on measured bandwidth
US8897359B2 (en) 2008-06-03 2014-11-25 Microsoft Corporation Adaptive quantization for enhancement layer video coding
DE102013220901A1 (de) 2013-10-15 2015-04-16 Continental Automotive Gmbh Verfahren zur Übertragung von digitalen Audio- und/oder Videodaten
US9967561B2 (en) 2006-05-05 2018-05-08 Microsoft Technology Licensing, Llc Flexible quantization

Families Citing this family (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020078253A1 (en) * 2000-12-20 2002-06-20 Gyorgy Szondy Translation of digital contents based on receiving device capabilities
US7242324B2 (en) 2000-12-22 2007-07-10 Sony Corporation Distributed on-demand media transcoding system and method
US6407680B1 (en) 2000-12-22 2002-06-18 Generic Media, Inc. Distributed on-demand media transcoding system and method
US20030028643A1 (en) * 2001-03-13 2003-02-06 Dilithium Networks, Inc. Method and apparatus for transcoding video and speech signals
US7054335B2 (en) * 2001-05-04 2006-05-30 Hewlett-Packard Development Company, L.P. Method and system for midstream transcoding of secure scalable packets in response to downstream requirements
WO2003001748A1 (fr) * 2001-06-21 2003-01-03 Ziplabs Pte Ltd. Procede et appareil de compression et de decompression de donnees
ITTO20010813A1 (it) * 2001-08-13 2003-02-13 Telecom Italia Lab Spa Procedimento per il trasferimento di messaggi tramite udp, relativo sistema e prodotto informatico.
US7480703B2 (en) 2001-11-09 2009-01-20 Sony Corporation System, method, and computer program product for remotely determining the configuration of a multi-media content user based on response of the user
US7356575B1 (en) 2001-11-09 2008-04-08 Sony Corporation System, method, and computer program product for remotely determining the configuration of a multi-media content user
US7730165B2 (en) 2001-11-09 2010-06-01 Sony Corporation System, method, and computer program product for remotely determining the configuration of a multi-media content user
JP2003152544A (ja) * 2001-11-12 2003-05-23 Sony Corp データ通信システム、データ送信装置、データ受信装置、および方法、並びにコンピュータ・プログラム
US7284069B2 (en) 2002-01-11 2007-10-16 Xerox Corporation Method for document viewing
US7200615B2 (en) 2003-10-16 2007-04-03 Xerox Corporation Viewing tabular data on small handheld displays and mobile phones
CN100458747C (zh) * 2003-10-31 2009-02-04 索尼株式会社 用于确定计算机的连接速度的方法
EP1738571A1 (fr) * 2004-04-20 2007-01-03 France Télécom Systeme de messagerie multimedia et poste telephonique comportant ce systeme
US7539341B2 (en) 2004-07-29 2009-05-26 Xerox Corporation Systems and methods for processing image data prior to compression
US7620892B2 (en) 2004-07-29 2009-11-17 Xerox Corporation Server based image processing for client display of documents
US7721204B2 (en) 2004-07-29 2010-05-18 Xerox Corporation Client dependent image processing for browser-based image document viewer for handheld client devices
US8812978B2 (en) 2005-12-22 2014-08-19 Xerox Corporation System and method for dynamic zoom to view documents on small displays
CN101662454A (zh) * 2008-08-29 2010-03-03 阿里巴巴集团控股有限公司 互联网中图像处理的方法、装置和系统
US9225762B2 (en) 2011-11-17 2015-12-29 Google Technology Holdings LLC Method and apparatus for network based adaptive streaming
US9747010B2 (en) 2014-01-16 2017-08-29 Xerox Corporation Electronic content visual comparison apparatus and method
US9521176B2 (en) 2014-05-21 2016-12-13 Sony Corporation System, method, and computer program product for media publishing request processing
CN112751886B (zh) * 2019-10-29 2023-05-26 贵州白山云科技股份有限公司 一种转码方法、系统、传输设备及存储介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0515101A2 (fr) * 1991-05-23 1992-11-25 AT&T Corp. Commande de mémoire tampon pour canal à débit binaire variable
EP0739140A2 (fr) * 1995-04-18 1996-10-23 Sun Microsystems, Inc. Codeur pour un système de distribution vidéo point-à point à échelle variable
WO1997030551A1 (fr) * 1996-02-14 1997-08-21 Olivr Corporation Ltd. Procede et systemes de transmission asynchrone et progressive de donnees multimedia
WO1998037698A1 (fr) * 1996-12-17 1998-08-27 Adaptive Media Technologies Procede et dispositif constituant un systeme de remise de supports conformables
US5822524A (en) * 1995-07-21 1998-10-13 Infovalue Computing, Inc. System for just-in-time retrieval of multimedia files over computer networks by transmitting data packets at transmission rate determined by frame size
WO1999000984A1 (fr) * 1997-06-26 1999-01-07 Citrix Systems, Inc. Systeme d'acheminement adaptatif de donnes video/audio sur un reseau
US5881176A (en) * 1994-09-21 1999-03-09 Ricoh Corporation Compression and decompression with wavelet style and binary style including quantization by device-dependent parser

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3071205B2 (ja) * 1990-01-23 2000-07-31 オリンパス光学工業株式会社 画像データの符号化装置および符号化方法
TW318315B (fr) * 1993-05-03 1997-10-21 At & T Corp
GB2278973B (en) * 1993-06-11 1997-10-29 Quantel Ltd Video image processing systems
JP3749752B2 (ja) * 1995-03-24 2006-03-01 アイティーティー・マニュファクチャリング・エンタープライジズ・インコーポレーテッド ブロック適応型差分パルスコード変調システム
US5706216A (en) * 1995-07-28 1998-01-06 Reisch; Michael L. System for data compression of an image using a JPEG compression circuit modified for filtering in the frequency domain
US5996022A (en) * 1996-06-03 1999-11-30 Webtv Networks, Inc. Transcoding data in a proxy computer prior to transmitting the audio data to a client
US5918013A (en) * 1996-06-03 1999-06-29 Webtv Networks, Inc. Method of transcoding documents in a network environment using a proxy server
US20010039615A1 (en) * 1997-04-15 2001-11-08 At &T Corp. Methods and apparatus for providing a broker application server
AU1285099A (en) * 1997-11-07 1999-05-31 Pipe Dream, Inc Method for compressing and decompressing motion video

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0515101A2 (fr) * 1991-05-23 1992-11-25 AT&T Corp. Commande de mémoire tampon pour canal à débit binaire variable
US5881176A (en) * 1994-09-21 1999-03-09 Ricoh Corporation Compression and decompression with wavelet style and binary style including quantization by device-dependent parser
EP0739140A2 (fr) * 1995-04-18 1996-10-23 Sun Microsystems, Inc. Codeur pour un système de distribution vidéo point-à point à échelle variable
US5822524A (en) * 1995-07-21 1998-10-13 Infovalue Computing, Inc. System for just-in-time retrieval of multimedia files over computer networks by transmitting data packets at transmission rate determined by frame size
WO1997030551A1 (fr) * 1996-02-14 1997-08-21 Olivr Corporation Ltd. Procede et systemes de transmission asynchrone et progressive de donnees multimedia
WO1998037698A1 (fr) * 1996-12-17 1998-08-27 Adaptive Media Technologies Procede et dispositif constituant un systeme de remise de supports conformables
WO1999000984A1 (fr) * 1997-06-26 1999-01-07 Citrix Systems, Inc. Systeme d'acheminement adaptatif de donnes video/audio sur un reseau

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
ANONYMOUS: "Method to Deliver Scalable Video across a Distributed Computer System", IBM TECHNICAL DISCLOSURE BULLETIN,US,IBM CORP. NEW YORK, vol. 5, no. 37, 1 May 1994 (1994-05-01), pages 251 - 256, XP002079392, ISSN: 0018-8689 *
CHADDHA N ET AL: "SCALABLE COMPRESSION BASED ON TREE STRUCTURED VECTOR QUANTIZATION OF PERCEPTUALLY WEIGHTED BLOCK, LAPPED, AND WAVELET TRANSFORMS", PROCEEDINGS OF THE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING. (ICIP),US,LOS ALAMITOS, IEEE COMP. SOC. PRESS, 1995, pages 89 - 92, XP000623109, ISBN: 0-7803-3122-2 *
CHAN Y -L ET AL: "VARIABLE TEMPORAL-LENGTH 3-D DISCRETE COSINE TRANSFORM CODING", IEEE TRANSACTIONS ON IMAGE PROCESSING,US,IEEE INC. NEW YORK, vol. 6, no. 5, 1 May 1997 (1997-05-01), pages 758 - 763, XP000656000, ISSN: 1057-7149 *

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9967561B2 (en) 2006-05-05 2018-05-08 Microsoft Technology Licensing, Llc Flexible quantization
US8139487B2 (en) 2007-02-28 2012-03-20 Microsoft Corporation Strategies for selecting a format for data transmission based on measured bandwidth
US8897359B2 (en) 2008-06-03 2014-11-25 Microsoft Corporation Adaptive quantization for enhancement layer video coding
US9185418B2 (en) 2008-06-03 2015-11-10 Microsoft Technology Licensing, Llc Adaptive quantization for enhancement layer video coding
US9571840B2 (en) 2008-06-03 2017-02-14 Microsoft Technology Licensing, Llc Adaptive quantization for enhancement layer video coding
US10306227B2 (en) 2008-06-03 2019-05-28 Microsoft Technology Licensing, Llc Adaptive quantization for enhancement layer video coding
DE102013220901A1 (de) 2013-10-15 2015-04-16 Continental Automotive Gmbh Verfahren zur Übertragung von digitalen Audio- und/oder Videodaten

Also Published As

Publication number Publication date
AU2652900A (en) 2000-12-12
AU2652800A (en) 2000-12-12
CA2280662A1 (fr) 2000-11-21
WO2000072602A1 (fr) 2000-11-30
WO2000072517A1 (fr) 2000-11-30
AU2653000A (en) 2000-12-12

Similar Documents

Publication Publication Date Title
WO2000072599A1 (fr) Serveur de moyens de communication a compression evolutive et multidimensionnelle de donnees
US8929436B2 (en) Method and apparatus for video coding, predecoding, and video decoding for video streaming service, and image filtering method
US6091777A (en) Continuously adaptive digital video compression system and method for a web streamer
US6337881B1 (en) Multimedia compression system with adaptive block sizes
US7082164B2 (en) Multimedia compression system with additive temporal layers
US7477688B1 (en) Methods for efficient bandwidth scaling of compressed video data
US6788740B1 (en) System and method for encoding and decoding enhancement layer data using base layer quantization data
US6680976B1 (en) Robust, reliable compression and packetization scheme for transmitting video
Sun et al. An overview of scalable video streaming
JP2006087125A (ja) ビデオフレームシーケンスを符号化する方法、符号化ビットストリーム、画像又は画像シーケンスを復号する方法、データの送信又は受信を含む使用、データを送信する方法、符号化及び/又は復号装置、コンピュータプログラム、システム、並びにコンピュータ読み取り可能な記憶媒体
US8571027B2 (en) System and method for multi-rate video delivery using multicast stream
KR100952185B1 (ko) 순방향 에러 정정 코드를 이용하여 비디오의 드리프트 없는 단편적인 다중 설명 채널 코딩을 위한 시스템 및 방법
JP2004512785A (ja) Dctに基づくスケーラブルなビデオ圧縮
EP0892557A1 (fr) Compression d'images
WO2003005699A2 (fr) Transformation d'echelle: cadre de codage et de multi-diffusion video pour services internet multimedia sur reseau sans fil
AU8759198A (en) A robust, reliable compression and packetization scheme for transmitting video
Girod et al. Scalable codec architectures for internet video-on-demand
Johanson Scalable video conferencing using subband transform coding and layered multicast transmission
Abd Al-azeez et al. Optimal quality ultra high video streaming based H. 265
Ortiz et al. Interactive transmission of JPEG2000 images using web proxy caching
Mrak et al. Scalable video coding in network applications
Pereira et al. Multiple description coding for internet video streaming
Tham et al. Layered coding for a scalable video delivery system
Johanson et al. Layered encoding and transmission of video in heterogeneous environments
Song et al. PVH-3DDCT: an algorithm for layered video coding and transmission

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AL AM AT AU AZ BA BB BG BR BY CA CH CN CR CU CZ DE DK DM EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG US UZ VN YU ZA ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): GH GM KE LS MW SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

122 Ep: pct application non-entry in european phase
DPE2 Request for preliminary examination filed before expiration of 19th month from priority date (pct application filed from 20040101)