EP1520422B1 - Decodeur fgs base sur une qualite estimee au decodeur - Google Patents

Decodeur fgs base sur une qualite estimee au decodeur Download PDF

Info

Publication number
EP1520422B1
EP1520422B1 EP03730430A EP03730430A EP1520422B1 EP 1520422 B1 EP1520422 B1 EP 1520422B1 EP 03730430 A EP03730430 A EP 03730430A EP 03730430 A EP03730430 A EP 03730430A EP 1520422 B1 EP1520422 B1 EP 1520422B1
Authority
EP
European Patent Office
Prior art keywords
bit rate
quality
data
stream
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime
Application number
EP03730430A
Other languages
German (de)
English (en)
Other versions
EP1520422A1 (fr
Inventor
Mihaela Van Der Schaar
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Koninklijke Philips NV
Original Assignee
Koninklijke Philips Electronics NV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Koninklijke Philips Electronics NV filed Critical Koninklijke Philips Electronics NV
Publication of EP1520422A1 publication Critical patent/EP1520422A1/fr
Application granted granted Critical
Publication of EP1520422B1 publication Critical patent/EP1520422B1/fr
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/462—Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
    • H04N21/4621—Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/20—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding
    • H04N19/29—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding involving scalability at the object level, e.g. video object layer [VOL]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
    • H04N19/34—Scalability techniques involving progressive bit-plane based encoding of the enhancement layer, e.g. fine granular scalability [FGS]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234327—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by decomposing into layers, e.g. base layer and one or more enhancement layers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/258—Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data
    • H04N21/25808—Management of client data
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/266—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
    • H04N21/2662—Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440227—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by decomposing into layers, e.g. base layer and one or more enhancement layers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/637—Control signals issued by the client directed to the server or network components
    • H04N21/6373—Control signals issued by the client directed to the server or network components for rate control, e.g. request to the server to modify its transmission rate
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/637—Control signals issued by the client directed to the server or network components
    • H04N21/6377—Control signals issued by the client directed to the server or network components directed to server
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/65—Transmission of management data between client and server
    • H04N21/658—Transmission by the client directed to the server

Definitions

  • the present invention relates to the implementation of a fine granular scalability (FGS) decoder.
  • FGS fine granular scalability
  • IP Internet Protocol
  • IP Internet Protocol
  • QoS Quality-of-Service
  • BL Base Layer
  • EL Enhancement Layer
  • the BL part of the scalable video stream represents, in general, the minimum amount of data needed for decoding that stream.
  • the EL part of the stream represents additional information, and therefore enhances the video signal representation when decoded by the receiver.
  • Fine Granular Scalability is a new video compression framework that has been recently adopted by the MPEG-4 standard for streaming applications. FGS is capable of supporting a wide range of bandwidth-variation scenarios that characterize IP-based networks, in general, and the Internet, in particular. Images coded with this type of scalability can be decoded progressively. That is, the decoder can start decoding and displaying the image after receiving a very small amount of data. As the decoder receives more data, the quality of the decoded image is progressively enhanced until the complete information is received, decoded, and displayed.
  • progressive image coding is one of the modes supported in JPEG and the still-image, texture coding tool in MPEG-4 video.
  • WO00/35187 describes how the encoder can decide not to include data for certain bit-planes in the encoded stream if it is detected that all pixels in these bit-planes have a predetermined value.
  • the EL compresses the SNR and temporal residual data using a progressive (embedded) codec.
  • the FGS residual signal is compressed bit-plane by bit- plane, starting with the most significant bit-plane and ending with the least significant (see Fig. 5).
  • Fig. 5 shows an example of such FGS enhancement-layer bit-planes, with progressive (Bit-plane by bit-plane) coding from the most significant bit (MSB) to the least significant bit (LSB), across the entire frame.
  • a decoder can stop decoding after a given number of bit-planes is received and decoded. For example, given a data rate constraint R and a frame rate fr, the decoder can be stopped after decoding R/fr bits per frame.
  • the decoder can be stopped after decoding R/fr bits per frame.
  • discarding bit-planes in order to decrease the decoder complexity can lead to undesired visual quality.
  • decoding is stopped when the prescribed bit rate is exactly fulfilled (i.e., exactly R/fr bits are transmitted and decoded per frame), that number of bits will most likely correspond to a non-integer number of bit-planes. For example, the beginning of the image may have three bit-planes, while the end of the image has two bit-planes, leading to inconsistent quality within the same image or sequence.
  • the invention includes a method and a decoder for decoding a stream of data.
  • a relationship is identified between quality of a decoded image and a bit rate for decoding the image.
  • a desired quality of images to be decoded from the stream of data is selected.
  • a bit rate is determined corresponding to the desired quality using the relationship.
  • a minimum number of bit-planes corresponding to the determined bit rate are decoded from the stream of data.
  • the effect on image quality obtained by discarding a fixed number of bit-planes depends on the sequence characteristics. For instance, sequences with high detail and low-motion are coded by FGS with many bits, and thus for example, discarding two bit-planes might have a considerably higher effect on image quality than discarding, for example, four bit-planes for an image with high motion and low frequency detail. Consequently, this might result in high quality fluctuations as the content changes.
  • the decoder before deciding to discard bit-planes to reduce decoding complexity, the decoder takes into account the effect of discarding the bit-planes on image quality.
  • FIGS. 1-4 reduce the complexity of low-power, low-capability decoder devices for decoding FGS streams.
  • the FGS decoder has an inherent complexity scalability feature, because a lower complexity is associated with decoding a smaller number of bit-planes. However, if a fixed number of entire bit-planes are always discarded, the resulting decoded image quality is unpredictable.
  • a method and a decoder are described that estimate the visual quality (e.g., in peak signal to noise ratio, PSNR) associated with decoding a certain number of bit-planes by using a base-layer quality-tag (e.g. PSNR). This allows reduction in the complexity of scalability while fulfilling a desired quality constraint.
  • PSNR peak signal to noise ratio
  • the quality of the FGS decoded images is determined at the decoder side (where the original image is not present), and subsequently, the decoder uses the computed quality to determine how many bit-planes can be discarded to reduce the complexity without lowering image quality below a desired quality level.
  • FIG. 1 is a flow chart diagram of an exemplary embodiment of the invention.
  • FIG. 1 shows a method and apparatus for decoding a stream of data.
  • a relationship is identified between quality of a decoded image and a bit rate for decoding the image.
  • An example of such a relationship is described further below in the section entitled, "Relationship Between Quality and Bit-Rate.”
  • the value of the base-layer quality, QBL, at the base-layer bit-rate, RBL is included in the data stream, then at step 104, the value of QBL can be received as side information (in the user data fields) by the decoder.
  • QBL can be computed based on the decoding information available to the decoder, i.e., based on quantization steps, motion-vectors, macroblock prediction types, or combinations of the three.
  • QBL can be expressed either in terms of PSNR or using other subjective metric (i.e., objective metrics that have been correlated with subjective evaluations of image quality and tabulated). Other metrics may include, but are not limited to blockiness, sharpness, flicker noise.
  • the other parameters that are used in the image quality calculation are determined. For example, for the relationship described further below, the additional parameters include the base layer bit-rate and the slope of the quality versus bit-rate curve.
  • a selection is made of a desired quality of images to be decoded from the stream of data, a processing speed to be used, or a decoding time.
  • An exemplary selection process is described below with reference to FIG. 3.
  • a bit rate corresponding to the desired quality, speed or decoding time is determined using the relationship.
  • decoding of a minimum number of bit-planes is begun, corresponding to the determined bit rate from the stream of data.
  • the minimum number of bit-planes may be a real number corresponding to the calculated bit rate or the smallest integer greater than or equal to the calculated bit rate.
  • the number of bit-planes decoded is dynamically adjusted within a single video sequence, to increase during a sequence of frames when there is higher detail, and decrease during a sequence of frames when there is lower detail. For example, preferably, each time a new scene change is detected (e.g., based on motion-vectors or the like), the base layer quality QBL is evaluated, and a determination is made whether a change in decoding bit rate is appropriate. It is contemplated that, when there is higher motion and lower frequency detail, more bit-planes can be discarded in these embodiments than would have been discarded if a fixed number of bit-planes is used for the entire video sequence.
  • FIG. 2 is a flow chart diagram showing optional dynamic adjustment of the bit-rate in the exemplary embodiments.
  • the steps of FIG. 2 may be automatically executed at appropriate times, for example upon detection of a scene change.
  • a scene change is detected. Detection may be based on motion vectors, for example.
  • a change in the relationship between image quality and bit-rate is identified. For example, a change in QBL may be identified.
  • step 204 the amount of detail in the images is considered. If the amount of detail in the image increases, steps 206 and 208 are executed. If the amount of detail in the image decreases, steps 207 and 209 are executed. These steps dynamically adjust the number of bit-planes decoded to correspond to the desired quality and the changed relationship.
  • the number of bit-planes decoded is increased (and the number discarded decreased).
  • a relatively high bit rate is used.
  • step 207 for an image sequence having low detail the number of bit-planes decoded is decreased (and the number discarded increased).
  • step 209 a relatively low bit rate is used.
  • FIG. 2 shows a dynamic adjustment technique
  • the number of bit-planes to be discarded may be determined once for each image sequence, and remain constant throughout that sequence.
  • the value of QBL may be based on the first frame of the sequence, or on some type of average. For an average value of QBL, motion vectors between subsequent frames can be used to determine the amount of motion within the sequence.
  • FIG. 3 is a flow chart diagram of an exemplary optional user interaction with the decoding process and decoder.
  • the user requests a menu of image quality or decoding complexity options.
  • the user may make such a request by selecting the menu from a higher level menu of functions and controls in the particular device containing the decoder.
  • the device containing the decoder displays at least two values of image quality and a respective measure of a resource required to decode the image for each of the at least two values of image quality. Any number of quality-resource measure pairs may be displayed, but preferably, only possible combinations should be displayed. For example, an image quality value corresponding to a decoding time that exceeds the capacity of the battery of the device containing the decoder should not be displayed as an option.
  • the displayed resource measure may be the time to decode the stream of data with the respective image quality, a fraction of battery charge used to decode the stream of data at the respective image quality, or a bit rate used to decode the stream of data at the respective image quality.
  • the user selects one of the pairs, which results in a selection of a decoding bit-rate (even if the bit-rate is not the resource measure displayed to the user in step 302).
  • the decoder may optionally transmit the determined bit rate to a source of the stream of data, requesting that the source of the stream of data only transmit future streams of data at the determined bit rate. If this option is included, then at step 310, the source of the data stream can set its transmitted bit-rate to the bit-rate requested by the decoder.
  • the decoder can omit step 308, and discard any bit-planes that are not desired to be decoded by the user.
  • the user or the terminal can either select a desired quality or a desired processing power.
  • the user can manually adjust the fixed quality setting QFIX, for example, to lengthen the remaining battery time or to improve image quality.
  • the device can easily trade image quality for battery time based on user request or a predetermined quality agreement or it can maximize battery time by reducing image quality as low as QBL (i.e., by lowering the bit rate to RBL).
  • such a feature can exist at the mobile terminal where, based on the selected quality, the device can inform the user how much battery time is still left for streaming applications using FGS.
  • the battery level is monitored, which corresponds to an approximate amount of time that the mobile device can be operated without recharging the battery.
  • An average time to decode a given amount of data in the mobile device can be estimated. (This average could be estimated as a composite of both base layer and enhancement layer decoding time.)
  • R By specifying a transmission rate R, the user controls the amount of data decoded, and thus the amount of time that the mobile device must spend decoding a given stream.
  • the mobile unit can compare the amount of operating time left on the battery with the predicted amount of time needed to decode a stream at either rate, and present the information to the user, either in minutes, or as a percentage of the total battery capacity. The user can then select the rate based on this information.
  • FIG. 4 is a block diagram of an exemplary embodiment featuring a decoder 400.
  • Decoder 400 has a base layer decoder 410 and an enhancement layer decoder 450.
  • BL decoder 410 includes a demultiplexer that receives the BL bitstream and provides the encoded discrete cosine transform (DCT) coefficient data to a BL variable length decoder (VLD) 414 and provides BL motion vectors (MVs) to a motion compensation block 422.
  • the BL VLD 414 provides decoded data to inverse quantization block 416, which outputs the DCT coefficient data to inverse DCT block 418.
  • Inverse DCT block 418 performs the inverse DCT operation and provides video frames to adder 424.
  • the motion compensation block 422 receives the BL MVs and BL frame data from the BL frame memory 420, and performs motion compensation on the BL frames in memory 420 to provides additional frames to the adder 424.
  • the decoded BL data stream is output from adder 424 to the BL frame memory 420 and the EL decoding section 450.
  • the EL decoder 450 has an FGS bit-plane VLD 452 that receives the EL bitstream and provides DCT coefficient data to the inverse DCT block 454.
  • the inverse DCT block 454 performs the inverse DCT operation and outputs the EL frame data that are added to the BL data in adder 456 to provide an enhanced video stream.
  • Decoder 400 includes an additional block 460 that determines how many bit-planes to decode.
  • Block 460 receives information from the inverse quantization block 416, the BL motion vectors, and the EL bitstream, and provides a number of bit-planes to be decoded to the FGS bit-plane VLD. The operation of block 460 may be as shown in FIGS. 1-3 and described above.
  • FIG. 6 Various rate-distortion curves are shown in FIG. 6 for various sequences 602, 604, and 606 coded within the same range of bit rates.
  • the leftmost point corresponds to the base layer bit rate, and the corresponding quality QBL. From Fig. 6, two observations can be made:
  • An exemplary function is set forth below that can be used to determine a bit-rate that will provide a desired image quality, given a base layer image quality and base layer image rate. This function takes advantage of empirical observations to provide a quality computation that is quickly performed with a small number of arithmetic operations.
  • the value of the term (R-RBL)/c is less than ten.
  • the value R1 corresponding to the pre-set QFIX can be easily computed since the quality of an FGS stream Q at rate R (as described above) is a linear function.
  • the complexity saving associated with this strategy can also be a-priori computed on average for a large set of sequences.
  • a quality versus average complexity table can be stored and used at the (proprietary) decoder. For any given Quality QFIX, base layer bit-rate RBL, encoding frame-rate and spatial-resolution, this table provides an average complexity factor.
  • complexity may be measured by the average CPU time (in milliseconds per frame) required to decode the sequence.
  • CPU time in milliseconds per frame
  • the complexity does not depend on the sequence characteristics but rather only depends on the bit rate R.
  • FIG. 7 shows an example with three different image sequences. The complexity is shown by the number of milliseconds per frame to decode the sequences, as a function of the bit rate. The complexity is highly correlated with the bit rate, and the relationship is nearly linear. There is little variation among the sequences.
  • the exemplary embodiments are especially useful for products such as wireless terminals, such as mobile-phones, personal computers, television set-top boxes, storage devices, digital video cameras, (in-home) transcoding gateways and the like.
  • the exemplary embodiments may be used in services, such as video-conferencing, real-time video decoding/streaming, storage applications and the like. This list is not exclusive, and other applications for the exemplary decoding technique are also contemplated.
  • the present invention may be embodied in the form of computer-implemented processes and apparatus for practicing those processes.
  • the present invention may also be embodied in the form of computer program code embodied in tangible media, such as random access memory (RAM), floppy diskettes, read only memories (ROMs), CD-ROMs, DVD-ROMs, hard drives, high density (e:g., "ZIP TM " or "JAZZ TM ”) removable disks, or any other computer-readable storage medium, wherein, when the computer program code is loaded into and executed by a computer, the computer becomes an apparatus for practicing the invention.
  • the present invention may also be embodied in the form of computer program code, for example, whether stored in a storage medium, loaded into and/or executed by a computer, or transmitted over some transmission medium, such as over the electrical wiring or cabling, through fiber optics, or via electromagnetic radiation, wherein, when the computer program code is loaded into and executed by a computer, the computer becomes an apparatus for practicing the invention.
  • computer program code segments configure the processor to create specific logic circuits.
  • the decoder may be implemented in specific logic circuits.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Computer Graphics (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Error Detection And Correction (AREA)
  • Image Processing (AREA)

Claims (27)

  1. Procédé de décodage d'un flux de données codé qui comprend une pluralité de plans binaires, le procédé comprenant les étapes exécutées sur machine, de :
    - sélection (110, 306) d'une qualité souhaitée d'images devant être décodées à partir du flux codé de données ;
    - détermination (112) d'un débit binaire pour un décodage correspondant à la qualité souhaitée ;
    - décodage (116) uniquement d'un nombre minimum des plans binaires à partir du flux codé, le nombre minimum correspondant au débit binaire déterminé,
    caractérisé par
    - l'identification (202, 204, 206, 207) d'un contenu de flux dépendant de la relation entre la qualité d'une image décodée et un débit binaire pour décoder l'image, le débit binaire du décodage étant déterminé (112) d'une façon qui correspond à la qualité souhaitée en utilisant la relation.
  2. Procédé selon la revendication 1, caractérisé en ce que la relation est définie par l'équation : Q(R)=QBL + (R-RBL)/c, où Q(R) est la qualité de l'image à un débit binaire R, QBL est une mesure de qualité d'une couche de base du flux de données , RBL est un débit binaire correspondant à la transmission de la couche de base, et c est une constante.
  3. Procédé selon la revendication 1, comprenant en outre les étapes de :
    (e) identification d'un changement de la relation ; et ajustement dynamique du nombre de plans binaires décodés de façon qu'il corresponde à la qualité souhaitée et à la relation modifiée.
  4. Procédé selon la revendication 3, caractérisé en ce que l'étape (f) comprend l'utilisation d'un débit binaire relativement faible pour une séquence d'images ayant un faible niveau de détail.
  5. Procédé selon la revendication 3, caractérisé en ce que les étapes (e) et (f) sont effectuées lorsqu'un changement de scène est détecté.
  6. Procédé selon la revendication 1, comprenant l'affichage d'au moins deux valeurs de qualité d'image et d'une mesure correspondante d'une ressource exigée pour décoder l'image pour chacune des au moins deux valeurs de qualité d'image, avant l'étape (b).
  7. Procédé selon la revendication 1, comprenant en outre la transmission du débit binaire déterminé à une source du flux de données, caractérisé en ce qu'il est uniquement demandé à la source du flux de données de transmettre des flux de données futurs au débit binaire déterminé.
  8. Support lisible par ordinateur ayant un code de programme informatique codé sur celui-ci, caractérisé en ce que, lorsque le code de programme informatique est exécuté par un processeur, le processeur exécute un procédé pour décoder un flux de données codé qui comprend une pluralité de plans binaires, le procédé comprenant les étapes de :
    - sélection (110, 306) d'une qualité souhaitée d'images devant être décodées à partir du flux codé de données ;
    - détermination (112) d'un débit binaire pour un décodage correspondant à la qualité souhaitée ; et
    - décodage (116) uniquement d'un nombre minimum des plans binaires provenant du flux codé, le nombre minimum correspondant au débit binaire déterminé,
    caractérisé par
    - l'identification (202, 204, 206, 207) d'une relation dépendant du contenu du flux entre la qualité d'une image décodée et un débit binaire pour décoder l'image, le débit binaire de décodage étant déterminé en utilisant ladite relation.
  9. Support lisible par ordinateur selon la revendication 8, caractérisé en ce que la relation est définie par l'équation : Q(R) = QBL + (R-RBL)/c, où Q(R) est la qualité de l'image à un débit binaire R, QBL est une mesure de qualité d'une couche de base du flux de données , RBL est un débit binaire correspondant à la transmission de la couche de base, et c est une constante.
  10. Support lisible par ordinateur selon la revendication 9, caractérisé en ce que c est approximativement égal à RBL.
  11. Support lisible par ordinateur selon la revendication 9, comprenant en outre la réception d'une valeur de QBL d'une source du flux de données.
  12. Support lisible par ordinateur selon la revendication 9, comprenant en outre le calcul de QBL sur la base d'un ou plusieurs du groupe constitué des pas de quantification, des vecteurs de mouvement et des types de prédiction de macroblocs.
  13. Support lisible par ordinateur selon la revendication 8, caractérisé en ce que le nombre minimum de plans binaires est l'un du groupe constitué d'un nombre réel correspondant au débit binaire déterminé à l'étape (c) et d'un plus petit entier supérieur ou égal au débit binaire déterminé à l'étape (c).
  14. Support lisible par ordinateur selon la revendication 8, comprenant en outre les étapes de :
    (e) identification d'un changement de la relation ; et (f) ajustement dynamique du nombre de plans binaires décodés de façon qu'il corresponde à la qualité souhaitée et à la relation changée.
  15. Support lisible par ordinateur selon la revendication 14, caractérisé en ce que l'étape (f) comprend l'utilisation d'un débit binaire relativement élevé pour une séquence d'images ayant un niveau de détail élevé et un faible mouvement.
  16. Support lisible par ordinateur selon la revendication 14, caractérisé en ce que l'étape (f) comprend l'utilisation d'un débit binaire relativement faible pour une séquence d'images ayant un niveau de détail relativement faible.
  17. Support lisible par ordinateur selon la revendication 14, caractérisé en ce que les étapes (e) et (f) sont effectuées lorsqu'un changement de scène est détecté.
  18. Support lisible par ordinateur selon la revendication 8, caractérisé en ce que le procédé comprend l'affichage d'au moins deux valeurs de la qualité d'image et d'une mesure correspondante d'une ressource exigée pour décoder l'image pour chacune des au moins deux valeurs de qualité d'image, avant l'étape (b).
  19. Support lisible par ordinateur selon la revendication 18, caractérisé en ce que la mesure de la ressource est l'un du groupe constitué d'un temps mis pour décoder le flux de données avec la qualité d'image correspondante, d'une fraction de la charge de la batterie utilisée pour décoder le flux de données avec la qualité d'image correspondante, et d'un débit binaire utilisé pour décoder le flux de données avec la qualité d'image correspondante.
  20. Support lisible par ordinateur selon la revendication 8, caractérisé en ce que le procédé comprend en outre la transmission du débit binaire déterminé à une source du flux de données, et en ce qu'il est demandé à la source du flux de données de ne transmettre des flux de données futurs qu'au débit binaire déterminé.
  21. Décodeur pour décoder un flux codé de données qui comprend une pluralité de plans binaires, le décodeur comprenant :
    - des moyens pour recevoir une sélection ayant une qualité souhaitée des images devant être décodées à partir du flux codé de données ;
    - des moyens pour déterminer un débit binaire pour décoder une image correspondant à la qualité souhaitée ;
    - des moyens pour commander un nombre de plans binaires à décoder afin de ne décoder qu'un nombre minimum des plans binaires à partir du flux codé pour former une ou plusieurs images, le nombre minimum correspondant au débit binaire souhaité, caractérisé en ce que les moyens pour déterminer le débit binaire du décodage sont conçus pour déterminer le débit binaire sur la base d'une qualité d'une image décodée, telle qu'elle est définie par une relation dépendant du contenu du flux entre la qualité de l'image décodée et un débit binaire pour l'image décodée.
  22. Décodeur selon la revendication 21, comprenant en outre : des moyens pour identifier un changement d'une relation entre le débit binaire et la qualité de l'image décodée ; et des moyens pour ajuster dynamiquement le nombre de plans binaires décodés afin qu'il corresponde à la qualité souhaitée et à la relation changée.
  23. Décodeur selon la revendication 22, caractérisé en ce que les moyens d'ajustement utilisent un débit binaire relativement faible pour une séquence d'images ayant un niveau de détail relativement faible.
  24. Décodeur selon la revendication 22, caractérisé en ce que les moyens d'identification de changement identifient un changement de la relation lorsqu'un changement de scène est détecté.
  25. Décodeur selon la revendication 21, caractérisé en ce que le décodeur est incorporé à un dispositif mobile qui affiche au moins deux valeurs de qualité d'image et une mesure correspondante d'une ressource exigée pour décoder l'image pour chacune des au moins deux valeurs de qualité d'image.
  26. Décodeur selon la revendication 25, caractérisé en ce que la mesure de la ressource est l'un du groupe constitué d'un temps pour décoder le flux de données avec la qualité d'image correspondante, d'une fraction de la charge de la batterie utilisée pour décoder le flux de données avec la qualité d'image correspondante, et d'un débit binaire utilisé pour décoder le flux de données avec la qualité d'image correspondante.
  27. Décodeur selon la revendication 21, caractérisé en ce que le décodeur est présent dans un dispositif mobile qui transmet le débit binaire déterminé à une source du flux de données, afin de demander que la source du flux de données ne transmette les flux de données futurs qu'au débit binaire déterminé.
EP03730430A 2002-06-27 2003-06-11 Decodeur fgs base sur une qualite estimee au decodeur Expired - Lifetime EP1520422B1 (fr)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US183760 2002-06-27
US10/183,760 US7136532B2 (en) 2002-06-27 2002-06-27 FGS decoder based on quality estimated at the decoder
PCT/IB2003/002566 WO2004004352A1 (fr) 2002-06-27 2003-06-11 Decodeur fgs base sur une qualite estimee au decodeur

Publications (2)

Publication Number Publication Date
EP1520422A1 EP1520422A1 (fr) 2005-04-06
EP1520422B1 true EP1520422B1 (fr) 2006-05-10

Family

ID=29779193

Family Applications (1)

Application Number Title Priority Date Filing Date
EP03730430A Expired - Lifetime EP1520422B1 (fr) 2002-06-27 2003-06-11 Decodeur fgs base sur une qualite estimee au decodeur

Country Status (9)

Country Link
US (1) US7136532B2 (fr)
EP (1) EP1520422B1 (fr)
JP (1) JP2005531240A (fr)
KR (1) KR20050013619A (fr)
CN (1) CN1669325A (fr)
AT (1) ATE326122T1 (fr)
AU (1) AU2003241110A1 (fr)
DE (1) DE60305181T2 (fr)
WO (1) WO2004004352A1 (fr)

Families Citing this family (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003199019A (ja) * 2001-12-26 2003-07-11 Sony Corp 撮像装置および方法、記録媒体、並びにプログラム
JP3952459B2 (ja) * 2002-11-15 2007-08-01 株式会社リコー 画像処理装置、プログラム、記憶媒体及び画像処理方法
US7474792B2 (en) * 2002-12-02 2009-01-06 Ricoh Company, Ltd. Image processing apparatus for using distortion amount information
US7903734B2 (en) * 2003-04-24 2011-03-08 Canon Kabushiki Kaisha Moving image decoding apparatus, moving image decoding method, image decoding method, and image decoding apparatus
DE10326168B4 (de) * 2003-06-10 2010-01-28 Siemens Ag Verfahren und Funktionseinheit zur Optimierung der Darstellung progressiv kodierter Bilddaten
JP4618676B2 (ja) 2005-04-28 2011-01-26 株式会社リコー 構造化文書符号の転送方法、画像処理システム、サーバ装置、プログラム及び情報記録媒体
KR100891663B1 (ko) * 2005-10-05 2009-04-02 엘지전자 주식회사 비디오 신호 디코딩 및 인코딩 방법
KR20070096751A (ko) * 2006-03-24 2007-10-02 엘지전자 주식회사 영상 데이터를 코딩/디코딩하는 방법 및 장치
KR100891662B1 (ko) * 2005-10-05 2009-04-02 엘지전자 주식회사 비디오 신호 디코딩 및 인코딩 방법
KR100904440B1 (ko) * 2005-10-05 2009-06-26 엘지전자 주식회사 레지듀얼 데이터 스트림을 생성하는 방법과 장치 및 이미지블록을 복원하는 방법과 장치
KR20070038396A (ko) 2005-10-05 2007-04-10 엘지전자 주식회사 영상 신호의 인코딩 및 디코딩 방법
KR100802180B1 (ko) * 2005-12-08 2008-02-12 한국전자통신연구원 엠펙-4 비디오 신호의 비트율을 동적인 통신 용량 변화에따라 제어하는 방법
WO2007114641A1 (fr) * 2006-04-03 2007-10-11 Lg Electronics, Inc. Procédé et dispositif pour coder/décoder un signal vidéo à échelle modifiable
CA2657267C (fr) * 2006-07-13 2013-07-16 Qualcomm Incorporated Codage video presentant une echelonnabilite a granularite fine (fgs) faisant intervenir des fragments alignes par cycle
US8942292B2 (en) * 2006-10-13 2015-01-27 Qualcomm Incorporated Efficient significant coefficients coding in scalable video codecs
US8068608B2 (en) * 2007-12-17 2011-11-29 Broadcom Corporation Video processing system for scrambling video streams with dependent portions and methods for use therewith
US9078024B2 (en) * 2007-12-18 2015-07-07 Broadcom Corporation Video processing system with user customized graphics for use with layered video coding and methods for use therewith
US8130823B2 (en) * 2007-12-19 2012-03-06 Broadcom Corporation Channel adaptive video transmission system for use with layered video coding and methods for use therewith
US9210480B2 (en) * 2007-12-20 2015-12-08 Broadcom Corporation Video processing system with layered video coding and methods for use therewith
US8416848B2 (en) * 2007-12-21 2013-04-09 Broadcom Corporation Device adaptive video transmission system for use with layered video coding and methods for use therewith
US9143731B2 (en) * 2008-01-02 2015-09-22 Broadcom Corporation Mobile video device for use with layered video coding and methods for use therewith
US8594191B2 (en) * 2008-01-03 2013-11-26 Broadcom Corporation Video processing system and transcoder for use with layered video coding and methods for use therewith
US8144781B2 (en) * 2008-01-04 2012-03-27 Broadcom Corporation Video processing system for scrambling layered video streams and methods for use therewith
US8290285B2 (en) * 2008-06-23 2012-10-16 Mediatek Inc. Method and related apparatuses for decoding multimedia data
FR2954035B1 (fr) * 2009-12-11 2012-01-20 Thales Sa Procede d'estimation de la qualite video a une resolution quelconque
JP5916624B2 (ja) 2010-01-06 2016-05-11 ドルビー ラボラトリーズ ライセンシング コーポレイション マルチ−レイヤ化されたビデオシステムのための複雑度を適応的にするスケーラブル復号及びストリーミング
CN102668579A (zh) * 2010-10-05 2012-09-12 英特尔公司 用于动态调整视频质量的方法和装置
CN105191329B (zh) * 2013-03-06 2018-10-19 交互数字专利控股公司 用于视频流的功率感知自适应
US9872255B2 (en) * 2015-04-01 2018-01-16 Ebay Inc. Battery charge aware communications
US11546611B2 (en) 2018-08-01 2023-01-03 Samsung Display Co., Ltd. Rate control for fixed rate foveated display compression
US11871052B1 (en) * 2018-09-27 2024-01-09 Apple Inc. Multi-band rate control

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4330040B2 (ja) * 1995-06-29 2009-09-09 トムソン マルチメデイア ソシエテ アノニム 階層化圧縮ビデオ・データを符号化し復号化するためのシステム
US5748903A (en) 1995-07-21 1998-05-05 Intel Corporation Encoding images using decode rate control
US6115421A (en) * 1996-04-25 2000-09-05 Matsushita Electric Industrial Co., Ltd. Moving picture encoding apparatus and method
US6480541B1 (en) * 1996-11-27 2002-11-12 Realnetworks, Inc. Method and apparatus for providing scalable pre-compressed digital video with reduced quantization based artifacts
JPH10164581A (ja) * 1996-12-03 1998-06-19 Sony Corp 画像信号符号化方法及び装置、信号記録媒体
US6631403B1 (en) * 1998-05-11 2003-10-07 At&T Corp. Architecture and application programming interfaces for Java-enabled MPEG-4 (MPEG-J) systems
US6292512B1 (en) * 1998-07-06 2001-09-18 U.S. Philips Corporation Scalable video coding system
KR20010080666A (ko) 1998-12-04 2001-08-22 매클린토크 샤운 엘 변환계수의 비트플레인 코딩을 이용한 세밀한 세분성스칼라빌러티의 향상
US6882711B1 (en) * 1999-09-20 2005-04-19 Broadcom Corporation Packet based network exchange with rate synchronization
CA2395605C (fr) * 1999-12-22 2011-04-05 General Instrument Corporation Compression video pour environnements de multi-diffusion avec codage d'echelonnabilite spatiale et de diffusion simultanee
US6668090B1 (en) 2000-05-26 2003-12-23 Eastman Kodak Company Producing a compressed digital image organized into layers corresponding to increasing visual quality levels and providing rate-control of such compressed digital image
US6859446B1 (en) 2000-09-11 2005-02-22 Lucent Technologies Inc. Integrating power-controlled and rate-controlled transmissions on a same frequency carrier
US7266148B2 (en) * 2001-01-05 2007-09-04 Lg Electronics Inc. Video transcoding apparatus
US6925120B2 (en) 2001-09-24 2005-08-02 Mitsubishi Electric Research Labs, Inc. Transcoder for scalable multi-layer constant quality video bitstreams
US7391807B2 (en) * 2002-04-24 2008-06-24 Mitsubishi Electric Research Laboratories, Inc. Video transcoding of scalable multi-layer videos to single layer video

Also Published As

Publication number Publication date
KR20050013619A (ko) 2005-02-04
US7136532B2 (en) 2006-11-14
AU2003241110A1 (en) 2004-01-19
JP2005531240A (ja) 2005-10-13
DE60305181T2 (de) 2007-03-08
ATE326122T1 (de) 2006-06-15
WO2004004352A1 (fr) 2004-01-08
CN1669325A (zh) 2005-09-14
DE60305181D1 (de) 2006-06-14
EP1520422A1 (fr) 2005-04-06
US20040001635A1 (en) 2004-01-01

Similar Documents

Publication Publication Date Title
EP1520422B1 (fr) Decodeur fgs base sur une qualite estimee au decodeur
US7773672B2 (en) Scalable rate control system for a video encoder
US8442122B2 (en) Complexity scalable video transcoder and encoder
KR100942395B1 (ko) 다층 비디오 설계를 위한 레이트 제어
KR101643790B1 (ko) 적응적 비디오 스트리밍용의 가변 비트 레이트 및 동적 해상도를 이용한 다중 비트 레이트 비디오 인코딩
Seeling et al. Network performance evaluation using frame size and quality traces of single-layer and two-layer video: A tutorial
US9071841B2 (en) Video transcoding with dynamically modifiable spatial resolution
JP5180294B2 (ja) ビデオ符号化において、フレームの複雑さ、バッファレベル、およびイントラフレームの位置を利用するバッファベースのレート制御
US20070201554A1 (en) Video transcoding method and apparatus
US20100166060A1 (en) Video transcoder rate control
JP2006087125A (ja) ビデオフレームシーケンスを符号化する方法、符号化ビットストリーム、画像又は画像シーケンスを復号する方法、データの送信又は受信を含む使用、データを送信する方法、符号化及び／又は復号装置、コンピュータプログラム、システム、並びにコンピュータ読み取り可能な記憶媒体
US9325989B2 (en) Automatic temporal layer bit allocation
Lee et al. Design of a mobile video streaming system using adaptive spatial resolution control
Vetro et al. Rate‐reduction transcoding design for wireless video streaming
Zhang et al. Constant-quality constrained-rate allocation for FGS video coded bitstreams
US20070031050A1 (en) Method and apparatus for bit rate control for image encoding
Chen et al. MPEG-4 FGS coding performance improvement using adaptive inter-layer prediction
Rhee et al. Bitrate control using a heuristic spatial resolution adjustment for a real-time H. 264/AVC encoder
Cho et al. Smooth quality streaming with bit-plane labeling
JP4273386B2 (ja) 符号化装置および符号化方法、プログラム、並びに記録媒体
JPH11196423A (ja) 画像処理装置および方法、並びに提供媒体
Lee et al. Adaptive spatial resolution control scheme for mobile video applications
Ramkishor et al. Bandwidth adaptation for MPEG-4 video streaming over the internet
Awaghad et al. Efficient Design of Multimedia Transcoding for Clientside Application over Mobile Interface
Chen et al. Complexity-scalable MPEG-4 FGS streaming for UMA

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20050127

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL LT LV MK

17Q First examination report despatched

Effective date: 20050531

DAX Request for extension of the european patent (deleted)
GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PT RO SE SI SK TR

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT;WARNING: LAPSES OF ITALIAN PATENTS WITH EFFECTIVE DATE BEFORE 2007 MAY HAVE OCCURRED AT ANY TIME BEFORE 2007. THE CORRECT EFFECTIVE DATE MAY BE DIFFERENT FROM THE ONE RECORDED.

Effective date: 20060510

Ref country code: BE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060510

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060510

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060510

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060510

Ref country code: CH

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060510

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060510

Ref country code: LI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060510

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060510

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060510

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060510

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20060612

REF Corresponds to:

Ref document number: 60305181

Country of ref document: DE

Date of ref document: 20060614

Kind code of ref document: P

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20060630

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060810

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060810

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20060811

Year of fee payment: 4

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060821

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20061010

NLV1 Nl: lapsed or annulled due to failure to fulfill the requirements of art. 29p and 29m of the patents act
REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

ET Fr: translation filed
PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20070213

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20080101

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060811

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060510

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060810

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060510

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20060611

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20061111

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060510

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20080626

Year of fee payment: 6

REG Reference to a national code

Ref country code: GB

Ref legal event code: 732E

Free format text: REGISTERED BETWEEN 20090312 AND 20090318

REG Reference to a national code

Ref country code: FR

Ref legal event code: ST

Effective date: 20100226

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20090630

REG Reference to a national code

Ref country code: GB

Ref legal event code: 732E

Free format text: REGISTERED BETWEEN 20120524 AND 20120530

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20160608

Year of fee payment: 14

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20170611

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170611