EP1338004A1 - Method and device for the generation or decoding of a scalable data stream with provision for a bit-store, encoder and scalable encoder - Google Patents
Method and device for the generation or decoding of a scalable data stream with provision for a bit-store, encoder and scalable encoderInfo
- Publication number
- EP1338004A1 EP1338004A1 EP02718023A EP02718023A EP1338004A1 EP 1338004 A1 EP1338004 A1 EP 1338004A1 EP 02718023 A EP02718023 A EP 02718023A EP 02718023 A EP02718023 A EP 02718023A EP 1338004 A1 EP1338004 A1 EP 1338004A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- encoder
- output data
- block
- data
- bit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 20
- 230000005540 biological transmission Effects 0.000 claims abstract description 27
- 230000003111 delayed effect Effects 0.000 claims description 6
- 230000003139 buffering effect Effects 0.000 claims description 2
- 230000001934 delay Effects 0.000 claims 1
- 230000005236 sound signal Effects 0.000 description 13
- 230000008901 benefit Effects 0.000 description 7
- 230000008569 process Effects 0.000 description 5
- 238000005070 sampling Methods 0.000 description 5
- 230000011664 signaling Effects 0.000 description 4
- 230000002123 temporal effect Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000001052 transient effect Effects 0.000 description 3
- 230000007175 bidirectional communication Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000002349 favourable effect Effects 0.000 description 2
- 238000013139 quantization Methods 0.000 description 2
- 230000003595 spectral effect Effects 0.000 description 2
- 101000873502 Homo sapiens S-adenosylmethionine decarboxylase proenzyme Proteins 0.000 description 1
- 102100035914 S-adenosylmethionine decarboxylase proenzyme Human genes 0.000 description 1
- 230000006854 communication Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000000873 masking effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000011144 upstream manufacturing Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
- G10L19/16—Vocoder architecture
- G10L19/18—Vocoders using multiple modes
- G10L19/24—Variable rate codecs, e.g. for generating different qualities using a scalable representation such as hierarchical encoding or layered encoding
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
- G10L19/08—Determination or coding of the excitation function; Determination or coding of the long-term prediction parameters
- G10L19/12—Determination or coding of the excitation function; Determination or coding of the long-term prediction parameters the excitation function being a code excitation, e.g. in code excited linear prediction [CELP] vocoders
Definitions
- the present invention relates to scalable encoders and decoders, and more particularly to the generation of scalable data streams.
- Scalable encoders are shown in EP 0 846 375 B1.
- scalability is understood to mean the possibility of a subset of a bit stream that contains an encoded data signal, e.g. represents an audio signal or a video signal to be decoded into a usable signal. This property is particularly desirable when e.g. a data transmission channel does not provide the full bandwidth required to transmit a full bit stream.
- incomplete decoding on a decoder with lower complexity is possible.
- different discrete scalability layers are defined in practice.
- FIG. 1 An example of a scalable encoder as defined in subpart 4 (general audio) of part 3 (audio) of the MPEG-4 standard (ISO / IEC 14496-3: 1999 subpart 4) is shown in FIG. 1 .
- An audio signal s (t) to be coded is fed into the scalable encoder on the input side.
- the scalable encoder shown in Fig. 1 includes a first encoder 12, which is an MPEG-Celp encoder.
- the second encoder 14 is an AAC encoder that provides high quality audio coding and is defined in the MPEG-2 AAC (ISO / IEC 13818) standard.
- the Celp encoder 12 supplies a first scaling layer via an output line 16, while the AAC encoder 14 supplies a second scaling layer to a bit stream multiplexer (BitMux) 20 via a second output line 18.
- the LATM format is described in Section 6.5 of Part 3 (Audio) of the first addition to the MPEG-4 standard (ISO / IEC 14496-3: 1999 / AMD1: 2000).
- the scalable audio encoder also includes some other elements. First there is a delay stage 24 in the AAC branch and a delay stage 26 in the Celp branch. An optional delay can be set for each branch using both delay levels.
- the delay stage 26 of the Celp branch is followed by a downsampling stage 28 in order to adapt the sampling rate of the input signal s (t) to the sampling rate required by the Celp encoder.
- An inverse celp decoder 30 is connected downstream of the celp encoder 12, the celp-coded / decoded signal being fed to an upsampling stage 32.
- the sampled up signal is then fed to a further delay stage 34, which is referred to in the MPEG-4 standard as "core encoder or delay".
- the CoreDoderDelay 34 level has the following function. If the delay is set to zero, the first encoder 14 and the second encoder 16 process exactly the same samples of the audio input signal in a so-called superframe.
- a superframe can consist, for example, of three AAC fraraes, which together have a certain number of samples H. Represent x to number y of the audio signal.
- a CoreCoderDelay D is set as a time variable other than zero, the three blocks of AAC frames nevertheless represent the same sample values No. x to No. y.
- the eight blocks of CELP frames represent sample values No. x - Fs D to No. y - Fs D, where Fs is the sampling frequency of the input signal.
- a CoreCoderDelay 0 is assumed for the following explanations without restricting generality, so that the current time period of the input signal for the first encoder and the current time period for the second encoder are identical.
- the only requirement for a superframe is that the AAC block (s) and the CELP blocks (s) in a superframe represent the same number of samples, the samples themselves not necessarily being identical, but also around CoreCoderDelay can be shifted to each other.
- the Celp encoder processes a section of the input signal s (t) faster than the AAC encoder 14.
- the optional delay stage 24 is followed by a block decision stage 26, which may be used. a. determines whether short or long windows should be used to window the input signal s (t), whereby short windows should be selected for strongly transient signals, while long windows are preferred for less transient signals, since the relationship between the amount of user data and side information is better for them than with short windows.
- a fixed delay of z. B. performs 5/8 times a block. This is referred to in technology as the look-ahead function.
- the output signals of the MDCT blocks 36, 38 are then fed to a subtractor 40.
- the subsequent block 44 determines whether it is more favorable to feed the input signal per se to the AAC encoder 14. This is made possible by the bypass branch 42. However, if it is determined that the difference signal at the output of the subtractor 40 is e.g. is lower in energy than the signal output by the MDCT block 38, the difference signal is not taken, but the difference signal, in order to be encoded by the AAC encoder 14 in order to finally form the second scaling layer 18. This comparison can be carried out in bands, which is indicated by a frequency-selective switching device (FSS) 44.
- FSS frequency-selective switching device
- An essential feature of the MPEG-4 standard and also of other encoder standards is that the transmission of the compressed data signal should take place over a channel with a constant bit rate.
- All high-quality audio codecs work block-based, ie they process blocks of audio data (order of magnitude 480-1024 samples) to pieces of a compressed bit stream, which are also referred to as frames.
- the bitstream format must be constructed in such a way that a decoder without a priori information, where a frame begins, is able to recognize the beginning of a frame in order to delay the output of the deco with the least possible delay. audio signal data. Therefore, each header or destination data block of a frame begins with a particular synchronization word that can be searched for in a continuous bit stream.
- Other common components in the data stream in addition to the determination data block are the main data or "payload data" of the individual layers, in which the actual compressed audio data are contained.
- bit stream format shows a bit stream format with a fixed frame length, in this bit stream format the headers or determination data blocks are inserted equidistantly into the bit stream.
- the side information and main data associated with this header follow immediately behind.
- the length, i.e. Number of bits, for the main data is the same in every frame.
- Such a bit stream format is used for example in MPEG Layer 2 or MPEG-CELP.
- Fig. 5 shows another bit stream format with a fixed frame length and a back pointer or backward pointer.
- the header and page information are arranged equidistantly as in the format shown in FIG. 4.
- the start of the associated main data only occurs in exceptional cases immediately after a header. In most cases, the start is in one of the previous frames.
- the number of bits by which the start of the main data in the bit stream is shifted is transmitted by the side information variable back pointer.
- the end of this main data can be in this frame or in a previous frame.
- the length of the main data is no longer constant.
- This technique is called "bit savings bank” and increases the theoretical delay in the transmission chain.
- Such a bitstream format is used for example in MPEG Layer 3 (MP3).
- MP3 MPEG Layer 3
- the Bit savings bank technology is also described in the MPEG Layer 3 standard.
- the bit savings bank represents a buffer of bits that can be used to provide more bits for coding a block of temporal samples than are actually permitted by the constant output data rate.
- the technology of the bit savings bank takes into account the fact that some blocks of audio samples can be coded with fewer bits than specified by the constant transmission rate, so that these blocks fill the bit bank, while still other blocks of audio samples have psychoacoustic properties that are not so allow large compression, so that the available bits would not be sufficient for these blocks for low-interference or interference-free coding.
- the required surplus bits are taken from the bit savings bank, so that the bit savings bank is emptied in such blocks.
- Such an audio signal could also be transmitted in a format with a variable frame length.
- bit stream format “variable frame length”
- the fixed order of the bit stream elements header, page information and main data is maintained as with the "fixed frame length”. Since the length of the main data is not constant, the bit savings bank technique can also be used here, but no back pointers as in FIG. 5 are required.
- An example of a bit stream format, as shown in FIG. 6, is the transport format ADTS (Audio Data Transport Stream), as defined in the MPEG 2 AAC standard.
- encoders are not all scalable encoders, but only comprise a single audio encoder.
- the portions of the first scaling layer in the bit stream are given priority over the second and further scaling layers during transmission, which ensures the transmission of the first scaling layer in the event of capacity bottlenecks in the transmission network, while the second scaling layer may be lost in whole or in part.
- the MPEG 4 version 2 standardizes the LATM transport format, which can also transmit scalable data streams.
- FIG. 2a is a schematic representation of the samples of the input signal s (t).
- the input signal can be divided into different successive sections 0, 1, 2, 3, each section having a certain fixed number of temporal samples.
- the AAC encoder 14 (FIG. 1) usually processes an entire section 0, 1, 2 or 3 to provide an encoded data signal for that section.
- the celp encoder 12 (FIG. 1) usually processes a smaller amount of temporal samples per coding step.
- FIG. 2b it is shown in FIG. 2b that the celp encoder, or generally speaking the first encoder or coder 1, has a block length which is one quarter of the block length of the second encoder. It should be noted that this division is completely arbitrary.
- the block length of the first encoder could also be half as long, but could also be one eleventh of the block length of the second encoder.
- the first encoder will generate four blocks (11, 12, 13, 14) from the section of the input signal, from which the second encoder supplies a block of data.
- a conventional LATM bitstream format is shown in FIG. 2c.
- a superframe can have different ratios of the number of AAC frames to the number of CELP frames, as is tabulated in MPEG 4. So a superframe z. B. an AAC block and 1 to 12 CELP blocks, 3 AAC blocks and 8 CELP blocks but also z. B. have more AAC blocks than CELP blocks depending on the configuration.
- a LATM frame that has a LATM determination data block comprises one or more superframes.
- the generation of the LATM frame opened by header 1 is described as an example.
- the output data blocks 11, 12, 13, 14 of the Celp encoder 12 (FIG. 1) are generated and buffered.
- the output data block of the AAC encoder which is labeled "1" in FIG. 2c, is generated.
- the determination data block (header 1) is only written.
- the output data block of the first encoder which is generated first and is designated 11 in FIG. 2c, can then be written, ie transmitted, directly after the header 1.
- bit stream formats shown in FIGS. 4 to 6 are known only for simple encoders, but not for scalable encoders and in particular not for scalable encoders with a bit savings bank function.
- the bit savings bank is used so that the variable output data rate that a psychoacoustic encoder inherently generates can be adapted to a constant output data rate.
- the number of bits an audio encoder requires depends on the signal properties. If the signal is such that it can be quantized relatively roughly, a relatively small amount of bits is required to encode this signal. However, if the signal is such that it has to be quantized very finely in order not to introduce audible interference, a larger number of bits are required to encode this signal.
- an average number of bits is set for a section of a signal to be encoded. If the amount of bits actually required for coding a section is smaller than the specified number of bits, the bits that are not required can be inserted into the bit savings bank. The bit savings bank is filling up. If, on the other hand, a section of a If the signal is obtained in such a way that a larger number of bits than the specified number is required for coding so that no audible interference is introduced into the signal, the additional bits required can be taken from the bit savings bank. This will empty the bit savings bank. This can ensure that a constant output data rate is obtained and that no audible interference is introduced into the audio signal. The prerequisite for this is that the bit savings bank is chosen to be sufficiently large.
- bit savings bank In the MPEG AAC standard (13818-7: 1997), the bit savings bank is referred to as a "bit reservoir".
- the maximum size of the bit savings bank for constant data rate channels can be calculated by subtracting the average number of bits per block from the maximum decoder input buffer size. According to the MPEG AAC standard, its value is fixed at a transmission rate of 96 kbit / s for a stereo signal with a sampling rate of 48 kHz to a value of 10,240 bits.
- the maximum value of the bit savings bank i.e.
- the size of the bit savings bank is dimensioned so large that audible interference is introduced into the audio signal even under bad circumstances, that is to say even if the signal contains many sections which cannot be coded with the specified number of bits to maintain the constant output data rate. This is only possible if the bit savings bank is of a sufficiently large size that it will never become empty.
- the size of the bit savings bank is 10,240 bits. This leads to an inherent initial delay of about 0.1 s due to the bit savings bank. The longer the maximum size of the bit savings bank is chosen, and the smaller the transmission rate is, the greater the delay.
- the object of the present invention is to provide an encoder with a bit savings bank function, by means of which a lower transmission delay can be achieved.
- Another object of the present invention is to provide a method and an apparatus for generating a scalable data stream in which a bit savings bank function can be signaled.
- Another object of the present invention is to provide a method and an apparatus for decoding a scalable data stream in which a bit savings bank function is signaled.
- the present invention is based on the finding that the previous concept of the fixed bit savings bank size has to be abandoned in order to achieve deceleration with less delay.
- This is achieved according to the invention by making the maximum size of the bit savings bank of an encoder adjustable, a specific setting of the bit savings bank being achieved depending on the application and depending on the intended decoder function.
- a large bit savings bank can be selected to meet the highest audio quality requirements, while in the case of bidirectional communication in which there is a frequent change of transmitter and receiver or a frequent change of speakers, a smaller bit savings bank size is to be set.
- the bit savings size In order for the decoder to benefit from a smaller bit savings size setting, the bit savings size must somehow be communicated to the decoder. This can be achieved on the one hand by transmitting additional information in the data stream, but it can also, as is shown in particular on the basis of the scalable case, implicitly without transmitting additional information Side information or signaling information is provided.
- An advantage of the present invention is that the decoder delay can now be influenced directly by setting the maximum size of the bit savings bank. If the maximum size of the bit savings bank is chosen to be smaller, the decoder can also insert a smaller delay before it begins decoding, without the risk that it will run out of output data during decoding, which is to be avoided in any case.
- the "price" to be paid for this is that one or the other section of the audio signal has not been encoded with 100% audio quality, since the bit savings bank was empty and no extra bits were available. In such a case, an audio encoder usually reacts by violating the psychoacoustic masking threshold during the quantization and by choosing a coarser quantization than is actually necessary in order to manage with the number of bits available.
- the variability of the bit savings bank according to the invention and the associated variability of the decoder-side connections Delay is particularly advantageous in the case of a scalable audio encoder, since decoder with less delay can now be achieved not only in the first lowest scaling layer, but also decoder with lower delay and higher scaling layers, which are generated, for example, by an AAC encoder.
- only one scaling layer is influenced by the variable setting of the bit savings bank size, while the other or the other scaling layers remain unaffected. This means that individual scaling layers can be targeted, while no changes are brought about in the other scaling layers.
- an adjustable bit savings bank size can be achieved simply by positioning a determination data block in the scalable data stream without additional page information.
- the determination data block is positioned in the bit stream such that the decoder, when it receives the determination data block, must receive as many bits for the corresponding layer as is predetermined by the average block length.
- the decoder can start decoding without calculating or inserting a delay. This is achieved in that the determination data block with regard to the useful data is already in the scalable data stream the first and second scaling layers are written with a delay, preferably with a delay that corresponds to the setting of the bit savings bank size. This ensures that the encoder can select any size of the bit savings bank, depending on the requirement, and simply implicitly signals the selected bit savings size to the decoder to the effect that it enters the determination data block in the bit stream with a delay with respect to the useful data.
- the current status of the bit savings bank can then be signaled by a so-called back pointer, where the data of a previous section ends and where the data of the current section begin.
- the decoder can also interpret the offset information as a backpointer in order to know which data of the bit stream now belong to a determination data block and thus correspond to a time segment of the input signal, taking into account the variable core encoder delay, if necessary.
- An important advantage of this arrangement is that the decoder does not have to calculate and insert a delay when it receives a data stream according to the invention, but that the delay has already been taken into account on the coding side solely by the positioning of the determination data block.
- the decoder can therefore output a frame immediately upon receipt. This also opens up the possibility of signaling a set maximum bit savings bank size in a simple manner, namely without additional bits.
- the signaling can be carried out simply and without effort, namely by the position of the determination data block, it is also possible without further ado and in particular without access to the decoder to vary the size of the bit savings bank in order to be able to set the transmission delay as required.
- Fig. La shows a scalable encoder according to MPEG 4, which has the present invention
- 2a shows a schematic representation of an input signal which is divided into successive time segments
- 2b shows a schematic representation of an input signal which is divided into successive time segments, the ratio of the block length of the first encoder to the block length of the second encoder being shown;
- 2c is a schematic representation of a scalable data stream with high delay in decoding. tion of the first scaling layer;
- 2d shows a schematic illustration of a scalable data stream with low delay in the decoding of the first scaling layer
- 2e shows a schematic representation of a scalable data stream according to the invention, in which the determination data block is delayed compared to the user data;
- FIG. 3 shows a detailed illustration of the scalable data stream according to the invention using the example of a Celp encoder as the first encoder and an AAC encoder as the second encoder with bit savings bank function.
- FIG. 5 shows an example of a bit stream format with a fixed frame length and back pointer
- FIG. 6 shows an example of a bit stream format with a variable frame length.
- the scalable data stream contains successive determination data blocks, which are designated as header 1 and header 2.
- the determination data blocks are LATM headers.
- the LATM header 200 is that from the top left parts of the output data block of the AAC encoder hatched to the lower right, which are entered in remaining gaps between output data blocks of the first encoder.
- the frame started by the LATM header 200 no longer only contains output data blocks of the first encoder which belong to this frame, such as the output data blocks 13 and 14, but also the output data blocks 21 and 22 of the subsequent section of input data.
- the two output data blocks of the first encoder which are designated by 11 and 12, are present in the transmission direction (arrow 202) in front of the LATM header 200 in the bit stream.
- the offset information 204 indicate an offset of the output data blocks of the first encoder from two output data blocks. If FIG. 2d is compared with FIG.
- the decoder can already decode the lowest scaling layer earlier by a time corresponding to this offset than in the case of FIG. 2c if the decoder is only interested in the first scaling layer is.
- the offset information e.g. B. can be signaled in the form of a "core frame offset" are used to determine the position of the first output data block 11 in the bit stream.
- the bit stream designated in FIG. 2c results.
- the core frame offset is> zero
- the corresponding output data block of the first encoder 11 is transmitted earlier by the number of core frame offset of output data blocks of the first encoder.
- the delay between the first output data block of the first encoder after the LATM header and the first AAC frame results from the core encoder delay (FIG. 1) + core frame offset x core block length (block oil length of the encoder 1 in FIG. 2b) ).
- core frame offset zero (FIG. 2c) after the LATM header 200
- the output data blocks 11 and 12 of the first encoder are transmitted.
- the output data blocks 13 and 14 can follow the LATM header 200, whereby the delay in the case of pure celp decoding, that is to say decoding of the first scaling layer, is reduced by two celp block lengths.
- an offset of three blocks would be optimal.
- an offset of one or two blocks also brings a delay advantage.
- This bitstream structure enables the celp encoder to transmit the generated celp block immediately after encoding.
- no additional delay is added to the Celp encoder by the bit stream multiplexer (20).
- no additional delay is added to the celp delay by the scalable combination, so that the delay becomes minimal.
- the case shown in FIG. 2d is only exemplary. So different ratios of the block length of the first encoder to the block length of the second encoder are possible, the z. B. can vary from 1: 2 to 1:12 or can take other ratios.
- FIG. 2c shows a scalable data stream according to the LATM format, in which the data blocks of the first encoder have to be buffered, that is to say have to be delayed.
- this is due to the fact that the header can only be written when the output data of the second encoder are present, since the header provides information about the length or the number of bits in the output data block of the second encoder.
- FIG. 2d an improvement is already shown in that the output data blocks of the first encoder are written earlier in the bit stream in order to reduce the delay if a decoder only wants to decode the lowest scaling layer. Nevertheless, the determination data block still precedes the output data block of the second encoder, which is labeled "1" in Fig. 2d.
- Fig. 2e compared to Fig. 2c, the scalable data stream according to the invention is now shown, in which the determination data block (header 1 200) is no longer written immediately when it is available, that is to say before the output data block of the first encoder which is associated with is designated "11", but in which the determination data block 200 is written into the data stream with a delay compared to the case of FIG. 2c.
- this time period is equal to the maximum size of the bit savings bank (Max Bufferfullness 250).
- the output encoder block of the second encoder for the current portion of the input signal designated by the destination data block 200 begins a number of bits equal to buffer fullness 260 in the direction of transmission from an encoder to a decoder before the destination data. block, while when looking at Fig. 2c, the AAC data has started after the determination data block.
- the pointer 260 is thus a back pointer.
- the ratio of four blocks of output data of the first encoder to one block of output data of the second encoder in the example shown in FIG. 2e For the same number of samples only as an example, starting from the determination data block, a core frame offset is now also signaled, as in the case of FIG. 2d, so that a decoder knows which blocks of output data of the first encoder form, for example, a block of output data of the belong to the second encoder or are related to one another via a core encoder or delay.
- FIG. 2d is now compared with FIG. 2e, it can be seen that an offset 204 is also present in FIG. 2d.
- the offset 204 of FIG. 2d which has a value of 2 in FIG. 2d, would increase to a value of 5 in relation to the case of FIG. 2e, since the determination data block 200 in FIG. 2e increases in comparison with FIG 2d has shifted 3 output data blocks of the first encoder backwards.
- the scalable encoder according to the invention which is shown in FIG. 1 a, contains a block of bit savings bank controller 50 and a control line 52 from the AAC encoder 14 to the bit stream multiplexer 20, via which the maximum size of the bit bank savings bank, the has been set by the bit savings bank controller 50, can be communicated to the bit stream multiplexer so that it can carry out the bit stream formatting required in FIG. 2e.
- Fig. 1b there is a schematic block diagram of a scalable decoder which is complementary to the scalable encoder in Fig. La.
- the scalable bit stream which is supplied to the encoder via line 60, is fed into an input buffer / bit stream demultiplexer 62 of the decoder.
- the bit stream is split to extract the blocks needed for a CELP decoder 64 and an AAC decoder 66.
- the decoder according to the invention further comprises an AAC delay stage 68 which is there to introduce a delay corresponding to the size of the bit savings bank so that the AAC decoder 66 never runs out of data for output.
- this AAC delay stage is now designed to be variable, the delay being controlled as a function of bit savings bank information, which are extracted from the bit stream by bit stream demultiplexer 62 and fed to AAC delay stage 68 via a bit savings bank information line 70.
- the delay of the AAC delay stage 68 is now set. If a small bit savings bank is set by the bit savings bank control device 50 from FIG. 1 a, the AAC delay stage 68 can also be set to a smaller delay, so that a deceleration of the second scaling layer with less delay can be achieved.
- the scalable decoder of Fig. 1b further includes an MDCT 72 to transform the time domain output signals of the CELP decoder 64 to the frequency domain and an up-sampling stage upstream thereof.
- the spectrum is delayed by a delay stage 74, which compensates for the time differences between the two branches, so that the same conditions are present in a device 76, which is designated with adder / FSS -1 .
- the device 66 essentially performs the analog function as the subtractor 40 and the FSS 44 of FIG.
- the spectral values are processed by a device 78 for performing a reverse transformation Frequency range transformed into the time domain, so that at output 80 either only the second scaling layer or else the first and second scaling layers are present in the time domain. In contrast, only the first scaling layer in the time domain, which is generated by the CELP decoder 64, is present at an output 82.
- FIG. 3 which is similar to FIG. 2, but represents the special implementation using the example of MPEG 4.
- a current time period is shown hatched in the first line.
- the windowing used in the AAC encoder is shown schematically. As is known, an overlap-and-add of 50% is used, so that a window is usually twice the length of time samples as the current time period, which is hatched in the top line of FIG. 3.
- FIG. 3 also shows the delay tdip, which corresponds to block 26 of FIG. 1 and which in the selected example has a size of 5/8 of the block length.
- a block length of the current time segment of 960 samples is typically used, so that the delay tdip of 5/8 of the block length is 600 samples.
- the AAC encoder delivers a bit stream of 24 kbit / s
- the Celp encoder shown schematically below delivers a bit stream at a rate of 8 kbit / s.
- the total bit rate is then 32 kbit / s.
- the output data blocks zero and one of the Celp encoder correspond to the current time period for the first encoder.
- the output data block with number 2 of the Celp encoder already corresponds to the next time period.
- the delay of the downsampling stage 28 and of the celp encoder 12 is also shown by an arrow, which is represented by the reference symbol 302. This results in the delay that must be set by the stage 34 in order for the Subtracting point 40 from FIG. 1 has the same conditions, the delay, which is denoted by core code or delay and is illustrated by an arrow 304 in FIG. 3. Alternatively, this delay can also be generated by block 26. For example:
- FIG. 3 In contrast to FIG. 2d, in which four output data blocks of the first encoder are generated in accordance with an output data block of the second encoder, in FIG. 3 two for an output data block of the second encoder, which is drawn in black in the last two lines of FIG. 3 Output data blocks of the Celp encoder, designated "0" and "1", are generated.
- the output data block of the Celp encoder with the number "0" is no longer written behind a first LATM header 306, but rather the output data block of the Celp encoder with the number "one", especially since the output data block with the number "zero" has already been transmitted to the decoder.
- the celp block 1 is followed by the celp block 2 for the next period of time, with the rest of the data of the output data block of the AAC encoder being written into the data stream until a frame is completed until another LATM header 308 follows for the next time period.
- the present invention can, as shown in the last line of FIG. 3, simply with the bit savings bank function can be combined.
- the variable "Bufferfullness" which indicates the filling of the bit savings bank, is less than the maximum value, this means that the AAC frame required more bits than actually permitted for the immediately preceding period of time.
- the at least one output data block of the AAC encoder from one or more previous time segments must first be written into the bit stream before writing the output data block of the AAC encoder can be started for the current period. From the comparison of the last two lines of FIG.
- bit savings bank function also leads directly to a delay in the encoder for the AAC frame.
- the data for the AAC frame of the current time period which is designated by 310 in FIG. 3, is present at exactly the same time as in the case "1”, but can only be written into the bit stream after the AAC Data 312 for the immediately preceding period of time has been written into the bit stream. The starting position of the AAC frame is thus shifted depending on the bit savings bank level of the AAC encoder.
- the bit savings bank status should be transferred in the LATM element StreamMuxConfig by the variable "Bufferfullness".
- the variable buffer fullness is calculated from the variable bit reservoir divided by 32 times the currently existing number of channels of the audio channels.
- pointer 314 is deliberately drawn interrupted below the celp block 2, since it does not take into account the length of the celp block 2 or the length of the celp block 1, since this data naturally has nothing to do with the bit savings bank of the AAC encoder. Furthermore, no header data and bits from any other layers that may be present are taken into account.
- the celp frames are first extracted from the bit stream, which is readily possible since, for example, they are arranged equidistantly and have a fixed length.
- the length and spacing of all CELP blocks can be signaled in the LATM header anyway, so that immediate decoding is possible in any case.
- variable core frame offset can now be used to shift output data blocks of the first encoder forward in the bitstream, while arrow 314 (max bufferfullness - bufferfullness) can shift the output data block of the second encoder can be reached backwards in the scalable data stream, so that the bit savings bank function can also be implemented in the scalable data stream in a simple and safe manner, while the basic grid of the bit stream is maintained by the successive LATM determination data blocks which are written whenever the AAC encoder has encoded a time period, and which can therefore serve as a reference point, even if, as shown in the last line in FIG.
- a large part of the data in the frame designated by a LATM header originates on the one hand from the next time period (regarding the C elp frames) or from previous time segments (with regard to the AAC frame), the respective shifts, however, being communicated to a decoder by the two variables additionally to be transmitted in the bit stream.
- the last line of FIG. 3 describes the case in which the LATM header 306 is written to the bit stream immediately after it is generated, so that the LATM header 306 still has output data of the second encoder (312) of the previous period, the output data of the second encoder for the current period to which the LATM header 306 relates only follow the LATM header at a distance in the direction of transmission, the distance being determined by the There is a difference between Max Bufferfullness and Bufferfullness, as shown in FIG. 3.
- the LATM header 306 is no longer written when it is created is written, but delayed by a time period that corresponds to Max Bufferfullness. Depending on the value of buffer fullness, the LATM header 306 would therefore be located after a position 330 in the bit stream, and the forward pointer 314 is replaced by a backward pointer (260 in FIG. 2e).
- the following priority distribution is preferably preferred when writing data into the scalable bit stream, in order to achieve both low-delay decoding of the first scaling layer and low-delay decoding of the second scaling layer.
- the output data blocks of the first encoder enjoy high priority. Whenever an output data block of the first encoder has been completely written, this output data block is written into the bit stream. Thus, when using a CELP encoder, the equidistant grid of output data blocks of the first encoder, which also have the same length, is automatically obtained.
- the writing of the output data of the AAC encoder for the current time period is also interrupted, when a LATM header is ready and has been delayed by Max Bufferfullness 250 (Fig. 2e).
- the scalable bit stream is ready when the corresponding values for buffer fullness 260 and offset 270 are entered in the bit stream either separately or via the determination data block.
- a decoding of a bit stream generated in this way is discussed below. If the decoder is only interested in the first scaling layer, i.e. in the output data blocks of the first encoder (CELP encoder), it will simply fetch one CELP block after the other from the bit stream and regardless of LATM header or AAC data decode. Since the CELP blocks are preferably written into the bit stream immediately after they have been generated, deceleration of the CELP blocks is ensured with little delay.
- the decoder wants to decode both the first and the second scaling layer, i.e. if it wants to receive an audio signal with high quality, it must assign the CELP blocks and the AAC blocks for a superframe, i.e. for a certain number of sample values, with a core encoder delay (34 from FIG. 1 a) possibly also having to be taken into account if the current time segment of the input signal of the AAC encoder is shifted with respect to a superframe from the current time segment of the CELP encoder.
- the decoder buffering the bit stream until it hits a LATM header, e.g. B. header 200 of FIG. 2e. Knowing the offset 270, the decoder can then determine which output data blocks of the first encoder belong to the LATM header 200. Taking the variable buffer fullness into account, the decoder also knows where in the data stored in the decoder input buffer the AAC frame of the time period to which the LATM header refers.
- the entire AAC frame of interest is already contained in the decoder input buffer; in the case of buffer fullness equal to 0, the AAC frame of interest begins immediately after the LATM header, so that the decoder uses the data already stored in the input buffer or decode using a portion of the data stored in the input buffer and using an immediately arriving portion of data that is behind the LATM header in the direction of transmission without delay.
- the bit savings bank size is thus implicitly signaled solely by the position of the determination data block with respect to the useful data in the bit stream, without any side information being required.
- the variable delay stage in the decoder block 68 of FIG. 1b
- the line 70 of FIG. 1b also become obsolete.
Landscapes
- Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Quality & Reliability (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
Description
Claims
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE10102159 | 2001-01-18 | ||
DE10102159A DE10102159C2 (en) | 2001-01-18 | 2001-01-18 | Method and device for generating or decoding a scalable data stream taking into account a bit savings bank, encoder and scalable encoder |
PCT/EP2002/000294 WO2002063611A1 (en) | 2001-01-18 | 2002-01-14 | Method and device for the generation or decoding of a scalable data stream with provision for a bit-store, encoder and scalable encoder |
Publications (3)
Publication Number | Publication Date |
---|---|
EP1338004A1 true EP1338004A1 (en) | 2003-08-27 |
EP1338004B1 EP1338004B1 (en) | 2004-09-08 |
EP1338004B8 EP1338004B8 (en) | 2005-08-31 |
Family
ID=7670988
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP02718023A Expired - Lifetime EP1338004B8 (en) | 2001-01-18 | 2002-01-14 | Method and device for the generation or decoding of a scalable data stream with provision for a bit-store, encoder and scalable encoder |
Country Status (10)
Country | Link |
---|---|
US (1) | US7516230B2 (en) |
EP (1) | EP1338004B8 (en) |
JP (1) | JP3890300B2 (en) |
KR (1) | KR100576034B1 (en) |
AT (1) | ATE275751T1 (en) |
AU (1) | AU2002249122B2 (en) |
CA (1) | CA2434882C (en) |
DE (2) | DE10102159C2 (en) |
HK (1) | HK1056641A1 (en) |
WO (1) | WO2002063611A1 (en) |
Families Citing this family (30)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP3881943B2 (en) * | 2002-09-06 | 2007-02-14 | 松下電器産業株式会社 | Acoustic encoding apparatus and acoustic encoding method |
DE10328777A1 (en) * | 2003-06-25 | 2005-01-27 | Coding Technologies Ab | Apparatus and method for encoding an audio signal and apparatus and method for decoding an encoded audio signal |
GB2403634B (en) * | 2003-06-30 | 2006-11-29 | Nokia Corp | An audio encoder |
DE10353793B4 (en) * | 2003-11-13 | 2012-12-06 | Deutsche Telekom Ag | Method for improving the reproduction quality in the case of packet-oriented transmission of audio / video data |
US7756594B2 (en) * | 2004-06-14 | 2010-07-13 | Microsoft Corporation | Systems and methods for parsing flexible audio codec topologies |
US7590065B2 (en) * | 2004-08-04 | 2009-09-15 | Microsoft Corporation | Equal-opportunity bandwidth regulation |
US20060041895A1 (en) * | 2004-08-04 | 2006-02-23 | Microsoft Corporation | Systems and methods for interfacing with codecs across an architecture optimized for audio |
US7706901B2 (en) * | 2004-10-01 | 2010-04-27 | Microsoft Corporation | Low latency real-time audio streaming |
BRPI0518133A (en) * | 2004-10-13 | 2008-10-28 | Matsushita Electric Ind Co Ltd | scalable encoder, scalable decoder, and scalable coding method |
BRPI0516376A (en) * | 2004-12-27 | 2008-09-02 | Matsushita Electric Ind Co Ltd | sound coding device and sound coding method |
US8826093B2 (en) * | 2005-01-19 | 2014-09-02 | Qualcomm Incorporated | Power saving method for coded transmission |
US7809018B2 (en) * | 2005-12-16 | 2010-10-05 | Coding Technologies Ab | Apparatus for generating and interpreting a data stream with segments having specified entry points |
EP1961181B1 (en) * | 2005-12-16 | 2009-04-15 | Dolby Sweden AB | Apparatus for generating and interpreting a data stream having a series of segments using data in subsequent data frames |
US7590523B2 (en) * | 2006-03-20 | 2009-09-15 | Mindspeed Technologies, Inc. | Speech post-processing using MDCT coefficients |
EP1841072B1 (en) * | 2006-03-30 | 2016-06-01 | Unify GmbH & Co. KG | Method and apparatus for decoding layer encoded data |
US8190441B2 (en) * | 2006-09-11 | 2012-05-29 | Apple Inc. | Playback of compressed media files without quantization gaps |
JP4358215B2 (en) * | 2006-09-27 | 2009-11-04 | 株式会社東芝 | Video encoding apparatus and method |
WO2010032992A2 (en) * | 2008-09-18 | 2010-03-25 | 한국전자통신연구원 | Encoding apparatus and decoding apparatus for transforming between modified discrete cosine transform-based coder and hetero coder |
CN101771417B (en) * | 2008-12-30 | 2012-04-18 | 华为技术有限公司 | Methods, devices and systems for coding and decoding signals |
BR112012026326B1 (en) * | 2010-04-13 | 2021-05-04 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V | method and encoder and decoder for accurate sampling representation of an audio signal |
US8532804B2 (en) * | 2010-06-18 | 2013-09-10 | Microsoft Corporation | Predictive resampler scheduler algorithm |
US8433823B2 (en) * | 2010-09-03 | 2013-04-30 | Tibco Software Inc. | Random access data compression |
KR20120084234A (en) | 2011-01-19 | 2012-07-27 | 삼성전자주식회사 | Method for delivering mmt access unit for mpeg media transport |
TWI476761B (en) * | 2011-04-08 | 2015-03-11 | Dolby Lab Licensing Corp | Audio encoding method and system for generating a unified bitstream decodable by decoders implementing different decoding protocols |
WO2013142650A1 (en) | 2012-03-23 | 2013-09-26 | Dolby International Ab | Enabling sampling rate diversity in a voice communication system |
TWI505262B (en) | 2012-05-15 | 2015-10-21 | Dolby Int Ab | Efficient encoding and decoding of multi-channel audio signal with multiple substreams |
US10199043B2 (en) * | 2012-09-07 | 2019-02-05 | Dts, Inc. | Scalable code excited linear prediction bitstream repacked from a higher to a lower bitrate by discarding insignificant frame data |
US9564136B2 (en) * | 2014-03-06 | 2017-02-07 | Dts, Inc. | Post-encoding bitrate reduction of multiple object audio |
US10283091B2 (en) * | 2014-10-13 | 2019-05-07 | Microsoft Technology Licensing, Llc | Buffer optimization |
US12001680B2 (en) * | 2022-08-24 | 2024-06-04 | Micron Technology, Inc. | Utilizing last successful read voltage level in memory access operations |
Family Cites Families (25)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE3943881B4 (en) * | 1989-04-17 | 2008-07-17 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Digital coding method |
DE4218695A1 (en) * | 1992-06-09 | 1993-12-16 | Thomson Brandt Gmbh | Methods, encoders and decoders for data transmission and / or storage |
US5365552A (en) * | 1992-11-16 | 1994-11-15 | Intel Corporation | Buffer fullness indicator |
US5835033A (en) * | 1994-11-08 | 1998-11-10 | Canon Kabushiki Kaisha | Decoding apparatus and method for coded data |
US5896099A (en) * | 1995-06-30 | 1999-04-20 | Sanyo Electric Co., Ltd. | Audio decoder with buffer fullness control |
DE19537338C2 (en) * | 1995-10-06 | 2003-05-22 | Fraunhofer Ges Forschung | Method and device for encoding audio signals |
US5758092A (en) * | 1995-11-14 | 1998-05-26 | Intel Corporation | Interleaved bitrate control for heterogeneous data streams |
US5768537A (en) * | 1996-02-22 | 1998-06-16 | International Business Machines Corporation | Scalable MPEG2 compliant video encoder |
JP3246715B2 (en) | 1996-07-01 | 2002-01-15 | 松下電器産業株式会社 | Audio signal compression method and audio signal compression device |
JP3344944B2 (en) * | 1997-05-15 | 2002-11-18 | 松下電器産業株式会社 | Audio signal encoding device, audio signal decoding device, audio signal encoding method, and audio signal decoding method |
US6092041A (en) * | 1996-08-22 | 2000-07-18 | Motorola, Inc. | System and method of encoding and decoding a layered bitstream by re-applying psychoacoustic analysis in the decoder |
KR100261253B1 (en) * | 1997-04-02 | 2000-07-01 | 윤종용 | Scalable audio encoder/decoder and audio encoding/decoding method |
KR100335609B1 (en) | 1997-11-20 | 2002-10-04 | 삼성전자 주식회사 | Scalable audio encoding/decoding method and apparatus |
WO1999033274A1 (en) * | 1997-12-19 | 1999-07-01 | Kenneth Rose | Scalable predictive coding method and apparatus |
WO2000008767A1 (en) * | 1998-08-06 | 2000-02-17 | Samsung Electronics Co., Ltd. | Channel encoding/decoding in communication system |
US6553086B1 (en) * | 1998-10-02 | 2003-04-22 | Lg Electronics, Inc. | Method and apparatus for recording time information for digital data streams |
JP2000307661A (en) | 1999-04-22 | 2000-11-02 | Matsushita Electric Ind Co Ltd | Encoding device and decoding device |
US6904089B1 (en) * | 1998-12-28 | 2005-06-07 | Matsushita Electric Industrial Co., Ltd. | Encoding device and decoding device |
FR2791167B1 (en) * | 1999-03-17 | 2003-01-10 | Matra Nortel Communications | AUDIO ENCODING, DECODING AND TRANSCODING METHODS |
US6195989B1 (en) | 1999-05-04 | 2001-03-06 | Caterpillar Inc. | Power control system for a machine |
KR100349329B1 (en) * | 1999-06-23 | 2002-08-21 | 한국전자통신연구원 | Method of processing of MPEG-2 AAC algorithm |
DE19959156C2 (en) * | 1999-12-08 | 2002-01-31 | Fraunhofer Ges Forschung | Method and device for processing a stereo audio signal to be encoded |
US6369722B1 (en) * | 2000-03-17 | 2002-04-09 | Matra Nortel Communications | Coding, decoding and transcoding methods |
US6675148B2 (en) * | 2001-01-05 | 2004-01-06 | Digital Voice Systems, Inc. | Lossless audio coder |
US6926526B2 (en) * | 2002-05-24 | 2005-08-09 | Kenneth G. Hudak | Endodontic adapter for a sonic scaler |
-
2001
- 2001-01-18 DE DE10102159A patent/DE10102159C2/en not_active Expired - Lifetime
-
2002
- 2002-01-14 AT AT02718023T patent/ATE275751T1/en active
- 2002-01-14 EP EP02718023A patent/EP1338004B8/en not_active Expired - Lifetime
- 2002-01-14 CA CA002434882A patent/CA2434882C/en not_active Expired - Lifetime
- 2002-01-14 WO PCT/EP2002/000294 patent/WO2002063611A1/en active IP Right Grant
- 2002-01-14 DE DE50200953T patent/DE50200953D1/en not_active Expired - Lifetime
- 2002-01-14 AU AU2002249122A patent/AU2002249122B2/en not_active Expired
- 2002-01-14 KR KR1020037009445A patent/KR100576034B1/en active IP Right Grant
- 2002-01-14 JP JP2002563470A patent/JP3890300B2/en not_active Expired - Lifetime
- 2002-01-14 US US10/466,781 patent/US7516230B2/en active Active
-
2003
- 2003-12-11 HK HK03108993A patent/HK1056641A1/en not_active IP Right Cessation
Non-Patent Citations (1)
Title |
---|
See references of WO02063611A1 * |
Also Published As
Publication number | Publication date |
---|---|
JP3890300B2 (en) | 2007-03-07 |
DE10102159C2 (en) | 2002-12-12 |
KR20030076611A (en) | 2003-09-26 |
DE10102159A1 (en) | 2002-08-08 |
US20040162911A1 (en) | 2004-08-19 |
EP1338004B1 (en) | 2004-09-08 |
ATE275751T1 (en) | 2004-09-15 |
WO2002063611A1 (en) | 2002-08-15 |
KR100576034B1 (en) | 2006-05-02 |
CA2434882C (en) | 2008-04-15 |
CA2434882A1 (en) | 2002-08-15 |
HK1056641A1 (en) | 2004-02-20 |
DE50200953D1 (en) | 2004-10-14 |
AU2002249122B2 (en) | 2005-06-23 |
US7516230B2 (en) | 2009-04-07 |
JP2004523790A (en) | 2004-08-05 |
EP1338004B8 (en) | 2005-08-31 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP1338004B1 (en) | Method and device for the generation or decoding of a scalable data stream with provision for a bit-store, encoder and scalable encoder | |
DE19549621B4 (en) | Device for encoding audio signals | |
DE3688980T2 (en) | Method for multi-speed coding of signals and device for carrying out this method. | |
DE60121592T2 (en) | CODING AND DECODING A DIGITAL SIGNAL | |
DE60012860T2 (en) | Method for processing a plurality of digital audio data streams | |
DE19628292B4 (en) | Method for coding and decoding stereo audio spectral values | |
EP1647010B1 (en) | Audio file format conversion | |
EP2119078B1 (en) | Device and method for generating a signal for transmission or a decoded signal | |
DE60117471T2 (en) | BROADBAND SIGNAL TRANSMISSION SYSTEM | |
EP1382038A2 (en) | Device and method for embedding a watermark in an audio signal | |
EP2245621B1 (en) | Method and means for encoding background noise information | |
EP1327243B1 (en) | Method and device for the generation of a scalable data stream and method and device for decoding a scalable data stream | |
DE19742655C2 (en) | Method and device for coding a discrete-time stereo signal | |
EP1953739A2 (en) | Method and device for reducing noise | |
EP1023777B1 (en) | Method and device for limiting a stream of audio data with a scaleable bit rate | |
DE102006055737A1 (en) | Method for the scalable coding of stereo signals | |
DE60217612T2 (en) | Method and device for coding and decoding speech signals | |
DE4430864A1 (en) | Transmission and storage method for embedding additional information in data stream | |
EP1354314B1 (en) | Method and device for producing a scalable data stream, and method and device for decoding a scalable data stream while taking a bit bank function into account | |
EP2245622B1 (en) | Method and means for decoding background noise information | |
DE2303497C2 (en) | Method for the transmission of speech signals | |
DE10339498B4 (en) | Audio file format conversion | |
DE10065363B4 (en) | Apparatus and method for decoding a coded data signal | |
WO2005034091A1 (en) | Audio transmission method and arrangement | |
DE102005032079A1 (en) | Noise suppression process for decoded signal comprise first and second decoded signal portion and involves determining a first energy envelope generating curve, forming an identification number, deriving amplification factor |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20030626 |
|
AK | Designated contracting states |
Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE TR |
|
AX | Request for extension of the european patent |
Extension state: AL LT LV MK RO SI |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: FRAUNHOFER-GESELLSCHAFT ZUR FOERDERUNG DERANGEWAND |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE TR |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20040908 |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D Free format text: NOT ENGLISH |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D Free format text: GERMAN |
|
REF | Corresponds to: |
Ref document number: 50200953 Country of ref document: DE Date of ref document: 20041014 Kind code of ref document: P |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20041208 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20041208 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20041219 |
|
GBT | Gb: translation of ep patent filed (gb section 77(6)(a)/1977) |
Effective date: 20041201 |
|
REG | Reference to a national code |
Ref country code: SE Ref legal event code: TRGR |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20050114 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20050131 |
|
REG | Reference to a national code |
Ref country code: HK Ref legal event code: GR Ref document number: 1056641 Country of ref document: HK |
|
LTIE | Lt: invalidation of european patent or patent extension |
Effective date: 20040908 |
|
ET | Fr: translation filed | ||
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
RIN2 | Information on inventor provided after grant (corrected) |
Inventor name: SPERSCHNEIDER, RALPH Inventor name: GRILL, BERNHARD Inventor name: TEICHMANN, BODO Inventor name: LUTZKY, MANFRED |
|
26N | No opposition filed |
Effective date: 20050609 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20050208 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 15 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 16 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 17 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: LU Payment date: 20210120 Year of fee payment: 20 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20210120 Year of fee payment: 20 Ref country code: FI Payment date: 20210119 Year of fee payment: 20 Ref country code: CH Payment date: 20210122 Year of fee payment: 20 Ref country code: IE Payment date: 20210120 Year of fee payment: 20 Ref country code: NL Payment date: 20210120 Year of fee payment: 20 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20210120 Year of fee payment: 20 Ref country code: SE Payment date: 20210122 Year of fee payment: 20 Ref country code: AT Payment date: 20210119 Year of fee payment: 20 Ref country code: BE Payment date: 20210120 Year of fee payment: 20 Ref country code: GB Payment date: 20210122 Year of fee payment: 20 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: IT Payment date: 20210129 Year of fee payment: 20 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R071 Ref document number: 50200953 Country of ref document: DE Ref country code: CH Ref legal event code: PL |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MK Effective date: 20220113 |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: PE20 Expiry date: 20220113 |
|
REG | Reference to a national code |
Ref country code: BE Ref legal event code: MK Effective date: 20220114 |
|
REG | Reference to a national code |
Ref country code: FI Ref legal event code: MAE |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: MK9A |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK07 Ref document number: 275751 Country of ref document: AT Kind code of ref document: T Effective date: 20220114 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF EXPIRATION OF PROTECTION Effective date: 20220114 Ref country code: GB Free format text: LAPSE BECAUSE OF EXPIRATION OF PROTECTION Effective date: 20220113 |