EP2179415A1 - Apparatus and method for encoding and decoding enhancement layer - Google Patents
Apparatus and method for encoding and decoding enhancement layerInfo
- Publication number
- EP2179415A1 EP2179415A1 EP08793289A EP08793289A EP2179415A1 EP 2179415 A1 EP2179415 A1 EP 2179415A1 EP 08793289 A EP08793289 A EP 08793289A EP 08793289 A EP08793289 A EP 08793289A EP 2179415 A1 EP2179415 A1 EP 2179415A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- exponent
- sample
- bits
- additional mantissa
- allocated
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
- G10L19/16—Vocoder architecture
- G10L19/18—Vocoders using multiple modes
- G10L19/24—Variable rate codecs, e.g. for generating different qualities using a scalable representation such as hierarchical encoding or layered encoding
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/002—Dynamic bit allocation
Definitions
- the present invention relates to a G.711 encoder and decoder, and more particularly, to a method of encoding and decoding an enhancement layer to reduce a quantization error in a G.711 codec.
- Examples of a method of compressing and decompressing a speech signal include pulse coded modulation (PCM) and code-excited linear prediction (CELP).
- PCM pulse coded modulation
- CELP code-excited linear prediction
- PCM is a method of compressing a respective voice sample to a predetermined number of bits
- CELP is a method of framing a speech signal into blocks and compressing the speech signal on the basis of a speech production model.
- codecs have been developed and standardized in a wide range of fields.
- One of the most widely used codecs is a log PCM codec that is adopted on a public switched telephone network (PSTN), internet telephony, and so on.
- PSTN public switched telephone network
- the log PCM codec varies a quantization step depending on the magnitude of an input signal. That is, the log PCM codec uses a small quantization step when the level of an input signal is low, and uses a large quantization step when the level of an input signal is high.
- the log PCM codec can compress a digital sample with 16 bits per sample to a codeword with 8 bits per sample. Accordingly, if a signal is sampled at 8 KHz, a bit rate of 64,000 bps is obtained using the log PCM codec.
- the A-law and u-law have been standardized as ITU-T( Internationa I Telecommunication Union - Telecommunication Sector) Recommendation G.711 since 1972.
- the values of ⁇ and A in the G.711 are 255 ( ⁇ ) and 87.56 (A), respectively.
- the G.711 codec actually uses a floating point quantization method, rather than directly calculates Equation 1.
- available bits per sample e.g., 8 bits for the G.711
- some bits are used to determine a quantization step and the remaining bits are used to express a position in the quantization step.
- the former bits are exponent bits and the latter bits are mantissa bits.
- 3 bits are used for the exponent
- 4 bits are used for the mantissa
- the remaining 1 bit is used to express a sign of a sample.
- the G.711 provides a high quality of about 4 MOS (Mean Opinion Score) for narrowband speech, with less computational complexity and memory requirements. However, there is still quality degradation caused by quantization error, compared with an original speech signal.
- MOS Mel Opinion Score
- FIG. 1 is a block diagram of an encoder and a decoder for quality improvement of speech in a G.711 codec;
- FIG. 2 is a schematic view illustrating an input sample and an output bitstream of an encoder to which a log pulse coded modulation (PCM) codec of a conventional G.711 codec is applied;
- FIG. 3 is a schematic view illustrating an input sample and an output bitstream of an encoder to which enhancement layer encoding according to the present invention is applied;
- PCM log pulse coded modulation
- FIGS. 4A and 4B illustrate an exponent maps for dynamically calculating the number of bits for additional mantissa information of each sample in a frame depending on the magnitude of an input signal according to an embodiment of the present invention
- FIG. 5 is a flowchart illustrating a method of generating a bit allocation table according to an embodiment of the present invention
- FIG. 6 is a block diagram of an enhancement layer encoder according to an embodiment of the present invention.
- FIG. 7 is a block diagram of an enhancement layer decoder according to an embodiment of the present invention
- FIG. 8 is a block diagram of an enhancement layer encoder according to another embodiment of the present invention.
- FIG. 9 is a flowchart illustrating a method of encoding an enhancement layer according to an embodiment of the present invention.
- FIG. 10 is a block diagram of an enhancement layer decoder according to another embodiment of the present invention.
- FIG. 11 is a flowchart illustrating a method of decoding an enhancement layer according to an embodiment of the present invention.
- the present invention provides an apparatus and method for encoding and decoding an enhancement layer, which can reduce quantization error in a G.711 codec by adding extra bits to each G.711 -coded sample.
- the present invention also provides an enhancement layer encoder and decoder which can reduce quantization error in a G.711 codec.
- a method of encoding an enhancement layer comprising: calculating exponent indices of additional mantissa information of each sample based on exponent information of each sample in a frame; repeating a process of allocating 1 bit to all of samples with a current exponent index starting from the maximum value while decreasing by 1 at every repetition until the total number of bits allocated to the samples is equal to the total number of available bits in the frame; and extracting the most significant bit(s), as many as the number of bit(s) allocated to each sample, from the additional mantissa information of each sample in the frame.
- an enhancement layer encoder comprising: an exponent map generating unit generating an exponent map that is a matrix, including exponent indices of additional mantissa information derived from exponent value of each sample and sample indices for each sample in a frame; a bit allocation table generating unit generating a bit allocation table containing the number of bits allocated to each sample by referring to the exponent map, in the way of repeating a process of allocating 1 bit to each sample with a respective exponent index starting from the maximum value while decreasing by 1 at every repetition until the total number of bits allocated to the samples is equal to the total number of available bits in the frame; and a bit output unit outputting the most significant bit(s), as many as the number of the bit(s) allocated to each sample, from the additional mantissa information of each sample, by referring to the bit allocation table.
- an encoder comprising: a G.711 encoding unit encoding an input frame; an enhancement layer encoding unit encoding the input frame in the way dynamically allocating the number of bits for additional mantissa information to each sample in the input frame based on a exponent information of each sample which is obtained from the G.711 encoding unit; and a multiplexing unit multiplexing bitstream of the G.711 encoding unit and bitstream of the enhancement layer encoding unit.
- a method of decoding an enhancement layer comprising: calculating exponent indices of additional mantissa information of each sample in a frame based on exponent information of each sample; repeating a process of allocating 1 bit to each sample with a current exponent index starting from the maximum while decreasing by 1 at every repetition until the total number of bits allocated to the samples is equal to the total number of available bits in the frame; and extracting an additional mantissa bit(s), as many as the number of bits allocated to each sample, from an enhancement bitstream and decoding the extracted additional mantissa bit(s).
- an enhancement layer decoder comprising: an exponent map generating unit generating an exponent map that is a matrix, including exponent indices of additional mantissa information derived from exponent value of each sample and sample indices for each sample in a frame; a bit allocation table generating unit generating a bit allocation table containing the number of bits allocated to each sample by referring to the exponent map, in the way of repeating a process of allocating 1 bit to each sample with a respective exponent index starting from the maximum value while decreasing by 1 at every repetition until the total number of bits allocated to the samples is equal to the total number of available bits in the frame; and an additional mantissa decoding unit extracting an additional mantissa bit(s), as many as the number of bit(s) allocated to each sample, from an enhancement bitstream by referring to the bit allocation table and decoding the additional mantissa bits.
- a decoder comprising: a demultiplexing unit demultiplexing a received frame to a G.711 bitstream and an enhancement bitstream; a G.711 decoding unit decoding the G.711 bitstream; an enhancement layer decoding unit calculating the number of bits for additional mantissa information allocated to each sample using exponent value of each sample obtained from the G.711 decoding unit, extracting an additional mantissa bit(s), as many as the number of bits allocated to each sample, from an enhancement bitstream, and decoding the extracted additional mantissa bit(s); and a signal synthesizing unit combining an output signal of the G.711 decoding unit and an output signal of the enhancement layer decoding unit to output the synthesized signal.
- a method of encoding an enhancement layer in a hierarchical encoding combined with a base layer including exponent information and mantissa information of a sample comprising: calculating the number of allocated bits for additional mantissa information to extend the mantissa information; and encoding the additional mantissa information according to the number of the allocated bits, wherein the calculating the number of allocated bits is adaptively performed by referring to exponent information of samples existing in an interval of predetermined length.
- a method of decoding an enhancement layer signal in a hierarchically encoded bitstream comprising a base layer bitstream, which includes exponent information and mantissa information of a sample, and an enhancement layer bitstream, which includes additional mantissa information of the sample, the method comprising: calculating the number of bits allocated to each sample for the additional mantissa information, based upon the exponent information extracted from the base layer bitstream; and extracting the additional mantissa information of the calculated number of the bits allocated to each sample from the enhancement layer bitstream.
- ADVANTAGEOUS EFFECTS As described above, according to the present invention, since additional bits for mantissa information are dynamically allocated to each sample by using exponent information obtained from a conventional log PCM codec like a conventional G.711 codec, a quantization error resulting from the log PCM codec can be reduced and the quality of speech can be improved. Also, since the present invention does not change the conventional log PCM codec, compatibility with a network employing an existing log PCM codec can be achieved.
- a method of encoding an enhancement layer comprising: calculating exponent indices of additional mantissa information of each sample based on exponent information of each sample in a frame; repeating a process of allocating 1 bit to all of samples with a current exponent index starting from the maximum value while decreasing by 1 at every repetition until the total number of bits allocated to the samples is equal to the total number of available bits in the frame; and extracting the most significant bit(s), as many as the number of bit(s) allocated to each sample, from the additional mantissa information of each sample in the frame.
- an enhancement layer encoder comprising: an exponent map generating unit generating an exponent map that is a matrix, including exponent indices of additional mantissa information derived from exponent value of each sample and sample indices for each sample in a frame; a bit allocation table generating unit generating a bit allocation table containing the number of bits allocated to each sample by referring to the exponent map, in the way of repeating a process of allocating 1 bit to each sample with a respective exponent index starting from the maximum value while decreasing by 1 at every repetition until the total number of bits allocated to the samples is equal to the total number of available bits in the frame; and a bit output unit outputting the most significant bit(s), as many as the number of the bit(s) allocated to each sample, from the additional mantissa information of each sample, by referring to the bit allocation table.
- an encoder comprising: a G.711 encoding unit encoding an input frame; an enhancement layer encoding unit encoding the input frame in the way dynamically allocating the number of bits for additional mantissa information to each sample in the input frame based on a exponent information of each sample which is obtained from the G.711 encoding unit; and a multiplexing unit multiplexing bitstream of the G.711 encoding unit and bitstream of the enhancement layer encoding unit.
- a method of decoding an enhancement layer comprising: calculating exponent indices of additional mantissa information of each sample in a frame based on exponent information of each sample; repeating a process of allocating 1 bit to each sample with a current exponent index starting from the maximum while decreasing by 1 at every repetition until the total number of bits allocated to the samples is equal to the total number of available bits in the frame; and extracting an additional mantissa bit(s), as many as the number of bits allocated to each sample, from an enhancement bitstream and decoding the extracted additional mantissa bit(s).
- an enhancement layer decoder comprising: an exponent map generating unit generating an exponent map that is a matrix, including exponent indices of additional mantissa information derived from exponent value of each sample and sample indices for each sample in a frame; a bit allocation table generating unit generating a bit allocation table containing the number of bits allocated to each sample by referring to the exponent map, in the way of repeating a process of allocating 1 bit to each sample with a respective exponent index starting from the maximum value while decreasing by 1 at every repetition until the total number of bits allocated to the samples is equal to the total number of available bits in the frame; and an additional mantissa decoding unit extracting an additional mantissa bit(s), as many as the number of bit(s) allocated to each sample, from an enhancement bitstream by referring to the bit allocation table and decoding the additional mantissa bits.
- a decoder comprising: a demultiplexing unit demultiplexing a received frame to a G.711 bitstream and an enhancement bitstream; a G.711 decoding unit decoding the G.711 bitstream; an enhancement layer decoding unit calculating the number of bits for additional mantissa information allocated to each sample using exponent value of each sample obtained from the G.711 decoding unit, extracting an additional mantissa bit(s), as many as the number of bits allocated to each sample, from an enhancement bitstream, and decoding the extracted additional mantissa bit(s); and a signal synthesizing unit combining an output signal of the G.711 decoding unit and an output signal of the enhancement layer decoding unit to output the synthesized signal.
- a method of encoding an enhancement layer in a hierarchical encoding combined with a base layer including exponent information and mantissa information of a sample comprising: calculating the number of allocated bits for additional mantissa information to extend the mantissa information; and encoding the additional mantissa information according to the number of the allocated bits, wherein the calculating the number of allocated bits is adaptively performed by referring to exponent information of samples existing in an interval of predetermined length.
- a method of decoding an enhancement layer signal in a hierarchically encoded bitstream comprising a base layer bitstream, which includes exponent information and mantissa information of a sample, and an enhancement layer bitstream, which includes additional mantissa information of the sample, the method comprising: calculating the number of bits allocated to each sample for the additional mantissa information, based upon the exponent information extracted from the base layer bitstream; and extracting the additional mantissa information of the calculated number of the bits allocated to each sample from the enhancement layer bitstream.
- FIG. 1 is a block diagram of an encoder 100 and a decoder 150 for improving the quality of speech in a G.711 codec.
- the encoder 100 includes an input buffer 105, a G.711 encoding unit 110, an enhancement layer encoding unit 115, and a multiplexing unit 120.
- the decoder 150 includes a demultiplexing unit 155, a G.711 decoding unit 160, an enhancement layer decoding unit 165, a signal synthesizing unit 170, and an output buffer 175.
- the encoder 10O is connected to the decoder 150 through a communication channel 140.
- the encoder 100 will be first explained.
- the G.711 encoding unit 110 encodes the frame stored in the input buffer 105 using the conventional G.711 codec, and outputs a G.711 bitstream. Since the G.711 is a well-known codec standardized by the ITU-T, a detailed explanation thereof will not be given here.
- the enhancement layer encoding unit 115 encodes a quantization error, which is not represented by the G.711 encoding unit 110, using additional bits and outputs an enhancement bitstream.
- the multiplexing unit 120 multiplexes the G.711 bitstream of the G.711 encoding unit 110 and the enhancement bitstream of the enhancement layer encoding unit 115.
- the multiplexed bitstream is transmitted to the decoder 150 through the communication channel 140.
- the decoder 150 will now be explained.
- the demultiplexing unit 155 demultiplexes the multiplexed bitstream received from the encoder 100 through the communication channel 140 to a G.711 bitstream and an enhancement bitstream.
- the G.711 decoding unit 160 decodes the G.711 bitstream by using the G.711 codec and outputs a G.711 decoded signal.
- the enhancement layer decoding unit 165 decodes the enhancement bitstream and outputs an enhancement layer decoded signal.
- the signal synthesizing unit 170 combines the G.711 decoded signal from the G.711 decoding unit 160 and the enhancement layer decoded signal from the enhancement layer decoding unit 165 and outputs a a synthesized signal.
- the output buffer 175 stores the synthesized signal from the signal synthesizing unit 170, and outputs the stored signal in frame-by-frame.
- FIG. 2 is a schematic view illustrating an input bitstream and an output bitstream of an encoder to which a log PCM codec of a conventional G.711 codec is applied.
- FIG. 3 is a schematic view illustrating an input bitstream and an output bitstream of an encoder to which enhancement layer encoding according to the present invention is applied.
- the 8-bit sample 250 is composed of 1-bit sign information 260, 3-bit exponent information 270, and 4-bit mantissa information 280.
- the exponent information 270 indicates a compander segment
- the mantissa information 280 indicates a position in the segment indicated by the exponent information 270.
- a coded sample when enhancement layer encoding according to the present invention is used, includes additional mantissa information 390 as well as a G.711 -coded sample of 8 bits.
- the additional mantissa information 390 sub-divides a specific position indicated by mantissa information 380 in a segment indicated by exponent information 370, thereby reducing quantization error resulted from the G.711 codec.
- a bit rate of the enhancement layer is 16 Kbit/s and the enhancement layer operates on 5 ms frame
- the total number of available bits for enhancement layer per frame is 80 bits. That is, when the number of samples in the frame is 40 samples, an additional 2 bits per sample can be allocated on the average.
- FIGS. 4A and 4B illustrate exponent maps for dynamically allocating the number of bits of additional mantissa information to each sample in a frame depending on the magnitude of an input sample according to an embodiment of the present invention.
- the exponent map is a matrix in which row indices are exponent indices of additional mantissa information derived from an exponent value of each sample and column indices are sample indices representing each sample. For example, when 0 to 3-bits for additional mantissa information are allocated to each sample in a frame of 40 samples, the exponent map is a 10 x 40 matrix.
- exponent indices of each sample are proportional to the exponent value of each sample, are sequential, and the number of the exponent indices are the same as the number of bits of additional mantissa information. That is, exponent indices are obtained by incrementing the exponent value of each sample. For example, when the exponent value of a sample is "000" in binary, exponent indices corresponding to the sample are 0 (the exponent value + 0), 1 (the exponent value + 1 ), and 2 (the exponent value + 2). As another example, when the exponent value is 7 ("111" in binary), exponent indices are 7 (the exponent value + 0), 8 (the exponent value + 1 ), and 9 (the exponent value + 2). Accordingly, exponent indices of each sample for additional exponent information can range from 0 to 9.
- the exponent map is established.
- a bit allocation table indicating the number of additional bits allocated to each sample is created based on the exponent map. That is, 1 bit is allocated to each sample corresponding to the current exponent index which starts from the maximum value (i.e., 9) while decreasing by 1.
- the bit allocation is performed until the total number of bits allocated to the samples is equal to the available bits in a frame. The generation of the bit allocation table will be explained in detail later with reference to FIG. 5.
- an exponent map is a matrix in which row indices are exponent indices of additional mantissa information derived from the exponent value of each sample and column indices are order indices indicating the number of samples with the same exponent index.
- Each of elements of the exponent map represents a sample index with an exponent index.
- the exponent map would be a 10 x 40 matrix since all of samples in a frame can have the common exponent index.
- the current numbers of the samples with the exponent indices are increased by 1, respectively. For example, when the exponent value of the 0-th sample in a frame is "110" in binary, the corresponding exponent indices are 6, 7, and 8.
- the corresponding exponent indices are 4, 5, and 6.
- Elements corresponding to the exponent indices 4, 5, and 6, i.e., (4,0), (5,0), (6,1 ) are set to sample index 1.
- the reason why the (6,1 )-th element is set to 1 is that the number of samples having the exponent index of 6 is already 1 in the previous sample.
- the numbers of the samples corresponding to the exponent indices 4, 5, 6, 7, and 8 are 1 , 1 , 2, 1 , and 1 , respectively.
- FIG. 5 is a flowchart illustrating a method of generating a bit allocation table according to an embodiment of the present invention. Referring to FIG. 5, when it is assumed that the maximum number of additional bits per sample is 3 and the total number of available bits per frame is 80, an enhancement layer encoder outputs additional mantissa information to each sample based on an exponent value of each sample.
- the enhancement layer encoder initializes all elements of a bit allocation table to 0, sets current available bits to the total number of available bits in a current frame, and sets a maximum value of the exponent indices as a current exponent index.
- the enhancement layer encoder calculates the number of samples corresponding to each exponent index, by referring to the exponent maps illustrated in FIGS. 4A and 4B. For example, there are two samples, whose sample indices are 0 and 39, corresponding to an exponent index 8 in the exponential map illustrated in FIG. 4A.
- the enhancement layer encoder compares the number of samples corresponding to the current exponent index with current available bits, and then sets the smaller one of the two numbers as the number of usable bits.
- the enhancement layer encoder updates the current exponent index by decrementing the current exponent index by 1 , and returns to operation S520. Then the enhancement layer encoder repeats operations from S520 to S550.
- FIG. 6 is a block diagram of an enhancement layer encoder according to an embodiment of the present invention.
- the enhancement layer encoder includes an exponent map generating unit 600, a bit allocation table generating unit 610, and a bit output unit 620.
- the enhancement layer encoder corresponds to an enhancement layer encoding unit
- the exponent map generating unit 600 computes exponent indices of additional mantissa information per sample based on the exponent value of each sample, and then generates an exponent map including the exponent indices of each sample.
- the exponent value of each sample can be extracted from the G.711 encoding unit 110 illustrated in FIG. 1. Since the exponent map is illustrated in FIG. 4A and 4 B, a detailed explanation thereof will not be given here.
- the bit allocation table generating unit 610 retrieves samples corresponding to each exponent index and allocates 1 bit to the retrieved samples by sequentially decrementing the exponent index by 1 from the maximum value. When this bit allocation procedure is completed for all of exponent indices, the bit allocation table generating unit 610 generates a bit allocation table containing the number of bits allocated to each sample. A method of generating the bit allocation table is illustrated in FIG. 5.
- the bit output unit 620 outputs the most significant bits of the additional mantissa information in amount of the number of the bits allocated to each sample in an input frame by using the bit allocation table. That is, the bit output unit 620 outputs [the additional mantissa information] / (2 ⁇ [the number of the bits of the additional mantissa information - the allocated number of the bits]).
- FIG. 7 is a block diagram of an enhancement layer decoder according to an embodiment of the present invention.
- the enhancement layer decoder includes an exponent map generating unit 700, a bit allocation table generating unit 710, and an additional mantissa decoding unit 720.
- the exponent map generating unit 700 and the bit allocation table generating unit 710 are the same as the exponent map generating unit 600 and the bit allocation table generating unit 610 illustrated in FIG. 6, and thus a detailed explanation thereof will not be given here.
- the additional mantissa decoding unit 720 extracts and decodes additional mantissa information of each sample from an enhancement bitstream based on the number of bits of the additional mantissa information allocated to each sample.
- FIG. 8 is a block diagram of an enhancement layer encoder according to another embodiment of the present invention.
- the enhancement layer encoder includes an additional mantissa extracting unit 800, a bit allocating unit 810, and a bit output unit 820.
- the additional mantissa extracting unit 800 extracts additional mantissa information from an input frame based on an exponent value obtained from the G.711 encoding performed on the input frame. That is, the additional mantissa extracting unit 800 extracts some bits followed by mantissa information (i.e., G.711 mantissa information of 4 bits) as additional mantissa information, based on the exponent value obtained from the G.711 encoding performed on the input frame. Although all of bits followed by the mantissa information may be the additional mantissa information, it is assumed in FIG. 8 that the maximum number of bits of the additional mantissa information is 3 in the current embodiment. In this case, a pseudo-code for the additional mantissa extracting unit 800 is as follows.
- L is the number of samples in a frame
- exp[i] is an exponent value of an i-th sample
- ext_bits[i] is the number of bits of additional mantissa information of the i-th sample
- ext_mantissa[i] is the additional mantissa information of the i-th sample
- x[i] is the i-th sample
- & is a bitwise AND operation.
- an input sample is "0000 0001 1010 1001 " in binary.
- the input sample is encoded in G.711 A-law, which is a combination of a sign bit of "1", exponent bits of "001 ", and mantissa bits of "1010".
- the resulting additional mantissa information is "100", assuming that the maximum number of bits of additional mantissa information is 3.
- the bit allocating unit 810 calculates the number of bits allocated to each sample by using the total number of available bits and exponent values of each sample. For example, if the bit-rate of an enhancement layer is 16 Kbit/s and the frame size is 5 ms, the total number of available bits per frame is 80 bits.
- the bit allocating unit 810 dynamically computes the number of bits for additional mantissa information per each sample based on the priorities of the additional mantissa information of each sample with criterion to minimize quantization error in each frame. For example, the priority may be determined from the exponent value of each sample which means a quantization step in a G.711 codec.
- the bit output unit 820 outputs the additional mantissa information in amount of the number of the bits allocated to each sample computed by the bit allocating unit 810.
- a pseudo-code for the bit output unit 820 is as follows.
- bit__alloc[i] is the number of bits allocated to the i-th sample
- tx_bit_enh[i] is an coded enhancement bitstream of the i-th sample
- x»a" is an operation in which "x" is right-shifted by "a” bits.
- bit output unit 820 outputs "10" as its encoded enhancement bitstream.
- the maximum number of bits per sample for additional mantissa information is extracted from the sample, and then is truncated to allocated number of bits for the sample.
- the allocated number of bits is extracted from a sample after a bit allocation table is established, in another embodiment.
- FIG. 9 is a flowchart illustrating a method of an enhancement layer encoding according to an embodiment of the present invention.
- an enhancement layer encoder extracts additional mantissa information based on an exponent value obtained from the G.711 encoding performed on the input frame.
- the additional mantissa information may be the whole remaining bits followed by mantissa information in the input frame, it is assumed in FIG. 9 that the maximum number of bits for the additional mantissa information is 3 in the current embodiment. Accordingly, the enhancement layer encoder extracts additional mantissa information of 3 bits only.
- the enhancement layer encoder computes the number of additional bits for mantissa information per sample by using the exponent value obtained from the G.711 encoding.
- the enhancement layer encoder sets priorities for 3 respective bits of the additional mantissa information of each sample as being proportional to the exponent value of each sample. For example, an exponent index may be used as a priority as shown in FIGS. 4A and 4B.
- the enhancement layer encoder sequentially allocates 1 bit to samples with from the highest priority to the lowest priority until a total number of allocated bits exceeds a total number of available bits in the frame.
- the enhancement layer encoder outputs the most significant bits of the additional mantissa information in amount of the number of the bits allocated to each sample in an input frame.
- FIG. 10 is a block diagram of an enhancement layer decoder according to another embodiment of the present invention.
- the enhancement layer decoder includes a decoder bit allocating unit 1000, an additional mantissa decoding unit 1010, and an enhancement signal synthesizing unit 1020.
- the decoder bit allocating unit 1000 obtains the number of bits for additional mantissa information per sample by using the total number of available bits in a frame and exponent values obtained by G.711 decoding.
- the operation of the decoder bit allocating unit 1000 is the same as that of the bit allocating unit illustrated in FIG. 7, and thus a detailed explanation thereof will not be given here.
- the additional mantissa decoding unit 1010 restores additional mantissa information from an enhancement bitstream, using the exponent value and the number of the bits for the additional mantissa information obtained by the decoder bit allocating unit 1000. That is, the additional mantissa decoding unit 1010 restores the additional mantissa information by extracting the same number of bits as the number of the bits of the additional mantissa information allocated to each sample from the enhancement bitstream.
- the maximum number of the bits for the additional mantissa information is 3
- a pseudo-code for operating the additional mantissa decoding unit 1010 is shown below.
- the additional mantissa decoding unit 1010 inserts a bit string of "0"s to the decoded additional mantissa information in amount of difference between the maximum number of bits for the additional mantissa information and the number of allocated bits for the additional mantissa information.
- the enhancement signal synthesizing unit 1020 restores an enhancement signal by using the restored additional mantissa information, exponent value and sign information obtained by G.711 decoding.
- a pseudo-code for operating the enhancement signal synthesizing unit 1020 is as follows. At first, the restored additional mantissa information is bit-shifted to left in amount of the exponent value, and then sign information is added in the way that if the sign information indicates a negative value, the bit-shifted additional mantissa information is multiplied by (-1 ), and otherwise, the bit-shifted additional mantissa information itself is an enhancement sample.
- sign[i] is sign information of the i-th sample which is obtained by G.711 decoding.
- FIG. 11 is a flowchart illustrating a method of decoding an enhancement layer according to an embodiment of the present invention.
- an enhancement layer decoder obtains the number of additional bits allocated to each sample by using the maximum number of available bits and exponent values in a frame obtained by a G.711 decoding unit.
- the enhancement layer decoder extracts additional mantissa information, as many as the number of the additional bits allocated to each sample, from an enhancement bitstream.
- the enhancement layer decoder restores additional mantissa information.
- the present invention may be embodied as computer-readable codes on a computer-readable recording medium.
- the computer-readable recording medium is any data storage device that can store data which can be thereafter read by a computer system. Examples of the computer-readable recording medium include read-only memories (ROMs), random-access memories (RAMs), CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices.
- ROMs read-only memories
- RAMs random-access memories
- CD-ROMs compact discs
- magnetic tapes magnetic tapes
- floppy disks and optical data storage devices.
- the computer-readable recording medium can be dispersively installed in a computer system connected to a network, and stored and executed as a computer-readable code in a distributed computing environment. While the present invention has been particularly shown and described with reference to exemplary embodiments thereof using specific terms, the embodiments and terms have been used to explain the present invention and should not be construed as limiting the scope of the present invention defined by the claims. Accordingly, it will be understood by those of ordinary
Landscapes
- Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Quality & Reliability (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
Description
Claims
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| EP10153693.6A EP2187387B1 (en) | 2007-08-16 | 2008-08-18 | Method for encoding and decoding an enhancement layer |
| EP10153630A EP2202728B1 (en) | 2007-08-16 | 2008-08-18 | Apparatus and method for decoding an enhancement layer |
Applications Claiming Priority (4)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| KR20070082346 | 2007-08-16 | ||
| KR1020080024919A KR100912826B1 (en) | 2007-08-16 | 2008-03-18 | A enhancement layer encoder/decoder for improving a voice quality in G.711 codec and method therefor |
| KR1020080079929A KR100912827B1 (en) | 2007-08-16 | 2008-08-14 | A enhancement layer encoder/decoder for improving a voice quality in G.711 codec and method therefor |
| PCT/KR2008/004775 WO2009022884A1 (en) | 2007-08-16 | 2008-08-18 | Apparatus and method for encoding and decoding enhancement layer |
Related Child Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| EP10153693.6 Division-Into | 2010-02-16 | ||
| EP10153630.8 Division-Into | 2010-02-16 |
Publications (3)
| Publication Number | Publication Date |
|---|---|
| EP2179415A1 true EP2179415A1 (en) | 2010-04-28 |
| EP2179415A4 EP2179415A4 (en) | 2011-07-20 |
| EP2179415B1 EP2179415B1 (en) | 2013-03-20 |
Family
ID=40350862
Family Applications (3)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| EP08793289A Not-in-force EP2179415B1 (en) | 2007-08-16 | 2008-08-18 | Apparatus and method for encoding an enhancement layer |
| EP10153630A Not-in-force EP2202728B1 (en) | 2007-08-16 | 2008-08-18 | Apparatus and method for decoding an enhancement layer |
| EP10153693.6A Not-in-force EP2187387B1 (en) | 2007-08-16 | 2008-08-18 | Method for encoding and decoding an enhancement layer |
Family Applications After (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| EP10153630A Not-in-force EP2202728B1 (en) | 2007-08-16 | 2008-08-18 | Apparatus and method for decoding an enhancement layer |
| EP10153693.6A Not-in-force EP2187387B1 (en) | 2007-08-16 | 2008-08-18 | Method for encoding and decoding an enhancement layer |
Country Status (2)
| Country | Link |
|---|---|
| EP (3) | EP2179415B1 (en) |
| WO (1) | WO2009022884A1 (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012002348A1 (en) * | 2010-06-29 | 2012-01-05 | 日本電信電話株式会社 | Bit allocation device, method, program and recording medium thereof |
| FR2981781A1 (en) | 2011-10-19 | 2013-04-26 | France Telecom | IMPROVED HIERARCHICAL CODING |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6785261B1 (en) * | 1999-05-28 | 2004-08-31 | 3Com Corporation | Method and system for forward error correction with different frame sizes |
| US6724814B1 (en) * | 1999-06-24 | 2004-04-20 | Intel Corporation | Pad and CODEC detection |
| US6650762B2 (en) * | 2001-05-31 | 2003-11-18 | Southern Methodist University | Types-based, lossy data embedding |
-
2008
- 2008-08-18 WO PCT/KR2008/004775 patent/WO2009022884A1/en not_active Ceased
- 2008-08-18 EP EP08793289A patent/EP2179415B1/en not_active Not-in-force
- 2008-08-18 EP EP10153630A patent/EP2202728B1/en not_active Not-in-force
- 2008-08-18 EP EP10153693.6A patent/EP2187387B1/en not_active Not-in-force
Non-Patent Citations (4)
| Title |
|---|
| "Pulse code modulation (PCM) of voice frequencies; G.711 (11/88)", ITU-T STANDARD IN FORCE (I), INTERNATIONAL TELECOMMUNICATION UNION, GENEVA, CH, no. G.711 (11/88), 25 November 1988 (1988-11-25), XP017400850, * |
| "Wideband embedded extension for G.711 pulse code modulation; G.711.1 (03/08)", ITU-T STANDARD, INTERNATIONAL TELECOMMUNICATION UNION, GENEVA ; CH, no. G.711.1 (03/08), 15 March 2008 (2008-03-15), XP017433779, * |
| HIWASAKI Y ET AL: "A G.711 EMBEDDED WIDEBAND SPEECH CODING FOR VOIP CONFERENCES", IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS, INFORMATION & SYSTEMS SOCIETY, TOKYO, JP, vol. E89D, no. 9, 1 September 2006 (2006-09-01), pages 2542-2552, XP001542710, ISSN: 0916-8532 * |
| See also references of WO2009022884A1 * |
Also Published As
| Publication number | Publication date |
|---|---|
| EP2179415A4 (en) | 2011-07-20 |
| WO2009022884A1 (en) | 2009-02-19 |
| EP2179415B1 (en) | 2013-03-20 |
| EP2202728A1 (en) | 2010-06-30 |
| EP2187387A2 (en) | 2010-05-19 |
| EP2187387B1 (en) | 2013-07-31 |
| EP2187387A3 (en) | 2012-01-25 |
| EP2202728B1 (en) | 2013-03-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US6807526B2 (en) | Method of and apparatus for processing at least one coded binary audio flux organized into frames | |
| CN1735928B (en) | Method for variable rate audio codec | |
| EP0573216A2 (en) | CELP vocoder | |
| KR20070038041A (en) | Method and apparatus for speech trans-rating in a multi-rate speech coder for telecommunications | |
| EP1595248A2 (en) | System and method for enhancing bit error tolerance over a bandwith limited channel | |
| US8055499B2 (en) | Transmitter and receiver for speech coding and decoding by using additional bit allocation method | |
| US8498875B2 (en) | Apparatus and method for encoding and decoding enhancement layer | |
| JP5047263B2 (en) | Encoding device and decoding device | |
| EP2187387B1 (en) | Method for encoding and decoding an enhancement layer | |
| Ding | Wideband audio over narrowband low-resolution media | |
| EP0573215A2 (en) | Vocoder synchronization | |
| JP3257386B2 (en) | Vector quantization method | |
| SECTOR et al. | ITU-Tg. 711.0 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
| 17P | Request for examination filed |
Effective date: 20100316 |
|
| AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR |
|
| AX | Request for extension of the european patent |
Extension state: AL BA MK RS |
|
| DAX | Request for extension of the european patent (deleted) | ||
| A4 | Supplementary search report drawn up and despatched |
Effective date: 20110620 |
|
| RIC1 | Information provided on ipc code assigned before grant |
Ipc: G10L 19/00 20060101ALI20110614BHEP Ipc: G10L 19/14 20060101AFI20090304BHEP |
|
| GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
| GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
| REG | Reference to a national code |
Ref country code: DE Ref legal event code: R079 Ref document number: 602008023065 Country of ref document: DE Free format text: PREVIOUS MAIN CLASS: G10L0019140000 Ipc: G10L0019035000 |
|
| GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
| AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR |
|
| REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
| RIC1 | Information provided on ipc code assigned before grant |
Ipc: G10L 19/035 20130101AFI20130208BHEP Ipc: G10L 19/002 20130101ALI20130208BHEP |
|
| REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
| REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
| REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 602501 Country of ref document: AT Kind code of ref document: T Effective date: 20130415 |
|
| REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602008023065 Country of ref document: DE Effective date: 20130516 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130620 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130701 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130320 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130620 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130320 |
|
| REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 602501 Country of ref document: AT Kind code of ref document: T Effective date: 20130320 |
|
| REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130621 Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130320 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130320 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130320 |
|
| REG | Reference to a national code |
Ref country code: NL Ref legal event code: VDEP Effective date: 20130320 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130320 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130320 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130720 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130320 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130320 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130320 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130320 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130320 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130722 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130320 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130320 Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130320 |
|
| PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130320 |
|
| 26N | No opposition filed |
Effective date: 20140102 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130320 |
|
| REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
| REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602008023065 Country of ref document: DE Effective date: 20140102 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130320 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20130831 Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20130831 |
|
| REG | Reference to a national code |
Ref country code: IE Ref legal event code: MM4A |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20130818 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130320 Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20130320 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20080818 Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20130818 |
|
| REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 9 |
|
| REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 10 |
|
| REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 11 |
|
| P01 | Opt-out of the competence of the unified patent court (upc) registered |
Effective date: 20230625 |
|
| PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20230720 Year of fee payment: 16 |
|
| PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20230725 Year of fee payment: 16 Ref country code: DE Payment date: 20230720 Year of fee payment: 16 |
|
| REG | Reference to a national code |
Ref country code: DE Ref legal event code: R119 Ref document number: 602008023065 Country of ref document: DE |
|
| GBPC | Gb: european patent ceased through non-payment of renewal fee |
Effective date: 20240818 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20250301 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GB Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20240818 |
|
| PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: FR Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20240831 |